Compare commits

...

50 Commits

Author SHA1 Message Date
唐小鸭 aa9f77f0f2 fix(kms): stop an oversized Vault credential window from panicking the request path
refresh_safety_window_secs is operator-supplied and unbounded, so a window like
u64::MAX passed validation and then reached `Instant::now() + safety_window` in
VaultCredentialProvider::current. After a lease-bearing login the first request
panicked with "overflow when adding duration to instant" — reachable from the
admin configure API for every login-based auth method.

Use checked arithmetic and collapse an unrepresentable window to "refuse": such
a window means every token is always inside it, so that is both the fail-closed
answer and the one the arithmetic was reaching for. The comparison moves into
one helper because current() and record_credential_gauges() must apply the same
gate, which their doc comments already require.

The other operand had the same defect: expires_at and renew_at add a TTL built
from the lease_duration the Vault server sent, an unvalidated u64 off the wire.
An unrepresentable TTL now collapses to None, which is indistinguishable from
the no-expiry case Vault already produces for zero-lease tokens; the token stays
in use and Vault still validates it on every call. Leaving that side unchecked
would have kept the same panic reachable through the lease instead of the
window.
2026-08-14 13:32:23 +08:00
唐小鸭 e4781e763a fix(kms): apply the configured skip-TLS-verify to every Vault client
VaultConnectionSettings carried no TLS state, so both backend constructors
dropped VaultConfig::tls on the floor and build_client never called
VaultClientSettingsBuilder::verify. vaultrs 0.8.0 then fell back to its own
default, leaving verification on: with RUSTFS_KMS_VAULT_SKIP_TLS_VERIFY=true
and RUSTFS_KMS_ALLOW_INSECURE_DEV_DEFAULTS=true against a self-signed Vault,
startup still failed the handshake with UnknownIssuer.

Carry skip_tls_verify on the connection settings and set verify explicitly on
every client generation, authenticated and login alike. Setting it
unconditionally also closes a bypass in the other direction: left unset, vaultrs
derives verify from its own VAULT_SKIP_VERIFY variable, so a stray value in the
environment disabled certificate verification without passing the KMS
insecure-defaults gate.

The restore path pins verification on: VaultRestoreTarget carries no TLS
settings, and recovery is the last path that should accept an unauthenticated
Vault. The remaining TlsConfig fields (ca_cert_path, client_cert_path,
client_key_path) are still unused, but no supported input can set them — every
constructor leaves them None.
2026-08-14 12:54:29 +08:00
唐小鸭 ab7e777e55 fix(kms): resolve Vault auth from the environment at startup and add Kubernetes auth
The server startup path built its Vault backend config field by field from the
command-line struct, hardcoding VaultAuthMethod::Token and requiring a token.
KmsConfig::from_env(), which already resolved AppRole and token-file auth plus
namespace, TLS and mount settings, was never called outside tests, so those
environment variables were silently dropped whenever RUSTFS_KMS_ENABLE=true and
the documented AppRole / Vault Agent deployments could not start.

Move the assembly into vault_kv2_config_from_env / vault_transit_config_from_env
in the KMS crate and route both from_env() and init.rs through them, with the
command line supplying only the values it owns. One implementation now serves
both entry points, so they cannot drift apart again.

On top of that, add VaultAuthMethod::Kubernetes: the pod's projected
ServiceAccount token is exchanged for a lease-bound Vault token and renewed like
AppRole. The token is re-read on every login because the kubelet rotates it, and
the file mode is deliberately not checked since the kubelet mounts it
world-readable. This removes the Vault Agent sidecar requirement on Kubernetes
and leaves no credential to distribute.

VaultCliOverrides deliberately does not derive Debug: it carries the raw token,
so denying the derive turns a future interpolation into a compile error.
2026-08-14 10:24:43 +08:00
houseme aa4d3317ed perf(ecstore): guard inline data-read metadata early-stop (#6069)
Add a default-off inline-only data-read metadata early-stop gate that verifies inline plaintext before cancelling pending metadata tasks.

Keep non-inline, prepared, and request-shape-sensitive reads on full fanout, and record scheduled/completed/cancelled ReadVersion lifecycle metrics for normal fanout completion.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 21:40:14 +08:00
houseme f704d015d6 fix(copy): keep copy commit owner alive (#6070)
Keep S3 CopyObject's real outer owner task alive across caller cancellation so the source/destination bucket guards, same-key copy guard, storage commit, and post-commit publication hooks complete as one request-owned transaction boundary.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 20:19:33 +08:00
houseme 6b86d44cac fix(ecstore): retain commit owners across cancellation (#6068) 2026-08-13 18:08:58 +08:00
Zhengchao An e3c15f012c test(table-catalog): extract the shared avro/json fixture constructors (#6066)
The two table_catalog test files (27.5K lines combined) each maintained a parallel constructor stack for Iceberg metadata JSON and avro manifest-list/manifest bytes. Per the issue's adversarial ruling the parameterized admin variants are canonical (the store file hardcoded sequence 7 / snapshot 20); the two stacks were verified structurally identical first — schemas byte-equal, field lists and values aligned.

New #[cfg(test)] table_catalog/test_support.rs owns the seven constructors (metadata JSON, three manifest-list variants, two manifest variants, nullable_long). The admin tests import them under their old names; the store tests keep their historical signatures as thin delegates passing the fixed values explicitly — every produced byte is identical to the pre-extraction fixtures (the delegate's argument order was cross-checked against the canonical destructuring after an initial swap surfaced as five sequence-bound validation failures).

Ref rustfs/backlog#1837 (PR1).
2026-08-13 09:45:47 +00:00
houseme d2b1003612 perf(storage): converge Wave 2 hot-path optimizations (#6065)
* perf(get): share inline shards and lock clients

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): converge PUT encoding on contiguous blocks

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(get): cache codec streaming gate config

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(sse): redact projected customer headers

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): collapse GET metadata snapshots

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): reuse decode stripe scratch

Co-Authored-By: heihutu <heihutu@gmail.com>

* refactor(ecstore): trim decode scratch adapters

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): adapt transition checks to metadata snapshots

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(get): release metadata snapshots at ownership boundary

Co-Authored-By: heihutu <heihutu@gmail.com>

* refactor(ecstore): close cumulative fast-path findings

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(storage): preserve lock and header invariants

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): adapt cumulative paths after rebase

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(rio-v2): adapt generated metadata fixture

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 16:34:28 +08:00
GatewayJ 36deab8670 perf(ecstore): retain remote shard HTTP chunks (#5991)
* perf(ecstore): retain remote shard HTTP chunks

* fix(ecstore): bound remote shard chunk retention

* fix(rio): persist empty chunk limit across polls
2026-08-13 15:00:44 +08:00
cxymds e11fcfbd08 fix(rebalance): converge multipart data movement retries (#6057)
* fix(rebalance): converge multipart data movement retries

* fix(rebalance): harden multipart retry replacement

* fix(rebalance): isolate internal multipart uploads

* test(ecstore): adapt metadata mutation fixtures

* fix(rebalance): preserve transition metadata semantics

* refactor(ecstore): reuse internal metadata matcher

* Revert "refactor(ecstore): reuse internal metadata matcher"

This reverts commit c87ca0328f.

* refactor(rebalance): reuse data movement log constants

* fix(rebalance): isolate migration-owned state

* fix(rebalance): preserve pre-gate retry compatibility
2026-08-13 06:12:26 +00:00
houseme 11eecdc888 perf(put): avoid eager body zero fill (#6063)
Use BytesMut spare capacity for direct and pooled small PUT body reads while preserving exact-length validation.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 05:30:00 +00:00
houseme 80eb4244a3 chore(deps): refresh mimalloc revision (#6062) 2026-08-13 12:31:01 +08:00
Zhengchao An e4da9bd718 refactor(rustfs): move layer-neutral shared types out of server (#6061)
RemoteAddr, the DependencyReadiness family (DependencyReadiness, ReadinessDegradedReason, DependencyReadinessReport), and convert_ecstore_object_info (with its offset_date_time_to_timestamp helper) are consumed across app, infra, and interface layers but lived under server, so every lower-layer import was an upward app->interface or infra->interface edge the layer guard had to baseline.

They now live in a new layer-neutral rustfs/src/shared_types.rs (classified infra by the guard, making all consumer imports downward or lateral). server::readiness and server::event re-export for their own internals; the eight consumer sites (admin_usecase, bucket_usecase, object_usecase, cluster_snapshot, storage/access, storage/helper, plus the admin handler tests) import from the new home. Pure move: no type, impl, or behavior change.

The regenerated layer-dependency baseline shrinks by exactly eight lines with zero additions — the ratchet's intended direction. The two remaining readiness entries (collect/snapshot fn imports) need the collection machinery itself extracted from server and are left for the issue's PR5 scope.

Ref rustfs/backlog#1834 (PR4).
2026-08-13 12:29:49 +08:00
Zhengchao An e28430ab3d test(rustfs): un-ignore the fourteen ecfs_test global-state tests (#6046)
The 14 tests carried #[ignore = "requires isolated global object layer state"], and the only CI lane that runs ignored tests filters for lifecycle tests — so they executed nowhere. Under nextest, the authoritative runner, every test owns its process and the stale reason no longer applies; all 14 pass.

Ten of them assert the InternalError path taken while the global object layer is uninitialized, a premise a sibling test can destroy under the documented shared-process cargo test fallback. Those ten now start with an explicit premise guard: when a sibling already initialized the store the test skips with a message instead of asserting against a scenario it does not describe. Under nextest the guard never fires and the assertions always run.

Dual-runner evidence: nextest 79 passed; cargo test module-scoped 79 passed; the full storage-tree cargo test sweep returns to its pre-existing baseline (8 unrelated in-process failures, none introduced or worsened here). No test deleted.

Ref rustfs/backlog#1830 (PR1).
2026-08-13 12:29:06 +08:00
Zhengchao An db4707f187 chore(io-metrics): make the server label injected, drop two leaf-violating deps (#6051) 2026-08-13 03:20:18 +00:00
houseme 3a0dbccc2e perf(ecstore): reduce inline PUT commit overhead (#6033)
* perf(metrics): attribute PUT stage costs

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): move PUT metadata during shuffle

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(s3): reuse PUT object lock state

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): trim PUT metadata fanout clones

Build per-disk PUT metadata only for committed writer slots, move the response metadata out of the fanout vector, and preserve fresh FileInfo shuffle semantics.

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(metrics): make PUT stage attribution opt-in

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): commit inline PUT shards directly

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): streamline rename staging cleanup

Use the directory-specific removal operation for rename_data staging parents. This avoids a guaranteed failed file-removal probe on Unix-like hosts and lets Windows remove the empty directory directly while preserving best-effort non-empty handling.

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): cover inline PUT rename failures

Cache the detailed stage metrics gate once per PUT and exercise exact-quorum and quorum-minus-one failures after inline shard encoding.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 02:04:20 +00:00
houseme 846517625b fix(ecstore): bound remote metadata read retries (#6035)
* fix(ecstore): bound remote metadata read retries

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): handle oversized read timeouts

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 02:01:05 +00:00
houseme f21e88b112 perf(ecstore): avoid per-block shard vector allocation (#6037)
Keep encoded shards in one contiguous Bytes buffer while they cross the streaming write queue, and materialize Vec<Bytes> only for the existing public APIs.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 01:55:24 +00:00
Zhengchao An a5594c3d89 fix(kms): construct wrap_budget_reserved in the VaultKeyData deserializer (#6055) 2026-08-13 00:59:13 +00:00
Zhengchao An fc927caadd docs(guards): make the layer-dependency baseline an explicit shrink-only ratchet (#6053) 2026-08-13 00:27:19 +00:00
Henry Guo b7e6334c13 fix(table-catalog): isolate metadata after table rename (#6050) 2026-08-13 00:15:31 +00:00
Zhengchao An 65091aa6a8 test: give the twenty-one bare #[ignore] attributes their reasons (#6049) 2026-08-13 08:10:47 +08:00
Zhengchao An 3c78a56ab0 test: un-ignore the remaining seven global-state tests, drop one stale premise (#6048) 2026-08-13 08:10:28 +08:00
Zhengchao An bdd7ecd205 test(rustfs): un-ignore the nine node_service global-state tests (#6047) 2026-08-13 08:10:11 +08:00
Zhengchao An a70a3787d8 fix(kms): tell an empty KV2 prefix from a missing mount on Vault's 404 (#6043) 2026-08-13 08:09:17 +08:00
Zhengchao An b2ae430805 chore(ecstore): compile the list-objects chaos injector out of production builds (#6042) 2026-08-13 08:09:03 +08:00
Zhengchao An 299eb0d965 docs(operations): record the two experimental GET-path switches (#6041) 2026-08-13 08:08:48 +08:00
Zhengchao An f5a780099b fix(kms): let the Vault Transit backend start against an empty transit engine (#6040) 2026-08-13 08:08:30 +08:00
Zhengchao An 5cfafcf39b chore(rustfs): remove the orphan starshard bucket-cache backend (#6038) 2026-08-13 08:08:15 +08:00
Zhengchao An a49243c671 test(kms): pin ILM behavior on SSE-KMS buckets under key-policy enforcement (#6027) 2026-08-13 08:06:42 +08:00
Zhengchao An c2a15f5214 refactor(utils): add shared retry_with_backoff and migrate target_descriptor (#6026) 2026-08-13 08:06:21 +08:00
Zhengchao An ee54f1e618 docs(checksums): cross-reference the three checksum registries (#6024) 2026-08-13 08:06:00 +08:00
Zhengchao An e6b85b60a8 test(io-metrics): assert metric emission in six modules of record_* smoke tests (#6021) 2026-08-13 08:05:43 +08:00
Zhengchao An 8c1e3c09ff refactor(admin): add authorize_admin_request and fold four local wrappers (#6020) 2026-08-13 08:05:28 +08:00
Zhengchao An ca06c7ec2c feat(kms): reserve and expose KV2 wrap-budget consumption (#6019) 2026-08-13 08:05:00 +08:00
Zhengchao An 4a41325d1a feat(sse): report the wrapping master-key version on S3 audit entries (#6005) 2026-08-13 08:04:30 +08:00
Zhengchao An 45e2bd0c28 chore(ecstore): collapse the expiry worker knobs to one documented env var (#6034)
init_background_expiry resolved its worker count through three env vars, none documented, none set in any known deployment: RUSTFS_MAX_EXPIRY_WORKERS, silently overridden by the underscore-prefixed _RUSTFS_ILM_EXPIRATION_WORKERS (a MinIO fossil, comment included), with a zero value then falling through to RUSTFS_DEFAULT_EXPIRY_WORKERS.

RUSTFS_MAX_EXPIRY_WORKERS stays as the canonical name per the rc constraint (no new env var names): the count is now resolved once — a set, parseable, non-zero value wins, anything else falls back to min(cpus, 16). The constant moves to rustfs-config's runtime constants alongside ENV_TRANSITION_WORKERS, the two dead names are gone repo-wide (rg-verified), and a serial four-state unit test (unset/zero/valid/garbage) pins the resolution, modeled on the transition-worker env harness.

Ref rustfs/backlog#1832 (PR2).
2026-08-12 20:46:06 +00:00
Zhengchao An e2fb0427f9 docs(io-metrics): remove stale unified-config docs left by #6008 (#6039)
PR #6008 deleted crates/io-metrics/src/config.rs but the crate docs still taught the deleted API: both READMEs kept the Unified Configuration sections, module-tree entries and ./src/config.rs links, the example kept an orphaned numbered comment, and the io-core/io-metrics changelog had no removal record. Scrub all of it; keep cache_config.rs references, which are still real.
2026-08-12 20:41:40 +00:00
Zhengchao An a825326ede test(e2e): fold seven identical POST-policy exact-mismatch tests into one table (#6016)
* test(e2e): fold seven identical POST-policy exact-mismatch tests into one table

The seven *_policy_mismatch tests in multipart_auth_test.rs were body-identical after literal normalization: the policy pins one field to an exact value, the form sends a different value, and the upload must be rejected with 400 InvalidPolicyDocument naming the field. Each test booted its own full server.

They fold into one table-driven test on the run_post_object_policy_case helper introduced by the PR1 fold. Every row keeps its original test's exact bucket, key, field name, policy value, mismatched form value, body bytes, and expected error strings — including the three rows that asserted the stronger <Code>InvalidPolicyDocument</Code> form. The pinned condition is built with an explicit serde_json::Map since the field name is now a table parameter.

cargo nextest list reports 97 tests for this module; the inventory row is updated in the same diff (103 -> 97).

Ref rustfs/backlog#1838 (PR2).

* test(e2e): fold the remaining POST-policy duplicate groups (15 tests) (#6018)

Completes the multipart_auth table-driven fold: the six remaining body-identical groups collapse onto the shared run_post_object_policy_case helper.

- Seven single-field exact-mismatch tests (cache-control, expires, tagging, storage-class, content-type, success_action_status, metadata-field-exact) join the existing exact-condition mismatch table as rows — same shape as the PR2 fold.
- The two object-lock mismatch tests become a two-row table (policy pins mode + retain-until-date, one form field mismatches).
- The three SSE-KMS parameter mismatch tests become a three-row table (policy pins the SSE mode plus one KMS parameter, form differs).
- The three SSE-KMS outside-policy tests become a three-row table pinning the distinct contract: an undeclared KMS parameter sails past policy validation and is rejected at runtime with 501 NotImplemented, not a policy error.

Every row keeps its original test's exact bucket, key, field names, values, body bytes, and expected status/code strings. cargo nextest list reports 85 tests for the module; the inventory row is updated in the same diff (97 -> 85).

Ref rustfs/backlog#1838 (PR3).
2026-08-12 20:18:36 +00:00
Zhengchao An e313276e49 fix(sse): align copy-path unknown-algorithm fallback with put path (#6022) 2026-08-13 03:42:03 +08:00
Zhengchao An 66af487978 docs(policy): pin the deliberate slash-only path.Clean duplication (#6013)
The policy crate's Go path.Clean port and rustfs-utils' Windows-aware clean look like duplicates but are not interchangeable: S3 ARN/resource matching must treat backslashes as object-name data, never as separators, so adopting the utils version would change policy evaluation semantics on Windows — a security-adjacent behavior change. Record that judgment as bidirectional do-not-merge notes on both implementations, per the issue's adversarial ruling.

Comment-only change.

Ref rustfs/backlog#1833 (PR7).
2026-08-13 03:41:09 +08:00
Zhengchao An d668a9293f chore(ecstore): remove test-only BitrotErrorType and pin wire-only disk variants (#6032)
BitrotErrorType (disk/error.rs) was constructed only by its own unit test: production bitrot mismatches never flow through it (they surface as DiskError::other strings). Delete the enum, its From<BitrotErrorType> for DiskError impl, the self-test, and the api facade re-export. The facade inventory doc does not name the type, so no doc change is needed.

DiskError::SourceStalled and DiskError::CrossDeviceLink are never constructed locally — they are reachable only through wire decoding and no current node sends them. Their decode arms stay per the cross-version compatibility constraint; each variant now carries a doc comment saying exactly that so the next dead-code sweep does not re-litigate them. Their consumer arms (heal classifier, batch processor) are left untouched — the values cannot appear, so removing the arms would be unobservable, and the heal classifier is pinned by the issue as do-not-touch.

Ref rustfs/backlog#1831 (PR4).
2026-08-12 19:37:00 +00:00
Zhengchao An ace28c1f85 chore(common): remove dead bucket_stats module and LastMinuteHistogram (#6011)
crates/common/src/bucket_stats.rs (ReplicationLatency plus a commented-out ReplicationLastMinute corpse) had zero consumers anywhere in the workspace — the live replication statistics implementation is crates/replication/src/stats.rs. LastMinuteHistogram in last_minute.rs (already carrying allow(dead_code)) was equally unreferenced, and size_to_tag / SIZE_LAST_ELEM_MARKER had no user besides the histogram, so the whole block goes with it. LastMinuteLatency and AccElem stay: common's metrics.rs uses them.

Ref rustfs/backlog#1833 (PR5).
2026-08-13 03:27:17 +08:00
Zhengchao An 2ad8ab534e fix(site-replication): admit same-generation peer-edit fan-out bodies (#6007)
The peer-edit delivery fence from #5882 treated an equal applied generation as stale. One edit legitimately fans out one delivery per peer record under a single generation (the ILM-expiry edit sends every peer's record), so the receiver applied only the first body, raised its high-water mark, and silently acked-success while dropping the rest — enableILMExpiryReplication never converged on receiving sites and the three-node nightly e2e failed deterministically (issue #5767).

Only a strictly newer applied generation is stale now. Equal generation implies the same logical edit and re-applying a delivery is idempotent (update_peer overwrites the peer record; the mark is raised with max), while strictly older deliveries — the cross-node ordering case the fence exists for — stay rejected.

Adds a composed unit test driving three same-generation bodies through the receiver's fenced sequence, and widens the replication e2e's two site-replication wait helpers from a 10s polling ceiling to the 30s deadline the file's other waits use.
2026-08-13 03:26:34 +08:00
Zhengchao An f7df4fa62a fix(versioning): reject suspending versioning while a replication config exists (#6006)
PutBucketVersioning with Status=Suspended on a bucket that carries a replication configuration now fails with InvalidBucketState, matching AWS S3 and MinIO. Suspension would start minting null versions that the versioned replication engine can never converge — the state is unreachable on AWS and MinIO, and the nightly acceptance-matrix e2e that tried to exercise it failed every night since it landed (issue #5767).

The acceptance-matrix test tail now pins the rejection contract (InvalidBucketState) and verifies a fresh matched PUT still replicates with a real version id after the rejected suspension.
2026-08-13 03:26:17 +08:00
Zhengchao An ca4e66daab chore(io-metrics): remove the zero-consumer config module (391 lines) (#6008)
crates/io-metrics/src/config.rs was a near-copy of io-core's Backpressure/Deadlock configuration with already-drifted field names (high_watermark vs io-core's high_water_mark) and had no consumer outside the crate's own example: the canonical BackpressureConfig lives in crates/io-core/src/backpressure.rs. Delete the module, its lib.rs re-exports, and the example's unified-config section, and settle the corresponding ARCHITECTURE.md ledger line that tracked this copy's removal.

Ref rustfs/backlog#1833 (PR4).
2026-08-13 03:25:30 +08:00
Zhengchao An 5b9c5289c2 fix(iam): remove eight dead error variants and make Clone variant-preserving (#6030)
* chore(iam): remove eight dead error variants

iam::Error mirrored policy::Error variant-for-variant, and eight of the twins had zero construction and zero match sites anywhere in the workspace: InvalidServiceType, ErrCredMalformed, CredNotInitialized, JWTError, NoAccessKey, InvalidToken, InvalidAccessKey, InvalidExpiration (each verified by repo-wide sweep; the InvalidToken hits elsewhere are KeystoneError's unrelated variant). Delete the variants along with their Clone and PartialEq arms.

The From<policy::Error> mapping keeps its exhaustive match: the eight orphaned arms now route through a grouped binding to Error::StringError(err.to_string()), so the rendered message is preserved; nothing could observe the old discriminants because no site ever matched on them.

Ref rustfs/backlog#1831 (PR1).

* fix(iam): make Error clone variant-preserving via Arc payloads

iam::Error's hand-written Clone demoted PolicyError and CryptoError to StringError because their payloads are not cloneable — a clone changed the variant identity. There is no production clone site today (the issue's refuter confirmed this is preventive hardening, not a live bug), but any future holder of a cloned error would match the wrong variant.

The two payloads are now Arc-wrapped, so Clone is a cheap reference bump that keeps the variant. Display strings are unchanged ({0} and crypto: {0}); the #[from] derives become manual From impls wrapping in Arc; the one behavioral trade-off is that source() is no longer forwarded for these two variants (Arc<E> does not implement std::error::Error), which nothing in the workspace consumed. A regression test pins discriminant and rendered message across clone for the hard-to-clone variants.

Ref rustfs/backlog#1831 (PR2).
2026-08-12 19:14:07 +00:00
Zhengchao An 3f9b84ec70 feat(kms): observe unknown fields in the last three silent persisted formats (#6003) 2026-08-13 03:08:49 +08:00
houseme 73bd5d9d95 perf(get): reduce request entry allocations (#6029)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-12 19:07:59 +00:00
houseme 398d2d87c8 fix(ecstore): retry manual ILM job CAS updates (#6012)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-12 18:40:10 +00:00
181 changed files with 20153 additions and 5453 deletions
+1 -1
View File
@@ -85,7 +85,7 @@ runs:
repo-token: ${{ github.token }}
- name: Install flatc
uses: Nugine/setup-flatc@e7855e994773ce90094a3f1626d4afc9080c23ae # v1
uses: Nugine/setup-flatc@698800de72a96bfb22cf60431dc21a2ff9a7e07b # v1
with:
version: "25.12.19"
+3 -3
View File
@@ -131,9 +131,9 @@ module split is tracked under `docs/architecture/`.
why it stays local).
- ✅ RESOLVED: `BackpressureConfig` and `DataUsageInfo` each have exactly one
definition (`crates/io-core/src/backpressure.rs`,
`crates/data-usage/src/data_usage.rs`). A zero-consumer
`BackpressureSettings` copy lingers in `crates/io-metrics/src/config.rs`;
its removal is tracked in rustfs/backlog#1833.
`crates/data-usage/src/data_usage.rs`). The zero-consumer
`BackpressureSettings` copy that lingered in io-metrics was removed
(rustfs/backlog#1833).
4. **ecstore does not know about HTTP or S3 protocol details.** It operates on
storage-level abstractions (objects, buckets, disks, pools).
Generated
+39 -31
View File
@@ -1162,9 +1162,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.61.1"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a9381123ab62d20c13082b151f30f962a3b112b727345394536dfa39a482944"
checksum = "6de526c7b567420a31bc283657a7921b45c4cafe0827fdf2490713dcc770c28f"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -1195,9 +1195,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http-client"
version = "1.2.0"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "635d23afda0a6ab48d666c4d447c4873e8d1e83518a2be2093122397e50b838e"
checksum = "3c1c8a04cb31ba74d0115af5a890bb8c0d48fba64b52812fa13929a6ef0cc83c"
dependencies = [
"aws-smithy-async",
"aws-smithy-protocol-test",
@@ -1277,9 +1277,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.12.1"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07505b34e8f4b3591a4fa69e9792b52289b95488dbbc68c3c0075b7bedb245e1"
checksum = "483b858ff67522011c4786310c5cd8fd88d0be7ea3d5f1a48328446300c4269e"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
@@ -1343,9 +1343,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.6.1"
version = "1.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6dc683efb34b9e755675b37fedbe0103141e5b6df7bdc9eb6967756a8c167d8"
checksum = "fce83ce9abbb198d25bc7131e468d0f9fe1257125e58c39f3f9fc9f5098c9647"
dependencies = [
"base64-simd",
"bytes",
@@ -5133,9 +5133,9 @@ dependencies = [
[[package]]
name = "http-body-util"
version = "0.1.4"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9f41fd6a08e4d4ec69df65976da761afd5ad5e58a9d4acb46bd1c953a9e3ff2"
checksum = "23169fe34a5fbcdd3f3862e78fb9b6fccd5f02a6dc6f732547005d45631ce71c"
dependencies = [
"bytes",
"futures-core",
@@ -5957,7 +5957,7 @@ checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
[[package]]
name = "libmimalloc-sys"
version = "0.1.49"
source = "git+https://github.com/xonatius/mimalloc_rust.git?rev=ce6338661179c8be22e516b00af7483f151485a7#ce6338661179c8be22e516b00af7483f151485a7"
source = "git+https://github.com/xonatius/mimalloc_rust.git?rev=6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11#6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11"
dependencies = [
"cc",
"cty",
@@ -6259,9 +6259,9 @@ dependencies = [
[[package]]
name = "metrique"
version = "0.1.29"
version = "0.1.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2e394c63e2d1a30aeb3b9392ecf3439d8475d2df810a8f4f6e66d6866754017"
checksum = "dedbf06ffeef4c37990c73636fbd993aa34fb1948afd736e6114f239220993db"
dependencies = [
"itoa",
"jiff",
@@ -6289,9 +6289,9 @@ dependencies = [
[[package]]
name = "metrique-macro"
version = "0.1.20"
version = "0.1.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "786df1fd0abebd0db685f7e9a353c78756d4b370fb98a52376c2015fa55f141f"
checksum = "f4fb1f30185f53f7f6e4c9e46745c1a1350af8e77fda5a88aded44b0637a82e0"
dependencies = [
"Inflector",
"darling 0.23.0",
@@ -6318,9 +6318,9 @@ checksum = "2faca4e4480069ff02b1763b3b79f5cec7e8628e24d9dc5b6073f53d2577a4d9"
[[package]]
name = "metrique-writer"
version = "0.1.25"
version = "0.1.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82cdde44d241dab7fc8b7a32e0eb5dae6cd28f8de80b59f9a1e9f2f0b05e485e"
checksum = "20bd17c1a3ca2719e31f19ce77a853948dc2102f35976b92276c42a64fdc5f3f"
dependencies = [
"ahash",
"crossbeam-queue",
@@ -6339,9 +6339,9 @@ dependencies = [
[[package]]
name = "metrique-writer-core"
version = "0.1.19"
version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e57379b7ee2272efaeaaa6de062503563e57333b24aadc7f2255b3d602899e8b"
checksum = "f1a55b6aae1d85c557c729564c4e2b32a26dc65ba2d90d9647ca01f2bd4854c4"
dependencies = [
"derive-where",
"itertools 0.14.0",
@@ -6366,7 +6366,7 @@ dependencies = [
[[package]]
name = "mimalloc"
version = "0.1.52"
source = "git+https://github.com/xonatius/mimalloc_rust.git?rev=ce6338661179c8be22e516b00af7483f151485a7#ce6338661179c8be22e516b00af7483f151485a7"
source = "git+https://github.com/xonatius/mimalloc_rust.git?rev=6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11#6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11"
dependencies = [
"libmimalloc-sys",
]
@@ -6882,7 +6882,7 @@ version = "5.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51e219e79014df21a225b1860a479e2dcd7cbd9130f4defd4bd0e191ea31d67d"
dependencies = [
"base64 0.22.1",
"base64 0.21.7",
"chrono",
"getrandom 0.2.17",
"http 1.5.0",
@@ -8043,7 +8043,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf"
dependencies = [
"heck 0.5.0",
"itertools 0.14.0",
"itertools 0.10.5",
"log",
"multimap",
"once_cell",
@@ -8063,7 +8063,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03da047801ff44bb6a4d407d4860c05fd70bb81714e6b2f3812603d5b145b042"
dependencies = [
"heck 0.5.0",
"itertools 0.14.0",
"itertools 0.10.5",
"log",
"multimap",
"petgraph 0.8.3",
@@ -8084,7 +8084,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d"
dependencies = [
"anyhow",
"itertools 0.14.0",
"itertools 0.10.5",
"proc-macro2",
"quote",
"syn 2.0.119",
@@ -8097,7 +8097,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf"
dependencies = [
"anyhow",
"itertools 0.14.0",
"itertools 0.10.5",
"proc-macro2",
"quote",
"syn 2.0.119",
@@ -9201,7 +9201,6 @@ dependencies = [
"sha2 0.11.0",
"shadow-rs",
"socket2",
"starshard",
"subtle",
"sysinfo",
"temp-env",
@@ -9621,9 +9620,7 @@ dependencies = [
"metrics",
"metrics-util",
"num_cpus",
"rustfs-common",
"rustfs-s3-ops",
"rustfs-utils",
"sysinfo",
"thiserror 2.0.20",
"tokio",
@@ -9737,6 +9734,7 @@ dependencies = [
"rustfs-utils",
"rustify",
"serde",
"serde_ignored",
"serde_json",
"sha2 0.11.0",
"subtle",
@@ -10561,9 +10559,9 @@ dependencies = [
[[package]]
name = "rustls-connector"
version = "0.23.7"
version = "0.23.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09a5abe04eec18f8b9fbe87885bcaee6426de80bbc579958c0bc064b728ee617"
checksum = "1babecfcc65b139b812e74bcc7f9ec7b4e00db659fd42d99567b7e77f0c714c6"
dependencies = [
"futures-io",
"futures-rustls",
@@ -10939,6 +10937,16 @@ dependencies = [
"syn 3.0.3",
]
[[package]]
name = "serde_ignored"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115dffd5f3853e06e746965a20dcbae6ee747ae30b543d91b0e089668bb07798"
dependencies = [
"serde",
"serde_core",
]
[[package]]
name = "serde_json"
version = "1.0.151"
@@ -11792,7 +11800,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.3",
"getrandom 0.3.4",
"once_cell",
"rustix",
"windows-sys 0.61.2",
+6 -5
View File
@@ -154,7 +154,7 @@ hyper-rustls = { default-features = false, version = "0.27.9" }
hyper-util = { version = "0.1.20" }
http = "1.5.0"
http-body = "1.1.0"
http-body-util = "0.1.4"
http-body-util = "0.1.5"
minlz = "1.2.3"
reqwest = "0.13.4"
rustfs-kafka-async = { version = "1.2.0" }
@@ -182,6 +182,7 @@ quick-xml = "0.41.0"
rmp = { version = "0.8.15" }
rmp-serde = { version = "1.3.1" }
serde = { version = "1.0.229" }
serde_ignored = { version = "0.1" }
serde_json = { version = "1.0.151" }
serde_urlencoded = "0.7.1"
@@ -230,9 +231,9 @@ aws-credential-types = { version = "1.3.0" }
aws-sdk-kms = { default-features = false, version = "1.114.0" }
aws-sdk-s3 = { default-features = false, version = "1.141.0" }
aws-sdk-sts = { default-features = false, version = "1.110.0" }
aws-smithy-http-client = { default-features = false, version = "1.2.0" }
aws-smithy-http-client = { default-features = false, version = "1.3.0" }
aws-smithy-runtime-api = { version = "1.14.0" }
aws-smithy-types = { version = "1.6.1" }
aws-smithy-types = { version = "1.6.2" }
base64 = "0.23.1"
base64-simd = "0.8.0"
brotli = "8.0.4"
@@ -347,8 +348,8 @@ russh-sftp = "2.4.0"
dav-server = "0.11.0"
# Performance Analysis and Memory Profiling
mimalloc = { version = "0.1.52", git = "https://github.com/xonatius/mimalloc_rust.git", rev = "ce6338661179c8be22e516b00af7483f151485a7" }
libmimalloc-sys = { version = "0.1.49", git = "https://github.com/xonatius/mimalloc_rust.git", rev = "ce6338661179c8be22e516b00af7483f151485a7", features = ["extended"] }
mimalloc = { version = "0.1.52", git = "https://github.com/xonatius/mimalloc_rust.git", rev = "6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11" }
libmimalloc-sys = { version = "0.1.49", git = "https://github.com/xonatius/mimalloc_rust.git", rev = "6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11", features = ["extended"] }
hotpath = { version = "0.23.2", default-features = false }
# Snapshot testing for output format regression detection
insta = { version = "1.48" }
+7
View File
@@ -21,6 +21,13 @@ use crate::{
Xxhash3, Xxhash64, Xxhash128,
};
// DELIBERATE DUPLICATION of the x-amz-checksum-* names that also exist as
// AMZ_CHECKSUM_* in rustfs-utils' headers module (crates/utils/src/http/
// headers.rs): this crate is a zero-internal-dependency leaf, so it cannot
// import them, and it additionally owns the RustFS extension names
// (sha512/xxhash*) that utils does not carry. Values are pinned by the S3
// wire protocol; do not merge without a maintainer decision on the leaf
// boundary (backlog#1833).
pub const CRC_32_HEADER_NAME: &str = "x-amz-checksum-crc32";
pub const CRC_32_C_HEADER_NAME: &str = "x-amz-checksum-crc32c";
pub const SHA_1_HEADER_NAME: &str = "x-amz-checksum-sha1";
+8
View File
@@ -41,6 +41,14 @@ pub const XXHASH_64_NAME: &str = "xxhash64";
pub const XXHASH_128_NAME: &str = "xxhash128";
pub const MD5_NAME: &str = "md5";
/// One of three deliberately separate checksum registries (backlog#1833):
/// this enum owns the **streaming-hash algorithm registry**, including the
/// RustFS extensions (sha512, xxhash3/64/128). The on-disk xl.meta bitset
/// lives in `rustfs_rio::ChecksumType` (crates/rio/src/checksum.rs, varint
/// bits are append-only), and the MinIO-port client keeps its own
/// `ChecksumMode` (crates/ecstore/src/client/checksum.rs). When adding an
/// algorithm, extend all three (or record why not) — they do not derive from
/// each other.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
#[non_exhaustive]
pub enum ChecksumAlgorithm {
-87
View File
@@ -1,87 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::last_minute::{self};
use std::collections::HashMap;
pub struct ReplicationLatency {
// Delays for single and multipart PUT requests
upload_histogram: last_minute::LastMinuteHistogram,
}
impl ReplicationLatency {
// Merge two ReplicationLatency
pub fn merge(&mut self, other: &mut ReplicationLatency) -> &ReplicationLatency {
self.upload_histogram.merge(&other.upload_histogram);
self
}
// Get upload delay (categorized by object size interval)
pub fn get_upload_latency(&mut self) -> HashMap<String, u64> {
let mut ret = HashMap::new();
let avg = self.upload_histogram.get_avg_data();
for (i, v) in avg.iter().enumerate() {
let avg_duration = v.avg();
ret.insert(self.size_tag_to_string(i), avg_duration.as_millis() as u64);
}
ret
}
pub fn update(&mut self, size: i64, during: std::time::Duration) {
self.upload_histogram.add(size, during);
}
// Simulate the conversion from size tag to string
fn size_tag_to_string(&self, tag: usize) -> String {
match tag {
0 => String::from("Size < 1 KiB"),
1 => String::from("Size < 1 MiB"),
2 => String::from("Size < 10 MiB"),
3 => String::from("Size < 100 MiB"),
4 => String::from("Size < 1 GiB"),
_ => String::from("Size > 1 GiB"),
}
}
}
// #[derive(Debug, Clone, Default)]
// pub struct ReplicationLastMinute {
// pub last_minute: LastMinuteLatency,
// }
// impl ReplicationLastMinute {
// pub fn merge(&mut self, other: ReplicationLastMinute) -> ReplicationLastMinute {
// let mut nl = ReplicationLastMinute::default();
// nl.last_minute = self.last_minute.merge(&mut other.last_minute);
// nl
// }
// pub fn add_size(&mut self, n: i64) {
// let t = SystemTime::now()
// .duration_since(UNIX_EPOCH)
// .expect("Time went backwards")
// .as_secs();
// self.last_minute.add_all(t - 1, &AccElem { total: t - 1, size: n as u64, n: 1 });
// }
// pub fn get_total(&self) -> AccElem {
// self.last_minute.get_total()
// }
// }
// impl fmt::Display for ReplicationLastMinute {
// fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
// let t = self.last_minute.get_total();
// write!(f, "ReplicationLastMinute sz= {}, n= {}, dur= {}", t.size, t.n, t.total)
// }
// }
-41
View File
@@ -572,44 +572,3 @@ mod tests {
assert_eq!(total.n, 6);
}
}
const SIZE_LAST_ELEM_MARKER: usize = 10; // Assumed marker size is 10, modify according to actual situation
#[allow(dead_code)]
#[derive(Debug, Default)]
pub struct LastMinuteHistogram {
histogram: Vec<LastMinuteLatency>,
size: u32,
}
impl LastMinuteHistogram {
pub fn merge(&mut self, other: &LastMinuteHistogram) {
for i in 0..self.histogram.len() {
self.histogram[i].merge(&other.histogram[i]);
}
}
pub fn add(&mut self, size: i64, t: Duration) {
let index = size_to_tag(size);
self.histogram[index].add(&t);
}
pub fn get_avg_data(&mut self) -> [AccElem; SIZE_LAST_ELEM_MARKER] {
let mut res = [AccElem::default(); SIZE_LAST_ELEM_MARKER];
for (i, elem) in self.histogram.iter_mut().enumerate() {
res[i] = elem.get_total();
}
res
}
}
fn size_to_tag(size: i64) -> usize {
match size {
_ if size < 1024 => 0, // sizeLessThan1KiB
_ if size < 1024 * 1024 => 1, // sizeLessThan1MiB
_ if size < 10 * 1024 * 1024 => 2, // sizeLessThan10MiB
_ if size < 100 * 1024 * 1024 => 3, // sizeLessThan100MiB
_ if size < 1024 * 1024 * 1024 => 4, // sizeLessThan1GiB
_ => 5, // sizeGreaterThan1GiB
}
}
-1
View File
@@ -12,7 +12,6 @@
// See the License for the specific language governing permissions and
// limitations under the License.
pub mod bucket_stats;
// pub mod error;
pub mod globals;
pub mod heal_channel;
+5
View File
@@ -353,6 +353,11 @@ pub const DEFAULT_OBS_TRACES_EXPORT_ENABLED: bool = true;
/// Environment variable: RUSTFS_OBS_METRICS_EXPORT_ENABLED
pub const DEFAULT_OBS_METRICS_EXPORT_ENABLED: bool = true;
/// Default detailed PUT stage metrics enabled
/// Default value: false
/// Environment variable: RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED
pub const DEFAULT_OBS_PUT_STAGE_METRICS_ENABLED: bool = false;
/// Default logs export enabled
/// It is used to enable or disable exporting logs
/// Default value: true
+24
View File
@@ -137,6 +137,21 @@ pub const DEFAULT_TIER_REMOTE_VERSION_STATE_FLEET_CONFIRMED: bool = false;
const _: () = assert!(!DEFAULT_TIER_REMOTE_VERSION_STATE_WRITE);
const _: () = assert!(!DEFAULT_TIER_REMOTE_VERSION_STATE_FLEET_CONFIRMED);
/// Request preserving legacy per-part checksum metadata during data movement.
///
/// This remains ineffective until
/// [`ENV_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED`] is also enabled.
pub const ENV_DATA_MOVEMENT_PART_CHECKSUMS_WRITE: &str = "RUSTFS_DATA_MOVEMENT_PART_CHECKSUMS_WRITE";
pub const DEFAULT_DATA_MOVEMENT_PART_CHECKSUMS_WRITE: bool = false;
/// Operator-attested confirmation that every serving node understands the
/// data-movement per-part checksum sidecar.
pub const ENV_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED: &str = "RUSTFS_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED";
pub const DEFAULT_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED: bool = false;
const _: () = assert!(!DEFAULT_DATA_MOVEMENT_PART_CHECKSUMS_WRITE);
const _: () = assert!(!DEFAULT_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED);
// =============================================================================
// Concurrent Request Fix - Timeout and Backpressure Configuration
// =============================================================================
@@ -649,4 +664,13 @@ mod remote_version_state_tests {
"RUSTFS_TIER_REMOTE_VERSION_STATE_FLEET_CONFIRMED"
);
}
#[test]
fn data_movement_part_checksum_gate_uses_stable_environment_names() {
assert_eq!(super::ENV_DATA_MOVEMENT_PART_CHECKSUMS_WRITE, "RUSTFS_DATA_MOVEMENT_PART_CHECKSUMS_WRITE");
assert_eq!(
super::ENV_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED,
"RUSTFS_DATA_MOVEMENT_PART_CHECKSUMS_FLEET_CONFIRMED"
);
}
}
+3
View File
@@ -81,6 +81,9 @@ pub const ENV_TEST_IAM_FAIL_INIT_ATTEMPTS: &str = "RUSTFS_TEST_IAM_FAIL_INIT_ATT
pub const ENV_TEST_IAM_RETRY_INTERVAL_MS: &str = "RUSTFS_TEST_IAM_RETRY_INTERVAL_MS";
/// Runtime env var controlling the transition worker count.
pub const ENV_TRANSITION_WORKERS: &str = "RUSTFS_MAX_TRANSITION_WORKERS";
/// Runtime env var controlling the ILM expiry worker count. A set, parsable,
/// non-zero value wins; anything else falls back to `min(cpus, 16)`.
pub const ENV_MAX_EXPIRY_WORKERS: &str = "RUSTFS_MAX_EXPIRY_WORKERS";
/// Runtime env var controlling the absolute maximum transition workers.
pub const ENV_TRANSITION_WORKERS_ABSOLUTE_MAX: &str = "RUSTFS_ABSOLUTE_MAX_WORKERS";
/// Runtime env var controlling the transition queue capacity.
+5
View File
@@ -44,6 +44,10 @@ pub const ENV_OBS_METRICS_EXPORT_ENABLED: &str = "RUSTFS_OBS_METRICS_EXPORT_ENAB
pub const ENV_OBS_LOGS_EXPORT_ENABLED: &str = "RUSTFS_OBS_LOGS_EXPORT_ENABLED";
pub const ENV_OBS_PROFILING_EXPORT_ENABLED: &str = "RUSTFS_OBS_PROFILING_EXPORT_ENABLED";
/// Enables detailed per-stage PUT metrics. Disabled by default because each
/// PUT records multiple timers and histograms when attribution is active.
pub const ENV_OBS_PUT_STAGE_METRICS_ENABLED: &str = "RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED";
pub const ENV_OBS_LOGGER_LEVEL: &str = "RUSTFS_OBS_LOGGER_LEVEL";
pub const ENV_OBS_LOG_STDOUT_ENABLED: &str = "RUSTFS_OBS_LOG_STDOUT_ENABLED";
pub const ENV_OBS_LOG_DIRECTORY: &str = "RUSTFS_OBS_LOG_DIRECTORY";
@@ -141,6 +145,7 @@ mod tests {
assert_eq!(ENV_OBS_METRICS_EXPORT_ENABLED, "RUSTFS_OBS_METRICS_EXPORT_ENABLED");
assert_eq!(ENV_OBS_LOGS_EXPORT_ENABLED, "RUSTFS_OBS_LOGS_EXPORT_ENABLED");
assert_eq!(ENV_OBS_PROFILING_EXPORT_ENABLED, "RUSTFS_OBS_PROFILING_EXPORT_ENABLED");
assert_eq!(ENV_OBS_PUT_STAGE_METRICS_ENABLED, "RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED");
// Test log cleanup related env keys
assert_eq!(ENV_OBS_LOG_MAX_TOTAL_SIZE_BYTES, "RUSTFS_OBS_LOG_MAX_TOTAL_SIZE_BYTES");
assert_eq!(ENV_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES, "RUSTFS_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES");
@@ -0,0 +1,612 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! ILM on SSE-KMS buckets while per-key SSE authorization is enforced (backlog#1582).
//!
//! Per-key KMS authorization (`RUSTFS_KMS_ENFORCE_SSE_KEY_POLICY=true`) scopes the
//! SSE-KMS data path to the requesting principal's `kms:GenerateDataKey` /
//! `kms:Decrypt` grants. Internal callers — the lifecycle scanner's expiry deletes
//! and the tier transition worker's reads — carry no request principal, and
//! `authorize_sse_kms_key` (rustfs/src/storage/sse.rs) exempts a `None` principal
//! so background maintenance keeps working on encrypted buckets.
//!
//! These tests pin that exemption end to end. If enforcement ever starts applying
//! to the scanner's internal operations, expiry stops happening on SSE-KMS buckets
//! and [`ilm_expiration_on_sse_kms_bucket_under_enforcement`] times out; if it
//! starts applying to the transition worker or the read-through path,
//! [`ilm_transition_on_sse_kms_bucket_under_enforcement_reads_back`] fails at the
//! transition wait or the plaintext round-trip.
//!
//! The replication half of the same acceptance item lives in
//! `crates/e2e_test/src/replication_extension_test.rs`
//! (`test_bucket_replication_sse_kms_failure_contract`); ILM had no coverage
//! before this file.
//!
//! Deployment constraint pinned by the transition test's setup: the RustFS warm
//! backend forwards the object's stored `x-amz-server-side-encryption*` metadata
//! as raw headers on the tier data PUT (`build_transition_put_options` +
//! `api_put_object.rs` header mapping), so a RustFS tier target must itself have
//! KMS enabled and hold the named key or it rejects every transition upload with
//! 400 InvalidRequest. That rejection is independent of the enforcement switch;
//! the cold server here therefore runs its own Local KMS with the same key id.
use super::common::{LocalKMSTestEnvironment, create_key_with_specific_id};
use crate::common::{RustFSTestEnvironment, admin_request, init_logging};
use aws_sdk_s3::Client;
use aws_sdk_s3::primitives::ByteStream;
use aws_sdk_s3::types::{
BucketLifecycleConfiguration, ExpirationStatus, LifecycleExpiration, LifecycleRule, LifecycleRuleFilter, RestoreRequest,
ServerSideEncryption, ServerSideEncryptionByDefault, ServerSideEncryptionConfiguration, ServerSideEncryptionRule, Transition,
TransitionStorageClass,
};
use serde::Deserialize;
use serial_test::serial;
use std::time::{Duration as StdDuration, Instant};
use tracing::info;
type TestResult = Result<(), Box<dyn std::error::Error + Send + Sync>>;
const SSE_KEY: &str = "kms-ilm-sse-key";
const PAYLOAD: &[u8] = b"kms ilm sse payload: survives enforcement, expires and transitions on schedule";
const EXPIRY_BUCKET: &str = "kms-ilm-expiry";
const EXPIRE_KEY: &str = "expire/object.bin";
const SURVIVOR_KEY: &str = "keep/object.bin";
const TIER_NAME: &str = "KMSCOLD";
const TIER_BUCKET: &str = "kms-ilm-cold-tier";
const TIER_PREFIX: &str = "tiered";
const TRANSITION_BUCKET: &str = "kms-ilm-transition";
const TRANSITION_KEY: &str = "tier/object.bin";
/// Generous CI safety net; with a 1s scanner cycle and 2s lifecycle days the
/// terminal state normally lands within a few seconds.
const ILM_DEADLINE: StdDuration = StdDuration::from_secs(90);
/// Start a Local-KMS server with per-key SSE authorization enforced and the
/// lifecycle clock accelerated.
///
/// KMS wiring matches `kms_authorization_negative_matrix_test.rs` (local backend,
/// `--kms-default-key-id`, insecure dev defaults). The lifecycle env matches
/// `reliant/lifecycle.rs::fast_lifecycle_env` plus `RUSTFS_ILM_DEBUG_DAY_SECS=2`,
/// so a `Days=1` rule is due about two seconds after the write.
async fn start_enforcing_ilm_server(env: &mut LocalKMSTestEnvironment) -> TestResult {
create_key_with_specific_id(&env.kms_keys_dir, SSE_KEY).await?;
let key_dir = env.kms_keys_dir.clone();
let args = vec![
"--kms-enable",
"--kms-backend",
"local",
"--kms-key-dir",
key_dir.as_str(),
"--kms-default-key-id",
SSE_KEY,
];
let envs = [
("RUSTFS_KMS_ALLOW_INSECURE_DEV_DEFAULTS", "true"),
("RUSTFS_KMS_ENFORCE_SSE_KEY_POLICY", "false"),
("RUSTFS_SCANNER_CYCLE", "1"),
("RUSTFS_ILM_PROCESS_TIME", "1"),
("RUSTFS_ILM_DEBUG_DAY_SECS", "2"),
];
env.base_env.start_rustfs_server_with_env(args, &envs).await?;
Ok(())
}
/// Set the bucket's default encryption to SSE-KMS under [`SSE_KEY`], so plain
/// PUTs (and internal rewrites) are encrypted without per-request SSE headers.
async fn set_bucket_default_sse_kms(client: &Client, bucket: &str) -> TestResult {
let encryption_config = ServerSideEncryptionConfiguration::builder()
.rules(
ServerSideEncryptionRule::builder()
.apply_server_side_encryption_by_default(
ServerSideEncryptionByDefault::builder()
.sse_algorithm(ServerSideEncryption::AwsKms)
.kms_master_key_id(SSE_KEY)
.build()?,
)
.build(),
)
.build()?;
client
.put_bucket_encryption()
.bucket(bucket)
.server_side_encryption_configuration(encryption_config)
.send()
.await?;
Ok(())
}
/// Assert via `HeadObject` that the stored object is SSE-KMS encrypted under
/// [`SSE_KEY`]. Without this, a bucket-default misconfiguration would let the
/// tests pass on an unencrypted object and prove nothing about KMS.
async fn assert_head_sse_kms(client: &Client, bucket: &str, key: &str) -> TestResult {
let head = client.head_object().bucket(bucket).key(key).send().await?;
assert_eq!(
head.server_side_encryption(),
Some(&ServerSideEncryption::AwsKms),
"{bucket}/{key} must be SSE-KMS encrypted via the bucket default"
);
assert_eq!(
head.ssekms_key_id(),
Some(SSE_KEY),
"{bucket}/{key} must be wrapped under the configured KMS key"
);
Ok(())
}
/// Returns `true` once `GET bucket/key` fails with `NoSuchKey`, `false` while it
/// still succeeds. Any other error is surfaced. (Copied from
/// `reliant/lifecycle.rs`; that helper is private to the reliant module.)
async fn object_is_gone(client: &Client, bucket: &str, key: &str) -> Result<bool, Box<dyn std::error::Error + Send + Sync>> {
match client.get_object().bucket(bucket).key(key).send().await {
Ok(output) => {
output.body.collect().await?;
Ok(false)
}
Err(e) => {
if let Some(service_error) = e.as_service_error() {
if service_error.is_no_such_key() {
return Ok(true);
}
return Err(format!("expected NoSuchKey, got: {e:?}").into());
}
Err(format!("expected a service error, got: {e:?}").into())
}
}
}
/// Poll until `GET bucket/key` returns `NoSuchKey`, or fail after `deadline`.
async fn wait_for_object_expired(client: &Client, bucket: &str, key: &str, deadline: StdDuration) -> TestResult {
let start = Instant::now();
loop {
if object_is_gone(client, bucket, key).await? {
return Ok(());
}
if start.elapsed() >= deadline {
return Err(format!(
"object {bucket}/{key} was not expired by the lifecycle scanner within {}s; \
SSE key-policy enforcement may have started blocking the scanner's internal deletes",
deadline.as_secs()
)
.into());
}
tokio::time::sleep(StdDuration::from_millis(500)).await;
}
}
/// Install a prefix-scoped `Days`-based expiration rule.
async fn put_expiration_rule(client: &Client, bucket: &str, id: &str, prefix: &str, days: i32) -> TestResult {
let rule = LifecycleRule::builder()
.id(id)
.filter(LifecycleRuleFilter::builder().prefix(prefix).build())
.expiration(LifecycleExpiration::builder().days(days).build())
.status(ExpirationStatus::Enabled)
.build()?;
let lifecycle = BucketLifecycleConfiguration::builder().rules(rule).build()?;
client
.put_bucket_lifecycle_configuration()
.bucket(bucket)
.lifecycle_configuration(lifecycle)
.send()
.await?;
Ok(())
}
/// Install a prefix-scoped `Days`-based transition rule targeting [`TIER_NAME`].
async fn put_transition_rule(client: &Client, bucket: &str, id: &str, prefix: &str, days: i32) -> TestResult {
let rule = LifecycleRule::builder()
.id(id)
.filter(LifecycleRuleFilter::builder().prefix(prefix).build())
.transitions(
Transition::builder()
.days(days)
.storage_class(TransitionStorageClass::from(TIER_NAME))
.build(),
)
.status(ExpirationStatus::Enabled)
.build()?;
let lifecycle = BucketLifecycleConfiguration::builder().rules(rule).build()?;
client
.put_bucket_lifecycle_configuration()
.bucket(bucket)
.lifecycle_configuration(lifecycle)
.send()
.await?;
Ok(())
}
/// Start a plain Local-KMS server (no enforcement, no lifecycle acceleration)
/// holding [`SSE_KEY`], to serve as the cold tier target.
///
/// The RustFS warm backend forwards the stored SSE-KMS headers on the tier data
/// PUT, so the target re-applies managed SSE-KMS under the named key and must
/// be able to resolve it; without KMS it answers 400 InvalidRequest and the
/// transition can never complete. Enforcement stays off here: the tier writes
/// arrive under `cold`'s root credentials, and one enforcing side is enough to
/// pin the exemption.
async fn start_cold_tier_kms_server(env: &mut LocalKMSTestEnvironment) -> TestResult {
create_key_with_specific_id(&env.kms_keys_dir, SSE_KEY).await?;
let key_dir = env.kms_keys_dir.clone();
let args = vec![
"--kms-enable",
"--kms-backend",
"local",
"--kms-key-dir",
key_dir.as_str(),
"--kms-default-key-id",
SSE_KEY,
];
env.base_env
.start_rustfs_server_with_env(args, &[("RUSTFS_KMS_ALLOW_INSECURE_DEV_DEFAULTS", "true")])
.await?;
Ok(())
}
/// The subset of the manual transition run report these tests assert on.
///
/// Unknown fields are ignored, so this stays compatible with report growth; the
/// full shape is pinned by `reliant/tiering.rs`.
#[derive(Debug, Deserialize)]
struct ManualTransitionRunReport {
#[serde(default)]
scanned: u64,
#[serde(default)]
enqueued: u64,
#[serde(default)]
skipped_already_in_flight: u64,
#[serde(default)]
skipped_tier: u64,
}
#[derive(Debug, Deserialize)]
struct ManualTransitionRunResponse {
state: String,
report: ManualTransitionRunReport,
}
/// One synchronous (enqueue-only) manual transition run over `bucket/prefix`,
/// via the same admin endpoint `reliant/tiering.rs` drives.
async fn manual_transition_run(
hot: &RustFSTestEnvironment,
bucket: &str,
prefix: &str,
) -> Result<ManualTransitionRunResponse, Box<dyn std::error::Error + Send + Sync>> {
let bucket = urlencoding::encode(bucket);
let prefix = urlencoding::encode(prefix);
let tier = urlencoding::encode(TIER_NAME);
let path =
format!("/rustfs/admin/v3/ilm/transition/run?bucket={bucket}&prefix={prefix}&tier={tier}&dryRun=false&maxObjects=10");
let (status, body) = admin_request(&hot.url, http::Method::POST, &path, None, &hot.access_key, &hot.secret_key).await?;
if !status.is_success() {
return Err(format!("manual transition run failed: status={status}, body={body}").into());
}
Ok(serde_json::from_str(&body)?)
}
/// Drive manual transition runs until one reports the object as processed.
///
/// The `Days=1` rule becomes due about two seconds after the write
/// (`RUSTFS_ILM_DEBUG_DAY_SECS=2`), so early runs may legitimately report the
/// object as not yet eligible; the loop keeps running the endpoint until it
/// either enqueues the transition, sees it already in flight (the 1s scanner
/// backstop got there first), or finds it already on the tier.
async fn run_manual_transition_until_processed(
hot: &RustFSTestEnvironment,
bucket: &str,
prefix: &str,
deadline: StdDuration,
) -> TestResult {
let start = Instant::now();
loop {
let run = manual_transition_run(hot, bucket, prefix).await?;
assert_eq!(run.report.scanned, 1, "manual transition run must scan the object: {run:#?}");
if run.report.enqueued + run.report.skipped_already_in_flight + run.report.skipped_tier >= 1 {
info!(state = %run.state, report = ?run.report, "manual transition run processed the SSE-KMS object");
return Ok(());
}
if start.elapsed() >= deadline {
return Err(format!(
"manual transition runs never processed {bucket}/{prefix} within {}s; last report: {run:#?}",
deadline.as_secs()
)
.into());
}
tokio::time::sleep(StdDuration::from_millis(500)).await;
}
}
/// Wire `hot` -> `cold` as a `TierType::RustFS` remote tier via `AddTier`.
///
/// No `force`, so the server runs the real connectivity probe against `cold`
/// (the tier bucket must already exist there). Mirrors
/// `reliant/tiering.rs::add_rustfs_tier`, which is private to that module.
async fn add_rustfs_tier(hot: &RustFSTestEnvironment, cold: &RustFSTestEnvironment) -> TestResult {
let body = serde_json::json!({
"type": "rustfs",
"rustfs": {
"name": TIER_NAME,
"endpoint": cold.url.as_str(),
"accessKey": cold.access_key.as_str(),
"secretKey": cold.secret_key.as_str(),
"bucket": TIER_BUCKET,
"prefix": TIER_PREFIX,
"region": "us-east-1",
"storageClass": ""
}
})
.to_string();
let (status, resp) = admin_request(
&hot.url,
http::Method::PUT,
"/rustfs/admin/v3/tier",
Some(body),
&hot.access_key,
&hot.secret_key,
)
.await?;
if !status.is_success() {
return Err(format!("AddTier(RustFS) failed: status={status}, body={resp}").into());
}
Ok(())
}
/// Poll `HEAD` until the object's storage class is the tier name (transition
/// complete), or fail after `deadline`. (From `reliant/tiering.rs`.)
async fn wait_for_transition(client: &Client, bucket: &str, key: &str, deadline: StdDuration) -> TestResult {
let start = Instant::now();
loop {
let head = client.head_object().bucket(bucket).key(key).send().await?;
if head.storage_class().map(|sc| sc.as_str()) == Some(TIER_NAME) {
return Ok(());
}
if start.elapsed() >= deadline {
return Err(format!(
"object {bucket}/{key} was not transitioned to {TIER_NAME} within {}s (storage_class={:?}); \
SSE key-policy enforcement may have started blocking the transition worker's internal reads",
deadline.as_secs(),
head.storage_class()
)
.into());
}
tokio::time::sleep(StdDuration::from_millis(500)).await;
}
}
/// Poll `HEAD` until `x-amz-restore` reports a finished restore
/// (`ongoing-request="false"`), or fail after `deadline`.
async fn wait_for_restore_complete(client: &Client, bucket: &str, key: &str, deadline: StdDuration) -> TestResult {
let start = Instant::now();
loop {
let head = client.head_object().bucket(bucket).key(key).send().await?;
if head.restore().is_some_and(|r| r.contains("ongoing-request=\"false\"")) {
return Ok(());
}
if start.elapsed() >= deadline {
return Err(format!(
"object {bucket}/{key} restore did not complete within {}s (restore={:?}); \
SSE key-policy enforcement may have started blocking the restore copy-back's internal reads",
deadline.as_secs(),
head.restore()
)
.into());
}
tokio::time::sleep(StdDuration::from_millis(500)).await;
}
}
/// ILM expiration keeps working on an SSE-KMS bucket while per-key SSE
/// authorization is enforced.
///
/// The lifecycle scanner deletes expired objects with an internal (no-principal)
/// identity that holds no `kms` grant. If enforcement ever starts applying to
/// those internal deletes (or to the scanner's metadata reads) on encrypted
/// buckets, expiry stops happening and this test times out.
///
/// A survivor object under a non-matching prefix isolates the rule's prefix
/// filter as the cause of the deletion and proves the encrypted bucket stays
/// readable end to end after the scanner has run.
#[tokio::test]
#[serial]
async fn ilm_expiration_on_sse_kms_bucket_under_enforcement() -> TestResult {
init_logging();
let mut env = LocalKMSTestEnvironment::new().await?;
start_enforcing_ilm_server(&mut env).await?;
env.base_env.create_test_bucket(EXPIRY_BUCKET).await?;
let client = env.base_env.create_s3_client();
set_bucket_default_sse_kms(&client, EXPIRY_BUCKET).await?;
for key in [EXPIRE_KEY, SURVIVOR_KEY] {
client
.put_object()
.bucket(EXPIRY_BUCKET)
.key(key)
.body(ByteStream::from_static(PAYLOAD))
.send()
.await?;
assert_head_sse_kms(&client, EXPIRY_BUCKET, key).await?;
}
info!("both objects stored SSE-KMS encrypted under enforcement");
put_expiration_rule(&client, EXPIRY_BUCKET, "kms-ilm-expire", "expire/", 1).await?;
// The regression this pins: the scanner's internal delete must stay exempt
// from per-key SSE authorization, so the encrypted object actually expires.
wait_for_object_expired(&client, EXPIRY_BUCKET, EXPIRE_KEY, ILM_DEADLINE).await?;
info!("SSE-KMS object expired by the lifecycle scanner under enforcement");
// Negative control: same bucket, same encryption, non-matching prefix. It
// must survive the scanner and still decrypt for the requesting principal.
assert!(
!object_is_gone(&client, EXPIRY_BUCKET, SURVIVOR_KEY).await?,
"non-matching-prefix object must not be expired by a prefix-scoped rule"
);
let survivor = client.get_object().bucket(EXPIRY_BUCKET).key(SURVIVOR_KEY).send().await?;
assert_eq!(
survivor.body.collect().await?.into_bytes().as_ref(),
PAYLOAD,
"surviving SSE-KMS object must still decrypt after the scanner has run"
);
Ok(())
}
/// ILM transition to a remote tier keeps working on an SSE-KMS bucket while
/// per-key SSE authorization is enforced, and the transitioned object reads
/// back as plaintext.
///
/// The transition worker moves the stored (encrypted) bytes to the cold tier
/// with an internal (no-principal) identity; the read-through `GET` then
/// decrypts the envelope for the requesting principal. If enforcement ever
/// starts applying to the worker's internal reads, the transition wait times
/// out; if the stored envelope is mishandled across the tier round trip, the
/// plaintext comparison fails.
///
/// The transition is driven through the manual transition-run admin endpoint
/// (the mechanism `reliant/tiering.rs` established), so the test does not
/// depend on scanner scheduling; the 1s scanner cycle stays on as a backstop.
#[tokio::test]
#[serial]
#[ignore = "pins rustfs/rustfs#6025: GET on a transitioned managed-SSE object silently returns corrupt bytes (fails with enforcement on AND off, so it is not an authorization regression); un-ignore with the fix"]
async fn ilm_transition_on_sse_kms_bucket_under_enforcement_reads_back() -> TestResult {
init_logging();
// Cold-tier server: independent credentials, its own Local KMS holding the
// same key id (see the module docs for why the tier target needs KMS).
// Started first; each server's startup cleanup only matches its own unique
// address and temp dir, so the two instances coexist.
let mut cold = LocalKMSTestEnvironment::new().await?;
cold.base_env.access_key = "kmscoldtieradmin".to_string();
cold.base_env.secret_key = "kmscoldtiersecret".to_string();
start_cold_tier_kms_server(&mut cold).await?;
let cold_client = cold.base_env.create_s3_client();
cold_client.create_bucket().bucket(TIER_BUCKET).send().await?;
// Hot server: Local KMS + enforcement + accelerated lifecycle clock.
let mut env = LocalKMSTestEnvironment::new().await?;
start_enforcing_ilm_server(&mut env).await?;
let hot_client = env.base_env.create_s3_client();
add_rustfs_tier(&env.base_env, &cold.base_env).await?;
env.base_env.create_test_bucket(TRANSITION_BUCKET).await?;
set_bucket_default_sse_kms(&hot_client, TRANSITION_BUCKET).await?;
hot_client
.put_object()
.bucket(TRANSITION_BUCKET)
.key(TRANSITION_KEY)
.body(ByteStream::from_static(PAYLOAD))
.send()
.await?;
assert_head_sse_kms(&hot_client, TRANSITION_BUCKET, TRANSITION_KEY).await?;
info!("object stored SSE-KMS encrypted under enforcement");
// Days=1 is due ~2s after the write with RUSTFS_ILM_DEBUG_DAY_SECS=2.
put_transition_rule(&hot_client, TRANSITION_BUCKET, "kms-ilm-transition", "tier/", 1).await?;
// Drive the transition deterministically via the manual run endpoint, then
// wait for HEAD to report the tier as the object's storage class.
run_manual_transition_until_processed(&env.base_env, TRANSITION_BUCKET, "tier/", ILM_DEADLINE).await?;
wait_for_transition(&hot_client, TRANSITION_BUCKET, TRANSITION_KEY, ILM_DEADLINE).await?;
info!("SSE-KMS object transitioned to the remote tier under enforcement");
let head = hot_client
.head_object()
.bucket(TRANSITION_BUCKET)
.key(TRANSITION_KEY)
.send()
.await?;
assert!(
head.restore().is_none(),
"a freshly transitioned object must not advertise x-amz-restore, got {:?}",
head.restore()
);
// The remote copy exists on the cold tier. The payload the tier holds is the
// hot server's stored ciphertext, wrapped once more under the cold server's
// own managed SSE-KMS layer (the forwarded headers re-request encryption).
let remote = cold_client.list_objects_v2().bucket(TIER_BUCKET).send().await?;
assert!(!remote.contents().is_empty(), "cold-tier bucket must hold the transitioned object's data");
// Read-through GET under enforcement must succeed (not AccessDenied) and
// keep advertising SSE-KMS. Its BODY is deliberately not compared here:
// the transitioned read path skips managed-SSE decryption — a product gap
// unrelated to enforcement — so a direct GET streams the stored ciphertext
// (`new_getobjectreader` in crates/ecstore/src/client/object_api_utils.rs
// hardcodes `is_encrypted = false` and never applies the
// `ReadTransform::Encrypted` wrapping the hot-read path builds in
// crates/ecstore/src/object_api/readers.rs). Plaintext recovery is pinned
// through restore semantics below; when the read-through gap is fixed, a
// byte assertion can be added here too.
let read_through = hot_client
.get_object()
.bucket(TRANSITION_BUCKET)
.key(TRANSITION_KEY)
.send()
.await?;
assert_eq!(
read_through.server_side_encryption(),
Some(&ServerSideEncryption::AwsKms),
"transitioned object must still report SSE-KMS on read-through"
);
let read_through_body = read_through.body.collect().await?.into_bytes();
assert_eq!(
read_through_body.len(),
PAYLOAD.len(),
"read-through GET must stream the object's full logical size under enforcement"
);
// RestoreObject copies the ciphertext back from the tier under the original
// envelope metadata; the restored copy is then served by the normal
// decrypting read path. The copy-back runs with an internal (no-principal)
// identity, so this also pins the exemption on the restore path. Days=300
// because RUSTFS_ILM_DEBUG_DAY_SECS=2 accelerates the restored copy's
// expiry as well (300 accelerated days == 600s of validity).
hot_client
.restore_object()
.bucket(TRANSITION_BUCKET)
.key(TRANSITION_KEY)
.restore_request(RestoreRequest::builder().days(300).build())
.send()
.await?;
wait_for_restore_complete(&hot_client, TRANSITION_BUCKET, TRANSITION_KEY, ILM_DEADLINE).await?;
info!("SSE-KMS object restored from the remote tier under enforcement");
// The KMS-relevant half: the restored envelope decrypts back to the exact
// plaintext for the requesting principal.
let restored = hot_client
.get_object()
.bucket(TRANSITION_BUCKET)
.key(TRANSITION_KEY)
.send()
.await?;
assert_eq!(
restored.server_side_encryption(),
Some(&ServerSideEncryption::AwsKms),
"restored object must still report SSE-KMS"
);
let body = restored.body.collect().await?.into_bytes();
assert_eq!(body.as_ref(), PAYLOAD, "restored SSE-KMS object must round-trip byte-identical plaintext");
Ok(())
}
+3
View File
@@ -59,3 +59,6 @@ mod configured_roundtrip_test;
#[cfg(test)]
mod kms_authorization_negative_matrix_test;
#[cfg(test)]
mod kms_ilm_sse_kms_test;
File diff suppressed because it is too large Load Diff
@@ -2854,7 +2854,7 @@ pub(crate) mod cmptst_30 {
result
}
#[ignore]
#[ignore = "timing-sensitive backend-pressure latency probe; run explicitly with --ignored"]
#[tokio::test]
async fn regression() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
crate::common::init_logging();
@@ -2401,15 +2401,20 @@ async fn wait_for_site_replication_info<F>(
where
F: Fn(&SiteReplicationInfo) -> bool,
{
for _ in 0..40 {
// 30s to match wait_for_replication_state: the three-node site tests run
// several full rustfs processes on one runner, so peer-state propagation
// can take well over 10s under CI load.
let deadline = tokio::time::Instant::now() + Duration::from_secs(30);
loop {
let info = site_replication_info(env).await?;
if predicate(&info) {
return Ok(info);
}
if tokio::time::Instant::now() >= deadline {
return Err(format!("site replication info did not reach expected state on {}", env.address).into());
}
sleep(Duration::from_millis(250)).await;
}
Err(format!("site replication info did not reach expected state on {}", env.address).into())
}
async fn wait_for_site_replication_status<F>(
@@ -2420,15 +2425,19 @@ async fn wait_for_site_replication_status<F>(
where
F: Fn(&SRStatusInfo) -> bool,
{
for _ in 0..40 {
// Same 30s ceiling as wait_for_site_replication_info: the status probes
// fan out to every peer, so they see the same multi-process CI load.
let deadline = tokio::time::Instant::now() + Duration::from_secs(30);
loop {
let status = site_replication_status(env, query).await?;
if predicate(&status) {
return Ok(status);
}
if tokio::time::Instant::now() >= deadline {
return Err(format!("site replication status did not reach expected state on {}", env.address).into());
}
sleep(Duration::from_millis(250)).await;
}
Err(format!("site replication status did not reach expected state on {}", env.address).into())
}
async fn wait_for_replication_reset_target<F>(
@@ -4235,37 +4244,49 @@ async fn test_bucket_replication_acceptance_matrix_local_dual_targets() -> TestR
"tag rule with disabled delete-marker replication created a marker: {tagged_state:?}"
);
set_bucket_versioning(&source_env, source_bucket, BucketVersioningStatus::Suspended).await?;
set_bucket_versioning(&target_env_a, target_bucket_a, BucketVersioningStatus::Suspended).await?;
let null_put = source_client
// AWS S3 and MinIO both reject suspending versioning on a bucket that
// carries a replication configuration (InvalidBucketState): suspension
// would mint null versions that versioned replication can never converge.
let suspend_err = source_client
.put_bucket_versioning()
.bucket(source_bucket)
.versioning_configuration(
VersioningConfiguration::builder()
.status(BucketVersioningStatus::Suspended)
.build(),
)
.send()
.await
.expect_err("suspending versioning on a replication source must be rejected");
assert_eq!(
suspend_err.as_service_error().and_then(|error| error.code()),
Some("InvalidBucketState"),
"suspension on a replication source must fail with InvalidBucketState: {suspend_err:?}"
);
// The rejected suspension must leave the versioning + replication state
// fully intact: a fresh matched PUT still replicates with a real version.
let post_reject_put = source_client
.put_object()
.bucket(source_bucket)
.key("prefix/null.txt")
.body(ByteStream::from_static(b"null version"))
.key("prefix/after-rejected-suspend.txt")
.body(ByteStream::from_static(b"still replicating"))
.send()
.await?;
assert!(null_put.version_id().is_none(), "suspended source PUT must create a null version");
wait_for_replication_state(&target_client_a, target_bucket_a, "null version did not replicate", |state| {
state
.iter()
.any(|entry| entry.key == "prefix/null.txt" && entry.version_id == "null" && !entry.delete_marker)
})
.await?;
let null_delete = source_client
.delete_object()
.bucket(source_bucket)
.key("prefix/null.txt")
.send()
.await?;
assert!(
null_delete.version_id().is_none(),
"suspended source DELETE must create a null delete marker"
);
wait_for_replication_state(&target_client_a, target_bucket_a, "null delete marker did not replicate", |state| {
state
.iter()
.any(|entry| entry.key == "prefix/null.txt" && entry.version_id == "null" && entry.delete_marker)
})
let post_reject_version_id = post_reject_put
.version_id()
.ok_or("PUT after rejected suspension omitted version ID")?
.to_string();
wait_for_replication_state(
&target_client_a,
target_bucket_a,
"replication stopped after rejected versioning suspension",
|state| {
state
.iter()
.any(|entry| entry.key == "prefix/after-rejected-suspend.txt" && entry.version_id == post_reject_version_id)
},
)
.await?;
Ok(())
+5
View File
@@ -32,6 +32,11 @@ workspace = true
[features]
default = []
# Compiles the controlled list-objects namespace-journal chaos injector into a
# production binary (it is always available to tests). Off by default so the
# RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_* env vars cannot rewrite journal
# state in a stock build (backlog#1832).
list-chaos = []
rio-v2 = ["dep:rustfs-rio-v2"]
hotpath = [
"hotpath/hotpath",
+6 -4
View File
@@ -61,9 +61,11 @@ pub mod bucket {
delete_manual_transition_scope_admission_if_current, load_manual_transition_job_record,
load_manual_transition_job_record_with_etag, load_manual_transition_scope_admission,
manual_transition_job_lease_expired, manual_transition_scope_admission_lease_expired,
manual_transition_scope_key, persist_manual_transition_job_progress, renew_manual_transition_job_lease,
request_manual_transition_job_cancel, save_manual_transition_job_record,
save_manual_transition_job_record_if_current, save_manual_transition_scope_admission_if_absent,
manual_transition_scope_key, persist_manual_transition_job_progress,
persist_manual_transition_job_progress_if_owned, renew_manual_transition_job_lease,
renew_manual_transition_job_lease_if_owned, request_manual_transition_job_cancel,
save_manual_transition_job_record, save_manual_transition_job_record_if_current,
save_manual_transition_scope_admission_if_absent, update_manual_transition_job_record,
};
}
@@ -344,7 +346,7 @@ pub mod disk {
}
pub mod error {
pub use crate::disk::error::{BitrotErrorType, DiskError, Error, FileAccessDeniedWithContext, Result};
pub use crate::disk::error::{DiskError, Error, FileAccessDeniedWithContext, Result};
}
pub mod error_reduce {
File diff suppressed because it is too large Load Diff
@@ -86,6 +86,21 @@ where
com::save_config_with_opts(api, file, data, opts).await
}
pub(crate) async fn save_config_with_opts_quiet<S>(api: Arc<S>, file: &str, data: Vec<u8>, opts: &ObjectOptions) -> Result<()>
where
S: ObjectIO<
Error = Error,
RangeSpec = HTTPRangeSpec,
HeaderMap = HeaderMap,
ObjectOptions = ObjectOptions,
ObjectInfo = ObjectInfo,
GetObjectReader = GetObjectReader,
PutObjectReader = PutObjReader,
>,
{
com::save_config_with_opts_quiet(api, file, data, opts).await
}
pub(crate) async fn delete_config<S>(api: Arc<S>, file: &str) -> Result<()>
where
S: ObjectOperations<
@@ -45,6 +45,104 @@ const MANUAL_TRANSITION_JOB_LEASE_SECONDS: i128 = 60;
const MANUAL_TRANSITION_LEGACY_SCOPE_SCAN_LIMIT: i32 = 1000;
const MANUAL_TRANSITION_TASK_SCAN_LIMIT: i32 = 1000;
const MANUAL_TRANSITION_WORKER_RESULT_SCAN_LIMIT: i32 = 1000;
const MANUAL_TRANSITION_JOB_CAS_RETRIES: usize = 4;
#[cfg(test)]
struct ManualTransitionJobCasBarrierState {
job_id: Uuid,
paused: std::sync::atomic::AtomicBool,
arrived: tokio::sync::Notify,
release: tokio::sync::Semaphore,
}
#[cfg(test)]
pub(crate) struct ManualTransitionJobCasBarrier {
state: Arc<ManualTransitionJobCasBarrierState>,
}
#[cfg(test)]
static MANUAL_TRANSITION_JOB_CAS_BARRIER: std::sync::OnceLock<std::sync::Mutex<Option<Arc<ManualTransitionJobCasBarrierState>>>> =
std::sync::OnceLock::new();
#[cfg(test)]
impl ManualTransitionJobCasBarrier {
pub(crate) fn install(job_id: Uuid) -> Self {
let state = Arc::new(ManualTransitionJobCasBarrierState {
job_id,
paused: std::sync::atomic::AtomicBool::new(false),
arrived: tokio::sync::Notify::new(),
release: tokio::sync::Semaphore::new(0),
});
let mut slot = MANUAL_TRANSITION_JOB_CAS_BARRIER
.get_or_init(|| std::sync::Mutex::new(None))
.lock()
.expect("manual transition progress CAS barrier mutex should not poison");
assert!(
slot.is_none(),
"manual transition job CAS barrier must be installed by one test at a time"
);
*slot = Some(Arc::clone(&state));
drop(slot);
Self { state }
}
pub(crate) async fn wait_until_paused(&self) {
tokio::time::timeout(std::time::Duration::from_secs(30), async {
loop {
let arrived = self.state.arrived.notified();
if self.state.paused.load(std::sync::atomic::Ordering::Acquire) {
return;
}
arrived.await;
}
})
.await
.expect("manual transition job update should reach the deterministic CAS barrier");
}
pub(crate) fn release(&self) {
self.state.release.add_permits(1);
}
}
#[cfg(test)]
impl Drop for ManualTransitionJobCasBarrier {
fn drop(&mut self) {
self.release();
let mut slot = MANUAL_TRANSITION_JOB_CAS_BARRIER
.get_or_init(|| std::sync::Mutex::new(None))
.lock()
.expect("manual transition progress CAS barrier mutex should not poison");
if slot.as_ref().is_some_and(|state| Arc::ptr_eq(state, &self.state)) {
*slot = None;
}
}
}
#[cfg(test)]
async fn pause_manual_transition_job_before_first_cas(job_id: Uuid) {
let barrier = MANUAL_TRANSITION_JOB_CAS_BARRIER
.get_or_init(|| std::sync::Mutex::new(None))
.lock()
.expect("manual transition progress CAS barrier mutex should not poison")
.as_ref()
.filter(|barrier| barrier.job_id == job_id)
.cloned();
if let Some(barrier) = barrier
&& barrier
.paused
.compare_exchange(false, true, std::sync::atomic::Ordering::AcqRel, std::sync::atomic::Ordering::Acquire)
.is_ok()
{
barrier.arrived.notify_one();
barrier
.release
.acquire()
.await
.expect("manual transition job CAS barrier should remain open")
.forget();
}
}
fn is_false(value: &bool) -> bool {
!*value
@@ -148,7 +246,6 @@ impl ManualTransitionJobRecord {
pub fn fail(&mut self, error: impl Into<String>) {
self.state = ManualTransitionJobState::Failed;
self.report.tier_failure = self.report.tier_failure.saturating_add(1);
self.error = Some(error.into());
self.mark_updated_terminal();
}
@@ -1040,7 +1137,7 @@ pub async fn save_manual_transition_job_record_if_current(
}
let object = manual_transition_job_record_object_name(job.job_id).map_err(manual_transition_job_store_error)?;
let data = job.encode().map_err(manual_transition_job_store_error)?;
config_boundary::save_config_with_opts(
config_boundary::save_config_with_opts_quiet(
api,
&object,
data,
@@ -1056,6 +1153,54 @@ pub async fn save_manual_transition_job_record_if_current(
.await
}
/// Applies a job-record mutation with optimistic concurrency control.
///
/// The mutation returns whether the record needs to be persisted. When a lease
/// is supplied, ownership is checked again after every conflicting write.
pub async fn update_manual_transition_job_record<F>(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Option<Uuid>,
update: F,
) -> EcstoreResult<ManualTransitionJobRecord>
where
F: FnMut(&mut ManualTransitionJobRecord) -> bool,
{
update_manual_transition_job_record_from(api, job_id, expected_lease_id, None, update).await
}
async fn update_manual_transition_job_record_from<F>(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Option<Uuid>,
mut current: Option<(ManualTransitionJobRecord, String)>,
mut update: F,
) -> EcstoreResult<ManualTransitionJobRecord>
where
F: FnMut(&mut ManualTransitionJobRecord) -> bool,
{
for _ in 0..MANUAL_TRANSITION_JOB_CAS_RETRIES {
let (mut record, etag) = match current.take() {
Some(current) => current,
None => load_manual_transition_job_record_with_etag(api.clone(), job_id).await?,
};
if expected_lease_id.is_some_and(|lease_id| record.lease_id != lease_id) {
return Err(Error::PreconditionFailed);
}
if !update(&mut record) {
return Ok(record);
}
#[cfg(test)]
pause_manual_transition_job_before_first_cas(job_id).await;
match save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await {
Ok(()) => return Ok(record),
Err(Error::PreconditionFailed) => continue,
Err(err) => return Err(err),
}
}
Err(Error::PreconditionFailed)
}
pub(crate) async fn save_manual_transition_worker_result_if_absent(
api: Arc<ECStore>,
record: &ManualTransitionWorkerResultRecord,
@@ -1314,99 +1459,113 @@ pub async fn reconcile_manual_transition_worker_results(
api: Arc<ECStore>,
job_id: Uuid,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
reconcile_manual_transition_worker_results_inner(api, job_id, None, queue_snapshot, false).await
}
pub(crate) async fn reconcile_manual_transition_worker_results_if_owned(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Uuid,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
reconcile_manual_transition_worker_results_inner(api, job_id, Some(expected_lease_id), queue_snapshot, false).await
}
async fn reconcile_manual_transition_worker_results_inner(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Option<Uuid>,
queue_snapshot: ManualTransitionQueueSnapshot,
mark_missing_results_unknown: bool,
) -> EcstoreResult<ManualTransitionJobRecord> {
let task_stats = match scan_manual_transition_task_journal(api.clone(), job_id).await? {
ManualTransitionTaskJournal::Stats(stats) => stats,
ManualTransitionTaskJournal::Corrupt(error) => {
return mark_manual_transition_job_unknown_for_task_journal_error(api, job_id, error, queue_snapshot).await;
return mark_manual_transition_job_unknown_for_task_journal_error(
api,
job_id,
expected_lease_id,
error,
queue_snapshot,
)
.await;
}
};
let stats = match scan_manual_transition_worker_result_journal(api.clone(), job_id).await? {
ManualTransitionWorkerResultJournal::Stats(stats) => stats,
ManualTransitionWorkerResultJournal::Corrupt(error) => {
return mark_manual_transition_job_unknown_for_worker_result_journal_error(api, job_id, error, queue_snapshot).await;
return mark_manual_transition_job_unknown_for_worker_result_journal_error(
api,
job_id,
expected_lease_id,
error,
queue_snapshot,
)
.await;
}
};
for _ in 0..4 {
let (mut record, etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
let changed = record.apply_worker_result_counts(
let mut changed = false;
let record = update_manual_transition_job_record(api.clone(), job_id, expected_lease_id, |record| {
let counts_changed = record.apply_worker_result_counts(
stats.stats.completed,
stats.stats.failed,
&stats.stats.tier_failure_by_reason,
task_stats.queued,
queue_snapshot,
);
if !changed {
return Ok(record);
}
match save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await {
Ok(()) => {
if record.is_terminal() {
delete_manual_transition_scope_admission_if_current(
api.clone(),
&record.scope_key,
record.job_id,
record.lease_id,
)
.await?;
} else {
renew_manual_transition_scope_admission_from_job(api, &record).await?;
}
return Ok(record);
}
Err(Error::PreconditionFailed) => continue,
Err(err) => return Err(err),
}
let became_unknown = mark_missing_results_unknown && record.mark_unknown_if_worker_results_lost(queue_snapshot);
changed = counts_changed || became_unknown;
changed
})
.await?;
if !changed {
return Ok(record);
}
Err(Error::PreconditionFailed)
if record.is_terminal() {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id).await?;
} else {
renew_manual_transition_scope_admission_from_job(api, &record).await?;
}
Ok(record)
}
async fn mark_manual_transition_job_unknown_for_task_journal_error(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Option<Uuid>,
error: String,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
for _ in 0..4 {
let (mut record, etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
if !record.mark_unknown_for_task_journal_error(error.clone(), queue_snapshot) {
return Ok(record);
}
match save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await {
Ok(()) => {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id)
.await?;
return Ok(record);
}
Err(Error::PreconditionFailed) => continue,
Err(err) => return Err(err),
}
let mut changed = false;
let record = update_manual_transition_job_record(api.clone(), job_id, expected_lease_id, |record| {
changed = record.mark_unknown_for_task_journal_error(error.clone(), queue_snapshot);
changed
})
.await?;
if changed && record.is_terminal() {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id).await?;
}
Err(Error::PreconditionFailed)
Ok(record)
}
async fn mark_manual_transition_job_unknown_for_worker_result_journal_error(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Option<Uuid>,
error: String,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
for _ in 0..4 {
let (mut record, etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
if !record.mark_unknown_for_worker_result_journal_error(error.clone(), queue_snapshot) {
return Ok(record);
}
match save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await {
Ok(()) => {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id)
.await?;
return Ok(record);
}
Err(Error::PreconditionFailed) => continue,
Err(err) => return Err(err),
}
let mut changed = false;
let record = update_manual_transition_job_record(api.clone(), job_id, expected_lease_id, |record| {
changed = record.mark_unknown_for_worker_result_journal_error(error.clone(), queue_snapshot);
changed
})
.await?;
if changed && record.is_terminal() {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id).await?;
}
Err(Error::PreconditionFailed)
Ok(record)
}
pub async fn save_manual_transition_scope_admission_if_absent(
@@ -1603,19 +1762,14 @@ async fn find_active_legacy_manual_transition_scope_conflict(
}
pub async fn request_manual_transition_job_cancel(api: Arc<ECStore>, job_id: Uuid) -> EcstoreResult<ManualTransitionJobRecord> {
for _ in 0..4 {
let (mut record, etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
update_manual_transition_job_record(api, job_id, None, |record| {
if record.is_terminal() || record.cancel_requested {
return Ok(record);
return false;
}
record.mark_cancel_requested();
match save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await {
Ok(()) => return Ok(record),
Err(Error::PreconditionFailed) => continue,
Err(err) => return Err(err),
}
}
Err(Error::PreconditionFailed)
true
})
.await
}
pub async fn persist_manual_transition_job_progress(
@@ -1624,10 +1778,39 @@ pub async fn persist_manual_transition_job_progress(
report: &ManualTransitionRunReport,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
let (mut record, etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
record.update_running_progress(report.clone(), queue_snapshot);
save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await?;
renew_manual_transition_scope_admission_from_job(api, &record).await?;
let current = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
persist_manual_transition_job_progress_inner(api, job_id, current.0.lease_id, Some(current), report, queue_snapshot).await
}
pub async fn persist_manual_transition_job_progress_if_owned(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Uuid,
report: &ManualTransitionRunReport,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
persist_manual_transition_job_progress_inner(api, job_id, expected_lease_id, None, report, queue_snapshot).await
}
async fn persist_manual_transition_job_progress_inner(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Uuid,
current: Option<(ManualTransitionJobRecord, String)>,
report: &ManualTransitionRunReport,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
let record = update_manual_transition_job_record_from(api.clone(), job_id, Some(expected_lease_id), current, |record| {
if record.state != ManualTransitionJobState::Running {
return false;
}
record.update_running_progress(report.clone(), queue_snapshot);
true
})
.await?;
if record.state == ManualTransitionJobState::Running {
renew_manual_transition_scope_admission_from_job(api, &record).await?;
}
Ok(record)
}
@@ -1661,25 +1844,58 @@ pub async fn renew_manual_transition_job_lease(
job_id: Uuid,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
let (mut record, mut etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
if record.state == ManualTransitionJobState::Running {
if record.scan_completed && queue_snapshot.queued == 0 && queue_snapshot.active == 0 {
record = reconcile_manual_transition_worker_results(api.clone(), job_id, queue_snapshot).await?;
if record.is_terminal() || !record.report.worker_transition_pending() {
return Ok(record);
let current = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
renew_manual_transition_job_lease_inner(api, job_id, current.0.lease_id, Some(current), queue_snapshot).await
}
pub async fn renew_manual_transition_job_lease_if_owned(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Uuid,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
renew_manual_transition_job_lease_inner(api, job_id, expected_lease_id, None, queue_snapshot).await
}
async fn renew_manual_transition_job_lease_inner(
api: Arc<ECStore>,
job_id: Uuid,
expected_lease_id: Uuid,
current: Option<(ManualTransitionJobRecord, String)>,
queue_snapshot: ManualTransitionQueueSnapshot,
) -> EcstoreResult<ManualTransitionJobRecord> {
let (current, current_etag) = match current {
Some(current) => current,
None => load_manual_transition_job_record_with_etag(api.clone(), job_id).await?,
};
if current.lease_id != expected_lease_id {
return Err(Error::PreconditionFailed);
}
if current.state != ManualTransitionJobState::Running {
return Ok(current);
}
if current.scan_completed && queue_snapshot.queued == 0 && queue_snapshot.active == 0 {
return reconcile_manual_transition_worker_results_inner(api, job_id, Some(expected_lease_id), queue_snapshot, true)
.await;
}
let record = update_manual_transition_job_record_from(
api.clone(),
job_id,
Some(expected_lease_id),
Some((current, current_etag)),
|record| {
if record.state != ManualTransitionJobState::Running {
return false;
}
(record, etag) = load_manual_transition_job_record_with_etag(api.clone(), job_id).await?;
}
let became_terminal = record.mark_unknown_if_worker_results_lost(queue_snapshot);
if !became_terminal {
record.renew_lease(queue_snapshot);
}
save_manual_transition_job_record_if_current(api.clone(), &record, &etag).await?;
if became_terminal {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id).await?;
} else {
renew_manual_transition_scope_admission_from_job(api, &record).await?;
}
true
},
)
.await?;
if record.is_terminal() {
delete_manual_transition_scope_admission_if_current(api, &record.scope_key, record.job_id, record.lease_id).await?;
} else if record.state == ManualTransitionJobState::Running {
renew_manual_transition_scope_admission_from_job(api, &record).await?;
}
Ok(record)
}
@@ -1688,15 +1904,31 @@ async fn renew_manual_transition_scope_admission_from_job(
api: Arc<ECStore>,
record: &ManualTransitionJobRecord,
) -> EcstoreResult<()> {
if let Ok((admission, admission_etag)) =
load_manual_transition_scope_admission_with_etag(api.clone(), &record.scope_key).await
&& admission.job_id == record.job_id
&& admission.lease_id == record.lease_id
{
let renewed_admission = ManualTransitionScopeAdmission::from_job(record);
save_manual_transition_scope_admission_if_current(api, &renewed_admission, &admission_etag).await?;
for _ in 0..MANUAL_TRANSITION_JOB_CAS_RETRIES {
let (admission, admission_etag) =
match load_manual_transition_scope_admission_with_etag(api.clone(), &record.scope_key).await {
Ok(admission) => admission,
Err(Error::ConfigNotFound) => return Ok(()),
Err(err) => return Err(err),
};
if admission.job_id != record.job_id || admission.lease_id != record.lease_id {
return Err(Error::PreconditionFailed);
}
let mut renewed_admission = ManualTransitionScopeAdmission::from_job(record);
renewed_admission.lease_expires_at_unix_nanos = renewed_admission
.lease_expires_at_unix_nanos
.max(admission.lease_expires_at_unix_nanos);
renewed_admission.updated_at_unix_nanos = renewed_admission.updated_at_unix_nanos.max(admission.updated_at_unix_nanos);
if renewed_admission == admission {
return Ok(());
}
match save_manual_transition_scope_admission_if_current(api.clone(), &renewed_admission, &admission_etag).await {
Ok(()) => return Ok(()),
Err(Error::PreconditionFailed) => continue,
Err(err) => return Err(err),
}
}
Ok(())
Err(Error::PreconditionFailed)
}
pub async fn delete_manual_transition_scope_admission_if_current(
@@ -2386,14 +2618,14 @@ mod tests {
}
#[test]
fn manual_transition_job_record_failure_counts_tier_failure() {
fn manual_transition_job_record_control_plane_failure_does_not_count_tier_failure() {
let options = ManualTransitionRunOptions::default();
let mut record = ManualTransitionJobRecord::new(Uuid::new_v4(), "bucket", &options, TEST_OWNER);
record.fail("missing tier");
assert_eq!(record.state, ManualTransitionJobState::Failed);
assert_eq!(record.report.tier_failure, 1);
assert_eq!(record.report.tier_failure, 0);
assert_eq!(record.error.as_deref(), Some("missing tier"));
}
+12 -2
View File
@@ -27,12 +27,24 @@ use crate::client::utils::base64_decode;
use crate::client::utils::base64_encode;
use crate::client::{api_put_object::PutObjectOptions, api_s3_datatypes::ObjectPart};
use crate::{disk::DiskAPI, object_api::GetObjectReader};
// s3s::header has no CRC64NVME constant yet; the canonical RustFS copy lives
// in rustfs-utils' headers module.
use rustfs_utils::http::headers::AMZ_CHECKSUM_CRC64NVME;
use s3s::header::{
X_AMZ_CHECKSUM_ALGORITHM, X_AMZ_CHECKSUM_CRC32, X_AMZ_CHECKSUM_CRC32C, X_AMZ_CHECKSUM_SHA1, X_AMZ_CHECKSUM_SHA256,
};
use enumset::{EnumSet, EnumSetType, enum_set};
/// One of three deliberately separate checksum registries (backlog#1833):
/// this enum is the MinIO-port client's wire vocabulary and stops at the
/// standard S3 set (CRC64NVME is its newest member; the RustFS extensions do
/// not exist on this client path). The streaming-hash registry lives in
/// `rustfs_checksums::ChecksumAlgorithm` (crates/checksums/src/lib.rs) and
/// the on-disk xl.meta bitset in `rustfs_rio::ChecksumType`
/// (crates/rio/src/checksum.rs, varint bits are append-only). When adding an
/// algorithm, extend all three (or record why not) — they do not derive from
/// each other.
#[derive(Debug, EnumSetType, Default)]
#[enumset(repr = "u8")]
pub enum ChecksumMode {
@@ -57,8 +69,6 @@ lazy_static! {
static ref C_ChecksumFullObjectCRC32C: EnumSet<ChecksumMode> =
enum_set!(ChecksumMode::ChecksumCRC32C | ChecksumMode::ChecksumFullObject);
}
const AMZ_CHECKSUM_CRC64NVME: &str = "x-amz-checksum-crc64nvme";
impl ChecksumMode {
//pub const CRC64_NVME_POLYNOMIAL: i64 = 0xad93d23594c93659;
@@ -31,7 +31,7 @@ use rustfs_config::{
DEFAULT_INTERNODE_DATA_TRANSPORT, ENV_RUSTFS_INTERNODE_DATA_TRANSPORT, INTERNODE_DATA_TRANSPORT_TCP,
KNOWN_INTERNODE_DATA_TRANSPORT_BACKENDS,
};
use rustfs_rio::{HttpReader, HttpWriter};
use rustfs_rio::{ChunkReaderBox, HttpChunkReader, HttpReader, HttpWriter};
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::future::Future;
@@ -221,6 +221,11 @@ pub struct NsScannerCapabilityRequest {
#[async_trait]
pub trait InternodeDataTransport: Send + Sync + std::fmt::Debug {
async fn open_read(&self, request: ReadStreamRequest) -> Result<FileReader>;
/// Opens an owned-chunk stream when this transport can retain receive-buffer
/// ownership. `None` preserves the established `open_read` fallback.
async fn open_read_chunks(&self, _request: ReadStreamRequest) -> Result<Option<ChunkReaderBox>> {
Ok(None)
}
async fn open_write(&self, request: WriteStreamRequest) -> Result<FileWriter>;
async fn open_walk_dir(&self, request: WalkDirStreamRequest) -> Result<FileReader>;
async fn open_ns_scanner(&self, _request: NsScannerStreamRequest) -> Result<FileReader> {
@@ -247,6 +252,15 @@ impl InternodeDataTransport for TcpHttpInternodeDataTransport {
))
}
async fn open_read_chunks(&self, request: ReadStreamRequest) -> Result<Option<ChunkReaderBox>> {
let url = build_read_file_stream_url(&request);
let mut headers = json_headers();
build_auth_headers(&url, &Method::GET, &mut headers)?;
Ok(Some(Box::new(
HttpChunkReader::new_with_stall_timeout(url, Method::GET, headers, None, request.stall_timeout).await?,
)))
}
async fn open_write(&self, request: WriteStreamRequest) -> Result<FileWriter> {
let server_epoch = self.put_file_auth_capability(&request.endpoint).await?;
let nonce = server_epoch.map(|_| Uuid::new_v4());
+594 -26
View File
@@ -522,6 +522,33 @@ impl RemoteDisk {
}
}
async fn open_read_chunks_with_retry(&self, request: ReadStreamRequest) -> Result<Option<rustfs_rio::ChunkReaderBox>> {
let mut attempt = 1;
let mut last_retry_classification = None;
loop {
match self.data_transport.open_read_chunks(request.clone()).await {
Ok(reader) => {
if attempt > 1
&& let Some(classification) = last_retry_classification
{
crate::cluster::rpc::runtime_sources::record_remote_disk_open_read_retry_success(classification);
}
return Ok(reader);
}
Err(err) if attempt < REMOTE_DISK_OPEN_READ_MAX_ATTEMPTS && Self::is_retryable_open_read_error(&err) => {
if let Some(classification) = err.internode_http_error_kind() {
let classification = classification.metric_label();
crate::cluster::rpc::runtime_sources::record_remote_disk_open_read_retry(classification);
last_retry_classification = Some(classification);
}
tokio::time::sleep(REMOTE_DISK_OPEN_READ_RETRY_BACKOFF).await;
attempt += 1;
}
Err(err) => return Err(err),
}
}
}
pub fn record_capacity_probe(&self, total: u64, used: u64, free: u64) {
self.health.record_capacity_probe(total, used, free);
}
@@ -846,31 +873,49 @@ impl RemoteDisk {
/// default to 1 (see [`internode_idempotent_read_retries`]). MUST NOT be used for write/lock
/// RPCs — those must never auto-retry (quorum/idempotency safety). The `operation` closure is
/// re-invoked per attempt, so it must be `Fn` (rebuild the request from borrowed inputs, do not
/// move captured state out).
/// move captured state out). Attempts and backoff share one total timeout budget.
async fn execute_read_with_retry<T, F, Fut>(&self, op: &'static str, operation: F, timeout_duration: Duration) -> Result<T>
where
F: Fn() -> Fut,
Fut: std::future::Future<Output = Result<T>>,
{
let deadline = (!timeout_duration.is_zero()).then(|| {
time::Instant::now()
.checked_add(timeout_duration)
.unwrap_or_else(|| time::sleep(timeout_duration).deadline())
});
let max_retries = internode_idempotent_read_retries();
let mut attempt = 0usize;
loop {
// Only the final attempt marks the disk faulty / evicts the channel. Earlier retries
// ignore the failure, so a transient error cannot flip the disk into a faulty
// short-circuit (which would defeat the retry) or over-count failures.
let attempt_timeout = deadline
.map(|deadline| deadline.saturating_duration_since(time::Instant::now()))
.unwrap_or(Duration::ZERO);
if deadline.is_some() && attempt_timeout.is_zero() {
self.record_timeout(op, timeout_duration);
return Err(DiskError::Timeout);
}
let health_action = if attempt >= max_retries {
FailureHealthAction::MarkFailure
} else {
FailureHealthAction::IgnoreFailure
};
match self
.execute_with_timeout_for_op_and_health_action(op, &operation, timeout_duration, health_action)
.execute_with_timeout_for_op_and_health_action(op, &operation, attempt_timeout, health_action)
.await
{
Err(err) if attempt < max_retries && is_network_like_disk_error(&err) => {
if matches!(err, DiskError::Timeout) && deadline.is_some_and(|deadline| time::Instant::now() >= deadline) {
self.mark_faulty("read_operation_deadline");
return Err(err);
}
attempt += 1;
let backoff = REMOTE_DISK_READ_RETRY_BASE_BACKOFF
.saturating_mul(1u32 << u32::try_from(attempt - 1).unwrap_or(4).min(4));
if deadline.is_some_and(|deadline| deadline.saturating_duration_since(time::Instant::now()) <= backoff) {
attempt = max_retries;
continue;
}
debug!(
endpoint = %self.endpoint,
addr = %self.addr,
@@ -878,7 +923,17 @@ impl RemoteDisk {
attempt,
"retrying idempotent read-only RPC after transient network error"
);
tokio::time::sleep(backoff).await;
if let Some(deadline) = deadline {
if time::timeout_at(deadline, time::sleep(backoff)).await.is_err() {
self.record_timeout(op, timeout_duration);
return Err(DiskError::Timeout);
}
} else {
time::sleep(backoff).await;
}
if self.health.is_faulty() {
return Err(DiskError::FaultyDisk);
}
}
other => return other,
}
@@ -957,32 +1012,35 @@ impl RemoteDisk {
operation_result
}
Err(_) => {
// Timeout occurred, mark disk as potentially faulty
counter!(
"rustfs_drive_op_timeout_total",
"endpoint" => self.endpoint.to_string(),
"op" => op.to_string()
)
.increment(1);
self.record_timeout(op, timeout_duration);
if failure_health_action == FailureHealthAction::MarkFailure {
self.mark_faulty_and_evict("operation_timeout").await;
}
warn!(
event = EVENT_REMOTE_DISK_RPC,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
endpoint = %self.endpoint,
addr = %self.addr,
op,
timeout_ms = timeout_duration.as_millis(),
state = "timeout",
"Remote disk operation timed out"
);
Err(DiskError::Timeout)
}
}
}
fn record_timeout(&self, op: &'static str, timeout_duration: Duration) {
counter!(
"rustfs_drive_op_timeout_total",
"endpoint" => self.endpoint.to_string(),
"op" => op.to_string()
)
.increment(1);
warn!(
event = EVENT_REMOTE_DISK_RPC,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
endpoint = %self.endpoint,
addr = %self.addr,
op,
timeout_ms = timeout_duration.as_millis(),
state = "timeout",
"Remote disk operation timed out"
);
}
async fn handle_network_like_error<T>(
&self,
op: &'static str,
@@ -1016,7 +1074,7 @@ impl RemoteDisk {
}
}
async fn mark_faulty_and_evict(&self, reason: &'static str) {
fn mark_faulty(&self, reason: &'static str) -> bool {
let previous_state = self.runtime_state();
let transitioned_to_offline = self.mark_suspect_or_offline(reason);
let state = self.runtime_state();
@@ -1053,6 +1111,12 @@ impl RemoteDisk {
"Remote disk marked suspect"
);
}
}
state != previous_state
}
async fn mark_faulty_and_evict(&self, reason: &'static str) {
if self.mark_faulty(reason) {
counter!(
"rustfs_drive_connection_evict_total",
"endpoint" => self.endpoint.to_string(),
@@ -2068,7 +2132,7 @@ impl DiskAPI for RemoteDisk {
Ok(file_info)
},
get_max_timeout_duration(),
get_drive_metadata_timeout(),
)
.await
}
@@ -2417,6 +2481,30 @@ impl DiskAPI for RemoteDisk {
.await
}
async fn read_file_stream_chunks(
&self,
volume: &str,
path: &str,
offset: usize,
length: usize,
) -> Result<Option<rustfs_rio::ChunkReaderBox>> {
if self.health.is_faulty() {
return Err(DiskError::FaultyDisk);
}
let disk = self.disk_ref().await;
let stall_timeout = get_object_disk_read_timeout();
self.open_read_chunks_with_retry(ReadStreamRequest {
endpoint: self.endpoint.grid_host(),
disk,
volume: volume.to_string(),
path: path.to_string(),
offset,
length,
stall_timeout: (!stall_timeout.is_zero()).then_some(stall_timeout),
})
.await
}
/// Buffered read for remote disks.
/// The transport stream is collected into owned Bytes for caller sharing.
#[tracing::instrument(level = "trace", skip_all)]
@@ -5094,6 +5182,452 @@ mod tests {
);
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_reset_during_backoff_preserves_recovery() {
let remote_disk = Arc::new(new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await);
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let first_attempt = Arc::new(tokio::sync::Notify::new());
let started = time::Instant::now();
let task_disk = Arc::clone(&remote_disk);
let task_attempts = Arc::clone(&attempts);
let task_first_attempt = Arc::clone(&first_attempt);
let task = tokio::spawn(async move {
task_disk
.execute_read_with_retry(
"read_version",
move || {
let attempt = task_attempts.fetch_add(1, Ordering::SeqCst);
let first_attempt = Arc::clone(&task_first_attempt);
async move {
if attempt == 0 {
time::sleep(Duration::from_millis(20)).await;
first_attempt.notify_one();
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionRefused,
"connection refused",
)));
}
Ok(())
}
},
Duration::from_millis(100),
)
.await
});
first_attempt.notified().await;
tokio::task::yield_now().await;
remote_disk.health.reset_for_store_init_retry(&remote_disk.endpoint);
let channel = TonicEndpoint::from_shared(remote_disk.addr.clone())
.expect("remote disk address should parse")
.connect_lazy();
runtime_sources::cache_test_node_channel(remote_disk.addr.clone(), channel).await;
task.await
.expect("retry task should finish")
.expect("the retry should succeed after the health reset");
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert_eq!(started.elapsed(), Duration::from_millis(70));
assert_eq!(
remote_disk.health.waiting_count(),
0,
"health reset must not underflow the waiting counter"
);
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Online);
assert!(
runtime_sources::test_node_channel_is_cached(&remote_disk.addr).await,
"a recovered channel must survive the retry backoff"
);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_still_retries_within_shared_deadline() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let channel = TonicEndpoint::from_shared(remote_disk.addr.clone())
.expect("remote disk address should parse")
.connect_lazy();
runtime_sources::cache_test_node_channel(remote_disk.addr.clone(), channel).await;
remote_disk
.execute_read_with_retry(
"read_version",
|| {
let attempt = attempts.fetch_add(1, Ordering::SeqCst);
async move {
if attempt == 0 {
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionReset,
"connection reset",
)));
}
Ok(())
}
},
Duration::from_millis(100),
)
.await
.expect("a retry that fits the shared deadline should succeed");
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Online);
assert!(runtime_sources::test_node_channel_is_cached(&remote_disk.addr).await);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_uses_remaining_budget_for_final_attempt() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let started = time::Instant::now();
let channel = TonicEndpoint::from_shared(remote_disk.addr.clone())
.expect("remote disk address should parse")
.connect_lazy();
runtime_sources::cache_test_node_channel(remote_disk.addr.clone(), channel).await;
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
let attempt = attempts.fetch_add(1, Ordering::SeqCst);
async move {
if attempt == 0 {
time::sleep(Duration::from_millis(20)).await;
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionRefused,
"connection refused",
)));
}
std::future::pending::<Result<()>>().await
}
},
Duration::from_millis(100),
)
.await
.expect_err("the final retry should consume only the remaining total budget");
assert_eq!(err, DiskError::Timeout);
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert_eq!(started.elapsed(), Duration::from_millis(100));
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Suspect);
assert!(!runtime_sources::test_node_channel_is_cached(&remote_disk.addr).await);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_uses_final_attempt_at_exact_backoff_boundary() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let started = time::Instant::now();
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
let attempt = attempts.fetch_add(1, Ordering::SeqCst);
async move {
if attempt == 0 {
time::sleep(Duration::from_millis(50)).await;
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionRefused,
"connection refused",
)));
}
std::future::pending::<Result<()>>().await
}
},
Duration::from_millis(100),
)
.await
.expect_err("the exact backoff boundary should be reserved for a final attempt");
assert_eq!(err, DiskError::Timeout);
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert_eq!(started.elapsed(), Duration::from_millis(100));
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Suspect);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_uses_final_attempt_below_backoff_budget() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let started = time::Instant::now();
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
let attempt = attempts.fetch_add(1, Ordering::SeqCst);
async move {
if attempt == 0 {
time::sleep(Duration::from_millis(80)).await;
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionRefused,
"connection refused",
)));
}
std::future::pending::<Result<()>>().await
}
},
Duration::from_millis(100),
)
.await
.expect_err("remaining budget below backoff should be reserved for a final attempt");
assert_eq!(err, DiskError::Timeout);
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert_eq!(started.elapsed(), Duration::from_millis(100));
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Suspect);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_zero_timeout_disables_the_deadline() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let started = time::Instant::now();
remote_disk
.execute_read_with_retry(
"read_version",
|| {
let attempt = attempts.fetch_add(1, Ordering::SeqCst);
async move {
if attempt == 0 {
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionReset,
"connection reset",
)));
}
Ok(())
}
},
Duration::ZERO,
)
.await
.expect("zero timeout should allow a retry without a deadline");
assert_eq!(attempts.load(Ordering::SeqCst), 2);
assert_eq!(started.elapsed(), REMOTE_DISK_READ_RETRY_BASE_BACKOFF);
remote_disk.cancel_token.cancel();
}
#[tokio::test]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_accepts_max_metadata_timeout() {
temp_env::async_with_vars([(rustfs_config::ENV_DRIVE_METADATA_TIMEOUT_SECS, Some(u64::MAX.to_string()))], async {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
remote_disk
.execute_read_with_retry("read_version", || async { Ok::<(), Error>(()) }, get_drive_metadata_timeout())
.await
.expect("the maximum configured metadata timeout must not panic");
remote_disk.cancel_token.cancel();
})
.await;
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_zero_retries_runs_once() {
temp_env::async_with_vars([(rustfs_config::ENV_INTERNODE_IDEMPOTENT_READ_RETRIES, Some("0"))], async {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let started = time::Instant::now();
let channel = TonicEndpoint::from_shared(remote_disk.addr.clone())
.expect("remote disk address should parse")
.connect_lazy();
runtime_sources::cache_test_node_channel(remote_disk.addr.clone(), channel).await;
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
attempts.fetch_add(1, Ordering::SeqCst);
async {
Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionReset,
"connection reset",
)))
}
},
Duration::from_secs(1),
)
.await
.expect_err("zero retries should return the first network error");
assert!(matches!(err, DiskError::Io(ref io_err) if io_err.kind() == std_io::ErrorKind::ConnectionReset));
assert_eq!(attempts.load(Ordering::SeqCst), 1);
assert_eq!(started.elapsed(), Duration::ZERO);
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Suspect);
assert!(!runtime_sources::test_node_channel_is_cached(&remote_disk.addr).await);
remote_disk.cancel_token.cancel();
})
.await;
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_attempt_timeout_marks_health_without_evicting() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let recorder = crate::test_metrics::CapturingRecorder::default();
let _recorder_guard = metrics::set_default_local_recorder(&recorder);
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let channel = TonicEndpoint::from_shared(remote_disk.addr.clone())
.expect("remote disk address should parse")
.connect_lazy();
runtime_sources::cache_test_node_channel(remote_disk.addr.clone(), channel).await;
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
attempts.fetch_add(1, Ordering::SeqCst);
std::future::pending::<Result<()>>()
},
Duration::from_millis(100),
)
.await
.expect_err("an in-flight attempt that consumes the deadline should time out");
assert_eq!(err, DiskError::Timeout);
assert_eq!(attempts.load(Ordering::SeqCst), 1);
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Suspect);
assert!(runtime_sources::test_node_channel_is_cached(&remote_disk.addr).await);
assert_eq!(
recorder.counter_value(
"rustfs_drive_op_timeout_total",
&[
("endpoint", remote_disk.endpoint.to_string().as_str()),
("op", "read_version")
]
),
1
);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_does_not_retry_business_errors() {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
attempts.fetch_add(1, Ordering::SeqCst);
async { Err::<(), Error>(DiskError::FileNotFound) }
},
Duration::from_secs(1),
)
.await
.expect_err("business errors should be returned directly");
assert_eq!(err, DiskError::FileNotFound);
assert_eq!(attempts.load(Ordering::SeqCst), 1);
remote_disk.cancel_token.cancel();
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_honors_configured_retry_count() {
temp_env::async_with_vars([(rustfs_config::ENV_INTERNODE_IDEMPOTENT_READ_RETRIES, Some("2"))], async {
let remote_disk = new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await;
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let started = time::Instant::now();
let channel = TonicEndpoint::from_shared(remote_disk.addr.clone())
.expect("remote disk address should parse")
.connect_lazy();
runtime_sources::cache_test_node_channel(remote_disk.addr.clone(), channel).await;
let err = remote_disk
.execute_read_with_retry(
"read_version",
|| {
attempts.fetch_add(1, Ordering::SeqCst);
async {
Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionReset,
"connection reset",
)))
}
},
Duration::from_secs(1),
)
.await
.expect_err("exhausted retries should return the last network error");
assert!(matches!(err, DiskError::Io(ref io_err) if io_err.kind() == std_io::ErrorKind::ConnectionReset));
assert_eq!(attempts.load(Ordering::SeqCst), 3);
assert_eq!(started.elapsed(), Duration::from_millis(150));
assert_eq!(remote_disk.runtime_state(), RuntimeDriveHealthState::Suspect);
assert!(!runtime_sources::test_node_channel_is_cached(&remote_disk.addr).await);
remote_disk.cancel_token.cancel();
})
.await;
}
#[tokio::test(start_paused = true)]
#[serial(remote_disk_read_retry)]
async fn execute_read_with_retry_stops_when_disk_turns_offline_during_backoff() {
let remote_disk = Arc::new(new_remote_disk_with_transport(Arc::new(RecordingInternodeDataTransport::default())).await);
let attempts = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let first_attempt = Arc::new(tokio::sync::Notify::new());
let task_disk = Arc::clone(&remote_disk);
let task_attempts = Arc::clone(&attempts);
let task_first_attempt = Arc::clone(&first_attempt);
let task = tokio::spawn(async move {
task_disk
.execute_read_with_retry(
"read_version",
move || {
let attempt = task_attempts.fetch_add(1, Ordering::SeqCst);
let first_attempt = Arc::clone(&task_first_attempt);
async move {
if attempt == 0 {
first_attempt.notify_one();
return Err::<(), Error>(DiskError::Io(std_io::Error::new(
std_io::ErrorKind::ConnectionReset,
"connection reset",
)));
}
Ok(())
}
},
Duration::from_secs(1),
)
.await
});
first_attempt.notified().await;
tokio::task::yield_now().await;
remote_disk
.health
.force_runtime_state_for_test(RuntimeDriveHealthState::Offline);
time::advance(REMOTE_DISK_READ_RETRY_BASE_BACKOFF).await;
let err = task
.await
.expect("retry task should finish")
.expect_err("an offline disk must stop before the next attempt");
assert_eq!(err, DiskError::FaultyDisk);
assert_eq!(attempts.load(Ordering::SeqCst), 1);
remote_disk.cancel_token.cancel();
}
#[tokio::test]
async fn test_execute_with_timeout_evicts_cached_connection() {
let addr = "http://127.0.0.1:59991".to_string();
@@ -5603,6 +6137,40 @@ mod tests {
accept_task.abort();
}
#[tokio::test]
async fn read_version_uses_the_metadata_timeout_on_a_stalled_peer() {
runtime_sources::ensure_test_rpc_secret();
let Some((base_addr, accept_task)) = spawn_stalled_grpc_peer().await else {
return;
};
let remote_disk = remote_disk_for_addr(&base_addr).await;
temp_env::async_with_vars(
[
(rustfs_config::ENV_DRIVE_METADATA_TIMEOUT_SECS, Some("1")),
(rustfs_config::ENV_DRIVE_MAX_TIMEOUT_DURATION, Some("10")),
],
async {
let started = time::Instant::now();
let err = tokio::time::timeout(
Duration::from_secs(5),
remote_disk.read_version("bucket", "bucket", "object", "", &ReadOptions::default()),
)
.await
.expect("read_version must use the shorter metadata deadline")
.expect_err("a stalled peer must fail read_version");
assert!(matches!(err, DiskError::Timeout), "expected the metadata deadline to fire, got {err:?}");
assert!(started.elapsed() >= Duration::from_millis(900));
assert!(started.elapsed() < Duration::from_secs(2));
},
)
.await;
remote_disk.cancel_token.cancel();
accept_task.abort();
}
#[tokio::test]
async fn delete_volume_bounds_the_wait_on_a_stalled_peer() {
runtime_sources::ensure_test_rpc_secret();
+76 -9
View File
@@ -2266,15 +2266,19 @@ fn decommission_delete_marker_opts(
version: &rustfs_filemeta::FileInfo,
version_id: Option<String>,
src_pool_idx: usize,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> ObjectOptions {
let version_suspended = version.version_id.is_none() && version_id.is_none();
ObjectOptions {
versioned: true,
version_id,
versioned: !version_suspended,
version_suspended,
version_id: version_id.or_else(|| version_suspended.then(|| uuid::Uuid::nil().to_string())),
mod_time: version.mod_time,
src_pool_idx,
data_movement: true,
delete_marker: true,
skip_decommissioned: true,
expected_bucket_incarnation_id,
delete_replication: version
.replication_state_internal
.as_ref()
@@ -2299,6 +2303,7 @@ fn decommission_remote_tiered_opts(
version: &rustfs_filemeta::FileInfo,
version_id: Option<String>,
src_pool_idx: usize,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> ObjectOptions {
ObjectOptions {
versioned: version_id.is_some(),
@@ -2307,6 +2312,9 @@ fn decommission_remote_tiered_opts(
user_defined: version.metadata.clone(),
src_pool_idx,
data_movement: true,
include_part_checksums: true,
http_preconditions: Some(crate::data_movement::data_movement_target_precondition()),
expected_bucket_incarnation_id,
..Default::default()
}
}
@@ -2805,6 +2813,7 @@ impl ECStore {
lifecycle_config: Option<BucketLifecycleConfiguration>,
object_lock_config: Option<ObjectLockConfiguration>,
replication_config: Option<(ReplicationConfiguration, OffsetDateTime)>,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> Result<()> {
debug!(
event = EVENT_DECOMMISSION_ENTRY,
@@ -2834,6 +2843,11 @@ impl ECStore {
}
decommission_cancel_signal_result(rx.is_cancelled())?;
let bucket_incarnation_fence = match expected_bucket_incarnation_id {
Some(expected) => Some(self.acquire_bucket_incarnation_fence(&bucket, expected).await?),
None => None,
};
let mut fivs = load_decommission_entry_exact_versions(&set, &entry, &bucket, "file_info_versions").await?;
fivs.versions
@@ -2894,7 +2908,7 @@ impl ECStore {
.delete_object(
bucket.as_str(),
&version.name,
decommission_delete_marker_opts(version, version_id.clone(), idx),
decommission_delete_marker_opts(version, version_id.clone(), idx, expected_bucket_incarnation_id),
)
.await
{
@@ -2984,7 +2998,7 @@ impl ECStore {
bucket.as_str(),
&version.name,
version,
&decommission_remote_tiered_opts(version, version_id.clone(), idx),
&decommission_remote_tiered_opts(version, version_id.clone(), idx, expected_bucket_incarnation_id),
)
.await
{
@@ -3056,7 +3070,11 @@ impl ECStore {
)
.await?;
if let Err(err) = self.clone().decommission_object(idx, bucket, rd).await {
if let Err(err) = self
.clone()
.decommission_object(idx, bucket, rd, expected_bucket_incarnation_id)
.await
{
if is_decommission_copy_cleanup_safe_error(&err) {
ignore = true;
cleanup_ignored = true;
@@ -3133,6 +3151,9 @@ impl ECStore {
}
if should_cleanup_decommission_source_entry(decommissioned, fivs.versions.len(), expired) {
if bucket_incarnation_fence.as_ref().is_some_and(|guard| guard.is_lock_lost()) {
return Err(Error::other("decommission bucket incarnation fence was lost before source cleanup"));
}
decommission_cancel_signal_result(rx.is_cancelled())?;
self.save_decommission_entry_progress_stage(
@@ -3157,6 +3178,12 @@ impl ECStore {
entry.name.as_str(),
&fivs,
&cleanup_preflight_allowed_missing,
data_movement::SourceCleanupBucketFence {
expected_incarnation_id: expected_bucket_incarnation_id,
lifecycle_guard: bucket_incarnation_fence
.as_ref()
.and_then(|guard| guard.namespace_lock_guard()),
},
"decommission",
)
.await
@@ -3268,6 +3295,11 @@ impl ECStore {
let mut lifecycle_config = None;
let mut object_lock_config = None;
let mut replication_config = None;
let expected_bucket_incarnation_id = if bi.name == RUSTFS_META_BUCKET {
None
} else {
Some(self.bucket_incarnation_id_from_disk(&bi.name).await?)
};
if bi.name != RUSTFS_META_BUCKET {
let _ = resolve_decommission_optional_bucket_config_result(
@@ -3321,6 +3353,7 @@ impl ECStore {
let lifecycle_config = lifecycle_config.clone();
let object_lock_config = object_lock_config.clone();
let replication_config = replication_config.clone();
let expected_bucket_incarnation_id = expected_bucket_incarnation_id;
let entry_error = entry_error.clone();
let callback_rx = callback_rx.clone();
@@ -3383,6 +3416,7 @@ impl ECStore {
lifecycle_config,
object_lock_config,
replication_config,
expected_bucket_incarnation_id,
)
.await
{
@@ -4168,10 +4202,24 @@ impl ECStore {
}
#[tracing::instrument(skip(self, rd))]
async fn decommission_object(self: Arc<Self>, pool_idx: usize, bucket: String, rd: GetObjectReader) -> Result<()> {
async fn decommission_object(
self: Arc<Self>,
pool_idx: usize,
bucket: String,
rd: GetObjectReader,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> Result<()> {
warn!("decommission_object: start {} {}", &bucket, &rd.object_info.name);
let object_name = rd.object_info.name.clone();
let result = data_movement::migrate_object(self, pool_idx, bucket.clone(), rd, "decommission_object").await;
let result = data_movement::migrate_object(
self,
pool_idx,
bucket.clone(),
rd,
expected_bucket_incarnation_id,
"decommission_object",
)
.await;
if result.is_ok() {
warn!("decommission_object: migrated {} {}", &bucket, &object_name);
}
@@ -4347,7 +4395,8 @@ mod tests {
..Default::default()
};
let opts = decommission_delete_marker_opts(&version, Some("version-id".to_string()), 7);
let incarnation = uuid::Uuid::new_v4();
let opts = decommission_delete_marker_opts(&version, Some("version-id".to_string()), 7, Some(incarnation));
let replication = opts.delete_replication.expect("replication state should be preserved");
assert!(opts.versioned);
@@ -4357,11 +4406,25 @@ mod tests {
assert_eq!(opts.src_pool_idx, 7);
assert_eq!(opts.version_id.as_deref(), Some("version-id"));
assert_eq!(opts.mod_time, Some(mod_time));
assert_eq!(opts.expected_bucket_incarnation_id, Some(incarnation));
assert_eq!(replication.replica_status, ReplicationStatusType::Replica);
assert!(replication.delete_marker);
assert_eq!(replication.replicate_decision_str, "existing");
}
#[test]
fn decommission_delete_marker_opts_preserves_suspended_null_version() {
let version = rustfs_filemeta::FileInfo {
deleted: true,
..Default::default()
};
let opts = decommission_delete_marker_opts(&version, None, 7, None);
assert!(!opts.versioned);
assert!(opts.version_suspended);
assert_eq!(opts.version_id.as_deref(), Some(uuid::Uuid::nil().to_string().as_str()));
}
#[test]
fn test_decommission_object_migration_read_opts_are_raw_data_movement() {
let opts = decommission_object_migration_read_opts(Some("vid-1".to_string()));
@@ -4383,7 +4446,8 @@ mod tests {
..Default::default()
};
let opts = decommission_remote_tiered_opts(&version, Some("version-id".to_string()), 9);
let incarnation = uuid::Uuid::new_v4();
let opts = decommission_remote_tiered_opts(&version, Some("version-id".to_string()), 9, Some(incarnation));
assert!(opts.versioned);
assert!(opts.data_movement);
@@ -4391,6 +4455,9 @@ mod tests {
assert_eq!(opts.version_id.as_deref(), Some("version-id"));
assert_eq!(opts.mod_time, Some(mod_time));
assert_eq!(opts.user_defined.get("x-amz-meta-key").map(String::as_str), Some("value"));
assert!(opts.include_part_checksums);
assert!(opts.http_preconditions.is_some());
assert_eq!(opts.expected_bucket_incarnation_id, Some(incarnation));
}
#[test]
File diff suppressed because it is too large Load Diff
+2
View File
@@ -137,6 +137,7 @@ pub(crate) const GET_METADATA_CACHE_REASON_NO_LOCK: &str = "no_lock";
pub(crate) const GET_METADATA_CACHE_REASON_NOT_FOUND_OR_EXPIRED: &str = "not_found_or_expired";
pub(crate) const GET_METADATA_CACHE_REASON_NOT_READ_DATA: &str = "not_read_data";
pub(crate) const GET_METADATA_CACHE_REASON_PART_NUMBER: &str = "part_number";
pub(crate) const GET_METADATA_CACHE_REASON_PART_CHECKSUMS: &str = "part_checksums";
pub(crate) const GET_METADATA_CACHE_REASON_RAW_DATA_MOVEMENT_READ: &str = "raw_data_movement_read";
pub(crate) const GET_METADATA_CACHE_REASON_STALE_PUBLICATION: &str = "stale_publication";
pub(crate) const GET_METADATA_CACHE_REASON_USABLE: &str = "usable";
@@ -480,6 +481,7 @@ mod tests {
assert_eq!(GET_METADATA_CACHE_REASON_NO_LOCK, "no_lock");
assert_eq!(GET_METADATA_CACHE_REASON_NOT_FOUND_OR_EXPIRED, "not_found_or_expired");
assert_eq!(GET_METADATA_CACHE_REASON_NOT_READ_DATA, "not_read_data");
assert_eq!(GET_METADATA_CACHE_REASON_PART_CHECKSUMS, "part_checksums");
assert_eq!(GET_METADATA_CACHE_REASON_PART_NUMBER, "part_number");
assert_eq!(GET_METADATA_CACHE_REASON_RAW_DATA_MOVEMENT_READ, "raw_data_movement_read");
assert_eq!(GET_METADATA_CACHE_REASON_STALE_PUBLICATION, "stale_publication");
+15
View File
@@ -2022,6 +2022,21 @@ impl DiskAPI for LocalDiskWrapper {
.await
}
async fn read_file_stream_chunks(
&self,
volume: &str,
path: &str,
offset: usize,
length: usize,
) -> Result<Option<rustfs_rio::ChunkReaderBox>> {
self.track_disk_health_with_op(
"read_file_stream_chunks",
|| async { self.disk.read_file_stream_chunks(volume, path, offset, length).await },
get_max_timeout_duration(),
)
.await
}
async fn read_file_mmap_copy(&self, volume: &str, path: &str, offset: usize, length: usize) -> Result<bytes::Bytes> {
self.track_disk_health_with_op(
"read_file_mmap_copy",
+6 -26
View File
@@ -113,6 +113,9 @@ pub enum DiskError {
#[error("bit-rot hash algorithm is invalid")]
BitrotHashAlgoInvalid,
/// Never constructed locally by RustFS (only reachable through wire
/// decoding, and no current node sends it). The wire code is kept for
/// cross-version compatibility — do not renumber or remove (backlog#1831).
#[error("Rename across devices not allowed, please fix your backend configuration")]
CrossDeviceLink,
@@ -143,6 +146,9 @@ pub enum DiskError {
#[error("io error {0}")]
Io(#[source] io::Error),
/// Never constructed locally by RustFS (only reachable through wire
/// decoding, and no current node sends it). The wire code is kept for
/// cross-version compatibility — do not renumber or remove (backlog#1831).
#[error("source stalled")]
SourceStalled,
@@ -642,19 +648,6 @@ impl Hash for DiskError {
// is currently commented out to avoid complexity. These can be re-enabled
// when needed for specific disk quorum checking and error aggregation logic.
/// Bitrot errors
#[derive(Debug, thiserror::Error)]
pub enum BitrotErrorType {
#[error("bitrot checksum verification failed")]
BitrotChecksumMismatch { expected: String, got: String },
}
impl From<BitrotErrorType> for DiskError {
fn from(e: BitrotErrorType) -> Self {
DiskError::other(e)
}
}
/// Context wrapper for file access errors
#[derive(Debug, thiserror::Error)]
pub struct FileAccessDeniedWithContext {
@@ -869,19 +862,6 @@ mod tests {
let _disk_error: DiskError = json_error.into();
}
#[test]
fn test_bitrot_error_type() {
let bitrot_error = BitrotErrorType::BitrotChecksumMismatch {
expected: "abc123".to_string(),
got: "def456".to_string(),
};
assert!(bitrot_error.to_string().contains("bitrot checksum verification failed"));
let disk_error: DiskError = bitrot_error.into();
assert!(matches!(disk_error, DiskError::Io(_)));
}
#[test]
fn test_file_access_denied_with_context() {
let path = PathBuf::from("/test/path");
+47 -3
View File
@@ -9174,7 +9174,7 @@ impl DiskAPI for LocalDisk {
if let Some(src_file_path_parent) = src_file_path.parent() {
if src_volume != super::RUSTFS_META_MULTIPART_BUCKET {
let _ = remove_std(src_file_path_parent);
let _ = std::fs::remove_dir(src_file_path_parent);
} else {
let _ = self
.delete_file(&dst_volume_dir, &src_file_path_parent.to_path_buf(), true, false)
@@ -9499,7 +9499,7 @@ impl DiskAPI for LocalDisk {
if let Some(ref cleanup) = cleanup_path {
let _ = self.delete_file(&dst_volume_dir, cleanup, true, false).await;
} else if let Some(parent) = src_file_path.parent() {
let _ = remove_std(parent);
let _ = std::fs::remove_dir(parent);
}
// Heal reuses a version's `data_dir` and lands the rebuilt shard on
@@ -9875,6 +9875,7 @@ impl DiskAPI for LocalDisk {
FileInfoOpts {
data: read_data,
include_free_versions: opts.incl_free_versions,
include_part_checksums: false,
},
)?;
@@ -12439,6 +12440,10 @@ mod test {
.join(RUSTFS_META_TMP_BUCKET)
.join(tmp_object)
.join(new_data_dir.to_string());
let tmp_parent = tmp_data_dir
.parent()
.expect("tmp data dir should have a parent")
.to_path_buf();
fs::create_dir_all(&tmp_data_dir)
.await
.expect("new tmp data dir should be created");
@@ -12450,6 +12455,10 @@ mod test {
disk.rename_data(RUSTFS_META_TMP_BUCKET, tmp_object, new_fi, bucket, object)
.await
.expect("rename_data should commit");
assert!(
!tmp_parent.exists(),
"successful non-inline commit should remove the empty staging parent"
);
// The tmp xl.meta write point uses SyncMode::FileOnly: its parent dir
// ({tmp}/{tmp_object}) must not be fsynced.
@@ -12654,6 +12663,9 @@ mod test {
let tmp_object = "tmp-new-inline";
ensure_test_volume(&disk, bucket).await;
ensure_test_volume(&disk, RUSTFS_META_TMP_BUCKET).await;
let tmp_parent = disk
.get_object_path(RUSTFS_META_TMP_BUCKET, tmp_object)
.expect("tmp parent should resolve");
let _mode = durability_mode_override::set(DurabilityMode::Strict);
let version_id = Uuid::parse_str("99999999-9999-9999-9999-999999999999").expect("version id should parse");
@@ -12662,6 +12674,7 @@ mod test {
disk.rename_data(RUSTFS_META_TMP_BUCKET, tmp_object, new_fi, bucket, object)
.await
.expect("inline rename_data should commit the new object");
assert!(!tmp_parent.exists(), "successful inline commit should remove the empty staging parent");
let bucket_dir = disk.get_bucket_path(bucket).expect("bucket path should resolve");
let prefix_dir = disk.get_object_path(bucket, "prefix").expect("prefix path should resolve");
@@ -12685,6 +12698,34 @@ mod test {
);
}
#[tokio::test]
async fn rename_data_inline_preserves_non_empty_staging_parent() {
use tempfile::tempdir;
let dir = tempdir().expect("temp dir should be created");
let endpoint = Endpoint::try_from(dir.path().to_str().expect("temp dir should be utf8")).expect("endpoint should parse");
let disk = LocalDisk::new(&endpoint, false).await.expect("local disk should be created");
let bucket = "inline-staging-sentinel-bucket";
let object = "inline-object";
let tmp_object = "inline-stage-with-sentinel";
ensure_test_volume(&disk, bucket).await;
ensure_test_volume(&disk, RUSTFS_META_TMP_BUCKET).await;
let tmp_parent = disk
.get_object_path(RUSTFS_META_TMP_BUCKET, tmp_object)
.expect("tmp parent should resolve");
fs::create_dir_all(&tmp_parent).await.expect("tmp parent should be created");
let sentinel = tmp_parent.join("sentinel");
fs::write(&sentinel, b"keep").await.expect("sentinel should be written");
let fi = test_file_info(object, Uuid::new_v4(), None, Some(Bytes::from_static(b"inline-payload")));
disk.rename_data(RUSTFS_META_TMP_BUCKET, tmp_object, fi, bucket, object)
.await
.expect("non-empty staging cleanup must not negate the committed object");
assert_eq!(fs::read(&sentinel).await.expect("sentinel should remain"), b"keep");
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
#[allow(clippy::await_holding_lock)]
@@ -12859,7 +12900,10 @@ mod test {
.expect("non-inline rename_data should commit");
assert!(!replacement_dir.exists(), "the destination object directory must not be replaced");
assert!(staging_parent.exists(), "the guarded staging parent must retain its identity");
assert!(
!staging_parent.exists(),
"successful commit should remove the empty staging parent after releasing its guard"
);
assert!(
!replacement_staging_parent.exists(),
"the staging parent must not be replaced between data and metadata publication"
+26
View File
@@ -65,6 +65,7 @@ use error::{Error, Result};
use local::LocalDisk;
use rustfs_filemeta::{FileInfo, ObjectPartInfo, RawFileInfo};
use rustfs_madmin::info_commands::DiskMetrics;
use rustfs_rio::ChunkReaderBox;
use serde::{Deserialize, Serialize};
use std::{fmt::Debug, path::PathBuf, sync::Arc, time::Duration};
use time::OffsetDateTime;
@@ -427,6 +428,19 @@ impl DiskAPI for Disk {
}
}
async fn read_file_stream_chunks(
&self,
volume: &str,
path: &str,
offset: usize,
length: usize,
) -> Result<Option<ChunkReaderBox>> {
match self {
Disk::Local(_) => Ok(None),
Disk::Remote(remote_disk) => remote_disk.read_file_stream_chunks(volume, path, offset, length).await,
}
}
#[tracing::instrument(level = "trace", skip_all)]
async fn read_file_mmap_copy(&self, volume: &str, path: &str, offset: usize, length: usize) -> Result<Bytes> {
match self {
@@ -865,6 +879,18 @@ pub trait DiskAPI: Debug + Send + Sync + 'static {
async fn read_file(&self, volume: &str, path: &str) -> Result<FileReader>;
async fn read_file_stream(&self, volume: &str, path: &str, offset: usize, length: usize) -> Result<FileReader>;
/// Returns an owned-chunk stream when the backing transport can preserve
/// receive-buffer ownership. `None` retains the ordinary reader path.
async fn read_file_stream_chunks(
&self,
_volume: &str,
_path: &str,
_offset: usize,
_length: usize,
) -> Result<Option<ChunkReaderBox>> {
Ok(None)
}
/// File read using mmap-then-copy on Unix or an efficient read on non-Unix.
async fn read_file_mmap_copy(&self, volume: &str, path: &str, offset: usize, length: usize) -> Result<Bytes>;
@@ -76,6 +76,13 @@ impl ShardBufferPool {
self.buffers[index] = Some(buf);
}
#[cfg(test)]
pub(crate) fn stored_allocation(&self, index: usize) -> Option<(*const u8, usize)> {
self.buffers
.get(index)
.and_then(|buf| buf.as_ref().map(|buf| (buf.as_ptr(), buf.capacity())))
}
#[cfg(test)]
fn stored_capacity(&self, index: usize) -> Option<usize> {
self.buffers.get(index).and_then(|buf| buf.as_ref().map(Vec::capacity))
+526 -9
View File
@@ -14,7 +14,10 @@
use pin_project_lite::pin_project;
use rustfs_utils::HashAlgorithm;
use std::future::poll_fn;
use std::io::IoSlice;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;
use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};
use tracing::error;
@@ -23,6 +26,18 @@ const LOG_COMPONENT_ECSTORE: &str = "ecstore";
const LOG_SUBSYSTEM_ERASURE: &str = "erasure";
const EVENT_BITROT_SHORT_SHARD_READ: &str = "bitrot_short_shard_read";
const EVENT_BITROT_HASH_MISMATCH: &str = "bitrot_hash_mismatch";
const MAX_RETAINED_CHUNKS_PER_BLOCK: usize = 64;
const MAX_CHUNK_POLLS_PER_YIELD: usize = MAX_RETAINED_CHUNKS_PER_BLOCK + 1;
/// Result of polling an optional owned-chunk handoff.
pub enum ShardChunkRead {
/// The source does not support owned-chunk handoff and remains untouched.
Unsupported,
/// The source reached EOF.
Eof,
/// A non-empty chunk containing at most the requested number of bytes.
Chunk(bytes::Bytes),
}
/// A shard source that may already hold its bytes in memory.
///
@@ -42,6 +57,12 @@ pub trait ShardSource: AsyncRead + Send + Sync + Unpin {
fn try_take_block(&mut self, _n: usize) -> Option<bytes::Bytes> {
None
}
/// Polls one owned chunk when the source supports chunk handoff.
/// `Unsupported` must leave the source untouched.
fn poll_read_chunk(self: Pin<&mut Self>, _cx: &mut Context<'_>, _max: usize) -> Poll<std::io::Result<ShardChunkRead>> {
Poll::Ready(Ok(ShardChunkRead::Unsupported))
}
}
/// Borrowed and owned byte slices are ordinary streaming sources: they carry no
@@ -75,6 +96,9 @@ pin_project! {
// contiguous on-disk `[hash][data]` block so both are pulled in a single
// pass; grown lazily and never shrunk.
buf: Vec<u8>,
// Reused owned chunk vector for the remote HTTP fast path. Keeping the
// allocation with the reader avoids allocating once per bitrot block.
chunks: Vec<bytes::Bytes>,
skip_verify: bool,
last_verify_duration: Duration,
}
@@ -91,6 +115,7 @@ where
hash_algo: algo,
shard_size,
buf: Vec::new(),
chunks: Vec::new(),
skip_verify,
last_verify_duration: Duration::ZERO,
}
@@ -100,6 +125,11 @@ where
self.last_verify_duration
}
#[cfg(test)]
pub(crate) fn inner_ref(&self) -> &R {
&self.inner
}
/// Read a single (hash+data) block, verify hash, and copy `out.len()` bytes
/// into `out`. Returns an error if the shard is short, the hash mismatches,
/// or `out` is larger than one shard. On error `out`'s contents are
@@ -260,11 +290,6 @@ where
let need = hash_size + want;
// In-memory fast path: the block is already resident, so slice it instead
// of copying it into the scratch buffer first (rustfs/backlog#1159). One
// copy (`extend_from_slice`) instead of two. A source that cannot serve
// `need` bytes returns `None` and falls through to the scratch path,
// keeping the short-read contract.
if let Some(block) = self.inner.try_take_block(need) {
let (data, verify) = split_and_verify(&self.hash_algo, self.skip_verify, &block)?;
out.extend_from_slice(data);
@@ -272,6 +297,126 @@ where
return Ok(want);
}
self.chunks.clear();
let handed_off = {
let inner = &mut self.inner;
let chunks = &mut self.chunks;
let tail_buf = &mut self.buf;
let mut received = 0usize;
poll_fn(|cx| {
for _ in 0..MAX_CHUNK_POLLS_PER_YIELD {
let next = match Pin::new(&mut *inner).poll_read_chunk(cx, need - received) {
Poll::Ready(Ok(next)) => next,
Poll::Ready(Err(err)) => return Poll::Ready(Err(err)),
Poll::Pending => return Poll::Pending,
};
let chunk = match next {
ShardChunkRead::Unsupported if received == 0 => return Poll::Ready(Ok(false)),
ShardChunkRead::Unsupported => {
return Poll::Ready(Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"chunk handoff became unavailable after transferring data",
)));
}
ShardChunkRead::Eof => {
return Poll::Ready(Err(short_shard_read(received.saturating_sub(hash_size), want)));
}
ShardChunkRead::Chunk(chunk) => chunk,
};
if received == 0 {
tail_buf.clear();
}
if chunk.is_empty() {
return Poll::Ready(Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"chunk handoff returned an empty chunk",
)));
}
let remaining = need - received;
if chunk.len() > remaining {
return Poll::Ready(Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"chunk handoff exceeded its requested boundary",
)));
}
received += chunk.len();
if chunks.len() == MAX_RETAINED_CHUNKS_PER_BLOCK {
if tail_buf.is_empty() {
tail_buf.reserve_exact(need - (received - chunk.len()));
}
tail_buf.extend_from_slice(&chunk);
} else {
chunks.push(chunk);
}
if received == need {
return Poll::Ready(Ok(true));
}
}
cx.waker().wake_by_ref();
Poll::Pending
})
.await?
};
if handed_off {
if self.chunks.len() == 1 && self.buf.is_empty() {
let block = &self.chunks[0];
let (data, verify) = split_and_verify(&self.hash_algo, self.skip_verify, block)?;
out.extend_from_slice(data);
self.last_verify_duration = verify;
return Ok(want);
}
let block_chunks = || {
self.chunks
.iter()
.map(|chunk| chunk.as_ref())
.chain((!self.buf.is_empty()).then_some(self.buf.as_slice()))
};
if !self.skip_verify {
let verify_start = std::time::Instant::now();
let actual_hash = self
.hash_algo
.hash_encode_slices(block_chunks().scan(hash_size, |skip, chunk| {
let start = (*skip).min(chunk.len());
*skip -= start;
Some(&chunk[start..])
}));
let verify = verify_start.elapsed();
let mut hash_offset = 0;
let mut remaining = hash_size;
for chunk in block_chunks() {
let take = remaining.min(chunk.len());
if actual_hash.as_ref()[hash_offset..hash_offset + take] != chunk[..take] {
error!(
event = EVENT_BITROT_HASH_MISMATCH,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_ERASURE,
state = "failed",
data_len = want,
"bitrot hash mismatch"
);
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, "bitrot hash mismatch"));
}
hash_offset += take;
remaining -= take;
if remaining == 0 {
break;
}
}
self.last_verify_duration = verify;
}
let mut skip = hash_size;
for chunk in block_chunks() {
let start = skip.min(chunk.len());
skip -= start;
out.extend_from_slice(&chunk[start..]);
}
return Ok(want);
}
// Streaming path: same single pass and same verification as `read`; only
// the sink differs (`extend_from_slice` into `out` instead of
// `copy_from_slice` into a pre-zeroed buffer).
@@ -677,18 +822,167 @@ impl BitrotWriterWrapper {
#[cfg(test)]
mod tests {
use super::ShardSource;
use super::{
BitrotReader, BitrotWriter, BitrotWriterWrapper, CustomWriter, bitrot_shard_file_size, bitrot_verify, write_all_vectored,
};
use super::{MAX_RETAINED_CHUNKS_PER_BLOCK, ShardChunkRead, ShardSource};
use bytes::Bytes;
use rustfs_utils::HashAlgorithm;
use std::io::{Cursor, IoSlice};
use std::collections::VecDeque;
use std::io::{self, Cursor, IoSlice};
use std::pin::Pin;
use std::sync::{
Arc,
atomic::{AtomicUsize, Ordering},
};
use std::task::{Context, Poll};
use tokio::io::{AsyncWrite, AsyncWriteExt};
use std::time::Duration;
use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt, ReadBuf};
struct FragmentedSource {
chunks: VecDeque<Bytes>,
}
impl FragmentedSource {
fn new(bytes: Vec<u8>, fragment_sizes: &[usize]) -> Self {
let mut chunks = VecDeque::new();
let mut offset = 0;
for &size in fragment_sizes {
let end = (offset + size).min(bytes.len());
if offset < end {
chunks.push_back(Bytes::copy_from_slice(&bytes[offset..end]));
}
offset = end;
}
if offset < bytes.len() {
chunks.push_back(Bytes::copy_from_slice(&bytes[offset..]));
}
Self { chunks }
}
}
impl AsyncRead for FragmentedSource {
fn poll_read(self: Pin<&mut Self>, _cx: &mut Context<'_>, _buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Err(io::Error::other("fragmented source must use chunk handoff")))
}
}
impl ShardSource for FragmentedSource {
fn poll_read_chunk(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, max: usize) -> Poll<io::Result<ShardChunkRead>> {
let Some(mut chunk) = self.chunks.pop_front() else {
return Poll::Ready(Ok(ShardChunkRead::Eof));
};
if chunk.len() > max {
self.chunks.push_front(chunk.split_off(max));
chunk.truncate(max);
}
Poll::Ready(Ok(ShardChunkRead::Chunk(chunk)))
}
}
struct GeneratedChunkSource {
bytes: Bytes,
offset: usize,
fragment_size: usize,
fail_at: Option<usize>,
}
impl GeneratedChunkSource {
fn new(bytes: Vec<u8>, fragment_size: usize) -> Self {
assert!(fragment_size > 0);
Self {
bytes: Bytes::from(bytes),
offset: 0,
fragment_size,
fail_at: None,
}
}
fn failing(bytes: Vec<u8>, fragment_size: usize, fail_at: usize) -> Self {
Self {
fail_at: Some(fail_at),
..Self::new(bytes, fragment_size)
}
}
}
impl AsyncRead for GeneratedChunkSource {
fn poll_read(self: Pin<&mut Self>, _cx: &mut Context<'_>, _buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Err(io::Error::other("generated source must use chunk handoff")))
}
}
impl ShardSource for GeneratedChunkSource {
fn poll_read_chunk(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, max: usize) -> Poll<io::Result<ShardChunkRead>> {
if self.fail_at == Some(self.offset) {
return Poll::Ready(Err(rustfs_rio::new_test_internode_http_io_error(
rustfs_rio::InternodeHttpErrorKind::BodyStreamAborted,
)));
}
if self.offset == self.bytes.len() {
return Poll::Ready(Ok(ShardChunkRead::Eof));
}
let error_limit = self.fail_at.unwrap_or(self.bytes.len());
let take = self
.fragment_size
.min(max)
.min(error_limit - self.offset)
.min(self.bytes.len() - self.offset);
let start = self.offset;
self.offset += take;
Poll::Ready(Ok(ShardChunkRead::Chunk(self.bytes.slice(start..start + take))))
}
}
struct InvalidChunkSource {
mode: InvalidChunkMode,
}
#[derive(Clone, Copy)]
enum InvalidChunkMode {
Empty,
Oversized,
UnsupportedAfterChunk,
Unsupported,
}
impl AsyncRead for InvalidChunkSource {
fn poll_read(self: Pin<&mut Self>, _cx: &mut Context<'_>, _buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Err(io::Error::other("invalid source must use chunk handoff")))
}
}
impl ShardSource for InvalidChunkSource {
fn poll_read_chunk(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, max: usize) -> Poll<io::Result<ShardChunkRead>> {
match self.mode {
InvalidChunkMode::Empty => Poll::Ready(Ok(ShardChunkRead::Chunk(Bytes::new()))),
InvalidChunkMode::Oversized => Poll::Ready(Ok(ShardChunkRead::Chunk(Bytes::from(vec![0; max + 1])))),
InvalidChunkMode::UnsupportedAfterChunk => {
self.mode = InvalidChunkMode::Unsupported;
Poll::Ready(Ok(ShardChunkRead::Chunk(Bytes::from_static(b"x"))))
}
InvalidChunkMode::Unsupported => Poll::Ready(Ok(ShardChunkRead::Unsupported)),
}
}
}
struct ScratchReuseSource {
block: Option<Bytes>,
saw_reused_scratch: bool,
}
impl AsyncRead for ScratchReuseSource {
fn poll_read(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
let Some(block) = self.block.take() else {
return Poll::Ready(Ok(()));
};
self.saw_reused_scratch = buf.initialize_unfilled()[..block.len()].iter().all(|byte| *byte == 0xa5);
buf.put_slice(&block);
Poll::Ready(Ok(()))
}
}
impl ShardSource for ScratchReuseSource {}
#[derive(Default)]
struct VectoredCountingWriter {
@@ -1446,6 +1740,70 @@ mod tests {
assert!(out.is_empty(), "corrupt bytes must never reach the caller's buffer");
}
#[tokio::test]
async fn chunked_handoff_verifies_data_split_across_hash_boundaries() {
const SHARD: usize = 4096;
let algo = HashAlgorithm::HighwayHash256S;
let data: Vec<u8> = (0..SHARD).map(|index| (index % 251) as u8).collect();
let mut encoded = Vec::new();
BitrotWriter::new(&mut encoded, SHARD, algo.clone())
.write(&data)
.await
.expect("write shard");
let mut output = Vec::with_capacity(SHARD);
BitrotReader::new(FragmentedSource::new(encoded, &[3, 11, 19, 37, 128]), SHARD, algo, false)
.read_appending(&mut output, SHARD)
.await
.expect("fragmented shard must verify");
assert_eq!(output, data);
}
#[tokio::test]
async fn chunked_handoff_never_appends_a_corrupt_shard() {
const SHARD: usize = 4096;
let algo = HashAlgorithm::HighwayHash256S;
let mut encoded = Vec::new();
BitrotWriter::new(&mut encoded, SHARD, algo.clone())
.write(&vec![9u8; SHARD])
.await
.expect("write shard");
let last = encoded.len() - 1;
encoded[last] ^= 0xff;
let mut output = Vec::with_capacity(SHARD);
let err = BitrotReader::new(FragmentedSource::new(encoded, &[7, 17, 31]), SHARD, algo, false)
.read_appending(&mut output, SHARD)
.await
.expect_err("corrupt fragmented shard must fail");
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
assert!(output.is_empty());
}
#[tokio::test]
async fn chunked_handoff_does_not_hash_when_verification_is_skipped() {
const SHARD: usize = 4096;
let algo = HashAlgorithm::HighwayHash256S;
let mut encoded = Vec::new();
BitrotWriter::new(&mut encoded, SHARD, algo.clone())
.write(&vec![9u8; SHARD])
.await
.expect("write shard");
encoded[0] ^= 0xff;
let mut output = Vec::with_capacity(SHARD);
let mut reader = BitrotReader::new(FragmentedSource::new(encoded, &[7, 17, 31]), SHARD, algo, true);
reader
.read_appending(&mut output, SHARD)
.await
.expect("skipped verification must accept fragmented shard bytes");
assert_eq!(reader.last_verify_duration(), Duration::ZERO);
assert_eq!(output, vec![9u8; SHARD]);
}
#[tokio::test]
async fn read_appending_rejects_a_want_larger_than_the_shard() {
let algo = HashAlgorithm::HighwayHash256;
@@ -1497,10 +1855,21 @@ mod tests {
// Equivalence: same bytes out of both paths.
let mut via_mem: Vec<u8> = Vec::with_capacity(SHARD);
BitrotReader::new(Cursor::new(Bytes::from(encoded.clone())), SHARD, algo.clone(), false)
let mut memory_reader = BitrotReader::new(Cursor::new(Bytes::from(encoded.clone())), SHARD, algo.clone(), false);
memory_reader
.read_appending(&mut via_mem, SHARD)
.await
.expect("in-memory read");
assert_eq!(
memory_reader.chunks.capacity(),
0,
"the synchronous fast path must not allocate chunk storage"
);
assert_eq!(
memory_reader.buf.capacity(),
0,
"the synchronous fast path must not allocate scratch storage"
);
let mut via_stream: Vec<u8> = Vec::with_capacity(SHARD);
BitrotReader::new(Cursor::new(encoded), SHARD, algo, false)
@@ -1537,4 +1906,152 @@ mod tests {
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
assert!(out.is_empty(), "corrupt bytes must never reach the caller's buffer");
}
#[tokio::test]
async fn streaming_fallback_reuses_initialized_scratch() {
const SHARD: usize = 4096;
let algo = HashAlgorithm::HighwayHash256S;
let data = vec![7u8; SHARD];
let encoded = encode_one_block(&data, SHARD, algo.clone()).await;
let source = ScratchReuseSource {
block: Some(Bytes::copy_from_slice(&encoded)),
saw_reused_scratch: false,
};
let mut reader = BitrotReader::new(source, SHARD, algo, false);
reader.buf = vec![0xa5; encoded.len()];
let mut output = Vec::new();
reader
.read_appending(&mut output, SHARD)
.await
.expect("streaming fallback should verify");
assert!(reader.inner.saw_reused_scratch, "capability probing must not clear reusable scratch");
assert_eq!(output, data);
}
#[tokio::test]
async fn chunked_handoff_bounds_production_sized_one_byte_fragments() {
const SHARD: usize = 1024 * 1024 / 4;
let algo = HashAlgorithm::HighwayHash256S;
let data: Vec<u8> = (0..SHARD).map(|index| (index % 251) as u8).collect();
let encoded = encode_one_block(&data, SHARD, algo.clone()).await;
let encoded_len = encoded.len();
let mut reader = BitrotReader::new(GeneratedChunkSource::new(encoded, 1), SHARD, algo, false);
let mut output = Vec::with_capacity(SHARD);
reader
.read_appending(&mut output, SHARD)
.await
.expect("one-byte fragments should verify with bounded retained state");
assert_eq!(output, data);
assert_eq!(reader.chunks.len(), MAX_RETAINED_CHUNKS_PER_BLOCK);
assert!(reader.chunks.capacity() <= MAX_RETAINED_CHUNKS_PER_BLOCK);
assert_eq!(reader.buf.len(), encoded_len - MAX_RETAINED_CHUNKS_PER_BLOCK);
}
#[tokio::test]
async fn chunked_handoff_keeps_sixty_four_frames_zero_copy_and_respects_poll_budget() {
const SHARD: usize = 1024 * 1024;
const FRAME: usize = 16 * 1024;
let algo = HashAlgorithm::HighwayHash256S;
let small_data = vec![3u8; 4096];
let small_encoded = encode_one_block(&small_data, 4096, algo.clone()).await;
let mut exact_reader =
BitrotReader::new(FragmentedSource::new(small_encoded.clone(), &[1; 63]), 4096, algo.clone(), false);
let mut exact_output = Vec::new();
exact_reader
.read_appending(&mut exact_output, 4096)
.await
.expect("exactly sixty-four frames should verify");
assert_eq!(exact_output, small_data);
assert_eq!(exact_reader.chunks.len(), MAX_RETAINED_CHUNKS_PER_BLOCK);
assert!(exact_reader.buf.is_empty(), "the threshold itself must remain zero-copy");
let mut yielded_reader = BitrotReader::new(FragmentedSource::new(small_encoded, &[1; 65]), 4096, algo.clone(), false);
let mut yielded_output = Vec::new();
let mut yielded_read = Box::pin(yielded_reader.read_appending(&mut yielded_output, 4096));
let mut cx = Context::from_waker(std::task::Waker::noop());
assert!(std::future::Future::poll(yielded_read.as_mut(), &mut cx).is_pending());
assert!(matches!(std::future::Future::poll(yielded_read.as_mut(), &mut cx), Poll::Ready(Ok(4096))));
drop(yielded_read);
assert_eq!(yielded_output, small_data);
let data = vec![7u8; SHARD];
let encoded = encode_one_block(&data, SHARD, algo.clone()).await;
let mut reader = BitrotReader::new(FragmentedSource::new(encoded, &[FRAME; 64]), SHARD, algo, false);
let mut output = Vec::with_capacity(SHARD);
let mut read = Box::pin(reader.read_appending(&mut output, SHARD));
assert!(
matches!(std::future::Future::poll(read.as_mut(), &mut cx), Poll::Ready(Ok(SHARD))),
"sixty-five normal HTTP frames should complete without a cooperative yield"
);
drop(read);
assert_eq!(output, data);
assert_eq!(reader.chunks.len(), MAX_RETAINED_CHUNKS_PER_BLOCK);
assert_eq!(reader.buf.len(), HashAlgorithm::HighwayHash256S.size());
}
#[tokio::test]
async fn chunked_tail_failures_preserve_errors_and_output() {
const SHARD: usize = 4096;
let algo = HashAlgorithm::HighwayHash256S;
let data = vec![7u8; SHARD];
let encoded = encode_one_block(&data, SHARD, algo.clone()).await;
let sentinel = vec![1u8, 2, 3];
let mut short_output = sentinel.clone();
let short_err = BitrotReader::new(GeneratedChunkSource::new(encoded[..100].to_vec(), 1), SHARD, algo.clone(), false)
.read_appending(&mut short_output, SHARD)
.await
.expect_err("EOF after the retention threshold must stay a short read");
assert_eq!(short_err.kind(), io::ErrorKind::UnexpectedEof);
assert_eq!(short_output, sentinel);
let mut corrupt = encoded.clone();
let last = corrupt.len() - 1;
corrupt[last] ^= 0xff;
let mut corrupt_output = sentinel.clone();
let corrupt_err = BitrotReader::new(GeneratedChunkSource::new(corrupt, 1), SHARD, algo.clone(), false)
.read_appending(&mut corrupt_output, SHARD)
.await
.expect_err("corrupt coalesced tail must fail verification");
assert_eq!(corrupt_err.kind(), io::ErrorKind::InvalidData);
assert_eq!(corrupt_output, sentinel);
let mut failed_output = sentinel.clone();
let body_err = BitrotReader::new(GeneratedChunkSource::failing(encoded, 1, 65), SHARD, algo, false)
.read_appending(&mut failed_output, SHARD)
.await
.expect_err("a terminal body error must not become EOF");
let source = body_err
.get_ref()
.and_then(|source| source.downcast_ref::<rustfs_rio::InternodeHttpError>())
.expect("body error should retain internode classification");
assert_eq!(source.kind(), rustfs_rio::InternodeHttpErrorKind::BodyStreamAborted);
assert_eq!(failed_output, sentinel);
}
#[tokio::test]
async fn chunked_handoff_rejects_invalid_source_contracts() {
const SHARD: usize = 64;
for mode in [
InvalidChunkMode::Empty,
InvalidChunkMode::Oversized,
InvalidChunkMode::UnsupportedAfterChunk,
] {
let source = InvalidChunkSource { mode };
let mut output = vec![9u8];
let err = BitrotReader::new(source, SHARD, HashAlgorithm::HighwayHash256S, false)
.read_appending(&mut output, SHARD)
.await
.expect_err("invalid chunk contracts must fail closed");
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
assert_eq!(output, vec![9u8]);
}
}
}
+107 -32
View File
@@ -25,7 +25,9 @@ use crate::disk::error_reduce::reduce_errs;
use crate::erasure::codec::workspace::ShardBufferPool;
use crate::erasure::coding::{BitrotReader, Erasure};
use crate::io_support::bitrot::DeferredReaderStripeHandle;
use crate::set_disk::shard_source::{ShardReadCost, ShardStripeSource, StripeReadState};
use crate::set_disk::shard_source::{
INLINE_SHARD_SLOTS, ShardBuffers, ShardErrors, ShardReadCost, ShardStripeSource, StripeReadState,
};
use futures::FutureExt;
use futures::stream::{FuturesUnordered, StreamExt};
use pin_project_lite::pin_project;
@@ -41,9 +43,6 @@ use tracing::{debug, error, warn};
type ShardReadFuture<'a> = Pin<Box<dyn Future<Output = (usize, ShardReadCost, Result<Vec<u8>, Error>, bool)> + Send + 'a>>;
const INLINE_SHARD_SLOTS: usize = 32;
type ShardBuffers = SmallVec<[Option<Vec<u8>>; INLINE_SHARD_SLOTS]>;
type ShardErrors = SmallVec<[Option<Error>; INLINE_SHARD_SLOTS]>;
type ShardIndexes = SmallVec<[usize; INLINE_SHARD_SLOTS]>;
type ActiveReaders = SmallVec<[bool; INLINE_SHARD_SLOTS]>;
@@ -392,6 +391,7 @@ pub(crate) struct ParallelReader<R> {
// Request-scoped shard buffers keyed by shard index. Keeping ownership in
// `ParallelReader` avoids dropping unused parity/backup slot buffers between stripes.
buffers: ShardBufferPool,
stripe_state: Option<Box<StripeReadState>>,
// Lockstep-path state (verify_reconstruction == true). `engaged[i]` marks
// readers that participate in each stripe read: all data slots from the
// start, parity slots only once a data shard is missing/dead. Unengaged
@@ -596,6 +596,7 @@ where
verify_reconstruction,
locality_preference_enabled: get_shard_locality_preference_enabled(),
buffers: ShardBufferPool::new(e.data_shards + e.parity_shards),
stripe_state: None,
engaged,
deferred_handles: Vec::new(),
stripe_index: 0,
@@ -700,6 +701,12 @@ where
{
#[hotpath::measure(impl_type = "ParallelReader")]
pub async fn read(&mut self) -> StripeReadOutput {
let mut state = StripeReadState::with_slot_count(self.readers.len(), self.data_shards);
self.read_into_state(&mut state).await;
state.into_parts()
}
async fn read_into_state(&mut self, state: &mut StripeReadState) {
// On the reconstruction-verifying GET path, read every live shard reader
// in lockstep so all readers advance one block per stripe and stay
// mutually aligned. The adaptive data-first path below only reads
@@ -709,12 +716,14 @@ where
// than the data shards, producing "inconsistent read source shards" and
// truncating large-object GETs under concurrency (backlog#832).
if self.verify_reconstruction {
return self.read_lockstep().await;
self.read_lockstep(state).await;
return;
}
// if self.readers.len() != self.total_shards {
// return Err(io::Error::new(ErrorKind::InvalidInput, "Invalid number of readers"));
// }
let num_readers = self.readers.len();
state.reset(num_readers, self.data_shards);
let shard_size = if self.offset + self.shard_size > self.shard_file_size {
self.shard_file_size - self.offset
@@ -723,7 +732,7 @@ where
};
if shard_size == 0 {
return (smallvec![None; num_readers], smallvec![None; num_readers]);
return;
}
// Advance to the next stripe so the following read() computes the correct
@@ -734,8 +743,7 @@ where
// is only read above to derive `shard_size`, so advancing here is safe.
self.offset += shard_size;
let mut shards: ShardBuffers = smallvec![None; num_readers];
let mut errs: ShardErrors = smallvec![None; num_readers];
let (shards, errs) = state.parts_mut();
let read_costs = self.read_costs.as_slice();
let locality_preference_enabled = self.locality_preference_enabled;
let low_cost_available = self
@@ -882,8 +890,8 @@ where
}
let result_is_err = record_shard_read_result(
&mut shards,
&mut errs,
shards,
errs,
&mut retire_readers,
&mut success,
&mut successful_costs,
@@ -944,8 +952,8 @@ where
active_readers[i] = false;
completed += 1;
if record_shard_read_result(
&mut shards,
&mut errs,
shards,
errs,
&mut retire_readers,
&mut success,
&mut successful_costs,
@@ -957,7 +965,7 @@ where
failed += 1;
}
}
retire_abandoned_readers(&mut errs, &mut retire_readers, &active_readers);
retire_abandoned_readers(errs, &mut retire_readers, &active_readers);
}
if let Some(path) = self.metrics_path {
@@ -1001,8 +1009,6 @@ where
for i in retire_readers {
self.readers[i] = None;
}
(shards, errs)
}
/// Lockstep stripe read for the reconstruction-verifying GET path.
@@ -1030,18 +1036,18 @@ where
/// stripe would reintroduce the desync. A parity reader that cannot be
/// realigned (no pending deferred handle) is likewise retired instead of
/// being read out of position.
async fn read_lockstep(&mut self) -> StripeReadOutput {
async fn read_lockstep(&mut self, state: &mut StripeReadState) {
let num_readers = self.readers.len();
state.reset(num_readers, self.data_shards);
let shard_size = if self.offset + self.shard_size > self.shard_file_size {
self.shard_file_size - self.offset
} else {
self.shard_size
};
let mut shards: ShardBuffers = smallvec![None; num_readers];
let mut errs: ShardErrors = smallvec![None; num_readers];
let (shards, errs) = state.parts_mut();
if shard_size == 0 {
return (shards, errs);
return;
}
// Advance to the next stripe (see the matching note in `read`); the
@@ -1279,8 +1285,6 @@ where
for i in retire_readers {
self.readers[i] = None;
}
(shards, errs)
}
/// Attempt to bring an as-yet-unread parity reader into the lockstep read
@@ -1337,10 +1341,20 @@ impl<R> ShardStripeSource for ParallelReader<R>
where
R: crate::erasure::coding::ShardSource,
{
async fn read_next_stripe(&mut self) -> StripeReadState {
let read_quorum = self.data_shards;
let (shards, errors) = ParallelReader::read(self).await;
StripeReadState::from_parts_with_read_costs(shards, errors, &self.read_costs, read_quorum)
async fn read_next_stripe(&mut self) -> Box<StripeReadState> {
let mut state = self
.stripe_state
.take()
.unwrap_or_else(|| Box::new(StripeReadState::with_slot_count(self.readers.len(), self.data_shards)));
self.read_into_state(&mut state).await;
state
}
fn recycle_stripe(&mut self, mut state: Box<StripeReadState>) {
self.recycle_shards(state.shards_mut());
state.reset(0, self.data_shards);
debug_assert!(self.stripe_state.is_none(), "a stripe cannot be recycled twice");
self.stripe_state = Some(state);
}
}
@@ -1972,13 +1986,18 @@ mod tests {
type BoxedShardReader = crate::io_support::bitrot::ShardReader;
#[test]
fn shard_scratch_stays_inline_through_the_common_limit_and_spills_safely() {
let inline: ShardBuffers = smallvec![None; INLINE_SHARD_SLOTS];
assert!(!inline.spilled(), "the common shard-count boundary must not allocate");
let spilled: ShardBuffers = smallvec![None; INLINE_SHARD_SLOTS + 1];
assert!(spilled.spilled(), "larger supported shard counts must fall back to the heap");
assert_eq!(spilled.len(), INLINE_SHARD_SLOTS + 1);
fn parallel_reader_keeps_stripe_scratch_out_of_line() {
eprintln!(
"parallel_reader={} stripe_state={} cached_state={}",
std::mem::size_of::<ParallelReader<Cursor<Vec<u8>>>>(),
std::mem::size_of::<StripeReadState>(),
std::mem::size_of::<Option<Box<StripeReadState>>>()
);
assert_eq!(
std::mem::size_of::<Option<Box<StripeReadState>>>(),
std::mem::size_of::<usize>(),
"the request-scoped cache must remain pointer-sized",
);
}
#[tokio::test]
@@ -1997,6 +2016,62 @@ mod tests {
assert_eq!(errors.len(), TOTAL_SHARDS);
}
#[tokio::test]
async fn codec_reader_reuses_inline_and_spilled_stripe_scratch_between_reads() {
for total_shards in [INLINE_SHARD_SLOTS, INLINE_SHARD_SLOTS + 1] {
let data_shards = total_shards - 1;
let readers = std::iter::repeat_with(|| None).take(total_shards).collect();
let erasure = Erasure::new(data_shards, 1, data_shards * 2);
let mut reader: ParallelReader<Cursor<Vec<u8>>> = ParallelReader::new(readers, erasure, 0, data_shards * 2);
let first = ShardStripeSource::read_next_stripe(&mut reader).await;
let first_state = (&*first) as *const StripeReadState;
let first_storage = first.scratch_storage();
assert_eq!(first_storage.2, total_shards > INLINE_SHARD_SLOTS);
assert_eq!(first_storage.3, total_shards > INLINE_SHARD_SLOTS);
ShardStripeSource::recycle_stripe(&mut reader, first);
let second = ShardStripeSource::read_next_stripe(&mut reader).await;
let second_storage = second.scratch_storage();
assert_eq!(
(&*second) as *const StripeReadState,
first_state,
"the request-scoped state must be reused"
);
assert_eq!(second_storage.0, first_storage.0, "shard slots must reuse their allocation");
assert_eq!(second_storage.1, first_storage.1, "error slots must reuse their allocation");
assert_eq!(second.into_parts().0.len(), total_shards);
}
}
#[tokio::test]
async fn codec_reader_returns_shard_allocations_to_the_request_pool() {
const SHARD_SIZE: usize = 16;
let hash_algo = HashAlgorithm::None;
let readers = vec![Some(create_reader(SHARD_SIZE, 2, 0x5a, &hash_algo, false).await)];
let erasure = Erasure::new(1, 0, SHARD_SIZE);
let mut reader = ParallelReader::new(readers, erasure, 0, SHARD_SIZE * 2);
let first = ShardStripeSource::read_next_stripe(&mut reader).await;
let first_allocation = first
.shard_allocation(0)
.expect("the first stripe should own its shard allocation");
ShardStripeSource::recycle_stripe(&mut reader, first);
assert_eq!(
reader.buffers.stored_allocation(0),
Some(first_allocation),
"recycling a stripe must return its shard allocation to the request pool"
);
let second = ShardStripeSource::read_next_stripe(&mut reader).await;
assert_eq!(
second.shard_allocation(0),
Some(first_allocation),
"the next stripe must reuse the pooled shard allocation"
);
}
/// Counts the raw bytes pulled from a shard stream, to prove which shards
/// a decode path actually touches (backlog#923 call-count evidence).
struct CountingShardReader {
@@ -65,7 +65,7 @@ enum FillPolicy {
}
impl FillPolicy {
fn from_env() -> Self {
fn load() -> Self {
match rustfs_utils::get_env_usize(
ENV_RUSTFS_GET_CODEC_STREAMING_MAX_INFLIGHT,
DEFAULT_RUSTFS_GET_CODEC_STREAMING_MAX_INFLIGHT,
@@ -75,6 +75,22 @@ impl FillPolicy {
}
}
fn from_env() -> Self {
#[cfg(test)]
{
Self::load()
}
#[cfg(not(test))]
{
Self::cached_core(Self::load)
}
}
fn cached_core(load: impl FnOnce() -> Self) -> Self {
static CACHED: std::sync::OnceLock<FillPolicy> = std::sync::OnceLock::new();
*CACHED.get_or_init(load)
}
const fn max_inflight(self) -> usize {
match self {
Self::SingleInFlight => 1,
@@ -479,22 +495,30 @@ where
let mut deferred_error = None;
let fill_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let stripe_read_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let state = source.read_next_stripe().await;
let mut state = source.read_next_stripe().await;
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_STRIPE_READ, stripe_read_stage_start);
let decode_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let mut output_buf = reusable_buffers.pop().unwrap_or_default();
let result =
match decode_stripe_into(metrics_path, stage_metrics_enabled, engine, workspace, state, remaining, &mut output_buf) {
Ok(true) => Ok(Some(output_buf)),
Ok(false) => {
reusable_buffers.push(output_buf);
Ok(None)
}
Err(err) => {
reusable_buffers.push(output_buf);
Err(err)
}
};
let result = match decode_stripe_into(
metrics_path,
stage_metrics_enabled,
engine,
workspace,
&mut state,
remaining,
&mut output_buf,
) {
Ok(true) => Ok(Some(output_buf)),
Ok(false) => {
reusable_buffers.push(output_buf);
Ok(None)
}
Err(err) => {
reusable_buffers.push(output_buf);
Err(err)
}
};
source.recycle_stripe(state);
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_DECODE, decode_stage_start);
if let Ok(Some(first_buf)) = result.as_ref() {
let mut remaining_after_first = remaining.saturating_sub(first_buf.len());
@@ -503,7 +527,7 @@ where
break;
}
let stripe_read_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let state = source.read_next_stripe().await;
let mut state = source.read_next_stripe().await;
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_STRIPE_READ, stripe_read_stage_start);
let decode_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let mut queued_buf = reusable_buffers.pop().unwrap_or_default();
@@ -512,10 +536,11 @@ where
stage_metrics_enabled,
engine,
workspace,
state,
&mut state,
remaining_after_first,
&mut queued_buf,
);
source.recycle_stripe(state);
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_DECODE, decode_stage_start);
match queued_result {
Ok(true) => {
@@ -717,7 +742,7 @@ fn decode_stripe_into<E>(
stage_metrics_enabled: bool,
engine: &E,
workspace: &mut E::Workspace,
state: StripeReadState,
state: &mut StripeReadState,
remaining: usize,
output: &mut Vec<u8>,
) -> io::Result<bool>
@@ -725,7 +750,7 @@ where
E: ErasureDecodeEngine,
{
output.clear();
if state.slots().is_empty() {
if state.is_empty() {
return Ok(false);
}
if !state.can_decode() {
@@ -741,13 +766,12 @@ where
);
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_RECONSTRUCT, reconstruct_stage_start);
let emit_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
emit_data_shards_into(&state, engine.data_shards(), engine.block_size(), remaining, output)?;
emit_data_shards_into(state, engine.data_shards(), engine.block_size(), remaining, output)?;
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_EMIT, emit_stage_start);
return Ok(true);
}
let (mut shards, _errs) = state.into_parts();
let reconstruct_outcome = match engine.reconstruct_into(&mut shards, workspace) {
let reconstruct_outcome = match engine.reconstruct_into(state.shards_mut(), workspace) {
Ok(outcome) => outcome,
Err(err) => {
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_RECONSTRUCT, reconstruct_stage_start);
@@ -757,7 +781,7 @@ where
rustfs_io_metrics::record_get_object_reconstruct_outcome(metrics_path, engine.engine_name(), reconstruct_outcome);
record_get_stage_duration_if_enabled(metrics_path, GET_STAGE_RECONSTRUCT, reconstruct_stage_start);
if shards.len() < engine.data_shards() {
if state.shards_mut().len() < engine.data_shards() {
return Err(io::Error::new(
ErrorKind::UnexpectedEof,
"decoded stripe has fewer shards than data shard count",
@@ -766,7 +790,7 @@ where
let emit_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
reserve_output_capacity(output, engine.block_size().min(remaining));
for shard in shards.iter().take(engine.data_shards()) {
for shard in state.shards_mut().iter().take(engine.data_shards()) {
if output.len() >= remaining {
break;
}
@@ -806,10 +830,7 @@ fn emit_data_shards_into(
if output.len() >= remaining {
break;
}
let Some(slot) = state.slot_by_index(index) else {
return Err(io::Error::new(ErrorKind::UnexpectedEof, "decoded stripe is missing a data shard"));
};
let Some(shard) = slot.data_bytes() else {
let Some(shard) = state.data_bytes(index) else {
return Err(io::Error::new(ErrorKind::UnexpectedEof, "decoded stripe is missing a data shard"));
};
let copy_len = shard.len().min(remaining - output.len());
@@ -826,7 +847,7 @@ mod tests {
};
use crate::erasure::coding::decode::ParallelReader;
use crate::erasure::coding::{BitrotReader, BitrotWriter, Erasure};
use crate::set_disk::shard_source::{ShardSlot, StripeReadState};
use crate::set_disk::shard_source::StripeReadState;
use rustfs_utils::HashAlgorithm;
use std::collections::VecDeque;
use std::future::{pending, poll_fn};
@@ -845,6 +866,13 @@ mod tests {
read_count: Option<Arc<AtomicUsize>>,
}
struct RecordingStripeSource {
stripes: VecDeque<StripeReadState>,
read_quorum: usize,
reads: usize,
recycles: usize,
}
struct BlockingSource {
started: Arc<Notify>,
dropped: Arc<AtomicUsize>,
@@ -899,25 +927,43 @@ mod tests {
#[async_trait::async_trait]
impl ShardStripeSource for VecStripeSource {
async fn read_next_stripe(&mut self) -> StripeReadState {
async fn read_next_stripe(&mut self) -> Box<StripeReadState> {
if let Some(read_count) = &self.read_count {
read_count.fetch_add(1, Ordering::SeqCst);
}
self.stripes
.pop_front()
.unwrap_or_else(|| StripeReadState::new(Vec::new(), self.read_quorum))
Box::new(
self.stripes
.pop_front()
.unwrap_or_else(|| StripeReadState::from_parts(Vec::new(), Vec::new(), self.read_quorum)),
)
}
}
#[async_trait::async_trait]
impl ShardStripeSource for RecordingStripeSource {
async fn read_next_stripe(&mut self) -> Box<StripeReadState> {
self.reads += 1;
Box::new(
self.stripes
.pop_front()
.unwrap_or_else(|| StripeReadState::from_parts(Vec::new(), Vec::new(), self.read_quorum)),
)
}
fn recycle_stripe(&mut self, _state: Box<StripeReadState>) {
self.recycles += 1;
}
}
#[async_trait::async_trait]
impl ShardStripeSource for BlockingSource {
async fn read_next_stripe(&mut self) -> StripeReadState {
async fn read_next_stripe(&mut self) -> Box<StripeReadState> {
let _guard = BlockingSourceDropGuard {
dropped: Arc::clone(&self.dropped),
};
self.started.notify_one();
pending::<()>().await;
StripeReadState::new(Vec::new(), self.read_quorum)
Box::new(StripeReadState::from_parts(Vec::new(), Vec::new(), self.read_quorum))
}
}
@@ -1090,6 +1136,23 @@ mod tests {
});
}
#[test]
fn fill_policy_production_cache_loads_once() {
use std::cell::Cell;
let loads = Cell::new(0);
for _ in 0..3 {
assert_eq!(
FillPolicy::cached_core(|| {
loads.set(loads.get() + 1);
FillPolicy::DualInFlight
}),
FillPolicy::DualInFlight
);
}
assert_eq!(loads.get(), 1, "the production fill policy must not re-read the environment per reader");
}
#[test]
fn erasure_decode_reader_rejects_invalid_engine_shape() {
let source = VecStripeSource {
@@ -1689,7 +1752,10 @@ mod tests {
.pop_front()
.expect("first stripe should exist");
let mut source = VecStripeSource {
stripes: VecDeque::from([first_state, StripeReadState::new(Vec::new(), erasure.data_shards)]),
stripes: VecDeque::from([
first_state,
StripeReadState::from_parts(Vec::new(), Vec::new(), erasure.data_shards),
]),
read_quorum: erasure.data_shards,
read_count: None,
};
@@ -1724,13 +1790,14 @@ mod tests {
.stripes
.pop_front()
.expect("first stripe should exist");
let mut source = VecStripeSource {
let mut source = RecordingStripeSource {
stripes: VecDeque::from([
first_state,
StripeReadState::new(vec![ShardSlot::data(0, vec![1])], erasure.data_shards),
StripeReadState::from_parts(vec![Some(vec![1])], Vec::new(), erasure.data_shards),
]),
read_quorum: erasure.data_shards,
read_count: None,
reads: 0,
recycles: 0,
};
let engine = LegacyEcDecodeEngine::new(erasure);
let mut workspace = engine.prepare_workspace(4).expect("workspace should be prepared");
@@ -1756,6 +1823,8 @@ mod tests {
.kind(),
ErrorKind::Other
);
assert_eq!(source.reads, 2, "the fill must read the primary and queued stripe");
assert_eq!(source.recycles, source.reads, "every completed stripe read must be recycled");
}
#[tokio::test]
@@ -1768,7 +1837,7 @@ mod tests {
.stripes
.pop_front()
.expect("first stripe should exist"),
StripeReadState::new(Vec::new(), erasure.data_shards),
StripeReadState::from_parts(Vec::new(), Vec::new(), erasure.data_shards),
]),
read_quorum: erasure.data_shards,
read_count: None,
@@ -2028,17 +2097,11 @@ mod tests {
}
#[test]
fn emit_data_shards_preserves_output_order_for_out_of_order_slots() {
let state = StripeReadState::new(
vec![
ShardSlot::data(1, b"cd".to_vec()),
ShardSlot::data(0, b"ab".to_vec()),
ShardSlot::data(2, b"ef".to_vec()),
],
2,
);
fn emit_data_shards_preserves_output_order() {
let state =
StripeReadState::from_parts(vec![Some(b"ab".to_vec()), Some(b"cd".to_vec()), Some(b"ef".to_vec())], Vec::new(), 2);
let output = emit_data_shards(&state, 3, 6, 5).expect("out-of-order data slots should emit by shard index");
let output = emit_data_shards(&state, 3, 6, 5).expect("data slots should emit by shard index");
assert_eq!(output, b"abcde");
}
@@ -2051,27 +2114,27 @@ mod tests {
};
let mut workspace = engine.prepare_workspace(4).expect("workspace should be prepared");
let mut output = Vec::with_capacity(1);
let short_state = StripeReadState::new(vec![ShardSlot::data(0, vec![1, 2, 3, 4])], 1);
let mut short_state = StripeReadState::from_parts(vec![Some(vec![1, 2, 3, 4])], Vec::new(), 1);
let err = decode_stripe_into(
GET_OBJECT_PATH_CODEC_STREAMING,
false,
&engine,
&mut workspace,
short_state,
&mut short_state,
8,
&mut output,
)
.expect_err("decoded stripe shorter than data shard count must fail");
assert_eq!(err.kind(), ErrorKind::UnexpectedEof);
let missing_state = StripeReadState::from_parts(vec![None, Some(vec![5, 6, 7, 8])], Vec::new(), 1);
let mut missing_state = StripeReadState::from_parts(vec![None, Some(vec![5, 6, 7, 8])], Vec::new(), 1);
let err = decode_stripe_into(
GET_OBJECT_PATH_CODEC_STREAMING,
false,
&engine,
&mut workspace,
missing_state,
&mut missing_state,
8,
&mut output,
)
@@ -2082,6 +2145,35 @@ mod tests {
assert!(output.capacity() >= 32);
}
#[test]
fn decode_stripe_reconstructs_in_place_without_replacing_slot_storage() {
let erasure = Erasure::new(2, 1, 8);
let engine = LegacyEcDecodeEngine::new(erasure.clone());
let mut workspace = engine.prepare_workspace(4).expect("workspace should be prepared");
let encoded = erasure.encode_data(b"abcdefgh").expect("test stripe should encode");
let mut shards = encoded.into_iter().map(|shard| Some(shard.to_vec())).collect::<Vec<_>>();
shards[0] = None;
let mut state = StripeReadState::from_parts(shards, vec![Some(DiskError::FileCorrupt)], 2);
let before = state.scratch_storage();
let mut output = Vec::new();
let decoded = decode_stripe_into(
GET_OBJECT_PATH_CODEC_STREAMING,
false,
&engine,
&mut workspace,
&mut state,
8,
&mut output,
)
.expect("degraded stripe should reconstruct");
assert!(decoded);
assert_eq!(output, b"abcdefgh");
assert_eq!(state.scratch_storage().0, before.0, "reconstruction must retain shard slot storage");
assert_eq!(state.scratch_storage().1, before.1, "unused error storage must not be rebuilt");
}
#[tokio::test]
async fn erasure_decode_reader_reports_short_source() {
let erasure = Erasure::new(4, 2, 32);
+196 -42
View File
@@ -18,10 +18,12 @@ use crate::disk::error_reduce::{
};
use crate::erasure::coding::BitrotWriterWrapper;
use crate::erasure::coding::Erasure;
use crate::erasure::coding::erasure::EncodedBlock;
use crate::runtime::sources as runtime_sources;
use bytes::{Bytes, BytesMut};
use futures::StreamExt;
use futures::stream::FuturesUnordered;
use rustfs_utils::HashAlgorithm;
use std::sync::Arc;
use std::time::Instant;
use std::vec;
@@ -223,8 +225,8 @@ async fn send_queued<T>(
sender.send(InflightEntry::new(entry, bytes)).await
}
fn queued_batch_bytes(batch: &[Vec<Bytes>]) -> usize {
batch.iter().map(|block| queued_block_bytes(block)).sum()
fn queued_batch_bytes(batch: &[EncodedBlock]) -> usize {
batch.iter().map(EncodedBlock::queued_bytes).sum()
}
fn dominant_error_summary_label(summary: &WriteQuorumFailureSummary) -> &'static str {
@@ -336,7 +338,7 @@ impl<'a> MultiWriter<'a> {
}
}
async fn write_shard(writer_opt: &mut Option<BitrotWriterWrapper>, err: &mut Option<Error>, shard: &Bytes) {
async fn write_shard(writer_opt: &mut Option<BitrotWriterWrapper>, err: &mut Option<Error>, shard: &[u8]) {
match writer_opt {
Some(writer) => {
match writer.write(shard).await {
@@ -361,12 +363,20 @@ impl<'a> MultiWriter<'a> {
}
pub async fn write(&mut self, data: Vec<Bytes>) -> std::io::Result<()> {
assert_eq!(data.len(), self.writers.len());
self.write_shards(data.iter().map(Bytes::as_ref)).await
}
async fn write_block(&mut self, block: &EncodedBlock) -> std::io::Result<()> {
self.write_shards(block.shards()).await
}
async fn write_shards<'b>(&mut self, shards: impl ExactSizeIterator<Item = &'b [u8]>) -> std::io::Result<()> {
assert_eq!(shards.len(), self.writers.len());
let budget = self.next_progress_budget();
{
let mut futures = FuturesUnordered::new();
for ((writer_opt, err), shard) in self.writers.iter_mut().zip(self.errs.iter_mut()).zip(data.iter()) {
for ((writer_opt, err), shard) in self.writers.iter_mut().zip(self.errs.iter_mut()).zip(shards) {
if err.is_some() {
continue; // Skip if we already have an error for this writer
}
@@ -490,10 +500,10 @@ impl<'a> MultiWriter<'a> {
}
impl Erasure {
async fn encode_block(self: Arc<Self>, encode_buf: Vec<u8>, len: usize) -> std::io::Result<(Vec<Bytes>, Vec<u8>)> {
async fn encode_block(self: Arc<Self>, encode_buf: Vec<u8>, len: usize) -> std::io::Result<(EncodedBlock, Vec<u8>)> {
let encode_stage_start = stage_timer_if_enabled();
let encode_once = move || {
let res = self.encode_data(&encode_buf[..len]);
let res = self.encode_data_block(&encode_buf[..len]);
(res, encode_buf)
};
@@ -518,9 +528,9 @@ impl Erasure {
Ok((res?, returned_buf))
}
async fn encode_block_bytes_mut(self: Arc<Self>, encode_buf: BytesMut, len: usize) -> std::io::Result<Vec<Bytes>> {
async fn encode_block_bytes_mut(self: Arc<Self>, encode_buf: BytesMut, len: usize) -> std::io::Result<EncodedBlock> {
let encode_stage_start = stage_timer_if_enabled();
let encode_once = move || self.encode_data_bytes_mut(encode_buf, len);
let encode_once = move || self.encode_data_bytes_mut_block(encode_buf, len);
let res = match tokio::runtime::Handle::current().runtime_flavor() {
// Same rationale as encode_block: inline the short EC burst on the
@@ -576,13 +586,46 @@ impl Erasure {
));
}
let shards = self.encode_data_owned(buf)?;
let block = self.encode_data_owned_block(buf)?;
let mut mw = MultiWriter::new(writers, quorum);
mw.write(shards).await?;
mw.write_block(&block).await?;
mw.shutdown().await?;
Ok((reader, total))
}
/// Encode a small inline object directly into its per-disk bitrot payloads.
/// The returned bytes are the same `[hash][shard]` representation produced
/// by `BitrotWriter`, ready to be embedded in each disk's staged `xl.meta`.
#[hotpath::measure(impl_type = "Erasure")]
pub(crate) async fn encode_inline_shards_with_size_hint<R>(
self: Arc<Self>,
mut reader: R,
size_hint: usize,
) -> std::io::Result<(R, usize, Vec<Bytes>)>
where
R: AsyncRead + Send + Sync + Unpin,
{
use tokio::io::AsyncReadExt;
let mut buf = Vec::with_capacity(small_ingest_capacity(&self, size_hint));
let total = reader.read_to_end(&mut buf).await?;
if total == 0 {
return Ok((reader, 0, Vec::new()));
}
let block = self.encode_data_owned_block(buf)?;
let mut inline_shards = Vec::with_capacity(block.shards().len());
for shard in block.shards() {
let hash = HashAlgorithm::HighwayHash256S.hash_encode(shard);
let mut encoded = BytesMut::with_capacity(hash.as_ref().len() + shard.len());
encoded.extend_from_slice(hash.as_ref());
encoded.extend_from_slice(shard);
inline_shards.push(encoded.freeze());
}
Ok((reader, total, inline_shards))
}
#[hotpath::measure(impl_type = "Erasure")]
pub async fn encode<R>(
self: Arc<Self>,
@@ -624,7 +667,7 @@ impl Erasure {
let expanded_block_bytes = self.shard_size().saturating_mul(self.total_shard_count());
let max_inflight_bytes = erasure_encode_max_inflight_bytes();
let inflight_blocks = encode_channel_capacity(expanded_block_bytes, max_inflight_bytes);
let (tx, mut rx) = mpsc::channel::<InflightEntry<Vec<Bytes>>>(inflight_blocks);
let (tx, mut rx) = mpsc::channel::<InflightEntry<EncodedBlock>>(inflight_blocks);
let mut task = AbortOnDropTask::new(tokio::spawn(async move {
let block_size = self.block_size;
@@ -646,7 +689,7 @@ impl Erasure {
let encode_buf = buf;
let res = self.clone().encode_block_bytes_mut(encode_buf, n).await?;
buf = BytesMut::with_capacity(ingest_capacity);
let queued_bytes = queued_block_bytes(&res);
let queued_bytes = res.queued_bytes();
let _producer_stage = rustfs_io_metrics::track_ec_encode_producer_bytes(queued_bytes);
let send_wait_stage_start = stage_timer_if_enabled();
if let Err(err) = send_queued(&tx, res, queued_bytes).await {
@@ -676,7 +719,7 @@ impl Erasure {
let encode_buf = std::mem::take(&mut buf);
let (res, returned_buf) = self.clone().encode_block(encode_buf, n).await?;
buf = returned_buf;
let queued_bytes = queued_block_bytes(&res);
let queued_bytes = res.queued_bytes();
let _producer_stage = rustfs_io_metrics::track_ec_encode_producer_bytes(queued_bytes);
let send_wait_stage_start = stage_timer_if_enabled();
if let Err(err) = send_queued(&tx, res, queued_bytes).await {
@@ -720,9 +763,9 @@ impl Erasure {
if block.is_empty() {
break;
}
let _writer_stage = rustfs_io_metrics::track_ec_encode_writer_bytes(queued_block_bytes(&block));
let _writer_stage = rustfs_io_metrics::track_ec_encode_writer_bytes(block.queued_bytes());
let write_stage_start = stage_timer_if_enabled();
if let Err(err) = writers.write(block).await {
if let Err(err) = writers.write_block(&block).await {
write_err = Some(err);
break;
}
@@ -769,7 +812,7 @@ impl Erasure {
let inflight_blocks = encode_channel_capacity(expanded_block_bytes, max_inflight_bytes);
let batch_blocks = encode_batch_block_count().min(inflight_blocks);
let channel_capacity = inflight_blocks.div_ceil(batch_blocks).max(1);
let (tx, mut rx) = mpsc::channel::<InflightEntry<Vec<Vec<Bytes>>>>(channel_capacity);
let (tx, mut rx) = mpsc::channel::<InflightEntry<Vec<EncodedBlock>>>(channel_capacity);
let mut task = AbortOnDropTask::new(tokio::spawn(async move {
let block_size = self.block_size;
@@ -786,7 +829,7 @@ impl Erasure {
let encode_buf = std::mem::take(&mut buf);
let (res, returned_buf) = self.clone().encode_block(encode_buf, n).await?;
buf = returned_buf;
let queued_bytes = queued_block_bytes(&res);
let queued_bytes = res.queued_bytes();
pending_batch_bytes = pending_batch_bytes.saturating_add(queued_bytes);
pending_batch.push(res);
drop(pending_batch_stage.take());
@@ -845,7 +888,7 @@ impl Erasure {
let _writer_stage = rustfs_io_metrics::track_ec_encode_writer_bytes(queued_batch_bytes(&batch));
let write_stage_start = stage_timer_if_enabled();
for block in batch {
if let Err(err) = writers.write(block).await {
if let Err(err) = writers.write_block(&block).await {
write_err = Some(err);
break;
}
@@ -1895,7 +1938,11 @@ mod tests {
let baseline = rustfs_io_metrics::current_ec_encode_inflight_bytes();
let (tx, rx) = mpsc::channel(2);
let mut rx = rx;
let batch = vec![vec![Bytes::from_static(b"queued")], vec![Bytes::from_static(b"batch")]];
let erasure = Erasure::new(1, 0, 16);
let batch = vec![
erasure.encode_data_block(b"queued").expect("first block should encode"),
erasure.encode_data_block(b"batch").expect("second block should encode"),
];
let batch_bytes = queued_batch_bytes(&batch);
send_queued(&tx, batch, batch_bytes).await.expect("batch should be queued");
@@ -2117,6 +2164,39 @@ mod tests {
);
}
#[tokio::test]
async fn cancelling_inline_small_drops_stalled_write() {
const BLOCK_SIZE: usize = 16;
let (writer_entered_tx, writer_entered) = oneshot::channel();
let writes = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let mut writers = vec![Some(bitrot_writer_plain(
StallOnWriteWithSignal {
entered: Some(writer_entered_tx),
writes: writes.clone(),
},
BLOCK_SIZE,
))];
let erasure = Arc::new(Erasure::new(1, 0, BLOCK_SIZE));
let reader = tokio::io::BufReader::new(Cursor::new(vec![0xA5; BLOCK_SIZE - 1]));
let encode = tokio::spawn(async move { erasure.encode_inline_small(reader, &mut writers, 1).await });
tokio::time::timeout(Duration::from_secs(1), writer_entered)
.await
.expect("inline writer should enter before cancellation")
.expect("stalling writer should signal entry");
encode.abort();
assert!(
matches!(encode.await, Err(err) if err.is_cancelled()),
"inline encode task should be cancelled"
);
assert_eq!(
writes.load(std::sync::atomic::Ordering::SeqCst),
1,
"cancellation must drop the stalled write instead of polling it again"
);
}
#[tokio::test]
async fn encode_returns_unexpected_eof_for_truncated_limited_reader() {
let committed = Arc::new(Mutex::new(Vec::new()));
@@ -2236,11 +2316,11 @@ mod tests {
.expect("bytesmut encode should succeed on current-thread runtime");
let expected_shard_size = payload.len().div_ceil(erasure.data_shards);
assert_eq!(shards.len(), erasure.total_shard_count());
assert!(shards.iter().all(|shard| shard.len() == expected_shard_size));
assert_eq!(shards.shards().len(), erasure.total_shard_count());
assert!(shards.shards().all(|shard| shard.len() == expected_shard_size));
let mut restored = Vec::new();
for shard in shards.iter().take(erasure.data_shards) {
for shard in shards.shards().take(erasure.data_shards) {
restored.extend_from_slice(shard);
}
restored.truncate(payload.len());
@@ -2343,6 +2423,41 @@ mod tests {
assert!(committed.lock().unwrap().is_empty());
}
#[tokio::test]
async fn encode_inline_shards_matches_writer_bitrot_layout() {
const DATA_SHARDS: usize = 2;
const PARITY_SHARDS: usize = 2;
const BLOCK_SIZE: usize = 64;
let checksum_algo = HashAlgorithm::HighwayHash256S;
for uses_legacy in [false, true] {
let erasure = Arc::new(Erasure::new_with_options(DATA_SHARDS, PARITY_SHARDS, BLOCK_SIZE, uses_legacy));
for payload in [Vec::new(), vec![0xA5], vec![0x5A; BLOCK_SIZE - 1], vec![0xC3; BLOCK_SIZE]] {
let reader = tokio::io::BufReader::new(Cursor::new(payload.clone()));
let (_reader, total, inline_shards) = erasure
.clone()
.encode_inline_shards_with_size_hint(reader, payload.len())
.await
.expect("inline shards should encode");
assert_eq!(total, payload.len());
if payload.is_empty() {
assert!(inline_shards.is_empty());
continue;
}
let raw_shards = erasure.encode_data(&payload).expect("reference shards should encode");
assert_eq!(inline_shards.len(), DATA_SHARDS + PARITY_SHARDS);
for (inline, raw) in inline_shards.iter().zip(raw_shards) {
let mut writer =
BitrotWriterWrapper::new(CustomWriter::new_inline_buffer(), raw.len(), checksum_algo.clone());
writer.write(&raw).await.expect("reference writer should accept shard");
writer.shutdown().await.expect("reference writer should shutdown");
assert_eq!(inline.as_ref(), writer.into_inline_data().expect("reference writer should retain bytes"));
}
}
}
}
/// encode_inline_small: small payload is encoded into the correct number of shards
/// and each writer receives data after shutdown.
#[tokio::test]
@@ -2506,7 +2621,7 @@ mod tests {
assert_eq!(&next[..], &data[16..]);
}
async fn committed_shards_for_ingest_mode(use_bytesmut_ingest: bool, uses_legacy: bool, payload: &[u8]) -> Vec<Vec<u8>> {
async fn committed_shards_for_pipeline(pipeline: EncodePipeline, uses_legacy: bool, payload: &[u8]) -> Vec<Vec<u8>> {
const DATA_SHARDS: usize = 2;
const PARITY_SHARDS: usize = 2;
const TOTAL_SHARDS: usize = DATA_SHARDS + PARITY_SHARDS;
@@ -2520,10 +2635,16 @@ mod tests {
let erasure = Arc::new(Erasure::new_with_options(DATA_SHARDS, PARITY_SHARDS, BLOCK_SIZE, uses_legacy));
let reader = tokio::io::BufReader::new(Cursor::new(payload.to_vec()));
let (_reader, total) = erasure
.encode_with_ingest_mode(reader, &mut writers, DATA_SHARDS, use_bytesmut_ingest)
.await
.expect("encode should succeed");
let (_reader, total) = match pipeline {
EncodePipeline::Vec => {
erasure
.encode_with_ingest_mode(reader, &mut writers, DATA_SHARDS, false)
.await
}
EncodePipeline::BytesMut => erasure.encode_with_ingest_mode(reader, &mut writers, DATA_SHARDS, true).await,
EncodePipeline::Batched => erasure.encode_batched(reader, &mut writers, DATA_SHARDS).await,
}
.expect("encode should succeed");
assert_eq!(total, payload.len());
committed
@@ -2532,31 +2653,64 @@ mod tests {
.collect()
}
/// HP-10 (rustfs/backlog#931) merge gate: the BytesMut ingest path must produce
/// byte-for-byte identical shard streams to the default Vec ingest path, for both
/// legacy-aware shard-size formulas, across empty, sub-block, exactly-full-block,
/// and multi-block-with-partial-tail payloads.
async fn expected_committed_shards(uses_legacy: bool, payload: &[u8]) -> Vec<Vec<u8>> {
const DATA_SHARDS: usize = 2;
const PARITY_SHARDS: usize = 2;
const TOTAL_SHARDS: usize = DATA_SHARDS + PARITY_SHARDS;
const BLOCK_SIZE: usize = 64;
let committed: Vec<Arc<Mutex<Vec<u8>>>> = (0..TOTAL_SHARDS).map(|_| Arc::new(Mutex::new(Vec::new()))).collect();
let mut writers: Vec<BitrotWriterWrapper> = committed
.iter()
.map(|c| bitrot_writer(DeferredCommitWriter::new(c.clone()), BLOCK_SIZE / DATA_SHARDS))
.collect();
let erasure = Erasure::new_with_options(DATA_SHARDS, PARITY_SHARDS, BLOCK_SIZE, uses_legacy);
for block in payload.chunks(BLOCK_SIZE) {
let shards = erasure.encode_data(block).expect("reference block should encode");
for (writer, shard) in writers.iter_mut().zip(shards) {
let written = writer.write(&shard).await.expect("reference shard should write");
assert_eq!(written, shard.len());
}
}
for writer in &mut writers {
writer.shutdown().await.expect("reference writer should commit");
}
committed
.iter()
.map(|c| c.lock().expect("committed buffer should be lockable").clone())
.collect()
}
/// The streaming and batched paths must produce the same bitrot-wrapped shard
/// bytes as the public block encoder for both shard-size formulas and all block
/// boundary shapes.
#[tokio::test]
async fn bytesmut_ingest_matches_vec_ingest_byte_for_byte() {
const BLOCK_SIZE: usize = 64;
let payloads: Vec<Vec<u8>> = vec![
Vec::new(),
b"tiny".to_vec(),
vec![1],
vec![2; BLOCK_SIZE - 1],
(0..BLOCK_SIZE as u32).map(|i| i as u8).collect(), // exactly one full block
vec![3u8; BLOCK_SIZE * 4], // whole number of blocks
vec![4; BLOCK_SIZE + 1],
vec![3u8; BLOCK_SIZE * 4], // whole number of blocks
(0..(BLOCK_SIZE * 3 + 7) as u32).map(|i| (i % 251) as u8).collect(), // partial tail
];
for uses_legacy in [false, true] {
for payload in &payloads {
let vec_path = committed_shards_for_ingest_mode(false, uses_legacy, payload).await;
let bytesmut_path = committed_shards_for_ingest_mode(true, uses_legacy, payload).await;
assert_eq!(
vec_path,
bytesmut_path,
"ingest paths must be byte-identical (legacy={uses_legacy}, payload_len={})",
payload.len()
);
let expected = expected_committed_shards(uses_legacy, payload).await;
for pipeline in [EncodePipeline::Vec, EncodePipeline::BytesMut, EncodePipeline::Batched] {
let actual = committed_shards_for_pipeline(pipeline, uses_legacy, payload).await;
assert_eq!(
actual,
expected,
"streaming shards must match the public block encoder (legacy={uses_legacy}, payload_len={})",
payload.len()
);
}
}
}
}
+140 -100
View File
@@ -29,6 +29,46 @@ use tokio::io::AsyncRead;
use tracing::warn;
use uuid::Uuid;
pub(crate) struct EncodedBlock {
data: Bytes,
shard_size: usize,
}
impl EncodedBlock {
fn empty() -> Self {
Self {
data: Bytes::new(),
shard_size: 0,
}
}
pub(crate) fn is_empty(&self) -> bool {
self.data.is_empty()
}
pub(crate) fn queued_bytes(&self) -> usize {
self.data.len()
}
pub(crate) fn shards(&self) -> impl ExactSizeIterator<Item = &[u8]> {
debug_assert!(self.shard_size > 0, "only non-empty encoded blocks reach shard writers");
debug_assert_eq!(self.data.len() % self.shard_size, 0);
self.data.chunks_exact(self.shard_size)
}
fn into_shards(mut self, shard_count: usize) -> Vec<Bytes> {
if self.shard_size == 0 {
return vec![Bytes::new(); shard_count];
}
let mut shards = Vec::with_capacity(shard_count);
for _ in 0..shard_count {
shards.push(self.data.split_to(self.shard_size));
}
shards
}
}
const MODERN_MAX_TOTAL_SHARDS: usize = <reed_solomon_erasure::galois_8::Field as reed_solomon_erasure::Field>::ORDER;
const MODERN_REED_SOLOMON_CACHE_MAX_ENTRIES: usize = 64;
@@ -675,106 +715,48 @@ impl Erasure {
#[tracing::instrument(level = "debug", skip_all, fields(data_len=data.len()))]
#[hotpath::measure(impl_type = "Erasure")]
pub fn encode_data(&self, data: &[u8]) -> io::Result<Vec<Bytes>> {
let shard_size_fn = if self.uses_legacy {
calc_shard_size_legacy
} else {
calc_shard_size
};
let per_shard_size = shard_size_fn(data.len(), self.data_shards);
if per_shard_size == 0 {
return Ok(vec![Bytes::new(); self.total_shard_count()]);
}
let need_total_size = per_shard_size * self.total_shard_count();
self.encode_data_block_inner(data)
.map(|block| block.into_shards(self.total_shard_count()))
}
let mut data_buffer = BytesMut::with_capacity(need_total_size);
#[tracing::instrument(level = "debug", skip_all, fields(data_len=data.len()))]
#[hotpath::measure(label = "Erasure::encode_data", impl_type = "Erasure")]
pub(crate) fn encode_data_block(&self, data: &[u8]) -> io::Result<EncodedBlock> {
self.encode_data_block_inner(data)
}
fn encode_data_block_inner(&self, data: &[u8]) -> io::Result<EncodedBlock> {
let mut data_buffer = BytesMut::with_capacity(self.encoded_capacity_for_data_len(data.len()));
data_buffer.extend_from_slice(data);
data_buffer.resize(need_total_size, 0u8);
{
let data_slices: SmallVec<[&mut [u8]; 16]> = data_buffer.chunks_exact_mut(per_shard_size).collect();
if self.parity_shards > 0 {
if self.uses_legacy {
if let Some(encoder) = self.legacy_encoder.as_ref() {
encoder.encode(data_slices)?;
} else {
warn!("parity_shards > 0, uses_legacy but legacy_encoder is None");
}
} else if let Some(encoder) = self.encoder.as_ref() {
encoder.encode(data_slices)?;
} else {
warn!("parity_shards > 0, but encoder is None");
}
}
}
// Zero-copy split, all shards reference data_buffer
let mut data_buffer = data_buffer.freeze();
let mut shards = Vec::with_capacity(self.total_shard_count());
for _ in 0..self.total_shard_count() {
let shard = data_buffer.split_to(per_shard_size);
shards.push(shard);
}
Ok(shards)
self.encode_buffer(data_buffer, data.len())
}
/// Encode owned data, avoiding a copy when the caller already has a heap buffer.
/// Falls back to copying into a new buffer if zero-copy conversion fails.
#[hotpath::measure(impl_type = "Erasure")]
pub fn encode_data_owned(&self, data: Vec<u8>) -> io::Result<Vec<Bytes>> {
let shard_size_fn = if self.uses_legacy {
calc_shard_size_legacy
} else {
calc_shard_size
};
let per_shard_size = shard_size_fn(data.len(), self.data_shards);
if per_shard_size == 0 {
return Ok(vec![Bytes::new(); self.total_shard_count()]);
}
let need_total_size = per_shard_size * self.total_shard_count();
self.encode_data_owned_block_inner(data)
.map(|block| block.into_shards(self.total_shard_count()))
}
#[hotpath::measure(label = "Erasure::encode_data_owned", impl_type = "Erasure")]
pub(crate) fn encode_data_owned_block(&self, data: Vec<u8>) -> io::Result<EncodedBlock> {
self.encode_data_owned_block_inner(data)
}
fn encode_data_owned_block_inner(&self, data: Vec<u8>) -> io::Result<EncodedBlock> {
let data_len = data.len();
// Try zero-copy: Vec<u8> -> Bytes -> BytesMut (succeeds when refcount == 1)
let mut data_buffer = match Bytes::from(data).try_into_mut() {
Ok(mut bm) => {
bm.resize(need_total_size, 0u8);
bm
}
let data_buffer = match Bytes::from(data).try_into_mut() {
Ok(data_buffer) => data_buffer,
Err(b) => {
// Rare path: refcount != 1, fall back to copy
let mut bm = BytesMut::with_capacity(need_total_size);
bm.extend_from_slice(&b);
bm.resize(need_total_size, 0u8);
bm
let mut data_buffer = BytesMut::with_capacity(self.encoded_capacity_for_data_len(data_len));
data_buffer.extend_from_slice(&b);
data_buffer
}
};
{
let data_slices: SmallVec<[&mut [u8]; 16]> = data_buffer.chunks_exact_mut(per_shard_size).collect();
if self.parity_shards > 0 {
if self.uses_legacy {
if let Some(encoder) = self.legacy_encoder.as_ref() {
encoder.encode(data_slices)?;
} else {
warn!("parity_shards > 0, uses_legacy but legacy_encoder is None");
}
} else if let Some(encoder) = self.encoder.as_ref() {
encoder.encode(data_slices)?;
} else {
warn!("parity_shards > 0, but encoder is None");
}
}
}
let mut data_buffer = data_buffer.freeze();
let mut shards = Vec::with_capacity(self.total_shard_count());
for _ in 0..self.total_shard_count() {
let shard = data_buffer.split_to(per_shard_size);
shards.push(shard);
}
Ok(shards)
self.encode_buffer(data_buffer, data_len)
}
/// Encode data from an owned `BytesMut` buffer, avoiding the initial copy
@@ -786,7 +768,17 @@ impl Erasure {
/// `data_len <= block_size` — both shard-size formulas are monotone in
/// `data_len` — so this function never reallocates the buffer.
#[hotpath::measure(impl_type = "Erasure")]
pub fn encode_data_bytes_mut(&self, mut data_buffer: BytesMut, data_len: usize) -> io::Result<Vec<Bytes>> {
pub fn encode_data_bytes_mut(&self, data_buffer: BytesMut, data_len: usize) -> io::Result<Vec<Bytes>> {
self.encode_buffer(data_buffer, data_len)
.map(|block| block.into_shards(self.total_shard_count()))
}
#[hotpath::measure(label = "Erasure::encode_data_bytes_mut", impl_type = "Erasure")]
pub(crate) fn encode_data_bytes_mut_block(&self, data_buffer: BytesMut, data_len: usize) -> io::Result<EncodedBlock> {
self.encode_buffer(data_buffer, data_len)
}
fn encode_buffer(&self, mut data_buffer: BytesMut, data_len: usize) -> io::Result<EncodedBlock> {
let shard_size_fn = if self.uses_legacy {
calc_shard_size_legacy
} else {
@@ -794,7 +786,7 @@ impl Erasure {
};
let per_shard_size = shard_size_fn(data_len, self.data_shards);
if per_shard_size == 0 {
return Ok(vec![Bytes::new(); self.total_shard_count()]);
return Ok(EncodedBlock::empty());
}
let need_total_size = per_shard_size * self.total_shard_count();
@@ -821,14 +813,10 @@ impl Erasure {
}
}
let mut data_buffer = data_buffer.freeze();
let mut shards = Vec::with_capacity(self.total_shard_count());
for _ in 0..self.total_shard_count() {
let shard = data_buffer.split_to(per_shard_size);
shards.push(shard);
}
Ok(shards)
Ok(EncodedBlock {
data: data_buffer.freeze(),
shard_size: per_shard_size,
})
}
/// Decode and reconstruct missing data shards in-place.
@@ -1498,10 +1486,16 @@ mod tests {
fn encode_data_owned_matches_borrowed_path() {
for uses_legacy in [false, true] {
let erasure = Erasure::new_with_options(4, 2, 64, uses_legacy);
assert_owned_encode_matches_borrowed(&erasure, Vec::new());
assert_owned_encode_matches_borrowed(&erasure, b"small payload".to_vec());
assert_owned_encode_matches_borrowed(&erasure, (0_u8..37).collect());
for data in [
Vec::new(),
vec![0xA5; 1],
b"small payload".to_vec(),
(0_u8..37).collect(),
vec![0xA5; erasure.block_size - 1],
vec![0x5A; erasure.block_size],
] {
assert_owned_encode_matches_borrowed(&erasure, data);
}
}
}
@@ -1547,6 +1541,52 @@ mod tests {
}
}
#[test]
fn streaming_encoded_block_uses_one_contiguous_backing_buffer() {
for uses_legacy in [false, true] {
let erasure = Erasure::new_with_options(8, 8, 64, uses_legacy);
for data_len in [0, 1, 63, 64] {
let data = (0..data_len).map(|i| i as u8).collect::<Vec<_>>();
let expected = erasure.encode_data(&data).expect("public encode should succeed");
let borrowed = erasure
.encode_data_block(&data)
.expect("borrowed streaming encode should succeed");
let owned = erasure
.encode_data_owned_block(data.clone())
.expect("owned streaming encode should succeed");
let bytes_mut = erasure
.encode_data_bytes_mut_block(BytesMut::from(&data[..]), data.len())
.expect("BytesMut streaming encode should succeed");
assert_eq!(borrowed.queued_bytes(), owned.queued_bytes());
assert_eq!(borrowed.queued_bytes(), bytes_mut.queued_bytes());
if data_len == 0 {
assert!(expected.iter().all(Bytes::is_empty));
assert!(borrowed.is_empty());
assert!(owned.is_empty());
assert!(bytes_mut.is_empty());
continue;
}
assert!(borrowed.shards().eq(expected.iter().map(Bytes::as_ref)));
assert!(owned.shards().eq(expected.iter().map(Bytes::as_ref)));
assert!(bytes_mut.shards().eq(expected.iter().map(Bytes::as_ref)));
assert_eq!(borrowed.shards().len(), 16);
let first = borrowed.shards().next().expect("encoded block should have shards").as_ptr();
for (index, shard) in borrowed.shards().enumerate() {
assert_eq!(shard.as_ptr(), first.wrapping_add(index * shard.len()));
}
}
}
assert_eq!(
std::mem::size_of::<EncodedBlock>(),
std::mem::size_of::<Bytes>() + std::mem::size_of::<usize>(),
"queue entries must contain one backing buffer handle, not per-shard handles"
);
}
/// HP-10 capacity invariant: both shard-size formulas are monotone in `data_len`,
/// so pre-reserving `shard_size(block_size) * total_shard_count` covers the
/// `need_total_size` of every block-or-smaller payload and the ingest buffer
+128 -3
View File
@@ -22,12 +22,13 @@ use crate::diagnostics::get::{
#[cfg(feature = "hotpath")]
use crate::disk::FileWriter;
use crate::disk::{self, DiskAPI as _, DiskStore, FileReader, MmapCopyStageMetrics, error::DiskError};
use crate::erasure::coding::{BitrotReader, BitrotWriterWrapper, CustomWriter};
use crate::erasure::coding::{BitrotReader, BitrotWriterWrapper, CustomWriter, ShardChunkRead};
use bytes::Bytes;
use rustfs_config::{
DEFAULT_OBJECT_MMAP_READ_ENABLE, DEFAULT_OBJECT_MMAP_READ_MAX_LENGTH, ENV_OBJECT_MMAP_READ_ENABLE,
ENV_OBJECT_MMAP_READ_MAX_LENGTH, ENV_OBJECT_ZERO_COPY_ENABLE,
};
use rustfs_rio::ChunkReaderBox;
use rustfs_utils::HashAlgorithm;
use std::future::Future;
use std::io::{self, Cursor};
@@ -51,13 +52,25 @@ tokio::task_local! {
/// (rustfs/backlog#1159). Everything else is a stream and keeps the old path.
pub enum ShardReader {
InMemory(Cursor<Bytes>),
Chunked(ChunkReaderBox),
Stream(Box<dyn AsyncRead + Send + Sync + Unpin>),
}
#[cfg(test)]
impl ShardReader {
pub(crate) fn inline_bytes(&self) -> Option<&Bytes> {
match self {
Self::InMemory(cursor) => Some(cursor.get_ref()),
Self::Chunked(_) | Self::Stream(_) => None,
}
}
}
impl AsyncRead for ShardReader {
fn poll_read(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut tokio::io::ReadBuf<'_>) -> Poll<std::io::Result<()>> {
match self.get_mut() {
Self::InMemory(cursor) => Pin::new(cursor).poll_read(cx, buf),
Self::Chunked(reader) => Pin::new(&mut **reader).poll_read(cx, buf),
Self::Stream(reader) => Pin::new(reader).poll_read(cx, buf),
}
}
@@ -67,7 +80,19 @@ impl crate::erasure::coding::ShardSource for ShardReader {
fn try_take_block(&mut self, n: usize) -> Option<Bytes> {
match self {
Self::InMemory(cursor) => cursor.try_take_block(n),
Self::Stream(_) => None,
Self::Chunked(_) | Self::Stream(_) => None,
}
}
fn poll_read_chunk(self: Pin<&mut Self>, cx: &mut Context<'_>, max: usize) -> Poll<io::Result<ShardChunkRead>> {
let Self::Chunked(reader) = self.get_mut() else {
return Poll::Ready(Ok(ShardChunkRead::Unsupported));
};
match Pin::new(&mut **reader).poll_read_chunk(cx, max) {
Poll::Ready(Ok(Some(chunk))) => Poll::Ready(Ok(ShardChunkRead::Chunk(chunk))),
Poll::Ready(Ok(None)) => Poll::Ready(Ok(ShardChunkRead::Eof)),
Poll::Ready(Err(err)) => Poll::Ready(Err(err)),
Poll::Pending => Poll::Pending,
}
}
}
@@ -345,6 +370,17 @@ async fn open_disk_reader(
let metrics_path = metrics_path.filter(|_| rustfs_io_metrics::get_stage_metrics_enabled());
let stage_metrics_enabled = metrics_path.is_some();
// Preserve HTTP body ownership only on healthy remote reads. Instrumented
// and local paths retain their existing AsyncRead wrappers.
if use_mmap_read
&& !disk.is_local()
&& !stage_metrics_enabled
&& !cfg!(feature = "hotpath")
&& let Some(reader) = disk.read_file_stream_chunks(bucket, path, offset, length).await?
{
return Ok(ShardReader::Chunked(reader));
}
// Mmap-copy materializes the whole `offset..offset+length` range as one
// owned allocation before any byte is served, and GET/heal shard reads
// request the entire part span in one call. Over-cap reads (e.g. a huge
@@ -620,7 +656,7 @@ pub async fn create_bitrot_reader_from_bytes(
}
#[allow(clippy::too_many_arguments)]
async fn create_bitrot_reader_from_bytes_with_stage_metrics(
pub(crate) async fn create_bitrot_reader_from_bytes_with_stage_metrics(
inline_data: Option<Bytes>,
disk: Option<&DiskStore>,
bucket: &str,
@@ -780,6 +816,50 @@ pub async fn create_bitrot_writer(
#[cfg(test)]
mod tests {
use super::*;
use rustfs_rio::ChunkReader;
use std::collections::VecDeque;
struct TestChunkReader {
chunks: VecDeque<Bytes>,
}
impl TestChunkReader {
fn new(bytes: Bytes, fragment_sizes: &[usize]) -> Self {
let mut chunks = VecDeque::new();
let mut offset = 0;
for &size in fragment_sizes {
let end = (offset + size).min(bytes.len());
if offset < end {
chunks.push_back(bytes.slice(offset..end));
}
offset = end;
}
if offset < bytes.len() {
chunks.push_back(bytes.slice(offset..));
}
Self { chunks }
}
}
impl AsyncRead for TestChunkReader {
fn poll_read(self: Pin<&mut Self>, _cx: &mut Context<'_>, _buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Err(io::Error::other("test chunk reader must use chunk handoff")))
}
}
impl ChunkReader for TestChunkReader {
fn poll_read_chunk(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, max: usize) -> Poll<io::Result<Option<Bytes>>> {
let Some(mut chunk) = self.chunks.pop_front() else {
return Poll::Ready(Ok(None));
};
let take = chunk.len().min(max);
if take < chunk.len() {
self.chunks.push_front(chunk.split_off(take));
}
chunk.truncate(take);
Poll::Ready(Ok(Some(chunk)))
}
}
#[cfg(feature = "hotpath")]
use crate::cluster::rpc::RemoteDisk;
@@ -1669,4 +1749,49 @@ mod tests {
println!("error: {error:?}");
assert_eq!(error, DiskError::DiskNotFound);
}
#[tokio::test]
async fn shard_reader_chunked_path_verifies_fragmented_remote_block() {
const SHARD_SIZE: usize = 1024;
let algo = HashAlgorithm::HighwayHash256S;
let data = vec![42u8; SHARD_SIZE];
let mut encoded = Vec::new();
crate::erasure::coding::BitrotWriter::new(&mut encoded, SHARD_SIZE, algo.clone())
.write(&data)
.await
.expect("test shard should encode");
let source = TestChunkReader::new(Bytes::from(encoded), &[3, 7, 17, 31]);
let mut reader = BitrotReader::new(ShardReader::Chunked(Box::new(source)), SHARD_SIZE, algo, false);
let mut output = Vec::with_capacity(SHARD_SIZE);
reader
.read_appending(&mut output, SHARD_SIZE)
.await
.expect("fragmented remote shard should verify");
assert_eq!(output, data);
}
#[tokio::test]
async fn shard_reader_chunked_path_handles_more_than_one_poll_budget() {
const SHARD_SIZE: usize = 1024;
let algo = HashAlgorithm::HighwayHash256S;
let data = vec![42u8; SHARD_SIZE];
let mut encoded = Vec::new();
crate::erasure::coding::BitrotWriter::new(&mut encoded, SHARD_SIZE, algo.clone())
.write(&data)
.await
.expect("test shard should encode");
let fragment_sizes = vec![1; encoded.len()];
let source = TestChunkReader::new(Bytes::from(encoded), &fragment_sizes);
let mut reader = BitrotReader::new(ShardReader::Chunked(Box::new(source)), SHARD_SIZE, algo, false);
let mut output = Vec::with_capacity(SHARD_SIZE);
reader
.read_appending(&mut output, SHARD_SIZE)
.await
.expect("fragmented remote shard should verify after multiple polls");
assert_eq!(output, data);
}
}
+3 -2
View File
@@ -21,7 +21,8 @@ use tracing::debug;
/// Supported set sizes this is used to find the optimal
/// single set size.
const SET_SIZES: [usize; 15] = [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16];
pub(crate) const MAX_ERASURE_SET_DRIVE_COUNT: usize = 16;
const SET_SIZES: [usize; 15] = [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, MAX_ERASURE_SET_DRIVE_COUNT];
const ENV_RUSTFS_ERASURE_SET_DRIVE_COUNT: &str = "RUSTFS_ERASURE_SET_DRIVE_COUNT";
#[derive(Deserialize, Debug, Default)]
@@ -327,7 +328,7 @@ fn possible_set_counts(set_size: usize) -> Vec<usize> {
/// checks whether given count is a valid set size for erasure coding.
fn is_valid_set_size(count: usize) -> bool {
count >= SET_SIZES[0] && count <= SET_SIZES[SET_SIZES.len() - 1]
count >= SET_SIZES[0] && count <= MAX_ERASURE_SET_DRIVE_COUNT
}
/// Final set size with all the symmetry accounted for.
+3
View File
@@ -260,6 +260,9 @@ pub struct ObjectOptions {
pub data_movement: bool,
pub raw_data_movement_read: bool,
/// Materialize the data-movement per-part checksum sidecar for APIs that
/// return part checksums. Ordinary object reads leave it encoded.
pub include_part_checksums: bool,
pub src_pool_idx: usize,
pub user_defined: HashMap<String, String>,
pub preserve_etag: Option<String>,
+3
View File
@@ -164,6 +164,9 @@ pub(crate) async fn local_node_name() -> String {
}
pub(crate) async fn set_local_node_name(node_name: String) {
// Also stamp the internode-metrics server label: io-metrics is a leaf
// crate and no longer resolves node identity itself (backlog#1834).
rustfs_io_metrics::internode_metrics::set_internode_server_label(node_name.as_str());
rustfs_common::set_global_local_node_name(&node_name).await;
}
+31 -5
View File
@@ -16,7 +16,7 @@ use super::meta::{
clone_arc_by_index, ensure_valid_rebalance_pool_index, invalid_rebalance_pool_index_error,
rebalance_metadata_not_initialized_error, should_ignore_rebalance_data_usage_cache,
};
use super::migration::migrate_entry_version;
use super::migration::{RebalanceMigrationBackend, migrate_entry_version};
use super::worker::{
RebalanceEntryCleanupResult, RebalanceEntryTask, load_rebalance_bucket_configs, rebalance_max_attempts,
resolve_rebalance_bucket_error, resolve_rebalance_entry_cleanup_delete_result, resolve_rebalance_file_info_versions_result,
@@ -144,6 +144,11 @@ impl ECStore {
return Ok(RebalanceEntryOutcome::Completed);
}
let bucket_incarnation_fence = match bucket_configs.bucket_incarnation_id {
Some(expected) => Some(self.acquire_bucket_incarnation_fence(&bucket, expected).await?),
None => None,
};
let mut fivs =
resolve_rebalance_file_info_versions_result(entry.file_info_versions(&bucket), bucket.as_str(), entry.name.as_str())?;
@@ -203,9 +208,14 @@ impl ECStore {
}
let version_id = version.version_id.map(|v| v.to_string());
let expected_bucket_incarnation_id = bucket_configs.bucket_incarnation_id;
let mut transfer = |src_pool_idx: usize, bucket: String, rd: GetObjectReader| {
let store = self.clone();
async move { store.rebalance_object(src_pool_idx, bucket, rd).await }
async move {
store
.rebalance_object(src_pool_idx, bucket, rd, expected_bucket_incarnation_id)
.await
}
};
// Route delete-marker migration through the store layer so it lands on the
// cross-pool target (excluding the source pool), not back onto the source set.
@@ -214,11 +224,12 @@ impl ECStore {
async move { store.delete_object(&bucket, &object, opts).await }
};
let result = migrate_entry_version(
set.as_ref(),
&RebalanceMigrationBackend::new(set.as_ref(), self.as_ref()),
bucket.clone(),
pool_index,
version,
version_id.clone(),
expected_bucket_incarnation_id,
rebalance_max_attempts(),
should_ignore_rebalance_data_usage_cache(bucket.as_str()),
&mut transfer,
@@ -303,6 +314,9 @@ impl ECStore {
}
if should_cleanup_rebalance_source_entry(rebalanced, fivs.versions.len(), expired) {
if bucket_incarnation_fence.as_ref().is_some_and(|guard| guard.is_lock_lost()) {
return Err(Error::other("rebalance bucket incarnation fence was lost before source cleanup"));
}
let cleanup_result = self
.finish_rebalance_entry_after_cleanup(
pool_index,
@@ -315,6 +329,12 @@ impl ECStore {
entry.name.as_str(),
&fivs,
&cleanup_preflight_allowed_missing,
data_movement::SourceCleanupBucketFence {
expected_incarnation_id: bucket_configs.bucket_incarnation_id,
lifecycle_guard: bucket_incarnation_fence
.as_ref()
.and_then(|guard| guard.namespace_lock_guard()),
},
"rebalance",
),
)
@@ -389,8 +409,14 @@ impl ECStore {
}
#[tracing::instrument(skip(self, rd))]
async fn rebalance_object(self: Arc<Self>, pool_idx: usize, bucket: String, rd: GetObjectReader) -> Result<()> {
data_movement::migrate_object(self, pool_idx, bucket, rd, "rebalance_object").await
async fn rebalance_object(
self: Arc<Self>,
pool_idx: usize,
bucket: String,
rd: GetObjectReader,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> Result<()> {
data_movement::migrate_object(self, pool_idx, bucket, rd, expected_bucket_incarnation_id, "rebalance_object").await
}
async fn update_rebalance_last_error(&self, pool_idx: usize, message: String) -> Result<()> {
@@ -5,6 +5,7 @@ use crate::error::{Error, Result, is_err_object_not_found, is_err_version_not_fo
use crate::object_api::{GetObjectReader, ObjectInfo, ObjectOptions};
use crate::set_disk::SetDisks;
use crate::storage_api_contracts::{object::ObjectIO, range::HTTPRangeSpec};
use crate::store::ECStore;
use http::HeaderMap;
use rustfs_filemeta::FileInfo;
use rustfs_utils::path::encode_dir_object;
@@ -21,15 +22,23 @@ pub(crate) struct MigrationVersionResult {
pub error: Option<Error>,
}
pub(super) fn rebalance_delete_marker_opts(version: &FileInfo, version_id: Option<String>, src_pool_idx: usize) -> ObjectOptions {
pub(super) fn rebalance_delete_marker_opts(
version: &FileInfo,
version_id: Option<String>,
src_pool_idx: usize,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> ObjectOptions {
let version_suspended = version.version_id.is_none() && version_id.is_none();
ObjectOptions {
versioned: true,
version_id,
versioned: !version_suspended,
version_suspended,
version_id: version_id.or_else(|| version_suspended.then(|| uuid::Uuid::nil().to_string())),
mod_time: version.mod_time,
src_pool_idx,
data_movement: true,
delete_marker: true,
skip_decommissioned: true,
expected_bucket_incarnation_id,
delete_replication: version
.replication_state_internal
.as_ref()
@@ -38,7 +47,12 @@ pub(super) fn rebalance_delete_marker_opts(version: &FileInfo, version_id: Optio
}
}
fn rebalance_remote_tiered_opts(version: &FileInfo, version_id: Option<String>, src_pool_idx: usize) -> ObjectOptions {
fn rebalance_remote_tiered_opts(
version: &FileInfo,
version_id: Option<String>,
src_pool_idx: usize,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
) -> ObjectOptions {
ObjectOptions {
versioned: version_id.is_some(),
version_id,
@@ -46,6 +60,21 @@ fn rebalance_remote_tiered_opts(version: &FileInfo, version_id: Option<String>,
user_defined: version.metadata.clone(),
src_pool_idx,
data_movement: true,
include_part_checksums: true,
http_preconditions: Some(crate::data_movement::data_movement_target_precondition()),
expected_bucket_incarnation_id,
..Default::default()
}
}
pub(super) fn rebalance_object_migration_read_opts(version_id: Option<String>) -> ObjectOptions {
ObjectOptions {
version_id,
no_lock: true,
data_movement: true,
raw_data_movement_read: true,
skip_decommissioned: true,
skip_rebalancing: true,
..Default::default()
}
}
@@ -70,8 +99,19 @@ pub(crate) trait MigrationBackend: Send + Sync {
) -> Result<()>;
}
pub(crate) struct RebalanceMigrationBackend<'a> {
source: &'a SetDisks,
store: &'a ECStore,
}
impl<'a> RebalanceMigrationBackend<'a> {
pub(crate) fn new(source: &'a SetDisks, store: &'a ECStore) -> Self {
Self { source, store }
}
}
#[async_trait::async_trait]
impl MigrationBackend for SetDisks {
impl MigrationBackend for RebalanceMigrationBackend<'_> {
async fn get_object_reader_for_migration(
&self,
bucket: &str,
@@ -80,7 +120,7 @@ impl MigrationBackend for SetDisks {
h: HeaderMap,
opts: &ObjectOptions,
) -> Result<GetObjectReader> {
self.get_object_reader(bucket, object, range, h, opts).await
self.source.get_object_reader(bucket, object, range, h, opts).await
}
async fn move_remote_version_for_migration(
@@ -90,7 +130,7 @@ impl MigrationBackend for SetDisks {
fi: &FileInfo,
opts: &ObjectOptions,
) -> Result<()> {
self.decommission_tiered_object(bucket, object, fi, opts).await
self.store.decommission_tiered_object(bucket, object, fi, opts).await
}
}
@@ -101,6 +141,7 @@ pub(crate) async fn migrate_entry_version<Backend, F, Fut, D, DFut>(
pool_index: usize,
version: &FileInfo,
version_id: Option<String>,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
max_attempts: usize,
ignore_data_usage_cache: bool,
transfer: F,
@@ -113,12 +154,13 @@ where
D: FnMut(String, String, ObjectOptions) -> DFut + Send,
DFut: Future<Output = Result<ObjectInfo>> + Send,
{
migrate_entry_version_with_retry_wait(
migrate_entry_version_with_retry_wait_and_incarnation(
set,
bucket,
pool_index,
version,
version_id,
expected_bucket_incarnation_id,
max_attempts,
ignore_data_usage_cache,
transfer,
@@ -137,6 +179,45 @@ pub(super) async fn migrate_entry_version_with_retry_wait<Backend, F, Fut, D, DF
version_id: Option<String>,
max_attempts: usize,
ignore_data_usage_cache: bool,
transfer: F,
delete_marker: D,
wait_retry: W,
) -> MigrationVersionResult
where
Backend: MigrationBackend + ?Sized,
F: FnMut(usize, String, GetObjectReader) -> Fut + Send,
Fut: Future<Output = Result<()>> + Send,
D: FnMut(String, String, ObjectOptions) -> DFut + Send,
DFut: Future<Output = Result<ObjectInfo>> + Send,
W: FnMut(Duration) -> WFut + Send,
WFut: Future<Output = ()> + Send,
{
migrate_entry_version_with_retry_wait_and_incarnation(
set,
bucket,
pool_index,
version,
version_id,
None,
max_attempts,
ignore_data_usage_cache,
transfer,
delete_marker,
wait_retry,
)
.await
}
#[allow(clippy::too_many_arguments)]
async fn migrate_entry_version_with_retry_wait_and_incarnation<Backend, F, Fut, D, DFut, W, WFut>(
set: &Backend,
bucket: String,
pool_index: usize,
version: &FileInfo,
version_id: Option<String>,
expected_bucket_incarnation_id: Option<uuid::Uuid>,
max_attempts: usize,
ignore_data_usage_cache: bool,
mut transfer: F,
mut delete_marker: D,
mut wait_retry: W,
@@ -169,7 +250,7 @@ where
&bucket,
&version.name,
version,
&rebalance_remote_tiered_opts(version, version_id, pool_index),
&rebalance_remote_tiered_opts(version, version_id, pool_index, expected_bucket_incarnation_id),
)
.await
{
@@ -212,7 +293,7 @@ where
if let Err(err) = delete_marker(
bucket.clone(),
version.name.clone(),
rebalance_delete_marker_opts(version, version_id, pool_index),
rebalance_delete_marker_opts(version, version_id, pool_index, expected_bucket_incarnation_id),
)
.await
{
@@ -255,11 +336,7 @@ where
&encode_dir_object(&version.name),
None,
HeaderMap::new(),
&ObjectOptions {
version_id: version_id.clone(),
no_lock: true,
..Default::default()
},
&rebalance_object_migration_read_opts(version_id.clone()),
)
.await
{
@@ -113,6 +113,8 @@ struct LegacyRebalanceMeta {
struct MigrationBackendSpy {
get_object_reader: Mutex<Option<core::result::Result<GetObjectReader, Error>>>,
move_remote: Mutex<Option<core::result::Result<(), Error>>>,
get_opts: Mutex<Vec<ObjectOptions>>,
move_remote_opts: Mutex<Vec<ObjectOptions>>,
get_calls: AtomicUsize,
move_remote_calls: AtomicUsize,
}
@@ -125,6 +127,8 @@ impl MigrationBackendSpy {
Self {
get_object_reader: Mutex::new(get_object_reader),
move_remote: Mutex::new(move_remote),
get_opts: Mutex::new(Vec::new()),
move_remote_opts: Mutex::new(Vec::new()),
get_calls: AtomicUsize::new(0),
move_remote_calls: AtomicUsize::new(0),
}
@@ -138,6 +142,24 @@ impl MigrationBackendSpy {
self.move_remote_calls.load(Ordering::SeqCst)
}
fn last_get_opts(&self) -> ObjectOptions {
self.get_opts
.lock()
.unwrap()
.last()
.cloned()
.expect("reader opts should be captured")
}
fn last_move_remote_opts(&self) -> ObjectOptions {
self.move_remote_opts
.lock()
.unwrap()
.last()
.cloned()
.expect("remote opts should be captured")
}
fn make_reader() -> GetObjectReader {
GetObjectReader {
stream: Box::new(Cursor::new(vec![0_u8; 3])),
@@ -156,9 +178,10 @@ impl MigrationBackend for MigrationBackendSpy {
_object: &str,
_range: Option<HTTPRangeSpec>,
_h: http::HeaderMap,
_opts: &ObjectOptions,
opts: &ObjectOptions,
) -> Result<GetObjectReader> {
self.get_calls.fetch_add(1, Ordering::SeqCst);
self.get_opts.lock().unwrap().push(opts.clone());
if let Some(result) = self.get_object_reader.lock().unwrap().take() {
return result;
}
@@ -171,9 +194,10 @@ impl MigrationBackend for MigrationBackendSpy {
_bucket: &str,
_object: &str,
_fi: &FileInfo,
_opts: &ObjectOptions,
opts: &ObjectOptions,
) -> Result<()> {
self.move_remote_calls.fetch_add(1, Ordering::SeqCst);
self.move_remote_opts.lock().unwrap().push(opts.clone());
if let Some(result) = self.move_remote.lock().unwrap().take() {
return result;
}
@@ -217,7 +241,8 @@ fn test_rebalance_delete_marker_opts_preserves_replication_state() {
..version_deleted()
};
let opts = rebalance_delete_marker_opts(&version, Some("version-id".to_string()), 7);
let incarnation = uuid::Uuid::new_v4();
let opts = rebalance_delete_marker_opts(&version, Some("version-id".to_string()), 7, Some(incarnation));
let replication = opts.delete_replication.expect("replication state should be preserved");
assert!(opts.versioned);
@@ -227,11 +252,22 @@ fn test_rebalance_delete_marker_opts_preserves_replication_state() {
assert_eq!(opts.src_pool_idx, 7);
assert_eq!(opts.version_id.as_deref(), Some("version-id"));
assert_eq!(opts.mod_time, Some(mod_time));
assert_eq!(opts.expected_bucket_incarnation_id, Some(incarnation));
assert_eq!(replication.replica_status, ReplicationStatusType::Replica);
assert!(replication.delete_marker);
assert_eq!(replication.replicate_decision_str, "existing");
}
#[test]
fn test_rebalance_delete_marker_opts_preserves_suspended_null_version() {
let version = version_deleted();
let opts = rebalance_delete_marker_opts(&version, None, 7, None);
assert!(!opts.versioned);
assert!(opts.version_suspended);
assert_eq!(opts.version_id.as_deref(), Some(uuid::Uuid::nil().to_string().as_str()));
}
#[tokio::test]
async fn test_migrate_entry_version_remote_version_is_moved_without_transfer() {
let backend = MigrationBackendSpy::new(None, Some(Ok(())));
@@ -248,12 +284,14 @@ async fn test_migrate_entry_version_remote_version_is_moved_without_transfer() {
}
};
let incarnation = uuid::Uuid::new_v4();
let result = migrate_entry_version(
&backend,
"bucket".to_string(),
0,
&version,
version.version_id.map(|v| v.to_string()),
Some(incarnation),
3,
false,
&mut transfer,
@@ -269,6 +307,10 @@ async fn test_migrate_entry_version_remote_version_is_moved_without_transfer() {
assert_eq!(transfer_count.load(Ordering::SeqCst), 0);
assert_eq!(backend.move_remote_calls(), 1);
assert_eq!(backend.get_calls(), 0);
let remote_opts = backend.last_move_remote_opts();
assert!(remote_opts.include_part_checksums);
assert!(remote_opts.http_preconditions.is_some());
assert_eq!(remote_opts.expected_bucket_incarnation_id, Some(incarnation));
}
#[tokio::test]
@@ -294,6 +336,7 @@ async fn test_migrate_entry_version_remote_not_found_is_cleanup_ignored() {
0,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -330,6 +373,7 @@ async fn test_migrate_entry_version_remote_overwrite_is_not_ignored() {
0,
&version,
Some("vid-1".to_string()),
None,
3,
false,
&mut transfer,
@@ -368,6 +412,7 @@ async fn test_migrate_entry_version_remote_failure_is_reported() {
0,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -410,6 +455,7 @@ async fn test_migrate_entry_version_deleted_version_routes_delete_through_store_
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -449,6 +495,7 @@ async fn test_migrate_entry_version_deleted_version_not_found_is_ignored() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -491,6 +538,7 @@ async fn test_migrate_entry_version_deleted_version_overwrite_is_not_ignored() {
1,
&version,
Some("vid-1".to_string()),
None,
3,
false,
&mut transfer,
@@ -520,6 +568,7 @@ async fn test_migrate_entry_version_reader_not_found_is_ignored() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -647,6 +696,7 @@ async fn test_migrate_entry_version_reader_fails_after_retries() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -685,6 +735,7 @@ async fn test_migrate_entry_version_zero_max_attempts_still_attempts_once() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
0,
false,
&mut transfer,
@@ -750,6 +801,13 @@ async fn test_migrate_entry_version_transfer_retries_before_success() {
assert_eq!(backend.get_calls(), 2);
assert_eq!(transfer_count.load(Ordering::SeqCst), 2);
assert_eq!(wait_count.load(Ordering::SeqCst), 1);
let read_opts = backend.last_get_opts();
assert_eq!(read_opts.version_id.as_deref(), version.version_id.map(|id| id.to_string()).as_deref());
assert!(read_opts.no_lock);
assert!(read_opts.data_movement);
assert!(read_opts.raw_data_movement_read);
assert!(read_opts.skip_decommissioned);
assert!(read_opts.skip_rebalancing);
}
#[tokio::test]
@@ -822,6 +880,7 @@ async fn test_migrate_entry_version_transfer_fails_after_retries() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
2,
false,
&mut transfer,
@@ -860,6 +919,7 @@ async fn test_migrate_entry_version_transfer_not_found_is_ignored() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
3,
false,
&mut transfer,
@@ -901,6 +961,7 @@ async fn test_migrate_entry_version_transfer_overwrite_is_not_ignored() {
1,
&version,
Some("vid-1".to_string()),
None,
3,
false,
&mut transfer,
@@ -943,6 +1004,7 @@ async fn test_migrate_entry_version_ignores_data_usage_cache_when_enabled() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
2,
true,
&mut transfer,
@@ -985,6 +1047,7 @@ async fn test_migrate_entry_version_data_usage_cache_moves_when_ignore_disabled(
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
2,
false,
&mut transfer,
@@ -2026,6 +2089,7 @@ async fn test_migrate_entry_version_transfer_failure_reports_write_target_stage(
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
1,
false,
&mut transfer,
@@ -2050,6 +2114,7 @@ async fn test_migrate_entry_version_reader_failure_reports_read_source_stage() {
1,
&version,
version.version_id.map(|v| v.to_string()),
None,
1,
false,
&mut transfer,
@@ -36,6 +36,7 @@ pub type RStats = Vec<Arc<RebalanceStats>>;
#[derive(Debug, Default)]
pub(super) struct RebalanceBucketConfigs {
pub(super) bucket_incarnation_id: Option<uuid::Uuid>,
pub(super) lifecycle_config: Option<s3s::dto::BucketLifecycleConfiguration>,
pub(super) object_lock_config: Option<s3s::dto::ObjectLockConfiguration>,
pub(super) replication_config: Option<(s3s::dto::ReplicationConfiguration, OffsetDateTime)>,
@@ -406,6 +406,7 @@ pub(super) async fn load_rebalance_bucket_configs(api: &ECStore, bucket: &str) -
let expiry_configs = crate::bucket::lifecycle::get_expiry_configs(api, bucket).await?;
Ok(RebalanceBucketConfigs {
bucket_incarnation_id: Some(api.bucket_incarnation_id_from_disk(bucket).await?),
lifecycle_config: expiry_configs.lifecycle.map(|config| (*config).clone()),
object_lock_config: expiry_configs.object_lock.map(|config| (*config).clone()),
replication_config: resolve_rebalance_optional_bucket_config_result(
File diff suppressed because it is too large Load Diff
+83
View File
@@ -85,6 +85,15 @@ impl SetDisks {
format!("{}/{}", Self::get_multipart_sha_dir(bucket, object), upload_uuid)
}
pub(super) fn get_multipart_upload_dir(bucket: &str, object: &str, upload_id: &str, data_movement: bool) -> String {
let upload_dir = Self::get_upload_id_dir(bucket, object, upload_id);
if data_movement {
format!("{DATA_MOVEMENT_MULTIPART_PREFIX}/{upload_dir}")
} else {
upload_dir
}
}
pub(super) fn get_multipart_sha_dir(bucket: &str, object: &str) -> String {
let path = format!("{bucket}/{object}");
let mut hasher = Sha256::new();
@@ -466,6 +475,28 @@ impl SetDisks {
Self::find_file_info_in_quorum(metas, &mod_time, &etag, quorum)
}
pub(crate) fn hydrate_selected_fileinfo_part_checksums(fi: &mut FileInfo) -> disk::error::Result<()> {
fi.hydrate_data_movement_part_checksums().map_err(DiskError::from)?;
for part in &fi.parts {
let Some(checksums) = part.checksums.as_ref() else {
continue;
};
let mut algorithms = HashSet::with_capacity(checksums.len());
for (name, value) in checksums {
let Some(checksum) = rustfs_rio::Checksum::new_from_string(name, value) else {
return Err(DiskError::FileCorrupt);
};
if checksum.checksum_type.is(rustfs_rio::ChecksumType::MULTIPART) {
return Err(DiskError::FileCorrupt);
}
if !algorithms.insert(checksum.checksum_type.base().0) {
return Err(DiskError::FileCorrupt);
}
}
}
Ok(())
}
fn update_hash_bytes(hasher: &mut Sha256, value: &[u8]) {
hasher.update(value.len().to_le_bytes());
hasher.update(value);
@@ -1079,6 +1110,25 @@ impl SetDisks {
shuffled_disks
}
pub(super) fn shuffle_disks_owned(mut disks: Vec<Option<DiskStore>>, distribution: &[usize]) -> Vec<Option<DiskStore>> {
if distribution.is_empty() {
return disks;
}
let mut shuffled_disks = vec![None; disks.len()];
for (index, disk) in disks.iter_mut().enumerate() {
let Some(slot) = distribution
.get(index)
.and_then(|block_index| block_index.checked_sub(1))
.filter(|slot| *slot < shuffled_disks.len())
else {
continue;
};
shuffled_disks[slot] = disk.take();
}
shuffled_disks
}
pub(super) fn shuffle_check_parts(parts_errs: &[usize], distribution: &[usize]) -> Vec<usize> {
if distribution.is_empty() {
return parts_errs.to_vec();
@@ -1390,6 +1440,23 @@ mod tests {
assert_eq!(owned_slots, expected_slots, "fallback disk slots must match the borrowing variant");
}
#[tokio::test]
async fn owned_shuffle_preserves_fresh_put_metadata() {
let tempdir = tempfile::tempdir().expect("tempdir should be created");
let fi = FileInfo::new("bucket/object", 2, 1);
let parts = vec![fi.clone(); fi.erasure.distribution.len()];
let disks = shuffle_test_disks(&tempdir, parts.len()).await;
let (owned_disks, owned_parts) = SetDisks::shuffle_disks_and_parts_metadata_by_index_owned(disks, parts, &fi);
assert!(owned_disks.iter().all(Option::is_some), "fresh PUT must retain every online disk");
assert_eq!(
owned_parts,
vec![fi; owned_disks.len()],
"fresh PUT metadata with pending shard indexes must survive init fallback"
);
}
// backlog#949: corrupt/adversarial distribution values (0 or > N) must not
// trigger a `usize` underflow / out-of-bounds panic in the shuffle helpers.
#[test]
@@ -1419,6 +1486,22 @@ mod tests {
assert_eq!(result.len(), disks.len(), "output length must be preserved");
}
#[tokio::test]
async fn owned_disk_shuffle_matches_borrowing_variant() {
let tempdir = tempfile::tempdir().expect("tempdir should be created");
let mut disks = shuffle_test_disks(&tempdir, 4).await;
disks[1] = None;
disks[3] = None;
let distribution = [3, 1, 4, 2];
let expected = SetDisks::shuffle_disks(&disks, &distribution);
let actual = SetDisks::shuffle_disks_owned(disks, &distribution);
let expected_slots = expected.iter().map(Option::is_some).collect::<Vec<_>>();
let actual_slots = actual.iter().map(Option::is_some).collect::<Vec<_>>();
assert_eq!(actual_slots, expected_slots, "owned shuffle must preserve disk placement");
}
#[tokio::test]
async fn shuffle_disks_and_parts_metadata_survives_corrupt_distribution() {
let tempdir = tempfile::tempdir().expect("tempdir should be created");
File diff suppressed because it is too large Load Diff
+4 -2
View File
@@ -542,7 +542,8 @@ impl SetDisks {
let filter_by_etag = quorum_etag.is_some();
match Self::pick_valid_fileinfo(&parts_metadata, quorum_mod_time, quorum_etag.clone(), read_quorum as usize) {
Ok(latest_meta) => {
Ok(mut latest_meta) => {
Self::hydrate_selected_fileinfo_part_checksums(&mut latest_meta)?;
trace!(
event = EVENT_SET_DISK_HEAL,
component = LOG_COMPONENT_ECSTORE,
@@ -3170,10 +3171,11 @@ mod heal_result_report_tests {
.await
.expect("object should be written");
let (fi, _, _) = set
let snapshot = set
.get_object_fileinfo(bucket, object, &opts, true, false)
.await
.expect("object metadata should resolve");
let fi = snapshot.fi();
assert_eq!(fi.erasure.parity_blocks, 0);
let data_dir = fi.data_dir.expect("non-inline object should have a data directory");
let part_path = dir.path().join(bucket).join(object).join(data_dir.to_string()).join("part.1");
+14 -3
View File
@@ -36,10 +36,21 @@ impl crate::storage_api_contracts::namespace::NamespaceLocking for SetDisks {
// test's transient DistErasure window) would push this set's namespace
// locking onto its own — possibly empty — dist locker list.
let set_lock = if self.ctx.is_dist_erasure().await {
// Calculate quorum based on lockers count (majority)
let lockers_count = self.lockers.len();
let lockers = if self.lockers.len() == self.shared_lockers.len()
&& self
.lockers
.iter()
.zip(self.shared_lockers.iter())
.all(|(current, shared)| Arc::ptr_eq(current, shared))
{
self.shared_lockers.clone()
} else {
Arc::from(self.lockers.clone())
};
// Calculate quorum from the exact client domain used by this lock.
let lockers_count = lockers.len();
let write_quorum = if lockers_count > 1 { (lockers_count / 2) + 1 } else { 1 };
NamespaceLock::with_clients_and_quorum_shared(self.set_lock_namespace.clone(), self.lockers.clone(), write_quorum)
NamespaceLock::with_clients_and_quorum_shared(self.set_lock_namespace.clone(), lockers, write_quorum)
} else {
NamespaceLock::with_local_manager_shared(self.set_lock_namespace.clone(), self.local_lock_manager.clone())
};
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+216 -45
View File
@@ -19,7 +19,7 @@ use crate::diagnostics::get::{
GET_METADATA_CACHE_REASON_DATA_MOVEMENT, GET_METADATA_CACHE_REASON_DELETE_MARKER, GET_METADATA_CACHE_REASON_DIST_ERASURE,
GET_METADATA_CACHE_REASON_INCL_FREE_VERSIONS, GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM,
GET_METADATA_CACHE_REASON_META_BUCKET, GET_METADATA_CACHE_REASON_NO_LOCK, GET_METADATA_CACHE_REASON_NOT_FOUND_OR_EXPIRED,
GET_METADATA_CACHE_REASON_NOT_READ_DATA, GET_METADATA_CACHE_REASON_PART_NUMBER,
GET_METADATA_CACHE_REASON_NOT_READ_DATA, GET_METADATA_CACHE_REASON_PART_CHECKSUMS, GET_METADATA_CACHE_REASON_PART_NUMBER,
GET_METADATA_CACHE_REASON_RAW_DATA_MOVEMENT_READ, GET_METADATA_CACHE_REASON_STALE_PUBLICATION,
GET_METADATA_CACHE_REASON_USABLE, GET_METADATA_CACHE_REASON_VERSION_ID, GET_METADATA_CACHE_REASON_VERSION_SUSPENDED,
GET_METADATA_CACHE_REASON_VERSIONED, GET_METADATA_EARLY_STOP_REASON_CONFLICTING_METADATA,
@@ -180,9 +180,9 @@ impl SetDisks {
let key = GetObjectMetadataCacheKey::new(bucket, object, generation);
let entry = Arc::new(GetObjectMetadataCacheEntry {
created_at: Instant::now(),
fi: Arc::new(fi.clone()),
parts_metadata: Arc::new(parts_metadata.to_vec()),
online_disks: Arc::new(online_disks.to_vec()),
fi: fi.clone(),
parts_metadata: parts_metadata.to_vec(),
online_disks: online_disks.to_vec(),
read_quorum,
});
self.insert_get_object_metadata_cache_entry_after_insert(key, generation, entry, || {})
@@ -300,11 +300,7 @@ impl SetDisks {
GET_STAGE_METADATA_CACHE_LOOKUP,
metadata_cache_lookup_start,
);
return Ok((
GetObjectMetadata::Shared(Arc::clone(&cached.fi)),
GetObjectMetadata::Shared(Arc::clone(&cached.parts_metadata)),
GetObjectMetadata::Shared(Arc::clone(&cached.online_disks)),
));
return Ok(GetObjectFileInfo::shared(cached));
}
MetadataCacheLookup::Miss => {
rustfs_io_metrics::record_get_object_metadata_cache_decision(
@@ -340,7 +336,7 @@ impl SetDisks {
// read_all_fileinfo_observed (see read_all_fileinfo_early_stop in
// core/io_primitives.rs); unsafe requests and callers that opt out
// (allow_early_stop=false) fall back to full-wait.
let (parts_metadata, errs, metadata_fanout_diagnostics) = Self::read_all_fileinfo_observed(
let (mut parts_metadata, errs, metadata_fanout_diagnostics) = Self::read_all_fileinfo_observed(
&disks,
"",
bucket,
@@ -394,8 +390,17 @@ impl SetDisks {
return Err(to_object_err(err.into(), vec![bucket, object]));
}
let (op_online_disks, fi, fileinfo_selection_quorum) =
let (op_online_disks, mut fi, fileinfo_selection_quorum) =
Self::select_valid_fileinfo(&disks, &parts_metadata, &errs, vid.as_str(), read_quorum, write_quorum)?;
let include_part_checksums =
opts.include_part_checksums || opts.part_number.is_some() || opts.data_movement || opts.raw_data_movement_read;
if include_part_checksums {
Self::hydrate_selected_fileinfo_part_checksums(&mut fi)?;
} else {
for metadata in std::iter::once(&mut fi).chain(parts_metadata.iter_mut()) {
rustfs_utils::http::remove_str(&mut metadata.metadata, rustfs_utils::http::SUFFIX_PART_CHECKSUMS);
}
}
metadata_fanout_diagnostics.record_quorum_candidate_latency(metadata_metrics_path, fileinfo_selection_quorum);
if errs.iter().any(|err| err.is_some()) {
let version_id = resolved_read_repair_version_id(&fi, opts.version_id.as_deref());
@@ -427,11 +432,7 @@ impl SetDisks {
// let online_disks: Vec<Option<DiskStore>> = op_online_disks.iter().filter(|v| v.is_some()).cloned().collect();
Ok((
GetObjectMetadata::Owned(fi),
GetObjectMetadata::Owned(parts_metadata),
GetObjectMetadata::Owned(op_online_disks),
))
Ok(GetObjectFileInfo::owned(fi, parts_metadata, op_online_disks))
}
#[hotpath::measure(impl_type = "SetDisks")]
@@ -441,14 +442,15 @@ impl SetDisks {
object: &str,
opts: &ObjectOptions,
) -> (ObjectInfo, usize, Option<StorageError>) {
let fi = match self.get_object_fileinfo(bucket, object, opts, false, false).await {
Ok((fi, _, _)) => fi,
let snapshot = match self.get_object_fileinfo(bucket, object, opts, false, false).await {
Ok(snapshot) => snapshot,
Err(e) => return (ObjectInfo::default(), 0, Some(e)),
};
let fi = snapshot.fi();
let write_quorum = fi.write_quorum(self.default_write_quorum());
let oi = ObjectInfo::from_file_info(&fi, bucket, object, opts.versioned || opts.version_suspended);
let oi = ObjectInfo::from_file_info(fi, bucket, object, opts.versioned || opts.version_suspended);
if !fi.version_purge_status().is_empty() && opts.version_id.is_some() {
return (
@@ -1826,6 +1828,9 @@ fn get_object_metadata_cache_request_bypass_reason(bucket: &str, opts: &ObjectOp
if opts.part_number.is_some() {
return Some(GET_METADATA_CACHE_REASON_PART_NUMBER);
}
if opts.include_part_checksums {
return Some(GET_METADATA_CACHE_REASON_PART_CHECKSUMS);
}
if opts.data_movement {
return Some(GET_METADATA_CACHE_REASON_DATA_MOVEMENT);
}
@@ -2497,6 +2502,16 @@ mod metadata_cache_tests {
Some(GET_METADATA_CACHE_REASON_PART_NUMBER)
);
opts = ObjectOptions {
include_part_checksums: true,
..Default::default()
};
assert!(!is_get_object_metadata_cache_request_eligible("bucket", &opts, true));
assert_eq!(
get_object_metadata_cache_request_bypass_reason("bucket", &opts, true),
Some(GET_METADATA_CACHE_REASON_PART_CHECKSUMS)
);
opts = ObjectOptions {
data_movement: true,
..Default::default()
@@ -2701,22 +2716,14 @@ mod metadata_cache_tests {
.await
.expect("fresh cache entry should be returned");
let (returned_fi, returned_parts_metadata, returned_online_disks) = set
let returned = set
.get_object_fileinfo("bucket", "object", &ObjectOptions::default(), true, false)
.await
.expect("cache-backed metadata lookup should succeed");
assert!(
matches!(returned_fi, GetObjectMetadata::Shared(ref value) if Arc::ptr_eq(value, &cached.fi)),
"cache hits must share FileInfo ownership"
);
assert!(
matches!(returned_parts_metadata, GetObjectMetadata::Shared(ref value) if Arc::ptr_eq(value, &cached.parts_metadata)),
"cache hits must share the metadata vector"
);
assert!(
matches!(returned_online_disks, GetObjectMetadata::Shared(ref value) if Arc::ptr_eq(value, &cached.online_disks)),
"cache hits must share the online-disk vector"
returned.shared_entry().is_some_and(|value| Arc::ptr_eq(value, &cached)),
"cache hits must share the complete metadata snapshot"
);
}
@@ -2759,9 +2766,9 @@ mod metadata_cache_tests {
),
Arc::new(GetObjectMetadataCacheEntry {
created_at: Instant::now(),
fi: Arc::new(fi.clone()),
parts_metadata: Arc::new(vec![fi]),
online_disks: Arc::new(vec![None]),
fi: fi.clone(),
parts_metadata: vec![fi],
online_disks: vec![None],
read_quorum: 1,
}),
)
@@ -2855,13 +2862,12 @@ mod metadata_cache_tests {
barrier.wait_until_paused().await;
set.invalidate_get_object_metadata_cache(bucket, object).await;
barrier.release();
let (fi, parts_metadata, online_disks) = read
let snapshot = read
.await
.expect("metadata read task should not panic")
.expect("metadata fanout should still return its selected FileInfo");
assert!(matches!(fi, GetObjectMetadata::Owned(_)));
assert!(matches!(parts_metadata, GetObjectMetadata::Owned(_)));
assert!(matches!(online_disks, GetObjectMetadata::Owned(_)));
assert!(snapshot.owned.is_some());
assert!(snapshot.has_valid_representation());
assert!(
set.get_object_metadata_cache
@@ -2908,9 +2914,9 @@ mod metadata_cache_tests {
let key = GetObjectMetadataCacheKey::new("bucket", "object", generation);
let entry = Arc::new(GetObjectMetadataCacheEntry {
created_at: Instant::now(),
fi: Arc::new(fi.clone()),
parts_metadata: Arc::new(vec![fi]),
online_disks: Arc::new(Vec::new()),
fi: fi.clone(),
parts_metadata: vec![fi],
online_disks: Vec::new(),
read_quorum: 0,
});
@@ -3018,9 +3024,9 @@ mod metadata_cache_tests {
let entry = |fi: FileInfo| {
Arc::new(GetObjectMetadataCacheEntry {
created_at: Instant::now(),
parts_metadata: Arc::new(vec![fi.clone()]),
fi: Arc::new(fi),
online_disks: Arc::new(Vec::new()),
parts_metadata: vec![fi.clone()],
fi,
online_disks: Vec::new(),
read_quorum: 0,
})
};
@@ -4083,8 +4089,8 @@ mod tests {
get_codec_streaming_reader_gate(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
range,
None,
classify_get_codec_streaming_object_class(range, object_info, fi),
object_info,
fi,
lock_optimization_enabled,
@@ -4101,8 +4107,8 @@ mod tests {
get_codec_streaming_reader_gate(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
range,
part_number,
classify_get_codec_streaming_object_class(range, object_info, fi),
object_info,
fi,
lock_optimization_enabled,
@@ -4828,6 +4834,114 @@ mod tests {
.await
}
async fn encoded_inline_blocks(blocks: &[&[u8]], shard_size: usize, hash_algo: HashAlgorithm) -> Bytes {
let mut writer = BitrotWriter::new(Cursor::new(Vec::new()), shard_size, hash_algo);
for block in blocks {
writer.write(block).await.expect("test block should be encoded");
}
Bytes::from(writer.into_inner().into_inner())
}
fn assert_reader_shares_inline_allocation(reader: &ObjectBitrotReader, source: &Bytes) {
let reader_bytes = reader
.inner_ref()
.inline_bytes()
.expect("inline scheduler should retain an in-memory Bytes source");
assert_eq!(
reader_bytes.as_ptr(),
source.as_ptr(),
"the scheduler must clone Bytes ownership instead of copying the inline shard payload"
);
}
#[tokio::test]
async fn inline_range_scheduler_shares_bytes_and_rejects_bitrot_mismatch() {
const SHARD_SIZE: usize = 16;
let hash_algo = HashAlgorithm::HighwayHash256S;
let first = [b'a'; SHARD_SIZE];
let second = [b'b'; SHARD_SIZE];
let mut source = encoded_inline_blocks(&[&first, &second], SHARD_SIZE, hash_algo.clone()).await;
let second_payload = hash_algo.size() * 2 + SHARD_SIZE;
source = {
let mut corrupt = source.to_vec();
corrupt[second_payload] ^= 0xff;
Bytes::from(corrupt)
};
let files = vec![encoded_reader_setup_fileinfo(Some(source.to_vec()))];
let source = files[0].data.clone().expect("inline shard should exist");
let disks = vec![None];
let mut setup = create_bitrot_readers_until_quorum_with_preference(
&files,
&disks,
"bucket",
"object",
1,
SHARD_SIZE,
SHARD_SIZE,
SHARD_SIZE,
hash_algo,
false,
false,
1,
0,
BitrotReaderSetupMode::ReadQuorum,
true,
None,
None,
)
.await;
let mut reader = setup.readers[0].take().expect("range reader should be ready");
assert_reader_shares_inline_allocation(&reader, &source);
let err = reader
.read(&mut [0; SHARD_SIZE])
.await
.expect_err("corrupt ranged inline block must fail bitrot verification");
assert_eq!(err.kind(), ErrorKind::InvalidData);
}
#[tokio::test]
async fn inline_part_scheduler_shares_bytes_and_rejects_bitrot_mismatch() {
const SHARD_SIZE: usize = 16;
let hash_algo = HashAlgorithm::HighwayHash256S;
let block = [b'p'; SHARD_SIZE];
let encoded = encoded_inline_blocks(&[&block], SHARD_SIZE, hash_algo.clone()).await;
let mut corrupt = encoded.to_vec();
corrupt[hash_algo.size()] ^= 0xff;
let files = vec![encoded_reader_setup_fileinfo(Some(corrupt))];
let source = files[0].data.clone().expect("inline shard should exist");
let disks = vec![None];
let mut setup = create_bitrot_readers_until_quorum_all_shards(
&files,
&disks,
"bucket",
"object",
7,
0,
SHARD_SIZE,
SHARD_SIZE,
hash_algo,
false,
false,
1,
0,
BitrotReaderSetupMode::VerifyReconstruction,
None,
None,
)
.await;
let mut reader = setup.readers[0].take().expect("part reader should be ready");
assert_reader_shares_inline_allocation(&reader, &source);
let err = reader
.read(&mut [0; SHARD_SIZE])
.await
.expect_err("corrupt inline part must fail bitrot verification");
assert_eq!(err.kind(), ErrorKind::InvalidData);
}
async fn decode_codec_data_blocks_first_setup(
erasure: coding::Erasure,
data: &[u8],
@@ -5528,6 +5642,63 @@ mod tests {
});
}
#[test]
fn codec_streaming_config_cache_loads_once() {
use std::cell::Cell;
let loads = Cell::new(0);
let expected = GetCodecStreamingConfig {
enabled: true,
rollout: GetCodecStreamingRollout::Off,
rollout_pct: 100,
body_compat_confirmed: true,
header_compat_confirmed: true,
engine: GetCodecStreamingEngine::Legacy,
min_size: DEFAULT_RUSTFS_GET_CODEC_STREAMING_MIN_SIZE,
};
for _ in 0..3 {
assert_eq!(
get_codec_streaming_config_cached_core(|| {
loads.set(loads.get() + 1);
expected
}),
expected
);
}
assert_eq!(loads.get(), 1, "production config cache must not reload env per GET");
}
#[test]
fn codec_streaming_config_loader_preserves_all_gate_env_overrides() {
temp_env::with_vars(
[
(ENV_RUSTFS_GET_CODEC_STREAMING_ENABLE, Some("false")),
(ENV_RUSTFS_GET_CODEC_STREAMING_ENGINE, Some(GET_CODEC_STREAMING_ENGINE_RUSTFS)),
(ENV_RUSTFS_GET_CODEC_STREAMING_ROLLOUT, Some("production")),
(ENV_RUSTFS_GET_CODEC_STREAMING_ROLLOUT_PCT, Some("37")),
(ENV_RUSTFS_GET_CODEC_STREAMING_BODY_COMPAT_CONFIRMED, Some("false")),
(ENV_RUSTFS_GET_CODEC_STREAMING_HEADER_COMPAT_CONFIRMED, Some("false")),
(ENV_RUSTFS_GET_CODEC_STREAMING_MIN_SIZE, None::<&str>),
(ENV_RUSTFS_GET_CODEC_STREAMING_RUSTFS_MIN_SIZE, Some("262144")),
],
|| {
assert_eq!(
load_get_codec_streaming_config(),
GetCodecStreamingConfig {
enabled: false,
rollout: GetCodecStreamingRollout::On,
rollout_pct: 37,
body_compat_confirmed: false,
header_compat_confirmed: false,
engine: GetCodecStreamingEngine::Rustfs,
min_size: 262144,
}
);
},
);
}
#[test]
fn codec_streaming_default_min_size_meets_direct_memory_ceiling() {
for engine in [None, Some(GET_CODEC_STREAMING_ENGINE_RUSTFS)] {
+8 -6
View File
@@ -75,12 +75,13 @@ impl SetDisks {
version_id,
versioned: opts.versioned,
version_suspended: opts.version_suspended,
include_part_checksums: true,
..Default::default()
};
let (fi, _, disks) = self
let (mut fi, _, disks) = self
.get_object_fileinfo_gated(bucket, object, &read_opts, false, false)
.await?;
let mut fi = fi.into_owned();
.await?
.into_owned();
if let Some(expected_operation_id) = expected_operation_id {
require_restore_operation_id(&fi.metadata, expected_operation_id)?;
}
@@ -142,12 +143,13 @@ impl SetDisks {
version_id,
versioned: opts.versioned,
version_suspended: opts.version_suspended,
include_part_checksums: true,
..Default::default()
};
let (fi, _, disks) = self
let (mut fi, _, disks) = self
.get_object_fileinfo_gated(bucket, object, &read_opts, false, false)
.await?;
let mut fi = fi.into_owned();
.await?
.into_owned();
if let Some(expected_operation_id) = expected_operation_id {
match restore_operation_id_from_metadata(&fi.metadata)? {
Some(actual_operation_id) if actual_operation_id == expected_operation_id => {}
+94 -159
View File
@@ -16,6 +16,14 @@ use crate::diagnostics::get::{
GET_SHARD_READ_COST_LOCAL, GET_SHARD_READ_COST_REMOTE, GET_SHARD_READ_COST_SAME_NODE, GET_SHARD_READ_COST_UNKNOWN,
};
use crate::disk::error::Error;
use crate::layout::disks_layout::MAX_ERASURE_SET_DRIVE_COUNT;
use smallvec::SmallVec;
/// Generic codec callers may exceed the production set limit; `SmallVec` then
/// spills without changing slot semantics.
pub(crate) const INLINE_SHARD_SLOTS: usize = MAX_ERASURE_SET_DRIVE_COUNT;
pub(crate) type ShardBuffers = SmallVec<[Option<Vec<u8>>; INLINE_SHARD_SLOTS]>;
pub(crate) type ShardErrors = SmallVec<[Option<Error>; INLINE_SHARD_SLOTS]>;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum ShardReadCost {
@@ -43,202 +51,137 @@ impl ShardReadCost {
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct ShardSlot {
index: usize,
read_cost: ShardReadCost,
data: Option<Vec<u8>>,
error: Option<Error>,
}
impl ShardSlot {
pub(crate) fn new(index: usize, data: Option<Vec<u8>>, error: Option<Error>) -> Self {
Self::with_read_cost(index, ShardReadCost::Unknown, data, error)
}
pub(crate) fn with_read_cost(index: usize, read_cost: ShardReadCost, data: Option<Vec<u8>>, error: Option<Error>) -> Self {
Self {
index,
read_cost,
data,
error,
}
}
pub(crate) fn data(index: usize, data: Vec<u8>) -> Self {
Self::new(index, Some(data), None)
}
pub(crate) fn data_with_read_cost(index: usize, read_cost: ShardReadCost, data: Vec<u8>) -> Self {
Self::with_read_cost(index, read_cost, Some(data), None)
}
pub(crate) fn missing(index: usize, error: Error) -> Self {
Self::new(index, None, Some(error))
}
pub(crate) fn missing_with_read_cost(index: usize, read_cost: ShardReadCost, error: Error) -> Self {
Self::with_read_cost(index, read_cost, None, Some(error))
}
pub(crate) fn index(&self) -> usize {
self.index
}
pub(crate) fn read_cost(&self) -> ShardReadCost {
self.read_cost
}
pub(crate) fn has_data(&self) -> bool {
self.data.is_some()
}
pub(crate) fn data_bytes(&self) -> Option<&[u8]> {
self.data.as_deref()
}
pub(crate) fn error(&self) -> Option<&Error> {
self.error.as_ref()
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct StripeReadState {
slots: Vec<ShardSlot>,
shards: ShardBuffers,
errors: ShardErrors,
read_quorum: usize,
}
impl StripeReadState {
pub(crate) fn new(slots: Vec<ShardSlot>, read_quorum: usize) -> Self {
Self { slots, read_quorum }
}
#[cfg(test)]
pub(crate) fn from_parts(shards: Vec<Option<Vec<u8>>>, errors: Vec<Option<Error>>, read_quorum: usize) -> Self {
Self::from_parts_with_read_costs(shards, errors, &[], read_quorum)
let mut shards = SmallVec::from_vec(shards);
let mut errors = SmallVec::from_vec(errors);
let slot_count = shards.len().max(errors.len());
shards.resize_with(slot_count, || None);
errors.resize_with(slot_count, || None);
Self {
shards,
errors,
read_quorum,
}
}
pub(crate) fn from_parts_with_read_costs<S, E>(shards: S, errors: E, read_costs: &[ShardReadCost], read_quorum: usize) -> Self
where
S: IntoIterator<Item = Option<Vec<u8>>>,
S::IntoIter: ExactSizeIterator,
E: IntoIterator<Item = Option<Error>>,
E::IntoIter: ExactSizeIterator,
{
let mut shards = shards.into_iter();
let mut errors = errors.into_iter();
let slot_count = shards.len().max(errors.len());
let mut slots = Vec::with_capacity(slot_count);
for index in 0..slot_count {
let read_cost = read_costs.get(index).copied().unwrap_or(ShardReadCost::Unknown);
slots.push(ShardSlot::with_read_cost(
index,
read_cost,
shards.next().flatten(),
errors.next().flatten(),
));
}
Self::new(slots, read_quorum)
pub(crate) fn with_slot_count(slot_count: usize, read_quorum: usize) -> Self {
let mut state = Self {
shards: SmallVec::new(),
errors: SmallVec::new(),
read_quorum,
};
state.reset(slot_count, read_quorum);
state
}
pub(crate) fn reset(&mut self, slot_count: usize, read_quorum: usize) {
self.shards.clear();
self.shards.resize_with(slot_count, || None);
self.errors.clear();
self.errors.resize_with(slot_count, || None);
self.read_quorum = read_quorum;
}
pub(crate) fn available_shards(&self) -> usize {
self.slots.iter().filter(|slot| slot.has_data()).count()
self.shards.iter().filter(|shard| shard.is_some()).count()
}
pub(crate) fn can_decode(&self) -> bool {
self.available_shards() >= self.read_quorum
}
pub(crate) fn slots(&self) -> &[ShardSlot] {
&self.slots
pub(crate) fn is_empty(&self) -> bool {
self.shards.is_empty()
}
pub(crate) fn slot_by_index(&self, index: usize) -> Option<&ShardSlot> {
if let Some(slot) = self.slots.get(index)
&& slot.index == index
{
return Some(slot);
}
self.slots.iter().find(|slot| slot.index == index)
pub(crate) fn data_bytes(&self, index: usize) -> Option<&[u8]> {
self.shards.get(index).and_then(Option::as_deref)
}
#[cfg(test)]
pub(crate) fn error(&self, index: usize) -> Option<&Error> {
self.errors.get(index).and_then(Option::as_ref)
}
pub(crate) fn data_shards_complete(&self, data_shards: usize) -> bool {
(0..data_shards).all(|index| self.slot_by_index(index).is_some_and(ShardSlot::has_data))
self.shards.len() >= data_shards && self.shards.iter().take(data_shards).all(Option::is_some)
}
pub(crate) fn into_parts(self) -> (Vec<Option<Vec<u8>>>, Vec<Option<Error>>) {
let part_count = self.slots.iter().map(|slot| slot.index).max().map_or(0, |index| index + 1);
let mut shards = Vec::with_capacity(part_count);
shards.resize_with(part_count, || None);
let mut errors = Vec::with_capacity(part_count);
errors.resize_with(part_count, || None);
for slot in self.slots {
shards[slot.index] = slot.data;
errors[slot.index] = slot.error;
}
(shards, errors)
pub(crate) fn parts_mut(&mut self) -> (&mut ShardBuffers, &mut ShardErrors) {
(&mut self.shards, &mut self.errors)
}
pub(crate) fn shards_mut(&mut self) -> &mut ShardBuffers {
&mut self.shards
}
pub(crate) fn into_parts(self) -> (ShardBuffers, ShardErrors) {
(self.shards, self.errors)
}
#[cfg(test)]
pub(crate) fn scratch_storage(&self) -> (*const Option<Vec<u8>>, *const Option<Error>, bool, bool) {
(self.shards.as_ptr(), self.errors.as_ptr(), self.shards.spilled(), self.errors.spilled())
}
#[cfg(test)]
pub(crate) fn shard_allocation(&self, index: usize) -> Option<(*const u8, usize)> {
self.shards
.get(index)
.and_then(|shard| shard.as_ref().map(|shard| (shard.as_ptr(), shard.capacity())))
}
}
#[async_trait::async_trait]
pub(crate) trait ShardStripeSource: Send {
async fn read_next_stripe(&mut self) -> StripeReadState;
async fn read_next_stripe(&mut self) -> Box<StripeReadState>;
fn recycle_stripe(&mut self, _state: Box<StripeReadState>) {}
}
#[cfg(test)]
mod tests {
use super::*;
use std::mem::size_of;
#[test]
fn stripe_read_state_tracks_decode_quorum() {
let state = StripeReadState::new(
vec![
ShardSlot::data_with_read_cost(0, ShardReadCost::Local, vec![1]),
ShardSlot::missing_with_read_cost(1, ShardReadCost::Remote, Error::FileNotFound),
ShardSlot::data_with_read_cost(2, ShardReadCost::SameNode, vec![2]),
],
2,
);
fn stripe_scratch_capacity_matches_the_production_set_limit() {
type OversizedShardBuffers = SmallVec<[Option<Vec<u8>>; 32]>;
type OversizedShardErrors = SmallVec<[Option<Error>; 32]>;
assert_eq!(INLINE_SHARD_SLOTS, MAX_ERASURE_SET_DRIVE_COUNT);
assert!(size_of::<ShardBuffers>() < size_of::<OversizedShardBuffers>());
assert!(size_of::<ShardErrors>() < size_of::<OversizedShardErrors>());
}
#[test]
fn stripe_read_state_tracks_decode_quorum_and_slot_access() {
let state =
StripeReadState::from_parts(vec![Some(vec![1]), None, Some(vec![2])], vec![None, Some(Error::FileNotFound), None], 2);
assert_eq!(state.available_shards(), 2);
assert!(state.can_decode());
assert_eq!(state.slots()[1].index(), 1);
assert_eq!(state.slots()[0].read_cost(), ShardReadCost::Local);
assert!(state.slots()[2].read_cost().is_low_cost());
assert_eq!(state.data_bytes(0), Some(&[1][..]));
assert_eq!(state.error(1), Some(&Error::FileNotFound));
}
#[test]
fn stripe_read_state_preserves_shards_and_errors() {
let state = StripeReadState::new(vec![ShardSlot::missing(1, Error::FileCorrupt), ShardSlot::data(0, vec![1, 2, 3])], 2);
let state = StripeReadState::from_parts(vec![Some(vec![1, 2, 3]), None], vec![None, Some(Error::FileCorrupt)], 2);
assert!(!state.can_decode());
let (shards, errors) = state.into_parts();
assert_eq!(shards, vec![Some(vec![1, 2, 3]), None]);
assert_eq!(errors, vec![None, Some(Error::FileCorrupt)]);
}
#[test]
fn stripe_read_state_builds_slots_from_parallel_reader_parts() {
let state =
StripeReadState::from_parts(vec![Some(vec![1]), None, Some(vec![3])], vec![None, Some(Error::FileNotFound)], 2);
assert!(state.can_decode());
assert_eq!(state.slots()[1].index(), 1);
assert_eq!(state.slots()[1].error(), Some(&Error::FileNotFound));
}
#[test]
fn stripe_read_state_preserves_read_cost_hints() {
let state = StripeReadState::from_parts_with_read_costs(
vec![Some(vec![1]), None, Some(vec![3])],
vec![None, Some(Error::FileNotFound)],
&[ShardReadCost::Local, ShardReadCost::Remote, ShardReadCost::Unknown],
2,
);
assert_eq!(state.slots()[0].read_cost(), ShardReadCost::Local);
assert_eq!(state.slots()[1].read_cost(), ShardReadCost::Remote);
assert_eq!(state.slots()[2].read_cost(), ShardReadCost::Unknown);
assert_eq!(ShardReadCost::SameNode.as_str(), GET_SHARD_READ_COST_SAME_NODE);
assert_eq!(shards.as_slice(), &[Some(vec![1, 2, 3]), None]);
assert_eq!(errors.as_slice(), &[None, Some(Error::FileCorrupt)]);
}
#[test]
@@ -258,8 +201,8 @@ mod tests {
let state = StripeReadState::from_parts(vec![Some(vec![1]), Some(vec![2]), None], Vec::new(), 2);
assert!(state.data_shards_complete(2));
assert_eq!(state.slots()[0].data_bytes(), Some(&[1][..]));
assert_eq!(state.slot_by_index(1).and_then(ShardSlot::data_bytes), Some(&[2][..]));
assert_eq!(state.data_bytes(0), Some(&[1][..]));
assert_eq!(state.data_bytes(1), Some(&[2][..]));
}
#[test]
@@ -268,12 +211,4 @@ mod tests {
assert!(!state.data_shards_complete(2));
}
#[test]
fn stripe_read_state_finds_out_of_order_slots_by_index() {
let state = StripeReadState::new(vec![ShardSlot::data(2, vec![3]), ShardSlot::data(0, vec![1])], 2);
assert_eq!(state.slot_by_index(0).and_then(ShardSlot::data_bytes), Some(&[1][..]));
assert!(state.slot_by_index(1).is_none());
}
}
+4
View File
@@ -161,6 +161,10 @@ impl BucketIncarnationFenceGuard {
pub(crate) fn is_lock_lost(&self) -> bool {
self.inner.as_ref().is_some_and(NamespaceLockGuard::is_lock_lost)
}
pub(crate) fn namespace_lock_guard(&self) -> Option<&NamespaceLockGuard> {
self.inner.as_ref()
}
}
impl Drop for BucketIncarnationFenceGuard {
File diff suppressed because it is too large Load Diff
+28
View File
@@ -309,9 +309,17 @@ const ENV_API_LIST_OBJECTS_INDEX_PROVIDER: &str = "RUSTFS_LIST_OBJECTS_INDEX_PRO
const ENV_API_LIST_OBJECTS_INDEX_PROVIDER_PATH: &str = "RUSTFS_LIST_OBJECTS_INDEX_PROVIDER_PATH";
const ENV_API_LIST_OBJECTS_INDEX_PROVIDER_GENERATION: &str = "RUSTFS_LIST_OBJECTS_INDEX_PROVIDER_GENERATION";
const ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_PATH: &str = "RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_PATH";
// The chaos machinery below is compiled only for tests and the opt-in
// `list-chaos` feature (backlog#1832): a production binary without the
// feature carries no chaos symbols, so the two env vars cannot silently
// rewrite a bucket's namespace-journal state.
#[cfg(any(test, feature = "list-chaos"))]
const ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_ENABLED: &str = "RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_ENABLED";
#[cfg(any(test, feature = "list-chaos"))]
const ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_BUCKET: &str = "RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_BUCKET";
#[cfg(any(test, feature = "list-chaos"))]
const ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_SEQUENCE: &str = "RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_SEQUENCE";
#[cfg(any(test, feature = "list-chaos"))]
const ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_STATUS: &str = "RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_STATUS";
const ENV_API_LIST_OBJECTS_METADATA_FAST_ENABLED: &str = "RUSTFS_LIST_OBJECTS_METADATA_FAST_ENABLED";
const ENV_API_LIST_OBJECTS_METADATA_FAST_STALENESS_MS: &str = "RUSTFS_LIST_OBJECTS_METADATA_FAST_STALENESS_MS";
@@ -552,7 +560,9 @@ static LIST_OBJECTS_MUTATION_SEQUENCE: AtomicU64 = AtomicU64::new(0);
static SCANNER_NAMESPACE_MUTATION_GENERATION: AtomicU64 = AtomicU64::new(0);
static LIST_OBJECTS_BUCKET_MUTATION_SEQUENCE: OnceCell<RwLock<HashMap<String, u64>>> = OnceCell::const_new();
static LIST_OBJECTS_NAMESPACE_JOURNAL_DEGRADED_BUCKETS: OnceCell<RwLock<HashSet<String>>> = OnceCell::const_new();
#[cfg(any(test, feature = "list-chaos"))]
static LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_CONFIG: OnceCell<Option<NamespaceMutationJournalChaosConfig>> = OnceCell::const_new();
#[cfg(any(test, feature = "list-chaos"))]
static LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_APPLIED: OnceCell<RwLock<HashSet<String>>> = OnceCell::const_new();
async fn persistent_key_only_index_cache() -> &'static RwLock<Option<PersistentKeyOnlyIndexCache>> {
@@ -579,6 +589,7 @@ async fn list_objects_namespace_journal_degraded_buckets() -> &'static RwLock<Ha
.await
}
#[cfg(any(test, feature = "list-chaos"))]
async fn list_objects_namespace_journal_chaos_config() -> Option<&'static NamespaceMutationJournalChaosConfig> {
LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_CONFIG
.get_or_init(|| async { namespace_mutation_journal_chaos_config_from_env() })
@@ -586,6 +597,7 @@ async fn list_objects_namespace_journal_chaos_config() -> Option<&'static Namesp
.as_ref()
}
#[cfg(any(test, feature = "list-chaos"))]
async fn list_objects_namespace_journal_chaos_applied() -> &'static RwLock<HashSet<String>> {
LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_APPLIED
.get_or_init(|| async { RwLock::new(HashSet::new()) })
@@ -681,6 +693,7 @@ enum NamespaceMutationJournalStatus {
}
impl NamespaceMutationJournalStatus {
#[cfg(any(test, feature = "list-chaos"))]
fn from_env_value(value: &str) -> Option<Self> {
if value.eq_ignore_ascii_case(LIST_OBJECTS_NAMESPACE_JOURNAL_STATUS_HEALTHY) {
Some(Self::Healthy)
@@ -691,6 +704,7 @@ impl NamespaceMutationJournalStatus {
}
}
#[cfg(any(test, feature = "list-chaos"))]
fn env_value(self) -> &'static str {
match self {
Self::Healthy => LIST_OBJECTS_NAMESPACE_JOURNAL_STATUS_HEALTHY,
@@ -712,6 +726,7 @@ struct NamespaceMutationJournalSnapshot {
degraded: bool,
}
#[cfg(any(test, feature = "list-chaos"))]
#[derive(Debug, Clone, PartialEq, Eq)]
struct NamespaceMutationJournalChaosConfig {
bucket: String,
@@ -795,30 +810,35 @@ fn list_objects_namespace_journal_root_from_env() -> Option<PathBuf> {
.filter(|path| !path.as_os_str().is_empty())
}
#[cfg(any(test, feature = "list-chaos"))]
fn namespace_mutation_journal_chaos_enabled_from_env() -> bool {
std::env::var(ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_ENABLED)
.ok()
.is_some_and(|value| value == "1" || value.eq_ignore_ascii_case("on") || value.eq_ignore_ascii_case("true"))
}
#[cfg(any(test, feature = "list-chaos"))]
fn namespace_mutation_journal_chaos_bucket_from_env() -> Option<String> {
std::env::var(ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_BUCKET)
.ok()
.filter(|bucket| !bucket.is_empty())
}
#[cfg(any(test, feature = "list-chaos"))]
fn namespace_mutation_journal_chaos_sequence_from_env() -> Option<u64> {
std::env::var(ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_SEQUENCE)
.ok()
.and_then(|value| value.parse::<u64>().ok())
}
#[cfg(any(test, feature = "list-chaos"))]
fn namespace_mutation_journal_chaos_status_from_env() -> Option<NamespaceMutationJournalStatus> {
std::env::var(ENV_API_LIST_OBJECTS_NAMESPACE_JOURNAL_CHAOS_STATUS)
.ok()
.and_then(|value| NamespaceMutationJournalStatus::from_env_value(&value))
}
#[cfg(any(test, feature = "list-chaos"))]
fn namespace_mutation_journal_chaos_config_from_env() -> Option<NamespaceMutationJournalChaosConfig> {
if !namespace_mutation_journal_chaos_enabled_from_env() {
return None;
@@ -846,6 +866,7 @@ fn namespace_mutation_journal_chaos_config_from_env() -> Option<NamespaceMutatio
})
}
#[cfg(any(test, feature = "list-chaos"))]
fn namespace_mutation_journal_chaos_applied_key(bucket: &str, status: NamespaceMutationJournalStatus) -> String {
let mut key = String::with_capacity(bucket.len() + 1 + status.env_value().len());
key.push_str(bucket);
@@ -854,6 +875,13 @@ fn namespace_mutation_journal_chaos_applied_key(bucket: &str, status: NamespaceM
key
}
/// Production no-op twin of the chaos injector: without `list-chaos` the
/// injection point compiles to nothing (backlog#1832).
#[cfg(not(any(test, feature = "list-chaos")))]
#[inline]
async fn maybe_apply_system_namespace_mutation_journal_chaos(_store: &ECStore, _bucket: &str, _default_sequence: u64) {}
#[cfg(any(test, feature = "list-chaos"))]
async fn maybe_apply_system_namespace_mutation_journal_chaos(store: &ECStore, bucket: &str, default_sequence: u64) {
let Some(config) = list_objects_namespace_journal_chaos_config().await else {
return;
+1 -1
View File
@@ -389,7 +389,7 @@ impl crate::storage_api_contracts::object::ObjectIO for ECStore {
type GetObjectReader = GetObjectReader;
type PutObjectReader = PutObjReader;
#[instrument(level = "debug", skip(self))]
#[instrument(level = "debug", skip(self, h))]
async fn get_object_reader(
&self,
bucket: &str,
+188 -5
View File
@@ -66,6 +66,76 @@ fn ensure_multipart_bucket_lifecycle_guard_held(
Ok(())
}
#[cfg(test)]
struct DataMovementMultipartCompletionBarrierState {
bucket: String,
arrived: tokio::sync::Notify,
release: tokio::sync::Notify,
}
#[cfg(test)]
pub(crate) struct DataMovementMultipartCompletionBarrier {
state: Arc<DataMovementMultipartCompletionBarrierState>,
}
#[cfg(test)]
static DATA_MOVEMENT_MULTIPART_COMPLETION_BARRIER: std::sync::OnceLock<
std::sync::Mutex<Option<Arc<DataMovementMultipartCompletionBarrierState>>>,
> = std::sync::OnceLock::new();
#[cfg(test)]
impl DataMovementMultipartCompletionBarrier {
pub(crate) fn install(bucket: &str) -> Self {
let state = Arc::new(DataMovementMultipartCompletionBarrierState {
bucket: bucket.to_string(),
arrived: tokio::sync::Notify::new(),
release: tokio::sync::Notify::new(),
});
let mut slot = DATA_MOVEMENT_MULTIPART_COMPLETION_BARRIER
.get_or_init(|| std::sync::Mutex::new(None))
.lock()
.expect("data movement multipart completion barrier mutex should not poison");
assert!(slot.is_none(), "data movement multipart completion barrier must be unique");
*slot = Some(Arc::clone(&state));
Self { state }
}
pub(crate) async fn wait_until_paused(&self) {
tokio::time::timeout(std::time::Duration::from_secs(30), self.state.arrived.notified())
.await
.expect("data movement multipart operation should reach selected completion");
}
}
#[cfg(test)]
impl Drop for DataMovementMultipartCompletionBarrier {
fn drop(&mut self) {
self.state.release.notify_one();
let mut slot = DATA_MOVEMENT_MULTIPART_COMPLETION_BARRIER
.get_or_init(|| std::sync::Mutex::new(None))
.lock()
.expect("data movement multipart completion barrier mutex should not poison");
if slot.as_ref().is_some_and(|state| Arc::ptr_eq(state, &self.state)) {
*slot = None;
}
}
}
#[cfg(test)]
async fn pause_data_movement_multipart_before_selected_completion(bucket: &str) {
let barrier = DATA_MOVEMENT_MULTIPART_COMPLETION_BARRIER
.get_or_init(|| std::sync::Mutex::new(None))
.lock()
.expect("data movement multipart completion barrier mutex should not poison")
.as_ref()
.filter(|barrier| barrier.bucket == bucket)
.cloned();
if let Some(barrier) = barrier {
barrier.arrived.notify_one();
barrier.release.notified().await;
}
}
async fn list_pool_multipart_uploads_for_incarnation(
pool: &crate::core::sets::Sets,
bucket: &str,
@@ -332,7 +402,7 @@ impl ECStore {
) -> Result<MultipartUploadResult> {
self.handle_new_multipart_upload_with_pool_idx(bucket, object, opts)
.await
.map(|(res, _)| res)
.map(|(res, _, _)| res)
}
pub(crate) async fn handle_new_multipart_upload_with_pool_idx(
@@ -340,7 +410,7 @@ impl ECStore {
bucket: &str,
object: &str,
opts: &ObjectOptions,
) -> Result<(MultipartUploadResult, usize)> {
) -> Result<(MultipartUploadResult, usize, Option<Uuid>)> {
check_new_multipart_args(bucket, object)?;
let (opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
let opts = &opts;
@@ -349,7 +419,20 @@ impl ECStore {
return self.pools[0]
.new_multipart_upload(bucket, object, opts)
.await
.map(|res| (res, 0));
.map(|res| (res, 0, opts.expected_bucket_incarnation_id));
}
if opts.data_movement && opts.version_id.is_some() {
let idx = self.select_data_movement_pool_idx(bucket, object, -1, opts, false).await?;
if idx == opts.src_pool_idx {
return Err(StorageError::DataMovementOverwriteErr(
bucket.to_owned(),
object.to_owned(),
opts.version_id.clone().unwrap_or_default(),
));
}
let res = self.pools[idx].new_multipart_upload(bucket, object, opts).await?;
return Ok((res, idx, opts.expected_bucket_incarnation_id));
}
for (idx, pool) in self.pools.iter().enumerate() {
@@ -372,7 +455,7 @@ impl ECStore {
if !res.uploads.is_empty() {
let res = self.pools[idx].new_multipart_upload(bucket, object, opts).await?;
return Ok((res, idx));
return Ok((res, idx, opts.expected_bucket_incarnation_id));
}
}
let idx = self.get_pool_idx(bucket, object, -1).await?;
@@ -385,7 +468,7 @@ impl ECStore {
}
let res = self.pools[idx].new_multipart_upload(bucket, object, opts).await?;
Ok((res, idx))
Ok((res, idx, opts.expected_bucket_incarnation_id))
}
#[instrument(skip(self))]
@@ -456,6 +539,30 @@ impl ECStore {
Err(StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned()))
}
pub(crate) async fn put_object_part_for_data_movement(
&self,
target_pool_idx: usize,
bucket: &str,
object: &str,
upload_id: &str,
data: &mut PutObjReader,
opts: &ObjectOptions,
) -> Result<PartInfo> {
let part_id = opts
.part_number
.ok_or_else(|| Error::other("targeted multipart upload requires a part number"))?;
check_put_object_part_args(bucket, object, upload_id)?;
if !opts.data_movement {
return Err(Error::other("targeted multipart upload requires data_movement options"));
}
let (opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
let pool = self
.pools
.get(target_pool_idx)
.ok_or_else(|| Error::other(format!("data movement target pool {target_pool_idx} is out of range")))?;
pool.put_object_part(bucket, object, upload_id, part_id, data, &opts).await
}
#[instrument(skip(self))]
pub(super) async fn handle_get_multipart_info(
&self,
@@ -530,6 +637,26 @@ impl ECStore {
Err(StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned()))
}
pub(crate) async fn abort_multipart_upload_for_data_movement(
&self,
target_pool_idx: usize,
bucket: &str,
object: &str,
upload_id: &str,
opts: &ObjectOptions,
) -> Result<()> {
check_abort_multipart_args(bucket, object, upload_id)?;
if !opts.data_movement {
return Err(Error::other("targeted multipart abort requires data_movement options"));
}
let (opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
let pool = self
.pools
.get(target_pool_idx)
.ok_or_else(|| Error::other(format!("data movement target pool {target_pool_idx} is out of range")))?;
pool.abort_multipart_upload(bucket, object, upload_id, &opts).await
}
#[instrument(skip(self))]
pub(super) async fn handle_complete_multipart_upload(
self: Arc<Self>,
@@ -574,6 +701,62 @@ impl ECStore {
Err(StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned()))
}
pub(crate) async fn complete_multipart_upload_for_data_movement(
self: Arc<Self>,
target_pool_idx: usize,
bucket: &str,
object: &str,
upload_id: &str,
uploaded_parts: Vec<CompletePart>,
opts: &ObjectOptions,
) -> Result<ObjectInfo> {
check_complete_multipart_args(bucket, object, upload_id)?;
if !opts.data_movement {
return Err(Error::other("targeted multipart completion requires data_movement options"));
}
let (mut opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
if opts.overwrites_existing_version() && !is_meta_bucketname(bucket) {
let expected_incarnation_id = opts
.expected_bucket_incarnation_id
.ok_or_else(|| Error::other("data movement completion is missing its bucket incarnation"))?;
let lifecycle_fence = opts
.bucket_lifecycle_lock_fence
.as_ref()
.ok_or_else(|| Error::other("data movement completion is missing its bucket lifecycle fence"))?;
let snapshot = match opts.object_lock_config_snapshot.as_ref() {
Some(snapshot) => Arc::clone(snapshot),
None => {
self.object_lock_config_snapshot_under_lifecycle_fence(bucket, lifecycle_fence)
.await?
}
};
if !snapshot.is_valid_for_destructive_put(self.id, bucket, expected_incarnation_id) {
return Err(Error::other(
"data movement Object Lock snapshot does not match the target bucket generation",
));
}
snapshot.add_lock_fences(&mut opts);
opts.object_lock_config_snapshot = Some(snapshot);
}
#[cfg(test)]
pause_data_movement_multipart_before_selected_completion(bucket).await;
let pool = self
.pools
.get(target_pool_idx)
.ok_or_else(|| Error::other(format!("data movement target pool {target_pool_idx} is out of range")))?
.clone();
let result = enqueue_transition_after_write(
pool.complete_multipart_upload(bucket, object, upload_id, uploaded_parts, &opts)
.await,
LcEventSrc::S3CompleteMultipartUpload,
)
.await;
if result.is_ok() {
list_objects::observe_list_objects_mutation(self.as_ref(), bucket).await;
}
result
}
}
/// Merges per-pool `ListMultipartUploads` pages into a single globally paginated
+627 -70
View File
@@ -667,18 +667,6 @@ impl SelectObjectSnapshotLockLossWake {
}
}
fn select_object_ssec_headers(headers: &HeaderMap) -> HeaderMap {
use rustfs_utils::http::headers::{SSEC_ALGORITHM_HEADER, SSEC_KEY_HEADER, SSEC_KEY_MD5_HEADER};
let mut selected = HeaderMap::new();
for name in [SSEC_ALGORITHM_HEADER, SSEC_KEY_HEADER, SSEC_KEY_MD5_HEADER] {
if let Some(value) = headers.get(name) {
selected.insert(name, value.clone());
}
}
selected
}
// LockRegistry clones its canonical client Arc for each endpoint host, so an
// exact Arc set identifies one distributed namespace-lock quorum domain.
fn same_distributed_lock_domain(left: &[Arc<dyn rustfs_lock::LockClient>], right: &[Arc<dyn rustfs_lock::LockClient>]) -> bool {
@@ -918,7 +906,7 @@ fn is_equivalent_data_movement_delete_marker(source: &ObjectInfo, target: &Objec
&& is_data_movement_delete_marker(target)
&& source.version_id == target.version_id
&& source.mod_time == target.mod_time
&& source.user_defined == target.user_defined
&& is_equivalent_data_movement_delete_marker_metadata(&source.user_defined, &target.user_defined)
&& source.user_tags == target.user_tags
&& source.replication_status_internal == target.replication_status_internal
&& source.replication_status == target.replication_status
@@ -926,24 +914,185 @@ fn is_equivalent_data_movement_delete_marker(source: &ObjectInfo, target: &Objec
&& source.version_purge_status == target.version_purge_status
}
fn is_equivalent_data_movement_delete_marker_metadata(
source: &HashMap<String, String>,
target: &HashMap<String, String>,
) -> bool {
matches!(
(
data_movement_delete_marker_metadata_identity(source),
data_movement_delete_marker_metadata_identity(target)
),
(Some(source), Some(target)) if source == target
)
}
fn data_movement_delete_marker_metadata_identity(metadata: &HashMap<String, String>) -> Option<HashMap<String, String>> {
let mut identity = HashMap::with_capacity(metadata.len());
let mut local_tier_free_version_id = None;
for (key, value) in metadata {
let Some(suffix) = rustfs_utils::http::strip_internal_prefix_preserving_case(key) else {
identity.insert(key.clone(), value.clone());
continue;
};
if suffix.eq_ignore_ascii_case(rustfs_utils::http::SUFFIX_TIER_FV_ID) {
let version_id = Uuid::parse_str(value).ok().filter(|version_id| !version_id.is_nil())?;
if local_tier_free_version_id.is_some_and(|expected| expected != version_id) {
return None;
}
local_tier_free_version_id = Some(version_id);
continue;
}
let canonical_suffix = [
rustfs_utils::http::SUFFIX_REPLICA_TIMESTAMP,
rustfs_utils::http::SUFFIX_REPLICA_STATUS,
rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP,
rustfs_utils::http::SUFFIX_REPLICATION_STATUS,
rustfs_utils::http::SUFFIX_PURGESTATUS,
]
.into_iter()
.find(|candidate| suffix.eq_ignore_ascii_case(candidate))
.map(str::to_string)
.or_else(|| {
[
rustfs_utils::http::SUFFIX_REPLICATION_RESET_ARN_PREFIX,
rustfs_utils::http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX,
]
.into_iter()
.find_map(|prefix| {
suffix
.get(..prefix.len())
.is_some_and(|candidate| candidate.eq_ignore_ascii_case(prefix))
.then(|| format!("{prefix}{}", &suffix[prefix.len()..]))
})
})
.unwrap_or_else(|| suffix.to_string());
let canonical_value = if canonical_suffix.eq_ignore_ascii_case(rustfs_utils::http::SUFFIX_REPLICA_TIMESTAMP)
|| canonical_suffix.eq_ignore_ascii_case(rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP)
{
rustfs_filemeta::parse_replication_timestamp(value)?
.unix_timestamp_nanos()
.to_string()
} else {
value.clone()
};
let canonical_key = format!("{}{canonical_suffix}", rustfs_utils::http::RUSTFS_INTERNAL_PREFIX);
if identity
.insert(canonical_key, canonical_value.clone())
.is_some_and(|existing| existing != canonical_value)
{
return None;
}
}
for (status_suffix, timestamp_suffix) in [
(rustfs_utils::http::SUFFIX_REPLICA_STATUS, rustfs_utils::http::SUFFIX_REPLICA_TIMESTAMP),
(
rustfs_utils::http::SUFFIX_REPLICATION_STATUS,
rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP,
),
] {
let status_key = format!("{}{status_suffix}", rustfs_utils::http::RUSTFS_INTERNAL_PREFIX);
let timestamp_key = format!("{}{timestamp_suffix}", rustfs_utils::http::RUSTFS_INTERNAL_PREFIX);
match (identity.contains_key(&status_key), identity.contains_key(&timestamp_key)) {
(true, false) => {
identity.insert(timestamp_key, OffsetDateTime::UNIX_EPOCH.unix_timestamp_nanos().to_string());
}
(false, true) => return None,
_ => {}
}
}
Some(identity)
}
fn is_data_movement_delete_marker(info: &ObjectInfo) -> bool {
info.delete_marker
}
fn is_expected_data_movement_delete_marker_source(source: &ObjectInfo, expected_mod_time: Option<OffsetDateTime>) -> bool {
is_data_movement_delete_marker(source)
&& source.mod_time.is_some()
&& source.mod_time == expected_mod_time
&& data_movement_delete_marker_metadata_identity(&source.user_defined).is_some()
}
fn current_data_movement_delete_marker_opts(source: &ObjectInfo, opts: &ObjectOptions) -> Option<ObjectOptions> {
let replica_status = rustfs_utils::http::get_str(&source.user_defined, rustfs_utils::http::SUFFIX_REPLICA_STATUS);
let replica_timestamp = rustfs_utils::http::get_str(&source.user_defined, rustfs_utils::http::SUFFIX_REPLICA_TIMESTAMP);
let (replica_status, replica_timestamp) = match (replica_status, replica_timestamp) {
(None, None) => Default::default(),
(Some(status), timestamp) => {
let status = crate::bucket::replication::ReplicationStatusType::from(status.as_str());
if status.is_empty() {
return None;
}
let timestamp = match timestamp {
Some(timestamp) => rustfs_filemeta::parse_replication_timestamp(&timestamp)?,
None => OffsetDateTime::UNIX_EPOCH,
};
(status, Some(timestamp))
}
(None, Some(_)) => return None,
};
let replication_status = rustfs_utils::http::get_str(&source.user_defined, rustfs_utils::http::SUFFIX_REPLICATION_STATUS);
let replication_timestamp =
rustfs_utils::http::get_str(&source.user_defined, rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP);
let (replication_status, replication_timestamp, replication_targets) = match (replication_status, replication_timestamp) {
(None, None) => Default::default(),
(Some(status), timestamp) => {
let direct_status = crate::bucket::replication::ReplicationStatusType::from(status.as_str());
let targets = crate::bucket::replication::replication_statuses_map(status.as_str());
if direct_status.is_empty() && targets.is_empty() {
return None;
}
let timestamp = match timestamp {
Some(timestamp) => rustfs_filemeta::parse_replication_timestamp(&timestamp)?,
None => OffsetDateTime::UNIX_EPOCH,
};
(Some(status), Some(timestamp), targets)
}
(None, Some(_)) => return None,
};
let mut state = source.replication_state();
if state.target_delete_marker_version_ids_corrupt {
return None;
}
state.replica_status = replica_status;
state.replica_timestamp = replica_timestamp;
state.replication_status_internal = replication_status;
state.replication_timestamp = replication_timestamp;
state.targets = replication_targets;
state.replicate_decision_str = source.replication_decision.clone();
state.delete_marker = true;
let mut target_opts = opts.clone();
target_opts.mod_time = source.mod_time;
target_opts.delete_replication = Some(state);
Some(target_opts)
}
fn expected_data_movement_tiered_object(source: &rustfs_filemeta::FileInfo) -> ObjectInfo {
ObjectInfo::from_file_info(source, "", &source.name, source.version_id.is_some())
}
fn is_equivalent_data_movement_tiered_object(source: &rustfs_filemeta::FileInfo, target: &ObjectInfo) -> bool {
let expected = expected_data_movement_tiered_object(source);
let Some(source_actual_size) = effective_object_actual_size(&expected) else {
return false;
};
let Some(target_actual_size) = effective_object_actual_size(target) else {
return false;
};
source.version_id == target.version_id
&& !target.delete_marker
&& source.size == target.size
&& source.get_etag() == target.etag
&& source.checksum == target.checksum
&& crate::data_movement::are_equivalent_data_movement_parts(&source.parts, &target.parts)
&& source.mod_time == target.mod_time
&& expected.user_defined == target.user_defined
&& crate::data_movement::is_equivalent_data_movement_metadata(&expected, target, source_actual_size, target_actual_size)
&& expected.user_tags == target.user_tags
&& expected.expires == target.expires
&& expected.storage_class == target.storage_class
@@ -952,11 +1101,12 @@ fn is_equivalent_data_movement_tiered_object(source: &rustfs_filemeta::FileInfo,
&& expected.version_purge_status_internal == target.version_purge_status_internal
&& expected.version_purge_status == target.version_purge_status
&& expected.transitioned_object.status == target.transitioned_object.status
&& expected.transition_version_state == target.transition_version_state
&& expected.transitioned_object.name == target.transitioned_object.name
&& expected.transitioned_object.tier == target.transitioned_object.tier
&& expected.transitioned_object.version_id == target.transitioned_object.version_id
&& expected.transitioned_object.free_version == target.transitioned_object.free_version
&& effective_object_actual_size(target) == Some(source.size)
&& source_actual_size == target_actual_size
}
fn should_check_data_movement_resume_target(src_pool_idx: usize, target_pool_idx: usize) -> bool {
@@ -1054,7 +1204,7 @@ impl ECStore {
)))
}
async fn object_lock_config_snapshot_under_lifecycle_fence(
pub(super) async fn object_lock_config_snapshot_under_lifecycle_fence(
&self,
bucket: &str,
lifecycle_fence: &NamespaceLockFence,
@@ -1134,7 +1284,7 @@ impl ECStore {
pool,
bucket: bucket.to_owned(),
object,
headers: select_object_ssec_headers(headers),
headers: rustfs_utils::http::project_ssec_transport_headers(headers),
opts,
object_info,
logical_size,
@@ -1454,7 +1604,8 @@ impl ECStore {
target_pool_idx: usize,
opts: &ObjectOptions,
) -> Result<Option<ObjectInfo>> {
let lookup_opts = version_aware_lookup_opts(opts, true);
let mut lookup_opts = version_aware_lookup_opts(opts, true);
lookup_opts.include_part_checksums = true;
let Some(pool) = self.pools.get(target_pool_idx) else {
return Err(Error::other(format!(
@@ -1521,6 +1672,25 @@ impl ECStore {
) -> Result<()> {
check_put_object_args(bucket, object)?;
let mut opts = opts.clone();
let bucket_incarnation_fence = if is_meta_bucketname(bucket) {
None
} else {
let expected = opts
.expected_bucket_incarnation_id
.ok_or_else(|| Error::other("tiered data movement is missing its bucket incarnation snapshot"))?;
let guard = self.acquire_bucket_incarnation_fence(bucket, expected).await?;
if let Some(namespace_guard) = guard.namespace_lock_guard() {
opts.add_bucket_lifecycle_lock_guard(namespace_guard);
}
Some(guard)
};
let mut fi = fi.clone();
if opts.data_movement {
crate::data_movement::prepare_tiered_data_movement_file_info(&mut fi)?;
}
let object = encode_dir_object(object);
if self.single_pool() {
@@ -1533,7 +1703,8 @@ impl ECStore {
let idx = if opts.data_movement && opts.version_id.is_some() {
Self::resolve_decommission_target_pool_idx_result(
self.select_data_movement_pool_idx(bucket, &object, fi.size, opts, true).await,
self.select_data_movement_pool_idx(bucket, &object, fi.size, &opts, true)
.await,
bucket,
&object,
)?
@@ -1550,7 +1721,7 @@ impl ECStore {
.await;
let target_pool_idx = resolve_data_movement_resume_target_pool(idx, resume_target_pool_idx, opts.src_pool_idx);
if self
.has_equivalent_data_movement_tiered_object(bucket, &object, fi, opts, target_pool_idx)
.has_equivalent_data_movement_tiered_object(bucket, &object, &fi, &opts, target_pool_idx)
.await?
{
return Ok(());
@@ -1563,17 +1734,30 @@ impl ECStore {
));
}
Self::resolve_decommission_tiered_object_result(
self.pools[idx]
.get_disks_by_key(&object)
.decommission_tiered_object(bucket, &object, fi, opts)
.await,
bucket,
&object,
)
let result = self.pools[idx]
.get_disks_by_key(&object)
.decommission_tiered_object(bucket, &object, &fi, &opts)
.await;
if matches!(result, Err(Error::PreconditionFailed)) {
if self
.has_equivalent_data_movement_tiered_object(bucket, &object, &fi, &opts, idx)
.await?
{
return Ok(());
}
return Err(StorageError::DataMovementOverwriteErr(
bucket.to_owned(),
object,
opts.version_id.clone().unwrap_or_default(),
));
}
if bucket_incarnation_fence.as_ref().is_some_and(|guard| guard.is_lock_lost()) {
return Err(Error::other("tiered data movement bucket incarnation fence was lost during target write"));
}
Self::resolve_decommission_tiered_object_result(result, bucket, &object)
}
#[instrument(level = "debug", skip(self))]
#[instrument(level = "debug", skip(self, h))]
#[hotpath::measure(impl_type = "ECStore")]
pub(super) async fn handle_get_object_reader(
&self,
@@ -1585,7 +1769,7 @@ impl ECStore {
) -> Result<GetObjectReader> {
check_get_obj_args(bucket, object)?;
let object = encode_dir_object(object);
let object = rustfs_utils::path::encode_dir_object_ref(object);
let mut opts = opts.clone();
let read_lock_guard = self
.acquire_object_read_lock_if_needed("get_object", bucket, &object, &mut opts)
@@ -1593,29 +1777,21 @@ impl ECStore {
let reader = if self.single_pool() {
self.pools[0]
.get_object_reader(bucket, object.as_str(), range, h, &opts)
.get_object_reader(bucket, object.as_ref(), range, h, &opts)
.await?
} else {
let (_, idx) = self
.get_latest_accessible_object_info_with_idx(bucket, &object, &opts)
.await?;
self.pools[idx]
.get_object_reader(bucket, object.as_str(), range, h, &opts)
.get_object_reader(bucket, object.as_ref(), range, h, &opts)
.await?
};
Ok(Self::attach_read_lock_guard(reader, read_lock_guard))
}
#[instrument(level = "debug", skip(self, data))]
#[hotpath::measure(impl_type = "ECStore")]
pub(super) async fn handle_put_object(
&self,
bucket: &str,
object: &str,
data: &mut PutObjReader,
opts: &ObjectOptions,
) -> Result<(ObjectInfo, Option<OldCurrentSize>)> {
async fn prepare_put_object(&self, bucket: &str, object: &str, opts: &ObjectOptions) -> Result<(String, ObjectOptions)> {
check_put_object_args(bucket, object)?;
let object = encode_dir_object(object);
@@ -1642,22 +1818,20 @@ impl ECStore {
};
snapshot.add_lock_fences(&mut opts);
}
Ok((object, opts))
}
// Keep PUT atomic-read friendly: SetDisks takes the object write lock only
// around precondition checks and the final rename/commit.
async fn select_put_object_pool_idx(&self, bucket: &str, object: &str, size: i64, opts: &ObjectOptions) -> Result<usize> {
if self.single_pool() {
return self.pools[0]
.put_object_with_old_current_size(bucket, object.as_str(), data, &opts)
.await;
return Ok(0);
}
let idx = if opts.data_movement && opts.version_id.is_some() {
self.select_data_movement_pool_idx(bucket, &object, data.size(), &opts, false)
.await?
self.select_data_movement_pool_idx(bucket, object, size, opts, false).await?
} else if opts.no_lock {
self.get_pool_idx_no_lock(bucket, &object, data.size()).await?
self.get_pool_idx_no_lock(bucket, object, size).await?
} else {
self.get_pool_idx(bucket, &object, data.size()).await?
self.get_pool_idx(bucket, object, size).await?
};
if opts.data_movement && idx == opts.src_pool_idx {
@@ -1667,7 +1841,50 @@ impl ECStore {
opts.version_id.clone().unwrap_or_default(),
));
}
Ok(idx)
}
pub(crate) async fn put_object_for_data_movement(
&self,
bucket: &str,
object: &str,
data: &mut PutObjReader,
opts: &ObjectOptions,
) -> Result<(usize, Result<ObjectInfo>)> {
if !opts.data_movement {
return Err(Error::other("data movement PUT requires data_movement options"));
}
let (object, opts) = self.prepare_put_object(bucket, object, opts).await?;
let idx = self
.select_put_object_pool_idx(bucket, object.as_str(), data.size(), &opts)
.await?;
let result = self.pools[idx]
.put_object_with_old_current_size(bucket, &object, data, &opts)
.await
.map(|(object_info, _)| object_info);
let result = enqueue_transition_after_write(result, LcEventSrc::S3PutObject).await;
if result.is_ok() {
list_objects::observe_list_objects_mutation(self, bucket).await;
}
Ok((idx, result))
}
#[instrument(level = "debug", skip(self, data))]
#[hotpath::measure(impl_type = "ECStore")]
pub(super) async fn handle_put_object(
&self,
bucket: &str,
object: &str,
data: &mut PutObjReader,
opts: &ObjectOptions,
) -> Result<(ObjectInfo, Option<OldCurrentSize>)> {
let (object, opts) = self.prepare_put_object(bucket, object, opts).await?;
let idx = self
.select_put_object_pool_idx(bucket, object.as_str(), data.size(), &opts)
.await?;
// Keep PUT atomic-read friendly: SetDisks takes the object write lock only
// around precondition checks and the final rename/commit.
self.pools[idx]
.put_object_with_old_current_size(bucket, &object, data, &opts)
.await
@@ -2110,6 +2327,50 @@ impl ECStore {
};
let target_pool_idx =
resolve_data_movement_resume_target_pool(selected_target_pool_idx, resume_target_pool_idx, opts.src_pool_idx);
let mut delete_marker_target_opts = None;
if opts.delete_marker && should_check_data_movement_resume_target(opts.src_pool_idx, target_pool_idx) {
let source = self
.find_data_movement_target_info(bucket, object, opts.src_pool_idx, &opts)
.await?;
let Some(source) = source else {
return Err(StorageError::DataMovementOverwriteErr(
bucket.to_owned(),
object.to_owned(),
opts.version_id.unwrap_or_default(),
));
};
if !is_expected_data_movement_delete_marker_source(&source, opts.mod_time) {
return Err(StorageError::DataMovementOverwriteErr(
bucket.to_owned(),
object.to_owned(),
opts.version_id.unwrap_or_default(),
));
}
let Some(target_opts) = current_data_movement_delete_marker_opts(&source, &opts) else {
return Err(StorageError::DataMovementOverwriteErr(
bucket.to_owned(),
object.to_owned(),
opts.version_id.unwrap_or_default(),
));
};
let target = self
.find_data_movement_target_info(bucket, object, target_pool_idx, &target_opts)
.await?;
if let Some(target) = target {
if is_equivalent_data_movement_delete_marker(&source, &target) {
let mut target = target;
target.name = decode_dir_object(object);
return Ok(target);
}
return Err(StorageError::DataMovementOverwriteErr(
bucket.to_owned(),
object.to_owned(),
opts.version_id.unwrap_or_default(),
));
}
delete_marker_target_opts = Some(target_opts);
}
if !should_check_data_movement_resume_target(opts.src_pool_idx, target_pool_idx) {
if let Ok((source_pool_info, _)) = existing_pool_info
@@ -2137,7 +2398,8 @@ impl ECStore {
));
}
let mut obj = self.pools[target_pool_idx].delete_object(bucket, object, opts).await?;
let target_opts = delete_marker_target_opts.unwrap_or(opts);
let mut obj = self.pools[target_pool_idx].delete_object(bucket, object, target_opts).await?;
obj.name = decode_dir_object(obj.name.as_str());
return Ok(obj);
}
@@ -3086,25 +3348,6 @@ mod tests {
assert!(second_signal.is_lost());
}
#[test]
fn select_snapshot_retains_only_ssec_headers() {
use rustfs_utils::http::headers::{SSEC_ALGORITHM_HEADER, SSEC_KEY_HEADER, SSEC_KEY_MD5_HEADER};
let mut headers = HeaderMap::new();
headers.insert(SSEC_ALGORITHM_HEADER, "AES256".parse().expect("valid SSE-C algorithm header"));
headers.insert(SSEC_KEY_HEADER, "secret-key".parse().expect("valid SSE-C key header"));
headers.insert(SSEC_KEY_MD5_HEADER, "key-md5".parse().expect("valid SSE-C key digest header"));
headers.insert("authorization", "credential".parse().expect("valid authorization header"));
let selected = select_object_ssec_headers(&headers);
assert_eq!(selected.len(), 3);
assert_eq!(selected.get(SSEC_ALGORITHM_HEADER), headers.get(SSEC_ALGORITHM_HEADER));
assert_eq!(selected.get(SSEC_KEY_HEADER), headers.get(SSEC_KEY_HEADER));
assert_eq!(selected.get(SSEC_KEY_MD5_HEADER), headers.get(SSEC_KEY_MD5_HEADER));
assert!(selected.get("authorization").is_none());
}
#[test]
fn tier_delete_entry_is_prepared_and_bound_to_source_generation() {
let identity = [9_u8; 32];
@@ -3200,6 +3443,247 @@ mod tests {
assert!(!is_equivalent_data_movement_delete_marker(&source, &mismatched));
}
#[test]
fn equivalent_data_movement_delete_marker_accepts_distinct_local_free_version_ids() {
let mut source = ObjectInfo {
version_id: Some(Uuid::from_u128(1)),
delete_marker: true,
mod_time: Some(OffsetDateTime::UNIX_EPOCH),
..Default::default()
};
rustfs_utils::http::insert_str(
Arc::make_mut(&mut source.user_defined),
rustfs_utils::http::SUFFIX_TIER_FV_ID,
Uuid::from_u128(2).to_string(),
);
let mut target = source.clone();
rustfs_utils::http::insert_str(
Arc::make_mut(&mut target.user_defined),
rustfs_utils::http::SUFFIX_TIER_FV_ID,
Uuid::from_u128(3).to_string(),
);
assert!(is_equivalent_data_movement_delete_marker(&source, &target));
Arc::make_mut(&mut target.user_defined).insert(
format!("{}{}", rustfs_utils::http::MINIO_INTERNAL_PREFIX, rustfs_utils::http::SUFFIX_TIER_FV_ID),
Uuid::from_u128(4).to_string(),
);
assert!(!is_equivalent_data_movement_delete_marker(&source, &target));
}
#[test]
fn equivalent_data_movement_delete_marker_accepts_replication_alias_expansion() {
let key = format!(
"{}{}",
rustfs_utils::http::MINIO_INTERNAL_PREFIX,
rustfs_utils::http::SUFFIX_REPLICATION_STATUS
);
let timestamp_key = format!(
"{}{}",
rustfs_utils::http::MINIO_INTERNAL_PREFIX,
rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP
);
let source = ObjectInfo {
version_id: Some(Uuid::from_u128(1)),
delete_marker: true,
mod_time: Some(OffsetDateTime::UNIX_EPOCH),
user_defined: Arc::new(HashMap::from([
(key.clone(), "arn=COMPLETED;".to_string()),
(timestamp_key, "1970-01-01T00:00:01Z".to_string()),
])),
..Default::default()
};
let mut target = source.clone();
rustfs_utils::http::insert_str(
Arc::make_mut(&mut target.user_defined),
rustfs_utils::http::SUFFIX_REPLICATION_STATUS,
"arn=COMPLETED;".to_string(),
);
rustfs_utils::http::insert_str(
Arc::make_mut(&mut target.user_defined),
rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP,
(OffsetDateTime::UNIX_EPOCH + time::Duration::SECOND).to_string(),
);
assert!(is_equivalent_data_movement_delete_marker(&source, &target));
Arc::make_mut(&mut target.user_defined).insert(key, "arn=FAILED;".to_string());
assert!(!is_equivalent_data_movement_delete_marker(&source, &target));
}
#[test]
fn data_movement_delete_marker_source_requires_persisted_mod_time() {
let source = ObjectInfo {
delete_marker: true,
..Default::default()
};
assert!(!is_expected_data_movement_delete_marker_source(&source, None));
let source = ObjectInfo {
mod_time: Some(OffsetDateTime::UNIX_EPOCH),
..source
};
assert!(is_expected_data_movement_delete_marker_source(&source, Some(OffsetDateTime::UNIX_EPOCH)));
assert!(!is_expected_data_movement_delete_marker_source(&source, None));
}
#[test]
fn data_movement_delete_marker_uses_current_source_replication_state() {
let expected_timestamp = OffsetDateTime::UNIX_EPOCH + time::Duration::SECOND;
let timestamp = expected_timestamp.to_string();
let mut metadata = HashMap::new();
rustfs_utils::http::insert_str(
&mut metadata,
rustfs_utils::http::SUFFIX_REPLICA_STATUS,
ReplicationStatusType::Replica.to_string(),
);
rustfs_utils::http::insert_str(&mut metadata, rustfs_utils::http::SUFFIX_REPLICA_TIMESTAMP, timestamp.clone());
rustfs_utils::http::insert_str(&mut metadata, rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP, timestamp);
rustfs_utils::http::insert_str(
&mut metadata,
rustfs_utils::http::SUFFIX_REPLICATION_STATUS,
"arn=COMPLETED;".to_string(),
);
rustfs_utils::http::insert_str(
&mut metadata,
&format!(
"{}{}",
rustfs_utils::http::SUFFIX_REPLICATION_RESET_ARN_PREFIX,
"arn:minio:replication::TenantA:bucket"
),
"reset-id".to_string(),
);
rustfs_utils::http::insert_str(
&mut metadata,
&format!(
"{}{}",
rustfs_utils::http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX,
"arn:minio:replication::TenantA:bucket"
),
"target-version".to_string(),
);
let source = ObjectInfo {
delete_marker: true,
mod_time: Some(OffsetDateTime::UNIX_EPOCH),
replication_status_internal: Some("arn=COMPLETED;".to_string()),
replication_decision: "arn=replicate;".to_string(),
user_defined: Arc::new(metadata),
..Default::default()
};
let opts = ObjectOptions {
mod_time: source.mod_time,
delete_replication: Some(ReplicationState {
replication_status_internal: Some("arn=PENDING;".to_string()),
..Default::default()
}),
..Default::default()
};
let target_opts = current_data_movement_delete_marker_opts(&source, &opts).expect("valid current source state");
let state = target_opts.delete_replication.as_ref().expect("current replication state");
assert_eq!(state.replication_status_internal.as_deref(), Some("arn=COMPLETED;"));
assert_eq!(state.replica_status, crate::bucket::replication::ReplicationStatusType::Replica);
assert_eq!(state.replica_timestamp, Some(expected_timestamp));
assert_eq!(state.replication_timestamp, state.replica_timestamp);
assert_eq!(state.replicate_decision_str, "arn=replicate;");
assert_eq!(
state
.reset_statuses_map
.get("arn:minio:replication::TenantA:bucket")
.map(String::as_str),
Some("reset-id")
);
assert_eq!(
state
.target_delete_marker_version_ids
.get("arn:minio:replication::TenantA:bucket")
.map(String::as_str),
Some("target-version")
);
}
#[test]
fn data_movement_delete_marker_rejects_corrupt_target_version_maps() {
let suffix = format!("{}not-an-arn", rustfs_utils::http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX);
let mut malformed = HashMap::new();
rustfs_utils::http::insert_str(&mut malformed, &suffix, "target-version".to_string());
let malformed_source = ObjectInfo {
delete_marker: true,
mod_time: Some(OffsetDateTime::UNIX_EPOCH),
user_defined: Arc::new(malformed),
..Default::default()
};
assert!(current_data_movement_delete_marker_opts(&malformed_source, &ObjectOptions::default()).is_none());
let mut conflicted = HashMap::new();
let suffix = format!(
"{}arn:minio:replication::target:bucket",
rustfs_utils::http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX
);
rustfs_utils::http::insert_str(&mut conflicted, &suffix, "target-version-a".to_string());
conflicted.insert(
format!("{}{suffix}", rustfs_utils::http::MINIO_INTERNAL_PREFIX),
"target-version-b".to_string(),
);
let conflicted_source = ObjectInfo {
user_defined: Arc::new(conflicted),
..malformed_source.clone()
};
assert!(current_data_movement_delete_marker_opts(&conflicted_source, &ObjectOptions::default()).is_none());
let mut over_cap = HashMap::new();
for index in 0..=1_000 {
let suffix = format!(
"{}arn:minio:replication::target:bucket-{index}",
rustfs_utils::http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX
);
rustfs_utils::http::insert_str(&mut over_cap, &suffix, format!("target-version-{index}"));
}
let over_cap_source = ObjectInfo {
user_defined: Arc::new(over_cap),
..malformed_source
};
assert!(current_data_movement_delete_marker_opts(&over_cap_source, &ObjectOptions::default()).is_none());
}
#[test]
fn data_movement_delete_marker_normalizes_legacy_missing_replication_timestamps() {
let mut source_metadata = HashMap::new();
rustfs_utils::http::insert_str(
&mut source_metadata,
rustfs_utils::http::SUFFIX_REPLICA_STATUS,
ReplicationStatusType::Replica.to_string(),
);
rustfs_utils::http::insert_str(
&mut source_metadata,
rustfs_utils::http::SUFFIX_REPLICATION_STATUS,
"arn=COMPLETED;".to_string(),
);
let source = ObjectInfo {
delete_marker: true,
mod_time: Some(OffsetDateTime::UNIX_EPOCH),
replication_status_internal: Some("arn=COMPLETED;".to_string()),
user_defined: Arc::new(source_metadata),
..Default::default()
};
let target_opts = current_data_movement_delete_marker_opts(&source, &ObjectOptions::default())
.expect("legacy status-only metadata should remain migratable");
let state = target_opts
.delete_replication
.expect("replication state should be reconstructed");
assert_eq!(state.replica_timestamp, Some(OffsetDateTime::UNIX_EPOCH));
assert_eq!(state.replication_timestamp, Some(OffsetDateTime::UNIX_EPOCH));
let mut target_metadata = (*source.user_defined).clone();
let epoch = OffsetDateTime::UNIX_EPOCH
.format(&time::format_description::well_known::Rfc3339)
.unwrap();
rustfs_utils::http::insert_str(&mut target_metadata, rustfs_utils::http::SUFFIX_REPLICA_TIMESTAMP, epoch.clone());
rustfs_utils::http::insert_str(&mut target_metadata, rustfs_utils::http::SUFFIX_REPLICATION_TIMESTAMP, epoch);
assert!(is_equivalent_data_movement_delete_marker_metadata(&source.user_defined, &target_metadata));
}
#[test]
fn equivalent_data_movement_delete_marker_rejects_metadata_and_replication_mismatch() {
let version_id = Uuid::nil();
@@ -3344,6 +3828,67 @@ mod tests {
assert!(is_equivalent_data_movement_tiered_object(&source, &target));
}
#[test]
fn equivalent_data_movement_tiered_object_uses_logical_compressed_and_encrypted_sizes() {
let mut compressed = tiered_equivalence_source();
compressed.size = 600;
rustfs_utils::http::insert_str(&mut compressed.metadata, rustfs_utils::http::SUFFIX_COMPRESSION, "S2".to_string());
rustfs_utils::http::insert_str(&mut compressed.metadata, rustfs_utils::http::SUFFIX_ACTUAL_SIZE, "1024".to_string());
let compressed_target = tiered_equivalence_target(&compressed);
assert!(is_equivalent_data_movement_tiered_object(&compressed, &compressed_target));
let mut encrypted = tiered_equivalence_source();
encrypted.size = 640;
encrypted.metadata.insert(
rustfs_utils::http::object_encryption_keys::INTERNAL_ENCRYPTION_KEY_ID_HEADER.to_string(),
"key-id".to_string(),
);
encrypted.metadata.insert(
rustfs_utils::http::object_encryption_keys::INTERNAL_ENCRYPTION_ORIGINAL_SIZE_HEADER.to_string(),
"1024".to_string(),
);
let encrypted_target = tiered_equivalence_target(&encrypted);
assert!(is_equivalent_data_movement_tiered_object(&encrypted, &encrypted_target));
}
#[test]
fn equivalent_data_movement_tiered_object_accepts_transition_alias_expansion() {
let mut source = tiered_equivalence_source();
let suffix = rustfs_utils::http::SUFFIX_TRANSITION_TIER;
source.metadata.insert(
format!("{}{suffix}", rustfs_utils::http::MINIO_INTERNAL_PREFIX),
source.transition_tier.clone(),
);
let mut target = tiered_equivalence_target(&source);
Arc::make_mut(&mut target.user_defined)
.insert(rustfs_utils::http::internal_key_rustfs(suffix), source.transition_tier.clone());
assert!(is_equivalent_data_movement_tiered_object(&source, &target));
}
#[test]
fn equivalent_data_movement_tiered_object_requires_hydrated_part_checksums() {
let mut source = tiered_equivalence_source();
source.parts = vec![rustfs_filemeta::ObjectPartInfo {
number: 1,
mod_time: Some(OffsetDateTime::UNIX_EPOCH + time::Duration::SECOND),
checksums: Some(HashMap::from([("CRC32C".to_string(), "AAAAAA==".to_string())])),
..Default::default()
}];
rustfs_utils::http::insert_str(
&mut source.metadata,
rustfs_utils::http::SUFFIX_PART_CHECKSUMS,
r#"[[1,[["CRC32C","AAAAAA=="]]]]"#.to_string(),
);
let mut target = tiered_equivalence_target(&source);
Arc::make_mut(&mut target.parts)[0].mod_time = None;
assert!(is_equivalent_data_movement_tiered_object(&source, &target));
let mut missing = target;
Arc::make_mut(&mut missing.parts)[0].checksums = None;
assert!(!is_equivalent_data_movement_tiered_object(&source, &missing));
}
#[test]
fn equivalent_data_movement_tiered_object_rejects_transition_mismatch() {
let source = tiered_equivalence_source();
@@ -3353,6 +3898,16 @@ mod tests {
assert!(!is_equivalent_data_movement_tiered_object(&source, &target));
}
#[test]
fn equivalent_data_movement_tiered_object_rejects_transition_version_state_mismatch() {
let mut source = tiered_equivalence_source();
source.transition_version_state = rustfs_filemeta::TransitionVersionState::Exact;
let mut target = tiered_equivalence_target(&source);
target.transition_version_state = rustfs_filemeta::TransitionVersionState::Unknown;
assert!(!is_equivalent_data_movement_tiered_object(&source, &target));
}
#[test]
fn equivalent_data_movement_tiered_object_rejects_user_metadata_mismatch() {
let source = tiered_equivalence_source();
@@ -4022,6 +4577,8 @@ mod tests {
assert_eq!(snapshot.headers.get(SSEC_KEY_HEADER), request_headers.get(SSEC_KEY_HEADER));
assert_eq!(snapshot.headers.get(SSEC_KEY_MD5_HEADER), request_headers.get(SSEC_KEY_MD5_HEADER));
assert!(snapshot.headers.get("authorization").is_none());
assert!(snapshot.headers.values().all(http::HeaderValue::is_sensitive));
assert!(!format!("{:?}", snapshot.headers).contains("secret-key"));
assert_eq!(
snapshot.logical_size(),
u64::try_from(payload.len()).expect("test payload length should fit in u64")
@@ -109,6 +109,7 @@ async fn run_legacy_bitrot_test_for_object(root: &std::path::Path, disk_name: &s
FileInfoOpts {
data: true, // need inline data for inline objects
include_free_versions: false,
include_part_checksums: true,
},
) {
Ok(f) => f,
+1 -1
View File
@@ -37,6 +37,7 @@ crc-fast = { workspace = true }
rmp.workspace = true
rmp-serde.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
time = { workspace = true, features = ["parsing", "formatting", "macros", "serde"] }
uuid = { workspace = true, features = ["v4", "fast-rng", "serde", "macro-diagnostics"] }
tokio = { workspace = true, features = ["io-util", "macros", "sync", "fs", "rt-multi-thread"] }
@@ -54,7 +55,6 @@ arc-swap.workspace = true
criterion = { workspace = true, features = ["html_reports"] }
tempfile = { workspace = true }
proptest = "1"
serde_json.workspace = true
[[bench]]
name = "xl_meta_bench"
@@ -52,6 +52,7 @@ fn main() {
FileInfoOpts {
data: false,
include_free_versions: true,
include_part_checksums: true,
},
)
.expect("decode file info");
+21 -3
View File
@@ -17,9 +17,9 @@ use bytes::Bytes;
use rmp_serde::Serializer;
use rustfs_utils::HashAlgorithm;
use rustfs_utils::http::{
SUFFIX_COMPRESSION, SUFFIX_DATA_MOVED, SUFFIX_FREE_VERSION, SUFFIX_HEALING, SUFFIX_INLINE_DATA, SUFFIX_TIER_FV_ID,
SUFFIX_TIER_FV_MARKER, SUFFIX_TIER_SKIP_FV_ID, contains_key_str, get_str, has_internal_suffix, insert_str,
is_encryption_metadata_key, starts_with_ignore_ascii_case,
AMZ_OBJECT_TAGGING, SUFFIX_COMPRESSION, SUFFIX_DATA_MOVED, SUFFIX_DATA_MOVED_TAGS, SUFFIX_FREE_VERSION, SUFFIX_HEALING,
SUFFIX_INLINE_DATA, SUFFIX_TIER_FV_ID, SUFFIX_TIER_FV_MARKER, SUFFIX_TIER_SKIP_FV_ID, contains_key_str, get_str,
has_internal_suffix, insert_str, is_encryption_metadata_key, starts_with_ignore_ascii_case,
};
use s3s::dto::{RestoreStatus, Timestamp};
use s3s::header::X_AMZ_RESTORE;
@@ -232,6 +232,17 @@ pub enum TransitionVersionState {
Exact,
}
impl TransitionVersionState {
pub const fn as_str(self) -> &'static str {
match self {
Self::Unknown => "unknown",
Self::KnownDisabled => "known-disabled",
Self::SuspendedNull => "suspended-null",
Self::Exact => "exact",
}
}
}
#[derive(PartialEq, Clone, Default)]
pub struct FileInfo {
pub volume: String,
@@ -1151,9 +1162,16 @@ impl FileInfo {
}
pub fn set_data_moved(&mut self) {
let tags_proof = format!("v1:{}", self.metadata.get(AMZ_OBJECT_TAGGING).map(String::as_str).unwrap_or_default());
insert_str(&mut self.metadata, SUFFIX_DATA_MOVED_TAGS, tags_proof);
insert_str(&mut self.metadata, SUFFIX_DATA_MOVED, "true".to_string());
}
pub fn acknowledge_data_movement(&mut self) {
// Keep both empty aliases so mixed-version disks retain one metadata identity.
insert_str(&mut self.metadata, SUFFIX_DATA_MOVED, String::new());
}
pub fn inline_data(&self) -> bool {
contains_key_str(&self.metadata, SUFFIX_INLINE_DATA) && !self.is_remote()
}
+156 -97
View File
@@ -206,6 +206,42 @@ fn persist_reset_statuses(meta_sys: &mut HashMap<String, Vec<u8>>, reset_statuse
}
}
pub fn parse_replication_timestamp(value: &str) -> Option<OffsetDateTime> {
const DISPLAY_FORMAT: &[time::format_description::BorrowedFormatItem<'_>] = time::macros::format_description!(
"[year sign:automatic]-[month]-[day] [hour padding:none]:[minute]:[second].[subsecond] [offset_hour sign:mandatory]:[offset_minute]:[offset_second]"
);
OffsetDateTime::parse(value, &Rfc3339)
.or_else(|_| OffsetDateTime::parse(value, DISPLAY_FORMAT))
.ok()
}
fn format_replication_timestamp(value: Option<OffsetDateTime>) -> String {
let value = value.unwrap_or(OffsetDateTime::UNIX_EPOCH);
value
.to_offset(time::UtcOffset::UTC)
.format(&Rfc3339)
.unwrap_or_else(|_| value.to_string())
}
fn persist_delete_marker_replication_state(meta_sys: &mut HashMap<String, Vec<u8>>, state: &ReplicationState) {
if !state.replica_status.is_empty() {
insert_bytes(meta_sys, SUFFIX_REPLICA_STATUS, state.replica_status.as_str().as_bytes().to_vec());
insert_bytes(
meta_sys,
SUFFIX_REPLICA_TIMESTAMP,
format_replication_timestamp(state.replica_timestamp).into_bytes(),
);
}
if let Some(status) = state.replication_status_internal.as_ref().filter(|status| !status.is_empty()) {
insert_bytes(meta_sys, SUFFIX_REPLICATION_STATUS, status.as_bytes().to_vec());
insert_bytes(
meta_sys,
SUFFIX_REPLICATION_TIMESTAMP,
format_replication_timestamp(state.replication_timestamp).into_bytes(),
);
}
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct FileMeta {
pub versions: Vec<FileMetaShallowVersion>,
@@ -213,6 +249,13 @@ pub struct FileMeta {
pub meta_ver: u8,
}
struct FileInfoDecodeOptions {
read_data: bool,
include_free_versions: bool,
all_parts: bool,
include_part_checksums: bool,
}
impl FileMeta {
pub fn new() -> Self {
Self {
@@ -511,53 +554,8 @@ impl FileMeta {
}
if fi.deleted {
if !fi.delete_marker_replication_status().is_empty()
&& let Some(delete_marker) = ventry.delete_marker.as_mut()
{
if fi.delete_marker_replication_status() == ReplicationStatusType::Replica {
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICA_STATUS,
fi.replication_state_internal
.as_ref()
.map(|v| v.replica_status.clone())
.unwrap_or_default()
.as_str()
.as_bytes()
.to_vec(),
);
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICA_TIMESTAMP,
fi.replication_state_internal
.as_ref()
.map(|v| v.replica_timestamp.unwrap_or(OffsetDateTime::UNIX_EPOCH).to_string())
.unwrap_or_default()
.as_bytes()
.to_vec(),
);
} else {
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICATION_STATUS,
fi.replication_state_internal
.as_ref()
.map(|v| v.replication_status_internal.clone().unwrap_or_default())
.unwrap_or_default()
.as_bytes()
.to_vec(),
);
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICATION_TIMESTAMP,
fi.replication_state_internal
.as_ref()
.map(|v| v.replication_timestamp.unwrap_or(OffsetDateTime::UNIX_EPOCH).to_string())
.unwrap_or_default()
.as_bytes()
.to_vec(),
);
}
if let (Some(delete_marker), Some(state)) = (ventry.delete_marker.as_mut(), fi.replication_state_internal.as_ref()) {
persist_delete_marker_replication_state(&mut delete_marker.meta_sys, state);
}
if !fi.version_purge_status().is_empty()
@@ -609,51 +607,8 @@ impl FileMeta {
}
if let Some(delete_marker) = v.delete_marker.as_mut() {
if !fi.delete_marker_replication_status().is_empty() {
if fi.delete_marker_replication_status() == ReplicationStatusType::Replica {
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICA_STATUS,
fi.replication_state_internal
.as_ref()
.map(|v| v.replica_status.clone())
.unwrap_or_default()
.as_str()
.as_bytes()
.to_vec(),
);
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICA_TIMESTAMP,
fi.replication_state_internal
.as_ref()
.map(|v| v.replica_timestamp.unwrap_or(OffsetDateTime::UNIX_EPOCH).to_string())
.unwrap_or_default()
.as_bytes()
.to_vec(),
);
} else {
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICATION_STATUS,
fi.replication_state_internal
.as_ref()
.map(|v| v.replication_status_internal.clone().unwrap_or_default())
.unwrap_or_default()
.as_bytes()
.to_vec(),
);
insert_bytes(
&mut delete_marker.meta_sys,
SUFFIX_REPLICATION_TIMESTAMP,
fi.replication_state_internal
.as_ref()
.map(|v| v.replication_timestamp.unwrap_or(OffsetDateTime::UNIX_EPOCH).to_string())
.unwrap_or_default()
.as_bytes()
.to_vec(),
);
}
if let Some(state) = fi.replication_state_internal.as_ref() {
persist_delete_marker_replication_state(&mut delete_marker.meta_sys, state);
}
if let Some(state) = fi.replication_state_internal.as_ref() {
@@ -773,6 +728,47 @@ impl FileMeta {
read_data: bool,
include_free_versions: bool,
all_parts: bool,
) -> Result<FileInfo> {
self.to_fileinfo_with_part_checksums(
volume,
path,
version_id,
FileInfoDecodeOptions {
read_data,
include_free_versions,
all_parts,
include_part_checksums: true,
},
)
}
pub fn into_fileinfo_without_part_checksums(
&self,
volume: &str,
path: &str,
version_id: &str,
read_data: bool,
include_free_versions: bool,
) -> Result<FileInfo> {
self.to_fileinfo_with_part_checksums(
volume,
path,
version_id,
FileInfoDecodeOptions {
read_data,
include_free_versions,
all_parts: true,
include_part_checksums: false,
},
)
}
fn to_fileinfo_with_part_checksums(
&self,
volume: &str,
path: &str,
version_id: &str,
opts: FileInfoDecodeOptions,
) -> Result<FileInfo> {
let vid = {
if !version_id.is_empty() {
@@ -795,7 +791,7 @@ impl FileMeta {
if header.free_version() {
non_free_versions -= 1;
if include_free_versions
if opts.include_free_versions
&& found_free_version.is_none()
&& let Ok(found_free_fi) = ver.parse_version_meta()
&& found_free_fi.version_type != VersionType::Invalid
@@ -806,7 +802,8 @@ impl FileMeta {
// Known side effect: if a disk holds only free versions and they are
// corrupt, `into_fileinfo` falls through to `FileNotFound` (not
// `FileCorrupt`), so that disk is not enqueued for heal.
match found_free_fi.into_fileinfo(volume, path, all_parts) {
match found_free_fi.to_fileinfo_with_part_checksums(volume, path, opts.all_parts, opts.include_part_checksums)
{
Ok(mut free_fi) => {
free_fi.is_latest = true;
found_free_version = Some(free_fi);
@@ -834,14 +831,14 @@ impl FileMeta {
found = true;
let mut fi = ver.into_fileinfo(volume, path, all_parts)?;
let mut fi = ver.to_fileinfo_with_part_checksums(volume, path, opts.all_parts, opts.include_part_checksums)?;
fi.is_latest = is_latest;
if let Some(_d) = succ_mod_time {
fi.successor_mod_time = succ_mod_time;
}
if read_data && fi.inline_data() {
if opts.read_data && fi.inline_data() {
fi.data = self.find_inline_data_for_version(fi.version_id)?.map(bytes::Bytes::from);
}
@@ -850,7 +847,7 @@ impl FileMeta {
if !found {
if version_id.is_empty() {
if include_free_versions
if opts.include_free_versions
&& non_free_versions == 0
&& let Some(free_version) = found_free_version
{
@@ -1537,6 +1534,68 @@ mod test {
assert_eq!(meta_sys2.len(), 2, "must not create a double-prefixed key");
}
#[test]
fn persist_delete_marker_replication_state_keeps_replica_and_target_statuses() {
let replica_timestamp = OffsetDateTime::UNIX_EPOCH + time::Duration::SECOND;
let replication_timestamp = replica_timestamp + time::Duration::SECOND;
let state = ReplicationState {
replica_status: ReplicationStatusType::Replica,
replica_timestamp: Some(replica_timestamp),
replication_status_internal: Some("arn:target=COMPLETED;".to_string()),
replication_timestamp: Some(replication_timestamp),
..Default::default()
};
let mut meta_sys = HashMap::new();
let replica_timestamp_string = replica_timestamp
.format(&Rfc3339)
.expect("timestamp should format as RFC3339");
let replication_timestamp_string = replication_timestamp
.format(&Rfc3339)
.expect("timestamp should format as RFC3339");
persist_delete_marker_replication_state(&mut meta_sys, &state);
assert_eq!(
rustfs_utils::http::get_bytes(&meta_sys, SUFFIX_REPLICA_STATUS).as_deref(),
Some(b"REPLICA".as_slice())
);
assert_eq!(
rustfs_utils::http::get_bytes(&meta_sys, SUFFIX_REPLICA_TIMESTAMP).as_deref(),
Some(replica_timestamp_string.as_bytes())
);
assert_eq!(
rustfs_utils::http::get_bytes(&meta_sys, SUFFIX_REPLICATION_STATUS).as_deref(),
Some(b"arn:target=COMPLETED;".as_slice())
);
assert_eq!(
rustfs_utils::http::get_bytes(&meta_sys, SUFFIX_REPLICATION_TIMESTAMP).as_deref(),
Some(replication_timestamp_string.as_bytes())
);
}
#[test]
fn persist_delete_marker_replication_timestamp_normalizes_second_offset_to_utc() {
let timestamp = OffsetDateTime::UNIX_EPOCH.to_offset(time::UtcOffset::from_hms(5, 30, 15).expect("valid offset"));
assert_eq!(parse_replication_timestamp(&timestamp.to_string()), Some(timestamp));
let state = ReplicationState {
replica_status: ReplicationStatusType::Replica,
replica_timestamp: Some(timestamp),
replication_status_internal: Some("arn:target=COMPLETED;".to_string()),
replication_timestamp: Some(timestamp),
..Default::default()
};
let mut meta_sys = HashMap::new();
persist_delete_marker_replication_state(&mut meta_sys, &state);
for suffix in [SUFFIX_REPLICA_TIMESTAMP, SUFFIX_REPLICATION_TIMESTAMP] {
let persisted = rustfs_utils::http::get_bytes(&meta_sys, suffix).expect("timestamp must be persisted");
let persisted = std::str::from_utf8(&persisted).expect("timestamp must be UTF-8");
assert_eq!(parse_replication_timestamp(persisted), Some(timestamp));
assert_ne!(persisted, OffsetDateTime::UNIX_EPOCH.to_string());
}
}
/// Regression test for rustfs/rustfs#2715: a corrupted version count in
/// xl.meta must yield a decode error instead of sizing a huge allocation
/// from the bogus count (which aborts the whole process).
+603 -120
View File
@@ -29,11 +29,12 @@ use super::*;
use crate::{ChecksumInfo, TransitionVersionState};
use rustfs_utils::HashAlgorithm;
use rustfs_utils::http::{
RUSTFS_INTERNAL_PREFIX, SUFFIX_CRC, SUFFIX_FREE_VERSION, SUFFIX_INLINE_DATA, SUFFIX_PURGESTATUS, SUFFIX_TIER_FV_ID,
RUSTFS_INTERNAL_PREFIX, SUFFIX_CRC, SUFFIX_FREE_VERSION, SUFFIX_INLINE_DATA, SUFFIX_PART_CHECKSUMS, SUFFIX_PURGESTATUS,
SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX, SUFFIX_REPLICATION_RESET_ARN_PREFIX, SUFFIX_TIER_FV_ID,
SUFFIX_TIER_FV_MARKER, SUFFIX_TRANSITION_STATUS, SUFFIX_TRANSITION_TIER, SUFFIX_TRANSITION_TIER_DESTINATION_ID,
SUFFIX_TRANSITIONED_OBJECTNAME, SUFFIX_TRANSITIONED_VERSION_ID, SUFFIX_TRANSITIONED_VERSION_STATE, contains_key_bytes,
get_bytes, get_consistent_bytes, get_str, has_internal_suffix, insert_bytes, is_internal_key, remove_bytes,
strip_internal_prefix, target_delete_marker_versions,
strip_internal_prefix, strip_internal_prefix_preserving_case, target_delete_marker_versions,
};
const MSGPACK_EXT8: u8 = 0xc7;
@@ -258,63 +259,181 @@ fn parse_legacy_uuid_bytes(bytes: &[u8], field: &str) -> Result<Option<Uuid>> {
/// Legacy RustFS writes used 16 raw UUID bytes. New writes and MinIO-migrated
/// records use the provider's exact UTF-8 version text. Empty, nil UUID, and
/// malformed bytes are not usable remote versions.
fn transitioned_version_from_meta_sys(meta_sys: &HashMap<String, Vec<u8>>) -> Result<Option<String>> {
if !contains_key_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_ID) {
return Ok(None);
}
let Some(value) = get_consistent_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_ID) else {
return Ok(None);
fn transition_version_state_from_bytes(value: Option<&[u8]>) -> Result<TransitionVersionState> {
let Some(value) = value else {
return Ok(TransitionVersionState::Unknown);
};
let value = value.to_vec();
match value {
b"known-disabled" => Ok(TransitionVersionState::KnownDisabled),
b"suspended-null" => Ok(TransitionVersionState::SuspendedNull),
b"exact" => Ok(TransitionVersionState::Exact),
b"unknown" => Ok(TransitionVersionState::Unknown),
_ => Err(Error::FileCorrupt),
}
}
fn transitioned_version_from_bytes(value: Option<&[u8]>, state: TransitionVersionState) -> Option<String> {
let value = value?;
if value.is_empty() {
return Ok(None);
return None;
}
if let Ok(id) = Uuid::from_slice(&value) {
return Ok((!id.is_nil()).then(|| id.to_string()));
if state == TransitionVersionState::Unknown
&& let Ok(id) = Uuid::from_slice(value)
{
return (!id.is_nil()).then(|| id.to_string());
}
let Ok(value) = String::from_utf8(value) else {
return Ok(None);
let Ok(value) = std::str::from_utf8(value) else {
return None;
};
if value.is_empty()
|| value.len() > MAX_TRANSITION_VERSION_LEN
|| value.chars().any(char::is_control)
|| Uuid::parse_str(&value).is_ok_and(|id| id.is_nil())
|| Uuid::parse_str(value).is_ok_and(|id| id.is_nil())
{
Ok(None)
None
} else {
Ok(Some(value))
Some(value.to_string())
}
}
fn transition_version_state_from_meta_sys(
meta_sys: &HashMap<String, Vec<u8>>,
version: Option<&str>,
) -> Result<TransitionVersionState> {
if !contains_key_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_STATE) {
return Ok(TransitionVersionState::Unknown);
}
let value = get_consistent_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_STATE).ok_or(Error::FileCorrupt)?;
let state = match value {
b"known-disabled" => TransitionVersionState::KnownDisabled,
b"suspended-null" => TransitionVersionState::SuspendedNull,
b"exact" => TransitionVersionState::Exact,
b"unknown" => TransitionVersionState::Unknown,
_ => return Err(Error::FileCorrupt),
};
fn validate_transition_version_state(state: TransitionVersionState, version: Option<&str>) -> Result<()> {
let valid = match state {
TransitionVersionState::Unknown | TransitionVersionState::KnownDisabled => version.is_none(),
TransitionVersionState::SuspendedNull => version == Some("null"),
TransitionVersionState::Exact => version.is_some_and(|value| value != "null"),
};
valid.then_some(state).ok_or(Error::FileCorrupt)
valid.then_some(()).ok_or(Error::FileCorrupt)
}
fn transition_version_state_bytes(state: TransitionVersionState) -> &'static [u8] {
match state {
TransitionVersionState::Unknown => b"unknown",
TransitionVersionState::KnownDisabled => b"known-disabled",
TransitionVersionState::SuspendedNull => b"suspended-null",
TransitionVersionState::Exact => b"exact",
#[derive(Default)]
struct DerivedInternalMetadata<'a> {
checksum: Option<&'a [u8]>,
part_checksums: Option<&'a [u8]>,
transition_status: Option<&'a [u8]>,
transitioned_object: Option<&'a [u8]>,
transitioned_version: Option<&'a [u8]>,
transitioned_version_state: Option<&'a [u8]>,
transition_tier: Option<&'a [u8]>,
}
impl<'a> DerivedInternalMetadata<'a> {
fn from_meta_sys(meta_sys: &'a HashMap<String, Vec<u8>>) -> Result<Self> {
let mut canonical = Self::default();
let mut legacy = Self::default();
for (key, value) in meta_sys {
let Some(suffix) = rustfs_utils::http::strip_internal_prefix_preserving_case(key) else {
continue;
};
let (canonical_slot, legacy_slot, expected_suffix) = if suffix.eq_ignore_ascii_case(SUFFIX_CRC) {
(&mut canonical.checksum, &mut legacy.checksum, SUFFIX_CRC)
} else if suffix.eq_ignore_ascii_case(SUFFIX_PART_CHECKSUMS) {
(&mut canonical.part_checksums, &mut legacy.part_checksums, SUFFIX_PART_CHECKSUMS)
} else if suffix.eq_ignore_ascii_case(SUFFIX_TRANSITION_STATUS) {
(&mut canonical.transition_status, &mut legacy.transition_status, SUFFIX_TRANSITION_STATUS)
} else if suffix.eq_ignore_ascii_case(SUFFIX_TRANSITIONED_OBJECTNAME) {
(
&mut canonical.transitioned_object,
&mut legacy.transitioned_object,
SUFFIX_TRANSITIONED_OBJECTNAME,
)
} else if suffix.eq_ignore_ascii_case(SUFFIX_TRANSITIONED_VERSION_ID) {
(
&mut canonical.transitioned_version,
&mut legacy.transitioned_version,
SUFFIX_TRANSITIONED_VERSION_ID,
)
} else if suffix.eq_ignore_ascii_case(SUFFIX_TRANSITIONED_VERSION_STATE) {
(
&mut canonical.transitioned_version_state,
&mut legacy.transitioned_version_state,
SUFFIX_TRANSITIONED_VERSION_STATE,
)
} else if suffix.eq_ignore_ascii_case(SUFFIX_TRANSITION_TIER) {
(&mut canonical.transition_tier, &mut legacy.transition_tier, SUFFIX_TRANSITION_TIER)
} else {
continue;
};
let slot = if suffix == expected_suffix
&& (key.starts_with(RUSTFS_INTERNAL_PREFIX) || key.starts_with(rustfs_utils::http::MINIO_INTERNAL_PREFIX))
{
canonical_slot
} else {
legacy_slot
};
if slot.is_some_and(|current| current != value.as_slice()) {
return Err(Error::FileCorrupt);
}
*slot = Some(value.as_slice());
}
Ok(Self {
checksum: canonical.checksum.or(legacy.checksum),
part_checksums: canonical.part_checksums.or(legacy.part_checksums),
transition_status: canonical.transition_status.or(legacy.transition_status),
transitioned_object: canonical.transitioned_object.or(legacy.transitioned_object),
transitioned_version: canonical.transitioned_version.or(legacy.transitioned_version),
transitioned_version_state: canonical.transitioned_version_state.or(legacy.transitioned_version_state),
transition_tier: canonical.transition_tier.or(legacy.transition_tier),
})
}
}
struct UniquePartChecksums(HashMap<String, String>);
impl<'de> serde::Deserialize<'de> for UniquePartChecksums {
fn deserialize<D>(deserializer: D) -> std::result::Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
struct UniquePartChecksumsVisitor;
impl<'de> serde::de::Visitor<'de> for UniquePartChecksumsVisitor {
type Value = UniquePartChecksums;
fn expecting(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("an array of unique checksum name and value pairs")
}
fn visit_seq<A>(self, mut seq: A) -> std::result::Result<Self::Value, A::Error>
where
A: serde::de::SeqAccess<'de>,
{
let mut checksums = HashMap::with_capacity(seq.size_hint().unwrap_or_default());
while let Some((key, value)) = seq.next_element::<(String, String)>()? {
if checksums.insert(key, value).is_some() {
return Err(serde::de::Error::custom("duplicate part checksum name"));
}
}
Ok(UniquePartChecksums(checksums))
}
}
deserializer.deserialize_seq(UniquePartChecksumsVisitor)
}
}
impl FileInfo {
pub fn hydrate_data_movement_part_checksums(&mut self) -> Result<()> {
let present = self
.metadata
.keys()
.any(|key| has_internal_suffix(key, SUFFIX_PART_CHECKSUMS));
if !present {
return Ok(());
}
let encoded = rustfs_utils::http::get_consistent_str(&self.metadata, SUFFIX_PART_CHECKSUMS).ok_or(Error::FileCorrupt)?;
let persisted = serde_json::from_str::<Vec<(usize, UniquePartChecksums)>>(encoded).map_err(|_| Error::FileCorrupt)?;
let mut part_indices = HashMap::with_capacity(self.parts.len());
for (index, part) in self.parts.iter().enumerate() {
if part_indices.insert(part.number, index).is_some() {
return Err(Error::FileCorrupt);
}
}
for (part_number, UniquePartChecksums(checksums)) in persisted {
let index = part_indices.remove(&part_number).ok_or(Error::FileCorrupt)?;
let part = self.parts.get_mut(index).ok_or(Error::FileCorrupt)?;
part.checksums = Some(checksums);
}
Ok(())
}
}
@@ -322,21 +441,10 @@ fn set_transition_version_state(meta_sys: &mut HashMap<String, Vec<u8>>, state:
if state == TransitionVersionState::Unknown {
remove_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_STATE);
} else {
insert_bytes(
meta_sys,
SUFFIX_TRANSITIONED_VERSION_STATE,
transition_version_state_bytes(state).to_vec(),
);
insert_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_STATE, state.as_str().as_bytes().to_vec());
}
}
fn legacy_transitioned_version_id_from_meta_sys(meta_sys: &HashMap<String, Vec<u8>>) -> Option<Uuid> {
transitioned_version_from_meta_sys(meta_sys)
.ok()
.flatten()
.and_then(|value| Uuid::parse_str(&value).ok())
}
fn transitioned_version_bytes(fi: &FileInfo) -> Option<Vec<u8>> {
fi.transition_version
.as_ref()
@@ -448,6 +556,17 @@ impl FileMetaShallowVersion {
pub fn into_fileinfo(&self, volume: &str, path: &str, all_parts: bool) -> Result<FileInfo> {
self.parse_version_meta()?.into_fileinfo(volume, path, all_parts)
}
pub(super) fn to_fileinfo_with_part_checksums(
&self,
volume: &str,
path: &str,
all_parts: bool,
include_part_checksums: bool,
) -> Result<FileInfo> {
self.parse_version_meta()?
.to_fileinfo_with_part_checksums(volume, path, all_parts, include_part_checksums)
}
}
impl TryFrom<FileMetaVersion> for FileMetaShallowVersion {
@@ -769,8 +888,16 @@ impl FileMetaVersion {
}
pub fn into_fileinfo(&self, volume: &str, path: &str, all_parts: bool) -> Result<FileInfo> {
// Only the Object arm carries part arrays and can fail the length guard; the
// Legacy and Delete arms have no part arrays and stay infallible.
self.to_fileinfo_with_part_checksums(volume, path, all_parts, true)
}
pub(super) fn to_fileinfo_with_part_checksums(
&self,
volume: &str,
path: &str,
all_parts: bool,
include_part_checksums: bool,
) -> Result<FileInfo> {
let mut fi = match self.version_type {
VersionType::Invalid | VersionType::Legacy => {
if let Some(ref legacy) = self.legacy_object {
@@ -788,14 +915,14 @@ impl FileMetaVersion {
self.object
.as_ref()
.unwrap_or(&default_object)
.into_fileinfo(volume, path, all_parts)?
.to_fileinfo_with_part_checksums(volume, path, all_parts, include_part_checksums)?
}
VersionType::Delete => {
let default_marker = MetaDeleteMarker::default();
self.delete_marker
.as_ref()
.unwrap_or(&default_marker)
.into_fileinfo(volume, path, all_parts)
.into_fileinfo(volume, path, all_parts)?
}
};
fi.uses_legacy_checksum = self.uses_legacy_checksum;
@@ -2390,7 +2517,18 @@ impl MetaObject {
}
pub fn into_fileinfo(&self, volume: &str, path: &str, all_parts: bool) -> Result<FileInfo> {
self.to_fileinfo_with_part_checksums(volume, path, all_parts, true)
}
fn to_fileinfo_with_part_checksums(
&self,
volume: &str,
path: &str,
all_parts: bool,
include_part_checksums: bool,
) -> Result<FileInfo> {
let version_id = self.version_id.filter(|&vid| !vid.is_nil());
let derived_metadata = DerivedInternalMetadata::from_meta_sys(&self.meta_sys)?;
let parts = if all_parts {
let n = self.part_numbers.len();
@@ -2474,7 +2612,10 @@ impl MetaObject {
}
}
let checksum = get_bytes(&self.meta_sys, SUFFIX_CRC).map(Bytes::from);
let checksum = derived_metadata
.checksum
.filter(|checksum| !checksum.is_empty())
.map(Bytes::copy_from_slice);
let erasure = ErasureInfo {
algorithm: self.erasure_algorithm.to_string(),
@@ -2486,20 +2627,29 @@ impl MetaObject {
..Default::default()
};
let transition_status = get_bytes(&self.meta_sys, SUFFIX_TRANSITION_STATUS)
.map(|v| String::from_utf8_lossy(&v).to_string())
let transition_status = derived_metadata
.transition_status
.filter(|value| !value.is_empty())
.map(|v| String::from_utf8_lossy(v).to_string())
.unwrap_or_default();
let transitioned_objname = get_bytes(&self.meta_sys, SUFFIX_TRANSITIONED_OBJECTNAME)
.map(|v| String::from_utf8_lossy(&v).to_string())
let transitioned_objname = derived_metadata
.transitioned_object
.filter(|value| !value.is_empty())
.map(|v| String::from_utf8_lossy(v).to_string())
.unwrap_or_default();
let transition_version = transitioned_version_from_meta_sys(&self.meta_sys)?;
let transition_version_state = transition_version_state_from_meta_sys(&self.meta_sys, transition_version.as_deref())?;
let transition_version_state = transition_version_state_from_bytes(derived_metadata.transitioned_version_state)?;
let transition_version = transitioned_version_from_bytes(derived_metadata.transitioned_version, transition_version_state);
if derived_metadata.transitioned_version_state.is_some() {
validate_transition_version_state(transition_version_state, transition_version.as_deref())?;
}
let transition_version_id = transition_version.as_deref().and_then(|value| Uuid::parse_str(value).ok());
let transition_tier = get_bytes(&self.meta_sys, SUFFIX_TRANSITION_TIER)
.map(|v| String::from_utf8_lossy(&v).to_string())
let transition_tier = derived_metadata
.transition_tier
.filter(|value| !value.is_empty())
.map(|v| String::from_utf8_lossy(v).to_string())
.unwrap_or_default();
Ok(FileInfo {
let mut file_info = FileInfo {
version_id,
erasure,
data_dir: self.data_dir,
@@ -2519,7 +2669,11 @@ impl MetaObject {
transition_version_state,
transition_tier,
..Default::default()
})
};
if all_parts && include_part_checksums {
file_info.hydrate_data_movement_part_checksums()?;
}
Ok(file_info)
}
pub fn set_transition(&mut self, fi: &FileInfo) {
@@ -2710,39 +2864,31 @@ fn get_internal_replication_state(metadata: &HashMap<String, String>) -> Option<
continue;
}
let sub_key_opt = strip_internal_prefix(k);
if let Some(ref sub_key) = sub_key_opt {
match sub_key.as_str() {
"replica-timestamp" => {
has = true;
rs.replica_timestamp = Some(OffsetDateTime::parse(v, &Rfc3339).unwrap_or(OffsetDateTime::UNIX_EPOCH));
}
"replica-status" => {
has = true;
rs.replica_status = ReplicationStatusType::from(v.as_str());
}
"replication-timestamp" => {
has = true;
rs.replication_timestamp = Some(OffsetDateTime::parse(v, &Rfc3339).unwrap_or(OffsetDateTime::UNIX_EPOCH))
}
"replication-status" => {
has = true;
rs.replication_status_internal = Some(v.clone());
rs.targets = replication_statuses_map(v.as_str());
}
_ => {
if let Some(arn) = sub_key.strip_prefix("replication-reset-") {
has = true;
// Store the canonical full-header key so the map matches
// the key `target_reset_header()` produces on the
// write/lookup side. Storing the bare ARN keyed the map
// inconsistently (bare on read, full on write), which
// could drop reset state across merge/reflatten cycles
// (backlog#799 B16).
rs.reset_statuses_map
.insert(crate::replication::target_reset_header(arn), v.clone());
}
}
if let Some(sub_key) = strip_internal_prefix_preserving_case(k) {
if sub_key.eq_ignore_ascii_case(SUFFIX_REPLICA_TIMESTAMP) {
has = true;
rs.replica_timestamp = Some(parse_replication_timestamp(v).unwrap_or(OffsetDateTime::UNIX_EPOCH));
} else if sub_key.eq_ignore_ascii_case(SUFFIX_REPLICA_STATUS) {
has = true;
rs.replica_status = ReplicationStatusType::from(v.as_str());
} else if sub_key.eq_ignore_ascii_case(SUFFIX_REPLICATION_TIMESTAMP) {
has = true;
rs.replication_timestamp = Some(parse_replication_timestamp(v).unwrap_or(OffsetDateTime::UNIX_EPOCH))
} else if sub_key.eq_ignore_ascii_case(SUFFIX_REPLICATION_STATUS) {
has = true;
rs.replication_status_internal = Some(v.clone());
rs.targets = replication_statuses_map(v.as_str());
} else if let Some(arn) = rustfs_utils::http::internal_key_strip_suffix_prefix(k, SUFFIX_REPLICATION_RESET_ARN_PREFIX)
{
has = true;
// Store the canonical full-header key so the map matches
// the key `target_reset_header()` produces on the
// write/lookup side. Storing the bare ARN keyed the map
// inconsistently (bare on read, full on write), which
// could drop reset state across merge/reflatten cycles
// (backlog#799 B16).
rs.reset_statuses_map
.insert(crate::replication::target_reset_header(&arn), v.clone());
}
}
}
@@ -2786,7 +2932,7 @@ impl MetaDeleteMarker {
contains_key_bytes(&self.meta_sys, SUFFIX_FREE_VERSION)
}
pub fn into_fileinfo(&self, volume: &str, path: &str, _all_parts: bool) -> FileInfo {
pub fn into_fileinfo(&self, volume: &str, path: &str, _all_parts: bool) -> Result<FileInfo> {
let metadata = self
.meta_sys
.clone()
@@ -2808,22 +2954,27 @@ impl MetaDeleteMarker {
if self.free_version() {
fi.set_tier_free_version();
fi.transition_tier = get_bytes(&self.meta_sys, SUFFIX_TRANSITION_TIER)
.map(|v| String::from_utf8_lossy(&v).to_string())
let derived_metadata = DerivedInternalMetadata::from_meta_sys(&self.meta_sys)?;
fi.transition_tier = derived_metadata
.transition_tier
.filter(|value| !value.is_empty())
.map(|value| String::from_utf8_lossy(value).to_string())
.unwrap_or_default();
fi.transitioned_objname = get_bytes(&self.meta_sys, SUFFIX_TRANSITIONED_OBJECTNAME)
.map(|v| String::from_utf8_lossy(&v).to_string())
fi.transitioned_objname = derived_metadata
.transitioned_object
.filter(|value| !value.is_empty())
.map(|value| String::from_utf8_lossy(value).to_string())
.unwrap_or_default();
fi.transition_version = transitioned_version_from_meta_sys(&self.meta_sys).ok().flatten();
fi.transition_version_id = legacy_transitioned_version_id_from_meta_sys(&self.meta_sys);
fi.transition_version_state =
transition_version_state_from_meta_sys(&self.meta_sys, fi.transition_version.as_deref())
.unwrap_or(TransitionVersionState::Unknown);
fi.transition_version_state = transition_version_state_from_bytes(derived_metadata.transitioned_version_state)?;
fi.transition_version =
transitioned_version_from_bytes(derived_metadata.transitioned_version, fi.transition_version_state);
fi.transition_version_id = fi.transition_version.as_deref().and_then(|value| Uuid::parse_str(value).ok());
if derived_metadata.transitioned_version_state.is_some() {
validate_transition_version_state(fi.transition_version_state, fi.transition_version.as_deref())?;
}
}
fi
Ok(fi)
}
pub fn encode_to<W: std::io::Write>(&self, wr: &mut W) -> Result<()> {
@@ -2941,6 +3092,13 @@ impl From<FileInfo> for MetaDeleteMarker {
if !is_internal_key(key) || is_skip_meta_key(key) {
continue;
}
if rustfs_utils::http::internal_key_strip_suffix_prefix(key, SUFFIX_REPLICATION_RESET_ARN_PREFIX).is_some()
|| rustfs_utils::http::internal_key_strip_suffix_prefix(key, SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX)
.is_some()
{
meta_sys.insert(key.clone(), metadata_value.as_bytes().to_vec());
continue;
}
let Some(suffix) = strip_internal_prefix(key) else {
continue;
};
@@ -2982,6 +3140,11 @@ impl From<FileInfo> for MetaDeleteMarker {
if !value.transition_tier.is_empty() {
insert_bytes(&mut meta_sys, SUFFIX_TRANSITION_TIER, value.transition_tier.as_bytes().to_vec());
}
if let Some(state) = value.replication_state_internal.as_ref() {
persist_delete_marker_replication_state(&mut meta_sys, state);
persist_reset_statuses(&mut meta_sys, &state.reset_statuses_map);
persist_target_delete_marker_versions(&mut meta_sys, &state.target_delete_marker_version_ids, &value.metadata);
}
Self {
version_id: value.version_id,
mod_time: value.mod_time,
@@ -3278,6 +3441,7 @@ pub fn file_info_from_raw(
FileInfoOpts {
data: read_data,
include_free_versions,
include_part_checksums: true,
},
)
}
@@ -3285,6 +3449,7 @@ pub fn file_info_from_raw(
pub struct FileInfoOpts {
pub data: bool,
pub include_free_versions: bool,
pub include_part_checksums: bool,
}
pub fn get_file_info(buf: &[u8], volume: &str, path: &str, version_id: &str, opts: FileInfoOpts) -> Result<FileInfo> {
@@ -3309,7 +3474,11 @@ pub fn get_file_info(buf: &[u8], volume: &str, path: &str, version_id: &str, opt
});
}
let fi = meta.into_fileinfo(volume, path, version_id, opts.data, opts.include_free_versions, true)?;
let fi = if opts.include_part_checksums {
meta.into_fileinfo(volume, path, version_id, opts.data, opts.include_free_versions, true)?
} else {
meta.into_fileinfo_without_part_checksums(volume, path, version_id, opts.data, opts.include_free_versions)?
};
Ok(fi)
}
@@ -3555,6 +3724,56 @@ mod tests {
assert!(!converted.meta_sys.contains_key("content-type"));
}
#[test]
fn delete_marker_conversion_does_not_lowercase_dynamic_replication_targets() {
let arn = "arn:rustfs:replication:us-east-1:TenantA:bucket";
let reset_suffix = format!("{SUFFIX_REPLICATION_RESET_ARN_PREFIX}{arn}");
let version_suffix = format!("{SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX}{arn}");
let mut marker = FileInfo::default();
marker.metadata.insert(
format!("{}{reset_suffix}", rustfs_utils::http::MINIO_INTERNAL_PREFIX),
"2026-08-12T00:00:00Z;COMPLETED".to_string(),
);
marker.metadata.insert(
format!("{}{version_suffix}", rustfs_utils::http::MINIO_INTERNAL_PREFIX),
"remote-version".to_string(),
);
marker.replication_state_internal = get_internal_replication_state(&marker.metadata);
let converted = MetaDeleteMarker::from(marker);
assert!(converted.meta_sys.keys().any(|key| key.ends_with(&reset_suffix)));
assert!(converted.meta_sys.keys().any(|key| key.ends_with(&version_suffix)));
assert!(!converted.meta_sys.keys().any(|key| key.contains("tenanta")));
let mut conflicting = FileInfo::default();
conflicting
.metadata
.insert(format!("{RUSTFS_INTERNAL_PREFIX}{version_suffix}"), "remote-version-a".to_string());
conflicting.metadata.insert(
format!("{}{version_suffix}", rustfs_utils::http::MINIO_INTERNAL_PREFIX),
"remote-version-b".to_string(),
);
conflicting.replication_state_internal = get_internal_replication_state(&conflicting.metadata);
assert!(
conflicting
.replication_state_internal
.as_ref()
.is_some_and(|state| state.target_delete_marker_version_ids_corrupt)
);
let roundtrip = MetaDeleteMarker::from(conflicting)
.into_fileinfo("bucket", "object", false)
.expect("dynamic replication aliases should remain decodable");
assert!(
roundtrip
.replication_state_internal
.as_ref()
.is_some_and(|state| state.target_delete_marker_version_ids_corrupt),
"conflicting dynamic aliases must remain corrupt across persistence"
);
}
#[derive(Serialize)]
enum LegacyDeleteVersionTypeFixture {
#[serde(rename = "DeleteMarker")]
@@ -3663,6 +3882,138 @@ mod tests {
assert!(matches!(res, Err(Error::FileCorrupt)), "short part_sizes must map to FileCorrupt");
}
#[test]
fn into_fileinfo_rejects_conflicting_derived_internal_aliases() {
for suffix in [
SUFFIX_CRC,
SUFFIX_TRANSITION_STATUS,
SUFFIX_TRANSITIONED_OBJECTNAME,
SUFFIX_TRANSITIONED_VERSION_ID,
SUFFIX_TRANSITIONED_VERSION_STATE,
SUFFIX_TRANSITION_TIER,
] {
let mut meta_sys = HashMap::from([(format!("{RUSTFS_INTERNAL_PREFIX}{suffix}"), vec![0xff, 1])]);
meta_sys.insert(format!("{}{suffix}", rustfs_utils::http::MINIO_INTERNAL_PREFIX), vec![0xfe, 2]);
let object = MetaObject {
meta_sys,
..Default::default()
};
assert_eq!(
object
.into_fileinfo("bucket", "key", false)
.expect_err("conflicting aliases must fail closed"),
Error::FileCorrupt,
"suffix {suffix}"
);
}
}
#[test]
fn into_fileinfo_recovers_noncanonical_binary_checksum_alias() {
let checksum = vec![0xff, 0x00, 0x80, 0x01];
let object = MetaObject {
meta_sys: HashMap::from([("X-Minio-Internal-crc".to_string(), checksum.clone())]),
..Default::default()
};
let file_info = object
.into_fileinfo("bucket", "key", false)
.expect("a single legacy checksum alias should remain readable");
assert_eq!(file_info.checksum.as_deref(), Some(checksum.as_slice()));
}
#[test]
fn into_fileinfo_prefers_canonical_rewrite_over_stale_mixed_case_alias() {
let checksum = vec![0xff, 0x00, 0x80, 0x01];
let mut object = MetaObject {
meta_sys: HashMap::from([("X-Minio-Internal-crc".to_string(), b"stale".to_vec())]),
..Default::default()
};
insert_bytes(&mut object.meta_sys, SUFFIX_CRC, checksum.clone());
let file_info = object
.into_fileinfo("bucket", "key", false)
.expect("canonical rewrites should supersede legacy mixed-case aliases");
assert_eq!(file_info.checksum.as_deref(), Some(checksum.as_slice()));
}
#[test]
fn into_fileinfo_recovers_data_movement_part_checksums() {
let mut object = object_with_parts(vec![1], vec![16], vec![16]);
insert_bytes(&mut object.meta_sys, SUFFIX_PART_CHECKSUMS, br#"[[1,[["CRC32C","AAAAAA=="]]]]"#.to_vec());
let file_info = object
.into_fileinfo("bucket", "key", true)
.expect("data movement part checksums should decode");
assert_eq!(
file_info.parts[0]
.checksums
.as_ref()
.and_then(|checksums| checksums.get("CRC32C"))
.map(String::as_str),
Some("AAAAAA==")
);
let mut deferred = object
.to_fileinfo_with_part_checksums("bucket", "key", true, false)
.expect("quorum candidates should retain raw checksum metadata");
assert!(deferred.parts[0].checksums.is_none());
deferred
.hydrate_data_movement_part_checksums()
.expect("the selected candidate should hydrate checksums once");
assert_eq!(deferred.parts[0].checksums, file_info.parts[0].checksums);
insert_bytes(&mut object.meta_sys, SUFFIX_PART_CHECKSUMS, b"not-json".to_vec());
assert_eq!(
object
.into_fileinfo("bucket", "key", true)
.expect_err("malformed data movement part checksums must fail closed"),
Error::FileCorrupt
);
for encoded in [
br#"[[1,[["CRC32C","AAAAAA=="]]],[1,[["CRC32C","BBBBBB=="]]]]"#.as_slice(),
br#"[[1,[["CRC32C","AAAAAA=="],["CRC32C","BBBBBB=="]]]]"#.as_slice(),
] {
insert_bytes(&mut object.meta_sys, SUFFIX_PART_CHECKSUMS, encoded.to_vec());
assert_eq!(
object
.into_fileinfo("bucket", "key", true)
.expect_err("duplicate part checksum keys must fail closed"),
Error::FileCorrupt
);
}
insert_bytes(&mut object.meta_sys, SUFFIX_PART_CHECKSUMS, br#"[[2,[["CRC32C","AAAAAA=="]]]]"#.to_vec());
assert_eq!(
object
.into_fileinfo("bucket", "key", true)
.expect_err("a sidecar for an unknown part must fail closed"),
Error::FileCorrupt
);
object.meta_sys = HashMap::from([
(
format!("{RUSTFS_INTERNAL_PREFIX}{SUFFIX_PART_CHECKSUMS}"),
br#"[[1,[["CRC32C","AAAAAA=="]]]]"#.to_vec(),
),
(
format!("{}{}", rustfs_utils::http::MINIO_INTERNAL_PREFIX, SUFFIX_PART_CHECKSUMS),
br#"[[1,[["CRC32C","BBBBBB=="]]]]"#.to_vec(),
),
]);
assert_eq!(
object
.into_fileinfo("bucket", "key", true)
.expect_err("conflicting sidecar aliases must fail closed"),
Error::FileCorrupt
);
}
#[test]
fn into_fileinfo_rejects_short_part_actual_sizes_including_empty() {
let obj = object_with_parts(vec![1, 2], vec![10, 20], vec![]);
@@ -4230,6 +4581,31 @@ mod tests {
assert_eq!(fi.transition_version_state, TransitionVersionState::Unknown);
}
#[test]
fn meta_object_transition_exact_rejects_legacy_raw_uuid_encoding() {
let mut sys = HashMap::new();
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_VERSION_ID, sample_version_id().as_bytes().to_vec());
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_VERSION_STATE, b"exact".to_vec());
let err = make_meta_object_with_sys(sys)
.into_fileinfo("b", "k", false)
.expect_err("exact remote versions must use their UTF-8 provider representation");
assert_eq!(err, Error::FileCorrupt);
}
#[test]
fn meta_object_transition_version_id_mixed_case_alias_is_recovered() {
let id = sample_version_id();
let sys = HashMap::from([("X-Minio-Internal-transitioned-versionID".to_string(), id.as_bytes().to_vec())]);
let fi = make_meta_object_with_sys(sys)
.into_fileinfo("b", "k", false)
.expect("a legacy mixed-case transition version alias should decode");
assert_eq!(fi.transition_version_id, Some(id));
assert_eq!(fi.transition_version, Some(id.to_string()));
}
#[test]
fn meta_object_transition_version_id_opaque_text_is_preserved() {
let mut sys = HashMap::new();
@@ -4271,8 +4647,10 @@ mod tests {
.map(Vec::as_slice),
Some(b"exact".as_slice())
);
let persisted_version = get_consistent_bytes(&object.meta_sys, SUFFIX_TRANSITIONED_VERSION_ID);
assert_eq!(
legacy_transitioned_version_id_from_meta_sys(&object.meta_sys),
transitioned_version_from_bytes(persisted_version, TransitionVersionState::Unknown)
.and_then(|value| Uuid::parse_str(&value).ok()),
Some(id),
"UUID exact writes must remain readable by the legacy UUID consumer"
);
@@ -4281,6 +4659,26 @@ mod tests {
assert_eq!(decoded.transition_version.as_deref(), Some(expected_version.as_str()));
}
#[test]
fn meta_object_transition_version_state_exact_preserves_sixteen_byte_opaque_text() {
let expected_version = "opaque.wasabi_01";
assert_eq!(expected_version.len(), 16);
let fi = FileInfo {
transition_status: "complete".to_string(),
transition_version: Some(expected_version.to_string()),
transition_version_state: TransitionVersionState::Exact,
..Default::default()
};
let decoded = MetaObject::from(fi)
.into_fileinfo("b", "k", false)
.expect("exact opaque transition version should round trip");
assert_eq!(decoded.transition_version.as_deref(), Some(expected_version));
assert_eq!(decoded.transition_version_id, None);
assert_eq!(decoded.transition_version_state, TransitionVersionState::Exact);
}
#[test]
fn set_transition_known_disabled_removes_stale_version_dual_keys() {
let mut meta_sys = HashMap::new();
@@ -4375,7 +4773,8 @@ mod tests {
mod_time: None,
meta_sys: sys,
}
.into_fileinfo("b", "k", false);
.into_fileinfo("b", "k", false)
.expect("nil tier version should remain an absent remote version");
assert_eq!(fi.transition_version_id, None);
}
@@ -4390,7 +4789,8 @@ mod tests {
mod_time: None,
meta_sys: sys,
}
.into_fileinfo("b", "k", false);
.into_fileinfo("b", "k", false)
.expect("legacy binary UUID tier version should decode");
assert_eq!(fi.transition_version_id, Some(id));
assert_eq!(fi.transition_version, Some(id.to_string()));
}
@@ -4407,7 +4807,8 @@ mod tests {
mod_time: Some(sample_mod_time()),
meta_sys: sys,
}
.into_fileinfo("b", "k", false);
.into_fileinfo("b", "k", false)
.expect("opaque tier version should remain readable");
assert_eq!(fi.transition_version_id, None);
assert_eq!(fi.transition_version.as_deref(), Some("opaque-generation-42"));
@@ -4429,12 +4830,67 @@ mod tests {
mod_time: Some(sample_mod_time()),
meta_sys: sys,
}
.into_fileinfo("b", "k", false);
.into_fileinfo("b", "k", false)
.expect("mixed-case tier aliases should decode");
assert_eq!(fi.transition_version_id, Some(id));
assert_eq!(fi.transition_version, Some(id.to_string()));
}
#[test]
fn delete_marker_free_version_recovers_mixed_case_transition_aliases() {
let id = sample_version_id();
let id_text = id.to_string();
let mut sys = HashMap::new();
insert_bytes(&mut sys, SUFFIX_FREE_VERSION, vec![]);
for (suffix, value) in [
(SUFFIX_TRANSITIONED_VERSION_ID, id_text.as_bytes()),
(SUFFIX_TRANSITIONED_VERSION_STATE, b"exact".as_slice()),
(SUFFIX_TRANSITION_TIER, b"WARM".as_slice()),
(SUFFIX_TRANSITIONED_OBJECTNAME, b"remote-object".as_slice()),
] {
sys.insert(format!("X-Minio-Internal-{suffix}"), value.to_vec());
}
let fi = MetaDeleteMarker {
version_id: Some(sample_version_id()),
mod_time: Some(sample_mod_time()),
meta_sys: sys,
}
.into_fileinfo("b", "k", false)
.expect("mixed-case tier aliases should decode");
assert_eq!(fi.transition_version_id, Some(id));
assert_eq!(fi.transition_version, Some(id.to_string()));
assert_eq!(fi.transition_version_state, TransitionVersionState::Exact);
assert_eq!(fi.transition_tier, "WARM");
assert_eq!(fi.transitioned_objname, "remote-object");
}
#[test]
fn delete_marker_free_version_rejects_conflicting_transition_aliases() {
let mut sys = HashMap::new();
insert_bytes(&mut sys, SUFFIX_FREE_VERSION, vec![]);
sys.insert(
format!("{RUSTFS_INTERNAL_PREFIX}{SUFFIX_TRANSITIONED_VERSION_ID}"),
b"source-version".to_vec(),
);
sys.insert(
format!("{}{}", rustfs_utils::http::MINIO_INTERNAL_PREFIX, SUFFIX_TRANSITIONED_VERSION_ID),
b"target-version".to_vec(),
);
let err = MetaDeleteMarker {
version_id: Some(sample_version_id()),
mod_time: Some(sample_mod_time()),
meta_sys: sys,
}
.into_fileinfo("b", "k", false)
.expect_err("conflicting transition aliases must fail closed");
assert_eq!(err, Error::FileCorrupt);
}
#[test]
fn version_header_sorts_before_prefers_object_over_delete_marker_on_equal_mod_time() {
let object = FileMetaVersionHeader {
@@ -4758,7 +5214,7 @@ mod tests {
#[test]
fn target_delete_marker_version_metadata_is_forward_and_backward_compatible() {
let arn = "arn:rustfs:replication:us-east-1:target:bucket";
let arn = "arn:rustfs:replication:us-east-1:TenantA:bucket";
let suffix = format!("{}{arn}", rustfs_utils::http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX);
let mut metadata = HashMap::from([(format!("{RUSTFS_INTERNAL_PREFIX}replication-status"), format!("{arn}=COMPLETED;"))]);
@@ -4803,7 +5259,7 @@ mod tests {
// must keep it keyed by `target_reset_header(arn)` (not the bare ARN) so
// `ReplicationState::target_state` finds it after a round trip
// (backlog#799 B16).
let arn = "arn:rustfs:replication:us-east-1:target:bucket";
let arn = "arn:rustfs:replication:us-east-1:TenantA:bucket";
let ts = "2026-06-30T00:00:00Z;reset-1".to_string();
let key = crate::replication::target_reset_header(arn);
let mut metadata = HashMap::new();
@@ -4822,6 +5278,33 @@ mod tests {
);
}
#[test]
fn get_internal_replication_state_accepts_rfc3339_and_rustfs_display_timestamps() {
let replica_timestamp = OffsetDateTime::UNIX_EPOCH + time::Duration::SECOND;
let replication_timestamp = replica_timestamp + time::Duration::SECOND;
let metadata = HashMap::from([
(format!("{RUSTFS_INTERNAL_PREFIX}{SUFFIX_REPLICA_STATUS}"), "REPLICA".to_string()),
(
format!("{RUSTFS_INTERNAL_PREFIX}{SUFFIX_REPLICA_TIMESTAMP}"),
replica_timestamp.to_string(),
),
(
format!("{RUSTFS_INTERNAL_PREFIX}{SUFFIX_REPLICATION_STATUS}"),
"arn:rustfs:replication:us-east-1:TenantA:bucket=COMPLETED;".to_string(),
),
(
format!("{RUSTFS_INTERNAL_PREFIX}{SUFFIX_REPLICATION_TIMESTAMP}"),
replication_timestamp
.format(&Rfc3339)
.expect("RFC3339 timestamp should format"),
),
]);
let state = get_internal_replication_state(&metadata).expect("replication metadata should parse");
assert_eq!(state.replica_timestamp, Some(replica_timestamp));
assert_eq!(state.replication_timestamp, Some(replication_timestamp));
}
// ---- Header signature (backlog#861 / B12) ----
fn signed_object() -> MetaObject {
+36 -1
View File
@@ -157,7 +157,7 @@ impl MetaCacheEntry {
});
}
let fi = fm.into_fileinfo(bucket, self.name.as_str(), "", false, false, true)?;
let fi = fm.into_fileinfo_without_part_checksums(bucket, self.name.as_str(), "", false, false)?;
return Ok(fi);
}
@@ -169,6 +169,7 @@ impl MetaCacheEntry {
FileInfoOpts {
data: false,
include_free_versions: false,
include_part_checksums: false,
},
)
}
@@ -2157,4 +2158,38 @@ mod tests {
let inner = caught.expect("file_info_versions must not panic");
assert!(matches!(inner, Err(Error::FileCorrupt)), "expected FileCorrupt");
}
#[test]
fn metacache_to_fileinfo_ignores_part_checksum_sidecar_regardless_of_cache_state() {
let mut meta = FileMeta::load(&create_real_xlmeta().expect("create real xl.meta")).expect("load real xl.meta");
let version_id = Uuid::parse_str("01234567-89ab-cdef-0123-456789abcdef").expect("valid fixture version id");
let (index, mut version) = meta.find_version(Some(version_id)).expect("find fixture object version");
rustfs_utils::http::insert_bytes(
&mut version.object.as_mut().expect("fixture object").meta_sys,
rustfs_utils::http::SUFFIX_PART_CHECKSUMS,
b"not-json".to_vec(),
);
meta.versions[index] = FileMetaShallowVersion::try_from(version).expect("replace fixture object version");
let encoded = meta.marshal_msg().expect("marshal object metadata");
let uncached = MetaCacheEntry {
name: "object".to_string(),
metadata: encoded.clone(),
cached: None,
reusable: false,
}
.to_fileinfo("bucket")
.expect("uncached metacache conversion must stay lazy");
let cached = MetaCacheEntry {
name: "object".to_string(),
metadata: encoded,
cached: Some(meta),
reusable: false,
}
.to_fileinfo("bucket")
.expect("cached metacache conversion must stay lazy");
assert_eq!(cached, uncached);
assert!(cached.parts.iter().all(|part| part.checksums.is_none()));
}
}
+2
View File
@@ -14,6 +14,8 @@
//! test endpoint index settings
#![recursion_limit = "256"]
use std::net::SocketAddr;
use tempfile::TempDir;
use tokio_util::sync::CancellationToken;
@@ -22,6 +22,8 @@
//! bucket-metadata-sys OnceCell) — under `cargo nextest` each test runs
//! in its own process so the OnceCell never collides.
#![recursion_limit = "256"]
use http::HeaderMap;
use rustfs_common::heal_channel::{HealOpts, HealScanMode};
use rustfs_heal::heal::{
@@ -21,6 +21,8 @@
//! These drive the REAL `ECStoreHealStorage` + `ECStore` against real disks.
//! Every test is `#[serial]`; under `cargo nextest` each runs in its own process.
#![recursion_limit = "256"]
use http::HeaderMap;
use rustfs_common::heal_channel::{HealOpts, HealScanMode};
use rustfs_heal::heal::storage::{
@@ -12,6 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
#![recursion_limit = "256"]
use http::HeaderMap;
use rustfs_common::heal_channel::{HealOpts, HealScanMode};
use rustfs_heal::heal::{
+60 -49
View File
@@ -14,19 +14,23 @@
use crate::IamStorageError;
use rustfs_policy::policy::Error as PolicyError;
use std::sync::Arc;
pub type Result<T> = core::result::Result<T, Error>;
#[derive(thiserror::Error, Debug)]
pub enum Error {
#[error(transparent)]
PolicyError(#[from] PolicyError),
// Arc payloads keep Clone variant-preserving for the non-cloneable inner
// errors (backlog#1831 PR2). Display is unchanged; the source() chain is
// not forwarded (Arc<E> does not implement std::error::Error).
#[error("{0}")]
PolicyError(Arc<PolicyError>),
#[error("{0}")]
StringError(String),
#[error("crypto: {0}")]
CryptoError(#[from] rustfs_crypto::Error),
CryptoError(Arc<rustfs_crypto::Error>),
#[error("user '{0}' does not exist")]
NoSuchUser(String),
@@ -58,15 +62,6 @@ pub enum Error {
#[error("not initialized")]
IamSysNotInitialized,
#[error("invalid service type: {0}")]
InvalidServiceType(String),
#[error("malformed credential")]
ErrCredMalformed,
#[error("CredNotInitialized")]
CredNotInitialized,
#[error("invalid access key length")]
InvalidAccessKeyLength,
@@ -79,27 +74,12 @@ pub enum Error {
#[error("group name contains reserved characters =,")]
GroupNameContainsReservedChars,
#[error("jwt err {0}")]
JWTError(jsonwebtoken::errors::Error),
#[error("no access key")]
NoAccessKey,
#[error("invalid token")]
InvalidToken,
#[error("invalid access_key")]
InvalidAccessKey,
#[error("access key is already in use")]
AccessKeyAlreadyExists,
#[error("action not allowed")]
IAMActionNotAllowed,
#[error("invalid expiration")]
InvalidExpiration,
#[error("no secret key with access key")]
NoSecretKeyWithAccessKey,
@@ -128,9 +108,8 @@ impl PartialEq for Error {
(Error::NoSuchServiceAccount(a), Error::NoSuchServiceAccount(b)) => a == b,
(Error::NoSuchTempAccount(a), Error::NoSuchTempAccount(b)) => a == b,
(Error::NoSuchGroup(a), Error::NoSuchGroup(b)) => a == b,
(Error::InvalidServiceType(a), Error::InvalidServiceType(b)) => a == b,
(Error::Io(a), Error::Io(b)) => a.kind() == b.kind() && a.to_string() == b.to_string(),
// For complex types like PolicyError, CryptoError, JWTError, compare string representations
// For complex types like PolicyError and CryptoError, compare string representations
(a, b) => std::mem::discriminant(a) == std::mem::discriminant(b) && a.to_string() == b.to_string(),
}
}
@@ -139,9 +118,9 @@ impl PartialEq for Error {
impl Clone for Error {
fn clone(&self) -> Self {
match self {
Error::PolicyError(e) => Error::StringError(e.to_string()), // Convert to string since PolicyError may not be cloneable
Error::PolicyError(e) => Error::PolicyError(Arc::clone(e)),
Error::StringError(s) => Error::StringError(s.clone()),
Error::CryptoError(e) => Error::StringError(format!("crypto: {e}")), // Convert to string
Error::CryptoError(e) => Error::CryptoError(Arc::clone(e)),
Error::NoSuchUser(s) => Error::NoSuchUser(s.clone()),
Error::NoSuchAccount(s) => Error::NoSuchAccount(s.clone()),
Error::NoSuchServiceAccount(s) => Error::NoSuchServiceAccount(s.clone()),
@@ -152,20 +131,12 @@ impl Clone for Error {
Error::GroupNotEmpty => Error::GroupNotEmpty,
Error::InvalidArgument => Error::InvalidArgument,
Error::IamSysNotInitialized => Error::IamSysNotInitialized,
Error::InvalidServiceType(s) => Error::InvalidServiceType(s.clone()),
Error::ErrCredMalformed => Error::ErrCredMalformed,
Error::CredNotInitialized => Error::CredNotInitialized,
Error::InvalidAccessKeyLength => Error::InvalidAccessKeyLength,
Error::InvalidSecretKeyLength => Error::InvalidSecretKeyLength,
Error::ContainsReservedChars => Error::ContainsReservedChars,
Error::GroupNameContainsReservedChars => Error::GroupNameContainsReservedChars,
Error::JWTError(e) => Error::StringError(format!("jwt err {e}")), // Convert to string
Error::NoAccessKey => Error::NoAccessKey,
Error::InvalidToken => Error::InvalidToken,
Error::InvalidAccessKey => Error::InvalidAccessKey,
Error::AccessKeyAlreadyExists => Error::AccessKeyAlreadyExists,
Error::IAMActionNotAllowed => Error::IAMActionNotAllowed,
Error::InvalidExpiration => Error::InvalidExpiration,
Error::NoSecretKeyWithAccessKey => Error::NoSecretKeyWithAccessKey,
Error::NoAccessKeyWithSecretKey => Error::NoAccessKeyWithSecretKey,
Error::PolicyTooLarge => Error::PolicyTooLarge,
@@ -176,6 +147,18 @@ impl Clone for Error {
}
}
impl From<PolicyError> for Error {
fn from(e: PolicyError) -> Self {
Error::PolicyError(Arc::new(e))
}
}
impl From<rustfs_crypto::Error> for Error {
fn from(e: rustfs_crypto::Error) -> Self {
Error::CryptoError(Arc::new(e))
}
}
impl Error {
pub fn other<E>(error: E) -> Self
where
@@ -208,16 +191,10 @@ impl From<rustfs_policy::error::Error> for Error {
match e {
rustfs_policy::error::Error::PolicyTooLarge => Error::PolicyTooLarge,
rustfs_policy::error::Error::InvalidArgument => Error::InvalidArgument,
rustfs_policy::error::Error::InvalidServiceType(s) => Error::InvalidServiceType(s),
rustfs_policy::error::Error::IAMActionNotAllowed => Error::IAMActionNotAllowed,
rustfs_policy::error::Error::InvalidExpiration => Error::InvalidExpiration,
rustfs_policy::error::Error::NoAccessKey => Error::NoAccessKey,
rustfs_policy::error::Error::InvalidToken => Error::InvalidToken,
rustfs_policy::error::Error::InvalidAccessKey => Error::InvalidAccessKey,
rustfs_policy::error::Error::NoSecretKeyWithAccessKey => Error::NoSecretKeyWithAccessKey,
rustfs_policy::error::Error::NoAccessKeyWithSecretKey => Error::NoAccessKeyWithSecretKey,
rustfs_policy::error::Error::Io(e) => Error::Io(e),
rustfs_policy::error::Error::JWTError(e) => Error::JWTError(e),
rustfs_policy::error::Error::NoSuchUser(s) => Error::NoSuchUser(s),
rustfs_policy::error::Error::NoSuchAccount(s) => Error::NoSuchAccount(s),
rustfs_policy::error::Error::NoSuchServiceAccount(s) => Error::NoSuchServiceAccount(s),
@@ -230,13 +207,22 @@ impl From<rustfs_policy::error::Error> for Error {
rustfs_policy::error::Error::InvalidSecretKeyLength => Error::InvalidSecretKeyLength,
rustfs_policy::error::Error::ContainsReservedChars => Error::ContainsReservedChars,
rustfs_policy::error::Error::GroupNameContainsReservedChars => Error::GroupNameContainsReservedChars,
rustfs_policy::error::Error::CredNotInitialized => Error::CredNotInitialized,
rustfs_policy::error::Error::IamSysNotInitialized => Error::IamSysNotInitialized,
rustfs_policy::error::Error::PolicyError(e) => Error::PolicyError(e),
rustfs_policy::error::Error::PolicyError(e) => Error::PolicyError(Arc::new(e)),
rustfs_policy::error::Error::StringError(s) => Error::StringError(s),
rustfs_policy::error::Error::CryptoError(e) => Error::CryptoError(e),
rustfs_policy::error::Error::ErrCredMalformed => Error::ErrCredMalformed,
rustfs_policy::error::Error::CryptoError(e) => Error::CryptoError(Arc::new(e)),
rustfs_policy::error::Error::IamSysAlreadyInitialized => Error::IamSysAlreadyInitialized,
// These policy variants had dead same-name twins on iam::Error (zero
// construction and zero match sites, removed in backlog#1831); the
// message is preserved through StringError instead.
err @ (rustfs_policy::error::Error::InvalidServiceType(_)
| rustfs_policy::error::Error::InvalidExpiration
| rustfs_policy::error::Error::NoAccessKey
| rustfs_policy::error::Error::InvalidToken
| rustfs_policy::error::Error::InvalidAccessKey
| rustfs_policy::error::Error::JWTError(_)
| rustfs_policy::error::Error::CredNotInitialized
| rustfs_policy::error::Error::ErrCredMalformed) => Error::StringError(err.to_string()),
}
}
}
@@ -415,6 +401,31 @@ mod tests {
assert!(converted_io.to_string().contains("access denied"));
}
#[test]
fn clone_preserves_variant_identity_and_message() {
// backlog#1831 PR2: cloning must never demote a variant to a different
// one (the old Clone stringified PolicyError/CryptoError into
// StringError). Pin discriminant and rendered message across clone.
let errors = vec![
Error::PolicyError(Arc::new(PolicyError::NonAction)),
Error::CryptoError(Arc::new(rustfs_crypto::Error::ErrInvalidKeyLength)),
Error::Io(std::io::Error::other("io payload")),
Error::StringError("plain".to_string()),
Error::NoSuchUser("u".to_string()),
Error::ConfigNotFound,
];
for error in errors {
let cloned = error.clone();
assert_eq!(
std::mem::discriminant(&error),
std::mem::discriminant(&cloned),
"clone must keep the variant of {error:?}"
);
assert_eq!(error.to_string(), cloned.to_string(), "clone must keep the rendered message");
}
}
#[test]
fn test_error_display_format() {
let test_cases = vec![
+7
View File
@@ -5,6 +5,13 @@ All notable changes to the rustfs-io-core and rustfs-io-metrics crates will be d
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Removed
#### rustfs-io-metrics
- **Unified configuration** (added in 0.0.5): the zero-consumer `IoConfig`, `CacheSettings`, `IoSchedulerSettings`, `BackpressureSettings`, `TimeoutSettings`, `DeadlockDetectionSettings` types and their `DEFAULT_*` constants were removed (rustfs/rustfs#6008); rustfs-io-core's `IoSchedulerConfig`/`BackpressureConfig` remain the canonical configuration types.
## [0.0.5] - 2025-01-XX
### Added
-8
View File
@@ -33,31 +33,23 @@ default = []
hotpath = [
"hotpath/hotpath",
"hotpath/tokio",
"rustfs-common/hotpath",
"rustfs-s3-ops/hotpath",
"rustfs-utils/hotpath",
]
hotpath-alloc = [
"hotpath",
"hotpath/hotpath-alloc",
"rustfs-common/hotpath-alloc",
"rustfs-s3-ops/hotpath-alloc",
"rustfs-utils/hotpath-alloc",
]
hotpath-cpu = [
"hotpath",
"hotpath/hotpath-cpu",
"rustfs-common/hotpath-cpu",
"rustfs-s3-ops/hotpath-cpu",
"rustfs-utils/hotpath-cpu",
]
[dependencies]
hotpath.workspace = true
metrics = { workspace = true }
rustfs-common = { workspace = true }
rustfs-s3-ops = { workspace = true }
rustfs-utils = { workspace = true, features = ["ip"] }
num_cpus = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["sync", "fs", "rt-multi-thread"] }
-27
View File
@@ -27,7 +27,6 @@
- **Metrics Collection**: Unified metrics recording and reporting
- **Bandwidth Monitoring**: Real-time bandwidth observation and analysis
- **Performance Metrics**: I/O performance metrics collection
- **Unified Configuration**: Centralized configuration management
- **Exporter Boundary**: Emit via `metrics`, export via `rustfs-obs`, no Prometheus HTTP endpoint
## Features
@@ -203,30 +202,6 @@ path and include:
deltas with `operation` and `backend` columns, so the TCP baseline can attribute
bytes and request/error counts to `tcp-http` transport operations.
### Unified Configuration
Centralized configuration management:
```rust
use rustfs_io_metrics::{
IoConfig, CacheSettings, IoSchedulerSettings,
BackpressureSettings, TimeoutSettings,
};
let config = IoConfig::new()
.with_cache(CacheSettings::new()
.with_max_capacity(10_000)
.with_ttl(std::time::Duration::from_secs(300)))
.with_scheduler(IoSchedulerSettings::new()
.with_max_concurrent_reads(64))
.with_backpressure(BackpressureSettings::new())
.with_timeout(TimeoutSettings::new());
// Access configuration
println!("Cache capacity: {}", config.cache.max_capacity);
println!("Max concurrent reads: {}", config.scheduler.max_concurrent_reads);
```
## Module Structure
```
@@ -235,7 +210,6 @@ rustfs-io-metrics/
│ ├── lib.rs # Module entry
│ ├── cache_config.rs # Cache configuration
│ ├── adaptive_ttl.rs # Adaptive TTL
│ ├── config.rs # Unified configuration
│ ├── io_metrics.rs # I/O metrics
│ ├── backpressure_metrics.rs # Backpressure metrics
│ ├── deadlock_metrics.rs # Deadlock metrics
@@ -278,7 +252,6 @@ Useful source references:
- [Crate API overview](./src/lib.rs)
- [Metrics example](./examples/metrics_example.rs)
- [Configuration module](./src/config.rs)
- [Adaptive TTL module](./src/adaptive_ttl.rs)
## Related Modules
-42
View File
@@ -27,7 +27,6 @@
- **指标收集**:统一的指标记录和上报
- **带宽监控**:实时带宽观测和分析
- **性能指标**I/O 性能指标收集
- **统一配置**:集中式配置管理
- **导出边界**:通过 `metrics` 主动上报,由 `rustfs-obs` 负责 OTEL 导出,不提供 Prometheus HTTP 端点
## ✨ 核心功能
@@ -172,30 +171,6 @@ println!("读取速率: {} bytes/s", snapshot.read_bytes_per_sec);
println!("写入速率: {} bytes/s", snapshot.write_bytes_per_sec);
```
### 统一配置 (IoConfig)
集中式配置管理:
```rust
use rustfs_io_metrics::{
IoConfig, CacheSettings, IoSchedulerSettings,
BackpressureSettings, TimeoutSettings,
};
let config = IoConfig::new()
.with_cache(CacheSettings::new()
.with_max_capacity(10_000)
.with_ttl(std::time::Duration::from_secs(300)))
.with_scheduler(IoSchedulerSettings::new()
.with_max_concurrent_reads(64))
.with_backpressure(BackpressureSettings::new())
.with_timeout(TimeoutSettings::new());
// 访问配置
println!("缓存容量: {}", config.cache.max_capacity);
println!("最大并发读: {}", config.scheduler.max_concurrent_reads);
```
## 📊 指标类型
### I/O 调度指标
@@ -233,21 +208,6 @@ println!("最大并发读: {}", config.scheduler.max_concurrent_reads);
| `operation_duration_secs` | 操作时长 | Histogram |
| `operation_progress` | 操作进度 | Gauge |
## 🔧 配置
### 代码配置
```rust
use rustfs_io_metrics::{CacheSettings, IoConfig};
let settings = CacheSettings::new()
.with_max_capacity(5000)
.with_ttl(std::time::Duration::from_secs(600))
.with_max_memory(200 * 1024 * 1024);
let config = IoConfig::new().with_cache(settings);
```
## 📁 模块结构
```
@@ -256,7 +216,6 @@ rustfs-io-metrics/
│ ├── lib.rs # 模块入口
│ ├── cache_config.rs # 缓存配置
│ ├── adaptive_ttl.rs # 自适应 TTL
│ ├── config.rs # 统一配置
│ ├── io_metrics.rs # I/O 指标
│ ├── backpressure_metrics.rs # 背压指标
│ ├── deadlock_metrics.rs # 死锁指标
@@ -297,7 +256,6 @@ cargo doc --package rustfs-io-metrics --no-deps --open
- [Crate API 概览](./src/lib.rs)
- [指标示例](./examples/metrics_example.rs)
- [配置模块](./src/config.rs)
- [自适应 TTL 模块](./src/adaptive_ttl.rs)
## 🔗 相关模块
+2 -27
View File
@@ -14,9 +14,7 @@
//! Example demonstrating metrics and configuration usage.
use rustfs_io_metrics::{
AccessTracker, AdaptiveTTL, CacheConfig, CacheSettings, IoConfig, IoSchedulerSettings, record_cache_size,
};
use rustfs_io_metrics::{AccessTracker, AdaptiveTTL, CacheConfig, record_cache_size};
use std::time::Duration;
fn main() {
@@ -31,10 +29,7 @@ fn main() {
// 3. Access tracking example
access_tracker_example();
// 4. Unified configuration example
unified_config_example();
// 5. Metrics recording example
// 4. Metrics recording example
metrics_recording_example();
}
@@ -109,26 +104,6 @@ fn access_tracker_example() {
println!();
}
fn unified_config_example() {
println!("--- Unified Configuration ---");
let config = IoConfig::new()
.with_cache(
CacheSettings::new()
.with_max_capacity(5000)
.with_ttl(Duration::from_secs(600)),
)
.with_scheduler(IoSchedulerSettings::new().with_max_concurrent_reads(64));
println!(" Cache capacity: {}", config.cache.max_capacity);
println!(" Cache TTL: {:?}", config.cache.default_ttl);
println!(" Max concurrent reads: {}", config.scheduler.max_concurrent_reads);
println!(" Backpressure high watermark: {}", config.backpressure.high_watermark);
println!(" Default timeout: {:?}", config.timeout.default_timeout);
println!();
}
fn metrics_recording_example() {
println!("--- Metrics Recording ---");
+38 -24
View File
@@ -315,6 +315,44 @@ impl Default for AccessTracker {
mod tests {
use super::*;
/// Replaces the per-helper smoke tests that called the record_* helpers
/// and asserted nothing: the calls (same literals) now run against a local
/// DebuggingRecorder and every metric name the helpers own must actually
/// be emitted (rustfs/backlog#1836 PR3).
#[test]
fn record_helpers_emit_their_metrics() {
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
record_ttl_adjustment("test-key", 100, 150);
record_ttl_adjustment("test-key", 100, 50);
record_ttl_expiration();
record_early_eviction("cold");
record_early_eviction("low_priority");
record_access_pattern_change("sequential", "random");
record_access_pattern_change("random", "sequential");
});
let emitted: std::collections::HashSet<String> = snapshotter
.snapshot()
.into_vec()
.into_iter()
.map(|(composite, _, _, _)| composite.key().name().to_string())
.collect();
for expected in [
"rustfs_cache_ttl_adjustments",
"rustfs_cache_ttl_base",
"rustfs_cache_ttl_adjusted",
"rustfs_cache_ttl_extensions",
"rustfs_cache_ttl_reductions",
"rustfs_cache_ttl_expirations",
"rustfs_cache_evictions_early",
"rustfs_cache_access_pattern_changes",
] {
assert!(emitted.contains(expected), "{expected} must be emitted by its record helper");
}
}
#[test]
fn test_adaptive_ttl_stats() {
let mut stats = AdaptiveTTLStats::new();
@@ -335,30 +373,6 @@ mod tests {
assert!((stats.reduction_rate() - 0.3333333333333333).abs() < 0.01);
}
#[test]
fn test_record_ttl_adjustment() {
// This test verifies the function compiles and runs
record_ttl_adjustment("test-key", 100, 150);
record_ttl_adjustment("test-key", 100, 50);
}
#[test]
fn test_record_ttl_expiration() {
record_ttl_expiration();
}
#[test]
fn test_record_early_eviction() {
record_early_eviction("cold");
record_early_eviction("low_priority");
}
#[test]
fn test_record_access_pattern_change() {
record_access_pattern_change("sequential", "random");
record_access_pattern_change("random", "sequential");
}
#[test]
fn test_access_record() {
let mut record = AccessRecord::new();
+31 -23
View File
@@ -53,30 +53,38 @@ pub fn record_backpressure_deactivation() {
mod tests {
use super::*;
/// Replaces the per-helper smoke tests that called the record_* helpers
/// and asserted nothing: the calls (same literals) now run against a local
/// DebuggingRecorder and every metric name the helpers own must actually
/// be emitted (rustfs/backlog#1836 PR3).
#[test]
fn test_record_backpressure_state_change() {
record_backpressure_state_change("normal", "warning");
record_backpressure_state_change("warning", "critical");
}
fn record_helpers_emit_their_metrics() {
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
record_backpressure_state_change("normal", "warning");
record_backpressure_state_change("warning", "critical");
record_backpressure_rejection();
record_concurrent_operations(10);
record_concurrent_operations(32);
record_backpressure_activation();
record_backpressure_deactivation();
});
#[test]
fn test_record_backpressure_rejection() {
record_backpressure_rejection();
}
#[test]
fn test_record_concurrent_operations() {
record_concurrent_operations(10);
record_concurrent_operations(32);
}
#[test]
fn test_record_backpressure_activation() {
record_backpressure_activation();
}
#[test]
fn test_record_backpressure_deactivation() {
record_backpressure_deactivation();
let emitted: std::collections::HashSet<String> = snapshotter
.snapshot()
.into_vec()
.into_iter()
.map(|(composite, _, _, _)| composite.key().name().to_string())
.collect();
for expected in [
"rustfs_backpressure_state_changes",
"rustfs_backpressure_rejections",
"rustfs_backpressure_concurrent",
"rustfs_backpressure_activations",
"rustfs_backpressure_deactivations",
] {
assert!(emitted.contains(expected), "{expected} must be emitted by its record helper");
}
}
}
-391
View File
@@ -1,391 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Unified configuration interface for I/O operations.
//!
//! This module provides a centralized configuration interface
//! for all I/O-related settings.
use std::time::Duration;
// ============================================================================
// Configuration Constants
// ============================================================================
/// Default cache max capacity.
pub const DEFAULT_CACHE_MAX_CAPACITY: u64 = 10_000;
/// Default cache TTL in seconds.
pub const DEFAULT_CACHE_TTL_SECS: u64 = 300;
/// Default cache max memory in bytes (100 MB).
pub const DEFAULT_CACHE_MAX_MEMORY: u64 = 100 * 1024 * 1024;
/// Default I/O scheduler max concurrent reads.
pub const DEFAULT_MAX_CONCURRENT_READS: usize = 32;
/// Default high priority size threshold (64 KB).
pub const DEFAULT_HIGH_PRIORITY_SIZE_THRESHOLD: usize = 64 * 1024;
/// Default low priority size threshold (4 MB).
pub const DEFAULT_LOW_PRIORITY_SIZE_THRESHOLD: usize = 4 * 1024 * 1024;
/// Default backpressure high watermark.
pub const DEFAULT_BACKPRESSURE_HIGH_WATERMARK: f64 = 0.8;
/// Default backpressure low watermark.
pub const DEFAULT_BACKPRESSURE_LOW_WATERMARK: f64 = 0.5;
/// Default lock acquire timeout in seconds.
pub const DEFAULT_LOCK_ACQUIRE_TIMEOUT_SECS: u64 = 5;
/// Default deadlock detection interval in seconds.
pub const DEFAULT_DEADLOCK_DETECTION_INTERVAL_SECS: u64 = 1;
/// Default base buffer size (128 KB).
pub const DEFAULT_BASE_BUFFER_SIZE: usize = 128 * 1024;
/// Default max buffer size (1 MB).
pub const DEFAULT_MAX_BUFFER_SIZE: usize = 1024 * 1024;
/// Default min buffer size (4 KB).
pub const DEFAULT_MIN_BUFFER_SIZE: usize = 4 * 1024;
// ============================================================================
// Cache Configuration
// ============================================================================
/// Cache configuration settings.
#[derive(Debug, Clone)]
pub struct CacheSettings {
/// Maximum cache capacity.
pub max_capacity: u64,
/// Default TTL.
pub default_ttl: Duration,
/// Maximum memory usage.
pub max_memory: u64,
/// Whether adaptive TTL is enabled.
pub adaptive_ttl_enabled: bool,
}
impl Default for CacheSettings {
fn default() -> Self {
Self {
max_capacity: DEFAULT_CACHE_MAX_CAPACITY,
default_ttl: Duration::from_secs(DEFAULT_CACHE_TTL_SECS),
max_memory: DEFAULT_CACHE_MAX_MEMORY,
adaptive_ttl_enabled: true,
}
}
}
impl CacheSettings {
/// Create new cache settings.
pub fn new() -> Self {
Self::default()
}
/// Builder: set max capacity.
pub fn with_max_capacity(mut self, capacity: u64) -> Self {
self.max_capacity = capacity;
self
}
/// Builder: set TTL.
pub fn with_ttl(mut self, ttl: Duration) -> Self {
self.default_ttl = ttl;
self
}
/// Builder: set max memory.
pub fn with_max_memory(mut self, memory: u64) -> Self {
self.max_memory = memory;
self
}
}
// ============================================================================
// I/O Scheduler Configuration
// ============================================================================
/// I/O scheduler configuration settings.
#[derive(Debug, Clone)]
pub struct IoSchedulerSettings {
/// Maximum concurrent reads.
pub max_concurrent_reads: usize,
/// High priority size threshold.
pub high_priority_threshold: usize,
/// Low priority size threshold.
pub low_priority_threshold: usize,
/// Base buffer size.
pub base_buffer_size: usize,
/// Max buffer size.
pub max_buffer_size: usize,
/// Min buffer size.
pub min_buffer_size: usize,
/// Whether priority scheduling is enabled.
pub priority_enabled: bool,
}
impl Default for IoSchedulerSettings {
fn default() -> Self {
Self {
max_concurrent_reads: DEFAULT_MAX_CONCURRENT_READS,
high_priority_threshold: DEFAULT_HIGH_PRIORITY_SIZE_THRESHOLD,
low_priority_threshold: DEFAULT_LOW_PRIORITY_SIZE_THRESHOLD,
base_buffer_size: DEFAULT_BASE_BUFFER_SIZE,
max_buffer_size: DEFAULT_MAX_BUFFER_SIZE,
min_buffer_size: DEFAULT_MIN_BUFFER_SIZE,
priority_enabled: true,
}
}
}
impl IoSchedulerSettings {
/// Create new settings.
pub fn new() -> Self {
Self::default()
}
/// Builder: set max concurrent reads.
pub fn with_max_concurrent_reads(mut self, max: usize) -> Self {
self.max_concurrent_reads = max;
self
}
/// Builder: set buffer sizes.
pub fn with_buffer_sizes(mut self, base: usize, min: usize, max: usize) -> Self {
self.base_buffer_size = base;
self.min_buffer_size = min;
self.max_buffer_size = max;
self
}
}
// ============================================================================
// Backpressure Configuration
// ============================================================================
/// Backpressure configuration settings.
#[derive(Debug, Clone)]
pub struct BackpressureSettings {
/// Whether backpressure is enabled.
pub enabled: bool,
/// High watermark (percentage).
pub high_watermark: f64,
/// Low watermark (percentage).
pub low_watermark: f64,
/// Cooldown duration.
pub cooldown: Duration,
}
impl Default for BackpressureSettings {
fn default() -> Self {
Self {
enabled: true,
high_watermark: DEFAULT_BACKPRESSURE_HIGH_WATERMARK,
low_watermark: DEFAULT_BACKPRESSURE_LOW_WATERMARK,
cooldown: Duration::from_millis(100),
}
}
}
impl BackpressureSettings {
/// Create new settings.
pub fn new() -> Self {
Self::default()
}
/// Get high watermark threshold for a given max value.
pub fn high_threshold(&self, max: usize) -> usize {
(max as f64 * self.high_watermark) as usize
}
/// Get low watermark threshold for a given max value.
pub fn low_threshold(&self, max: usize) -> usize {
(max as f64 * self.low_watermark) as usize
}
}
// ============================================================================
// Timeout Configuration
// ============================================================================
/// Timeout configuration settings.
#[derive(Debug, Clone)]
pub struct TimeoutSettings {
/// Default operation timeout.
pub default_timeout: Duration,
/// Maximum retries.
pub max_retries: usize,
/// Retry backoff factor.
pub retry_backoff_factor: f64,
/// Lock acquire timeout.
pub lock_acquire_timeout: Duration,
}
impl Default for TimeoutSettings {
fn default() -> Self {
Self {
default_timeout: Duration::from_secs(30),
max_retries: 3,
retry_backoff_factor: 2.0,
lock_acquire_timeout: Duration::from_secs(DEFAULT_LOCK_ACQUIRE_TIMEOUT_SECS),
}
}
}
impl TimeoutSettings {
/// Create new settings.
pub fn new() -> Self {
Self::default()
}
/// Calculate timeout with backoff for a given retry count.
pub fn timeout_with_backoff(&self, retry_count: usize) -> Duration {
let multiplier = self.retry_backoff_factor.powi(retry_count as i32);
Duration::from_secs_f64(self.default_timeout.as_secs_f64() * multiplier)
}
}
// ============================================================================
// Deadlock Detection Configuration
// ============================================================================
/// Deadlock detection configuration settings.
#[derive(Debug, Clone)]
pub struct DeadlockDetectionSettings {
/// Whether detection is enabled.
pub enabled: bool,
/// Detection interval.
pub detection_interval: Duration,
/// Maximum lock hold time before warning.
pub max_hold_time: Duration,
}
impl Default for DeadlockDetectionSettings {
fn default() -> Self {
Self {
enabled: true,
detection_interval: Duration::from_secs(DEFAULT_DEADLOCK_DETECTION_INTERVAL_SECS),
max_hold_time: Duration::from_secs(30),
}
}
}
impl DeadlockDetectionSettings {
/// Create new settings.
pub fn new() -> Self {
Self::default()
}
}
// ============================================================================
// Unified Configuration
// ============================================================================
/// Unified configuration for all I/O operations.
#[derive(Debug, Clone, Default)]
pub struct IoConfig {
/// Cache settings.
pub cache: CacheSettings,
/// I/O scheduler settings.
pub scheduler: IoSchedulerSettings,
/// Backpressure settings.
pub backpressure: BackpressureSettings,
/// Timeout settings.
pub timeout: TimeoutSettings,
/// Deadlock detection settings.
pub deadlock_detection: DeadlockDetectionSettings,
}
impl IoConfig {
/// Create new unified configuration.
pub fn new() -> Self {
Self::default()
}
/// Builder: set cache settings.
pub fn with_cache(mut self, cache: CacheSettings) -> Self {
self.cache = cache;
self
}
/// Builder: set scheduler settings.
pub fn with_scheduler(mut self, scheduler: IoSchedulerSettings) -> Self {
self.scheduler = scheduler;
self
}
/// Builder: set backpressure settings.
pub fn with_backpressure(mut self, backpressure: BackpressureSettings) -> Self {
self.backpressure = backpressure;
self
}
/// Builder: set timeout settings.
pub fn with_timeout(mut self, timeout: TimeoutSettings) -> Self {
self.timeout = timeout;
self
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_cache_settings() {
let settings = CacheSettings::new()
.with_max_capacity(5000)
.with_ttl(Duration::from_secs(600));
assert_eq!(settings.max_capacity, 5000);
assert_eq!(settings.default_ttl, Duration::from_secs(600));
}
#[test]
fn test_io_scheduler_settings() {
let settings =
IoSchedulerSettings::new()
.with_max_concurrent_reads(64)
.with_buffer_sizes(256 * 1024, 8 * 1024, 2 * 1024 * 1024);
assert_eq!(settings.max_concurrent_reads, 64);
assert_eq!(settings.base_buffer_size, 256 * 1024);
}
#[test]
fn test_backpressure_settings() {
let settings = BackpressureSettings::new();
assert_eq!(settings.high_threshold(100), 80);
assert_eq!(settings.low_threshold(100), 50);
}
#[test]
fn test_timeout_settings() {
let settings = TimeoutSettings::new();
// First retry: 30s * 2 = 60s
let timeout1 = settings.timeout_with_backoff(1);
assert!(timeout1.as_secs() >= 60);
// Second retry: 30s * 4 = 120s
let timeout2 = settings.timeout_with_backoff(2);
assert!(timeout2.as_secs() >= 120);
}
#[test]
fn test_unified_config() {
let config = IoConfig::new()
.with_cache(CacheSettings::new().with_max_capacity(5000))
.with_scheduler(IoSchedulerSettings::new().with_max_concurrent_reads(64));
assert_eq!(config.cache.max_capacity, 5000);
assert_eq!(config.scheduler.max_concurrent_reads, 64);
}
}
+41 -32
View File
@@ -72,39 +72,48 @@ pub fn record_wait_edge_removed() {
mod tests {
use super::*;
/// Replaces the per-helper smoke tests that called the record_* helpers
/// and asserted nothing: the calls (same literals) now run against a local
/// DebuggingRecorder and every metric name the helpers own must actually
/// be emitted (rustfs/backlog#1836 PR3).
#[test]
fn test_record_deadlock_detected() {
record_deadlock_detected(3);
record_deadlock_detected(5);
}
fn record_helpers_emit_their_metrics() {
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
record_deadlock_detected(3);
record_deadlock_detected(5);
record_long_held_lock(1, Duration::from_secs(30));
record_long_held_lock(2, Duration::from_secs(60));
record_lock_acquisition("mutex");
record_lock_acquisition("rwlock");
record_lock_release("mutex", Duration::from_millis(10));
record_lock_release("rwlock", Duration::from_millis(5));
record_lock_contention("mutex");
record_lock_contention("rwlock");
record_wait_edge_added();
record_wait_edge_removed();
});
#[test]
fn test_record_long_held_lock() {
record_long_held_lock(1, Duration::from_secs(30));
record_long_held_lock(2, Duration::from_secs(60));
}
#[test]
fn test_record_lock_acquisition() {
record_lock_acquisition("mutex");
record_lock_acquisition("rwlock");
}
#[test]
fn test_record_lock_release() {
record_lock_release("mutex", Duration::from_millis(10));
record_lock_release("rwlock", Duration::from_millis(5));
}
#[test]
fn test_record_lock_contention() {
record_lock_contention("mutex");
record_lock_contention("rwlock");
}
#[test]
fn test_record_wait_edge() {
record_wait_edge_added();
record_wait_edge_removed();
let emitted: std::collections::HashSet<String> = snapshotter
.snapshot()
.into_vec()
.into_iter()
.map(|(composite, _, _, _)| composite.key().name().to_string())
.collect();
for expected in [
"rustfs_deadlock_detected_total",
"rustfs_deadlock_cycle_length",
"rustfs_deadlock_long_held",
"rustfs_deadlock_hold_time_secs",
"rustfs_lock_acquisitions",
"rustfs_lock_releases",
"rustfs_lock_hold_time_secs",
"rustfs_lock_contentions",
"rustfs_deadlock_wait_edges_added",
"rustfs_deadlock_wait_edges_removed",
] {
assert!(emitted.contains(expected), "{expected} must be emitted by its record helper");
}
}
}
+12 -15
View File
@@ -196,22 +196,19 @@ pub const INTERNODE_OPERATION_METRICS: &[InternodeOperationMetricDescriptor] = &
},
];
static STABLE_SERVER_LABEL: OnceLock<String> = OnceLock::new();
/// Injects the stable server label (node name or address) stamped on
/// internode metrics. The runtime calls this when the local node name is
/// published (see ecstore's `set_local_node_name`); the first write wins.
/// io-metrics is a leaf crate and no longer resolves node identity itself
/// (backlog#1834) — before injection the label reads "unset".
pub fn set_internode_server_label(label: impl Into<String>) {
let _ = STABLE_SERVER_LABEL.set(label.into());
}
fn current_server_label() -> &'static str {
static STABLE_SERVER_LABEL: OnceLock<String> = OnceLock::new();
static FALLBACK_SERVER_LABEL: LazyLock<String> = LazyLock::new(rustfs_utils::get_local_ip_with_default);
if let Some(server) = STABLE_SERVER_LABEL.get() {
return server.as_str();
}
if let Some(server) = rustfs_common::try_get_global_local_node_name() {
let _ = STABLE_SERVER_LABEL.set(server);
if let Some(server) = STABLE_SERVER_LABEL.get() {
return server.as_str();
}
}
FALLBACK_SERVER_LABEL.as_str()
STABLE_SERVER_LABEL.get().map(String::as_str).unwrap_or("unset")
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
+50 -38
View File
@@ -169,46 +169,58 @@ impl IoSchedulerStats {
mod tests {
use super::*;
/// Replaces the per-helper smoke tests that called the record_* helpers
/// and asserted nothing: the calls (same literals) now run against a local
/// DebuggingRecorder and every metric name the helpers own must actually
/// be emitted (rustfs/backlog#1836 PR3).
#[test]
fn test_record_io_scheduler_decision() {
record_io_scheduler_decision(128 * 1024, "low", "sequential");
record_io_scheduler_decision(64 * 1024, "high", "random");
}
fn record_helpers_emit_their_metrics() {
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
record_io_scheduler_decision(128 * 1024, "low", "sequential");
record_io_scheduler_decision(64 * 1024, "high", "random");
record_io_priority_decision("high", 1024);
record_io_priority_decision("normal", 1024 * 1024);
record_io_priority_decision("low", 10 * 1024 * 1024);
record_load_level_change("low", "medium");
record_load_level_change("medium", "high");
record_bandwidth_observation(100 * 1024 * 1024);
record_bandwidth_observation(500 * 1024 * 1024);
record_buffer_size_adjustment(128 * 1024, 64 * 1024, "concurrency");
record_buffer_size_adjustment(128 * 1024, 256 * 1024, "sequential");
record_queue_operation("enqueue", "high", 10);
record_queue_operation("dequeue", "high", 9);
record_starvation_event("low");
});
#[test]
fn test_record_io_priority_decision() {
record_io_priority_decision("high", 1024);
record_io_priority_decision("normal", 1024 * 1024);
record_io_priority_decision("low", 10 * 1024 * 1024);
}
#[test]
fn test_record_load_level_change() {
record_load_level_change("low", "medium");
record_load_level_change("medium", "high");
}
#[test]
fn test_record_bandwidth_observation() {
record_bandwidth_observation(100 * 1024 * 1024);
record_bandwidth_observation(500 * 1024 * 1024);
}
#[test]
fn test_record_buffer_size_adjustment() {
record_buffer_size_adjustment(128 * 1024, 64 * 1024, "concurrency");
record_buffer_size_adjustment(128 * 1024, 256 * 1024, "sequential");
}
#[test]
fn test_record_queue_operation() {
record_queue_operation("enqueue", "high", 10);
record_queue_operation("dequeue", "high", 9);
}
#[test]
fn test_record_starvation_event() {
record_starvation_event("low");
let emitted: std::collections::HashSet<String> = snapshotter
.snapshot()
.into_vec()
.into_iter()
.map(|(composite, _, _, _)| composite.key().name().to_string())
.collect();
for expected in [
"rustfs_io_scheduler_decisions",
"rustfs_io_scheduler_buffer_size",
"rustfs_io_scheduler_load",
"rustfs_io_scheduler_strategy",
"rustfs_io_scheduler_buffer_size_histogram",
"rustfs_io_priority_decisions",
"rustfs_io_priority_by_level",
"rustfs_io_priority_request_size",
"rustfs_io_load_changes",
"rustfs_io_bandwidth_bps",
"rustfs_io_bandwidth_histogram",
"rustfs_io_buffer_adjustments",
"rustfs_io_buffer_original",
"rustfs_io_buffer_adjusted",
"rustfs_io_queue_operations",
"rustfs_io_queue_size",
"rustfs_io_starvation_events",
] {
assert!(emitted.contains(expected), "{expected} must be emitted by its record helper");
}
}
#[test]
+112 -13
View File
@@ -58,7 +58,7 @@ use std::sync::{
/// When `false`, `record_put_object_path` and `record_put_object_stage_duration`
/// become no-ops, and callers can skip the `Instant::now()` syscalls entirely.
///
/// Set to `true` during startup when OTEL metric export is enabled.
/// Enabled only through an explicit runtime opt-in.
static PUT_STAGE_METRICS_ENABLED: AtomicBool = AtomicBool::new(false);
static GET_STAGE_METRICS_ENABLED: AtomicBool = AtomicBool::new(false);
@@ -78,7 +78,7 @@ static METRICS_ENABLED: AtomicBool = AtomicBool::new(false);
/// Enable or disable detailed per-stage PUT metrics.
///
/// Called once during startup, typically gated by `rustfs_obs::observability_metric_enabled()`.
/// Called once during startup after applying the detailed PUT attribution opt-in.
pub fn set_put_stage_metrics_enabled(enabled: bool) {
PUT_STAGE_METRICS_ENABLED.store(enabled, Ordering::Relaxed);
}
@@ -103,6 +103,12 @@ pub fn put_stage_metrics_enabled() -> bool {
PUT_STAGE_METRICS_ENABLED.load(Ordering::Relaxed)
}
/// Start a PUT-stage timer only when detailed PUT attribution is enabled.
#[inline(always)]
pub fn put_stage_timer() -> Option<std::time::Instant> {
put_stage_metrics_enabled().then(std::time::Instant::now)
}
#[inline(always)]
pub fn get_stage_metrics_enabled() -> bool {
GET_STAGE_METRICS_ENABLED.load(Ordering::Relaxed)
@@ -173,7 +179,6 @@ pub mod backpressure_metrics;
pub mod cache_config;
pub mod capacity_metrics;
pub mod collector;
pub mod config;
pub mod deadlock_metrics;
pub mod internode_metrics;
pub mod io_metrics;
@@ -260,13 +265,6 @@ pub use timeout_metrics::{
record_operation_progress, record_stalled_operation, record_timeout_event,
};
// Config exports
pub use config::{
BackpressureSettings, CacheSettings, DEFAULT_BASE_BUFFER_SIZE, DEFAULT_CACHE_MAX_CAPACITY, DEFAULT_CACHE_MAX_MEMORY,
DEFAULT_CACHE_TTL_SECS, DEFAULT_MAX_BUFFER_SIZE, DEFAULT_MAX_CONCURRENT_READS, DEFAULT_MIN_BUFFER_SIZE,
DeadlockDetectionSettings, IoConfig, IoSchedulerSettings, TimeoutSettings,
};
// Re-exports for convenience
pub use collector::MetricsCollector;
pub use performance::PerformanceMetrics;
@@ -442,7 +440,7 @@ pub fn record_get_object_request_result(status: &str, duration_secs: f64) {
/// Record PutObject request start.
#[inline(always)]
pub fn record_put_object_request_start(concurrent_requests: usize) {
if !put_stage_metrics_enabled() {
if !metrics_enabled() {
return;
}
counter!("rustfs_io_put_object_requests_total").increment(1);
@@ -452,7 +450,7 @@ pub fn record_put_object_request_start(concurrent_requests: usize) {
/// Record PutObject request result.
#[inline(always)]
pub fn record_put_object_request_result(status: &str, duration_secs: f64) {
if !put_stage_metrics_enabled() {
if !metrics_enabled() {
return;
}
counter!("rustfs_io_put_object_request_results_total", "status" => status.to_string()).increment(1);
@@ -814,6 +812,17 @@ pub fn record_get_object_metadata_fanout_shape(path: &'static str, total: usize,
.record(metadata_fanout_count_to_f64(non_valid));
}
/// Record task lifecycle shape for one GetObject metadata fanout.
#[inline(always)]
pub fn record_get_object_metadata_fanout_lifecycle(path: &'static str, scheduled: usize, completed: usize, cancelled: usize) {
if !get_stage_metrics_enabled() {
return;
}
histogram!("rustfs_io_get_object_metadata_fanout_scheduled", "path" => path).record(metadata_fanout_count_to_f64(scheduled));
histogram!("rustfs_io_get_object_metadata_fanout_completed", "path" => path).record(metadata_fanout_count_to_f64(completed));
histogram!("rustfs_io_get_object_metadata_fanout_cancelled", "path" => path).record(metadata_fanout_count_to_f64(cancelled));
}
/// Record a guarded metadata early-stop hit for GetObject.
#[inline(always)]
pub fn record_get_object_metadata_early_stop_hit(path: &'static str, reason: &'static str) {
@@ -1913,7 +1922,7 @@ pub fn record_get_object(duration_ms: f64, size_bytes: i64) {
/// * `zero_copy_eligible` - Whether the request was eligible for a zero-copy path
#[inline(always)]
pub fn record_put_object(duration_ms: f64, size_bytes: i64, zero_copy_eligible: bool) {
if !put_stage_metrics_enabled() {
if !metrics_enabled() {
return;
}
counter!("rustfs_s3_put_object_total").increment(1);
@@ -2012,6 +2021,13 @@ pub fn record_put_object_stage_duration(stage: &'static str, duration_ms: f64) {
histogram!("rustfs_s3_put_object_stage_duration_ms", "stage" => stage).record(duration_ms);
}
#[inline(always)]
pub fn record_put_object_stage_duration_from(stage: &'static str, started_at: Option<std::time::Instant>) {
if let Some(started_at) = started_at {
record_put_object_stage_duration(stage, started_at.elapsed().as_secs_f64() * 1000.0);
}
}
/// Record generic internal operation stage duration (non-PUT paths).
/// Use this for metacache walks, listing, lifecycle, and other background
/// operations that are NOT part of the PUT object hot path.
@@ -2693,6 +2709,7 @@ mod tests {
record_get_object_quorum_reached_latency("legacy_duplex", 0.002);
record_get_object_metadata_response("legacy_duplex", "valid");
record_get_object_metadata_fanout_shape("legacy_duplex", 4, 3, 1, 1);
record_get_object_metadata_fanout_lifecycle("legacy_duplex", 4, 3, 1);
record_get_object_metadata_early_stop_hit("legacy_duplex", "valid_quorum");
record_get_object_metadata_early_stop_miss("legacy_duplex", "insufficient_quorum");
record_get_object_metadata_early_stop_saved_responses("legacy_duplex", 1);
@@ -2763,6 +2780,38 @@ mod tests {
assert!(remote_scheduled >= remote_avoid_potential);
}
#[test]
fn metadata_fanout_lifecycle_records_named_histograms() {
let _guard = METRICS_FLAG_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let recorder = DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
set_get_stage_metrics_enabled(true);
record_get_object_metadata_fanout_lifecycle("legacy_duplex", 4, 3, 1);
set_get_stage_metrics_enabled(false);
});
let metrics = snapshotter.snapshot().into_vec();
for (name, expected) in [
("rustfs_io_get_object_metadata_fanout_scheduled", 4.0),
("rustfs_io_get_object_metadata_fanout_completed", 3.0),
("rustfs_io_get_object_metadata_fanout_cancelled", 1.0),
] {
let value = metrics.iter().find_map(|(composite, _, _, value)| {
let has_path = composite
.key()
.labels()
.any(|label| label.key() == "path" && label.value() == "legacy_duplex");
(composite.kind() == MetricKind::Histogram && composite.key().name() == name && has_path).then_some(value)
});
assert!(
matches!(value, Some(DebugValue::Histogram(values)) if values.len() == 1 && values[0].0 == expected),
"{name} must record the exact fanout lifecycle sample"
);
}
}
#[test]
fn test_record_get_object_fill_metrics() {
record_get_object_fill_queued("codec_streaming", "single_inflight", 1);
@@ -2827,6 +2876,56 @@ mod tests {
assert!(!put_stage_metrics_enabled());
}
#[test]
fn put_stage_gate_does_not_disable_basic_put_metrics() {
let _guard = METRICS_FLAG_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let recorder = DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
set_metrics_enabled(true);
set_put_stage_metrics_enabled(false);
record_put_object_request_start(1);
record_put_object_request_result("ok", 0.001);
record_put_object(1.0, 1024, false);
record_put_object_stage_duration("disabled_stage", 0.5);
set_put_stage_metrics_enabled(true);
record_put_object_stage_duration("enabled_stage", 0.5);
set_put_stage_metrics_enabled(false);
set_metrics_enabled(false);
});
let metrics = snapshotter.snapshot().into_vec();
assert!(metrics.iter().any(|(composite, _, _, _)| {
composite.kind() == MetricKind::Counter && composite.key().name() == "rustfs_s3_put_object_total"
}));
assert!(metrics.iter().any(|(composite, _, _, _)| {
composite.kind() == MetricKind::Counter && composite.key().name() == "rustfs_io_put_object_requests_total"
}));
let stages = metrics
.iter()
.filter(|(composite, _, _, _)| {
composite.kind() == MetricKind::Histogram && composite.key().name() == "rustfs_s3_put_object_stage_duration_ms"
})
.flat_map(|(composite, _, _, _)| composite.key().labels().map(|label| label.value().to_string()))
.collect::<Vec<_>>();
assert_eq!(stages, ["enabled_stage"]);
}
#[test]
fn test_put_stage_timer_follows_metrics_switch() {
let _guard = METRICS_FLAG_LOCK.lock().unwrap_or_else(|e| e.into_inner());
set_put_stage_metrics_enabled(false);
assert!(put_stage_timer().is_none());
set_put_stage_metrics_enabled(true);
assert!(put_stage_timer().is_some());
set_put_stage_metrics_enabled(false);
}
#[test]
fn test_record_get_object_path_and_stage() {
let _guard = METRICS_FLAG_LOCK.lock().unwrap_or_else(|e| e.into_inner());
+40 -34
View File
@@ -163,6 +163,46 @@ impl LockMetricsSummary {
#[cfg(test)]
mod tests {
use super::*;
/// Replaces the per-helper smoke tests that called the record_* helpers
/// and asserted nothing: the calls (same literals) now run against a local
/// DebuggingRecorder and every metric name the helpers own must actually
/// be emitted (rustfs/backlog#1836 PR3).
#[test]
fn record_helpers_emit_their_metrics() {
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
record_lock_optimization_enabled(true);
record_lock_optimization_enabled(false);
record_spin_attempt(true);
record_spin_attempt(false);
record_spin_count_change(100);
record_spin_count_change(200);
record_lock_hold_time(Duration::from_millis(10));
record_lock_hold_time(Duration::from_millis(100));
record_early_release();
record_contention_event();
});
let emitted: std::collections::HashSet<String> = snapshotter
.snapshot()
.into_vec()
.into_iter()
.map(|(composite, _, _, _)| composite.key().name().to_string())
.collect();
for expected in [
"rustfs_lock_optimization_enabled",
"rustfs_lock_spin_successes",
"rustfs_lock_spin_failures",
"rustfs_lock_spin_count",
"rustfs_lock_hold_time_secs",
"rustfs_lock_early_releases",
"rustfs_lock_contentions",
] {
assert!(emitted.contains(expected), "{expected} must be emitted by its record helper");
}
}
use metrics::{Counter, CounterFn, Gauge, GaugeFn, Histogram, HistogramFn, Key, KeyName, Metadata, SharedString, Unit};
use std::sync::{Arc, Mutex};
@@ -255,40 +295,6 @@ mod tests {
fn record(&self, _value: f64) {}
}
#[test]
fn test_record_lock_optimization_enabled() {
record_lock_optimization_enabled(true);
record_lock_optimization_enabled(false);
}
#[test]
fn test_record_spin_attempt() {
record_spin_attempt(true);
record_spin_attempt(false);
}
#[test]
fn test_record_spin_count_change() {
record_spin_count_change(100);
record_spin_count_change(200);
}
#[test]
fn test_record_lock_hold_time() {
record_lock_hold_time(Duration::from_millis(10));
record_lock_hold_time(Duration::from_millis(100));
}
#[test]
fn test_record_early_release() {
record_early_release();
}
#[test]
fn test_record_contention_event() {
record_contention_event();
}
#[test]
fn test_record_object_lock_diag_enabled() {
let recorder = SeenMetricsRecorder::default();
+38 -31
View File
@@ -114,39 +114,46 @@ impl TimeoutMetricsSummary {
mod tests {
use super::*;
/// Replaces the per-helper smoke tests that called the record_* helpers
/// and asserted nothing: the calls (same literals) now run against a local
/// DebuggingRecorder and every metric name the helpers own must actually
/// be emitted (rustfs/backlog#1836 PR3).
#[test]
fn test_record_timeout_event() {
record_timeout_event("get_object");
record_timeout_event("put_object");
}
fn record_helpers_emit_their_metrics() {
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
record_timeout_event("get_object");
record_timeout_event("put_object");
record_operation_duration("get_object", Duration::from_millis(100));
record_operation_duration("put_object", Duration::from_millis(500));
record_dynamic_timeout(1024 * 1024, Duration::from_secs(10));
record_dynamic_timeout(100 * 1024 * 1024, Duration::from_secs(30));
record_operation_progress("get_object", 50.0);
record_operation_progress("get_object", 100.0);
record_stalled_operation("get_object");
record_operation_completion("get_object", true);
record_operation_completion("get_object", false);
});
#[test]
fn test_record_operation_duration() {
record_operation_duration("get_object", Duration::from_millis(100));
record_operation_duration("put_object", Duration::from_millis(500));
}
#[test]
fn test_record_dynamic_timeout() {
record_dynamic_timeout(1024 * 1024, Duration::from_secs(10));
record_dynamic_timeout(100 * 1024 * 1024, Duration::from_secs(30));
}
#[test]
fn test_record_operation_progress() {
record_operation_progress("get_object", 50.0);
record_operation_progress("get_object", 100.0);
}
#[test]
fn test_record_stalled_operation() {
record_stalled_operation("get_object");
}
#[test]
fn test_record_operation_completion() {
record_operation_completion("get_object", true);
record_operation_completion("get_object", false);
let emitted: std::collections::HashSet<String> = snapshotter
.snapshot()
.into_vec()
.into_iter()
.map(|(composite, _, _, _)| composite.key().name().to_string())
.collect();
for expected in [
"rustfs_io_timeout_events_total",
"rustfs_io_operation_duration_seconds",
"rustfs_timeout_dynamic_size",
"rustfs_timeout_dynamic_secs",
"rustfs_timeout_dynamic_size_histogram",
"rustfs_operation_progress",
"rustfs_operation_stalled",
"rustfs_operation_completions",
] {
assert!(emitted.contains(expected), "{expected} must be emitted by its record helper");
}
}
#[test]
+4
View File
@@ -35,6 +35,10 @@ tokio = { workspace = true, features = ["fs", "io-util", "macros", "rt-multi-thr
uuid = { workspace = true, features = ["serde", "v4", "fast-rng", "macro-diagnostics"] }
jiff = { workspace = true, features = ["serde"] }
serde = { workspace = true, features = ["derive"] }
# Observes fields a persisted-format deserialization ignored, per the
# repository rule that formats too compatibility-bound for
# deny_unknown_fields must at least warn (AGENTS.md).
serde_ignored = { workspace = true }
serde_json = { workspace = true, features = ["raw_value"] }
tracing = { workspace = true }
thiserror = { workspace = true }
+58
View File
@@ -293,6 +293,15 @@ enum StrictVaultAuthMethod {
#[serde(default)]
refresh_safety_window_secs: Option<u64>,
},
Kubernetes {
role: String,
#[serde(default)]
mount: Option<String>,
#[serde(default)]
jwt_path: Option<std::path::PathBuf>,
#[serde(default)]
refresh_safety_window_secs: Option<u64>,
},
TokenFile {
path: std::path::PathBuf,
#[serde(default)]
@@ -319,6 +328,17 @@ impl From<StrictVaultAuthMethod> for VaultAuthMethod {
mount: mount.unwrap_or_else(|| crate::config::DEFAULT_VAULT_APPROLE_MOUNT.to_string()),
refresh_safety_window_secs,
},
StrictVaultAuthMethod::Kubernetes {
role,
mount,
jwt_path,
refresh_safety_window_secs,
} => Self::Kubernetes {
role,
mount: mount.unwrap_or_else(|| crate::config::DEFAULT_VAULT_KUBERNETES_MOUNT.to_string()),
jwt_path: jwt_path.unwrap_or_else(|| std::path::PathBuf::from(crate::config::DEFAULT_VAULT_KUBERNETES_JWT_PATH)),
refresh_safety_window_secs,
},
StrictVaultAuthMethod::TokenFile {
path,
poll_interval_secs,
@@ -499,6 +519,7 @@ impl From<&KmsConfig> for KmsConfigSummary {
auth_method_type: match &vault_config.auth_method {
VaultAuthMethod::Token { .. } => "token".to_string(),
VaultAuthMethod::AppRole { .. } => "approle".to_string(),
VaultAuthMethod::Kubernetes { .. } => "kubernetes".to_string(),
VaultAuthMethod::TokenFile { .. } => "token_file".to_string(),
},
has_stored_credentials: true,
@@ -513,6 +534,7 @@ impl From<&KmsConfig> for KmsConfigSummary {
auth_method_type: match &vault_config.auth_method {
VaultAuthMethod::Token { .. } => "token".to_string(),
VaultAuthMethod::AppRole { .. } => "approle".to_string(),
VaultAuthMethod::Kubernetes { .. } => "kubernetes".to_string(),
VaultAuthMethod::TokenFile { .. } => "token_file".to_string(),
},
has_stored_credentials: true,
@@ -901,6 +923,42 @@ mod tests {
assert!(request.to_kms_config().validate().is_ok());
}
/// The admin API reaches Kubernetes auth with the role alone; the mount and
/// the projected token path fall back to the cluster defaults, so a Tenant
/// manifest carries no credential and no cluster-specific paths.
#[test]
fn test_deserialize_vault_configure_request_accepts_kubernetes_auth() {
let raw = serde_json::json!({
"backend_type": "vault-transit",
"address": "https://vault.example.com:8200",
"mount_path": "rustfs",
"auth_method": { "Kubernetes": { "role": "rustfs" } }
});
let request: ConfigureKmsRequest = serde_json::from_value(raw).expect("kubernetes auth should deserialize");
let config = request.to_kms_config();
config.validate().expect("kubernetes auth must validate");
let vault = config.vault_transit_config().expect("vault transit backend config");
let VaultAuthMethod::Kubernetes {
role, mount, jwt_path, ..
} = &vault.auth_method
else {
panic!("expected Kubernetes auth, got {:?}", vault.auth_method);
};
assert_eq!(role, "rustfs");
assert_eq!(mount, crate::config::DEFAULT_VAULT_KUBERNETES_MOUNT);
assert_eq!(jwt_path, std::path::Path::new(crate::config::DEFAULT_VAULT_KUBERNETES_JWT_PATH));
let unknown_field = serde_json::json!({
"backend_type": "vault-transit",
"address": "https://vault.example.com:8200",
"auth_method": { "Kubernetes": { "role": "rustfs", "service_account": "rustfs" } }
});
serde_json::from_value::<ConfigureKmsRequest>(unknown_field)
.expect_err("an unknown auth field must be rejected rather than silently dropped");
}
#[test]
fn test_deserialize_aws_configure_request_accepts_type_aliases() {
for backend_type in ["AWS", "AwsKms", "aws", "aws-kms", "aws_kms"] {
+1
View File
@@ -657,6 +657,7 @@ impl KmsBackend for AwsKmsBackend {
created_by: None,
rotation_due: false,
rotation_due_reason: None,
wrap_budget_reserved: None,
});
}
+11
View File
@@ -57,6 +57,17 @@ impl ScriptedResponse {
}
}
/// The 404 Vault answers a LIST of an empty path with: something routed the
/// request and found nothing under it, so the `errors` array comes back
/// empty. [`ScriptedResponse::error`] cannot stand in — it always fills
/// `errors`, which is what marks a 404 as an unrouted path instead.
pub(crate) fn empty_list_404() -> Self {
Self::Http {
status: 404,
body: serde_json::json!({ "errors": [] }).to_string(),
}
}
/// Close the connection after consuming a request without sending an HTTP response.
pub(crate) fn close() -> Self {
Self::Close
+1
View File
@@ -271,6 +271,7 @@ impl StaticKmsBackend {
created_by: None,
rotation_due: false,
rotation_due_reason: None,
wrap_budget_reserved: None,
})
}
File diff suppressed because it is too large Load Diff
+271 -5
View File
@@ -326,6 +326,97 @@ impl fmt::Debug for AppRoleLogin {
}
}
/// Token source for [`VaultAuthMethod::Kubernetes`]: exchanges the pod's
/// projected ServiceAccount token for a lease-bound Vault token.
///
/// The JWT is re-read on every login because the kubelet rotates a projected
/// token well inside the pod's lifetime; caching it would strand the source on
/// an expired assertion once the current Vault token can no longer be renewed.
///
/// Unlike [`TokenFileSource`], the file mode is not checked: the kubelet owns
/// the projected token and mounts it world-readable by default, so rejecting
/// group/other bits would refuse every standard pod rather than catch a
/// deployment error.
pub(crate) struct KubernetesLogin {
/// Unauthenticated client used only for the login exchange.
login_client: VaultClient,
mount: String,
role: String,
jwt_path: PathBuf,
}
impl KubernetesLogin {
pub(crate) fn new(settings: &VaultConnectionSettings, mount: String, role: String, jwt_path: PathBuf) -> Result<Self> {
Ok(Self {
login_client: settings.build_login_client()?,
mount,
role,
jwt_path,
})
}
/// Read the ServiceAccount token for one login attempt.
///
/// Mirrors [`AppRoleLogin::resolve_secret_id`]: a read failure is fatal for
/// the attempt but the refresh loop keeps retrying, so a token the kubelet
/// has not projected yet heals the source without a restart.
async fn resolve_jwt(&self) -> AttemptResult<SecretString> {
let mut raw = tokio::fs::read_to_string(&self.jwt_path)
.await
.map_err(|error| AttemptError {
class: ErrorClass::Fatal,
error: KmsError::configuration_error(format!(
"Failed to read Kubernetes ServiceAccount token {}: {error}",
self.jwt_path.display()
)),
})?;
let trimmed = raw.trim();
if trimmed.is_empty() {
raw.zeroize();
return Err(AttemptError {
class: ErrorClass::Fatal,
error: KmsError::configuration_error(format!(
"Kubernetes ServiceAccount token {} is empty",
self.jwt_path.display()
)),
});
}
let jwt = SecretString::new(trimmed.to_string());
raw.zeroize();
Ok(jwt)
}
}
#[async_trait]
impl TokenSource for KubernetesLogin {
async fn acquire(&self) -> AttemptResult<TokenLease> {
let jwt = self.resolve_jwt().await?;
let auth = vaultrs::auth::kubernetes::login(&self.login_client, &self.mount, &self.role, jwt.expose())
.await
.map_err(|error| attempt_error("Kubernetes login", error))?;
Ok(TokenLease::from_auth(auth))
}
async fn renew(&self, client: &VaultClient) -> AttemptResult<TokenLease> {
let auth = vaultrs::token::renew_self(client, None)
.await
.map_err(|error| attempt_error("token renewal", error))?;
Ok(TokenLease::from_auth(auth))
}
}
impl fmt::Debug for KubernetesLogin {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
// The login client embeds Vault client settings and must stay out of
// Debug output; the role name is not a secret, and the JWT is never held.
f.debug_struct("KubernetesLogin")
.field("mount", &self.mount)
.field("role", &self.role)
.field("jwt_path", &self.jwt_path)
.finish_non_exhaustive()
}
}
/// Token source for [`VaultAuthMethod::TokenFile`]: reads an agent-managed
/// token file (for example a Vault Agent auto-auth sink).
///
@@ -464,6 +555,9 @@ pub(crate) fn token_source_for(
secret_id.clone(),
secret_id_file.clone(),
)?)),
VaultAuthMethod::Kubernetes {
role, mount, jwt_path, ..
} => Ok(Box::new(KubernetesLogin::new(settings, mount.clone(), role.clone(), jwt_path.clone())?)),
VaultAuthMethod::TokenFile {
path,
poll_interval_secs,
@@ -486,6 +580,9 @@ pub(crate) struct VaultConnectionSettings {
pub(crate) namespace: Option<String>,
/// Per-attempt HTTP timeout applied to the underlying reqwest client.
pub(crate) attempt_timeout: Duration,
/// Whether to accept an unverified Vault server certificate. Gated on
/// `allow_insecure_dev_defaults` by `KmsConfig::validate`.
pub(crate) skip_tls_verify: bool,
}
impl VaultConnectionSettings {
@@ -499,6 +596,11 @@ impl VaultConnectionSettings {
// operation-level retry policy.
settings_builder.timeout(Some(self.attempt_timeout));
settings_builder.token(token);
// Always set explicitly: left unset, vaultrs derives this from its own
// VAULT_SKIP_VERIFY variable, so a stray value in the environment would
// disable certificate verification behind the KMS configuration and its
// insecure-defaults gate.
settings_builder.verify(!self.skip_tls_verify);
if let Some(namespace) = &self.namespace {
settings_builder.namespace(Some(namespace.clone()));
@@ -551,6 +653,10 @@ impl VaultCredentialPolicy {
refresh_safety_window_secs: Some(secs),
..
}
| VaultAuthMethod::Kubernetes {
refresh_safety_window_secs: Some(secs),
..
}
| VaultAuthMethod::TokenFile {
refresh_safety_window_secs: Some(secs),
..
@@ -584,15 +690,25 @@ pub(crate) struct VaultClientHandle {
impl VaultClientHandle {
/// Absolute expiry of this generation's token.
///
/// `lease.ttl` is built from the `lease_duration` the Vault server sent, so
/// a value too large to add to `issued_at` would panic on the bare `+`. A
/// TTL that cannot be represented is indistinguishable from no expiry, so it
/// collapses to `None` — the same answer already given for the zero-lease
/// tokens Vault issues, which keeps the token in use and still fully
/// validated by Vault on every call.
fn expires_at(&self) -> Option<Instant> {
self.lease.map(|lease| self.issued_at + lease.ttl)
self.lease.and_then(|lease| self.issued_at.checked_add(lease.ttl))
}
/// When the renewal task should refresh this generation: half the TTL,
/// leaving the second half as budget for retries before the fail-closed
/// window is reached.
///
/// Unrepresentable TTLs collapse to `None` as in [`Self::expires_at`],
/// leaving a token that never expires with nothing to renew.
fn renew_at(&self) -> Option<Instant> {
self.lease.map(|lease| self.issued_at + lease.ttl / 2)
self.lease.and_then(|lease| self.issued_at.checked_add(lease.ttl / 2))
}
}
@@ -662,7 +778,7 @@ impl VaultCredentialProvider {
let handle = self.current.load_full();
if let Some(expires_at) = handle.expires_at() {
let now = Instant::now();
if now + self.policy.safety_window >= expires_at {
if self.inside_safety_window(now, expires_at) {
return Err(KmsError::credentials_unavailable(format!(
"Vault token (generation {}) is within {:?} of expiry and has not been refreshed; refusing to use it",
handle.generation, self.policy.safety_window
@@ -672,6 +788,18 @@ impl VaultCredentialProvider {
Ok(handle)
}
/// Whether the token expiring at `expires_at` is close enough to refuse.
///
/// `safety_window` reaches here from persisted configuration, so it is not
/// guaranteed to have passed this version's validation: a window too large
/// to add to the current instant would panic on the bare `+`. Such a window
/// means every token is always inside it, so saturating to "refuse" is both
/// the fail-closed answer and the one the arithmetic was reaching for.
fn inside_safety_window(&self, now: Instant, expires_at: Instant) -> bool {
now.checked_add(self.policy.safety_window)
.is_none_or(|deadline| deadline >= expires_at)
}
/// Publish the credential gauges for the generation currently installed.
///
/// The fail-closed gauge re-evaluates the very gate
@@ -683,7 +811,7 @@ impl VaultCredentialProvider {
let fail_closed = match handle.expires_at() {
Some(expires_at) => {
metrics::gauge!(METRIC_TOKEN_TTL_SECONDS).set(expires_at.saturating_duration_since(now).as_secs_f64());
now + self.policy.safety_window >= expires_at
self.inside_safety_window(now, expires_at)
}
// A generation without an expiry has no remaining TTL to report
// and can never lapse, so it can never fail closed either.
@@ -860,7 +988,7 @@ impl Drop for CredentialTaskHandle {
#[cfg(test)]
mod tests {
use super::*;
use crate::config::REDACTED_SECRET;
use crate::config::{DEFAULT_VAULT_KUBERNETES_MOUNT, REDACTED_SECRET};
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
const TEST_TOKEN: &str = "vault-token-debug-leak-canary";
@@ -871,6 +999,7 @@ mod tests {
address: "http://127.0.0.1:8200".to_string(),
namespace: Some("team-namespace".to_string()),
attempt_timeout: Duration::from_secs(30),
skip_tls_verify: false,
}
}
@@ -1057,6 +1186,143 @@ mod tests {
assert!(format!("{source:?}").contains("AppRoleLogin"));
}
#[tokio::test]
async fn test_kubernetes_auth_method_maps_to_login_source() {
let settings = test_settings();
let source = token_source_for(&VaultAuthMethod::kubernetes("rustfs".to_string()), &settings)
.expect("kubernetes auth must map to a login source");
assert!(format!("{source:?}").contains("KubernetesLogin"));
}
/// `refresh_safety_window_secs` is operator-supplied and reaches the request
/// path from persisted configuration, so the fail-closed comparison must
/// survive a window too large to add to the current instant. Before the
/// checked arithmetic this panicked with "overflow when adding duration to
/// instant" on the first request after a lease-bearing login.
#[tokio::test]
async fn test_current_refuses_rather_than_panics_on_an_unrepresentable_safety_window() {
let (provider, _state) = scripted_provider(
Duration::from_secs(60),
true,
test_policy(Duration::from_secs(u64::MAX), Duration::from_secs(5)),
)
.await;
let error = provider
.current()
.expect_err("a window wider than any lease must refuse the token");
assert!(
matches!(error, KmsError::CredentialsUnavailable { .. }),
"expected CredentialsUnavailable, got {error:?}"
);
}
/// `lease_duration` is a bare u64 straight off the Vault response and forms
/// the other side of the same comparison, so an absurd one must not panic
/// either. It is indistinguishable from a non-expiring token, which is how
/// the zero-lease case already behaves.
#[tokio::test]
async fn test_an_unrepresentable_lease_is_treated_as_non_expiring() {
let (provider, _state) = scripted_provider(
Duration::from_secs(u64::MAX),
true,
test_policy(Duration::from_secs(30), Duration::from_secs(5)),
)
.await;
provider
.current()
.expect("a token whose expiry cannot be represented must stay usable");
}
/// The configured flag has to reach the HTTP client, not just the config
/// struct: every generation (authenticated and login) builds its own client,
/// and a Vault with a self-signed certificate fails the handshake unless
/// each one carries the setting.
#[test]
fn test_skip_tls_verify_reaches_every_vault_client_generation() {
for skip_tls_verify in [false, true] {
let settings = VaultConnectionSettings {
address: "https://vault.example.com:8200".to_string(),
namespace: None,
attempt_timeout: Duration::from_secs(30),
skip_tls_verify,
};
let authenticated = settings.build_client(TEST_TOKEN).expect("authenticated client must build");
assert_eq!(authenticated.settings.verify, !skip_tls_verify);
let login = settings.build_login_client().expect("login client must build");
assert_eq!(login.settings.verify, !skip_tls_verify);
}
}
/// vaultrs derives `verify` from its own VAULT_SKIP_VERIFY variable when the
/// builder leaves it unset, which would disable certificate verification
/// without passing the KMS insecure-defaults gate.
#[test]
fn test_vaultrs_skip_verify_env_cannot_override_the_configured_setting() {
temp_env::with_var("VAULT_SKIP_VERIFY", Some("true"), || {
let client = test_settings().build_client(TEST_TOKEN).expect("client must build");
assert!(
client.settings.verify,
"a stray VAULT_SKIP_VERIFY must not disable verification behind the KMS configuration"
);
});
}
/// The projected token is read fresh per login attempt and trimmed, so a
/// kubelet rotation is picked up without a restart and a trailing newline
/// does not corrupt the assertion sent to Vault.
#[tokio::test]
async fn test_kubernetes_login_rereads_and_trims_the_service_account_token() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("token");
tokio::fs::write(&path, " first-jwt\n").await.expect("write token");
let login = KubernetesLogin::new(
&test_settings(),
DEFAULT_VAULT_KUBERNETES_MOUNT.to_string(),
"rustfs".to_string(),
path.clone(),
)
.expect("login source must build");
assert_eq!(login.resolve_jwt().await.expect("first read").expose(), "first-jwt");
tokio::fs::write(&path, "rotated-jwt").await.expect("rotate token");
assert_eq!(
login.resolve_jwt().await.expect("second read").expose(),
"rotated-jwt",
"a rotated projected token must be picked up without a restart"
);
}
/// The ServiceAccount token is re-read per attempt, so an unreadable or
/// empty one fails that attempt without reaching Vault; the refresh loop
/// keeps retrying, which is what lets a late projection heal the source.
#[tokio::test]
async fn test_kubernetes_login_rejects_an_unusable_service_account_token() {
let dir = tempfile::tempdir().expect("temp dir");
let missing = dir.path().join("absent-token");
let empty = dir.path().join("empty-token");
tokio::fs::write(&empty, " \n").await.expect("write empty token");
for (path, expected) in [(missing, "Failed to read"), (empty, "is empty")] {
let login =
KubernetesLogin::new(&test_settings(), DEFAULT_VAULT_KUBERNETES_MOUNT.to_string(), "rustfs".to_string(), path)
.expect("login source must build");
let error = login
.acquire()
.await
.expect_err("an unusable ServiceAccount token must fail the attempt");
assert!(matches!(error.class, ErrorClass::Fatal));
assert!(error.error.to_string().contains(expected), "got {}", error.error);
}
}
#[tokio::test(start_paused = true)]
async fn test_renewal_task_renews_at_half_ttl() {
let (provider, state) = scripted_provider(
+345 -12
View File
@@ -27,6 +27,7 @@ use crate::backends::{
use crate::config::{KmsConfig, VaultTransitConfig};
use crate::encryption::{DataKeyEnvelope, generate_key_material};
use crate::error::{KmsError, Result};
use crate::persisted_observability::{BoundedUnknownFieldName, UnknownFieldSummary};
use crate::policy::{self, AttemptError, OpClass, RetryPolicy};
use crate::types::*;
use async_trait::async_trait;
@@ -100,6 +101,23 @@ fn is_cas_conflict(error: &ClientError) -> bool {
)
}
/// Whether a transit LIST failed with the 404 Vault uses for "mounted, but no
/// keys yet".
///
/// Vault answers a LIST on a mounted transit engine that holds no keys with a
/// 404 whose `errors` array is empty — the mount routed and answered the
/// request, so the engine is reachable. A 404 for a path with no mount behind
/// it instead carries a "no handler for route" message, so the empty `errors`
/// array is what separates "engine reachable but empty" from "engine missing".
///
/// An empty non-transit engine (e.g. KV v1) at the configured path answers
/// with byte-identical 404s, so this probe cannot detect that misconfiguration
/// — no LIST-based probe can. The data path still fails hard on the first real
/// transit operation against such a mount.
fn is_empty_transit_list(error: &ClientError) -> bool {
matches!(error, ClientError::APIError { code: 404, errors } if errors.is_empty())
}
#[derive(Debug, Clone)]
struct TransitKeyMetadata {
key_usage: KeyUsage,
@@ -114,7 +132,12 @@ struct TransitKeyMetadata {
}
/// Serializable version of TransitKeyMetadata for KV v2 persistence.
#[derive(Debug, Clone, Serialize, Deserialize)]
///
/// `Deserialize` is hand-written so fields the current build does not know
/// are counted and warned about instead of vanishing silently — this record
/// is compatibility-bound in both directions (older and newer builds read
/// each other's writes), so `deny_unknown_fields` is not an option.
#[derive(Debug, Clone, Serialize)]
struct TransitKeyMetadataPersisted {
key_usage: KeyUsage,
description: Option<String>,
@@ -127,6 +150,168 @@ struct TransitKeyMetadataPersisted {
current_version: u32,
}
impl UnknownFieldSummary {
fn record_for_transit_key_metadata(&self) {
let Some((field, field_name_truncated, field_count)) = self.record("vault-transit-key-metadata") else {
return;
};
static RECORDS_WITH_UNKNOWN_FIELDS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let observed_records = RECORDS_WITH_UNKNOWN_FIELDS
.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
.saturating_add(1);
if observed_records.is_power_of_two() {
tracing::warn!(
field = ?field,
field_name_truncated,
field_count,
observed_records,
"Vault Transit key metadata record contains unknown fields"
);
}
}
}
impl<'de> Deserialize<'de> for TransitKeyMetadataPersisted {
fn deserialize<D>(deserializer: D) -> std::result::Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
use serde::de::{self, IgnoredAny, MapAccess, Visitor};
use std::fmt;
enum Field {
KeyUsage,
Description,
Tags,
KeyState,
CreatedAt,
DeletionDate,
Origin,
CreatedBy,
CurrentVersion,
Unknown(BoundedUnknownFieldName),
}
impl<'de> Deserialize<'de> for Field {
fn deserialize<D>(deserializer: D) -> std::result::Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
struct FieldVisitor;
impl Visitor<'_> for FieldVisitor {
type Value = Field;
fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("a Vault Transit key metadata field name")
}
fn visit_str<E>(self, value: &str) -> std::result::Result<Self::Value, E>
where
E: de::Error,
{
Ok(match value {
"key_usage" => Field::KeyUsage,
"description" => Field::Description,
"tags" => Field::Tags,
"key_state" => Field::KeyState,
"created_at" => Field::CreatedAt,
"deletion_date" => Field::DeletionDate,
"origin" => Field::Origin,
"created_by" => Field::CreatedBy,
"current_version" => Field::CurrentVersion,
_ => Field::Unknown(BoundedUnknownFieldName::new(value)),
})
}
}
deserializer.deserialize_identifier(FieldVisitor)
}
}
struct TransitKeyMetadataPersistedVisitor;
impl<'de> Visitor<'de> for TransitKeyMetadataPersistedVisitor {
type Value = TransitKeyMetadataPersisted;
fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("a Vault Transit key metadata record")
}
fn visit_map<A>(self, mut map: A) -> std::result::Result<Self::Value, A::Error>
where
A: MapAccess<'de>,
{
macro_rules! read_field {
($slot:ident, $name:literal) => {{
if $slot.is_some() {
return Err(de::Error::duplicate_field($name));
}
$slot = Some(map.next_value()?);
}};
}
let mut key_usage = None;
let mut description = None;
let mut tags = None;
let mut key_state = None;
let mut created_at = None;
let mut deletion_date = None;
let mut origin = None;
let mut created_by = None;
let mut current_version = None;
let mut unknown_fields = UnknownFieldSummary::default();
while let Some(field) = map.next_key()? {
match field {
Field::KeyUsage => read_field!(key_usage, "key_usage"),
Field::Description => read_field!(description, "description"),
Field::Tags => read_field!(tags, "tags"),
Field::KeyState => read_field!(key_state, "key_state"),
Field::CreatedAt => read_field!(created_at, "created_at"),
Field::DeletionDate => read_field!(deletion_date, "deletion_date"),
Field::Origin => read_field!(origin, "origin"),
Field::CreatedBy => read_field!(created_by, "created_by"),
Field::CurrentVersion => read_field!(current_version, "current_version"),
Field::Unknown(field) => {
let _: IgnoredAny = map.next_value()?;
unknown_fields.observe(field);
}
}
}
let metadata = TransitKeyMetadataPersisted {
key_usage: key_usage.ok_or_else(|| de::Error::missing_field("key_usage"))?,
description: description.unwrap_or(None),
tags: tags.ok_or_else(|| de::Error::missing_field("tags"))?,
key_state: key_state.ok_or_else(|| de::Error::missing_field("key_state"))?,
created_at: created_at.ok_or_else(|| de::Error::missing_field("created_at"))?,
deletion_date: deletion_date.unwrap_or(None),
origin: origin.ok_or_else(|| de::Error::missing_field("origin"))?,
created_by: created_by.unwrap_or(None),
current_version: current_version.ok_or_else(|| de::Error::missing_field("current_version"))?,
};
unknown_fields.record_for_transit_key_metadata();
Ok(metadata)
}
}
const FIELDS: &[&str] = &[
"key_usage",
"description",
"tags",
"key_state",
"created_at",
"deletion_date",
"origin",
"created_by",
"current_version",
];
deserializer.deserialize_struct("TransitKeyMetadataPersisted", FIELDS, TransitKeyMetadataPersistedVisitor)
}
}
impl TransitKeyMetadata {
fn from_create_request(request: &CreateKeyRequest) -> Self {
Self {
@@ -230,6 +415,7 @@ impl VaultTransitKmsClient {
address: config.address.clone(),
namespace: config.namespace.clone(),
attempt_timeout: kms_config.effective_timeout(),
skip_tls_verify: config.tls.as_ref().is_some_and(|tls| tls.skip_verify),
};
let source = token_source_for(&config.auth_method, &settings)?;
let policy = VaultCredentialPolicy::from_kms_config(
@@ -706,6 +892,7 @@ impl VaultTransitKmsClient {
created_by: metadata.created_by,
rotation_due: false,
rotation_due_reason: None,
wrap_budget_reserved: None,
})
}
@@ -1074,12 +1261,17 @@ impl VaultTransitKmsClient {
let mut all_keys = self
.run("vault_transit_list_keys", OpClass::ReadIdempotent, move || async move {
let vault = self.vault().map_err(AttemptError::fatal)?;
key::list(&vault.client, &self.config.mount_path).await.map_err(|e| {
AttemptError::from_vaultrs(e, |e| KmsError::backend_error(format!("Failed to list Vault Transit keys: {e}")))
})
match key::list(&vault.client, &self.config.mount_path).await {
Ok(response) => Ok(response.keys),
// An empty transit engine answers LIST with a bare 404;
// that is an empty listing, not a backend failure.
Err(error) if is_empty_transit_list(&error) => Ok(Vec::new()),
Err(e) => Err(AttemptError::from_vaultrs(e, |e| {
KmsError::backend_error(format!("Failed to list Vault Transit keys: {e}"))
})),
}
})
.await?
.keys;
.await?;
// Vault's own LIST ordering is not part of its contract, so the sort is
// what makes the marker a stable cursor across calls.
all_keys.sort_unstable();
@@ -1252,12 +1444,17 @@ impl VaultTransitKmsClient {
pub(crate) async fn health_check(&self) -> Result<()> {
self.run("vault_transit_health_check", OpClass::ReadIdempotent, move || async move {
let vault = self.vault().map_err(AttemptError::fatal)?;
key::list(&vault.client, &self.config.mount_path)
.await
.map(|_| ())
.map_err(|e| {
AttemptError::from_vaultrs(e, |e| KmsError::backend_error(format!("Vault Transit health check failed: {e}")))
})
match key::list(&vault.client, &self.config.mount_path).await {
Ok(_) => Ok(()),
// A brand-new transit mount holds no keys until something
// creates one, and this check gates startup before the service
// creates its own probe key — treating "empty" as unhealthy
// would keep a first-ever deployment from ever starting.
Err(error) if is_empty_transit_list(&error) => Ok(()),
Err(e) => Err(AttemptError::from_vaultrs(e, |e| {
KmsError::backend_error(format!("Vault Transit health check failed: {e}"))
})),
}
})
.await
}
@@ -1916,6 +2113,107 @@ mod tests {
);
}
/// Regression test for the first-boot chicken-and-egg on a fresh transit
/// mount (rustfs/backlog#1774).
///
/// Vault answers a LIST on a mounted-but-empty transit engine with a 404
/// carrying an empty `errors` array. The health check gates startup before
/// the service creates its probe key, so this 404 must count as healthy —
/// failing it means a first-ever deployment on a fresh mount can never
/// start until an operator creates some transit key out-of-band.
#[tokio::test]
async fn health_check_passes_on_an_empty_transit_engine() {
let (vault, client) = scripted_client(vec![ScriptedResponse::Http {
status: 404,
body: serde_json::json!({ "errors": [] }).to_string(),
}])
.await;
client
.health_check()
.await
.expect("an empty transit engine is reachable and must pass the health check");
let requests = vault.requests();
assert_eq!(
requests,
vec!["LIST /v1/transit/keys".to_string()],
"the empty-list 404 must be accepted on the first attempt, not retried"
);
}
/// A 404 whose body says "no handler for route" means no transit engine is
/// mounted at the configured path at all; that must keep failing the
/// health check instead of riding the empty-engine allowance.
#[tokio::test]
async fn health_check_fails_when_the_transit_mount_is_missing() {
let (_vault, client) = scripted_client(vec![ScriptedResponse::error(
404,
"no handler for route \"transit/keys\". route entry not found.",
)])
.await;
let error = client
.health_check()
.await
.expect_err("a missing transit mount must fail the health check");
assert!(matches!(error, KmsError::BackendError { .. }), "got {error:?}");
}
/// The empty-engine allowance is scoped to 404 alone: any other status
/// whose body happens to carry an empty `errors` array (an intermediary
/// answering for Vault, for instance) must keep failing the health check.
#[tokio::test]
async fn health_check_fails_on_a_non_404_error_with_an_empty_errors_body() {
let (_vault, client) = scripted_client(vec![ScriptedResponse::Http {
status: 403,
body: serde_json::json!({ "errors": [] }).to_string(),
}])
.await;
let error = client
.health_check()
.await
.expect_err("only a 404 may ride the empty-engine allowance");
assert!(matches!(error, KmsError::BackendError { .. }), "got {error:?}");
}
/// The listing's own copy of the discriminator must not widen into "every
/// LIST failure is an empty listing" — a missing mount still fails loudly.
#[tokio::test]
async fn list_fails_when_the_transit_mount_is_missing() {
let (_vault, client) = scripted_client(vec![ScriptedResponse::error(
404,
"no handler for route \"transit/keys\". route entry not found.",
)])
.await;
let error = client
.list_keys(&ListKeysRequest::default(), None)
.await
.expect_err("a missing transit mount must fail the listing, not empty it");
assert!(matches!(error, KmsError::BackendError { .. }), "got {error:?}");
}
/// The same empty-engine 404 on the listing path is an empty result set,
/// not a backend failure.
#[tokio::test]
async fn list_keys_returns_an_empty_page_on_an_empty_transit_engine() {
let (_vault, client) = scripted_client(vec![ScriptedResponse::Http {
status: 404,
body: serde_json::json!({ "errors": [] }).to_string(),
}])
.await;
let response = client
.list_keys(&ListKeysRequest::default(), None)
.await
.expect("an empty transit engine must list as empty, not fail");
assert!(response.keys.is_empty(), "got {:?}", response.keys);
assert!(!response.truncated, "an empty listing has nothing left to page through");
assert_eq!(response.next_marker, None);
}
fn test_vault_transit_config() -> VaultTransitConfig {
VaultTransitConfig {
address: "http://127.0.0.1:8200".to_string(),
@@ -2133,6 +2431,41 @@ mod tests {
assert!(metadata.deletion_date.is_none());
}
#[test]
fn transit_key_metadata_unknown_fields_remain_readable_and_are_observed() {
// A record written by a newer build carries fields this build does not
// know. It must stay readable — and the drop must be visible, not
// silent (rustfs/backlog#1641). Only the field name may be logged.
let persisted: TransitKeyMetadataPersisted = TransitKeyMetadata::synthesized().into();
let mut value = serde_json::to_value(&persisted).expect("serialize metadata record");
let object = value.as_object_mut().expect("metadata record serializes to an object");
object.insert("field_from_the_future".to_string(), serde_json::json!("field value must not be logged"));
let logs = crate::test_support::CapturedLogs::default();
let subscriber = tracing_subscriber::fmt()
.with_ansi(false)
.with_max_level(tracing::Level::WARN)
.with_writer(logs.clone())
.finish();
let dispatch = tracing::Dispatch::new(subscriber);
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let parsed: TransitKeyMetadataPersisted = metrics::with_local_recorder(&recorder, || {
tracing::dispatcher::with_default(&dispatch, || {
serde_json::from_value(value).expect("unknown fields must remain readable")
})
});
assert_eq!(parsed.key_state, KeyState::Enabled);
assert_eq!(crate::test_support::unknown_field_metric(&recorder, "vault-transit-key-metadata"), 1);
let output = logs.output();
assert!(
output.contains("Vault Transit key metadata record contains unknown fields"),
"got: {output}"
);
assert!(output.contains("field_from_the_future"));
assert!(!output.contains("field value must not be logged"));
}
/// KV2 write acknowledgement (`SecretVersionMetadata`) for `kv2::set`.
fn kv2_write_ack() -> serde_json::Value {
serde_json::json!({
+4
View File
@@ -450,6 +450,10 @@ impl VaultRestoreClient {
address: target.address.clone(),
namespace: target.namespace.clone(),
attempt_timeout: kms_config.effective_timeout(),
// A restore target carries no TLS settings, so certificates are
// always verified: recovery is the last path that should accept an
// unauthenticated Vault.
skip_tls_verify: false,
};
let source = token_source_for(&target.auth_method, &settings)?;
let policy = VaultCredentialPolicy::from_kms_config(
+430 -54
View File
@@ -25,6 +25,10 @@ use url::Url;
pub const ENV_KMS_ALLOW_INSECURE_DEV_DEFAULTS: &str = "RUSTFS_KMS_ALLOW_INSECURE_DEV_DEFAULTS";
pub const ENV_KMS_ALLOW_IMMEDIATE_DELETION: &str = "RUSTFS_KMS_ALLOW_IMMEDIATE_DELETION";
pub const ENV_KMS_VAULT_ADDRESS: &str = "RUSTFS_KMS_VAULT_ADDRESS";
pub const ENV_KMS_VAULT_TOKEN: &str = "RUSTFS_KMS_VAULT_TOKEN";
pub const ENV_KMS_VAULT_NAMESPACE: &str = "RUSTFS_KMS_VAULT_NAMESPACE";
pub const ENV_KMS_VAULT_MOUNT_PATH: &str = "RUSTFS_KMS_VAULT_MOUNT_PATH";
pub const ENV_KMS_VAULT_SKIP_TLS_VERIFY: &str = "RUSTFS_KMS_VAULT_SKIP_TLS_VERIFY";
pub const ENV_KMS_VAULT_TRANSIT_METADATA_KV_MOUNT: &str = "RUSTFS_KMS_VAULT_TRANSIT_METADATA_KV_MOUNT";
pub const ENV_KMS_VAULT_TRANSIT_METADATA_PREFIX: &str = "RUSTFS_KMS_VAULT_TRANSIT_METADATA_PREFIX";
@@ -35,6 +39,9 @@ pub const ENV_KMS_VAULT_APPROLE_SECRET_ID: &str = "RUSTFS_KMS_VAULT_APPROLE_SECR
pub const ENV_KMS_VAULT_APPROLE_SECRET_ID_FILE: &str = "RUSTFS_KMS_VAULT_APPROLE_SECRET_ID_FILE";
pub const ENV_KMS_VAULT_APPROLE_MOUNT: &str = "RUSTFS_KMS_VAULT_APPROLE_MOUNT";
pub const ENV_KMS_VAULT_TOKEN_FILE: &str = "RUSTFS_KMS_VAULT_TOKEN_FILE";
pub const ENV_KMS_VAULT_KUBERNETES_ROLE: &str = "RUSTFS_KMS_VAULT_KUBERNETES_ROLE";
pub const ENV_KMS_VAULT_KUBERNETES_MOUNT: &str = "RUSTFS_KMS_VAULT_KUBERNETES_MOUNT";
pub const ENV_KMS_VAULT_KUBERNETES_JWT_PATH: &str = "RUSTFS_KMS_VAULT_KUBERNETES_JWT_PATH";
pub const ENV_KMS_AWS_REGION: &str = "RUSTFS_KMS_AWS_REGION";
pub const ENV_KMS_AWS_ENDPOINT_URL: &str = "RUSTFS_KMS_AWS_ENDPOINT_URL";
/// Age in whole seconds beyond which a key is reported as due for rotation;
@@ -44,6 +51,9 @@ pub const ENV_KMS_ROTATION_MAX_AGE_SECS: &str = "RUSTFS_KMS_ROTATION_MAX_AGE_SEC
pub const DEFAULT_VAULT_TRANSIT_METADATA_KV_MOUNT: &str = "secret";
pub const DEFAULT_VAULT_TRANSIT_METADATA_KEY_PREFIX: &str = "rustfs/kms/transit-metadata";
pub const DEFAULT_VAULT_APPROLE_MOUNT: &str = "approle";
pub const DEFAULT_VAULT_KUBERNETES_MOUNT: &str = "kubernetes";
/// Where the kubelet projects a pod's ServiceAccount token by default.
pub const DEFAULT_VAULT_KUBERNETES_JWT_PATH: &str = "/var/run/secrets/kubernetes.io/serviceaccount/token";
/// Upper bound applied to `KmsConfig::timeout` when deriving backend behavior.
///
@@ -83,6 +93,14 @@ fn default_vault_approle_mount() -> String {
DEFAULT_VAULT_APPROLE_MOUNT.to_string()
}
fn default_vault_kubernetes_mount() -> String {
DEFAULT_VAULT_KUBERNETES_MOUNT.to_string()
}
fn default_vault_kubernetes_jwt_path() -> PathBuf {
PathBuf::from(DEFAULT_VAULT_KUBERNETES_JWT_PATH)
}
pub const KMS_CONFIG_REDACTION_RULES: &[RedactionRule] = &[
RedactionRule::new("kms.local.master_key", RedactionLevel::Secret, "local backend key encryption material"),
RedactionRule::new("kms.vault.token", RedactionLevel::Secret, "vault authentication token"),
@@ -489,6 +507,23 @@ pub enum VaultAuthMethod {
#[serde(default)]
refresh_safety_window_secs: Option<u64>,
},
/// Kubernetes authentication: the pod's ServiceAccount token is exchanged
/// for a lease-bound Vault token that is renewed in the background.
Kubernetes {
/// Vault role bound to this ServiceAccount.
role: String,
/// Kubernetes auth engine mount path.
#[serde(default = "default_vault_kubernetes_mount")]
mount: String,
/// Projected ServiceAccount token to present. Re-read on every login so
/// a token the kubelet rotates is picked up without a restart.
#[serde(default = "default_vault_kubernetes_jwt_path")]
jwt_path: PathBuf,
/// Fail-closed margin in seconds, as on `AppRole`. Defaults to the
/// per-attempt timeout.
#[serde(default)]
refresh_safety_window_secs: Option<u64>,
},
/// Agent-managed token file (for example a Vault Agent auto-auth sink):
/// the token is read from `path` and re-read periodically so a token
/// rotated by the agent is picked up without a restart.
@@ -519,6 +554,16 @@ impl VaultAuthMethod {
}
}
/// Kubernetes authentication with the default mount and projected token path.
pub fn kubernetes(role: String) -> Self {
Self::Kubernetes {
role,
mount: default_vault_kubernetes_mount(),
jwt_path: default_vault_kubernetes_jwt_path(),
refresh_safety_window_secs: None,
}
}
/// Agent-managed token file with the default poll interval.
pub fn token_file(path: PathBuf) -> Self {
Self::TokenFile {
@@ -547,6 +592,20 @@ impl fmt::Debug for VaultAuthMethod {
.field("mount", mount)
.field("refresh_safety_window_secs", refresh_safety_window_secs)
.finish(),
// No redaction: the role and mount name a Vault binding, and the
// ServiceAccount token itself is never held on this type.
Self::Kubernetes {
role,
mount,
jwt_path,
refresh_safety_window_secs,
} => f
.debug_struct("Kubernetes")
.field("role", role)
.field("mount", mount)
.field("jwt_path", jwt_path)
.field("refresh_safety_window_secs", refresh_safety_window_secs)
.finish(),
Self::TokenFile {
path,
poll_interval_secs,
@@ -868,6 +927,14 @@ impl KmsConfig {
// `mount_path` is deprecated and unused by this backend, so an empty value
// is deliberately not an error.
// `kv_mount` is: it is the mount every read, write and listing is
// routed through, and an empty one produces a path Vault has no
// handler for. Rejecting it here names the setting; letting it
// through spends a round-trip to report an unroutable path.
if config.kv_mount.is_empty() {
return Err(KmsError::configuration_error("Vault KV2 mount cannot be empty"));
}
// Validate TLS configuration if using HTTPS
if config.address.starts_with("https://")
&& let Some(ref tls) = config.tls
@@ -1019,50 +1086,12 @@ impl KmsConfig {
});
}
KmsBackend::VaultKv2 => {
let address = get_env_str("RUSTFS_KMS_VAULT_ADDRESS", "http://localhost:8200");
let auth_method = vault_auth_method_from_env()?;
let skip_tls_verify = get_env_bool(ENV_KMS_VAULT_SKIP_TLS_VERIFY, false);
let mount_path = match get_env_opt_str("RUSTFS_KMS_VAULT_MOUNT_PATH") {
Some(path) => {
tracing::warn!(
"RUSTFS_KMS_VAULT_MOUNT_PATH is deprecated for the Vault KV2 backend: it never calls the Transit engine and the value is stored but unused"
);
path
}
None => default_vault_kv2_mount_path(),
};
config.backend_config = BackendConfig::VaultKv2(Box::new(VaultConfig {
address,
auth_method,
namespace: get_env_opt_str("RUSTFS_KMS_VAULT_NAMESPACE"),
mount_path,
kv_mount: get_env_str("RUSTFS_KMS_VAULT_KV_MOUNT", "secret"),
key_path_prefix: get_env_str("RUSTFS_KMS_VAULT_KEY_PREFIX", "rustfs/kms/keys"),
tls: vault_tls_config(skip_tls_verify),
}));
config.backend_config =
BackendConfig::VaultKv2(Box::new(vault_kv2_config_from_env(VaultCliOverrides::default())?));
}
KmsBackend::VaultTransit => {
let address = get_env_str("RUSTFS_KMS_VAULT_ADDRESS", "http://localhost:8200");
let auth_method = vault_auth_method_from_env()?;
let skip_tls_verify = get_env_bool(ENV_KMS_VAULT_SKIP_TLS_VERIFY, false);
config.backend_config = BackendConfig::VaultTransit(Box::new(VaultTransitConfig {
address,
auth_method,
namespace: get_env_opt_str("RUSTFS_KMS_VAULT_NAMESPACE"),
mount_path: get_env_str("RUSTFS_KMS_VAULT_MOUNT_PATH", "transit"),
metadata_kv_mount: get_env_str(
ENV_KMS_VAULT_TRANSIT_METADATA_KV_MOUNT,
DEFAULT_VAULT_TRANSIT_METADATA_KV_MOUNT,
),
metadata_key_prefix: get_env_str(
ENV_KMS_VAULT_TRANSIT_METADATA_PREFIX,
DEFAULT_VAULT_TRANSIT_METADATA_KEY_PREFIX,
),
tls: vault_tls_config(skip_tls_verify),
}));
config.backend_config =
BackendConfig::VaultTransit(Box::new(vault_transit_config_from_env(VaultCliOverrides::default())?));
}
KmsBackend::Static => {
// Read from file first, then fall back to direct env var
@@ -1129,6 +1158,53 @@ pub fn allow_immediate_deletion_from_env() -> bool {
get_env_bool(ENV_KMS_ALLOW_IMMEDIATE_DELETION, false)
}
impl crate::persisted_observability::UnknownFieldSummary {
fn record_for_kms_config(&self) {
let Some((field, field_name_truncated, field_count)) = self.record("kms-config") else {
return;
};
static RECORDS_WITH_UNKNOWN_FIELDS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let observed_records = RECORDS_WITH_UNKNOWN_FIELDS
.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
.saturating_add(1);
if observed_records.is_power_of_two() {
tracing::warn!(
field = ?field,
field_name_truncated,
field_count,
observed_records,
"persisted KMS configuration contains unknown fields"
);
}
}
}
/// Deserialize a persisted KMS configuration, observing ignored fields.
///
/// The persisted configuration deliberately tolerates unknown fields — a
/// rolling upgrade writes fields the previous build does not know, and
/// rejecting them would turn every upgrade into a hard stop (see the
/// regression test pinning that tolerance). Tolerated must not mean
/// invisible: this loader wraps the deserializer with `serde_ignored`, so
/// every field the configuration silently dropped is counted and sampled
/// into a warning, per the repository rule that formats too
/// compatibility-bound for `deny_unknown_fields` must at least log unknown
/// fields. Only field paths are recorded, never values — a mistyped field
/// name can sit next to a secret.
pub fn kms_config_from_persisted_json(data: &[u8]) -> serde_json::Result<KmsConfig> {
use crate::persisted_observability::{BoundedUnknownFieldName, UnknownFieldSummary};
let mut deserializer = serde_json::Deserializer::from_slice(data);
let mut unknown_fields = UnknownFieldSummary::default();
let config: KmsConfig = serde_ignored::deserialize(&mut deserializer, |path| {
unknown_fields.observe(BoundedUnknownFieldName::new(&path.to_string()));
})?;
deserializer.end()?;
unknown_fields.record_for_kms_config();
Ok(config)
}
fn vault_tls_config(skip_tls_verify: bool) -> Option<TlsConfig> {
skip_tls_verify.then_some(TlsConfig {
ca_cert_path: None,
@@ -1146,6 +1222,78 @@ fn is_under_temp_dir(path: &Path) -> bool {
path.starts_with(std::env::temp_dir())
}
/// Command-line values that take precedence over the matching environment
/// variables when assembling a Vault backend configuration.
///
/// Every field has a `RUSTFS_KMS_VAULT_*` equivalent that the CLI layer already
/// reads, so these are only set when the operator passed an explicit flag.
///
/// Deliberately not `Debug`: `token` holds the raw Vault token, and the
/// redacting `Debug` impls elsewhere in this module exist because a derived one
/// would print it. Denying the derive makes a future `{overrides:?}` a compile
/// error instead of a leak.
#[derive(Default, Clone, Copy)]
pub struct VaultCliOverrides<'a> {
pub address: Option<&'a str>,
pub token: Option<&'a str>,
pub mount_path: Option<&'a str>,
}
/// Assemble the Vault KV2 backend configuration from the environment.
///
/// Shared by [`KmsConfig::from_env`] and the server's command-line startup path
/// so both resolve the same auth method, namespace, TLS and mount settings.
pub fn vault_kv2_config_from_env(overrides: VaultCliOverrides<'_>) -> Result<VaultConfig> {
let mount_path = match overrides
.mount_path
.map(str::to_string)
.or_else(|| get_env_opt_str(ENV_KMS_VAULT_MOUNT_PATH))
{
Some(path) => {
tracing::warn!(
"RUSTFS_KMS_VAULT_MOUNT_PATH is deprecated for the Vault KV2 backend: it never calls the Transit engine and the value is stored but unused"
);
path
}
None => default_vault_kv2_mount_path(),
};
Ok(VaultConfig {
address: vault_address_from_env(overrides.address),
auth_method: vault_auth_method_from_env(overrides.token)?,
namespace: get_env_opt_str(ENV_KMS_VAULT_NAMESPACE),
mount_path,
kv_mount: get_env_str("RUSTFS_KMS_VAULT_KV_MOUNT", "secret"),
key_path_prefix: get_env_str("RUSTFS_KMS_VAULT_KEY_PREFIX", "rustfs/kms/keys"),
tls: vault_tls_config(get_env_bool(ENV_KMS_VAULT_SKIP_TLS_VERIFY, false)),
})
}
/// Assemble the Vault Transit backend configuration from the environment.
///
/// Companion to [`vault_kv2_config_from_env`]; see there for why both entry
/// points share it.
pub fn vault_transit_config_from_env(overrides: VaultCliOverrides<'_>) -> Result<VaultTransitConfig> {
Ok(VaultTransitConfig {
address: vault_address_from_env(overrides.address),
auth_method: vault_auth_method_from_env(overrides.token)?,
namespace: get_env_opt_str(ENV_KMS_VAULT_NAMESPACE),
mount_path: overrides
.mount_path
.map(str::to_string)
.unwrap_or_else(|| get_env_str(ENV_KMS_VAULT_MOUNT_PATH, "transit")),
metadata_kv_mount: get_env_str(ENV_KMS_VAULT_TRANSIT_METADATA_KV_MOUNT, DEFAULT_VAULT_TRANSIT_METADATA_KV_MOUNT),
metadata_key_prefix: get_env_str(ENV_KMS_VAULT_TRANSIT_METADATA_PREFIX, DEFAULT_VAULT_TRANSIT_METADATA_KEY_PREFIX),
tls: vault_tls_config(get_env_bool(ENV_KMS_VAULT_SKIP_TLS_VERIFY, false)),
})
}
fn vault_address_from_env(override_value: Option<&str>) -> String {
override_value
.map(str::to_string)
.unwrap_or_else(|| get_env_str(ENV_KMS_VAULT_ADDRESS, "http://localhost:8200"))
}
/// Resolve the Vault auth method from environment variables.
///
/// Setting `RUSTFS_KMS_VAULT_APPROLE_ROLE_ID` selects AppRole authentication;
@@ -1153,27 +1301,59 @@ fn is_under_temp_dir(path: &Path) -> bool {
/// (re-read on every login, mirroring the `RUSTFS_KMS_STATIC_SECRET_KEY_FILE`
/// precedent) or inline from `RUSTFS_KMS_VAULT_APPROLE_SECRET_ID`, with the
/// file taking precedence. Without a role id the legacy token flow applies.
fn vault_auth_method_from_env() -> Result<VaultAuthMethod> {
///
/// `RUSTFS_KMS_VAULT_KUBERNETES_ROLE` selects Kubernetes authentication, which
/// presents the pod's projected ServiceAccount token.
///
/// `token_override` carries a token supplied on the command line; it stands in
/// for `RUSTFS_KMS_VAULT_TOKEN` everywhere below, including the conflict checks,
/// so a flag and the variable it mirrors select the same method.
fn vault_auth_method_from_env(token_override: Option<&str>) -> Result<VaultAuthMethod> {
let token = token_override
.map(str::to_string)
.or_else(|| get_env_opt_str(ENV_KMS_VAULT_TOKEN));
let role_id = get_env_opt_str(ENV_KMS_VAULT_APPROLE_ROLE_ID);
let kubernetes_role = get_env_opt_str(ENV_KMS_VAULT_KUBERNETES_ROLE);
if let Some(token_file) = get_env_opt_str(ENV_KMS_VAULT_TOKEN_FILE) {
// A token file names one authoritative credential source; combining it
// with another one would leave the effective identity ambiguous, so
// that is a configuration error rather than a precedence rule.
if get_env_opt_str(ENV_KMS_VAULT_APPROLE_ROLE_ID).is_some() {
return Err(KmsError::configuration_error(format!(
"{ENV_KMS_VAULT_TOKEN_FILE} cannot be combined with {ENV_KMS_VAULT_APPROLE_ROLE_ID}; configure exactly one Vault auth method"
)));
}
if get_env_opt_str("RUSTFS_KMS_VAULT_TOKEN").is_some() {
return Err(KmsError::configuration_error(format!(
"{ENV_KMS_VAULT_TOKEN_FILE} cannot be combined with RUSTFS_KMS_VAULT_TOKEN; configure exactly one Vault auth method"
)));
for (name, configured) in [
(ENV_KMS_VAULT_APPROLE_ROLE_ID, role_id.is_some()),
(ENV_KMS_VAULT_KUBERNETES_ROLE, kubernetes_role.is_some()),
(ENV_KMS_VAULT_TOKEN, token.is_some()),
] {
if configured {
return Err(KmsError::configuration_error(format!(
"{ENV_KMS_VAULT_TOKEN_FILE} cannot be combined with {name}; configure exactly one Vault auth method"
)));
}
}
return Ok(VaultAuthMethod::token_file(PathBuf::from(token_file)));
}
let Some(role_id) = get_env_opt_str(ENV_KMS_VAULT_APPROLE_ROLE_ID) else {
if let Some(role) = kubernetes_role {
// Unlike a leftover static token, a second login method is never a
// stale remnant: both were configured deliberately and neither can be
// ranked over the other.
if role_id.is_some() {
return Err(KmsError::configuration_error(format!(
"{ENV_KMS_VAULT_KUBERNETES_ROLE} cannot be combined with {ENV_KMS_VAULT_APPROLE_ROLE_ID}; configure exactly one Vault auth method"
)));
}
return Ok(VaultAuthMethod::Kubernetes {
role,
mount: get_env_str(ENV_KMS_VAULT_KUBERNETES_MOUNT, DEFAULT_VAULT_KUBERNETES_MOUNT),
jwt_path: get_env_opt_str(ENV_KMS_VAULT_KUBERNETES_JWT_PATH)
.map_or_else(default_vault_kubernetes_jwt_path, PathBuf::from),
refresh_safety_window_secs: None,
});
}
let Some(role_id) = role_id else {
return Ok(VaultAuthMethod::Token {
token: get_env_str("RUSTFS_KMS_VAULT_TOKEN", "dev-token"),
token: token.unwrap_or_else(|| "dev-token".to_string()),
});
};
@@ -1217,6 +1397,22 @@ fn validate_vault_auth_method(backend_name: &str, auth_method: &VaultAuthMethod)
}
Ok(())
}
VaultAuthMethod::Kubernetes {
role, mount, jwt_path, ..
} => {
if role.is_empty() {
return Err(KmsError::configuration_error(format!("{backend_name} Kubernetes role cannot be empty")));
}
if mount.is_empty() {
return Err(KmsError::configuration_error(format!("{backend_name} Kubernetes mount cannot be empty")));
}
if jwt_path.as_os_str().is_empty() {
return Err(KmsError::configuration_error(format!(
"{backend_name} Kubernetes ServiceAccount token path cannot be empty"
)));
}
Ok(())
}
VaultAuthMethod::TokenFile {
path,
poll_interval_secs,
@@ -1920,6 +2116,134 @@ mod tests {
.expect("well-formed token file auth must validate");
}
/// A Kubernetes role alone configures the method: the credential is the
/// pod's projected ServiceAccount token, so nothing secret is in the
/// environment and the mount and token path fall back to the cluster
/// defaults.
#[test]
fn test_from_env_selects_kubernetes() {
with_vars(
vec![
("RUSTFS_KMS_BACKEND", Some("vault-transit")),
(ENV_KMS_VAULT_ADDRESS, Some("https://vault.example.com")),
(ENV_KMS_VAULT_KUBERNETES_ROLE, Some("rustfs")),
(ENV_KMS_VAULT_KUBERNETES_MOUNT, None),
(ENV_KMS_VAULT_KUBERNETES_JWT_PATH, None),
(ENV_KMS_VAULT_TOKEN, None),
(ENV_KMS_VAULT_TOKEN_FILE, None),
(ENV_KMS_VAULT_APPROLE_ROLE_ID, None),
],
|| {
let config = KmsConfig::from_env().expect("kms config should load from env");
let vault = config.vault_transit_config().expect("vault transit backend config");
let VaultAuthMethod::Kubernetes {
role,
mount,
jwt_path,
refresh_safety_window_secs,
} = &vault.auth_method
else {
panic!(
"a kubernetes role in the environment must select Kubernetes auth, got {:?}",
vault.auth_method
);
};
assert_eq!(role, "rustfs");
assert_eq!(mount, DEFAULT_VAULT_KUBERNETES_MOUNT);
assert_eq!(jwt_path, Path::new(DEFAULT_VAULT_KUBERNETES_JWT_PATH));
assert_eq!(refresh_safety_window_secs, &None);
},
);
}
#[test]
fn test_from_env_kubernetes_is_mutually_exclusive_with_other_auth() {
with_vars(
vec![
("RUSTFS_KMS_BACKEND", Some("vault-transit")),
(ENV_KMS_VAULT_KUBERNETES_ROLE, Some("rustfs")),
(ENV_KMS_VAULT_APPROLE_ROLE_ID, Some("env-role-id")),
(ENV_KMS_VAULT_TOKEN, None),
(ENV_KMS_VAULT_TOKEN_FILE, None),
],
|| {
let error = KmsConfig::from_env().expect_err("kubernetes combined with approle must be rejected");
assert!(error.to_string().contains(ENV_KMS_VAULT_KUBERNETES_ROLE));
assert!(error.to_string().contains(ENV_KMS_VAULT_APPROLE_ROLE_ID));
},
);
}
#[test]
fn test_validate_rejects_bad_kubernetes_settings() {
let vault_config = |auth_method: VaultAuthMethod| KmsConfig {
backend: KmsBackend::VaultTransit,
backend_config: BackendConfig::VaultTransit(Box::new(VaultTransitConfig {
address: "https://vault.example.com:8200".to_string(),
auth_method,
..Default::default()
})),
..Default::default()
};
let error = vault_config(VaultAuthMethod::kubernetes(String::new()))
.validate()
.expect_err("an empty kubernetes role must be rejected");
assert!(error.to_string().contains("role"), "got {error}");
let error = vault_config(VaultAuthMethod::Kubernetes {
role: "rustfs".to_string(),
mount: String::new(),
jwt_path: PathBuf::from(DEFAULT_VAULT_KUBERNETES_JWT_PATH),
refresh_safety_window_secs: None,
})
.validate()
.expect_err("an empty kubernetes mount must be rejected");
assert!(error.to_string().contains("mount"), "got {error}");
let error = vault_config(VaultAuthMethod::Kubernetes {
role: "rustfs".to_string(),
mount: DEFAULT_VAULT_KUBERNETES_MOUNT.to_string(),
jwt_path: PathBuf::new(),
refresh_safety_window_secs: None,
})
.validate()
.expect_err("an empty ServiceAccount token path must be rejected");
assert!(error.to_string().contains("token path"), "got {error}");
vault_config(VaultAuthMethod::kubernetes("rustfs".to_string()))
.validate()
.expect("well-formed kubernetes auth must validate");
}
/// Every KV2 read, write and listing is routed through `kv_mount`, so an
/// empty one names a path no Vault engine answers. The Transit backend
/// already rejects its own empty mounts; this closes the same gap on the
/// setting whose absence otherwise surfaces as an unroutable-path failure at
/// the first Vault call.
#[test]
fn test_validate_rejects_an_empty_kv2_mount() {
let kv2_config = |kv_mount: &str| KmsConfig {
backend: KmsBackend::VaultKv2,
backend_config: BackendConfig::VaultKv2(Box::new(VaultConfig {
address: "https://vault.example.com:8200".to_string(),
auth_method: VaultAuthMethod::Token {
token: "a-real-token".to_string(),
},
kv_mount: kv_mount.to_string(),
..Default::default()
})),
..Default::default()
};
let error = kv2_config("")
.validate()
.expect_err("an empty KV2 mount must be rejected as a configuration error");
assert!(error.to_string().contains("mount"), "got {error}");
kv2_config("secret").validate().expect("a named KV2 mount must validate");
}
#[test]
fn test_approle_config_deserializes_legacy_shape_with_defaults() {
// Persisted configurations from before the AppRole implementation only
@@ -1979,6 +2303,58 @@ mod tests {
});
}
#[test]
fn persisted_config_unknown_fields_remain_readable_and_are_observed() {
// Unknown fields in a persisted config are deliberately tolerated (a
// rolling upgrade writes fields the previous build does not know), but
// tolerated must not mean invisible (rustfs/backlog#1641): the
// observing loader counts and warns, naming only the field path —
// never the value, which can sit next to a secret. Coverage includes a
// field nested inside the backend variant, which the externally tagged
// enum exposes to the observer.
let mut value = serde_json::to_value(KmsConfig::default()).expect("serialize config");
value.as_object_mut().expect("config serializes to an object").insert(
"top_level_field_from_the_future".to_string(),
serde_json::json!("top-level value must not be logged"),
);
value
.pointer_mut("/backend_config/Local")
.expect("default config has a Local backend section")
.as_object_mut()
.expect("Local backend section is an object")
.insert(
"nested_field_from_the_future".to_string(),
serde_json::json!("nested value must not be logged"),
);
let data = serde_json::to_vec(&value).expect("encode config");
let logs = crate::test_support::CapturedLogs::default();
let subscriber = tracing_subscriber::fmt()
.with_ansi(false)
.with_max_level(tracing::Level::WARN)
.with_writer(logs.clone())
.finish();
let dispatch = tracing::Dispatch::new(subscriber);
let recorder = metrics_util::debugging::DebuggingRecorder::new();
let config = metrics::with_local_recorder(&recorder, || {
tracing::dispatcher::with_default(&dispatch, || {
kms_config_from_persisted_json(&data).expect("unknown fields must remain readable")
})
});
assert!(matches!(config.backend_config, BackendConfig::Local(_)));
assert_eq!(crate::test_support::unknown_field_metric(&recorder, "kms-config"), 2);
let output = logs.output();
assert!(output.contains("persisted KMS configuration contains unknown fields"), "got: {output}");
assert!(!output.contains("must not be logged"));
// A clean config observes nothing and logs nothing.
let clean = serde_json::to_vec(&KmsConfig::default()).expect("encode clean config");
let recorder = metrics_util::debugging::DebuggingRecorder::new();
metrics::with_local_recorder(&recorder, || kms_config_from_persisted_json(&clean).expect("clean config must parse"));
assert_eq!(crate::test_support::unknown_field_metric(&recorder, "kms-config"), 0);
}
#[test]
fn test_validate_rejects_incomplete_approle() {
let mut config = KmsConfig::vault_approle(

Some files were not shown because too many files have changed in this diff Show More