Compare commits

..

84 Commits

Author SHA1 Message Date
overtrue 569b946454 fix(startup): skip update checks when client initialization fails 2026-11-18 04:58:53 +08:00
overtrue 75b3dce2e0 fix(test): preserve parity parameter identifiers 2026-11-18 03:12:19 +08:00
overtrue 9df834a955 feat(gateway): bind preflighted POST archive replay 2026-11-18 03:10:37 +08:00
overtrue 969191ebe6 fix(s3): refresh archive member tag conditions 2026-11-18 01:12:50 +08:00
overtrue 17f1164581 test(gateway): reject incomplete declared POST forms 2026-11-18 01:06:03 +08:00
overtrue 6b60b64b81 fix(s3): enforce archive member policy overrides 2026-11-18 01:04:20 +08:00
overtrue 3ab62090c2 fix(gateway): fail closed on unbound POST archives 2026-11-18 00:32:50 +08:00
overtrue 1c4a7c2f8b feat(gateway): serve browser object uploads 2026-11-18 00:32:50 +08:00
overtrue dc83d478f8 fix(test): preserve colons in pytest parameter IDs 2026-11-18 00:29:35 +08:00
overtrue a18231a2b3 build(gateway): pin verified POST policy support 2026-11-18 00:25:06 +08:00
overtrue 0717dc1b0d fix(gateway): honor configured unread body timeout 2026-11-18 00:18:06 +08:00
overtrue a68e299baa fix(s3): enforce copy source bucket owner 2026-11-17 23:58:32 +08:00
Hauser Zhi f2003af73d fix(ecstore): recover deletes on full disks (#9465)
* fix(ecstore): reserve emergency space for delete journals

Keep bounded physical metadata headroom for recoverable deletion on Unix without bypassing quorum or rollback.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(ecstore): recover delete metadata space on demand

Share a pinned reserve owner per physical disk and keep healthy deletion free of reserve filesystem probes. Retry only disposable staging and idempotent metadata primitives after actual ENOSPC, preserving transaction publication and quorum boundaries.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(ecstore): retry concurrent full-disk deletes

Serialize ENOSPC waiters per physical disk and give each safe metadata primitive one bounded retry against shared released capacity.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ci): account for delete reserves in receipt fixture

Reject undersized tmpfs mounts after including the eight per-disk delete reserves and keep a self-test for the capacity boundary.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(heal): refresh resume admission fixture lease

Refresh the existing node admission sample immediately before starting resume GC so the cancellation test reaches the native-open ownership boundary.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): inject receipt ENOSPC on hardlink path

Keep the receipt fault at the atomic allocation boundary after the stage-two hardlink fast path landed, so reserve recovery remains covered.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-17 07:53:38 +00:00
overtrue f47840b3c2 feat(gateway): serve native UploadPartCopy
Resolve multipart ownership from storage metadata and share authorization across both request edges. Preserve the existing legacy adapter while binding gateway claims to the recorded namespace.
2026-11-17 11:00:34 +08:00
Hauser Zhi 7fb9b3501f fix(ecstore): allow missing deletes at degraded quorum (#9478) 2026-11-17 09:28:48 +08:00
Hauser Zhi cbe5b16758 test(rpc): verify timeout boundary evidence (#9482) 2026-11-17 09:28:21 +08:00
GatewayJ a33fa8a226 fix(rpc): retain cancelled streams and repair CI validation (#9439)
* fix(build): disable cuobject defaults at the workspace level

* fix(rpc): retain cancelled streams during metadata read bursts

* fix(deps): use published NIXL bindings with scoped advisory policy

* fix(test): size multipart refusal fixture threads for real storage

* fix(release): wait for uploaded artifact metadata visibility

---------

Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-16 16:22:06 +00:00
Ramakrishna Chilaka bfff795635 perf(ecstore): defer flat inline preflight and lazy-pin commit dirs (#9459)
* perf(ecstore): defer flat inline preflight and lazy-pin commit dirs

Eliminate redundant directory opens, ancestor prefix probes, and metadata stats on the Linux local-disk object commit path:
- Short-circuit validate_existing_local_disk_prefix_at on ENOENT for normalized relative paths under RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS.
- Defer source directory pinning and new-destination fstat in RenameCommitGuard on flat inline commits.
- Defer destination volume preflight to the commit guard preparation step on flat inline commits while preserving VolumeNotFound/VolumeAccessDenied preflight rejection proofs.
- Stage rollback backups by linking the held predecessor file descriptor via /proc/self/fd/<fd> and extend hardlinked rollback staging to non-inline overwrites.

* fix(ecstore): sync linked predecessor metadata when sync_backup is enabled

---------

Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-16 15:53:23 +00:00
overtrue fb104ab2c1 feat(gateway): serve extension catalog on the native admin stack
Wire both admin aliases through AdminBridge and the existing owned ServerInfo authorization. Keep the business handler and gateway pin unchanged; add no API or instance endpoint.

Add three test entry points covering eight allowed and 80 refusal/fallback rows, both hosts and aliases, complete JSON/XML, response headers, and unread unsigned bodies with signed controls. Production diff: +12/-3 lines.

Baseline and mutation base: 94a138b994. The unmodified native-selection test failed as expected. One compiled cohort passed eight controls, then eight faults failed their intended assertions: authorization bypass, native selection fallback, JSON, content type, Content-Length, refusal XML, premature body polling, and premature handler entry.

Restored sources were hash-checked and actually recompiled on 0278e1cd0e: 31 scoped tests, fmt, both RustFS Clippy modes, script-tests and test wiring passed. Existing snapshot checks now accumulate wire failures before asserting.

Not verified: CI, socket end-to-end or the full workspace suite.

Validation: cargo fmt --all --check; both rustfs all-target Clippy modes with -D warnings; 31 scoped restored tests; make script-tests; test wiring. No CI, socket-level end-to-end run, or full-workspace suite was run.

One compiled fault cohort first passed eight controls. Authorization bypass, native-route fallback, corrupted JSON, content type, content length and refusal XML each failed native_admin_extensions_catalog_real_signature_iam_and_wire; route fallback also failed native_admin_selection_is_exact_and_host_independent. Premature body polling and handler entry each failed native_admin_extensions_catalog_unsigned_before_body. Temporary instrumentation was removed, exact hashes restored and touched, and the product actually recompiled.

Restored test and lint base: 0278e1cd0e. Existing snapshot wire assertions remain enforced through the shared failure collector.

Integration verification base: 913edbc536 with the Iceberg config increment retained. The final dialect registers ten operations and four claims. Twenty-six selected tests passed: fourteen native admin HTTP tests, four mounted admin fixture tests, seven original extension handler tests and the exact edge selector. Both all-target RustFS Clippy modes, formatting, script-tests, current test wiring and prepush guards passed. Source hashes matched after verification. No new assertions or business behavior were introduced during integration.
2026-11-17 05:43:38 +08:00
overtrue 251eee0cba feat(gateway): serve Iceberg catalog config natively
Route both config GET aliases through GetCatalogConfigHandler. Keep its owned IAM, session-claim and warehouse handling, with the shared Iceberg JSON error outlet. Only these exact GET paths use the native bridge.

The matrix covers 12 allowed and 52 refused rows across both aliases and hosts, including full JSON, temporary credentials, query ordering and observed handler entry. Pin early duplicate-query rejection and the native anonymous message explicitly. The legacy baseline ran on b84985286.

Seven faults made the IAM, table-token, warehouse-prefix, alias-selection, handler-counter, JSON-outlet and complete anonymous-message assertions fail. Each had a passing control with the same compiled source. Restore hashes and a real restored build were checked.

Validation on 0278e1cd0: 16 scoped tests, cargo fmt and RustFS Clippy with and without gateway-stack passed. make script-tests passed with unchanged scripts. Production diff: +18/-9, excluding test-only probes.

No catalog storage, credential vending, catalog commits or other Iceberg operations were migrated. Full workspace, TCP/external SDK e2e and CI were not run.
2026-11-17 05:43:38 +08:00
overtrue 676875576f refactor(protocols): own UploadPartCopy client boundary
Route the protocol client and legacy adapter through one owned multipart
copy use case, preserving storage ownership, fallback context, completion
accounting, conditional order, XML ETags, and error messages and causes.
Share copy-source validation and retain the original 1024-byte key limit.

Validation: original-main fixtures, 23 focused runtime tests, 38 effective
behavior mutations, restored-source runtime, default and gateway-stack
all-target Clippy, script tests, and final integration check and prepush.
2026-11-17 05:40:29 +08:00
hector a54e2ea386 feat(runtime): raise default thread stack size to 2 MiB (#9452)
Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-16 15:35:37 +00:00
Pernav Jain 913edbc536 fix(s3): return NoSuchVersion for missing Object Lock versions (#9468)
fix(s3): return NoSuchVersion for lock reads

Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-16 11:09:55 +00:00
Ramakrishna Chilaka 909b7c3c6f perf(ecstore): hardlink stage-2 delete receipts and pin live proc roots (#9470)
Reduce syscall and sync overhead on local-disk DELETE and PUT paths:
- Hardlink the Stage 1 delete-prepared regular file to delete-quorum-committed on the cached Stage 2 local publish path when Stage 1 synced the journal, avoiding a duplicate temp-file create, write, fdatasync, and rename per drive while retaining the staged wrapper fallback on conflict or durability escalation.
- Accept bare DeleteRecoveryJournal payloads as unfinished (finished = false) commit receipts in parse_commit_receipt while keeping delete-prepared validation strictly DeleteRecoveryJournal.
- Reuse validated io_get_bucket_and_object_path components when constructing the root-relative metadata lease key in delete_version_inner on Linux.
- Track live /proc/self/fd/<fd> PublicationRoot handles via Weak<File> in NATIVE_PUBLICATION_HISTORY so record_native_publication skips redundant statx calls on pinned uncollided procfs roots.
2026-11-16 11:00:11 +00:00
Hauser Zhi 571d100a27 perf(ecstore): align inline placement and batch deletion codecs (#9449)
* feat(ecstore): align inline budgets and binary batch decoding

Use a fixed 128 KiB per-shard inline budget with versioned admission and
preserve historical configuration markers and persisted object layouts.

Decode binary delete batches independently of the legacy JSON list while
retaining JSON-only requests and dual-write senders. Reject malformed
binary entries before disk mutation and preallocate the selected batch.

Cover budget boundaries, layout coexistence, MinIO metadata fixtures,
binary-only batches and JSON compatibility. Document geometry and
rollback boundaries.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): add isolated Docker inline geometry validation

Build clean-source Linux validation images from a frozen Git bundle.
Pin running image identities and binary hashes, and inspect persisted
object geometry through the existing filemeta decoder.

Provide an owned four-node Docker adapter for the existing ABBA runner.
Guard resource ownership, isolate loopback access, serialize state
mutations, and verify HEAD, full GET and Range contents.

Keep PUT content probes available through an isolated Linux mc adapter
when a host client cannot initialize. Cover ownership and failure paths
with script tests wired into the existing runner.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): reuse bounded verified inline bodies

Share the materialization contract between metadata pre-verification
and buffered inline GETs. Bound responses to 2 MiB and a limited frame
count, and reuse an already verified body within the selected snapshot.

Route proof-protected parts through the readers that bind PartProofReader.
Cover larger single-block bodies, allocation bounds and proof validation
after a shard's bitrot checksum has been recomputed.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(ecstore): support bounded multi-block inline objects

Keep the default EC block and accumulated shard budget while bounding inline frame storage and verified-body reuse. Preserve transformed and protected write admission and existing metadata formats.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): isolate inline transport comparison modes

Pin and verify both MessagePack-only gates for each Docker cell, preserve an explicit JSON compatibility cell and probe exact accumulated shard boundaries.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): cover inline rollback and mixed-reader fleets

Add reusable compatibility fixtures for null and versioned objects, cross-block ranges, unfinished multipart uploads and batch version deletion. Record coordinator identity and per-version geometry, preserve retained-data manifests and archive manifests on dataset resets.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(ecstore): unify bounded inline read and write paths

Share checked frame decoding across metadata preverification and GET, resolve write admission once, and use one checked physical shard length for inline and disk writers. Preserve proof-bound readers and erasure recovery for damaged or missing shards.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(testing): persist inline compatibility mutation receipts

Save successful multipart and delete receipts before fallible content and geometry checks. Resume completed mutations through verification without repeating writes, and cover read failures, geometry failures, and command retries.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(gateway): isolate capability probes from multipart listings

Give successful capability probes their own buckets so CreateMultipartUpload does not change the exact two-upload pagination fixtures. Preserve all listing-boundary assertions on both HTTP stacks.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(rpc): honor fleet-confirmed codecs for batch deletion

Build DeleteVersions requests with one codec policy for the complete batch. Emit JSON compatibility fields by default and binary-only batches after both existing fleet gates pass. Cover request fields, canonical binding and actual remote sender, handler and disk deletion in both modes.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(ecstore): verify inline shards through borrowed frames

Share borrowed frame verification across buffered GET, bitrot readers and deep inline healing. Validate exact physical framing before scanning memory slices. Cover short prefixes, legacy padded frames, damage and trailing data, and provide a reproducible manual ABBA kernel comparison.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ci): account for manual inline verification benchmark

Raise the ECStore ignored-test ceiling from 2 to 3 for the opt-in
ABBA comparison of borrowed and streaming bitrot verification.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): reject inexact inline shard framing

Validate the complete encoded length of every selected inline data shard
before decoding or caching a verified object body. Return to the erasure
reader on missing, truncated, or trailing payload bytes.

Extend decoder fixtures across single and multi-block geometry, legacy
padding, unhashed payloads, and explicit bitrot verification bypass. Cover
metadata early-stop rejection and GET recovery for trailing shard bytes.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(ecstore): share inline shard reader construction

Build owned-metadata and borrowed-metadata readers through one Send iterator-based factory. Preserve shard ordering, empty slots, bitrot settings and caller-bound integrity verification.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(rpc): move endpoint buffers into batch delete requests

Transfer the prepared endpoint String into the protobuf request and assert allocation identity across every batch codec mode. This removes the intermediate endpoint copy during request construction.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(gateway): remove superseded pagination isolation

Use the upstream capability probes that clean up their own multipart sessions. Run probes against the seeded pagination bucket while keeping all exact page and cursor assertions.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(table-catalog): recognize borrowed multipart guard calls

Count owned and borrowed bucket/key guard forms while requiring all four production calls. Cover mixed split-file calls and reject wrong resources, missing error propagation and test-only mentions.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): size heal cleanup fixtures above inline budgets

Use an environment-independent per-pool fixture policy and payload sizes above the aggregate inline budget. Assert external placement before pausing old-data cleanup, rebuilding the committed generation and checking physical shards and the final payload.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(table-catalog): count borrowed object metadata guards

Recognize the borrowed bucket/key guards in migrated legal-hold and retention entrypoints. Validate two object and two key guards and cover incorrect resources, missing propagation and test-only mentions.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-16 10:11:19 +00:00
unixglo e4ac38a011 fix(s3-client): decode URL-encoded listing names (#9448)
Signed-off-by: unixglo <unixunit@outlook.com>
Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-16 09:31:08 +00:00
Hauser Zhi 849cdd6319 feat(ecstore): stage quorum-proof metadata cache (#9454)
* feat(lock): carry cache fence quorum proofs

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(ecstore): stage proof-bound distributed cache

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* docs(ecstore): refine cache activation blockers

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-11-16 08:38:54 +00:00
Hauser Zhi dc7f4e943c fix(ecstore): unify commit durability and reduce inline dispatches (#9451)
* fix(ecstore): honor file-sync policy in inline fallback commits

Apply metadata, directory, and recovery synchronization policies separately while retaining rollback publication barriers. Add sync-budget regressions and small-object I/O profiling guidance.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(ecstore): unify recovery-aware commit synchronization

Share recovery-directory policy across fused and fallback commits. Complete fresh file-sync publication within preparation and verify queue budgets, rollback recovery, and inline-to-external overwrites.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): isolate sync counts across reused disk handles

Consume the metadata synchronization count for every external write attempt before releasing a Linux procfs-backed fixture, including failed publication and retry.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-16 08:06:58 +00:00
GatewayJ 0278e1cd0e fix(iceberg): recover renames and unblock namespace deletion (#9440)
* fix(iceberg): recover renames and unblock namespace deletion

* fix(iceberg): fence rename recovery with restore lease

---------

Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-16 23:54:03 +08:00
overtrue 94a138b994 feat(gateway): serve cluster snapshots on the native admin stack
Register GET /{rustfs,minio}/admin/v4/cluster/snapshot with the existing owned handler and ServerInfo authorization. The read-only use case is unchanged.

Reuse the admin fixture for full JSON and refusal XML comparisons, response headers, exact route selection, and unsigned rejection before reading a body. The native matrix covers 8 allowed and 80 refusal or fallback rows; the body observer adds four signed controls and four unsigned refusals. Update the existing registration census.

Validation on b84985286: 35 scoped tests, RustFS Clippy with and without gateway-stack, cargo fmt, and make script-tests passed. Full workspace, distributed e2e, and CI were not run.

Mutations: IAM bypass, route removal, JSON replacement, and event-time corruption turned the signature/IAM/wire contract red. Reading the unsigned body turned the body observer red. Content-Type, Content-Length, and an extra XML Resource node each turned the corresponding wire assertion red. A temporary entry counter observed signed calls and no unsigned handler entry.

Production diff: +12/-2. No new public API or business logic.
2026-11-16 23:18:03 +08:00
Hauser Zhi b3cdea0c15 fix(targets): decode Kafka notification record keys (#9450)
* fix(targets): decode Kafka notification record keys

Build Kafka records from the decoded object identity so record keys match
the notification envelope for immediate delivery and persisted replay.
Cover form encoding boundaries, historical queue round trips, and early
rejection of invalid UTF-8 metadata.

Fixes #9437

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(targets): share allocation-efficient event key construction

Use a borrowed form decoder and exact-capacity key buffer for notification
envelopes and Kafka records. Reuse the established percent decoder and
preserve the owned public decoder API and persisted replay metadata.

Add exhaustive malformed-escape and UTF-8 compatibility checks, plus a
regression for borrowing plain object names.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-16 07:49:17 +00:00
overtrue b849852863 refactor(protocols): call owned UploadPart use case
Route the protocol client through its installed multipart context and owned
values. Preserve request headers, response and error projection, counting,
and audit completion. UploadId keeps one scalar compatibility projection.
UploadPartCopy, PostObject, gateway handlers and ownership rules are unchanged.

On f68ef7d5f: fmt, both all-targets Clippy modes, 37 scoped tests, and script-tests
passed. The 14-fault cohort had a green 37-test control; restored sources were
hash-checked, touched and recompiled before the final 37-test pass. On 489316e7a:
gateway-stack all-targets check and prepush-quick passed after rebase.

New runtime cases: 13 negative, 2 positive. Projection matrices cover 60 input,
64 output and 6 error rows. Non-test lines: +122/-20. No existing test was
deleted, weakened or ignored.

Effective mutations and failing tests (under protocols::client::multipart::
tests::upload_part_tests):
- part-number, length, md5, sse, eager-body, metric, audit:
  upload_part_refusals_match_original_without_persisting_parts
- checksum: upload_part_input_projection_matches_original_fields
- output-field, etag-order:
  upload_part_output_projection_matches_original_fields_and_refusals
- error-default, error-explicit:
  upload_part_error_projection_matches_original_defaults
- ambient, observer: upload_part_client_preserves_stored_bytes_and_bound_context

Not run: full workspace, protocol wire lanes, released-binary differential,
live KMS/SSE integration, or CI.
2026-11-16 21:40:00 +08:00
overtrue 489316e7a2 refactor(protocol): dispatch CopyObject through the owned usecase
Replace the protocol client compatibility dispatch with owned request and
response projections while preserving source validation, conditions, dates,
ETags, error answers, bound storage, fallback behavior, and operation counts.

Validated against the existing client and library behavior on f68ef7d5.
Final gateway-feature tests passed 11/11, including existing PutObject
regressions; default-mode tests passed 6/6. Both all-target Clippy modes,
script tests, wiring checks, and static preflight passed.

Twenty-two effective mutations covered observation controls, operation
counts, context selection, storage effects, source and condition parsing,
optional fields, dates, output fields, error facts, and request credentials.
Restored sources were rebuilt before the final passing controls.

This does not register UploadPartCopy or change the default S3 stack.
External migration lanes and release-reference testing remain outstanding.
2026-11-16 21:06:46 +08:00
overtrue f68ef7d5f6 refactor(protocols): use owned multipart lifecycle paths
Route CreateMultipartUpload, CompleteMultipartUpload and AbortMultipartUpload through the installed server's multipart use case. Keep the measured protocol projections and operation counts.

The upload-id scalar compatibility helper remains in Complete and Abort. UploadPart, UploadPartCopy, PostObject and persisted-owner work are outside this change.

Validation: 11 new tests; the runtime contract has 8 refusal cases and 2 successful flows. The restored selection passed 59/59 on d3abf8023. Both Clippy modes, fmt, script-tests, test wiring and prepush-quick passed. Non-test diff: +231/-42. The original client baseline ran on a98bd9028.

Effective mutations (38), grouped by the test that failed:
- multipart_create_projection_matches_original_fields_and_times: create-input-name, create-input-metadata, create-input-tagging, create-input-storage, create-input-sse, create-input-kms, create-input-md5, create-input-mode, create-input-hold, create-input-date, timestamp.
- multipart_refusals_match_original_client_errors: complete-error-message.
- complete_input_preserves_part_fields: legacy-part-checksum.
- multipart_create_projection_preserves_refusals: create-input-refusal.
- multipart_create_output_matches_original_for_stored_names_and_empty_values: create-output-name, create-output-strings.
- multipart_complete_projection_matches_original_parts_and_conditions: complete-condition, complete-parts, complete-part-number, complete-part-etag, complete-part-checksum, complete-condition-refusal.
- multipart_completion_projection_preserves_original_escaped_tags: complete-output-escape, complete-output-weak.
- multipart_completion_projection_preserves_header_only_field_behavior: complete-output-headerdrop, complete-output-header-validation.
- protocol_multipart_lifecycle_stores_parts_and_aborts: count-create, count-complete, count-abort, complete-location, complete-storage.
- multipart_uses_bound_store_with_a_foreign_ambient_context: create-storage, abort-storage, bound-store.
- multipart_envelopes_preserve_original_request_facts: request-facts.
- protocol_policy_denials_and_request_conditions: permission-create, permission-complete, permission-abort.

The final restored run had no leaky summary; an earlier control marked one pure projection test leaky. The linker reported the non-failing __eh_frame warning. The ambient fault's test exited 100 with target=false/ambient=true; its wrapper exited 1 because it expected generic assertion wording.

Not verified: CI, network protocol suites and the released-reference comparison.
2026-11-16 19:09:59 +08:00
overtrue d3abf80237 refactor(protocols): call owned CreateBucket use case
Call the existing bucket use case directly from the protocol client. Preserve
bucket-name materialization, response facts, edge counts and bound or fallback
store selection. Keep legacy DTO imports confined to tests.

Tests: one new runtime matrix, with seven rejection and four success rows,
measured first against main at 70022e51c. Restored gateway bucket tests passed 4/4.
The old dispatch failed the source wiring guard; owned dispatch passed it.

Mutation checks: normal controls passed; the runtime matrix rejected storage
bypass, missing fallback, metric loss, ambient context selection, strict names,
and changed error code, message, status or headers. Isolated storage, fallback
and ambient-context faults retained normal operation counts. Restored sources
were touched and actually recompiled before the runtime pass.

Validation: fmt, package Clippy in both feature modes, script-tests and test
wiring. Production diff: +19/-14 lines. No new API or storage use case.
Not verified: CI, full workspace tests or live protocol end-to-end tests.
2026-11-16 18:17:06 +08:00
overtrue 70022e51c4 refactor(protocols): call owned bucket use cases for listings
Use the bucket use case for ListObjectsV2 and both bucket listing entry points. Keep entry encoding, timestamps, ETags and refreshed session facts while removing three legacy FS calls. Other protocol operations are outside this increment. Non-test diff: +102/-82.

Added six tests: projection rows have 34 refusals and six accepted values; runtime cases have five refusals and eight successes, plus bucket exclusions and four credential rows. Existing assertions were retained.

Validation: baseline 2fea17d57 (2/2); restored build on a98bd9028 (15/15); all-target Clippy default and gateway-stack; fmt, script-tests, test wiring and prepush-quick. Rebased onto 88515bb7d with no task-file or pin overlap; all task blobs match the tested source.

Eight independent controls and 31 intended mutation failures:
- protocol_listing_output_matches_legacy_projection: strict Name, oversized key, invalid/missing ETag, XML escaping, weak tag, prefix, marker, encoding, timestamp.
- protocol_listing_bucket_projection_preserves_strict_names_and_milliseconds: bucket filtering, timestamp.
- protocol_listing_pages_and_encoding: cursor, count, encoding.
- protocol_listing_refusals_and_limits: empty/unknown credentials, counts.
- protocol_listing_bucket_context_and_defaults: count, bound context.
- protocol_listing_session_credential_projection_matches_legacy: token, secret.
- protocol_session_bucket_listing_revalidates_the_session_key: request context, IP, secure transport, headers, refresh, accepted count, refusal count, store context.
- request_extensions_preserve_authenticated_identity_and_source_ip: owner.

Experiments were removed, hashes checked and the restored product actually recompiled. Full protocol listing E2E and CI were not run.
2026-11-16 16:58:48 +08:00
overtrue 88515bb7d0 feat(gateway): serve admin usage and catalog reads
Register data usage, plugin catalog and module-switch GET reads through
the existing AdminBridge and handlers under both admin aliases. Keep
neighboring methods and object-lambda routing on their existing paths.

Validation on 65057691: unchanged-production legacy baseline passed;
restored-source scoped tests passed 78/78. The new native HTTP matrix
covers 28 success and 236 refusal or routing-boundary rows. Exact replies
are pinned for both aliases and ordinary and bucket Hosts.

Three independent no-fault controls and 21 effective mutations passed
in one compiled cohort: selector removal/widening/lambda, each handler
binding, each IAM gate and action, both narrowed data-usage any-of gates,
signature verification, lambda dispatch, response bytes/status/request
id/content type and Keystone precedence. Exact refusal observations also
caught the disabled-key signature mutation. No existing assertion was
removed or weakened.

Gateway and default all-targets Clippy, script tests, test wiring and the
local prepush guards passed. No gateway pin or API changed.

Not verified: production listeners and full middleware, session
credentials, non-default persisted usage/module-switch data, full
workspace tests or CI.
2026-11-16 16:43:17 +08:00
overtrue a98bd9028d refactor(protocols): use owned bucket head and delete paths
Call the existing bucket use cases directly from the protocol client. Preserve root lookups, bucket validation, request context, response facts and operation counts. Other protocol operations stay as they are.

Non-test lines: +33/-40. Added cases: 17 refusals, 16 successes; reused three HeadBucket policy refusals. Protocol tests passed 39/39 in default and gateway-stack builds. Both rustfs all-target Clippy modes and make script-tests passed.

Four no-fault controls passed; 22 faulted test runs failed as intended. Mutations covered input routing, name refusals, output and error facts, counters, deletion, context resolution and authorization allow/deny. The force-delete fault also changed the observed bucket/object state. The raw HeadBucket NUL-name projection went red before its response text was restored.

Tests and Clippy ran on 2fea17d57. Script-tests ran on 2ca689f48; their sources are unchanged. Rebased onto 65057691 with unchanged client and gateway-pin blobs. Network transport e2e, workspace tests and CI were not run.
2026-11-16 16:12:18 +08:00
overtrue 65057691b6 feat(gateway): serve signed admin system reads
Serve the storage info, runtime capabilities and object cache statistics
GET aliases through the native admin service. Call the existing full IAM
gates and owned handlers, preserve raw request facts and keep the Keystone
guard first.

Add two contract matrices (42 allowed and 168 refused rows), with expanded
selector, registry, envelope and readiness coverage. On 2fea17d57d, the
restored source passed 72 scoped tests, fmt and both rustfs all-targets
Clippy modes. Script tests and test wiring also passed.

29 effective fault runs cover selection, lambda precedence, all three
dispatch/action/authorization bindings, signature verification, source IP,
raw URI/query, request context, readiness, response bytes/status/headers/
length and the registry. Nine controls ran in separate processes against
the same binary before those faults.

Not verified: live listeners and full production middleware, session
credentials, active cache statistics, CI or the full workspace tests.
2026-11-16 15:35:13 +08:00
overtrue 2fea17d57d feat(auth): bridge Keystone identities into the gateway stack
Pair a single-use Keystone proof with sanitized owned authorization facts.
Preserve the original session-presence fact for policy principal projection,
and scope legacy credentials during both service call construction and polling.
Bind the gateway host-token principal to that owned identity without an IAM
credential or task-local fallback.

Initialize Keystone before gateway HTTP assembly and reuse the provider for
listeners and later auth integration setup. Refuse token-bearing claimed
non-S3 requests before console, admin or legacy dispatch.

Add 14 test functions. The primary ListBuckets/HeadBucket matrix has 13 rejected
and 7 allowed requests. Change enabled-Keystone startup to succeed after provider
initialization; retain both configuration tests, including disabled and invalid
settings. Production changes: +304/-77 lines, excluding test bodies.

Mutation checks (20 effective boundaries; each has a green control):
- owner, groups, username: keystone_http_tokens_authorize_real_s3_by_roles_groups_and_username
- pair-name: keystone_envelope_refuses_missing_and_mismatched_owned_identity
- provider, expiry: keystone_bridge_refuses_provider_mismatch_and_post_validation_expiry
- stale-established, stale-owned: keystone_stack_clears_both_payloads_before_a_signed_request
- legacy-before-deny, native-before-deny, admin-before-deny: gateway_keystone_tokens_are_refused_before_non_s3_legacy_dispatch
- scope: disabled_middleware_clears_identity_before_inner_call_and_poll
- principal-session: resource_free_conditions_preserve_legacy_projection_and_erased_keystone_session_facts
- secret, session, session-fact, replay, poison: request_identity_erases_secrets_and_consumes_its_proof_once
- repeat-init, missing-child: keystone_initialization_reuses_the_provider_for_concurrent_calls

Verification on 2ca689f: 214 focused tests passed after restoring and rebuilding
the production sources; all 33 Keystone package tests passed. Both package-scoped
all-target Clippy configurations, formatting, source guards, script tests and
prepush checks passed. Temporary fault code was removed.

No gateway API, dependency pin or PostObject changes. A deployed Keystone server,
the complete listener token lane and CI were not exercised locally.
2026-11-16 15:02:13 +08:00
overtrue 2ca689f486 refactor(protocols): call the owned PutObject use case
Project protocol PutObject directly into the owned write input and project the answer back without the legacy DTO path. Keep the measured normalization, refusals, error facts, server context and operation counts at the client edge.

Non-test lines: +196/-18. Eight added tests include 25 runtime cases (18 negative, 7 positive).

Verified on f4266a4576 with gateway pin 44f9a2422d745ffba952347ec91b0332ef2667a4: protocol tests 36/36 in default and gateway-stack builds; both all-target Clippy builds; script-tests; prepush-quick.

Sixteen runtime mutations failed their intended tests from one fixed binary after all eight controls passed:
- empty: protocol_put_optional_values_match_legacy
- condition, specific-none: protocol_put_condition_grammar_matches_legacy
- input-checksum: protocol_put_checksum_input_matches_legacy
- retain-nanos, retain-range: protocol_put_retain_until_matches_legacy
- output-etag, output-fields, output-checksum: protocol_put_output_matches_legacy
- error-default, error-facts: protocol_put_error_projection_matches_legacy
- event-hold, event-days, event-years, body, metrics: protocol_put_runtime_matches_legacy

The experimental code was removed and the restored implementation was recompiled and tested in both builds. The earlier incorrect metric lookup is not counted as evidence.

Scope is protocol PutObject. Network protocol E2E, the full workspace and CI were not run.
2026-11-16 14:15:53 +08:00
overtrue b2c30d69c2 feat(gateway): bridge the signed admin info aliases
Serve GET /rustfs/admin/v3/info and /minio/admin/v3/info through a
scoped native service using the existing signature verifier and the real
ServerInfoHandler IAM gate. Keep object-lambda routing ahead of these
aliases and retain the raw URI, optional ReqInfo, headers and body.
The pinned RustFS profile already handles claims on bucket-style hosts.

Outside tests and the generated lockfile: +180/-14 lines. Add seven tests,
including 32 authorization refusals and 16 authorized info responses, both
SourceIp directions and signed legacy-service comparisons. No existing
assertion was weakened. Duplicate-query signing is tested as a fixture
refusal, with routing coverage retained separately.

Validation on f4266a4576: fmt, both rustfs
Clippy modes, 42 scoped tests, script-tests and prepush-quick.

Twenty-four mutation runs failed as intended across 22 faults:
- native_admin_selection_is_exact_and_host_independent: remove/widen
  selection and lose object-lambda priority.
- native_admin_real_iam_and_legacy_contract: bypass signature verification
  or IAM, change the action or source IP, omit pools, rewrite the URI,
  or lose object-lambda priority.
- native_admin_envelope_preserves_raw_alias_host_facts_and_optional_req_info:
  rewrite the URI, drop/default ReqInfo, or drop client, peer or request ID.
- native_admin_complete_response_keeps_bytes_status_and_content_type:
  corrupt bytes, status, content type, extra headers or content length.
- native_admin_authenticated_request_refuses_an_uninstalled_server_context:
  remove the readiness check.
- native_admin_registers_one_operation_and_both_claims: register an extra op.
- native_admin_signing_fixture_refuses_duplicate_query: accept duplicate keys.
The same mutation binary passed all seven controls before fault runs;
the restored production tree was then rebuilt and tested.

Not exercised: network listeners, session credentials, the combined
Keystone change, CI or other admin endpoints.
2026-11-16 14:01:00 +08:00
Hauser Zhi fb9f19c6d2 fix(rpc): scope canary errors to grpc failures (#9453)
* fix(rpc): scope canary errors to grpc failures

Exclude expected read-version remote responses without hiding transport, gRPC status, response-decode, or other gRPC operation failures.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(rpc): verify canary revisions from image history

Use immutable image history only when recognized config labels are absent, keep mismatches fail-closed, and record the provenance source in verifier output.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(audit): restore bucket config audits and metadata guards

Audit accelerate, request-payment, and website configuration mutations through the shared owned request scope, including failures and the website deletion status.

Validate the replication mutation delegation before checking peer reload notifications. Count borrowed reserved-prefix guard calls and cover missing production calls, test-only mentions, audit completion, and stale bucket incarnations.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-11-16 13:57:56 +08:00
overtrue f4266a4576 refactor(protocols): use owned object reads and deletion
Route both GetObject calls, HeadObject and DeleteObject to the owned use cases. Keep the request facts and millisecond timestamp behavior. PutObject stays unchanged; 12 operations and 13 trait calls remain.

The client edge is an interface in the layer guard. Its public re-export is covered too, and app-to-client imports are rejected by fixtures.

Runtime wiring: +177/-101 non-test lines. The new matrix has 18 negative and 13 positive operation cases, plus the extended identity-copy control.

Validation: 28 protocol tests passed in both default and gateway-stack modes; both all-target Clippy modes passed. Script tests, layer controls and focused architecture/footprint/test guards passed. Five source faults failed at the intended claims, range, Head ETag, deletion-witness and version assertions. The precision oracles failed before the millisecond projection was added.

Runtime and Clippy ran on e9da1d309; initial controls and faults ran on 2308dee4b2. Rebase to 00ecde125 changed none of the seven task files or gateway pins. CI and HTTP transport were not checked.
2026-11-16 12:11:38 +08:00
overtrue 00ecde1250 feat(gateway): dispatch console routes through native Axum
Use the existing console router when the gateway console is enabled. Keep lambda and extension precedence, the legacy selection, and outer middleware assembly unchanged. The common body conversion stays at the edge.

Add test-only assets and one contract with 10 native and 52 excluded routing probes. Non-test source: +89/-16. The restored run passed 22 console/stack tests on e9da1d309; both Clippy modes, script-tests and prepush-quick passed.

native_console_route_contract went red for three independent mutations: remove the native branch (service-call counts), widen the shared prefix (neighboring S3 path), and route every claim to console (anonymous admin refusal). Restoring the sources triggered a real product recompile.

Full outer HTTP wire checks and a production UI bundle were not tested. Admin, STS and website migration remain separate work.
2026-11-16 11:47:40 +08:00
overtrue e9da1d3096 feat(gateway): serve DeleteObjects through the owned engine
Share the batch deletion engine between legacy and native edges with owned inputs and results. Keep exact per-entry authorization, version and retention handling, ordered errors, quiet responses, metrics, and audit aggregation. Preserve frozen resource resolution and rejected-key echoes.

On 2308dee4 with gateway 44f9a242, the focused suite passed 208 tests before and after deliberate mutations. Both rustfs Clippy configurations, script tests, test wiring, formatting, and prepush guards passed. Four added runtime tests cover mixed successes and per-entry refusals, frozen fields, and rejected-key state.

Five targeted tests failed when batch target clearing, denial error mapping, and raw-key echo were changed. Sources were restored byte for byte and product artifacts recompiled.

The legacy edge matched 62 observed baseline responses including all headers, XML and object states. Native decoder and expected-owner differences are tracked separately. Actual webhook records matched for mixed and quiet requests across the baseline and both candidate stacks.

Full workspace tests and CI were not run. Verification is scoped to this migration increment.
2026-11-16 10:24:21 +08:00
overtrue 2308dee4b2 fix(gateway): preserve object retention date offsets
Use the validated request offset and full timestamp precision when projecting PutObject and PutObjectRetention dates into authorization and storage.

Signed requests verify exact stored dates and version isolation for positive and negative offsets and UTC. Policy tests cover denied dates and the typed date supplied by PutObject authorization.

On b2b4f0c4, 175 focused tests and six raw xl.meta checks passed. The UTC-only control and five targeted mutations failed at their intended assertions. Both Clippy feature configurations passed.
2026-11-16 08:39:36 +08:00
overtrue b2b4f0c491 feat(gateway): serve SelectObjectContent natively
Share GetObject authorization and the existing Select query producer across both stacks. Carry live events through the typed response with the projected SSE headers. The producer retains its single deadline, snapshot checks, bounded channel and receiver cancellation.

On c5b4a8577, the restored build passed 207 focused tests, both RustFS Clippy feature modes, formatting, script tests and test wiring. Signed socket probes matched all 17 legacy cases and 12 gateway cases. Five gateway refusals omit the legacy default message, tracked in rustfs/backlog#2684.

The fault build passed all 98 controls. Bypassing authorization failed on both stacks. Removing receiver-close handling, dropping the terminal Records permit, skipping the Stats fence, replacing an error with End, and dropping progress or scan range each failed the corresponding test.

Verification used local stores and SSE-C. The full workspace suite and external KMS were outside the local run.
2026-11-16 07:29:41 +08:00
overtrue c5b4a85778 feat(gateway): serve RestoreObject through the owned use case
Share the authorization snapshot and restore admission between both stacks.
Preserve HTTP 200 for both success paths and emit ObjectRestorePost on new
admission only.

Verified on c0763a43 with 114 focused tests, four serialized ILM tests, and
16 signed wire responses per stack. Faults covered shared authorization,
generation capture and admission, SELECT rejection, worker parameters and
liveness, and both audit and event paths. Cluster lock loss and mixed-version
rollout remain unverified.
2026-11-16 06:30:48 +08:00
overtrue 4ae180ea40 feat(gateway): serve object legal hold and retention writes
Share each operation's authorization and metadata write with the legacy edge.
Keep their request ordering, version and bucket-incarnation fences, replication
completion, and notification suppression. Retention checks the locked metadata
again before committing a change.

The two stacks were exercised through signed HTTP requests and independent
disk metadata reads. Faults in authorization, body projection, version
selection, counting, auditing, notification suppression, scanner invalidation,
the bucket fence, and the commit-time retention check reached their assertions.
2026-11-16 06:19:43 +08:00
overtrue c0763a43f5 feat(gateway): serve bucket object lock configuration
Share the bucket Object Lock write between the legacy and gateway edges. Keep lock configuration and versioning as sequential writes against the authorized bucket incarnation, followed by peer reload, site replication and scanner invalidation.

Cover eight refusal or failure cases and three successful cases with durable metadata and signed requests. The two stacks run the same request cases and refusal matrix.

The mutation run detected both missing write fences, swallowed peer failure, propagated site failure, early or absent scanner invalidation, missing audit, wrong authorization action or resource, dropped request body and incorrect operation counting.

Validation used debug builds on d461908bf. The final lint pass elides an equivalent lifetime in a test helper. Peer and site failures were injected locally, with real multi-site delivery left unverified.
2026-11-16 05:35:18 +08:00
overtrue d461908bf0 feat(gateway): serve bucket replication configuration writes
Route PutBucketReplication and DeleteBucketReplication through the owned bucket use case. Share site checks, metadata fencing, target cleanup recovery, and audit completion with the legacy edge.

Signed requests exercise 28 refusals and six accepted edits across both stacks. The direct storage tests cover six failure paths and two successful edits. Twenty-four faults turned their expected disk, authorization, locking, callback, audit, or metric assertions red.

The restored gateway-stack build passed 132 selected tests on 75b293f09, including both HTTP stacks. Both Clippy variants and the mutation cohort passed on fb3e2818c. The fixed legacy CLI matched all six response records. The standalone default runtime suite, multi-site delivery, and CI were unverified.
2026-11-16 04:32:24 +08:00
overtrue 5b8876ad0e feat(gateway): serve bucket policy, encryption, and tagging writes
Share the owned write paths across both S3 stacks. Preserve the policy
document bytes, default KMS key resolution, bucket tag validation, and
incarnation fences before reload and replication callbacks.

Add three successful writes and three rejected stale writes on disk.
Extend the two-stack refusal matrix with six successful and 24 denied
requests. Cover the encryption and tag validation order before lookup.

The fault cohort exercised the action and resource checks, value
validation and order, raw policy bytes, default KMS readback, legacy
policy count, durable writes, incarnation fences, reload, replication,
audit, and scanner invalidation. Each selected assertion failed.

Local checks used base 75b293f09. Fixed CLI comparisons covered legacy
responses, audit entries, and operation counts. Live peer replication,
external KMS providers, and CI were outside the local checks.
2026-11-16 04:24:32 +08:00
overtrue 75b293f094 feat(gateway): serve encryption, policy, and tagging deletes
Share the three generation-fenced deletes across both S3 stacks. Keep
their existing IAM actions and use the shared IAM scope for tag deletion.

Add six focused tests covering 21 refusals and nine successes, and add the
three operations to the refusal matrix. Substituting IAM actions and
buckets, removing the tagging scope, and suppressing the gateway counter
fail the authorization and metrics checks. The storage tests detect a
lost fence, a skipped write, missing reload or replication calls, omitted
scanner invalidation, and a changed audit status.

Verified on 4294c41cc with scoped nextest, both Clippy modes, script tests,
quick checks, and local CLI comparisons. Peer network delivery was
outside the local checks.
2026-11-16 03:21:06 +08:00
overtrue fb3e2818c7 feat(gateway): serve bucket ACL and logging writes
Route PutBucketAcl and PutBucketLogging through shared bucket use cases.
Preserve the ACL response for header-only requests and the bucket lookup
before the XML policy refusal. Write logging configuration using the
authorized bucket incarnation and the existing persistence format.

Verify 111 focused tests on 4294c41cc, including the storage access suite,
and run both Clippy modes, script tests, and local wire comparisons. The
wire matrix exercises 15 refusal cases and seven successful writes on
each stack.

Change both IAM actions and resources, audit statuses, and counters in
the fault cohort. Reverse ACL lookup and XML policy checks, drop policy
presence, corrupt logging bytes, reverse logging conversion order, and
remove the incarnation guard. The regressions detect each fault, and
the stale-authorization test reads the replacement bucket from disk.
2026-11-16 03:08:10 +08:00
overtrue 4294c41ccf feat(gateway): serve bucket public access block operations
Route Get, Put, and DeletePublicAccessBlock through shared bucket use
cases. Preserve the persisted switch values, mutation incarnation fence,
reload behavior, audit status, and missing-configuration response.

Add six focused tests covering 25 refusal cases and 17 success cases
across both stacks and storage. Extend the refusal and counter matrix
for all three operations.

Substitute each IAM action and bucket, corrupt each operation's value,
change the missing-config status, and suppress each edge counter. The
round-trip and matrix tests reject those faults. Put and Delete tests
reject a missing incarnation fence, the wrong reload mode, and wrong
audit status. Read the durable replacement-bucket bytes after disabling
both fences.

Verify on 92050cdca with scoped nextest, both Clippy modes, script tests,
and local wire tests. Peer reload delivery was outside these local tests.
2026-11-16 02:10:24 +08:00
overtrue 3cb0aa137e feat(gateway): serve core bucket configuration mutations
Serve accelerate, request-payment and website writes, including website
deletion, through shared use cases. Both S3 edges enforce the bucket
policy and the metadata incarnation fence before changing stored data.

Verified on 92050cdca with formatting, Clippy on both RustFS feature
sets, script tests, 21 RustFS tests and 27 operation and metrics tests.
The fixed legacy CLI supplied 42 HTTP comparison cases per stack.

The wire assertions caught altered IAM actions and resources, missing
denial audits and operation counters, and skipped website validation.
The stale-incarnation test caught changes to the replacement bucket's
disk bytes. The lookup-order test caught early configuration conversion.

Refs rustfs/backlog#2734
2026-11-16 02:07:32 +08:00
Hauser Zhi a8cf62f8f3 fix(metrics): label grouped fdatasync batches accurately (#9446)
* fix(metrics): label grouped fdatasync batches accurately

Record the serialized user-space fdatasync coordinator as grouped and reserve parallel for independent per-file fanout. This keeps performance dashboards from conflating two different dispatch models.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): separate grouping from timeout race

Keep the same-disk batching test focused on queue aggregation. Claimed-follower timeout ownership remains covered by its dedicated deterministic regression, avoiding a valid pre-claim timeout race in the batching fixture.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(table-catalog): accept borrowed multipart guard calls

Count both owned String and borrowed &str spellings of the same reserved-prefix guard. UploadPart now obtains borrowed bucket and key values from its typed target, while the production guard remains in place.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-15 12:51:57 +00:00
Hauser Zhi d0fd6425e4 feat(rpc): guard stale generations before dispatch (#9435)
* perf(grpc): attribute rebuilds and bound bulk responses

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(grpc): attribute rebuild success after publication

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(obs): add exact HTTP/2 ping timeout diagnostics

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(multipart): run refusal matrix on large stack

Run the real-store completion refusal matrix through the shared 32 MiB test harness so its deep async storage future does not overflow libtest's default stack.

Keep the original test name and all refusal-order assertions intact.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* chore(deps): pin nixl-sys workspace source

Manage the nixl-sys Git revision from the workspace dependency table and point the optional Linux transport dependency at houseme/nixl commit cc636c83f13d692e85b67e55aed092413d16fb41.

Allow the reviewed source in cargo-deny and refresh the lockfile, replacing the unmaintained bincode dependency with the fork's wincode graph.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): authenticate msgpack peer capability

Add an optional boot-epoch-bound HMAC capability proof and cache the verified per-peer observation without enabling any msgpack send policy.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(grpc): bind peer capability to connection generations (#9433)

* feat(grpc): track physical connection generations

Observe every tonic physical reconnect through a stable custom connector while preserving existing TCP, TLS, HTTP/2, cache, and public channel behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): bind msgpack capability to connections

Scope authenticated msgpack observations to channel identity and physical connection generation so transparent reconnects require a fresh proof without enabling the send policy.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): guard stale generations before dispatch

Reject generation-bound requests before inner transport call or first payload frame when tonic readiness observes a different physical connection, while leaving ordinary mutations unguarded until the send-policy follow-up.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): add per-peer msgpack rollout gate

Add the default-off peer-capability environment gate, cached evaluation, fixed-cardinality send-decision metrics, and rollout constraints for stable peer-affine connections.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): apply per-peer msgpack send policy

Capture one capability decision per eligible RPC, bind peer-mode requests to boot and connection generations, and rebuild at most one dual-write request only after a locally proven pre-dispatch rejection.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(rpc): verify msgpack peer canary evidence

Add a strict three-phase four-node evidence verifier with raw Prometheus and workload receipts, adversarial tests, script-test wiring, and the operational manifest contract.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(rpc): align batch read manifest encoder

Match the BatchReadVersion compatibility manifest to the request-owned encoder expression used by the production send path.

Restore the owning ECStore source guard while preserving wire and runtime behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(rpc): orchestrate msgpack peer canary

Run the fixed mixed/all-new/rollback four-node matrix with prebuilt images, real workload and readback, raw Prometheus capture, strict manifest generation, and adversarial fake integration tests.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(rpc): verify canary image and telemetry identities

* test(rpc): bind canary evidence provenance

Scope canary metrics to one injected cluster identity and bind candidate, baseline, and per-phase running containers to immutable image IDs and exact build revisions.

Reject foreign-cluster metrics, stale revision labels, same-image tags, and phase image mismatches.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(table-catalog): accept borrowed multipart guard calls

Count the exact borrowed bucket/key guard call introduced by UploadPart target ownership while preserving the four-production-call contract.

Cover the borrowed spelling in both the positive fixture and the test-only false-positive regression.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
2026-11-15 12:37:25 +00:00
overtrue 92050cdca6 feat(gateway): serve PutObjectTagging through the shared use case
Share XML tag authorization and storage execution across both stacks.
Preserve the requested version, replication-target proxy, and metadata
completion order.

The dual-stack tests exercise 37 refusal and 13 success scenarios.
The restored integration run passed 148 tests on 26db9b19a, and the
proxy cleanup verification passed 134 run, 134 passed.
Both feature configurations passed Clippy, and script tests passed.

Fault injection caught native and shared authorization bypasses,
missing XML facts, and incorrect action, resource and tag key facts.
It also caught duplicate counting, incorrect persisted tags, missing
or duplicate success events, and event or dirty publication on a miss.

Live replication targets and ECStore write faults remain unverified.
2026-11-16 00:29:49 +08:00
overtrue 26db9b19af feat(gateway): serve core bucket configuration reads
Serve bucket acceleration, request payment, website, and logging
configuration through shared persisted read paths. Keep each operation's
existing authorization and legacy response projection. Append distinct
metric labels without shifting existing indexes.

The scoped RustFS, S3 operation, and metrics suite passed 283 tests on
993464fbb. The integrated configuration and refusal suites passed 21
tests on 292974baf. Both feature modes passed all-targets Clippy.

Standalone checks compared 23 successful and 17 refused legacy requests
against cfdd4b268 and exercised the gateway answers on the same data.
Action, resource, anonymous, projection, default, missing-bucket, counter,
and audit faults failed their original assertions. Metric registry tests
also detected duplicate labels, changed index order, and an added event.
Verification is scoped to these local suites and wire probes.
2026-11-15 23:29:07 +08:00
overtrue 292974baff feat(gateway): serve bucket CORS, encryption and tagging reads
Share bucket-scoped authorization and persisted configuration reads across
both stacks. Carry the stored configurations in gateway responses and
preserve the operation-specific errors for missing configs.

On RustFS 993464fbb, 120 scoped tests passed, including both refusal
matrices and six new tests with 24 refusal and 12 success cases.
Live HTTP checks covered response documents, operation counts and
authorization-denial audit records. Both feature configurations passed
Clippy, and formatting, script tests and source guards passed.

Changing each IAM action or resource admitted a forbidden read.
Duplicating each edge count and changing each stored output failed its
test. Returning 404 for missing encryption failed the 400 assertion.
Suppressing gateway denial audit removed all nine expected records.
Runtime verification used single-node debug servers.
2026-11-15 22:48:03 +08:00
overtrue 993464fbbb feat(gateway): serve GetObjectAttributes through the owned path
Share the read use case and both IAM decisions across the legacy and
gateway handlers. Keep attribute selection, stored metadata and
pagination in the use case, and encode the result at each edge.

The 6f93ac5f control matched 55 legacy wire cases and the restored
sources passed 139 scoped tests. Both stacks exercised 59 signed
requests, including 31 legacy refusals and 32 gateway refusals.

Seventeen injected faults each turned a targeted assertion red. They
cover the four IAM actions, resource and version binding, empty-version
selection, handler authorization, SSE-C, selection, both checksum paths,
paging, ETag, Last-Modified, counting and notification suppression.
2026-11-15 21:41:16 +08:00
overtrue cfdd4b2686 feat(gateway): serve PutBucketNotificationConfiguration
Share one owned write path across the gateway and legacy edges. Project
runtime destinations at each edge and defer consumption of the persisted
conversion result until after filter validation. Keep IAM scope, the
incarnation fence, and the existing validate, write, reload, clear, and
add order. Count the request at its edge and audit it in the use case.

Add three refusal tests and four tests that exercise success and failure. The legacy HTTP baseline passed on 97a7639f. On 6f93ac5f the
38-test control passed. Six fault injections exposed wrong IAM action,
wrong IAM resource, a missing incarnation fence, duplicate counting,
the wrong audit status, and missing reload callbacks. The first three
changed the stored bytes guarded by their refusal tests.

The restored tree rebuilt and passed all 38 scoped tests. Both package
Clippy configurations, script tests, test wiring, and prepush passed.
Cross-node reload delivery and remote audit sink delivery were outside
the local test scope.
2026-11-15 21:18:37 +08:00
overtrue 6f93ac5f47 feat(gateway): serve AbortMultipartUpload
Use the same authorization and audited use case on both S3 stacks. Bind
the gateway claim to the owner selected under the upload write lock
before staged parts can be removed. Preserve the existing generation,
lifecycle and quorum checks.

Verified the fixed legacy baseline, the integrated native wire flow, 212
focused tests and both Clippy modes. Fault checks exercised owner
selection, deletion and orphan handling, scoped IAM, generation
transport, audit, operation counts and dirty usage.

Audit delivery and separate fault injection of the existing final
lock-loss and metadata-incarnation fences were outside this verification.
2026-11-15 18:05:23 +08:00
overtrue 620fdc286a feat(gateway): serve CopyObject through the owned path
Resolve the source from the gateway read proof. Share source and
destination authorization, tag checks, and the existing storage copy
between both S3 stacks.

Checked the legacy wire baseline, both CopyObject wire contracts, and the
access and refusal suites. Nine temporary faults triggered the intended
assertions. The restored build passed seven focused tests with gateway
75757292, and Clippy passed with and without gateway-stack.
2026-11-15 18:02:37 +08:00
overtrue 58310c569b chore: pin RustFS to gateway mainflow update
Refresh the four RustFS gateway dependency pins to gateway commit 75757292b9617080cbb1bb7f880db8da9b34912c. Update the gateway facade and types versions; keep stream and HTTP versions unchanged.

Verification: cargo check --locked -p rustfs --all-targets --features gateway-stack, layer dependency guard, architecture migration guard, and prepush-quick passed. No Rust sources or tests changed.
2026-11-15 17:42:10 +08:00
overtrue e36c7452ac test: cover DeleteBucketLifecycle in the refusal matrix
Seed a lifecycle configuration so the admitted request removes real state.

Verified the matrix census and both refusal matrices in the CopyObject development checkout on 93947f22c.
2026-11-15 16:41:34 +08:00
overtrue 48e97253b5 feat(gateway): serve ListParts through the owned path
Share authorization and pagination across both S3 edges. Resolve the
native upload claim using the owner from the same quorum-selected
metadata as the parts page. Preserve typed storage errors and write
the page through each edge's response projection.

The scoped suite includes all 98 storage access tests, real upload
sessions, stored-owner combinations, both wire paths, and the storage
API. Twenty-three fault modes exercised the owner, authorization,
lookup, error, counter, and projection assertions.

Validated 168 focused tests after restoring and recompiling the candidate
on c2b5fbb plus the isolated fixture cleanup. The owner matrix covers
three positive and six negative cases; each wire IAM matrix covers one
positive and seven negative cases. Both feature configurations passed
Clippy. Full migration acceptance lanes remain pending.
2026-11-15 16:24:31 +08:00
overtrue ce5bd8b117 test: isolate CreateMultipartUpload in the refusal matrix
Abort the session returned by the successful matrix call through the
fixture store so later pagination assertions see their own sessions.

Both listing boundary tests failed on c2b5fbb and passed with the
fixture cleanup. Both refusal matrices passed with the same cleanup.
2026-11-15 15:32:04 +08:00
overtrue 93947f22ce feat(gateway): serve DeleteBucketLifecycle through the owned use case
Share the incarnation-fenced lifecycle deletion across both edges, preserving reload and site-hook ordering and one audit record.

Validated 37 focused tests on 77da63397, both all-target Clippy configurations, script tests and wiring guards. Thirteen effective faults covered deletion, authorization wiring, incarnation fences, counting, reload and hook ordering, and audit behavior. Live peer delivery and final migration acceptance lanes remain unverified.
2026-11-15 15:28:43 +08:00
Hauser Zhi 8f361e3a6f perf(ecstore): share rename fanout metadata (#9432)
* perf(ecstore): share rename fanout metadata

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(storage): stabilize async regression fixtures

Run the multipart refusal matrix on the shared large-stack harness, wait for scanner metadata commit tails before restart discovery, and observe io_uring owner release alongside in-flight retirement.

Keep the production paths unchanged while preserving each original failure oracle.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-11-15 15:05:20 +08:00
Hauser Zhi 3cee9c154b perf(grpc): attribute rebuilds and bound bulk responses (#9420)
* perf(grpc): attribute rebuilds and bound bulk responses

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(grpc): attribute rebuild success after publication

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(obs): add exact HTTP/2 ping timeout diagnostics

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(multipart): run refusal matrix on large stack

Run the real-store completion refusal matrix through the shared 32 MiB test harness so its deep async storage future does not overflow libtest's default stack.

Keep the original test name and all refusal-order assertions intact.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* chore(deps): pin nixl-sys workspace source

Manage the nixl-sys Git revision from the workspace dependency table and point the optional Linux transport dependency at houseme/nixl commit cc636c83f13d692e85b67e55aed092413d16fb41.

Allow the reviewed source in cargo-deny and refresh the lockfile, replacing the unmaintained bincode dependency with the fork's wincode graph.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): authenticate msgpack peer capability

Add an optional boot-epoch-bound HMAC capability proof and cache the verified per-peer observation without enabling any msgpack send policy.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(grpc): bind peer capability to connection generations (#9433)

* feat(grpc): track physical connection generations

Observe every tonic physical reconnect through a stable custom connector while preserving existing TCP, TLS, HTTP/2, cache, and public channel behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(rpc): bind msgpack capability to connections

Scope authenticated msgpack observations to channel identity and physical connection generation so transparent reconnects require a fresh proof without enabling the send policy.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-15 15:05:09 +08:00
Ramakrishna Chilaka 504cc06d0c perf(ecstore): stage inline rollback backups and fuse backup reclaim (#9445)
Stage inline overwrite rollback backups (xl.meta.bkp) directly relative
to the pinned destination directory descriptor in RenameCommitGuard
instead of staging in .rustfs.sys/tmp and walking both paths a second
time. Also fuse post-commit xl.meta.bkp removal and empty-ancestor
directory pruning into a single blocking dispatch while stopping at live
object roots.
2026-11-15 15:04:18 +08:00
Hauser Zhi 44145ef327 chore(deps): update flake.lock (#9441)
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4fe25e1' (2026-11-06)
  → 'github:NixOS/nixpkgs/2074cc6' (2026-11-12)
• Updated input 'rust-overlay':
    'github:oxalica/rust-overlay/d3fe742' (2026-11-07)
  → 'github:oxalica/rust-overlay/5b87ee0' (2026-11-14)
2026-11-15 15:04:11 +08:00
Hauser Zhi ba9e8342f8 test(storage): add fail-closed quiescence gate (#9436)
* test(storage): add fail-closed quiescence gate

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(storage): gate benchmark rounds on quiescence

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(bench): bound quiescence Docker calls

Start the storage-quiescence deadline before Docker discovery, pass the remaining budget to every subprocess, and reject samples completed after the deadline.

Cover delayed initial lookup and delayed exec through the supported Docker binary override.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-11-15 15:04:04 +08:00
Hauser Zhi a41019802d test(storage): expose direct-write validation controls (#9430)
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-15 15:03:53 +08:00
Hauser Zhi 4313812e24 perf(ecstore): keep inline commits fused with file grouping (#9427)
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-15 15:03:45 +08:00
hector 5c58a56c4c fix: recognize gVisor directfs bind mounts in the FS guard (#9426)
gVisor presents a host bind mount as 9p, so statfs reports V9FS.
When the mountinfo options include both trans=fd and directfs, the
startup filesystem guard treats that endpoint as local POSIX storage
and logs that the host type is not visible. Real 9p and virtio shares
still warn.

Co-authored-by: hector <majinghe@users.noreply.github.com>
Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-15 15:03:37 +08:00
GatewayJ 8196603a76 fix(iam): authorize delegated service account creation (#9408)
* fix(iam): authorize delegated service account creation

* docs(iam): clarify explicit grants for custom administrator policies

* fix(iam): include service account delegation in admin wildcards

* style(policy): format delegated authorization test cases

* fix(iam): keep delegated authorization and replication consistent

* fix(ci): include delegated account test in E2E selections
2026-11-15 15:03:24 +08:00
Hauser Zhi 1e62cbaeaf perf(ecstore): integrate safe sync and allocator diagnostics (#9428)
* feat(diagnostics): restore allocator and runtime metrics

Port the missing live-byte accounting and serving-runtime observations from 13a30abca. Reuse the current MiMalloc GlobalAlloc wrapper, expose explicit metric availability, and cover allocation operations and feature selection.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(ecstore): detach queued fdatasync followers safely

Adapt the follower timeout experiment from 138bc69f6 with exclusive queue ownership transfer. Move queued paths and resource units into independent sync, retain claimed batch ownership through completion, and record observed queue time. Cover publication before the original batch resumes, sync failures, claimed batches, and queue accounting.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(ecstore): index sync followers and isolate missing sources

Use reusable slab slots and monotonic identities for constant-time queued follower extraction. Avoid retaining follower group references when the timeout is disabled. Preserve claimed-batch ownership and reject stale tickets after slot reuse.

Keep missing-source results local to their requests so healthy requests still synchronize their own files. Preserve fail-fast behavior for other disk I/O failures. Repair failure-probe synchronization and exercise source disappearance, reused slots, disabled tracking, cancellation and bounded admission on Linux.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ci): apply storage stack to multipart refusal fixture

Route the real-store absent-binding refusal-order fixture through the existing bounded 4 MiB storage setup in both default and CI profiles. The Linux CI failure reproduced with the default 2 MiB stack. Quota, legal-hold, error, cause, audit and notification assertions execute through the established storage fixture budget.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): make sync worker idle handoff atomic

Move batch claim, in-flight accounting and completion onto the existing worker guard. Publish the idle state under the queue lock so an enqueue during cleanup establishes successor-worker ownership. Keep stale cleanup conditional on current ownership.

Add a controlled interleaving regression that detaches the final queued follower and enqueues a successor after the empty claim. The old worker leaves that request without an owner; the refactored worker hands it off and completes one physical sync.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): await grouped sync resource release

Wait for native producer admission units to reach zero after the blocking sync finishes. This removes a scheduler-dependent assertion window exposed by the latest main integration while preserving the cancellation and ownership checks.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-11-15 14:58:54 +08:00
Hauser Zhi da1817ae01 test(rpc): expose msgpack fleet confirmation (#9429) 2026-11-15 14:54:23 +08:00
overtrue c2b5fbb138 feat(gateway): serve CreateMultipartUpload through the owned path
Share authorization between both CreateMultipartUpload edges. Keep the
decoded legacy fields intact, preserve the replication destination
version and apply the captured bucket incarnation before creating the
upload. Leave checksum, encryption and retention metadata on the same
storage path.

Verify the frozen legacy behavior on 2c2af51a before the migration and
exercise both stacks with 66 refusal cases and 22 success controls.
Read back sessions, version, retention and encryption metadata.

Fault probes made authorization tests fail on both edges when an action
or the resource binding was lost. Storage tests detected a different
destination version and unexpected sessions when generation, quota, KMS
or SSE checks were removed. The wire and
helper tests failed for early name rejection, lost timestamp precision,
metadata, checksum, SSE response, the operation count and unexpected
notification.

On 77da6339, 185 scoped tests passed with a 32 MiB test stack. The default
stack run passed 184 tests and reproduced the existing overflow in
completion_absent_binding_refusal_order_baseline. Both all-target
Clippy variants, test wiring, script tests and prepush passed. After
integration with 97a7639f, gateway all-target check and prepush passed.
Whole-workspace CI and rio-v2 were outside the local verification.
2026-11-15 14:08:38 +08:00
overtrue 97a7639f59 feat(s3): serve UploadPart through the gateway
Share the streaming multipart write path and staged authorization across both edges. Resolve the native upload ID against the active storage record at the existing lookup. Carry checksums, trailers, SSE values and the stored ETag through each response projection.

Add seven tests for application lookup, route authorization, framing and durable part state. Exercise 62 refusals and 10 successful requests across those fixtures. The existing SSE-C completion test also covers the migrated UploadPart core.

Verification on 1467d32695: fixed legacy contract, 147 gateway-feature tests and 119 default tests, both all-target Clippy legs, script tests, test wiring and prepush checks. The test scopes include the entire access module. The gateway scope includes PutObject and both refusal matrices.

Twenty-two local fault controls exercised edge authorization, policy and generation, resource and upload ID projection, owner binding, lookup count and error ordering, logical length, trailer publication, checksum validation and typed producer errors. Each fault failed its intended test after the same artifact passed with the selector unset.

Linux RDMA and a native KMS denial through UploadPart remain unverified.
2026-11-15 13:41:36 +08:00
299 changed files with 48290 additions and 6135 deletions
+2 -2
View File
@@ -1,2 +1,2 @@
sha256-darwin=8fef326a3fddd8a548e2cbcc6ea8159f831113595fb003eeebbd9aeb4ac23e48
sha256-linux=8b4381578377704ac8b1b5156128b83f036ec4aedc9d863fff25df15112420fc
sha256-darwin=24a8cfaa1b9125bd69d7d4a7ff752a8ab8ed350c761d90ae8472ef52393e2686
sha256-linux=014257197fbbfbb48cab2864da9e10d96ee877d478fa29d119f8186532ea293d
+1 -1
View File
@@ -1 +1 @@
sha256=061d0a697967d2dc8f472ff885886e069b1cf4aeaa5f230e177b7236017c23b6
sha256=fd5683273bba1eae3b4402a4e854a5d168ace845bd0e5fc801f8e336485a4394
+2
View File
@@ -39,9 +39,11 @@ script-tests: ## Run shell script tests
./scripts/test_entrypoint_directories.sh
./scripts/test_internode_grpc_ab_bench.sh
$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_grpc_window.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_rpc_timeout_boundary.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_msgpack_peer_canary.py
./scripts/test_internode_msgpack_peer_canary_runner.sh
./scripts/test_object_batch_bench_enhanced.sh
$(RUSTFS_PYTHON_BIN) ./scripts/test_inline_docker_lab.py
./scripts/test_package_service_scripts.sh
./scripts/test_hotpath_warp_ab_gate.sh
./scripts/test_hotpath_warp_abba.sh
+3 -2
View File
@@ -141,8 +141,9 @@ setup = 'ecstore-large-stack'
filter = '(package(rustfs) & test(/^(admin::handlers::replication::target_repair_tests::[a-z_]+|admin::handlers::plugin_attempt::v2_tests::delegation_v2_persists_and_vends_with_real_parent_intersection|admin::handlers::site_replication::tests::durable_iam_state_apply_iam_item_rejects_a_stale_recreate_after_a_replicated_delete|admin::service::federated_identity::tests::(binding_keeps_explicit_iam_mapping_strict|markerless_legacy_oidc_session_crosses_auth_and_iam_deletion_boundaries)|storage::scanner_dirty_journal::tests::(clear_callback_keeps_unknown_coverage_until_all_pending_work_is_confirmed|owned_journals_keep_peer_records_after_interleaved_writes_and_clear))$/)) | (package(rustfs-heal) & ((binary(heal_b5_versioned_regression_test) & test(/^serial_tests::(test_heal_unversioned_bucket_e2e|test_historical_null_recursive_heal_restores_physical_versions|test_recursive_deep_heal_reports_stale_delete_marker_repair)$/)) | (binary(usage_observation_read_repair_test) & test(=usage_observation_read_repair_survives_snapshot_cleanup)))) | (package(rustfs-test-utils) & test(=data_usage_snapshot_tests::data_usage_snapshot_recovery_wires_primary_backup_and_negative_cache))'
setup = 'ecstore-base-stack'
# The multipart refusal fixture composes real-store setup, abort, quota and legal-hold futures.
[[profile.default.scripts]]
filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::complete::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::complete::tests::completion_absent_binding_refusal_order_baseline|app::multipart_usecase::complete::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
setup = 'ecstore-base-stack'
# These handler/store fixtures and the relocated-pool GET compose deep storage
@@ -465,7 +466,7 @@ filter = '(package(rustfs) & test(/^(admin::handlers::replication::target_repair
setup = 'ecstore-base-stack'
[[profile.ci.scripts]]
filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::complete::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::complete::tests::completion_absent_binding_refusal_order_baseline|app::multipart_usecase::complete::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
setup = 'ecstore-base-stack'
[[profile.ci.scripts]]
+1 -1
View File
@@ -22,7 +22,7 @@ rustfs-connect-symbol-catalog 6 0
rustfs-credentials 24 0
rustfs-crypto 67 0
rustfs-data-usage 58 0
rustfs-ecstore 7858 2
rustfs-ecstore 7858 3
rustfs-extension-schema 23 0
rustfs-filemeta 361 0
rustfs-gateway-adapter 74 0
@@ -35,6 +35,7 @@ services:
- RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
- RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
- RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
- RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE=${RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
# Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
@@ -46,6 +47,8 @@ services:
- RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
- RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
- RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
- RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
- RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
- RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -59,7 +62,7 @@ services:
- node1_data_2:/data/rustfs2
- node1_data_3:/data/rustfs3
ports:
- "9000:9000"
- "${RUSTFS_NODE1_PORT:-9000}:9000"
networks:
- rustfs-cluster-net
@@ -85,6 +88,7 @@ services:
- RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
- RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
- RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
- RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE=${RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
# Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
@@ -96,6 +100,8 @@ services:
- RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
- RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
- RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
- RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
- RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
- RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -109,7 +115,7 @@ services:
- node2_data_2:/data/rustfs2
- node2_data_3:/data/rustfs3
ports:
- "9001:9000"
- "${RUSTFS_NODE2_PORT:-9001}:9000"
networks:
- rustfs-cluster-net
@@ -135,6 +141,7 @@ services:
- RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
- RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
- RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
- RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE=${RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
# Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
@@ -146,6 +153,8 @@ services:
- RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
- RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
- RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
- RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
- RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
- RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -159,7 +168,7 @@ services:
- node3_data_2:/data/rustfs2
- node3_data_3:/data/rustfs3
ports:
- "9002:9000"
- "${RUSTFS_NODE3_PORT:-9002}:9000"
networks:
- rustfs-cluster-net
@@ -185,6 +194,7 @@ services:
- RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
- RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
- RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
- RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE=${RUSTFS_OBJECT_DIRECT_IO_WRITE_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
- RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
# Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
@@ -196,6 +206,8 @@ services:
- RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
- RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
- RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
- RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
- RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
- RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
- RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -209,7 +221,7 @@ services:
- node4_data_2:/data/rustfs2
- node4_data_3:/data/rustfs3
ports:
- "9003:9000"
- "${RUSTFS_NODE4_PORT:-9003}:9000"
networks:
- rustfs-cluster-net
+58
View File
@@ -0,0 +1,58 @@
# syntax=docker/dockerfile:1.6
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Server-only Linux validation image. The build context supplies a Git bundle.
FROM rust:1.100.0-trixie AS build-tools
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates clang lld libssl-dev pkg-config protobuf-compiler python3 python3-boto3 \
&& rm -rf /var/lib/apt/lists/*
ENV RUSTUP_TOOLCHAIN=1.100.0 \
CARGO_BUILD_JOBS=4 \
CARGO_NET_GIT_FETCH_WITH_CLI=true \
CARGO_TARGET_DIR=/work/rustfs/target \
RUSTFLAGS="-C link-arg=-fuse-ld=lld"
FROM build-tools AS build
ARG SOURCE_REVISION
COPY repository.bundle /tmp/repository.bundle
COPY inspect_geometry.rs /tmp/inspect_geometry.rs
RUN git clone /tmp/repository.bundle /work/rustfs \
&& git -C /work/rustfs checkout --detach "$SOURCE_REVISION" \
&& test "$(git -C /work/rustfs rev-parse HEAD)" = "$SOURCE_REVISION"
WORKDIR /work/rustfs
RUN --mount=type=cache,id=rustfs-inline-registry,target=/usr/local/cargo/registry \
--mount=type=cache,id=rustfs-inline-git,target=/usr/local/cargo/git \
--mount=type=cache,id=rustfs-inline-linux-target,target=/work/rustfs/target \
cargo build --locked --release --bin rustfs \
&& install -Dm755 target/release/rustfs /out/rustfs
# Build the diagnostic tool after preserving the clean-source server binary.
RUN cp /tmp/inspect_geometry.rs crates/filemeta/examples/inline_lab_inspect.rs
RUN --mount=type=cache,id=rustfs-inline-registry,target=/usr/local/cargo/registry \
--mount=type=cache,id=rustfs-inline-git,target=/usr/local/cargo/git \
--mount=type=cache,id=rustfs-inline-linux-target,target=/work/rustfs/target \
cargo build --locked --release -p rustfs-filemeta --example inline_lab_inspect \
&& install -Dm755 target/release/examples/inline_lab_inspect /out/inline_lab_inspect
FROM debian:trixie-slim AS runtime
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates curl libssl3t64 \
&& rm -rf /var/lib/apt/lists/*
ARG SOURCE_REVISION
LABEL org.opencontainers.image.revision="$SOURCE_REVISION" \
rustfs.validation="inline"
COPY --from=build /out/rustfs /usr/local/bin/rustfs
COPY --from=build /out/inline_lab_inspect /usr/local/bin/inline_lab_inspect
ENTRYPOINT ["/usr/local/bin/rustfs"]
+5
View File
@@ -0,0 +1,5 @@
FROM debian:trixie-slim
COPY mc /usr/bin/mc
RUN chmod 0755 /usr/bin/mc && /usr/bin/mc --version
LABEL rustfs.validation="inline"
ENTRYPOINT ["/usr/bin/mc"]
+133
View File
@@ -0,0 +1,133 @@
# Inline geometry Linux lab
**Use this when** validating inline placement, block geometry, Range reads and reader rollback in a task-owned local Docker cluster.
The adapter reuses the repository's four-node compose topology with 16 logical Docker volumes and EC12+4. It binds one S3 port to loopback, keeps strict durability and checks running binary hashes and persisted object geometry. Docker volumes share the host storage substrate. Performance observations describe this local virtualized lab.
## Build a frozen server image
Use the current stable toolchain pinned in the lab Dockerfile. Supply a Git bundle and the metadata inspector source as the build context. The server is compiled from a clean checkout before the diagnostic example is added.
```bash
set -euo pipefail
LAB_BUILD_DIR="$(mktemp -d)"
LAB_REVISION="$(git rev-parse HEAD)"
git bundle create "$LAB_BUILD_DIR/repository.bundle" HEAD
cp crates/filemeta/examples/inline_lab_inspect.rs "$LAB_BUILD_DIR/inspect_geometry.rs"
docker --context "$LAB_CONTEXT" build --target build-tools \
-t rustfs-inline-build-tools:1.100 \
-f .docker/inline-lab/Dockerfile .docker/inline-lab
docker --context "$LAB_CONTEXT" build \
--build-arg SOURCE_REVISION="$LAB_REVISION" \
-t rustfs-inline-validation:local \
-f .docker/inline-lab/Dockerfile "$LAB_BUILD_DIR"
LAB_EXPORT_ID="$(docker --context "$LAB_CONTEXT" create \
--label rustfs.validation=inline rustfs-inline-validation:local)"
docker --context "$LAB_CONTEXT" cp \
"$LAB_EXPORT_ID:/usr/local/bin/rustfs" "$LAB_BUILD_DIR/rustfs-linux"
docker --context "$LAB_CONTEXT" rm "$LAB_EXPORT_ID"
```
## Configure and probe
Select a new compose project and a private state directory. The adapter generates test credentials there. Existing resources require matching ownership labels. Configure checks the image's source revision and pins immutable image IDs. Each operation takes an exclusive state lock. Deployment waits for `/health/ready` on every node, covering storage, IAM, lock quorum and peer dependencies.
```bash
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" configure \
--context "$LAB_CONTEXT" --project "$LAB_PROJECT" \
--image rustfs-inline-validation:local --revision "$LAB_REVISION" \
--binary "$LAB_BUILD_DIR/rustfs-linux" --port 19080
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" deploy \
--bucket inline-geometry-probe --shard-bytes 0
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" probe \
--bucket inline-geometry-probe
```
The probe verifies complete bytes, HEAD sizes, Range contents, EC12+4 and the actual persisted block. It records inline markers for 4 KiB, 1 MiB, 1 MiB + 1 byte, 1.25 MiB, 1,572,856 bytes, 1,572,857 bytes and 1.5 MiB objects in `probe.json`. The server-only image includes the existing filemeta decoder through a small geometry inspector. Console assets and the administration CLI are outside this lab image.
## ABBA deployment hook
Use `inline_docker_lab.py ... deploy` in the existing `run_hotpath_warp_abba.sh` external deploy hook. Export `RUSTFS_ERASURE_SHARD_BYTES` according to `HOTPATH_ABBA_PHASE`. The adapter uses the supplied leg, phase, bucket, binary and evidence-file context, resets only owned lab volumes, checks running configuration and binary hashes, creates the fresh bucket and writes the runner's deployment evidence.
Use the same frozen binary for both sides of a geometry experiment. Fix the inline budget and encoding mode, then compare zero with one candidate shard value. Include the CPU, memory and image identities from every node in each result. The state `config.json` supplies the generated test credentials for the load driver. Set `DOCKER_CONTEXT` to the selected context when the driver captures Docker resource snapshots.
For content compatibility across an image or geometry change, configure the next selected image and artifact using the same state, then deploy with `--keep-data` and an existing bucket. This retains owned volumes. Compare existing objects and version IDs with the manifest prepared before the change.
## Cleanup
```bash
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" down --purge
```
Cleanup checks ownership for containers, volumes and networks. A configuration cannot adopt another project's resources. Raw measurements, state, Git bundles and build outputs stay in task artifact directories.
Unit checks run with `scripts/python_bin.sh scripts/test_inline_docker_lab.py` and are included in `make script-tests`.
## Linux content-probe client
The PUT after-probe requires a working `mc`. For a macOS client affected by cgo initialization failures, compile the same frozen public mc module for Linux with CGO disabled, build it with `Dockerfile.mc`, and use `scripts/inline_docker_mc.sh` as the process-local `mc` command. Record the module version, checksum and resulting image ID with the run.
The adapter rewrites only the lab loopback alias endpoint to the private `node1` endpoint. Alias configuration stays in the probe's own `MC_CONFIG_DIR`, and `cat` streams raw bytes. The existing HEAD, GET and hash checks continue to execute.
```bash
export RUSTFS_INLINE_LAB_STATE="$LAB_STATE"
export RUSTFS_INLINE_MC_IMAGE="$LAB_MC_IMAGE"
mkdir -p "$LAB_TOOLS"
ln -s "$PWD/scripts/inline_docker_mc.sh" "$LAB_TOOLS/mc"
export PATH="$LAB_TOOLS:$PATH"
```
## Compare transport and placement independently
`deploy --rpc-mode json-compat` explicitly clears both MessagePack-only flags on every node. `deploy --rpc-mode msgpack-only` enables both flags for the selected homogeneous lab fleet. Deployment evidence records the mode and checks both running flags. The default is `json-compat`. The ABBA hook may select the same mode through `RUSTFS_INLINE_LAB_RPC_MODE`.
First compare the base and bounded-multi-block binaries at `--shard-bytes 0` in JSON compatibility mode. Then repeat with MessagePack-only on both sides. A same-binary transport ABBA isolates the encoding effect. Keep the 128 KiB budget, CPU sets, strict durability, load and object set fixed. Check actual markers and block sizes at the boundary sizes before timing. Retain baseline-drift and candidate gates, error logs and raw request distributions.
For a rolling downgrade, redeploy the current binary with `--keep-data --rpc-mode json-compat` before introducing an older node. Check old and new reads, version IDs and batch deletes against the persisted manifest. A running MessagePack-only flag pair establishes the lab configuration. Compatibility with a particular old reader requires the explicit reader test.
## Preserve a compatibility manifest
Run the seed once in a fresh lab bucket. It records full byte oracles and the observed version inventory, including any additional version matching a known body, and leaves a 5 MiB multipart part ready for the next reader. Each bucket has its own private `compat-<bucket>.json` manifest. Keep the same state and volumes across image changes. A fresh deployment or cleanup that resets volumes moves existing compatibility manifests into `compat-history`. Retained-data deployments preserve the active manifests.
```bash
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-seed --bucket inline-compat
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check --bucket inline-compat
# Configure the next frozen image and binary, then retain the data.
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" deploy \
--bucket inline-compat --shard-bytes 0 --rpc-mode json-compat --keep-data
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check --bucket inline-compat
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-finish-mpu --bucket inline-compat
```
The check compares every persisted version ID, HEAD length, complete SHA256, cross-block Range and per-version geometry from `xl.meta`. The seed includes zero-length objects, null versions, the 192 KiB versioned admission boundary and the accumulated 128 KiB shard boundary. Seed a second bucket on the new writer before the downgrade, then check both manifests using the selected old reader. Complete an old upload on the new binary and a new upload on the old binary. After returning to the current fleet, `compat-delete --bucket <bucket>` deletes the recorded versions in one S3 batch and checks the exact remaining inventory and bytes.
Successful multipart-completion and batch-delete receipts are saved before subsequent content and geometry checks. If either check fails, rerun the same command with the same state directory to resume verification from the saved completion state. After a seed check fails with a saved manifest, use `compat-check` to resume verification.
For mixed-version testing, retain JSON compatibility mode throughout the rolling replacement. Verify each node's immutable image and binary, then perform manifest checks and new writes through both coordinator versions using `--coordinator node1` and `--coordinator node4`. The manifest records the actual writer coordinator revision from the running image. Stop one owned node to check read availability at EC12+4, restore it and check the manifests again. Record the exact oldest tested source revision, toolchain and build profile with the evidence. The version boundary is established by these executed cells.
An owned lab can replace just its fourth node through a compose image override. Select the frozen old image first and verify its revision label. Keep `compose.json` in JSON compatibility mode for this cell.
```bash
set -euo pipefail
OLD_IMAGE_ID="$(docker --context "$LAB_CONTEXT" image inspect "$OLD_IMAGE" --format '{{.Id}}')"
test "$(docker --context "$LAB_CONTEXT" image inspect "$OLD_IMAGE_ID" --format '{{index .Config.Labels "org.opencontainers.image.revision"}}')" = "$OLD_REVISION"
INLINE_OLD_IMAGE_ID="$OLD_IMAGE_ID" scripts/python_bin.sh - <<'PYTHON' > "$LAB_STATE/mixed-image.json"
import json
import os
import sys
override = {"services": {"node4": {"image": os.environ["INLINE_OLD_IMAGE_ID"]}}}
json.dump(override, sys.stdout)
PYTHON
docker --context "$LAB_CONTEXT" compose --project-name "$LAB_PROJECT" \
--file "$LAB_STATE/compose.json" --file "$LAB_STATE/mixed-image.json" \
up --detach --no-deps --no-build --pull never node4
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check \
--bucket inline-compat --coordinator node4
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-seed \
--bucket inline-mixed-writer --coordinator node4
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check \
--bucket inline-mixed-writer --coordinator node1
```
Inspect all four running image IDs and binary hashes before recording the mixed-fleet result. Redeploy the configured current image with `--keep-data --rpc-mode json-compat` to restore a homogeneous fleet. The same manifest checks apply after stopping and restarting one owned node through the base compose file.
Generated
+33 -25
View File
@@ -4133,7 +4133,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.59.0",
"windows-sys 0.60.2",
]
[[package]]
@@ -4641,7 +4641,7 @@ checksum = "94e7099f6313ecacbe1256e8ff9d617b75d1bcb16a6fddef94866d225a01a14a"
dependencies = [
"io-lifetimes 2.0.4",
"rustix",
"windows-sys 0.59.0",
"windows-sys 0.52.0",
]
[[package]]
@@ -5844,7 +5844,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20fd6de4ccfcc187e38bc21cfa543cb5a302cb86a8b114eb7f0bf0dc9f8ac00f"
dependencies = [
"io-lifetimes 3.0.1",
"windows-sys 0.60.2",
"windows-sys 0.52.0",
]
[[package]]
@@ -7073,7 +7073,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.59.0",
"windows-sys 0.60.2",
]
[[package]]
@@ -9865,6 +9865,8 @@ dependencies = [
"rustfs-filemeta",
"rustfs-gateway",
"rustfs-gateway-adapter",
"rustfs-gateway-core",
"rustfs-gateway-dialect-rustfs-admin",
"rustfs-gateway-stream",
"rustfs-gateway-types",
"rustfs-heal",
@@ -10187,6 +10189,7 @@ dependencies = [
"serial_test",
"sha2 0.11.1",
"shadow-rs",
"slab",
"smallvec",
"temp-env",
"tempfile",
@@ -10265,8 +10268,8 @@ dependencies = [
[[package]]
name = "rustfs-gateway"
version = "0.63.0"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
version = "0.66.0"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"arc-swap",
"bytes",
@@ -10312,8 +10315,8 @@ dependencies = [
[[package]]
name = "rustfs-gateway-core"
version = "0.60.0"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
version = "0.63.0"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"bytes",
"futures-core",
@@ -10331,10 +10334,24 @@ dependencies = [
"zeroize",
]
[[package]]
name = "rustfs-gateway-dialect-rustfs-admin"
version = "0.13.0"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"bytes",
"form_urlencoded",
"http 1.5.0",
"rustfs-gateway-core",
"rustfs-gateway-sig",
"rustfs-gateway-stream",
"rustfs-gateway-types",
]
[[package]]
name = "rustfs-gateway-http"
version = "0.1.4"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"hmac 0.13.0",
"http 1.5.0",
@@ -10352,7 +10369,7 @@ dependencies = [
[[package]]
name = "rustfs-gateway-macros"
version = "0.1.4"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"proc-macro2",
"quote",
@@ -10362,7 +10379,7 @@ dependencies = [
[[package]]
name = "rustfs-gateway-sig"
version = "0.15.0"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"hmac 0.13.0",
"http 1.5.0",
@@ -10377,7 +10394,7 @@ dependencies = [
[[package]]
name = "rustfs-gateway-stream"
version = "0.2.0"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"bitflags 2.13.2",
"bytes",
@@ -10387,8 +10404,8 @@ dependencies = [
[[package]]
name = "rustfs-gateway-types"
version = "0.40.0+aws.2026-10-02"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
version = "0.42.0+aws.2026-10-02"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"bytes",
"crc-fast",
@@ -10411,7 +10428,7 @@ dependencies = [
[[package]]
name = "rustfs-gateway-xml"
version = "0.4.0"
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
dependencies = [
"quick-xml 0.41.0",
]
@@ -14272,15 +14289,6 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.59.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.60.2"
@@ -14453,7 +14461,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f3fd376f71958b862e7afb20cfe5a22830e1963462f3a17f49d82a6c1d1f42d"
dependencies = [
"bitflags 2.13.2",
"windows-sys 0.59.0",
"windows-sys 0.52.0",
]
[[package]]
+8 -5
View File
@@ -151,14 +151,16 @@ rustfs-zip = { path = "./crates/zip", version = "1.0.5" }
# engine crates move onto from s3s (rustfs/backlog#2745). The one pin of the
# gateway in this workspace; no crate may enable a `compat-s3s*` feature except
# the s3s-facing edge of the rustfs binary.
rustfs-gateway-types = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", version = "0.40.0", default-features = false }
rustfs-gateway-stream = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", version = "0.2.0", default-features = false }
rustfs-gateway-types = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.42.0", default-features = false }
rustfs-gateway-stream = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.2.0", default-features = false }
# The gateway framework itself, consumed for its extension traits by the ring-2
# adapter. `default-features = false` keeps the `server` feature, and with it the
# server crate, out of the build.
rustfs-gateway = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", version = "0.63.0", default-features = false }
rustfs-gateway = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.66.0", default-features = false }
# Test-only: builds the `EffectiveHost` a host-resolution test queries with.
rustfs-gateway-http = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", default-features = false }
rustfs-gateway-http = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", default-features = false }
rustfs-gateway-core = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.63.0", default-features = false }
rustfs-gateway-dialect-rustfs-admin = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.13.0", default-features = false }
# Async Runtime and Networking
async-channel = "2.5.0"
@@ -186,6 +188,7 @@ memchr = "2.8.3"
minlz = "1.2.3"
reqwest = "0.13.5"
rustfs-kafka-async = { version = "1.4.1" }
slab = { version = "0.4.12" }
socket2 = { version = "0.6.5" }
tokio = { version = "1.54.1" }
tokio-rustls = { default-features = false, version = "0.26.6" }
@@ -308,7 +311,7 @@ hostname = "0.4.2"
ipnetwork = { version = "0.21.1" }
lazy_static = "1.5.1"
libc = "0.2.191"
cuobject = "0.1.0"
cuobject = { version = "0.1.0", default-features = false }
libsystemd = "0.7.2"
local-ip-address = "0.6.13"
log = "0.4"
+2
View File
@@ -302,6 +302,8 @@ Drive timeout profile preset:
RustFS production guidance remains direct-attached local POSIX filesystems. Network-mounted filesystems (for example `nfs`, `cifs`, `smb2`, and `fuse.*`) are treated as unsupported by this startup guard. On Windows, endpoints on network drives (UNC shares such as `\\server\share` and mapped drive letters) are treated the same way, even though Windows reports the server's filesystem (usually `NTFS`) for them.
A local POSIX filesystem bind-mounted into a gVisor sandbox with directfs is a supported storage path. gVisor presents that bind mount as `9p`/`V9FS`, so the host filesystem type is not visible; the startup guard recognizes mountinfo options `trans=fd` and `directfs` and does not treat the mount as unsupported. Genuine 9p, virtio, and NFS shares remain unsupported.
## 📄 License
This project is licensed under the Apache License 2.0 - see the [LICENSE](../../LICENSE) file for details.
+1 -1
View File
@@ -48,7 +48,7 @@ pub const ENV_RUNTIME_DIAL9_S3_PREFIX: &str = "RUSTFS_RUNTIME_DIAL9_S3_PREFIX";
pub const DEFAULT_WORKER_THREADS: usize = 16;
pub const DEFAULT_MAX_BLOCKING_THREADS: usize = 1024;
pub const DEFAULT_THREAD_PRINT_ENABLED: bool = false;
pub const DEFAULT_THREAD_STACK_SIZE: usize = MI_B; // 1 MiB
pub const DEFAULT_THREAD_STACK_SIZE: usize = 2 * MI_B; // 2 MiB
pub const DEFAULT_THREAD_KEEP_ALIVE: u64 = 60; // seconds
pub const DEFAULT_GLOBAL_QUEUE_INTERVAL: u32 = 31;
pub const DEFAULT_THREAD_NAME: &str = "rustfs-worker";
+133 -1
View File
@@ -146,7 +146,11 @@ async fn create_service_account_for(
Some(serde_json::json!({ "targetUser": parent }).to_string()),
)
.await?;
let response: serde_json::Value = serde_json::from_str(&response)?;
service_account_keys(&response)
}
fn service_account_keys(response: &str) -> Result<(String, String), Box<dyn Error + Send + Sync>> {
let response: serde_json::Value = serde_json::from_str(response)?;
let access_key = response["credentials"]["accessKey"]
.as_str()
.ok_or("service account response should contain credentials.accessKey")?
@@ -344,6 +348,134 @@ async fn test_update_service_account_enforces_owner_and_parent_scope() -> TestRe
Ok(())
}
#[tokio::test]
async fn test_delegated_service_account_creation_through_console_sessions() -> TestResult {
init_logging();
let mut env = RustFSTestEnvironment::new().await?;
env.start_rustfs_server_without_cleanup(vec![]).await?;
let bucket = "delegated-backups";
let target = "backup-user";
let operator = "backup-operator";
let operator_secret = "operatorSecret123";
let wildcard_admin = "backup-admin";
let wildcard_secret = "adminSecret123";
let s3 = env.create_s3_client();
s3.create_bucket().bucket(bucket).send().await?;
for (name, secret) in [
(target, "backupSecret123"),
(operator, operator_secret),
(wildcard_admin, wildcard_secret),
("other-user", "otherUserSecret123"),
] {
admin_ok(
&env,
http::Method::PUT,
&format!("/rustfs/admin/v3/add-user?accessKey={name}"),
Some(serde_json::json!({"secretKey":secret,"status":"enabled"}).to_string()),
)
.await?;
}
let delegation = serde_json::json!({
"Version":"2012-10-17",
"Statement":[
{"Effect":"Allow","Action":"admin:CreateServiceAccount"},
{"Effect":"Allow","Action":"sts:AssumeRole","Resource":"arn:aws:s3:::*"},
{"Effect":"Allow","Action":"admin:CreateServiceAccountForUser","Condition":{"StringEquals":{"svc:TargetUser":target}}}
]
});
let wildcard = serde_json::json!({
"Version":"2012-10-17",
"Statement":[
{"Effect":"Allow","Action":"admin:*"},
{"Effect":"Allow","Action":"sts:AssumeRole","Resource":"arn:aws:s3:::*"}
]
});
for (user, name, policy) in [
(wildcard_admin, "backup-admin-wildcard", wildcard.to_string()),
(target, "backup-writer", bucket_rw_policy(bucket)),
(operator, "backup-key-manager", delegation.to_string()),
] {
admin_ok(
&env,
http::Method::PUT,
&format!("/rustfs/admin/v3/add-canned-policy?name={name}"),
Some(policy),
)
.await?;
admin_ok(
&env,
http::Method::POST,
"/rustfs/admin/v3/idp/builtin/policy/attach",
Some(serde_json::json!({"policies":[name],"user":user}).to_string()),
)
.await?;
}
for (name, secret) in [
(operator, operator_secret),
(wildcard_admin, wildcard_secret),
(env.access_key.as_str(), env.secret_key.as_str()),
] {
let session = build_test_sts_client(&env.url, name, secret, None, "delegated-create")
.assume_role()
.role_arn("arn:aws:iam::123456789012:role/console")
.role_session_name("console")
.send()
.await?;
let caller = session.credentials().ok_or("missing console session")?;
let create = |user: &str| {
admin_request_with_session_token(
&env.url,
http::Method::PUT,
"/rustfs/admin/v3/add-service-account",
Some(serde_json::json!({"targetUser":user}).to_string()),
caller.access_key_id(),
caller.secret_access_key(),
Some(caller.session_token()),
)
};
let (status, response) = create(target).await?;
assert_eq!(status, StatusCode::OK, "create for target as {name}");
let (key, secret) = service_account_keys(&response)?;
let client = build_s3_client(&env.url, &key, &secret);
wait_for_s3_put(&client, bucket, "backup", Duration::from_secs(10)).await?;
let denied = client
.create_bucket()
.bucket("outside-backup-policy")
.send()
.await
.expect_err("target cannot create unrelated buckets");
assert_eq!(denied.as_service_error().and_then(|e| e.code()), Some("AccessDenied"));
if name != env.access_key {
let (status, _) = create(&env.access_key).await?;
assert_eq!(status, StatusCode::FORBIDDEN, "delegated targets exclude root");
}
if name == operator {
let (status, _) = create("other-user").await?;
assert_eq!(status, StatusCode::FORBIDDEN, "outside delegated target");
}
}
let restricted = build_test_sts_client(&env.url, &env.access_key, &env.secret_key, None, "restricted-root")
.assume_role()
.role_arn("arn:aws:iam::123456789012:role/console")
.role_session_name("restricted-console")
.policy(r#"{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"admin:CreateServiceAccount"}]}"#)
.send()
.await?;
let caller = restricted.credentials().ok_or("missing restricted root session")?;
let (status, _) = admin_request_with_session_token(
&env.url,
http::Method::PUT,
"/rustfs/admin/v3/add-service-account",
Some(serde_json::json!({"targetUser":target}).to_string()),
caller.access_key_id(),
caller.secret_access_key(),
Some(caller.session_token()),
)
.await?;
assert_eq!(status, StatusCode::FORBIDDEN, "restricted root session requires delegated permission");
Ok(())
}
/// Full user -> policy -> service-account lifecycle, proving each management
/// call takes effect on the data plane, not just that the endpoint answers 200.
#[tokio::test]
@@ -17,7 +17,7 @@
use async_trait::async_trait;
use rustfs_lock::{
LockClient, LockError, LockId, LockInfo, LockRequest, LockResponse, LockStats, LockStatus, LockType, Result,
CacheFenceStamp, LockClient, LockError, LockId, LockInfo, LockRequest, LockResponse, LockStats, LockStatus, LockType, Result,
types::{LockMetadata, LockPriority},
};
use rustfs_protos::proto_gen::node_service::{BatchGenerallyLockRequest, GenerallyLockRequest, PingRequest};
@@ -64,6 +64,7 @@ impl GrpcLockClient {
deadlock_detection: false,
suppress_contention_logs: false,
refresh_interval: None,
cache_fence_stamp_requested: false,
}
}
@@ -124,10 +125,15 @@ impl LockClient for GrpcLockClient {
// Check if the lock acquisition was successful
if resp.success {
Ok(LockResponse::success(
Self::build_lock_info(request, resp.lock_info),
std::time::Duration::ZERO,
))
let info = Self::build_lock_info(request, resp.lock_info);
match resp
.cache_fence_stamp
.as_deref()
.and_then(|encoded| CacheFenceStamp::decode(encoded).ok())
{
Some(stamp) => Ok(LockResponse::success_with_cache_fence_stamp(info, std::time::Duration::ZERO, stamp)),
None => Ok(LockResponse::success(info, std::time::Duration::ZERO)),
}
} else {
// Lock acquisition failed
Ok(LockResponse::failure(
@@ -160,7 +166,15 @@ impl LockClient for GrpcLockClient {
.enumerate()
.map(|(idx, request)| match resp.results.get(idx) {
Some(result) if result.success => {
LockResponse::success(Self::build_lock_info(request, result.lock_info.clone()), std::time::Duration::ZERO)
let info = Self::build_lock_info(request, result.lock_info.clone());
match result
.cache_fence_stamp
.as_deref()
.and_then(|encoded| CacheFenceStamp::decode(encoded).ok())
{
Some(stamp) => LockResponse::success_with_cache_fence_stamp(info, std::time::Duration::ZERO, stamp),
None => LockResponse::success(info, std::time::Duration::ZERO),
}
}
Some(result) => LockResponse::failure(
result
@@ -43,6 +43,7 @@ fn lock_result_from_response(response: rustfs_lock::LockResponse) -> GenerallyLo
success: response.success,
error_info: response.error,
lock_info: response.lock_info.and_then(|info| serde_json::to_string(&info).ok()),
cache_fence_stamp: response.cache_fence_stamp.map(|stamp| stamp.encode().to_vec().into()),
..Default::default()
}
}
@@ -180,6 +181,7 @@ impl NodeService for MinimalLockNodeService {
success: result.success,
error_info: result.error,
lock_info: lock_info_json,
cache_fence_stamp: result.cache_fence_stamp.map(|stamp| stamp.encode().to_vec().into()),
..Default::default()
}))
}
+1
View File
@@ -126,6 +126,7 @@ test-util = []
e2e-test-hooks = []
[dependencies]
slab.workspace = true
hashbrown.workspace = true
hotpath.workspace = true
rustfs-filemeta.workspace = true
+2 -1
View File
@@ -393,7 +393,8 @@ pub mod cluster {
pub mod compression {
pub use crate::io_support::compress::{
MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible, is_disk_compression_enabled, is_multipart_disk_compression_enabled,
DISK_COMPRESSION_EXCLUDED_CONTENT_TYPES, MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible, is_disk_compression_enabled,
is_multipart_disk_compression_enabled,
};
}
+134 -57
View File
@@ -2559,9 +2559,44 @@ fn encode_file_info_versions_msgpack(value: &FileInfoVersions) -> Result<Vec<u8>
encode_msgpack_with_capacity(value, capacity)
}
/// JSON compatibility string for a dual-encoded (`_bin` + text) request field. Returns an empty
/// string only when msgpack-only mode and its explicit fleet confirmation guard are both enabled;
/// otherwise the legacy JSON encoding is retained for old peers.
fn encode_delete_versions_request(
disk: String,
volume: &str,
versions: &[FileInfoVersions],
opts: &DeleteOptions,
) -> Result<DeleteVersionsRequest> {
// Binary-only batches require fleet-confirmed receivers that traverse the
// binary list. Capture that policy for the complete batch and its options.
let msgpack_only = rustfs_protos::internode_rpc_msgpack_only();
let opts_json = if msgpack_only {
String::new()
} else {
serde_json::to_string(opts)?
};
let mut versions_json = if msgpack_only {
Vec::new()
} else {
Vec::with_capacity(versions.len())
};
let mut versions_bin = Vec::with_capacity(versions.len());
for version in versions {
if !msgpack_only {
versions_json.push(serde_json::to_string(version)?);
}
versions_bin.push(encode_file_info_versions_msgpack(version)?.into());
}
Ok(DeleteVersionsRequest {
disk,
volume: volume.to_owned(),
versions: versions_json,
opts: opts_json,
versions_bin,
opts_bin: encode_delete_options_msgpack(opts)?.into(),
})
}
/// JSON compatibility string for a dual-encoded request field. The captured
/// send policy selects the encoding, and its generation guard binds dispatch.
fn compat_json<T: Serialize>(value: &T, policy: &MsgpackSendPolicy) -> Result<String> {
if policy.omit_eligible_json() {
return Ok(String::new());
@@ -3547,52 +3582,10 @@ impl DiskAPI for RemoteDisk {
return vec![Some(DiskError::FaultyDisk); versions.len()];
}
// `_bin` support for DeleteVersions is new (grpc-optimization P2); always dual-write JSON +
// msgpack until its fallback counter has read zero across a release window.
let opts_bin = match encode_delete_options_msgpack(&opts) {
Ok(opts_bin) => opts_bin,
Err(err) => {
let mut errors = Vec::with_capacity(versions.len());
for _ in 0..versions.len() {
errors.push(Some(Error::other(err.to_string())));
}
return errors;
}
let request = match encode_delete_versions_request(self.endpoint.to_string(), volume, &versions, &opts) {
Ok(request) => request,
Err(err) => return vec![Some(Error::other(err.to_string())); versions.len()],
};
let opts = match serde_json::to_string(&opts) {
Ok(opts) => opts,
Err(err) => {
let mut errors = Vec::with_capacity(versions.len());
for _ in 0..versions.len() {
errors.push(Some(Error::other(err.to_string())));
}
return errors;
}
};
let mut versions_str = Vec::with_capacity(versions.len());
let mut versions_bin = Vec::with_capacity(versions.len());
for file_info_versions in versions.iter() {
versions_str.push(match serde_json::to_string(file_info_versions) {
Ok(versions_str) => versions_str,
Err(err) => {
let mut errors = Vec::with_capacity(versions.len());
for _ in 0..versions.len() {
errors.push(Some(Error::other(err.to_string())));
}
return errors;
}
});
versions_bin.push(match encode_file_info_versions_msgpack(file_info_versions) {
Ok(versions_bin) => Bytes::from(versions_bin),
Err(err) => {
let mut errors = Vec::with_capacity(versions.len());
for _ in 0..versions.len() {
errors.push(Some(Error::other(err.to_string())));
}
return errors;
}
});
}
let mut client = match self.get_client().await {
Ok(client) => client,
Err(err) => {
@@ -3604,14 +3597,7 @@ impl DiskAPI for RemoteDisk {
}
};
let mut request = Request::new(DeleteVersionsRequest {
disk: self.endpoint.to_string(),
volume: volume.to_string(),
versions: versions_str,
opts,
versions_bin,
opts_bin: opts_bin.into(),
});
let mut request = Request::new(request);
let canonical_body = rustfs_protos::canonical_delete_versions_request_body(request.get_ref());
if let Err(err) = attach_mutation_body_digest(&mut request, canonical_body, "delete_versions") {
let mut errors = Vec::with_capacity(versions.len());
@@ -6337,6 +6323,94 @@ mod tests {
}
}
#[test]
fn delete_versions_request_encoding_preserves_batches_and_fleet_guards() {
use tonic_prost::prost::Message;
let groups: Vec<_> = ["first", "second"]
.into_iter()
.map(|name| FileInfoVersions {
volume: "bucket".to_owned(),
name: name.to_owned(),
versions: vec![sample_rename_data_file_info()],
..Default::default()
})
.collect();
let opts = DeleteOptions {
undo_write: true,
old_data_dir: Some(Uuid::from_u128(7)),
..Default::default()
};
let mut binary_wire_size = 0;
let mut compat_wire_size = 0;
for (requested, confirmed, peer_capability, binary_only) in [
(None, None, None, false),
(Some("true"), None, None, false),
(Some("false"), Some("true"), None, false),
(Some("true"), Some("false"), None, false),
(Some("true"), None, Some("true"), false),
(Some("true"), Some("false"), Some("true"), false),
(Some("true"), Some("true"), None, true),
(Some("true"), Some("true"), Some("true"), true),
] {
with_internode_msgpack_env(
[
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_ONLY, requested),
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED, confirmed),
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY, peer_capability),
],
|| {
for batch in [&groups[..], &[]] {
let disk = "disk".to_owned();
let disk_buffer = disk.as_ptr();
let request = encode_delete_versions_request(disk, "bucket", batch, &opts).expect("encode batch");
assert_eq!(request.disk, "disk");
assert_eq!(request.disk.as_ptr(), disk_buffer, "move the prepared endpoint into the request");
assert_eq!(request.volume, "bucket");
assert_eq!(request.versions_bin.len(), batch.len());
assert_eq!(request.opts.is_empty(), binary_only);
let decoded_opts: DeleteOptions = rmp_serde::from_slice(&request.opts_bin).expect("binary options");
assert!(decoded_opts.undo_write);
assert_eq!(decoded_opts.old_data_dir, opts.old_data_dir);
for (encoded, expected) in request.versions_bin.iter().zip(batch) {
let decoded: FileInfoVersions = rmp_serde::from_slice(encoded).expect("binary batch item");
assert_eq!(decoded.name, expected.name);
assert_eq!(decoded.versions, expected.versions);
}
if binary_only {
assert!(request.versions.is_empty());
} else {
assert_eq!(request.opts, serde_json::to_string(&opts).expect("legacy options"));
assert_eq!(request.versions.len(), batch.len());
for (encoded, expected) in request.versions.iter().zip(batch) {
let decoded: FileInfoVersions = serde_json::from_str(encoded).expect("legacy batch item");
assert_eq!(decoded.name, expected.name);
assert_eq!(decoded.versions, expected.versions);
}
}
let canonical = rustfs_protos::canonical_delete_versions_request_body(&request).expect("bound batch");
let mut modified = request.clone();
modified.opts_bin = encode_delete_options_msgpack(&DeleteOptions::default())
.expect("other options")
.into();
assert_ne!(
canonical,
rustfs_protos::canonical_delete_versions_request_body(&modified).expect("other bound batch")
);
if !batch.is_empty() {
if binary_only {
binary_wire_size = request.encoded_len();
} else {
compat_wire_size = request.encoded_len();
}
}
}
},
);
}
assert!(binary_wire_size > 0 && binary_wire_size < compat_wire_size);
println!("delete batch encoded bytes compat={compat_wire_size} msgpack_only={binary_wire_size}");
}
#[test]
fn msgpack_eligible_requests_bind_one_policy_and_bounded_local_retry() {
let source = rustfs_protos::compat_manifest::production_source(include_str!("remote_disk.rs"), "remote_disk.rs");
@@ -7287,7 +7361,10 @@ mod tests {
);
}
fn with_internode_msgpack_env<R>(vars: [(&'static str, Option<&'static str>); 2], f: impl FnOnce() -> R) -> R {
fn with_internode_msgpack_env<R, const N: usize>(
vars: [(&'static str, Option<&'static str>); N],
f: impl FnOnce() -> R,
) -> R {
temp_env::with_vars(vars, || {
rustfs_protos::reset_internode_rpc_msgpack_only_cache();
let result = f();
+57 -11
View File
@@ -20,8 +20,8 @@ use async_trait::async_trait;
use bytes::Bytes;
use futures::{FutureExt, future::BoxFuture};
use rustfs_lock::{
BatchAcquireOutcome, LockCleanupOutcome, LockClient, LockError, LockInfo, LockRequest, LockResponse, LockStats, LockStatus,
LockType, Result,
BatchAcquireOutcome, CacheFenceStamp, LockCleanupOutcome, LockClient, LockError, LockInfo, LockRequest, LockResponse,
LockStats, LockStatus, LockType, Result,
types::{LockId, LockMetadata, LockPriority},
};
use rustfs_protos::proto_gen::node_service::{
@@ -52,6 +52,10 @@ fn attach_lock_mutation_body_digest<T: rustfs_protos::CanonicalMutationBody>(req
set_tonic_rolling_mutation_body_digest(request)
}
fn decode_cache_fence_stamp(encoded: Option<&Bytes>) -> Option<CacheFenceStamp> {
encoded.and_then(|encoded| CacheFenceStamp::decode(encoded).ok())
}
/// Work to run if an RPC completes after its caller stops waiting.
type LateCompletion<T> = Option<Box<dyn FnOnce(T) -> Pin<Box<dyn Future<Output = ()> + Send>> + Send>>;
type LateFailure = LateCompletion<()>;
@@ -324,6 +328,7 @@ fn batch_acquire_outcome(lock_id: &LockId, lock_type: LockType, result: Option<&
return BatchAcquireOutcome::Granted {
lock_info,
wait_time: Duration::ZERO,
cache_fence_stamp: decode_cache_fence_stamp(result.cache_fence_stamp.as_ref()),
};
}
if result.lock_info.is_some() {
@@ -452,6 +457,7 @@ impl RemoteClient {
deadlock_detection: false,
suppress_contention_logs: false,
refresh_interval: None,
cache_fence_stamp_requested: false,
}
}
@@ -1573,10 +1579,11 @@ impl LockClient for RemoteClient {
// Check if the lock acquisition was successful
if resp.success {
Ok(LockResponse::success(
Self::build_lock_info(request, resp.lock_info),
std::time::Duration::ZERO,
))
let lock_info = Self::build_lock_info(request, resp.lock_info);
match decode_cache_fence_stamp(resp.cache_fence_stamp.as_ref()) {
Some(stamp) => Ok(LockResponse::success_with_cache_fence_stamp(lock_info, std::time::Duration::ZERO, stamp)),
None => Ok(LockResponse::success(lock_info, std::time::Duration::ZERO)),
}
} else {
// Lock acquisition failed
Ok(LockResponse::failure(
@@ -1594,12 +1601,19 @@ impl LockClient for RemoteClient {
.iter()
.zip(outcomes)
.map(|(request, outcome)| match outcome {
BatchAcquireOutcome::Granted { lock_info, wait_time } => LockResponse::success(
lock_info
.map(|info| *info)
.unwrap_or_else(|| Self::build_lock_info(request, None)),
BatchAcquireOutcome::Granted {
lock_info,
wait_time,
),
cache_fence_stamp,
} => {
let lock_info = lock_info
.map(|info| *info)
.unwrap_or_else(|| Self::build_lock_info(request, None));
match cache_fence_stamp {
Some(stamp) => LockResponse::success_with_cache_fence_stamp(lock_info, wait_time, stamp),
None => LockResponse::success(lock_info, wait_time),
}
}
BatchAcquireOutcome::Ambiguous {
error: error @ LockError::Timeout { .. },
..
@@ -2367,6 +2381,38 @@ mod tests {
assert!(matches!(classify(None), BatchAcquireOutcome::Ambiguous { .. }));
}
#[test]
fn cache_fence_stamp_decoder_accepts_current_wire_shape_and_rejects_malformed_values() {
let mut encoded = Vec::with_capacity(27);
encoded.push(1);
encoded.extend_from_slice(uuid::Uuid::from_u128(7).as_bytes());
encoded.extend_from_slice(&17_u16.to_le_bytes());
encoded.extend_from_slice(&42_u64.to_le_bytes());
let encoded = Bytes::from(encoded);
let stamp = decode_cache_fence_stamp(Some(&encoded)).expect("current stamp wire shape");
assert_eq!(stamp.voter_boot_id(), uuid::Uuid::from_u128(7));
assert_eq!(stamp.shard_index(), 17);
assert_eq!(stamp.generation(), 42);
assert!(decode_cache_fence_stamp(None).is_none(), "legacy responses remain valid without proof");
assert!(decode_cache_fence_stamp(Some(&Bytes::from_static(b"malformed"))).is_none());
let request = test_lock_request(Duration::from_secs(30));
let result = GenerallyLockResult {
success: true,
cache_fence_stamp: Some(encoded),
..Default::default()
};
let BatchAcquireOutcome::Granted {
cache_fence_stamp: Some(batch_stamp),
..
} = batch_acquire_outcome(&request.lock_id, request.lock_type, Some(&result))
else {
panic!("valid batch grant must retain its cache-fence stamp");
};
assert_eq!(batch_stamp, stamp);
}
#[tokio::test(start_paused = true)]
async fn batch_success_crossing_deadline_in_one_poll_runs_late_cleanup_once() {
use crate::cluster::rpc::client::NoOpInterceptor;
+39 -40
View File
@@ -101,7 +101,6 @@ const DEFAULT_RRS_STORAGE_CLASS: &str = "EC:1";
const ZERO_SET_DRIVE_COUNT_ERROR: &str = "set drive count must be greater than zero";
pub static DEFAULT_INLINE_BLOCK: usize = 128 * 1024;
const DEFAULT_INLINE_OBJECT_BUDGET: usize = 2 * DEFAULT_INLINE_BLOCK;
pub static DEFAULT_KVS: LazyLock<KVS> = LazyLock::new(|| {
let kvs = vec![
@@ -152,6 +151,7 @@ pub struct Config {
inline_block: usize,
initialized: bool,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
// Preserve the explicit-limit marker when configurations round-trip through older readers.
inline_block_explicit: bool,
#[serde(skip)]
standard_parities: Vec<PoolParity>,
@@ -245,8 +245,10 @@ impl Config {
return false;
}
let shard_size = shard_size as usize;
let inline_block = self.effective_inline_block(data_shards);
let Ok(shard_size) = usize::try_from(shard_size) else {
return false;
};
let inline_block = self.inline_block();
if versioned {
shard_size <= inline_block / 8
@@ -255,27 +257,7 @@ impl Config {
}
}
/// Returns the per-shard inline budget used by both write admission and
/// legacy read fallback.
///
/// The default budget is scaled by the number of data shards so a wider EC
/// layout does not silently increase the maximum inline object size. An
/// explicitly configured `inline_block` remains a fixed per-shard limit for
/// compatibility with deployments that opted into the historical policy.
pub(crate) fn effective_inline_block(&self, data_shards: usize) -> usize {
if data_shards == 0 {
return 0;
}
if self.initialized && self.inline_block_explicit {
self.inline_block
} else {
// Keep the historical two-data-shard object budget while preventing
// wider EC layouts from multiplying the maximum inline object size.
DEFAULT_INLINE_OBJECT_BUDGET.div_ceil(data_shards).min(DEFAULT_INLINE_BLOCK)
}
}
/// Per-shard inline data budget, defaulting to 128 KiB across EC layouts.
pub fn inline_block(&self) -> usize {
if !self.initialized {
DEFAULT_INLINE_BLOCK
@@ -567,7 +549,7 @@ mod tests {
}
#[test]
fn should_inline_scales_default_threshold_by_data_shards() {
fn should_inline_uses_fixed_default_threshold_for_every_ec_layout() {
let config = lookup_config_for_pools_with_env(&KVS::new(), &[3, 12], no_env_overrides())
.expect("default inline policy should resolve for EC2+1 and EC8+4");
@@ -576,10 +558,14 @@ mod tests {
("EC2+1 unversioned above", 128 * 1024 + 1, 2, false, false),
("EC2+1 versioned exact", 16 * 1024, 2, true, true),
("EC2+1 versioned above", 16 * 1024 + 1, 2, true, false),
("EC8+4 unversioned exact", 32 * 1024, 8, false, true),
("EC8+4 unversioned above", 32 * 1024 + 1, 8, false, false),
("EC8+4 versioned exact", 4 * 1024, 8, true, true),
("EC8+4 versioned above", 4 * 1024 + 1, 8, true, false),
("EC8+4 unversioned exact", 128 * 1024, 8, false, true),
("EC8+4 unversioned above", 128 * 1024 + 1, 8, false, false),
("EC8+4 versioned exact", 16 * 1024, 8, true, true),
("EC8+4 versioned above", 16 * 1024 + 1, 8, true, false),
("EC12+4 unversioned exact", 128 * 1024, 12, false, true),
("EC12+4 unversioned above", 128 * 1024 + 1, 12, false, false),
("EC12+4 versioned exact", 16 * 1024, 12, true, true),
("EC12+4 versioned above", 16 * 1024 + 1, 12, true, false),
("negative", -1, 2, false, false),
("zero data shards", 0, 0, false, false),
] {
@@ -626,37 +612,35 @@ mod tests {
let shard_size = erasure.shard_file_size(object_size);
let inline = config.should_inline(shard_size, data_shards, false);
// The effective policy is monotonic across object sizes. This
// table covers the boundaries that previously exposed the
// fixed-shard read-ahead mismatch, including the 1 MiB case.
// The per-shard budget stays monotonic across object sizes.
// PUT separately bounds inline body materialization.
assert!(!inline || previous, "inline decision must not re-enable at {object_size} bytes");
previous = inline;
}
assert!(
!config.should_inline(erasure.shard_file_size(1024 * 1024), data_shards, false),
"1 MiB must use the non-inline path for EC{data_shards}+{parity_shards}"
config.should_inline(erasure.shard_file_size(1024 * 1024), data_shards, false),
"1 MiB fits the per-shard budget for EC{data_shards}+{parity_shards}"
);
}
}
#[test]
fn effective_inline_block_scales_default_budget_and_preserves_explicit_limit() {
fn inline_block_preserves_fixed_default_and_explicit_limits() {
let config = Config::default();
assert_eq!(config.effective_inline_block(8), 32 * 1024);
assert_eq!(config.effective_inline_block(12), 21_846);
assert_eq!(config.effective_inline_block(0), 0);
assert_eq!(config.inline_block(), 128 * 1024);
assert!(!config.should_inline(0, 0, false));
let explicit = lookup_config_for_pools_with_env(
&KVS::new(),
&[12],
StorageClassEnvOverrides {
inline_block: Some("128KiB".to_string()),
inline_block: Some("32KiB".to_string()),
..Default::default()
},
)
.expect("explicit inline block should resolve");
assert_eq!(explicit.effective_inline_block(12), 128 * 1024);
assert_eq!(explicit.inline_block(), 32 * 1024);
}
#[test]
@@ -905,6 +889,21 @@ mod tests {
assert!(decoded.should_inline(100 * 1024, 8, false));
}
#[test]
fn legacy_scalar_inline_config_preserves_limit_without_explicit_marker() {
let cfg = lookup_config_for_pools_with_env(&KVS::new(), &[12], no_env_overrides())
.expect("default storage class should resolve");
let mut legacy = serde_json::to_value(cfg).expect("serialize scalar config");
legacy["inline_block"] = serde_json::json!(64 * 1024);
legacy.as_object_mut().expect("config object").remove("inline_block_explicit");
let decoded: Config = serde_json::from_value(legacy).expect("read historical scalar config");
assert!(decoded.should_inline(64 * 1024, 8, false));
assert!(!decoded.should_inline(64 * 1024 + 1, 8, false));
assert!(decoded.should_inline(8 * 1024, 8, true));
assert!(!decoded.should_inline(8 * 1024 + 1, 8, true));
}
#[test]
fn lookup_config_reads_rrs_from_class_rrs_key() {
// Regression: kvs.get(RRS) used RRS="REDUCED_REDUNDANCY" instead of
+3
View File
@@ -180,6 +180,9 @@ pub(crate) const GET_METADATA_CACHE_DECISION_SKIP: &str = "skip";
pub(crate) const GET_METADATA_CACHE_REASON_DATA_MOVEMENT: &str = "data_movement";
pub(crate) const GET_METADATA_CACHE_REASON_DELETE_MARKER: &str = "delete_marker";
pub(crate) const GET_METADATA_CACHE_REASON_DIST_ERASURE: &str = "dist_erasure";
pub(crate) const GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST: &str = "fence_lock_lost";
pub(crate) const GET_METADATA_CACHE_REASON_FENCE_PROOF_MISMATCH: &str = "fence_proof_mismatch";
pub(crate) const GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE: &str = "fence_proof_unavailable";
pub(crate) const GET_METADATA_CACHE_REASON_INCL_FREE_VERSIONS: &str = "incl_free_versions";
pub(crate) const GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM: &str = "insufficient_cached_quorum";
pub(crate) const GET_METADATA_CACHE_REASON_BODY_LAYOUT_REQUIRED: &str = "body_layout_required";
File diff suppressed because it is too large Load Diff
+245 -102
View File
@@ -231,14 +231,26 @@ fn rollback_metadata_std(destination_metadata: &Path, rollback_data_dir: Option<
let Some(dst_parent) = destination_metadata.parent() else {
return Err(std::io::Error::new(ErrorKind::InvalidInput, "missing object metadata parent"));
};
let backup_path = dst_parent.join(old_data_dir.to_string()).join(STORAGE_FORMAT_FILE_BACKUP);
std::fs::rename(backup_path, destination_metadata)?;
let rollback_parent = dst_parent.join(old_data_dir.to_string());
let backup_path = rollback_parent.join(STORAGE_FORMAT_FILE_BACKUP);
std::fs::rename(&backup_path, destination_metadata)?;
let _ = remove_file_if_exists(&backup_path);
let _ = std::fs::remove_dir(rollback_parent);
} else {
remove_file_if_exists(destination_metadata)?;
}
Ok(())
}
#[cfg(target_os = "linux")]
pub(super) fn discard_staged_inline_rollback_backup_std(
old_data_dir: Uuid,
has_old_data_dir: Option<Uuid>,
commit_guard: &os::RenameCommitGuard,
) {
os::discard_inline_rollback_backup_with_commit_guard_std(&old_data_dir.to_string(), has_old_data_dir.is_none(), commit_guard);
}
fn rollback_inline_metadata_std(
destination_metadata: &Path,
rollback_data_dir: Option<Uuid>,
@@ -377,34 +389,77 @@ pub(super) async fn lock_rename_commit_directories(
result.map_err(to_file_error).map_err(DiskError::from)
}
#[cfg(target_os = "linux")]
#[derive(Clone, Copy)]
enum SourcePreparationMode {
CreateParent,
ExistingParent,
FlatInlineTmp { defer_dst_preflight: bool },
}
#[cfg(target_os = "linux")]
enum PrepareCommitAndDestinationError {
PreflightAccess(std::io::Error),
Io(std::io::Error),
}
#[cfg(target_os = "linux")]
fn prepare_rename_commit_and_destination_metadata_std(
create_source_parent: bool,
mode: SourcePreparationMode,
source_parent: &Path,
destination_parent: &Path,
destination_metadata: &Path,
base_dir: &Path,
publication_root: &os::PublicationRoot,
) -> std::io::Result<(os::RenameCommitGuard, Option<Bytes>)> {
if create_source_parent {
std::fs::create_dir_all(source_parent)?;
) -> core::result::Result<(os::RenameCommitGuard, Option<os::ReadDestinationFile>), PrepareCommitAndDestinationError> {
if matches!(mode, SourcePreparationMode::CreateParent) {
std::fs::create_dir_all(source_parent).map_err(PrepareCommitAndDestinationError::Io)?;
}
let guard = match os::prepare_rename_commit_guard(source_parent, destination_parent, base_dir, publication_root) {
let defer_dst_preflight = matches!(
mode,
SourcePreparationMode::FlatInlineTmp {
defer_dst_preflight: true
}
);
let guard_res = if matches!(mode, SourcePreparationMode::FlatInlineTmp { .. }) {
os::prepare_rename_commit_guard_flat_inline(source_parent, destination_parent, base_dir, publication_root)
} else {
os::prepare_rename_commit_guard(source_parent, destination_parent, base_dir, publication_root)
};
let guard = match guard_res {
Ok(guard) => guard,
Err(err) => {
return Err(match std::fs::symlink_metadata(base_dir) {
if err
.get_ref()
.and_then(|inner| inner.downcast_ref::<DiskError>())
.is_some_and(|disk_err| matches!(disk_err, DiskError::InvalidPath))
{
return Err(PrepareCommitAndDestinationError::Io(err));
}
if defer_dst_preflight && let Err(access_err) = super::super::fs::access_std(base_dir) {
return Err(PrepareCommitAndDestinationError::PreflightAccess(access_err));
}
return Err(PrepareCommitAndDestinationError::Io(match std::fs::symlink_metadata(base_dir) {
Err(base_err) if base_err.kind() == ErrorKind::NotFound => base_err,
_ => err,
});
}));
}
};
let data = os::read_destination_file_with_commit_guard(destination_metadata, &guard)?;
Ok((guard, data.map(Bytes::from)))
if defer_dst_preflight
&& !guard.has_prepared_destination()
&& let Err(access_err) = super::super::fs::access_std(base_dir)
{
return Err(PrepareCommitAndDestinationError::PreflightAccess(access_err));
}
let data = os::read_destination_file_handle_with_commit_guard(destination_metadata, &guard)
.map_err(PrepareCommitAndDestinationError::Io)?;
Ok((guard, data))
}
#[cfg(target_os = "linux")]
enum FusedCommitAttempt {
Committed(Box<RenameDataResp>),
PreflightAccessErr(std::io::Error),
CommitErr(DiskError),
RestoreDataSource(os::RenameCommitGuard, DiskError),
Fallback(os::RenameCommitGuard, Option<Bytes>, Box<FileInfo>),
@@ -412,6 +467,14 @@ enum FusedCommitAttempt {
ResumePanic(Box<dyn std::any::Any + Send + 'static>),
}
/// File fdatasync grouping only owns staged shard files. Inline commits have
/// no shard directory, so the switch must not evict them from the flat/fused
/// metadata path where grouping can never run.
#[cfg(any(target_os = "linux", test))]
pub(super) fn file_fdatasync_grouping_requires_async_shard_sync(no_inline: bool) -> bool {
no_inline && os::file_fdatasync_group_commit_enabled()
}
async fn prepare_rename_commit_and_destination_metadata(
create_source_parent: bool,
source_parent: &Path,
@@ -567,7 +630,20 @@ impl LocalDisk {
// object commit is publishing into it. The peer's empty scan remains
// optimistic; this lease establishes the local commit/delete order and
// remains owned by any blocking syscall that outlives async cancellation.
let (dst_volume_dir, destination_object_path) = self.io_get_bucket_and_object_path(dst_volume, dst_path)?;
let dst_dir_group_commit = os::dst_dir_fsync_group_commit_enabled();
let flat_inline_tmp = cfg!(target_os = "linux")
&& (fi.data.is_some() || fi.size <= 0)
&& src_volume == super::super::RUSTFS_META_TMP_BUCKET
&& pool_meta_condition.is_none()
&& !fi.is_healing()
&& !dst_dir_group_commit
&& !has_rename_data_test_failpoints(dst_path);
let defer_dst_preflight = flat_inline_tmp && !skip_access_checks(dst_volume);
let (dst_volume_dir, destination_object_path) = if defer_dst_preflight {
self.io_get_bucket_and_object_open_path(dst_volume, dst_path)?
} else {
self.io_get_bucket_and_object_path(dst_volume, dst_path)?
};
let quota_fence_token =
match rustfs_utils::http::metadata_compat::get_consistent_str(&fi.metadata, QUOTA_MUTATION_FENCE_METADATA_SUFFIX) {
Some(value) => {
@@ -716,7 +792,8 @@ impl LocalDisk {
return Err(to_access_error(e, DiskError::VolumeAccessDenied).into());
}
if !skip_access_checks(dst_volume)
if !defer_dst_preflight
&& !skip_access_checks(dst_volume)
&& let Err(e) = super::super::fs::access_std(&dst_volume_dir)
{
info!(
@@ -742,16 +819,6 @@ impl LocalDisk {
} else {
STORAGE_FORMAT_FILE
};
let flat_inline_tmp = cfg!(target_os = "linux")
&& !no_inline
&& src_volume == super::super::RUSTFS_META_TMP_BUCKET
&& !preserve_upload_metadata
&& multipart_publication.is_none()
&& pool_meta_condition.is_none()
&& !fi_healing
&& !os::dst_dir_fsync_group_commit_enabled()
&& !os::file_fdatasync_group_commit_enabled()
&& !has_rename_data_test_failpoints(dst_path);
let src_file_path = match multipart_publication.as_ref() {
Some(publication) => publication.canonical_head(),
None if flat_inline_tmp => {
@@ -833,12 +900,13 @@ impl LocalDisk {
.ok_or_else(|| DiskError::other("missing object metadata parent"))?;
#[cfg(target_os = "linux")]
let (rename_commit_guard, has_dst_buf) = if !preserve_upload_metadata
&& multipart_publication.is_none()
&& pool_meta_condition.is_none()
&& !fi_healing
&& !os::dst_dir_fsync_group_commit_enabled()
&& !os::file_fdatasync_group_commit_enabled()
let (rename_commit_guard, has_dst_buf) = if flat_inline_tmp
|| (!preserve_upload_metadata
&& multipart_publication.is_none()
&& pool_meta_condition.is_none()
&& !fi_healing
&& !dst_dir_group_commit
&& !file_fdatasync_grouping_requires_async_shard_sync(no_inline))
{
#[cfg(test)]
if !no_inline {
@@ -873,25 +941,45 @@ impl LocalDisk {
let root_owned = self.root.clone();
let fused_attempt = move |is_cancelled: &dyn Fn() -> bool| -> std::io::Result<FusedCommitAttempt> {
let (guard, has_dst_buf) = prepare_rename_commit_and_destination_metadata_std(
!no_inline && !flat_inline_tmp,
let source_mode = if flat_inline_tmp {
SourcePreparationMode::FlatInlineTmp { defer_dst_preflight }
} else if !no_inline {
SourcePreparationMode::CreateParent
} else {
SourcePreparationMode::ExistingParent
};
let (guard, old_destination_file) = match prepare_rename_commit_and_destination_metadata_std(
source_mode,
&src_parent_owned,
&dst_parent_owned,
&dst_file_path_owned,
&dst_volume_dir_owned,
&publication_root_owned,
)?;
) {
Ok(res) => res,
Err(PrepareCommitAndDestinationError::PreflightAccess(err)) => {
return Ok(FusedCommitAttempt::PreflightAccessErr(err));
}
Err(PrepareCommitAndDestinationError::Io(err)) => return Err(err),
};
// An existing xl.meta that replaces an existing version slot
// requires a rollback directory; appending a new version keeps
// an inline rollback backup for post-rename directory sync errors.
if is_cancelled() {
return Ok(FusedCommitAttempt::Fallback(guard, has_dst_buf, Box::new(fi_moved)));
if flat_inline_tmp {
guard.ensure_source_pinned()?;
}
return Ok(FusedCommitAttempt::Fallback(
guard,
old_destination_file.map(|read| read.bytes),
Box::new(fi_moved),
));
}
let mut xlmeta = FileMeta::new();
let mut dst_meta_unparsable = false;
if let Some(ref dst_buf) = has_dst_buf {
if FileMeta::is_xl2_v1_format(dst_buf)
&& let Ok(nmeta) = FileMeta::load(dst_buf)
if let Some(ref old_file) = old_destination_file {
if FileMeta::is_xl2_v1_format(&old_file.bytes)
&& let Ok(nmeta) = FileMeta::load(&old_file.bytes)
{
xlmeta = nmeta;
} else {
@@ -901,14 +989,21 @@ impl LocalDisk {
let version_id = fi_moved.version_id.unwrap_or_default();
let has_old_data_dir = xlmeta.find_unshared_data_dir_for_version(Some(version_id));
let rollback_data_dir = has_old_data_dir.or_else(|| {
if has_dst_buf.is_some() && xlmeta.find_version(Some(version_id)).is_ok() {
if old_destination_file.is_some() && xlmeta.find_version(Some(version_id)).is_ok() {
Some(inline_metadata_rollback_dir(version_id, &xlmeta))
} else {
None
}
});
if rollback_data_dir.is_some() && has_rename_data_test_failpoints(&dst_path_str) {
return Ok(FusedCommitAttempt::Fallback(guard, has_dst_buf, Box::new(fi_moved)));
if flat_inline_tmp {
guard.ensure_source_pinned()?;
}
return Ok(FusedCommitAttempt::Fallback(
guard,
old_destination_file.map(|read| read.bytes),
Box::new(fi_moved),
));
}
#[cfg(test)]
let probe_active = os::file_sync_probe::is_active();
@@ -920,10 +1015,17 @@ impl LocalDisk {
&& (probe_active || !super::is_presynced_tmp_data_dir(src_data_path))
&& os::regular_files(src_data_path).is_ok_and(|files| files.len() >= os::PARALLEL_FILE_SYNC_THRESHOLD)
{
return Ok(FusedCommitAttempt::Fallback(guard, has_dst_buf, Box::new(fi_moved)));
if flat_inline_tmp {
guard.ensure_source_pinned()?;
}
return Ok(FusedCommitAttempt::Fallback(
guard,
old_destination_file.map(|read| read.bytes),
Box::new(fi_moved),
));
}
let (old_current_size, old_current_source_checked, old_current_source) = observe_old_current(
has_dst_buf.is_some(),
old_destination_file.is_some(),
dst_meta_unparsable,
&xlmeta,
&dst_volume_str,
@@ -936,9 +1038,9 @@ impl LocalDisk {
xlmeta.add_version(fi_moved)?;
let version_signature = rename_data_versions_signature(&xlmeta);
let new_dst_buf = xlmeta.marshal_msg().map_err(std::io::Error::other)?;
let destination_was_absent = has_dst_buf.is_none();
let destination_was_absent = old_destination_file.is_none();
let mut skip_parent = dst_volume_dir_owned.clone();
if has_dst_buf.is_some() {
if old_destination_file.is_some() {
skip_parent = dst_parent_owned.clone();
}
@@ -948,7 +1050,7 @@ impl LocalDisk {
os::create_prepared_rename_source_with_commit_guard(&src_file_path_owned, &dst_file_path_owned, &guard)?;
let cleanup_flat_tmp = || {
if flat_inline_tmp {
let _ = remove_file_if_exists(&src_file_path_owned);
os::unlink_flat_tmp_source_std(&src_file_path_owned, &src_parent_owned, &publication_root_owned);
}
};
if let Err(err) = prepared_metadata_source.write_all(&new_dst_buf, false) {
@@ -961,14 +1063,14 @@ impl LocalDisk {
}
let mut staged_rollback_path = None;
if (!no_inline || rollback_data_dir.is_none())
&& let Some(ref old_metadata) = has_dst_buf
&& (rollback_data_dir.is_some() || cfg!(test))
if rollback_data_dir.is_none()
&& let Some(ref old_file) = old_destination_file
&& cfg!(test)
{
let backup_path = match create_local_inline_rollback_backup(
&dst_file_path_owned,
&src_file_path_owned,
old_metadata,
&old_file.bytes,
sync_recovery_protocol,
) {
Ok(path) => path,
@@ -1088,28 +1190,22 @@ impl LocalDisk {
}
return Ok(FusedCommitAttempt::RestoreDataSource(guard, DiskError::Unexpected));
}
if let (Some(old_data_dir), Some(dst_buf)) = (rollback_data_dir, has_dst_buf.as_ref()) {
if let (Some(old_data_dir), Some(old_file)) = (rollback_data_dir, old_destination_file.as_ref()) {
let backup_parent = dst_parent_owned.join(old_data_dir.to_string());
if let Err(err) =
os::mkdir_all_below_existing_base_std(&backup_parent, &skip_parent, &publication_root_owned)
{
return Ok(FusedCommitAttempt::RestoreDataSource(guard, to_file_error(err).into()));
}
let backup_path_guard = match guard.create_destination_directory_for_path_access(&backup_parent) {
Ok(g) => g,
Err(err) => return Ok(FusedCommitAttempt::RestoreDataSource(guard, to_file_error(err).into())),
};
let backup_path = backup_parent.join(STORAGE_FORMAT_FILE_BACKUP);
if let Err(err) = check_path_length(backup_path.to_string_lossy().as_ref()) {
return Ok(FusedCommitAttempt::RestoreDataSource(guard, err));
}
#[cfg(test)]
run_owned_file_before_open(&backup_path);
if let Err(err) = backup_path_guard.write_file_for_path_access(
if let Err(err) = os::stage_inline_rollback_backup_with_commit_guard_std(
&dst_file_path_owned,
&backup_parent,
&backup_path,
dst_buf.as_ref(),
sync_recovery_protocol,
old_file,
sync_recovery_protocol,
!should_fail_local_inline_rollback_hardlink(&dst_file_path_owned),
&guard,
) {
return Ok(FusedCommitAttempt::RestoreDataSource(guard, to_file_error(err).into()));
}
@@ -1117,25 +1213,32 @@ impl LocalDisk {
if crash_inject::should_crash_at(CrashPoint::RenameAfterBackupBeforeMetaCommit, &dst_path_str) {
return Ok(FusedCommitAttempt::CommitErr(DiskError::Unexpected));
}
} else if let (Some(old_data_dir), Some(staged_backup)) =
(rollback_data_dir, staged_rollback_path.as_ref().map(StagedRollbackBackup::path))
{
} else if let (Some(old_data_dir), Some(old_file)) = (rollback_data_dir, old_destination_file.as_ref()) {
let backup_parent = dst_parent_owned.join(old_data_dir.to_string());
let backup_path = backup_parent.join(STORAGE_FORMAT_FILE_BACKUP);
if let Err(err) = os::rename_all_with_precondition_and_parent_sync_std(
staged_backup,
if let Err(err) = check_path_length(backup_path.to_string_lossy().as_ref()) {
drop(prepared_metadata_source);
cleanup_flat_tmp();
return Ok(FusedCommitAttempt::CommitErr(err));
}
#[cfg(test)]
run_owned_file_before_open(&backup_path);
if let Err(err) = os::stage_inline_rollback_backup_with_commit_guard_std(
&dst_file_path_owned,
&backup_parent,
&backup_path,
&skip_parent,
&publication_root_owned,
None,
old_file,
sync_recovery_protocol,
!should_fail_local_inline_rollback_hardlink(&dst_file_path_owned),
&guard,
) {
drop(prepared_metadata_source);
cleanup_flat_tmp();
let _ = remove_file_if_exists(staged_backup);
return Ok(FusedCommitAttempt::CommitErr(to_file_error(err).into()));
}
staged_rollback_path = None;
if crash_inject::should_crash_at(CrashPoint::RenameAfterBackupBeforeMetaCommit, &dst_path_str) {
return Ok(FusedCommitAttempt::CommitErr(DiskError::Unexpected));
}
}
let rename_res = if should_fail_commit_rename(&dst_path_str) {
@@ -1156,6 +1259,9 @@ impl LocalDisk {
if let Some(backup_path) = staged_rollback_path.as_ref().map(StagedRollbackBackup::path) {
let _ = remove_file_if_exists(backup_path);
}
if let Some(old_data_dir) = rollback_data_dir {
discard_staged_inline_rollback_backup_std(old_data_dir, has_old_data_dir, &guard);
}
if no_inline {
return Ok(FusedCommitAttempt::RestoreDataSource(guard, err));
}
@@ -1181,10 +1287,10 @@ impl LocalDisk {
return Ok(FusedCommitAttempt::CommitErr(DiskError::Unexpected));
}
if sync_commit_directories {
if durability.syncs_metadata_commit_directory(rollback_data_dir.is_some()) {
if staged_rollback_path.is_none()
&& rollback_data_dir.is_none()
&& let Some(ref old_metadata) = has_dst_buf
&& let Some(ref old_file) = old_destination_file
&& (!no_inline || rollback_data_dir.is_none())
{
let fsync_res = (|| -> std::io::Result<()> {
@@ -1195,7 +1301,7 @@ impl LocalDisk {
})();
if let Err(err) = fsync_res {
let rollback_ok =
write_local_inline_rollback_backup_from_bytes(&src_file_path_owned, old_metadata, true)
write_local_inline_rollback_backup_from_bytes(&src_file_path_owned, &old_file.bytes, true)
.and_then(|backup_path| {
let backup = StagedRollbackBackup(backup_path);
let res =
@@ -1270,6 +1376,20 @@ impl LocalDisk {
}
return Ok(*resp);
}
Ok(FusedCommitAttempt::PreflightAccessErr(e)) => {
info!(
target: "rustfs_ecstore::disk::local",
event = EVENT_DISK_LOCAL_ACCESS_FAILED,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_DISK_LOCAL,
path = ?dst_volume_dir,
operation = "rename_data_dst_access",
error = %e,
"Disk local access check failed"
);
state.preflight_rejection = Some(LocalRenamePreflightRejection(()));
return Err(to_access_error(e, DiskError::VolumeAccessDenied).into());
}
Ok(FusedCommitAttempt::RestoreDataSource(directory_guard, err)) => {
let commit = RenameCommitContext {
directory_guard,
@@ -1306,7 +1426,7 @@ impl LocalDisk {
&& !preserve_upload_metadata
&& !publication.legacy
&& !fi_healing
&& !os::dst_dir_fsync_group_commit_enabled()
&& !dst_dir_group_commit
{
enum MultipartFusedCommitOutcome {
Committed(Box<RenameDataResp>),
@@ -1336,15 +1456,19 @@ impl LocalDisk {
true,
move || -> std::io::Result<MultipartFusedCommitOutcome> {
let (directory_guard_owned, has_dst_buf_owned) = match prepare_rename_commit_and_destination_metadata_std(
false,
SourcePreparationMode::ExistingParent,
&src_parent_owned,
&dst_parent_owned,
&dst_file_path_owned,
&dst_volume_dir_owned,
&publication_root_owned,
) {
Ok(res) => res,
Err(err) => return Ok(MultipartFusedCommitOutcome::AbortBeforeDataRename(to_file_error(err).into())),
Ok((guard, old_file)) => (guard, old_file.map(|read| read.bytes)),
Err(
PrepareCommitAndDestinationError::PreflightAccess(err) | PrepareCommitAndDestinationError::Io(err),
) => {
return Ok(MultipartFusedCommitOutcome::AbortBeforeDataRename(to_file_error(err).into()));
}
};
let mut xlmeta = FileMeta::new();
let mut dst_meta_unparsable = false;
@@ -1882,9 +2006,10 @@ impl LocalDisk {
// durable. It is part of the metadata commit machinery, so relaxed
// tiers leave it to the page cache like the xl.meta it mirrors.
let backup_sync = durability.recovery_sync_mode();
if let (Some(old_data_dir), Some(dst_buf)) = (rollback_data_dir, has_dst_buf.as_ref())
&& multipart_publication.as_ref().is_none_or(|publication| publication.legacy)
{
let recovery_backup = rollback_data_dir
.zip(has_dst_buf.as_ref())
.filter(|_| multipart_publication.as_ref().is_none_or(|publication| publication.legacy));
if let Some((old_data_dir, dst_buf)) = recovery_backup {
let backup_parent = dst_file_parent.join(old_data_dir.to_string());
#[cfg(not(windows))]
if let Err(err) = os::make_dir_all(&backup_parent, &skip_parent).await {
@@ -1998,7 +2123,7 @@ impl LocalDisk {
// Persist the directory entries for both the data dir and xl.meta renames;
// without this the commit itself can vanish on power loss. Relaxed tiers
// accept that window (documented in docs/operations/durability-modes.md).
if durability.syncs_commit_directories()
if durability.syncs_metadata_commit_directory(recovery_backup.is_some())
&& let Some(parent) = dst_file_path.parent()
{
let fsync_started = rustfs_io_metrics::put_stage_timer();
@@ -2147,13 +2272,15 @@ impl LocalDisk {
let dst_path_for_source = dst_path.to_string();
let dst_volume_dir_for_commit = dst_volume_dir.clone();
let publication_root_for_commit = self.publication_root.clone();
let cleanup_src_parent = multipart_publication.is_none() && !preserve_upload_metadata;
let cleanup_src_parent = !flat_inline_tmp && multipart_publication.is_none() && !preserve_upload_metadata;
#[cfg(windows)]
let source_parent = src_file_parent.to_path_buf();
let rename_commit_guard_for_preparation = commit.directory_guard.clone();
let sync = durability.syncs_recovery_protocol_state();
let sync_metadata_files = durability.syncs_metadata_files();
let sync_commit_directories = durability.syncs_commit_directories();
let sync_recovery_protocol = durability.syncs_recovery_protocol_state();
#[cfg(unix)]
let receipt_identity = if sync
let receipt_identity = if sync_recovery_protocol
&& pool_meta_condition
.as_ref()
.is_some_and(|condition| condition.parent_raw_sha256.is_none())
@@ -2181,7 +2308,7 @@ impl LocalDisk {
}
#[cfg(test)]
run_inline_before_file_sync_admission(dst_path);
let mut file_sync_admission = if sync {
let mut file_sync_admission = if sync_metadata_files || sync_commit_directories || sync_recovery_protocol {
let _admission = os::StageTimer::new(native_timing, "pool_meta_native_file_sync_admission");
Some(
os::acquire_file_sync_admission(self.file_sync_permits.clone())
@@ -2249,14 +2376,10 @@ impl LocalDisk {
xlmeta.add_version(fi)?;
let version_signature = rename_data_versions_signature(&xlmeta);
let new_buf = xlmeta.marshal_msg()?;
// Write the staged xl.meta. Inline objects carry their data inside
// xl.meta, so this is the durable preparation for the metadata commit:
// relaxed tiers do no per-object fsync here at all (aligned
// with MinIO's default), trading a documented power-loss
// window for latency.
// Inline payload shares xl.meta's file synchronization policy.
{
let _new_file = os::StageTimer::new(native_timing, "pool_meta_native_staged_write_sync");
prepared_metadata_source.write_all(&new_buf, sync)?;
prepared_metadata_source.write_all(&new_buf, sync_metadata_files)?;
}
run_inline_preparation_before_backup(&dst_path_for_failpoint);
#[cfg(unix)]
@@ -2287,10 +2410,10 @@ impl LocalDisk {
if !backup.is_linked() {
receipt_candidate = None;
}
if sync && !reuses_durability {
if sync_recovery_protocol && !reuses_durability {
let _backup_sync = os::StageTimer::new(native_timing, "pool_meta_native_backup_file_sync");
backup.sync_data()?;
} else if sync {
} else if sync_recovery_protocol {
rustfs_io_metrics::record_put_object_stage_count("pool_meta_native_backup_sync_reused", 1);
}
Some(backup)
@@ -2304,7 +2427,7 @@ impl LocalDisk {
let owned_backup = None;
if let Some(ref old_metadata) = has_dst_buf
&& owned_backup.is_none()
&& (rollback_data_dir.is_some() || fi_healing || sync || cfg!(test))
&& (rollback_data_dir.is_some() || fi_healing || sync_recovery_protocol || cfg!(test))
{
#[cfg(windows)]
let backup_path = {
@@ -2313,13 +2436,18 @@ impl LocalDisk {
.parent()
.ok_or_else(|| std::io::Error::new(ErrorKind::InvalidInput, "missing staging metadata parent"))?
.join(STORAGE_FORMAT_FILE_BACKUP);
source_metadata_guard.write_file_for_path_access(&backup_path, old_metadata, sync, false)?;
source_metadata_guard.write_file_for_path_access(
&backup_path,
old_metadata,
sync_recovery_protocol,
false,
)?;
backup_path
};
#[cfg(not(windows))]
let backup_path = {
let _backup = os::StageTimer::new(native_timing, "pool_meta_native_backup_create");
create_local_inline_rollback_backup(&dst, &src, old_metadata, sync)?
create_local_inline_rollback_backup(&dst, &src, old_metadata, sync_recovery_protocol)?
};
staged_rollback_path = Some(StagedRollbackBackup(backup_path));
}
@@ -2327,7 +2455,7 @@ impl LocalDisk {
let destination_was_absent = has_dst_buf.is_none();
if !is_cancelled()
&& rollback_data_dir.is_none()
&& !os::dst_dir_fsync_group_commit_enabled()
&& (!sync_commit_directories || !os::dst_dir_fsync_group_commit_enabled())
&& !has_inline_commit_test_failpoints(&dst_path_for_failpoint)
{
let rename_res = if should_fail_commit_rename(&dst_path_for_failpoint) {
@@ -2344,6 +2472,10 @@ impl LocalDisk {
)
};
if let Err(err) = rename_res {
#[cfg(target_os = "linux")]
if flat_inline_tmp {
os::unlink_prepared_rename_source_with_commit_guard_std(&src, &rename_commit_guard_for_preparation);
}
if let Some(backup_path) = staged_rollback_path.as_ref().map(StagedRollbackBackup::path) {
let _ = remove_file_if_exists(backup_path);
}
@@ -2357,7 +2489,7 @@ impl LocalDisk {
return Ok(InlinePreparationOutcome::CommitErr(DiskError::from(err)));
}
if sync
if sync_commit_directories
&& let Err(err) = fsync_inline_commit_std(
&dst,
&dst_volume_dir_for_commit,
@@ -2459,6 +2591,10 @@ impl LocalDisk {
}
Ok(InlinePreparationOutcome::CommitErr(err)) | Err(err) => {
drop(file_sync_admission.take());
#[cfg(target_os = "linux")]
if flat_inline_tmp {
os::unlink_prepared_rename_source_with_commit_guard_std(&src_file_path, &commit.directory_guard);
}
for part_path in &invalidate_part_paths {
self.io_backend.invalidate_cached_fd(dst_volume, part_path).await;
}
@@ -2552,7 +2688,7 @@ impl LocalDisk {
}
#[cfg(test)]
run_rename_data_after_first_publication(&self.root, dst_volume, dst_path);
if sync {
if sync_commit_directories || sync_recovery_protocol {
let _admission = os::StageTimer::new(native_timing, "pool_meta_native_file_sync_readmission");
file_sync_admission = Some(
os::acquire_file_sync_admission(self.file_sync_permits.clone())
@@ -2601,6 +2737,10 @@ impl LocalDisk {
};
drop(head_publish);
if let Err(err) = commit_result {
#[cfg(target_os = "linux")]
if flat_inline_tmp {
os::unlink_prepared_rename_source_with_commit_guard_std(&src_file_path, &commit.directory_guard);
}
if let Some(backup_path) = local_rollback_path.as_ref().map(StagedRollbackBackup::path) {
let _ = remove_file_if_exists(backup_path);
}
@@ -2616,8 +2756,10 @@ impl LocalDisk {
return Err(std::io::Error::other("test fail after metadata commit"));
}
// Persist the commit rename's directory entry across power loss.
if let Some(admission) = file_sync_admission.as_ref()
// The object directory also publishes the recovery backup's
// directory entry, which file-sync must keep durable.
if durability.syncs_metadata_commit_directory(rollback_data_dir.is_some())
&& let Some(admission) = file_sync_admission.as_ref()
&& let Some(dst_parent) = dst_file_path.parent()
{
let _head_dir = os::StageTimer::new(native_timing, "pool_meta_native_head_dir_sync");
@@ -2650,7 +2792,8 @@ impl LocalDisk {
// not its own entry, so for a new inline object fsync the ancestor
// chain up to and including the bucket. Overwrites already have a
// durable object dir; the starts_with guard bounds the walk.
if let Some(admission) = file_sync_admission.as_ref()
if sync_commit_directories
&& let Some(admission) = file_sync_admission.as_ref()
&& destination_was_absent
{
let mut ancestor = dst_file_path.parent().and_then(|object_dir| object_dir.parent());
+620 -127
View File
@@ -209,7 +209,7 @@ impl DiscoverySession {
None => regular_bytes(&proof_root.join(DELETE_QUORUM_COMMITTED_FILE))
.map_err(std::io::Error::from)?
.filter(|bytes| !bytes.is_empty())
.and_then(|bytes| serde_json::from_slice::<CommitReceipt>(&bytes).ok())
.and_then(|bytes| parse_commit_receipt(&bytes).ok())
.map(|receipt| receipt.journal),
};
let logical = record
@@ -273,6 +273,21 @@ struct CommitReceipt {
finished: bool,
}
/// Parse a commit receipt from `delete-quorum-committed`.
/// A bare `DeleteRecoveryJournal` (from hardlinking `delete-prepared`) always
/// represents `finished = false`; `finished = true` is only persisted via the
/// `CommitReceipt` wrapper when replacing the receipt entry.
fn parse_commit_receipt(bytes: &[u8]) -> Result<CommitReceipt> {
if let Ok(receipt) = serde_json::from_slice::<CommitReceipt>(bytes) {
return Ok(receipt);
}
let journal = serde_json::from_slice::<DeleteRecoveryJournal>(bytes).map_err(|_| DiskError::FileCorrupt)?;
Ok(CommitReceipt {
journal,
finished: false,
})
}
pub(super) struct DeletePreparationContext {
pub transaction: Uuid,
pub bucket_incarnation: Option<Uuid>,
@@ -866,9 +881,8 @@ pub(super) fn inspection_with_markers_bounded(
let receipt: Option<CommitReceipt> = receipt_bytes
.as_deref()
.filter(|bytes| !bytes.is_empty())
.map(serde_json::from_slice)
.transpose()
.map_err(|_| DiskError::FileCorrupt)?;
.map(parse_commit_receipt)
.transpose()?;
let committed = receipt.is_some();
let finished = regular_marker(&rollback.join(FINISHED))? || receipt.as_ref().is_some_and(|receipt| receipt.finished);
let journal: Option<DeleteRecoveryJournal> = prepared
@@ -1185,6 +1199,7 @@ struct PreparedDeleteReceiptEntry {
journal: DeleteRecoveryJournal,
sole_transaction: bool,
had_old_data_dir: bool,
prepared_synced: bool,
rollback_dir_fd: Option<Arc<std::fs::File>>,
file_path_dir: Option<Arc<std::fs::File>>,
}
@@ -1227,6 +1242,16 @@ fn read_child_regular_bytes(
std::fs::read(fallback_path)
}
fn stage_recovery_file(root: &Path, bytes: &[u8], sync: bool) -> std::io::Result<tempfile::NamedTempFile> {
std::fs::create_dir_all(root)?;
let mut file = tempfile::NamedTempFile::new_in(root)?;
std::io::Write::write_all(file.as_file_mut(), bytes)?;
if sync && !bytes.is_empty() {
file.as_file().sync_data()?;
}
Ok(file)
}
#[cfg(unix)]
fn create_and_write_child_file(
parent_fd: Option<&std::fs::File>,
@@ -1538,11 +1563,7 @@ impl LocalDisk {
.push(serde_json::from_slice::<DeleteRecoveryJournal>(&bytes).map_err(|_| DiskError::FileCorrupt)?);
}
if let Some(bytes) = receipt.filter(|bytes| !bytes.is_empty()) {
records.push(
serde_json::from_slice::<CommitReceipt>(&bytes)
.map_err(|_| DiskError::FileCorrupt)?
.journal,
);
records.push(parse_commit_receipt(&bytes)?.journal);
}
for record in records {
if record.protocol_version != DELETE_RECOVERY_PROTOCOL_VERSION
@@ -1872,34 +1893,40 @@ impl LocalDisk {
let record_destination = destination.clone();
let staging_root = temporary;
let lease = os::acquire_namespace_mutation_lease_with_owner(&destination, owner.clone()).await;
let reserve = self.delete_reserve.clone();
let (staged, target) = os::run_blocking_namespace_operation_with_sync(lease, sync, move || {
std::fs::create_dir_all(&staging_root)?;
let staged = Arc::new(
tempfile::Builder::new()
.prefix("delete-preparation-")
.tempdir_in(&staging_root)?,
);
let missing_marker = original.is_none() && target.is_some();
for (name, bytes) in [
(STORAGE_FORMAT_FILE_BACKUP, original.as_deref()),
(TARGET, target.as_deref()),
(PREPARED, Some(journal.as_slice())),
(DELETE_MARKER_ROLLBACK_FILE, missing_marker.then_some(&[][..])),
] {
let Some(bytes) = bytes else {
continue;
};
#[cfg(test)]
run_owned_file_before_open(&record_destination.join(name));
let mut file = std::fs::File::create(staged.path().join(name))?;
std::io::Write::write_all(&mut file, bytes)?;
if sync && !bytes.is_empty() {
file.sync_data()?;
let staged = reserve.retry_after_enospc(|| {
std::fs::create_dir_all(&staging_root)?;
let staged = Arc::new(
tempfile::Builder::new()
.prefix("delete-preparation-")
.tempdir_in(&staging_root)?,
);
let missing_marker = original.is_none() && target.is_some();
for (name, bytes) in [
(STORAGE_FORMAT_FILE_BACKUP, original.as_deref()),
(TARGET, target.as_deref()),
(PREPARED, Some(journal.as_slice())),
(DELETE_MARKER_ROLLBACK_FILE, missing_marker.then_some(&[][..])),
] {
let Some(bytes) = bytes else {
continue;
};
#[cfg(test)]
run_owned_file_before_open(&record_destination.join(name));
let mut file = std::fs::File::create(staged.path().join(name))?;
std::io::Write::write_all(&mut file, bytes)?;
if sync && !bytes.is_empty() {
file.sync_data()?;
}
}
}
if sync {
os::fsync_dir_std(staged.path())?;
}
if sync {
os::fsync_dir_std(staged.path())?;
}
#[cfg(test)]
preparation_io_fault::run(&record_destination)?;
Ok(staged)
})?;
Ok((staged, target))
})
.await
@@ -2812,6 +2839,7 @@ impl LocalDisk {
let object_str = object.to_owned();
let recovery_scope = recovery_operation_scope();
let reserve = self.delete_reserve.clone();
type FusedStage1Outcome = Option<(Option<Uuid>, PathBuf, PreparedDeleteReceiptEntry)>;
let res = os::run_blocking_namespace_operation_with_sync(
lease,
@@ -2918,11 +2946,7 @@ impl LocalDisk {
);
}
if let Some(bytes) = receipt.filter(|bytes| !bytes.is_empty()) {
records.push(
serde_json::from_slice::<CommitReceipt>(&bytes)
.map_err(|_| DiskError::FileCorrupt)?
.journal,
);
records.push(parse_commit_receipt(&bytes)?.journal);
}
for record in records {
if record.protocol_version != DELETE_RECOVERY_PROTOCOL_VERSION
@@ -2976,52 +3000,61 @@ impl LocalDisk {
Ok(None) => return Ok(Ok(None)),
Err(err) => return Ok(Err(err)),
};
let staged = match tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root) {
Ok(dir) => dir,
Err(err) if err.kind() == ErrorKind::NotFound => {
std::fs::create_dir_all(&staging_root)?;
tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root)?
let (staged, rollback_dir_fd) = reserve.retry_after_enospc(|| {
let staged = match tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root) {
Ok(dir) => dir,
Err(err) if err.kind() == ErrorKind::NotFound => {
std::fs::create_dir_all(&staging_root)?;
tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root)?
}
Err(err) => return Err(err),
};
let staged_dir = staged.path().to_path_buf();
#[cfg(unix)]
let rollback_dir_fd = open_dir_fd(&staged_dir).ok().map(Arc::new);
#[cfg(not(unix))]
let rollback_dir_fd: Option<Arc<std::fs::File>> = None;
let backup_path = staged_dir.join(STORAGE_FORMAT_FILE_BACKUP);
if link_child_files(
file_path_dir.as_deref(),
STORAGE_FORMAT_FILE,
&xl_path,
rollback_dir_fd.as_deref(),
STORAGE_FORMAT_FILE_BACKUP,
&backup_path,
)
.is_err()
{
let file = create_and_write_child_file(
rollback_dir_fd.as_deref(),
STORAGE_FORMAT_FILE_BACKUP,
&backup_path,
&buf,
)?;
if sync && !buf.is_empty() {
file.sync_data()?;
}
}
Err(err) => return Err(err),
};
let staged_dir = staged.path().to_path_buf();
#[cfg(unix)]
let rollback_dir_fd = open_dir_fd(&staged_dir).ok().map(Arc::new);
#[cfg(not(unix))]
let rollback_dir_fd: Option<Arc<std::fs::File>> = None;
let backup_path = staged_dir.join(STORAGE_FORMAT_FILE_BACKUP);
if link_child_files(
file_path_dir.as_deref(),
STORAGE_FORMAT_FILE,
&xl_path,
rollback_dir_fd.as_deref(),
STORAGE_FORMAT_FILE_BACKUP,
&backup_path,
)
.is_err()
{
let file =
create_and_write_child_file(rollback_dir_fd.as_deref(), STORAGE_FORMAT_FILE_BACKUP, &backup_path, &buf)?;
if sync && !buf.is_empty() {
let mut written_files = Vec::with_capacity(2);
for (name, bytes) in [(TARGET, target.as_deref()), (PREPARED, Some(journal_raw.as_slice()))] {
let Some(bytes) = bytes else {
continue;
};
let file = create_and_write_child_file(rollback_dir_fd.as_deref(), name, &staged_dir.join(name), bytes)?;
if sync && !bytes.is_empty() {
written_files.push(file);
}
}
for file in written_files {
file.sync_data()?;
}
}
let mut written_files = Vec::with_capacity(2);
for (name, bytes) in [(TARGET, target.as_deref()), (PREPARED, Some(journal_raw.as_slice()))] {
let Some(bytes) = bytes else {
continue;
};
let file = create_and_write_child_file(rollback_dir_fd.as_deref(), name, &staged_dir.join(name), bytes)?;
if sync && !bytes.is_empty() {
written_files.push(file);
if sync {
os::fsync_directory_handle_std(&staged_dir, rollback_dir_fd.as_deref())?;
}
}
for file in written_files {
file.sync_data()?;
}
if sync {
os::fsync_directory_handle_std(&staged_dir, rollback_dir_fd.as_deref())?;
}
#[cfg(test)]
preparation_io_fault::run(&destination)?;
Ok((staged, rollback_dir_fd))
})?;
let tx_name = transaction.to_string();
if !destination_missing {
match std::fs::symlink_metadata(&destination) {
@@ -3040,7 +3073,9 @@ impl LocalDisk {
}
let staged_dir = staged.keep();
os::record_native_publication(&destination);
if let Err(err) = rename_into_dir_fd(&staged_dir, file_path_dir.as_deref(), &tx_name, &destination) {
if let Err(err) = reserve
.retry_after_enospc(|| rename_into_dir_fd(&staged_dir, file_path_dir.as_deref(), &tx_name, &destination))
{
let _ = std::fs::remove_dir_all(&staged_dir);
return Err(err);
}
@@ -3058,14 +3093,16 @@ impl LocalDisk {
let reserved_name = format!("{RESERVED_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}");
let reserved_marker = old_path.join(&reserved_name);
let old_dir_file = open_child_dir_fd(file_path_dir.as_deref(), &old_dir_name, &old_path);
let reserve_res = (|| -> std::io::Result<()> {
let reserve_res = reserve.retry_after_enospc(|| -> std::io::Result<()> {
#[cfg(test)]
preparation_io_fault::run(&reserved_marker)?;
let marker = create_and_write_child_file(old_dir_file.as_ref(), &reserved_name, &reserved_marker, &[])?;
if sync {
marker.sync_all()?;
os::fsync_directory_handle_std(&old_path, old_dir_file.as_ref())?;
}
Ok(())
})();
});
if let Err(err) = reserve_res {
let _ = unlink_child_file(old_dir_file.as_ref(), &reserved_name, &reserved_marker);
let _ = std::fs::remove_dir_all(&destination);
@@ -3087,19 +3124,20 @@ impl LocalDisk {
Err(_) => false,
};
let write_res = if linked {
let rename_res = rename_into_dir_fd(&linked_xl, file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path);
let rename_res = reserve.retry_after_enospc(|| {
rename_into_dir_fd(&linked_xl, file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path)
});
if rename_res.is_err() {
let _ = std::fs::remove_file(&linked_xl);
}
rename_res
} else {
(|| -> std::io::Result<()> {
let mut staged_xl = tempfile::NamedTempFile::new_in(&staging_root)?;
std::io::Write::write_all(staged_xl.as_file_mut(), target_bytes)?;
if sync && !target_bytes.is_empty() {
staged_xl.as_file().sync_data()?;
}
rename_into_dir_fd(staged_xl.path(), file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path)
let staged_xl =
reserve.retry_after_enospc(|| stage_recovery_file(&staging_root, target_bytes, sync))?;
reserve.retry_after_enospc(|| {
rename_into_dir_fd(staged_xl.path(), file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path)
})
})()
};
if let Err(err) = write_res {
@@ -3150,6 +3188,7 @@ impl LocalDisk {
journal,
sole_transaction,
had_old_data_dir,
prepared_synced: sync,
rollback_dir_fd,
file_path_dir,
},
@@ -3213,6 +3252,7 @@ impl LocalDisk {
entry.journal,
entry.sole_transaction,
entry.had_old_data_dir,
entry.prepared_synced,
entry.rollback_dir_fd,
entry.file_path_dir,
)),
@@ -3242,38 +3282,79 @@ impl LocalDisk {
let volume_str = volume.to_owned();
let object_str = object.to_owned();
let reserve = self.delete_reserve.clone();
type PublishDeleteFastReturn = (DeleteRecoveryJournal, bool, bool, Option<(PathBuf, PublishedDeleteReceiptEntry)>);
let res = os::run_blocking_namespace_operation_with_sync(
lease,
sync,
move || -> std::io::Result<Result<Option<PublishDeleteFastReturn>>> {
if let Some((journal_raw, journal, sole_transaction, had_old_data_dir, rollback_dir_fd, file_path_dir)) =
cached_receipt
if let Some((
journal_raw,
journal,
sole_transaction,
had_old_data_dir,
prepared_synced,
rollback_dir_fd,
file_path_dir,
)) = cached_receipt
&& child_regular_file_exists(rollback_dir_fd.as_deref(), PREPARED, &rollback.join(PREPARED))
{
let tmp_receipt = staging_root.join(format!("receipt-{transaction}.tmp"));
let receipt_bytes = format_commit_receipt_bytes(&journal_raw, false);
let write_res = (|| -> std::io::Result<()> {
let mut file = std::fs::File::create(&tmp_receipt)?;
std::io::Write::write_all(&mut file, &receipt_bytes)?;
if sync && !receipt_bytes.is_empty() {
file.sync_data()?;
}
// Share the immutable Stage 1 PREPARED inode; any later receipt update must replace the entry by rename.
let linked = if !sync || prepared_synced {
os::record_native_publication(&receipt_path);
rename_into_dir_fd(
&tmp_receipt,
rollback_dir_fd.as_deref(),
DELETE_QUORUM_COMMITTED_FILE,
&receipt_path,
)?;
reserve
.retry_after_enospc(|| {
#[cfg(test)]
preparation_io_fault::run(&receipt_path)?;
link_child_files(
rollback_dir_fd.as_deref(),
PREPARED,
&rollback.join(PREPARED),
rollback_dir_fd.as_deref(),
DELETE_QUORUM_COMMITTED_FILE,
&receipt_path,
)
})
.is_ok()
} else {
false
};
if linked {
if sync {
os::fsync_directory_handle_std(&rollback, rollback_dir_fd.as_deref())?;
}
Ok(())
})();
if let Err(err) = write_res {
let _ = std::fs::remove_file(&tmp_receipt);
return Err(err);
} else {
let tmp_receipt = staging_root.join(format!("receipt-{transaction}.tmp"));
let receipt_bytes = format_commit_receipt_bytes(&journal_raw, false);
let write_res = (|| -> std::io::Result<()> {
reserve.retry_after_enospc(|| {
let mut file = std::fs::File::create(&tmp_receipt)?;
std::io::Write::write_all(&mut file, &receipt_bytes)?;
if sync && !receipt_bytes.is_empty() {
file.sync_data()?;
}
#[cfg(test)]
preparation_io_fault::run(&receipt_path)?;
Ok(())
})?;
os::record_native_publication(&receipt_path);
reserve.retry_after_enospc(|| {
rename_into_dir_fd(
&tmp_receipt,
rollback_dir_fd.as_deref(),
DELETE_QUORUM_COMMITTED_FILE,
&receipt_path,
)
})?;
if sync {
os::fsync_directory_handle_std(&rollback, rollback_dir_fd.as_deref())?;
}
Ok(())
})();
if let Err(err) = write_res {
let _ = std::fs::remove_file(&tmp_receipt);
return Err(err);
}
}
let published_entry = bucket_path.map(|bucket_path| {
(
@@ -3358,14 +3439,9 @@ impl LocalDisk {
finished: false,
};
let receipt_bytes = serde_json::to_vec(&receipt).map_err(std::io::Error::other)?;
std::fs::create_dir_all(&staging_root)?;
let mut staged = tempfile::NamedTempFile::new_in(&staging_root)?;
std::io::Write::write_all(staged.as_file_mut(), &receipt_bytes)?;
if sync && !receipt_bytes.is_empty() {
staged.as_file().sync_data()?;
}
let staged = reserve.retry_after_enospc(|| stage_recovery_file(&staging_root, &receipt_bytes, sync))?;
os::record_native_publication(&receipt_path);
std::fs::rename(staged.path(), &receipt_path)?;
reserve.retry_after_enospc(|| std::fs::rename(staged.path(), &receipt_path))?;
if sync {
os::fsync_dir_std(&rollback)?;
}
@@ -3517,6 +3593,7 @@ impl LocalDisk {
.collect()
};
let reserve = self.delete_reserve.clone();
let res = os::run_blocking_namespace_operation_with_sync(lease, sync, move || -> std::io::Result<()> {
let persist_receipt = |finished: bool| -> std::io::Result<()> {
let receipt_bytes = match &cached_journal_raw {
@@ -3531,14 +3608,9 @@ impl LocalDisk {
serde_json::to_vec(&receipt).map_err(std::io::Error::other)?
}
};
std::fs::create_dir_all(&staging_root)?;
let mut staged_receipt = tempfile::NamedTempFile::new_in(&staging_root)?;
std::io::Write::write_all(staged_receipt.as_file_mut(), &receipt_bytes)?;
if sync && !receipt_bytes.is_empty() {
staged_receipt.as_file().sync_data()?;
}
let staged_receipt = reserve.retry_after_enospc(|| stage_recovery_file(&staging_root, &receipt_bytes, sync))?;
os::record_native_publication(&receipt_path);
std::fs::rename(staged_receipt.path(), &receipt_path)?;
reserve.retry_after_enospc(|| std::fs::rename(staged_receipt.path(), &receipt_path))?;
if sync {
let _ = os::fsync_dir_std(&rollback);
}
@@ -3724,6 +3796,11 @@ impl LocalDisk {
for claim in claims {
claim.finish(ok);
}
if ok && !self.delete_reserve.is_armed() {
let reserve = self.delete_reserve.clone();
let lease = os::acquire_namespace_mutation_lease_with_owner(self.io_root(), None).await;
let _ = os::run_blocking_namespace_operation_with_sync(lease, true, move || reserve.replenish()).await;
}
res.map(|()| true).map_err(|err| to_file_error(err).into())
}
}
@@ -3859,6 +3936,43 @@ mod fused_prepare_checkpoint {
}
}
#[cfg(test)]
mod preparation_io_fault {
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{LazyLock, Mutex};
type Hook = Box<dyn FnOnce() -> std::io::Result<()> + Send>;
static HOOKS: LazyLock<Mutex<HashMap<PathBuf, Hook>>> = LazyLock::new(|| Mutex::new(HashMap::new()));
pub(super) struct Guard(PathBuf);
impl Drop for Guard {
fn drop(&mut self) {
HOOKS.lock().expect("checkpoint hooks").remove(&self.0);
}
}
pub(super) fn install(root: &Path, hook: impl FnOnce() -> std::io::Result<()> + Send + 'static) -> Guard {
assert!(
HOOKS
.lock()
.expect("checkpoint hooks")
.insert(root.to_path_buf(), Box::new(hook))
.is_none()
);
Guard(root.to_path_buf())
}
pub(super) fn run(root: &Path) -> std::io::Result<()> {
let hook = HOOKS.lock().expect("checkpoint hooks").remove(root);
match hook {
Some(hook) => hook(),
None => Ok(()),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -8450,6 +8564,9 @@ mod tests {
journal: journal.clone(),
finished: true,
};
fs::remove_file(rollback.join(DELETE_QUORUM_COMMITTED_FILE))
.await
.expect("remove receipt before replacement");
fs::write(
rollback.join(DELETE_QUORUM_COMMITTED_FILE),
serde_json::to_vec(&finished_receipt).expect("serialize finished receipt"),
@@ -8484,6 +8601,55 @@ mod tests {
#[tokio::test]
async fn delete_recovery_fused_unversioned_physical_delete_retains_receipt_until_finalize() {
assert_unversioned_delete_reserve_recovery(None).await;
}
#[cfg(unix)]
#[tokio::test]
async fn delete_recovery_fused_enospc_recovers_scratch_marker_and_receipt_without_replaying_transaction() {
for stage in ["scratch", "marker", "receipt"] {
assert_unversioned_delete_reserve_recovery(Some(stage)).await;
}
}
#[cfg(unix)]
#[tokio::test]
async fn delete_recovery_fallback_enospc_recovers_before_activation_and_preserves_original() {
for kind in [ErrorKind::StorageFull, ErrorKind::PermissionDenied] {
let (_temp, disk, info, _) = fixture().await;
let root = disk.io_get_object_path("recovery", "object").expect("object path");
let original = std::fs::read(root.join(STORAGE_FORMAT_FILE)).expect("original metadata");
let transaction = Uuid::new_v4();
let destination = root.join(transaction.to_string());
let _fault = preparation_io_fault::install(&destination, move || Err(std::io::Error::from(kind)));
let result = disk
.prepare_delete_recovery(
"recovery",
"object",
Some(&original),
&[info],
None,
DeletePreparationContext {
transaction,
bucket_incarnation: None,
namespace_owner: None,
},
)
.await;
if kind == ErrorKind::StorageFull {
assert!(result.is_ok(), "scratch ENOSPC must recover: {result:?}");
assert!(destination.join(PREPARED).is_file());
assert!(!disk.delete_reserve.is_armed());
} else {
assert!(result.is_err(), "non-capacity failures must not retry");
assert!(!destination.exists());
assert!(disk.delete_reserve.is_armed());
}
assert_eq!(std::fs::read(root.join(STORAGE_FORMAT_FILE)).expect("unchanged metadata"), original);
}
}
async fn assert_unversioned_delete_reserve_recovery(fault: Option<&str>) {
let (_temp, disk, mut info, _) = fixture().await;
let root = disk.io_get_object_path("recovery", "object").expect("object path");
@@ -8506,7 +8672,35 @@ mod tests {
.await
.expect("write xl.meta");
#[cfg(unix)]
let reserve_path = disk.io_root().join(".rustfs.sys/delete-reserve");
#[cfg(unix)]
assert_eq!(
std::fs::metadata(&reserve_path).expect("initialized reserve").len(),
super::super::delete_reserve::RESERVE_BYTES
);
let transaction = Uuid::new_v4();
let rollback = root.join(transaction.to_string());
let _fault = fault.map(|stage| {
let path = match stage {
"scratch" => rollback.clone(),
"marker" => root
.join(data_dir.to_string())
.join(format!("{RESERVED_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}")),
"receipt" => rollback.join(DELETE_QUORUM_COMMITTED_FILE),
_ => panic!("unknown fault stage"),
};
preparation_io_fault::install(&path, || Err(std::io::Error::from(ErrorKind::StorageFull)))
});
#[cfg(unix)]
let checkpoints = Arc::new(std::sync::atomic::AtomicUsize::new(0));
#[cfg(unix)]
let _checkpoint = {
let checkpoints = checkpoints.clone();
fused_prepare_checkpoint::install(&root, move || {
checkpoints.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
})
};
let opts = DeleteOptions {
recoverable_delete: true,
..Default::default()
@@ -8516,6 +8710,32 @@ mod tests {
.await
.expect("fused unversioned physical delete must succeed");
assert_eq!(fused, Some(()));
#[cfg(unix)]
assert_eq!(
std::fs::metadata(&reserve_path).expect("released reserve").len(),
if matches!(fault, Some("scratch" | "marker")) {
0
} else {
super::super::delete_reserve::RESERVE_BYTES
},
"only an actual capacity failure should consume the reserve"
);
#[cfg(unix)]
assert_eq!(
checkpoints.load(std::sync::atomic::Ordering::Relaxed),
1,
"transaction publication must not replay"
);
assert!(
std::fs::read_dir(disk.io_root().join(RUSTFS_META_TMP_BUCKET))
.expect("staging root")
.all(|entry| !entry
.expect("staging entry")
.file_name()
.to_string_lossy()
.starts_with("delete-prep-")),
"failed scratch attempts must be cleaned up"
);
let (journal, sole_transaction, target_satisfied) = disk
.try_publish_prepared_delete_local("recovery", "object", transaction, None)
@@ -8525,6 +8745,15 @@ mod tests {
assert!(sole_transaction);
assert!(target_satisfied);
assert!(journal.target_digest.is_none());
#[cfg(unix)]
assert_eq!(
std::fs::metadata(&reserve_path).expect("reserve after receipt").len(),
if fault.is_some() {
0
} else {
super::super::delete_reserve::RESERVE_BYTES
}
);
let rollback = root.join(transaction.to_string());
assert!(rollback.join(DELETE_QUORUM_COMMITTED_FILE).is_file());
@@ -8534,6 +8763,11 @@ mod tests {
.await
.expect("local finalize must succeed");
assert!(finalized);
#[cfg(unix)]
assert_eq!(
std::fs::metadata(&reserve_path).expect("replenished reserve").len(),
super::super::delete_reserve::RESERVE_BYTES
);
assert!(!rollback.exists(), "rollback directory must be pruned");
assert!(!root.join(data_dir.to_string()).exists(), "physical data_dir must be removed");
assert!(!root.exists(), "empty object directory must be pruned");
@@ -8618,6 +8852,265 @@ mod tests {
);
assert!(!rollback.exists());
}
#[cfg(unix)]
#[tokio::test]
async fn delete_recovery_stage2_hardlinks_prepared_journal_and_falls_back_on_conflict() {
use std::os::unix::fs::MetadataExt;
let _strict = durability_mode_override::set(DurabilityMode::Strict);
let (_temp, disk, info, _) = fixture().await;
let root = disk.io_get_object_path("recovery", "object").expect("object path");
let opts = DeleteOptions {
recoverable_delete: true,
..Default::default()
};
#[cfg(target_os = "linux")]
{
let proc_vol = format!("{}/recovery", disk.io_root().display());
assert!(matches!(
disk.delete_version(&proc_vol, "object", info.clone(), false, opts.clone())
.await,
Err(DiskError::InvalidPath)
));
}
let tx1 = Uuid::new_v4();
assert_eq!(
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx1, None)
.await
.expect("fused prepare tx1"),
Some(())
);
let (journal1, sole1, satisfied1) = disk
.try_publish_prepared_delete_local("recovery", "object", tx1, None)
.await
.expect("publish tx1")
.expect("tx1 satisfied");
assert!(sole1 && satisfied1);
let rollback1 = root.join(tx1.to_string());
assert!(
os::fsync_dir_recorder::was_fsynced(&rollback1),
"strict Stage 2 hardlink must fsync rollback directory"
);
let prep_meta1 = std::fs::metadata(rollback1.join(PREPARED)).expect("prepared metadata");
let rec_meta1 = std::fs::metadata(rollback1.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt metadata");
assert_eq!(
(prep_meta1.dev(), prep_meta1.ino()),
(rec_meta1.dev(), rec_meta1.ino()),
"Stage 2 fast path must hardlink PREPARED to DELETE_QUORUM_COMMITTED_FILE"
);
let raw_receipt1 = std::fs::read(rollback1.join(DELETE_QUORUM_COMMITTED_FILE)).expect("read bare receipt");
let parsed_bare = parse_commit_receipt(&raw_receipt1).expect("parse bare journal receipt");
assert!(!parsed_bare.finished && parsed_bare.journal == journal1);
let wrapped_true = format_commit_receipt_bytes(&raw_receipt1, true);
let parsed_wrapped = parse_commit_receipt(&wrapped_true).expect("parse wrapped finished receipt");
assert!(parsed_wrapped.finished && parsed_wrapped.journal == journal1);
assert!(matches!(parse_commit_receipt(b"{\"invalid\":1}"), Err(DiskError::FileCorrupt)));
assert_eq!(
disk.inspect_delete_transaction("recovery", "object", tx1)
.await
.expect("inspect QuorumCommitted")
.state,
DeleteRecoveryState::QuorumCommitted
);
// Verify DiscoverySession::advance_bounded (:212) extracts the bound logical key ("directory/")
// from a bare-journal receipt when PREPARED is absent (physical dir name is "directory").
let dir_root = disk.io_get_object_path("recovery", "directory/").expect("dir key path");
let mut dir_info = info.clone();
dir_info.name = "directory/".to_owned();
let mut dir_meta = FileMeta::new();
dir_meta.add_version(dir_info.clone()).expect("add dir version");
fs::create_dir_all(&dir_root).await.expect("create dir_root");
fs::write(dir_root.join(STORAGE_FORMAT_FILE), dir_meta.marshal_msg().expect("marshal dir meta"))
.await
.expect("write dir xl.meta");
let tx_dir = Uuid::new_v4();
assert_eq!(
disk.try_fused_recoverable_delete_version("recovery", "directory/", &dir_info, &opts, tx_dir, None)
.await
.expect("fused prepare tx_dir"),
Some(())
);
let (journal_dir, _, _) = disk
.try_publish_prepared_delete_local("recovery", "directory/", tx_dir, None)
.await
.expect("publish tx_dir")
.expect("tx_dir satisfied");
let rollback_dir = dir_root.join(tx_dir.to_string());
fs::remove_file(rollback_dir.join(PREPARED))
.await
.expect("unlink dir PREPARED");
fs::remove_file(rollback_dir.join(STORAGE_FORMAT_FILE_BACKUP))
.await
.expect("unlink dir backup");
let discovered_dir = disk
.discover_delete_transactions("recovery", None, None, 16)
.await
.expect("bucket-wide discovery of directory/ bare-journal receipt");
assert!(
discovered_dir
.transactions
.iter()
.any(|id| id.object == "directory/" && id.transaction == tx_dir),
"DiscoverySession must parse logical key 'directory/' from bare-journal receipt"
);
assert!(
disk.try_finish_and_finalize_prepared_delete_local("recovery", "directory/", tx_dir, &journal_dir, None)
.await
.expect("finalize tx_dir")
);
// Removing PREPARED and backup while retaining the bare-journal receipt infers CommitFinished
// and exercises all bare-journal parse_commit_receipt sites (bucket-wide discovery + predecessor scans).
fs::remove_file(rollback1.join(PREPARED)).await.expect("unlink PREPARED");
fs::remove_file(rollback1.join(STORAGE_FORMAT_FILE_BACKUP))
.await
.expect("unlink backup");
assert_eq!(
disk.inspect_delete_transaction("recovery", "object", tx1)
.await
.expect("inspect CommitFinished from bare-journal receipt")
.state,
DeleteRecoveryState::CommitFinished
);
let discovered_bucket = disk
.discover_delete_transactions("recovery", None, None, 16)
.await
.expect("bucket-wide discovery of bare-journal receipt");
assert!(discovered_bucket.transactions.iter().any(|id| id.transaction == tx1));
let preds = disk
.delete_recovery_predecessors("recovery", "object", Uuid::new_v4(), None, None)
.await
.expect("predecessor scan of bare-journal receipt");
assert_eq!(preds, vec![tx1]);
// Re-create xl.meta while tx1's bare-journal receipt is still present so Stage 1's predecessor scan
// parses tx1's bare-journal receipt, and pre-populate tx2's receipt before Stage 2 to force EEXIST fallback.
let mut meta = FileMeta::new();
meta.add_version(info.clone()).expect("add version");
fs::create_dir_all(&root).await.expect("recreate object dir");
fs::write(root.join(STORAGE_FORMAT_FILE), meta.marshal_msg().expect("marshal meta"))
.await
.expect("write xl.meta");
let tx2 = Uuid::new_v4();
assert_eq!(
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx2, None)
.await
.expect("fused prepare tx2"),
Some(())
);
let rollback2 = root.join(tx2.to_string());
fs::write(rollback2.join(DELETE_QUORUM_COMMITTED_FILE), b"stale")
.await
.expect("pre-create conflicting receipt");
let (journal2, sole2, satisfied2) = disk
.try_publish_prepared_delete_local("recovery", "object", tx2, None)
.await
.expect("publish tx2 fallback")
.expect("tx2 satisfied");
assert!(!sole2 && satisfied2);
assert_eq!(journal2.predecessors, vec![tx1]);
let prep_meta2 = std::fs::metadata(rollback2.join(PREPARED)).expect("prepared2 metadata");
let rec_meta2 = std::fs::metadata(rollback2.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt2 metadata");
assert_ne!(
(prep_meta2.dev(), prep_meta2.ino()),
(rec_meta2.dev(), rec_meta2.ino()),
"EEXIST fallback must replace receipt with a distinct wrapper inode"
);
assert!(
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx1, &journal1, None)
.await
.expect("finalize tx1")
);
assert!(
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx2, &journal2, None)
.await
.expect("finalize tx2")
);
// Prepare tx3 under Relaxed (`prepared_synced == false`), then escalate to Strict before Stage 2:
// Stage 2 must not hardlink the unsynced PREPARED inode and must instead write a fresh synced receipt.
fs::create_dir_all(&root).await.expect("recreate object dir for tx3");
fs::write(root.join(STORAGE_FORMAT_FILE), meta.marshal_msg().expect("marshal meta"))
.await
.expect("write xl.meta for tx3");
drop(_strict);
let tx3 = Uuid::new_v4();
{
let _relaxed = durability_mode_override::set(DurabilityMode::Relaxed);
assert_eq!(
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx3, None)
.await
.expect("fused prepare tx3 relaxed"),
Some(())
);
}
let _strict_again = durability_mode_override::set(DurabilityMode::Strict);
let (journal3, sole3, satisfied3) = disk
.try_publish_prepared_delete_local("recovery", "object", tx3, None)
.await
.expect("publish tx3 escalated to strict")
.expect("tx3 satisfied");
assert!(sole3 && satisfied3);
let rollback3 = root.join(tx3.to_string());
let prep_meta3 = std::fs::metadata(rollback3.join(PREPARED)).expect("prepared3 metadata");
let rec_meta3 = std::fs::metadata(rollback3.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt3 metadata");
assert_ne!(
(prep_meta3.dev(), prep_meta3.ino()),
(rec_meta3.dev(), rec_meta3.ino()),
"unsynced Stage 1 preparation must not be hardlinked when Stage 2 requires sync"
);
assert!(
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx3, &journal3, None)
.await
.expect("finalize tx3")
);
drop(_strict_again);
// Prepare and publish tx4 under Relaxed (`!sync`): Stage 2 must hardlink PREPARED without directory fsync.
fs::create_dir_all(&root).await.expect("recreate object dir for tx4");
fs::write(root.join(STORAGE_FORMAT_FILE), meta.marshal_msg().expect("marshal meta"))
.await
.expect("write xl.meta for tx4");
let tx4 = Uuid::new_v4();
{
let _relaxed = durability_mode_override::set(DurabilityMode::Relaxed);
assert_eq!(
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx4, None)
.await
.expect("fused prepare tx4 relaxed"),
Some(())
);
let (journal4, sole4, satisfied4) = disk
.try_publish_prepared_delete_local("recovery", "object", tx4, None)
.await
.expect("publish tx4 relaxed")
.expect("tx4 satisfied");
assert!(sole4 && satisfied4);
let rollback4 = root.join(tx4.to_string());
assert!(
!os::fsync_dir_recorder::was_fsynced(&rollback4),
"relaxed Stage 2 hardlink must not fsync rollback directory"
);
let prep_meta4 = std::fs::metadata(rollback4.join(PREPARED)).expect("prepared4 metadata");
let rec_meta4 = std::fs::metadata(rollback4.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt4 metadata");
assert_eq!(
(prep_meta4.dev(), prep_meta4.ino()),
(rec_meta4.dev(), rec_meta4.ino()),
"relaxed Stage 2 must hardlink PREPARED to DELETE_QUORUM_COMMITTED_FILE"
);
assert!(
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx4, &journal4, None)
.await
.expect("finalize tx4")
);
}
}
}
#[cfg(test)]
@@ -0,0 +1,597 @@
// Copyright 2026 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Bounded emergency headroom for durable deletion. Healthy operations access
//! only the shared owner's atomic state. Only allocation failures consume the
//! reserve, without holding its mutex across transaction publication or quorum.
use std::{io, path::Path, sync::Arc};
#[cfg(unix)]
pub(super) const RESERVE_BYTES: u64 = 8 * 1024 * 1024;
pub(super) struct DeleteReserve {
#[cfg(unix)]
inner: unix::State,
}
impl DeleteReserve {
pub(super) fn acquire(root: &Path) -> io::Result<Arc<Self>> {
#[cfg(unix)]
{
unix::acquire(root)
}
#[cfg(not(unix))]
{
let _ = root;
Ok(Arc::new(Self {}))
}
}
pub(super) fn is_armed(&self) -> bool {
#[cfg(unix)]
{
self.inner.is_armed()
}
#[cfg(not(unix))]
{
true
}
}
/// Populate the reserve on the disk's dedicated fsync lane.
pub(super) fn initialize(&self) -> io::Result<()> {
self.replenish()
}
/// Refill only when sufficient free space remains for recovery. An armed
/// owner returns immediately without opening files or probing capacity.
pub(super) fn replenish(&self) -> io::Result<()> {
#[cfg(unix)]
{
self.inner.replenish()
}
#[cfg(not(unix))]
{
Ok(())
}
}
/// Probe capacity during startup so a persisted reserve can make the disk
/// writable before its normal writeability check.
pub(super) fn release_if_low(&self) -> io::Result<()> {
#[cfg(unix)]
{
self.inner.release_if_low()
}
#[cfg(not(unix))]
{
Ok(())
}
}
/// Retry an allocation-safe scratch operation once after ENOSPC. The first
/// waiter releases the reserve; concurrent waiters serialize behind it and
/// may use the same released capacity. The caller must ensure that a failed
/// attempt leaves only disposable scratch or an idempotent primitive under
/// the existing namespace owner. Never replay a transaction or a
/// publication-plus-sync sequence with this helper.
///
/// The owner mutex covers release and the retry, excluding replenishment
/// during this bounded operation. Later transaction phases and unrelated
/// allocators can still compete for the released space.
pub(super) fn retry_after_enospc<T>(&self, mut operation: impl FnMut() -> io::Result<T>) -> io::Result<T> {
match operation() {
#[cfg(unix)]
Err(error) if error.kind() == io::ErrorKind::StorageFull => self.inner.retry_after_enospc(operation, error),
result => result,
}
}
}
#[cfg(unix)]
mod unix {
use super::*;
use parking_lot::Mutex;
use rustix::fs::{Mode, OFlags, mkdirat, open, openat};
use std::{
collections::HashMap,
fs::File,
io::{Seek, SeekFrom, Write},
os::unix::fs::MetadataExt,
sync::{
OnceLock, Weak,
atomic::{AtomicBool, Ordering},
},
};
const METADATA_DIR: &str = ".rustfs.sys";
const RESERVE_NAME: &str = "delete-reserve";
type RootIdentity = (u64, u64);
type OwnerRegistry = Mutex<HashMap<RootIdentity, Weak<DeleteReserve>>>;
static OWNERS: OnceLock<OwnerRegistry> = OnceLock::new();
pub(super) struct State {
root: File,
mutation: Mutex<()>,
armed: AtomicBool,
}
pub(super) fn acquire(path: &Path) -> io::Result<Arc<DeleteReserve>> {
let root = File::from(open(
path,
OFlags::RDONLY | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
Mode::empty(),
)?);
let metadata = root.metadata()?;
let identity = (metadata.dev(), metadata.ino());
let mut registry = OWNERS.get_or_init(Mutex::default).lock();
if let Some(owner) = registry.get(&identity).and_then(Weak::upgrade) {
return Ok(owner);
}
registry.retain(|_, owner| owner.strong_count() != 0);
let owner = Arc::new(DeleteReserve {
inner: State {
root,
mutation: Mutex::new(()),
armed: AtomicBool::new(false),
},
});
registry.insert(identity, Arc::downgrade(&owner));
Ok(owner)
}
fn metadata_directory(root: &File, create: bool) -> io::Result<File> {
if create {
match mkdirat(root, METADATA_DIR, Mode::from_raw_mode(0o700)) {
Ok(()) => root.sync_all()?,
Err(rustix::io::Errno::EXIST) => {}
Err(error) => return Err(error.into()),
}
}
Ok(File::from(openat(
root,
METADATA_DIR,
OFlags::RDONLY | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
Mode::empty(),
)?))
}
fn reserve_file(parent: &File, create: bool) -> io::Result<File> {
let flags = OFlags::RDWR | OFlags::NOFOLLOW | OFlags::NONBLOCK | OFlags::CLOEXEC;
let file = File::from(openat(
parent,
RESERVE_NAME,
if create { flags | OFlags::CREATE } else { flags },
Mode::from_raw_mode(0o600),
)?);
let metadata = file.metadata()?;
if !metadata.is_file() || metadata.nlink() != 1 || metadata.len() > RESERVE_BYTES {
return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid delete reserve file"));
}
Ok(file)
}
fn absent_to_none<T>(result: io::Result<T>) -> io::Result<Option<T>> {
match result {
Ok(value) => Ok(Some(value)),
Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(None),
Err(error) => Err(error),
}
}
impl State {
pub(super) fn is_armed(&self) -> bool {
self.armed.load(Ordering::Acquire)
}
pub(super) fn replenish(&self) -> io::Result<()> {
if self.is_armed() {
return Ok(());
}
let _guard = self.mutation.lock();
if self.is_armed() {
return Ok(());
}
let existing = match absent_to_none(metadata_directory(&self.root, false))? {
Some(parent) => absent_to_none(reserve_file(&parent, false))?.map(|file| (parent, file)),
None => None,
};
if let Some((_, file)) = &existing {
let metadata = file.metadata()?;
if metadata.len() == RESERVE_BYTES && metadata.blocks().saturating_mul(512) >= RESERVE_BYTES {
self.armed.store(true, Ordering::Release);
return Ok(());
}
}
// A depleted reserve must not take newly released recovery space
// away from a disk that is still nearly full.
if available_bytes(&self.root)? < RESERVE_BYTES * 2 {
return Ok(());
}
let (parent, mut file) = match existing {
Some(existing) => existing,
None => {
let parent = metadata_directory(&self.root, true)?;
let file = reserve_file(&parent, true)?;
(parent, file)
}
};
let metadata = file.metadata()?;
// A length alone does not reserve blocks. Repair sparse files left
// by interrupted allocation or older installations.
if metadata.blocks().saturating_mul(512) < metadata.len() {
file.set_len(0)?;
}
let length = file.seek(SeekFrom::End(0))?;
let mut remaining = RESERVE_BYTES - length;
let mut bytes = [0_u8; 64 * 1024];
while remaining != 0 {
let count = usize::try_from(remaining.min(64 * 1024)).map_err(io::Error::other)?;
// Zero blocks can become holes on transparent-compression filesystems.
rand::fill(&mut bytes[..count]);
file.write_all(&bytes[..count])?;
remaining -= u64::try_from(count).map_err(io::Error::other)?;
}
file.sync_all()?;
if file.metadata()?.blocks().saturating_mul(512) < RESERVE_BYTES {
return Err(io::Error::new(
io::ErrorKind::Unsupported,
"filesystem did not allocate delete reserve blocks",
));
}
parent.sync_all()?;
self.armed.store(true, Ordering::Release);
Ok(())
}
fn release_locked(&self) -> io::Result<bool> {
let Some(parent) = absent_to_none(metadata_directory(&self.root, false))? else {
self.armed.store(false, Ordering::Release);
return Ok(false);
};
let Some(file) = absent_to_none(reserve_file(&parent, false))? else {
self.armed.store(false, Ordering::Release);
return Ok(false);
};
let metadata = file.metadata()?;
if metadata.len() == 0 {
self.armed.store(false, Ordering::Release);
return Ok(false);
}
file.set_len(0)?;
self.armed.store(false, Ordering::Release);
// The reserve contains no recovery authority. Truncation releases
// blocks immediately and needs no durability barrier before retry.
Ok(metadata.blocks() != 0)
}
pub(super) fn release_if_low(&self) -> io::Result<()> {
if available_bytes(&self.root)? < RESERVE_BYTES / 2 {
let _guard = self.mutation.lock();
self.release_locked()?;
}
Ok(())
}
pub(super) fn retry_after_enospc<T>(
&self,
mut operation: impl FnMut() -> io::Result<T>,
original: io::Error,
) -> io::Result<T> {
let _guard = self.mutation.lock();
match self.release_locked() {
// A peer may have consumed the shared reserve while this
// operation was failing. Waiting for the mutation lock and
// replaying once lets it use the capacity that peer released.
// The public contract only accepts disposable scratch or an
// idempotent primitive, so the retry remains safe even when no
// reserve was present. Invalid reserve state still fails closed.
Ok(_) => operation(),
Err(_) => Err(original),
}
}
}
fn available_bytes(root: &File) -> io::Result<u64> {
let stat = rustix::fs::fstatvfs(root)?;
let block_size = if stat.f_frsize == 0 { stat.f_bsize } else { stat.f_frsize };
let available = u128::from(stat.f_bavail) * u128::from(block_size);
Ok(u64::try_from(available).unwrap_or(u64::MAX))
}
#[cfg(test)]
mod tests {
use super::*;
fn reserve_path(root: &Path) -> std::path::PathBuf {
root.join(METADATA_DIR).join(RESERVE_NAME)
}
#[test]
fn delete_reserve_allocates_retries_once_and_replenishes_real_blocks() {
let root = tempfile::tempdir().expect("create disk root");
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
owner.initialize().expect("allocate reserve");
let path = reserve_path(root.path());
let initial = std::fs::metadata(&path).expect("reserve metadata");
assert_eq!(initial.len(), RESERVE_BYTES);
assert!(initial.blocks() * 512 >= RESERVE_BYTES);
assert!(owner.is_armed());
let mut calls = 0;
let result = owner
.retry_after_enospc(|| {
calls += 1;
if calls == 1 {
Err(io::Error::new(io::ErrorKind::StorageFull, "scratch allocation"))
} else {
Ok(42)
}
})
.expect("retry scratch allocation");
assert_eq!(result, 42);
assert_eq!(calls, 2);
assert!(!owner.is_armed());
assert_eq!(std::fs::metadata(&path).expect("released reserve").len(), 0);
owner.replenish().expect("replenish reserve");
assert!(owner.is_armed());
assert_eq!(std::fs::metadata(&path).expect("replenished reserve").len(), RESERVE_BYTES);
}
#[test]
fn delete_reserve_retries_once_without_local_reserve_capacity() {
let root = tempfile::tempdir().expect("create disk root");
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
let mut calls = 0;
let error = owner
.retry_after_enospc::<()>(|| {
calls += 1;
Err(io::Error::new(io::ErrorKind::StorageFull, format!("allocation {calls}")))
})
.expect_err("retry without a reserve still has no capacity");
assert_eq!(calls, 2);
assert_eq!(error.to_string(), "allocation 2");
owner.initialize().expect("allocate reserve");
let parent = metadata_directory(&owner.inner.root, false).expect("metadata directory");
let reserve = reserve_file(&parent, false).expect("reserve file");
reserve.set_len(0).expect("deplete reserve");
calls = 0;
assert!(
owner
.retry_after_enospc::<()>(|| {
calls += 1;
Err(io::Error::new(io::ErrorKind::StorageFull, "empty reserve"))
})
.is_err()
);
assert_eq!(calls, 2);
}
#[test]
fn delete_reserve_retry_ignores_other_errors_and_stays_bounded() {
let root = tempfile::tempdir().expect("create disk root");
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
owner.initialize().expect("allocate reserve");
let mut calls = 0;
let error = owner
.retry_after_enospc::<()>(|| {
calls += 1;
Err(io::Error::new(io::ErrorKind::PermissionDenied, "permission failure"))
})
.expect_err("permission failures must propagate");
assert_eq!(error.kind(), io::ErrorKind::PermissionDenied);
assert_eq!(calls, 1);
assert!(owner.is_armed());
calls = 0;
let error = owner
.retry_after_enospc::<()>(|| {
calls += 1;
Err(io::Error::new(io::ErrorKind::StorageFull, format!("allocation {calls}")))
})
.expect_err("second allocation failure propagates");
assert_eq!(calls, 2);
assert_eq!(error.to_string(), "allocation 2");
}
#[test]
fn delete_reserve_rejects_symlinks_and_hardlinks_without_hiding_operation_errors() {
use std::os::unix::fs::symlink;
let root = tempfile::tempdir().expect("create disk root");
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
let external = tempfile::tempdir().expect("create external directory");
let victim = external.path().join("victim");
std::fs::write(&victim, b"preserve").expect("write victim");
let metadata = root.path().join(METADATA_DIR);
symlink(external.path(), &metadata).expect("symlink metadata directory");
assert!(owner.replenish().is_err());
std::fs::remove_file(&metadata).expect("remove directory link");
std::fs::create_dir(&metadata).expect("create metadata directory");
let reserve = reserve_path(root.path());
symlink(&victim, &reserve).expect("symlink reserve");
assert!(owner.replenish().is_err());
let mut calls = 0;
let error = owner
.retry_after_enospc::<()>(|| {
calls += 1;
Err(io::Error::new(io::ErrorKind::StorageFull, "original error"))
})
.expect_err("invalid reserve cannot mask allocation failure");
assert_eq!(calls, 1);
assert_eq!(error.to_string(), "original error");
std::fs::remove_file(&reserve).expect("remove reserve link");
std::fs::hard_link(&victim, &reserve).expect("hardlink reserve");
assert!(owner.replenish().is_err());
assert!(owner.inner.release_locked().is_err());
assert_eq!(std::fs::read(&victim).expect("read victim"), b"preserve");
}
#[test]
fn delete_reserve_repairs_partial_and_sparse_files() {
let root = tempfile::tempdir().expect("create disk root");
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
let parent = metadata_directory(&owner.inner.root, true).expect("create metadata directory");
let mut file = reserve_file(&parent, true).expect("create reserve file");
file.write_all(b"partial allocation").expect("write partial reserve");
owner.replenish().expect("finish partial allocation");
let metadata = file.metadata().expect("partial reserve metadata");
assert_eq!(metadata.len(), RESERVE_BYTES);
assert!(metadata.blocks() * 512 >= RESERVE_BYTES);
file.set_len(0).expect("truncate reserve");
file.set_len(RESERVE_BYTES).expect("create sparse reserve");
owner.inner.armed.store(false, Ordering::Release);
assert!(file.metadata().expect("sparse reserve metadata").blocks() * 512 < RESERVE_BYTES);
owner.replenish().expect("repair sparse allocation");
let metadata = file.metadata().expect("repaired reserve metadata");
assert_eq!(metadata.len(), RESERVE_BYTES);
assert!(metadata.blocks() * 512 >= RESERVE_BYTES);
}
#[test]
fn delete_reserve_shares_reopened_owner_and_serializes_recovery_with_replenishment() {
let root = tempfile::tempdir().expect("create disk root");
let first = DeleteReserve::acquire(root.path()).expect("open first root");
let second = DeleteReserve::acquire(root.path()).expect("reopen root");
assert!(Arc::ptr_eq(&first, &second));
first.initialize().expect("allocate reserve");
let (retry_entered_tx, retry_entered_rx) = std::sync::mpsc::channel();
let (allow_retry_tx, allow_retry_rx) = std::sync::mpsc::channel();
let (replenished_tx, replenished_rx) = std::sync::mpsc::channel();
let recovery_owner = first.clone();
let root_path = root.path().to_path_buf();
std::thread::scope(|scope| {
let recovery = scope.spawn(move || {
let mut calls = 0;
recovery_owner
.retry_after_enospc(|| {
calls += 1;
if calls == 1 {
return Err(io::Error::new(io::ErrorKind::StorageFull, "scratch allocation"));
}
assert!(recovery_owner.inner.mutation.try_lock().is_none(), "retry must retain the reserve mutex");
retry_entered_tx.send(()).expect("notify retry");
allow_retry_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("allow retry completion");
assert_eq!(std::fs::metadata(reserve_path(&root_path)).expect("depleted reserve").len(), 0);
Ok(())
})
.expect("retry allocation");
});
retry_entered_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("retry entered");
let replenish = scope.spawn(move || {
second.replenish().expect("concurrent replenishment");
replenished_tx.send(()).expect("notify replenishment");
});
assert!(matches!(replenished_rx.try_recv(), Err(std::sync::mpsc::TryRecvError::Empty)));
allow_retry_tx.send(()).expect("finish retry");
recovery.join().expect("recovery worker");
replenish.join().expect("replenishment worker");
});
assert!(first.is_armed());
assert_eq!(std::fs::metadata(reserve_path(root.path())).expect("final reserve").len(), RESERVE_BYTES);
}
#[test]
fn delete_reserve_gives_concurrent_enospc_waiters_one_retry() {
let root = tempfile::tempdir().expect("create disk root");
let first = DeleteReserve::acquire(root.path()).expect("open first owner");
let second = DeleteReserve::acquire(root.path()).expect("open second owner");
first.initialize().expect("allocate reserve");
let (first_retry_tx, first_retry_rx) = std::sync::mpsc::channel();
let (allow_first_tx, allow_first_rx) = std::sync::mpsc::channel();
let (second_failed_tx, second_failed_rx) = std::sync::mpsc::channel();
let (second_done_tx, second_done_rx) = std::sync::mpsc::channel();
std::thread::scope(|scope| {
let first_waiter = scope.spawn(move || {
let mut calls = 0;
first
.retry_after_enospc(|| {
calls += 1;
if calls == 1 {
return Err(io::Error::new(io::ErrorKind::StorageFull, "first ENOSPC"));
}
first_retry_tx.send(()).expect("notify first retry");
allow_first_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("allow first retry to finish");
Ok(())
})
.expect("first waiter retries");
assert_eq!(calls, 2);
});
first_retry_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("first waiter holds recovery lock");
let second_waiter = scope.spawn(move || {
let mut calls = 0;
second
.retry_after_enospc(|| {
calls += 1;
if calls == 1 {
second_failed_tx.send(()).expect("notify second ENOSPC");
return Err(io::Error::new(io::ErrorKind::StorageFull, "second ENOSPC"));
}
Ok(())
})
.expect("second waiter retries after shared release");
assert_eq!(calls, 2);
second_done_tx.send(()).expect("notify second completion");
});
second_failed_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("second waiter observed ENOSPC");
assert!(matches!(second_done_rx.try_recv(), Err(std::sync::mpsc::TryRecvError::Empty)));
allow_first_tx.send(()).expect("finish first retry");
first_waiter.join().expect("first waiter");
second_done_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("second waiter completed its bounded retry");
second_waiter.join().expect("second waiter");
});
assert_eq!(std::fs::metadata(reserve_path(root.path())).expect("depleted reserve").len(), 0);
}
#[test]
fn delete_reserve_rejects_oversize_files_and_fifos() {
let root = tempfile::tempdir().expect("create disk root");
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
let parent = metadata_directory(&owner.inner.root, true).expect("create metadata directory");
let file = reserve_file(&parent, true).expect("create reserve file");
file.set_len(RESERVE_BYTES + 1).expect("create oversized reserve");
assert!(owner.replenish().is_err());
assert!(owner.inner.release_locked().is_err());
assert_eq!(file.metadata().expect("oversize metadata").len(), RESERVE_BYTES + 1);
let path = reserve_path(root.path());
std::fs::remove_file(&path).expect("remove oversized file");
assert!(
std::process::Command::new("mkfifo")
.arg(&path)
.status()
.expect("create FIFO fixture")
.success()
);
assert!(owner.replenish().is_err());
assert!(owner.inner.release_locked().is_err());
use std::os::unix::fs::FileTypeExt;
assert!(std::fs::symlink_metadata(path).expect("FIFO metadata").file_type().is_fifo());
}
}
}
File diff suppressed because it is too large Load Diff
+166 -15
View File
@@ -303,6 +303,24 @@ fn log_read_error(error: &std::io::Error) {
}
}
/// Verify one borrowed `[hash][data]` frame before exposing its payload.
/// A short digest prefix and a digest mismatch both fail without copying data.
pub(crate) fn verified_bitrot_frame<'a>(
hash_algo: &HashAlgorithm,
skip_verify: bool,
block: &'a [u8],
) -> std::io::Result<&'a [u8]> {
let hash_size = hash_algo.size();
if block.len() < hash_size {
return Err(std::io::Error::new(std::io::ErrorKind::UnexpectedEof, "truncated bitrot frame digest"));
}
let (hash, data) = block.split_at(hash_size);
if !skip_verify && hash_algo.hash_encode(data).as_ref() != hash {
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, DiskError::FileCorrupt));
}
Ok(data)
}
/// Split a `[hash][data]` block, verify the hash (unless `skip_verify`), and
/// return the data slice plus the time spent hashing. The caller writes `data`
/// into its sink **only after** this returns `Ok`, so a shard that fails the
@@ -310,17 +328,13 @@ fn log_read_error(error: &std::io::Error) {
/// rather than stored so this stays a free function usable while `self` is
/// borrowed for the block.
fn split_and_verify<'a>(hash_algo: &HashAlgorithm, skip_verify: bool, block: &'a [u8]) -> std::io::Result<(&'a [u8], Duration)> {
let (hash, data) = block.split_at(hash_algo.size());
if skip_verify {
return Ok((data, Duration::ZERO));
}
let verify_start = std::time::Instant::now();
let actual_hash = hash_algo.hash_encode(data);
let verify = verify_start.elapsed();
if actual_hash.as_ref() != hash {
log_hash_mismatch(data.len());
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, DiskError::FileCorrupt));
}
let verify_start = (!skip_verify).then(std::time::Instant::now);
let data = verified_bitrot_frame(hash_algo, skip_verify, block).inspect_err(|err| {
if err.kind() == std::io::ErrorKind::InvalidData {
log_hash_mismatch(block.len().saturating_sub(hash_algo.size()));
}
})?;
let verify = verify_start.map_or(Duration::ZERO, |start| start.elapsed());
Ok((data, verify))
}
@@ -965,10 +979,11 @@ pub struct InlineBuffer {
foreground: Vec<u8>,
background: Option<crate::disk::background_memory::BackgroundBuffer>,
filled: usize,
max_len: Option<usize>,
}
impl InlineBuffer {
fn new() -> Self {
fn new(max_len: Option<usize>) -> Self {
let background = crate::disk::current_resource_unit()
.and_then(|unit| unit.memory_budget())
.map(|budget| crate::disk::background_memory::BackgroundBuffer::new(Some(budget)));
@@ -976,6 +991,7 @@ impl InlineBuffer {
foreground: Vec::new(),
background,
filled: 0,
max_len,
}
}
@@ -990,10 +1006,22 @@ impl InlineBuffer {
let total = bufs
.iter()
.try_fold(0usize, |length, buf| length.checked_add(buf.len()).ok_or(std::io::ErrorKind::OutOfMemory))?;
let length = self
.as_slice()
.len()
.checked_add(total)
.ok_or(std::io::ErrorKind::OutOfMemory)?;
if self.max_len.is_some_and(|limit| length > limit) {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"inline shard exceeds its admitted buffer limit",
));
}
if let Some(buffer) = &mut self.background {
let length = self.filled.checked_add(total).ok_or(std::io::ErrorKind::OutOfMemory)?;
if length > buffer.as_slice().len() {
let capacity = buffer.as_slice().len().checked_mul(2).unwrap_or(length).max(length);
let capacity = self
.max_len
.unwrap_or_else(|| buffer.as_slice().len().checked_mul(2).unwrap_or(length).max(length));
// Admission includes both allocations until the prefix has been copied.
buffer.grow(capacity, self.filled)?;
}
@@ -1004,6 +1032,13 @@ impl InlineBuffer {
}
self.filled = length;
} else {
if let Some(limit) = self.max_len
&& length > self.foreground.capacity()
{
self.foreground
.try_reserve_exact(limit - self.foreground.len())
.map_err(std::io::Error::other)?;
}
for buf in bufs {
self.foreground.extend_from_slice(buf);
}
@@ -1022,7 +1057,11 @@ impl InlineBuffer {
impl CustomWriter {
/// Create a new inline buffer writer
pub fn new_inline_buffer() -> Self {
Self::InlineBuffer(InlineBuffer::new())
Self::InlineBuffer(InlineBuffer::new(None))
}
pub(crate) fn new_bounded_inline_buffer(max_len: usize) -> Self {
Self::InlineBuffer(InlineBuffer::new(Some(max_len)))
}
/// Create a new disk writer from any AsyncWrite implementation
@@ -1475,6 +1514,80 @@ mod tests {
use std::time::Duration;
use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt, ReadBuf};
#[test]
fn borrowed_bitrot_frame_verification_preserves_slices_and_rejects_short_prefixes() {
for algo in [
HashAlgorithm::HighwayHash256S,
HashAlgorithm::HighwayHash256SLegacy,
HashAlgorithm::SHA256,
HashAlgorithm::None,
] {
let payload = b"borrowed frame";
let mut frame = algo.hash_encode(payload).as_ref().to_vec();
frame.extend_from_slice(payload);
let verified = super::verified_bitrot_frame(&algo, false, &frame).expect("valid borrowed frame");
assert_eq!(verified, payload);
assert_eq!(verified.as_ptr(), frame[algo.size()..].as_ptr());
for prefix_len in 0..algo.size() {
for skip_verify in [false, true] {
assert_eq!(
super::split_and_verify(&algo, skip_verify, &frame[..prefix_len])
.expect_err("short prefix must be an error")
.kind(),
io::ErrorKind::UnexpectedEof
);
}
}
if algo.size() != 0 {
frame[0] ^= 1;
assert_eq!(
super::verified_bitrot_frame(&algo, false, &frame)
.expect_err("damaged digest")
.kind(),
io::ErrorKind::InvalidData
);
assert_eq!(
super::verified_bitrot_frame(&algo, true, &frame).expect("explicit diagnostic skip"),
payload
);
}
}
}
#[tokio::test]
#[ignore = "manual ABBA comparison of borrowed and streaming verification kernels"]
async fn inline_bitrot_borrowed_verification_benchmark() {
let algo = HashAlgorithm::HighwayHash256S;
for (part_size, shard_size, iterations) in [(4096, 4096, 20_000), (128 * 1024, 64 * 1024, 2000)] {
let data = vec![0x7b; part_size];
let mut encoded = Vec::new();
for part in data.chunks(shard_size) {
encoded.extend_from_slice(algo.hash_encode(part).as_ref());
encoded.extend_from_slice(part);
}
for borrowed in [false, true, true, false] {
let start = std::time::Instant::now();
for _ in 0..iterations {
let encoded = std::hint::black_box(encoded.as_slice());
if borrowed {
for frame in encoded.chunks(shard_size + algo.size()) {
std::hint::black_box(super::verified_bitrot_frame(&algo, false, frame).expect("borrowed frame"));
}
} else {
bitrot_verify(Cursor::new(encoded), encoded.len(), part_size, algo.clone(), shard_size)
.await
.expect("streamed frame");
}
}
println!(
"inline_verify debug={} size={part_size} borrowed={borrowed} iterations={iterations} elapsed_ns={}",
cfg!(debug_assertions),
start.elapsed().as_nanos()
);
}
}
}
#[test]
fn bitrot_damage_preserves_typed_repair_evidence() {
use crate::disk::error::Error as DiskError;
@@ -2943,6 +3056,25 @@ mod tests {
}
}
#[tokio::test]
async fn bounded_inline_buffer_rejects_vectored_overflow_atomically() {
use tokio::io::AsyncWriteExt;
let mut writer = CustomWriter::new_bounded_inline_buffer(64);
writer.write_all(&[7; 32]).await.expect("initial inline prefix");
let error = writer
.write_vectored(&[IoSlice::new(&[8; 16]), IoSlice::new(&[9; 17])])
.await
.expect_err("combined vectored data exceeds the bound");
assert_eq!(error.kind(), io::ErrorKind::InvalidData);
assert_eq!(writer.get_inline_data(), Some(&[7; 32][..]));
writer.write_all(&[8; 32]).await.expect("fill exactly to the bound");
assert!(writer.write_all(&[1]).await.is_err());
if let CustomWriter::InlineBuffer(buffer) = &writer {
assert!(buffer.foreground.capacity() <= 64);
}
assert_eq!(writer.into_inline_data().expect("export bounded bytes").len(), 64);
}
mod inline_memory_regressions {
use super::*;
use crate::disk::background_memory::is_memory_pressure;
@@ -2957,6 +3089,25 @@ mod tests {
.await
}
#[tokio::test]
async fn bounded_inline_writer_retains_its_memory_receipt() {
let memory = BackgroundMemoryBudget::new();
memory.set_sampled_limit(256, Duration::from_secs(30));
let (resources, _) = test_resource_units(1);
let mut writer = with_optional_resource_unit(Some(resources.with_memory_budget(Arc::clone(&memory))), async {
CustomWriter::new_bounded_inline_buffer(64)
})
.await;
writer.write_all(&[3; 32]).await.expect("bounded background write");
let bytes = writer.into_inline_data().expect("bounded bytes");
let clone = bytes.clone();
assert_eq!(memory.snapshot().used, 64);
drop(bytes);
assert_eq!(memory.snapshot().used, 64);
drop(clone);
assert_eq!(memory.snapshot().used, 0);
}
#[tokio::test]
async fn inline_writer_rejects_closed_and_expired_owners_before_first_allocation() {
for expired in [false, true] {
+65 -13
View File
@@ -812,12 +812,20 @@ pub(crate) fn create_deferred_bitrot_reader_with_stripe_handle(
/// a fixed body length when locating the authenticated trailer. Clamping it
/// to `0` would claim an empty body and misframe the stream. `0` stays `0`
/// because a genuinely empty object still means an empty body.
fn bitrot_create_file_size(length: i64, shard_size: usize, checksum_algo: &HashAlgorithm) -> i64 {
fn bitrot_create_file_size(length: i64, shard_size: usize, checksum_algo: &HashAlgorithm) -> disk::error::Result<i64> {
if length <= 0 {
return length;
return Ok(length);
}
let length = length as usize;
(length.div_ceil(shard_size) * checksum_algo.size() + length) as i64
if shard_size == 0 {
return Err(DiskError::other("bitrot shard size is zero"));
}
let payload = usize::try_from(length).map_err(|_| DiskError::other("bitrot shard length exceeds usize"))?;
let framed = payload
.div_ceil(shard_size)
.checked_mul(checksum_algo.size())
.and_then(|overhead| payload.checked_add(overhead))
.ok_or_else(|| DiskError::other("bitrot framed length overflows"))?;
i64::try_from(framed).map_err(|_| DiskError::other("bitrot framed length exceeds i64"))
}
pub async fn create_bitrot_writer(
@@ -843,12 +851,16 @@ pub async fn create_bitrot_writer_with_orig_volume(
shard_size: usize,
checksum_algo: HashAlgorithm,
) -> disk::error::Result<BitrotWriterWrapper> {
let framed_length = bitrot_create_file_size(length, shard_size, &checksum_algo)?;
let writer = if is_inline_buffer {
CustomWriter::new_inline_buffer()
if framed_length < 0 {
CustomWriter::new_inline_buffer()
} else {
let limit = usize::try_from(framed_length).map_err(|_| DiskError::other("inline buffer limit exceeds usize"))?;
CustomWriter::new_bounded_inline_buffer(limit)
}
} else if let Some(disk) = disk {
let length = bitrot_create_file_size(length, shard_size, &checksum_algo);
let file = disk.create_file(origvolume, volume, path, length).await?;
let file = disk.create_file(origvolume, volume, path, framed_length).await?;
#[cfg(feature = "hotpath")]
let file = instrument_raw_shard_writer(file, disk.is_local());
CustomWriter::from_file_writer(file)
@@ -867,23 +879,41 @@ mod tests {
#[test]
fn bitrot_create_file_size_grows_known_length_by_checksums() {
// 10 bytes over 4-byte shards = 3 shards, each followed by a 32-byte hash.
assert_eq!(bitrot_create_file_size(10, 4, &HashAlgorithm::HighwayHash256), 10 + 3 * 32);
assert_eq!(bitrot_create_file_size(10, 4, &HashAlgorithm::None), 10);
// 10 bytes over 4-byte shards = 3 frames with a 32-byte prefix each.
assert_eq!(
bitrot_create_file_size(10, 4, &HashAlgorithm::HighwayHash256).expect("known frame size"),
10 + 3 * 32
);
assert_eq!(bitrot_create_file_size(10, 4, &HashAlgorithm::None).expect("unhashed frame size"), 10);
}
#[test]
fn bitrot_create_file_size_keeps_empty_and_unknown_distinct() {
assert_eq!(bitrot_create_file_size(0, 4, &HashAlgorithm::HighwayHash256), 0);
assert_eq!(
bitrot_create_file_size(0, 4, &HashAlgorithm::HighwayHash256).expect("empty frame size"),
0
);
// SSE/compression streams advertise SIZE_PRESERVE_LAYER (-1); the remote
// put_file_stream receiver relies on a non-positive size to parse the auth
// trailer from the stream tail, so the sentinel must survive untouched.
assert_eq!(
bitrot_create_file_size(rustfs_rio::HashReader::SIZE_PRESERVE_LAYER, 4, &HashAlgorithm::HighwayHash256),
bitrot_create_file_size(rustfs_rio::HashReader::SIZE_PRESERVE_LAYER, 4, &HashAlgorithm::HighwayHash256)
.expect("unknown length sentinel"),
rustfs_rio::HashReader::SIZE_PRESERVE_LAYER
);
}
#[test]
fn bitrot_create_file_size_rejects_invalid_geometry_and_overflow() {
assert!(bitrot_create_file_size(1, 0, &HashAlgorithm::HighwayHash256S).is_err());
assert!(bitrot_create_file_size(i64::MAX, 1, &HashAlgorithm::HighwayHash256S).is_err());
assert_eq!(
bitrot_create_file_size(-7, 0, &HashAlgorithm::HighwayHash256S).expect("negative sentinel"),
-7
);
assert_eq!(bitrot_create_file_size(0, 0, &HashAlgorithm::HighwayHash256S).expect("empty body"), 0);
}
struct TestChunkReader {
chunks: VecDeque<Bytes>,
}
@@ -1762,6 +1792,28 @@ mod tests {
assert!(result.unwrap().is_none());
}
#[tokio::test]
async fn bounded_inline_bitrot_writer_counts_each_frame_header() {
let mut writer = create_bitrot_writer(true, None, "bucket", "object", 6, 4, HashAlgorithm::HighwayHash256S)
.await
.expect("bounded inline writer");
writer.write(b"abcd").await.expect("first full frame");
writer.write(b"ef").await.expect("final partial frame");
let raw = writer.into_inline_data().expect("export framed bytes");
assert_eq!(raw.len(), 6 + 2 * 32);
let mut exact = create_bitrot_writer(true, None, "bucket", "object", 4, 4, HashAlgorithm::HighwayHash256S)
.await
.expect("exact bounded writer");
exact.write(b"abcd").await.expect("fill admitted frame");
assert!(exact.write(b"e").await.is_err());
assert_eq!(exact.into_inline_data().expect("intact first frame").len(), 4 + 32);
assert!(
create_bitrot_writer(true, None, "bucket", "object", 1, 0, HashAlgorithm::HighwayHash256S)
.await
.is_err()
);
}
#[tokio::test]
async fn test_create_bitrot_writer_inline() {
use rustfs_utils::HashAlgorithm;
+15
View File
@@ -1280,6 +1280,10 @@ pub struct ObjectOptions {
/// True when an upper layer already holds the object read lock before
/// forwarding a no_lock read to the set layer.
pub metadata_cache_safe: bool,
/// Read-quorum proof captured from the live distributed namespace guard.
/// This transient value is never serialized or persisted.
#[doc(hidden)]
pub metadata_cache_fence_proof: Option<Arc<rustfs_lock::CacheFenceProof>>,
pub versioned: bool,
pub version_suspended: bool,
@@ -1388,6 +1392,10 @@ pub struct ObjectOptions {
#[doc(hidden)]
pub multipart_completion_authorization:
Option<Arc<dyn crate::storage_api_contracts::multipart::MultipartCompletionAuthorization<Error = Error>>>,
/// In-process authorization against the selected upload, while Abort holds its write lock.
#[doc(hidden)]
pub multipart_abort_authorization:
Option<Arc<dyn crate::storage_api_contracts::multipart::MultipartAbortAuthorization<Error = Error>>>,
pub eval_metadata: Option<HashMap<String, String>>,
/// Internal compare-and-set condition for replication workers publishing
/// terminal status after remote I/O. Storage validates it while holding
@@ -1501,6 +1509,7 @@ impl std::fmt::Debug for ObjectOptions {
.field("expected_bucket_incarnation_id", &self.expected_bucket_incarnation_id)
.field("no_lock", &self.no_lock)
.field("metadata_cache_safe", &self.metadata_cache_safe)
.field("metadata_cache_fence_proof", &self.metadata_cache_fence_proof.is_some())
.field("versioned", &self.versioned)
.field("version_suspended", &self.version_suspended)
.field("incl_free_versions", &self.incl_free_versions)
@@ -1580,6 +1589,7 @@ impl ObjectOptions {
let mut opts = self.clone();
opts.no_lock = true;
opts.metadata_cache_safe = false;
opts.metadata_cache_fence_proof = None;
opts.include_part_checksums = true;
opts
}
@@ -1650,6 +1660,11 @@ impl ObjectOptions {
}
}
pub(crate) fn add_metadata_cache_read_guard(&mut self, guard: &rustfs_lock::NamespaceLockGuard) {
self.add_namespace_lock_guard(guard);
self.metadata_cache_fence_proof = guard.cache_fence_proof().cloned().map(Arc::new);
}
pub fn add_bucket_lifecycle_lock_guard(&mut self, guard: &rustfs_lock::NamespaceLockGuard) {
let fence = self.bucket_lifecycle_lock_fence.get_or_insert_with(NamespaceLockFence::new);
if let Some(signal) = guard.lock_lost_signal() {
@@ -242,6 +242,7 @@ pub(in crate::set_disk) enum MetadataCacheLookup {
Hit(Arc<GetObjectMetadataCacheEntry>),
Miss,
RejectedInsufficientQuorum,
RejectedFenceProof,
}
pub(in crate::set_disk) fn is_confirmed_missing_part_error(err: Option<&str>) -> bool {
@@ -4009,6 +4010,7 @@ impl SetDisks {
file_info.validate_for_erasure_write()?;
}
}
let file_infos = Arc::new(file_infos);
let disk_count = disks.len();
let fanout_disks = disks.to_vec();
@@ -4031,19 +4033,16 @@ impl SetDisks {
let successful_rename_completion_rank =
rustfs_io_metrics::put_stage_metrics_enabled().then(|| Arc::new(AtomicUsize::new(0)));
let mut tasks = JoinSet::new();
for (i, ((disk, file_info), scanner_publication_lease_token)) in fanout_disks
.into_iter()
.zip(file_infos.iter())
.zip(fanout_fence_tokens.into_iter())
.enumerate()
for (i, (disk, scanner_publication_lease_token)) in
fanout_disks.into_iter().zip(fanout_fence_tokens.into_iter()).enumerate()
{
let src_bucket = fanout_src_bucket.clone();
let src_object = fanout_src_object.clone();
let dst_bucket = fanout_dst_bucket.clone();
let dst_object = fanout_dst_object.clone();
let file_info = file_info.clone();
let successful_rename_completion_rank = successful_rename_completion_rank.clone();
let namespace_commit_guard = namespace_commit_guard.clone();
let file_infos = file_infos.clone();
tasks.spawn(async move {
let _namespace_commit_guard = namespace_commit_guard.clone();
let mut dispatch_state = RenameDispatchState::NotDispatched;
@@ -4060,12 +4059,16 @@ impl SetDisks {
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_DISK_WAIT_REMOTE
};
let file_info = &file_infos[i];
let is_delete_marker = file_info.is_canonical_delete_marker();
// Clone FileInfo and set erasure.index for this disk
let mut file_info = file_info.clone();
if file_info.erasure.index == 0 {
file_info.erasure.index = i + 1;
}
let mut indexed_file_info;
let file_info = if file_info.erasure.index == 0 {
indexed_file_info = file_info.clone();
indexed_file_info.erasure.index = i + 1;
&indexed_file_info
} else {
file_info
};
if file_info.erasure.index == 0 || (!is_delete_marker && !file_info.has_valid_erasure_geometry()) {
return Err(DiskError::FileCorrupt);
}
@@ -4083,7 +4086,7 @@ impl SetDisks {
.rename_data_borrowed_with_fence_observed(
&src_bucket,
&src_object,
&file_info,
file_info,
&dst_bucket,
&dst_object,
disk::RenameDataGuards {
@@ -4191,7 +4194,7 @@ impl SetDisks {
let snapshot_commit = Self::rename_data_commit_from_observations(
RenameDataObservations {
disks: &coordinator_disks,
file_infos: &file_infos,
file_infos: file_infos.as_slice(),
disk_versions: &disk_versions,
errs: &errs,
cleanup_data_dirs: &cleanup_data_dirs,
@@ -4236,6 +4239,7 @@ impl SetDisks {
if !sent_commit {
let ret_err =
reduce_write_quorum_errs(&errs, OBJECT_OP_IGNORED_ERRS, write_quorum).unwrap_or(DiskError::Unexpected);
let file_infos = Arc::try_unwrap(file_infos).unwrap_or_else(|file_infos| file_infos.as_ref().clone());
rollback_failed_rename(
&coordinator_disks,
file_infos,
@@ -9900,6 +9904,19 @@ mod tests {
("RUSTFS_GET_METADATA_EARLY_STOP_BOUNDED_FANOUT", Some("true")),
],
async {
let fanout_order = bounded_metadata_fanout_order(bucket, &object, DISKS, 2);
let distribution = FileInfo::new(&metadata_distribution_key(bucket, &object), 2, 2)
.erasure
.distribution;
let initial_parity_disk = *fanout_order
.iter()
.take(3)
.find(|disk_index| distribution[**disk_index] > 2)
.expect("initial fanout should include one parity shard");
// Hold the initial parity response so the single-pending-read
// hedge is exercised independently of task completion order.
let initial_barrier =
rename_fanout_barrier::arm(&object, initial_parity_disk, rename_fanout_barrier::PHASE_READ_VERSION);
let spare_disk = bounded_spare_disk_index(bucket, &object, 2, 2);
let barrier = rename_fanout_barrier::arm(&object, spare_disk, rename_fanout_barrier::PHASE_READ_VERSION);
let tracker = rename_fanout_barrier::observe_tasks(&object);
@@ -9922,9 +9939,13 @@ mod tests {
.await
});
tokio::time::timeout(BARRIER_PAUSE_GUARD, initial_barrier.wait_until_paused())
.await
.expect("initial parity response should pause before quorum");
tokio::time::timeout(BARRIER_PAUSE_GUARD, barrier.wait_until_paused())
.await
.expect("bounded fanout should hedge and pause the spare disk");
initial_barrier.release();
let (parts_metadata, errs, diagnostics) = tokio::time::timeout(BARRIER_PAUSE_GUARD, &mut read)
.await
.expect("verified inline quorum should return without the paused spare")
@@ -10218,6 +10239,15 @@ mod tests {
data_read_early_stop_inline_body_miss_reason(bucket, object, &candidate, &corrupt, &disks).await,
Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
);
let mut trailing = parts_metadata.clone();
let mut inline_data = trailing[data_disk].data.as_ref().expect("fixture inline bytes").to_vec();
inline_data.push(0x7b);
trailing[data_disk].data = Some(Bytes::from(inline_data));
assert_eq!(
data_read_early_stop_inline_body_miss_reason(bucket, object, &candidate, &trailing, &disks).await,
Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
);
}
#[test]
@@ -36,12 +36,12 @@ use crate::disk::{
self, BATCH_READ_VERSION_MAX_ITEMS, BatchReadVersionItem, BatchReadVersionReq, BatchReadVersionResp, Disk, DiskAPI,
};
use crate::set_disk::{
DiskError, DiskStore, FileInfo, HashAlgorithm, ReadOptions, SetDisks, can_try_inline_data_shards_direct,
codec_streaming_rollout_applies, collect_inline_data_shard_fileinfos_from_observations, file_info_is_valid_for_metadata,
get_metadata_slowtail_fault_request, inline_erasure_shard_file_offset, inline_erasure_shard_size,
is_get_metadata_data_read_early_stop_enabled, is_get_metadata_early_stop_bounded_fanout_enabled,
is_get_metadata_early_stop_enabled, is_get_metadata_non_inline_data_read_early_stop_enabled, is_version_early_stop_enabled,
object_fits_single_block,
DiskError, DiskStore, FileInfo, ReadOptions, SetDisks, codec_streaming_rollout_applies,
collect_inline_data_shard_fileinfos_from_observations, file_info_is_valid_for_metadata, get_metadata_slowtail_fault_request,
inline_body_buffering_miss_reason, is_get_metadata_data_read_early_stop_enabled,
is_get_metadata_early_stop_bounded_fanout_enabled, is_get_metadata_early_stop_enabled,
is_get_metadata_non_inline_data_read_early_stop_enabled, is_version_early_stop_enabled, object_fits_single_block,
try_decode_inline_data_shards,
};
use futures::future::join_all;
use metrics::counter;
@@ -656,68 +656,8 @@ fn verify_inline_body_from_observations<'a>(
disks.get(index).is_some_and(Option::is_some)
})?;
let Some(part) = candidate.parts.first() else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_PART_SHAPE);
};
let Ok(object_size) = usize::try_from(candidate.size) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
};
let checksum_info = candidate.erasure.get_checksum_info(part.number);
let checksum_algo = if candidate.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
HashAlgorithm::HighwayHash256SLegacy
} else {
checksum_info.algorithm
};
let read_length = inline_erasure_shard_file_offset(
0,
object_size,
object_size,
candidate.erasure.block_size,
data_shards,
candidate.uses_legacy_checksum,
);
let shard_size = inline_erasure_shard_size(candidate.erasure.block_size, data_shards, candidate.uses_legacy_checksum);
if object_size == 0 || read_length == 0 || read_length > shard_size || data_files.len() < data_shards {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
}
let shards_needed = object_size.div_ceil(read_length);
if shards_needed > data_shards {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
}
let Some(encoded_capacity) = read_length.checked_mul(shards_needed) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
#[cfg(test)]
{
for file in &data_files {
if file.data.is_some() {
crate::set_disk::disk_call_counters::record_shard_open(_object, true);
}
}
crate::set_disk::disk_call_counters::record(_object, crate::set_disk::disk_call_counters::KIND_BODY_DECODE, 0);
}
let hash_size = checksum_algo.size();
let need = hash_size + read_length;
let mut body = Vec::with_capacity(encoded_capacity);
for file in data_files.iter().take(shards_needed) {
let Some(raw) = file.data.as_deref() else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let Some(block) = raw.get(..need) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let (hash, data) = block.split_at(hash_size);
if hash_size > 0 && checksum_algo.hash_encode(data).as_ref() != hash {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
}
body.extend_from_slice(data);
if body.len() >= object_size {
body.truncate(object_size);
return Ok(bytes::Bytes::from(body));
}
}
Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
try_decode_inline_data_shards(_object, candidate, &data_files, false)
.ok_or(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
}
fn data_read_early_stop_inline_candidate_miss_reason(candidate: &FileInfo) -> Option<&'static str> {
@@ -755,10 +695,7 @@ fn data_read_early_stop_inline_candidate_miss_reason(candidate: &FileInfo) -> Op
if candidate.parts.first().is_none_or(|part| part.size != object_size) {
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_PART_SHAPE);
}
if !can_try_inline_data_shards_direct(object_size, candidate.erasure.block_size) {
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
}
None
inline_body_buffering_miss_reason(candidate)
}
pub(in crate::set_disk) fn non_inline_data_read_candidate_is_safe(candidate: &FileInfo) -> bool {
+340 -164
View File
@@ -51,12 +51,13 @@ use crate::bucket::replication::{ReplicateDecision, ReplicationObjectBridge, Rep
use crate::cluster::rpc::heal_bucket_local_on_disks;
use crate::data_usage::record_compression_total_memory;
use crate::diagnostics::get::{
GET_CODEC_STREAMING_OBJECT_CLASS_PLAIN_SINGLE_PART, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY,
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_IDENTITY_MISMATCH,
GET_CODEC_STREAMING_OBJECT_CLASS_PLAIN_SINGLE_PART, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY,
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_IDENTITY_MISMATCH,
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_PAYLOAD,
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_SHARD, GET_OBJECT_PATH_BODY_CACHE, GET_OBJECT_PATH_CODEC_STREAMING,
GET_OBJECT_PATH_CODEC_STREAMING_LEGACY_ENGINE, GET_OBJECT_PATH_CODEC_STREAMING_RUSTFS_ENGINE, GET_OBJECT_PATH_DIRECT_MEMORY,
GET_OBJECT_PATH_EMPTY, GET_OBJECT_PATH_INLINE_DIRECT, GET_OBJECT_PATH_INTERNAL_META, GET_OBJECT_PATH_LEGACY_DUPLEX,
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_SHARD, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE,
GET_OBJECT_PATH_BODY_CACHE, GET_OBJECT_PATH_CODEC_STREAMING, GET_OBJECT_PATH_CODEC_STREAMING_LEGACY_ENGINE,
GET_OBJECT_PATH_CODEC_STREAMING_RUSTFS_ENGINE, GET_OBJECT_PATH_DIRECT_MEMORY, GET_OBJECT_PATH_EMPTY,
GET_OBJECT_PATH_INLINE_DIRECT, GET_OBJECT_PATH_INTERNAL_META, GET_OBJECT_PATH_LEGACY_DUPLEX,
GET_OBJECT_PATH_REMOTE_TRANSITION, GET_OBJECT_PATH_SET_DISK, GET_STAGE_DECODE, GET_STAGE_EMIT, GET_STAGE_INLINE_PREPARE,
GET_STAGE_LOCK_ACQUIRE, GET_STAGE_METADATA, GET_STAGE_OBJECT_INFO, GET_STAGE_PATH_DECISION, GET_STAGE_READER_SETUP,
classify_storage_error, get_stage_timer_if_enabled, record_get_object_pipeline_failure,
@@ -727,6 +728,11 @@ const DISK_HEALTH_CACHE_TTL: Duration = Duration::from_millis(750);
const GET_OBJECT_METADATA_CACHE_TTL: Duration = Duration::from_secs(2); // Increased from 250ms to 2s
const DEFAULT_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES: usize = 4096; // Increased from 1024 to 4096
const ENV_RUSTFS_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES: &str = "RUSTFS_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES";
#[cfg(any(test, feature = "e2e-test-hooks"))]
const ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE: &str =
"RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE";
#[cfg(any(test, feature = "e2e-test-hooks"))]
const DEFAULT_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE: bool = false;
const GET_OBJECT_METADATA_CACHE_FENCE_SHARDS: u16 = 4096;
// --- Codec Streaming Configuration ---
@@ -787,7 +793,8 @@ const DEFAULT_RUSTFS_GET_SMALL_OBJECT_DIRECT_MEMORY_THRESHOLD: usize = 128 * 102
// Bound the in-memory inline decoder independently from the configurable
// storage-class admission policy. This protects its Vec allocation while
// allowing explicitly configured EC layouts to inline objects above 128 KiB.
const INLINE_FAST_PATH_MAX_OBJECT_SIZE: i64 = 1024 * 1024;
const INLINE_FAST_PATH_MAX_OBJECT_SIZE: usize = 2 * 1024 * 1024;
const INLINE_FAST_PATH_MAX_BLOCKS: usize = 32;
// --- Metadata Early-Stop Configuration ---
@@ -1515,6 +1522,7 @@ mod prepared_get_object_metadata_tests {
online_disks: vec![None],
read_quorum: 0,
verified_inline_body: None,
distributed_fence_proof: None,
});
let snapshot = GetObjectFileInfo::shared(Arc::clone(&cached));
@@ -3443,6 +3451,20 @@ fn get_object_metadata_cache_max_entries() -> usize {
}
}
pub(crate) fn is_get_object_metadata_cache_distributed_proof_enabled() -> bool {
#[cfg(any(test, feature = "e2e-test-hooks"))]
{
rustfs_utils::get_env_bool(
ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE,
DEFAULT_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE,
)
}
#[cfg(not(any(test, feature = "e2e-test-hooks")))]
{
false
}
}
fn is_get_small_object_direct_memory_enabled() -> bool {
#[cfg(test)]
{
@@ -5016,6 +5038,7 @@ struct GetObjectMetadataCacheEntry {
read_quorum: usize,
capability: GetObjectMetadataCacheCapability,
verified_inline_body: Option<Bytes>,
distributed_fence_proof: Option<Arc<rustfs_lock::CacheFenceProof>>,
}
#[derive(Clone, Debug)]
@@ -6049,6 +6072,51 @@ fn should_use_inline_fast_path(
should_use_inline_fast_path_with_plan(range, object_info, fi, opts, ReadPathPlan::new(object_info, fi))
}
fn inline_body_fits_buffer(size: usize, block_size: usize) -> bool {
size > 0
&& size <= INLINE_FAST_PATH_MAX_OBJECT_SIZE
&& block_size != 0
&& size.div_ceil(block_size) <= INLINE_FAST_PATH_MAX_BLOCKS
}
fn should_inline_put(
storage_class: &crate::config::storageclass::Config,
erasure: &coding::Erasure,
stored_size: i64,
actual_size: i64,
versioned: bool,
protected: bool,
) -> bool {
let admission_size = if stored_size >= 0 {
stored_size
} else if !protected && actual_size > 0 {
actual_size
} else {
return false;
};
let Ok(size) = usize::try_from(admission_size) else {
return false;
};
if size > erasure.block_size && (stored_size < 0 || protected || !inline_body_fits_buffer(size, erasure.block_size)) {
return false;
}
storage_class.should_inline(erasure.shard_file_size(admission_size), erasure.data_shards, versioned)
}
fn inline_body_buffering_miss_reason(fi: &FileInfo) -> Option<&'static str> {
let Ok(size) = usize::try_from(fi.size) else {
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
};
if !inline_body_fits_buffer(size, fi.erasure.block_size) {
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
}
// Proof-protected parts use the readers that bind their PartProofReader.
if fi.parts.iter().any(|part| part.integrity.is_some()) {
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
}
None
}
fn should_use_inline_fast_path_with_plan(
range: &Option<HTTPRangeSpec>,
object_info: &ObjectInfo,
@@ -6060,18 +6128,12 @@ fn should_use_inline_fast_path_with_plan(
return false;
}
let Some(shape) = plan.shape() else {
if plan.shape().is_none() {
return false;
};
// The persisted marker is authoritative for the storage decision, but it
// is still untrusted metadata at this boundary. Revalidate its geometry so
// a stale/corrupt marker cannot route an oversized payload into the
// in-memory decoder. The independent direct-memory reader remains capped
// at 128 KiB below.
plan.is_plain()
&& shape.object_size <= usize::try_from(INLINE_FAST_PATH_MAX_OBJECT_SIZE).expect("inline fast path limit fits usize")
&& fi.erasure.data_blocks != 0
&& fi.erasure.block_size != 0
}
// Apply the same materialization contract as metadata pre-verification.
// The independent direct-memory reader retains its 128 KiB gate.
plan.is_plain() && inline_body_buffering_miss_reason(fi).is_none() && fi.erasure.data_blocks != 0
}
enum SmallWritePath {
@@ -6160,27 +6222,9 @@ fn known_put_object_storage_size(data_size: i64) -> i64 {
}
}
/// Shard size the inline admission check evaluates for a single PUT.
///
/// A compressed or encrypted stream reports `SIZE_PRESERVE_LAYER` as its
/// stored size because the transformed length is only known after the write.
/// MinIO's `putObject` sizes such objects for inline admission by their
/// plaintext `ActualSize`; without that fallback every transformed object,
/// however small, lands in `part.1` files. A stream with neither size known
/// yields a negative shard size, which `should_inline` rejects.
fn inline_admission_shard_size(erasure: &coding::Erasure, stored_size: i64, actual_size: i64) -> i64 {
if stored_size >= 0 {
return erasure.shard_file_size(stored_size);
}
if actual_size > 0 {
return erasure.shard_file_size(actual_size);
}
HashReader::SIZE_PRESERVE_LAYER
}
#[allow(clippy::too_many_arguments)]
async fn build_inline_bitrot_readers(
files: &[FileInfo],
async fn build_inline_bitrot_readers<'a>(
files: impl Iterator<Item = &'a FileInfo> + Send,
total_shards: usize,
bucket: &str,
object: &str,
@@ -6190,7 +6234,7 @@ async fn build_inline_bitrot_readers(
skip_verify_bitrot: bool,
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
let mut readers = Vec::with_capacity(total_shards);
for file in files.iter().take(total_shards) {
for file in files.take(total_shards) {
let reader = if let Some(data) = &file.data {
create_bitrot_reader_from_bytes(
Some(data.clone()),
@@ -6213,38 +6257,83 @@ async fn build_inline_bitrot_readers(
Ok(readers)
}
#[allow(clippy::too_many_arguments)]
async fn build_inline_bitrot_readers_from_refs(
files: &[&FileInfo],
bucket: &str,
object: &str,
read_length: usize,
shard_size: usize,
checksum_algo: &HashAlgorithm,
skip_verify_bitrot: bool,
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
let mut readers = Vec::with_capacity(files.len());
for file in files {
let reader = if let Some(data) = &file.data {
create_bitrot_reader_from_bytes(
Some(data.clone()),
None,
bucket,
object,
0,
read_length,
shard_size,
checksum_algo.clone(),
skip_verify_bitrot,
false,
)
.await?
} else {
None
};
readers.push(reader);
/// Resolve the bitrot checksum algorithm for one part, honoring the legacy
/// HighwayHash flag.
fn part_checksum_algorithm(fi: &FileInfo, part_number: usize) -> HashAlgorithm {
let checksum_info = fi.erasure.get_checksum_info(part_number);
if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
HashAlgorithm::HighwayHash256SLegacy
} else {
checksum_info.algorithm
}
Ok(readers)
}
/// Decode already identity-checked data shards without reader wrappers or RS.
/// A failed frame leaves recovery to the caller's ordinary erasure reader.
fn try_decode_inline_data_shards(
_object: &str,
fi: &FileInfo,
data_files: &[&FileInfo],
skip_verify_bitrot: bool,
) -> Option<Bytes> {
if inline_body_buffering_miss_reason(fi).is_some() {
return None;
}
let object_size = usize::try_from(fi.size).ok()?;
let part = fi.parts.first()?;
let data_shards = fi.erasure.data_blocks;
if fi.parts.len() != 1 || part.size != object_size || data_files.len() < data_shards {
return None;
}
let shard_size = inline_erasure_shard_size(fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
if shard_size == 0 {
return None;
}
let checksum_algo = part_checksum_algorithm(fi, part.number);
let hash_size = checksum_algo.size();
let stride = shard_size.checked_add(hash_size)?;
let part_size = inline_erasure_shard_file_size(part.size, fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
let encoded_size = part_size
.div_ceil(shard_size)
.checked_mul(hash_size)?
.checked_add(part_size)?;
// Metadata early-stop can cache this body, so require exact framing even
// when the caller explicitly skips bitrot digest verification.
for file in data_files.iter().take(data_shards) {
if file.data.as_ref()?.len() != encoded_size {
return None;
}
}
#[cfg(test)]
{
for _ in data_files.iter().take(data_shards) {
disk_call_counters::record_shard_open(_object, true);
}
disk_call_counters::record(_object, disk_call_counters::KIND_BODY_DECODE, 0);
}
let mut body = Vec::with_capacity(object_size);
for (frame, offset) in (0..object_size).step_by(fi.erasure.block_size).enumerate() {
let block_length = fi.erasure.block_size.min(object_size - offset);
let read_length =
inline_erasure_shard_file_size(block_length, fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
if read_length == 0 || read_length > shard_size {
return None;
}
let start = frame.checked_mul(stride)?;
let end = start.checked_add(hash_size)?.checked_add(read_length)?;
let mut remaining = block_length;
for file in data_files.iter().take(block_length.div_ceil(read_length)) {
let block = file.data.as_deref()?.get(start..end)?;
let data = coding::verified_bitrot_frame(&checksum_algo, skip_verify_bitrot, block).ok()?;
let take = remaining.min(data.len());
body.extend_from_slice(&data[..take]);
remaining -= take;
}
if remaining != 0 {
return None;
}
}
Some(Bytes::from(body))
}
async fn try_read_inline_data_shards_direct(
@@ -6285,10 +6374,6 @@ async fn try_read_inline_data_shards_direct(
None
}
fn can_try_inline_data_shards_direct(object_size: usize, block_size: usize) -> bool {
object_size > 0 && object_size <= block_size
}
fn inline_erasure_shard_size(block_size: usize, data_shards: usize, uses_legacy: bool) -> usize {
if block_size == 0 || data_shards == 0 {
return 0;
@@ -6317,31 +6402,14 @@ fn inline_erasure_shard_file_size(total_length: usize, block_size: usize, data_s
num_shards * shard_size + last_shard_size
}
fn inline_erasure_shard_file_offset(
start_offset: usize,
length: usize,
total_length: usize,
block_size: usize,
data_shards: usize,
uses_legacy: bool,
) -> usize {
if block_size == 0 || data_shards == 0 {
return 0;
}
let shard_size = inline_erasure_shard_size(block_size, data_shards, uses_legacy);
let shard_file_size = inline_erasure_shard_file_size(total_length, block_size, data_shards, uses_legacy);
let end_shard = (start_offset + length) / block_size;
let till_offset = end_shard * shard_size + shard_size;
till_offset.min(shard_file_size)
}
type InlineDataShardFiles<'a> = smallvec::SmallVec<[&'a FileInfo; shard_source::INLINE_SHARD_SLOTS]>;
fn collect_inline_data_shard_fileinfos_by_index<'a>(
parts_metadata: &'a [FileInfo],
fi: &FileInfo,
data_shards: usize,
disk_is_online: impl FnMut(usize) -> bool,
) -> Option<Vec<&'a FileInfo>> {
) -> Option<InlineDataShardFiles<'a>> {
collect_inline_data_shard_fileinfos_by_index_or_reason(parts_metadata, fi, data_shards, disk_is_online).ok()
}
@@ -6350,7 +6418,7 @@ fn collect_inline_data_shard_fileinfos_by_index_or_reason<'a>(
fi: &FileInfo,
data_shards: usize,
disk_is_online: impl FnMut(usize) -> bool,
) -> std::result::Result<Vec<&'a FileInfo>, &'static str> {
) -> std::result::Result<InlineDataShardFiles<'a>, &'static str> {
collect_inline_data_shard_fileinfos_from_observations(
parts_metadata
.iter()
@@ -6367,9 +6435,13 @@ fn collect_inline_data_shard_fileinfos_from_observations<'a>(
fi: &FileInfo,
data_shards: usize,
mut disk_is_online: impl FnMut(usize) -> bool,
) -> std::result::Result<Vec<&'a FileInfo>, &'static str> {
) -> std::result::Result<InlineDataShardFiles<'a>, &'static str> {
let distribution = &fi.erasure.distribution;
let mut data_files = vec![None; data_shards];
if data_shards == 0 || data_shards != fi.erasure.data_blocks || data_shards > distribution.len() {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY);
}
let mut data_files: smallvec::SmallVec<[Option<&FileInfo>; shard_source::INLINE_SHARD_SLOTS]> =
smallvec::smallvec![None; data_shards];
for (disk_index, file_info) in observations {
if !disk_is_online(disk_index) {
@@ -6402,7 +6474,7 @@ fn collect_inline_data_shard_fileinfos_from_observations<'a>(
data_files
.into_iter()
.collect::<Option<Vec<_>>>()
.collect::<Option<InlineDataShardFiles<'_>>>()
.ok_or(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_SHARD)
}
@@ -7368,23 +7440,20 @@ fn join_errs(errs: &[Option<DiskError>]) -> String {
errs.join(", ")
}
async fn verify_inline_part_bitrot(meta: &FileInfo) -> disk::error::Result<()> {
fn verify_inline_part_bitrot(meta: &FileInfo) -> disk::error::Result<()> {
meta.validate_for_metadata_read()?;
let [part] = meta.parts.as_slice() else {
return Err(DiskError::FileCorrupt);
};
let data = meta.data.as_deref().ok_or(DiskError::FileCorrupt)?;
let checksum = meta.erasure.get_checksum_info(part.number);
let algo = match checksum.algorithm {
HashAlgorithm::HighwayHash256S if meta.uses_legacy_checksum => HashAlgorithm::HighwayHash256SLegacy,
algo @ (HashAlgorithm::HighwayHash256S | HashAlgorithm::HighwayHash256SLegacy) => algo,
_ => return Err(DiskError::BitrotHashAlgoInvalid),
};
let algo = part_checksum_algorithm(meta, part.number);
if !matches!(algo, HashAlgorithm::HighwayHash256S | HashAlgorithm::HighwayHash256SLegacy) {
return Err(DiskError::BitrotHashAlgoInvalid);
}
let shard_size = inline_erasure_shard_size(meta.erasure.block_size, meta.erasure.data_blocks, meta.uses_legacy_checksum);
let part_size =
inline_erasure_shard_file_size(part.size, meta.erasure.block_size, meta.erasure.data_blocks, meta.uses_legacy_checksum);
// Check framing arithmetic and the physical buffer length before the shared
// verifier sizes its scratch buffer from the metadata.
// Validate the complete framing before visiting borrowed payloads.
let encoded_size = part_size
.div_ceil(shard_size)
.checked_mul(algo.size())
@@ -7393,9 +7462,11 @@ async fn verify_inline_part_bitrot(meta: &FileInfo) -> disk::error::Result<()> {
if encoded_size != data.len() {
return Err(DiskError::FileCorrupt);
}
coding::bitrot_verify(Cursor::new(data), encoded_size, part_size, algo, shard_size)
.await
.map_err(|_| DiskError::FileCorrupt)
let stride = shard_size.checked_add(algo.size()).ok_or(DiskError::FileCorrupt)?;
for frame in data.chunks(stride) {
coding::verified_bitrot_frame(&algo, false, frame).map_err(|_| DiskError::FileCorrupt)?;
}
Ok(())
}
fn validate_deep_scan_results(results: &[usize], expected_parts: usize) -> disk::error::Result<()> {
@@ -7613,7 +7684,7 @@ async fn disks_with_all_parts(
// before deciding whether reconstruction (and its bitrot readers) is needed.
if (meta.data.is_some() || meta.size == 0) && !meta.parts.is_empty() {
let part_status = if scan_mode == HealScanMode::Deep && meta.data.is_some() {
match verify_inline_part_bitrot(meta).await {
match verify_inline_part_bitrot(meta) {
Ok(()) => CHECK_PART_SUCCESS,
Err(DiskError::FileCorrupt) => CHECK_PART_FILE_CORRUPT,
Err(err) => return Err(err),
@@ -14991,21 +15062,19 @@ mod tests {
let (_, files, _, _) = inline_bitrot_files_for_payload_with_mode(&payload, legacy).await;
for mut meta in files {
assert!(meta.erasure.checksums.is_empty(), "legacy metadata may omit per-part checksum entries");
verify_inline_part_bitrot(&meta)
.await
.expect("healthy data and parity shards should verify");
verify_inline_part_bitrot(&meta).expect("healthy data and parity shards should verify");
let original = meta.data.clone().expect("fixture should retain inline bytes");
for offset in [0, 32, original.len() - 1] {
let mut damaged = original.to_vec();
damaged[offset] ^= 1;
meta.data = Some(Bytes::from(damaged));
assert_eq!(verify_inline_part_bitrot(&meta).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&meta), Err(DiskError::FileCorrupt));
}
let mut trailing = original.to_vec();
trailing.push(0x7b);
for damaged in [Vec::new(), original[..original.len() - 1].to_vec(), trailing] {
meta.data = Some(Bytes::from(damaged));
assert_eq!(verify_inline_part_bitrot(&meta).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&meta), Err(DiskError::FileCorrupt));
}
}
}
@@ -15019,19 +15088,17 @@ mod tests {
empty.parts[0].size = 0;
empty.parts[0].actual_size = 0;
empty.data = Some(Bytes::new());
verify_inline_part_bitrot(&empty)
.await
.expect("empty inline objects have no bitrot frames");
verify_inline_part_bitrot(&empty).expect("empty inline objects have no bitrot frames");
empty.data = Some(Bytes::from_static(b"unexpected"));
assert_eq!(verify_inline_part_bitrot(&empty).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&empty), Err(DiskError::FileCorrupt));
let mut invalid = files[0].clone();
invalid.erasure.block_size = 0;
assert_eq!(verify_inline_part_bitrot(&invalid).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&invalid), Err(DiskError::FileCorrupt));
invalid = files[0].clone();
invalid.parts.push(invalid.parts[0].clone());
invalid.parts[1].number = 2;
assert_eq!(verify_inline_part_bitrot(&invalid).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&invalid), Err(DiskError::FileCorrupt));
let mut oversized = files[0].clone();
oversized.erasure.block_size = 2;
@@ -15040,7 +15107,7 @@ mod tests {
oversized
.validate_for_metadata_read()
.expect("logical shard length should fit metadata bounds");
assert_eq!(verify_inline_part_bitrot(&oversized).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&oversized), Err(DiskError::FileCorrupt));
oversized.erasure.block_size = 1 << 40;
oversized.size = 1 << 40;
@@ -15048,7 +15115,7 @@ mod tests {
oversized
.validate_for_metadata_read()
.expect("large metadata geometry should remain representable");
assert_eq!(verify_inline_part_bitrot(&oversized).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&oversized), Err(DiskError::FileCorrupt));
}
#[tokio::test]
@@ -15071,13 +15138,11 @@ mod tests {
assert_eq!(meta.uses_legacy_checksum, legacy);
assert!(meta.inline_data());
assert!(meta.data.is_some(), "the production decoder should extract the historical inline value");
verify_inline_part_bitrot(&meta)
.await
.expect("historical inline shard should pass deep verification");
verify_inline_part_bitrot(&meta).expect("historical inline shard should pass deep verification");
let mut damaged = meta.data.as_ref().expect("fixture should be inline").to_vec();
*damaged.last_mut().expect("fixture shard should not be empty") ^= 1;
meta.data = Some(Bytes::from(damaged));
assert_eq!(verify_inline_part_bitrot(&meta).await, Err(DiskError::FileCorrupt));
assert_eq!(verify_inline_part_bitrot(&meta), Err(DiskError::FileCorrupt));
}
}
@@ -15121,6 +15186,48 @@ mod tests {
fi
}
#[test]
fn inline_put_admission_bounds_only_the_multi_block_extension() {
let block = 1024 * 1024;
let config = crate::config::storageclass::Config::default();
let erasure = coding::Erasure::new(12, 4, block);
let admits = |stored, actual, protected| should_inline_put(&config, &erasure, stored, actual, false, protected);
assert!(admits(1280 * 1024, 1280 * 1024, false));
assert!(!admits(1280 * 1024, 1280 * 1024, true));
assert!(!admits(-1, 1280 * 1024, false));
assert!(admits(-1, block as i64, false));
assert!(!admits(-1, block as i64, true));
assert!(!admits(2 * 1024 * 1024 + 1, 0, false));
assert!(admits(0, 0, false));
assert!(admits(1_572_856, 0, false));
assert!(!admits(1_572_857, 0, false));
assert!(should_inline_put(&config, &erasure, 196_608, 0, true, false));
assert!(!should_inline_put(&config, &erasure, 196_609, 0, true, false));
let small_blocks = coding::Erasure::new(12, 4, 4096);
assert!(should_inline_put(&config, &small_blocks, 32 * 4096, 0, false, false));
assert!(!should_inline_put(&config, &small_blocks, 32 * 4096 + 1, 0, false, false));
}
#[test]
fn inline_body_buffering_limits_size_and_frame_count() {
let mut info = FileInfo::new("bucket/object", 12, 4);
info.size = 2 * 1024 * 1024;
info.erasure.block_size = 1024 * 1024;
assert!(inline_body_buffering_miss_reason(&info).is_none());
info.size += 1;
assert!(inline_body_buffering_miss_reason(&info).is_some());
info.erasure.block_size = 4096;
info.size = 32 * 4096;
assert!(inline_body_buffering_miss_reason(&info).is_none());
info.size += 1;
assert!(inline_body_buffering_miss_reason(&info).is_some());
info.size = -1;
assert!(inline_body_buffering_miss_reason(&info).is_some());
info.size = 1;
info.erasure.block_size = 0;
assert!(inline_body_buffering_miss_reason(&info).is_some());
}
#[test]
fn collect_inline_data_shards_by_index_uses_distribution_order() {
let distribution = vec![3, 1, 5, 2, 4, 6];
@@ -15162,6 +15269,100 @@ mod tests {
];
assert!(collect_inline_data_shard_fileinfos_by_index(&files, &fi, 2, |_| true).is_none());
assert_eq!(
collect_inline_data_shard_fileinfos_by_index_or_reason(&files, &fi, usize::MAX, |_| true).expect_err("invalid count"),
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY,
);
}
#[tokio::test]
async fn inline_frame_decoder_handles_legacy_padding_hashes_and_unhashed_parts() {
for size in [1, 3, 8, 11, 16] {
let payload = &b"abcdefghijklmnop"[..size];
for (legacy, algorithm) in [
(false, HashAlgorithm::HighwayHash256S),
(true, HashAlgorithm::HighwayHash256SLegacy),
(false, HashAlgorithm::None),
] {
let erasure = coding::Erasure::new_with_options(2, 2, 8, legacy);
let mut writers: Vec<_> = (0..4)
.map(|_| {
coding::BitrotWriterWrapper::new(
coding::CustomWriter::new_inline_buffer(),
erasure.shard_size(),
algorithm.clone(),
)
})
.collect();
for block in payload.chunks(8) {
let shards = erasure.encode_data(block).expect("encode golden frame");
for (writer, shard) in writers.iter_mut().zip(shards) {
writer.write(&shard).await.expect("write golden frame");
}
}
let mut info = FileInfo::new("bucket/object", 2, 2);
info.size = i64::try_from(payload.len()).expect("test payload size fits i64");
info.erasure.block_size = 8;
info.uses_legacy_checksum = legacy;
info.add_object_part(1, "etag".to_owned(), payload.len(), None, info.size, None, None);
info.erasure.checksums = vec![rustfs_filemeta::ChecksumInfo {
part_number: 1,
algorithm: algorithm.clone(),
..Default::default()
}];
let mut files: Vec<_> = writers
.into_iter()
.enumerate()
.map(|(index, writer)| {
let mut file = info.clone();
file.erasure.index = index + 1;
file.data = writer.into_inline_data();
file
})
.collect();
let refs = [&files[0], &files[1]];
let body = try_decode_inline_data_shards("golden", &info, &refs, false).expect("decode every golden frame");
assert_eq!(body.as_ref(), payload);
if algorithm.size() > 0 {
for file in &files {
verify_inline_part_bitrot(file).expect("deep verification covers both padded frames");
}
}
for shard_index in 0..info.erasure.data_blocks {
let original = files[shard_index].data.as_ref().expect("selected data shard").clone();
let mut trailing = original.to_vec();
trailing.push(0x7b);
for malformed in [original[..original.len() - 1].to_vec(), trailing] {
files[shard_index].data = Some(Bytes::from(malformed));
for skip_verify_bitrot in [false, true] {
assert!(
try_decode_inline_data_shards("inexact", &info, &[&files[0], &files[1]], skip_verify_bitrot)
.is_none(),
"reject inexact shard {shard_index}: legacy={legacy}, algorithm={algorithm:?}, size={}",
payload.len()
);
}
}
files[shard_index].data = Some(original);
}
let original = files[0].data.as_ref().expect("first data shard").clone();
let mut truncated = original.to_vec();
truncated.pop();
files[0].data = Some(Bytes::from(truncated));
assert!(try_decode_inline_data_shards("truncated", &info, &[&files[0], &files[1]], false).is_none());
if algorithm.size() > 0 {
assert_eq!(verify_inline_part_bitrot(&files[0]), Err(DiskError::FileCorrupt));
let mut corrupt = original.to_vec();
*corrupt.last_mut().expect("last frame payload") ^= 1;
files[0].data = Some(Bytes::from(corrupt));
assert!(try_decode_inline_data_shards("corrupt", &info, &[&files[0], &files[1]], false).is_none());
assert_eq!(verify_inline_part_bitrot(&files[0]), Err(DiskError::FileCorrupt));
assert!(try_decode_inline_data_shards("diagnostic", &info, &[&files[0], &files[1]], true).is_some());
} else {
assert_eq!(verify_inline_part_bitrot(&files[0]), Err(DiskError::BitrotHashAlgoInvalid));
}
}
}
}
#[tokio::test]
@@ -15169,7 +15370,7 @@ mod tests {
let payload = b"small inline object payload that spans data shards";
let (erasure, files, read_length, checksum_algo) = inline_bitrot_files_for_payload(payload).await;
let mut readers = build_inline_bitrot_readers(
&files,
files.iter(),
erasure.data_shards,
"bucket",
"object",
@@ -15195,7 +15396,7 @@ mod tests {
let (erasure, files, read_length, checksum_algo) = inline_bitrot_files_for_payload_with_mode(payload, true).await;
assert_ne!(payload.len() % erasure.data_shards, 0, "test payload must exercise EC padding");
let mut readers = build_inline_bitrot_readers(
&files,
files.iter(),
erasure.data_shards,
"bucket",
"object",
@@ -15226,7 +15427,7 @@ mod tests {
*second = Bytes::from(corrupted);
let mut readers = build_inline_bitrot_readers(
&files,
files.iter(),
erasure.total_shard_count(),
"bucket",
"object",
@@ -15244,31 +15445,17 @@ mod tests {
}
#[test]
fn inline_data_shards_direct_read_requires_single_block() {
assert!(can_try_inline_data_shards_direct(1024, 1024));
assert!(!can_try_inline_data_shards_direct(0, 1024));
assert!(!can_try_inline_data_shards_direct(1025, 1024));
}
#[test]
fn inline_erasure_offset_helpers_match_erasure_methods() {
fn inline_erasure_size_helpers_match_erasure_methods() {
for uses_legacy in [false, true] {
let erasure = coding::Erasure::new_with_options(4, 2, 1024 * 1024, uses_legacy);
for object_size in [1usize, 1024, 100 * 1024, 1024 * 1024] {
for object_size in [1usize, 1024, 100 * 1024, 1024 * 1024, 1024 * 1024 + 1, 2 * 1024 * 1024 + 1] {
assert_eq!(
inline_erasure_shard_size(erasure.block_size, erasure.data_shards, uses_legacy),
erasure.shard_size()
);
assert_eq!(
inline_erasure_shard_file_offset(
0,
object_size,
object_size,
erasure.block_size,
erasure.data_shards,
uses_legacy,
),
erasure.shard_file_offset(0, object_size, object_size)
inline_erasure_shard_file_size(object_size, erasure.block_size, erasure.data_shards, uses_legacy),
usize::try_from(erasure.shard_file_size(object_size as i64)).expect("test shard size")
);
}
}
@@ -15789,25 +15976,14 @@ mod tests {
#[test]
fn inline_admission_falls_back_to_actual_size_for_transformed_streams() {
let config = crate::config::storageclass::Config::default();
let erasure = coding::Erasure::new(2, 2, 1024 * 1024);
let unknown = HashReader::SIZE_PRESERVE_LAYER;
// A known stored size is authoritative, whatever the plaintext size says.
assert_eq!(
inline_admission_shard_size(&erasure, 16 * 1024, 4 * 1024 * 1024),
erasure.shard_file_size(16 * 1024)
);
assert_eq!(inline_admission_shard_size(&erasure, 0, 4 * 1024), 0);
// A transformed stream is sized by its plaintext length (MinIO parity).
assert_eq!(
inline_admission_shard_size(&erasure, unknown, 16 * 1024),
erasure.shard_file_size(16 * 1024)
);
// Neither size known, or an empty transformed stream, cannot be admitted.
assert!(inline_admission_shard_size(&erasure, unknown, 0) < 0);
assert!(inline_admission_shard_size(&erasure, unknown, unknown) < 0);
assert!(should_inline_put(&config, &erasure, 16 * 1024, 4 * 1024 * 1024, false, false));
assert!(should_inline_put(&config, &erasure, 0, 4 * 1024, false, false));
assert!(should_inline_put(&config, &erasure, unknown, 16 * 1024, false, false));
assert!(!should_inline_put(&config, &erasure, unknown, 0, false, false));
assert!(!should_inline_put(&config, &erasure, unknown, unknown, false, false));
}
#[test]
@@ -133,7 +133,8 @@ pub(crate) struct DeleteRecoverySummary {
pub reclaimed: usize,
/// Observed removals for this exact key, not aggregated across prefixes.
pub removed_versions: HashSet<Uuid>,
/// Read-only evidence for ordinary missing-key S3 semantics. This never
/// Complete discovery on a strict majority found no transaction for this key.
/// Reusable only under the same key locks before a mutation. This never
/// authorizes settlement while a configured participant is unavailable.
pub no_transactions_at_quorum: bool,
}
@@ -862,16 +863,18 @@ impl SetDisks {
Err(error) => return Err(error),
}
}
let no_transactions_at_quorum = transactions.is_empty() && !unsupported && inspected > self.set_drive_count / 2;
if offline {
return Ok(DeleteRecoverySummary {
discovered: transactions.len(),
pending: 1,
no_transactions_at_quorum: transactions.is_empty() && !unsupported && inspected > self.set_drive_count / 2,
no_transactions_at_quorum,
..Default::default()
});
}
let mut summary = DeleteRecoverySummary {
discovered: transactions.len(),
no_transactions_at_quorum,
..Default::default()
};
let mut all_states = BTreeMap::new();
@@ -1400,6 +1403,7 @@ fn unavailable(error: &DiskError) -> bool {
match error {
DiskError::DiskNotFound
| DiskError::FaultyDisk
| DiskError::FaultyRemoteDisk
| DiskError::DiskOngoingReq
| DiskError::Timeout
| DiskError::RemoteClientUnavailable(_) => true,
@@ -1427,6 +1431,32 @@ mod tests {
use super::*;
use crate::bucket::metadata_sys::{self, init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
#[test]
fn recovery_availability_classifies_local_and_remote_disk_failures() {
for error in [
DiskError::DiskNotFound,
DiskError::FaultyDisk,
DiskError::FaultyRemoteDisk,
DiskError::Timeout,
DiskError::DiskOngoingReq,
] {
assert!(unavailable(&error), "drive availability failure must defer recovery: {error:?}");
}
for error in [
DiskError::FileCorrupt,
DiskError::DiskAccessDenied,
DiskError::FileAccessDenied,
DiskError::InvalidPath,
DiskError::DeleteRecoveryWorkLimit,
DiskError::Io(std::io::Error::from_raw_os_error(libc::EIO)),
] {
assert!(
!unavailable(&error),
"integrity, access and unclassified IO errors must propagate: {error:?}"
);
}
}
#[test]
fn unresolved_heal_error_distinguishes_legacy_ambiguity_from_transient_blocks() {
// rustfs/rustfs#8608: a legacy residue is deterministic and must not be
@@ -3334,6 +3364,64 @@ mod tests {
.expect("all proof is eventually collected");
}
#[tokio::test]
async fn delete_recovery_missing_delete_reuses_locked_discovery() {
use crate::bucket::metadata_sys::ObjectLockConfigState;
use crate::object_api::ObjectLockConfigSnapshot;
for offline in [false, true] {
let bucket = if offline {
"missing-probe-degraded"
} else {
"missing-probe-healthy"
};
let (_dirs, store, disks) = fixture(bucket).await;
let set = &store.pools[0].disk_set[0];
if offline {
disks[3].force_offline_for_test();
}
for object in ["missing", "missing/deep/key"] {
let online = &disks[..if offline { 3 } else { 4 }];
let before: Vec<_> = online
.iter()
.map(|disk| {
disk.metrics_snapshot()
.expect("local disk metrics")
.api_calls
.get("discover_delete_transactions")
.copied()
.unwrap_or(0)
})
.collect();
let opts = ObjectOptions {
object_lock_config_snapshot: Some(Arc::new(ObjectLockConfigSnapshot::new(
ObjectLockConfigState::ConfirmedAbsent,
))),
..Default::default()
};
let error = set
.delete_object(bucket, object, opts)
.await
.expect_err("set layer retains typed absence");
assert!(crate::error::is_err_object_not_found(&error), "typed absence: {error:?}");
for (disk, before) in online.iter().zip(before) {
let after = disk
.metrics_snapshot()
.expect("local disk metrics")
.api_calls
.get("discover_delete_transactions")
.copied()
.unwrap_or(0);
assert_eq!(
after - before,
1,
"one locked discovery per online drive, offline={offline}, key={object}"
);
}
}
}
}
#[tokio::test]
async fn delete_recovery_set_ordinary_missing_single_and_batch_keep_quorum_semantics() {
let bucket = "recover-missing-quorum";
+275 -45
View File
@@ -85,6 +85,7 @@ use crate::set_disk::metadata_sys;
use crate::set_disk::runtime_sources;
#[cfg(test)]
use crate::storage_api_contracts::multipart::MultipartOperations;
use crate::storage_api_contracts::multipart::{ListPartsOwner, MultipartOwner};
#[cfg(test)]
use crate::storage_api_contracts::object::HTTPPreconditions;
use crate::storage_api_contracts::object::ObjectOperations;
@@ -439,6 +440,8 @@ pub enum MultipartCommitPause {
NewUploadBeforeLockLost,
#[cfg(test)]
ListPartsAfterMetadata,
#[cfg(test)]
PartBindingAfterMetadata,
PutPartBeforeLockAcquire,
PutPartBeforeLockLost,
PutPartAfterCapacityAdmission,
@@ -683,6 +686,22 @@ fn fence_commit_on_lock_loss(guard: Option<&ObjectLockDiagGuard>, mode: &'static
Ok(())
}
fn recorded_multipart_owner(
metadata: &HashMap<String, String>,
bucket: &str,
encoded_object: &str,
) -> Option<crate::storage_api_contracts::multipart::MultipartOwner> {
let recorded_bucket = get_consistent_str(metadata, "multipart-bucket")?;
let recorded_object = get_consistent_str(metadata, "multipart-object")?;
if recorded_bucket != bucket || recorded_object != encoded_object {
return None;
}
Some(crate::storage_api_contracts::multipart::MultipartOwner {
bucket: recorded_bucket.to_owned(),
object: decode_dir_object(recorded_object),
})
}
fn multipart_bucket_incarnation_id(metadata: &HashMap<String, String>) -> Result<Option<Uuid>> {
let Some(value) = rustfs_utils::http::metadata_compat::get_consistent_str(metadata, SUFFIX_BUCKET_INCARNATION_ID) else {
if rustfs_utils::http::metadata_compat::contains_key_str(metadata, SUFFIX_BUCKET_INCARNATION_ID) {
@@ -1098,6 +1117,57 @@ struct MultipartUploadListScope<'a> {
}
impl SetDisks {
async fn read_multipart_info<const INCLUDE_OWNER: bool>(
&self,
bucket: &str,
object: &str,
upload_id: &str,
opts: &ObjectOptions,
) -> Result<(MultipartInfo, Option<MultipartOwner>)> {
let _upload_guard = self
.acquire_multipart_upload_read_lock("get_multipart_info", bucket, object, upload_id, opts)
.await?;
let (mut fi, _) = self
.check_upload_id_exists_with_opts(bucket, object, upload_id, false, opts)
.await
.map_err(|e| to_object_err(e, vec![bucket, object, upload_id]))?;
ensure_data_movement_upload_access(&fi, bucket, object, upload_id, opts)?;
ensure_multipart_bucket_incarnation(&self.ctx, &fi, bucket, object, upload_id, opts.expected_bucket_incarnation_id)
.await?;
ensure_multipart_bucket_lifecycle_lock_held(bucket, object, opts)?;
#[cfg(test)]
if INCLUDE_OWNER {
pause_multipart_commit(bucket, object, MultipartCommitPause::PartBindingAfterMetadata).await;
}
let owner = if INCLUDE_OWNER {
recorded_multipart_owner(&fi.metadata, bucket, object)
} else {
None
};
let info = MultipartInfo {
bucket: bucket.to_owned(),
object: decode_dir_object(object),
upload_id: upload_id.to_owned(),
user_defined: {
strip_internal_multipart_metadata(&mut fi.metadata);
fi.metadata.clone()
},
..Default::default()
};
Ok((info, owner))
}
pub(crate) async fn lookup_multipart_part_binding(
&self,
bucket: &str,
object: &str,
upload_id: &str,
opts: &ObjectOptions,
) -> Result<(MultipartInfo, Option<MultipartOwner>)> {
self.read_multipart_info::<true>(bucket, object, upload_id, opts).await
}
fn validated_multipart_upload_path(bucket: &str, object: &str, upload_id: &str, data_movement: bool) -> Result<String> {
let path = Self::get_upload_id_dir(bucket, object, upload_id);
if !path
@@ -3190,7 +3260,13 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
.cloned()
.unwrap_or_else(|| storageclass::STANDARD.to_string());
// Use the same metadata snapshot that produced the page.
let recorded_owner = recorded_multipart_owner(&fi.metadata, bucket, object).map(|owner| ListPartsOwner {
bucket: owner.bucket,
object: owner.object,
});
let mut ret = ListPartsInfo {
recorded_owner,
bucket: bucket.to_owned(),
object: decode_dir_object(object),
upload_id: upload_id.to_owned(),
@@ -3557,28 +3633,9 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
upload_id: &str,
opts: &ObjectOptions,
) -> Result<MultipartInfo> {
let _upload_guard = self
.acquire_multipart_upload_read_lock("get_multipart_info", bucket, object, upload_id, opts)
.await?;
let (mut fi, _) = self
.check_upload_id_exists_with_opts(bucket, object, upload_id, false, opts)
self.read_multipart_info::<false>(bucket, object, upload_id, opts)
.await
.map_err(|e| to_object_err(e, vec![bucket, object, upload_id]))?;
ensure_data_movement_upload_access(&fi, bucket, object, upload_id, opts)?;
ensure_multipart_bucket_incarnation(&self.ctx, &fi, bucket, object, upload_id, opts.expected_bucket_incarnation_id)
.await?;
ensure_multipart_bucket_lifecycle_lock_held(bucket, object, opts)?;
Ok(MultipartInfo {
bucket: bucket.to_owned(),
object: decode_dir_object(object),
upload_id: upload_id.to_owned(),
user_defined: {
strip_internal_multipart_metadata(&mut fi.metadata);
fi.metadata.clone()
},
..Default::default()
})
.map(|(info, _)| info)
}
#[tracing::instrument(skip(self))]
@@ -3597,6 +3654,10 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
ensure_multipart_bucket_incarnation(&self.ctx, &fi, bucket, object, upload_id, opts.expected_bucket_incarnation_id)
.await?;
ensure_multipart_bucket_lifecycle_lock_held(bucket, object, opts)?;
if let Some(authorization) = &opts.multipart_abort_authorization {
let owner = recorded_multipart_owner(&fi.metadata, bucket, object);
authorization.authorize(owner.as_ref())?;
}
let disks = self.get_disks_internal().await;
let cleanup = prepare_source_cleanup(
&disks,
@@ -3622,6 +3683,8 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
.await;
if result.is_ok() {
remove_capped_multipart_staging_semaphore(&upload_id_path);
#[cfg(any(test, feature = "test-util"))]
pause_multipart_commit(bucket, object, MultipartCommitPause::AbortAfterDelete).await;
}
result
}
@@ -6434,6 +6497,8 @@ mod tests {
assert!(empty.parts.is_empty());
assert!(!empty.is_truncated);
assert_eq!(empty.next_part_number_marker, None);
let owner = empty.recorded_owner.expect("the empty page has the recorded owner");
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, object));
for part_number in [10, 1, 3] {
put_test_part(&set_disks, bucket, object, &upload.upload_id, part_number, b"part", 4).await;
@@ -6460,9 +6525,184 @@ mod tests {
assert_eq!(page.parts.iter().map(|part| part.part_num).collect::<Vec<_>>(), expected_parts);
assert_eq!(page.is_truncated, next_marker.is_some());
assert_eq!(page.next_part_number_marker, next_marker);
assert_eq!(page.recorded_owner.as_ref(), Some(&owner));
}
}
#[tokio::test]
async fn list_parts_baseline_legacy_owner_metadata() {
let (_roots, disks, set) = hermetic_set_disks(4).await;
let bucket = "list-parts-owner-observation";
let object = "object";
let opts = ObjectOptions::default();
make_bucket_on_all(&disks, bucket).await;
let upload = set.new_multipart_upload(bucket, object, &opts).await.expect("real upload");
let path = SetDisks::get_upload_id_dir(bucket, object, &upload.upload_id);
let owner_keys = [
RUSTFS_MULTIPART_BUCKET_KEY,
"x-minio-internal-multipart-bucket",
RUSTFS_MULTIPART_OBJECT_KEY,
"x-minio-internal-multipart-object",
];
for (label, values) in [
("rustfs", [Some(bucket), None, Some(object), None]),
("minio", [None, Some(bucket), None, Some(object)]),
("both", [Some(bucket), Some(bucket), Some(object), Some(object)]),
("no bucket", [None, None, Some(object), None]),
("no object", [Some(bucket), None, None, None]),
("other bucket", [Some("other-bucket"), None, Some(object), None]),
("other object", [Some(bucket), None, Some("other-object"), None]),
("bucket aliases differ", [Some(bucket), Some("other-bucket"), Some(object), None]),
("object aliases differ", [Some(bucket), None, Some(object), Some("other-object")]),
] {
for disk in &disks {
let mut fi = disk
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
.await
.expect("read actual upload metadata");
for (key, value) in owner_keys.into_iter().zip(values) {
if let Some(value) = value {
fi.metadata.insert(key.to_owned(), value.to_owned());
} else {
fi.metadata.remove(key);
}
}
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, fi)
.await
.expect("write the synthetic owner metadata");
}
let page = set
.list_object_parts(bucket, object, &upload.upload_id, None, 1, &opts)
.await
.unwrap_or_else(|error| panic!("{label}: {error:?}"));
assert_eq!((page.bucket.as_str(), page.object.as_str()), (bucket, object), "{label}");
assert!(page.parts.is_empty(), "{label}");
let expected = matches!(label, "rustfs" | "minio" | "both").then_some((bucket, object));
assert_eq!(
page.recorded_owner
.as_ref()
.map(|owner| (owner.bucket.as_str(), owner.object.as_str())),
expected,
"{label}"
);
let info = set
.get_multipart_info(bucket, object, &upload.upload_id, &opts)
.await
.unwrap_or_else(|error| panic!("{label}: {error:?}"));
assert_eq!((info.bucket.as_str(), info.object.as_str()), (bucket, object), "{label}");
assert_eq!(info.upload_id, upload.upload_id, "{label}");
assert_eq!(info.user_defined, page.user_defined, "{label}");
let (bound_info, owner) = set
.lookup_multipart_part_binding(bucket, object, &upload.upload_id, &opts)
.await
.unwrap_or_else(|error| panic!("{label}: {error:?}"));
assert_eq!((bound_info.bucket.as_str(), bound_info.object.as_str()), (bucket, object), "{label}");
assert_eq!(bound_info.upload_id, upload.upload_id, "{label}");
assert_eq!(
owner.as_ref().map(|owner| (owner.bucket.as_str(), owner.object.as_str())),
page.recorded_owner
.as_ref()
.map(|owner| (owner.bucket.as_str(), owner.object.as_str())),
"{label}"
);
assert_eq!(bound_info.user_defined, page.user_defined, "{label}");
}
}
#[test]
fn multipart_owner_matches_encoded_namespace_before_decoding() {
let bucket = "owner-bucket";
let object = encode_dir_object("directory/");
let mut metadata = HashMap::new();
insert_str(&mut metadata, "multipart-bucket", bucket.to_owned());
insert_str(&mut metadata, "multipart-object", object.clone());
let owner = recorded_multipart_owner(&metadata, bucket, &object).expect("matching metadata");
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, "directory/"));
assert!(recorded_multipart_owner(&metadata, bucket, "directory/").is_none());
assert!(recorded_multipart_owner(&metadata, "other-bucket", &object).is_none());
metadata.insert("X-Minio-Internal-Multipart-Object".to_owned(), "foreign".to_owned());
assert!(recorded_multipart_owner(&metadata, bucket, &object).is_none());
metadata.remove("X-Minio-Internal-Multipart-Object");
insert_str(&mut metadata, "multipart-object", String::new());
assert!(recorded_multipart_owner(&metadata, bucket, &object).is_none());
}
#[tokio::test(flavor = "multi_thread")]
#[serial]
async fn multipart_part_binding_uses_one_snapshot_under_upload_read_lock() {
let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
let lockers: Vec<Arc<dyn LockClient>> = vec![Arc::new(LocalClient::with_manager(manager))];
let (_dirs, disks, set) = hermetic_set_disks_with_lockers(4, 0, 2, lockers).await;
let bucket = "part-binding-snapshot";
let object = "object";
make_bucket_on_all(&disks, bucket).await;
let upload = set
.new_multipart_upload(bucket, object, &ObjectOptions::default())
.await
.expect("create upload");
let path = SetDisks::get_upload_id_dir(bucket, object, &upload.upload_id);
for disk in &disks {
let mut fi = disk
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
.await
.expect("read upload metadata");
fi.metadata.insert("x-amz-meta-observation".to_owned(), "before".to_owned());
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, fi)
.await
.expect("write initial observation");
}
let _setup_type_guard = SetupTypeGuard::switch_to(SetupType::DistErasure).await;
let barrier = MultipartCommitBarrier::install(bucket, object, MultipartCommitPause::PartBindingAfterMetadata);
let lookup_set = Arc::clone(&set);
let upload_id = upload.upload_id.clone();
let lookup = tokio::spawn(async move {
lookup_set
.lookup_multipart_part_binding(bucket, object, &upload_id, &ObjectOptions::default())
.await
});
barrier.wait_until_paused().await;
let upload_lock = set
.new_ns_lock(RUSTFS_META_MULTIPART_BUCKET, &path)
.await
.expect("create upload lock");
let lock_error = upload_lock
.get_write_lock(Duration::from_millis(50))
.await
.expect_err("the binding must retain its upload read lock while observing metadata");
assert!(matches!(lock_error, rustfs_lock::LockError::Timeout { .. }));
// Bypass the upload lock to detect an accidental second metadata read.
for disk in &disks {
let mut fi = disk
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
.await
.expect("read upload metadata");
insert_str(&mut fi.metadata, "multipart-object", "foreign".to_owned());
fi.metadata.insert("x-amz-meta-observation".to_owned(), "after".to_owned());
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, fi)
.await
.expect("write changed observation");
}
barrier.release();
let (info, owner) = lookup.await.expect("lookup task").expect("initial binding");
assert_eq!(info.user_defined.get("x-amz-meta-observation").map(String::as_str), Some("before"));
let owner = owner.expect("initial owner");
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, object));
drop(barrier);
let (updated, owner) = set
.lookup_multipart_part_binding(bucket, object, &upload.upload_id, &ObjectOptions::default())
.await
.expect("updated binding");
assert_eq!(updated.user_defined.get("x-amz-meta-observation").map(String::as_str), Some("after"));
assert!(owner.is_none(), "a changed recorded namespace must not produce owner evidence");
upload_lock
.get_write_lock(Duration::from_secs(5))
.await
.expect("the upload read lock is released after the observation");
}
async fn object_transaction_epochs(disks: &[DiskStore], bucket: &str, object: &str) -> Vec<Option<Uuid>> {
let mut epochs = Vec::with_capacity(disks.len());
let deadline = tokio::time::Instant::now() + Duration::from_secs(30);
@@ -7514,12 +7754,13 @@ mod tests {
let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
let signaling = Arc::new(SignalingLockClient::new(Arc::new(LocalClient::with_manager(manager))));
let lockers: Vec<Arc<dyn LockClient>> = vec![signaling.clone()];
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers(4, 0, 2, lockers).await;
let instance_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
instance_ctx.update_erasure_type(SetupType::DistErasure).await;
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers_and_ctx(4, 0, 2, lockers, instance_ctx).await;
make_bucket_on_all(&disk_stores, bucket).await;
let (upload_id, parts) = stage_upload_with_create_opts(&set_disks, bucket, object, &[0x47; 4096], &create_opts).await;
let upload_id_path = SetDisks::get_upload_id_dir(bucket, object, &upload_id);
signaling.set_target(rustfs_lock::ObjectKey::new(RUSTFS_META_MULTIPART_BUCKET, upload_id_path));
let _setup_type_guard = SetupTypeGuard::switch_to(SetupType::DistErasure).await;
let barrier = MultipartCommitBarrier::install(bucket, object, MultipartCommitPause::AfterRename);
let complete_store = set_disks.clone();
@@ -7565,27 +7806,14 @@ mod tests {
let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
let signaling = Arc::new(SignalingLockClient::new(Arc::new(LocalClient::with_manager(manager))));
let lockers: Vec<Arc<dyn LockClient>> = vec![signaling.clone()];
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers(4, 0, 2, lockers).await;
let instance_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
instance_ctx.update_erasure_type(SetupType::DistErasure).await;
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers_and_ctx(4, 0, 2, lockers, instance_ctx).await;
make_bucket_on_all(&disk_stores, bucket).await;
let (upload_id, parts) = stage_upload_with_create_opts(&set_disks, bucket, object, &[0x48; 4096], &create_opts).await;
let upload_id_path = SetDisks::get_upload_id_dir(bucket, object, &upload_id);
signaling.set_target(rustfs_lock::ObjectKey::new(RUSTFS_META_MULTIPART_BUCKET, upload_id_path.clone()));
let _setup_type_guard = SetupTypeGuard::switch_to(SetupType::DistErasure).await;
let object_holder = set_disks
.new_ns_lock(bucket, object)
.await
.expect("object namespace lock should be created")
.get_write_lock(Duration::from_secs(5))
.await
.expect("test should hold the object lock");
let holder = set_disks
.new_ns_lock(RUSTFS_META_MULTIPART_BUCKET, &upload_id_path)
.await
.expect("upload namespace lock should be created")
.get_write_lock(Duration::from_secs(5))
.await
.expect("test should hold the upload lock");
signaling.wait_for_attempts(1).await;
let barrier = MultipartCommitBarrier::install(bucket, object, MultipartCommitPause::AbortAfterDelete);
let abort_store = set_disks.clone();
let abort_upload_id = upload_id.clone();
@@ -7594,7 +7822,9 @@ mod tests {
.abort_multipart_upload(bucket, object, &abort_upload_id, &ObjectOptions::default())
.await
});
signaling.wait_for_attempts(2).await;
// Abort finishes disk cleanup before completion starts its lock deadline,
// and retains upload ownership until the competing request is waiting.
barrier.wait_until_paused().await;
let complete_store = set_disks.clone();
let complete_upload_id = upload_id.clone();
@@ -7603,21 +7833,21 @@ mod tests {
.complete_multipart_upload(bucket, object, &complete_upload_id, parts, &ObjectOptions::default())
.await
});
drop(holder);
signaling.wait_for_attempts(2).await;
assert!(!complete.is_finished(), "completion must wait for the abort upload lock");
barrier.release();
abort
.await
.expect("abort task should not panic")
.expect("abort should win the upload finalization");
drop(object_holder);
let complete_err = complete
.await
.expect("completion task should not panic")
.expect_err("completion must observe the aborted upload");
assert!(
matches!(complete_err, StorageError::InvalidUploadID(..)),
"abort-first refusal case={bucket} code={}",
complete_err.code().as_u32()
"abort-first refusal case={bucket}: {complete_err:?}"
);
let object_err = set_disks
.get_object_info(bucket, object, &ObjectOptions::default())
@@ -1029,3 +1029,229 @@ mod native_abort_closure_tests {
include!("multipart_initial_probe_tests.rs");
}
}
mod abort_owner_tests {
use super::*;
use crate::storage_api_contracts::multipart::{MultipartAbortAuthorization, MultipartOwner};
#[test]
fn recorded_multipart_owner_requires_unambiguous_metadata_for_the_selected_namespace() {
const BUCKET: &str = "abort-owner";
const ENCODED: &str = "directory__XLDIR__";
const RUSTFS: &str = RUSTFS_MULTIPART_BUCKET_KEY;
const MINIO: &str = "x-minio-internal-multipart-bucket";
const RUSTFS_OBJECT: &str = RUSTFS_MULTIPART_OBJECT_KEY;
const MINIO_OBJECT: &str = "x-minio-internal-multipart-object";
let expected = MultipartOwner {
bucket: BUCKET.to_owned(),
object: "directory/".to_owned(),
};
let rustfs = [(RUSTFS, BUCKET), (RUSTFS_OBJECT, ENCODED)];
let minio = [(MINIO, BUCKET), (MINIO_OBJECT, ENCODED)];
for fields in [rustfs.to_vec(), minio.to_vec(), [rustfs, minio].concat()] {
let metadata = fields
.into_iter()
.map(|(key, value)| (key.to_owned(), value.to_owned()))
.collect();
assert_eq!(recorded_multipart_owner(&metadata, BUCKET, ENCODED), Some(expected.clone()));
}
for fields in [
vec![(RUSTFS, BUCKET)],
vec![(RUSTFS_OBJECT, ENCODED)],
vec![(RUSTFS, ""), (RUSTFS_OBJECT, ENCODED)],
vec![(RUSTFS, BUCKET), (RUSTFS_OBJECT, "")],
vec![(RUSTFS, "foreign"), (RUSTFS_OBJECT, ENCODED)],
vec![(RUSTFS, BUCKET), (RUSTFS_OBJECT, "directory/")],
[rustfs, [(MINIO, "foreign"), (MINIO_OBJECT, ENCODED)]].concat(),
[rustfs, [(MINIO, BUCKET), ("X-MINIO-INTERNAL-MULTIPART-OBJECT", "other")]].concat(),
] {
let metadata = fields
.iter()
.map(|(key, value)| (key.to_string(), value.to_string()))
.collect();
assert_eq!(recorded_multipart_owner(&metadata, BUCKET, ENCODED), None, "{fields:?}");
}
}
struct AbortGate {
expected: Option<MultipartOwner>,
accepted: bool,
calls: AtomicUsize,
}
impl MultipartAbortAuthorization for AbortGate {
type Error = StorageError;
fn authorize(&self, owner: Option<&MultipartOwner>) -> Result<()> {
self.calls.fetch_add(1, Ordering::Relaxed);
assert_eq!(owner, self.expected.as_ref(), "gate sees the actual selected upload owner");
if self.accepted {
Ok(())
} else {
Err(StorageError::InvalidUploadID(String::new(), String::new(), String::new()))
}
}
}
#[tokio::test]
async fn abort_owner_callback_controls_physical_deletion() {
let (_dirs, disks, set) = hermetic_set_disks(4).await;
let bucket = "abort-locked-owner";
let key = "staged-object";
make_bucket_on_all(&disks, bucket).await;
let (id, _) = stage_upload_with_create_opts(&set, bucket, key, b"staged part", &ObjectOptions::default()).await;
let path = SetDisks::get_upload_id_dir(bucket, key, &id);
let (fi, _) = set
.check_upload_id_exists(bucket, key, &id, true)
.await
.expect("staged upload");
let part = format!("{path}/{}/part.1", fi.data_dir.expect("staged data directory"));
let expected = MultipartOwner {
bucket: bucket.to_owned(),
object: key.to_owned(),
};
let mut before = Vec::new();
for disk in &disks {
before.push(
disk.read_all(RUSTFS_META_MULTIPART_BUCKET, &part)
.await
.expect("staged shard"),
);
}
let rejected = Arc::new(AbortGate {
expected: Some(expected.clone()),
accepted: false,
calls: AtomicUsize::new(0),
});
let opts = ObjectOptions {
multipart_abort_authorization: Some(rejected.clone()),
..Default::default()
};
let result = set.abort_multipart_upload(bucket, key, &id, &opts).await;
for (disk, staged) in disks.iter().zip(before) {
let after = disk
.read_all(RUSTFS_META_MULTIPART_BUCKET, &part)
.await
.expect("denied Abort preserves the shard");
assert_eq!(staged, after);
}
assert!(matches!(result, Err(StorageError::InvalidUploadID(..))));
assert_eq!(rejected.calls.load(Ordering::Relaxed), 1);
let allowed = Arc::new(AbortGate {
expected: Some(expected),
accepted: true,
calls: AtomicUsize::new(0),
});
let opts = ObjectOptions {
multipart_abort_authorization: Some(allowed.clone()),
..Default::default()
};
set.abort_multipart_upload(bucket, key, &id, &opts)
.await
.expect("owner permits Abort");
assert_eq!(allowed.calls.load(Ordering::Relaxed), 1);
for disk in &disks {
assert!(matches!(
disk.read_all(RUSTFS_META_MULTIPART_BUCKET, &part).await,
Err(DiskError::FileNotFound)
));
}
}
#[tokio::test]
async fn abort_owner_comes_from_the_selected_metadata_on_all_disks() {
use rustfs_utils::http::{get_str, remove_str};
let (_dirs, disks, set) = hermetic_set_disks(4).await;
let bucket = "abort-owner-metadata";
make_bucket_on_all(&disks, bucket).await;
for (key, bucket_fields, object_fields, accepted) in [
("rustfs", [Some(bucket), None], [Some("rustfs"), None], true),
("minio", [None, Some(bucket)], [None, Some("minio")], true),
("matching", [Some(bucket), Some(bucket)], [Some("matching"), Some("matching")], true),
("empty", [Some(bucket), Some(bucket)], [Some("empty"), Some("")], false),
("missing", [None, None], [Some("missing"), None], false),
("foreign-bucket", [Some("foreign"), None], [Some("foreign-bucket"), None], false),
("foreign-key", [Some(bucket), None], [Some("unrelated"), None], false),
("conflict", [Some(bucket), Some("foreign")], [Some("conflict"), Some("conflict")], false),
] {
let (id, _) = stage_upload_with_create_opts(&set, bucket, key, b"staged shard data", &ObjectOptions::default()).await;
let path = SetDisks::get_upload_id_dir(bucket, key, &id);
pin_historical_unmarked_upload(&disks, bucket, &path).await;
let mut part_path = None;
let mut staged = Vec::new();
let fields = [
(RUSTFS_MULTIPART_BUCKET_KEY, bucket_fields[0]),
("x-minio-internal-multipart-bucket", bucket_fields[1]),
(RUSTFS_MULTIPART_OBJECT_KEY, object_fields[0]),
("x-minio-internal-multipart-object", object_fields[1]),
];
for disk in &disks {
let mut head = disk
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
.await
.expect("read the staged metadata on each disk");
let part = part_path.get_or_insert_with(|| format!("{path}/{}/part.1", head.data_dir.expect("part directory")));
staged.push(
disk.read_all(RUSTFS_META_MULTIPART_BUCKET, part)
.await
.expect("read staged part before Abort"),
);
assert_eq!(get_str(&head.metadata, "multipart-bucket").as_deref(), Some(bucket));
remove_str(&mut head.metadata, "multipart-bucket");
remove_str(&mut head.metadata, "multipart-object");
for (name, value) in fields {
if let Some(value) = value {
head.metadata.insert(name.to_owned(), value.to_owned());
}
}
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, head)
.await
.expect("write the test owner on each disk");
let persisted = disk
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
.await
.expect("verify the actual persisted metadata");
for (name, value) in fields {
assert_eq!(persisted.metadata.get(name).map(String::as_str), value);
}
}
let gate = Arc::new(AbortGate {
expected: accepted.then(|| MultipartOwner {
bucket: bucket.to_owned(),
object: key.to_owned(),
}),
accepted,
calls: AtomicUsize::new(0),
});
let opts = ObjectOptions {
multipart_abort_authorization: Some(gate.clone()),
..Default::default()
};
let result = set.abort_multipart_upload(bucket, key, &id, &opts).await;
for (disk, previous) in disks.iter().zip(staged) {
let after = disk
.read_all(RUSTFS_META_MULTIPART_BUCKET, part_path.as_deref().expect("part path"))
.await;
if accepted {
assert!(matches!(after, Err(DiskError::FileNotFound)), "{key}: authorized data removed");
} else {
assert_eq!(after.expect("refused Abort preserves the staged part"), previous, "{key}");
}
}
let calls = gate.calls.load(Ordering::Relaxed);
if accepted {
assert_eq!(calls, 1, "{key}");
result.expect("Abort with agreeing persisted owner");
} else {
assert!(result.is_err(), "{key}: invalid metadata must not authorize Abort");
if key == "empty" {
// The existing source inspection refuses this head before selecting an owner.
assert_eq!(calls, 0, "malformed physical owner was refused before the callback");
}
}
}
}
}
File diff suppressed because it is too large Load Diff
+319 -67
View File
@@ -17,17 +17,18 @@ use super::{
GetObjectFileInfo, GetObjectMetadataCacheCapability, GetObjectMetadataCacheEntry, GetObjectMetadataCacheGeneration,
GetObjectMetadataCacheKey, GetObjectMetadataCacheLookupKey, GetObjectReadPolicy, HashAlgorithm, LOG_COMPONENT_ECSTORE,
LOG_SUBSYSTEM_SET_DISK, OBJECT_OP_IGNORED_ERRS, ObjectInfo, ObjectOptions, RUSTFS_META_BUCKET, ReadOptions, Result, SetDisks,
StorageError, adaptive_duplex_buffer_size, build_get_codec_streaming_decode_engine, build_inline_bitrot_readers_from_refs,
StorageError, adaptive_duplex_buffer_size, build_get_codec_streaming_decode_engine, build_inline_bitrot_readers,
collect_inline_data_shard_fileinfos_by_index, debug, error, get_codec_streaming_metrics_path,
get_codec_streaming_multipart_max_parts, get_object_read_policy, is_codec_streaming_multipart_enabled,
is_multipart_reader_setup_prefetch_enabled, object_fits_single_block, reduce_read_quorum_errs, to_object_err,
try_read_inline_data_shards_direct, warn,
is_multipart_reader_setup_prefetch_enabled, object_fits_single_block, part_checksum_algorithm, reduce_read_quorum_errs,
to_object_err, try_read_inline_data_shards_direct, warn,
};
use crate::diagnostics::get::{
GET_DIRECT_MEMORY_SUBPATH_DISK_DATA_BLOCKS, GET_DIRECT_MEMORY_SUBPATH_INLINE_BUFFERED, GET_METADATA_CACHE_DECISION_HIT,
GET_METADATA_CACHE_DECISION_MISS, GET_METADATA_CACHE_DECISION_REJECT, GET_METADATA_CACHE_DECISION_SKIP,
GET_METADATA_CACHE_REASON_BODY_LAYOUT_REQUIRED, GET_METADATA_CACHE_REASON_DATA_MOVEMENT,
GET_METADATA_CACHE_REASON_DELETE_MARKER, GET_METADATA_CACHE_REASON_DIST_ERASURE,
GET_METADATA_CACHE_REASON_DELETE_MARKER, GET_METADATA_CACHE_REASON_DIST_ERASURE, GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST,
GET_METADATA_CACHE_REASON_FENCE_PROOF_MISMATCH, GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE,
GET_METADATA_CACHE_REASON_INCL_FREE_VERSIONS, GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM,
GET_METADATA_CACHE_REASON_META_BUCKET, GET_METADATA_CACHE_REASON_NO_LOCK, GET_METADATA_CACHE_REASON_NOT_FOUND_OR_EXPIRED,
GET_METADATA_CACHE_REASON_NOT_READ_DATA, GET_METADATA_CACHE_REASON_PART_CHECKSUMS, GET_METADATA_CACHE_REASON_PART_NUMBER,
@@ -57,11 +58,39 @@ use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
pub(super) const GET_OBJECT_PATH_MID_SIZE_STREAMING: &str = "mid_size_streaming";
#[derive(Clone)]
struct MetadataCacheAuthority {
proof: Arc<rustfs_lock::CacheFenceProof>,
fence: crate::object_api::NamespaceLockFence,
}
impl MetadataCacheAuthority {
fn from_options(opts: &ObjectOptions) -> Option<Self> {
Some(Self {
proof: opts.metadata_cache_fence_proof.clone()?,
fence: opts.namespace_lock_fence.clone()?,
})
}
fn is_current(&self) -> bool {
!self.fence.is_lock_lost()
}
}
struct MetadataCachePublication {
fi: FileInfo,
capability: GetObjectMetadataCacheCapability,
verified_inline_body: Option<Bytes>,
authority: Option<MetadataCacheAuthority>,
}
#[cfg(test)]
use super::DEFAULT_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES;
#[cfg(test)]
use super::DEFAULT_RUSTFS_GET_CODEC_STREAMING_MIN_SIZE;
#[cfg(test)]
use super::ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE;
#[cfg(test)]
use super::ENV_RUSTFS_GET_CODEC_STREAMING_BODY_COMPAT_CONFIRMED;
#[cfg(test)]
use super::ENV_RUSTFS_GET_CODEC_STREAMING_DATA_BLOCKS_FIRST_ENABLE;
@@ -454,28 +483,39 @@ impl SetDisks {
if let Some(reason) = get_object_metadata_cache_request_bypass_reason(bucket, opts, read_data) {
return Some(reason);
}
self.ctx
.is_dist_erasure()
.await
.then_some(GET_METADATA_CACHE_REASON_DIST_ERASURE)
if self.ctx.is_dist_erasure().await {
if !super::is_get_object_metadata_cache_distributed_proof_enabled() {
return Some(GET_METADATA_CACHE_REASON_DIST_ERASURE);
}
return match MetadataCacheAuthority::from_options(opts) {
None => Some(GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE),
Some(authority) if !authority.is_current() => Some(GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST),
Some(_) => None,
};
}
None
}
#[allow(dead_code, reason = "asserted by this file's tests (backlog#1823)")]
async fn cached_get_object_fileinfo(&self, bucket: &str, object: &str) -> Option<Arc<GetObjectMetadataCacheEntry>> {
match self.lookup_cached_get_object_fileinfo(bucket, object).await {
MetadataCacheLookup::Hit(entry) => Some(entry),
MetadataCacheLookup::Miss | MetadataCacheLookup::RejectedInsufficientQuorum => None,
MetadataCacheLookup::Miss
| MetadataCacheLookup::RejectedInsufficientQuorum
| MetadataCacheLookup::RejectedFenceProof => None,
}
}
async fn lookup_cached_get_object_fileinfo(&self, bucket: &str, object: &str) -> MetadataCacheLookup {
self.lookup_cached_get_object_fileinfo_after_get(bucket, object, || {}).await
self.lookup_cached_get_object_fileinfo_with_authority(bucket, object, None, || {})
.await
}
async fn lookup_cached_get_object_fileinfo_after_get(
async fn lookup_cached_get_object_fileinfo_with_authority(
&self,
bucket: &str,
object: &str,
authority: Option<&MetadataCacheAuthority>,
after_get: impl FnOnce(),
) -> MetadataCacheLookup {
let Some(generation) = self.get_object_metadata_cache_generation(bucket, object) else {
@@ -491,6 +531,12 @@ impl SetDisks {
self.get_object_metadata_cache.invalidate(&key).await;
return MetadataCacheLookup::Miss;
}
if authority.is_some_and(|authority| !authority.is_current())
|| entry.distributed_fence_proof.as_deref() != authority.map(|authority| authority.proof.as_ref())
{
self.get_object_metadata_cache.invalidate(&key).await;
return MetadataCacheLookup::RejectedFenceProof;
}
if entry.online_disks.iter().filter(|disk| disk.is_some()).count() >= entry.read_quorum {
MetadataCacheLookup::Hit(entry)
} else {
@@ -498,6 +544,17 @@ impl SetDisks {
}
}
#[cfg(test)]
async fn lookup_cached_get_object_fileinfo_after_get(
&self,
bucket: &str,
object: &str,
after_get: impl FnOnce(),
) -> MetadataCacheLookup {
self.lookup_cached_get_object_fileinfo_with_authority(bucket, object, None, after_get)
.await
}
#[cfg(test)]
async fn cache_get_object_fileinfo(
&self,
@@ -513,7 +570,12 @@ impl SetDisks {
.cache_get_object_fileinfo_owned(
identity,
generation,
(fi.clone(), capability, None),
MetadataCachePublication {
fi: fi.clone(),
capability,
verified_inline_body: None,
authority: None,
},
parts_metadata.to_vec(),
online_disks.to_vec(),
read_quorum,
@@ -525,12 +587,17 @@ impl SetDisks {
&self,
identity: (&str, &str),
generation: Option<GetObjectMetadataCacheGeneration>,
selected: (FileInfo, GetObjectMetadataCacheCapability, Option<Bytes>),
publication: MetadataCachePublication,
parts_metadata: Vec<FileInfo>,
online_disks: Vec<Option<DiskStore>>,
read_quorum: usize,
) -> GetObjectFileInfo {
let (fi, capability, verified_inline_body) = selected;
let MetadataCachePublication {
fi,
capability,
verified_inline_body,
authority,
} = publication;
if fi.deleted || !fi.has_valid_erasure_geometry() {
let mut snapshot = GetObjectFileInfo::owned(fi, parts_metadata, online_disks);
snapshot.set_verified_inline_body(verified_inline_body);
@@ -559,18 +626,38 @@ impl SetDisks {
read_quorum,
capability,
verified_inline_body,
distributed_fence_proof: authority.as_ref().map(|authority| Arc::clone(&authority.proof)),
});
self.insert_get_object_metadata_cache_entry_after_insert(key, generation, Arc::clone(&entry), || {})
.await;
self.insert_get_object_metadata_cache_entry_with_authority_after_insert(
key,
generation,
Arc::clone(&entry),
authority,
|| {},
)
.await;
GetObjectFileInfo::owned_published(entry)
}
#[cfg(test)]
async fn insert_get_object_metadata_cache_entry_after_insert(
&self,
key: GetObjectMetadataCacheKey,
generation: GetObjectMetadataCacheGeneration,
entry: Arc<GetObjectMetadataCacheEntry>,
after_insert: impl FnOnce(),
) {
self.insert_get_object_metadata_cache_entry_with_authority_after_insert(key, generation, entry, None, after_insert)
.await;
}
async fn insert_get_object_metadata_cache_entry_with_authority_after_insert(
&self,
key: GetObjectMetadataCacheKey,
generation: GetObjectMetadataCacheGeneration,
entry: Arc<GetObjectMetadataCacheEntry>,
authority: Option<MetadataCacheAuthority>,
after_insert: impl FnOnce(),
) {
// Every publisher shares Moka's per-key compute lock. A partial snapshot
// cannot downgrade a full layout or extend its original TTL.
@@ -579,6 +666,7 @@ impl SetDisks {
.and_compute_with(|current| {
std::future::ready(
if !self.is_get_object_metadata_cache_generation_current(generation)
|| authority.as_ref().is_some_and(|authority| !authority.is_current())
|| (entry.capability != GetObjectMetadataCacheCapability::Full
&& current
.as_ref()
@@ -592,7 +680,9 @@ impl SetDisks {
})
.await;
after_insert();
if self.is_get_object_metadata_cache_generation_current(generation) {
if self.is_get_object_metadata_cache_generation_current(generation)
&& !authority.as_ref().is_some_and(|authority| !authority.is_current())
{
return;
}
@@ -840,6 +930,9 @@ impl SetDisks {
let metadata_cache_lookup_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let cache_bypass_reason = self.get_object_metadata_cache_bypass_reason(bucket, opts, read_data).await;
let use_metadata_cache = cache_bypass_reason.is_none();
let metadata_cache_authority = use_metadata_cache
.then(|| MetadataCacheAuthority::from_options(opts))
.flatten();
if let Some(reason) = cache_bypass_reason {
rustfs_io_metrics::record_get_object_metadata_cache_decision(
GET_OBJECT_PATH_SET_DISK,
@@ -847,7 +940,10 @@ impl SetDisks {
reason,
);
} else if vid.is_empty() {
match self.lookup_cached_get_object_fileinfo(bucket, object).await {
match self
.lookup_cached_get_object_fileinfo_with_authority(bucket, object, metadata_cache_authority.as_ref(), || {})
.await
{
MetadataCacheLookup::Hit(cached)
if cached.capability == GetObjectMetadataCacheCapability::Full
|| (cached.capability == GetObjectMetadataCacheCapability::QuorumSelected
@@ -904,6 +1000,13 @@ impl SetDisks {
GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM,
);
}
MetadataCacheLookup::RejectedFenceProof => {
rustfs_io_metrics::record_get_object_metadata_cache_decision(
GET_OBJECT_PATH_SET_DISK,
GET_METADATA_CACHE_DECISION_REJECT,
GET_METADATA_CACHE_REASON_FENCE_PROOF_MISMATCH,
);
}
}
}
record_get_stage_duration_if_enabled(
@@ -1104,9 +1207,9 @@ impl SetDisks {
self.cache_get_object_fileinfo_owned(
(bucket, object),
metadata_cache_generation,
(
MetadataCachePublication {
fi,
if metadata_fanout_complete {
capability: if metadata_fanout_complete {
GetObjectMetadataCacheCapability::Full
} else if condition_stopped {
GetObjectMetadataCacheCapability::ConditionOnly
@@ -1114,7 +1217,8 @@ impl SetDisks {
GetObjectMetadataCacheCapability::QuorumSelected
},
verified_inline_body,
),
authority: metadata_cache_authority,
},
parts_metadata,
op_online_disks,
read_quorum,
@@ -1372,12 +1476,7 @@ impl SetDisks {
let erasure = erasure_cache.get_for_file_info(fi)?;
let checksum_info = fi.erasure.get_checksum_info(part.number);
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
HashAlgorithm::HighwayHash256SLegacy
} else {
checksum_info.algorithm
};
let checksum_algo = part_checksum_algorithm(fi, part.number);
let read_length = erasure.shard_file_offset(0, object_size, object_size);
if fi.data.is_some() {
@@ -1392,8 +1491,9 @@ impl SetDisks {
};
let reader_setup_stage_start = Instant::now();
let mut readers = build_inline_bitrot_readers_from_refs(
&data_files,
let mut readers = build_inline_bitrot_readers(
data_files.iter().copied(),
data_files.len(),
bucket,
object,
read_length,
@@ -1665,7 +1765,7 @@ impl SetDisks {
"Streaming multipart part"
);
let checksum_algo = multipart_part_checksum_algo(&fi, part_number);
let checksum_algo = part_checksum_algorithm(&fi, part_number);
let read_length = till_offset.saturating_sub(read_offset);
let read_costs = coding::decode::should_collect_shard_read_costs().then(|| shard_read_costs_for_disks(&disks));
@@ -1729,7 +1829,7 @@ impl SetDisks {
part_number: next_number,
read_offset: 0,
read_length: erasure.shard_file_offset(0, next_length, next_size),
checksum_algo: multipart_part_checksum_algo(&fi, next_number),
checksum_algo: part_checksum_algorithm(&fi, next_number),
};
let files = Arc::clone(&files);
let disks = Arc::clone(&disks);
@@ -2326,12 +2426,7 @@ impl SetDisks {
if part_length > part_size {
return Err(Error::other("codec streaming reader part length exceeds part size"));
}
let checksum_info = fi.erasure.get_checksum_info(part_number);
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
HashAlgorithm::HighwayHash256SLegacy
} else {
checksum_info.algorithm
};
let checksum_algo = part_checksum_algorithm(fi, part_number);
let use_mmap_read = object_mmap_read_enabled();
let till_offset = erasure.shard_file_offset(part_offset, part_length, part_size);
let read_offset = (part_offset / erasure.block_size) * erasure.shard_size();
@@ -2620,17 +2715,6 @@ struct PartReaderSetupSpec {
checksum_algo: HashAlgorithm,
}
/// Resolve the bitrot checksum algorithm for one part, honoring the legacy
/// HighwayHash flag.
fn multipart_part_checksum_algo(fi: &FileInfo, part_number: usize) -> HashAlgorithm {
let checksum_info = fi.erasure.get_checksum_info(part_number);
if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
HashAlgorithm::HighwayHash256SLegacy
} else {
checksum_info.algorithm
}
}
fn multipart_reader_setup_prefetch_enabled(policy: GetObjectReadPolicy) -> bool {
policy.allows_multipart_setup_prefetch() && is_multipart_reader_setup_prefetch_enabled()
}
@@ -3140,6 +3224,7 @@ mod metadata_cache_tests {
use super::*;
use crate::set_disk::core::io_primitives::disk_call_counters;
use rustfs_heal_contracts::heal_channel::HealAdmissionDropReason;
use rustfs_lock::{GlobalLockManager, LocalClient, LockClient, NamespaceLock, ObjectKey};
use rustfs_utils::path::object_name_disk_identity;
use serial_test::serial;
use std::sync::atomic::{AtomicUsize, Ordering};
@@ -4427,6 +4512,7 @@ mod metadata_cache_tests {
online_disks: vec![None],
read_quorum: 1,
verified_inline_body: None,
distributed_fence_proof: None,
}),
)
.await;
@@ -5089,6 +5175,7 @@ mod metadata_cache_tests {
read_quorum: 0,
capability,
verified_inline_body: None,
distributed_fence_proof: None,
})
}
@@ -5582,6 +5669,7 @@ mod metadata_cache_tests {
online_disks: vec![None],
read_quorum: 1,
verified_inline_body: None,
distributed_fence_proof: None,
}),
)
.await;
@@ -5701,6 +5789,7 @@ mod metadata_cache_tests {
online_disks: Vec::new(),
read_quorum: 0,
verified_inline_body: None,
distributed_fence_proof: None,
});
set.insert_get_object_metadata_cache_entry_after_insert(key, generation, entry, || {
@@ -5749,30 +5838,192 @@ mod metadata_cache_tests {
}
#[tokio::test]
#[serial(metadata_cache_distributed_proof_env)]
async fn metadata_cache_distribution_bypass_uses_set_instance_context() {
let distributed_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
distributed_ctx
.update_erasure_type(crate::layout::endpoints::SetupType::DistErasure)
.await;
let standalone_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
standalone_ctx
.update_erasure_type(crate::layout::endpoints::SetupType::Erasure)
.await;
let distributed = new_metadata_cache_test_set_with_ctx(distributed_ctx).await;
let standalone = new_metadata_cache_test_set_with_ctx(standalone_ctx).await;
temp_env::async_with_vars(
[(ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE, None::<&str>)],
async {
let distributed_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
distributed_ctx
.update_erasure_type(crate::layout::endpoints::SetupType::DistErasure)
.await;
let standalone_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
standalone_ctx
.update_erasure_type(crate::layout::endpoints::SetupType::Erasure)
.await;
let distributed = new_metadata_cache_test_set_with_ctx(distributed_ctx).await;
let standalone = new_metadata_cache_test_set_with_ctx(standalone_ctx).await;
assert_eq!(
distributed
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
assert_eq!(
distributed
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
.await,
Some(GET_METADATA_CACHE_REASON_DIST_ERASURE)
);
assert_eq!(
standalone
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
.await,
None
);
},
)
.await;
}
fn cache_fence_test_lock() -> NamespaceLock {
let clients = (0..3)
.map(|_| Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))) as Arc<dyn LockClient>)
.collect();
NamespaceLock::with_clients("metadata-cache-proof".to_owned(), clients)
}
#[tokio::test]
#[serial(metadata_cache_distributed_proof_env)]
async fn distributed_metadata_cache_requires_live_quorum_proof() {
temp_env::async_with_vars(
[(ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE, Some("true"))],
async {
let ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
ctx.update_erasure_type(crate::layout::endpoints::SetupType::DistErasure)
.await;
let set = new_metadata_cache_test_set_with_ctx(ctx).await;
assert_eq!(
set.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
.await,
Some(GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE)
);
let lock = cache_fence_test_lock();
let resource = ObjectKey::new("bucket", "object");
let guard = lock
.get_read_lock_with_cache_fence(resource, "reader", Duration::from_secs(1))
.await
.expect("proof-carrying read lock");
let mut opts = ObjectOptions {
no_lock: true,
metadata_cache_safe: true,
..Default::default()
};
opts.add_metadata_cache_read_guard(&guard);
assert!(opts.metadata_cache_fence_proof.is_some());
assert_eq!(set.get_object_metadata_cache_bypass_reason("bucket", &opts, true).await, None);
opts.namespace_lock_fence = Some(crate::object_api::NamespaceLockFence::lost_for_test());
assert_eq!(
set.get_object_metadata_cache_bypass_reason("bucket", &opts, true).await,
Some(GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST)
);
},
)
.await;
}
#[tokio::test]
async fn distributed_metadata_cache_rejects_proof_after_write_quorum() {
let set = new_metadata_cache_test_set().await;
let lock = cache_fence_test_lock();
let resource = ObjectKey::new("bucket", "object");
let mut first_guard = lock
.get_read_lock_with_cache_fence(resource.clone(), "reader-a", Duration::from_secs(1))
.await
.expect("first proof-carrying read lock");
let mut first_opts = ObjectOptions {
no_lock: true,
metadata_cache_safe: true,
..Default::default()
};
first_opts.add_metadata_cache_read_guard(&first_guard);
let authority = MetadataCacheAuthority::from_options(&first_opts).expect("first read authority");
let fi = valid_test_fileinfo("object");
let generation = set.get_object_metadata_cache_generation("bucket", "object");
set.cache_get_object_fileinfo_owned(
("bucket", "object"),
generation,
MetadataCachePublication {
fi: fi.clone(),
capability: GetObjectMetadataCacheCapability::Full,
verified_inline_body: None,
authority: Some(authority.clone()),
},
vec![fi],
Vec::new(),
0,
)
.await;
assert!(matches!(
set.lookup_cached_get_object_fileinfo_with_authority("bucket", "object", Some(&authority), || {})
.await,
Some(GET_METADATA_CACHE_REASON_DIST_ERASURE)
);
assert_eq!(
standalone
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
MetadataCacheLookup::Hit(_)
));
assert!(first_guard.release());
let mut writer = lock
.get_write_lock(resource.clone(), "writer", Duration::from_secs(1))
.await
.expect("intervening write quorum");
assert!(writer.release());
let mut second_guard = lock
.get_read_lock_with_cache_fence(resource, "reader-b", Duration::from_secs(1))
.await
.expect("second proof-carrying read lock");
let mut second_opts = ObjectOptions {
no_lock: true,
metadata_cache_safe: true,
..Default::default()
};
second_opts.add_metadata_cache_read_guard(&second_guard);
let second_authority = MetadataCacheAuthority::from_options(&second_opts).expect("second read authority");
assert!(matches!(
set.lookup_cached_get_object_fileinfo_with_authority("bucket", "object", Some(&second_authority), || {})
.await,
None
MetadataCacheLookup::RejectedFenceProof
));
assert!(second_guard.release());
}
#[tokio::test]
async fn distributed_metadata_cache_does_not_publish_after_read_lock_loss() {
let set = new_metadata_cache_test_set().await;
let lock = cache_fence_test_lock();
let resource = ObjectKey::new("bucket", "lost-reader");
let mut guard = lock
.get_read_lock_with_cache_fence(resource, "reader", Duration::from_secs(1))
.await
.expect("proof-carrying read lock");
let mut opts = ObjectOptions {
no_lock: true,
metadata_cache_safe: true,
..Default::default()
};
opts.add_metadata_cache_read_guard(&guard);
opts.namespace_lock_fence = Some(crate::object_api::NamespaceLockFence::lost_for_test());
let authority = MetadataCacheAuthority::from_options(&opts).expect("lost authority retains its proof for rejection");
let fi = valid_test_fileinfo("lost-reader");
let generation = set
.get_object_metadata_cache_generation("bucket", "lost-reader")
.expect("active local generation");
let key = GetObjectMetadataCacheKey::new("bucket", "lost-reader", generation);
set.cache_get_object_fileinfo_owned(
("bucket", "lost-reader"),
Some(generation),
MetadataCachePublication {
fi: fi.clone(),
capability: GetObjectMetadataCacheCapability::Full,
verified_inline_body: None,
authority: Some(authority),
},
vec![fi],
Vec::new(),
0,
)
.await;
assert!(
set.get_object_metadata_cache.get(&key).await.is_none(),
"a lost read guard cannot publish a reusable entry"
);
assert!(guard.release());
}
#[tokio::test]
@@ -5837,6 +6088,7 @@ mod metadata_cache_tests {
online_disks: Vec::new(),
read_quorum: 0,
verified_inline_body: None,
distributed_fence_proof: None,
})
};
@@ -52,9 +52,10 @@ pub(crate) mod list {
}
pub(crate) mod multipart {
pub(crate) use rustfs_storage_api::multipart::ListPartsOwner;
pub(crate) use rustfs_storage_api::{
CompletePart, ListMultipartsInfo, ListPartsInfo, MultipartCompletionAuthorization, MultipartInfo, MultipartOperations,
MultipartUploadInspection, MultipartUploadResult, PartInfo,
CompletePart, ListMultipartsInfo, ListPartsInfo, MultipartAbortAuthorization, MultipartCompletionAuthorization,
MultipartInfo, MultipartOperations, MultipartOwner, MultipartUploadInspection, MultipartUploadResult, PartInfo,
};
}
+40 -1
View File
@@ -6825,6 +6825,28 @@ mod tests {
tokio::fs::remove_file(conflicting_metadata)
.await
.expect("remove conflicting fixture");
struct RequiresOwner;
impl crate::storage_api_contracts::multipart::MultipartAbortAuthorization for RequiresOwner {
type Error = StorageError;
fn authorize(&self, _: Option<&crate::storage_api_contracts::multipart::MultipartOwner>) -> Result<()> {
panic!("metadata-absent recovery cannot observe an upload owner");
}
}
let native_opts = ObjectOptions {
multipart_abort_authorization: Some(Arc::new(RequiresOwner)),
..Default::default()
};
let rejected = store
.abort_multipart_upload(&bucket, object, &upload.upload_id, &native_opts)
.await;
for disk_index in 0..3 {
assert!(
staging_dir(0, disk_index).join("part.1").exists(),
"owner-required Abort must preserve late data"
);
}
assert!(matches!(rejected, Err(StorageError::InvalidUploadID(..))));
let (paused, release) = crate::store::multipart::pause_orphan_multipart_cleanup_for_test(&bucket);
let recovery_store = store.clone();
let recovery_bucket = bucket.clone();
@@ -7102,7 +7124,6 @@ mod tests {
.expect("create upload in active target pool");
drop(lifecycle_guard);
mark_test_pool_decommissioning(&store, 0).await;
// Corrupt only the source pool's entry for this UploadID. Taking the
// whole source set offline would also make the bucket-incarnation
// sidecar unreadable before multipart routing is reached.
@@ -7125,6 +7146,17 @@ mod tests {
.expect("inject source upload metadata read failure");
}
let failed_binding = store
.lookup_multipart_part_binding(&bucket, object, &upload.upload_id, &ObjectOptions::default())
.await;
assert!(
failed_binding
.as_ref()
.is_err_and(|err| !crate::error::is_err_invalid_upload_id(err)),
"a hard error in the first active pool must not be hidden by the later upload: {failed_binding:?}"
);
mark_test_pool_decommissioning(&store, 0).await;
let source_result = store.pools[0]
.get_multipart_info(&bucket, object, &upload.upload_id, &ObjectOptions::default())
.await;
@@ -7140,6 +7172,13 @@ mod tests {
);
let routed = routed_result.expect("the active target UploadID must be resolved before the faulted suspended source");
assert_eq!(routed.upload_id, upload.upload_id);
let (binding, owner) = store
.lookup_multipart_part_binding(&bucket, object, &upload.upload_id, &ObjectOptions::default())
.await
.expect("binding must prefer the active target before the faulted suspended source");
assert_eq!(binding.upload_id, upload.upload_id);
let owner = owner.expect("active target must have an owner");
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket.as_str(), object));
shutdown.cancel();
}
+105 -1
View File
@@ -17,6 +17,7 @@ use crate::core::pools::{DecommissionCapacityOwner, ensure_decommission_capacity
use crate::multipart_listing::paginate_multipart_listing;
use crate::set_disk::get_lock_acquire_timeout;
use crate::storage_api_contracts::multipart::MultipartOperations as _;
use crate::storage_api_contracts::multipart::MultipartOwner;
use crate::storage_api_contracts::object::ObjectOperations as _;
use futures::{StreamExt, stream};
use rustfs_utils::path::encode_dir_object_ref;
@@ -845,6 +846,51 @@ impl ECStore {
.await
}
/// Observe active upload metadata and its recorded owner from the same storage snapshot.
///
/// A missing or inconsistent recorded owner is returned as `None`. This observation
/// does not reserve the upload or authorize a later write. The lookup is read-only;
/// cancellation leaves storage unchanged, and callers may retry.
///
/// The caller must retain any supplied lifecycle guard. With `opts.no_lock`, the
/// caller is responsible for the upload lock normally held during the observation.
///
/// # Errors
/// Preserves argument, bucket lifetime, access, lock, I/O and quorum errors from
/// the ordinary active-upload lookup.
pub async fn lookup_multipart_part_binding(
&self,
bucket: &str,
object: &str,
upload_id: &str,
opts: &ObjectOptions,
) -> Result<(MultipartInfo, Option<MultipartOwner>)> {
check_list_parts_args(bucket, object, upload_id)?;
let encoded_object = encode_dir_object_ref(object);
let object = encoded_object.as_ref();
// Lock order: bucket lifecycle before the selected set's upload lock.
let (opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
if self.single_pool() {
return self.pools[0]
.get_disks_by_key(object)
.lookup_multipart_part_binding(bucket, object, upload_id, &opts)
.await;
}
for pool_idx in self.existing_multipart_pool_order().await {
match self.pools[pool_idx]
.get_disks_by_key(object)
.lookup_multipart_part_binding(bucket, object, upload_id, &opts)
.await
{
Ok(binding) => return Ok(binding),
Err(err) if is_err_invalid_upload_id(&err) => continue,
Err(err) => return Err(err),
}
}
Err(StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned()))
}
/// Observe an upload's original namespace without recovering or acknowledging it.
///
/// Only ordinary active metadata is returned for application preflight. This
@@ -966,7 +1012,8 @@ impl ECStore {
bucket_lifecycle_guard: Option<super::BucketIncarnationFenceGuard>,
) -> Result<()> {
let invalid_upload = || StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned());
if self.ctx.lock_manager().is_disabled() {
// Without upload metadata this path has no recorded owner to authorize.
if opts.multipart_abort_authorization.is_some() || self.ctx.lock_manager().is_disabled() {
return Err(invalid_upload());
}
let Some(bucket_lifecycle_guard) = bucket_lifecycle_guard else {
@@ -1251,6 +1298,7 @@ impl ECStore {
lookup_opts.no_lock = true;
let oi = if require_all_pools {
lookup_opts.metadata_cache_safe = false;
lookup_opts.metadata_cache_fence_proof = None;
lookup_opts.suppress_read_repair = true;
// Only completion uses this branch, after validating bucket and object.
let encoded_object = encode_dir_object_ref(object);
@@ -1620,6 +1668,62 @@ mod tests {
.await
.expect("old bucket incarnations must not affect placement")
);
let directory = "binding directory/";
let encoded = encode_dir_object(directory);
let upload = store.pools[1]
.new_multipart_upload(bucket, &encoded, &opts)
.await
.expect("create directory upload in second pool");
let object_lock = store
.handle_new_ns_lock(bucket, &encoded)
.await
.expect("create object namespace lock");
let _object_writer = object_lock
.get_write_lock(Duration::from_secs(5))
.await
.expect("hold the object write lock");
let (info, owner) = tokio::time::timeout(
Duration::from_secs(5),
store.lookup_multipart_part_binding(bucket, directory, &upload.upload_id, &opts),
)
.await
.expect("binding must not acquire the object lock")
.expect("find the directory upload in the second pool");
assert_eq!((info.bucket.as_str(), info.object.as_str()), (bucket, directory));
assert_eq!(info.upload_id, upload.upload_id);
let owner = owner.expect("owner from the active upload");
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, directory));
let mut lost_opts = opts.clone();
lost_opts.bucket_lifecycle_lock_fence = Some(crate::object_api::NamespaceLockFence::lost_for_test());
assert!(matches!(
store.pools[1]
.get_disks_by_key(&encoded)
.lookup_multipart_part_binding(bucket, &encoded, &upload.upload_id, &lost_opts)
.await,
Err(StorageError::NamespaceLockQuorumUnavailable { .. })
));
let mut stale_opts = opts.clone();
stale_opts.expected_bucket_incarnation_id = Some(Uuid::new_v4());
assert!(matches!(
store
.lookup_multipart_part_binding(bucket, directory, &upload.upload_id, &stale_opts)
.await,
Err(StorageError::BucketNotFound(_))
));
assert!(matches!(
store
.lookup_multipart_part_binding(bucket, directory, "/invalid", &opts)
.await,
Err(StorageError::MalformedUploadID(_))
));
assert!(matches!(
store
.lookup_multipart_part_binding(bucket, "different/", &upload.upload_id, &opts)
.await,
Err(StorageError::InvalidUploadID(..))
));
cancel.cancel();
}
+13 -1
View File
@@ -3124,6 +3124,7 @@ impl ECStore {
let mut lookup_opts = opts.clone();
lookup_opts.no_lock = true;
lookup_opts.metadata_cache_safe = false;
lookup_opts.metadata_cache_fence_proof = None;
lookup_opts.suppress_read_repair = true;
lookup_opts.http_preconditions = None;
lookup_opts.version_id = None;
@@ -3312,6 +3313,7 @@ impl ECStore {
// (or another SetDisks owner) can commit without invalidating this cache.
let mut opts = opts.clone();
opts.metadata_cache_safe = false;
opts.metadata_cache_fence_proof = None;
Ok(if self.single_pool() {
let pool = Arc::clone(&self.pools[0]);
let metadata = pool.prepare_get_object_reader_metadata(bucket, object, &opts).await?;
@@ -3351,6 +3353,7 @@ impl ECStore {
let mut opts = opts.clone();
opts.no_lock = false;
opts.metadata_cache_safe = false;
opts.metadata_cache_fence_proof = None;
let read_lock_guards = self
.acquire_all_object_read_locks("select_object", bucket, &object, &mut opts)
.await?;
@@ -3668,7 +3671,13 @@ impl ECStore {
let acquire_start = Instant::now();
let timeout = get_lock_acquire_timeout();
let mut attempt = ObjectLockAttempt::start(op, bucket, object, opts.version_id.as_deref(), &ns_lock, "read", timeout);
let result = ns_lock.get_read_lock(timeout).await;
let distributed_cache_proof =
crate::set_disk::is_get_object_metadata_cache_distributed_proof_enabled() && self.ctx.is_dist_erasure().await;
let result = if distributed_cache_proof {
ns_lock.get_read_lock_with_cache_fence(timeout).await
} else {
ns_lock.get_read_lock(timeout).await
};
attempt.observe(&result);
let guard = result.map_err(|err| Self::map_namespace_lock_error(bucket, object, "read", err))?;
let owner = diag_enabled.then(|| ns_lock.owner().to_string());
@@ -3683,6 +3692,9 @@ impl ECStore {
);
opts.no_lock = true;
opts.metadata_cache_safe = true;
if distributed_cache_proof {
opts.add_metadata_cache_read_guard(&guard);
}
Ok(Some(
ObjectLockDiagGuard::new(
@@ -0,0 +1,47 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use rustfs_filemeta::{FileInfoOpts, get_file_info};
use std::{env, fs, io};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let path = env::args()
.nth(1)
.ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "usage: inline_lab_inspect <xl.meta> [version]"))?;
let version = env::args().nth(2).unwrap_or_default();
let data = fs::read(path)?;
let info = get_file_info(
&data,
"validation-bucket",
"validation-object",
&version,
FileInfoOpts {
data: false,
include_free_versions: false,
include_part_checksums: false,
},
)?;
println!(
"{}",
serde_json::json!({
"size": info.size,
"block_size": info.erasure.block_size,
"data_blocks": info.erasure.data_blocks,
"parity_blocks": info.erasure.parity_blocks,
"inline": info.inline_data(),
"version_id": info.version_id,
})
);
Ok(())
}
@@ -822,6 +822,11 @@ fn resume_admission_cancel_returns_node_but_native_keeps_physical_and_memory() {
.unwrap()
.set_test_capacity(vec![physical.clone()])
.expect("explicit native physical capacity");
// Building the real ECStore fixture can outlive the node admission
// sample's three-second lease. Refresh the same owner immediately
// before starting the producer so this test reaches the native-open
// cancellation boundary instead of testing stale-sample deferral.
publish_resume_node(&node, true);
let mut pause = LocalPublicationPause::install(
fixture.disk.as_ref(),
crate::heal::RUSTFS_META_BUCKET,
+68
View File
@@ -866,6 +866,74 @@ where
}
let _mutation_guard = self.cache.service_account_mutation_lock().lock().await;
self.save_new_service_account_locked(cred, updated_at).await
}
/// Order a cross-user issue with changes to either identity. Parent locks
/// are acquired by name, then the service-account lock, then the caller's
/// STS lock, matching user updates. Hold them through durable publication.
pub(crate) async fn add_delegated_service_account(&self, cred: &Credentials, caller: &Credentials) -> Result<OffsetDateTime> {
use subtle::ConstantTimeEq;
let target = cred.parent_user.as_str();
let principal = if caller.parent_user.is_empty() {
caller.access_key.as_str()
} else {
caller.parent_user.as_str()
};
if target == principal || target == caller.access_key || crate::is_root_access_key(target) {
return Err(Error::IAMActionNotAllowed);
}
let mut parents = [principal, target];
parents.sort_unstable();
let _first = self.cache.parent_mutation_lock(parents[0]).lock_owned().await;
let _second = self.cache.parent_mutation_lock(parents[1]).lock_owned().await;
let _service = self.cache.service_account_mutation_lock().lock().await;
let _session = if !caller.session_token.is_empty() && !caller.is_service_account() {
Some(self.cache.sts_account_mutation_lock(&caller.access_key).lock_owned().await)
} else {
None
};
let active_local_user = |user: &UserIdentity| {
user.credentials.is_valid() && user.credentials.parent_user.is_empty() && user.credentials.session_token.is_empty()
};
let snapshot = self.cache.snapshot();
if !snapshot.users.get(target).is_some_and(active_local_user) {
return Err(Error::IAMActionNotAllowed);
}
let root = crate::root_credentials::credentials().ok_or(Error::IAMActionNotAllowed)?;
let current = if caller.access_key == root.access_key {
&root
} else {
&snapshot
.users
.get(&caller.access_key)
.or_else(|| snapshot.sts_accounts.get(&caller.access_key))
.ok_or(Error::IAMActionNotAllowed)?
.credentials
};
if !current.is_valid()
|| current.parent_user != caller.parent_user
|| !bool::from(current.secret_key.as_bytes().ct_eq(caller.secret_key.as_bytes()))
|| !bool::from(current.session_token.as_bytes().ct_eq(caller.session_token.as_bytes()))
{
return Err(Error::IAMActionNotAllowed);
}
if !caller.parent_user.is_empty()
&& principal != root.access_key
&& !crate::sys::is_rustfs_oidc_claims(caller.claims_or_empty())
&& !snapshot.users.get(principal).is_some_and(active_local_user)
{
return Err(Error::IAMActionNotAllowed);
}
drop(snapshot);
self.save_new_service_account_locked(cred.clone(), OffsetDateTime::now_utc())
.await
}
/// The caller holds the service-account mutation lock through cache publication.
async fn save_new_service_account_locked(&self, cred: Credentials, updated_at: OffsetDateTime) -> Result<OffsetDateTime> {
let cache = self.cache.snapshot();
if cache.users.contains_key(&cred.access_key) || cache.sts_accounts.contains_key(&cred.access_key) {
return Err(Error::AccessKeyAlreadyExists);
+321 -8
View File
@@ -960,6 +960,19 @@ impl<T: Store> IamSys<T> {
opts: NewServiceAccountOpts,
updated_at: OffsetDateTime,
) -> Result<(Credentials, OffsetDateTime)> {
let cred = Self::service_account_credentials(parent_user, groups, opts)?;
let create_at = self.store.add_service_account_at(cred.clone(), updated_at).await?;
self.notify_for_service_account(&cred.access_key).await;
Ok((cred, create_at))
}
fn service_account_credentials(
parent_user: &str,
groups: Option<Vec<String>>,
opts: NewServiceAccountOpts,
) -> Result<Credentials> {
if !opts.access_key.is_empty()
&& (opts.access_key.trim() == "."
|| opts.access_key.trim() == ".."
@@ -1046,11 +1059,7 @@ impl<T: Store> IamSys<T> {
cred.name = opts.name;
cred.description = opts.description;
let create_at = self.store.add_service_account_at(cred.clone(), updated_at).await?;
self.notify_for_service_account(&cred.access_key).await;
Ok((cred, create_at))
Ok(cred)
}
pub async fn update_service_account(&self, name: &str, opts: UpdateServiceAccountOpts) -> Result<OffsetDateTime> {
@@ -1689,6 +1698,80 @@ impl<T: Store> IamSys<T> {
Ok(Some(verified_claims))
}
/// Issue a credential for a different local user. Both actions must allow the
/// authenticated caller and the same resolved target. Once admitted, the
/// write continues if the caller stops waiting.
///
/// Returns an error if the caller lacks either grant, the target is missing
/// or ineligible, the caller has been revoked, or the key cannot be saved.
pub async fn new_service_account_for_user(
&self,
target: &str,
opts: NewServiceAccountOpts,
caller: &Credentials,
conditions: &HashMap<String, Vec<String>>,
authenticated_owner: bool,
) -> Result<(Credentials, OffsetDateTime, HashMap<String, Value>)> {
use rustfs_policy::policy::{
SVC_TARGET_USER,
action::{Action, AdminAction},
};
if target.is_empty() || crate::is_root_access_key(target) || target == caller.access_key || target == caller.parent_user {
return Err(IamError::IAMActionNotAllowed);
}
// Bind the policy decision to the identity that will be written.
if !conditions
.get(SVC_TARGET_USER)
.is_some_and(|values| values.len() == 1 && values[0] == target)
{
return Err(IamError::IAMActionNotAllowed);
}
let mut args = Args {
account: &caller.access_key,
groups: &caller.groups,
action: Action::AdminAction(AdminAction::CreateServiceAccountAdminAction),
bucket: "",
conditions,
is_owner: authenticated_owner,
object: "",
claims: caller.claims_or_empty(),
deny_only: false,
};
// Both built-in decisions must use the same identity and policy revision.
let prepared = self.prepare_auth(&args).await;
if !self.eval_prepared(&prepared, &args).await {
return Err(IamError::IAMActionNotAllowed);
}
args.action = Action::AdminAction(AdminAction::CreateServiceAccountForUserAdminAction);
if !self.eval_prepared(&prepared, &args).await {
return Err(IamError::IAMActionNotAllowed);
}
if self.check_key(target).await?.0.is_none() {
return Err(IamError::NoSuchUser(target.to_owned()));
}
// A delegated credential belongs entirely to its local target. Only
// its requested session policy and expiration constrain that identity.
let opts = NewServiceAccountOpts {
claims: None,
allow_site_replicator_account: false,
..opts
};
let cred = Self::service_account_credentials(target, None, opts)?;
let iam = self.clone();
let caller = caller.clone();
let runtime = tokio::runtime::Handle::try_current().map_err(Error::other)?;
runtime
.spawn(async move {
let updated_at = iam.store.add_delegated_service_account(&cred, &caller).await?;
iam.notify_for_service_account(&cred.access_key).await;
Ok((cred, updated_at, HashMap::new()))
})
.await
.map_err(Error::other)?
}
/// Create within the authenticated caller's scope and current parent revision.
/// An admitted write owns its parent lock until publication even if the caller
/// stops waiting; errors never publish a credential from a changed parent.
@@ -1719,8 +1802,7 @@ impl<T: Store> IamSys<T> {
mut opts: NewServiceAccountOpts,
caller: &Credentials,
) -> Result<(Credentials, OffsetDateTime, HashMap<String, Value>)> {
if parent_user != caller.access_key && parent_user != caller.parent_user && !crate::is_root_access_key(&caller.access_key)
{
if parent_user != caller.access_key && parent_user != caller.parent_user {
return Err(IamError::IAMActionNotAllowed);
}
@@ -3347,6 +3429,7 @@ mod tests {
/// When true, parent user has no groups and no mapped policies (empty `policy_db_get`).
empty_policies: bool,
bucket_tags: Arc<Mutex<Result<HashMap<String, String>>>>,
policy_load_barrier: Option<Arc<tokio::sync::Barrier>>,
saved_sts_users: Arc<Mutex<HashMap<String, UserIdentity>>>,
saved_service_account_count: Arc<Mutex<usize>>,
fail_delete: Arc<std::sync::atomic::AtomicBool>,
@@ -3369,6 +3452,7 @@ mod tests {
fn new(empty_policies: bool) -> Self {
Self {
empty_policies,
policy_load_barrier: None,
bucket_tags: Arc::new(Mutex::new(Err(Error::other("bucket metadata unavailable")))),
saved_sts_users: Arc::new(Mutex::new(HashMap::new())),
saved_service_account_count: Arc::new(Mutex::new(0)),
@@ -3561,7 +3645,14 @@ mod tests {
Err(Error::InvalidArgument)
}
async fn load_policy_doc(&self, _name: &str, _m: &mut HashMap<String, rustfs_policy::policy::PolicyDoc>) -> Result<()> {
async fn load_policy_doc(&self, name: &str, docs: &mut HashMap<String, PolicyDoc>) -> Result<()> {
if let Some(barrier) = &self.policy_load_barrier {
// merge_policies has already captured the cached policy documents.
barrier.wait().await;
barrier.wait().await;
docs.insert(name.to_owned(), PolicyDoc::new(Policy::default()));
return Ok(());
}
Err(Error::InvalidArgument)
}
@@ -4391,6 +4482,67 @@ mod tests {
}
}
#[tokio::test]
async fn delegated_service_account_holds_both_parents_and_session_through_publication() {
use std::sync::atomic::Ordering;
ensure_test_global_credentials();
let store = StsTestMockStore::new(false);
let iam = IamSys::new(IamCache::new(store.clone()).await.expect("initialize IAM"));
let caller_parent = "operator";
let target = "backup";
for user in [caller_parent, target] {
iam.create_user(
user,
&AddOrUpdateUserReq {
secret_key: "originalSecret123".into(),
status: rustfs_madmin::AccountStatus::Enabled,
policy: None,
},
)
.await
.expect("create local user");
}
let session = rotation_test_session(caller_parent, "operator-console-session");
iam.set_temp_user(&session.access_key, &session, None)
.await
.expect("publish STS session");
store.block_derived_save.store(true, Ordering::SeqCst);
let pending = IamSys::<StsTestMockStore>::service_account_credentials(target, None, NewServiceAccountOpts::default())
.expect("prepare credential");
let creation = {
let iam = iam.clone();
let session = session.clone();
tokio::spawn(async move { iam.store.add_delegated_service_account(&pending, &session).await })
};
tokio::time::timeout(std::time::Duration::from_secs(5), store.derived_save_started.notified())
.await
.expect("credential write started");
for principal in [caller_parent, target] {
assert!(iam.store.cache.parent_mutation_lock(principal).try_lock().is_err());
}
assert!(
iam.store
.cache
.sts_account_mutation_lock(&session.access_key)
.try_lock()
.is_err()
);
store.block_derived_save.store(false, Ordering::SeqCst);
store.release_derived_save.notify_one();
creation.await.expect("join writer").expect("publish credential");
iam.set_user_status(target, rustfs_madmin::AccountStatus::Disabled)
.await
.expect("disable target");
let pending = IamSys::<StsTestMockStore>::service_account_credentials(target, None, NewServiceAccountOpts::default())
.expect("prepare credential for disabled user");
assert!(matches!(
iam.store.add_delegated_service_account(&pending, &session).await,
Err(Error::IAMActionNotAllowed)
));
assert_eq!(store.saved_service_account_count(), 1);
}
#[tokio::test]
async fn revoked_sts_cannot_mint_service_account_between_rotation_and_sweep() {
ensure_test_global_credentials();
@@ -5332,6 +5484,167 @@ mod tests {
);
}
#[tokio::test]
async fn delegated_service_account_requires_both_actions_in_one_policy_revision() {
use rustfs_policy::policy::SVC_TARGET_USER;
ensure_test_global_credentials();
let barrier = Arc::new(tokio::sync::Barrier::new(2));
let mut store = StsTestMockStore::new(true);
store.policy_load_barrier = Some(barrier.clone());
let cache = temp_env::async_with_vars([("RUSTFS_SKIP_BACKGROUND_TASK", Some("1"))], IamCache::new(store.clone()))
.await
.expect("initialize IAM");
let iam = IamSys::new(cache);
let operator = "backup-operator";
let target = "backup-user";
for name in [operator, target] {
iam.create_user(
name,
&AddOrUpdateUserReq {
secret_key: "testSecret123".into(),
status: rustfs_madmin::AccountStatus::Enabled,
policy: None,
},
)
.await
.expect("create local user");
}
let caller = iam.get_user(operator).await.expect("operator exists").credentials;
let set_grant = |action: &str| {
let policy = serde_json::json!({"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": action}]});
let policy = Policy::parse_config(policy.to_string().as_bytes()).expect("parse administrator policy");
iam.store
.cache
.add_or_update_policy_doc("operator-grant", &PolicyDoc::new(policy), OffsetDateTime::now_utc());
};
set_grant("admin:CreateServiceAccount");
iam.store.cache.add_or_update_user_policy(
operator,
&MappedPolicy::new("operator-grant,uncached-policy"),
OffsetDateTime::now_utc(),
);
let conditions = HashMap::from([(SVC_TARGET_USER.to_owned(), vec![target.to_owned()])]);
let create = iam.new_service_account_for_user(target, NewServiceAccountOpts::default(), &caller, &conditions, false);
let replace = async {
barrier.wait().await;
set_grant("admin:CreateServiceAccountForUser");
barrier.wait().await;
};
let (result, ()) = tokio::time::timeout(std::time::Duration::from_secs(5), async { tokio::join!(create, replace) })
.await
.expect("finish lookup and concurrent policy replacement");
assert!(matches!(result, Err(Error::IAMActionNotAllowed)));
assert_eq!(store.saved_service_account_count(), 0);
}
#[tokio::test]
async fn delegated_service_account_distinguishes_missing_target_from_storage_failure() {
use rustfs_policy::policy::SVC_TARGET_USER;
ensure_test_global_credentials();
let store = StsTestMockStore::new(true);
let iam = IamSys::new(IamCache::new(store.clone()).await.expect("initialize IAM"));
let caller = crate::root_credentials::credentials().expect("root credentials");
for target in ["missing-target", "load-failure-user"] {
let conditions = HashMap::from([(SVC_TARGET_USER.to_owned(), vec![target.to_owned()])]);
let error = iam
.new_service_account_for_user(target, NewServiceAccountOpts::default(), &caller, &conditions, true)
.await
.expect_err("cannot publish without a target");
match target {
"missing-target" => assert!(matches!(error, Error::NoSuchUser(name) if name == target)),
_ => assert!(matches!(error, Error::Io(_))),
}
}
assert_eq!(store.saved_service_account_count(), 0);
}
#[tokio::test]
async fn delegated_service_account_uses_local_target_permissions_and_requires_live_oidc_session() {
use rustfs_policy::policy::{
SVC_TARGET_USER,
action::{Action, AdminAction, S3Action},
};
ensure_test_global_credentials();
// This fixture seeds a policy mapping in memory. Its mock reload only
// persists identities and would replace the mapping with its seed data.
let cache =
temp_env::async_with_vars([("RUSTFS_SKIP_BACKGROUND_TASK", Some("1"))], IamCache::new(StsTestMockStore::new(true)))
.await
.expect("initialize IAM");
let iam = IamSys::new(cache);
let target = "backup-user";
iam.create_user(
target,
&AddOrUpdateUserReq {
secret_key: "backupUserSecret123".into(),
status: rustfs_madmin::AccountStatus::Enabled,
policy: None,
},
)
.await
.expect("create backup user");
iam.store
.cache
.add_or_update_user_policy(target, &MappedPolicy::new("readonly"), OffsetDateTime::now_utc());
let mut caller = rotation_test_session("oidc-admin-parent", "oidc-console-session");
caller.claims = Some(HashMap::from([
("iss".into(), Value::from("rustfs-oidc")),
("oidc_provider".into(), Value::from("default")),
("sub".into(), Value::from("operator")),
("parent".into(), Value::from(caller.parent_user.clone())),
(OIDC_VIRTUAL_PARENT_CLAIM.into(), Value::from(caller.parent_user.clone())),
(POLICYNAME.into(), Value::from("consoleAdmin")),
]));
iam.set_temp_user(&caller.access_key, &caller, None)
.await
.expect("publish OIDC session");
let conditions = HashMap::from([(SVC_TARGET_USER.to_owned(), vec![target.to_owned()])]);
let (created, _, replicated_claims) = iam
.new_service_account_for_user(target, NewServiceAccountOpts::default(), &caller, &conditions, false)
.await
.expect("delegate to local user");
assert_eq!(created.parent_user, target);
assert!(replicated_claims.is_empty());
let key = iam
.get_user(&created.access_key)
.await
.expect("published service account")
.credentials;
assert!(!is_rustfs_oidc_claims(key.claims_or_empty()));
let mut args = Args {
account: &key.access_key,
groups: &key.groups,
action: Action::S3Action(S3Action::GetObjectAction),
bucket: "backups",
object: "archive",
conditions: &HashMap::new(),
claims: key.claims_or_empty(),
is_owner: false,
deny_only: false,
};
assert!(iam.is_allowed(&args).await);
args.action = Action::S3Action(S3Action::PutObjectAction);
assert!(!iam.is_allowed(&args).await);
args.action = Action::AdminAction(AdminAction::CreateServiceAccountForUserAdminAction);
assert!(!iam.is_allowed(&args).await);
iam.store
.delete_user(&caller.access_key, UserType::Sts)
.await
.expect("revoke session");
let pending = IamSys::<StsTestMockStore>::service_account_credentials(target, None, NewServiceAccountOpts::default())
.expect("prepare credential");
assert!(matches!(
iam.store.add_delegated_service_account(&pending, &caller).await,
Err(Error::IAMActionNotAllowed)
));
assert!(iam.store.get_user(&pending.access_key).await.is_none());
}
#[tokio::test]
async fn oidc_service_account_uses_verified_policy_and_persisted_boundary() {
ensure_test_global_credentials();
+18 -3
View File
@@ -433,8 +433,10 @@ pub const PUT_COMMIT_LOCK_ADMISSION_OUTCOME_LOCK_ERROR: &str = "lock_error";
pub const PUT_RENAME_FDATASYNC_BATCH_MODE_SERIAL: &str = "serial";
pub const PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL: &str = "parallel";
pub const PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED: &str = "grouped";
pub const PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_LEADER: &str = "leader";
pub const PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER: &str = "follower";
pub const PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER_TIMEOUT: &str = "follower_timeout";
pub const PUT_RENAME_FDATASYNC_GROUP_OUTSTANDING_STATE_ENQUEUE_WAITERS: &str = "enqueue_waiters";
pub const PUT_RENAME_FDATASYNC_GROUP_OUTSTANDING_STATE_ENQUEUE_FILES: &str = "enqueue_files";
pub const PUT_RENAME_FDATASYNC_GROUP_OUTSTANDING_STATE_BATCH_WAITERS: &str = "batch_waiters";
@@ -4422,13 +4424,16 @@ mod tests {
set_put_stage_metrics_enabled(false);
record_put_rename_fdatasync_batch(PUT_RENAME_FDATASYNC_BATCH_MODE_SERIAL, 2);
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_LEADER, 1.0);
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER_TIMEOUT, 3.0);
record_put_rename_fdatasync_group_outstanding(PUT_RENAME_FDATASYNC_GROUP_OUTSTANDING_STATE_ENQUEUE_WAITERS, 2);
record_put_rename_disk_wait_completion(PUT_RENAME_DISK_WAIT_COMPLETION_POSITION_QUORUM_FIRST, 3.0);
record_put_rename_quorum_wait_fanout(4, 3, 3, 1, 0);
set_put_stage_metrics_enabled(true);
record_put_rename_fdatasync_batch(PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL, 9);
record_put_rename_fdatasync_batch(PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED, 4);
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_LEADER, 1.0);
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER_TIMEOUT, 3.0);
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER, 2.0);
record_put_rename_fdatasync_group_outstanding(PUT_RENAME_FDATASYNC_GROUP_OUTSTANDING_STATE_ENQUEUE_WAITERS, 2);
record_put_rename_fdatasync_group_outstanding(PUT_RENAME_FDATASYNC_GROUP_OUTSTANDING_STATE_ENQUEUE_FILES, 4);
@@ -4442,7 +4447,10 @@ mod tests {
});
let rows = snapshotter.snapshot().into_vec();
assert_eq!(histogram_samples(&rows, "rustfs_s3_put_object_rename_fdatasync_batch_files"), vec![9.0]);
assert_eq!(
histogram_samples(&rows, "rustfs_s3_put_object_rename_fdatasync_batch_files"),
vec![4.0, 9.0]
);
let batch_modes = rows
.iter()
.filter(|(composite, _, _, _)| {
@@ -4458,13 +4466,19 @@ mod tests {
.collect::<Vec<_>>()
})
.collect::<HashSet<_>>();
assert_eq!(batch_modes, HashSet::from([PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL.to_string()]));
assert_eq!(
batch_modes,
HashSet::from([
PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED.to_string(),
PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL.to_string(),
])
);
let quorum_samples = histogram_samples(&rows, "rustfs_s3_put_object_rename_quorum_wait_fanout_disks");
assert_eq!(quorum_samples, vec![0.0, 1.0, 3.0, 3.0, 4.0]);
assert_eq!(
histogram_samples(&rows, "rustfs_s3_put_object_rename_fdatasync_group_wait_ms"),
vec![1.0, 2.0]
vec![1.0, 2.0, 3.0]
);
assert_eq!(
histogram_samples(&rows, "rustfs_s3_put_object_rename_fdatasync_group_outstanding"),
@@ -4494,6 +4508,7 @@ mod tests {
HashSet::from([
PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_LEADER.to_string(),
PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER.to_string(),
PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER_TIMEOUT.to_string(),
])
);
let group_outstanding_states = rows
+1 -1
View File
@@ -74,7 +74,7 @@ pub use client::KeystoneClient;
pub use config::{KeystoneConfig, RoleMapping};
pub use error::{KeystoneError, Result};
pub use identity::KeystoneIdentityMapper;
pub use middleware::{KEYSTONE_CREDENTIALS, KeystoneAuthLayer};
pub use middleware::{KEYSTONE_CREDENTIALS, KeystoneAuthLayer, KeystoneRequestIdentity, KeystoneRequestNotValidated};
/// Keystone API version
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+178 -10
View File
@@ -16,8 +16,8 @@
//!
//! This middleware intercepts HTTP requests and checks for OpenStack Keystone
//! authentication headers (X-Auth-Token or Swift X-Storage-Token). If found, it validates the token
//! with Keystone and stores the authenticated credentials in task-local storage
//! for use by downstream authentication handlers.
//! with Keystone. The legacy edge receives credentials in task-local storage; the gateway
//! receives a single-use request identity containing the proof and sanitized authorization facts.
//!
//! ## Authentication Flow
//!
@@ -42,12 +42,12 @@ use http_body::Body;
use http_body_util::{BodyExt, Full};
use rustfs_credentials::Credentials;
use std::pin::Pin;
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
use tower::{Layer, Service};
use tracing::{debug, info, warn};
use crate::KeystoneAuthProvider;
use crate::{KeystoneAuthProvider, ValidatedKeystoneToken};
const HEADER_X_AUTH_TOKEN: &str = "X-Auth-Token";
const HEADER_X_STORAGE_TOKEN: &str = "X-Storage-Token";
@@ -59,6 +59,58 @@ tokio::task_local! {
pub static KEYSTONE_CREDENTIALS: Option<Credentials>;
}
/// A request-local, single-use result of one Keystone validation. Only this middleware can
/// create it. Clones share the same consumption slot and contain no token or secret key.
#[derive(Clone)]
pub struct KeystoneRequestIdentity(Arc<Mutex<Option<KeystoneRequestValidation>>>);
struct KeystoneRequestValidation {
provider: Arc<KeystoneAuthProvider>,
proof: ValidatedKeystoneToken,
credentials: Credentials,
had_session_token: bool,
}
/// A missing, consumed, poisoned or differently configured request identity.
#[derive(Debug, thiserror::Error)]
#[error("the request has no matching Keystone validation")]
pub struct KeystoneRequestNotValidated;
impl KeystoneRequestIdentity {
fn new(provider: Arc<KeystoneAuthProvider>, proof: ValidatedKeystoneToken, mut credentials: Credentials) -> Self {
let had_session_token = !credentials.session_token.is_empty();
credentials.secret_key.clear();
credentials.session_token.clear();
Self(Arc::new(Mutex::new(Some(KeystoneRequestValidation {
provider,
proof,
credentials,
had_session_token,
}))))
}
/// Consumes the paired proof and sanitized credential once, only for the validating provider.
/// The boolean records whether the original credential had a session token.
///
/// # Errors
/// Returns [`KeystoneRequestNotValidated`] for a different provider, a consumed slot or poison.
pub fn take_for(
self,
provider: &Arc<KeystoneAuthProvider>,
) -> Result<(ValidatedKeystoneToken, Credentials, bool), KeystoneRequestNotValidated> {
let value = self
.0
.lock()
.map_err(|_| KeystoneRequestNotValidated)?
.take()
.ok_or(KeystoneRequestNotValidated)?;
if !Arc::ptr_eq(provider, &value.provider) {
return Err(KeystoneRequestNotValidated);
}
Ok((value.proof, value.credentials, value.had_session_token))
}
}
/// Tower Layer for Keystone authentication
///
/// This layer wraps services with Keystone authentication middleware.
@@ -125,7 +177,8 @@ where
self.inner.poll_ready(cx)
}
fn call(&mut self, req: Request<ReqBody>) -> Self::Future {
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
req.extensions_mut().remove::<KeystoneRequestIdentity>();
let keystone_auth = self.keystone_auth.clone();
let mut inner = self.inner.clone();
@@ -136,7 +189,7 @@ where
None => {
// No Keystone configured, pass through to normal authentication
debug!("Keystone middleware: No provider configured, passing through");
let resp = inner.call(req).await?;
let resp = KEYSTONE_CREDENTIALS.scope(None, async move { inner.call(req).await }).await?;
let (parts, body) = resp.into_parts();
let body: BoxBody = body.map_err(Into::into).boxed_unsync();
return Ok(Response::from_parts(parts, body));
@@ -149,14 +202,22 @@ where
debug!("Keystone middleware: Found Keystone token header, validating");
// Validate token with Keystone
match keystone_auth.authenticate_with_token(token).await {
Ok(credentials) => {
match keystone_auth.authenticate_with_token_and_proof(token).await {
Ok((credentials, proof)) => {
// Authentication successful!
info!("Keystone middleware: Authentication successful for user: {}", credentials.parent_user);
req.extensions_mut().insert(KeystoneRequestIdentity::new(
Arc::clone(&keystone_auth),
proof,
credentials.clone(),
));
// Store credentials in task-local storage and continue processing
// The auth handlers will retrieve these credentials when needed
let resp = KEYSTONE_CREDENTIALS.scope(Some(credentials), inner.call(req)).await?;
let resp = KEYSTONE_CREDENTIALS
.scope(Some(credentials), async move { inner.call(req).await })
.await?;
let (parts, body) = resp.into_parts();
let body: BoxBody = body.map_err(Into::into).boxed_unsync();
return Ok(Response::from_parts(parts, body));
@@ -198,7 +259,7 @@ where
// No Keystone token header present, pass through to normal S3 authentication
debug!("Keystone middleware: No Keystone token header, passing through to S3 auth");
let resp = inner.call(req).await?;
let resp = KEYSTONE_CREDENTIALS.scope(None, async move { inner.call(req).await }).await?;
let (parts, body) = resp.into_parts();
let body: BoxBody = body.map_err(Into::into).boxed_unsync();
Ok(Response::from_parts(parts, body))
@@ -228,6 +289,113 @@ mod tests {
use crate::{KeystoneClient, KeystoneVersion};
use std::time::Duration;
#[tokio::test]
async fn request_identity_erases_secrets_and_consumes_its_proof_once() {
use std::io::{Read, Write};
use std::net::TcpListener;
let listener = TcpListener::bind("127.0.0.1:0").expect("bind Keystone stub");
let endpoint = format!("http://{}", listener.local_addr().expect("stub address"));
let server = std::thread::spawn(move || {
let (mut socket, _) = listener.accept().expect("validation request");
socket.set_read_timeout(Some(Duration::from_secs(5))).expect("stub timeout");
let mut request = [0; 4096];
assert!(socket.read(&mut request).expect("read validation request") > 0);
let expiry = (time::OffsetDateTime::now_utc() + time::Duration::hours(1))
.format(&time::format_description::well_known::Rfc3339)
.expect("expiry");
let body = serde_json::json!({"token": {"user": {"id": "user-id", "name": "user-name"}, "roles": [{"name": "member"}], "expires_at": expiry}}).to_string();
write!(
socket,
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
)
.expect("validation response");
});
let client = KeystoneClient::new(
endpoint,
KeystoneVersion::V3,
None,
None,
None,
"Default".into(),
true,
Duration::from_secs(5),
);
let provider = Arc::new(KeystoneAuthProvider::new(client, 1, Duration::from_secs(1), false));
let (mut credentials, proof) = provider
.authenticate_with_token_and_proof("request-token")
.await
.expect("validated token");
server.join().expect("stub answered");
credentials.secret_key = "original-secret".into();
credentials.session_token = "original-session".into();
let original = credentials.clone();
let carrier = KeystoneRequestIdentity::new(Arc::clone(&provider), proof, credentials);
let clone = carrier.clone();
let (proof, sanitized, had_session) = carrier.take_for(&provider).expect("first consumer");
let mut failures = Vec::new();
for (ok, fact) in [
(proof.user_id() == "user-id", "proof identity"),
(sanitized.secret_key.is_empty(), "secret erased"),
(sanitized.session_token.is_empty(), "session erased"),
(had_session, "original session presence"),
(sanitized.access_key == original.access_key, "access key retained"),
(sanitized.parent_user == original.parent_user, "parent retained"),
(sanitized.expiration == original.expiration, "expiry retained"),
(sanitized.groups == original.groups, "groups retained"),
(sanitized.claims == original.claims, "claims retained"),
(clone.take_for(&provider).is_err(), "single consumption"),
] {
if !ok {
failures.push(fact);
}
}
let poisoned = KeystoneRequestIdentity::new(Arc::clone(&provider), proof, sanitized);
let slot = Arc::clone(&poisoned.0);
assert!(
std::thread::spawn(move || {
let _guard = slot.lock().expect("unpoisoned slot");
panic!("poison the request slot");
})
.join()
.is_err()
);
if poisoned.take_for(&provider).is_ok() {
failures.push("poison refusal");
}
assert!(failures.is_empty(), "{}", failures.join(", "));
}
#[tokio::test]
async fn disabled_middleware_clears_identity_before_inner_call_and_poll() {
use tower::ServiceExt;
let service = tower::service_fn(|_request: Request<()>| {
assert!(KEYSTONE_CREDENTIALS.try_with(Clone::clone).expect("call scope").is_none());
async {
assert!(KEYSTONE_CREDENTIALS.try_with(Clone::clone).expect("poll scope").is_none());
Ok::<_, std::convert::Infallible>(Response::new(Full::new(Bytes::new())))
}
});
let outer = Credentials {
access_key: "outer-identity".into(),
..Default::default()
};
KEYSTONE_CREDENTIALS
.scope(Some(outer), async {
let response = KeystoneAuthLayer::new(None)
.layer(service)
.oneshot(Request::new(()))
.await
.expect("disabled middleware");
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(
KEYSTONE_CREDENTIALS.with(|credentials| credentials.as_ref().map(|value| value.access_key.clone())),
Some("outer-identity".into())
);
})
.await;
}
#[test]
fn test_layer_creation_no_keystone() {
// Test that layer can be created without Keystone provider
+185 -8
View File
@@ -16,14 +16,15 @@ use parking_lot::{RwLock, RwLockReadGuard, RwLockWriteGuard};
use std::collections::HashMap;
use std::ops::{Deref, DerefMut};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::time::{Duration, SystemTime};
use tokio::sync::Notify;
use tokio::time::{Instant, timeout, timeout_at};
use crate::{
FastLockGuard, GlobalLockManager, LockClient, LockId, LockInfo, LockManager, LockMetadata, LockPriority, LockRequest,
LockResponse, LockStats, LockStatus, LockType, Result,
CACHE_FENCE_SHARD_COUNT, CacheFenceStamp, FastLockGuard, GlobalLockManager, LockClient, LockId, LockInfo, LockManager,
LockMetadata, LockPriority, LockRequest, LockResponse, LockStats, LockStatus, LockType, Result,
types::cache_fence_shard_index,
};
/// Default shard count for guard storage (must be power of 2)
@@ -204,6 +205,13 @@ pub struct LocalClient {
manager: Option<Arc<GlobalLockManager>>,
reaper_started: AtomicBool,
reaper_interval: Duration,
cache_fence_boot_id: uuid::Uuid,
cache_fence_generations: Box<[AtomicU64]>,
}
struct LocalGrant {
info: LockInfo,
cache_fence_stamp: Option<CacheFenceStamp>,
}
#[derive(Debug)]
@@ -291,7 +299,7 @@ impl LocalClient {
&self,
request: &LockRequest,
acquire_deadline: Instant,
) -> std::result::Result<LockInfo, LocalAcquireFailure> {
) -> std::result::Result<LocalGrant, LocalAcquireFailure> {
let started = Instant::now();
self.ensure_reaper();
if request.lock_id.resource != request.resource {
@@ -402,7 +410,14 @@ impl LocalClient {
priority: request.priority,
wait_start_time: None,
};
return Ok(lock_info);
return Ok(LocalGrant {
info: lock_info,
cache_fence_stamp: self.cache_fence_stamp_for_grant(
&request.resource,
request.lock_type,
request.cache_fence_stamp_requested,
),
});
}
Err(crate::fast_lock::LockResult::Timeout) => {
if !retried_after_reclaim {
@@ -463,6 +478,36 @@ impl LocalClient {
manager,
reaper_started: AtomicBool::new(false),
reaper_interval,
cache_fence_boot_id: uuid::Uuid::new_v4(),
cache_fence_generations: (0..CACHE_FENCE_SHARD_COUNT)
.map(|_| AtomicU64::new(0))
.collect::<Vec<_>>()
.into_boxed_slice(),
}
}
fn cache_fence_stamp_for_grant(
&self,
resource: &crate::ObjectKey,
lock_type: LockType,
stamp_requested: bool,
) -> Option<CacheFenceStamp> {
match lock_type {
LockType::Exclusive => {
let shard_index = cache_fence_shard_index(resource);
let generation = &self.cache_fence_generations[usize::from(shard_index)];
let _ = generation
.try_update(Ordering::AcqRel, Ordering::Acquire, |current| current.checked_add(1))
.ok()?
.checked_add(1)?;
None
}
LockType::Shared if stamp_requested => {
let shard_index = cache_fence_shard_index(resource);
let generation = &self.cache_fence_generations[usize::from(shard_index)];
CacheFenceStamp::new(self.cache_fence_boot_id, shard_index, generation.load(Ordering::Acquire))
}
LockType::Shared => None,
}
}
@@ -579,7 +624,10 @@ impl LockClient for LocalClient {
let started = Instant::now();
let deadline = started.checked_add(request.acquire_timeout).unwrap_or(started);
Ok(match self.acquire_lock_until(request, deadline).await {
Ok(info) => LockResponse::success(info, Duration::ZERO),
Ok(grant) => match grant.cache_fence_stamp {
Some(stamp) => LockResponse::success_with_cache_fence_stamp(grant.info, Duration::ZERO, stamp),
None => LockResponse::success(grant.info, Duration::ZERO),
},
Err(failure) => failure.into_response(request),
})
}
@@ -591,9 +639,10 @@ impl LockClient for LocalClient {
let item_deadline = deadline.min(started.checked_add(request.acquire_timeout).unwrap_or(started));
async move {
match self.acquire_lock_until(request, item_deadline).await {
Ok(info) => super::BatchAcquireOutcome::Granted {
lock_info: Some(Box::new(info)),
Ok(grant) => super::BatchAcquireOutcome::Granted {
lock_info: Some(Box::new(grant.info)),
wait_time: Duration::ZERO,
cache_fence_stamp: grant.cache_fence_stamp,
},
Err(failure) => failure.into_batch_outcome(request),
}
@@ -1635,4 +1684,132 @@ mod tests {
let recovered = client.acquire_lock(&second).await.unwrap();
assert!(recovered.success, "the reclaimed guard must be available to the next request");
}
#[tokio::test]
async fn cache_fence_stamp_advances_only_for_exclusive_grants() {
let client = LocalClient::with_manager(Arc::new(GlobalLockManager::new()));
let resource = crate::ObjectKey::new("bucket", "cache-fence");
let ordinary = LockRequest::new(resource.clone(), LockType::Shared, "ordinary-reader");
let ordinary_response = client.acquire_lock(&ordinary).await.expect("ordinary shared grant");
assert!(
ordinary_response.cache_fence_stamp.is_none(),
"ordinary shared locks preserve their wire shape"
);
assert!(client.release(&ordinary.lock_id).await.expect("release ordinary reader"));
let first_read = LockRequest::new(resource.clone(), LockType::Shared, "reader-a").request_cache_fence_stamp();
let first = client
.acquire_lock(&first_read)
.await
.expect("first shared grant")
.cache_fence_stamp
.expect("new local voter supplies a stamp");
assert!(client.release(&first_read.lock_id).await.expect("release first reader"));
let write = LockRequest::new(resource.clone(), LockType::Exclusive, "writer");
let write_response = client.acquire_lock(&write).await.expect("exclusive grant");
assert!(
write_response.cache_fence_stamp.is_none(),
"exclusive grants advance without adding response bytes"
);
assert!(client.release(&write.lock_id).await.expect("release writer"));
let second_read = LockRequest::new(resource, LockType::Shared, "reader-b").request_cache_fence_stamp();
let second = client
.acquire_lock(&second_read)
.await
.expect("second shared grant")
.cache_fence_stamp
.expect("second shared grant supplies a stamp");
assert_eq!(second.voter_boot_id(), first.voter_boot_id());
assert_eq!(second.shard_index(), first.shard_index());
assert_eq!(second.generation(), first.generation() + 1);
}
#[tokio::test]
async fn cache_fence_voter_restart_changes_boot_identity() {
let resource = crate::ObjectKey::new("bucket", "restart");
let request = LockRequest::new(resource, LockType::Shared, "reader").request_cache_fence_stamp();
let first = LocalClient::with_manager(Arc::new(GlobalLockManager::new()))
.acquire_lock(&request)
.await
.expect("first voter grant")
.cache_fence_stamp
.expect("first voter stamp");
let second = LocalClient::with_manager(Arc::new(GlobalLockManager::new()))
.acquire_lock(&request)
.await
.expect("restarted voter grant")
.cache_fence_stamp
.expect("restarted voter stamp");
assert_ne!(first.voter_boot_id(), second.voter_boot_id());
}
#[tokio::test]
async fn cache_fence_generation_overflow_permanently_withholds_proof() {
let client = LocalClient::with_manager(Arc::new(GlobalLockManager::new()));
let resource = crate::ObjectKey::new("bucket", "overflow");
let shard = cache_fence_shard_index(&resource);
client.cache_fence_generations[usize::from(shard)].store(u64::MAX - 1, Ordering::Release);
let write = LockRequest::new(resource.clone(), LockType::Exclusive, "writer");
let response = client
.acquire_lock(&write)
.await
.expect("overflowing exclusive grant remains a valid lock");
assert!(response.success);
assert!(response.cache_fence_stamp.is_none(), "exhausted generation cannot authorize cache use");
assert!(client.release(&write.lock_id).await.expect("release overflow writer"));
let read = LockRequest::new(resource, LockType::Shared, "reader").request_cache_fence_stamp();
let response = client
.acquire_lock(&read)
.await
.expect("shared grant after overflow remains a valid lock");
assert!(response.success);
assert!(
response.cache_fence_stamp.is_none(),
"overflowed shard stays fail closed until voter restart"
);
}
#[tokio::test]
async fn cache_fence_hash_collision_causes_only_conservative_invalidation() {
let mut seen = std::collections::HashMap::new();
let (first_key, second_key) = (0..=CACHE_FENCE_SHARD_COUNT)
.find_map(|index| {
let key = crate::ObjectKey::new("bucket", format!("collision-{index}"));
let shard = cache_fence_shard_index(&key);
seen.insert(shard, key.clone()).map(|previous| (previous, key))
})
.expect("4097 keys must contain a collision across 4096 shards");
let client = LocalClient::with_manager(Arc::new(GlobalLockManager::new()));
let first_read = LockRequest::new(first_key.clone(), LockType::Shared, "reader-a").request_cache_fence_stamp();
let before = client
.acquire_lock(&first_read)
.await
.expect("first collision reader")
.cache_fence_stamp
.expect("first collision stamp");
assert!(
client
.release(&first_read.lock_id)
.await
.expect("release first collision reader")
);
let other_write = LockRequest::new(second_key, LockType::Exclusive, "writer");
assert!(client.acquire_lock(&other_write).await.expect("colliding writer").success);
assert!(client.release(&other_write.lock_id).await.expect("release colliding writer"));
let second_read = LockRequest::new(first_key, LockType::Shared, "reader-b").request_cache_fence_stamp();
let after = client
.acquire_lock(&second_read)
.await
.expect("second collision reader")
.cache_fence_stamp
.expect("second collision stamp");
assert_eq!(after.generation(), before.generation() + 1);
}
}
+3
View File
@@ -29,6 +29,7 @@ pub enum BatchAcquireOutcome {
Granted {
lock_info: Option<Box<LockInfo>>,
wait_time: Duration,
cache_fence_stamp: Option<crate::CacheFenceStamp>,
},
/// This attempt cannot grant later, and another attempt may succeed.
RetryableNoGrant { error: LockError },
@@ -49,6 +50,7 @@ impl BatchAcquireOutcome {
Self::Granted {
lock_info: response.lock_info.map(Box::new),
wait_time: response.wait_time,
cache_fence_stamp: response.cache_fence_stamp,
}
} else {
Self::Ambiguous {
@@ -231,6 +233,7 @@ mod tests {
error: None,
wait_time: Duration::ZERO,
position_in_queue: None,
cache_fence_stamp: None,
}),
BatchAcquireOutcome::Granted { lock_info: None, .. }
));
+153 -2
View File
@@ -16,7 +16,7 @@ use crate::{
ObjectKey,
client::{BatchAcquireOutcome, LockCleanupOutcome, LockClient},
error::{LockError, Result},
types::{LockId, LockInfo, LockRequest, LockResponse, LockStatus, LockType},
types::{CacheFenceProof, CacheFenceStamp, LockId, LockInfo, LockRequest, LockResponse, LockStatus, LockType},
};
use futures::{
future::join_all,
@@ -358,6 +358,7 @@ struct HeldLockEntry {
lock_id: LockId,
client: Arc<dyn LockClient>,
valid_until: Instant,
cache_fence_stamp: Option<CacheFenceStamp>,
}
impl HeldLockEntry {
@@ -453,6 +454,7 @@ pub struct DistributedLockGuard {
refresh_task: Option<JoinHandle<()>>,
/// Lock-loss signal, shared with the heartbeat task.
lock_lost: Arc<LockLostSignal>,
cache_fence_proof: Option<CacheFenceProof>,
}
impl DistributedLockGuard {
@@ -464,6 +466,15 @@ impl DistributedLockGuard {
/// - `heartbeat`: lease TTL, optional renewal cadence, quorum, and diagnostics.
fn new(lock_id: LockId, entries: Vec<HeldLockEntry>, lock_type: LockType, heartbeat: LockHeartbeatConfig) -> Self {
record_lock_held_acquire(lock_type);
let cache_fence_proof = (lock_type == LockType::Shared)
.then(|| {
CacheFenceProof::from_grants(
&heartbeat.resource,
entries.iter().filter_map(|entry| entry.cache_fence_stamp),
heartbeat.refresh_quorum,
)
})
.flatten();
let lock_lost = Arc::new(LockLostSignal::default());
match Self::quorum_valid_until(&entries, heartbeat.refresh_quorum) {
Some(valid_until) => lock_lost.set_valid_until(valid_until),
@@ -484,6 +495,7 @@ impl DistributedLockGuard {
disarmed: false,
refresh_task,
lock_lost,
cache_fence_proof,
}
}
@@ -607,6 +619,12 @@ impl DistributedLockGuard {
self.entries.iter().map(|entry| &entry.lock_id)
}
/// Read-quorum cache-fence proof carried by this guard, when every required
/// voter supplied one valid, distinct stamp.
pub fn cache_fence_proof(&self) -> Option<&CacheFenceProof> {
self.cache_fence_proof.as_ref()
}
/// Whether the guard's heartbeat has observed a refresh-quorum loss.
pub fn is_lock_lost(&self) -> bool {
self.lock_lost.is_lost()
@@ -911,6 +929,22 @@ impl DistributedLock {
self.acquire_guard(&req).await
}
/// Acquire a shared guard and request cache-fence stamps from capable voters.
#[hotpath::measure]
pub async fn rlock_guard_with_cache_fence(
&self,
resource: ObjectKey,
owner: &str,
timeout: Duration,
ttl: Duration,
) -> Result<Option<DistributedLockGuard>> {
let req = LockRequest::new(resource, LockType::Shared, owner)
.with_acquire_timeout(timeout)
.with_ttl(ttl)
.request_cache_fence_stamp();
self.acquire_guard(&req).await
}
/// Acquire independent exclusive locks with one sub-batch per peer and round.
/// A successful key starts its heartbeat immediately. Only completed, explicitly
/// retryable attempts whose grants have been cleaned may enter another round.
@@ -1113,11 +1147,12 @@ impl DistributedLock {
pending_by_key[key][reply.peer_index] = false;
let outcome = normalized_batch_outcome(request, outcomes.next());
match outcome {
BatchAcquireOutcome::Granted { .. } => {
BatchAcquireOutcome::Granted { cache_fence_stamp, .. } => {
let entry = HeldLockEntry {
lock_id: request.lock_id.clone(),
client: client.clone(),
valid_until,
cache_fence_stamp,
};
if results[key].is_some() {
redundant.push(entry.release_entry());
@@ -1477,6 +1512,7 @@ impl DistributedLock {
lock_id: request.lock_id.clone(),
client: client.clone(),
valid_until: reply.started,
cache_fence_stamp: None,
};
let completed = completed.clone();
// Each task owns its grant before the drain next awaits.
@@ -2049,6 +2085,7 @@ impl DistributedLock {
match join_result {
Ok((idx, Ok(resp))) => {
if resp.success {
let cache_fence_stamp = resp.cache_fence_stamp;
let lock_id = resp
.lock_info
.as_ref()
@@ -2060,6 +2097,7 @@ impl DistributedLock {
lock_id,
client: client.clone(),
valid_until: grant_valid_until,
cache_fence_stamp,
});
} else {
tracing::warn!("Missing lock client at index {} while recording success", idx);
@@ -2352,6 +2390,118 @@ mod tests {
RpcError, // Err RPC jitter
}
#[derive(Debug)]
struct LegacyProofClient {
inner: LocalClient,
}
impl LegacyProofClient {
fn new() -> Self {
Self {
inner: LocalClient::with_manager(Arc::new(GlobalLockManager::new())),
}
}
}
#[async_trait::async_trait]
impl LockClient for LegacyProofClient {
async fn acquire_lock(&self, request: &LockRequest) -> crate::Result<LockResponse> {
let mut response = self.inner.acquire_lock(request).await?;
response.cache_fence_stamp = None;
Ok(response)
}
async fn release(&self, lock_id: &LockId) -> crate::Result<bool> {
self.inner.release(lock_id).await
}
async fn refresh(&self, lock_id: &LockId) -> crate::Result<bool> {
self.inner.refresh(lock_id).await
}
async fn force_release(&self, lock_id: &LockId) -> crate::Result<bool> {
self.inner.force_release(lock_id).await
}
async fn check_status(&self, lock_id: &LockId) -> crate::Result<Option<LockInfo>> {
self.inner.check_status(lock_id).await
}
async fn get_stats(&self) -> crate::Result<LockStats> {
self.inner.get_stats().await
}
async fn close(&self) -> crate::Result<()> {
self.inner.close().await
}
async fn is_online(&self) -> bool {
true
}
async fn is_local(&self) -> bool {
true
}
}
fn stamped_voters(count: usize) -> Vec<Arc<dyn LockClient>> {
(0..count)
.map(|_| Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))) as Arc<dyn LockClient>)
.collect()
}
#[tokio::test]
async fn shared_guard_exposes_complete_cache_fence_quorum_proof() {
let lock = DistributedLock::new("cache-proof".to_owned(), stamped_voters(3), 2);
let resource = ObjectKey::new("bucket", "object");
let read = LockRequest::new(resource.clone(), LockType::Shared, "reader").request_cache_fence_stamp();
let mut first = lock
.acquire_guard(&read)
.await
.expect("read quorum acquisition")
.expect("read quorum guard");
let before = first.cache_fence_proof().expect("all new voters provide proof").clone();
assert_eq!(before.stamps().len(), 2);
assert!(first.release());
let write = LockRequest::new(resource.clone(), LockType::Exclusive, "writer");
let mut writer = lock
.acquire_guard(&write)
.await
.expect("write quorum acquisition")
.expect("write quorum guard");
assert!(writer.cache_fence_proof().is_none(), "write guards never authorize cache hits");
assert!(writer.release());
let mut second = lock
.acquire_guard(&read)
.await
.expect("second read quorum acquisition")
.expect("second read quorum guard");
let after = second.cache_fence_proof().expect("second read proof");
assert_ne!(&before, after, "an intervening write quorum must change every later read proof");
assert!(second.release());
}
#[tokio::test]
async fn shared_guard_with_legacy_quorum_keeps_lock_but_withholds_proof() {
let clients: Vec<Arc<dyn LockClient>> = vec![
Arc::new(LegacyProofClient::new()),
Arc::new(LegacyProofClient::new()),
Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))),
];
let lock = DistributedLock::new("mixed-cache-proof".to_owned(), clients, 2);
let read = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Shared, "reader").request_cache_fence_stamp();
let mut guard = lock
.acquire_guard(&read)
.await
.expect("mixed read quorum acquisition")
.expect("mixed read quorum guard remains available");
assert!(guard.cache_fence_proof().is_none(), "legacy voters fail closed only for cache proof");
assert!(!guard.is_lock_lost(), "missing proof must not weaken the acquired lock");
assert!(guard.release());
}
struct ConstantRng(u64);
impl TryRng for ConstantRng {
@@ -4730,6 +4880,7 @@ mod tests {
error: None,
wait_time: Duration::ZERO,
position_in_queue: None,
cache_fence_stamp: None,
};
let responses = Ok(vec![
success.clone(),
@@ -671,6 +671,7 @@ fn batch_diagnostics_preserve_typed_no_grant_and_ambiguity() {
BatchAcquireOutcome::Granted {
lock_info: None,
wait_time: Duration::ZERO,
cache_fence_stamp: None,
},
BatchAcquireOutcome::RetryableNoGrant {
error: LockError::internal("explicit"),
+2 -2
View File
@@ -51,8 +51,8 @@ pub use crate::{
namespace::{NamespaceLock, NamespaceLockGuard, NamespaceLockWrapper},
// Core types
types::{
HealthInfo, HealthStatus, LockId, LockInfo, LockLeaseInfo, LockMetadata, LockPriority, LockRequest, LockResponse,
LockStats, LockStatus, LockType,
CACHE_FENCE_SHARD_COUNT, CacheFenceProof, CacheFenceStamp, CacheFenceStampDecodeError, HealthInfo, HealthStatus, LockId,
LockInfo, LockLeaseInfo, LockMetadata, LockPriority, LockRequest, LockResponse, LockStats, LockStatus, LockType,
},
};
+59
View File
@@ -89,6 +89,18 @@ impl NamespaceLockWrapper {
self.lock.get_read_lock(self.resource.clone(), &self.owner, timeout).await
}
/// Acquire a shared lock and request cache-fence proof from capable voters.
/// Lock admission is unchanged; legacy or insufficient proof is represented
/// by [`NamespaceLockGuard::cache_fence_proof`] returning `None`.
pub async fn get_read_lock_with_cache_fence(
&self,
timeout: Duration,
) -> std::result::Result<NamespaceLockGuard, crate::error::LockError> {
self.lock
.get_read_lock_with_cache_fence(self.resource.clone(), &self.owner, timeout)
.await
}
/// Acquire write locks for `resources` in one batched round using this
/// wrapper's owner; see [`NamespaceLock::acquire_write_locks_batch`].
/// One guard or error per resource, in `resources` order.
@@ -110,6 +122,15 @@ impl NamespaceLockGuard {
}
}
/// Read-quorum cache-fence proof supplied by distributed lock voters.
/// Local fast locks and mixed/legacy cohorts return `None`.
pub fn cache_fence_proof(&self) -> Option<&crate::CacheFenceProof> {
match self {
Self::Standard(guard) => guard.cache_fence_proof(),
Self::Fast(_) => None,
}
}
/// Get the object key if available (only for Fast guards)
pub fn key(&self) -> Option<&ObjectKey> {
match self {
@@ -298,6 +319,27 @@ impl NamespaceLock {
}
}
/// Convenience: acquire a shared guard with optional cache-fence proof.
/// Missing proof never changes whether the underlying lock reaches quorum.
pub async fn rlock_guard_with_cache_fence(
&self,
resource: ObjectKey,
owner: &str,
timeout: Duration,
ttl: Duration,
) -> Result<Option<NamespaceLockGuard>> {
match self {
Self::Distributed(lock) => lock
.rlock_guard_with_cache_fence(resource, owner, timeout, ttl)
.await
.map(|opt| opt.map(NamespaceLockGuard::Standard)),
Self::Local(lock) => lock
.rlock_guard(resource, owner, timeout, ttl)
.await
.map(|opt| opt.map(NamespaceLockGuard::Fast)),
}
}
/// Acquire write lock (exclusive lock) with timeout
/// Returns the guard if acquisition succeeds, or an error if it fails
pub async fn get_write_lock(
@@ -360,6 +402,23 @@ impl NamespaceLock {
}
}
/// Acquire a shared lock while requesting a cache-fence proof from the
/// existing distributed quorum. Missing proof never changes lock success.
pub async fn get_read_lock_with_cache_fence(
&self,
resource: ObjectKey,
owner: &str,
timeout: Duration,
) -> std::result::Result<NamespaceLockGuard, crate::error::LockError> {
let ttl = crate::fast_lock::DEFAULT_LOCK_TIMEOUT;
let resource_str = format!("{}", resource);
match self.rlock_guard_with_cache_fence(resource, owner, timeout, ttl).await {
Ok(Some(guard)) => Ok(guard),
Ok(None) => Err(crate::error::LockError::timeout(resource_str, timeout)),
Err(error) => Err(error),
}
}
/// Acquire exclusive locks for many independent resources in one batched
/// round; see [`crate::distributed_lock::DistributedLock::acquire_guard_batch`].
/// Returns one guard or error per resource, in `resources` order.
+33
View File
@@ -664,6 +664,39 @@ async fn test_namespace_lock_get_read_lock_local() {
}
}
#[tokio::test]
async fn cache_fence_read_api_is_opt_in_and_does_not_change_lock_admission() {
let clients = (0..3)
.map(|_| Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))) as Arc<dyn LockClient>)
.collect();
let lock = NamespaceLock::with_clients("cache-fence-proof".to_string(), clients);
let resource = create_test_object_key("bucket", "object");
let mut ordinary = lock
.get_read_lock(resource.clone(), "ordinary", Duration::from_secs(1))
.await
.expect("ordinary read lock");
assert!(ordinary.cache_fence_proof().is_none(), "ordinary read locks do not add proof traffic");
assert!(ordinary.release());
let mut requested = lock
.get_read_lock_with_cache_fence(resource, "cache-reader", Duration::from_secs(1))
.await
.expect("proof-carrying read lock uses the same quorum admission");
assert_eq!(requested.cache_fence_proof().expect("new voter quorum proof").stamps().len(), 2);
assert!(requested.release());
let local = NamespaceLock::with_local_manager("local-cache-fence".to_string(), Arc::new(GlobalLockManager::new()));
let local_guard = local
.get_read_lock_with_cache_fence(create_test_object_key("bucket", "local-object"), "local-reader", Duration::from_secs(1))
.await
.expect("local fast lock remains available");
assert!(
local_guard.cache_fence_proof().is_none(),
"single-process fast locks need no distributed proof"
);
}
#[tokio::test]
async fn test_namespace_lock_guard_release() {
let manager = Arc::new(GlobalLockManager::new());
+295 -1
View File
@@ -13,11 +13,180 @@
// limitations under the License.
use serde::{Deserialize, Serialize};
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use std::{
sync::Arc,
time::{Duration, SystemTime, UNIX_EPOCH},
};
use uuid::Uuid;
use crate::ObjectKey;
/// Wire version for namespace-lock cache-fence stamps.
pub const CACHE_FENCE_STAMP_VERSION: u8 = 1;
/// Fixed number of mutation-counter shards maintained by each lock voter.
pub const CACHE_FENCE_SHARD_COUNT: usize = 4096;
const CACHE_FENCE_SHARD_COUNT_U16: u16 = 4096;
const CACHE_FENCE_SHARD_MASK: u16 = CACHE_FENCE_SHARD_COUNT_U16 - 1;
const CACHE_FENCE_STAMP_ENCODED_LEN: usize = 27;
/// Stable shard for namespace mutation generations. Hash collisions only cause
/// conservative cache invalidation; this is not an identity or authorization hash.
pub(crate) fn cache_fence_shard_index(resource: &ObjectKey) -> u16 {
const FNV_OFFSET: u64 = 0xcbf29ce484222325;
const FNV_PRIME: u64 = 0x100000001b3;
fn update(mut hash: u64, bytes: &[u8]) -> u64 {
for byte in bytes {
hash ^= u64::from(*byte);
hash = hash.wrapping_mul(FNV_PRIME);
}
hash
}
let mut hash = update(FNV_OFFSET, resource.bucket.as_bytes());
hash = update(hash, &[0]);
hash = update(hash, resource.object.as_bytes());
hash = update(hash, &[1]);
if let Some(version) = &resource.version {
hash = update(hash, &[1]);
hash = update(hash, version.as_bytes());
} else {
hash = update(hash, &[0]);
}
let bytes = hash.to_le_bytes();
u16::from_le_bytes([bytes[0], bytes[1]]) & CACHE_FENCE_SHARD_MASK
}
/// One lock voter's proof that identifies its process lifetime and the current
/// mutation generation for a stable object-key shard.
#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub struct CacheFenceStamp {
voter_boot_id: Uuid,
shard_index: u16,
generation: u64,
}
impl CacheFenceStamp {
pub(crate) fn new(voter_boot_id: Uuid, shard_index: u16, generation: u64) -> Option<Self> {
(voter_boot_id != Uuid::nil() && shard_index < CACHE_FENCE_SHARD_COUNT_U16 && generation != u64::MAX).then_some(Self {
voter_boot_id,
shard_index,
generation,
})
}
/// Unique identity for this lock-voter process lifetime.
pub fn voter_boot_id(self) -> Uuid {
self.voter_boot_id
}
/// Stable object-key shard whose generation this stamp describes.
pub fn shard_index(self) -> u16 {
self.shard_index
}
/// Monotonic generation observed or advanced by the lock grant.
pub fn generation(self) -> u64 {
self.generation
}
/// Encode the stamp for the optional lock-response proof field.
pub fn encode(self) -> [u8; CACHE_FENCE_STAMP_ENCODED_LEN] {
let mut encoded = [0; CACHE_FENCE_STAMP_ENCODED_LEN];
encoded[0] = CACHE_FENCE_STAMP_VERSION;
encoded[1..17].copy_from_slice(self.voter_boot_id.as_bytes());
encoded[17..19].copy_from_slice(&self.shard_index.to_le_bytes());
encoded[19..27].copy_from_slice(&self.generation.to_le_bytes());
encoded
}
/// Decode and strictly validate an optional lock-response proof field.
///
/// # Errors
/// Returns [`CacheFenceStampDecodeError`] for unknown versions, malformed
/// lengths, invalid voter identities, out-of-range shards, or an exhausted
/// generation sentinel.
pub fn decode(encoded: &[u8]) -> Result<Self, CacheFenceStampDecodeError> {
if encoded.len() != CACHE_FENCE_STAMP_ENCODED_LEN {
return Err(CacheFenceStampDecodeError::InvalidLength);
}
if encoded[0] != CACHE_FENCE_STAMP_VERSION {
return Err(CacheFenceStampDecodeError::UnsupportedVersion);
}
let voter_boot_id = Uuid::from_slice(&encoded[1..17]).map_err(|_| CacheFenceStampDecodeError::InvalidBootId)?;
if voter_boot_id.is_nil() {
return Err(CacheFenceStampDecodeError::InvalidBootId);
}
let shard_index = u16::from_le_bytes([encoded[17], encoded[18]]);
if shard_index >= CACHE_FENCE_SHARD_COUNT_U16 {
return Err(CacheFenceStampDecodeError::InvalidShard);
}
let generation = u64::from_le_bytes(
encoded[19..27]
.try_into()
.map_err(|_| CacheFenceStampDecodeError::InvalidLength)?,
);
if generation == u64::MAX {
return Err(CacheFenceStampDecodeError::ExhaustedGeneration);
}
Ok(Self {
voter_boot_id,
shard_index,
generation,
})
}
}
/// Invalid or unsupported cache-fence proof received from a lock peer.
#[derive(Clone, Copy, Debug, Eq, PartialEq, thiserror::Error)]
pub enum CacheFenceStampDecodeError {
#[error("cache-fence stamp has an invalid length")]
InvalidLength,
#[error("cache-fence stamp uses an unsupported protocol version")]
UnsupportedVersion,
#[error("cache-fence stamp has an invalid voter boot id")]
InvalidBootId,
#[error("cache-fence stamp has an invalid shard")]
InvalidShard,
#[error("cache-fence stamp generation is exhausted")]
ExhaustedGeneration,
}
/// A deterministic read-quorum proof suitable for binding a future metadata
/// cache entry to the namespace-lock cohort that authorized its fill.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct CacheFenceProof {
stamps: Arc<[CacheFenceStamp]>,
}
impl CacheFenceProof {
pub(crate) fn from_grants(
resource: &ObjectKey,
stamps: impl IntoIterator<Item = CacheFenceStamp>,
required_quorum: usize,
) -> Option<Self> {
if required_quorum == 0 {
return None;
}
let expected_shard = cache_fence_shard_index(resource);
let mut stamps = stamps.into_iter().collect::<Vec<_>>();
if stamps.iter().any(|stamp| stamp.shard_index != expected_shard) {
return None;
}
stamps.sort_unstable_by_key(|stamp| stamp.voter_boot_id);
if stamps.windows(2).any(|pair| pair[0].voter_boot_id == pair[1].voter_boot_id) || stamps.len() < required_quorum {
return None;
}
stamps.truncate(required_quorum);
Some(Self { stamps: stamps.into() })
}
/// Sorted, distinct voter stamps that form this proof.
pub fn stamps(&self) -> &[CacheFenceStamp] {
&self.stamps
}
}
/// Lock type enumeration
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum LockType {
@@ -251,6 +420,10 @@ pub struct LockRequest {
/// (which never encoded this field) deserializable.
#[serde(default)]
pub refresh_interval: Option<Duration>,
/// Request a cache-fence stamp from shared-lock voters. Exclusive grants
/// always advance their voter generation, including for legacy requests.
#[serde(default)]
pub cache_fence_stamp_requested: bool,
}
impl LockRequest {
@@ -268,6 +441,7 @@ impl LockRequest {
deadlock_detection: false,
suppress_contention_logs: false,
refresh_interval: None,
cache_fence_stamp_requested: false,
}
}
@@ -312,6 +486,12 @@ impl LockRequest {
self.refresh_interval = Some(interval);
self
}
/// Ask capable shared-lock voters to return their current cache-fence stamp.
pub fn request_cache_fence_stamp(mut self) -> Self {
self.cache_fence_stamp_requested = true;
self
}
}
/// Lock response structure
@@ -327,6 +507,9 @@ pub struct LockResponse {
pub wait_time: Duration,
/// Position in wait queue
pub position_in_queue: Option<usize>,
/// Optional cache-fence generation returned by the granting voter.
#[serde(skip)]
pub cache_fence_stamp: Option<CacheFenceStamp>,
}
impl LockResponse {
@@ -338,6 +521,19 @@ impl LockResponse {
error: None,
wait_time,
position_in_queue: None,
cache_fence_stamp: None,
}
}
/// Create a success response carrying a validated cache-fence stamp.
pub fn success_with_cache_fence_stamp(lock_info: LockInfo, wait_time: Duration, cache_fence_stamp: CacheFenceStamp) -> Self {
Self {
success: true,
lock_info: Some(lock_info),
error: None,
wait_time,
position_in_queue: None,
cache_fence_stamp: Some(cache_fence_stamp),
}
}
@@ -349,6 +545,7 @@ impl LockResponse {
error: Some(error.into()),
wait_time,
position_in_queue: None,
cache_fence_stamp: None,
}
}
@@ -360,6 +557,7 @@ impl LockResponse {
error: None,
wait_time,
position_in_queue: Some(position),
cache_fence_stamp: None,
}
}
@@ -623,3 +821,99 @@ impl WaitQueueItem {
self.wait_start_time.elapsed().unwrap_or_default()
}
}
#[cfg(test)]
mod cache_fence_tests {
use super::{
CACHE_FENCE_SHARD_COUNT_U16, CACHE_FENCE_STAMP_VERSION, CacheFenceProof, CacheFenceStamp, CacheFenceStampDecodeError,
cache_fence_shard_index,
};
use crate::{LockRequest, LockType, ObjectKey};
use uuid::Uuid;
fn stamp(boot: u128, shard: u16, generation: u64) -> CacheFenceStamp {
CacheFenceStamp::new(Uuid::from_u128(boot), shard, generation).expect("test stamp should be valid")
}
#[test]
fn cache_fence_stamp_round_trip_and_strict_validation() {
let original = stamp(1, 17, 42);
let encoded = original.encode();
assert_eq!(CacheFenceStamp::decode(&encoded), Ok(original));
assert_eq!(CacheFenceStamp::decode(&encoded[..26]), Err(CacheFenceStampDecodeError::InvalidLength));
let mut invalid = encoded;
invalid[0] = CACHE_FENCE_STAMP_VERSION + 1;
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::UnsupportedVersion));
let mut invalid = encoded;
invalid[1..17].fill(0);
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::InvalidBootId));
let mut invalid = encoded;
invalid[17..19].copy_from_slice(&CACHE_FENCE_SHARD_COUNT_U16.to_le_bytes());
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::InvalidShard));
let mut invalid = encoded;
invalid[19..27].copy_from_slice(&u64::MAX.to_le_bytes());
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::ExhaustedGeneration));
}
#[test]
fn cache_fence_proof_requires_distinct_matching_quorum_stamps() {
let resource = ObjectKey::new("bucket", "object");
let shard = cache_fence_shard_index(&resource);
assert_eq!(shard, 451, "stable object-key hash must not drift silently");
let first = stamp(1, shard, 7);
let second = stamp(2, shard, 7);
let proof = CacheFenceProof::from_grants(&resource, [second, first], 2).expect("two distinct voters form quorum proof");
assert_eq!(proof.stamps(), &[first, second], "proof order is deterministic");
assert_eq!(CacheFenceProof::from_grants(&resource, [first], 2), None);
assert_eq!(CacheFenceProof::from_grants(&resource, [first, first], 2), None);
let other_shard = (shard + 1) % CACHE_FENCE_SHARD_COUNT_U16;
assert_eq!(CacheFenceProof::from_grants(&resource, [first, stamp(3, other_shard, 7)], 2), None);
}
#[test]
fn read_and_write_quorums_always_intersect_for_supported_fleets() {
for voters in 1_u32..=16 {
let write_quorum = voters / 2 + 1;
let read_quorum = voters - voters / 2;
let all = (1_u32 << voters) - 1;
for cohort in 0..=all {
if cohort.count_ones() == write_quorum {
assert!(
(all ^ cohort).count_ones() < read_quorum,
"a read quorum must not fit outside a successful write quorum: voters={voters}, cohort={cohort:#x}"
);
}
if cohort.count_ones() == read_quorum {
assert!(
(all ^ cohort).count_ones() < write_quorum,
"a write quorum must not fit outside a successful read quorum: voters={voters}, cohort={cohort:#x}"
);
}
}
}
}
#[test]
fn cache_fence_request_flag_defaults_off_for_legacy_json() {
let request =
LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Shared, "reader").request_cache_fence_stamp();
let mut encoded = serde_json::to_value(&request).expect("encode current lock request");
encoded
.as_object_mut()
.expect("lock request encodes as an object")
.remove("cache_fence_stamp_requested");
let legacy: LockRequest = serde_json::from_value(encoded).expect("decode request from a legacy peer");
assert!(!legacy.cache_fence_stamp_requested);
let current: LockRequest = serde_json::from_str(&serde_json::to_string(&request).expect("encode requested proof"))
.expect("decode requested proof");
assert!(current.cache_fence_stamp_requested);
}
}
+1
View File
@@ -30,6 +30,7 @@ pub mod variables;
pub use action::ActionSet;
pub use doc::PolicyDoc;
pub use effect::Effect;
pub use function::key_name::SVC_TARGET_USER;
pub use function::{Functions, is_server_derived_condition_key};
pub use id::ID;
pub use policy::*;
+3
View File
@@ -495,6 +495,8 @@ pub enum AdminAction {
SiteReplicationOperationAction,
#[strum(serialize = "admin:CreateServiceAccount")]
CreateServiceAccountAdminAction,
#[strum(serialize = "admin:CreateServiceAccountForUser")]
CreateServiceAccountForUserAdminAction,
#[strum(serialize = "admin:UpdateServiceAccount")]
UpdateServiceAccountAdminAction,
#[strum(serialize = "admin:RemoveServiceAccount")]
@@ -705,6 +707,7 @@ impl AdminAction {
| AdminAction::SiteReplicationInfoAction
| AdminAction::SiteReplicationOperationAction
| AdminAction::CreateServiceAccountAdminAction
| AdminAction::CreateServiceAccountForUserAdminAction
| AdminAction::UpdateServiceAccountAdminAction
| AdminAction::RemoveServiceAccountAdminAction
| AdminAction::ListServiceAccountsAdminAction
+10 -1
View File
@@ -48,6 +48,9 @@ impl TryFrom<&str> for KeyName {
}
}
/// Canonical condition-map key for the resolved service-account parent.
pub const SVC_TARGET_USER: &str = "TargetUser";
impl KeyName {
pub const COMMON_KEYS: &'static [KeyName] = &[
// s3
@@ -136,7 +139,11 @@ impl KeyName {
/// Names (prefix stripped) of the well-known condition keys that request input
/// must never populate. See [`KeyName::is_server_derived`].
pub fn server_derived_key_names() -> impl Iterator<Item = &'static str> {
Self::COMMON_KEYS.iter().filter(|k| k.is_server_derived()).map(|k| k.name())
Self::COMMON_KEYS
.iter()
.filter(|k| k.is_server_derived())
.map(|k| k.name())
.chain([SVC_TARGET_USER])
}
/// Returns the policy variable form of the key, e.g. `${jwt:sub}`.
@@ -328,6 +335,8 @@ pub enum JwtKeyName {
pub enum SvcKeyName {
#[strum(serialize = "svc:DurationSeconds")]
SVCDurationSeconds,
#[strum(serialize = "svc:TargetUser")]
TargetUser,
}
#[derive(Clone, EnumString, Debug, IntoStaticStr, Eq, PartialEq, Serialize, Deserialize)]
+52
View File
@@ -644,6 +644,58 @@ mod test {
use crate::error::Result;
use crate::policy::action::{AdminAction, KmsAction, S3Action};
#[tokio::test]
async fn delegated_service_account_obeys_admin_wildcards_and_target_conditions() -> Result<()> {
let conditions = HashMap::from([("TargetUser".into(), vec!["backup".into()])]);
let args = Args {
account: "operator",
groups: &None,
action: Action::AdminAction(AdminAction::CreateServiceAccountForUserAdminAction),
bucket: "",
object: "",
conditions: &conditions,
claims: &HashMap::new(),
is_owner: false,
deny_only: false,
};
for (statement, allowed) in [
(serde_json::json!({"Effect":"Allow","Action":"admin:*"}), true),
(serde_json::json!({"Effect":"Allow","Action":"admin:CreateServiceAccount"}), false),
(serde_json::json!({"Effect":"Allow","NotAction":"admin:DeleteUser","Resource":"*"}), true),
(
serde_json::json!({"Effect":"Allow","NotAction":"admin:CreateServiceAccountForUser","Resource":"*"}),
false,
),
(
serde_json::json!({"Effect":"Allow","Action":"admin:CreateServiceAccountForUser","Condition":{"StringEquals":{"svc:TargetUser":"backup"}}}),
true,
),
(
serde_json::json!({"Effect":"Allow","Action":"admin:CreateServiceAccountForUser","Condition":{"StringEquals":{"svc:TargetUser":"archive"}}}),
false,
),
] {
let policy = Policy::parse_config(
&serde_json::to_vec(&serde_json::json!({"Version":DEFAULT_VERSION,"Statement":[statement]}))
.expect("encode policy"),
)?;
assert_eq!(policy.is_allowed(&args).await, allowed, "{statement}");
}
let admin = default::DEFAULT_POLICIES
.iter()
.find(|(name, _)| *name == "consoleAdmin")
.expect("built-in admin policy");
assert!(admin.1.is_allowed(&args).await);
for denied_action in ["admin:*", "admin:CreateServiceAccountForUser"] {
let deny = Policy::parse_config(&serde_json::to_vec(&serde_json::json!({
"Version": DEFAULT_VERSION,
"Statement": [{"Effect": "Deny", "Action": denied_action}]
}))?)?;
assert!(!Policy::merge_policies(vec![admin.1.clone(), deny]).is_allowed(&args).await);
}
Ok(())
}
#[tokio::test]
async fn test_parse_policy() -> Result<()> {
let data = r#"
+4 -4
View File
@@ -100,8 +100,8 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
json_field: "opts",
bin_field: "opts_bin",
},
json_encoder: "let opts = match serde_json::to_string(&opts) {",
policy: RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,
json_encoder: "serde_json::to_string(opts)?",
policy: RequestJsonPolicy::MsgpackOnlyEligible,
},
RequestCompatSendSite {
field: CompatPayloadField {
@@ -109,8 +109,8 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
json_field: "versions",
bin_field: "versions_bin",
},
json_encoder: "versions_str.push(match serde_json::to_string(file_info_versions) {",
policy: RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,
json_encoder: "versions_json.push(serde_json::to_string(version)?);",
policy: RequestJsonPolicy::MsgpackOnlyEligible,
},
RequestCompatSendSite {
field: CompatPayloadField {
@@ -913,8 +913,9 @@ pub struct DeleteVersionsRequest {
pub versions: ::prost::alloc::vec::Vec<::prost::alloc::string::String>,
#[prost(string, tag = "4")]
pub opts: ::prost::alloc::string::String,
/// msgpack payloads mirroring the JSON fields above (grpc-optimization P2). Senders dual-write
/// both; receivers prefer the \*\_bin form and fall back to the JSON string when it is empty.
/// Senders dual-write for legacy peers. A nonempty versions_bin list determines
/// the batch length and values. Malformed binary items reject the batch.
/// An empty binary list selects the legacy versions list. Options decode independently.
#[prost(bytes = "bytes", repeated, tag = "5")]
pub versions_bin: ::prost::alloc::vec::Vec<::prost::bytes::Bytes>,
#[prost(bytes = "bytes", tag = "6")]
@@ -1028,6 +1029,9 @@ pub struct GenerallyLockResponse {
/// Only populated by unlock RPCs.
#[prost(enumeration = "LockReleaseStatus", tag = "4")]
pub release_status: i32,
/// Fixed-width optional namespace mutation proof.
#[prost(bytes = "bytes", optional, tag = "6")]
pub cache_fence_stamp: ::core::option::Option<::prost::bytes::Bytes>,
}
#[allow(clippy::all)]
#[derive(Clone, PartialEq, Eq, Hash, ::prost::Message)]
@@ -1051,6 +1055,9 @@ pub struct GenerallyLockResult {
/// Only populated by lock batches.
#[prost(enumeration = "LockAcquireStatus", tag = "5")]
pub acquire_status: i32,
/// Fixed-width optional namespace mutation proof.
#[prost(bytes = "bytes", optional, tag = "6")]
pub cache_fence_stamp: ::core::option::Option<::prost::bytes::Bytes>,
}
#[allow(clippy::all)]
#[derive(Clone, PartialEq, ::prost::Message)]
+50 -2
View File
@@ -3657,6 +3657,11 @@ fn complete_pending_rebuild_on_cache_hit(dial_addr: &str, cache_key: &str) -> bo
// Limits transport response futures; client RPC metrics separately cover unary body decoding.
const INTERNODE_RPC_CONCURRENCY_LIMIT: usize = 64;
// Quorum reads cancel surplus metadata RPCs in bursts. Keep bounded reset state
// for their in-flight replies without spending the connection's protocol-error
// budget (h2 remembers only 50 resets by default).
const INTERNODE_RPC_MAX_CONCURRENT_RESET_STREAMS: usize = 4096;
/// Whether control/bulk channel isolation is enabled (env-gated, default off for safe rollout).
fn channel_isolation_enabled() -> bool {
rustfs_utils::get_env_bool(
@@ -4110,6 +4115,7 @@ async fn dial_channel(
let conn_window = checked_rpc_window(conn_window)?;
let mut endpoint = Endpoint::from_shared(dial_addr.to_string())?
.concurrency_limit(concurrency_limit)
.http2_max_concurrent_reset_streams(INTERNODE_RPC_MAX_CONCURRENT_RESET_STREAMS)
// HTTP/2 PING frames for application-layer health check
.http2_keep_alive_interval(http2_keepalive_interval)
// How long to wait for PING ACK before considering connection dead
@@ -4266,6 +4272,9 @@ pub async fn evict_failed_connection_with_reason(
}
}
#[cfg(test)]
mod reset_stream_tests;
#[cfg(test)]
mod tests {
use super::*;
@@ -4660,8 +4669,8 @@ mod tests {
),
("DeleteVersionRequest", "file_info", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,),
("DeleteVersionRequest", "opts", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero),
("DeleteVersionsRequest", "opts", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,),
("DeleteVersionsRequest", "versions", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,),
("DeleteVersionsRequest", "opts", RequestJsonPolicy::MsgpackOnlyEligible),
("DeleteVersionsRequest", "versions", RequestJsonPolicy::MsgpackOnlyEligible),
("ReadMultipleRequest", "read_multiple_req", RequestJsonPolicy::MsgpackOnlyEligible),
("ReadVersionRequest", "opts", RequestJsonPolicy::MsgpackOnlyEligible),
("RenameDataRequest", "file_info", RequestJsonPolicy::MsgpackOnlyEligible),
@@ -5871,6 +5880,7 @@ mod lock_release_tests {
error_info: error.clone(),
lock_info: None,
release_status: status.into(),
cache_fence_stamp: None,
};
let result = GenerallyLockResult {
success,
@@ -5926,4 +5936,42 @@ mod lock_release_tests {
assert_eq!(decoded.acquire_status, 99);
assert!(LockAcquireStatus::try_from(decoded.acquire_status).is_err());
}
#[test]
fn cache_fence_stamp_is_additive_for_single_and_batch_lock_responses() {
let stamp = prost::bytes::Bytes::from_static(b"fixed-width-cache-fence-proof");
let response = GenerallyLockResponse {
success: true,
cache_fence_stamp: Some(stamp.clone()),
..Default::default()
};
let result = GenerallyLockResult {
success: true,
cache_fence_stamp: Some(stamp),
..Default::default()
};
for encoded in [response.encode_to_vec(), result.encode_to_vec()] {
let legacy = LegacyReleaseResponse::decode(encoded.as_slice()).expect("old peer ignores additive cache proof");
assert!(legacy.success);
}
let legacy = LegacyReleaseResponse {
success: true,
error_info: None,
lock_info: None,
}
.encode_to_vec();
assert!(
GenerallyLockResponse::decode(legacy.as_slice())
.expect("new single response decodes legacy peer")
.cache_fence_stamp
.is_none()
);
assert!(
GenerallyLockResult::decode(legacy.as_slice())
.expect("new batch result decodes legacy peer")
.cache_fence_stamp
.is_none()
);
}
}
+5 -2
View File
@@ -583,8 +583,9 @@ message DeleteVersionsRequest {
string volume = 2;
repeated string versions = 3;
string opts = 4;
// msgpack payloads mirroring the JSON fields above (grpc-optimization P2). Senders dual-write
// both; receivers prefer the *_bin form and fall back to the JSON string when it is empty.
// Senders dual-write for legacy peers. A nonempty versions_bin list determines
// the batch length and values. Malformed binary items reject the batch.
// An empty binary list selects the legacy versions list. Options decode independently.
repeated bytes versions_bin = 5;
bytes opts_bin = 6;
}
@@ -662,6 +663,7 @@ message GenerallyLockResponse {
optional string error_info = 2;
optional string lock_info = 3; // JSON serialized LockInfo
LockReleaseStatus release_status = 4; // Only populated by unlock RPCs.
optional bytes cache_fence_stamp = 6; // Fixed-width optional namespace mutation proof.
}
message BatchGenerallyLockRequest {
@@ -681,6 +683,7 @@ message GenerallyLockResult {
optional string lock_info = 3; // JSON serialized LockInfo
LockReleaseStatus release_status = 4; // Per-ID result for unlock batches.
LockAcquireStatus acquire_status = 5; // Only populated by lock batches.
optional bytes cache_fence_stamp = 6; // Fixed-width optional namespace mutation proof.
}
message BatchGenerallyLockResponse {
+186
View File
@@ -0,0 +1,186 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::io;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::oneshot;
use tonic::codegen::{Service, http};
const CANCELLED_REQUESTS: usize = 64;
async fn write_frame(socket: &mut TcpStream, kind: u8, flags: u8, stream: u32, payload: &[u8]) -> io::Result<()> {
let length = u32::try_from(payload.len()).expect("fixture frame length");
let mut frame = Vec::with_capacity(9 + payload.len());
frame.extend_from_slice(&length.to_be_bytes()[1..]);
frame.extend_from_slice(&[kind, flags]);
frame.extend_from_slice(&stream.to_be_bytes());
frame.extend_from_slice(payload);
socket.write_all(&frame).await
}
async fn read_frame(socket: &mut TcpStream) -> io::Result<(u8, u8, u32, Vec<u8>)> {
let mut header = [0; 9];
socket.read_exact(&mut header).await?;
let length = u32::from_be_bytes([0, header[0], header[1], header[2]]);
let stream = u32::from_be_bytes(header[5..9].try_into().expect("stream ID bytes"));
let mut payload = vec![0; usize::try_from(length).expect("fixture frame length")];
socket.read_exact(&mut payload).await?;
Ok((header[3], header[4], stream, payload))
}
// A raw peer can release replies already in transit after observing CANCEL.
// An h2 server API would discard those replies upon receiving the reset.
async fn delayed_peer(
listener: TcpListener,
requests_seen: oneshot::Sender<()>,
resets_seen: oneshot::Sender<usize>,
) -> io::Result<usize> {
let (mut socket, _) = listener.accept().await?;
socket.set_nodelay(true)?;
let mut preface = [0; 24];
socket.read_exact(&mut preface).await?;
assert_eq!(&preface, b"PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n");
write_frame(&mut socket, 4, 0, 0, &[]).await?;
let mut requests = Vec::new();
while requests.len() < CANCELLED_REQUESTS {
let (kind, flags, stream, _) = read_frame(&mut socket).await?;
match (kind, flags) {
(4, 0) => write_frame(&mut socket, 4, 1, 0, &[]).await?,
(1, _) => requests.push(stream),
_ => {}
}
}
requests_seen.send(()).expect("caller waits for dispatched requests");
let mut cancellations = 0;
while cancellations < CANCELLED_REQUESTS {
let (kind, _, stream, payload) = read_frame(&mut socket).await?;
if kind == 3 {
assert!(requests.contains(&stream), "reset belongs to a dispatched request");
assert_eq!(payload, 8_u32.to_be_bytes(), "caller cancellation sends CANCEL");
cancellations += 1;
}
}
for stream in requests {
// HPACK static index 8 is ':status: 200'; END_HEADERS | END_STREAM.
write_frame(&mut socket, 1, 5, stream, &[0x88]).await?;
}
let ping = b"rst-test";
write_frame(&mut socket, 6, 0, 0, ping).await?;
let mut error_resets = 0;
loop {
let (kind, flags, _, payload) = read_frame(&mut socket).await?;
match (kind, flags) {
(3, _) => {
assert_eq!(payload, 5_u32.to_be_bytes(), "late reply rejected as STREAM_CLOSED");
error_resets += 1;
}
(6, 1) if payload == ping => break,
(7, _) => panic!("late replies closed the shared connection: {payload:?}"),
_ => {}
}
}
// PING acknowledgment establishes that all preceding replies were processed.
resets_seen.send(error_resets).expect("caller waits for reset accounting");
loop {
let (kind, _, stream, payload) = read_frame(&mut socket).await?;
if kind == 3 {
assert_eq!(payload, 5_u32.to_be_bytes());
error_resets += 1;
}
if kind == 1 {
write_frame(&mut socket, 1, 5, stream, &[0x88]).await?;
return Ok(error_resets);
}
}
}
async fn cancelled_metadata_replies_keep_channel_usable(bulk: bool) {
tokio::time::timeout(Duration::from_secs(10), async {
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind fixture peer");
let address = format!("http://{}", listener.local_addr().expect("fixture address"));
let key = if bulk {
super::bulk_cache_key(&address, 0)
} else {
address.clone()
};
let (requests_tx, requests_rx) = oneshot::channel();
let (resets_tx, resets_rx) = oneshot::channel();
let server = tokio::spawn(delayed_peer(listener, requests_tx, resets_tx));
let (channel, _) = super::dial_channel(&address, &key, Duration::from_secs(10))
.await
.expect("production channel connects");
let mut calls = Vec::new();
for _ in 0..CANCELLED_REQUESTS {
let mut channel = channel.clone();
let uri = format!("{address}/node_service.NodeService/ReadVersion");
calls.push(tokio::spawn(async move {
std::future::poll_fn(|cx| channel.poll_ready(cx))
.await
.expect("channel ready");
channel
.call(
http::Request::builder()
.uri(uri)
.body(tonic::body::Body::empty())
.expect("read request"),
)
.await
}));
}
requests_rx.await.expect("all metadata requests dispatched");
for call in &calls {
call.abort();
}
for call in calls {
assert!(call.await.expect_err("request cancelled before response").is_cancelled());
}
let _ = resets_rx.await.expect("peer accounts for late response frames");
let mut channel = channel;
std::future::poll_fn(|cx| channel.poll_ready(cx))
.await
.expect("channel remains ready");
let response = channel
.call(
http::Request::builder()
.uri(format!("{address}/node_service.NodeService/Ping"))
.body(tonic::body::Body::empty())
.expect("probe request"),
)
.await
.expect("subsequent RPC shares the connection");
assert_eq!(response.status(), http::StatusCode::OK);
let error_resets = server
.await
.expect("fixture task completes")
.expect("fixture transport succeeds");
assert_eq!(
error_resets, 0,
"cancelled metadata replies must not spend the connection's protocol-error budget"
);
})
.await
.expect("bounded cancellation exchange");
}
#[tokio::test]
async fn control_channel_tolerates_late_cancelled_metadata_replies() {
cancelled_metadata_replies_keep_channel_usable(false).await;
}
#[tokio::test]
async fn bulk_channel_tolerates_late_cancelled_metadata_replies() {
cancelled_metadata_replies_keep_channel_usable(true).await;
}
+168 -7
View File
@@ -411,13 +411,10 @@ impl ListObjectsOptions {
fn decode_s3_name(name: &str, encoding_type: &str) -> Result<String, std::io::Error> {
match encoding_type {
"url" => {
//return url::QueryUnescape(name);
return Ok(name.to_string());
}
_ => {
return Ok(name.to_string());
}
"url" => urlencoding::decode(name)
.map(|decoded| decoded.into_owned())
.map_err(|err| std::io::Error::new(ErrorKind::InvalidData, err)),
_ => Ok(name.to_string()),
}
}
@@ -449,6 +446,170 @@ mod tests {
}
}
#[test]
fn url_encoded_s3_names_are_decoded() {
for (encoded, expected) in [
("test_file%283%29.png", "test_file(3).png"),
("dir%20a/file.txt", "dir a/file.txt"),
("%E4%B8%AD%E6%96%87.txt", "中文.txt"),
("%F0%9F%98%80.txt", "😀.txt"),
("file%2bname+%25.txt", "file+name+%.txt"),
("literal%252F.txt", "literal%2F.txt"),
("dir%2Ffile%00%01.txt", "dir/file\0\u{1}.txt"),
("", ""),
] {
assert_eq!(decode_s3_name(encoded, "url").expect("valid URL-encoded S3 key"), expected);
}
}
#[test]
fn s3_names_without_url_encoding_are_preserved() {
for encoding in ["", "unknown"] {
let name = "中文+%20%FF.txt";
assert_eq!(decode_s3_name(name, encoding).expect("unencoded key should pass through"), name);
}
}
#[test]
fn url_encoded_s3_names_reject_invalid_utf8() {
for name in ["%FF", "%C3%28", "%E4%B8", "%ED%A0%80", "%C0%AF"] {
let err = decode_s3_name(name, "url").expect_err("invalid UTF-8 must not become a different object key");
assert_eq!(err.kind(), ErrorKind::InvalidData);
}
}
#[test]
fn url_encoded_s3_names_preserve_malformed_percent_escapes() {
// Match urlencoding's permissive handling without treating '+' as a space.
for name in ["file%", "file%2", "file%GG+name"] {
assert_eq!(decode_s3_name(name, "url").expect("literal malformed escape"), name);
}
}
async fn list_response_fixture(responses: Vec<&'static str>) -> (TransitionClient, tokio::task::JoinHandle<Vec<url::Url>>) {
let listener = TcpListener::bind("127.0.0.1:0").await.expect("list fixture should bind");
let endpoint = listener.local_addr().expect("list fixture address").to_string();
let fixture = tokio::spawn(async move {
tokio::time::timeout(Duration::from_secs(5), async move {
let mut requests = Vec::new();
for body in responses {
let (mut stream, _) = listener.accept().await.expect("list fixture should accept request");
let mut request = Vec::new();
let mut buffer = [0; 1024];
while !request.windows(4).any(|window| window == b"\r\n\r\n") {
let read = stream.read(&mut buffer).await.expect("list fixture should read request");
assert_ne!(read, 0, "request headers must be complete");
request.extend_from_slice(&buffer[..read]);
assert!(request.len() <= 16 * 1024, "fixture request headers should be bounded");
}
let request = std::str::from_utf8(&request).expect("request headers should be UTF-8");
let target = request.split_whitespace().nth(1).expect("request should have a target");
requests.push(url::Url::parse(&format!("http://localhost{target}")).expect("request URL should parse"));
let response = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/xml\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
);
stream.write_all(response.as_bytes()).await.expect("list fixture should send response");
}
requests
})
.await
.expect("list fixture should finish within its deadline")
});
let client = TransitionClient::new_with_timeouts(
&endpoint,
timeout_test_options(),
"",
TransitionClientTimeouts::new(Duration::from_secs(1), Duration::from_secs(1), Duration::from_secs(1)),
)
.await
.expect("list fixture client should build");
client
.bucket_loc_cache
.lock()
.expect("location cache should lock")
.set("bucket", "us-east-1");
(client, fixture)
}
#[tokio::test]
async fn list_objects_v2_decodes_prefixes_and_preserves_continuation_tokens() {
let (client, fixture) = list_response_fixture(vec![
r#"<ListBucketResult><EncodingType>url</EncodingType><IsTruncated>true</IsTruncated>
<CommonPrefixes><Prefix>dir%20a/</Prefix></CommonPrefixes>
<NextContinuationToken>opaque%2F+token</NextContinuationToken></ListBucketResult>"#,
"<ListBucketResult><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated></ListBucketResult>",
])
.await;
let first = client
.list_objects_v2_query("bucket", "", "", false, false, "/", "", 1, HeaderMap::new())
.await
.expect("first object page should decode");
assert_eq!(first.common_prefixes[0].prefix, "dir a/");
assert_eq!(first.next_continuation_token, "opaque%2F+token");
let second = client
.list_objects_v2_query("bucket", "", &first.next_continuation_token, false, false, "/", "", 1, HeaderMap::new())
.await
.expect("second object page should succeed");
assert!(!second.is_truncated);
let requests = fixture.await.expect("object list fixture should join");
let query: HashMap<_, _> = requests[1].query_pairs().into_owned().collect();
assert_eq!(query.get("continuation-token").map(String::as_str), Some("opaque%2F+token"));
assert_eq!(query.get("encoding-type").map(String::as_str), Some("url"));
}
#[tokio::test]
async fn list_versions_decodes_names_and_round_trips_pagination_markers() {
let (client, fixture) = list_response_fixture(vec![
r#"<ListVersionsResult><EncodingType>url</EncodingType><IsTruncated>true</IsTruncated>
<Version><Key>dir%20a/%E4%B8%AD%E6%96%87%2B%252F.txt</Key><VersionId>version%2F+id</VersionId></Version>
<DeleteMarker><Key>deleted%20file%2B.txt</Key><VersionId>delete%2F+id</VersionId></DeleteMarker>
<CommonPrefixes><Prefix>dir%20a/</Prefix></CommonPrefixes>
<NextKeyMarker>dir%20a/%E4%B8%AD%E6%96%87%2B%252F.txt</NextKeyMarker>
<NextVersionIdMarker>version%2F+id</NextVersionIdMarker></ListVersionsResult>"#,
"<ListVersionsResult><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated></ListVersionsResult>",
])
.await;
let mut opts = ListObjectsOptions::default();
opts.set("prefix", "dir a/");
let first = client
.list_object_versions_query("bucket", &opts, "", "", "/")
.await
.expect("first version page");
assert_eq!(first.versions[0].key, "dir a/中文+%2F.txt");
assert_eq!(first.versions[0].version_id, "version%2F+id");
assert_eq!(first.delete_markers[0].key, "deleted file+.txt");
assert_eq!(first.delete_markers[0].version_id, "delete%2F+id");
assert_eq!(first.common_prefixes[0].prefix, "dir a/");
assert_eq!(first.next_key_marker, "dir a/中文+%2F.txt");
assert_eq!(first.next_version_id_marker, "version%2F+id");
let second = client
.list_object_versions_query("bucket", &opts, &first.next_key_marker, &first.next_version_id_marker, "/")
.await
.expect("second version page");
assert!(!second.is_truncated);
let requests = fixture.await.expect("version list fixture should join");
let query: HashMap<_, _> = requests[1].query_pairs().into_owned().collect();
assert_eq!(query.get("key-marker").map(String::as_str), Some("dir a/中文+%2F.txt"));
assert_eq!(query.get("version-id-marker").map(String::as_str), Some("version%2F+id"));
assert_eq!(query.get("prefix").map(String::as_str), Some("dir a/"));
assert_eq!(query.get("encoding-type").map(String::as_str), Some("url"));
}
#[tokio::test]
async fn list_versions_rejects_invalid_utf8_keys() {
let (client, fixture) = list_response_fixture(vec![
"<ListVersionsResult><EncodingType>url</EncodingType><Version><Key>%FF</Key></Version></ListVersionsResult>",
])
.await;
let err = client
.list_object_versions_query("bucket", &ListObjectsOptions::default(), "", "", "")
.await
.expect_err("invalid UTF-8 in a remote key should fail the listing");
assert_eq!(err.kind(), ErrorKind::InvalidData);
assert_eq!(fixture.await.expect("invalid key fixture should join").len(), 1);
}
#[test]
fn list_versions_xml_preserves_versions_and_delete_markers() {
let xml = br#"
+50 -1
View File
@@ -83,6 +83,14 @@ pub enum S3Operation {
SelectObjectContent,
UploadPart,
UploadPartCopy,
GetBucketAccelerateConfiguration,
GetBucketRequestPayment,
GetBucketWebsite,
PutBucketAccelerateConfiguration,
PutBucketRequestPayment,
PutBucketWebsite,
DeleteBucketWebsite,
PostObject,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -126,7 +134,7 @@ impl EventMapping {
}
impl S3Operation {
pub const ALL: [Self; 66] = [
pub const ALL: [Self; 74] = [
Self::AbortMultipartUpload,
Self::CompleteMultipartUpload,
Self::CopyObject,
@@ -193,6 +201,14 @@ impl S3Operation {
Self::SelectObjectContent,
Self::UploadPart,
Self::UploadPartCopy,
Self::GetBucketAccelerateConfiguration,
Self::GetBucketRequestPayment,
Self::GetBucketWebsite,
Self::PutBucketAccelerateConfiguration,
Self::PutBucketRequestPayment,
Self::PutBucketWebsite,
Self::DeleteBucketWebsite,
Self::PostObject,
];
#[inline]
@@ -214,10 +230,12 @@ impl S3Operation {
Self::DeleteBucketPolicy => "s3:DeleteBucketPolicy",
Self::DeleteBucketReplication => "s3:DeleteBucketReplication",
Self::DeleteBucketTagging => "s3:DeleteBucketTagging",
Self::DeleteBucketWebsite => "s3:DeleteBucketWebsite",
Self::DeleteObject => "s3:DeleteObject",
Self::DeleteObjectTagging => "s3:DeleteObjectTagging",
Self::DeleteObjects => "s3:DeleteObjects",
Self::DeletePublicAccessBlock => "s3:DeletePublicAccessBlock",
Self::GetBucketAccelerateConfiguration => "s3:GetBucketAccelerateConfiguration",
Self::GetBucketAcl => "s3:GetBucketAcl",
Self::GetBucketCors => "s3:GetBucketCors",
Self::GetBucketEncryption => "s3:GetBucketEncryption",
@@ -228,8 +246,10 @@ impl S3Operation {
Self::GetBucketPolicy => "s3:GetBucketPolicy",
Self::GetBucketPolicyStatus => "s3:GetBucketPolicyStatus",
Self::GetBucketReplication => "s3:GetBucketReplication",
Self::GetBucketRequestPayment => "s3:GetBucketRequestPayment",
Self::GetBucketTagging => "s3:GetBucketTagging",
Self::GetBucketVersioning => "s3:GetBucketVersioning",
Self::GetBucketWebsite => "s3:GetBucketWebsite",
Self::GetObject => "s3:GetObject",
Self::GetObjectAcl => "s3:GetObjectAcl",
Self::GetObjectAttributes => "s3:GetObjectAttributes",
@@ -247,6 +267,7 @@ impl S3Operation {
Self::ListObjects => "s3:ListObjects",
Self::ListObjectsV2 => "s3:ListObjectsV2",
Self::ListParts => "s3:ListParts",
Self::PutBucketAccelerateConfiguration => "s3:PutBucketAccelerateConfiguration",
Self::PutBucketAcl => "s3:PutBucketAcl",
Self::PutBucketCors => "s3:PutBucketCors",
Self::PutBucketEncryption => "s3:PutBucketEncryption",
@@ -255,8 +276,11 @@ impl S3Operation {
Self::PutBucketNotificationConfiguration => "s3:PutBucketNotificationConfiguration",
Self::PutBucketPolicy => "s3:PutBucketPolicy",
Self::PutBucketReplication => "s3:PutBucketReplication",
Self::PutBucketRequestPayment => "s3:PutBucketRequestPayment",
Self::PutBucketTagging => "s3:PutBucketTagging",
Self::PutBucketVersioning => "s3:PutBucketVersioning",
Self::PutBucketWebsite => "s3:PutBucketWebsite",
Self::PostObject => "s3:PostObject",
Self::PutObject => "s3:PutObject",
Self::PutObjectAcl => "s3:PutObjectAcl",
Self::PutObjectLegalHold => "s3:PutObjectLegalHold",
@@ -288,6 +312,7 @@ impl S3Operation {
Self::GetObjectLegalHold => EventMapping::Single(EventName::ObjectAccessedGetLegalHold),
Self::GetObjectRetention => EventMapping::Single(EventName::ObjectAccessedGetRetention),
Self::HeadObject => EventMapping::Single(EventName::ObjectAccessedHead),
Self::PostObject => EventMapping::Single(EventName::ObjectCreatedPost),
Self::PutObject => EventMapping::PutObject,
Self::PutObjectAcl => EventMapping::Single(EventName::ObjectAclPut),
Self::PutObjectLegalHold => EventMapping::Single(EventName::ObjectCreatedPutLegalHold),
@@ -301,7 +326,9 @@ impl S3Operation {
| Self::DeleteBucketPolicy
| Self::DeleteBucketReplication
| Self::DeleteBucketTagging
| Self::DeleteBucketWebsite
| Self::DeletePublicAccessBlock
| Self::GetBucketAccelerateConfiguration
| Self::GetBucketAcl
| Self::GetBucketCors
| Self::GetBucketEncryption
@@ -312,8 +339,10 @@ impl S3Operation {
| Self::GetBucketPolicy
| Self::GetBucketPolicyStatus
| Self::GetBucketReplication
| Self::GetBucketRequestPayment
| Self::GetBucketTagging
| Self::GetBucketVersioning
| Self::GetBucketWebsite
| Self::GetObjectAcl
| Self::GetObjectLockConfiguration
| Self::GetObjectTagging
@@ -326,6 +355,7 @@ impl S3Operation {
| Self::ListObjects
| Self::ListObjectsV2
| Self::ListParts
| Self::PutBucketAccelerateConfiguration
| Self::PutBucketAcl
| Self::PutBucketCors
| Self::PutBucketEncryption
@@ -334,8 +364,10 @@ impl S3Operation {
| Self::PutBucketNotificationConfiguration
| Self::PutBucketPolicy
| Self::PutBucketReplication
| Self::PutBucketRequestPayment
| Self::PutBucketTagging
| Self::PutBucketVersioning
| Self::PutBucketWebsite
| Self::PutObjectLockConfiguration
| Self::PutPublicAccessBlock
| Self::UploadPart
@@ -516,12 +548,29 @@ mod tests {
assert_eq!(mask & EventName::ObjectRemovedDelete.mask(), 0);
}
#[test]
fn metric_indexes_and_labels_identify_each_operation() {
let mut labels = std::collections::HashSet::new();
for (index, op) in S3Operation::ALL.into_iter().enumerate() {
assert_eq!(op.metric_index(), index, "metric index for {op:?}");
assert!(labels.insert(op.as_str()), "repeated metric label for {op:?}");
}
assert_eq!(S3Operation::UploadPartCopy.metric_index(), 65);
}
#[test]
fn test_operations_without_event_mapping_remain_unmapped() {
let unmapped = [
S3Operation::DeleteBucketWebsite,
S3Operation::GetBucketAccelerateConfiguration,
S3Operation::GetBucketAcl,
S3Operation::GetBucketRequestPayment,
S3Operation::GetBucketWebsite,
S3Operation::ListObjectsV2,
S3Operation::PutBucketAccelerateConfiguration,
S3Operation::PutBucketNotificationConfiguration,
S3Operation::PutBucketRequestPayment,
S3Operation::PutBucketWebsite,
S3Operation::UploadPart,
];
for op in unmapped {
@@ -740,6 +740,7 @@ async fn scanner_recovery_intent_startup_rejects_corrupt_pending_record() {
save_config(store.clone(), &path, b"{corrupt".to_vec())
.await
.expect("corrupt intent payload should persist");
crate::scanner_io::tests::wait_for_namespace_commit_tails(store.as_ref()).await;
let restarted = restart_scanner_cycle_store_from(&store).await;
let error = scanner_usage_recovery_intents_for_startup(&CancellationToken::new(), restarted)
+1 -1
View File
@@ -109,7 +109,7 @@ pub use bucket::{BucketInfo, BucketOperations, BucketOptions, DeleteBucketOption
pub use capability::{CapabilitySnapshotError, CapabilityState, CapabilityStatus};
pub use error::{StorageErrorCode, StorageResult};
pub use multipart::{CompletePart, ListMultipartsInfo, ListPartsInfo, MultipartInfo, MultipartUploadResult, PartInfo};
pub use multipart::{MultipartCompletionAuthorization, MultipartUploadInspection};
pub use multipart::{MultipartAbortAuthorization, MultipartCompletionAuthorization, MultipartOwner, MultipartUploadInspection};
pub use object::AuthoritativeListOperations;
pub use object::DeleteAccounting;
pub use object::ObjectLockDeleteOptions;
+28
View File
@@ -15,6 +15,24 @@
use std::collections::HashMap;
use time::OffsetDateTime;
/// Ownership observed in the selected upload metadata.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct MultipartOwner {
/// The bucket recorded by the upload.
pub bucket: String,
/// The recorded object, decoded after checking the encoded storage namespace.
pub object: String,
}
/// Authorizes Abort against the upload selected under its existing write lock.
/// An implementation must reject missing ownership and must not acquire storage locks or mutate storage.
pub trait MultipartAbortAuthorization: Send + Sync {
type Error: std::error::Error + Send + Sync + 'static;
/// Returns a refusal before storage prepares or deletes any staged data.
fn authorize(&self, owner: Option<&MultipartOwner>) -> Result<(), Self::Error>;
}
/// Request authorization against the source selected under completion locks.
/// Implementations must not mutate storage or acquire object/upload locks again.
/// Called before comparing client checksums/parts or acknowledging a replay.
@@ -73,8 +91,18 @@ pub struct ListMultipartsInfo {
pub common_prefixes: Vec<String>,
}
/// The owner found in the upload metadata used to read a ListParts page.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ListPartsOwner {
pub bucket: String,
/// The recorded object after decoding the storage namespace spelling.
pub object: String,
}
#[derive(Debug, Clone, Default)]
pub struct ListPartsInfo {
/// Present when the recorded bucket and encoded object agree with the storage namespace.
pub recorded_owner: Option<ListPartsOwner>,
pub bucket: String,
pub object: String,
pub upload_id: String,
@@ -0,0 +1,63 @@
// Copyright 2026 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use rustfs_storage_api::multipart::{ListPartsInfo, ListPartsOwner, MultipartInfo, MultipartOwner};
use std::any::TypeId;
use std::collections::HashMap;
trait DownstreamOwner {
fn kind(&self) -> &'static str;
}
impl DownstreamOwner for ListPartsOwner {
fn kind(&self) -> &'static str {
"list-parts"
}
}
impl DownstreamOwner for MultipartOwner {
fn kind(&self) -> &'static str {
"multipart"
}
}
#[test]
fn multipart_owner_types_remain_distinct_for_downstream_implementations() {
let list_owner = ListPartsOwner {
bucket: "bucket".to_owned(),
object: "key".to_owned(),
};
let multipart_owner = MultipartOwner {
bucket: "bucket".to_owned(),
object: "key".to_owned(),
};
assert_eq!(list_owner.kind(), "list-parts");
assert_eq!(multipart_owner.kind(), "multipart");
assert_ne!(TypeId::of::<ListPartsOwner>(), TypeId::of::<MultipartOwner>());
let page = ListPartsInfo {
recorded_owner: Some(list_owner.clone()),
..Default::default()
};
assert_eq!(page.recorded_owner, Some(list_owner));
}
#[test]
fn multipart_info_retains_exhaustive_literal_and_pattern() {
let info = MultipartInfo {
bucket: "bucket".to_owned(),
object: "key".to_owned(),
upload_id: "upload".to_owned(),
initiated: None,
user_defined: HashMap::new(),
};
let MultipartInfo {
bucket,
object,
upload_id,
initiated,
user_defined,
} = info;
assert_eq!((bucket.as_str(), object.as_str(), upload_id.as_str()), ("bucket", "key", "upload"));
assert!(initiated.is_none());
assert!(user_defined.is_empty());
}
+76 -6
View File
@@ -24,7 +24,7 @@ use crate::{
store::{Key, Store},
target::{
ChannelTargetType, EntityTarget, QueuedPayload, QueuedPayloadMeta, TargetDeliveryCounters, TargetDeliverySnapshot,
TargetTlsState, TargetType, build_queued_payload, build_target_tls_fingerprint, is_connectivity_error,
TargetTlsState, TargetType, build_event_key, build_queued_payload, build_target_tls_fingerprint, is_connectivity_error,
open_target_queue_store, persist_queued_payload_to_store,
},
};
@@ -291,6 +291,16 @@ impl KafkaArgs {
}
}
fn build_kafka_record<'a>(
topic: &'a str,
body: &'a [u8],
meta: &QueuedPayloadMeta,
) -> Result<Record<'a, String, &'a [u8]>, TargetError> {
// Queue metadata carries the form-encoded S3 key, including on replay.
let key = build_event_key(&meta.bucket_name, &meta.object_name)?;
Ok(Record::from_key_value(topic, key, body))
}
/// A target that sends events to an Apache Kafka topic
pub struct KafkaTarget<E>
where
@@ -459,6 +469,8 @@ where
"Sending Kafka payload"
);
let record = build_kafka_record(&self.args.topic, &body, meta)?;
// rustfs-kafka-async does not validate response correlation IDs. Keep
// producer selection, send, and timeout invalidation serialized so a
// waiter cannot reuse a connection with an unread timed-out response.
@@ -467,12 +479,8 @@ where
&self.delivery_poisoned,
|| self.get_or_build_producer(),
|producer| async move {
// Use "<bucket>/<object>" as the message key so all events for the same
// object hash to the same partition and preserve per-object ordering
// across multiple partitions (backlog#983).
let partition_key = format!("{}/{}", meta.bucket_name, meta.object_name);
producer
.send(&Record::from_key_value(&self.args.topic, partition_key, body.as_slice()))
.send(&record)
.await
.map_err(|err| Self::map_kafka_error(err, "Failed to send message to Kafka"))
},
@@ -661,6 +669,68 @@ mod tests {
}
}
#[test]
fn kafka_record_key_matches_envelope_for_live_and_queued_events() {
let target = KafkaTarget::<serde_json::Value>::new("test".to_string(), base_args())
.expect("Kafka target configuration should be valid");
let cases = [
("file.txt", "file.txt"),
("test-ns%2Fsflm%2FusecaseB%2Ffile.txt", "test-ns/sflm/usecaseB/file.txt"),
("red+flower.jpg", "red flower.jpg"),
("red%20flower.jpg", "red flower.jpg"),
("a%2Bb%2Fc.txt", "a+b/c.txt"),
("literal%252F%2B%252B%2B%25.txt", "literal%2F+%2B+%.txt"),
("%E7%9B%AE%E5%BD%95%2F%E6%96%87%E4%BB%B6.txt", "目录/文件.txt"),
("%2Fleading%2F", "/leading/"),
("", ""),
];
for (encoded, decoded) in cases {
let event = EntityTarget {
object_name: encoded.to_string(),
bucket_name: "test-bucket".to_string(),
event_name: crate::EventName::ObjectCreatedPut,
data: serde_json::json!({"s3": {"object": {"key": encoded}}}),
};
let live = target.build_queued_payload(&event).expect("event should serialize");
let stored = live.encode().expect("queued payload should encode");
let replay = QueuedPayload::decode(&stored).expect("existing queue format should decode");
for queued in [&live, &replay] {
let record =
build_kafka_record(&target.args.topic, &queued.body, &queued.meta).expect("Kafka record should build");
let envelope: serde_json::Value = serde_json::from_slice(record.value).expect("record body should be JSON");
assert_eq!(record.key, format!("test-bucket/{decoded}"), "encoded object name: {encoded}");
assert_eq!(envelope["Key"], record.key);
assert_eq!(envelope["Records"][0]["s3"]["object"]["key"], encoded);
assert_eq!(queued.meta.object_name, encoded);
assert_eq!(record.topic, "rustfs-events");
assert_eq!(record.value.as_ptr(), queued.body.as_ptr(), "record should borrow the body");
}
}
}
#[tokio::test]
async fn kafka_record_rejects_invalid_utf8_before_delivery() {
let target = KafkaTarget::<serde_json::Value>::new("test".to_string(), base_args())
.expect("Kafka target configuration should be valid");
let meta = QueuedPayloadMeta::new(
crate::EventName::ObjectCreatedPut,
"test-bucket".to_string(),
"%FF".to_string(),
"application/json",
2,
);
let queued = QueuedPayload::new(meta, b"{}".to_vec());
let stored = queued.encode().expect("corrupt event metadata should serialize");
let replay = QueuedPayload::decode(&stored).expect("queue framing should decode");
let error = target
.send_body(replay.body, &replay.meta)
.await
.expect_err("invalid UTF-8 should reject delivery");
assert!(matches!(error, TargetError::Encoding(_)), "unexpected error: {error}");
assert!(target.producer.lock().await.is_none());
assert!(!target.delivery_poisoned.load(Ordering::Acquire));
}
#[tokio::test(start_paused = true)]
async fn timeout_invalidates_before_the_next_delivery_selects_a_producer() {
let delivery_lock = Arc::new(Mutex::new(()));
+102 -8
View File
@@ -19,6 +19,7 @@ use crate::{StoreError, TargetError, TargetLog};
use async_trait::async_trait;
use rustfs_s3_types::EventName;
use serde::{Deserialize, Serialize};
use std::borrow::Cow;
use std::cell::Cell;
use std::fmt::Formatter;
use std::future::Future;
@@ -673,8 +674,8 @@ pub(crate) fn queue_store_subdir_name(target_type: &str, target_id: &str) -> Str
/// Decodes a form-urlencoded object name to its original form.
///
/// This function properly handles form-urlencoded strings where spaces are
/// represented as `+` symbols. It first replaces `+` with spaces, then
/// performs standard percent-decoding.
/// represented as `+` symbols. Percent-encoded plus signs (`%2B`) remain
/// literal plus signs, and percent escapes are decoded once.
///
/// # Arguments
/// * `encoded` - The form-urlencoded string to decode
@@ -691,10 +692,30 @@ pub(crate) fn queue_store_subdir_name(target_type: &str, target_id: &str) -> Str
/// assert_eq!(decoded, "greeting file (2).csv");
/// ```
pub fn decode_object_name(encoded: &str) -> Result<String, TargetError> {
let replaced = encoded.replace("+", " ");
urlencoding::decode(&replaced)
.map(|s| s.into_owned())
.map_err(|e| TargetError::Encoding(format!("Failed to decode object key: {e}")))
decode_object_name_cow(encoded).map(Cow::into_owned)
}
fn decode_object_name_cow(encoded: &str) -> Result<Cow<'_, str>, TargetError> {
let form = if encoded.contains('+') {
Cow::Owned(encoded.replace('+', " "))
} else {
Cow::Borrowed(encoded)
};
let decoded =
urlencoding::decode(&form).map_err(|err| TargetError::Encoding(format!("Failed to decode object key: {err}")))?;
Ok(match decoded {
Cow::Borrowed(_) => form,
Cow::Owned(name) => Cow::Owned(name),
})
}
pub(crate) fn build_event_key(bucket: &str, encoded_object: &str) -> Result<String, TargetError> {
let object = decode_object_name_cow(encoded_object)?;
let mut key = String::with_capacity(bucket.len() + 1 + object.len());
key.push_str(bucket);
key.push('/');
key.push_str(&object);
Ok(key)
}
pub(crate) fn build_queued_payload<E>(event: &EntityTarget<E>) -> Result<QueuedPayload, TargetError>
@@ -712,8 +733,7 @@ where
E: PluginEvent,
R: Serialize,
{
let object_name = decode_object_name(&event.object_name)?;
let key = format!("{}/{}", event.bucket_name, object_name);
let key = build_event_key(&event.bucket_name, &event.object_name)?;
let log = TargetLog {
event_name: event.event_name,
@@ -1214,6 +1234,80 @@ mod tests {
assert_eq!(decoded.body, br#"{"ok":true}"#);
}
fn assert_object_name_matches_legacy_decoder(encoded: &str) {
let form = encoded.replace('+', " ");
match urlencoding::decode(&form) {
Ok(expected) => {
assert_eq!(decode_object_name(encoded).expect("object should decode"), expected, "input: {encoded:?}");
let key = build_event_key("bucket%2F+", encoded).expect("event key should build");
assert_eq!(key, format!("bucket%2F+/{expected}"), "input: {encoded:?}");
}
Err(_) => {
assert!(matches!(decode_object_name(encoded), Err(TargetError::Encoding(_))), "input: {encoded:?}");
assert!(
matches!(build_event_key("bucket", encoded), Err(TargetError::Encoding(_))),
"input: {encoded:?}"
);
}
}
}
#[test]
fn object_name_decoder_preserves_legacy_escape_and_utf8_semantics() {
for first in 0u8..=127 {
for second in 0u8..=127 {
let encoded = format!("prefix+%{}{}%2B%252F/目录", char::from(first), char::from(second));
assert_object_name_matches_legacy_decoder(&encoded);
}
}
for byte in 0u8..=255 {
assert_object_name_matches_legacy_decoder(&format!("prefix/%{byte:02X}+suffix"));
}
for encoded in [
"",
"plain.txt",
"目录/文件.txt",
"a&b=c?#.txt",
"%",
"%2",
"%GG",
"%2B",
"%252B",
"%252F",
"%+1",
"%1+",
"%%32",
"%2%32",
"+++",
"%00",
"%C3%A9",
"%E7%9B%AE%E5%BD%95",
"%F0%9F%98%80",
"%C0%AF",
"%ED%A0%80",
"%F4%90%80%80",
"%C3",
"%C3+%A9",
] {
assert_object_name_matches_legacy_decoder(encoded);
}
}
#[test]
fn object_name_decoder_borrows_plain_names() {
let plain = "directory/file.txt";
let decoded = decode_object_name_cow(plain).expect("plain name should decode");
assert!(matches!(decoded, Cow::Borrowed(_)));
assert_eq!(decoded.as_ptr(), plain.as_ptr());
for (encoded, expected) in [("red+flower.jpg", "red flower.jpg"), ("a%2Bb", "a+b"), ("a%2Bb+c", "a+b c")] {
let decoded = decode_object_name_cow(encoded).expect("encoded name should decode");
assert!(matches!(decoded, Cow::Owned(_)));
assert_eq!(decoded, expected);
}
let key = build_event_key("test-bucket", "directory%2Ffile.txt").expect("event key should build");
assert_eq!(key, "test-bucket/directory/file.txt");
}
#[test]
fn build_queued_payload_uses_event_data_shape() {
let event = EntityTarget {
+6 -4
View File
@@ -50,7 +50,7 @@ Two codec backends exist, selected per object from its metadata (never a runtime
- **Modern backend** — Reed–Solomon over **GF(2⁸)** using `rustfs-erasure-codec` (a RustFS fork of `reed-solomon-erasure` v8, `Cargo.toml`), imported as `reed_solomon_erasure::galois_8::ReedSolomon` ([erasure.rs](../../crates/ecstore/src/erasure/coding/erasure.rs)). Vandermonde generator matrix; algorithm string `"rs-vandermonde"` ([object_api/mod.rs](../../crates/ecstore/src/object_api/mod.rs)). The GF(2⁸) field bounds total shards per set far above the geometry cap of 16 (§2). Used for **all new writes** — and, because MinIO uses the same `rs-vandermonde` GF(2⁸) scheme, for **all MinIO-migrated objects** too.
- **Legacy backend** — Reed–Solomon over **GF(2¹⁶)** using `reed-solomon-simd` v3.1 (`Cargo.toml`), imported at [erasure.rs](../../crates/ecstore/src/erasure/coding/erasure.rs). Used **only** to read and heal objects written in **RustFS's own older ("main branch") format** — the `rmp_serde`-serialized layout detected by `uses_legacy_checksum` (see §11). This backend is **not** MinIO-compatible; MinIO-migrated data is decoded by the modern GF(2⁸) backend above (see [minio-file-format-compat.md](minio-file-format-compat.md)). The backend is chosen per object from its metadata (`uses_legacy_checksum`), never by a runtime toggle.
Industry alignment (all confirmed in code): byte-oriented RS over GF(2⁸) with a Vandermonde matrix, 1 MiB erasure block, and HighwayHash-256 bitrot checksums with a π-derived key — the same family and defaults MinIO uses. This is what makes byte-level `xl.meta` interoperability with MinIO possible (see [minio-file-format-compat.md](minio-file-format-compat.md)).
Industry alignment (all confirmed in code): byte-oriented RS over GF(2⁸) with a Vandermonde matrix and HighwayHash-256 bitrot checksums with a π-derived key share the format family used by MinIO. RustFS defaults to a 1 MiB erasure block. Reads use each object’s persisted block geometry. The supported byte-level `xl.meta` interoperability is defined by [minio-file-format-compat.md](minio-file-format-compat.md).
Where the code lives: the erasure engine is owned by `crates/ecstore/src/erasure/` and is crate-private; the `xl.meta` model is owned by `crates/filemeta`. See [ecstore-layout-boundary.md](ecstore-layout-boundary.md).
@@ -140,7 +140,7 @@ distribution[i-1] = 1 + ((start + i) % N) for i in 1..=N // a cyclic rotatio
### 4.3 Streaming pipeline and fan-out
`Erasure::encode` / `encode_batched` ([encode.rs](../../crates/ecstore/src/erasure/coding/encode.rs)) read the source in `block_size` chunks on a producer task, encode each block, and stream the `N` shards of each block over a bounded channel to a consumer that fans them out through `MultiWriter` to the `N` shard writers. `block_size == 0` is rejected up front (`InvalidInput`). In-flight memory is bounded by the channel depth (default budget 32 MiB). A clean EOF stops the loop; the final partial block is padded per §4.2.
`Erasure::encode` / `encode_batched` ([encode.rs](../../crates/ecstore/src/erasure/coding/encode.rs)) read the source in `block_size` chunks on a producer task, encode each block, and stream the `N` shards of each block over a bounded channel to a consumer that fans them out through `MultiWriter` to the `N` shard writers. `block_size == 0` is rejected up front (`InvalidInput`). Channel capacity is derived from the queue budget and the EC-expanded block size, with at least one slot. The default queue budget is 32 MiB. Producer, consumer, ingest, writer, and allocator buffers contribute additional memory, and a single expanded block can exceed that budget. Batched mode also owns pending batches. Queue accounting describes queued blocks. Per-PUT and process RSS also depend on those additional owners. A clean EOF stops the loop; the final partial block is padded per §4.2.
- **INVARIANT — write-quorum enforcement during encode.** `MultiWriter` writes shard `i` to writer `i`, drops any stalled/failed/short writer (sets it to `None`), and after each block requires `nil_count ≥ write_quorum`, else fails with a reduced-write-quorum error ([encode.rs](../../crates/ecstore/src/erasure/coding/encode.rs)). Shard index → writer index → on-disk position is fixed.
@@ -292,10 +292,12 @@ Fields: `version_id`, `mod_time`, `signature: [u8;4]`, `version_type`, `flags: u
### 6.6 Inline data
Small objects store their payload inline after the container CRC ([filemeta_inline.rs](../../crates/filemeta/src/filemeta_inline.rs)): 1 version byte (`INLINE_DATA_VER = 1`) then a msgpack map of `version-key → bin`. **INVARIANT — the map key** is the version-id string, `"null"` (`NULL_VERSION_ID`, [fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)) for the null/None version, else the lowercase hyphenated UUID. Presence is determined **on read** solely by the `meta_sys[inline-data]` body marker (`FileInfo::inline_data`); the read path gates inline extraction on that marker alone. The header `InlineData` flag is **written** (mirrored from the body on marshal) but is **not** consulted on read, and a disagreement is tolerated — MinIO may leave the header flag unset while inline data is present, so a reader must **not** require the flag and the marker to agree. The inline threshold is `should_inline` ([storageclass.rs](../../crates/ecstore/src/config/storageclass.rs)): inline if `shard_size ≤ inline_block/8` for versioned buckets, else `≤ inline_block`; `DEFAULT_INLINE_BLOCK = 128 KiB`. A compressed or encrypted single PUT has no known stored size up front, so its shard size is derived from the plaintext `actual_size` (`inline_admission_shard_size`, MinIO `putObject` parity); such objects are inlined through the streaming encoder with in-memory bitrot writers rather than the single-block fast path. When shard-integrity protected writes are enabled, inline placement additionally requires a known stored length no greater than one erasure block; unknown transformed streams retain external shards and proof indexes.
Small objects store their payload inline after the container CRC ([filemeta_inline.rs](../../crates/filemeta/src/filemeta_inline.rs)): 1 version byte (`INLINE_DATA_VER = 1`) then a msgpack map of `version-key → bin`. **INVARIANT — the map key** is the version-id string, `"null"` (`NULL_VERSION_ID`, [fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)) for the null/None version, else the lowercase hyphenated UUID. Presence is determined **on read** solely by the `meta_sys[inline-data]` body marker (`FileInfo::inline_data`); the read path gates inline extraction on that marker alone. The header `InlineData` flag is **written** (mirrored from the body on marshal) but is **not** consulted on read, and a disagreement is tolerated — MinIO may leave the header flag unset while inline data is present, so a reader must **not** require the flag and the marker to agree. `should_inline` ([storageclass.rs](../../crates/ecstore/src/config/storageclass.rs)) admits a shard up to `inline_block/8` for versioned buckets and up to `inline_block` otherwise. The default is a fixed `DEFAULT_INLINE_BLOCK = 128 KiB` per shard for every EC geometry, matching MinIO. Explicit `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` values use the same per-shard interpretation. Known-length unprotected PUTs may span multiple erasure blocks within a 2 MiB object limit and 32-frame limit, subject to the accumulated per-shard budget. The default block remains 1 MiB. The direct encoder remains single-block. Transformed streams and proof-protected writes retain their single-block admission bound. Known-length inline writers include every bitrot frame header in a checked capacity limit and reject overflow before appending data. The persisted marker governs existing-object reads independently of the current write budget. Config serialization retains the historical `inline_block_explicit` marker for readers that use it. A compressed or encrypted single PUT has no known stored size up front, so its shard size is derived from the plaintext `actual_size` (`should_inline_put`, MinIO `putObject` parity); such objects are inlined through the streaming encoder with in-memory bitrot writers rather than the single-block fast path. When shard-integrity protected writes are enabled, inline placement additionally requires a known stored length no greater than one erasure block; unknown transformed streams retain external shards and proof indexes.
---
Inline body materialization uses the shared `inline_body_buffering_miss_reason` contract ([set_disk/mod.rs](../../crates/ecstore/src/set_disk/mod.rs)) with a 2 MiB response limit and a bounded frame count. Metadata pre-verification and buffered GET consumption use this same contract, and an already verified body is reused within the selected snapshot. Parts carrying `PartIntegrity` use the readers that bind their proof verifier. Metadata early-stop and GET share `try_decode_inline_data_shards` for interleaved bitrot verification, per-block EC padding removal and body assembly. Data-shard metadata uses inline-capacity collections for normal EC sets. A healthy full metadata fanout decodes directly from cached bytes. Missing or damaged data shards enter the existing quorum-protected reconstruction path. Buffered GET, deep inline healing and ordinary bitrot readers share the borrowed `verified_bitrot_frame` primitive. Deep healing checks exact physical length before hashing the original frame slices, including every data and parity frame, and rejects trailing bytes, truncated prefixes and checksum mismatches.
## 7. Write path and write quorum
`put_object` and multipart resolve the write layout (§2.2), encode (§4), write shards with bitrot (§5), then commit atomically.
@@ -379,7 +381,7 @@ Geometry & math
Algorithm
- Modern RS over GF(2⁸) (`rs-vandermonde`) for new writes; legacy GF(2¹⁶) for old files, selected by `uses_legacy_checksum`.
- `block_size = 1 MiB` (`BLOCK_SIZE_V2`), stored per version.
- The default `block_size` is 1 MiB (`BLOCK_SIZE_V2`). `RUSTFS_ERASURE_SHARD_BYTES` derives new user-object geometry as `data_blocks × shard_bytes` through `erasure_block_size_for` ([fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)). System metadata keeps the default. Each version stores its own geometry for subsequent reads.
- Shard-size formulas: modern `div_ceil`; legacy `(div_ceil + 1) & !1`. Final block zero-padded before encode.
- Bitrot `HighwayHash256S` (legacy key variant for old files); interleaved `[hash][data]` per block; `bitrot_shard_file_size = ceil(size/shard_size)*32 + size`; verify before use.
@@ -1,7 +1,7 @@
# Cluster and erasure-coding lifecycle operations
**Use this when:** planning, starting, expanding, rebalancing, decommissioning, healing, replacing drives in, or restarting an erasure-coded RustFS deployment, or deciding whether an `EC:<parity>` setting (including `EC:0`) is safe for a workload.
**Source of truth:** `crates/ecstore/src/layout/disks_layout.rs` (`DisksLayout::from_volumes`, `SET_SIZES`, `RUSTFS_ERASURE_SET_DRIVE_COUNT`), `crates/ecstore/src/config/storageclass.rs` (`default_parity_count`, `validate_parity_inner`, `Config::should_inline`, `Config::effective_inline_block`), `crates/ecstore/src/set_disk/mod.rs` (`resolve_write_layout`, `WriteLayout::from_parity`), `crates/ecstore/src/set_disk/metadata.rs` (`object_quorum_from_meta`, `common_parity`), `crates/ecstore/src/set_disk/ops/heal.rs` (no-parity `cannot_heal` classification), `crates/ecstore/src/store/init_format.rs` (`check_format_erasure_value_for_topology`), `rustfs/src/admin/handlers/pools.rs`, `rustfs/src/admin/handlers/rebalance.rs`, `rustfs/src/admin/handlers/heal.rs`, and `rustfs/src/admin/route_policy.rs` (admin surface).
**Source of truth:** `crates/ecstore/src/layout/disks_layout.rs` (`DisksLayout::from_volumes`, `SET_SIZES`, `RUSTFS_ERASURE_SET_DRIVE_COUNT`), `crates/ecstore/src/config/storageclass.rs` (`default_parity_count`, `validate_parity_inner`, `Config::should_inline`, `Config::inline_block`), `crates/ecstore/src/set_disk/mod.rs` (`resolve_write_layout`, `WriteLayout::from_parity`), `crates/ecstore/src/set_disk/metadata.rs` (`object_quorum_from_meta`, `common_parity`), `crates/ecstore/src/set_disk/ops/heal.rs` (no-parity `cannot_heal` classification), `crates/ecstore/src/store/init_format.rs` (`check_format_erasure_value_for_topology`), `rustfs/src/admin/handlers/pools.rs`, `rustfs/src/admin/handlers/rebalance.rs`, `rustfs/src/admin/handlers/heal.rs`, and `rustfs/src/admin/route_policy.rs` (admin surface).
This runbook is the operator entry point. It restates only what an operator needs to act and links the normative contracts for everything else: [erasure-coding.md](../architecture/erasure-coding.md) (algorithm, quorum, on-disk format), [ecstore-layout-boundary.md](../architecture/ecstore-layout-boundary.md) (immutable layout), [decommission-compatibility.md](../architecture/decommission-compatibility.md) (decommission and rebalance contract), [heal-concurrency-model.md](../architecture/heal-concurrency-model.md) (heal versus write locking), [rolling-restart.md](rolling-restart.md) (restarts), [replacement-generation-recovery.md](replacement-generation-recovery.md) (drive replacement recovery), and [scanner-runtime-controls.md](scanner-runtime-controls.md) (heal and scanner knobs). Admin routes below require an authenticated request holding the matching `AdminAction`; the route to action mapping is owned by `rustfs/src/admin/route_policy.rs` (see [admin-route-action-snapshot.md](../architecture/admin-route-action-snapshot.md)).
@@ -60,7 +60,7 @@ A single local path (`rustfs D:\rustfs\data`) is a single-node single-drive depl
| `RUSTFS_STORAGE_CLASS_STANDARD` | STANDARD parity as `EC:<n>` | Validated per pool; affects new writes only |
| `RUSTFS_STORAGE_CLASS_RRS` | RRS parity as `EC:<n>` | Validated per pool; an explicit value fails on a single drive, the persisted default `EC:1` resolves to `0` there |
| `RUSTFS_STORAGE_CLASS_OPTIMIZE` | Accepted for MinIO compatibility | Stored on `Config`, but `Config::capacity_optimized` has no caller: it changes nothing at runtime today |
| `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` | Fixed per-shard inline limit (bytesize syntax such as `128KiB`) | Replaces the scaled default in §4.3; values above 128 KiB log `storage_class_inline_block_large` |
| `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` | Fixed per-shard inline limit (bytesize syntax such as `128KiB`) | Overrides the fixed 128 KiB default in §4.3. Values above 128 KiB log `storage_class_inline_block_large` |
An empty `RUSTFS_STORAGE_CLASS_STANDARD` or `RUSTFS_STORAGE_CLASS_RRS` restores the automatic policy and overrides a persisted value (`standard_policy` / `rrs_policy`). `MINIO_`-prefixed spellings of these keys, of `ERASURE_SET_DRIVE_COUNT`, and of `VOLUMES` are mapped onto the `RUSTFS_` names when the `RUSTFS_` name is unset (`apply_external_env_compat` in `crates/utils/src/envs.rs`); when both are set the `RUSTFS_` value wins and the conflict is reported at startup.
@@ -103,7 +103,9 @@ Sixteen drives at `EC:4`: `K = 12`, about 75 percent usable, four failures toler
### 4.3 Inline objects
An object is stored inline in `xl.meta` when every shard is at most the per-shard budget (`Config::should_inline`). The default budget is `DEFAULT_INLINE_OBJECT_BUDGET / K` capped at `DEFAULT_INLINE_BLOCK` (256 KiB / K, at most 128 KiB per shard), so a wider set does not raise the maximum inline object size; on a versioned bucket the budget is divided by 8. `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` replaces the scaled default with a fixed per-shard limit. Each drive keeps only its own shard inline, so inline data has the same loss semantics as external shards. Compressed or encrypted streams inline only when the stored size is known and within budget.
An object is stored inline in `xl.meta` when every shard is at most the per-shard budget (`Config::should_inline`). The default is `DEFAULT_INLINE_BLOCK` (128 KiB per shard), matching MinIO's default. A versioned bucket uses one eighth of that budget. `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` sets an explicit fixed per-shard limit. Each drive keeps its own bitrot-protected shard inline. PUT admission also requires the stored size, or the plaintext size for an unprotected transformed stream with unknown stored size, to fit one erasure block. Protected transformed streams require a known stored size. The complete format and transform rules are in [erasure-coding.md §6.6](../architecture/erasure-coding.md#66-inline-data).
Existing inline and external objects are read according to their persisted metadata after a budget change. The former scaled default admitted approximately 256 KiB per unversioned object across wider EC layouts. The fixed per-shard budget admits larger new objects as data-shard count grows, subject to the single-block bound. Operators can pin an explicit smaller per-shard limit for a selected geometry. Changing this setting affects future placement; rollback to a historical binary requires that binary's reader to accept the newly written layouts. Use [MinIO format compatibility](../architecture/minio-file-format-compat.md) for the supported migration direction and encrypted-object constraints.
### 4.4 Write and read paths
@@ -0,0 +1,15 @@
# Distributed GetObject metadata cache proof experiment
`RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE` opts a test or `e2e-test-hooks` build into proof-validated GetObject metadata caching for distributed erasure sets. It defaults to `false` and requires a restart to change. Ordinary production builds ignore the setting and keep the distributed cache hard-disabled until the mutation-authority audit described below is complete.
The experiment does not add a cache-control RPC or change write admission. Capable namespace-lock voters advance a bounded mutation-generation shard on every successful exclusive grant. An eligible GET requests stamps through its existing shared-lock quorum and may use a cached metadata entry only when the current proof exactly matches the proof recorded at cache publication.
The read falls back to authoritative metadata quorum without failing the S3 request when the setting is disabled or any required proof property is unavailable, including a legacy voter, malformed or unknown stamp, duplicate voter identity, lock loss, voter restart, generation exhaustion, changed quorum cohort, or proof mismatch.
Only request shapes already eligible for the process-local metadata cache participate: latest unversioned object reads whose ECStore entrypoint holds the fixed object namespace read guard, excluding direct SetDisks calls, system buckets, explicit versions, versioned or suspended buckets, delete-marker reads, part-number or part-checksum reads, and data-movement paths. Local erasure deployments retain the existing process-local generation behavior.
The proof state is in memory and has no on-disk migration. Hash-shard collisions cause conservative cache misses only. Mixed-version operation preserves lock and storage availability because old peers can grant locks without a stamp; a cache hit requires a complete capable read quorum.
Keep the experiment disabled for production rollout until every cache-visible metadata mutation path has been verified to hold the same exclusive object namespace authority and the multi-node fault and performance acceptance matrix has passed. Disabling the setting and restarting clears all process-local proof-bound cache state.
The current production-activation blockers include delete-recovery reconciliation, exact-marker cleanup, and selected restore/heal metadata paths that can operate under a set-local, lifecycle, or process-local fence instead of the ECStore fixed object lock domain. These paths must either acquire that fixed exclusive authority before publication or gain an equivalent quorum-stamp transition before the production gate is removed. Legacy transition-state reconciliation is not a blocker because it already acquires the fixed store domain followed by every distinct physical set domain before disk metadata publication.
+26
View File
@@ -92,6 +92,32 @@ power simultaneously, `relaxed` can lose recently acknowledged objects
cluster-wide. Single-node
deployments must stay on `strict`.
## Small-object I/O profile
Small-object cost scales with object count, key depth, and the selected durability mode. Each inline object retains its own object directory and `xl.meta`, which contains both metadata and inline payload. Non-inline objects also have a data directory and shard files. Compression can reduce stored bytes while retaining this per-object namespace. The default single-drive inline admission budget is 128 KiB per shard. Versioned writes use a smaller budget. Explicit storage-class configuration, stream transformations, the single-erasure-block bound, and the erasure layout affect admission. See `StorageClassConfig::should_inline` in `crates/ecstore/src/config/storageclass.rs` and its PUT caller in `crates/ecstore/src/set_disk/ops/object.rs`.
The following application-level synchronization budget covers a successful ordinary Linux inline commit to a fresh key in an existing bucket. It assumes directory group commit is disabled and excludes multipart, healing, system metadata, retries, and background work. Let `d` be the number of prefix components before the final key component, so `object` has `d = 0` and `a/b/object` has `d = 2`.
| Write shape | Metadata file synchronization | Directory synchronization |
| --- | --- | --- |
| Fresh inline key, `strict` | One metadata `sync_data`, including inline payload | One object-directory sync plus one per ancestor through the bucket, totaling `d + 2` |
| Fresh inline key, `file-sync` | One metadata `sync_data`, including inline payload | Zero ordinary commit-directory syncs |
| Fresh inline key, `relaxed` or `none` | Zero | Zero ordinary commit-directory syncs |
Linux stages ordinary inline metadata in a flat temporary file and publishes it by rename. The temporary name is removed by publication, so synchronizing that temporary directory entry would add work without protecting the acknowledged object. Strict mode synchronizes the final object directory and its ancestors. Even an existing shared prefix can have been created by another writer whose directory synchronization is still pending. Keeping the ancestor barriers protects that concurrent creation case. `rename_data_fresh_inline_sync_budget` tests exercise the real disk commit and pin the directory-call budget for flat and nested keys.
Overwrites and versioned writes can retain predecessor metadata for rollback and add backup, recovery, and cleanup work. Non-inline writes also synchronize shard contents under `strict`, `file-sync`, and `relaxed`. Their budgets depend on object shape and recovery state. A normal inline GET reads `xl.meta`. Non-inline GET also reads the required shard data and verifies integrity. Delete work depends on versioning, delete markers, retained versions, recovery state, and deferred trash cleanup. LIST, scanner, and heal perform additional directory and metadata reads, and repairs can perform writes. Measure each workload separately.
All metadata commit paths share the same recovery-aware object-directory policy. Publishing a persistent rollback directory requires synchronizing its entry in the object directory, including under `file-sync` and when replacing an inline object with an external-data object. The backup directory itself also remains synchronized. These barriers preserve the recovery source for a failed multi-drive publication.
A fresh inline `file-sync` commit can finish writing, synchronizing, and publishing metadata within its preparation task even when directory grouping is enabled. The sync-budget regression checks two disk-local queue submissions on Linux and one on macOS for that path. macOS also performs separate Tokio filesystem preparation. Persistent rollback publication and strict directory grouping retain their respective synchronization and scheduling requirements.
Application synchronization counts and device IOPS measure different layers. Filesystem journals, page-cache hits, writeback, request merging, and device behavior determine the physical operations and latency. For a reproducible profile, record the exact server revision, effective bucket durability mode, storage-class configuration, object-size distribution, key depth, versioning, concurrency, filesystem, and physical device mapping. Use an isolated test bucket with separate fresh-key PUT, overwrite, GET, and DELETE phases. Collect a background-only interval as well as each foreground interval, keeping scanner and heal state explicit. Pair application syscall traces with device telemetry over the same interval, then normalize counts by completed requests.
On Linux, an instrumented run can use `strace -f -c -p <pid> -e trace=openat,openat2,close,newfstatat,fstat,statx,getdents64,mkdir,mkdirat,rmdir,rename,renameat,renameat2,unlink,unlinkat,read,pread64,write,pwrite64,fsync,fdatasync` together with `iostat -x 1`. Tracing adds overhead and includes other server activity, so retain an untraced run for latency and throughput comparison. The [scanner benchmark runbook](scanner-benchmark-runbook.md) covers background-pressure measurement.
Server-side packing of multiple S3 keys into shared containers requires a persistent index, atomic publication, per-key version/delete semantics, garbage collection, and heal/recovery support. These requirements also apply when compacting historical objects. Current inline storage keeps each object's directory and metadata entry independently addressable.
## Recovery from corrupt object metadata
A zero-length or unreadable `xl.meta` is corruption, not an absent object.
@@ -228,6 +228,27 @@ Every offered Ping is recorded, including schedule drops, deadlines and errors.
Use identical probe source and settings on the two frozen library revisions, in separate Cargo targets, and record the harness overlay and binary identity independently from the already-built clean server binaries. Qualify repeated A cells and then A1/B1/B2/A2 before expanding to coalescing, isolation, pool or window experiments. A successful local mock regression verifies the probe's calling and accounting boundaries; acceptance against the actual authenticated four-node services is still required.
### Timeout-boundary evidence
Use `scripts/check_internode_rpc_timeout_boundary.py` after a task-owned fault injector has recorded three independent probe reports: a stall whose measured duration remains below the declared RPC timeout, a stall whose requested and measured durations both exceed it, and a no-fault recovery run. The gate does not inject faults, deploy services or infer a timeout from wall-clock labels; the manifest references separate measured fault receipts and the raw reports.
```json
{
"schema": 1,
"scope": "internode-rpc-timeout-boundary",
"rpc_timeout_ms": 10000,
"under_budget": {"fault_receipt": "under-fault.json", "report": "under.json"},
"over_budget": {"fault_receipt": "over-fault.json", "report": "over.json"},
"recovery": {"report": "recovery.json"}
}
```
```bash
python3 scripts/check_internode_rpc_timeout_boundary.py --manifest /path/to/manifest.json --out /path/to/verification.json
```
Each fault receipt is a bounded JSON object containing a non-empty `target`, `requested_stall_ms` and `actual_stall_ms`. Exit 0 requires complete bulk coverage, exact offered-control coverage, no errors below the timeout, at least one `deadline` above it, and a zero-error recovery report. `schedule_drop` and `rpc_error` never substitute for the expected deadline. Exit 1 means structurally complete evidence did not satisfy the boundary; exit 2 means the evidence is invalid. The verifier binds all reports and fault receipts by SHA-256, rejects path escape/reuse and refuses to overwrite or alias raw evidence. Deployment ownership, actual fault delivery, phase-scoped telemetry and performance attribution remain separate gates.
## Acceptance gates
Record the verdict (pass/fail plus measured delta) in the paired directory's summary.
@@ -66,20 +66,26 @@ Convergence-ready:
| request | `ReadMultiple.read_multiple_req` | `ReadMultipleReq` |
| request | `BatchReadVersion.batch_read_version_req` | `BatchReadVersionReq` |
| request | `Read*.opts` | `ReadOptions` |
| request | `DeleteVersions.versions` | binary-list-driven `FileInfoVersions` batch |
| request | `DeleteVersions.opts` | named-map `DeleteOptions` |
| response | `ReadVersion.file_info` | `FileInfo` (`cluster/rpc/remote_disk.rs`) |
| response | `ReadXL.raw_file_info` | `RawFileInfo` |
| response | `RenameData.rename_data_resp` | `RenameDataResp` |
| response | `ReadMultiple` response list | per-item plus list fallback |
| response | `BatchReadVersion` response list | per-item plus list fallback |
Delete messages, kept on dual-write until their own window reads zero with `_bin`-first decoders deployed fleet-wide (the client always dual-writes these regardless of the flags):
Single-version delete fields retain dual-write until their own observation window establishes fleet-wide readiness.
| Direction | Message / field | Status |
| --- | --- | --- |
| request | `DeleteVersion.file_info` (`FileInfo`) | `_bin` present; dual-write; converge after its own window |
| request | `DeleteVersion.opts` (`DeleteOptions`) | same |
| request | `DeleteVersions.versions` (`FileInfoVersions`) | same |
| request | `DeleteVersions.opts` (`DeleteOptions`) | same |
`DeleteVersions` receivers select the binary list whenever `versions_bin` contains entries. That list supplies the batch length and every decoded item, including requests whose JSON `versions` list is empty or has a different length. Empty or corrupt binary items reject the complete request before disk mutation. A request with an empty binary list uses the legacy JSON list and its length. Options retain their independent binary-first decoder. The sender captures the codec policy once per complete batch. Default mode emits both lists and both option encodings. Setting both `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=true` and `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=true` emits the binary list and named-map options, with empty JSON fields. Before confirming the fleet, deploy the corrected binary-list-driven receiver to every destination and verify the observation counters and binary-only deletion results. Rolling upgrades and rollback to a receiver that traverses the JSON list require JSON compatibility mode before replacing nodes. These fields remain in the protobuf schema for older peers.
User-object recoverable deletion uses `RecoverDeleteTransaction` and guarded mutation messages with their existing binary payloads. Measure those operations with transaction preparation, commit, rollback and cleanup stages. The `DeleteVersions` codec comparison covers the legacy/internal batch RPC and its consumers.
`DeleteVersions` binary-only sending uses the explicit fleet-confirmation gate. In per-peer capability mode, batch deletion emits both JSON and binary fields. The generic peer capability proof covers the other eligible request families; binary-list-driven deletion additionally requires the corrected batch receiver on every destination.
Still JSON-only:
@@ -108,16 +114,16 @@ The benchmark driver pins these operator states as dry-run phases (`before`, `re
The per-peer gate is not qualified by a successful command transcript alone. Record three ordered four-node measurement phases: `mixed-old-peer` (three candidate nodes and one previous-version node), `all-new` (four candidate nodes), and `rollback-old-peer` (replace one candidate with the previous-version image and verify readback). Every candidate node uses `MSGPACK_ONLY=true`, `MSGPACK_PEER_CAPABILITY=true`, and `MSGPACK_ONLY_FLEET_CONFIRMED=false`; previous-version nodes use all three false. Each phase must run real PUT/GET/metadata traffic and retain the exact candidate/baseline revisions, topology, timestamps, workload result, environment, and raw Prometheus HTTP API JSON.
Capture these PromQL vectors over each non-overlapping phase window and store the complete HTTP API response, not extracted numbers:
Assign one unique `msgpack-<32-hex>` cluster ID for this run, set `OTEL_RESOURCE_ATTRIBUTES=rustfs.cluster.id=<id>` on every node, and retain the collector’s resource-to-telemetry conversion so Prometheus exposes `rustfs_cluster_id`. Capture these PromQL vectors over each non-overlapping phase window and store the complete HTTP API response, not extracted numbers:
```promql
sum by (decision, reason) (increase(rustfs_system_network_internode_msgpack_send_decisions_total[$WINDOW]))
sum(increase(rustfs_system_network_internode_msgpack_json_fallback_total[$WINDOW])) or vector(0)
sum(increase(rustfs_system_network_internode_msgpack_json_decode_error_total[$WINDOW])) or vector(0)
(sum(increase(rustfs_system_network_internode_operation_errors_total{backend="grpc",operation!="grpc_read_version"}[$WINDOW])) or vector(0)) + (sum(increase(rustfs_system_network_internode_operation_classified_errors_total{backend="grpc",operation="grpc_read_version",classification!="remote_response"}[$WINDOW])) or vector(0))
sum by (decision, reason) (increase(rustfs_system_network_internode_msgpack_send_decisions_total{rustfs_cluster_id="$CLUSTER_ID"}[$WINDOW]))
sum(increase(rustfs_system_network_internode_msgpack_json_fallback_total{rustfs_cluster_id="$CLUSTER_ID"}[$WINDOW])) or vector(0)
sum(increase(rustfs_system_network_internode_msgpack_json_decode_error_total{rustfs_cluster_id="$CLUSTER_ID"}[$WINDOW])) or vector(0)
(sum(increase(rustfs_system_network_internode_operation_errors_total{rustfs_cluster_id="$CLUSTER_ID",backend="grpc",operation!="grpc_read_version"}[$WINDOW])) or vector(0)) + (sum(increase(rustfs_system_network_internode_operation_classified_errors_total{rustfs_cluster_id="$CLUSTER_ID",backend="grpc",operation="grpc_read_version",classification!="remote_response"}[$WINDOW])) or vector(0))
```
Build a schema-1 manifest with `scope="internode-msgpack-peer-canary"`, `source_revision`, `baseline_revision`, `stable_connection_affinity=true`, `max_metric_lag_secs`, and the three phase records. Each phase records integer `start_unix`/`end_unix`, exact topology counts plus a `node1..node4` revision map, the new/old node environment maps, the exact four PromQL strings with a window equal to the phase duration, a relative workload JSON path, and relative paths for `decisions`, `fallback`, `decode_errors`, and `rpc_errors`. The workload artifact records `status`, a non-zero integer `requests`, and `rollback_readback` (true only for `rollback-old-peer`). Validate without rewriting raw evidence:
Build a schema-1 manifest with `scope="internode-msgpack-peer-canary"`, `source_revision`, `baseline_revision`, `stable_connection_affinity=true`, `max_metric_lag_secs`, `cluster_id`, an `images` path to sanitized image inspections, and the three phase records. Each phase records integer `start_unix`/`end_unix`, exact topology counts plus a `node1..node4` revision map, the new/old node environment maps, the exact four PromQL strings with a window equal to the phase duration, an `images` path to the verified per-node image ID/revision map, a relative workload JSON path, and relative paths for `decisions`, `fallback`, `decode_errors`, and `rpc_errors`. The workload artifact records `status`, a non-zero integer `requests`, and `rollback_readback` (true only for `rollback-old-peer`). Validate without rewriting raw evidence:
```bash
python3 scripts/check_internode_msgpack_peer_canary.py \
@@ -127,7 +133,7 @@ python3 scripts/check_internode_msgpack_peer_canary.py \
The verifier rejects reused/escaping/overwritten artifacts, duplicate or non-exact decision labels, labeled scalar queries, mismatched PromQL windows, wrong per-node revisions, missing peer-capable traffic, missing mixed/rollback dual-write traffic, manual-fleet or not-requested leakage, failed workload/readback, non-zero fallback/decode/gRPC transport-or-protocol errors, stale timestamps, invalid topology, and candidate/baseline identity reuse. The gRPC error query checks every operation except `grpc_read_version` through the total error counter. Read-version uses its classified counter and excludes only `remote_response`, because a successful gRPC exchange can carry an expected application-level disk response; `grpc_status` and `response_decode` still block the gate. TCP/HTTP operation outcomes remain outside this msgpack gRPC compatibility gate and the workload's zero-error contract remains independently enforced. The workload artifact binds its raw benchmark `summary.csv` by relative path and SHA-256; rollback also records equal source/download content hashes. Verifier output includes SHA-256 receipts for every raw query, workload, and summary artifact. Passing this gate is compatibility evidence only; production soak and ABBA performance drift gates remain separate.
Use the orchestration runner to create the isolated Compose project, execute the three phases, preserve a known object across node2 rollback, capture the four raw Prometheus responses, build the manifest, and invoke the verifier. Without `--execute` it prints a redacted plan and makes no external changes. Execution refuses an existing Compose project/volume set and removes its isolated containers and volumes on exit unless `--keep-up` is explicit.
Use the orchestration runner to create the isolated Compose project, execute the three phases, preserve a known object across node2 rollback, capture the four raw Prometheus responses, build the manifest, and invoke the verifier. Without `--execute` it prints a redacted plan and makes no external changes. The runner verifies distinct immutable image IDs and full `vcs-ref` or OCI revision labels against the requested revisions, pins Compose to those IDs, and checks each running container’s image and telemetry identity in every phase. If an image backend omits both recognized labels from `image inspect`, the runner queries `image history` by the already resolved immutable image ID and accepts only a canonical 40-hex revision from the newest matching `LABEL` instruction; an existing mismatched config label never falls back. Images without matching full revision evidence are rejected. It records whether each sanitized identity receipt came from config labels or immutable history without retaining container credentials or unrelated history. Execution refuses an existing Compose project/volume set and removes its isolated containers and volumes on exit unless `--keep-up` is explicit.
```bash
scripts/run_internode_msgpack_peer_canary.sh \
+6
View File
@@ -31,6 +31,12 @@ A previously published gauge can remain visible after a statistic becomes unavai
When live requested bytes are unavailable, capture a supported heap profile or allocation/free trace. Preserve attribution to allocation stacks and lifetime; committed bytes and RSS alone cannot establish a leak. Other raw allocator statistics, including page accounting, depend on the allocator build and are not interchangeable with live application bytes.
### Diagnostic allocator and runtime metrics
Build with `--features diagnostic-allocator-live` to count live requested bytes for Rust allocations routed through the binary's MiMalloc `GlobalAlloc` wrapper. Read `rustfs_memory_allocator_rust_global_requested_live_bytes` together with `rustfs_memory_allocator_rust_global_requested_live_bytes_available`. Availability is `1` for this feature on non-Windows builds when the `hotpath` and `hotpath-alloc` combination is inactive. Direct native allocations are outside this counter. The wrapper records successful allocations, zeroed allocations, reallocations, and frees using allocation-free atomic updates. Per-allocation accounting adds overhead, so collect diagnostic and throughput measurements in separate runs.
The memory sampler also exports `rustfs_runtime_worker_threads`. With `RUSTFLAGS="--cfg tokio_unstable"`, it exports serving-runtime blocking-thread total, idle, and active gauges under `rustfs_runtime_blocking_threads_*`, with `rustfs_runtime_blocking_pool_metrics_available = 1`. An ordinary build reports availability `0`. These metrics describe the serving runtime; isolated disk fsync and cleanup runtimes have their own execution ownership. Enable the allocator feature separately when both views are needed. Tokio's `tokio_unstable` configuration uses the stable compiler and opts into Tokio's unstable telemetry API.
### Linux transparent huge pages
Linux RustFS binaries disable transparent huge pages for mimalloc by default. This reduces resident memory when reclaimable allocations occupy only part of a huge page. Check `THP_enabled` in `/proc/<pid>/status` and `AnonHugePages` in `/proc/<pid>/smaps_rollup` to verify the running process. Both values should be zero with the default. The Linux binary also defaults to immediately purging free arena ranges. Set `MIMALLOC_PURGE_DELAY` before startup to override the delay in milliseconds.
@@ -68,6 +68,7 @@ iostat -xz 5 > telemetry/iostat.txt &
| Knob | Default | Controls | Validating stage | Risk if widened |
| --- | --- | --- | --- | --- |
| `RUSTFS_ERASURE_SHARD_BYTES` | `0` | `erasure_block_size_for` in `crates/filemeta/src/fileinfo.rs` derives a new user-object block from data-shard count times this byte value. Zero selects the historical 1 MiB block. Validation limits live in `crates/config/src/constants/object.rs`. Cached at first use. | Stripe count, `erasure_encode_cpu`, GET path, range bytes, TTFB and RSS | Changes geometry across pools and storage classes, direct-write eligibility and bitrot frame size |
| `RUSTFS_ERASURE_ENCODE_MAX_INFLIGHT_BYTES` | 32MiB (`crates/ecstore/src/erasure/coding/encode.rs`) | EC encode producer/consumer memory budget (blocks queued between encode and shard write) | `set_disk_encode` P95 + `rustfs_ec_encode_inflight_bytes_current` | RSS growth under high concurrency |
| `RUSTFS_ERASURE_ENCODE_BYTESMUT_INGEST` | `true` | Streaming encode reads into an owned, EC-sized buffer, avoiding the input-to-encoder block copy. Set `false` to select the previous streaming Vec ingest path; batched encode always uses owned buffers. Cached at first use. | `erasure_encode_cpu`, PUT throughput and RSS | No disk-format change; queue budget does not bound total process memory |
| `RUSTFS_OBJECT_IO_BUFFER_SIZE` | 128KiB (`crates/config/src/constants/object.rs`) | Streaming read-in / write-out block size | `ingress_prepare`, `set_disk_encode` | Larger buffers = fewer polls, more resident memory |
@@ -108,6 +109,38 @@ Collect detailed diagnostics in dedicated rounds with `RUSTFS_OBS_PUT_STAGE_METR
These metrics cover the disk queue's Fsync lane. Direct fsync dispatches and cleanup work on the Isolated lane require their own stage observations. Peak gauges are process-lifetime high-water marks for workers whose diagnostics started while the detailed PUT gate was enabled, including startup and warmup. A retiring closure and its replacement can briefly overlap in this count while execution concurrency remains bounded. Restart each server per leg to establish a fresh lifetime. Guards finish accounting on normal exit and panic, including when the gate is disabled while a measured worker remains alive. Diagnostics use atomic accounting and cached metric handles outside the disk queue mutex; the recorder may acquire its own publication locks. Hold this observation overhead consistent within diagnostic comparisons.
#### Erasure block geometry experiment
The [isolated Docker inline lab](../../.docker/inline-lab/README.md) provides a frozen-source image and an owned external deployment adapter for the existing ABBA runner.
Freeze the inline per-shard budget, durability mode, codec mode, queue budget, and binary before changing `RUSTFS_ERASURE_SHARD_BYTES`. Use integer bytes for this setting. `configured_erasure_shard_bytes` validates its alignment and upper bound and caches the result. Restart the tested processes and create each leg's measurement dataset after the new geometry is active. Existing versions and active multipart uploads retain their recorded geometry, so pre-existing GET data cannot establish the new-write behavior.
The setting applies to every newly constructed user-object geometry. A fixed shard value produces different block sizes for different data-shard counts, including different storage classes in one pool. Record the persisted block size in each test cell. The per-shard placement budget and body materialization limits are separate contracts. Verified inline body reuse is bounded to 2 MiB and 32 frames. Trace metadata body verification and subsequent body decoding together, including reuse within the selected version snapshot.
Use one candidate shard value per ABBA run, starting with a small change appropriate to the target geometry. Include full GET, PUT, small Range GET, versioned objects, multipart, transformed objects, degraded reads and a reader rollback fixture. Large blocks reduce stripe transitions while expanding read/verification granularity and the maximum single-block direct-write input. The streaming queue has a minimum of one expanded block, and direct paths have their own buffering. Validate RSS, CPU scheduling, first-byte latency and error counts alongside throughput.
The pinned MinIO Community comparison uses a 1 MiB block for new objects. Its inline buffer writers can accumulate multiple blocks while the total erasure shard remains within the storage-class budget. RustFS supports bounded multi-block inline for known-length unprotected PUTs at its default 1 MiB block. The accumulated data-shard budget is 128 KiB, reduced to 16 KiB for versioned writes. EC12+4 therefore admits up to 1,572,856 logical bytes, with each bitrot frame header additionally charged to the bounded writer. Transformed and proof-protected writes retain their existing single-block admission contract. A geometry override is an independently measured configuration experiment with its own read-path and compatibility results. [MinIO object geometry](https://github.com/minio/minio/blob/7aac2a2c5b7c882e68c1ce017d8256be2feea27f/cmd/storage-datatypes.go#L388-L399) and [inline writer selection](https://github.com/minio/minio/blob/7aac2a2c5b7c882e68c1ce017d8256be2feea27f/cmd/erasure-object.go#L1398-L1440).
For transport comparisons, keep geometry fixed and independently evaluate JSON compatibility mode and the existing MessagePack-only flag pair. Inline RenameData carries its data through metadata RPCs, so JSON serialization and duplicate payloads can outweigh the saved part-file I/O. The Docker lab pins and checks both transport flags on every node. Use its version, geometry and multipart manifests for mixed-reader and full-reader rollback checks before a performance run. Restore JSON compatibility mode before introducing an older peer and record the oldest reader actually tested.
#### Borrowed inline verification kernel
The inline verifier operates on already loaded frame slices. Its manual ABBA comparison exercises the streaming verifier and the borrowed verifier over identical 4 KiB and 128 KiB shard payloads. Capture the reported debug/release profile, iteration counts, elapsed times and baseline drift. This measures frame verification CPU and copying costs. End-to-end acceptance uses the fixed-condition Docker ABBA matrix with request errors, throughput and tail latency recorded for every leg.
```bash
cargo test -p rustfs-ecstore --lib --features test-util \
inline_bitrot_borrowed_verification_benchmark -- --ignored --nocapture --test-threads=1
```
For batch-delete codec acceptance, run `delete_versions_request_encoding_preserves_batches_and_fleet_guards` to verify the emitted fields, named-map options and encoded byte counts. Run `delete_versions_wire_roundtrip_preserves_json_and_binary_only_batches` in the `rustfs` library tests to exercise the actual remote sender, protobuf handler and disk mutation with both codec modes. Its retained-object oracle protects against deleting an unselected entry. Deploy JSON compatibility mode during mixed-version and old-reader tests, then confirm the homogeneous upgraded fleet before enabling binary-only batches as described in the [codec convergence runbook](internode-msgpack-json-convergence-runbook.md).
```bash
cargo nextest run -p rustfs-ecstore --lib --features test-util --no-tests fail \
-E 'test(delete_versions_request_encoding_preserves_batches_and_fleet_guards)'
NO_PROXY='*' no_proxy='*' cargo nextest run -p rustfs --lib --features gateway-stack --no-tests fail \
-E 'test(delete_versions_wire_roundtrip_preserves_json_and_binary_only_batches)'
```
### 5.2 GET
| Knob | Default | Controls | Validating stage | Risk if enabled |
@@ -108,6 +108,28 @@ Claim values are used verbatim as policy names (after `claim_prefix`, if any). N
For first-contact testing only, `RUSTFS_IDENTITY_OPENID_ROLE_POLICY=consoleAdmin` grants every login full access; remove it before production.
### Creating access keys for a local user
In the Console, open **Users → select the local user → Access Keys → Create**. An unrestricted root Console session or a local or OIDC administrator with the built-in `consoleAdmin` policy can create a key for an enabled local user. The key uses the target user's permissions and any narrower session policy supplied at creation.
For a custom administrator, grant both actions. The following policy permits delegated creation only for `backup-user`:
```json
{
"Version": "2012-10-17",
"Statement": [
{"Effect": "Allow", "Action": "admin:CreateServiceAccount"},
{
"Effect": "Allow",
"Action": "admin:CreateServiceAccountForUser",
"Condition": {"StringEquals": {"svc:TargetUser": "backup-user"}}
}
]
}
```
Self-service uses the existing `admin:CreateServiceAccount` permission. The built-in `consoleAdmin` policy and custom policies granting `admin:*` cover both actions. This also applies to existing wildcard grants after an upgrade. To exclude delegation from a wildcard grant, add an explicit deny for `admin:CreateServiceAccountForUser`. Deploy the new server version to all nodes before installing policies that name the new action or condition.
## Validation
1. Discovery:
+16
View File
@@ -193,3 +193,19 @@ The 1024 PUT admission limit includes uploading requests and detached owners;
it is an object-count bound, not a memory or byte budget. The older deferred queue
remains optional and disabled by default; its overflow path cannot bypass the
new coordinator or disk limits. No unbounded foreground fallback is introduced.
## Experimental file-sync follower timeout
`RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE` enables file-sync grouping and defaults to `false`. `RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS` controls the batch collection window, defaults to `0`, and is capped at `1000` microseconds. `RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_FOLLOWER_TIMEOUT_MICROS` controls how long a follower waits before trying to leave a queued batch. It defaults to `0` (disabled) and is capped at `1000` microseconds. Configure these process-lifetime settings before startup.
After its deadline, a follower can atomically take ownership of its queued files and resource unit. It then executes file sync through the existing per-disk and global limits, followed by source-directory fsync in the normal caller. The original batch loses those paths before the follower can publish its data directory. A worker that has already claimed the batch retains exclusive ownership, and the follower waits for that worker's result. The timeout therefore bounds queued batching delay; disk admission and native sync can take longer. Cancellation retains the existing native-work resource lifetime and worker cleanup guarantees.
The worker guard owns batch claim, in-flight accounting and idle publication. Claiming an empty queue publishes the idle state under the same group lock used by enqueue. A request arriving during cleanup can establish successor-worker ownership, and stale idle cleanup checks that ownership before removing the group.
Pending followers use indexed slab slots and a monotonic waiter identity. Detaching a queued follower performs constant-time lookup under the coordinator locks. The identity check protects a claimed request when its old slot is reused by a later batch. Queue capacity remains bounded by the existing file and waiter limits.
A source that disappears before native open fails its own waiter, while healthy requests in that batch still complete file and directory sync. Other file-sync errors retain the existing fail-fast batch behavior, preventing repeated I/O against a failing disk. Admission or native-dispatch failure also fails the batch. File sets and their existing resource units move together into one blocking dispatch, and result senders remain with the async worker so cancellation keeps its established notification contract.
With detailed PUT metrics enabled, a successful queue ownership transfer adds a `follower_timeout` role sample to `rustfs_s3_put_object_rename_fdatasync_group_wait_ms`. Evaluate the experiment against the same source, durability setting, topology, and workload. Include sync failures, follower publication before batch completion, cancellation, and peak resource usage in acceptance.
Grouped worker batches use `mode="grouped"` on `rustfs_s3_put_object_rename_fdatasync_batch_files`. The `parallel` mode is reserved for the independent per-file fanout path, so dashboards can distinguish serialized user-space grouping from bounded parallel dispatch.
+1 -1
View File
@@ -114,7 +114,7 @@ Cloudflare's proxy may buffer the entire request body before forwarding and can
For HTTP `UploadPart`, the inactivity budget counts time waiting for raw request-body bytes while storage is requesting input. Positive raw bytes reset the budget, including fragments of a signed AWS chunk that has not yet finished decoding. Foreground admission, capped-session staging, and storage backpressure do not consume the budget. Finishing the declared payload does not bypass the signed terminator, required trailers, or final body validation. This is an inactivity limit, so an upload making progress can take longer than 300 seconds overall.
Setting the timeout to `0` disables it for ordinary uploads. Multipart sessions with an explicit total-object-size cap retain a minimum 300-second timeout, including when the configured value is `0`. After a body-stall timeout, HTTP/1 uses the existing raw-body drain and closes the connection; HTTP/2 releases the affected stream and keeps the connection usable.
Setting the timeout to `0` disables it for ordinary uploads. Multipart sessions with an explicit total-object-size cap retain a minimum 300-second timeout, including when the configured value is `0`. After a body-stall timeout, HTTP/1 uses the existing raw-body drain and closes the connection; HTTP/2 releases the affected stream and keeps the connection usable. On both stacks, the HTTP/1 early-response drain uses the configured idle timeout; `0` leaves that drain unbounded.
`UploadPart` requires a known logical byte length. RustFS uses the length normalized by S3S after authentication and decoding, with an exact logical stream length as a fallback. A bare `x-amz-decoded-content-length` or `Content-Encoding: aws-chunked` declaration cannot supply this length by itself. Requests reaching an ordinary upload session without a known length return `MissingContentLength` (HTTP 411) before body ingestion; capped sessions retain their `UnexpectedContent` rejection. Preserve the client's framing and signed headers through the proxy. The 5 GiB limit applies to each part request, not the combined size of an ordinary multipart upload.
@@ -1,6 +1,6 @@
# S3 Tables Catalog Lifecycle
**Use this when:** disabling an object-backed table catalog, cleaning up metadata left by deleted empty tables, or diagnosing catalog registration reads.
**Use this when:** disabling an object-backed table catalog, cleaning up metadata left by deleted empty tables, diagnosing catalog registration reads, or recovering an interrupted table rename.
**Source of truth:** `rustfs/src/admin/handlers/table_catalog/config.rs`, `rustfs/src/table_catalog/store/cleanup.rs`, and `rustfs/src/table_catalog/store/registration_cache.rs`.
## Disable A Table Catalog
@@ -30,3 +30,11 @@ Each server keeps a bounded memory cache of validated ownership records. Reuse r
Registration still enumerates namespace and warehouse-index prefixes, so enumeration grows with catalog size. Cache hits reduce object-body reads and JSON parsing. A hit reuses previously validated content without reading the current body again to prove physical readability. Conditional writes, rollback, and lock-loss fencing continue to control publication; the cache does not synchronize writers that bypass those guards.
Existing catalog, Iceberg metadata, and Console/client request formats remain unchanged. See the [S3 Tables support matrix](../architecture/s3-tables-support-matrix.md) for supported API boundaries.
## Interrupted Table Renames
Every server starts rename recovery once its own application context is ready, including embedded servers. This runs independently of `RUSTFS_TABLE_MAINTENANCE_SCHEDULER_ENABLED`. Recovery scans up to 32 bucket directories per pass, skips their child objects, and admits at most four recoveries concurrently. Passes are separated by five seconds plus jitter; failed buckets retry with exponential delays from 10 seconds up to five minutes. Larger catalogs may need multiple passes. These intervals do not guarantee a recovery deadline while storage or locks remain unavailable.
Recovery resumes the rename referenced by the bucket's `active_rename_id` through the same restore fence, publication locks, conditional writes, and state validation used by `RenameTable`. Reads remain fenced until finalization completes. Missing or invalid intent records retain the fence and produce `table_catalog_rename_recovery` warnings; preserve those records and collect the warnings for investigation. Shutdown stops new admissions and drains started recoveries.
A completed rename preserves the table UUID, warehouse location, and metadata location. Dropping the empty source namespace removes its namespace record and retains valid `DELETED` table/view records. Active objects, pending states, child namespaces, unreadable records, and active rename fences still prevent namespace deletion. Namespace deletion does not remove metadata or data files referenced by the destination table.
@@ -24,6 +24,22 @@ The authenticated admin metrics endpoint exposes the same counters through the o
A missing `http` field means an older or non-reporting node, not zero failures. Old map-encoded RPC readers ignore the additive field; new readers accept older snapshots. In mixed-version deployments, check reporting coverage before aggregating a fleet-wide ratio.
## Browser upload staging
On the gateway stack, a browser POST whose file length is unknown is staged in the process's system temporary directory before the ordinary PUT path begins. Staging uses an ephemeral in-memory key and authenticated encrypted frames. Two staging files may be admitted per process; further requests receive `SlowDown` immediately, with no fairness or queue guarantee. The slots remain occupied through replay and any in-flight file I/O, including after cancellation.
Staged ciphertext is limited to 6 GiB per process, including each frame's 4-byte header, 16-byte authentication tag, and authenticated end frame. The budget remains charged until the file closes. A request that cannot reserve its next frame receives `SlowDown`; a retry may keep failing until other staging files close. The S3 single-object plaintext limit remains 5 GiB, which needs approximately 5 GiB plus 1.6 MiB of framing. These limits are fixed and are not configuration settings.
On Unix, RustFS queries the open file's filesystem before consuming the file body and before each frame. It requires available space to remain at least 10% of filesystem capacity, capped at 1 GiB and floored at one byte, plus the rounded-up frame and other in-process pending writes. The initial check includes space for an end frame and rejects insufficient space with `SlowDown` before polling the file body. An unavailable or invalid space query, including inconsistent filesystem block counts, fails closed with `InternalError`. Other platforms fail closed for unknown-length POST staging because this file-handle space query is not implemented there. The check adds at least one filesystem statistics query per frame, currently 64 KiB of plaintext. If a concurrent write makes a snapshot stale, RustFS retries up to four times and then returns `SlowDown`, even if free space is otherwise sufficient.
These checks are process-local and the free-space result is only a snapshot, not a disk reservation: other processes, filesystem metadata, delayed allocation, quotas, and external writers can still exhaust the disk. Use a dedicated size-limited filesystem or an appropriate filesystem quota for hard isolation, accounting for every RustFS process. A tmpfs can also exhaust memory. Disk write errors still fail the request.
Staging, including file creation, body ingest, and preparing replay, has a fixed two-hour absolute deadline after slot admission. This can reject a slow but progressing client with `RequestTimeout`, independently of the existing body idle and minimum-progress guards. The deadline ends before PUT begins; it cannot cancel a committed write. Blocking file operations cannot be forcibly interrupted, so their slot and budget remain held until they finish, and replay or downstream storage stalls are not bounded by this deadline.
A process-wide worker drops staged file handles outside the asynchronous executor. It retains each file's slot and budget until the handle's drop returns; a stalled close can therefore delay other uploads. If the worker is unavailable or its bounded queue unexpectedly fills, the caller drops the handle synchronously rather than releasing resources early. If the worker cannot be started, unknown-length POST staging fails closed.
Bucket quota and the final commit checks remain unchanged and run after staging, so a staging resource error can precede a quota error. On Unix, a temporary file must be unlinked before it receives data; normal process termination or a crash closes its last handle, but inherited handles or unusual filesystems can delay reclamation. Temporary-file encryption does not protect plaintext in swap, hibernation images or core dumps.
## Interpret failed storage probes
Storage inventory includes an `observations` entry for each probed node. The aggregator owns this provenance even when a peer runs an older version.
@@ -68,3 +84,11 @@ No new environment variable, admin authorization action, or recovery command is
## Delete recovery during rolling upgrades
S3 deletes that require bounded recovery need peers that support the bounded prepare protocol. An older peer rejects that prepare before staging a mutation. If too few upgraded drives can satisfy the write quorum, the delete can fail with a retryable error until more peers are upgraded. Ordinary recovery inspection and settlement continue to use the original protocol.
## Full data disks and deletion recovery
An ordinary unversioned DELETE permanently removes data, but RustFS first persists a recoverable delete transaction so a failed quorum can restore the object. A filesystem with no allocatable space can reject that journal before any data is removed. This is distinct from PUT pool-capacity admission; changing only that admission does not make deletes work on a physically full disk. Full-disk failures retain the existing S3 `503 ServiceUnavailable` response.
On Unix, each local disk attempts to keep an 8 MiB physically allocated emergency reserve at `.rustfs.sys/delete-reserve`. The reserve uses incompressible content and is armed only after synced allocation is verified, so sparse or transparently compressed zero blocks cannot be mistaken for recovery headroom. Reopened disk handles share one owner pinned to the physical root. Healthy deletion does not open reserve paths or query capacity. If a repeatable scratch write or an idempotent delete-metadata primitive fails with ENOSPC, the first waiter releases existing reserve blocks and retries that operation once; concurrent waiters serialize behind it and also get one bounded retry against the released capacity. Replenishment cannot race those retries. A failed scratch attempt is discarded before retry; the transaction, publication-plus-sync sequences, and destructive cleanup are never replayed as a unit. Invalid reserve state preserves the original operation error. Successful local finalization only attempts replenishment after reserve depletion and with at least 16 MiB free, on the dedicated fsync lane. Startup may release a persisted reserve below 4 MiB available space before the writable probe. The reserve is outside temporary-object cleanup and changes no object formats, quorum, retention checks, or rollback rules. Rollback to an older server leaves an unused reserve file consuming at most 8 MiB per disk.
The reserve is bounded recovery headroom, not a guarantee under arbitrary concurrent writes. Large version histories, inode exhaustion, unrelated writers, or multiple simultaneous deletes can still exhaust space. A first upgrade on an already-full disk may have no reserve to release. Reservation failure does not disable otherwise usable storage, and Windows currently retains its existing behavior. Stop uploads and create filesystem headroom if API deletion still fails; do not remove object directories manually because that bypasses metadata and erasure consistency. Versioned DELETE normally creates a marker and does not reclaim the prior payload; permanently deleting a permitted version requires its version ID.
Generated
+6 -6
View File
@@ -2,11 +2,11 @@
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1793977579,
"narHash": "sha256-WrSODdZFVIs/Hh2NzC0bY+ROByi0bATg8WfePX8dPj8=",
"lastModified": 1794495034,
"narHash": "sha256-E0hYoioqUtM2XmPPvYYpJgdADF5AW9oOoPSEiM9dTHQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "4fe25e12eb9f4c744966dd3de5a05246cf098601",
"rev": "2074cc672349f14240fb7ea90d509e0b643e9dd4",
"type": "github"
},
"original": {
@@ -29,11 +29,11 @@
]
},
"locked": {
"lastModified": 1794040562,
"narHash": "sha256-7eEiezSku3lbZPUOXW8we1a7+xd5axkCQdyI3rzOKjo=",
"lastModified": 1794646145,
"narHash": "sha256-rpqDyUM/d5fACYB8W+wKxxO57ICWjjalzb2X25oFFkY=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "d3fe74269f2a6652437e21e78eb7e83be81f6982",
"rev": "5b87ee0513c29127a01ad06e9cb362ea5ce91e78",
"type": "github"
},
"original": {
+5 -3
View File
@@ -89,7 +89,7 @@ rio-v2 = ["rustfs-ecstore/rio-v2"]
# (rustfs/backlog#2734). Under construction: off by default and never enabled in a
# release build; the stack refuses to compile without debug assertions, and
# `scripts/check_connect_release_boundary.py` keeps every release-reachable feature from enabling it.
gateway-stack = ["dep:rustfs-gateway", "dep:rustfs-gateway-adapter"]
gateway-stack = ["dep:rustfs-gateway", "dep:rustfs-gateway-adapter", "dep:rustfs-gateway-core", "dep:rustfs-gateway-dialect-rustfs-admin"]
# Linux-only cuObject S3-over-RDMA GET into client (GPU) memory; needs libcuobjserver.
cuobject = ["dep:rustfs-cuobject", "rustfs-cuobject/server"]
# NIXL/UCX transport for S3-over-RDMA; replaces cuobject when both are enabled.
@@ -284,6 +284,8 @@ rustfs-policy = { workspace = true }
rustfs-protocols = { workspace = true }
rustfs-gateway = { workspace = true, optional = true }
rustfs-gateway-adapter = { workspace = true, optional = true }
rustfs-gateway-core = { workspace = true, optional = true }
rustfs-gateway-dialect-rustfs-admin = { workspace = true, optional = true }
rustfs-gateway-stream = { workspace = true }
rustfs-gateway-types = { workspace = true, features = ["compat-s3s-rustfs"] }
rustfs-protos = { workspace = true }
@@ -365,7 +367,7 @@ zstd.workspace = true
ed25519-dalek.workspace = true
rustls = { workspace = true, default-features = false, features = ["aws-lc-rs", "logging", "tls12", "prefer-post-quantum", "std"] }
rustls-pki-types = { workspace = true }
rustix = { workspace = true, features = ["param"] }
rustix = { workspace = true, features = ["param", "fs"] }
x509-parser = { workspace = true }
subtle = { workspace = true }
jiff = { workspace = true, features = ["serde"] }
@@ -405,7 +407,7 @@ zip = { workspace = true }
libc = { workspace = true }
rand = { workspace = true, features = ["serde"] }
regex = { workspace = true }
aes-gcm = { workspace = true, features = ["rand_core"] }
aes-gcm = { workspace = true, features = ["rand_core", "zeroize"] }
chacha20poly1305 = { workspace = true }
# Observability and Metrics
+1
View File
@@ -15,6 +15,7 @@
use std::io::Write;
fn main() -> shadow_rs::SdResult<()> {
println!("cargo:rustc-check-cfg=cfg(tokio_unstable)");
println!("cargo:rerun-if-env-changed=RUSTFS_BUILD_VERSION");
if let Ok(version) = std::env::var("RUSTFS_BUILD_VERSION")
&& !version.is_empty()
+2 -1
View File
@@ -57,7 +57,8 @@ use tracing::{Span, debug, error, info, instrument, warn};
use tracing_opentelemetry::OpenTelemetrySpanExt;
#[derive(RustEmbed)]
#[folder = "$CARGO_MANIFEST_DIR/static"]
#[cfg_attr(test, folder = "$CARGO_MANIFEST_DIR/tests/fixtures/native-console")]
#[cfg_attr(not(test), folder = "$CARGO_MANIFEST_DIR/static")]
struct StaticFiles;
/// Static file handler
@@ -207,6 +207,7 @@ impl From<ObjectZipDownloadReqInfoSnapshot> for ReqInfo {
Self {
cred: snapshot.cred,
is_owner: snapshot.is_owner,
keystone_session_present: None,
bucket: snapshot.bucket,
object: snapshot.object,
version_id: snapshot.version_id,
@@ -1690,6 +1691,7 @@ mod tests {
ReqInfo {
cred: Some(credentials),
is_owner: false,
keystone_session_present: None,
bucket: Some("photos".to_string()),
object: None,
version_id: None,
+194 -157
View File
@@ -41,7 +41,7 @@ use rustfs_madmin::{
ServiceAccountInfo, TemporaryAccountInfoResp, UpdateServiceAccountReq,
};
use rustfs_policy::policy::action::{Action, AdminAction};
use rustfs_policy::policy::{Args, DEFAULT_VERSION, Policy};
use rustfs_policy::policy::{Args, DEFAULT_VERSION, Policy, SVC_TARGET_USER};
use rustfs_s3_types::{Body, S3ErrorCode::InvalidRequest};
use rustfs_s3_types::{S3Error, S3ErrorCode, S3Result, s3_error};
use serde::Deserialize;
@@ -139,19 +139,8 @@ fn is_service_account_owner_of(caller: &StoredCredentials, target_parent_user: &
caller_parent == target_parent_user
}
/// GHSA-5354: whether a caller resolving to `owner` may create a service account
/// parented to `target_user`, given the caller's own key (`req_user`) and its
/// effective parent (`req_parent_user`, which equals `req_user` for a top-level
/// user or the parent for a derived credential).
///
/// The `CreateServiceAccountAdminAction` permission gates *whether* a caller may
/// create service accounts, not *for whom*. A non-owner is therefore confined to
/// its own non-root scope; only an owner may target another user — including the root
/// credential, whose existence check is deliberately lenient. Without this a
/// caller holding only that action could pass `target_user = <root access key>`
/// and mint a root-parented service account that authenticates as owner via
/// `prepare_service_account_auth`, a full-takeover privilege escalation. Mirrors
/// `imported_service_account_parent_allowed` on the ImportIam path (GHSA-566f).
/// Scope available with the base create permission. Creating for another local
/// user requires separate authorization in IAM, even when the caller is owner.
fn add_service_account_parent_within_scope(
root_access_key: &str,
target_user: &str,
@@ -332,6 +321,73 @@ pub fn register_service_account_route(r: &mut S3Router<AdminOperation>) -> std::
Ok(())
}
fn map_service_account_create_error(error: rustfs_iam::error::Error) -> S3Error {
match error {
rustfs_iam::error::Error::NoSuchUser(_) => s3_error!(InvalidRequest, "target user not exist"),
rustfs_iam::error::Error::InvalidArgument
| rustfs_iam::error::Error::InvalidAccessKeyLength
| rustfs_iam::error::Error::InvalidSecretKeyLength
| rustfs_iam::error::Error::AccessKeyAlreadyExists => iam_error_to_s3_error(error),
rustfs_iam::error::Error::IAMActionNotAllowed => s3_error!(AccessDenied, "access denied"),
_ => s3_error!(InternalError, "create service account failed"),
}
}
// The Admin operation owns the local write and site notification together.
// A disconnected requester can stop waiting without splitting those effects.
async fn create_and_replicate_service_account<R: Future<Output = S3Result<()>> + Send>(
create: impl Future<Output = S3Result<(StoredCredentials, OffsetDateTime, HashMap<String, serde_json::Value>)>> + Send + 'static,
replication_policy: Option<String>,
replication_expiration: Option<OffsetDateTime>,
replicate: impl FnOnce(SRIAMItem) -> R + Send + 'static,
) -> S3Result<StoredCredentials> {
super::supervise_admin_mutation("service account creation", async move {
let (new_cred, updated_at, replication_claims) = create.await?;
let (replication_session_policy, oidc_service_account_envelope) =
encode_service_account_replication_policy(&replication_claims, replication_policy.as_deref())?;
if let Err(err) = replicate(SRIAMItem {
r#type: "service-account".to_string(),
svc_acc_change: Some(SRSvcAccChange {
create: Some(SRSvcAccCreate {
parent: new_cred.parent_user.clone(),
access_key: new_cred.access_key.clone(),
secret_key: new_cred.secret_key.clone(),
groups: new_cred.groups.clone().unwrap_or_default(),
claims: replication_claims,
session_policy: replication_session_policy,
status: String::new(),
name: new_cred.name.clone().unwrap_or_default(),
description: new_cred.description.clone().unwrap_or_default(),
expiration: replication_expiration,
api_version: Some(SITE_REPL_API_VERSION.to_string()),
}),
oidc_service_account_envelope,
api_version: Some(SITE_REPL_API_VERSION.to_string()),
..Default::default()
}),
updated_at: Some(updated_at),
api_version: Some(SITE_REPL_API_VERSION.to_string()),
..Default::default()
})
.await
{
warn!(
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_SERVICE_ACCOUNT,
event = EVENT_ADMIN_SERVICE_ACCOUNT_STATE,
access_key = %new_cred.access_key,
action = "create",
result = "site_replication_hook_failed",
error = ?err,
"admin service account state"
);
}
Ok(new_cred)
})
.await
}
pub struct AddServiceAccount {}
#[async_trait::async_trait]
impl Operation for AddServiceAccount {
@@ -376,20 +432,13 @@ impl Operation for AddServiceAccount {
return Err(s3_error!(InvalidArgument, "can't create user with system access key"));
}
let mut target_user = if let Some(u) = create_req.target_user {
u
} else {
cred.access_key.clone()
};
let req_user = cred.access_key.clone();
let mut req_parent_user = cred.access_key.clone();
let req_groups = cred.groups.clone();
let mut req_is_derived_cred = false;
if cred.is_service_account() || cred.is_temp() {
req_parent_user = cred.parent_user.clone();
req_is_derived_cred = true;
let requested_user = create_req.target_user.as_deref().unwrap_or(&cred.access_key);
let derived = cred.is_service_account() || cred.is_temp();
let caller_parent = if derived { &cred.parent_user } else { &cred.access_key };
let self_scoped = requested_user == cred.access_key || requested_user == caller_parent;
let target_user = if self_scoped { caller_parent.as_str() } else { requested_user };
if target_user.is_empty() {
return Err(s3_error!(AccessDenied, "service account requires a parent"));
}
let Ok(iam_store) = crate::admin::runtime_sources::current_ready_iam_handle() else {
@@ -400,48 +449,45 @@ impl Operation for AddServiceAccount {
let mut conditions = get_condition_values(req.headers(), &cred, None, None, req.policy_source_addr());
set_service_account_duration_condition(&mut conditions, create_req.expiration, OffsetDateTime::now_utc());
// This family deliberately calls `is_allowed` directly instead of going
// through `validate_admin_request`, and must keep doing so
// (backlog#1886). The shared helper returns as soon as *any* candidate
// action is allowed — an OR. The checks here are an AND: each one must
// pass, and a later stage additionally needs `owner` for the GHSA-5354
// parent-scope guard and drives `deny_only` dynamically. Replacing these
// with the shared gate would widen authorization.
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::CreateServiceAccountAdminAction),
bucket: "",
conditions: &conditions,
is_owner: owner,
object: "",
claims: cred.claims_or_empty(),
deny_only: false, // Always require explicit Allow permission
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
if target_user != cred.access_key {
let has_user = iam_store.get_user(&target_user).await;
if has_user.is_none() && target_user != sys_cred.access_key {
// The policy and the write must see the same resolved parent, including
// when a derived credential names itself. OPA receives this map too.
conditions.retain(|key, _| !key.eq_ignore_ascii_case(SVC_TARGET_USER));
conditions.insert(SVC_TARGET_USER.to_owned(), vec![target_user.to_owned()]);
if self_scoped {
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::CreateServiceAccountAdminAction),
bucket: "",
conditions: &conditions,
is_owner: owner,
object: "",
claims: cred.claims_or_empty(),
deny_only: false,
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
if !add_service_account_parent_within_scope(
&sys_cred.access_key,
requested_user,
&cred.access_key,
caller_parent,
owner,
) {
return Err(s3_error!(AccessDenied, "service account parent is outside requester scope"));
}
if requested_user != cred.access_key
&& requested_user != sys_cred.access_key
&& iam_store.get_user(requested_user).await.is_none()
{
return Err(s3_error!(InvalidRequest, "target user not exist"));
}
}
let is_svc_acc = target_user == req_user || target_user == req_parent_user;
// GHSA-5354: confine a non-owner caller to its own scope so it cannot mint
// a root-parented (owner) service account. Evaluated on the original
// `target_user` the caller submitted, before the derived-credential rewrite
// to `req_parent_user` below, so the guard is bound to the parent actually
// requested and to the effective parent after that rewrite.
if !add_service_account_parent_within_scope(&sys_cred.access_key, &target_user, &req_user, &req_parent_user, owner) {
return Err(s3_error!(AccessDenied, "service account parent is outside requester scope"));
}
let mut target_groups = None;
let target_groups = if self_scoped { cred.groups.clone() } else { None };
let opts = NewServiceAccountOpts {
access_key: create_req.access_key,
secret_key: create_req.secret_key,
@@ -452,84 +498,39 @@ impl Operation for AddServiceAccount {
..Default::default()
};
if is_svc_acc {
if req_is_derived_cred {
if req_parent_user.is_empty() {
return Err(s3_error!(AccessDenied, "only derived cred can create service account"));
}
target_user = req_parent_user;
}
target_groups = req_groups;
}
let replication_groups = target_groups.clone().unwrap_or_default();
let replication_name = opts.name.clone().unwrap_or_default();
let replication_description = opts.description.clone().unwrap_or_default();
let target_user = target_user.to_owned();
let response_secret = cred.secret_key.clone();
let replication_expiration = opts.expiration;
let create_result = iam_store
.new_service_account_from_caller(&target_user, target_groups, opts, &cred)
.await;
let (new_cred, updated_at, replication_claims) = create_result.map_err(|e| {
debug!(
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_SERVICE_ACCOUNT,
event = EVENT_ADMIN_SERVICE_ACCOUNT_STATE,
target_user = %target_user,
result = "create_failed",
error = ?e,
"admin service account state"
);
match e {
rustfs_iam::error::Error::InvalidArgument
| rustfs_iam::error::Error::InvalidAccessKeyLength
| rustfs_iam::error::Error::InvalidSecretKeyLength
| rustfs_iam::error::Error::AccessKeyAlreadyExists => iam_error_to_s3_error(e),
rustfs_iam::error::Error::IAMActionNotAllowed => s3_error!(AccessDenied, "access denied"),
err => s3_error!(InternalError, "create service account failed, e: {:?}", err),
}
})?;
let (replication_session_policy, oidc_service_account_envelope) =
encode_service_account_replication_policy(&replication_claims, replication_policy.as_deref())?;
if let Err(err) = site_replication_iam_change_hook(SRIAMItem {
r#type: "service-account".to_string(),
svc_acc_change: Some(SRSvcAccChange {
create: Some(SRSvcAccCreate {
parent: target_user.clone(),
access_key: new_cred.access_key.clone(),
secret_key: new_cred.secret_key.clone(),
groups: replication_groups,
claims: replication_claims,
session_policy: replication_session_policy,
status: String::new(),
name: replication_name,
description: replication_description,
expiration: replication_expiration,
api_version: Some(SITE_REPL_API_VERSION.to_string()),
}),
oidc_service_account_envelope,
api_version: Some(SITE_REPL_API_VERSION.to_string()),
..Default::default()
}),
updated_at: Some(updated_at),
api_version: Some(SITE_REPL_API_VERSION.to_string()),
..Default::default()
})
.await
{
warn!(
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_SERVICE_ACCOUNT,
event = EVENT_ADMIN_SERVICE_ACCOUNT_STATE,
access_key = %new_cred.access_key,
action = "create",
result = "site_replication_hook_failed",
error = ?err,
"admin service account state"
);
}
let create = async move {
let create_result = if self_scoped {
iam_store
.new_service_account_from_caller(&target_user, target_groups, opts, &cred)
.await
} else {
iam_store
.new_service_account_for_user(&target_user, opts, &cred, &conditions, owner)
.await
};
create_result.map_err(|e| {
debug!(
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_SERVICE_ACCOUNT,
event = EVENT_ADMIN_SERVICE_ACCOUNT_STATE,
target_user = %target_user,
result = "create_failed",
error = ?e,
"admin service account state"
);
map_service_account_create_error(e)
})
};
let new_cred = create_and_replicate_service_account(
create,
replication_policy,
replication_expiration,
site_replication_iam_change_hook,
)
.await?;
let resp = AddServiceAccountResp {
credentials: Credentials {
@@ -541,7 +542,7 @@ impl Operation for AddServiceAccount {
};
let body = serde_json::to_vec(&resp).map_err(|e| s3_error!(InternalError, "marshal body failed, e: {:?}", e))?;
let (body, content_type) = encode_compatible_admin_payload(req.uri().path(), &cred.secret_key, body)?;
let (body, content_type) = encode_compatible_admin_payload(req.uri().path(), &response_secret, body)?;
let mut header = HeaderMap::new();
header.insert(CONTENT_TYPE, content_type.parse().expect("valid header value"));
@@ -1851,22 +1852,58 @@ mod tests {
assert_eq!(err.message(), Some("service account 'missing' does not exist"));
}
#[tokio::test]
async fn delegated_service_account_replication_survives_request_cancellation() {
use tokio::sync::oneshot;
use tokio::time::{Duration, timeout};
let (started_tx, started_rx) = oneshot::channel();
let (release_tx, release_rx) = oneshot::channel();
let (event_tx, event_rx) = oneshot::channel();
let updated_at = OffsetDateTime::now_utc();
let create = async move {
started_tx.send(()).expect("signal creation");
release_rx.await.expect("release write");
let credentials = StoredCredentials {
access_key: "backup-key".to_owned(),
parent_user: "backup-user".to_owned(),
..Default::default()
};
Ok((credentials, updated_at, HashMap::new()))
};
let waiter = tokio::spawn(create_and_replicate_service_account(create, None, None, |item| async move {
event_tx.send(item).expect("capture site replication event");
Ok(())
}));
let deadline = Duration::from_secs(5);
timeout(deadline, started_rx)
.await
.expect("creation started")
.expect("start signal");
waiter.abort();
assert!(waiter.await.expect_err("request cancelled").is_cancelled());
release_tx.send(()).expect("write still admitted");
let item = timeout(deadline, event_rx)
.await
.expect("replication completed")
.expect("replication event");
assert_eq!(item.updated_at, Some(updated_at));
let created = item.svc_acc_change.and_then(|change| change.create).expect("create event");
assert_eq!(created.access_key, "backup-key");
assert_eq!(created.parent, "backup-user");
}
#[test]
fn add_service_account_maps_iam_create_errors_to_s3_errors() {
let src = include_str!("service_account.rs");
let create_start = src
.find("impl Operation for AddServiceAccount")
.expect("AddServiceAccount operation should exist");
let create_block = &src[create_start..];
let create_end = create_block
.find("if let Err(err) = site_replication_iam_change_hook")
.expect("site replication hook marker should exist");
let create_block = &create_block[..create_end];
use rustfs_iam::error::Error;
assert!(
create_block.contains("AccessKeyAlreadyExists") && create_block.contains("iam_error_to_s3_error(e)"),
"duplicate access keys must map to client S3 errors"
);
for (error, expected) in [
(Error::AccessKeyAlreadyExists, S3ErrorCode::InvalidArgument),
(Error::NoSuchUser("missing".into()), S3ErrorCode::InvalidRequest),
(Error::Io(std::io::Error::other("storage unavailable")), S3ErrorCode::InternalError),
] {
assert_eq!(map_service_account_create_error(error).code(), &expected);
}
}
#[test]
+38
View File
@@ -922,6 +922,44 @@ pub(crate) fn claims_route(method: &Method, uri: &Uri, headers: &HeaderMap) -> b
|| is_control_plane_route(method, uri, headers)
}
/// Signed system-status and catalog-configuration reads, after object-lambda routing.
#[cfg(feature = "gateway-stack")]
pub(crate) fn claims_native_admin_route(method: &Method, uri: &Uri) -> bool {
method == Method::GET
&& matches!(
uri.path(),
"/rustfs/admin/v3/info"
| "/minio/admin/v3/info"
| "/rustfs/admin/v3/storageinfo"
| "/minio/admin/v3/storageinfo"
| "/rustfs/admin/v4/runtime/capabilities"
| "/minio/admin/v4/runtime/capabilities"
| "/rustfs/admin/v3/object-data-cache/stats"
| "/minio/admin/v3/object-data-cache/stats"
| "/rustfs/admin/v3/datausageinfo"
| "/minio/admin/v3/datausageinfo"
| "/rustfs/admin/v4/plugins/catalog"
| "/minio/admin/v4/plugins/catalog"
| "/rustfs/admin/v4/cluster/snapshot"
| "/minio/admin/v4/cluster/snapshot"
| "/rustfs/admin/v4/extensions/catalog"
| "/minio/admin/v4/extensions/catalog"
| "/rustfs/admin/v3/module-switches"
| "/minio/admin/v3/module-switches"
| "/iceberg/v1/config"
| "/_iceberg/v1/config"
)
&& parse_object_lambda_request(method, uri).is_none()
}
/// Console paths whose requests do not belong to an earlier extension route.
#[cfg(feature = "gateway-stack")]
pub(crate) fn claims_native_console_route(method: &Method, uri: &Uri) -> bool {
is_console_path(uri.path())
&& parse_object_lambda_request(method, uri).is_none()
&& s3_extension::route_key(method, uri).is_none()
}
fn is_public_health_path(path: &str) -> bool {
path == HEALTH_PREFIX || path == HEALTH_READY_PATH
}
+35 -9
View File
@@ -33,9 +33,9 @@ use super::storage_api::bucket_usecase::bucket::{
run_stale_multipart_upload_cleanup_once, validate_lifecycle_config, validate_transition_tier,
},
metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG,
BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TARGETS_FILE,
BUCKET_VERSIONING_CONFIG,
BUCKET_ACCELERATE_CONFIG, BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG,
BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG, BUCKET_REPLICATION_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_SSECONFIG,
BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, BUCKET_WEBSITE_CONFIG,
},
metadata_sys,
policy_sys::PolicySys,
@@ -61,10 +61,7 @@ use super::storage_api::bucket_usecase::s3_api::bucket::{
ListObjectVersionsParams, ListObjectsV2Params, encode_list_objects_v2_continuation_token, parse_list_object_versions_params,
parse_list_objects_v2_params, rustfs_owner,
};
use super::storage_api::bucket_usecase::{
get_validated_store, process_lambda_configurations, process_queue_configurations, process_topic_configurations,
request_context, validate_list_object_unordered_with_delimiter,
};
use super::storage_api::bucket_usecase::{get_validated_store, request_context, validate_list_object_unordered_with_delimiter};
#[cfg(feature = "diagnostic-read-version-caller")]
use super::storage_api::diagnostic;
use crate::app::RequestEnvelope;
@@ -95,7 +92,6 @@ use rustfs_gateway_types::persistence::{
PersistedVersioningConfiguration,
};
use rustfs_gateway_types::persistence::{PersistedLifecycleConfiguration, PersistedReplicationConfiguration};
use rustfs_io_metrics::record_s3_op;
use rustfs_madmin::{SITE_REPL_API_VERSION, SRBucketMeta};
use rustfs_policy::policy::{
action::{Action, S3Action},
@@ -128,6 +124,8 @@ mod core;
#[cfg(any(test, feature = "gateway-stack"))]
pub(crate) use self::core::requested_location_constraint;
mod cors;
#[cfg(test)]
mod delete_config_tests;
mod encryption;
#[cfg(test)]
mod guards;
@@ -138,6 +136,7 @@ pub(crate) mod list_objects;
mod listing;
mod logging;
mod notification;
pub(crate) use notification::NotificationDestination;
mod object_lock;
mod policy;
mod public_access_block;
@@ -275,6 +274,23 @@ pub(crate) async fn notify_table_bucket_metadata_reload(
notify_bucket_metadata_reload_with_context(bucket, operation, None, false, app_context).await;
}
#[cfg(test)]
type BucketMetadataReloadObserver = Box<dyn Fn(bool, &'static str, bool) + Send + Sync>;
#[cfg(test)]
tokio::task_local! {
static BUCKET_METADATA_RELOAD_PROBE: (String, BucketMetadataReloadObserver);
}
#[cfg(test)]
fn record_bucket_metadata_reload(bucket: &str, completed: bool, operation: &'static str, scanner_maintenance_change: bool) {
let _ = BUCKET_METADATA_RELOAD_PROBE.try_with(|(observed_bucket, record)| {
if observed_bucket == bucket {
record(completed, operation, scanner_maintenance_change);
}
});
}
async fn notify_bucket_metadata_reload_with_context(
bucket: String,
operation: &'static str,
@@ -282,6 +298,11 @@ async fn notify_bucket_metadata_reload_with_context(
scanner_maintenance_change: bool,
app_context: Option<Arc<AppContext>>,
) {
#[cfg(test)]
record_bucket_metadata_reload(&bucket, false, operation, scanner_maintenance_change);
#[cfg(test)]
let observed_bucket = bucket.clone();
record_local_scanner_maintenance_reload(&bucket, scanner_maintenance_change);
// Keep reload detached across request cancellation, but wait before a healthy peer can serve the previous config.
let (completed_tx, completed_rx) = tokio::sync::oneshot::channel();
@@ -303,7 +324,12 @@ async fn notify_bucket_metadata_reload_with_context(
}
let _ = completed_tx.send(());
});
let _ = completed_rx.await;
let completed = completed_rx.await;
#[cfg(test)]
if completed.is_ok() {
record_bucket_metadata_reload(&observed_bucket, true, operation, scanner_maintenance_change);
}
let _ = completed;
}
fn record_local_scanner_maintenance_reload(bucket: &str, scanner_maintenance_change: bool) {
+49 -1
View File
@@ -12,4 +12,52 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//! Bucket accelerate configuration use-case operations (reserved for the next batch).
//! Bucket accelerate configuration shared by the S3 edges.
use super::BUCKET_ACCELERATE_CONFIG;
use super::{
ApiError, DefaultBucketUsecase, S3Operation, StatusCode, StorageError, bucket_config_mutation_incarnation, metadata_sys,
serialize, update_bucket_config_for_incarnation,
};
use crate::app::RequestEnvelope;
use rustfs_gateway_types::persistence::PersistedAccelerateConfiguration;
use rustfs_s3_types::S3Result;
impl DefaultBucketUsecase {
pub(crate) async fn execute_get_bucket_accelerate_configuration(
&self,
bucket: &str,
) -> S3Result<PersistedAccelerateConfiguration> {
self.ensure_config_bucket(bucket).await?;
match metadata_sys::get_accelerate_config(bucket).await {
Ok((config, _)) => Ok(config),
Err(StorageError::ConfigNotFound) => Ok(PersistedAccelerateConfiguration::default()),
Err(err) => Err(ApiError::from(err).into()),
}
}
pub(crate) async fn execute_put_bucket_accelerate_configuration(
&self,
envelope: RequestEnvelope,
bucket: &str,
configuration: impl FnOnce() -> S3Result<PersistedAccelerateConfiguration> + Send,
) -> S3Result<()> {
RequestEnvelope::audit_only_operation(
envelope,
S3Operation::PutBucketAccelerateConfiguration,
StatusCode::OK,
|envelope| async move {
let expected_incarnation_id = bucket_config_mutation_incarnation(&envelope, bucket)?;
self.ensure_config_bucket(bucket).await?;
let config = configuration()?;
let data = serialize(&config).map_err(|err| rustfs_s3_types::s3_error!(MalformedXML, "{err}"))?;
update_bucket_config_for_incarnation(bucket, BUCKET_ACCELERATE_CONFIG, data, expected_incarnation_id)
.await
.map_err(ApiError::from)?;
Ok(())
},
)
.await
}
}
+19
View File
@@ -17,6 +17,25 @@
use super::*;
impl DefaultBucketUsecase {
pub(crate) async fn execute_put_bucket_acl(
&self,
envelope: RequestEnvelope,
bucket: &str,
has_xml_policy: bool,
) -> rustfs_s3_types::S3Result<()> {
RequestEnvelope::audit_only_operation(envelope, S3Operation::PutBucketAcl, StatusCode::OK, |_| async move {
self.ensure_config_bucket(bucket).await?;
if has_xml_policy {
return Err(rustfs_s3_types::s3_error!(
NotImplemented,
"ACL XML grants are not supported; use canned ACL headers or omit ACL"
));
}
Ok(())
})
.await
}
/// RustFS has one owner and no ACL storage: every bucket answers that owner with full control.
pub async fn execute_get_bucket_acl(
&self,
+12
View File
@@ -55,6 +55,18 @@ pub(crate) fn requested_location_constraint(input: &gateway_dto::CreateBucketInp
}
impl DefaultBucketUsecase {
/// Checks that the bucket exists before reading or writing a stored configuration.
pub(super) async fn ensure_config_bucket(&self, bucket: &str) -> rustfs_s3_types::S3Result<()> {
let Some(store) = self.object_store() else {
return Err(rustfs_s3_types::s3_error!(InternalError, "Not init"));
};
store
.get_bucket_info(bucket, &BucketOptions::default())
.await
.map_err(ApiError::from)?;
Ok(())
}
fn create_bucket_exists_result(&self) -> rustfs_s3_types::S3Result<gateway_dto::CreateBucketOutput> {
let region = self
.global_region()
+17 -21
View File
@@ -15,8 +15,6 @@
//! Bucket CORS configuration use-case operations.
use super::*;
use crate::app::storage_api::s3::dto::GetBucketCorsInput;
use crate::app::storage_api::s3::{S3Error, S3ErrorCode, S3Request, S3Result};
use rustfs_gateway_types::cors_tagging::parse_cors;
use rustfs_gateway_types::persistence::serialize_cors_dto;
@@ -64,26 +62,30 @@ impl DefaultBucketUsecase {
.await
}
#[instrument(level = "debug", skip(self, req))]
pub async fn execute_get_bucket_cors(&self, req: S3Request<GetBucketCorsInput>) -> S3Result<PersistedCorsConfiguration> {
let GetBucketCorsInput { bucket, .. } = req.input;
/// Returns the stored bucket CORS configuration after checking the bucket exists.
///
/// # Errors
///
/// Returns `InternalError` if the store is uninitialized, `NoSuchCORSConfiguration`
/// if the configuration is absent, or the storage error from either lookup.
#[instrument(level = "debug", skip(self, bucket))]
pub async fn execute_get_bucket_cors(&self, bucket: &str) -> rustfs_s3_types::S3Result<PersistedCorsConfiguration> {
let Some(store) = self.object_store() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
return Err(rustfs_s3_types::s3_error!(InternalError, "Not init"));
};
store
.get_bucket_info(&bucket, &BucketOptions::default())
.get_bucket_info(bucket, &BucketOptions::default())
.await
.map_err(ApiError::from)?;
let cors_configuration = match metadata_sys::get_cors_config(&bucket).await {
let cors_configuration = match metadata_sys::get_cors_config(bucket).await {
Ok((config, _)) => config,
Err(err) => {
if err == StorageError::ConfigNotFound {
return Err(S3Error::with_message(
S3ErrorCode::NoSuchCORSConfiguration,
"The CORS configuration does not exist".to_string(),
return Err(rustfs_s3_types::s3_error!(
NoSuchCORSConfiguration,
"The CORS configuration does not exist"
));
}
warn!(
@@ -154,7 +156,7 @@ impl DefaultBucketUsecase {
#[cfg(test)]
mod tests {
use super::*;
use crate::app::bucket_usecase::test_support::{build_request, gateway_bucket, owned_envelope};
use crate::app::bucket_usecase::test_support::{gateway_bucket, owned_envelope};
use http::Method;
#[tokio::test]
@@ -174,16 +176,10 @@ mod tests {
#[tokio::test]
async fn execute_get_bucket_cors_returns_internal_error_when_store_uninitialized() {
let input = GetBucketCorsInput::builder()
.bucket("test-bucket".to_string())
.build()
.unwrap();
let req = build_request(input, Method::GET);
let usecase = DefaultBucketUsecase::without_context();
let err = usecase.execute_get_bucket_cors(req).await.unwrap_err();
assert_eq!(err.code(), &S3ErrorCode::InternalError);
let err = usecase.execute_get_bucket_cors("test-bucket").await.unwrap_err();
assert_eq!(err.code(), &rustfs_s3_types::S3ErrorCode::InternalError);
}
#[tokio::test]
@@ -0,0 +1,350 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Bucket configuration mutation tests over the shared use case and durable metadata.
use super::*;
use crate::app::bucket_usecase::test_support::build_request;
use crate::app::gating_test_env::{run_large_stack_test, shared_gating_ambient};
use crate::app::runtime_sources::ServerContextSlot;
use crate::app::storage_api::bucket_usecase::helper::MutationCompletionProbe;
use crate::app::storage_api::owned_parity::authorize_bucket_config_mutation;
use crate::site_replication::hooks::BUCKET_META_HOOK_CALLS;
use http::Method;
use rustfs_credentials::Credentials;
use rustfs_s3_types::{S3ErrorCode, S3Result};
use serial_test::serial;
use std::{cell::RefCell, sync::Mutex};
#[derive(Clone, Copy)]
enum Config {
Encryption,
Policy,
Tagging,
}
impl Config {
async fn seed(self, bucket: &str) {
match self {
Self::Encryption => {
metadata_sys::update(
bucket,
BUCKET_SSECONFIG,
b"<ServerSideEncryptionConfiguration><Rule><ApplyServerSideEncryptionByDefault>\
<SSEAlgorithm>AES256</SSEAlgorithm></ApplyServerSideEncryptionByDefault></Rule></ServerSideEncryptionConfiguration>"
.to_vec(),
)
.await
}
Self::Policy => {
metadata_sys::update(bucket, BUCKET_POLICY_CONFIG, br#"{"Version":"2012-10-17","Statement":[]}"#.to_vec()).await
}
Self::Tagging => {
let tags = rustfs_gateway_types::cors_tagging::parse_tagging(
b"<Tagging><TagSet><Tag><Key>team</Key><Value>storage</Value></Tag></TagSet></Tagging>",
)
.expect("valid tagging fixture");
metadata_sys::update_s3_tagging_config(bucket, Some(tags), None).await
}
}
.expect("seed bucket configuration");
}
async fn stored(self, bucket: &str) -> Vec<u8> {
let metadata = metadata_sys::get_global_bucket_metadata_sys()
.expect("initialized metadata system")
.read()
.await
.get_config_from_disk(bucket)
.await
.expect("read bucket metadata from disk");
match self {
Self::Encryption => metadata.encryption_config_xml,
Self::Policy => metadata.policy_config_json,
Self::Tagging => metadata.tagging_config_xml,
}
}
async fn put(self, usecase: &DefaultBucketUsecase, envelope: RequestEnvelope, bucket: &str) -> S3Result<()> {
match self {
Self::Encryption => {
let config = rustfs_gateway_types::persistence::parse_bucket_encryption(ENCRYPTION_WRITE).unwrap();
usecase.execute_put_bucket_encryption(envelope, bucket, Ok(config)).await
}
Self::Policy => usecase.execute_put_bucket_policy(envelope, bucket, POLICY_WRITE).await,
Self::Tagging => {
let tags = rustfs_gateway_types::cors_tagging::parse_tagging(TAGGING_WRITE).unwrap();
usecase.execute_put_bucket_tagging(envelope, bucket, tags).await
}
}
}
async fn execute(self, usecase: &DefaultBucketUsecase, envelope: RequestEnvelope, bucket: &str) -> S3Result<()> {
match self {
Self::Encryption => usecase.execute_delete_bucket_encryption(envelope, bucket).await,
Self::Policy => usecase.execute_delete_bucket_policy(envelope, bucket).await,
Self::Tagging => usecase.execute_delete_bucket_tagging(envelope, bucket).await,
}
}
}
const ENCRYPTION_WRITE: &[u8] = b"<ServerSideEncryptionConfiguration><Rule><ApplyServerSideEncryptionByDefault><SSEAlgorithm>aws:kms</SSEAlgorithm><KMSMasterKeyID>fixture-key</KMSMasterKeyID></ApplyServerSideEncryptionByDefault></Rule></ServerSideEncryptionConfiguration>";
const POLICY_WRITE: &str = "{ \"Version\": \"2012-10-17\", \"Statement\": [] } ";
const TAGGING_WRITE: &[u8] = b"<Tagging><TagSet><Tag><Key>team</Key><Value>gateway</Value></Tag></TagSet></Tagging>";
async fn check_mutation(config: Config, stale: bool, write: bool) {
let (subresource, action, operation, reload, replicated, audit_status) = match (config, write) {
(Config::Encryption, false) => (
"encryption",
S3Action::PutBucketEncryptionAction,
S3Operation::DeleteBucketEncryption,
"delete bucket encryption",
"sse-config",
204,
),
(Config::Policy, false) => (
"policy",
S3Action::DeleteBucketPolicyAction,
S3Operation::DeleteBucketPolicy,
"delete bucket policy",
"policy",
200,
),
(Config::Tagging, false) => (
"tagging",
S3Action::PutBucketTaggingAction,
S3Operation::DeleteBucketTagging,
"delete bucket tagging",
"tags",
200,
),
(Config::Encryption, true) => (
"encryption",
S3Action::PutBucketEncryptionAction,
S3Operation::PutBucketEncryption,
"put bucket encryption",
"sse-config",
200,
),
(Config::Policy, true) => (
"policy",
S3Action::PutBucketPolicyAction,
S3Operation::PutBucketPolicy,
"put bucket policy",
"policy",
200,
),
(Config::Tagging, true) => (
"tagging",
S3Action::PutBucketTaggingAction,
S3Operation::PutBucketTagging,
"put bucket tagging",
"tags",
200,
),
};
let context = shared_gating_ambient().await;
let store = context.object_store();
let bucket = format!("config-mutation-{}", uuid::Uuid::new_v4().simple());
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("create bucket");
let server = ServerContextSlot::new();
assert!(server.install(Arc::clone(&context)));
let mut request = build_request((), if write { Method::PUT } else { Method::DELETE });
request.uri = format!("/{bucket}?{subresource}").parse().expect("request URI");
request.extensions.insert(Arc::clone(&server));
request.extensions.insert(ReqInfo {
cred: Some(Credentials {
access_key: "config-delete-owner".to_owned(),
..Default::default()
}),
is_owner: true,
..Default::default()
});
request.extensions.insert(request_context::RequestContext {
request_id: "bucket-config-delete".to_owned(),
x_amz_request_id: "bucket-config-delete".to_owned(),
trace_id: None,
span_id: None,
start_time: std::time::Instant::now(),
});
let mut envelope = RequestEnvelope::from_s3s(&request);
authorize_bucket_config_mutation(&server, &mut envelope, &bucket, Action::S3Action(action))
.await
.expect("capture authorized bucket incarnation");
if stale {
store
.delete_bucket(&bucket, &DeleteBucketOptions::default())
.await
.expect("delete authorized bucket");
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("recreate bucket");
}
config.seed(&bucket).await;
let seeded = config.stored(&bucket).await;
assert!(!seeded.is_empty(), "durable configuration fixture");
let before = rustfs_scanner::scanner_dirty_usage_bucket_generation(&bucket);
let maintenance_before = rustfs_scanner::scanner_maintenance_generation();
let audits = MutationCompletionProbe::install(&bucket);
let reloads = Arc::new(Mutex::new(Vec::new()));
let observe = Arc::clone(&reloads);
let usecase = DefaultBucketUsecase::with_context(Some(context));
let (result, hooks) = BUCKET_META_HOOK_CALLS
.scope(RefCell::default(), async {
let result = BUCKET_METADATA_RELOAD_PROBE
.scope(
(
bucket.clone(),
Box::new(move |completed, label, maintenance| {
let replication_calls = BUCKET_META_HOOK_CALLS.with(|calls| calls.borrow().len());
observe
.lock()
.expect("reload entries")
.push((completed, label, maintenance, replication_calls));
}),
),
async {
if write {
config.put(&usecase, envelope, &bucket).await
} else {
config.execute(&usecase, envelope, &bucket).await
}
},
)
.await;
(result, BUCKET_META_HOOK_CALLS.with(RefCell::take))
})
.await;
let stored = config.stored(&bucket).await;
let after = rustfs_scanner::scanner_dirty_usage_bucket_generation(&bucket);
let reloads = reloads.lock().expect("reload entries");
let entries = audits.audits.lock().expect("audit entries");
assert!(audits.events.lock().expect("event entries").is_empty());
let audit = entries
.iter()
.map(|entry| (entry.api.name.as_deref(), entry.api.status_code))
.collect::<Vec<_>>();
if stale {
assert_eq!(
(
result.err().map(|err| err.code().clone()),
stored,
reloads.len(),
hooks.len(),
after,
audit
),
(
Some(S3ErrorCode::NoSuchBucket),
seeded,
0,
0,
before,
vec![(Some(operation.as_str()), Some(404))]
),
"{operation:?} must refuse the stale authorization and preserve replacement bytes"
);
} else {
result.expect("authorized mutation");
if write {
match config {
Config::Encryption => {
let actual = rustfs_gateway_types::persistence::parse_bucket_encryption(&stored).unwrap();
let expected = rustfs_gateway_types::persistence::parse_bucket_encryption(ENCRYPTION_WRITE).unwrap();
assert_eq!(actual, expected);
}
Config::Policy => assert_eq!(stored, POLICY_WRITE.as_bytes()),
Config::Tagging => {
let actual = rustfs_gateway_types::cors_tagging::parse_tagging(&stored).unwrap();
let expected = rustfs_gateway_types::cors_tagging::parse_tagging(TAGGING_WRITE).unwrap();
assert_eq!(actual, expected);
}
}
} else {
assert!(stored.is_empty(), "{operation:?} must remove the durable configuration");
}
assert_eq!(*reloads, [(false, reload, false, 0), (true, reload, false, 0)]);
let [hook] = hooks.as_slice() else { panic!("{operation:?}: expected one replication call") };
assert_eq!((&hook.bucket, hook.item_type.as_str()), (&bucket, replicated));
assert_eq!(hook.scanner_maintenance_generation, maintenance_before);
assert_eq!(rustfs_scanner::scanner_maintenance_generation(), maintenance_before);
if matches!(config, Config::Tagging) {
assert!(after > hook.scanner_generation);
} else {
assert_eq!(after, hook.scanner_generation);
}
assert_eq!(audit, [(Some(operation.as_str()), Some(audit_status))]);
assert_eq!(entries[0].request_id.as_deref(), Some("bucket-config-delete"));
}
}
#[test]
#[serial]
fn encryption_delete_fences_recreated_bucket_and_orders_callbacks() {
run_large_stack_test("delete-encryption", || async {
check_mutation(Config::Encryption, false, false).await;
check_mutation(Config::Encryption, true, false).await;
});
}
#[test]
#[serial]
fn policy_delete_fences_recreated_bucket_and_orders_callbacks() {
run_large_stack_test("delete-policy", || async {
check_mutation(Config::Policy, false, false).await;
check_mutation(Config::Policy, true, false).await;
});
}
#[test]
#[serial]
fn tagging_delete_fences_recreated_bucket_and_orders_callbacks() {
run_large_stack_test("delete-tagging", || async {
check_mutation(Config::Tagging, false, false).await;
check_mutation(Config::Tagging, true, false).await;
});
}
#[test]
#[serial]
fn encryption_put_fences_recreated_bucket_and_orders_callbacks() {
run_large_stack_test("put-encryption", || async {
check_mutation(Config::Encryption, false, true).await;
check_mutation(Config::Encryption, true, true).await;
});
}
#[test]
#[serial]
fn policy_put_fences_recreated_bucket_and_orders_callbacks() {
run_large_stack_test("put-policy", || async {
check_mutation(Config::Policy, false, true).await;
check_mutation(Config::Policy, true, true).await;
});
}
#[test]
#[serial]
fn tagging_put_fences_recreated_bucket_and_orders_callbacks() {
run_large_stack_test("put-tagging", || async {
check_mutation(Config::Tagging, false, true).await;
check_mutation(Config::Tagging, true, true).await;
});
}

Some files were not shown because too many files have changed in this diff Show More