mirror of
https://github.com/rustfs/rustfs.git
synced 2026-11-18 14:05:27 +00:00
10cb203c76290c8b95cd90c94004c19bbfd5fd15
1126 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b417d25165 | fix(auth): require separate permission to change OIDC providers | ||
|
|
4aded279ae | docs(storage): clarify drive isolation for separate clusters (#9493) | ||
|
|
1c4a7c2f8b | feat(gateway): serve browser object uploads | ||
|
|
0717dc1b0d | fix(gateway): honor configured unread body timeout | ||
|
|
f2003af73d |
fix(ecstore): recover deletes on full disks (#9465)
* fix(ecstore): reserve emergency space for delete journals Keep bounded physical metadata headroom for recoverable deletion on Unix without bypassing quorum or rollback. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): recover delete metadata space on demand Share a pinned reserve owner per physical disk and keep healthy deletion free of reserve filesystem probes. Retry only disposable staging and idempotent metadata primitives after actual ENOSPC, preserving transaction publication and quorum boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(ecstore): retry concurrent full-disk deletes Serialize ENOSPC waiters per physical disk and give each safe metadata primitive one bounded retry against shared released capacity. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ci): account for delete reserves in receipt fixture Reject undersized tmpfs mounts after including the eight per-disk delete reserves and keep a self-test for the capacity boundary. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(heal): refresh resume admission fixture lease Refresh the existing node admission sample immediately before starting resume GC so the cancellation test reaches the native-open ownership boundary. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): inject receipt ENOSPC on hardlink path Keep the receipt fault at the atomic allocation boundary after the stage-two hardlink fast path landed, so reserve recovery remains covered. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
cbe5b16758 | test(rpc): verify timeout boundary evidence (#9482) | ||
|
|
571d100a27 |
perf(ecstore): align inline placement and batch deletion codecs (#9449)
* feat(ecstore): align inline budgets and binary batch decoding Use a fixed 128 KiB per-shard inline budget with versioned admission and preserve historical configuration markers and persisted object layouts. Decode binary delete batches independently of the legacy JSON list while retaining JSON-only requests and dual-write senders. Reject malformed binary entries before disk mutation and preallocate the selected batch. Cover budget boundaries, layout coexistence, MinIO metadata fixtures, binary-only batches and JSON compatibility. Document geometry and rollback boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): add isolated Docker inline geometry validation Build clean-source Linux validation images from a frozen Git bundle. Pin running image identities and binary hashes, and inspect persisted object geometry through the existing filemeta decoder. Provide an owned four-node Docker adapter for the existing ABBA runner. Guard resource ownership, isolate loopback access, serialize state mutations, and verify HEAD, full GET and Range contents. Keep PUT content probes available through an isolated Linux mc adapter when a host client cannot initialize. Cover ownership and failure paths with script tests wired into the existing runner. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): reuse bounded verified inline bodies Share the materialization contract between metadata pre-verification and buffered inline GETs. Bound responses to 2 MiB and a limited frame count, and reuse an already verified body within the selected snapshot. Route proof-protected parts through the readers that bind PartProofReader. Cover larger single-block bodies, allocation bounds and proof validation after a shard's bitrot checksum has been recomputed. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(ecstore): support bounded multi-block inline objects Keep the default EC block and accumulated shard budget while bounding inline frame storage and verified-body reuse. Preserve transformed and protected write admission and existing metadata formats. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): isolate inline transport comparison modes Pin and verify both MessagePack-only gates for each Docker cell, preserve an explicit JSON compatibility cell and probe exact accumulated shard boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): cover inline rollback and mixed-reader fleets Add reusable compatibility fixtures for null and versioned objects, cross-block ranges, unfinished multipart uploads and batch version deletion. Record coordinator identity and per-version geometry, preserve retained-data manifests and archive manifests on dataset resets. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): unify bounded inline read and write paths Share checked frame decoding across metadata preverification and GET, resolve write admission once, and use one checked physical shard length for inline and disk writers. Preserve proof-bound readers and erasure recovery for damaged or missing shards. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(testing): persist inline compatibility mutation receipts Save successful multipart and delete receipts before fallible content and geometry checks. Resume completed mutations through verification without repeating writes, and cover read failures, geometry failures, and command retries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(gateway): isolate capability probes from multipart listings Give successful capability probes their own buckets so CreateMultipartUpload does not change the exact two-upload pagination fixtures. Preserve all listing-boundary assertions on both HTTP stacks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(rpc): honor fleet-confirmed codecs for batch deletion Build DeleteVersions requests with one codec policy for the complete batch. Emit JSON compatibility fields by default and binary-only batches after both existing fleet gates pass. Cover request fields, canonical binding and actual remote sender, handler and disk deletion in both modes. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(ecstore): verify inline shards through borrowed frames Share borrowed frame verification across buffered GET, bitrot readers and deep inline healing. Validate exact physical framing before scanning memory slices. Cover short prefixes, legacy padded frames, damage and trailing data, and provide a reproducible manual ABBA kernel comparison. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ci): account for manual inline verification benchmark Raise the ECStore ignored-test ceiling from 2 to 3 for the opt-in ABBA comparison of borrowed and streaming bitrot verification. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): reject inexact inline shard framing Validate the complete encoded length of every selected inline data shard before decoding or caching a verified object body. Return to the erasure reader on missing, truncated, or trailing payload bytes. Extend decoder fixtures across single and multi-block geometry, legacy padding, unhashed payloads, and explicit bitrot verification bypass. Cover metadata early-stop rejection and GET recovery for trailing shard bytes. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): share inline shard reader construction Build owned-metadata and borrowed-metadata readers through one Send iterator-based factory. Preserve shard ordering, empty slots, bitrot settings and caller-bound integrity verification. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(rpc): move endpoint buffers into batch delete requests Transfer the prepared endpoint String into the protobuf request and assert allocation identity across every batch codec mode. This removes the intermediate endpoint copy during request construction. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(gateway): remove superseded pagination isolation Use the upstream capability probes that clean up their own multipart sessions. Run probes against the seeded pagination bucket while keeping all exact page and cursor assertions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(table-catalog): recognize borrowed multipart guard calls Count owned and borrowed bucket/key guard forms while requiring all four production calls. Cover mixed split-file calls and reject wrong resources, missing error propagation and test-only mentions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): size heal cleanup fixtures above inline budgets Use an environment-independent per-pool fixture policy and payload sizes above the aggregate inline budget. Assert external placement before pausing old-data cleanup, rebuilding the committed generation and checking physical shards and the final payload. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(table-catalog): count borrowed object metadata guards Recognize the borrowed bucket/key guards in migrated legal-hold and retention entrypoints. Validate two object and two key guards and cover incorrect resources, missing propagation and test-only mentions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
849cdd6319 |
feat(ecstore): stage quorum-proof metadata cache (#9454)
* feat(lock): carry cache fence quorum proofs Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(ecstore): stage proof-bound distributed cache Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * docs(ecstore): refine cache activation blockers Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
|
dc7f4e943c |
fix(ecstore): unify commit durability and reduce inline dispatches (#9451)
* fix(ecstore): honor file-sync policy in inline fallback commits Apply metadata, directory, and recovery synchronization policies separately while retaining rollback publication barriers. Add sync-budget regressions and small-object I/O profiling guidance. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): unify recovery-aware commit synchronization Share recovery-directory policy across fused and fallback commits. Complete fresh file-sync publication within preparation and verify queue budgets, rollback recovery, and inline-to-external overwrites. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): isolate sync counts across reused disk handles Consume the metadata synchronization count for every external write attempt before releasing a Linux procfs-backed fixture, including failed publication and retry. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
0278e1cd0e |
fix(iceberg): recover renames and unblock namespace deletion (#9440)
* fix(iceberg): recover renames and unblock namespace deletion * fix(iceberg): fence rename recovery with restore lease --------- Co-authored-by: Hauser Zhi <housemecn@gmail.com> |
||
|
|
fb9f19c6d2 |
fix(rpc): scope canary errors to grpc failures (#9453)
* fix(rpc): scope canary errors to grpc failures Exclude expected read-version remote responses without hiding transport, gRPC status, response-decode, or other gRPC operation failures. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(rpc): verify canary revisions from image history Use immutable image history only when recognized config labels are absent, keep mismatches fail-closed, and record the provenance source in verifier output. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(audit): restore bucket config audits and metadata guards Audit accelerate, request-payment, and website configuration mutations through the shared owned request scope, including failures and the website deletion status. Validate the replication mutation delegation before checking peer reload notifications. Count borrowed reserved-prefix guard calls and cover missing production calls, test-only mentions, audit completion, and stale bucket incarnations. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
|
a8cf62f8f3 |
fix(metrics): label grouped fdatasync batches accurately (#9446)
* fix(metrics): label grouped fdatasync batches accurately Record the serialized user-space fdatasync coordinator as grouped and reserve parallel for independent per-file fanout. This keeps performance dashboards from conflating two different dispatch models. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): separate grouping from timeout race Keep the same-disk batching test focused on queue aggregation. Claimed-follower timeout ownership remains covered by its dedicated deterministic regression, avoiding a valid pre-claim timeout race in the batching fixture. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(table-catalog): accept borrowed multipart guard calls Count both owned String and borrowed &str spellings of the same reserved-prefix guard. UploadPart now obtains borrowed bucket and key values from its typed target, while the production guard remains in place. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
d0fd6425e4 |
feat(rpc): guard stale generations before dispatch (#9435)
* perf(grpc): attribute rebuilds and bound bulk responses Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(grpc): attribute rebuild success after publication Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(obs): add exact HTTP/2 ping timeout diagnostics Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(multipart): run refusal matrix on large stack Run the real-store completion refusal matrix through the shared 32 MiB test harness so its deep async storage future does not overflow libtest's default stack. Keep the original test name and all refusal-order assertions intact. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * chore(deps): pin nixl-sys workspace source Manage the nixl-sys Git revision from the workspace dependency table and point the optional Linux transport dependency at houseme/nixl commit cc636c83f13d692e85b67e55aed092413d16fb41. Allow the reviewed source in cargo-deny and refresh the lockfile, replacing the unmaintained bincode dependency with the fork's wincode graph. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(rpc): authenticate msgpack peer capability Add an optional boot-epoch-bound HMAC capability proof and cache the verified per-peer observation without enabling any msgpack send policy. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(grpc): bind peer capability to connection generations (#9433) * feat(grpc): track physical connection generations Observe every tonic physical reconnect through a stable custom connector while preserving existing TCP, TLS, HTTP/2, cache, and public channel behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(rpc): bind msgpack capability to connections Scope authenticated msgpack observations to channel identity and physical connection generation so transparent reconnects require a fresh proof without enabling the send policy. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> * feat(rpc): guard stale generations before dispatch Reject generation-bound requests before inner transport call or first payload frame when tonic readiness observes a different physical connection, while leaving ordinary mutations unguarded until the send-policy follow-up. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(rpc): add per-peer msgpack rollout gate Add the default-off peer-capability environment gate, cached evaluation, fixed-cardinality send-decision metrics, and rollout constraints for stable peer-affine connections. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(rpc): apply per-peer msgpack send policy Capture one capability decision per eligible RPC, bind peer-mode requests to boot and connection generations, and rebuild at most one dual-write request only after a locally proven pre-dispatch rejection. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(rpc): verify msgpack peer canary evidence Add a strict three-phase four-node evidence verifier with raw Prometheus and workload receipts, adversarial tests, script-test wiring, and the operational manifest contract. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(rpc): align batch read manifest encoder Match the BatchReadVersion compatibility manifest to the request-owned encoder expression used by the production send path. Restore the owning ECStore source guard while preserving wire and runtime behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(rpc): orchestrate msgpack peer canary Run the fixed mixed/all-new/rollback four-node matrix with prebuilt images, real workload and readback, raw Prometheus capture, strict manifest generation, and adversarial fake integration tests. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(rpc): verify canary image and telemetry identities * test(rpc): bind canary evidence provenance Scope canary metrics to one injected cluster identity and bind candidate, baseline, and per-phase running containers to immutable image IDs and exact build revisions. Reject foreign-cluster metrics, stale revision labels, same-image tags, and phase image mismatches. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(table-catalog): accept borrowed multipart guard calls Count the exact borrowed bucket/key guard call introduced by UploadPart target ownership while preserving the four-production-call contract. Cover the borrowed spelling in both the positive fixture and the test-only false-positive regression. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
|
3cee9c154b |
perf(grpc): attribute rebuilds and bound bulk responses (#9420)
* perf(grpc): attribute rebuilds and bound bulk responses Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(grpc): attribute rebuild success after publication Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(obs): add exact HTTP/2 ping timeout diagnostics Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(multipart): run refusal matrix on large stack Run the real-store completion refusal matrix through the shared 32 MiB test harness so its deep async storage future does not overflow libtest's default stack. Keep the original test name and all refusal-order assertions intact. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * chore(deps): pin nixl-sys workspace source Manage the nixl-sys Git revision from the workspace dependency table and point the optional Linux transport dependency at houseme/nixl commit cc636c83f13d692e85b67e55aed092413d16fb41. Allow the reviewed source in cargo-deny and refresh the lockfile, replacing the unmaintained bincode dependency with the fork's wincode graph. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(rpc): authenticate msgpack peer capability Add an optional boot-epoch-bound HMAC capability proof and cache the verified per-peer observation without enabling any msgpack send policy. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(grpc): bind peer capability to connection generations (#9433) * feat(grpc): track physical connection generations Observe every tonic physical reconnect through a stable custom connector while preserving existing TCP, TLS, HTTP/2, cache, and public channel behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(rpc): bind msgpack capability to connections Scope authenticated msgpack observations to channel identity and physical connection generation so transparent reconnects require a fresh proof without enabling the send policy. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
8196603a76 |
fix(iam): authorize delegated service account creation (#9408)
* fix(iam): authorize delegated service account creation * docs(iam): clarify explicit grants for custom administrator policies * fix(iam): include service account delegation in admin wildcards * style(policy): format delegated authorization test cases * fix(iam): keep delegated authorization and replication consistent * fix(ci): include delegated account test in E2E selections |
||
|
|
1e62cbaeaf |
perf(ecstore): integrate safe sync and allocator diagnostics (#9428)
* feat(diagnostics): restore allocator and runtime metrics Port the missing live-byte accounting and serving-runtime observations from |
||
|
|
fd8dc41f0c |
feat(ecstore): add file-sync durability tier (#9422)
* refactor(ecstore): split durability sync capabilities Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): classify durability barriers Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(ecstore): add file-sync durability tier Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): harden durability docker validation Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): isolate durability PUT benchmarks Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): expose file sync grouping controls Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
645bb63809 |
feat: S3-over-RDMA with NIXL and cuObject transports (#9394)
* feat: S3-over-RDMA with NIXL and cuObject transports A signed S3 request with an empty body names a registered client buffer; the server moves the object bytes with one-sided RDMA (NIXL/UCX, or NVIDIA cuObject through the cuobject crate) instead of the HTTP body. Internode shard reads and writes can also move over NIXL. Applied onto main from #9305 (head |
||
|
|
5f9e5d62df |
feat(obs): measure remote rename field bytes (#9417)
* feat(obs): split PUT commit and remote rename timing Measure the app commit-owner scheduling gap and split remote rename client preparation into compatibility encoding, client acquisition, and request digest stages. Keep the timers behind the existing detailed PUT diagnostics gate. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * bench(filemeta): measure inline rename RPC encoding Compare JSON compatibility, msgpack, and dual encoding for metadata-only and representative inline FileInfo payloads used by remote rename requests. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(bench): use exact inline rename shard fixture Model the 100 KiB EC12 inline data as one rounded data shard plus its HighwayHash header, and report metadata-only throughput as operations. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(obs): measure remote rename field bytes Record detail-gated JSON and msgpack field lengths after successful remote rename request encoding so msgpack-only rehearsals can prove sender behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(scanner): preserve partial carry drain window Use the production drain window for real partial-bucket checkpoints so CI storage latency cannot truncate source-cache persistence. Align the certified partial-cycle assertions with carried-value semantics while retaining mixed-coverage and final convergence checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
247aecf7b8 |
fix(health): bound quorum probes and readiness observations (#9410)
* fix(health): bound quorum probes and record timeout observations Collect fresh local and remote observations concurrently and finish once each configured set has sufficient storage and lock quorum evidence. Validate configured peer identities and disk slots before counting votes. Apply the cluster deadline to cache and singleflight waiting, retain flight ownership through timeout publication, and record timeout observations once. Cover slow peers, unavailable quorum, identity validation, cancellation, cache ownership and compatibility with focused regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(health): coalesce probes and count quorum incrementally Share the completion generation with overlapping uncached requests while retaining fresh observations for later requests and bounded flight ownership. Build validated topology thresholds once and count unique disk and lock votes incrementally instead of rescanning collected inventories. Honor the existing inventory metrics flag through RPC and storage owners. Health probes omit operation metric snapshots while retaining fresh disk identity, capacity and runtime checks. Move owned inventories across local aggregation layers and preserve the full-metrics administrative APIs. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(health): cover failed uncached flight recovery Cover a later waiter sharing its overlapping failed round before an independent request starts a fresh successful observation. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(health): group lock membership in one endpoint pass Group configured lock members by set in one pass and reject impossible set counts before allocation. Retain per-set thresholds and add a bounded malformed-topology regression. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(health): publish startup topology in gating fixture Publish the shared four-disk fixture's endpoint topology before initializing disks, matching storage startup. Cluster health probes can then validate fresh quorum observations while the readiness test exercises S3 admission publication. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
fc22c11d1a |
feat(obs): split PUT commit and remote rename timing (#9416)
* feat(obs): split PUT commit and remote rename timing Measure the app commit-owner scheduling gap and split remote rename client preparation into compatibility encoding, client acquisition, and request digest stages. Keep the timers behind the existing detailed PUT diagnostics gate. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * bench(filemeta): measure inline rename RPC encoding Compare JSON compatibility, msgpack, and dual encoding for metadata-only and representative inline FileInfo payloads used by remote rename requests. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(bench): use exact inline rename shard fixture Model the 100 KiB EC12 inline data as one rounded data shard plus its HighwayHash header, and report metadata-only throughput as operations. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
c132329961 |
fix(scanner): preserve usage convergence across bounded cycles (#9397)
* feat(usage): record bucket provenance and complete observations * fix(scanner): preserve usage progress across bounded scan cycles * docs(scanner): explain usage provenance and bounded convergence * chore(scanner): correct requeuing spelling in runtime guidance * fix(scanner): route usage contracts through domain facades * fix(usage): keep admin projection futures sendable * fix(scanner): expose endpoint layout in production facade * fix(scanner): preserve checkpoint ownership during budget drain * test(scanner): remove redundant fixture clones * test(ecstore): update Windows rename publication calls * fix(ci): restore scanner test exports and lint assertions * test(scanner): preserve production drain for real checkpoints * fix(scanner): retire obsolete work without clearing responsibility * test(table-catalog): follow DeleteObject guard delegation --------- Co-authored-by: Hauser Zhi <housemecn@gmail.com> |
||
|
|
3c59e4f306 |
fix(kms): tolerate Vault reads that lag the client's own writes (#9406)
* fix(kms): tolerate Vault reads that lag the client's own writes A Vault or OpenBao node that answers reads from its own copy (an OpenBao standby, a Vault Enterprise performance standby) can serve the state from before a write for a short while after acknowledging it, and can refuse a write it is able to reject from that copy without forwarding it. - Create a KV2 key in one create-only write that carries usage, description and tags, instead of writing the record and reading it back to attach them. - Pause with jittered backoff before re-reading after a lost check-and-set write, and allow five attempts instead of three, in both backends. - Remember for five seconds what this client last wrote or deleted at each record and re-read an answer that is older. The last answer is returned once the attempts are spent, so this only delays a read. - Retry a Transit key delete that is refused because the deletion switch is not visible yet. * docs(kms): describe running the Vault backends against OpenBao * fix(kms): keep the remembered write version from moving backwards Two writes of one record by the same client can have their acknowledgements processed out of order. Recording the lower version last made it the floor again, so a read served from that older version was accepted. The marker is now updated atomically and only raised within a record's lifetime; a write after this client's own delete starts the count again. |
||
|
|
8d5faa5750 |
fix(ecstore): persist restartable format bootstrap authority (#9359)
* fix(ecstore): persist restartable format bootstrap authority Persist all-drive creator plans before format publication and resume exact deployment and drive slots after interrupted fresh formation. Consume authority durably before startup publication and validate regular control files on full and streamed reads. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): finalize format bootstrap once Order bootstrap retirement before initialized identity publication, preserve established expansion preflight repair, and require followers to observe committed topology and complete identities. Publish bootstrap controls through one authenticated atomic RPC on the dedicated fsync lane. Keep cancellation-safe cache ownership in the native operation and retain interoperable replacement healing. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): fence anchored format publication retries Select the durable first-slot plan before fanout and complete immutable anchored plans across retries. Retain physical-root namespace ownership through native publication, retirement and cache invalidation, rejecting conflicting or retired control records. Reuse main's unary RPC lifecycle recorder for validated outcomes and exact request, byte and cancellation accounting. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(test): align Windows and test-util compilation gates Match physical publication-hook tests to their non-Windows provider. Compile pool-capacity overrides with their test-util consumers, and keep alias-list mutation inside the Unix branch. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): target native format publication paths Co-authored-by: houseme <4829346+houseme@users.noreply.github.com> * style(rpc): format merged request imports Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: Chris An <anzhengchao@gmail.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: houseme <4829346+houseme@users.noreply.github.com> |
||
|
|
ef6af77026 |
feat(heal): add bounded MRF management and streaming audits (#9399)
* feat(heal): add bounded MRF management and streaming audits Publish lightweight responsibility summaries and expose authenticated, identity-bound cluster administration without reading object payloads. Stream audit-only reads while retaining staged verified bytes and full target readback for migration. * test(admin): keep MRF coverage within gateway boundaries Exercise owned MRF handlers through the shared admin test seam and keep wire-level authentication assertions in the router tests, without introducing another direct s3s consumer. * test(e2e): complete MRF lifecycle lock fixture RPC * docs: clarify integrity job resume refusals Explain that a failed lock acquisition does not prove a live worker, and require job-status reconciliation before retrying an unknown request outcome. * test(admin): include MRF cluster routes in route matrix Keep the exhaustive admin route matrix aligned with the four registered cluster management endpoints, including the concrete node and responsibility ID probe. |
||
|
|
0fde0f14a0 |
Revert "fix(ecstore): isolate unavailable local drives during startup (#9353)"
This reverts commit |
||
|
|
f6d64bc1fb |
fix(ecstore): isolate unavailable local drives during startup (#9353)
* fix(ecstore): isolate unavailable local drives during startup Keep missing local endpoints in their configured slots and initialize healthy local drives before applying the existing format and service quorums. Protect unformatted Linux URL endpoints from writes to the system filesystem, validate legacy admission, and retry missing local slots during format waits. Refs rustfs/rustfs#9350 Refs rustfs/backlog#3174 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): prepare local drives before registry publication Prepare local disks with bounded concurrency outside inventory locks and publish validated batches under per-instance writer coordination. Preserve existing registrations on failed or cancelled full initialization, reuse current-root handles, and withdraw stale reopened slots. Unify initial-format and reconnect publication, retain raw and explicit-context constructor contracts, and move filesystem queries into owned native jobs. Centralize format validation and reject special format files before startup writes. Refs rustfs/rustfs#9350 Refs rustfs/backlog#3174 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * chore(ecstore): tighten formatted-error ratchet after refactor Drop the three removed runtime registration call sites from the shrink-only baseline. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test: align drive fixtures with startup admission Expect the disk-scoped missing-root error and enable the existing directory test mode for the synthetic URL-based receiver fixture. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): unify local drive admission and publication Share bounded preparation and pre-I/O membership snapshots across local startup and retry paths, with coherent registry and topology publication. Coalesce native startup preflight and preserve root and buffer ownership. Keep reconnect ID caches instance-scoped and retain admitted health monitors. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): pin multipart index queue aliases to owner lifetime Derive the Linux index alias from the existing identity descriptor so a retired caller descriptor cannot redirect rebuild or reconnect queue lookup. Retain the original descriptor assertion and cover cross-disk alias reuse. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
d37640fbb9 |
fix(grpc): record TLS setup failures and cancelled dial attempts (#9388)
* perf(protos): balance peer channels and synchronize TLS eviction Use bounded per-peer bulk cursors and shared TLS generation reads. Keep eviction and generation removal coherent with concurrent publication, and expose opt-in bulk transport tuning with checked capacities. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(metrics): observe internode unary request lifetimes Record client attempts through errors and cancellation, preserve the published server-duration family, and add gated activity and completion metrics. Cache enabled instruments after recorder publication while retaining startup duration recording. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): make internode gRPC experiments reproducible Reset phase-owned settings before applying overrides, separate lifecycle experiments, and provide a bounded loopback workload with independently reported control and bulk latency. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): isolate transport knobs and redact dry runs Add single-parameter P0 presets with separate artifacts while preserving the legacy combined default. Mask credential arguments in dry-run output without changing actual dispatch, and document fleet stop barriers and measurement boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(observability): qualify gRPC windows and trace shutdown Reject stale, incomplete or maintenance-active measurement evidence using producer timestamps and independent process identities. Preserve raw inputs, bind evidence hashes, and exercise failure paths through the script entry point. Record HTTP/2 graceful, child-join and TLS reloader retirement phases without changing shared deadlines, join ownership or unclean-shutdown marker policy. Document RPC coverage, fixed-window behavior and separate maintenance workloads. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): add authenticated ReadAll and fixed-rate Ping probe Exercise production remote-disk and signed control clients against task-owned temporary fixtures. Preserve all offered control arrivals, dispatch delay, payload validation and bounded raw request samples, with optional external measurement markers. Register probe regressions with the default package test discovery and document its telemetry, retry and storage-acceptance boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(perf): resolve RPC probe disks by explicit UUID Allow a complete non-nil disk UUID mapping to be installed through the existing remote-disk client API before fixture preflight. This supports test services that register local paths instead of the probe's transport URLs, without modifying server metadata. Validate mapping shape and assert the actual wire target in the loopback regression. Keep preflight diagnostics limited to the numeric disk-error classification. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * docs(perf): reseed temporary RPC fixtures after restart Separate startup-cleaned RPC temporary files from confirmed S3 durability fixtures. Revalidate readiness and disk identity before reseeding the probe after recovery. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(server): cancel unfinished TLS handshakes during shutdown Cancel pre-request TLS acceptance when the listener retires so silent peers cannot retain graceful watchers beyond the shared drain budget. Keep established TLS/H2 response draining and original child joins intact. Add real TCP/TLS shutdown regressions and document fresh native OTLP observation fences and exec-confirmed diagnostic test units. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(grpc): reuse bulk keys and fix IPv6 TLS verification Reuse bounded per-peer slot keys and retain the existing cache identities for larger pools. Delegate custom-CA certificate names to tonic's URI parser and cover real IPv6 control/bulk TLS connections with a wrong-name rejection regression. Classify ReadVersion failure boundaries while preserving error totals, raise the bounded raw probe budget, and align the Linux-only rename test helper with its caller's platform condition. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(grpc): keep eviction race behind cache helpers Pause eviction on the owned TLS-generation lock and inspect the channel through the common cache API. Assert lock ordering while both mutations are queued, then retain the replacement-generation assertion after they complete. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(grpc): load native trust and preserve mTLS client identity Select explicit private trust or platform roots, then attach the published client identity for either mode. Build the TLS connector on the blocking pool under the existing acquisition deadline. Cover real control/bulk TLS connections, required client authentication, private-root scope and queued configuration timeout without late connection establishment. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(grpc): account for TLS setup failures and cancelled dials Own explicit dial metrics from validated endpoint setup through connection completion. Record TLS preparation failures and acquisition cancellation once while keeping peer-health decisions in the connect-result branches. Cover malformed client identity, queued configuration cancellation and successful coalesced dials, and document the metric scope. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
c43e415b09 |
feat(server): serve ListBuckets and HeadBucket on the gateway stack (#9296)
The walking skeleton of the gateway edge, behind the gateway-stack cargo feature and RUSTFS_S3_STACK=gateway. rustfs-gateway inside the RustFS binary serves ListBuckets and HeadBucket end to end and answers every other S3 operation 501 NotImplemented. A default build carries neither the stack selection nor the wrapper: its request path is unchanged. - One per-operation unit: impl Served for dto::Op with a metric label, an authorize without a default body, and an execute. The only impl Handler is the blanket one over Served (access phase, authorize, count, execute, one error mapping). authorize calls the function the legacy access hook calls. - One table laid out per family (ten family files named after the migration batches); it registers the handlers and is the whole early gate. - A shared access phase used by both edges, fed on the gateway edge from the gateway's verdict. - Stored bucket names are materialised under the gateway floor only, so names the AWS grammar refuses are listed as before; HeadBucket's legacy edge builds its input the same way until the gateway re-pin (rustfs/gateway#1610). - With the gateway selected the legacy service receives exactly what the legacy router claims (extension routes, object lambda, control plane). - A start-up line names the stack the mounted service answers with. - The release-boundary guard reads feature closures from cargo metadata. Squashed from gw/T2.5-gateway-stack at |
||
|
|
ce9d0b153e | fix(heal): retain legacy MRF responsibility after physical repair (#9391) | ||
|
|
ef79dcbe19 |
ci: conformance and client-matrix lanes against both S3 stacks
Acceptance lanes T3.4 (rustfs/backlog#2775) and T3.5 (rustfs/backlog#2776): .github/workflows/gateway-acceptance.yml runs the rustfs/gateway conformance corpus (--profile rustfs) and client matrix (ci/compat/run_matrix.sh --external) against the candidate binary on RUSTFS_S3_STACK=legacy and =gateway, one server per stack. The suites are checked out at the gateway rev Cargo.toml pins. Nightly schedule and workflow_dispatch only. - Each stack must be observed: the start step writes `ready` only when /health/ready answers while its own process is alive and the server's server_starting event names the requested stack (scripts/dual_stack.py stack); otherwise `did not start: ...`. Summaries print the observed stack. - Verdicts come from files the run leaves on disk (startup record, stack evidence, liveness after the corpus, exit codes, report.json, matrix.json), never from an earlier step's exit status. - Conformance: the rustfs profile describes the gateway edge, so only the gateway stack is judged (scripts/check_conformance_rulings.py against .config/conformance-rulings.toml). Red: an unruled failure, a ruled case failing differently from its ruling's `failure`, a stale ruling, and a skip on the server's own answer (endpoint not answering, refused fixture). The legacy run is an unjudged reference: it must serve, stay up and complete. - Client matrix (scripts/compat_matrix_report.py): per stack, red on a `fail` or `sut-unregistered` cell, a client that could not be installed, or an unfinished matrix; legacy is the baseline and every legacy pass must pass on gateway. - The gateway leg and the stack comparison are continue-on-error, report-only until T2.10. Not built: no re-pin (write cases need a gateway rev with --allow-external-writes plus that flag on the lane's run line); no legacy rulings; no use of the runner's own --rulings ledger (stale rulings do not block there); no call to the gateway's check_compat_matrix.sh (it does not fail `fail` cells or clients that never ran); no JUnit (the pinned runner writes a raw U+0001); no install cache; toolchain versions copied from the gateway's client-matrix.yml at 7915d069. Tests: --self-test of both checkers in make script-tests, 40 tests, 31 negative. Mutations, each red: unruled failure ignored (test_an_unruled_failure_is_red); stale only when passed (test_a_ruling_on_a_skipped_or_absent_case_is_stale); did-not-start green (test_a_server_that_did_not_start_is_red_and_says_why); swallowed runner exit (test_a_swallowed_exit_cannot_hide_a_failed_case); exit 3 accepted (test_an_environment_exit_is_red); legacy judged or tallied (test_the_legacy_reference_is_counted_never_judged); dead server green (test_a_server_that_died_during_the_corpus_is_red); endpoint and fixture skips not red (test_a_skip_on_the_endpoint_not_answering_is_red, test_a_skip_on_a_refused_fixture_is_red); ruling on id only (test_a_ruled_case_failing_differently_is_red); unknown verdict (test_an_unknown_verdict_is_refused); stack not compared (test_a_server_of_the_other_stack_is_red); no stack evidence (test_ready_without_stack_evidence_is_red); failed cell, skipped client, swallowed matrix exit (test_a_failed_cell_is_red, test_a_client_that_could_not_be_installed_is_skipped_never_passed, test_a_missing_manifest_is_red_even_when_the_exit_was_swallowed); superset gate (test_a_legacy_pass_lost_or_absent_on_gateway_is_red, test_a_gateway_superset_of_the_baseline_is_clean_and_its_extra_passes_are_listed, test_diff_without_both_manifests_is_red); diffs (test_diff_lists_changed_and_absent_cases_only, test_diff_without_both_reports_compares_nothing). Trial runs on rustfs/rustfs#9368: 43264289490, 43325101234. Gateway stack did not start (gateway stack is not assembled yet, T2.5); legacy reference completed, 140 of 890 cases executable against an external endpoint; legacy matrix 13 x 14: pass 121, fail 0, client-unsupported 61, sut-unregistered 0. Not verified: the stack observation and liveness record in CI (exercised locally under bash -eo pipefail with a stand-in server and main's dual_stack.py, both directions); a scheduled run; a diff with gateway content. |
||
|
|
eca95f2973 |
perf(ecstore): erasure read path for large GETs (#9304)
Lands the contributor's pull requests #9302 (opt-in lockless KV cache buckets), #9303 (per-bucket XXH3 ETags; MD5 off the request task) and #9304 (erasure read path for large GETs) as one change, because #9304 was stacked on the other two. Maintainer changes on top of the contributed work: - Merged main. crates/ecstore/src/erasure/coding/bitrot.rs: kept the PR's verify_chunked_frame/append_chunked_data restructuring and routed its hash-mismatch log through main's throttled log_hash_mismatch. - rustfs/src/app/multipart_usecase.rs: took main's per-operation split and re-applied the PR's changes in place: multipart_etag_algorithm and its test in the parent module, the XXH3 pin in create.rs, the stage timer and from_stream_with_etag in upload_part.rs. - Reworded one doc comment in crates/config/src/constants/object.rs that spelled an ecstore import path, which the architecture migration guard counts as a direct import. - fix(config): made the MD5 offload opt-in. Each offloaded upload holds one blocking-pool thread until its body ends, and that pool also serves disk I/O, so many concurrent large uploads could starve disk writes. The default threshold is now 0 (disabled); operators enable it with RUSTFS_OBJECT_ETAG_MD5_OFFLOAD_MIN_BYTES (for example 8388608). |
||
|
|
5a26862e2e |
ci: build macOS and Windows on a weekly schedule (#9286)
* ci: build macOS and Windows on a weekly schedule Keep the 01:13 UTC cron on Linux and add a Sunday 16:00 UTC cron for macOS and Windows. Select that subset from the firing cron so a delayed runner cannot guess the weekday. Co-authored-by: hector <majinghe@users.noreply.github.com> * ci: move the weekly desktop cron off minute zero --------- Co-authored-by: hector <majinghe@users.noreply.github.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
|
704b984c1f |
ci: both-stack e2e diff in the source workflows and a strict mint stack gate
T3.1/T3.3 (rustfs/backlog#2772, #2774). The existing nightly lanes get a stack dimension and their legacy leg is the legacy side: e2e-replication-nightly (repl, cluster, protocols), e2e-distributed and the ODM minio-source job. A gateway leg runs scripts/dual_stack.py preflight after its build and stops there when no gateway start is proven; the summary says 'skipped: gateway stack refuses startup: <why>'. One summary job per workflow diffs the legs of the same run with junit_diff.py. dual-stack-nightly.yml covers only e2e-smoke and e2e-full: one build, both stacks from that binary, gateway legs only when the preflight proved a start. junit_diff.py reports a missing, empty or case-less side as 'no result for <label>'. mint.yml stays weekly, builds the image once and runs it per stack; mint_stack_diff.py gates that every case legacy passes is present and passing on gateway with no new failure (missing, NA, unknown status and truncated lines never pass). Gateway legs and the compare job are report-only until T2.10. Not built: a shared-binary preflight job in the three source workflows (one preflight step per gateway leg); harness-only filtering for their five profiles; mc/Vault/iptables in the e2e-full leg; an s3s-e2e both-stack run; rustfs_gateway=debug. Tests: dual_stack 30 (preflight with fake binaries), mint_stack_diff 16, junit_diff 27; negatives outnumber positives. Mutations (red test named) are in the PR 9367 description. Not verified: the workflows have not run in CI in this shape (composites, stack matrices, image artifact round trip, needs result of a red continue-on-error leg). |
||
|
|
45156f313e |
perf(ecstore): reduce default fsync worker idle spin budget (#9386)
Use 256 idle iterations by default and preserve the configurable 1024 iteration ceiling. Update fallback regression cases and tuning guidance. Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
b42f444166 |
ci: stop rebuilding what the cache already built (#9371)
* ci: stop rebuilding what the cache already built cache-warm.yml built `--workspace --all-targets`, a selection no ci-dev reader builds. Feature resolution differs per selection, so the test archive recompiled 123 external units and the E2E archive about 290 on every exact cache hit. warm-ci-dev now runs the readers' own builds: `ci_test_artifact.py cargo warm` (the Cargo build of `nextest archive`) for both archive configurations, the hooks server and the lint command. Rust-cache never saves over an exact key match, so the warm jobs skip their builds after one; ci_gate.py --check-workflow pins both rules. The manual shard rehearsal ran only after a full execution, saved nothing, and could not run since #9071 renamed the archive it downloads. It is removed with its plan field, jobs, script and tests; the plan contract moves to 5. * ci: key ci-dev by its warm commands and close the warm guard's gaps The ci-dev key now ends in a digest of warm-ci-dev's build steps, so the reader selections this branch warms are written under a new key instead of being skipped as an exact hit of the old one. ci_gate.py --check-workflow derives the digest, requires the writer and every reader in ci.yml and reference-parity.yml to name that key, and prints the key a change needs. The presence check now reads only the steps that run whenever the key is written, so a selection moved into the on-request --timings step no longer counts. Configuration names are compared as sets in both directions and accept either option spelling. The plan contract moves to 6: #9369 already used 5 for a different plan. |
||
|
|
70525a9698 |
feat(ecstore): expose fsync worker policy experiments (#9357)
* feat(ecstore): expose fsync worker policy experiments Add independent idle-spin controls and gated worker diagnostics for controlled performance comparisons. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): forward Docker telemetry in ABBA runs Validate node mappings and record per-leg container resources for isolated fsync policy experiments. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(perf): reject baseline drift in either direction Apply absolute delta budgets to A2 baseline checks while preserving directional candidate checks and their exemptions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(perf): stop ABBA after a load driver failure Preserve the load driver exit status through command substitution and stop before later deployment legs while retaining cleanup traps. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
6b27fa4756 |
ci: accept the proving merge-group run as release evidence (#9376)
* ci: accept the proving merge-group run as release evidence A commit that lands through the merge queue gets a push run of ci.yml in mode proven: Quick Checks and Test and Lint succeed, and every validation job, End-to-End Tests (full merge gate) included, is skipped. source_ci refused such a commit, so no release candidate could be cut from it. source_ci now accepts that push run when Quick Checks and Test and Lint succeeded and a successful merge_group run of ci.yml on that branch's queue has the same head tree and passed all three required jobs itself; that run is recorded as the candidate's exact-source CI identity. The match is ci_merge_group_proof.proves, the same predicate the push plan uses. Also: a test that a full merge_group run requires every job a full integration push requires; the sibling queue-branch case (gh-readonly-queue/mainline/...); select_plan's proof is keyword-only; the release, CI gate and workflow docs describe proven pushes. * test(release): pin the proving run's position, attempt and branch The release test now lists the matching merge-group run among a run of another tree and a run of another branch's queue, in both orders; gives it attempt 3 and reads jobs by run and attempt; and accepts a release branch's own queue run while refusing a main queue run for it. |
||
|
|
4a34e512cd |
ci: skip the push run for a tree the merge queue already proved (#9369)
A merge-queue landing fast-forwards main to the group's commit, so the push run re-validates a tree a successful merge_group run of ci.yml has just passed. Select CI scope now looks that run up with the job's read-only Actions token (scripts/ci_merge_group_proof.py) and plans mode `proven`: Select CI scope, Typos and Quick Checks run, every validation job is skipped, and Test and Lint requires exactly that. The plan names the proving run, its not-applicable reasons and the aggregate's summary say the jobs were skipped as proven. Any doubt runs everything: no token, an API or network error, a rate limit, an unexpected payload, no successful queue run of the same tree, or a proof on another event. A direct push has no such run. The plan contract moves to 5. |
||
|
|
5f460c2383 |
perf(grpc): optimize peer pools and unary RPC observability (#9355)
* perf(protos): balance peer channels and synchronize TLS eviction Use bounded per-peer bulk cursors and shared TLS generation reads. Keep eviction and generation removal coherent with concurrent publication, and expose opt-in bulk transport tuning with checked capacities. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(metrics): observe internode unary request lifetimes Record client attempts through errors and cancellation, preserve the published server-duration family, and add gated activity and completion metrics. Cache enabled instruments after recorder publication while retaining startup duration recording. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): make internode gRPC experiments reproducible Reset phase-owned settings before applying overrides, separate lifecycle experiments, and provide a bounded loopback workload with independently reported control and bulk latency. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): isolate transport knobs and redact dry runs Add single-parameter P0 presets with separate artifacts while preserving the legacy combined default. Mask credential arguments in dry-run output without changing actual dispatch, and document fleet stop barriers and measurement boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(observability): qualify gRPC windows and trace shutdown Reject stale, incomplete or maintenance-active measurement evidence using producer timestamps and independent process identities. Preserve raw inputs, bind evidence hashes, and exercise failure paths through the script entry point. Record HTTP/2 graceful, child-join and TLS reloader retirement phases without changing shared deadlines, join ownership or unclean-shutdown marker policy. Document RPC coverage, fixed-window behavior and separate maintenance workloads. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(perf): add authenticated ReadAll and fixed-rate Ping probe Exercise production remote-disk and signed control clients against task-owned temporary fixtures. Preserve all offered control arrivals, dispatch delay, payload validation and bounded raw request samples, with optional external measurement markers. Register probe regressions with the default package test discovery and document its telemetry, retry and storage-acceptance boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(perf): resolve RPC probe disks by explicit UUID Allow a complete non-nil disk UUID mapping to be installed through the existing remote-disk client API before fixture preflight. This supports test services that register local paths instead of the probe's transport URLs, without modifying server metadata. Validate mapping shape and assert the actual wire target in the loopback regression. Keep preflight diagnostics limited to the numeric disk-error classification. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * docs(perf): reseed temporary RPC fixtures after restart Separate startup-cleaned RPC temporary files from confirmed S3 durability fixtures. Revalidate readiness and disk identity before reseeding the probe after recovery. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(server): cancel unfinished TLS handshakes during shutdown Cancel pre-request TLS acceptance when the listener retires so silent peers cannot retain graceful watchers beyond the shared drain budget. Keep established TLS/H2 response draining and original child joins intact. Add real TCP/TLS shutdown regressions and document fresh native OTLP observation fences and exec-confirmed diagnostic test units. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(grpc): reuse bulk keys and fix IPv6 TLS verification Reuse bounded per-peer slot keys and retain the existing cache identities for larger pools. Delegate custom-CA certificate names to tonic's URI parser and cover real IPv6 control/bulk TLS connections with a wrong-name rejection regression. Classify ReadVersion failure boundaries while preserving error totals, raise the bounded raw probe budget, and align the Linux-only rename test helper with its caller's platform condition. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(grpc): keep eviction race behind cache helpers Pause eviction on the owned TLS-generation lock and inspect the channel through the common cache API. Assert lock ordering while both mutations are queued, then retain the replacement-generation assertion after they complete. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(grpc): load native trust and preserve mTLS client identity Select explicit private trust or platform roots, then attach the published client identity for either mode. Build the TLS connector on the blocking pool under the existing acquisition deadline. Cover real control/bulk TLS connections, required client authentication, private-root scope and queued configuration timeout without late connection establishment. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
|
55d9bf08a2 |
ci: run workspace clippy beside the test binary build (#9345)
Clippy ran in front of the test-binary archive inside Workspace Lint and Build, so every workspace test waited 13 to 17 minutes for a lint that shares no compiler output with the archive. Move the unchanged Clippy step into its own workspace-clippy job with the same condition, needs, runner and cache key. Workspace Lint and Build keeps its ID and name as the archive producer. The aggregate and the scheduled alert depend on the new job, and the plan contract moves to 4 so a base planner that predates the job set falls back to the full matrix. ci_gate.py now rejects a workflow where the lint is conditional, duplicated, outside the aggregate, scoped differently from the archive producer, or back on the archive consumers' dependency chain, and where the workflow's plan-contract marker differs from the planner's. |
||
|
|
633c0d204a | fix(s3-tables): reclaim dropped table commits and maintenance reports (#9351) | ||
|
|
7f90eb7ae8 |
fix(replication): seal SSE-KMS replicas under the same-named key (#9349)
* fix(replication): replicate SSE-KMS objects under key enforcement With RUSTFS_KMS_ENFORCE_SSE_KEY_POLICY=true the receiving site authorizes a replica PUT of an SSE-KMS object for kms:GenerateDataKey against the key it resolves to. The managed site-replicator account held no kms grant, so every SSE-KMS object stayed FAILED with AccessDenied. Grant kms:GenerateDataKey on every key and treat a stored policy without it as outdated so the reconcile tick refreshes existing accounts. kms:Decrypt stays out: reads proxied to a peer run as this account. * fix(replication): seal SSE-KMS replicas under the same-named key A replica write carried only the aws:kms intent, so a site-replication peer sealed the replica under its bucket default or site default key. An object written under a key named in the request therefore ended up under a different key on each site, and with key enforcement a caller scoped to the named key could read it on one site and not on the other. Site-replication peers are required to hold every KMS key under the same name, so the replica write to a peer now names the source object's key. Operator-configured bucket-replication targets still get the intent only: their KMS has no reason to hold a key of that name. |
||
|
|
e6b2a576e3 |
feat(quota): report peer metadata reload acknowledgements (#9330)
Co-authored-by: Chris An <anzhengchao@gmail.com> |
||
|
|
f97fcf36b2 |
refactor(app): split bucket_usecase into one file per config family (#9317)
* feat(server): add the RUSTFS_S3_STACK selector and dual-stack e2e lane (#8411) * refactor(server): group the s3s-compat tower layers under legacy_compat (#8410) Move the ten tower layers that patch s3s behaviour out of rustfs/src/server/layer.rs into rustfs/src/server/legacy_compat/, one file per layer with its tests. Each file header names the s3s behaviour it patches, the gateway mechanism that replaces it (Replaced by:), and the task that removes it (Removed by: T2.12, rustfs/backlog#2771). The layers keep their code, their order in external_service_stack!, and their behaviour; no layer is deleted. Add server::http::tests::stack_census, which renders the ordered service chain of the real external_service_stack! expansion and compares it with a committed constant, so a reorder, insertion or removal is a visible diff. Route the two s3s error types the moved layers use through a storage_api::server::legacy_compat shim so the split adds no s3s-importing file, and lower the s3s footprint baselines to the measured values (files 209 to 208, s3_error! lines 1586 to 1585). Keep the server runtime-source bypass guard covering the moved code. Refs rustfs/backlog#2787, rustfs/backlog#2734 * feat(s3-types): add the RustFS-owned S3 error type and s3_error! macro (#8406) `rustfs-s3-types` now provides `S3ErrorCode` (generated by build.rs from src/codes.txt, the grep-derived call surface, plus `Custom`), `S3Error`, `S3Result`, `StdError` and the `s3_error` macro with the same call syntax as s3s, so the T1.2 codemod is a `use` swap. Statuses are hand-written and pinned to the s3s behaviour RustFS ships today by an oracle test that uses s3s as a dev-dependency only. The `From` conversions for the legacy edge live behind the off-by-default `compat-s3s` feature (the orphan rule forbids them in rustfs/src/error.rs); `rustfs` enables it. Refs rustfs/backlog#2742, rustfs/backlog#2734 * ci: add the per-crate test-count ratchet (#8404) Quick Checks now fails any PR that lowers the number of test attributes in a workspace crate or adds #[ignore], measured per crate against the exact rows of .config/test-count-baseline.txt (rustfs/backlog#2734 section 1 rule 3). The counter widens the spec's literal #[(tokio::)?test] to the tokio attributes that carry arguments and both spellings of #[test_case(...)], and anchors at line start so commented-out attributes do not count. The initial baseline covers 52 crates, 21077 tests and 164 ignored. Refs rustfs/backlog#2738 Refs rustfs/backlog#2734 * test(ecstore): pin the persisted bucket-config XML bytes for 13 families (#8408) Before any DTO or codec is replaced, pin the exact bytes that deserialize followed by serialize produces for every persisted bucket-configuration XML family in its three source shapes (AWS documentation, MinIO-written, RustFS-written), and the bytes RustFS writes today for a typed value. Byte equality only: a structural comparison would let a codec replacement change the persisted encoding unnoticed. The generator script refuses to overwrite existing goldens without --force, refuses on dirty codec paths, cannot report success from its write mode, and records the generating commit in the fixtures README. Refs rustfs/backlog#2744, rustfs/backlog#2734 * ci: raise the test-count baseline to the integration tree (#8426) * ci: make the s3s footprint baseline exact and lower-only (#8403) * ci: make the s3s footprint baseline exact and lower-only The s3s footprint guard compared three counters against ceilings held in the script itself, so a ceiling could sit above the tree and a drift was fixable by raising it (rustfs/rustfs#5964 raised the s3_error! ceiling to go green). The three values now live in .config/s3s-footprint-baseline.txt as files, s3_error_lines and ecstore_files, the script reads only that file, and a measurement that differs from its baseline in either direction fails: growth prints "s3s footprint grew: <measured> > <baseline>", shrinkage prints "lower the baseline to <measured> in this PR". The counting method is unchanged; the baseline is the current measurement of gateway/integration (209 / 1585 / 36). The header records the end state: once all three reach 0, T4.2 switches the script to failing on any occurrence. Refs rustfs/backlog#2739, rustfs/backlog#2734 * ci: add the s3s edge allowlist mode to the footprint guard (#8405) `scripts/check_s3s_footprint.sh --mode allowlist` fails on every Rust file that references s3s (`s3s::` or `s3s_sigv4`) and is not matched by an entry of the new `.config/s3s-edge-allowlist.txt`; `--dry-run` lists those files and exits 0. The allowlist carries the initial edge from the spec, and the script rejects an entry that names or matches anything under crates/ other than crates/e2e_test. The baseline mode and the argument-less CI invocation are unchanged. Refs rustfs/backlog#2741, rustfs/backlog#2734 * test(ecstore): reach the bucket-config codec through the test facade (#8428) * refactor(s3-types): own the HTTP Body and drop s3s from three crates (#8431) Add rustfs_s3_types::Body, a BoxBody<Bytes, StdError> wrapper with empty() and From<Bytes | Vec<u8> | String | &'static str>, and the legacy From conversions to and from s3s::Body behind compat-s3s. The signer is now generic over the request body: it only reads and writes the request head, so it needs no body type and its callers keep theirs. s3-client sends its requests with the owned Body, madmin signs a bodiless head, and targets drops its two S3Request/S3Response header wrappers in favour of extract_params_header. rustfs-signer, rustfs-madmin and rustfs-targets no longer depend on s3s; the s3s footprint baseline drops from 209 to 202 files. Refs rustfs/backlog#2747, rustfs/backlog#2734 * refactor(ecstore): move S3ErrorCode to rustfs-s3-types by codemod (#8432) * test(ecstore): pin the InvalidRange responses of new_getobjectreader Both error branches of new_getobjectreader build an ErrorResponse with S3ErrorCode::InvalidRange, and no test observed either code. Pin the code, its wire name and the location fields of each branch before the code type moves to rustfs-s3-types (rustfs/backlog#2743), so a changed code on either branch fails a test. The ecstore test-count row rises by the two tests. * chore(scripts): add the s3s-to-rustfs-s3-types error codemod Task T1.2 of rustfs/backlog#2734 (rustfs/backlog#2743) moves S3Error, S3ErrorCode, S3Result and s3_error! from s3s to rustfs-s3-types by script, one crate closure per PR. The codemod rewrites use trees (splitting mixed s3s imports) and qualified code paths, never comments or literals, adds the rustfs-s3-types dependency, drops s3s only from a crate that no longer names it, refuses every file of the s3s edge allowlist, takes --hold for a file whose type is fixed outside the closure, and prints a per-crate census. A second run changes nothing. Its self-test runs under make script-tests. * refactor(ecstore): take S3ErrorCode from rustfs-s3-types Output of scripts/codemods/s3_error_to_s3_types.py over crates/ecstore, crates/lifecycle, crates/scanner and crates/replication (rustfs/backlog#2743 batches 1 and 2). The ErrorResponse code field, the two InvalidRange constructions in new_getobjectreader and an unused tier import now use rustfs_s3_types::S3ErrorCode; the variant names and wire names are unchanged. lifecycle, scanner and replication reference none of the four error names. warm_backend.rs and warm_backend_s3.rs are held: they compare against rustfs_s3_client::ErrorResponse.code, which stays the s3s type until s3-client moves. * ci(footprint): count only s3_error! lines still on s3s's macro s3_error_lines counted every line holding the text `s3_error!`, which also matches the RustFS-owned `rustfs_s3_types::s3_error!`. A codemod that moves call sites onto rustfs-s3-types therefore could not lower it (rustfs/backlog#2743, decision 3; rustfs/rustfs#8432, gotcha 2). The counter now applies a textual, per-file and per-line rule that errs on the side of counting (full rule in the script header): - occurrences spelled `rustfs_s3_types::s3_error!` never count; - a line holding `s3s::s3_error!` always counts; - any other `s3_error!` line counts unless its file is cleared, that is, it imports the macro from rustfs_s3_types by name on a line starting with the use item (or defines it with `macro_rules! s3_error`) and does not also bring s3s's macro in (`use s3s::s3_error`, a `use s3s::{..}` tree naming it or a crate-root `*`, `use s3s::*`, or `#[macro_use] extern crate s3s`); - fail closed: a file matching neither form (`use super::*`, a crate-local re-export, a renamed or glob rustfs_s3_types import, a commented-out import, a comment-only mention) counts. Census of the 1585 lines on |
||
|
|
83ce509ff3 |
ci: run the gateway smoke lane only where the gateway stack is touched (#9344)
* ci: run the gateway smoke lane only where the gateway stack is touched The report-only `e2e-smoke (gateway, report-only until T2.10)` lane ran on every pull request and failed on every one of them: the gateway S3 stack does not serve the whole e2e-smoke suite before T2.10. A pull request that touches none of the lane's inputs can neither cause nor fix that failure, yet it was shown a red check and occupied an sm-standard-2 runner. Select CI scope now publishes one more output, `gateway_smoke`, computed by `scripts/ci_gate.py select` from the change set: true when a path in GATEWAY_SMOKE_PATHS changed or a `rustfs-gateway*` pin in Cargo.toml or Cargo.lock differs, and on every event that is not a pull request. The lane is skipped only on a pull request whose selection is the literal `false`; a missing or refused selection runs it. Job-level continue-on-error stays, and no step may carry it, so a failed suite is never rendered green. `ci_gate.py --check-workflow` pins the condition by truth table: the lane must run wherever End-to-End Tests runs except on unselected pull requests, must need classify-changes, and the output must be published. * ci: keep the e2e suite's own files out of the gateway smoke selection Feature pull requests add e2e tests and touch the smoke selection all the time, and until T2.10 the gateway lane cannot pass on any of them, so selecting on `crates/e2e_test/*`, `.config/nextest.toml` and `.config/e2e-smoke-selection.txt` kept producing the red check this change exists to remove. What the suite contains carries no signal while the stack cannot serve it. The gateway code paths, the pin comparison, the workflow, the test-runtime action and the two scripts stay. An e2e-only pull request being skipped is now a pinned self-test case. At T2.10 the selection is deleted, not extended. |
||
|
|
e2bd880fa4 |
fix(kms): make AWS-backed SSE objects readable and key policies hold (#9334)
* fix(kms): read back SSE objects written through the AWS backend
The AWS backend returned the GenerateDataKey blob bare, so the read path took the unrecognized bytes for a locally wrapped key and asked for RUSTFS_SSE_S3_MASTER_KEY without ever calling AWS Decrypt. Every SSE-KMS and SSE-S3 object written through this backend was unreadable, and UploadPart failed the same way.
The backend now wraps the blob in the data-key envelope the other backends use. A stored key that is neither an envelope nor a local-master-key wrap is offered to the KMS when its backend reports that released versions persisted it bare, so objects written by 1.0.4 and 1.0.5 read back without being rewritten.
* fix(admin): percent-decode the key id in the KMS describe path
GET /rustfs/admin/v3/kms/keys/{key_id} read the path segment verbatim, so an AWS key ARN, which has to be escaped to fit in one segment, was looked up, authorized and audited under its escaped spelling and answered 404.
* fix(policy): match an AWS key ARN as its key id in KMS statements
A key-scoped statement was matched against the key identifier exactly as the request spelled it. A Deny on key/<id> therefore did not cover a request naming the same key by its AWS ARN, on the SSE-KMS write, on the read of an object stored under the ARN spelling, and on the per-key admin endpoints.
A key ARN is now reduced to its key id before matching. A request naming a key by alias, which only the backend can resolve, fails closed against key-scoped statements. The deletion impact report compares key references the same way.
Adds the AWS KMS backend guide in English and Chinese.
* fix(kms): read rio-v2 objects that carry a bare AWS data key
A rio-v2 build stores a sealed object key beside the data key, so an AWS-backed object written before the envelope carries the same shape as a MinIO object and was sent to the MinIO decoder, failing before AWS Decrypt was tried.
MinIO's sealing is authenticated, so opening the data key under the local master key is a proof: the bare blob reaches the KMS only once that has failed, or when no local master key exists. Without a backend that reads bare ciphertext the MinIO route is unchanged.
* docs(kms): describe pairing an AWS KMS site with a Vault site
Site replication synchronizes key names and never keys, and AWS assigns key ids, so in a mixed pair the Vault site has to name its keys after the AWS key ids.
|
||
|
|
acf0d3ff72 |
fix(replication): replicate SSE-KMS objects under key-policy enforcement (#9340)
fix(replication): replicate SSE-KMS objects under key enforcement With RUSTFS_KMS_ENFORCE_SSE_KEY_POLICY=true the receiving site authorizes a replica PUT of an SSE-KMS object for kms:GenerateDataKey against the key it resolves to. The managed site-replicator account held no kms grant, so every SSE-KMS object stayed FAILED with AccessDenied. Grant kms:GenerateDataKey on every key and treat a stored policy without it as outdated so the reconcile tick refreshes existing accounts. kms:Decrypt stays out: reads proxied to a peer run as this account. |
||
|
|
056ddce8f1 |
fix(kms): require a scheduled deletion before destroying a key (#9337)
On the Vault KV2 and Vault Transit backends, an immediate deletion of a key that was not already pending deletion only marked it PendingDeletion and still reported success, so objects under the key stayed readable after the caller was told the key was gone. Key destruction is now two explicit steps on the Local, Vault KV2 and Vault Transit backends: a deletion is scheduled first, and `force_immediate` only destroys a key that is already PendingDeletion. A request against an Enabled or Disabled key is refused with an invalid state error and changes nothing. Vault Transit's "encryption key not found" answer for a destroyed key now maps to KeyNotFound instead of a backend error. Document the feature in docs/operations/kms-key-destruction.md with a Chinese translation. |
||
|
|
20c046af6c |
fix(targets): resolve default queue dirs per node (#9262)
The built-in notify and audit queue directories /opt/rustfs/events and /opt/rustfs/audit are persisted cluster-wide as literal defaults. On native Windows they have no drive prefix, so every target that kept the default failed validation as a non-absolute path. Treat those two literals as logical defaults and resolve them on each node when a queue directory is validated or opened. A new RUSTFS_TARGET_STORE_DIR relocates both default queues on any platform; without it, Windows uses %ProgramData%\RustFS. Explicit operator paths, the persisted configuration, and Linux/macOS behavior are unchanged, and pending entries are not migrated when the root changes. Expose the variable through the Helm chart as config.rustfs.target_store_dir and document the resolution rules. |