* perf(ilm): back off idle tier-delete journal recovery
Use bounded idle backoff only after complete empty journal and dispatch scans, while preserving immediate notifications and the retry cadence for observed work.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): reconcile interrupted pool metadata publication
Bind pool metadata CAS and Undo to exact native predecessors. Recover an interrupted physical publication only from an ordinary committed authority and complete verified identity, current and backup evidence, then require normal V3 CAS and quorum readback.
Preserve known-conflict gates, startup and heal lock ordering, targeted admission and the original kill-recovery deadline. Wake proof-ready recovery immediately and bound parallel capacity sampling without removing mutation fences.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* perf(ecstore): reuse proven pool metadata durability
Publish native rollback backups directly in their owned UUID directories. Reuse a one-shot data-durability receipt only for an exact hardlink to the pinned, fully synchronized head and root.
Preserve new-head and directory synchronization, native predecessor CAS, conditional Undo and uncertainty gates. Keep unknown, copied, raw-parent, cancelled and failed operations cold. Add bounded diagnostics, sync-call regressions and path-isolated test hooks.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(multipart): exercise selective RPC delay and peer loss
Add authenticated method-entry barriers for metadata and lifecycle locks under e2e-test-hooks. Verify real dispatch, write quorum, prior part payloads, lock drain and incarnation isolation without changing production budgets.
Record cancellation separately from normal exits so late filesystem completion cannot overwrite Drop evidence.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* docs: define multipart delay gates and receipt invariants
Document RPC entry evidence, peer-loss and cancellation oracles, remaining incident attribution limits, and immutable head inodes required for durability receipts.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(readiness): expose missing runtime drive slots
Report configured slots without a runtime handle separately from unavailable inventoried drives. Preserve quorum and probe status semantics. Wait for every coordinator to regain its complete live inventory before multipart RPC fault windows and single-attempt recovery oracles.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): retry committed promotion convergence safely
Require peer reload convergence on every worker attempt, including unchanged retries. Recheck durable identity, replicas, namespace fence, generation and cancellation before allowing any data movement.
Keep known committed promotions active on a verified propagation pause. Release precise owners, retain queued successors and reuse the existing watchdog. Preserve uncertainty gates and terminal operation behavior.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): guard native metadata mutation aliases
Reject protected RenamePart source and destination paths before I/O. Classify the protected namespace through the native path resolver, and require canonical conditional RenameData and Undo keys so aliases cannot create separate lease domains.
Preserve ordinary multipart and canonical CAS behavior, warm durability receipts and zero-allocation canonical fast paths. Cover real alias, byte identity, receipt and normal mutation behavior.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): register Darwin multipart RPC delay gates
Update distributed membership from the actual Darwin listing. Preserve the full-profile digest and leave Linux unchanged until its own listing is produced, as required by the contributor guide.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(multipart): fence native publication and recovery
Publish multipart data through private hardlinks and atomically consume the
prepared source head. Bind exact compensation capabilities and native recovery
to the physical cohort, source generation and durable publication state. Require
fresh terminal confirmation for native replay and preserve indeterminate data
when no safe recovery witness exists.
Release logical guards only after validated irreversible quorum success while
retaining physical mutation leases, accounting and resource ownership through
actual callbacks. Preserve local instance proof authority and historical Copy
metadata without granting healed donors fresh publication authority.
Keep general movement identity strict while preserving directory capacity
compatibility in a separate checked comparison.
Cover native fault and cancellation boundaries, zero-byte objects, real Copy,
heal and migration, exact physical readback and distributed RPC delay. Retain
all quorum and durability requirements and document rollout limitations.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): fix ratchet scan and Windows fault hooks
Move native pool metadata inventory tests after production items so the
error-format ratchet sees the existing call site. Preserve the baseline
instead of accepting a false count decrease.
Make the directory-barrier failure hook available on every test platform
and exercise the parent-barrier failure/retry cases in Windows CI. Keep
production synchronization behavior and all existing assertions intact.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(protos): remove redundant recovery request clone
Move the baseline into its final struct update while retaining the earlier
clone needed to construct the first request variant.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): align RPC delay gates with native abort
* fix(storage): validate native source recovery and CI gates
Preserve wire layouts while bounding native response sizes and fix the
last-use recovery request clone. Avoid destination replay reads only for
fully observed initial native sources; keep upgraded and recovered sources
conservative. Cover real legacy publication after a cold restart.
Align upgrade acceptance with the native mixed-version mutation pause and
use exact upload-scoped test completion evidence before advancing time.
Keep all original quorum, ownership, byte and deadline assertions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): verify native abort under partial admission
Delay native cleanup at the exact upload root after all-participant closure.
Verify quorum-only admissions retain every source byte on refused Abort and
complete the same upload without creating a missing original participant.
Keep sub-quorum orphans and their discoverable identities isolated while a
fresh all-participant upload proves service recovery. Route shared constants
through the existing E2E storage facade.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): avoid formatting a fixed invalid upload path
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): align native MPU fault recovery assertions
Verify real source participants, preserved orphan bytes and same-upload completion after quorum-only admission. Require full physical inventories so an expected read-quorum error cannot hide unrelated leakage. Preserve ordinary quorum and recovery budgets, and document listing availability through the affected set.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): keep MPU tail deadlines on the wall clock
Global Tokio time advancement can expire drive-health probes while real
filesystem callbacks are pending. Use a short real drive deadline with the
existing physical-entry and cleanup barriers, retaining the byte, lock,
resource-permit and final-shard assertions.
Fixes#8937
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(storage): align native fixtures with instance ownership
Open independent disks for the simulated capacity-loss instance, scope four coordinator journal disks without rebinding source authority, and probe configured namespace keys while retaining physical I/O hooks.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): isolate native multipart future construction
Construct optional native publication futures behind synchronous heap boundaries so ordinary rename callers do not carry their state machines. Preserve native guards and ownership without adding allocations to ordinary PUT, copy or heal branches.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(heal): assert coordinator journal scope after rejoin
Check the exact original four-member journal map after every rejoin and manager stop while preserving all sixteen native data slots. Reconnect expansion fails the test rather than being filtered away.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(heal): bound object repair future construction
Construct the normal and FileCorrupt deep-retry state machines behind a synchronous heap boundary. Preserve guards and cancellation while bounding the shared rename and deletion call stacks without increasing test stacks.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): qualify fixture disk identity checks
Call the same DiskAPI method explicitly so the independently reopened disk identity assertion compiles under every target and feature combination.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): isolate native inspection from mutation state
Select a dedicated read-only inspection future behind the existing single Box allocation. Preserve validation, source ownership and the canonical inspection body while keeping mutation state out of inline read callers. Reject misrouted operations with typed errors and retain source evidence.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): pause abort time after native cleanup
Wait for durable source deletion on the wall clock before advancing the
capacity lease clock at the post-delete barrier. Paused time can otherwise
expire the barrier while native filesystem cleanup is still pending.
Retain lease-loss injection, durable inflight-byte accounting and restart
reconciliation assertions. Document the test-util feature required to
select the native decommission fixtures in package-scoped regressions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): reclaim data from owned multipart undo
Retain exact runtime-only Undo ownership through deferred snapshot cleanup. Revalidate every target metadata version and the retirement anchor, keep the delete claim through the physical callback, and remove the anchor after the payload barrier. Preserve generic cleanup refusal when metadata is missing.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): preserve unformatted bootstrap routes
Keep explicitly unformatted local paths reachable by an in-flight elected formatter while quarantining their service slots and disk-ID membership. Own bounded legacy-read handles so the actual asynchronous creator future remains Send.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(s3): bound CopyObject tracing poll frames
Construct the business future behind an outlined single-Box boundary and lazily instrument its handle. Preserve Send, Unpin, span timing and cancellation without embedding concrete business-state construction in the tracing poll.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(mpu): observe recovery after direct legacy bootstrap
Seed all sixteen real previous-release sources before attaching the same-port method observers. Require exact per-peer disk-route Unimplemented responses with zero publication calls, retain byte and cleanup assertions, and remove superseded raw transport. Register the two new regressions from actual Linux and macOS listings.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(mpu): prepare the homogeneous previous-release fleet
Reuse the existing release baseline before creating the genuine legacy multipart sources. Restart old processes with their normal budgets and require every drive through the reduced-redundancy probes before installing the mixed-version observers and current fault budgets.
Keep standard EC:4, single-attempt calls, all mixed-version byte and reclamation assertions, and their original deadlines unchanged.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(mpu): isolate release compatibility from fault profiles
Keep normal component configuration for the observed mixed-release rollout, which injects no RPC delay or loss. Retain the short component profile in actual method-fault scenarios and preserve the single-attempt 45-second probe and every typed error, routing, source byte and reclamation assertion.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(mpu): use normal shutdown for planned release replacements
Use the existing bounded SIGTERM path during homogeneous release preparation and the observed mixed-version rollout. Preserve every multipart inventory, routing, quorum and byte assertion, and keep actual crash recovery on its original hard-kill path.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* revert: retain the verified release compatibility fixture
Restore the published fixture after the local configuration and lifecycle experiments failed to resolve the macOS startup failures. Keep their diagnostic records separate from the delivered implementation.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(copy): bound inherent handler polling stack
Build the inherent CopyObject future through a synchronous noinline
factory at the existing trait await. This keeps its large state out of
the trait wrapper's active polling frame during historical self-copy.
Preserve cancellation, tracing and post-write notifications, with one
additional allocation restricted to COPY.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): validate authoritative quota and native pool heals
Require actual authoritative quota usage in the existing readiness loop
instead of treating HTTP success or unknown usage as a numeric baseline.
Keep the original quota assertions and convergence budgets.
Allow the owning pool metadata CAS to republish into a fresh valid data
directory while retaining exact version, layout, completeness and shard
fingerprints. Require every node's real metadata/read/write readiness
before accepting recovery. Keep ordinary object manifests and the original
180-second recovery deadline unchanged, with negative oracle regressions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): distinguish certified native Abort recovery
Require native participants with a persisted full closure certificate to retire their heads and reclaim every surviving source on retry. Keep uncertified legacy authority below identity quorum conservative, while retaining actual partial-delete and committed-object byte checks.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
* refactor(listing): keep paginated listing on ECStore only
`Sets` and `SetDisks` carried their own copies of `inner_list_objects_v2`,
`list_objects_generic`, `inner_list_object_versions`, `list_path` and
`list_merged`, plus a third copy of the task that converts merged raw entries
into the walk result stream. Only `ECStore` can answer a paginated listing,
since a page merges every set of every pool; the copies drifted (the `Sets`
copy still rejected `//` prefixes that #8605 made literal on `ECStore`) and
were reachable only from tests and one decommission helper.
Remove the `ListOperations` impls for `Sets` and `SetDisks` together with the
duplicated pagination code. `SetDisks::walk` stays as an inherent method for
set-scoped callers and now honors `versions_sort` like the store-wide walk;
`SetDisks::list_path_result` stays for the recursive delete lister. The walk
entry conversion moves into one `spawn_walk_entry_forwarder` shared by both
walks. The decommission receipt lister walks its pool's sets instead of
paginating through the removed `Sets` listing.
Tests that exercised the removed layers now exercise `ECStore` for pagination
or `SetDisks::walk` for per-pool version inspection. The architecture and
logging guard rows that pinned the removed `SetDisks`/`Sets` listing entry
points are dropped, and the error-other ratchet baseline shrinks by one.
* fix(listing): exclude synthetic directories from receipt walks
* refactor(listing): remove env-gated ListObjects index provider
Delete the experimental, default-off key-only / verified-page /
metadata-fast index provider from `ECStore::list_objects_generic` together
with everything only it drove: the `RUSTFS_LIST_OBJECTS_INDEX_*`,
`RUSTFS_LIST_OBJECTS_NAMESPACE_JOURNAL_*` and
`RUSTFS_LIST_OBJECTS_METADATA_FAST_*` readers, the persistent key-only
index and its on-disk format, the namespace mutation journal (state,
chaos injector, system/local backends, degraded set, global lock), the
index lifecycle/health/source-selection types, the verified-index
candidate pager, the `list-chaos` cargo feature, the
`rustfs_s3_list_objects_index_*` metrics and the archived bench script.
`list_objects_generic` now always walks. The write-path hooks
`observe_list_objects_mutation(s)` are gone; their callers bump the scanner
namespace generation directly through `observe_scanner_namespace_mutations`,
which is the sub-step the old hook performed before taking the journal
lock, so the generation observed by the scanner, the node RPC service and
the data-movement lease checks is unchanged while PUT/DELETE no longer take
a global tokio RwLock or read environment variables.
`ListSourceMode::{metadata_authority, requires_live_metadata_verification,
can_satisfy_strong_listing}` and `ListMetadataAuthority` were only reachable
from the provider and are removed as well; the enum, its variants, the
cursor encoding and `ListContinuationV2` are left untouched.
* docs(listing): define index retirement upgrade prerequisites
* fix(ci): retire the removed listing index test requirement
test(e2e): assert Abort refusal below identity quorum
Keep exactly-quorum success and recoverable cleanup assertions. For
minority staging authority, require typed refusal, zero further Delete
RPCs, unchanged per-disk metadata and the previous object body. This
aligns the phase matrix with the existing conservative recovery contract.
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* test(e2e): assert Abort refusal below identity quorum
Keep exactly-quorum success and recoverable cleanup assertions. For
minority staging authority, require typed refusal, zero further Delete
RPCs, unchanged per-disk metadata and the previous object body. This
aligns the phase matrix with the existing conservative recovery contract.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): exercise simultaneous multipart terminal contenders
Retain the four pairwise resend cases and add live/cancelled three-way Complete, Abort and resend races. Observe distinct forwarded lock owners and verify terminal bytes and physical cleanup.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): freeze contender evidence before multipart cleanup
Retire the passive lock observer before terminal ListParts checks so the reported count identifies only the two contenders at the preparation barrier.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): require both multipart terminal race outcomes
Retain the free races and select both terminal winners with independently released, owner-scoped lock gates. Verify real contender admission, unchanged signed packets and exact terminal bytes for live and cancelled part waiters.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): isolate losing coordinator lock votes
Hold every remote vote for the losing coordinator while the same gate observes the winner. Prevent partial lock grants from delaying the requested terminal order and retain ordinary RPC caller lifetime.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* refactor(listing): keep paginated listing on ECStore only
`Sets` and `SetDisks` carried their own copies of `inner_list_objects_v2`,
`list_objects_generic`, `inner_list_object_versions`, `list_path` and
`list_merged`, plus a third copy of the task that converts merged raw entries
into the walk result stream. Only `ECStore` can answer a paginated listing,
since a page merges every set of every pool; the copies drifted (the `Sets`
copy still rejected `//` prefixes that #8605 made literal on `ECStore`) and
were reachable only from tests and one decommission helper.
Remove the `ListOperations` impls for `Sets` and `SetDisks` together with the
duplicated pagination code. `SetDisks::walk` stays as an inherent method for
set-scoped callers and now honors `versions_sort` like the store-wide walk;
`SetDisks::list_path_result` stays for the recursive delete lister. The walk
entry conversion moves into one `spawn_walk_entry_forwarder` shared by both
walks. The decommission receipt lister walks its pool's sets instead of
paginating through the removed `Sets` listing.
Tests that exercised the removed layers now exercise `ECStore` for pagination
or `SetDisks::walk` for per-pool version inspection. The architecture and
logging guard rows that pinned the removed `SetDisks`/`Sets` listing entry
points are dropped, and the error-other ratchet baseline shrinks by one.
* fix(listing): exclude synthetic directories from receipt walks
* fix(listing): stop abandoned walks and filter directory entries
While a remote namespace scanner walk runs, the fence watcher reads the
persisted cycle state every 5 s and stops the walk on any error. A read
that hits the 5 s distributed-lock timeout on .bloomcycle.bin is then
reported as a fence change: the bucket scan fails, the set cache is not
published and the cycle ends without a durable usage snapshot.
Return ScannerError::CycleStateUnavailable (same message as before) when
the state cannot be read, and let the watcher tolerate up to 12
consecutive such polls before stopping, logging a warning each time and
resetting the count after a successful poll. A mismatching fence and any
other error still stop the walk at once; the fence check when the stream
opens is unchanged.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ecstore): preserve uncertainty after partial multipart abort
Keep recovery callers from reporting NoSuchUpload while minority primary metadata remains. Retain all quorum and deletion checks. Exercise actual partial Abort and explicitly retained late RPC packets after caller retirement, staging cleanup and bucket recreation.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): prevent retired multipart staging resurrection
Require the upload root to exist before preparing recovery publication. Preserve rooted directory walks for nested bases on non-Linux Unix and cover missing-source native work, configured aliases and pinned mixed-version coordinator recovery.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): use IPv4 peer aliases for pinned mixed recovery
Keep direct IPv6 S3 listeners behind IPv4 RPC proxies for the pinned previous release, whose peer health resolver rejects bracketed IPv6 hosts. Preserve signed authorities, ports and strict recovery assertions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): separate pinned startup from MPU fault budgets
Use the old fleet defaults during bootstrap, then apply the original short fault budgets to the current replacement. Keep direct S3 URLs aligned with IPv6 listeners and preserve all quorum and physical cleanup assertions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): seed legacy MPU through transparent transport
Retire raw proxy connections before enabling method faults so legacy streaming setup stays byte-transparent and every later scoped mutation must traverse the fault barrier. Verify raw and adapted protocol authority, bodies and trailers.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(e2e): retain direct listeners for legacy MPU peers
Advertise same-port localhost peers for the pinned resolver while retaining direct IPv4 S3 listeners. Exact scoped RPC hit counts still reject bypassed injection and all physical authority assertions remain unchanged.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(server): keep startup gRPC on the authenticated router
Use the hybrid protocol predicate for startup RPC admission so early control responses retain production authentication and owner validation. Keep REST FullReady gating and pinned replay capability rejection intact. Add deterministic regressions, scoped diagnostics and bounded legacy fixture resources.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ci): verify MPU recovery across upgrade source versions
Run the exact sixteen-drive mixed MPU case for every checksum-verified upgrade source, independently of generic compatibility failures. Require one listed test and retain per-version logs while preserving the historical distributed source pin and every recovery assertion.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ci): trigger upgrade MPU checks for fixture changes
Include distributed fixtures and the RPC fault proxy in the upgrade
workflow paths so test-only changes schedule the strict recovery gate.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(storage): verify resend owners across MPU finalization
Hold real part-transaction RPCs while another coordinator completes or
aborts the upload, with both live and cancelled client response waiters.
Require full object bytes, terminal upload state, and all staging roots
to remain consistent after the part owner drains.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* feat(scanner): expose startup phases and attempt deadlines
Share one node-local startup snapshot between native status and metrics.
Track actual waits and first attempts without changing admission timing,
coverage acknowledgement, or publication semantics.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(scanner): observe startup recovery before initialization
Create the observation owner at enabled startup entry and move it through
the existing initializer, preserving admission timing and disabled cleanup.
Cover blocked recovery discovery and cancellation through the public entry.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* fix(s3): normalize UploadPartCopy source versionId and populate source versioning state
execute_upload_part_copy forwards CopySource::Bucket.version_id verbatim without normalizing 'null' to Uuid::nil() or validating UUID syntax up front (returning 500 InternalError instead of succeeding on versionId=null or returning 400 InvalidArgument on malformed versionId), and neither execute_upload_part_copy nor execute_copy_object populates src_opts.versioned / src_opts.version_suspended from BucketVersioningSys.
Root Cause:
Unlike execute_copy_object (issue #4238) and get_opts, execute_upload_part_copy in rustfs/src/app/multipart_usecase.rs extracts version_id from CopySource::Bucket without trimming, converting 'null' to Uuid::nil().to_string(), or validating UUID syntax via Uuid::parse_str, causing prepare_get_object_reader to fail with 500 InternalError when versionId=null or an invalid UUID is passed in x-amz-copy-source. Additionally, neither execute_upload_part_copy nor execute_copy_object populates src_opts.versioned and src_opts.version_suspended from BucketVersioningSys, and execute_upload_part_copy returns x-amz-copy-source-version-id without gating on whether versioning is enabled or suspended on the source bucket (issue #4976).
Fix:
Add normalize_copy_source_version_id() and format_copy_source_version_id() in rustfs/src/app/multipart_usecase.rs, populate src_opts.versioned and src_opts.version_suspended from BucketVersioningSys on both UploadPartCopy and CopyObject, and gate x-amz-copy-source-version-id on source bucket versioning state.
Signed-off-by: czcbangkai <czcbangkai@google.com>
* fix(s3): keep copy version validation within footprint guard
---------
Signed-off-by: czcbangkai <czcbangkai@google.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: Hauser <housemecn@gmail.com>
* test(ecstore): pin historical compression index reads
Exercise fixed pre-varint index bytes through the production reader,
including full and ranged reads, index fallback, and body CRC failure.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): use typed compression plan offsets
Use the existing usize storage offsets directly in the compatibility fixtures, fixing the same-type conversion errors reported by Clippy. Keep the planned-offset and decoded-byte assertions unchanged.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Workers::give() decrements in_use to 0 before calling semaphore.add_permits(1), while a queued Workers::take() task only increments in_use after its woken future is scheduled and polled, allowing Workers::wait() to return prematurely while a woken worker is still in flight.
Root Cause:
In crates/concurrency/src/workers.rs, Workers::give() decrements in_use inside self.in_use.send_modify() before calling self.semaphore.add_permits(1), while Workers::take() only increments in_use after self.semaphore.acquire().await returns and the woken task is scheduled and polled. During the handoff window when the last active worker calls give() and transfers its semaphore permit to a queued take() waiter (so semaphore.available_permits() < limit), in_use is 0, causing Workers::wait() (rx.wait_for(|&held| held == 0)) to complete prematurely while work is still in flight and Workers::available() to over-report available capacity.
Fix:
Require both held == 0 and self.semaphore.available_permits() == self.limit in Workers::wait(), notify self.in_use watchers after self.semaphore.add_permits(1) in Workers::give() and via an AcquireCancelGuard in Workers::take(), and bound Workers::available() by self.semaphore.available_permits().
Signed-off-by: czcbangkai <czcbangkai@google.com>
Co-authored-by: Hauser <housemecn@gmail.com>
* perf(ecstore): unify per-disk blocking workers with spin keep-alive and coalesce DELETE hops
Unify per-disk blocking I/O and fsync workers under a 3-lane DiskBlockingQueue
(Foreground, Isolated, Fsync) with MAX_NAMESPACE_WORKERS_PER_DISK = 1 and a
bounded user-space spin/yield keep-alive (WORKER_IDLE_SPIN_ITERS = 256), and
coalesce redundant blocking dispatches along the DELETE hot path:
- Expand DiskBlockingQueue to 3 lanes (Foreground, Isolated, Fsync) with 1
warm worker per disk/lane and a 256-iteration spin/yield keep-alive before
exiting spawn_blocking, preventing Tokio BlockingPool shared.idle futex
wake/wait thrashing between back-to-back stages of PUT and DELETE.
- Route fsync_dir, fsync_dir_with_owner, and namespace file-sync barriers
through the per-disk Fsync lane on FSYNC_RUNTIME, and replace LocalDisk's
duplicate MetadataReadQueue with LocalDisk::run_metadata_read backed by
self.blocking_queue.
- Route write_all_internal(Owned), open_file(O_TRUNC), remove_file_with_owner,
and remove_dir_with_owner through DiskBlockingQueue with pre-execution
cancellation checks.
- Coalesce DELETE directory scans in commit_reserved_version_delete and
finish_version_delete into single blocking passes with periodic cancellation
polling, batch rollback file writes in write_delete_rollback_files, batch
rollback unlinks + directory fsync in remove_files_and_fsync_with_owner, and
coalesce post-rename directory fsyncs in activate_delete_preparation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d85aaf47-2a4b-4c2c-97a3-dbe8c7717e2a
* perf(ecstore): isolate durable work and reduce idle queue locking
Route coalesced writes and delete settlement through the fsync lane while preserving namespace leases, cancellation ownership, and durability ordering. Keep relaxed work on the foreground lane and replace idle mutex polling with an advisory atomic hint; dequeue and worker retirement remain mutex-protected.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): yield disk workers across shared blocking pools
Limit each draining task to 64 completed jobs and reuse the worker guard to reschedule remaining work behind other disks. Preserve per-disk FIFO, worker limits, and per-operation ownership. Add a deterministic two-disk regression with one blocking thread.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): release queued jobs when workers cannot start
Construct the worker guard before submission and track whether execution began. Release pending jobs outside the lane mutex when the last worker is rejected, so waiters, resource units, and self-owning namespace leases cannot remain stranded. Cover closed runtimes and shutdown during batch handoff.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(connect): synchronize restart with observed inventory
Wait for the received POST instead of treating transport backoff as proof of delivery. Inject a pre-HTTP connection drop and use a verified loopback IP fixture without changing timeout or replay assertions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* refactor(ecstore): unify disk worker lifecycle and drain loop
Represent submitted, running, and retired workers explicitly. Share dequeue, execution, idle recheck, and retirement in one loop while preserving bounded handoff and rejected-task ownership cleanup.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(storage): separate real I/O from paused-clock assertions
Complete durable fixtures before pausing virtual time, drive the immediate scanner tick before freezing time during filesystem waits, and keep checkpoint-backed retry accounting on a real clock. Preserve default fsync isolation, disk deadlines, retry counts, and all outcome assertions.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Hauser <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Copilot-Session: d85aaf47-2a4b-4c2c-97a3-dbe8c7717e2a
A delete-all lifecycle expiry is a prefix delete that takes the bucket
lifecycle write lock, while the rebalance entry holds that lock for read
through the inline apply and its bucket incarnation fence. The write
waited out the lock timeout, stalled every lifecycle read in the bucket,
and failed the entry on every retry.
Like decommission (#8777), rebalance now moves such a version live and
lifecycle expires it on the target.
Refs rustfs/backlog#2873
The listing pipeline signalled "this page exhausted the namespace" by
setting `MetaCacheEntriesSortedResult.err = Some(Error::Unexpected)` and
had every consumer filter that variant, so any future `Unexpected` on the
producer path would have been read as a complete listing. A second truth
source, the shared `producer_limit_reached` atomic, was last-writer-wins
across the disks of a page and was never serialized to remote peers.
Collectors now return `ListPage { entries, exhausted }`, errors surface as
errors, and the early empty-listing exits return an exhausted empty page.
The producer's stop reason travels in the metacache stream itself:
`MetacacheWriter::close_with_limit_reached` appends it after the `false`
terminator as repeated terminator markers (the only trailing bytes a
deployed reader tolerates, because `list_path_raw_inner` keeps polling
readers after EOF), `MetacacheReader::limit_reached` decodes it, and
`list_path_raw_inner` folds every reader's trailer into a
`ListPathRawOutcome` that reaches the collector. Remote disks therefore
report their per-disk limit stops as well, and the `producer_limit_reached`
fields on `WalkDirOptions`, `ListPathRawOptions` and `ListPathOptions` are
removed.
* test(e2e): tighten scoped multipart fault acceptance
Target Abort root deletion instead of child transaction cleanup, require
actual metadata peer hits, and keep faulted APIs single-attempt. Validate
bounded ordinary recovery and safe quorum refusal for partial mutations,
with the recovery enhancement tracked in backlog#2940.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(e2e): drive concurrent TLS reads independently
Give each coordinator its own supervised task and cooperative polling
budget while retaining the existing workload, deadline and byte/hash
assertions. Preserve node logs after fixture teardown and report the
operation, response phase and received bytes when a transfer times out.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* fix(listing): honor persisted erasure quorum
Separate metadata admission and primary-reader sizing from each version's
persisted erasure requirements. Propagate agreed-entry uncertainty and
report LIST read-quorum failures as storage unavailability.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* perf(listing): borrow metadata during quorum resolution
Keep validated metadata inspection and agreed-version selection in
filemeta. Resolve independent reader votes through borrowed slots and
retain owned buffers until a filtered result is required.
Preserve fixed-baseline legacy comparisons, cached latest fast paths,
quorum policy, and disk fan-out. Add ownership regressions and paired
release benchmarks for cold and cached version histories.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): remove redundant fallback clone
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Run two cargo update and cargo upgrade rounds from origin/main.
Update mimalloc to 0.7.2, hotpath to 0.29.1, and ZIP to 9.1.0,
with compatible crypto, compression, and smallvec updates.
Retain the generic-array version required by crypto-common 0.1.7.
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* fix(scripts): collect scanner host telemetry on macOS
Use platform-supported iostat arguments and expose optional collector
failures without interrupting scanner-status collection.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* perf(ecstore): reuse validated metadata read path buffers
Share the checked bucket/object tuple across metadata read consumers and borrow clean normalization inputs without skipping filesystem validation. Keep Windows reconstruction and the second bucket check intact.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(scripts): finalize scanner telemetry for failed runs
Initialize host telemetry artifacts before external captures so reused output directories cannot retain prior successes. Share normal and error-path finalization, reap collectors on request failure, and preserve the primary exit status even if status recording fails.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): distinguish unreadable recovery inventories
Treat only the exact system-prefix read-quorum result as incomplete during bounded polling. Preserve strict final reads and the shutdown/drain assertions, record failed observations, and cover interrupted empty streaks and unrelated errors.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* fix(rio): ZigZag-encode signed delta varints and widen scratch buffer in compress_index
Index::append_to passes signed delta residuals (c_off and u_off, which are negative whenever a compressed block is smaller than est_block_uncomp / 2) to write_varint(buf, v: i64), which checks while v >= 0x80 without ZigZag encoding and truncates every negative delta to a single byte, while tmp = [0u8; 8] panics out-of-bounds on 9-10 byte varints.
Root Cause:
In crates/rio/src/compress_index.rs, Index::append_to delta-encodes u_off and c_off (c_off -= prev.compressed_offset + c_predict, where c_predict starts at est_block_uncomp / 2) as signed i64 residuals and passes them to write_varint(buf, v: i64). However, write_varint checks while v >= 0x80 on the signed i64 without ZigZag encoding (unlike Go binary.PutVarint / binary.Varint in the S2 index format): for any negative v, v >= 0x80 is false, so write_varint writes a single truncated byte v as u8 without continuation bits, either silently corrupting compressed_offset on Index::load (when (v as u8) < 0x80) or desynchronizing the varint stream (when (v as u8) >= 0x80). Additionally, tmp = [0u8; 8] at line 223 panics with an out-of-bounds index when encoding 9-10 byte varints, and read_varint lacks an overflow check on 10-byte inputs.
Fix:
Use ZigZag encoding and decoding in write_varint and read_varint, reject 64-bit varint overflows in read_varint, and widen the scratch buffer tmp in Index::append_to from [0u8; 8] to [0u8; 10].
Signed-off-by: czcbangkai <czcbangkai@google.com>
* test(rio): use est_block_uncomp field and compare IndexInfo fields in compress_index test
---------
Signed-off-by: czcbangkai <czcbangkai@google.com>
`LocalDisk::walk_dir` stored its own `limit_reached` into the
`producer_limit_reached` flag that every producer of a page shares, so a
walk reaching EOF after a sibling stopped at its per-disk limit cleared the
signal and the collector reported a complete page. Use `fetch_or` so the
flag answers "did any bounded walk stop early", whatever the finish order.
`MetaCacheEntry::is_in_dir` stripped the directory prefix repeatedly, so
`a/a/b` counted as a direct child of `a/`. `MetaCacheEntriesSorted::
forward_past` kept every entry when none sorted after the marker and split
on an index computed over the flattened entries. Both are masked today by
the collector's pre-filtering; fix them and pin the contracts with tests.
* fix(multipart): allow abort after partial commit
Read staging metadata at read quorum during Abort while retaining the
delete write quorum and generation checks. Bound post-fault lock recovery,
isolate quorum rows from peer breaker history, and cover the metadata and
deletion boundaries.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* perf(multipart): release topology guards before metadata I/O
Read multipart metadata from owned disk snapshots so slow I/O does not
block registry renewal. Cover read and write lookups with a deterministic
paused-I/O regression.
Reuse one cluster per quorum matrix after observing fresh responses from
every peer. Preserve fault-hit and quorum assertions without repeatedly
starting clusters or disabling peer request breakers.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* perf(ecstore): avoid head-of-line blocking in delete recovery
Recover independent keys through a bounded rolling window while preserving per-key hint generations, admission budgets, cancellation retry, and the existing transaction decision engine.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* perf(ecstore): bound durable delete preparation and bootstrap recovery
Prepare independent Linux delete transactions with bounded concurrency while retaining per-key result order and serial fallback for overlapping, rollback, and legacy requests. Preserve startup discovery until bucket metadata is initialized, and add release performance and process-crash acceptance runners.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): verify clean exits in recovery acceptance
Reject forced or OOM exits when measuring clean startup and verify original ETags across mixed-layout crash recovery and a second restart.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): distinguish prepared and decided batch crashes
Observe real preparation and receipt states separately, retain non-vacuous acknowledged-delete sentinels, and require read-only physical convergence before S3 reads can trigger cleanup.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test(ecstore): move final delete recovery options
Move the options value into the batch-ordering regression's final case
instead of cloning it. Keep the earlier clone needed for reuse.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(ecstore): replay delete receipts after interrupted proof GC
Retire only the exact empty transaction leaf before journal activation. Preserve occupied directories and prefix children, and cover partial cross-disk proof GC without a fresh delete. Tighten crash evidence to distinguish directory-only remnants from retained content.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
* fix(scanner): bound implicit startup delay and publish config early
Limit warm-cache startup to the smaller of the cycle interval and one
minute before jitter, while preserving explicit startup settings.
Publish configuration gauges when applying runtime settings and retain
timeout diagnostics until an actual new cycle begins.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(scanner): unify startup waits and skip unchanged configuration
Bound read-only startup inspection and make startup and pre-leadership
retries respond to cancellation and configuration changes. Preserve one
startup deadline and discover replication behind lifecycle-only buckets.
Avoid irrelevant bucket listings and repeated per-disk reconfiguration.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(scanner): retain effective startup policy and fallback metrics
Keep the sampled startup deadline when configuration changes leave its
normalized policy unchanged. Initialize retained runtime metrics after
an invalid first refresh without suppressing the validation error or
replacing a newer accepted configuration.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
get_online_offline_disks_stats() counts and demotes all root_disk entries regardless of drive state, underflowing online_disks[ep] (0usize - 1) and double-counting offline_disks[ep] when a root_disk drive is offline or unknown alongside healthy non-root drives.
Root Cause:
In crates/ecstore/src/diagnostics/admin_server_info.rs: (1) root_disk_count counts every drive with root_disk = true even if the drive was already classified into offline_disks or unknown_disks; (2) the subsequent demotion loop unconditionally decrements online_disks[ep] and increments offline_disks[ep] for every disk.root_disk without checking whether disk.state was classified as online (ok or unformatted). When an endpoint has an offline or unknown root_disk drive and zero online drives, *online_disks.get_mut(ep) -= 1 underflows 0usize (panicking in debug/checked builds or wrapping to usize::MAX) and inflates offline_disks.
Fix:
Introduce is_online_drive_state() and only count and demote root_disk drives whose state was classified as online (ok or unformatted), preserving the offline and unknown bucket tallies and keeping online + offline + unknown equal to disks_info.len().
Signed-off-by: czcbangkai <czcbangkai@google.com>
cxymds