refactor(ecstore): retire the set_disk lint blankets by making the prelude explicit
backlog#1823 step 1 / backlog#2029 road 2. Removes the last two module-level lint blankets in ecstore: set_disk/mod.rs #![allow(unused_imports)] and #![allow(unused_variables)], restoring both lints for the whole 40K-line subtree, and deletes the register line for the unused_variables blanket in the same diff (the guard from #6155 is a bidirectional exact match).
The unused_imports blanket existed because 14 submodules consumed mod.rs as a glob prelude (use super::* / use super::super::*), and rustc does not track consumption through glob re-exports. Each glob is now an explicit use super::{...} list, keeping mod.rs as the single import hub while making every import lint-checkable. Names consumed only by test or test-util units carry #[cfg(test)] / #[cfg(all(test, feature = "test-util"))] / #[cfg(any(test, feature = "test-util"))] gates matching their consumers; storage-api traits are routed through the storage_api_contracts facade per the architecture guard.
The sweep then deleted the genuinely dead imports the blanket was hiding (chrono::Utc, glob::Pattern, futures::task::AtomicWaker, rustfs_lock LocalLock, AsyncBatchProcessor, rand::Rng, std::future::Future among others in mod.rs, plus stale scoped imports and one empty test module shell across the subtree). One unused_variables finding surfaced: flush_read_version_coalescer_pending's lane_key is read only by the #[cfg(test)] counter block, handled with the cfg(not(test)) let _ pattern established in #6158.
Verification: cargo check zero warnings versus the 9cf276ed2 baseline on five lanes (default lib / --tests / rio-v2 --tests / test-util --tests / test-util,rio-v2 --tests; the --tests lane keeps the same three pre-existing core/pools.rs and store/object.rs dead-code warnings main already has); clippy --lib --tests -D warnings clean with test-util,rio-v2; cargo nextest run 4567 passed; make pre-commit exit 0.
* refactor(ecstore): drop the client shim, import rustfs-s3-client directly
Completes the migration window opened by the rustfs-s3-client extraction (rustfs/backlog#1842 PR3): every consumer now imports the client crate directly and the crate::client shim is deleted.
- All in-crate crate::client:: paths (tier warm backends, tier core, lifecycle tier_sweeper, replication storage boundary, set_disk) now import rustfs_s3_client::* directly; crates/ecstore/src/client/mod.rs and the lib.rs mod client declaration are gone.
- The two server-side modules historically misfiled under client/ move to their real homes: object_api_utils.rs to crates/ecstore/src/object_api/ (it builds engine-side object readers/writers), and object_handlers_common.rs to crates/ecstore/src/bucket/lifecycle/ (it is the lifecycle noncurrent-version cleanup helper). The latter now routes its replication calls through the lifecycle replication_sink boundary (schedule_delete wrapper and the sink's ReplicationObjectBridge re-export), as the lifecycle guard requires.
- The ecstore public facade drops api::client: object_api_utils is exposed as api::object_api_utils, and the rustfs crate takes admin_handler_utils (AdminError) from rustfs-s3-client directly (new dependency).
- Guard updates: the migration guard no longer pins mod client in ecstore's lib.rs or the admin_handler_utils facade module (it pins the new api::object_api_utils facade instead), and the module-lint register follows object_api_utils.rs to its new path.
Verification: cargo check -p rustfs-ecstore --all-targets and -p rustfs; cargo fmt --all; tier/transition/lifecycle-focused nextest (626 passed) and the decommission/rebalance/heal families in a filtered run (603 passed; the full-suite parallel run only fails on this machine's known decommission/rebalance baseline flakes, which pass in filtered reruns and fail identically on pristine origin/main); layer/migration/s3s/logging/error-format/doc-path guard scripts all pass.
* docs(architecture): record the S3 client extraction and reword invariant 4 (#6669)
Closes the documentation step of rustfs/backlog#1842. ARCHITECTURE.md invariant 4 now states the serving-vs-consuming distinction the adversarial ruling asked for: ecstore must not serve HTTP/S3 wire types, while consuming remote S3 endpoints is a legitimate engine capability that lives in the extracted rustfs-s3-client crate. The violation note is updated from the pre-extraction snapshot (58 files, embedded client) to the current ratcheted state (shrink-only S3S_ECSTORE_FILES_BASELINE in scripts/check_s3s_footprint.sh, object_lock converted first), and the crate map gains s3-client. ecstore-module-split-plan.md gets the client-directory entry the plan was missing: a Current Shape row and a completed-extraction section describing the pure-move + shim + direct-import sequence and the re-homing of the two misfiled server-side modules.
The object_lock module evaluated WORM state through s3s wire DTOs (ObjectLockRetention, ObjectLockLegalHold, DefaultRetention, Date) and s3s header constants, keeping the storage engine coupled to the serving protocol (rustfs/backlog#1842, ARCHITECTURE.md invariant 4). This PR gives the module its own storage-level vocabulary and pushes the DTO conversions to the boundaries that already speak s3s.
New crates/ecstore/src/bucket/object_lock/types.rs defines RetentionMode, LegalHoldStatus, ObjectRetention, ObjectLegalHold, and DefaultRetention with no s3s dependency. objectlock.rs parses persisted metadata into these types using the rustfs-utils lowercase header constants (the same literal keys as before, pinned by the existing g-key-002 test). objectlock_sys.rs evaluates retention/legal-hold/default-retention from them; the fail-closed error messages and decision logic are unchanged line for line where possible.
Boundary conversions:
- bucket/metadata_sys.rs gains default_retention_from_object_lock_config, converting the persisted s3s configuration into the storage-level DefaultRetention; a rule without a usable GOVERNANCE/COMPLIANCE mode converts to None exactly like the evaluation code always ignored it, and days/years pass through so an invalid period still fails closed at evaluation time.
- check_object_lock_for_deletion_with_config becomes check_object_lock_for_deletion_with_default_retention (it only ever read the default retention); the lifecycle object_lock_boundary keeps the old s3s-typed signature and converts.
- The ObjectLockApi / ObjectLockStatusExt trait impls for the s3s DTOs move next to the persisted configuration owner in bucket/metadata.rs; the traits stay in object_lock/mod.rs.
- check_retention_for_modification now takes Option<RetentionMode>. The serving-layer wrappers (rustfs storage_api, set_disk options path) convert the request string with the new RetentionMode::parse_exact, which accepts only the canonical spelling — preserving the historical literal comparison where a non-canonical requested mode reads as a mode change and stays blocked.
- rustfs app-layer wrappers return the storage types; the replication-overwrite gate in object_usecase.rs uses the typed API (legal_hold.is_on(), RetentionMode::Compliance).
Ratchet: the ecstore-scoped s3s counter drops 42 -> 39 and the repo-wide file counter 211 -> 208 in scripts/check_s3s_footprint.sh.
Verification: cargo check -p rustfs-ecstore --all-targets and -p rustfs (lib+bins); cargo clippy -p rustfs-ecstore --all-targets and -p rustfs --lib --bins (clean); cargo nextest run -p rustfs-ecstore --no-fail-fast (4534/4542; the 8 failures are the same store::rebalance / store::heal machine-baseline set that fails identically on pristine origin/main, plus one fencing flake that passes in isolation); all object_lock/retention/legal-hold tests pass; guard scripts (layer deps, migration rules, s3s footprint, logging, error-format ratchet, doc paths) pass.
The storage engine embedded a ~8.4K-line hand-written S3 HTTP client under crates/ecstore/src/client (rustfs/backlog#1842). That client is a legitimate engine capability — it consumes remote S3-compatible endpoints for ILM tier warm backends and transition targets — but it was misfiled inside the engine, dragging s3s/hyper wire types into ecstore and blocking ARCHITECTURE.md invariant 4.
This PR is the pure-move step: 21 modules move verbatim to the new crates/s3-client crate (rustfs-s3-client), and crates/ecstore/src/client/mod.rs becomes a re-export shim so every in-crate crate::client:: path keeps working. The two server-side modules that were historically misfiled under client/ — object_api_utils.rs and object_handlers_common.rs — stay in ecstore.
Three reverse dependencies from the client into engine internals are severed so the move can be pure:
- transition_api::ReaderImpl::ObjectBody held ecstore's GetObjectReader; the client only ever reads the body, so the variant now holds an ObjectReader newtype over Box<dyn AsyncRead + Send + Sync + Unpin> with the same read_all() surface. The single production construction site (set_disk transition upload) and the two engine-side consumers were adjusted.
- api_list/api_remove used ecstore's storage_api_contracts / object_api types; api_list now imports BucketInfo from rustfs-storage-api directly, and api_remove uses the client's own transition_api::ObjectInfo (only .name/.version_id were read; the error-path bucket name is now threaded as a parameter instead of read from the deleted objects).
- the api_put_object_streaming regression tests built a GetObjectReader by hand; they now wrap the duplex stream in ObjectReader::new.
Guard updates: the s3s footprint ratchet gains an ecstore-scoped counter (42 files, shrink-only, per rustfs/backlog#1842), the ecstore module-lint-blanket register follows the moved files into crates/s3-client so the blanket ratchet keeps covering them, the logging guardrail path pin follows transition_api.rs, and the ::other(format!) baseline is regenerated (moved call sites left ecstore).
Verification: cargo check -p rustfs-s3-client -p rustfs-ecstore; cargo nextest run -p rustfs-s3-client (43 passed) and -p rustfs-ecstore (4515/4523; the 8 failures reproduce identically on pristine origin/main on the same machine); cargo clippy --all-targets; scripts/check_layer_dependencies.sh, check_architecture_migration_rules.sh, check_s3s_footprint.sh, check_logging_guardrails.sh, check_error_other_format_ratchet.sh, check_doc_paths.sh, check_ci_paths_sync.sh all pass.
refactor(ecstore): make store-to-disk error narrowing a named fallible operation
Backlog#1845 step 4. The blanket impl From<StorageError> for DiskError let ? silently push store-only errors (locks, buckets, quotas) across the disk boundary into DiskError::other, where the rendered message fragments reduce_errs quorum buckets. Same story for the blanket From<StorageError> for rustfs_filemeta::Error and its other() catch-all.
Both impls are replaced by named, fallible methods: StorageError::narrow_to_disk() and StorageError::narrow_to_filemeta(). Variants with an identity on the far side map across unchanged - including the two documented lossy collapses (SlowDown -> TooManyOpenFiles, StorageFull -> DiskFull) that the round-trip tests pin - and everything else returns Err(self) so the call site decides what crossing the boundary means. Removing the impls let the compiler enumerate every conversion site; the census that scoped this issue had found 5, the compiler found 33.
Call sites keep their existing behavior: the io identity bridge and the generic sites fold Err into the io-backed other() exactly as the old catch-all did (identity still recoverable by downcast), listing paths use one shared to_filemeta_err helper, and the two sites that relied on the SlowDown collapse now construct DiskError::TooManyOpenFiles directly so the loss is visible where it happens. No behavior change intended anywhere; the io::Error bridge itself is untouched by design.
Ref rustfs/backlog#1845
The doc comment named RUSTFS_LOCK_ACQUIRE_TIMEOUT with a 30-second
default, but the function reads RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT
with a 5-second default. RUSTFS_LOCK_ACQUIRE_TIMEOUT is a real,
separate knob read by the lock and scanner crates, so tuning the
documented name silently has no effect on this path.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Single-part encrypted objects in the legacy format could not serve range
reads without decrypting from byte 0: v1 frames are emitted per upstream
read, so no closed-form plaintext-to-ciphertext mapping exists. The v2
layout fixed the frame length (8218 ciphertext bytes per 8 KiB plaintext
frame), making the mapping closed-form.
Consume it:
- Single-part PUTs that encrypt locally under the v2 write switch stamp
the frame-layout marker, valued with the object's data_dir token -
ciphertext passthrough, data movement and copies mint a new data_dir
or strip the marker, so a re-homed marker disqualifies itself.
- The encrypted read plan seeks marked, uncompressed single-part objects
to frame_index * 8218 and decrypts from that frame: the frame index
rides the plan's sequence-number slot into DecryptReader::new_at_block,
whose nonce and AAD bind absolute indices. New metric path label
frame_seek.
- A lying marker fails closed: v2 authentication rejects bytes at a fake
frame boundary; plaintext is never served from the wrong offset.
Compressed objects and multipart sub-part seeks keep the conservative
paths (follow-up work); reading needs no switch - seekability follows
the marker.
* feat(mimalloc): add arena diagnostics and configuration
Based on mimalloc maintainer feedback (microsoft/mimalloc#1372),
add diagnostics to check mimalloc arena configuration at runtime.
Changes:
- Add rustfs-mimalloc-sys to workspace dependencies
- Add log_mimalloc_diagnostics() function to check:
- arena_max_object_size
- pagemap_commit status
- mimalloc version
- Add memory_observability module with mimalloc diagnostics
This helps diagnose why allocations might be going outside arenas,
which is the suspected root cause of futex contention.
Ref: rustfs/backlog#2005
Ref: microsoft/mimalloc#1372
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): add Vec<u8> buffer pool for EC operations
Add a general-purpose buffer pool to reduce Vec<u8> allocations
in hot paths like EC encoding/decoding.
Changes:
- Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs
- Thread-safe pool with capacity-based bucketing (power-of-two)
- Global EC_BUFFER_POOL instance with 16 buffers per bucket
- Add buffer_pool module to codec/mod.rs
Expected impact:
- Reduce heap allocations in EC encode/decode paths
- Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool)
- Reduce mimalloc lock contention
Note: Main bottleneck remains mimalloc internal synchronization
(futex 98.64% time). Buffer pool provides modest improvement (+2-5%).
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and related files
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): add #[allow(dead_code)] to buffer pool
The BufferPool infrastructure is ready but not yet integrated
into the EC hot paths. Add #[allow(dead_code)] with clear
documentation about integration status.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): integrate BufferPool into bitrot verify path
Replace vec![0; shard_size] with get_ec_buffer() in the bitrot
verification hot path to reduce heap allocations and avoid memzero.
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and bitrot changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* refactor(ecstore): clean up buffer pool code
- Remove unnecessary #[allow(dead_code)] attributes
- Update module documentation to reflect current integration status
- Simplify code structure
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(runtime): cap default worker threads at 16
Testing showed 16 worker threads outperforms 32+ for 1KiB PUT
workloads due to reduced mimalloc lock contention.
A/B test results (testing 4-node cluster, c=64):
- worker_threads=32: 740 obj/s (baseline)
- worker_threads=16: 785 obj/s (+6.1%)
The default was detect_cores() which returned 32 on our testing
nodes. Cap at 16 for optimal small-object performance.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and runtime changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): remove unused BufferPool::new() function
The new() function was never used since EC_BUFFER_POOL
initializes directly with with_limits(16).
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): update buffer_pool tests to use with_limits
Replace BufferPool::new() with BufferPool::with_limits(16) in tests
since new() was removed in favor of with_limits().
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): optimize opts.clone() and FileInfo clone patterns
## Changes
1. ObjectOptions helper methods:
- add as_commit_opts(): creates commit options with no_lock=true,
metadata_cache_safe=false, include_part_checksums=true
- add as_read_opts(): creates read options with
include_part_checksums=true
- add with_no_lock(): creates options with modified no_lock field
2. Replace opts.clone() in hot paths:
- commit_opts = opts.as_commit_opts() (was 4-line manual clone)
- read_opts = opts.as_read_opts() (was 2-line manual clone)
3. Optimize FileInfo clone in rename path:
- avoid double clone: clone once and modify erasure.index in place
- pass &file_info reference to rename_data_borrowed_with_fence
## A/B Results (4-node cluster, c=64)
| Size | main | optimized | Change |
|------|------|-----------|--------|
| 1KiB | 892 obj/s | 920-976 obj/s | +3%~+9% |
| 4KiB | 957 obj/s | 903 obj/s | -5.7% |
| 16KiB | 922 obj/s | 855 obj/s | -7.3% |
Note: 1KiB improvement is consistent. 4KiB/16KiB variance
likely due to test noise; needs more rounds to confirm.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): add BytesMut buffer pool to EC encoding path
Pre-allocate a Vec<BytesMut> pool in the EC encoding loop to avoid
repeated heap allocations for ingest buffers.
Changes:
- Pre-allocate buffer pool with capacity 4
- Reuse buffers from pool after encoding
- Return buffers to pool when capacity is sufficient
Expected impact: +10-20% in EC encoding path by reducing
BytesMut allocation overhead.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: hector <hetor@rustfs.com>
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(ecstore): handle metadata-less bucket residue
Diagnose metadata-less on-disk residue before non-force DeleteBucket reaches physical deletion, and keep scanner-discovered metadata-missing objects on a non-destructive heal path.
Add explicit heal --remove cleanup for unversioned metadata-less data directories, using the existing data-dir delete primitive and fail-closed shape checks so pre-commit or unknown residue is preserved.
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(connect): adapt offline array validator
Wrap the filesystem summary validator in a closure so Option::is_some_and can pass the concrete array reference accepted by serde_json::Value::as_array.
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(connect): remove redundant offline test clones
Move the temporary path into the swap closure after deriving the output path, keeping clippy's redundant-clone lint clean for offline bundle tests.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* feat(mimalloc): add arena diagnostics and configuration
Based on mimalloc maintainer feedback (microsoft/mimalloc#1372),
add diagnostics to check mimalloc arena configuration at runtime.
Changes:
- Add rustfs-mimalloc-sys to workspace dependencies
- Add log_mimalloc_diagnostics() function to check:
- arena_max_object_size
- pagemap_commit status
- mimalloc version
- Add memory_observability module with mimalloc diagnostics
This helps diagnose why allocations might be going outside arenas,
which is the suspected root cause of futex contention.
Ref: rustfs/backlog#2005
Ref: microsoft/mimalloc#1372
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): add Vec<u8> buffer pool for EC operations
Add a general-purpose buffer pool to reduce Vec<u8> allocations
in hot paths like EC encoding/decoding.
Changes:
- Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs
- Thread-safe pool with capacity-based bucketing (power-of-two)
- Global EC_BUFFER_POOL instance with 16 buffers per bucket
- Add buffer_pool module to codec/mod.rs
Expected impact:
- Reduce heap allocations in EC encode/decode paths
- Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool)
- Reduce mimalloc lock contention
Note: Main bottleneck remains mimalloc internal synchronization
(futex 98.64% time). Buffer pool provides modest improvement (+2-5%).
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and related files
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): add #[allow(dead_code)] to buffer pool
The BufferPool infrastructure is ready but not yet integrated
into the EC hot paths. Add #[allow(dead_code)] with clear
documentation about integration status.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): integrate BufferPool into bitrot verify path
Replace vec![0; shard_size] with get_ec_buffer() in the bitrot
verification hot path to reduce heap allocations and avoid memzero.
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and bitrot changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* refactor(ecstore): clean up buffer pool code
- Remove unnecessary #[allow(dead_code)] attributes
- Update module documentation to reflect current integration status
- Simplify code structure
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(runtime): cap default worker threads at 16
Testing showed 16 worker threads outperforms 32+ for 1KiB PUT
workloads due to reduced mimalloc lock contention.
A/B test results (testing 4-node cluster, c=64):
- worker_threads=32: 740 obj/s (baseline)
- worker_threads=16: 785 obj/s (+6.1%)
The default was detect_cores() which returned 32 on our testing
nodes. Cap at 16 for optimal small-object performance.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and runtime changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): remove unused BufferPool::new() function
The new() function was never used since EC_BUFFER_POOL
initializes directly with with_limits(16).
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): update buffer_pool tests to use with_limits
Replace BufferPool::new() with BufferPool::with_limits(16) in tests
since new() was removed in favor of with_limits().
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: hector <hetor@rustfs.com>
Co-authored-by: heihutu <heihutu@gmail.com>
A single object's SourceChanged during decommission cleanup no longer
cancels the shared worker token and fails the whole pool operation.
Cleanup preflight and source-cleanup outcomes now retry per entry with
bounded attempts and cancellation-aware backoff, applied uniformly to
ordinary versions, delete markers, and tiered copies (removing the
try-once-only branches); every retry re-lists the entry and redoes
version multiset validation before touching the source. Only quorum
loss, unrecoverable system errors, or exceeding a pool-level
SourceChanged exhaustion threshold still fails the decommission, and
exhausted entries never delete their source versions.
Retry attempts, backoff, and deferred/exhausted reasons are logged per
entry for observability. Heavy regression tests spawn on dedicated
32MiB stacks following the existing store-test pattern.
Fixesrustfs/backlog#1913
Co-authored-by: houseme <housemecn@gmail.com>
* feat(allocator): replace mimalloc/libmimalloc-sys with rustfs-mimalloc/rustfs-mimalloc-sys
Replace the upstream xonatius/mimalloc_rust.git fork (mimalloc + libmimalloc-sys)
with the published rustfs-mimalloc (v0.5.0) and rustfs-mimalloc-sys (v0.5.0) crates
from crates.io.
The new crates are based on mimalloc V3 (v3.5.0) and provide:
- MiMalloc global allocator with safe API (collect, stats_json, process_info)
- Heap management and arena operations (heap module)
- Full FFI bindings to mimalloc V3
Changes:
- Workspace deps: mimalloc + libmimalloc-sys (git) → rustfs-mimalloc + rustfs-mimalloc-sys (crates.io)
- allocator_reclaim.rs: libmimalloc_sys::mi_collect → rustfs_mimalloc::MiMalloc::collect
- memory_observability.rs: raw FFI mi_stats_get_json → MiMalloc::stats_json()
- main.rs: heap ownership tests use Heap::contains() (V3 API)
- deny.toml: remove xonatius/mimalloc_rust.git from allow-git
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): default rename fanout to parallel early-ack path
Switch the default rename_data commit fanout from serial join_all to the
parallel JoinSet early-ack path. The serial path (#5987) was the primary
cause of the 1MiB PUT regression (-71.7%) observed in rc.3 benchmarks.
A/B verification on testing 4-node cluster (c=64, 1MiB PUT, 2min):
- Serial (join_all): 96.99 MiB/s, P50=644ms
- Early ack (JoinSet): 177.46 MiB/s, P50=407ms (+83%)
Also:
- Update rename_data_reclaims_synthetic_inline_rollback_dir_after_commit
to use rename_data_owned and await tail_drain for proper cleanup.
- Update rename_data_waits_for_tail_disk_after_write_quorum to explicitly
test the serial path (now non-default) via env override.
- Add error source chain to HTTP Body stream transport error log
(backlog#2005) so the underlying cause is visible.
Ref: rustfs/backlog#2005
Ref: rustfs/backlog#1792#issuecomment-5384346238
Ref: rustfs/backlog#1792#issuecomment-5384370938
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
The inline_block threshold used floor division (DEFAULT_INLINE_OBJECT_BUDGET
/ data_shards) while shard_file_size uses ceiling division (div_ceil). For
EC 12:4 with 256KiB objects, this caused a 1-byte discrepancy:
- inline_block = 262144 / 12 = 21845 (floor)
- shard_file_size = 262144.div_ceil(12) = 21846 (ceil)
- should_inline(21846, 12, false) = false (wrong!)
Fix by using div_ceil for the inline_block calculation, so both sides
use the same rounding and the inline path is correctly triggered.
Co-authored-by: heihutu <heihutu@gmail.com>
Add deterministic rename_data coverage for tail-disk success/failure, cancellation serialization, and strict quorum rollback visibility after disk reopen. This establishes the safety boundary before experimenting with write-quorum early ACK and background tail completion for backlog #925.
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(lifecycle): safely expire all object versions
* fix(lifecycle): preserve delete-all replication purges
* fix(lifecycle): remove dead replication journal
* fix(ci): avoid lifecycle transition test stack overflow
* fix(lifecycle): release recovery locks before tier IO
* test(lifecycle): align object-lock error assertions
* test(lifecycle): avoid scanner restore stack overflow
* test(scanner): avoid stack overflow in transition and restore flow test (#6300)
* refactor(scanner): split remote_scanner.rs into stream child module (#6289)
Split the 3080-line remote_scanner.rs (47% inline tests) into a
canonical foo.rs + foo/ module tree with zero behavior change:
- remote_scanner.rs (~320): protocol constants, process statics, and
the request decode/validate/admit/preflight/claim API plus root
re-exports
- remote_scanner/stream.rs (~1340): wire/frame types, replay cache,
FrameAuthenticator, serve path, local bucket scan + persist, client
scan, and the bounded stream plumbing
- remote_scanner/stream/tests.rs (~1470): the inline test module as a
child module of stream so it can reach both parents' private items
All crate paths are unchanged: lib.rs re-exports
(serve_remote_scanner_request, RemoteScannerRequest, ...) resolve
through root re-exports, and scanner_io's crate::remote_scanner::
{scan_remote_bucket, RemoteScannerScanSpec, RemoteScannerOutcome}
paths resolve through pub(crate) re-exports. Cross-module items gain
pub(super), whose scope equals the old single-module privacy domain;
no item's effective visibility widens. Code is moved verbatim apart
from those markers, per-module import headers, and rustfmt line
re-wraps.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(heal): split resume.rs into focused child modules (#6290)
Split the 4242-line resume.rs (46% inline tests) into a canonical
foo.rs + foo/ module tree with zero behavior change:
- resume.rs (~1020): state file constants, PersistThrottle, ResumeState,
ResumeManager core (constructors, load/discovery, progress mutators,
ordinary persistence) plus root re-exports
- resume/replacement.rs (~690): replacement-intent/proof types and the
ResumeManager replacement-lifecycle methods
- resume/checkpoint.rs (~350): ResumeCheckpoint + CheckpointManager
- resume/utils.rs (~310): ResumeUtils statics
- resume/tests.rs (~1980): the inline test module as a child module
All module paths are unchanged (heal::resume::CheckpointManager and
friends resolve through root re-exports), so no consumer inside or
outside the crate changes. Items defined in child modules keep
module-private visibility; only the ten cross-module helpers gain
pub(super), which is not part of the crate API. Code is moved verbatim
apart from those visibility markers, four super::storage_api path
fixes, and the new per-module import headers.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(scanner): split scanner_io.rs into child modules (#6294)
Split the 5369-line scanner_io.rs (39% inline tests) into a canonical
scanner_io.rs + scanner_io/ module tree with zero behavior change:
- scanner_io.rs (~660): constants, metadata-error constructors, the
bucket scan plan, cycle-status classification helpers, the ScannerIO /
ScannerIOCache / ScannerIODisk traits, and ScannerCycleResult
- scanner_io/dirty_usage.rs (~300): process-wide dirty-usage statics
and the acknowledgment protocol
- scanner_io/guards.rs (~270): concurrency gauges and RAII guards
- scanner_io/cache.rs (~410): scanner cache locks and the snapshot
persist/publish path
- scanner_io/io_cycle.rs (~390), io_cache.rs (~1160), io_disk.rs
(~230): the ECStore / SetDisks / Disk trait implementations
- scanner_io/publish_gate_tests.rs (~750) and tests.rs (~1340): the two
inline test modules as child modules
All crate paths are unchanged: the lib.rs scanner_io re-exports and
every crate::scanner_io:: consumer (scanner.rs, remote_scanner,
scanner_folder, and cross-crate rustfs users) resolve through root
re-exports with their original visibilities (pub stays pub, pub(crate)
stays pub(crate)). Cross-module items gain pub(super), whose scope
equals the old single-module privacy domain. Code is moved verbatim
apart from those markers, per-module import headers, and rustfmt
re-wraps.
The logging-guardrail nsscanner_disk skip-set_disks rule now points at
scanner_io/io_disk.rs where the function moved; the pattern and
thresholds are unchanged.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(scanner): split data_usage_define persistence and tests (#6292)
Split the 3655-line data_usage_define.rs (59% inline tests) into a
canonical foo.rs + foo/ module tree with zero behavior change:
- data_usage_define.rs (~950): cache constants and revision helpers,
the data-usage tree types, DataUsageCacheInfo with its hand-written
Serialize, the in-memory tree operations, dui, and marshal/unmarshal
- data_usage_define/persistence.rs (~580): the load/backup/restore
ladder (load, try_load_inner, revision_for_path) and the CAS save
path with its retry policy and save metrics
- data_usage_define/tests.rs (~2155): the inline test module as a child
module
All module paths are unchanged (the lib.rs data_usage_define::* glob
re-export and every crate::data_usage_define:: consumer resolve as
before). The hand-written map-encoded Serialize for
DataUsageCacheInfo is moved byte-for-byte per the AGENTS.md
cross-cutting invariant; on-disk names and the cache key format const
stay in the root. Four persistence helpers used by tests gain
pub(super), whose scope equals the old single-module privacy domain.
Code is moved verbatim apart from those markers, per-module import
headers, and rustfmt re-wraps.
Co-authored-by: heihutu <heihutu@gmail.com>
* chore(deps): bump datafusion to 55.0.0 (#6288)
* refactor(heal): split task.rs per heal kind (#6293)
* feat(ecstore): batch small file fdatasync commits (#6297)
* feat(ecstore): batch small file fdatasync commits
Add a default-off experimental file fdatasync group commit path for small rename_data shard directories. The coordinator batches same-disk waiters into one blocking task while preserving per-directory source fsync after shard contents are durable.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(e2e): wait for compression S3 readiness
Reuse the shared S3 API readiness probe for compression test servers so multipart requests do not race the startup readiness gate after the TCP port opens.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(tier): recover multi-committed mutation intents (#6296)
* fix(tier): recover multi-committed mutation intents
* fix(tier): recover committed mutations on standalone nodes
* test(scanner): avoid stack overflow in transition test
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
---------
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Add PUT-stage diagnostics for file fdatasync group commit wait time, per-group outstanding depth, and rename disk completion position. These metrics keep the existing default-off PUT stage gate and do not change group commit scheduling or quorum behavior.
Co-authored-by: heihutu <heihutu@gmail.com>
Zhengchao An