The batch `NewerNoncurrentVersions` expiry path took a lifecycle event
argument and ignored it: after `delete_objects` committed it only evicted
the cache and scheduled replication deletes, so a successful noncurrent
version expiry was invisible to notification subscribers while the
equivalent current-version path emitted a lifecycle expiration event.
Emit that event from the batch path too, reusing the existing lifecycle
audit sink and event contract. Only entries that actually mutated
something are announced, and cache eviction and replication scheduling
keep their existing order and admission — the event is derived from the
committed result and a send failure never rolls back a delete.
"No error" is not enough to prove a mutation: the disk layer skips an
absent version and reports success, so a batch entry for a version that
was already gone came back indistinguishable from a committed delete.
The delete plan already resolves whether the source exists, so carry that
`source_missing` result on `DeletedObject` and let the lifecycle path
stay silent for versions it did not remove.
backlog#2202
* fix(ilm): enqueue committed tier free versions
* fix(ilm): stabilize causal cleanup CI coverage
* test(ilm): make expire GET race deterministic
* test(ilm): synchronize expiry with active GET
Six set_disk::ops tests failed non-deterministically only under
concurrent full-suite load, rotating between runs while each passed in
isolation. All six share one root cause: a lock-owning put_object
quorum-acks once the rename fanout reaches write quorum and lets a
detached tail task finish the lagging disks, so a fixture that inspects
per-disk state immediately after PUT can observe a disk the tail has not
reached yet.
The two heal report fixtures, the inline-commit fixture, and the
transaction-fencing fixture read or delete physical shards right after
PUT, and hit FileNotFound on a lagging disk. The two metadata-cache
fixtures prime the cache after PUT, and the read fanout refuses to publish
a cache entry while any disk still reports an error, so the priming read
observably published nothing.
Keep every affected setup PUT on the full-fanout commit path with
no_lock: true, following the existing precedent in this module, so PUT
returns only after every disk has committed. The option only governs lock
acquisition, so it does not weaken what any of these fixtures assert; the
transaction-fencing gate in particular is driven by the fleet proof and
env vars, never by the lock option. Where a fixture also depends on cache
publication, re-prime until the current generation is observably cached
instead of asserting on a single read that a loaded host can stall past
the cache TTL. The heal race fixture's shard damage injection is
best-effort by construction, so it now skips injection when the previous
round's tail still lags rather than unwrapping a read that may
legitimately race.
No production code changes, and no retries or sleeps added.
Also replace deprecated Atomic::fetch_update calls with try_update so the
current Rust toolchain keeps lint and CI jobs warning-clean.
Co-authored-by: heihutu <heihutu@gmail.com>
A PUT with HTTP preconditions took the per-object namespace write lock
before ingesting the request body and held it until commit, so any
concurrent read of the same object queued behind client-paced body
ingestion until the 5s acquire timeout and surfaced as 503. Exposed as
a deterministic S3 Implemented Tests gate failure when #6770 routed
1 MB conditional writes onto the streaming path (rustfs/backlog#2074).
Keep a lock-free advisory precondition check before the body for fast
412/404, and evaluate the authoritative check under the put_object
commit lock, reusing the deferred shape data movement already uses.
Reads during ingestion now return the last committed version, and a
precondition invalidated mid-stream fails closed with 412 at commit.
* fix(s3): round-trip null-version delete-marker identity through listing and delete responses
On a versioning-suspended bucket, a null delete marker's identity was lost on the way back to the client at three points (issue #6745): ListObjectVersions advertised the marker's VersionId as the literal nil UUID instead of null; deleting by that id succeeded but the DeleteObjects/DeleteObject response reported the identity as null with no way to correlate it to the request; and the response lacked DeleteMarker/DeleteMarkerVersionId because the marker-ness comparison mixed the client-facing identity (Some(nil)) with the storage identity (None), so the removal also mis-recorded accounting and fired DeleteMarkerCreated semantics on later paths.
- Listing (bucket_usecase, s3_api/bucket, build_list_versions_next_marker) now maps the synthesized nil UUID to the literal null everywhere it reaches the wire, and VersionMarker::parse folds a nil-UUID marker from older listings into VersionMarker::Null so pagination resumes correctly.
- delete_objects normalizes both sides of the marker-ness comparison via delete_file_info_version_id (matching the adjacent explicit_delete_marker admission check) and reports DeleteMarkerVersionId as null for an explicit null-marker removal.
- resolve_delete_version_state reports delete_marker for an explicit-version delete whose target is a delete marker even when the bucket is versioning-suspended, fixing x-amz-delete-marker on the single-object path.
- The DeleteObjects response entry echoes the version identity the request addressed for marker removals, marker-removal accounting no longer records a marker creation, and notification events fire DeleteMarkerCreated only for actual marker creation.
Fixes#6745
* fix(s3): keep null-marker removal write shape undeleted and report marker semantics response-side
The first cut marked the storage delete request deleted for a null-marker removal, which FileMeta::delete_version interprets as the suspended-bucket delete-mints-a-marker write and re-creates the marker just removed. Carry marker-ness to responses via explicit_delete_removed_marker (single path) and a response-only branch flag (batch path) instead, keeping every storage write shape byte-identical to the pre-fix behavior. Adds an embedded end-to-end regression test covering the full issue #6745 round trip.
* Revert "perf(ecstore): use AHashMap for FileInfo metadata fields (#6738)"
This reverts commit 13a2ae212e.
* fix(filemeta): restore standard HashMap metadata (#6742)
Remove the direct ahash dependency added for FileInfo metadata and revert the affected filemeta/ecstore call sites back to std::collections::HashMap.
Co-authored-by: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
refactor(ecstore): retire the set_disk lint blankets by making the prelude explicit
backlog#1823 step 1 / backlog#2029 road 2. Removes the last two module-level lint blankets in ecstore: set_disk/mod.rs #![allow(unused_imports)] and #![allow(unused_variables)], restoring both lints for the whole 40K-line subtree, and deletes the register line for the unused_variables blanket in the same diff (the guard from #6155 is a bidirectional exact match).
The unused_imports blanket existed because 14 submodules consumed mod.rs as a glob prelude (use super::* / use super::super::*), and rustc does not track consumption through glob re-exports. Each glob is now an explicit use super::{...} list, keeping mod.rs as the single import hub while making every import lint-checkable. Names consumed only by test or test-util units carry #[cfg(test)] / #[cfg(all(test, feature = "test-util"))] / #[cfg(any(test, feature = "test-util"))] gates matching their consumers; storage-api traits are routed through the storage_api_contracts facade per the architecture guard.
The sweep then deleted the genuinely dead imports the blanket was hiding (chrono::Utc, glob::Pattern, futures::task::AtomicWaker, rustfs_lock LocalLock, AsyncBatchProcessor, rand::Rng, std::future::Future among others in mod.rs, plus stale scoped imports and one empty test module shell across the subtree). One unused_variables finding surfaced: flush_read_version_coalescer_pending's lane_key is read only by the #[cfg(test)] counter block, handled with the cfg(not(test)) let _ pattern established in #6158.
Verification: cargo check zero warnings versus the 9cf276ed2 baseline on five lanes (default lib / --tests / rio-v2 --tests / test-util --tests / test-util,rio-v2 --tests; the --tests lane keeps the same three pre-existing core/pools.rs and store/object.rs dead-code warnings main already has); clippy --lib --tests -D warnings clean with test-util,rio-v2; cargo nextest run 4567 passed; make pre-commit exit 0.
The storage engine embedded a ~8.4K-line hand-written S3 HTTP client under crates/ecstore/src/client (rustfs/backlog#1842). That client is a legitimate engine capability — it consumes remote S3-compatible endpoints for ILM tier warm backends and transition targets — but it was misfiled inside the engine, dragging s3s/hyper wire types into ecstore and blocking ARCHITECTURE.md invariant 4.
This PR is the pure-move step: 21 modules move verbatim to the new crates/s3-client crate (rustfs-s3-client), and crates/ecstore/src/client/mod.rs becomes a re-export shim so every in-crate crate::client:: path keeps working. The two server-side modules that were historically misfiled under client/ — object_api_utils.rs and object_handlers_common.rs — stay in ecstore.
Three reverse dependencies from the client into engine internals are severed so the move can be pure:
- transition_api::ReaderImpl::ObjectBody held ecstore's GetObjectReader; the client only ever reads the body, so the variant now holds an ObjectReader newtype over Box<dyn AsyncRead + Send + Sync + Unpin> with the same read_all() surface. The single production construction site (set_disk transition upload) and the two engine-side consumers were adjusted.
- api_list/api_remove used ecstore's storage_api_contracts / object_api types; api_list now imports BucketInfo from rustfs-storage-api directly, and api_remove uses the client's own transition_api::ObjectInfo (only .name/.version_id were read; the error-path bucket name is now threaded as a parameter instead of read from the deleted objects).
- the api_put_object_streaming regression tests built a GetObjectReader by hand; they now wrap the duplex stream in ObjectReader::new.
Guard updates: the s3s footprint ratchet gains an ecstore-scoped counter (42 files, shrink-only, per rustfs/backlog#1842), the ecstore module-lint-blanket register follows the moved files into crates/s3-client so the blanket ratchet keeps covering them, the logging guardrail path pin follows transition_api.rs, and the ::other(format!) baseline is regenerated (moved call sites left ecstore).
Verification: cargo check -p rustfs-s3-client -p rustfs-ecstore; cargo nextest run -p rustfs-s3-client (43 passed) and -p rustfs-ecstore (4515/4523; the 8 failures reproduce identically on pristine origin/main on the same machine); cargo clippy --all-targets; scripts/check_layer_dependencies.sh, check_architecture_migration_rules.sh, check_s3s_footprint.sh, check_logging_guardrails.sh, check_error_other_format_ratchet.sh, check_doc_paths.sh, check_ci_paths_sync.sh all pass.
refactor(ecstore): make store-to-disk error narrowing a named fallible operation
Backlog#1845 step 4. The blanket impl From<StorageError> for DiskError let ? silently push store-only errors (locks, buckets, quotas) across the disk boundary into DiskError::other, where the rendered message fragments reduce_errs quorum buckets. Same story for the blanket From<StorageError> for rustfs_filemeta::Error and its other() catch-all.
Both impls are replaced by named, fallible methods: StorageError::narrow_to_disk() and StorageError::narrow_to_filemeta(). Variants with an identity on the far side map across unchanged - including the two documented lossy collapses (SlowDown -> TooManyOpenFiles, StorageFull -> DiskFull) that the round-trip tests pin - and everything else returns Err(self) so the call site decides what crossing the boundary means. Removing the impls let the compiler enumerate every conversion site; the census that scoped this issue had found 5, the compiler found 33.
Call sites keep their existing behavior: the io identity bridge and the generic sites fold Err into the io-backed other() exactly as the old catch-all did (identity still recoverable by downcast), listing paths use one shared to_filemeta_err helper, and the two sites that relied on the SlowDown collapse now construct DiskError::TooManyOpenFiles directly so the loss is visible where it happens. No behavior change intended anywhere; the io::Error bridge itself is untouched by design.
Ref rustfs/backlog#1845
Single-part encrypted objects in the legacy format could not serve range
reads without decrypting from byte 0: v1 frames are emitted per upstream
read, so no closed-form plaintext-to-ciphertext mapping exists. The v2
layout fixed the frame length (8218 ciphertext bytes per 8 KiB plaintext
frame), making the mapping closed-form.
Consume it:
- Single-part PUTs that encrypt locally under the v2 write switch stamp
the frame-layout marker, valued with the object's data_dir token -
ciphertext passthrough, data movement and copies mint a new data_dir
or strip the marker, so a re-homed marker disqualifies itself.
- The encrypted read plan seeks marked, uncompressed single-part objects
to frame_index * 8218 and decrypts from that frame: the frame index
rides the plan's sequence-number slot into DecryptReader::new_at_block,
whose nonce and AAD bind absolute indices. New metric path label
frame_seek.
- A lying marker fails closed: v2 authentication rejects bytes at a fake
frame boundary; plaintext is never served from the wrong offset.
Compressed objects and multipart sub-part seeks keep the conservative
paths (follow-up work); reading needs no switch - seekability follows
the marker.
* feat(mimalloc): add arena diagnostics and configuration
Based on mimalloc maintainer feedback (microsoft/mimalloc#1372),
add diagnostics to check mimalloc arena configuration at runtime.
Changes:
- Add rustfs-mimalloc-sys to workspace dependencies
- Add log_mimalloc_diagnostics() function to check:
- arena_max_object_size
- pagemap_commit status
- mimalloc version
- Add memory_observability module with mimalloc diagnostics
This helps diagnose why allocations might be going outside arenas,
which is the suspected root cause of futex contention.
Ref: rustfs/backlog#2005
Ref: microsoft/mimalloc#1372
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): add Vec<u8> buffer pool for EC operations
Add a general-purpose buffer pool to reduce Vec<u8> allocations
in hot paths like EC encoding/decoding.
Changes:
- Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs
- Thread-safe pool with capacity-based bucketing (power-of-two)
- Global EC_BUFFER_POOL instance with 16 buffers per bucket
- Add buffer_pool module to codec/mod.rs
Expected impact:
- Reduce heap allocations in EC encode/decode paths
- Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool)
- Reduce mimalloc lock contention
Note: Main bottleneck remains mimalloc internal synchronization
(futex 98.64% time). Buffer pool provides modest improvement (+2-5%).
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and related files
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): add #[allow(dead_code)] to buffer pool
The BufferPool infrastructure is ready but not yet integrated
into the EC hot paths. Add #[allow(dead_code)] with clear
documentation about integration status.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): integrate BufferPool into bitrot verify path
Replace vec![0; shard_size] with get_ec_buffer() in the bitrot
verification hot path to reduce heap allocations and avoid memzero.
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and bitrot changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* refactor(ecstore): clean up buffer pool code
- Remove unnecessary #[allow(dead_code)] attributes
- Update module documentation to reflect current integration status
- Simplify code structure
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(runtime): cap default worker threads at 16
Testing showed 16 worker threads outperforms 32+ for 1KiB PUT
workloads due to reduced mimalloc lock contention.
A/B test results (testing 4-node cluster, c=64):
- worker_threads=32: 740 obj/s (baseline)
- worker_threads=16: 785 obj/s (+6.1%)
The default was detect_cores() which returned 32 on our testing
nodes. Cap at 16 for optimal small-object performance.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and runtime changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): remove unused BufferPool::new() function
The new() function was never used since EC_BUFFER_POOL
initializes directly with with_limits(16).
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): update buffer_pool tests to use with_limits
Replace BufferPool::new() with BufferPool::with_limits(16) in tests
since new() was removed in favor of with_limits().
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): optimize opts.clone() and FileInfo clone patterns
## Changes
1. ObjectOptions helper methods:
- add as_commit_opts(): creates commit options with no_lock=true,
metadata_cache_safe=false, include_part_checksums=true
- add as_read_opts(): creates read options with
include_part_checksums=true
- add with_no_lock(): creates options with modified no_lock field
2. Replace opts.clone() in hot paths:
- commit_opts = opts.as_commit_opts() (was 4-line manual clone)
- read_opts = opts.as_read_opts() (was 2-line manual clone)
3. Optimize FileInfo clone in rename path:
- avoid double clone: clone once and modify erasure.index in place
- pass &file_info reference to rename_data_borrowed_with_fence
## A/B Results (4-node cluster, c=64)
| Size | main | optimized | Change |
|------|------|-----------|--------|
| 1KiB | 892 obj/s | 920-976 obj/s | +3%~+9% |
| 4KiB | 957 obj/s | 903 obj/s | -5.7% |
| 16KiB | 922 obj/s | 855 obj/s | -7.3% |
Note: 1KiB improvement is consistent. 4KiB/16KiB variance
likely due to test noise; needs more rounds to confirm.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): add BytesMut buffer pool to EC encoding path
Pre-allocate a Vec<BytesMut> pool in the EC encoding loop to avoid
repeated heap allocations for ingest buffers.
Changes:
- Pre-allocate buffer pool with capacity 4
- Reuse buffers from pool after encoding
- Return buffers to pool when capacity is sufficient
Expected impact: +10-20% in EC encoding path by reducing
BytesMut allocation overhead.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: hector <hetor@rustfs.com>
Co-authored-by: heihutu <heihutu@gmail.com>
* feat(mimalloc): add arena diagnostics and configuration
Based on mimalloc maintainer feedback (microsoft/mimalloc#1372),
add diagnostics to check mimalloc arena configuration at runtime.
Changes:
- Add rustfs-mimalloc-sys to workspace dependencies
- Add log_mimalloc_diagnostics() function to check:
- arena_max_object_size
- pagemap_commit status
- mimalloc version
- Add memory_observability module with mimalloc diagnostics
This helps diagnose why allocations might be going outside arenas,
which is the suspected root cause of futex contention.
Ref: rustfs/backlog#2005
Ref: microsoft/mimalloc#1372
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): add Vec<u8> buffer pool for EC operations
Add a general-purpose buffer pool to reduce Vec<u8> allocations
in hot paths like EC encoding/decoding.
Changes:
- Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs
- Thread-safe pool with capacity-based bucketing (power-of-two)
- Global EC_BUFFER_POOL instance with 16 buffers per bucket
- Add buffer_pool module to codec/mod.rs
Expected impact:
- Reduce heap allocations in EC encode/decode paths
- Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool)
- Reduce mimalloc lock contention
Note: Main bottleneck remains mimalloc internal synchronization
(futex 98.64% time). Buffer pool provides modest improvement (+2-5%).
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and related files
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): add #[allow(dead_code)] to buffer pool
The BufferPool infrastructure is ready but not yet integrated
into the EC hot paths. Add #[allow(dead_code)] with clear
documentation about integration status.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): integrate BufferPool into bitrot verify path
Replace vec![0; shard_size] with get_ec_buffer() in the bitrot
verification hot path to reduce heap allocations and avoid memzero.
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and bitrot changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* refactor(ecstore): clean up buffer pool code
- Remove unnecessary #[allow(dead_code)] attributes
- Update module documentation to reflect current integration status
- Simplify code structure
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(runtime): cap default worker threads at 16
Testing showed 16 worker threads outperforms 32+ for 1KiB PUT
workloads due to reduced mimalloc lock contention.
A/B test results (testing 4-node cluster, c=64):
- worker_threads=32: 740 obj/s (baseline)
- worker_threads=16: 785 obj/s (+6.1%)
The default was detect_cores() which returned 32 on our testing
nodes. Cap at 16 for optimal small-object performance.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <heihutu@gmail.com>
* style: apply cargo fmt to buffer pool and runtime changes
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): remove unused BufferPool::new() function
The new() function was never used since EC_BUFFER_POOL
initializes directly with with_limits(16).
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): update buffer_pool tests to use with_limits
Replace BufferPool::new() with BufferPool::with_limits(16) in tests
since new() was removed in favor of with_limits().
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: hector <hetor@rustfs.com>
Co-authored-by: heihutu <heihutu@gmail.com>
The inline_block threshold used floor division (DEFAULT_INLINE_OBJECT_BUDGET
/ data_shards) while shard_file_size uses ceiling division (div_ceil). For
EC 12:4 with 256KiB objects, this caused a 1-byte discrepancy:
- inline_block = 262144 / 12 = 21845 (floor)
- shard_file_size = 262144.div_ceil(12) = 21846 (ceil)
- should_inline(21846, 12, false) = false (wrong!)
Fix by using div_ceil for the inline_block calculation, so both sides
use the same rounding and the inline path is correctly triggered.
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(lifecycle): safely expire all object versions
* fix(lifecycle): preserve delete-all replication purges
* fix(lifecycle): remove dead replication journal
* fix(ci): avoid lifecycle transition test stack overflow
* fix(lifecycle): release recovery locks before tier IO
* test(lifecycle): align object-lock error assertions
* test(lifecycle): avoid scanner restore stack overflow
* test(scanner): avoid stack overflow in transition and restore flow test (#6300)
* refactor(scanner): split remote_scanner.rs into stream child module (#6289)
Split the 3080-line remote_scanner.rs (47% inline tests) into a
canonical foo.rs + foo/ module tree with zero behavior change:
- remote_scanner.rs (~320): protocol constants, process statics, and
the request decode/validate/admit/preflight/claim API plus root
re-exports
- remote_scanner/stream.rs (~1340): wire/frame types, replay cache,
FrameAuthenticator, serve path, local bucket scan + persist, client
scan, and the bounded stream plumbing
- remote_scanner/stream/tests.rs (~1470): the inline test module as a
child module of stream so it can reach both parents' private items
All crate paths are unchanged: lib.rs re-exports
(serve_remote_scanner_request, RemoteScannerRequest, ...) resolve
through root re-exports, and scanner_io's crate::remote_scanner::
{scan_remote_bucket, RemoteScannerScanSpec, RemoteScannerOutcome}
paths resolve through pub(crate) re-exports. Cross-module items gain
pub(super), whose scope equals the old single-module privacy domain;
no item's effective visibility widens. Code is moved verbatim apart
from those markers, per-module import headers, and rustfmt line
re-wraps.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(heal): split resume.rs into focused child modules (#6290)
Split the 4242-line resume.rs (46% inline tests) into a canonical
foo.rs + foo/ module tree with zero behavior change:
- resume.rs (~1020): state file constants, PersistThrottle, ResumeState,
ResumeManager core (constructors, load/discovery, progress mutators,
ordinary persistence) plus root re-exports
- resume/replacement.rs (~690): replacement-intent/proof types and the
ResumeManager replacement-lifecycle methods
- resume/checkpoint.rs (~350): ResumeCheckpoint + CheckpointManager
- resume/utils.rs (~310): ResumeUtils statics
- resume/tests.rs (~1980): the inline test module as a child module
All module paths are unchanged (heal::resume::CheckpointManager and
friends resolve through root re-exports), so no consumer inside or
outside the crate changes. Items defined in child modules keep
module-private visibility; only the ten cross-module helpers gain
pub(super), which is not part of the crate API. Code is moved verbatim
apart from those visibility markers, four super::storage_api path
fixes, and the new per-module import headers.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(scanner): split scanner_io.rs into child modules (#6294)
Split the 5369-line scanner_io.rs (39% inline tests) into a canonical
scanner_io.rs + scanner_io/ module tree with zero behavior change:
- scanner_io.rs (~660): constants, metadata-error constructors, the
bucket scan plan, cycle-status classification helpers, the ScannerIO /
ScannerIOCache / ScannerIODisk traits, and ScannerCycleResult
- scanner_io/dirty_usage.rs (~300): process-wide dirty-usage statics
and the acknowledgment protocol
- scanner_io/guards.rs (~270): concurrency gauges and RAII guards
- scanner_io/cache.rs (~410): scanner cache locks and the snapshot
persist/publish path
- scanner_io/io_cycle.rs (~390), io_cache.rs (~1160), io_disk.rs
(~230): the ECStore / SetDisks / Disk trait implementations
- scanner_io/publish_gate_tests.rs (~750) and tests.rs (~1340): the two
inline test modules as child modules
All crate paths are unchanged: the lib.rs scanner_io re-exports and
every crate::scanner_io:: consumer (scanner.rs, remote_scanner,
scanner_folder, and cross-crate rustfs users) resolve through root
re-exports with their original visibilities (pub stays pub, pub(crate)
stays pub(crate)). Cross-module items gain pub(super), whose scope
equals the old single-module privacy domain. Code is moved verbatim
apart from those markers, per-module import headers, and rustfmt
re-wraps.
The logging-guardrail nsscanner_disk skip-set_disks rule now points at
scanner_io/io_disk.rs where the function moved; the pattern and
thresholds are unchanged.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(scanner): split data_usage_define persistence and tests (#6292)
Split the 3655-line data_usage_define.rs (59% inline tests) into a
canonical foo.rs + foo/ module tree with zero behavior change:
- data_usage_define.rs (~950): cache constants and revision helpers,
the data-usage tree types, DataUsageCacheInfo with its hand-written
Serialize, the in-memory tree operations, dui, and marshal/unmarshal
- data_usage_define/persistence.rs (~580): the load/backup/restore
ladder (load, try_load_inner, revision_for_path) and the CAS save
path with its retry policy and save metrics
- data_usage_define/tests.rs (~2155): the inline test module as a child
module
All module paths are unchanged (the lib.rs data_usage_define::* glob
re-export and every crate::data_usage_define:: consumer resolve as
before). The hand-written map-encoded Serialize for
DataUsageCacheInfo is moved byte-for-byte per the AGENTS.md
cross-cutting invariant; on-disk names and the cache key format const
stay in the root. Four persistence helpers used by tests gain
pub(super), whose scope equals the old single-module privacy domain.
Code is moved verbatim apart from those markers, per-module import
headers, and rustfmt re-wraps.
Co-authored-by: heihutu <heihutu@gmail.com>
* chore(deps): bump datafusion to 55.0.0 (#6288)
* refactor(heal): split task.rs per heal kind (#6293)
* feat(ecstore): batch small file fdatasync commits (#6297)
* feat(ecstore): batch small file fdatasync commits
Add a default-off experimental file fdatasync group commit path for small rename_data shard directories. The coordinator batches same-disk waiters into one blocking task while preserving per-directory source fsync after shard contents are durable.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(e2e): wait for compression S3 readiness
Reuse the shared S3 API readiness probe for compression test servers so multipart requests do not race the startup readiness gate after the TCP port opens.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(tier): recover multi-committed mutation intents (#6296)
* fix(tier): recover multi-committed mutation intents
* fix(tier): recover committed mutations on standalone nodes
* test(scanner): avoid stack overflow in transition test
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
---------
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
* feat(common): add MRF intent channel and Mrf request source (HS-01)
Introduce the producer-facing half of the mission repair feed: a global
bounded (8192) channel carrying lightweight MrfIntent values from IO
error paths, plus the RUSTFS_HEAL_MRF_ENABLE delivery kill-switch and
config constants for queue/journal sizing. Delivery is strictly
non-blocking (try_send, drop-on-full) so it can sit on decode-failure
and partial-write paths without adding latency. HealRequestSource grows
a 'mrf' variant so admission accounting can attribute replayed intents.
Part of backlog#1865 (option a: wire HealEvent-style intents with a
durable retry ledger).
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(heal): add MRF queue, durable journal, and intent consumer (HS-01)
Consumer half of the mission repair feed: a bounded pending queue
(100k intents / 8 MiB dual ceiling, drop-newest on overflow), a durable
journal at buckets/.heal/mrf/journal.bin holding the unaccepted pending
snapshot, and a consumer task that batches intents off the global
channel, translates them into prioritized heal requests (decode
failure -> Urgent ECDecode, metadata corruption -> High Metadata,
partial write -> Normal object heal), and retries full admissions with
a 5s backoff and a 3-attempt ceiling.
Durability: every journal record carries its own CRC32 and a
format/version header, so a torn tail truncates cleanly at replay; the
journal is deleted after a successful replay and when the pending set
drains (mirroring MinIO's post-replay list.bin unlink). Losing the last
500 ms flush window is acceptable: replayed duplicates merge via the
manager dedup key and read-repair remains the safety net.
Metrics: rustfs_heal_mrf_queue_depth/_queue_bytes, _dropped_total
{reason}, _replayed_total, _journal_bytes, _journal_fsync_total.
The consumer is wired at heal runtime bootstrap right after manager
start, honoring RUSTFS_HEAL_MRF_ENABLE (default on, rollback = off).
Tests: unit tests for the dual ceiling, record roundtrip, torn-tail
truncation, and the priority mapping; integration tests against a real
4-disk ECStore proving channel intents reach the manager queue as
Urgent/mrf-attributed requests and journal replay arms intents, drops
torn tails, and removes the file.
Part of backlog#1865 (option a).
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(ecstore,scanner): deliver MRF intents from error paths (HS-01)
Wire the three production delivery points, each a single non-blocking
try_send next to the existing in-memory heal paths, which stay as the
fast path:
- read.rs decode-error branch: DecodeFailure intent beside the existing
read-repair submit, so an Urgent ECDecode request survives restarts
even when the Low-priority read-repair request was dropped or lost.
- add_partial: PartialWrite intent, giving partial-write recovery a
durable Normal-priority object heal across restarts.
- scanner_folder metadata-corruption classification: MetadataCorruption
intent beside the existing High-priority scanner heal request.
All three are on error paths only: zero cost on healthy IO.
Part of backlog#1865 (option a).
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix: include mrf heal source counts
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix: keep node heal status wire compatibility
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Zhengchao An