Add deterministic rename_data coverage for tail-disk success/failure, cancellation serialization, and strict quorum rollback visibility after disk reopen. This establishes the safety boundary before experimenting with write-quorum early ACK and background tail completion for backlog #925.
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(lifecycle): safely expire all object versions
* fix(lifecycle): preserve delete-all replication purges
* fix(lifecycle): remove dead replication journal
* fix(ci): avoid lifecycle transition test stack overflow
* fix(lifecycle): release recovery locks before tier IO
* test(lifecycle): align object-lock error assertions
* test(lifecycle): avoid scanner restore stack overflow
* test(scanner): avoid stack overflow in transition and restore flow test (#6300)
* refactor(scanner): split remote_scanner.rs into stream child module (#6289)
Split the 3080-line remote_scanner.rs (47% inline tests) into a
canonical foo.rs + foo/ module tree with zero behavior change:
- remote_scanner.rs (~320): protocol constants, process statics, and
the request decode/validate/admit/preflight/claim API plus root
re-exports
- remote_scanner/stream.rs (~1340): wire/frame types, replay cache,
FrameAuthenticator, serve path, local bucket scan + persist, client
scan, and the bounded stream plumbing
- remote_scanner/stream/tests.rs (~1470): the inline test module as a
child module of stream so it can reach both parents' private items
All crate paths are unchanged: lib.rs re-exports
(serve_remote_scanner_request, RemoteScannerRequest, ...) resolve
through root re-exports, and scanner_io's crate::remote_scanner::
{scan_remote_bucket, RemoteScannerScanSpec, RemoteScannerOutcome}
paths resolve through pub(crate) re-exports. Cross-module items gain
pub(super), whose scope equals the old single-module privacy domain;
no item's effective visibility widens. Code is moved verbatim apart
from those markers, per-module import headers, and rustfmt line
re-wraps.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(heal): split resume.rs into focused child modules (#6290)
Split the 4242-line resume.rs (46% inline tests) into a canonical
foo.rs + foo/ module tree with zero behavior change:
- resume.rs (~1020): state file constants, PersistThrottle, ResumeState,
ResumeManager core (constructors, load/discovery, progress mutators,
ordinary persistence) plus root re-exports
- resume/replacement.rs (~690): replacement-intent/proof types and the
ResumeManager replacement-lifecycle methods
- resume/checkpoint.rs (~350): ResumeCheckpoint + CheckpointManager
- resume/utils.rs (~310): ResumeUtils statics
- resume/tests.rs (~1980): the inline test module as a child module
All module paths are unchanged (heal::resume::CheckpointManager and
friends resolve through root re-exports), so no consumer inside or
outside the crate changes. Items defined in child modules keep
module-private visibility; only the ten cross-module helpers gain
pub(super), which is not part of the crate API. Code is moved verbatim
apart from those visibility markers, four super::storage_api path
fixes, and the new per-module import headers.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(scanner): split scanner_io.rs into child modules (#6294)
Split the 5369-line scanner_io.rs (39% inline tests) into a canonical
scanner_io.rs + scanner_io/ module tree with zero behavior change:
- scanner_io.rs (~660): constants, metadata-error constructors, the
bucket scan plan, cycle-status classification helpers, the ScannerIO /
ScannerIOCache / ScannerIODisk traits, and ScannerCycleResult
- scanner_io/dirty_usage.rs (~300): process-wide dirty-usage statics
and the acknowledgment protocol
- scanner_io/guards.rs (~270): concurrency gauges and RAII guards
- scanner_io/cache.rs (~410): scanner cache locks and the snapshot
persist/publish path
- scanner_io/io_cycle.rs (~390), io_cache.rs (~1160), io_disk.rs
(~230): the ECStore / SetDisks / Disk trait implementations
- scanner_io/publish_gate_tests.rs (~750) and tests.rs (~1340): the two
inline test modules as child modules
All crate paths are unchanged: the lib.rs scanner_io re-exports and
every crate::scanner_io:: consumer (scanner.rs, remote_scanner,
scanner_folder, and cross-crate rustfs users) resolve through root
re-exports with their original visibilities (pub stays pub, pub(crate)
stays pub(crate)). Cross-module items gain pub(super), whose scope
equals the old single-module privacy domain. Code is moved verbatim
apart from those markers, per-module import headers, and rustfmt
re-wraps.
The logging-guardrail nsscanner_disk skip-set_disks rule now points at
scanner_io/io_disk.rs where the function moved; the pattern and
thresholds are unchanged.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(scanner): split data_usage_define persistence and tests (#6292)
Split the 3655-line data_usage_define.rs (59% inline tests) into a
canonical foo.rs + foo/ module tree with zero behavior change:
- data_usage_define.rs (~950): cache constants and revision helpers,
the data-usage tree types, DataUsageCacheInfo with its hand-written
Serialize, the in-memory tree operations, dui, and marshal/unmarshal
- data_usage_define/persistence.rs (~580): the load/backup/restore
ladder (load, try_load_inner, revision_for_path) and the CAS save
path with its retry policy and save metrics
- data_usage_define/tests.rs (~2155): the inline test module as a child
module
All module paths are unchanged (the lib.rs data_usage_define::* glob
re-export and every crate::data_usage_define:: consumer resolve as
before). The hand-written map-encoded Serialize for
DataUsageCacheInfo is moved byte-for-byte per the AGENTS.md
cross-cutting invariant; on-disk names and the cache key format const
stay in the root. Four persistence helpers used by tests gain
pub(super), whose scope equals the old single-module privacy domain.
Code is moved verbatim apart from those markers, per-module import
headers, and rustfmt re-wraps.
Co-authored-by: heihutu <heihutu@gmail.com>
* chore(deps): bump datafusion to 55.0.0 (#6288)
* refactor(heal): split task.rs per heal kind (#6293)
* feat(ecstore): batch small file fdatasync commits (#6297)
* feat(ecstore): batch small file fdatasync commits
Add a default-off experimental file fdatasync group commit path for small rename_data shard directories. The coordinator batches same-disk waiters into one blocking task while preserving per-directory source fsync after shard contents are durable.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(e2e): wait for compression S3 readiness
Reuse the shared S3 API readiness probe for compression test servers so multipart requests do not race the startup readiness gate after the TCP port opens.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(tier): recover multi-committed mutation intents (#6296)
* fix(tier): recover multi-committed mutation intents
* fix(tier): recover committed mutations on standalone nodes
* test(scanner): avoid stack overflow in transition test
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
---------
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Add PUT-stage diagnostics for file fdatasync group commit wait time, per-group outstanding depth, and rename disk completion position. These metrics keep the existing default-off PUT stage gate and do not change group commit scheduling or quorum behavior.
Co-authored-by: heihutu <heihutu@gmail.com>
* feat(ecstore): batch small file fdatasync commits
Add a default-off experimental file fdatasync group commit path for small rename_data shard directories. The coordinator batches same-disk waiters into one blocking task while preserving per-directory source fsync after shard contents are durable.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(e2e): wait for compression S3 readiness
Reuse the shared S3 API readiness probe for compression test servers so multipart requests do not race the startup readiness gate after the TCP port opens.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Two producer paths double-booked the same damage across repair records
(backlog#1894 axis A):
- The scanner's corrupt-metadata branch fired a durable MRF journal
intent, an immediate High heal request, and a pending-ledger entry for
the same object. When the MRF intent is accepted into the channel it
already covers the repair durably (the consumer files a High Metadata
heal and the journal replays it across restarts), so the immediate
request and ledger entry are dropped in that case; on delivery failure
(feature disabled, channel uninitialized, or full) the old immediate
request + ledger path runs unchanged, keeping the repair safety net.
- The read path filed a journal intent before the read-repair
reservation check, so a burst of reads failing on one object booked a
journal record per retry. The intent now rides the submission: it is
filed only when the sighting wins the dedup TTL, next to the Low
request, via a new optional mrf_intent field on
ReadRepairHealSubmission (None keeps the historical no-intent
behavior for the other read-repair call sites).
Manager dedup-key semantics are untouched; the fix is that competing
producers stop double-booking. With RUSTFS_HEAL_MRF_ENABLE off both
paths behave exactly as before.
Co-authored-by: heihutu <heihutu@gmail.com>
T2 of backlog#1827. `data_movement_stage_error` flattened every stage failure into `Error::other(format!(...))`, discarding the typed error. The cost was visible in tree: `is_decommission_target_capacity_error` had to match rendered text —
let message = err.to_string();
message.contains(&disk_full) || message.contains(&storage_full)
— to notice that the destination pool had filled up, and `is_decommission_copy_cleanup_safe_error` could not see a not-found that surfaced from inside a stage at all.
The wrapper now carries what it wrapped. `DataMovementStageError` renders the same string and returns the original through `source()`; `Error::other` boxes it through `std::io::Error`, so `data_movement_stage_source` recovers it by downcast. Both classifiers unwrap before matching, keeping their substring paths for errors that arrive through some other wrapper.
The rendered message is unchanged, which a test now pins against the exact string the old `format!` produced rather than against a `contains`. Three more cover the round trip for `DiskFull`, `StorageFull`, `FileNotFound` and `SlowDown`, that unrelated errors are not mistaken for stage wrappers, and — the case the issue names — that a not-found surfacing from inside a stage is judged cleanup-safe by the decommission loop exactly as a direct one is.
Refs backlog#1827
Add default-off PUT stage helpers for fdatasync batch shape and rename quorum fanout shape so #925 follow-up probes can distinguish shard sync batching opportunities from fanout convergence.
Co-authored-by: heihutu <heihutu@gmail.com>
`is_data_usage_cache_absent` matched `FileNotFound | VolumeNotFound`, but `SetDisks::get_object_reader` runs its failures through `to_object_err`, which rewrites those to `ObjectNotFound` and `BucketNotFound` before they reach the caller. The classifier therefore never matched in production: a cache object that simply does not exist was treated as a transient failure, retried five times with backoff, and then reported as an error instead of an empty cache. Admin server-info resolves one cache per erasure set, so that is roughly 1.5s of pointless backoff per set on any cluster whose scanner has not written a cache yet.
The same rewrite is why the pre-existing `FileNotFound | VolumeNotFound` arm in the old loop never fired either, which left the legacy-key fallback beside it unreachable — it only ever returned an empty cache through the catch-all break.
The classifier now covers the rewritten variants as well as the raw pair, the test store reports absence the way `to_object_err` does, and a new test pins which variants actually arrive.
Refs backlog#1828
* fix(ecstore): make the data-usage cache load actually retry
`load_data_usage_cache` wrapped its read in `while retries < 5`, but every arm of the match inside broke out of the loop, so `retries` was never incremented and the random sleep below it was unreachable: the loop always ran exactly once. The fallback arm compounded this by re-matching the *outer* error after the legacy-key read failed, which meant its second arm could not be reached either.
The read now goes through `rustfs_utils::retry::retry_with_backoff`. A key that is absent under both the prefixed and the legacy name still yields an empty cache without retrying, since retrying a definitive absence cannot turn it into a hit. A transient failure is retried with capped, jittered backoff and surfaces as an error once the attempts are exhausted, instead of being reported as an empty cache — the sole caller already maps `Err` to `usage_error = DATA_USAGE_UNAVAILABLE`, so a read failure now says "unavailable" rather than "zero usage".
`load_data_usage_cache` is generic over `ObjectIO` rather than taking `&SetDisks`, which is what makes the retry and fallback ordering testable at all; being untestable is why the inert loop survived. The call site passes `as_ref()` instead of cloning the `Arc` it immediately borrowed.
Refs backlog#1828
* fix(ecstore): route the load bound through the storage-api contracts
The generic bound named `rustfs_storage_api::ObjectIO` directly, which the architecture guard rejects: ecstore modules must reach storage-api symbols through `crates/ecstore/src/storage_api_contracts`. The bound is now the crate's own `EcstoreObjectIO` alias, which pins the same associated types in one place.
That alias is `pub(crate)`, so `load_data_usage_cache` becomes `pub(crate)` too rather than exposing a crate-private bound on a public signature. Nothing outside ecstore called it — its only caller is `diagnostics/admin_server_info.rs`, and it was never re-exported from the crate root.
---------
Co-authored-by: houseme <housemecn@gmail.com>
Route strict inline rename_data dst-parent fsync through the default-off group-commit helper when enabled while preserving the namespace file-sync limited path by default.
Co-authored-by: heihutu <heihutu@gmail.com>
Use Linux openat2 with RESOLVE_BENEATH and RESOLVE_NO_SYMLINKS for LocalDisk I/O path validation while keeping the existing lstat walk as the public-path and unsupported-kernel fallback. Add focused regression coverage for traversal, symlink swaps, missing leaves, recreated parents, high-cardinality prefixes, final symlink leaves, and concurrent validation.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(replication): split four oversized hot-path functions into focused helpers
Pure-move decomposition of the four oversized functions flagged by the
replication compatibility review (P1-18), unblocking migration milestone
M2 which requires resyncer moves to stay mechanical:
- resync_bucket (522 lines -> 61-line step sequence): leader lock,
target resolution, walk/collector/worker spawning, and dispatch loop
extracted into focused helpers; pure decision helpers (DTO builders,
HEAD-result classification) separated from IO orchestration.
- replicate_all (411 lines -> 113-line main body): initial target-info
seeding, read/stat option builders, skip-path notes, target HEAD
action resolution, and the multipart/single-put payload transport
extracted as private free functions.
- start_mrf_processor (306 lines -> 46-line spawn body): recovery guard,
ledger load, per-entry replay (delete/object/metadata), and retained
entry resolution extracted; retry bookkeeping semantics preserved
exactly (inner continue-paths push inside helpers, outer Missed push
stays in the loop).
- apply_iam_item (255 lines -> match dispatch skeleton): one helper per
IAM item type.
No behavior change: log texts, error paths, event emissions, and metric
counts are byte-identical; existing tests unchanged and green (238
ecstore replication/mrf/resync + 232 rustfs site-replication).
* feat(replication): proxy GET/HEAD/Tagging for unreplicated objects to replication targets (#6172)
* feat(replication): proxy GET/HEAD/Tagging for unreplicated objects to replication targets
Implements the MinIO active-active read-proxy protocol (P1-5 of the
replication compatibility review): when a GET/HEAD/GetObjectTagging/
PutObjectTagging/DeleteObjectTagging request fails locally with
not-found and the bucket has replication targets, the request is proxied
to the targets in rule order, mirroring bucket-replication.go
proxyGetToReplicationTarget/proxyHeadToRepTarget/proxyTaggingToRepTarget.
Protocol surface:
- Anti-loop: inbound {x-rustfs-,x-minio-}source-proxy-request is parsed
into ObjectOptions (proxy_request + proxy_header_set, matching MinIO
ProxyRequest/ProxyHeaderSet); a request carrying the marker with ANY
value is never re-proxied. Outbound client proxy calls send the marker
as "true"; replication worker convergence HEADs send it as "false" so
a peer's proxy layer cannot answer a convergence check by proxying
back to the source (which would fake Completed without a PUT).
- Target selection: new replication_proxy.rs get_proxy_targets — empty
when the marker is set, versioning is suspended, or no replication
config; otherwise filter_target_arns -> TargetClient lookup, skipping
targets with proxying disabled.
- TargetClient gains head_object_for_proxy/get_object (streaming) and
the three tagging calls. Proxy calls never send the replication-check
SSE-C exemption header; customer SSE-C keys are forwarded verbatim so
the target performs real decryption. Conditional (If-*) headers are
not forwarded (MinIO parity); Range and part_number are, with
parts_count/tag_count/storage_class/expiration passed through.
- Metrics: proxy counters now count only real client proxy traffic,
MinIO-aligned (one total per proxied request, one failed when no
target served it). The previous misattributed counters — replication
worker HEAD/PUT (#2672) and local tagging operations (#2682) — are
removed; ReplProxyMetric now maps the tagging counters instead of
dropping them.
e2e (fake_s3_target extended with tagging + header journaling): proxied
GET body + outbound header contract (marker present, no
replication-check, SSE-C passthrough), HEAD, anti-loop 404 with zero
outbound requests, GetObjectTagging, and metric mapping unit tests.
Rolling note: proxying only activates for buckets with replication
targets; requests carrying the marker keep pre-upgrade behavior.
Refs rustfs/backlog#1675 (P1-5)
* fix(replication): fail SSE-C passthrough closed on targets that drop transport headers (#6178)
SSE-C ciphertext passthrough replicates via X-Rustfs-Replication-* transport
headers. A MinIO/generic-S3 target silently discards them, storing bare
ciphertext with no decryption material — yet the PUT succeeded, so the object
reported COMPLETED with a silently unreadable replica (backlog#1675 N2).
Fail-closed design:
- SsecPassthroughCapability {Unknown, Supported, Unsupported} cached in
BucketTargetSys per target ARN with a recording timestamp. Entries reset
whenever the target is rebuilt, edited, or removed (arn_remotes_map
lifecycle) and expire after SSEC_PASSTHROUGH_CAPABILITY_TTL (10 minutes):
an expired verdict in either direction is re-earned through the audit, so
an Unsupported target recovers automatically after an upgrade (at most one
wasted PUT+HEAD audit per bad target per TTL window) and a Supported
verdict cannot outlive a backend swapped behind the same endpoint.
- Replication worker (replicate_object and replicate_all): fresh Unsupported
targets never receive the PUT — the attempt fails immediately into the
normal MRF retry channel with a "run ?replication-check to re-probe" hint.
Unknown or expired verdicts are audited: after the PUT the worker HEADs
the replica back through the replication-check channel (source version id
mapped through resolve_read_api_version_id, so null-version objects audit
correctly) and requires SSE-C evidence (the echoed customer-algorithm
header); missing evidence records Unsupported and fails the attempt.
Convergence HEADs are audited the same way, so a broken ciphertext replica
from an earlier attempt can never launder itself into COMPLETED via an
ETag match. The gate/evidence policy is pure (replication_target_boundary,
staleness folded in as an input) for the M2 worker migration.
- replication-check grows an SsecPassthrough probe phase: a probe PUT
carrying the live transport-header shape, HEAD-back for evidence, and a
machine-readable Code BucketRemoteSsecPassthroughUnsupported on failure.
The probe verdict is synced into the runtime capability cache. Unlike
VersionFidelity, a failed SsecPassthrough phase does NOT fail the target
overall — it is a capability limit, not a broken replication contract,
and a plaintext-only deployment against such a target must not turn red.
- fake_s3_target: default mode now models a RustFS target (stores the
transport headers, echoes SSE-C evidence); the new
drop_unlisted_replication_headers mode models MinIO. The journal records
whether a request carried transport headers.
Receiver-echo verification: the replication-check HEAD exemption only skips
SSE-C key validation; the response has always built sse-customer-algorithm
from stored metadata (rustfs/src/app/object_usecase.rs), so no receiver
change was needed — pinned end to end by the replication-check e2e against
a real RustFS target.
Rolling-upgrade constraint: RustFS targets older than the replication-check
HEAD exemption (#5898) answer the audit HEAD without SSE-C evidence (or fail
it outright), so SSE-C replication to such targets reports FAILED. This is
deliberate — FAILED-and-retryable beats a silently undecryptable replica —
and self-heals: once the target is upgraded, the next TTL expiry (or a
manual ?replication-check re-probe) re-audits and records Supported.
Plaintext and managed-SSE replication are unaffected. The capability cache
is per-node; each node audits independently.
Known limitations:
- The audit judges evidence from the echoed customer-algorithm header only.
A hypothetical target that preserves that one header while dropping other
transport headers (partial-drop) would pass the audit; no known target
behaves this way — observed targets drop the whole unknown-header family.
- A mixed-version target cluster can flap the verdict between audits routed
to different target nodes until the rollout completes; the TTL bounds how
long each stale verdict persists.
New e2e (backlog#1675 C1 + N2, red-first): fail-closed against a
header-dropping fake (FAILED + no second PUT via the capability cache,
journal-asserted; red run showed the old COMPLETED), replication-check
reports the SsecPassthrough phase Code while the target stays OK overall,
SSE-C heal convergence after a real target outage, and SSE-C
existing-object resync landing a REPLICA readable with the customer key.
TTL expiry in both directions is pinned at the cache and gate seams.
* refactor(replication): move resyncer pure decision logic into rustfs-replication (M2) (#6180)
* refactor(replication): move resyncer pure decision logic into rustfs-replication (M2)
Pure-move milestone M2 of the ECStore replication split (backlog#1675
P1-17): relocate the resyncer's IO-free decision helpers, with their unit
tests, into the crates they already belong to by type ownership. No
behavior change.
Moved into crates/replication:
- resync.rs: resync_status_duration
- delete.rs: resync_existing_delete_replication_info,
replicate_delete_outcome, target_delete_version_id,
delete_marker_purge_version_id, delete_marker_purge_mrf_entry
- object.rs: version_identity_drifted, is_replication_target_offline_error,
SsecPassthroughCapability, SsecPassthroughGate, ssec_passthrough_gate,
ssec_passthrough_evidence_present (param-demoted to the echoed
customer-algorithm string; ECStore keeps the HeadObjectOutput adapter)
- filemeta.rs: NULL_VERSION_ID wire literal (crate-owned copy per the
filemeta-independence contract)
ECStore rewiring (Rule #14: imports stay in *_boundary.rs):
- resync/object-decision/target boundaries re-export the moved symbols;
resyncer call sites are unchanged
- bucket_target_sys keeps only the verdict cache + TTL and re-exports the
capability enum so existing consumer paths keep compiling
Not moved (signatures carry ECStore or aws-sdk types):
verify_resync_head_result, resync_target_error_detail, the SdkError
classifiers, the replicate_all_* option/info builders, and the env-coupled
bounded_resync_max_jobs admission clamp. README milestone table updated.
* chore(replication): retire the datatypes.rs relay early
README sanctions retiring datatypes.rs ahead of M4. The module was a
pure relay (resync boundary -> datatypes -> mod.rs facade) with no
external consumer importing it directly, so the facade now re-exports
ResyncStatusType from replication_resync_boundary and the relay file is
deleted. Consumers stay behind the ECStore facade, keeping Migration
Rule #15 intact — the original retirement wording ("consumers import
through rustfs-replication directly") conflicted with that rule and is
corrected in the README.
* chore(arch): extend migration guards to the M2-moved decision contracts
The adversarial review of the M2 move found the per-symbol ratchet in
check_architecture_migration_rules.sh was not extended for the moved
symbols, leaving them free to be redefined in ECStore or imported past
their boundary without CI noticing:
- resync definition pin + boundary fences gain resync_status_duration;
- the object-decision boundary fences gain the five delete-family
helpers (delete_marker_purge_mrf_entry, delete_marker_purge_version_id,
replicate_delete_outcome, resync_existing_delete_replication_info,
target_delete_version_id);
- the target-boundary fence gains the SSE-C gate family, the offline
classifier, and version_identity_drifted;
- a new definition pin rejects ECStore redefinitions of the M2-moved
fns/enums (ssec_passthrough_evidence_present deliberately excluded:
ECStore keeps a thin HeadObjectOutput adapter under that name).
Mutation-verified: a probe fn ssec_passthrough_gate under
crates/ecstore/src/bucket/replication trips the new pin.
Also anchors the intentionally-duplicated NULL_VERSION_ID wire literal
from the filemeta side and tightens the M2 README note on
bounded_resync_max_jobs.
Add default-off PUT stage attribution for the rename_data sync tail so strict durability probes can split queue wait, fdatasync, directory fsync, rename, per-disk wait, and quorum wait without changing commit ordering or S3-visible behavior.
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(ecstore): single-flight remote disk recovery
* test(ecstore): cover remote recovery review cases
* test(ecstore): exercise recovery through disk slot
* test(ecstore): match format reads exactly
* test(ecstore): cover recovery teardown races
* style(ecstore): format recovery race tests
* fix(ecstore): remove unused health snapshot helper
* fix(ecstore): group recovery monitor test state
* fix(protos): preserve production source in compatibility checks
* feat(common): add MRF intent channel and Mrf request source (HS-01)
Introduce the producer-facing half of the mission repair feed: a global
bounded (8192) channel carrying lightweight MrfIntent values from IO
error paths, plus the RUSTFS_HEAL_MRF_ENABLE delivery kill-switch and
config constants for queue/journal sizing. Delivery is strictly
non-blocking (try_send, drop-on-full) so it can sit on decode-failure
and partial-write paths without adding latency. HealRequestSource grows
a 'mrf' variant so admission accounting can attribute replayed intents.
Part of backlog#1865 (option a: wire HealEvent-style intents with a
durable retry ledger).
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(heal): add MRF queue, durable journal, and intent consumer (HS-01)
Consumer half of the mission repair feed: a bounded pending queue
(100k intents / 8 MiB dual ceiling, drop-newest on overflow), a durable
journal at buckets/.heal/mrf/journal.bin holding the unaccepted pending
snapshot, and a consumer task that batches intents off the global
channel, translates them into prioritized heal requests (decode
failure -> Urgent ECDecode, metadata corruption -> High Metadata,
partial write -> Normal object heal), and retries full admissions with
a 5s backoff and a 3-attempt ceiling.
Durability: every journal record carries its own CRC32 and a
format/version header, so a torn tail truncates cleanly at replay; the
journal is deleted after a successful replay and when the pending set
drains (mirroring MinIO's post-replay list.bin unlink). Losing the last
500 ms flush window is acceptable: replayed duplicates merge via the
manager dedup key and read-repair remains the safety net.
Metrics: rustfs_heal_mrf_queue_depth/_queue_bytes, _dropped_total
{reason}, _replayed_total, _journal_bytes, _journal_fsync_total.
The consumer is wired at heal runtime bootstrap right after manager
start, honoring RUSTFS_HEAL_MRF_ENABLE (default on, rollback = off).
Tests: unit tests for the dual ceiling, record roundtrip, torn-tail
truncation, and the priority mapping; integration tests against a real
4-disk ECStore proving channel intents reach the manager queue as
Urgent/mrf-attributed requests and journal replay arms intents, drops
torn tails, and removes the file.
Part of backlog#1865 (option a).
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(ecstore,scanner): deliver MRF intents from error paths (HS-01)
Wire the three production delivery points, each a single non-blocking
try_send next to the existing in-memory heal paths, which stay as the
fast path:
- read.rs decode-error branch: DecodeFailure intent beside the existing
read-repair submit, so an Urgent ECDecode request survives restarts
even when the Low-priority read-repair request was dropped or lost.
- add_partial: PartialWrite intent, giving partial-write recovery a
durable Normal-priority object heal across restarts.
- scanner_folder metadata-corruption classification: MetadataCorruption
intent beside the existing High-priority scanner heal request.
All three are on error paths only: zero cost on healthy IO.
Part of backlog#1865 (option a).
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix: include mrf heal source counts
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix: keep node heal status wire compatibility
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* test(heal): add concurrency invariants for heal vs delete/overwrite races (HS-12)
Audit conclusion for backlog#1874: RustFS does not need a persistent
object-level healing marker (MinIO x-minio-healing) because every path
that can touch the same (bucket, object) commit surface serializes on
the same namespace write lock, and the heal lock guard spans the whole
rename commit including the HEAL_RENAME_INCOMPLETE partial path.
Lock the conclusion in with two race regression tests:
- heal_racing_version_delete_never_resurrects_the_deleted_version:
shard damage is injected on the doomed version so a Deep heal has real
reconstruction work while a versioned DELETE runs concurrently; the
deleted version must stay deleted and the survivor intact.
- heal_racing_unversioned_overwrites_preserves_the_last_commit:
unversioned overwrites (activating the post-commit tail that deletes
the replaced data dir without the ns lock) race a Deep heal in a
loop; the final current version must be exactly the last commit.
Also adds docs/operations/heal-concurrency-safety-notes-zh.md with the
full intersection matrix (17 intersections), lock-coverage argument,
and the residual-window classification (commit tail races are
fail-into-retry safe; bare prefix delete has zero production callers;
admin no_lock is an explicit operator opt-in).
Co-Authored-By: heihutu <heihutu@gmail.com>
* test: remove redundant heal etag clone
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
heal_bucket, list_bucket, get_bucket_info and delete_bucket returned Error::other("") when a peer answered success=false without an error payload, so operators saw a bare "io error " after quorum reduction. Route all five bucket RPCs through peer_failure_without_details, which names the operation and bucket while staying identical across the peers of one operation so reduce_errs keeps grouping them into a single dominant error.
* feat(scanner): emit excess alerts as S3 notification events
The excess-versions / excess-version-size / excess-folders alerts were
metrics-and-logs only; consoles and external auditors had no way to hear
them (rustfs/backlog#1868, HS-04). MinIO emits s3:ObjectManyVersions /
s3:ObjectLargeVersions / s3:PrefixManyFolders for the same conditions —
RustFS carries those as EventName::Scanner* with s3:Scanner:* wire names
that already existed unpublished.
The three alert sites now also dispatch through the standard event
pipeline (send_event via the storage_api owner facade), carrying the
actual values and thresholds in req_params and UserAgent "Scanner".
Without a cooldown a single over-threshold object would re-emit on every
~60s scan cycle, so emissions are edge-held per (kind, bucket, object)
for 24h (RUSTFS_SCANNER_ALERT_COOLDOWN_SECS, 0 = every cycle), backed by
a process-global map with a 4096-key hard cap that clears rather than
grows. Metrics and structured logs stay level-triggered every cycle;
only the notification events are held back. A restart resets the
cooldown deliberately: one re-emission per still-hot key buys back
visibility after the restarts that accompany incident response.
Tests pin the edge-hold semantics (first fires, immediate re-check held,
independent keys, cooldown expiry re-fires, zero cooldown always emits,
hard bound) in one sequential test for the process-global map, and pin
the emitted wire names against EventName's canonical string forms so a
subscribed bucket notification can never silently stop matching.
docs/operations/scanner-excess-alerts.md documents the three events,
the metric-vs-event cadence difference, and the HS-15 threshold deltas
(alert_excess_folders 65538 vs MinIO 50000 is deliberate: Proxmox
Backup Server chunk layout compatibility).
Closesrustfs/backlog#1868.
Co-Authored-By: heihutu <heihutu@gmail.com>
* docs(operations): split scanner excess alerts into English and Chinese pages
The page shipped Chinese-only; keep it as scanner-excess-alerts_zh.md and
add a faithful English translation at the original path, cross-linked at
the top of both.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* feat(heal): track erasure set progress baseline
Record erasure-set heal byte progress from per-object results and seed progress totals from complete usage-cache snapshots when available.
Keep usage-cache failures observational so heal execution continues without a baseline.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(heal): skip filtered erasure set versions
Skip erasure-set versions written after the durable heal start time, and queue lifecycle-expired versions for expiry before skipping them.
Track new-version and ILM-expired skips separately so progress can explain completed baseline work without treating these skips as retry-blocking failures.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(heal): wire abandoned data-dir cleanup check
Connect check_abandoned_parts through ECStore, pool, and set layers so heal can invoke the existing orphan data-dir reclaim path instead of returning NotImplemented.
Add dry-run support to the reclaim scan and cover dry-run plus scoped set behavior with regression tests.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(obs): add heal scanner trace bus
Introduce an in-process broadcast trace bus with typed heal and scanner events, lazy event construction, and bounded lagged-subscriber behavior.
Cover zero-subscriber publishing, subscription delivery, drop accounting, and lagged receivers with focused common-crate tests.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(obs): stream heal trace events from admin API
Wire the admin trace endpoint to the common trace bus for heal/scanner events, including kind, regex, and threshold filtering.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(obs): emit heal trace events
Publish heal task lifecycle and abandoned-parts cleanup events through the common trace bus so the admin trace stream has live heal diagnostics.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(obs): emit scanner trace events
Publish scanner folder, lifecycle action, and heal-candidate events through the common trace bus for live admin scanner diagnostics.
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(heal): route data usage loader through storage api
Keep ECStore data-usage facade access behind the heal storage_api boundary so architecture migration guards can validate the heal progress path.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(heal): avoid lifecycle snapshots on ordinary heal pages
Only request lifecycle object snapshots when the heal pass has lifecycle expiry context. This keeps ordinary listing and disk-walk pages from cloning FileInfo/ObjectInfo payloads while preserving the skip path that queues expired versions.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(heal): update bug-fix mocks for lifecycle snapshots
Carry the lifecycle snapshot opt-in argument through the remaining heal bug-fix test mocks so all-targets clippy covers the updated storage trait.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(rustfs): sync heal storage mock signature
Update the rustfs storage RPC test mock for the lifecycle snapshot opt-in argument and cover it with rustfs all-targets clippy.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(e2e): allocate smoke ports across nextest processes
Serialize E2E port selection with a small /tmp allocator so nextest workers do not reuse the same just-released ephemeral port before RustFS binds it.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Replace every bare `#[allow(dead_code)]` in ecstore with either a deletion or a per-item allow carrying a `reason`. Blanket allows at module, struct, and impl level silence the lint for future members too, so each is narrowed to the members that are actually dead.
Delete the dead cluster in `config/heal.rs` (`Config`, its three methods, `RUSTFS_BITROT_CYCLE_IN_MONTHS`, `parse_bitrot_config`) rather than annotate it: it has no callers and is unreachable outside the crate, and `parse_bitrot_config` would panic on its disabled path via `Duration::from_secs_f64(-1.0)`. `DEFAULT_KVS` stays, since the config registry uses it.
Correct two `reason` strings on `Checksum::new` and `PutObjReader::md5_current_hex_string`, which are methods but carried a field-only rationale.
Refs backlog#1823
Co-authored-by: houseme <housemecn@gmail.com>
feat(scanner): expose prefix-level bucket usage via admin API
The scanner's per-bucket, per-set usage caches already hold a path-keyed
prefix tree, but dui() flattened it only to bucket names — consoles and
operators had no way to ask "what does this prefix hold" without an S3
listing sweep (rustfs/backlog#1872, MinIO loadPrefixUsageFromBackend
parity).
Add:
- data-usage: prefix_usage_in_cache — a shared aggregation over the
entry map (arbitrary prefix, full counters, one-level sub-prefix
breakdown with names recovered from the literal-path cache keys),
hardened like the scanner's checked flatten: cycles, dangling child
links, over-deep trees, and overflowing counters yield None rather
than unbounded recursion or wrapped totals.
- ecstore: ECStore::all_set_disks — iterate every erasure set so a
query can read each set's own cache copy; the hash-routed store path
would always land on one set.
- scanner: bucket_prefix_usage — per-set loads (5s budget each, a slow
set degrades to not-reporting instead of stalling the caller),
merged across sets with partial/compacted/truncated flags, served
from a bounded 30s cache (128 entries, hard-capped) that bucket
writes invalidate through the dirty-usage hook.
- admin: GET /rustfs/admin/v3/usage/{bucket}?prefix=&max-entries=
behind the same any-of gate as datausageinfo (DataUsageInfoAdminAction
OR ListBucketAction), rejecting unknown query parameters and
clamping max-entries to 1..=10000. Route registered in the policy
table (deferred MultipleActions, matching datausageinfo) and the
route matrix test.
Closesrustfs/backlog#1872.
Co-authored-by: heihutu <heihutu@gmail.com>
feat(ecstore): pin bitrot algorithms with a startup self-test
A drifted HighwayHash implementation fails silently: every shard reads
back corrupt, heal rewrites healthy data, and cross-platform clusters
disagree about which copy is good. Mirror MinIO's bitrotSelfTest by
verifying, once at process start:
- known-answer digests for HighwayHash256S / HighwayHash256SLegacy over
a deterministic 4096-byte xorshift64* payload, plus the externally
verifiable FIPS SHA-256 "abc" vector guarding the HashAlgorithm
plumbing itself;
- an end-to-end roundtrip per streaming variant (encode -> size formula
-> bitrot_verify -> BitrotReader read-back), over full blocks and a
partial tail;
- tamper detection: one flipped byte in the final data block and one in
the leading hash must both be rejected as a hash mismatch, not by an
incidental read error.
The check costs microseconds and runs inline in
init_background_service_runtime before any shard can be written or
verified. Outcome surfaces as one structured bitrot_selftest log event,
the rustfs_bitrot_selftest_status gauge (1=passed / 0=failed / 2=skipped),
a bitrotSelftest field on the admin server-info response, and
RUSTFS_BITROT_SELFTEST_STRICT=on turns a failure into a startup error
(MinIO Fatal parity; the default only degrades the status so a bad build
cannot brick an existing fleet on upgrade).
Closesrustfs/backlog#1873 (HS-11).
Co-authored-by: heihutu <heihutu@gmail.com>
backlog#1823 step 1, the diagnosis half. Temporarily removing set_disk/mod.rs's #![allow(unused_variables)] surfaced nine bindings. The issue asks that values computed and then dropped on write/quorum paths be diagnosed before being underscored, and that turned out to matter: only four were plain leftovers.
Two errors were bound and then left out of the log they were bound for. complete_multipart_upload's checksum failures read `if let Err(err) = ...` and then log part_id, bucket and object with no `err` anywhere in the message, so a checksum failure in production told you which part failed but not why. Both messages now carry the error.
One is a lock guard. heal's write_lock_guard holds a namespace write lock for the rest of the scope; renaming it to a bare `_` would drop it immediately and release the lock. It is now `_write_lock_guard`, with a comment saying why it must not be `_`.
One was kept alive by a corpse. `errors` in read_multiple_files is read by nothing except two commented-out debug! lines directly below it; the binding and the commented lines go together.
One is a cfg split. heal's disk_index is read only inside the #[cfg(test)] fault-injection branch, so underscoring it would break the test build; a `#[cfg(not(test))] let _ = disk_index;` covers the non-test lane instead.
The remaining four are genuine leftovers: an unused enumerate index in list_object_parts, a discarded error in a heal reader loop, an inner binding shadowing its own iterator variable, and delete_object's write_quorum.
That last one is worth a separate look: delete_object asks get_object_info_and_quorum for a write quorum and never uses it, because delete_object_version below recomputes its own as disks.len() / 2 + 1. The two are not the same number — one comes from the object's erasure configuration, the other is a plain majority of the disk array. Pre-existing behaviour, untouched here.
The blankets stay for now. Removing #![allow(unused_imports)] exposes 76 unused imports in set_disk/mod.rs, and they cannot be removed per-lane: cargo fix, working from the lib lane, produced 54 compile errors in the test lane. That needs its own pass with both lanes checked per import.
Verification: cargo check -p rustfs-ecstore --tests and --features test-util --tests both warning-free; clippy --lib --tests -D warnings clean; cargo nextest run -p rustfs-ecstore 4101 passed; make pre-commit exit 0.
Ref rustfs/backlog#1823 (step 1).
Add a diagnostic metadata-only read_version delay hook for GET data-read fanout so bounded/default behavior can be compared under controlled slow-tail metadata responses.
Co-authored-by: heihutu <heihutu@gmail.com>
#6107 routed the transitioned read through the object's own ReadPlan so a tiered SSE object stops serving ciphertext. Compression rides that same plan and got fixed with it, but nothing pins it: revert the routing and a compressed object that ILM moved to a warm tier returns its stored (compressed) bytes under the compressed size, with every existing test still green.
The gap is easy to reopen because transition genuinely uploads the stored representation — the upload side is correct and the read side is the only place that can decode it. These tests state that contract at the boundary where it broke.
Four cases, all through SetDisks::get_object_reader against a mock warm tier:
- a full GET of a transitioned compressed object returns the plaintext and publishes the plaintext size (the test also asserts the remote copy holds the compressed bytes, so it fails loudly if the upload side ever changes instead);
- a ranged GET returns that plaintext slice, with the range deliberately starting past the compressed size so a range still measured in stored coordinates cannot produce it;
- a restore read still receives the stored bytes under the stored size — restore_request_active holds it on the Plain branch, and decompressing there would write plaintext under compressed metadata;
- a plain transitioned object still reads back byte-identical, full and ranged.
Verified as guards, not decoration: forcing the tiered read back onto the Plain branch turns the two compressed tests red and leaves the plain and restore tests green.
What these do not pin, so the gap stays recorded rather than implied covered: the fixture carries no compression index, so part.index stays None and the plan's storage offset is always 0 — the compressed-offset translation itself is still untested, as are multipart compressed objects, partNumber reads, and the encrypted tiered read that #6107 targeted.
* chore(ecstore): drop the bucket dead_code blanket
The last blanket of the backlog#1823 burn-down, and the largest: 71 items across lifecycle, replication, metadata, quota, object lock and bucket utils. Four are deleted.
Deleted, all trivial:
- check_valid_object_name and check_valid_object_name_prefix, a pair that only calls into each other with no external caller. Worth stating plainly so nobody reads this as a validation gap: object names are validated through check_object_name_for_length_and_slash, which is live; this pair is a second, unwired entry point.
- DEFAULT_HEALTH_CHECK_RELOAD_DURATION, a lone unused constant.
- The LifecycleReplicationConfig alias, which orphaned a re-export in replication/mod.rs that goes with it.
Everything else is kept, in four groups, because the blanket here was hiding structure rather than rot:
Windows platform gating. WINDOWS_RESERVED_NAMES, the two reason constants and object_name_has_windows_incompatible_segment are called from inside the #[cfg(target_os = "windows")] block in check_object_name_for_length_and_slash (utils.rs:228-255), so they only read as dead on non-Windows hosts. As with the Linux gating in the disk root, this cannot be adjudicated locally: cargo check for both x86_64-pc-windows-msvc and x86_64-unknown-linux-gnu fails in the aws-lc-sys build script for want of a cross C toolchain. CI covers both.
Declared boundary surface. The *_boundary.rs and *_bridge.rs files carry the replication split plan's contracts, which scripts/check_architecture_migration_rules.sh pins through the EcstoreReplicationBoundaryImports section of the split-plan doc. Their unused items are declarations, not leftovers.
test-util seams. ConfigWriteLockProbe with install/wait_until_attempted follows the same pattern as the barriers in the services and set_disk roots.
MinIO-parity tier/lifecycle entry points that this port never wired: apply_lifecycle_action, get_transitioned_object_reader, recover_tier_free_versions, delete_object_from_remote_tier, abort_tier_delete_journal_entry and the replication pool's worker-management surface. These are complete, substantial machinery with no caller — the same shape as data_usage's local_snapshot feature. Removing them is a product decision, so they are made explicit here rather than deleted.
Verification, four lanes warning-free: default, --tests, --features rio-v2 --tests, --features test-util --tests. cargo nextest run -p rustfs-ecstore 4096 passed; clippy --lib --tests -D warnings clean; make pre-commit exit 0. Note that clippy is what caught the orphaned re-export above: cargo check and pre-commit both treat unused_imports as a warning.
Ref rustfs/backlog#1823 (step 2, final root).
* chore(ecstore): correct inaccurate dead_code reasons in the bucket root
Six items were labelled 'asserted by this file's tests' or as MinIO-parity
entry points while having no caller at all - free get_bucket_acl_config and
created_at only reach their own live methods (production goes through
created_at_in), BucketVersioningSys::get_in, utils::serialize_content and
ServiceType have no reference anywhere, and with_transition_queue_env_async
is an unused test fixture, not a tier entry point. Name what each one is so
the next reader does not assume coverage that is not there.
Ref rustfs/backlog#1823.
* chore(ecstore): drop the set_disk dead_code blanket
Removing the blanket exposes 39 items; exactly one is deleted. The low share is a finding, not caution: unlike the disk root, where platform gating made local adjudication impossible, here the items were checked and nearly all of them are live.
Deleted: HealEntryResult, the only item with no reference anywhere.
What the checks turned up, in the order the warnings suggest deleting them:
SetDisks::rename_data looked like the head of a dead chain feeding into_legacy_tuple and RenameDataLegacyTuple. It is not: production goes through rename_data_owned, and rename_data itself has test callers at mod.rs:5809 and 5880. The chain below it is therefore live through the tests, and inferring "this is dead, so its callee is dead" would have removed three working items.
create_bitrot_readers_until_quorum, read_multiple_files and map_cleanup_join_result all have callers inside their files' test modules, so they only look dead in the lib target.
TransitionCommitBarrier and TransitionUploadedSaveProbe, with their install/wait_until_paused/release surfaces, are installed by tests behind #[cfg(all(test, feature = "test-util"))].
ctx.rs's SetDisksCtx accessors are the split seam left by the SetDisks god-object break-up (backlog#815).
heal_object_dir's two apparent references are comments, and they document an index-alignment contract that live code maintains for it, so they stay as they are.
Worth a maintainer decision: the metadata early-stop switch has a complete percentage-rollout facet — ENV_RUSTFS_GET_METADATA_EARLY_STOP_ROLLOUT_PCT, get_metadata_early_stop_rollout_pct and should_use_metadata_early_stop — with no caller, no test and no documentation, while its sibling enable flag is live. It is kept with an allow that says so rather than removed, since a rollout knob is a product call.
One placement note for anyone adding allows near heal code: check_logging_guardrails.sh requires #[instrument(level = "trace")] to sit immediately before async fn heal_object_dir, so the allow goes above the instrument attribute. Putting it between the two drops the guard's match count and fails the check.
Verification, four lanes warning-free: default, --tests, --features rio-v2 --tests, --features test-util --tests. cargo nextest run -p rustfs-ecstore 4096 passed; clippy --lib --tests -D warnings clean; make pre-commit exit 0.
Ref rustfs/backlog#1823 (step 2).
* chore(ecstore): fix duplicated and inaccurate dead_code reasons in set_disk
format_lock_error carried the same #[allow] twice. Five items in the
locking/heal roots were labelled 'asserted by this file's tests' while
having no reference at all - heal_object_dir's only two references are
comments, as this branch's own notes point out. Say what each item
actually is instead, so the next reader does not assume test coverage
that is not there.
Ref rustfs/backlog#1823.
* chore(ecstore): correct the bounded_spare_disk_index dead_code reason
The mod.rs copy is an unused test fixture, not something this module's
tests assert; the namesake that is exercised lives in the io_primitives
test module.
Ref rustfs/backlog#1823.
* chore(ecstore): drop the disk dead_code blanket
Removing the blanket exposes 36 items in the lowest storage layer: 7 deleted, 29 kept with reasoned item-level allows. That is the smallest deletion share of this burn-down, and the reason is a verification limit rather than a judgement call.
disk/local.rs carries 141 `#[cfg(target_os = "linux")]` sites — the densest platform gating in the tree, because O_DIRECT and io_uring only exist there. The direct-I/O cluster (six ENV_RUSTFS_OBJECT_DIRECT_IO_* constants plus is_direct_io_read_enabled, is_direct_io_write_enabled, get_direct_io_read_threshold, direct_write_staging_capacity, direct_write_tail_split and DIRECT_WRITE_STAGING_BYTES) reads as dead on macOS purely because its production callers at local.rs:1766, 3114 and 4605 sit inside Linux-gated blocks. direct_write_staging_capacity even documents itself as "Platform-independent (no O_DIRECT), so it is unit-tested on any host".
Deleting those would leave every local check green — 4096 tests pass, clippy is clean, make pre-commit exits 0 — and break the Linux build in CI, because all four local lanes compile for aarch64-apple-darwin. Cross-checking locally is not available either: cargo check --target x86_64-unknown-linux-gnu fails in the aws-lc-sys build script for want of a Linux C cross-compiler. Their allows name the platform reason so the next reader on a non-Linux host does not repeat the investigation.
Deleted, all in files with no target_os gating at all (os.rs, disk_store.rs):
- HealthDiskCtxKey and HealthDiskCtxValue with its private log_success. Note that DiskHealthTracker::log_success is a different method of the same name and is live from cluster/rpc/peer_s3_client.rs and remote_disk.rs — the two have to be told apart by type, not by name.
- LocalDiskWrapper::new_with_health and check_id.
- os.rs file_exists and lock_destination_directory_for_path_access.
Kept with allows: DiskHealthTracker's set_faulty, mark_offline, waiting_count and last_success have test callers in remote_disk.rs, so they only look dead in the lib target. to_disk_error, remove_all and sync_dir_files are asserted by their own files' tests. The reclaim, mmap and path-cache field groups are written but never read back.
Placement follows the same rule as the earlier roots: per-method allows inside impl DiskHealthTracker and impl LocalDisk, since both are mostly live and a block-level allow would be a smaller version of the blanket this issue removes. Struct-level allows are used only where the warning covers that struct's own fields. The three cached_read_env! functions take their allow inside the macro invocation, before the fn line, because the macro forwards $(#[$meta:meta])* onto the generated item.
Verification, four lanes warning-free: default, --tests, --features rio-v2 --tests, --features test-util --tests. cargo nextest run -p rustfs-ecstore 4096 passed; clippy --lib --tests -D warnings clean; make pre-commit exit 0. The Linux lane is not covered locally and is left to CI.
Ref rustfs/backlog#1823 (step 2).
* chore(ecstore): correct two dead_code reasons in the disk root
check_valid_path and reject_symlink_components have no caller at all -
not even a test - so 'asserted by this file's tests' misreads them as
covered. Both are method wrappers over live free functions; say that
instead.
Ref rustfs/backlog#1823.
houseme