Removing both blankets exposes 23 items, of which only four are deleted. The ratio is the point: close to the core data path the blankets were hiding test assertions and migration seams, not dead code.
A cfg-split function is the reason two symbols in the internode transport look dead when neither is. build_internode_data_transport_from_env has two bodies, one under #[cfg(test)] that calls build_internode_data_transport directly and one under #[cfg(not(test))] that goes through the INTERNODE_DATA_TRANSPORT static so tests do not share process-global transport state. Each half's helper is live in exactly one build, and because cargo check --tests compiles both the lib target and the test harness, both symbols appear in one warning list. Deleting either one breaks the other lane. Both are kept with allows naming their half.
Three deletion candidates were withdrawn after a per-name grep: ParallelReader::new, ErasureDecodeReader::new and SyncErasureDecodeReader::new all have test callers. The last two are exactly the shape of the dead wrapper deleted in #6084 — a thin forward to a new_with_metrics_path sibling — except that sibling is live in production (set_disk/read.rs) and the wrappers are used by tests.
Deleted:
- RemotePeerS3Client::get_addr and RemoteLocker::from_url, neither with a consumer in any lane.
- RemotePeerS3Client's node field, which new writes after using it to derive addr and nothing ever reads. Its only other writer was a test helper that built a whole Node solely to fill the field; that block goes too.
- ParallelReader::can_decode, superseded by an inlined copy. The copy's comment named the method it replaced, so deleting the method alone would have left a dangling reference; the comment now describes the check instead of pointing at a method that no longer exists.
Kept with allows: the erasure items are decode/encode invariants asserted by their own files' tests (shard_read_launch_order, decode_with_read_costs, emit_data_shards, queued_block_bytes, the engine trait facets, the ParallelReader and decode-reader constructors, encode_stream_callback_async). On the cluster side, peer_replay_state, heal_bucket_local and clone_drives are test-only, InternodeDataTransportCapabilities and tcp_http are constructed only by transport test doubles, and the InternodeDataTransport trait's name/capabilities pair is an unused capability-negotiation facet kept for the transport split (backlog#1350) — six impls provide them and no caller negotiates on them yet.
Verification, four lanes warning-free: default, --tests, --features rio-v2 --tests, --features test-util --tests. cargo nextest run -p rustfs-ecstore 4041 passed; clippy --lib --tests -D warnings clean; make pre-commit exit 0.
Ref rustfs/backlog#1823 (step 2).
heal::Error carried six variants with zero construction and zero match sites (ConfigurationError, NotFound, TaskAlreadyExists, ManagerNotRunning, EventProcessingFailed, ProgressTrackingFailed) plus IO(String), which was never constructed either — its only appearances were two or-pattern match arms that could never fire (task.rs's demotion match and the recoverability classifier). All seven are deleted and the two or-patterns lose their dead alternative.
Config(String) stays (live, four construction sites); Io(std::io::Error) stays; the retry classifier's behavior is untouched per the issue constraint — removing an arm that can never match is not a classification change.
Ref rustfs/backlog#1831 (PR3).
Co-authored-by: cxymds <cxymds@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Record local read_version path resolution, path length check, xl.meta read, and metadata decode durations through the existing GET stage metrics channel. The new samples are gated by GET stage metrics so metrics-off reads avoid timer and recorder work.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(sse): sink managed-SSE attribution into the shared encryption-keys module
Moves the managed-SSE classifier — stored_managed_encryption_key, contains_managed_encryption_metadata, normalize_managed_metadata — and the SSEType enum from rustfs/src/storage/sse.rs into crates/utils/src/http/object_encryption_keys.rs, the module that already owns every constant they read. This is PR-B0 of rustfs/backlog#1643: crates/scanner must never depend on the rustfs binary crate, so encryption attribution has to live in a shared lower layer before the scanner can report per-scheme coverage without growing a second classifier.
SSEType moves wholesale (option a): its only impl is the dependency-free audit_label(), so the enum relocates verbatim (audit_label becomes pub) and rustfs::storage::sse re-exports it, keeping every existing path compiling. The one piece that cannot move verbatim is normalize_managed_metadata's KMS-context branch, which needs base64 and serde_json — dependencies rustfs-utils does not have and does not gain here. The shared normalizer instead takes an injected Option<fn(&str) -> Option<String>> context recoder; sse.rs passes recode_minio_kms_context, the old inline chain verbatim including the silent skip on decode failure. stored_managed_encryption_key passes no recoder because the context mapping only ever inserts the context key, which the key-id lookup never reads, so its output is identical.
Every metadata lookup stays a case-sensitive exact match (lowercase x-amz-* stored forms, TitleCase MinIO-internal names) per the backlog#1775 trap; new shared-module tests pin that, and a source-scan test in sse.rs asserts the classifier has exactly one definition so a second copy cannot silently return.
* fix(utils): satisfy encryption key test clippy
---------
Co-authored-by: cxymds <cxymds@gmail.com>
* fix(kms): construct wrap_budget_reserved in the VaultKeyData deserializer
main does not compile: #6019 added VaultKeyData.wrap_budget_reserved on a base that predated #6003's hand-written Deserialize, so the visitor's struct literal never learned about the field. Each PR was green on its own base; the breakage only exists in their merge.
The field joins the other three lists the hand-written impl maintains (Field enum, match arm, struct literal, FIELDS) and defaults to 0 when absent — the value a record written before wrap accounting, or rewritten by an older build, carries; zero restarts the reservation rather than blocking a wrap.
vault_key_data_deserializer_covers_every_serialized_field turns this class of mistake into a test failure instead of a merge-order accident: it serializes a fully populated record and asserts the deserializer recognizes every emitted key (unknown-field counter stays zero) and reads every value back. Mutation-verified by dropping the new match arm.
* feat(kms): report a key as due for rotation once its wrap budget is spent
The rotation readiness verdict only knew about age; the wrap accounting landed by #6019 counted wraps and published an aggregate gauge but never fed the per-key verdict, leaving the criterion backlog#1636 asks for unimplemented.
RUSTFS_KMS_ROTATION_MAX_WRAPS adds the second, independent threshold, parsed with the same discipline as the age one: unset or unparsable leaves the verdict unreported rather than inventing a policy, and values below one million are raised to it because wraps are reserved in blocks of that size and a smaller threshold would trip on the first reservation regardless of how many wraps happened.
The wrap check runs before the age check so that a key crossing both reports 'wraps': the AES-GCM random-nonce ceiling is a cryptographic bound an operator cannot negotiate, while the age period is a policy they chose. Backends that report no count — Transit and AWS wrap externally, and pre-accounting records carry nothing — leave the wrap half silent instead of guessing, and a backend that cannot rotate is still never told to.
Refs rustfs/backlog#1636 (PR-3 acceptance criterion), rustfs/backlog#1562.
---------
Co-authored-by: houseme <housemecn@gmail.com>
Add a default-off inline-only data-read metadata early-stop gate that verifies inline plaintext before cancelling pending metadata tasks.
Keep non-inline, prepared, and request-shape-sensitive reads on full fanout, and record scheduled/completed/cancelled ReadVersion lifecycle metrics for normal fanout completion.
Co-authored-by: heihutu <heihutu@gmail.com>
* perf(metrics): attribute PUT stage costs
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): move PUT metadata during shuffle
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(s3): reuse PUT object lock state
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): trim PUT metadata fanout clones
Build per-disk PUT metadata only for committed writer slots, move the response metadata out of the fanout vector, and preserve fresh FileInfo shuffle semantics.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(metrics): make PUT stage attribution opt-in
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): commit inline PUT shards directly
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): streamline rename staging cleanup
Use the directory-specific removal operation for rename_data staging parents. This avoids a guaranteed failed file-removal probe on Unix-like hosts and lets Windows remove the empty directory directly while preserving best-effort non-empty handling.
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(ecstore): cover inline PUT rename failures
Cache the detailed stage metrics gate once per PUT and exercise exact-quorum and quorum-minus-one failures after inline shard encoding.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Keep encoded shards in one contiguous Bytes buffer while they cross the streaming write queue, and materialize Vec<Bytes> only for the existing public APIs.
Co-authored-by: heihutu <heihutu@gmail.com>
init_background_expiry resolved its worker count through three env vars, none documented, none set in any known deployment: RUSTFS_MAX_EXPIRY_WORKERS, silently overridden by the underscore-prefixed _RUSTFS_ILM_EXPIRATION_WORKERS (a MinIO fossil, comment included), with a zero value then falling through to RUSTFS_DEFAULT_EXPIRY_WORKERS.
RUSTFS_MAX_EXPIRY_WORKERS stays as the canonical name per the rc constraint (no new env var names): the count is now resolved once — a set, parseable, non-zero value wins, anything else falls back to min(cpus, 16). The constant moves to rustfs-config's runtime constants alongside ENV_TRANSITION_WORKERS, the two dead names are gone repo-wide (rg-verified), and a serial four-state unit test (unset/zero/valid/garbage) pins the resolution, modeled on the transition-worker env harness.
Ref rustfs/backlog#1832 (PR2).
PR #6008 deleted crates/io-metrics/src/config.rs but the crate docs still taught the deleted API: both READMEs kept the Unified Configuration sections, module-tree entries and ./src/config.rs links, the example kept an orphaned numbered comment, and the io-core/io-metrics changelog had no removal record. Scrub all of it; keep cache_config.rs references, which are still real.
* test(e2e): fold seven identical POST-policy exact-mismatch tests into one table
The seven *_policy_mismatch tests in multipart_auth_test.rs were body-identical after literal normalization: the policy pins one field to an exact value, the form sends a different value, and the upload must be rejected with 400 InvalidPolicyDocument naming the field. Each test booted its own full server.
They fold into one table-driven test on the run_post_object_policy_case helper introduced by the PR1 fold. Every row keeps its original test's exact bucket, key, field name, policy value, mismatched form value, body bytes, and expected error strings — including the three rows that asserted the stronger <Code>InvalidPolicyDocument</Code> form. The pinned condition is built with an explicit serde_json::Map since the field name is now a table parameter.
cargo nextest list reports 97 tests for this module; the inventory row is updated in the same diff (103 -> 97).
Ref rustfs/backlog#1838 (PR2).
* test(e2e): fold the remaining POST-policy duplicate groups (15 tests) (#6018)
Completes the multipart_auth table-driven fold: the six remaining body-identical groups collapse onto the shared run_post_object_policy_case helper.
- Seven single-field exact-mismatch tests (cache-control, expires, tagging, storage-class, content-type, success_action_status, metadata-field-exact) join the existing exact-condition mismatch table as rows — same shape as the PR2 fold.
- The two object-lock mismatch tests become a two-row table (policy pins mode + retain-until-date, one form field mismatches).
- The three SSE-KMS parameter mismatch tests become a three-row table (policy pins the SSE mode plus one KMS parameter, form differs).
- The three SSE-KMS outside-policy tests become a three-row table pinning the distinct contract: an undeclared KMS parameter sails past policy validation and is rejected at runtime with 501 NotImplemented, not a policy error.
Every row keeps its original test's exact bucket, key, field names, values, body bytes, and expected status/code strings. cargo nextest list reports 85 tests for the module; the inventory row is updated in the same diff (97 -> 85).
Ref rustfs/backlog#1838 (PR3).
The policy crate's Go path.Clean port and rustfs-utils' Windows-aware clean look like duplicates but are not interchangeable: S3 ARN/resource matching must treat backslashes as object-name data, never as separators, so adopting the utils version would change policy evaluation semantics on Windows — a security-adjacent behavior change. Record that judgment as bidirectional do-not-merge notes on both implementations, per the issue's adversarial ruling.
Comment-only change.
Ref rustfs/backlog#1833 (PR7).
BitrotErrorType (disk/error.rs) was constructed only by its own unit test: production bitrot mismatches never flow through it (they surface as DiskError::other strings). Delete the enum, its From<BitrotErrorType> for DiskError impl, the self-test, and the api facade re-export. The facade inventory doc does not name the type, so no doc change is needed.
DiskError::SourceStalled and DiskError::CrossDeviceLink are never constructed locally — they are reachable only through wire decoding and no current node sends them. Their decode arms stay per the cross-version compatibility constraint; each variant now carries a doc comment saying exactly that so the next dead-code sweep does not re-litigate them. Their consumer arms (heal classifier, batch processor) are left untouched — the values cannot appear, so removing the arms would be unobservable, and the heal classifier is pinned by the issue as do-not-touch.
Ref rustfs/backlog#1831 (PR4).
crates/common/src/bucket_stats.rs (ReplicationLatency plus a commented-out ReplicationLastMinute corpse) had zero consumers anywhere in the workspace — the live replication statistics implementation is crates/replication/src/stats.rs. LastMinuteHistogram in last_minute.rs (already carrying allow(dead_code)) was equally unreferenced, and size_to_tag / SIZE_LAST_ELEM_MARKER had no user besides the histogram, so the whole block goes with it. LastMinuteLatency and AccElem stay: common's metrics.rs uses them.
Ref rustfs/backlog#1833 (PR5).
The peer-edit delivery fence from #5882 treated an equal applied generation as stale. One edit legitimately fans out one delivery per peer record under a single generation (the ILM-expiry edit sends every peer's record), so the receiver applied only the first body, raised its high-water mark, and silently acked-success while dropping the rest — enableILMExpiryReplication never converged on receiving sites and the three-node nightly e2e failed deterministically (issue #5767).
Only a strictly newer applied generation is stale now. Equal generation implies the same logical edit and re-applying a delivery is idempotent (update_peer overwrites the peer record; the mark is raised with max), while strictly older deliveries — the cross-node ordering case the fence exists for — stay rejected.
Adds a composed unit test driving three same-generation bodies through the receiver's fenced sequence, and widens the replication e2e's two site-replication wait helpers from a 10s polling ceiling to the 30s deadline the file's other waits use.
PutBucketVersioning with Status=Suspended on a bucket that carries a replication configuration now fails with InvalidBucketState, matching AWS S3 and MinIO. Suspension would start minting null versions that the versioned replication engine can never converge — the state is unreachable on AWS and MinIO, and the nightly acceptance-matrix e2e that tried to exercise it failed every night since it landed (issue #5767).
The acceptance-matrix test tail now pins the rejection contract (InvalidBucketState) and verifies a fresh matched PUT still replicates with a real version id after the rejected suspension.
crates/io-metrics/src/config.rs was a near-copy of io-core's Backpressure/Deadlock configuration with already-drifted field names (high_watermark vs io-core's high_water_mark) and had no consumer outside the crate's own example: the canonical BackpressureConfig lives in crates/io-core/src/backpressure.rs. Delete the module, its lib.rs re-exports, and the example's unified-config section, and settle the corresponding ARCHITECTURE.md ledger line that tracked this copy's removal.
Ref rustfs/backlog#1833 (PR4).
* chore(iam): remove eight dead error variants
iam::Error mirrored policy::Error variant-for-variant, and eight of the twins had zero construction and zero match sites anywhere in the workspace: InvalidServiceType, ErrCredMalformed, CredNotInitialized, JWTError, NoAccessKey, InvalidToken, InvalidAccessKey, InvalidExpiration (each verified by repo-wide sweep; the InvalidToken hits elsewhere are KeystoneError's unrelated variant). Delete the variants along with their Clone and PartialEq arms.
The From<policy::Error> mapping keeps its exhaustive match: the eight orphaned arms now route through a grouped binding to Error::StringError(err.to_string()), so the rendered message is preserved; nothing could observe the old discriminants because no site ever matched on them.
Ref rustfs/backlog#1831 (PR1).
* fix(iam): make Error clone variant-preserving via Arc payloads
iam::Error's hand-written Clone demoted PolicyError and CryptoError to StringError because their payloads are not cloneable — a clone changed the variant identity. There is no production clone site today (the issue's refuter confirmed this is preventive hardening, not a live bug), but any future holder of a cloned error would match the wrong variant.
The two payloads are now Arc-wrapped, so Clone is a cheap reference bump that keeps the variant. Display strings are unchanged ({0} and crypto: {0}); the #[from] derives become manual From impls wrapping in Arc; the one behavioral trade-off is that source() is no longer forwarded for these two variants (Arc<E> does not implement std::error::Error), which nothing in the workspace consumed. A regression test pins discriminant and rendered message across clone for the hard-to-clone variants.
Ref rustfs/backlog#1831 (PR2).
马登山