* fix(scanner): require root publication proof before dirty ack
Bind ACK expectations to validated scan candidates and confirm the actual primary-root revision and readback. Retain saved outcomes and dirty responsibility when stronger evidence is unavailable. Isolate CAS attempt confirmation and invalidate proof after scope mutations.
Revalidate observed candidate reuse before issuing a new publication proof, preserve the exact validated authoritative baseline work digest, and settle fixture commit tails before stable maintenance scans. Keep scoped ACK production disabled.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix: restore ILM transition and lifecycle validation (#7316)
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
feat(scanner): retain bounded bootstrap admission fairness
Keep a leader-local bounded cohort across scanner retries and preserve
waiting bucket priority during dirty arrivals and capacity overflow.
Order source permits in the dispatcher without changing result identity,
parent budgets, explicit cycle timing, or persistent coverage evidence.
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Split the 5369-line scanner_io.rs (39% inline tests) into a canonical
scanner_io.rs + scanner_io/ module tree with zero behavior change:
- scanner_io.rs (~660): constants, metadata-error constructors, the
bucket scan plan, cycle-status classification helpers, the ScannerIO /
ScannerIOCache / ScannerIODisk traits, and ScannerCycleResult
- scanner_io/dirty_usage.rs (~300): process-wide dirty-usage statics
and the acknowledgment protocol
- scanner_io/guards.rs (~270): concurrency gauges and RAII guards
- scanner_io/cache.rs (~410): scanner cache locks and the snapshot
persist/publish path
- scanner_io/io_cycle.rs (~390), io_cache.rs (~1160), io_disk.rs
(~230): the ECStore / SetDisks / Disk trait implementations
- scanner_io/publish_gate_tests.rs (~750) and tests.rs (~1340): the two
inline test modules as child modules
All crate paths are unchanged: the lib.rs scanner_io re-exports and
every crate::scanner_io:: consumer (scanner.rs, remote_scanner,
scanner_folder, and cross-crate rustfs users) resolve through root
re-exports with their original visibilities (pub stays pub, pub(crate)
stays pub(crate)). Cross-module items gain pub(super), whose scope
equals the old single-module privacy domain. Code is moved verbatim
apart from those markers, per-module import headers, and rustfmt
re-wraps.
The logging-guardrail nsscanner_disk skip-set_disks rule now points at
scanner_io/io_disk.rs where the function moved; the pattern and
thresholds are unchanged.
Co-authored-by: heihutu <heihutu@gmail.com>
feat(scanner): expose prefix-level bucket usage via admin API
The scanner's per-bucket, per-set usage caches already hold a path-keyed
prefix tree, but dui() flattened it only to bucket names — consoles and
operators had no way to ask "what does this prefix hold" without an S3
listing sweep (rustfs/backlog#1872, MinIO loadPrefixUsageFromBackend
parity).
Add:
- data-usage: prefix_usage_in_cache — a shared aggregation over the
entry map (arbitrary prefix, full counters, one-level sub-prefix
breakdown with names recovered from the literal-path cache keys),
hardened like the scanner's checked flatten: cycles, dangling child
links, over-deep trees, and overflowing counters yield None rather
than unbounded recursion or wrapped totals.
- ecstore: ECStore::all_set_disks — iterate every erasure set so a
query can read each set's own cache copy; the hash-routed store path
would always land on one set.
- scanner: bucket_prefix_usage — per-set loads (5s budget each, a slow
set degrades to not-reporting instead of stalling the caller),
merged across sets with partial/compacted/truncated flags, served
from a bounded 30s cache (128 entries, hard-capped) that bucket
writes invalidate through the dirty-usage hook.
- admin: GET /rustfs/admin/v3/usage/{bucket}?prefix=&max-entries=
behind the same any-of gate as datausageinfo (DataUsageInfoAdminAction
OR ListBucketAction), rejecting unknown query parameters and
clamping max-entries to 1..=10000. Route registered in the policy
table (deferred MultipleActions, matching datausageinfo) and the
route matrix test.
Closesrustfs/backlog#1872.
Co-authored-by: heihutu <heihutu@gmail.com>
SizeSummary::tier_stats was populated for every scanned object but
apply_scanner_size_summary dropped it, so per-tier usage never reached
DataUsageInfo. Wire it through the same merge chain repl_target_stats
already uses, up to DataUsageInfo::tier_stats.
DataUsageEntry used the derived MessagePack encoding, which serialises
structs as arrays: appending a field turns the whole cache into a decode
error for older readers, so mixed-version nodes would invalidate each
other's cache every scan cycle. Give it the same hand-written
map-encoded Serialize DataUsageCacheInfo already carries, and record the
invariant in AGENTS.md.
Widen TierStats counters from i32 to u64 so a tier past 2^31 versions
cannot make checked_merge reject an entire usage snapshot, and drop the
duplicate TierStats/AllTierStats definitions in the scanner crate in
favour of the data-usage ones.
* fix(filemeta): add state-aware file info validation
* fix(filemeta): validate shard arithmetic and delete paths
* fix(ecstore): add fallible erasure construction
* fix(ecstore): resolve storage parity per pool
* fix(storage): report heterogeneous erasure layouts
* fix(admin): publish prepared storage config atomically
* fix(storage): harden per-pool parity boundaries
* fix(storage): address pre-PR validation findings
* test(ci): fix strict-topology validation fixtures
* fix(heal): preserve delete markers during repair
* refactor(filemeta): drop unused ValidatedFileInfo witness
ValidatedFileInfo wrapped an unread `_file_info` reference alongside an `Option<ValidatedErasureLayout>`, but only the layout was ever consumed. Return the layout directly from `FileInfo::validate` so the sole production consumer (`LocalDisk::check_parts`) and the two unit tests read it without the extra witness type and lifetime.
No behavior change.
* fix(filemeta): keep compressed and MinIO-migrated tiered objects readable
The new decode-path validation rejected several legitimate on-disk shapes that older RustFS and MinIO-migrated data carry, turning readable objects into FileCorrupt:
- Compressed objects written with an unknown upload size persist a negative per-part actual_size (the documented "unknown size" sentinel that ObjectInfo::get_actual_size already tolerates). validate_collection_contents rejected it via usize::try_from; now a negative actual_size skips shard validation and only real, non-negative sizes are checked.
- MinIO-migrated objects transitioned to a versioned remote tier store the tier version id as a UUID string, not 16 raw bytes. MetaObject::into_fileinfo returned FileCorrupt (main tolerated it as None), making all versions of the object unreadable; MetaDeleteMarker free-version records took a Some(nil) sentinel path with the same effect, which also breaks free-version expiry (remote-tier leak). Both now decode through a shared transitioned_version_id_from_meta_sys helper: 16 raw bytes or a UUID string are accepted, anything else is tolerated as None instead of failing the read.
Regression tests updated to assert the readable/compat behavior, with new tests covering MinIO string-form recovery.
* fix(scanner): build the delete-marker test fixture without erasure geometry
get_size_counts_delete_markers_separately_from_versions built its delete marker with `FileInfo::new(object, 1, 1)`, which attaches erasure geometry (data=1/parity=1/distribution). This PR classifies versions by shape via `is_storage_delete_marker()` (no geometry) rather than the raw `deleted` flag, so a geometry-bearing "delete marker" is correctly serialized as a purge-pending payload Object and counted as a version — CI saw summary.versions=3, expected 2.
Real delete markers carry no erasure geometry (delete paths build them as `FileInfo { deleted: true, ..Default::default() }`), so construct the fixture the same way. It then classifies as a storage delete marker and the counts (versions=2, delete_markers=1) hold. This keeps the PR's more-correct classification, which prevents a purge-pending object's geometry from being dropped when serialized as a bare delete marker.
* docs(changelog): note per-pool parity fix and storage-class startup upgrade caveat
Records the #4801 per-pool erasure parity fix under Fixed, and documents the upgrade behavior where a persisted storage class that a small or heterogeneous pool cannot satisfy now fails startup — with the RUSTFS_STORAGE_CLASS_STANDARD recovery steps. Docs-only; covers R4 from the on-disk compatibility audit.
* fix(heal): report parity from erasure geometry, not is_valid()
heal_object set HealResultItem.parity_blocks via `if lfi.is_valid()`, which was missed by the migration of the other quorum/metadata predicates. With the new `is_valid()` semantics (full payload validation; delete markers now return false), a delete marker or a geometry-bearing version with a benign collection quirk would misreport parity as the pool default instead of its own. Use `has_valid_erasure_geometry()` — the narrow "does this carry erasure geometry" predicate the rest of the migration uses — so reporting matches the object's actual layout. Reporting-only; no data-path change.
* fix(filemeta): do not silently serialize a non-canonical deleted FileInfo as an Object
`From<FileInfo> for FileMetaVersion` classifies by `is_storage_delete_marker()` (shape), which correctly routes canonical delete markers to Delete and purge-pending payloads (deleted=true with real erasure geometry) to Object. But a `deleted` FileInfo that is neither a canonical marker nor a valid erasure payload would silently serialize as a zero-geometry MetaObject that later fails `validate_for_metadata_read`. Write paths validate first (`validate_for_erasure_write` / `validate_for_metadata_read`), so this is a caller bug; `From` is infallible, so surface it with a structured `warn!` on the malformed branch instead of writing corrupt metadata silently. Legitimate purge-pending objects (valid geometry) are unaffected — the guard only fires for `deleted && !has_valid_erasure_geometry()`.
* test(filemeta): assert real historical xl.meta versions pass metadata-read validation
Empirical companion to the code-reasoned decode-tolerance invariants (docs/architecture/erasure-coding.md §11) and the rolling-upgrade / MinIO-migration compatibility concern: the tightened `validate_for_metadata_read` runs on every local disk read and peer-RPC-decoded FileInfo, so it must accept every version of real historically-written xl.meta, never reject it as FileCorrupt.
Loads five real fixtures — MinIO small-inline, MinIO versioned (two object versions + a delete marker), MinIO large multipart, a legacy V1 (xl.json-derived) object, and a legacy meta_ver 2 object — decodes every version with parts materialized, and asserts validate_for_metadata_read() is Ok for each. Reverting the tolerant handling (delete-marker shape, legacy per-part checksums, string/short transitioned-versionID, negative actual_size) turns this red.
* fix(ci): remove duplicate storage test re-exports
---------
Co-authored-by: overtrue <anzhengchao@gmail.com>
* fix(scanner): back off clean single-disk cycles
* fix(scanner): extend idle backoff across erasure clusters
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
versions_scanned for both the scanner and ILM collectors was read from
the Lifecycle work source's `checked` counter, which is never recorded on
the production scan path — so rustfs_scanner_versions_scanned_total and
rustfs_ilm_versions_scanned_total sat at zero even while objects_scanned
climbed. The two metrics also have distinct intended meanings that were
conflated: "versions scanned" (all versions, any bucket) vs "versions
checked for ILM actions" (lifecycle-configured buckets only).
Add a lifetime `versions_scanned` counter recorded for every version the
scanner walks (independent of ILM), and record the Lifecycle source's
`checked` counter from the ILM evaluator so the ILM metric reflects the
real checked subset. The scanner collector now reports total scanned
versions; the ILM collector keeps the ILM-checked subset.
Closes backlog#995 (OBS-09).
Co-authored-by: heihutu <heihutu@gmail.com>
houseme