086ee8e48a
fix(scanner): require root publication proof before dirty ack ( #7297 )
...
* fix(scanner): require root publication proof before dirty ack
Bind ACK expectations to validated scan candidates and confirm the actual primary-root revision and readback. Retain saved outcomes and dirty responsibility when stronger evidence is unavailable. Isolate CAS attempt confirmation and invalidate proof after scope mutations.
Revalidate observed candidate reuse before issuing a new publication proof, preserve the exact validated authoritative baseline work digest, and settle fixture commit tails before stable maintenance scans. Keep scoped ACK production disabled.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix: restore ILM transition and lifecycle validation (#7316 )
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-06 23:25:40 +08:00
3d46ed312a
feat(scanner): reuse complete observed scan candidates ( #7206 )
2026-09-06 17:52:52 +08:00
0ee5408b94
feat(scanner): preserve bounded bootstrap admission fairness ( #7260 )
...
feat(scanner): retain bounded bootstrap admission fairness
Keep a leader-local bounded cohort across scanner retries and preserve
waiting bucket priority during dirty arrivals and capacity overflow.
Order source permits in the dispatcher without changing result identity,
parent budgets, explicit cycle timing, or persistent coverage evidence.
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-06 16:45:57 +08:00
159dc13548
fix(scanner): bind resumable scans and cache publication coverage ( #7210 )
...
* chore(deps): refresh scanner heal batch dependency baseline
Regenerate compatible lockfile selections before the next implementation
batch. Cargo upgrade leaves direct requirements unchanged.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* chore(deps): refresh profiling dependencies for the next batch
Update hotpath and its macro crate to the compatible patch release before
the next dependency-ready implementation tasks.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(deps): preserve supported hotpath focus expressions
Keep the profiler runtime before its regex-lite compatibility regression.
Track the opt-in validation required to remove this constraint in backlog.
Refs rustfs/backlog#2302 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): require complete publication coverage
Refs rustfs/backlog#2261 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): retain scoped partial coverage across dirty plans
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): keep stable snapshot rescan behavior
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): verify coverage receipts and scan strength
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): use valid modification times in checkpoint fixtures
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): keep maintenance cycles outside dirty bucket scopes
Force complete bucket scope for deep scans and scheduled maintenance while
preserving the existing planner for verified ordinary dirty work.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): refresh scope safety independently of idle backoff
Inspect maintenance on multi-disk startup and refresh changed or failed
evidence even when explicit bitrot configuration disables idle backoff.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): bind bucket cache reuse to scan work requirements
Carry stable scan mode and full-maintenance requirements in the existing
opaque bucket digest before local and remote cache admission. Different
requirements cannot replay a same-cycle Normal cache after root delivery
failure; matching requirements remain reusable for the same intent.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): fence set snapshot reuse with the scan work proof
Prevent same-cycle set publication from replacing freshly scanned maintenance
results with an older Normal aggregate. Recognize uniform completed
maintenance baselines when planning later ordinary dirty-bucket work.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): reproduce same-cycle dirty aggregate replay
Cover a Normal-to-Normal retry with a new dirty bucket generation after
bucket persistence and root delivery failure.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): fence same-cycle caches with full activity coverage
Keep structural baseline identity separate from the full activity coverage
required by bucket admission and set publication. Require complete set
coverage proofs while retaining revision CAS and epoch regression checks.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): supply explicit coverage in publication fixtures
Keep the confirmed-empty namespace fixture authoritative under the required
coverage contract and qualify the bucket cache metadata test type.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): verify joint checkpoint coverage metadata
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): satisfy cache prefix sort lint
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-06 07:27:18 +08:00
e2a921bc16
fix(storage): harden ODM and scanner publication ( #7187 )
...
* fix(storage): harden ODM and scanner publication
* fix(app): simplify absent SSE configuration matching
* test(heal): settle PUT rename tails before disk-wipe fixtures
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(ci): satisfy new clippy lints
* style(scanner): order merged test imports
* fix(scanner): invalidate bucket work after namespace completion
* fix(scanner): fence cached snapshots by scan execution
---------
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 13:47:12 +00:00
2d159635ed
feat(scanner): plan dirty bucket cache refreshes ( #7146 )
...
* feat(scanner): plan dirty bucket cache refreshes
* fix(scanner): route peer snapshot through storage boundary
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: cxymds <cxymds@gmail.com >
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 16:32:02 +08:00
923bde6904
feat(scanner): prepare scoped bucket cache scans ( #7136 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-09-04 15:09:58 +00:00
62e66baf89
feat(scanner): add storage seam ( #7105 )
2026-09-03 20:31:59 +08:00
c235f7c05d
fix(scanner): retain usage across transient peer failures ( #6859 )
...
* test(scanner): cover bucket drive guard lifecycle
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): recover usage floor from fenced backups
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): retry transient activity probes
Retry one failed scanner activity probe after a bounded reconnect when the failure is transport-like or timed out. Keep protocol and response validation failures fail-closed.
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): retain post-scan observations
Preserve a complete scanner walk as a non-converged observation when the final activity probe is unavailable. Advance the cycle as partial without acknowledging dirty usage.\n\nCo-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): classify publication lease deferrals
Distinguish persistence budget and lease deadline deferrals from unavailable activity baselines, and ensure lease-gate deferrals update usage metrics. Keep the fixed lease gate fail-closed while storage-owned commit scope work remains pending.
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): recover usage floor from fenced backups
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): preserve publication lease defer reasons
Keep lease expiry and release failures distinct from activity baseline failures so scanner freshness metrics and cycle outcomes identify the publication barrier that blocked progress. Preserve fail-closed behavior.
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): reuse recovered usage baseline for publication
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): fence legacy usage floor fallback
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(scanner): use typed activity timeout error
---------
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-30 02:54:20 +08:00
eec0e0e056
fix(scanner): fence movement generation publication ( #6461 )
...
* feat(scanner): add movement generation fencing
* fix(scanner): prioritize unverified cycle deferral
* feat(ecstore): add scanner publication lease fence
* feat(rpc): add scanner publication lease protocol
* feat(scanner): hold remote leases through usage publish
* test(scanner): cover publication lease fencing
* fix(scanner): fence remote leases across restart and delay
* feat(rpc): fence scanner publication rename writes
* fix(scanner): fence observed cleanup deletes
* fix(proto): qualify lease release test types
* fix(scanner): pin movement notifications
* fix(scanner): clean publication imports
* fix(ecstore): satisfy scanner fence clippy
* refactor(scanner): group wait and publication options
* fix(scanner): satisfy final lint and facade guards
* fix(rpc): resolve facade export conflicts
* fix(ci): remove unused decommission and healing facades
* fix(ci): cfg-gate test-only usage overlay import
* fix(scanner): wake on remote scanner restart
2026-08-24 14:17:35 +08:00
b2e60be647
fix(scanner): fence unknown tier accounting ( #6396 )
2026-08-23 19:28:43 +08:00
e196a134cc
fix(scanner): fence system metadata publication ( #6444 )
...
* feat(scanner): fence usage publication during data movement
* fix(scanner): detect movement refresh state changes
* fix(scanner): fence publication during data movement
* fix(scanner): close movement epoch publication races
* fix(scanner): fence movement-sensitive publication paths
* fix(scanner): fence cache and heal recovery paths
* fix(scanner): carry publication epoch through scan cycle
* fix(scanner): recheck remote cache epoch after save
* fix(scanner): recheck local cache epoch before publish
* fix(scanner): fence data usage writers and baseline
* fix(scanner): expose decommission activity to publication fence
* fix(scanner): release publication gate before reads
* fix(scanner): complete publication fence integration
* fix(scanner): avoid empty usage baseline publication
* chore(scanner): gate test-only helpers
* fix: use decommission canceler in reload test
---------
Co-authored-by: houseme <housemecn@gmail.com >
2026-08-23 17:28:21 +08:00
8dc2537178
fix(scanner): publish per-set usage freshness ( #6432 )
...
* fix(scanner): publish partial usage observations
* fix(ecstore): preserve quota baseline across restart
* style: format usage freshness changes
* fix(scanner): correct observational usage arguments
---------
Co-authored-by: houseme <housemecn@gmail.com >
2026-08-23 16:44:45 +08:00
2fccfdeabe
fix(scanner): fence timed out scan cycles ( #6352 )
...
* fix(scanner): fence timed out scan cycles
* fix(scanner): cancel scan workers with cycle scope
* fix(scanner): reject persisted timer overflow
* fix(scanner): reject terminal leadership epochs
* fix(scanner): reject trailing cycle state bytes
2026-08-22 14:38:18 +00:00
adb90fc6e1
fix(scanner): defer usage publication during pool recovery ( #6333 )
...
* fix(scanner): defer usage publication during pool recovery
* fix(scanner): preserve metrics when publication is deferred
* fix(scanner): route test types through storage boundary
* fix(scanner): keep cache floor deferred during movement
2026-08-21 17:32:59 +08:00
769e66511f
refactor(scanner): split scanner_io.rs into child modules ( #6294 )
...
Split the 5369-line scanner_io.rs (39% inline tests) into a canonical
scanner_io.rs + scanner_io/ module tree with zero behavior change:
- scanner_io.rs (~660): constants, metadata-error constructors, the
bucket scan plan, cycle-status classification helpers, the ScannerIO /
ScannerIOCache / ScannerIODisk traits, and ScannerCycleResult
- scanner_io/dirty_usage.rs (~300): process-wide dirty-usage statics
and the acknowledgment protocol
- scanner_io/guards.rs (~270): concurrency gauges and RAII guards
- scanner_io/cache.rs (~410): scanner cache locks and the snapshot
persist/publish path
- scanner_io/io_cycle.rs (~390), io_cache.rs (~1160), io_disk.rs
(~230): the ECStore / SetDisks / Disk trait implementations
- scanner_io/publish_gate_tests.rs (~750) and tests.rs (~1340): the two
inline test modules as child modules
All crate paths are unchanged: the lib.rs scanner_io re-exports and
every crate::scanner_io:: consumer (scanner.rs, remote_scanner,
scanner_folder, and cross-crate rustfs users) resolve through root
re-exports with their original visibilities (pub stays pub, pub(crate)
stays pub(crate)). Cross-module items gain pub(super), whose scope
equals the old single-module privacy domain. Code is moved verbatim
apart from those markers, per-module import headers, and rustfmt
re-wraps.
The logging-guardrail nsscanner_disk skip-set_disks rule now points at
scanner_io/io_disk.rs where the function moved; the pattern and
thresholds are unchanged.
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-20 12:18:45 +08:00
houseme