54c11ef28b
test(scanner): bound segment observation diagnostics ( #7240 )
...
* test(scanner): bound segment observation diagnostics
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): observe committed fixture changes during walks
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): validate segment fixture metadata and off state
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
2026-09-06 10:51:30 +08:00
fddecf0afe
test(scanner): diagnose raw enumeration across restarts ( #7228 )
...
* test(scanner): diagnose raw enumeration across process restarts
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): observe the canonical synthetic disk path
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): reject unobserved enumeration budget evidence
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): remove redundant disk path clone
* fix(app): keep list-through header import test-only
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
2026-09-06 10:51:14 +08:00
159dc13548
fix(scanner): bind resumable scans and cache publication coverage ( #7210 )
...
* chore(deps): refresh scanner heal batch dependency baseline
Regenerate compatible lockfile selections before the next implementation
batch. Cargo upgrade leaves direct requirements unchanged.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* chore(deps): refresh profiling dependencies for the next batch
Update hotpath and its macro crate to the compatible patch release before
the next dependency-ready implementation tasks.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(deps): preserve supported hotpath focus expressions
Keep the profiler runtime before its regex-lite compatibility regression.
Track the opt-in validation required to remove this constraint in backlog.
Refs rustfs/backlog#2302 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): require complete publication coverage
Refs rustfs/backlog#2261 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): retain scoped partial coverage across dirty plans
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): keep stable snapshot rescan behavior
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): verify coverage receipts and scan strength
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): use valid modification times in checkpoint fixtures
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): keep maintenance cycles outside dirty bucket scopes
Force complete bucket scope for deep scans and scheduled maintenance while
preserving the existing planner for verified ordinary dirty work.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): refresh scope safety independently of idle backoff
Inspect maintenance on multi-disk startup and refresh changed or failed
evidence even when explicit bitrot configuration disables idle backoff.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): bind bucket cache reuse to scan work requirements
Carry stable scan mode and full-maintenance requirements in the existing
opaque bucket digest before local and remote cache admission. Different
requirements cannot replay a same-cycle Normal cache after root delivery
failure; matching requirements remain reusable for the same intent.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): fence set snapshot reuse with the scan work proof
Prevent same-cycle set publication from replacing freshly scanned maintenance
results with an older Normal aggregate. Recognize uniform completed
maintenance baselines when planning later ordinary dirty-bucket work.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): reproduce same-cycle dirty aggregate replay
Cover a Normal-to-Normal retry with a new dirty bucket generation after
bucket persistence and root delivery failure.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): fence same-cycle caches with full activity coverage
Keep structural baseline identity separate from the full activity coverage
required by bucket admission and set publication. Require complete set
coverage proofs while retaining revision CAS and epoch regression checks.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): supply explicit coverage in publication fixtures
Keep the confirmed-empty namespace fixture authoritative under the required
coverage contract and qualify the bucket cache metadata test type.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): verify joint checkpoint coverage metadata
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): satisfy cache prefix sort lint
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-06 07:27:18 +08:00
2e4ab045b6
test(scanner): add durable checkpoint diagnostics ( #7175 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): add durable checkpoint diagnostics
Refs rustfs/backlog#2260 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 07:34:29 +00:00
80c88a9031
fix(ilm): delete historical null versions by exact identity ( #7109 )
2026-09-04 08:14:47 +08:00
1585308f0f
fix(test): restore #[serial] markers the fallback runner still needs ( #6767 )
2026-08-28 08:41:36 +08:00
a96dd7d289
refactor: migrate consumers off rustfs-common heal/scanner shims ( #6623 )
...
* refactor(ecstore): import heal/scanner contracts crates directly (backlog#1843)
* refactor(heal): import heal/scanner contracts crates directly (backlog#1843)
* refactor(lifecycle): import heal/scanner contracts crates directly (backlog#1843)
* refactor(obs): import heal/scanner contracts crates directly (backlog#1843)
* refactor(protos): import heal/scanner contracts crates directly (backlog#1843)
* refactor(scanner): import heal/scanner contracts crates directly (backlog#1843)
* refactor(rustfs): import heal/scanner contracts crates directly (backlog#1843)
2026-08-26 21:13:18 +08:00
cd1363d519
fix(scanner): restore s3s footprint baseline ( #6486 )
...
chore(scanner): narrow s3s DTO references
2026-08-24 14:30:16 +08:00
0e015360cc
test(scanner): repair post-fence fixtures ( #6478 )
2026-08-24 09:33:21 +08:00
b2e60be647
fix(scanner): fence unknown tier accounting ( #6396 )
2026-08-23 19:28:43 +08:00
76a863b3ea
fix(scanner): unify unknown metadata size accounting ( #6394 )
...
* fix(scanner): unify unknown metadata size accounting
* fix(scanner): preserve restore expiry semantics
* fix(ci): resolve ecstore clippy warnings
* fix(scanner): close lifecycle review gaps
---------
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-08-23 17:28:52 +08:00
9cda615519
fix(scanner): discover sub-quorum heal candidates ( #6384 )
...
* fix(scanner): preserve unversioned heal retries
* fix(scanner): bound orphan heal discovery fallback
* fix(filemeta): fence unsafe heal key components
* fix(scanner): preserve exact overflow heal versions
---------
Co-authored-by: houseme <housemecn@gmail.com >
2026-08-23 16:46:36 +08:00
32cc7c8fcf
fix(heal): coalesce duplicate MRF intents ( #6425 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-08-23 16:45:22 +08:00
5d820df79c
chore(obs): rename ReplicationStats to ReplicationMetricsSnapshot ( #6344 )
...
* chore(obs): ReplicationStats -> ReplicationMetricsSnapshot, BucketReplicationStats -> BucketReplicationMetricsSnapshot
Rename in-obs-crate ReplicationStats and BucketReplicationStats to
ReplicationMetricsSnapshot and BucketReplicationMetricsSnapshot respectively.
No serde impact (these types are Prometheus metric collectors, not serialized).
No external consumers found outside the obs crate.
* cleanup: remove #[serial] annotations from e2e_test, scanner, lifecycle, and object-capacity crates
Remove no-op #[serial] attributes (nextest ignores serial_test) and the
serial_test dependency from four crates. All tests already use temp_env
for env-var isolation, making #[serial] purely redundant.
Crates cleaned:
- e2e_test (37 annotations, 9 imports, removed serial_test dep)
- rustfs-scanner (115 annotations across 7 files, removed serial_test dep)
- rustfs-lifecycle (46 annotations, removed serial_test dep)
- rustfs-object-capacity (38 annotations, removed serial_test dep)
Also converted scanner/tests/lifecycle_integration_test.rs
with_forced_immediate_enqueue_timeout helper from unsafe raw
env::set_var/remove_var to temp_env::async_with_vars for proper
isolation, and added async_closure feature to scanner's temp-env dep.
* fix(lifecycle): restore #[serial] on 2 tests that read env vars without temp_env
eval_inner_expires_latest_object_after_days_due and
eval_inner_does_not_panic_on_many_equal_due_events call eval_inner()
which reads ENV_ILM_PROCESS_TIME via std::env::var(). Without #[serial]
they race with other tests that set these vars via temp_env.
* style: cargo fmt
2026-08-22 00:30:30 +00:00
762919b1ba
perf(scanner): reduce per-object allocation churn ( #6318 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-20 16:47:48 +00:00
a247c79359
perf(heal): trim scanner and heal queue hot paths ( #6307 )
...
Cache heal queue dedup keys, avoid retry request double construction, clear task aliases after terminal completion, and age out stale scanner pending-heal ledger entries during retry sweeps.
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-20 20:28:22 +08:00
129677f0b3
refactor(scanner): split scanner_folder item actions and ledger ( #6302 )
...
* refactor(scanner): split scanner_folder item actions and ledger
Split the 6345-line scanner_folder.rs (46% inline tests) into a
canonical scanner_folder.rs + scanner_folder/ module tree with zero
behavior change:
- scanner_folder.rs (~2280): scan constants, alert cooldowns, metric
accounting, resume ordering, tracing helpers, the FolderScanner
struct with failed-object bookkeeping and the scan_folder traversal,
and scan_data_folder
- scanner_folder/item_actions.rs (~890): CachedFolder, the get-size
failure policy, ScannerItem with apply_actions and the heal/ILM
admission helpers
- scanner_folder/ledger.rs (~280): the pending-scanner-heal ledger
methods and their entry helpers (record/prune/clear-for-repaired/
retry)
- scanner_folder/tests.rs (~2950): the inline test module as a child
module
The ScannerItem path used by scanner_io resolves through a root
re-export, and every other crate path is unchanged. Cross-module items
gain pub(super), whose scope equals the old single-module privacy
domain. Code is moved verbatim apart from those markers, per-module
import headers, and rustfmt re-wraps.
Co-Authored-By: heihutu <heihutu@gmail.com >
* fmt
---------
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-20 19:11:55 +08:00
houseme