mirror of
https://github.com/rustfs/rustfs.git
synced 2026-11-01 08:40:23 +00:00
Chris An
b8ee0301e9
* feat(scanner): bound first-build enumeration with the cycle budget Charge raw directory entries and their name bytes, and let one bucket scan hold a child budget that stops its own token while charging the cycle. A count budget also bounds one directory listing, so a wide first build can no longer finish a full read_dir before the first object is charged. The new stop reason maps onto the persisted Directories variant so older readers keep decoding checkpoints. A directory with queued children keeps its index until they end, and every index on the active walk path is saved as an optional map field: losing either one would recharge its replay and stop a fixed budget from advancing. * feat(scanner): keep bucket service order across restarts Save each set's first-in-first-out bucket order as an optional map field of its usage cache, including after an incomplete cycle, which otherwise writes nothing. Durable age now outranks the dirty-first and leader-local cohort orders, which only break its ties, so a restart cannot hand the budget back to the same first buckets. Failed turns add visible debt without moving a bucket forward; a budget stop or a resource veto adds none. A bucket that spends its quantum and saves forward motion returns to the queue tail within the cycle. The order authorizes nothing: another movement epoch or a corrupt value starts a new one, and a recreated bucket joins as a new arrival. * test(scanner): cover quantum handoff and saved service order Run the real worker queue with injected bucket shares and verify a spent bucket yields to waiters, a share without durable motion ends its turn, and the saved order keeps certified coverage while rejecting older owners. * docs(scanner): describe budgeted first build and service order * test(scanner): pin index retention and requeue motion Bound the nested restart fixture tightly enough that dropping a directory index fails it, and cover a repeated share whose frontier never moves. * fix(scanner): keep service generations and owner stops intact Queue a new bucket behind waiters but ahead of buckets that already had a turn, matching the leader-local cohort, and keep the set cache untouched when the owner was cancelled so an interrupted cycle cannot advance its revision. The order holds one record per inventory bucket, like the set cache's entries, so a truncating bound that could starve the long-waiting buckets is removed. Publish the incomplete scope before saving so a contended lock cannot delay it. * ci(e2e): refresh the full selection digests * fix(scanner): correct a comment spelling * fix(scanner): separate service debt from vetoes and stale owners Keep a resource veto and a remote count-budget stop out of failure debt, settle the claim before a requeue exposes the bucket to another worker, and stamp the service order with its leader so an older owner cannot replace it after an incomplete save. The checkpoint fixture's budget now covers both service orders. * fix(ecstore): create parents before resource-scoped writes #9064 moved parent creation into the lazy writer, so a write that holds a resource unit reached the native open without its parent directory and failed with FileNotFound on nested paths. * fix(e2e): retry SlowDown codes during upgrade write readiness The rolling-upgrade readiness probe only retried 503 ServiceUnavailable, so a transient 503 SlowDown from a still-recovering node failed the four_node_rolling_upgrade test immediately. Accept SlowDown and SlowDownRead alongside ServiceUnavailable, matching the existing chaos test transient-error set.
RustFS Scanner
RustFS Scanner is the background maintenance scan loop. It handles usage accounting, lifecycle expiry and transition admission, bucket replication repair admission, scanner-driven heal/bitrot checks, and namespace alerts.
For operator-facing runtime controls, status fields, and tuning workflows, see Scanner Runtime Controls. For repeatable scanner-pressure validation, see Scanner Benchmark Runbook.
Chinese documentation is available in README.zh-CN.md.
Development
Build
cargo build --package rustfs-scanner
Test
cargo test --package rustfs-scanner
License
Apache License 2.0