feat(scanner): decouple bucket walkers from disk count (#8539)

* fix(scanner): treat replication status writebacks as status-class hits

A replication status writeback (persist_replication_state_if_current in
Update mode, including the heals the scanner queues during its own walk)
marked the bucket dirty on the namespace path: it advanced the dirty-usage
generation, woke the scanner, reached the durable dirty journal and
entered the activity snapshot. Any replicated bucket therefore made every
scanner cycle Superseded, the authoritative usage baseline was never
certified and usage freshness stayed stale for as long as replication ran
(#8375).

The writeback is now status class: record_status_class_mutation records
only the process-local producer coverage bit and nothing else. There is
no rescan queue for it, because a deployment with an active replication
rule already walks every bucket on every cycle
(ScannerMaintenanceFeatures::needs_regular_cycle) and that walk refreshes
replication_stats. The routing lives in a named, unit-tested function in
rustfs/src/storage/storage_api.rs; tier-expiration cleanup removes data
and stays on the namespace path.

No persisted format, protocol, admin API or configuration changes.

This is phase 1 of docs/architecture/scanner-per-bucket-convergence.md,
which this change adds together with its index entry.

Refs #8375, #8494, rustfs/backlog#2816.

* fix(scanner): refresh replication statistics during maintenance

Refresh compacted prefixes and completed bucket caches during replication maintenance. Bind set execution reuse to the observed replication statistics so a same-cycle retry cannot restore older counters.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(scanner): avoid expanding the legacy S3 DTO surface

Decode the existing replication configuration type from a JSON fixture instead of adding direct S3 DTO references to a new file. Preserve the compacted-prefix and same-cycle assertions and the footprint ratchet.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* perf(scanner): keep raw enumeration bookkeeping linear per entry

The page writer inserted into its building page with a push-and-sort on
every entry and committed each page through a clone. Insert in sorted
position and move the committed page into the index instead.

Add a regression test that records 20,000 entries in one prefix and pins
the cost to one hash per entry and well under a second, so the quadratic
re-normalization removed by #8114 (rustfs#8113) cannot return once pacing
builds on this walker.

* feat(scanner): decouple bucket walkers from disk count

Add the scanner key walkers_per_disk (RUSTFS_SCANNER_WALKERS_PER_DISK,
1 through 16, default 1). Each local coordinator disk of a set spawns that
many bucket walkers; remote disks keep one. The walker count is what
max_concurrent_disk_scans caps and the disk count stays the floor.

The walkers of one disk share its scanner admission through
LocalDiskAdmissionShare: the first walker to start a bucket takes the
per-disk permit and the last to finish releases it, so a peer's remote
request still sees one scan session per physical disk. Without the share
the second walker was rejected as busy and retired.

The disk_concurrency_budget log line reports walkers_per_disk, walkers and
the reason for the effective limit. The default stays 1 because no local
disk reports rotational media today; the design keeps a higher flash
default as the intent.

* docs(scanner): document walkers_per_disk in the Helm chart and configuration manuals

RUSTFS_SCANNER_WALKERS_PER_DISK was only described in the scanner runtime
controls table. Add it to the Helm chart (values, configmap rendering,
README table, template test), the configuration crate's environment
reference with operator guidance, and a short section in the operations
manual on when to raise it and how to confirm it took effect.

---------

Co-authored-by: Hauser <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

scripts/test_helm_templates.sh

@@ -180,6 +180,7 @@ scanner_config_output=$(render_chart \
   --set config.rustfs.scanner.cache_save_timeout_secs=30 \
   --set config.rustfs.scanner.max_concurrent_set_scans=2 \
   --set config.rustfs.scanner.max_concurrent_disk_scans=1 \
+  --set config.rustfs.scanner.walkers_per_disk=2 \
   --set config.rustfs.scanner.yield_every_n_objects=128 \
   --set config.rustfs.scanner.alert_excess_versions=100 \
   --set config.rustfs.scanner.alert_excess_version_size=1099511627776 \
@@ -197,6 +198,7 @@ grep -q 'RUSTFS_SCANNER_IDLE_MODE: "false"' <<<"$scanner_config_output"
 grep -q 'RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS: "30"' <<<"$scanner_config_output"
 grep -q 'RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS: "2"' <<<"$scanner_config_output"
 grep -q 'RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS: "1"' <<<"$scanner_config_output"
+grep -q 'RUSTFS_SCANNER_WALKERS_PER_DISK: "2"' <<<"$scanner_config_output"
 grep -q 'RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS: "128"' <<<"$scanner_config_output"
 grep -q 'RUSTFS_SCANNER_ALERT_EXCESS_VERSIONS: "100"' <<<"$scanner_config_output"
 grep -q 'RUSTFS_SCANNER_ALERT_EXCESS_VERSION_SIZE: "1099511627776"' <<<"$scanner_config_output"