Commit Graph

17 Commits

Author SHA1 Message Date
Zhengchao An 1585308f0f fix(test): restore #[serial] markers the fallback runner still needs (#6767) 2026-08-28 08:41:36 +08:00
Henry Guo 1590d9107b fix(scanner): rebuild missing usage floor after upgrade (#6624)
* fix(scanner): rebuild missing usage floor after upgrade

* fix(scanner): preserve missing-floor reset across conflicts

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
2026-08-26 12:30:53 +08:00
Zhengchao An ade7e320da test(scanner): align pristine startup fixtures (#6516) 2026-08-24 18:29:47 +08:00
Henry Guo f06b004f2d fix(scanner): clarify follower status 2026-08-24 16:12:29 +08:00
Zhengchao An 170a4c7640 fix(scanner): bootstrap pristine usage baseline (#6471)
* fix(ecstore): fence pool metadata replica updates

* fix(ecstore): block decommission on unsafe pool metadata

* fix(ecstore): block writes after pool metadata save errors

* fix(ecstore): latch pool metadata writes before await

* fix(scanner): bootstrap pristine usage baseline
2026-08-24 14:29:09 +08:00
cxymds eec0e0e056 fix(scanner): fence movement generation publication (#6461)
* feat(scanner): add movement generation fencing

* fix(scanner): prioritize unverified cycle deferral

* feat(ecstore): add scanner publication lease fence

* feat(rpc): add scanner publication lease protocol

* feat(scanner): hold remote leases through usage publish

* test(scanner): cover publication lease fencing

* fix(scanner): fence remote leases across restart and delay

* feat(rpc): fence scanner publication rename writes

* fix(scanner): fence observed cleanup deletes

* fix(proto): qualify lease release test types

* fix(scanner): pin movement notifications

* fix(scanner): clean publication imports

* fix(ecstore): satisfy scanner fence clippy

* refactor(scanner): group wait and publication options

* fix(scanner): satisfy final lint and facade guards

* fix(rpc): resolve facade export conflicts

* fix(ci): remove unused decommission and healing facades

* fix(ci): cfg-gate test-only usage overlay import

* fix(scanner): wake on remote scanner restart
2026-08-24 14:17:35 +08:00
Zhengchao An 0e015360cc test(scanner): repair post-fence fixtures (#6478) 2026-08-24 09:33:21 +08:00
Zhengchao An 99fb77b164 fix(ci): restore main checks (#6469)
* fix(ci): restore main checks

* fix(scanner): bootstrap pristine usage state

* fix(scanner): reject empty usage snapshots
2026-08-24 09:33:00 +08:00
cxymds a8e4b67d99 feat(metrics): expose deferred usage freshness (#6449) 2026-08-23 19:31:01 +08:00
cxymds e196a134cc fix(scanner): fence system metadata publication (#6444)
* feat(scanner): fence usage publication during data movement

* fix(scanner): detect movement refresh state changes

* fix(scanner): fence publication during data movement

* fix(scanner): close movement epoch publication races

* fix(scanner): fence movement-sensitive publication paths

* fix(scanner): fence cache and heal recovery paths

* fix(scanner): carry publication epoch through scan cycle

* fix(scanner): recheck remote cache epoch after save

* fix(scanner): recheck local cache epoch before publish

* fix(scanner): fence data usage writers and baseline

* fix(scanner): expose decommission activity to publication fence

* fix(scanner): release publication gate before reads

* fix(scanner): complete publication fence integration

* fix(scanner): avoid empty usage baseline publication

* chore(scanner): gate test-only helpers

* fix: use decommission canceler in reload test

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 17:28:21 +08:00
cxymds 8dc2537178 fix(scanner): publish per-set usage freshness (#6432)
* fix(scanner): publish partial usage observations

* fix(ecstore): preserve quota baseline across restart

* style: format usage freshness changes

* fix(scanner): correct observational usage arguments

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:44:45 +08:00
cxymds 2fccfdeabe fix(scanner): fence timed out scan cycles (#6352)
* fix(scanner): fence timed out scan cycles

* fix(scanner): cancel scan workers with cycle scope

* fix(scanner): reject persisted timer overflow

* fix(scanner): reject terminal leadership epochs

* fix(scanner): reject trailing cycle state bytes
2026-08-22 14:38:18 +00:00
cxymds 04e1ea227a fix(scanner): isolate corrupt cycle state (#6354)
* fix(scanner): isolate corrupt cycle state

* fix(scanner): preserve newer state during recovery reset

* fix(scanner): fence recovery reset state

* fix(scanner): reject terminal recovery epochs

* fix(scanner): reject trailing cycle state bytes

* fix(scanner): reject terminal leadership epochs

* fix(scanner): retain recovery wake notifications

* fix(scanner): recover from oversized markers
2026-08-22 11:11:48 +00:00
Zhengchao An 5d820df79c chore(obs): rename ReplicationStats to ReplicationMetricsSnapshot (#6344)
* chore(obs): ReplicationStats -> ReplicationMetricsSnapshot, BucketReplicationStats -> BucketReplicationMetricsSnapshot

Rename in-obs-crate ReplicationStats and BucketReplicationStats to
ReplicationMetricsSnapshot and BucketReplicationMetricsSnapshot respectively.
No serde impact (these types are Prometheus metric collectors, not serialized).
No external consumers found outside the obs crate.

* cleanup: remove #[serial] annotations from e2e_test, scanner, lifecycle, and object-capacity crates

Remove no-op #[serial] attributes (nextest ignores serial_test) and the
serial_test dependency from four crates. All tests already use temp_env
for env-var isolation, making #[serial] purely redundant.

Crates cleaned:
- e2e_test (37 annotations, 9 imports, removed serial_test dep)
- rustfs-scanner (115 annotations across 7 files, removed serial_test dep)
- rustfs-lifecycle (46 annotations, removed serial_test dep)
- rustfs-object-capacity (38 annotations, removed serial_test dep)

Also converted scanner/tests/lifecycle_integration_test.rs
with_forced_immediate_enqueue_timeout helper from unsafe raw
env::set_var/remove_var to temp_env::async_with_vars for proper
isolation, and added async_closure feature to scanner's temp-env dep.

* fix(lifecycle): restore #[serial] on 2 tests that read env vars without temp_env

eval_inner_expires_latest_object_after_days_due and
eval_inner_does_not_panic_on_many_equal_due_events call eval_inner()
which reads ENV_ILM_PROCESS_TIME via std::env::var(). Without #[serial]
they race with other tests that set these vars via temp_env.

* style: cargo fmt
2026-08-22 00:30:30 +00:00
cxymds adb90fc6e1 fix(scanner): defer usage publication during pool recovery (#6333)
* fix(scanner): defer usage publication during pool recovery

* fix(scanner): preserve metrics when publication is deferred

* fix(scanner): route test types through storage boundary

* fix(scanner): keep cache floor deferred during movement
2026-08-21 17:32:59 +08:00
Henry Guo 105b6fbfde fix(scanner): honor explicit cycle cadence (#6313)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-20 23:35:03 +08:00
houseme 095bf34086 refactor(scanner): split scanner.rs cycle/leadership/persist children (#6305)
Split the 8178-line scanner.rs (48% inline tests) into a canonical
scanner.rs + scanner/ module tree with zero behavior change:

- scanner.rs (~2140): cycle constants, schedule status, budget/config
  helpers, startup, maintenance features, the two run loops, and
  cycle-result finalization
- scanner/activity.rs (~770): wake/backoff policy and scanner activity
  observation (probing, generations, topology digest)
- scanner/heal_info.rs (~110): the background-heal info object
- scanner/cycle_state.rs (~500): cycle-state codec, persisted usage
  floors, and cycle-state persistence
- scanner/leadership.rs (~360): leader-lock claiming, usage-epoch
  fencing, and lock-loss handling
- scanner/usage_store.rs (~480): the CAS data-usage store pipeline and
  observed-snapshot cleanup
- scanner/tests.rs (~3920): the inline test module as a child module

All crate paths are unchanged: scanner::BackgroundHealInfo,
scanner::read_background_heal_info, scanner::store_data_usage_in_backend,
and scanner_topology_digest resolve through root re-exports with their
original visibilities, and the pub(crate) surface used by scanner_io and
remote_scanner re-exports at pub(crate). Cross-module items gain
pub(super), whose scope equals the old single-module privacy domain.
Code is moved verbatim apart from those markers, per-module import
headers, and rustfmt re-wraps.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-20 19:12:27 +08:00