Commit Graph

3 Commits

Author SHA1 Message Date
Zhengchao An 4d22ed4465 perf(capacity): drop per-PUT global lock and per-disk allocation from write dirty-scope (#4933)
perf(capacity): remove per-PUT global lock and per-disk allocation from write dirty-scope

Every successful write recorded its capacity dirty scope by allocating an endpoint/path String per online disk, deduplicating through a HashSet, entering the global dirty-scope Mutex, and — in the app response path — taking a global async RwLock to record the write frequency. Under small-object high concurrency this created a global serialization point and O(disks) allocation on the hot path (https://github.com/rustfs/backlog/issues/1315).

This change makes the steady-state write path allocation-free and lock-free without altering capacity accounting semantics:

- Memoize the per-set dirty scope. Each set resolves its disks' immutable endpoint/path identity lazily into a slot-indexed cache and reuses a shared `Arc<CapacityScope>`; steady-state writes clone the Arc under a read lock instead of rebuilding String/HashSet. The heal path keeps an ad-hoc scope builder because it passes disks in erasure-distribution order rather than physical-slot order.
- Add a monotonic generation to the global dirty-scope registry, advanced only when a non-empty drain removes disks. A set upgrades the global registry mutex only on the first write of each generation and then skips it while the generation is unchanged; the observed generation is read under the registry lock so a concurrent drain forces a re-mark, preventing lost updates. The write commits its bytes before recording the scope, so any drain that could remove the mark is ordered after the commit and the following refresh reads the committed bytes.
- Replace the write-frequency `RwLock<WriteRecord>` with lock-free atomics: per-second CAS buckets, an atomic last-write timestamp, and an atomic total counter. The frequency window and debounce semantics the refresh scheduler relies on are unchanged.

Capacity marking remains a conservative superset of the disks actually written, so admin/scan totals are byte-for-byte identical: extra dirty marks only trigger a re-read of a disk whose usage is unchanged. White-box tests assert the memoized scope equals the previous ad-hoc construction, that the global registry is upgraded exactly once per generation and re-marked after a drain, and that the lock-free write record is exact under concurrent contention.

Ref: https://github.com/rustfs/backlog/issues/1315
2026-07-17 08:29:38 +08:00
Zhengchao An ddb9d8ca3e fix(object-capacity): harden write timing, interval bounds, and scope expiry (#4575)
fix(object-capacity): harden clock handling in write window, background intervals and scope registry

Three low-severity robustness gaps (S32+S33+#35):

- WriteRecord keyed its 60-bucket write window by wall-clock unix
  seconds while the debounce used Instant. An NTP step backwards marked
  recent buckets as future and silently suppressed write-triggered
  refreshes until the wall clock caught up; a forward step aged the
  whole window at once. Bucket keys now derive from a monotonic
  per-record epoch, immune to clock steps.
- Background refresh/metrics intervals were only clamped at zero;
  RUSTFS_CAPACITY_SCHEDULED_INTERVAL=u64::MAX overflowed
  Instant + Duration and panicked the spawned task, silently killing
  scheduled refreshes. Intervals now clamp into [1s, 30 days] via one
  helper with a structured warn.
- record_capacity_scope merged new disks into an expired entry and
  refreshed its recorded_at, resurrecting a scope take_capacity_scope
  would have discarded (and the merge path never pruned). Expired
  entries are now replaced, not merged into.

Ref: rustfs/backlog#1022 (S32+S33+#35 from audit rustfs/backlog#1010)
2026-07-09 05:27:36 +08:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012)
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00