Commit Graph

19 Commits

Author SHA1 Message Date
Zhengchao An d31bd3cd10 fix(data-usage): make thin usage-cache types a read-only projection (#5981)
Delete the dead ecstore save_data_usage_cache and the thin
DataUsageCache::marshal_msg it was the only caller of, drop the
Serialize derive (and the dead DataUsageCacheStorage trait with its
save path) from the thin projection types so no write path can exist
outside the scanner's canonical map-encoded writer, and pin the
persisted .usage-cache.bin wire bytes with cross-crate fixture tests
on both the scanner writer and the thin reader.

Refs rustfs/backlog#1828 (T1-T3).
2026-08-12 09:09:09 +00:00
cxymds 6cce3d60bb fix(quota): reject oversized multipart completion (#5958)
* fix(quota): reject oversized multipart completion

* fix(arch): route quota test through app facade
2026-08-11 21:30:05 +08:00
anthonymartin 706a8b6061 fix(scanner): publish bounded observational usage (#5742)
* fix(scanner): publish bounded observational usage

* test(ci): serialize embedded integration ports

* test(cache): isolate generation-change timeout

* fix(scanner): address observational usage review

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: Anthony Martin <949506+anthonymartin@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-07 08:52:34 +08:00
Zhengchao An 4042bc0a5e fix(quota): admit writes against a persisted usage baseline while authoritative usage is unavailable (#5722)
Upgrading from a pre-v2 release leaves only the legacy .usage.json snapshot, which has no completeness marker and is demoted to non-authoritative, so every write to a quota-enabled bucket failed closed with a retryable 503 until the scanner's first complete cycle persisted .usage.v2.json — a production outage on large namespaces (issue #5716).

Quota admission now degrades to the last persisted per-bucket size: normalize_loaded_data_usage returns the pre-discard bucket sizes, the TTL-bounded snapshot cache retains them (carried forward through failed refreshes), and QuotaChecker::get_real_time_usage falls back to that baseline when the authoritative caches miss. The baseline is static between snapshot loads, so hard-quota enforcement is advisory for the duration of the degraded window — strictly tighter than beta.11 (usage treated as 0) and strictly more available than a blanket 503. Buckets absent from every persisted snapshot still fail closed, and removing a bucket's usage from the backend purges the baseline so a recreated bucket cannot inherit the dead incarnation's size.

Refs #5716
2026-08-05 01:42:09 +00:00
Henry Guo d5df66ac4f fix(scanner): require authoritative usage snapshots (#5333) 2026-07-28 07:29:57 +08:00
Henry Guo a63b79004c fix(scanner): make distributed usage convergence authoritative (#5151)
* fix(scanner): make distributed usage cycles authoritative

* fix(scanner): close distributed refresh races

* fix(config): align scanner reload integration

* fix(admin): scope config test helpers

* fix(scanner): harden distributed usage convergence

* fix(scanner): preserve rolling activity compatibility

* fix(admin): expose non-secret optional config values

* fix(scanner): acknowledge distributed dirty usage

* fix(ecstore): make bucket mutations cancellation safe

* fix(scanner): preserve pending dirty acknowledgements

* test(obs): account for superseded scanner metric

* fix(api): reject excess detached bucket mutations

* test: close scanner convergence coverage gaps

* fix(scanner): make path tracking cleanup one-shot

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-07-25 18:45:16 +08:00
Zhengchao An 87128682b0 fix(ecstore): recover data usage snapshots safely (#4979) 2026-07-17 19:02:16 +00:00
Zhengchao An 4589148f48 fix(admin): serve data usage endpoints from scanner snapshot instead of live listing (#4980) 2026-07-18 00:09:02 +08:00
Zhengchao An 627c396649 fix(scanner): allow usage snapshot save when existing timestamp is future-dated beyond clock tolerance (#4982) 2026-07-17 16:04:43 +00:00
houseme 13bdca6762 build(toolchain): switch Rust channel to stable (#4775)
* Change Rust toolchain channel to stable

Signed-off-by: houseme <housemecn@gmail.com>

* style: apply clippy --fix and cargo fix lint suggestions

Run `cargo clippy --fix --all-targets --all-features` and
`cargo fix --lib --all-targets` across the workspace, then resolve the
remaining warnings by hand:

- collapse needless borrows in `format!` args, prefer `?` over explicit
  early returns, and use `.values()` / `.flatten()` iterator adapters
- rewrite the `Md5` scan loop via `manual_flatten` and re-indent the
  `select!` macro body (rustfmt skips macro interiors)
- annotate the intentional dead-code `Md5` inherent methods (constructed
  only by the test factory) with `#[allow(dead_code)]`

Behavior is unchanged.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-12 18:59:43 +08:00
Henry Guo 3f25426534 fix(ecstore): reject incomplete listing usage refreshes (#4698)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-07-11 08:47:49 +08:00
Henry Guo 24d0c09347 fix(admin): keep datausage live refresh stable (#4630)
fix(ecstore): keep fuller data usage overlay

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-07-10 09:17:35 +08:00
Zhengchao An 4f999bb6b8 perf(ecstore): backfill rename_data old size and gate the PUT prelookup (#4598) 2026-07-09 12:08:01 +08:00
Henry Guo 757f9b3b7b fix(admin): refresh accountinfo bucket usage (#4472) 2026-07-08 18:34:11 +08:00
wood 7484a61fa3 feat(compression): show cluster-level compression stat in grafana (#4112) 2026-06-30 19:05:05 +08:00
Zhengchao An 055d3ad825 refactor(ecstore): scope dead code allowances (#4098) 2026-06-30 06:07:45 +08:00
Henry Guo 1e303e5be0 fix(data-usage): refresh versioned usage state (#3969)
fix(data-usage): refresh versioned usage from authoritative state

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-28 07:58:07 +08:00
Zhengchao An 0a5b1b1b3a refactor: consolidate ecstore owner module layout (#3934)
* refactor: shrink ecstore root owner facades

* refactor: remove ecstore core store root shims

* refactor: move ecstore erasure owner modules

* refactor: remove ecstore root rpc facade

* refactor: move ecstore services domain modules
2026-06-27 09:03:20 +08:00
Zhengchao An c6ecfae39e refactor: move ecstore owner layout modules (#3932) 2026-06-27 05:54:25 +08:00