Commit Graph

229 Commits

Author SHA1 Message Date
houseme 6ab8636734 fix(obs): count scanned versions independent of ILM (#4516)
versions_scanned for both the scanner and ILM collectors was read from
the Lifecycle work source's `checked` counter, which is never recorded on
the production scan path — so rustfs_scanner_versions_scanned_total and
rustfs_ilm_versions_scanned_total sat at zero even while objects_scanned
climbed. The two metrics also have distinct intended meanings that were
conflated: "versions scanned" (all versions, any bucket) vs "versions
checked for ILM actions" (lifecycle-configured buckets only).

Add a lifetime `versions_scanned` counter recorded for every version the
scanner walks (independent of ILM), and record the Lifecycle source's
`checked` counter from the ILM evaluator so the ILM metric reflects the
real checked subset. The scanner collector now reports total scanned
versions; the ILM collector keeps the ILM-checked subset.

Closes backlog#995 (OBS-09).

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:50:51 +00:00
houseme c9292688d3 fix(obs): wire dial9 runtime telemetry (#4491)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:16:06 +00:00
houseme 6cb47049e8 fix(obs): isolate process sampler windows (#4492)
* fix(obs): isolate process sampler windows

Refs rustfs/backlog#1004
Refs rustfs/backlog#986

- add a reusable ProcessSampler so callers can own independent sysinfo refresh windows
- wire separate sampler instances for obs metrics scheduling and memory observability
- keep compatibility helpers while avoiding cross-task CPU and disk delta interference

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(obs): import process sampler bundle helper

Refs rustfs/backlog#1004
Refs rustfs/backlog#986

- import collect_process_metric_bundle_with in the metrics scheduler
- drop the stale collect_process_metric_bundle import after switching scheduler sampling to independent process samplers

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(obs): move process sampler into blocking task

Refs rustfs/backlog#1004
Refs rustfs/backlog#986

- move the memory observability process sampler into the spawn_blocking closure
- satisfy the closure static lifetime required by tokio while keeping the isolated sampler design intact

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 15:51:34 +00:00
houseme 83bacd5b84 fix(obs): harden cleaner file matching (#4486)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 22:44:34 +08:00
houseme 87044b2378 fix(obs): tighten low-risk telemetry correctness (#4483)
Refs rustfs/backlog#1007
Refs rustfs/backlog#986

- respect explicit log-target directives when injecting noisy-crate suppressions
- fix daily rotation day-boundary checks and add a short retry cooldown after failed rotations
- preserve recorder metadata across label variants and serialize gauge updates before export
- correct profiling target_env reporting, trace all=true parsing, cleaner accounting/docs, and legacy system interval rounding

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 14:06:19 +00:00
houseme e7cfc510ec fix(obs): correct gpu collector coverage (#4482)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 13:35:49 +00:00
houseme f0bf8cfe03 fix(obs): hide unwired request metrics (#4481)
Refs rustfs/backlog#1006

- aggregate request traffic samples per type to avoid future counter collisions
- keep request schema and collector crate-internal until a production stats source exists
- preserve focused regression coverage for the internal request collector logic

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 13:08:57 +00:00
houseme 0cb76e9420 fix(obs): harden metrics scheduler (#4479)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 13:04:42 +00:00
houseme 4fe0789fb1 fix(obs): honor otlp export contracts (#4477)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 12:41:18 +00:00
houseme 1eb393cab1 fix(obs): align metrics schema and collector contracts (#4476)
fix(obs): align schema and collector contracts

Refs rustfs/backlog#1005

- align obs schema descriptors with emitted labels and metric types
- fix bucket traffic help text and TTFB bucket descriptor semantics
- add regression tests for drive, network host, bucket, and node bucket contracts

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 11:23:23 +00:00
houseme 3ddade24f2 fix(obs): validate numeric env settings (#4474)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 10:42:09 +00:00
houseme 718c051ff3 fix(obs): export cluster usage staleness (#4467)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 09:44:05 +00:00
houseme dea53616a4 fix(obs): correct versions scanned metrics (#4443)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
2026-07-08 09:12:02 +00:00
houseme a30a9c0aba fix(obs): align cluster capacity semantics (#4457)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 09:11:01 +00:00
houseme 021c955c21 fix(obs): report live replication backlog (#4448)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:39:06 +08:00
houseme f6433ebb8b fix(obs): drop placeholder drive series (#4440)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:29:48 +08:00
houseme c1cc0d189f fix(obs): retire stale bucket and target series (#4434)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:23:25 +08:00
houseme acb1b765db fix(obs): export resettable metrics as gauges (#4432)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:23:11 +08:00
houseme d7b55c9762 fix(obs): honor profiling export contracts (#4433)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:22:47 +08:00
houseme 5aa25a3047 fix(obs): reuse network snapshots across ticks (#4431)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:22:35 +08:00
houseme 8df474b41b fix(obs): shut down global telemetry guard (#4430)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:22:23 +08:00
houseme e0a46e940c fix(obs): drain cleaner results inside scope (#4429)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 16:21:52 +08:00
houseme f968129945 fix(obs): stop exporting fake cpu categories (#4439)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
2026-07-08 16:05:11 +08:00
Zhengchao An 9a7255540b fix(replication): add resync metrics (#4408) 2026-07-08 15:01:28 +08:00
houseme 70e1d79dfd feat: replace jemalloc with mimalloc (#4174)
* feat: replace jemalloc with mimalloc

* docs: record allocator rounds5 retest

* fix(replication): satisfy clippy unwrap lints

* docs: keep allocator migration plan local only

* feat(profiling): rely on pyroscope cpu profiling

* refactor(profiling): centralize unsupported pprof responses

* chore(deps): update s3s revision
2026-07-02 19:03:38 +08:00
Zhengchao An 644a472e52 refactor(obs): hide replication stats handle (#4149) 2026-07-01 23:37:23 +08:00
houseme b16120dbcc feat: optimize small GET read paths (#4022) 2026-07-01 15:40:00 +08:00
wood 7484a61fa3 feat(compression): show cluster-level compression stat in grafana (#4112) 2026-06-30 19:05:05 +08:00
Zhengchao An 27a491a730 refactor(runtime): guard replication stats globals (#4058) 2026-06-29 19:55:38 +08:00
Zhengchao An ea88f5b67b refactor(obs): use lifecycle runtime facades (#4031) 2026-06-29 09:29:50 +08:00
Zhengchao An 1104d630d1 docs(obs): inventory ecstore dependency boundary (#4023) 2026-06-29 07:07:47 +08:00
Zhengchao An cd165ab181 docs: document obs reverse dependency on ecstore (#735) (#4010)
docs: document obs reverse dependency on ecstore

Add comment explaining why obs depends on ecstore and the scope
of work required to break this dependency.

Refs #735

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-28 20:28:04 +08:00
Zhengchao An 05d201679c fix: replace unwrap() with expect() in more files (#729 batch 13) (#3993) 2026-06-28 11:45:13 +08:00
Zhengchao An e1272f2aba revert: restore #![allow(dead_code)] - CI clippy -D warnings conflict (#3979)
revert: restore #![allow(dead_code)] - clippy -D warnings treats warn as error

The #742 PR changed #![allow(dead_code)] to #![warn(dead_code)], but
CI runs clippy with -D warnings which turns warnings into errors.
This caused CI failures across multiple PRs.

Reverting to #![allow(dead_code)] until the dead code is actually
cleaned up. The 189 warnings in ecstore should be fixed incrementally
by deleting dead code and adding item-level allows, not by changing
the crate-level policy.
2026-06-28 08:32:34 +08:00
Zhengchao An 113058af54 chore: replace blanket #![allow(dead_code)] with #![warn(dead_code)] (#742) (#3974) 2026-06-28 07:50:51 +08:00
Zhengchao An 1b3dea012e refactor: route ecstore runtime globals through facade (#3941) 2026-06-27 12:27:03 +08:00
Zhengchao An 72ae43cb90 refactor: segment external storage contract imports (#3908) 2026-06-26 16:44:59 +08:00
Zhengchao An 1f7e159388 refactor: segment external storage api boundaries (#3903) 2026-06-26 15:10:49 +08:00
Zhengchao An e37e367390 refactor: route remaining external storage boundaries (#3889) 2026-06-26 06:22:51 +08:00
Zhengchao An 5c60f0cae9 refactor: centralize owner server config reads (#3793) 2026-06-23 21:27:30 +08:00
Zhengchao An 198fd4f150 refactor(runtime): route RustFS runtime consumers through storage owner (#3756) 2026-06-23 05:17:56 +08:00
Zhengchao An 6da61b44e5 refactor: expose remaining external owner symbols (#3751) 2026-06-22 23:35:50 +08:00
Zhengchao An 418b5d04f9 refactor: route external ECStore imports through owners (#3747) 2026-06-22 20:57:10 +08:00
Zhengchao An 88a87b3e8f refactor: centralize external ECStore facade aliases (#3746) 2026-06-22 20:37:13 +08:00
Zhengchao An 539c68778d refactor: remove standalone compat bridge modules (#3740) 2026-06-22 17:35:58 +08:00
安正超 cca9e83a8b refactor: use relative standalone compat consumers (#3734) 2026-06-22 15:07:58 +08:00
安正超 de158743c3 refactor: split compat facade imports (#3716) 2026-06-22 09:53:22 +08:00
安正超 d39815470e refactor: prune consumer storage compat paths (#3704) 2026-06-22 02:31:30 +08:00
安正超 30b46640b4 refactor: prune edge compat aliases (#3692) 2026-06-21 17:41:07 +08:00
安正超 0985225448 refactor: consolidate ecstore public facade (#3678)
* refactor: expand ecstore compatibility facade

* test: enforce ecstore api facade imports

* refactor: hide legacy ecstore layout modules

* refactor: hide ecstore facade root modules
2026-06-21 09:09:11 +08:00