Commit Graph

19 Commits

Author SHA1 Message Date
houseme 29fbdc2dbf feat(ecstore): add object lock diagnostics (#3178)
* feat(ecstore): add object lock diagnostics

Add configurable namespace lock diagnostics for object operations so production contention can be traced by operation, owner, and key.

Wrap object read/write lock acquisition in diagnostic guards across get, head, put, delete, copy, and multipart flows, and log slow acquisition and long hold durations behind new RUSTFS_OBJECT_LOCK_DIAG_* settings.

Verification:

- make pre-commit

* feat(obs): expose object lock diagnostics metrics

Add Prometheus metrics and Grafana panels for object namespace lock diagnostics, covering slow acquire counts, slow hold counts, acquire duration, hold duration, and the diagnostics-enabled state.

Adopt PR review feedback by keeping diagnostic guards alive through the guarded operation so long-hold warnings and metrics are emitted, reusing shared env helpers, and reducing default-path overhead when diagnostics are disabled.

Verification:

- cargo check -p rustfs-ecstore -p rustfs-io-metrics

- cargo test -p rustfs-ecstore store::object -- --nocapture

- make pre-commit

* perf(obs): reduce object lock diag overhead

Avoid repeated environment parsing on hot object-lock paths by caching the diagnostics-enabled flag, and stop allocating label strings for object lock metrics by recording static labels directly.

Strengthen io-metrics tests by using a local recorder and asserting that the expected object lock diagnostic counters, gauges, and histograms are emitted.

Verification:

- cargo check -p rustfs-ecstore -p rustfs-io-metrics

- cargo test -p rustfs-io-metrics -- --nocapture

- make pre-commit
2026-06-03 02:10:27 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012)
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
houseme c90bfe2b23 fix(ecstore): harden runtime read-path quorum handling (#2872) 2026-05-08 09:56:39 +00:00
houseme 50d03ef021 perf(memory): add reclaim signals and cache controls (#2689) 2026-04-26 16:42:35 +00:00
houseme 59f41eb86a feat(obs): improve metrics coverage and dashboard performance (#2682) 2026-04-25 18:51:29 +00:00
houseme 13b4500212 feat(obs): improve telemetry stack, replication metrics, and Grafana alignment (#2672)
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-24 13:50:17 +00:00
houseme 116db4f5d9 refactor(metrics): unify process sampling and split network IO (#2590) 2026-04-18 15:30:44 +00:00
ankohuu 0e5fc4bec1 fix(metrics): use Prometheus-compatible metric names (#2312) (#2317)
Signed-off-by: Shunchao Hu <ankohuu@gmail.com>
2026-04-05 21:39:10 +08:00
houseme 7172e151de fix: address correctness, safety, and concurrency issues (#2327)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-30 00:30:57 +08:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-20 18:52:33 +08:00
houseme 94cdb89e29 feat(obs): add init_obs_with_config API and signature guard test (#2175)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-16 18:17:55 +08:00
LeonWang0735 7f1cdaedad feat(replication): add bandwidth-aware reporting for bucket replication metrics (#2141) 2026-03-15 09:03:10 +08:00
houseme 05032cf887 chore: update dependencies and workspace resolver (#2073) 2026-03-04 19:22:54 +08:00
evan slack ac4b13def1 feat(obs): Optional continuous CPU profiling with grafana pyroscope (#2035)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-03-03 15:28:58 +08:00
evan slack fd1b903531 fix(obs): Update observability docker compose stack (#2010) 2026-03-01 03:03:50 +08:00
evan slack bdb2a9e9b7 fix(dashboard): Rename grafana dashboard rustfs.yaml -> rustfs.json (#1987)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 15:49:34 +08:00
heihutu d983638391 build: update docker config and refine s3s region handling (#1976)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-27 01:21:12 +08:00
evan slack 9fe3d5621e feat(observability): Add additional metric panels to grafana dashboard (#1778)
Signed-off-by: evan slack <51209817+evanofslack@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-14 21:41:07 +08:00
evan slack 9824171995 feat(observability): Add grafana dashboard, observability changes (#1770)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-11 15:55:08 +08:00