ad1a489f75
feat(scanner): add scanner budget progress controls ( #3185 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-06-03 14:37:58 +00:00
cc07946782
feat(scanner): add cycle budget observability ( #3166 )
...
* feat(scanner): add cycle budget observability
* fix(scanner): clear cycle state after budget stop
* test(scanner): stabilize timeout-based scanner tests
* fix(scanner): keep cycle ILM counts scanner-only
* test(scanner): avoid test-only pending import
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-06-03 03:32:41 +00:00
1d46047d6f
feat(scanner): expand scanner observability metrics ( #3159 )
...
* feat(scanner): expand scanner observability metrics
* chore(scanner): align bucket-drive metric wording
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-06-02 01:29:53 +00:00
f3bd838925
feat(scanner): expose cycle progress metrics ( #3152 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-06-01 07:54:39 +00:00
76da2a48d0
feat(scanner): expose cycle observability controls ( #3147 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-05-31 21:55:46 +00:00
a99ef64db2
feat(scanner): add scanner budgets and progress metrics ( #3145 )
...
* fix(scanner): preserve maintenance scan cadence
* feat(scanner): add scanner concurrency budget
* feat(scanner): expose scanner runtime progress
* fix(scanner): address scanner review feedback
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-31 16:42:38 +00:00
c684438625
fix(obs): add proxied PUT replication metrics ( #3020 )
...
fix(obs): add proxied put replication metrics
2026-05-20 05:30:57 +00:00
c727589161
fix(obs): remove stale replication metric TODOs ( #3024 )
2026-05-20 03:59:16 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
bd1e57293f
fix: harden lifecycle transition compensation and regression coverage ( #2995 )
...
* fix(ecstore): honor transition worker configuration
* fix(ecstore): add transition queue backpressure metrics
* fix(ecstore): schedule transition compensation on enqueue pressure
* fix(ecstore): log transition compensation scheduling
* test(rustfs): add transition compensation fault-injection coverage
* test(rustfs): cover delete after transition compensation
* test(scanner): cover cleanup after transition compensation
* test(rustfs): extend compensation transition coverage
* test(scanner): cover backfill idempotency after compensation
* test(scanner): cover noncurrent expiry after compensation
* test(rustfs): cover versioned delete after compensation
* test(rustfs): cover delete marker lifecycle after compensation
* test(scanner): extend versioned lifecycle compensation coverage
* test(scanner): model versioned delete after compensation
* test(scanner): clarify modeled versioned delete helper
* refactor(ecstore): optimize transition enqueue hot path
* refactor(ecstore): centralize transition runtime constants
* style(ecstore): apply rustfmt for transition timeout helper
* fix(ilm): align queue-full metric semantics
* refactor(ecstore): unify immediate enqueue failure handling
* refactor(ecstore): reuse transition worker env constant
* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
77c21dc5fd
test(obs): cover replication bandwidth tombstones ( #2906 )
2026-05-11 01:55:54 +00:00
61a7820651
optimize(obs):zero and expire removed replication bandwidth series ( #2901 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-10 14:26:18 +00:00
c90bfe2b23
fix(ecstore): harden runtime read-path quorum handling ( #2872 )
2026-05-08 09:56:39 +00:00
59f41eb86a
feat(obs): improve metrics coverage and dashboard performance ( #2682 )
2026-04-25 18:51:29 +00:00
13b4500212
feat(obs): improve telemetry stack, replication metrics, and Grafana alignment ( #2672 )
...
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt >
Co-authored-by: cxymds <Cxymds@qq.com >
Co-authored-by: weisd <im@weisd.in >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-04-24 13:50:17 +00:00
116db4f5d9
refactor(metrics): unify process sampling and split network IO ( #2590 )
2026-04-18 15:30:44 +00:00
1cbf156559
refactor(obs): migrate metrics runtime/schema and tighten migration guards ( #2584 )
2026-04-18 07:51:15 +00:00
4411c625e2
feat(metrics): async collection with configurable intervals & graceful shutdown ( #1768 )
2026-02-10 21:37:24 +08:00
29056a767a
Refactor Telemetry Initialization and Environment Utilities ( #811 )
...
* improve code for metrics
* improve code for metrics
* fix
* fix
* Refactor telemetry initialization and environment functions ordering
- Reorder functions in envs.rs by type size (8-bit to 64-bit, signed before unsigned) and add missing variants like get_env_opt_u16.
- Optimize init_telemetry to support three modes: stdout logging (default error level with span tracing), file rolling logs (size-based with retention), and HTTP-based observability with sub-endpoints (trace, metric, log) falling back to unified endpoint.
- Fix stdout logging issue by retaining WorkerGuard in OtelGuard to prevent premature release of async writer threads.
- Enhance observability mode with HTTP protocol, compression, and proper resource management.
- Update OtelGuard to include tracing_guard for stdout and flexi_logger_handles for file logging.
- Improve error handling and configuration extraction in OtelConfig.
* fix
* up
* fix
* fix
* improve code for obs
* fix
* fix
2025-11-07 20:01:54 +08:00
dd47fcf2a8
fix: restore localized samples in tests ( #749 )
...
* fix: restore required localized examples
* style: fix formatting issues
2025-10-29 13:16:31 +08:00
9d5ed1acac
Feature/scanner performance optimization ( #498 )
...
* Refactor: reimplement scanner
Signed-off-by: RustFS Developer <dandan@rustfs.com >
* comment lock
Signed-off-by: junxiang Mu <1948535941@qq.com >
* remove dirty file
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Fix: fix rebase
* fix(scanner): Improve error handling and logging
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: RustFS Developer <dandan@rustfs.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: RustFS Developer <dandan@rustfs.com >
2025-09-08 18:35:45 +08:00
6c37e1cb2a
refactor: replace lazy_static with LazyLock ( #318 )
...
* refactor: replace `lazy_static` with `LazyLock`
Replace `lazy_static` with `LazyLock`.
Compile time may reduce a little.
See https://github.com/rust-lang-nursery/lazy-static.rs/issues/214
* fmt
* fix
2025-07-31 14:25:39 +08:00
2e14b32ccd
chore: Add copyright and license headers ( #23 )
...
* chore: Add copyright and license headers
This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.
* cargo fmt
* fix
* fmt
* fix clippy
2025-07-02 15:07:47 +08:00
2b1241de96
fix: fmt
2025-07-01 10:17:20 +08:00
38220697e0
modify description
2025-06-30 23:03:29 +08:00
4559baaeeb
feat: migrate to reed-solomon-simd only implementation
...
- Remove reed-solomon-erasure dependency and all related code
- Simplify ReedSolomonEncoder from enum to struct with SIMD-only implementation
- Eliminate all conditional compilation (#[cfg(feature = ...)])
- Add instance caching with RwLock-based encoder/decoder reuse
- Implement reset mechanism to avoid unnecessary allocations
- Ensure thread safety with proper cache management
- Update documentation and benchmark scripts for SIMD-only approach
- Apply code formatting across all files
Breaking Changes:
- Removes support for reed-solomon-erasure feature flag
- API remains compatible but implementation is now SIMD-only
Performance Impact:
- Improved encoding/decoding performance through SIMD optimization
- Reduced memory allocations via instance caching
- Enhanced thread safety and concurrency support
2025-06-23 10:00:17 +08:00
fd1bf30de8
test metrics
2025-05-16 20:16:43 +08:00
6659b77f9a
improve comment and error
2025-05-16 17:00:56 +08:00
60635aeb65
add metrics
2025-05-15 23:34:36 +08:00
9e9f721c08
add metric info
2025-05-14 19:00:15 +08:00
cdd285dec8
add metric
2025-05-13 09:17:52 +08:00
Henry Guo