gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-09 14:49:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,829 Commits 25 Branches 121 Tags
809e77d5db7050125de4aaac15b8de994bb9748a
Commit Graph

55 Commits

Author SHA1 Message Date
Zhengchao An ea88f5b67b refactor(obs): use lifecycle runtime facades (#4031) 2026-06-29 09:29:50 +08:00
Zhengchao An 05d201679c fix: replace unwrap() with expect() in more files (#729 batch 13) (#3993) 2026-06-28 11:45:13 +08:00
Zhengchao An e1272f2aba revert: restore #![allow(dead_code)] - CI clippy -D warnings conflict (#3979) 
revert: restore #![allow(dead_code)] - clippy -D warnings treats warn as error

The #742 PR changed #![allow(dead_code)] to #![warn(dead_code)], but
CI runs clippy with -D warnings which turns warnings into errors.
This caused CI failures across multiple PRs.

Reverting to #![allow(dead_code)] until the dead code is actually
cleaned up. The 189 warnings in ecstore should be fixed incrementally
by deleting dead code and adding item-level allows, not by changing
the crate-level policy.
 2026-06-28 08:32:34 +08:00
Zhengchao An 113058af54 chore: replace blanket #![allow(dead_code)] with #![warn(dead_code)] (#742) (#3974) 2026-06-28 07:50:51 +08:00
Zhengchao An 1b3dea012e refactor: route ecstore runtime globals through facade (#3941) 2026-06-27 12:27:03 +08:00
Zhengchao An 72ae43cb90 refactor: segment external storage contract imports (#3908) 2026-06-26 16:44:59 +08:00
Zhengchao An 1f7e159388 refactor: segment external storage api boundaries (#3903) 2026-06-26 15:10:49 +08:00
Zhengchao An e37e367390 refactor: route remaining external storage boundaries (#3889) 2026-06-26 06:22:51 +08:00
Zhengchao An 5c60f0cae9 refactor: centralize owner server config reads (#3793) 2026-06-23 21:27:30 +08:00
Zhengchao An 198fd4f150 refactor(runtime): route RustFS runtime consumers through storage owner (#3756) 2026-06-23 05:17:56 +08:00
Zhengchao An 6da61b44e5 refactor: expose remaining external owner symbols (#3751) 2026-06-22 23:35:50 +08:00
Zhengchao An 418b5d04f9 refactor: route external ECStore imports through owners (#3747) 2026-06-22 20:57:10 +08:00
Zhengchao An 88a87b3e8f refactor: centralize external ECStore facade aliases (#3746) 2026-06-22 20:37:13 +08:00
Zhengchao An 539c68778d refactor: remove standalone compat bridge modules (#3740) 2026-06-22 17:35:58 +08:00
安正超 cca9e83a8b refactor: use relative standalone compat consumers (#3734) 2026-06-22 15:07:58 +08:00
安正超 a860f2b40c refactor: narrow storage compatibility surfaces (#3585) 
* refactor: narrow storage compatibility surfaces

* refactor: narrow observability compatibility surfaces
 2026-06-19 00:56:14 +08:00
安正超 c098184c49 refactor: clean runtime and test storage boundaries (#3573) 
* refactor: clean runtime observability select boundaries

* refactor: clean test harness fuzz storage boundaries
 2026-06-18 18:09:25 +08:00
安正超 ed55857bca refactor: move bucket operations contract (#3507) 2026-06-17 09:10:38 +08:00
Henry Guo dd6b4c35ad fix(scanner): harden lifecycle and tiering backlog (#3469) 
* fix(scanner): expose lifecycle transition backlog

* fix(scanner): expose lifecycle expiry backlog

* fix(scanner): preserve lifecycle backlog states

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-15 15:01:21 +08:00
安正超 84f7027e90 refactor: add object store resolver for standalone crates (#3437) 2026-06-14 18:02:39 +08:00
houseme efa89a98ed refactor(logging): standardize protocol and observability events (#3419) 
* refactor(logging): standardize object capacity events

* refactor(logging): standardize protocol server events

* refactor(logging): standardize swift protocol events

* refactor(logging): standardize observability events

* refactor(logging): move masking helper and extend guardrails
 2026-06-14 07:14:45 +08:00
安正超 6b1e114b39 feat(obs): add metrics runtime controller status (#3408) 2026-06-13 20:48:34 +08:00
安正超 7146c893cb refactor(storage-api): remove legacy bucket DTO re-export (#3364) 2026-06-11 21:24:56 +08:00
安正超 94c53af264 refactor(storage): use admin API for observability reads (#3335) 2026-06-11 08:13:15 +08:00
Henry Guo ad1a489f75 feat(scanner): add scanner budget progress controls (#3185) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-03 14:37:58 +00:00
Henry Guo cc07946782 feat(scanner): add cycle budget observability (#3166) 
* feat(scanner): add cycle budget observability

* fix(scanner): clear cycle state after budget stop

* test(scanner): stabilize timeout-based scanner tests

* fix(scanner): keep cycle ILM counts scanner-only

* test(scanner): avoid test-only pending import

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-06-03 03:32:41 +00:00
Henry Guo 1d46047d6f feat(scanner): expand scanner observability metrics (#3159) 
* feat(scanner): expand scanner observability metrics

* chore(scanner): align bucket-drive metric wording

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-02 01:29:53 +00:00
Henry Guo f3bd838925 feat(scanner): expose cycle progress metrics (#3152) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-01 07:54:39 +00:00
Henry Guo 76da2a48d0 feat(scanner): expose cycle observability controls (#3147) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-05-31 21:55:46 +00:00
Henry Guo a99ef64db2 feat(scanner): add scanner budgets and progress metrics (#3145) 
* fix(scanner): preserve maintenance scan cadence

* feat(scanner): add scanner concurrency budget

* feat(scanner): expose scanner runtime progress

* fix(scanner): address scanner review feedback

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-31 16:42:38 +00:00
安正超 c684438625 fix(obs): add proxied PUT replication metrics (#3020) 
fix(obs): add proxied put replication metrics
 2026-05-20 05:30:57 +00:00
安正超 c727589161 fix(obs): remove stale replication metric TODOs (#3024) 2026-05-20 03:59:16 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012) 
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
 2026-05-19 12:50:25 +00:00
houseme bd1e57293f fix: harden lifecycle transition compensation and regression coverage (#2995) 
* fix(ecstore): honor transition worker configuration

* fix(ecstore): add transition queue backpressure metrics

* fix(ecstore): schedule transition compensation on enqueue pressure

* fix(ecstore): log transition compensation scheduling

* test(rustfs): add transition compensation fault-injection coverage

* test(rustfs): cover delete after transition compensation

* test(scanner): cover cleanup after transition compensation

* test(rustfs): extend compensation transition coverage

* test(scanner): cover backfill idempotency after compensation

* test(scanner): cover noncurrent expiry after compensation

* test(rustfs): cover versioned delete after compensation

* test(rustfs): cover delete marker lifecycle after compensation

* test(scanner): extend versioned lifecycle compensation coverage

* test(scanner): model versioned delete after compensation

* test(scanner): clarify modeled versioned delete helper

* refactor(ecstore): optimize transition enqueue hot path

* refactor(ecstore): centralize transition runtime constants

* style(ecstore): apply rustfmt for transition timeout helper

* fix(ilm): align queue-full metric semantics

* refactor(ecstore): unify immediate enqueue failure handling

* refactor(ecstore): reuse transition worker env constant

* ci(actions): update setup action inputs
 2026-05-18 12:50:43 +00:00
安正超 77c21dc5fd test(obs): cover replication bandwidth tombstones (#2906) 2026-05-11 01:55:54 +00:00
LeonWang0735 61a7820651 optimize(obs):zero and expire removed replication bandwidth series (#2901) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-10 14:26:18 +00:00
houseme c90bfe2b23 fix(ecstore): harden runtime read-path quorum handling (#2872) 2026-05-08 09:56:39 +00:00
houseme 59f41eb86a feat(obs): improve metrics coverage and dashboard performance (#2682) 2026-04-25 18:51:29 +00:00
houseme 13b4500212 feat(obs): improve telemetry stack, replication metrics, and Grafana alignment (#2672) 
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-04-24 13:50:17 +00:00
houseme 116db4f5d9 refactor(metrics): unify process sampling and split network IO (#2590) 2026-04-18 15:30:44 +00:00
houseme 1cbf156559 refactor(obs): migrate metrics runtime/schema and tighten migration guards (#2584) 2026-04-18 07:51:15 +00:00
houseme 4411c625e2 feat(metrics): async collection with configurable intervals & graceful shutdown (#1768) 2026-02-10 21:37:24 +08:00
houseme 29056a767a Refactor Telemetry Initialization and Environment Utilities (#811) 
* improve code for metrics

* improve code for metrics

* fix

* fix

* Refactor telemetry initialization and environment functions ordering

- Reorder functions in envs.rs by type size (8-bit to 64-bit, signed before unsigned) and add missing variants like get_env_opt_u16.
- Optimize init_telemetry to support three modes: stdout logging (default error level with span tracing), file rolling logs (size-based with retention), and HTTP-based observability with sub-endpoints (trace, metric, log) falling back to unified endpoint.
- Fix stdout logging issue by retaining WorkerGuard in OtelGuard to prevent premature release of async writer threads.
- Enhance observability mode with HTTP protocol, compression, and proper resource management.
- Update OtelGuard to include tracing_guard for stdout and flexi_logger_handles for file logging.
- Improve error handling and configuration extraction in OtelConfig.

* fix

* up

* fix

* fix

* improve code for obs

* fix

* fix
 2025-11-07 20:01:54 +08:00
安正超 dd47fcf2a8 fix: restore localized samples in tests (#749) 
* fix: restore required localized examples

* style: fix formatting issues
 2025-10-29 13:16:31 +08:00
guojidan 9d5ed1acac Feature/scanner performance optimization (#498) 
* Refactor: reimplement scanner

Signed-off-by: RustFS Developer <dandan@rustfs.com>

* comment lock

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove dirty file

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Fix: fix rebase

* fix(scanner): Improve error handling and logging

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: RustFS Developer <dandan@rustfs.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: RustFS Developer <dandan@rustfs.com>
 2025-09-08 18:35:45 +08:00
houseme 6c37e1cb2a refactor: replace lazy_static with LazyLock (#318) 
* refactor: replace `lazy_static` with `LazyLock`

Replace `lazy_static` with `LazyLock`.

Compile time may reduce a little.

See https://github.com/rust-lang-nursery/lazy-static.rs/issues/214

* fmt

* fix
 2025-07-31 14:25:39 +08:00
houseme 2e14b32ccd chore: Add copyright and license headers (#23) 
* chore: Add copyright and license headers

This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.

* cargo fmt

* fix

* fmt

* fix clippy
 2025-07-02 15:07:47 +08:00
weisd 2b1241de96 fix: fmt 2025-07-01 10:17:20 +08:00
houseme 38220697e0 modify description 2025-06-30 23:03:29 +08:00
weisd 4559baaeeb feat: migrate to reed-solomon-simd only implementation 
- Remove reed-solomon-erasure dependency and all related code
- Simplify ReedSolomonEncoder from enum to struct with SIMD-only implementation
- Eliminate all conditional compilation (#[cfg(feature = ...)])
- Add instance caching with RwLock-based encoder/decoder reuse
- Implement reset mechanism to avoid unnecessary allocations
- Ensure thread safety with proper cache management
- Update documentation and benchmark scripts for SIMD-only approach
- Apply code formatting across all files

Breaking Changes:
- Removes support for reed-solomon-erasure feature flag
- API remains compatible but implementation is now SIMD-only

Performance Impact:
- Improved encoding/decoding performance through SIMD optimization
- Reduced memory allocations via instance caching
- Enhanced thread safety and concurrency support
 2025-06-23 10:00:17 +08:00