gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-06 05:17:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,129 Commits 31 Branches 119 Tags
9ce9ec22d13484fd66cb3ce3b91ea9adcff41b6f
Commit Graph

149 Commits

Author SHA1 Message Date
Henry Guo 76da2a48d0 feat(scanner): expose cycle observability controls (#3147) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-05-31 21:55:46 +00:00
Henry Guo a99ef64db2 feat(scanner): add scanner budgets and progress metrics (#3145) 
* fix(scanner): preserve maintenance scan cadence

* feat(scanner): add scanner concurrency budget

* feat(scanner): expose scanner runtime progress

* fix(scanner): address scanner review feedback

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-31 16:42:38 +00:00
houseme dcbffb084f chore(deps): refresh workspace deps and linux fs_type gating (#3030) 
* chore(deps): refresh workspace deps and linux fs_type gating

- refresh workspace dependency pins and lockfile updates

- remove now-unused crate dependency entries in multiple Cargo.toml files

- enable profiling export defaults in config and scripts/run.sh

- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings

* fix(utils): simplify fs_type linux gating

- keep fs_type module-level linux cfg in os::mod

- remove redundant linux cfg on get_fs_type and test module

* chore(deps): bump s3s git revision

- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5

- refresh Cargo.lock source entry for s3s

* chore(dev): allow mysql_async git source and env overrides

- allow mysql_async git source in deny.toml allow-git list

- make scripts/run.sh core env vars overrideable via existing shell env

* fix(utils): import get_fs_type in fs_type tests

- add explicit super::get_fs_type import in fs_type test module

- fix Linux E0425 unresolved function errors in unit tests

* chore(dev): tune run script observability defaults

- make profiling export env overrideable in scripts/run.sh

- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0

- update allow-git review window comments in deny.toml

* test(obs): stabilize profiling env alias tests
 2026-05-20 15:03:07 +00:00
安正超 c684438625 fix(obs): add proxied PUT replication metrics (#3020) 
fix(obs): add proxied put replication metrics
 2026-05-20 05:30:57 +00:00
安正超 c727589161 fix(obs): remove stale replication metric TODOs (#3024) 2026-05-20 03:59:16 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012) 
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
 2026-05-19 12:50:25 +00:00
houseme c4248078d6 chore: update dependencies (#2890) 
* chore: update dependencies

* build(deps): bump the dependencies group with 5 updates

* build(deps): bump the dependencies group with 6 updates (#2908)

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>

* fix(ecstore): narrow Windows URL drive path rewrite

* chore(deps): bump starshard to 1.2.0

* revert(ecstore): restore endpoint Windows path behavior

* up

* up

* up

* perf(notify): use cached snapshot mode for scans

* fix

* chore(deps): bump workspace dependency versions

* chore(deps): refresh pinned dependency references

* chore(ci): align profiling and decommission tooling updates

- enable tokio_unstable cfg in performance profiling build flags\n- bump Rust base image from 1.93 to 1.95 in source and decommission Dockerfiles\n- remove obsolete compose version key from docker-compose-simple.yml\n- add standard Apache-2.0 license header to docker-compose.decommission.yml

* chore(deps): bump the dependencies group across 1 directory with 7 updates (#2994)

Bumps the dependencies group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo) | `0.39.1` | `0.39.2` |
| [dial9-tokio-telemetry](https://github.com/dial9-rs/dial9-tokio-telemetry) | `0.3.9` | `0.3.10` |
| [opentelemetry](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-otlp](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.1` | `0.32.0` |
| [opentelemetry_sdk](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-semantic-conventions](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-stdout](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |



Updates `sysinfo` from 0.39.1 to 0.39.2
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/main/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.39.1...v0.39.2)

Updates `dial9-tokio-telemetry` from 0.3.9 to 0.3.10
- [Release notes](https://github.com/dial9-rs/dial9-tokio-telemetry/releases)
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dial9-rs/dial9-tokio-telemetry/compare/dial9-tokio-telemetry-v0.3.9...dial9-tokio-telemetry-v0.3.10)

Updates `opentelemetry` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/opentelemetry-prometheus-0.31.0...opentelemetry-0.32.0)

Updates `opentelemetry-otlp` from 0.31.1 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/opentelemetry-otlp-0.31.1...opentelemetry-otlp-0.32.0)

Updates `opentelemetry_sdk` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry_sdk-0.32.0)

Updates `opentelemetry-semantic-conventions` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry-semantic-conventions-0.32.0)

Updates `opentelemetry-stdout` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry-stdout-0.32.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-version: 0.39.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: dial9-tokio-telemetry
  dependency-version: 0.3.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: opentelemetry
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-otlp
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry_sdk
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-semantic-conventions
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-stdout
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

* chore(deps): bump workspace dependency versions

* chore(deps): refresh lockfile windows crate graph

* chore(bench): align snapshot mode labels with coverage

* chore(bench): clarify tested snapshot modes

* fix(review): address PR2890 Copilot comments

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 04:58:42 +00:00
houseme bd1e57293f fix: harden lifecycle transition compensation and regression coverage (#2995) 
* fix(ecstore): honor transition worker configuration

* fix(ecstore): add transition queue backpressure metrics

* fix(ecstore): schedule transition compensation on enqueue pressure

* fix(ecstore): log transition compensation scheduling

* test(rustfs): add transition compensation fault-injection coverage

* test(rustfs): cover delete after transition compensation

* test(scanner): cover cleanup after transition compensation

* test(rustfs): extend compensation transition coverage

* test(scanner): cover backfill idempotency after compensation

* test(scanner): cover noncurrent expiry after compensation

* test(rustfs): cover versioned delete after compensation

* test(rustfs): cover delete marker lifecycle after compensation

* test(scanner): extend versioned lifecycle compensation coverage

* test(scanner): model versioned delete after compensation

* test(scanner): clarify modeled versioned delete helper

* refactor(ecstore): optimize transition enqueue hot path

* refactor(ecstore): centralize transition runtime constants

* style(ecstore): apply rustfmt for transition timeout helper

* fix(ilm): align queue-full metric semantics

* refactor(ecstore): unify immediate enqueue failure handling

* refactor(ecstore): reuse transition worker env constant

* ci(actions): update setup action inputs
 2026-05-18 12:50:43 +00:00
Henry Guo bca8b08c2b fix: handle Windows paths in pre-commit tests (#2974) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-15 14:04:51 +00:00
安正超 77c21dc5fd test(obs): cover replication bandwidth tombstones (#2906) 2026-05-11 01:55:54 +00:00
LeonWang0735 61a7820651 optimize(obs):zero and expire removed replication bandwidth series (#2901) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-10 14:26:18 +00:00
houseme c90bfe2b23 fix(ecstore): harden runtime read-path quorum handling (#2872) 2026-05-08 09:56:39 +00:00
houseme c4d5c5c5ec fix(obs): disable profiling export by default and fix Helm env name (#2719) 2026-04-28 11:57:15 +00:00
houseme 59f41eb86a feat(obs): improve metrics coverage and dashboard performance (#2682) 2026-04-25 18:51:29 +00:00
houseme 13b4500212 feat(obs): improve telemetry stack, replication metrics, and Grafana alignment (#2672) 
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-04-24 13:50:17 +00:00
houseme 47247789ad feat(obs): support OTLP headers and timeout overrides for HTTP exporters (#2661) 2026-04-23 17:02:26 +00:00
houseme 116db4f5d9 refactor(metrics): unify process sampling and split network IO (#2590) 2026-04-18 15:30:44 +00:00
houseme 1cbf156559 refactor(obs): migrate metrics runtime/schema and tighten migration guards (#2584) 2026-04-18 07:51:15 +00:00
Tunglies 579b124726 lint: clippy rules or_fun_call (#2561) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-04-16 02:48:39 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
John 6a2f98b468 fix(obs): respect error log level for http warnings (#2492) 
Signed-off-by: 80347547 <jianglong@oppo.com>
Co-authored-by: 80347547 <jianglong@oppo.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-04-12 01:32:54 +08:00
ankohuu 8d27170ce4 fix: limit pyroscope profiling to supported Unix targets (#2399) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-04-06 23:05:30 +08:00
houseme fb2ced4d27 feat(obs): integrate dial9-tokio-telemetry for runtime tracing (#2285) 
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2026-03-25 14:23:58 +08:00
houseme 628481be7c fix(tracing): Fix distributed tracing context linking (#2247) 2026-03-21 11:07:35 +08:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-20 18:52:33 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
houseme 94cdb89e29 feat(obs): add init_obs_with_config API and signature guard test (#2175) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
 2026-03-16 18:17:55 +08:00
houseme 7f3459f5a8 fix(obs): fixed unresolved import super::local::ensure_dir_permissions (#2164) 2026-03-15 00:33:06 +08:00
majinghe f66a90c1b2 fix: fix github action error caused by oltp modification (#2163) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-14 22:15:35 +08:00
安正超 a1104b45f6 fix(obs): honor target-only rust_log directives (#2159) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-03-14 11:14:46 +08:00
houseme 6e0f034ad1 refactor(obs): enhance log rotation robustness and refine filter logic (#2155) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-03-14 09:20:35 +08:00
houseme 593a58c161 refactor(obs): optimize logging with custom RollingAppender and improved cleanup (#2151) 
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-03-13 13:20:27 +08:00
安正超 845ad1fa16 fix(obs): avoid panic in telemetry init and clamp sampler boundaries (#2118) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-11 01:32:46 +08:00
唐小鸭 f89cdfe5b3 update s3s 0.14.0-dev (#2070) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-04 01:07:24 +08:00
houseme 5e7495a042 build(obs): restrict pyroscope dependency to unix targets (#2064) 2026-03-03 20:41:37 +08:00
evan slack ac4b13def1 feat(obs): Optional continuous CPU profiling with grafana pyroscope (#2035) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
 2026-03-03 15:28:58 +08:00
安正超 08e1f4670b fix: restore default CORS fallback and STS object ACL ownership (#2053) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-03 01:08:50 +08:00
houseme 2ac07c95a8 refactor(obs): enhance log cleanup and rotation (#2040) 2026-03-02 16:28:32 +08:00
houseme c452f24487 Optimize log cleanup and rotation, update dependencies (#2032) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-03-01 20:09:52 +08:00
heihutu 2c01b8c49d feat(obs): add advanced log management configuration (#2016) 
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: 唐小鸭 <tangtang1251@qq.com>
 2026-03-01 03:23:48 +08:00
heihutu d983638391 build: update docker config and refine s3s region handling (#1976) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-02-27 01:21:12 +08:00
houseme d345ace326 perf(obs): optimize metrics recorder and telemetry initialization (#1859) 2026-02-18 07:05:43 +08:00
evan slack 9da332c47d perf(metrics): Cache metric handles instead of creating each call (#1852) 
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
 2026-02-18 01:03:35 +08:00
evan slack 8010284aa3 obs(export): Add env vars to selectivly disable exporting traces/metrics/logs (#1853) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-02-18 00:38:30 +08:00
evan slack 9824171995 feat(observability): Add grafana dashboard, observability changes (#1770) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-02-11 15:55:08 +08:00
houseme 4411c625e2 feat(metrics): async collection with configurable intervals & graceful shutdown (#1768) 2026-02-10 21:37:24 +08:00
Isaac Mills d635ee8d2e Propogate tracing context from HTTP requests into spans (#1739) 
Signed-off-by: Isaac Mills <57533634+StratusFearMe21@users.noreply.github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-02-07 11:52:31 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
houseme 08f1a31f3f Fix notification event stream cleanup, add bounded send concurrency, and reduce overhead (#1224) 2025-12-22 00:57:05 +08:00
yihong 834025d9e3 docs: fix some dead link (#1053) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-12-08 11:23:24 +08:00