gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-28 17:18:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,867 Commits 31 Branches 117 Tags
3b754f649d98fcc6d9fe98da5f74eaabcdb150bc
Commit Graph

458 Commits

Author SHA1 Message Date
dependabot[bot] df82e3e645 build(deps): bump the dependencies group with 3 updates (#4030) 
Bumps the dependencies group with 3 updates: [aes-gcm](https://github.com/RustCrypto/AEADs), [chacha20poly1305](https://github.com/RustCrypto/AEADs) and [arc-swap](https://github.com/vorner/arc-swap).


Updates `aes-gcm` from 0.11.0-rc.4 to 0.11.0
- [Commits](https://github.com/RustCrypto/AEADs/compare/aes-gcm-v0.11.0-rc.4...aes-gcm-v0.11.0)

Updates `chacha20poly1305` from 0.11.0-rc.3 to 0.11.0
- [Commits](https://github.com/RustCrypto/AEADs/compare/chacha20poly1305-v0.11.0-rc.3...chacha20poly1305-v0.11.0)

Updates `arc-swap` from 1.9.1 to 1.9.2
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/commits)

---
updated-dependencies:
- dependency-name: aes-gcm
  dependency-version: 0.11.0
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: chacha20poly1305
  dependency-version: 0.11.0
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: arc-swap
  dependency-version: 1.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-29 10:32:52 +08:00
Zhengchao An ed56ffb54a docs(io): clarify zero copy metric transition (#4029) 2026-06-29 08:21:07 +08:00
Zhengchao An b37c8c6435 build: pin rc crypto dependency versions (#4020) 2026-06-28 22:52:20 +08:00
Zhengchao An aa8b8b5706 docs: document crypto RC version dependencies (#732) (#4011) 
* docs: document crypto RC version dependencies

Add comment explaining why aes-gcm and chacha20poly1305 use RC
versions and the migration path when stable versions are released.

Refs #732

* fix: remove all backlog links from code and comments
 2026-06-28 20:47:51 +08:00
Zhengchao An ee82d6c026 test: add insta snapshot test for storage error display format (#740) (#4001) 
test: add insta snapshot test for storage error display format

Add snapshot test to detect unexpected changes in StorageError
display format. This catches output format regressions that
traditional assert tests might miss.

Refs #740
 2026-06-28 16:10:07 +08:00
houseme 27468ebfa9 feat(get): consolidate GET performance optimization (#3972) 
* feat(get): consolidate GET performance optimization

Consolidated implementation of all GET performance optimizations into
a single, well-organized commit replacing the previous patch-on-patch
approach.

## Changes

### Configuration (set_disk/mod.rs)
- Consolidated all GET optimization flags into a single organized section
- Enabled by default: codec streaming, metadata early-stop, page cache reclaim
- Added codec streaming multipart flag (default: disabled)
- Added version-aware early-stop flag (default: disabled)
- Added adaptive duplex buffer sizing based on object size
- All flags use OnceLock caching with rollout percentage support

### Metadata Early-Stop (set_disk/read.rs)
- Delete marker early-stop when quorum agrees
- Version-aware early-stop for versioned GET requests
- MetadataQuorumAccumulator enhanced with:
  - delete_marker_votes tracking
  - requested_version_id and matching_version_votes tracking
  - version_early_stop_decision() method
- 6 new tests for version early-stop scenarios

### Codec Streaming (erasure/coding/decode_reader.rs)
- DualInFlight (2-stripe lookahead) enabled by default

### Decode Pipeline (erasure/coding/decode.rs)
- Stripe prefetch count configuration
- Bitrot-decode overlap configuration

### Disk Layer (disk/local.rs)
- O_DIRECT read configuration constants (preparation)

### Metrics (io-metrics/lib.rs)
- BytesPool acquisition/return metrics
- Metadata phase duration with early-stop label
- Total duration with reader_path label

### Diagnostics (diagnostics/)
- Early-stop reason constants
- Pool tier/outcome label constants

### Observability (.docker/observability/)
- 3 Grafana dashboards for GET optimization monitoring
- Prometheus alert rules (6 alerts: 3 critical, 3 warning)
- Updated README.md and README_ZH.md with usage docs

### Config (config/src/constants/runtime.rs)
- Page cache reclaim read enabled by default

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| RUSTFS_GET_CODEC_STREAMING_ENABLE | true | Codec streaming base flag |
| RUSTFS_GET_CODEC_STREAMING_ROLLOUT_PCT | 100 | Codec streaming rollout % |
| RUSTFS_GET_CODEC_STREAMING_MULTIPART_ENABLE | false | Multipart codec streaming |
| RUSTFS_GET_METADATA_EARLY_STOP_ENABLE | true | Early-stop base flag |
| RUSTFS_GET_METADATA_EARLY_STOP_ROLLOUT_PCT | 100 | Early-stop rollout % |
| RUSTFS_GET_METADATA_VERSION_EARLY_STOP_ENABLE | false | Version-aware early-stop |
| RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE | true | Page cache reclaim |
| RUSTFS_OBJECT_DIRECT_IO_READ_ENABLE | false | O_DIRECT (preparation) |
| RUSTFS_GET_DECODE_STRIPE_PREFETCH_COUNT | 1 | Stripe prefetch |
| RUSTFS_GET_BITROT_DECODE_OVERLAP_ENABLE | false | Bitrot-decode overlap |
| RUSTFS_GET_CODEC_STREAMING_MAX_INFLIGHT | 2 | DualInFlight stripes |

## Rollback

All optimizations can be disabled via environment variables:
RUSTFS_GET_CODEC_STREAMING_ENABLE=false
RUSTFS_GET_METADATA_EARLY_STOP_ENABLE=false
RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE=false

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(get): add stress test scripts for GET optimization validation

- quick-validate-get-optimization.sh: Quick 5-minute validation
- stress-test-get-optimization.sh: Full 30+ minute stress test
- README-stress-test.md: Usage documentation

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): align file cache reclaim defaults

* chore(deps): update redis and erasure codec

* test(ecstore): align decode fill policy default

* test(ecstore): align metadata early-stop default

* fix(ecstore): keep metadata early stop opt-in

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-06-28 07:14:07 +08:00
houseme 58b76a3d45 feat(get): add codec engine ab matrix (#3940) 
* upgrade version

* feat(get): add codec engine ab matrix

* chore(get): drop unrelated dependency drift

* upgrade version

* upgrade version

* fix cargo deny
 2026-06-27 13:27:16 +08:00
Zhengchao An 840d21d201 fix(deps): remove vulnerable thrift dependency (#3926) 2026-06-27 00:16:51 +08:00
houseme f7a9a7142b chore: update deps and e2e cache action (#3885) 
* chore(deps): update workspace crate versions

* ci: update e2e s3tests cache action
 2026-06-26 04:19:28 +08:00
houseme 623fc801f1 feat(replication): support custom TLS for bucket targets (#3825) 
* feat(replication): support insecure https bucket targets

* feat(replication): support custom ca bucket targets

* test(replication): satisfy e2e clippy for TLS helpers

* fix(replication): avoid native root panic for custom trust stores

* test(replication): decouple private IP target test from TLS roots

* test(replication): use target TLS client in private IP unit test
 2026-06-24 20:23:20 +08:00
houseme 0a00d8d500 perf(server): lighten internode data-plane stack (#3735) 
* refactor(server): split internode dispatch scaffold

* test(server): cover internode dispatch prefix split

* refactor(server): name internode stack boundaries

* perf(server): skip internode request logging layer

* perf(server): skip internode trace layer

* perf(server): use lite internode request context

* feat(metrics): track internode rpc duration

* feat(ecstore): add put object stage summary logs

* test(metrics): update internode descriptor expectations

* fix(server): tighten internode path matching

* fix(pr): address review follow-up comments

* style(ecstore): simplify commit tail duration field

* refactor(ecstore): group put stage summary fields

* refactor(ecstore): inline put stage summary log

* fix(s3): return storage class for object attributes

* merge: sync latest main and resolve object attributes conflict

* fmt

* fix(server): remove duplicate rpc imports

* build(deps): bump memmap2 for RUSTSEC-2026-0186

* fix(s3select): align object_store with datafusion

* chore(deps): prune workspace dependencies

* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization

  Separate fuzz harness compilation from execution to eliminate redundant
  builds across targets. Introduce matrix-based parallel execution for
  PR smoke and nightly fuzz jobs.

  Changes:

  - Split CI workflow into `fuzz-build` (compile once) and matrix run jobs
    (`pr-fuzz-smoke`, `nightly-fuzz-corpus`) that execute targets in parallel
  - Add `BUILD_ONLY` mode to run_ci_targets.sh / run_nightly_targets.sh
  - Add run_single_target.sh for matrix jobs (no build phase)
  - Optimize `local_metadata` fuzz target: reduce prefix iterations from
    8-10 (4 functions each) to 5 critical prefixes (parser-only), cutting
    per-iteration cost by ~3-5x
  - Move archive path validation (`validate_extract_relative_path`,
    `normalize_extract_entry_key`) from `rustfs` to `rustfs-utils::path`,
    eliminating `rustfs` binary crate dependency from fuzz harness
  - Remove `rustfs` from fuzz/Cargo.toml (drops significant transitive deps)
  - Add unit tests for archive path validation in rustfs-utils
  - Update fuzz/README.md with new workflow and script documentation

  Expected CI improvement: PR smoke wall-clock from ~120min (frequent
  timeout) to ~40min; nightly from ~180min to ~60min.

* refactor(fuzz): consolidate scripts and fix prefix test alignment

Replace three duplicated shell scripts (run_ci_targets.sh,
run_nightly_targets.sh, run_single_target.sh) with a single
parameterized run.sh that supports BUILD_ONLY, SKIP_BUILD, and
MAX_TOTAL_TIME environment variables.

Fix local_metadata fuzz target prefix testing: replace always-true
'len > 0' guard with lengths aligned to xl.meta binary layout
(4/5/8/12 bytes for magic+version+header fields). Remove redundant
empty-slice test.

Hoist RUSTFLAGS to workflow top-level env to eliminate per-job
duplication. Update README with unified script documentation.

Net: -118 lines, zero functionality loss.

* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization

Restructure fuzz CI workflow to eliminate redundant compilation and
run targets in parallel via matrix strategy.

Workflow changes:
- Split into fuzz-build (compile once) and matrix run jobs
- PR smoke: 3 targets parallel, 60s each, timeout 30min (was 120min)
- Nightly: 3 targets parallel, 300s each, timeout 60min (was 180min)
- Pass compiled harness via actions/artifact between jobs
- Hoist RUSTFLAGS to workflow top-level env

Script consolidation:
- Replace 3 duplicated scripts with single parameterized run.sh
- Supports BUILD_ONLY, SKIP_BUILD, MAX_TOTAL_TIME env vars

Target optimizations:
- Remove rustfs binary crate from fuzz dependencies (was pulling
  979 transitive deps); move archive path validation to rustfs-utils
- Optimize local_metadata: reduce prefix iterations from 8-10x4
  calls to 5 prefixes with parser-only (no decompress), aligned
  with xl.meta binary layout (4/5/8/12 bytes)
- Add unit tests for archive path validation in rustfs-utils
- Update fuzz/README.md with unified script documentation

Expected: PR smoke wall-clock from ~120min (frequent timeout)
to ~40min; nightly from ~180min to ~60min.

* fix(rpc): resolve internode metrics via app context

* fmt

* ci: speed up fuzz smoke artifact restore

---------

Signed-off-by: houseme <housemecn@gmail.com>
 2026-06-23 21:36:39 +08:00
houseme 46775ae019 feat: harden runtime capability snapshots (#3784) 
* feat(admin): expose runtime capability snapshots

* feat(runtime): refine workload admission snapshots

* test(ci): align architecture migration checks

* build(deps): bump memmap2 for RUSTSEC-2026-0186

* build(deps): refresh cargo deny lockfile
 2026-06-23 18:10:49 +08:00
dependabot[bot] 5a601869f6 chore(deps): bump sysinfo from 0.39.3 to 0.39.4 in the dependencies group (#3712) 2026-06-22 09:53:57 +08:00
houseme 9e6d0c7292 chore(deps): refresh workflow actions and crate pins (#3696) 2026-06-21 20:29:17 +08:00
houseme 8cf3c0bfbd fix(ecstore): support MinIO DARE fixture compatibility (#3590) 2026-06-19 10:13:39 +08:00
Henry Guo 65c724c786 feat(table-catalog): add manifest reachability cleanup (#3484) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-15 22:29:23 +08:00
houseme adec195486 chore(scanner): enable pulsar telemetry feature (#3481) 2026-06-15 18:43:14 +08:00
dependabot[bot] fbf1f9e5b8 build(deps): bump brotli from 8.0.3 to 8.0.4 in the dependencies group (#3458) 
* build(deps): bump brotli from 8.0.3 to 8.0.4 in the dependencies group

Bumps the dependencies group with 1 update: [brotli](https://github.com/dropbox/rust-brotli).


Updates `brotli` from 8.0.3 to 8.0.4
- [Release notes](https://github.com/dropbox/rust-brotli/releases)
- [Commits](https://github.com/dropbox/rust-brotli/compare/8.0.3...8.0.4)

---
updated-dependencies:
- dependency-name: brotli
  dependency-version: 8.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump s3s from cf4c3346 to c59b62f7ba in the dependencies group

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-15 12:06:01 +08:00
cxymds 046d5386ba feat: stream object zip downloads (#3380) 
* feat: stream object zip downloads

* fix: stream zip downloads page by page

* fix: prepare zip downloads before streaming

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-14 14:26:44 +08:00
houseme e8012bd1ba refactor(logging): normalize admin telemetry and error messages (#3430) 2026-06-14 13:27:10 +08:00
houseme 0d68851f7a refactor(logging): standardize rustfs runtime events (#3396) 
* fix(rpc): adjust log levels and reduce noise in storage RPC layer

Issue #682: pool rebalance normal flow logs were using warn! instead
of info!. Additionally, several high-frequency RPC entry points (ping,
write_stream, read_at, walk_dir) were logging at info! level, causing
unnecessary log noise in production.

Changes:
- load_rebalance_meta: warn! → info! for normal flow (3 occurrences)
- ping body decode: info! → debug! (fires on every health check)
- write_stream/read_at/walk_dir entry: info! → debug!
- metrics.rs: lowercase error messages, add structured error field
- http_service.rs: add structured error field to walk_dir failure
- Add tracing::instrument fields for start_rebalance context

* refactor(logging): standardize rustfs runtime events

* build(deps): bump workspace dependency versions

* build(deps): pin time to 0.3.47

* build(deps): update postgres client versions
 2026-06-13 08:39:47 +08:00
安正超 7d6d56a547 feat(extension): add schema contracts (#3384) 2026-06-12 15:07:17 +08:00
houseme ee96e194a3 build(deps): bump s3s for header parsing fix (#3344) 2026-06-11 04:28:36 +00:00
houseme b9c924a6ed build(ci): update macOS runner and bump regex/uuid (#3318) 
* ci(build): use macos-26-intel for x86_64 job

* build(deps): bump regex and uuid

* ci(build): disable cross for macos x86_64 job

---------

Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
 2026-06-10 04:42:10 +00:00
houseme 68400933b5 chore(release): prepare 1.0.0-beta.8 (#3317) 
* chore(release): prepare 1.0.0-beta.8

* chore(release): align release assets for 1.0.0-beta.8
 2026-06-10 04:34:39 +00:00
安正超 45391ac776 feat(storage-api): add contract crate (#3310) 2026-06-09 20:06:15 +00:00
houseme aabda41ae5 build: upgrade Rust baseline to 1.96.0 (#3291) 
* build: upgrade Rust baseline to 1.96.0

* ci: pin Rust workflows to 1.96.0

* remove

* ci: pin setup action to Rust 1.96.0

* build: pin toolchain to Rust 1.96.0
 2026-06-08 20:32:37 +00:00
houseme 9354b939ca refactor(zip): simplify archive extraction path (#3290) 
* refactor(zip): simplify archive extraction path

* build(deps): align datafusion and http updates

* fix(test): restore pre-commit compatibility
 2026-06-08 20:03:02 +00:00
唐小鸭 f7724d223b feat(rio): rio_v2 is compatible with minio for storing data. (#3115) 
* Set up a compatibility layer for replacing old Rio components with new ones.

* fix(rio). compress range

* feat(rio). Add the experimental feature rio_v2 to support minio data at the binary level.

* feat(rio_v2): add sse-c test

* test compression component

* simple fix

* fix minlz encode

* fix metadata

* fix kms key cache error

* Update launch.json

* ci: set nix crate download user agent

* fix: gate obs pyroscope backend

* ignore minio test

* fix encrypt check

* fix

* fix

* fix

* Update object_usecase.rs

* Update ci.yml

* fix

* ci add rio-v2 test

* fix

* ci fix

* fix

* Reconstructed into a more reasonable compatibility mode

* fix

* fix

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-06-08 11:59:14 +00:00
houseme c479f3d0cb fix(ecstore): gate rustix fs diagnostics on windows (#3267) 
* fix(ecstore): gate rustix fs diagnostics on windows

* fix(ecstore): improve windows disk validation diagnostics

* build(deps): bump workspace dependency versions
 2026-06-08 00:05:55 +00:00
安正超 7a9bf707ee feat: add security governance contract types (#3270) 2026-06-07 23:14:11 +00:00
houseme 66b8699927 chore(release): prepare 1.0.0-beta.7 (#3184) 
* chore(release): prepare 1.0.0-beta.7

* chore(release): align release assets for 1.0.0-beta.7
 2026-06-03 04:18:50 +00:00
dependabot[bot] cbb6f9c76f chore(deps): bump the dependencies group with 6 updates (#3151) 
* chore(deps): bump the dependencies group with 6 updates

Bumps the dependencies group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [hyper](https://github.com/hyperium/hyper) | `1.10.0` | `1.10.1` |
| [serial_test](https://github.com/palfrey/serial_test) | `3.4.0` | `3.5.0` |
| [snafu](https://github.com/shepmaster/snafu) | `0.9.0` | `0.9.1` |
| [uuid](https://github.com/uuid-rs/uuid) | `1.23.1` | `1.23.2` |
| [dial9-tokio-telemetry](https://github.com/dial9-rs/dial9-tokio-telemetry) | `0.3.12` | `0.3.13` |
| [pyroscope](https://github.com/grafana/pyroscope-rs) | `2.0.5` | `2.0.6` |


Updates `hyper` from 1.10.0 to 1.10.1
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v1.10.0...v1.10.1)

Updates `serial_test` from 3.4.0 to 3.5.0
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v3.4.0...v3.5.0)

Updates `snafu` from 0.9.0 to 0.9.1
- [Changelog](https://github.com/shepmaster/snafu/blob/main/CHANGELOG.md)
- [Commits](https://github.com/shepmaster/snafu/compare/0.9.0...0.9.1)

Updates `uuid` from 1.23.1 to 1.23.2
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.23.1...v1.23.2)

Updates `dial9-tokio-telemetry` from 0.3.12 to 0.3.13
- [Release notes](https://github.com/dial9-rs/dial9-tokio-telemetry/releases)
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dial9-rs/dial9-tokio-telemetry/compare/dial9-tokio-telemetry-v0.3.12...dial9-tokio-telemetry-v0.3.13)

Updates `pyroscope` from 2.0.5 to 2.0.6
- [Release notes](https://github.com/grafana/pyroscope-rs/releases)
- [Changelog](https://github.com/grafana/pyroscope-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/grafana/pyroscope-rs/compare/lib-2.0.5...lib-2.0.6)

---
updated-dependencies:
- dependency-name: hyper
  dependency-version: 1.10.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serial_test
  dependency-version: 3.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: snafu
  dependency-version: 0.9.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: uuid
  dependency-version: 1.23.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: dial9-tokio-telemetry
  dependency-version: 0.3.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: pyroscope
  dependency-version: 2.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix(profiling): pin pyroscope to 2.0.5, add jemalloc memory profiling, unify platform gates

Pin pyroscope to =2.0.5 to avoid duplicate `perf_signal_handler` symbol
caused by 2.0.6 vendoring pprof-rs internally. 2.0.5 depends on external
pprof-pyroscope-fork (v0.1500.4), which Cargo unifies with the workspace's
pprof-pyroscope-fork — a single symbol instance, no linker collision.

Add jemalloc_backend for continuous memory profiling export to Pyroscope,
alongside the existing pprof-based CPU profiling. Both CPU and memory
profiling now work on linux and macos.

Unify platform gates from `cfg(all(target_os = "linux", target_env = "gnu",
target_arch = "x86_64"))` to `cfg(any(target_os = "linux",
target_os = "macos"))`, switching macOS global allocator from mimalloc to
jemalloc.

* style: format cfg attributes in allocator_reclaim.rs

* chore(deps): ignore pyroscope 2.x in dependabot, add profile_type tag

Add pyroscope 2.x to dependabot ignore list to prevent auto-upgrade
past 2.0.5 (>=2.0.6 vendors pprof-rs, causing duplicate symbol conflict).

Add profile_type=cpu tag to CPU profiling agent for differentiation
from memory profiling agent in Pyroscope UI.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-01 07:21:47 +00:00
houseme 9f78cd3896 chore(deps): bump workspace dependencies (#3118) 
* chore(deps): bump workspace dependencies

- bump direct dependency versions in Cargo.toml\n- refresh Cargo.lock to resolve updated crates

* fix(deps): avoid pyroscope and pprof linker collision

- downgrade pyroscope from 2.0.6 to 2.0.5\n- keep a single pprof implementation path via pprof-pyroscope-fork\n- fix duplicate perf_signal_handler during test linking
 2026-05-29 09:36:19 +00:00
houseme 28bac7fbd6 chore(release): prepare 1.0.0-beta.6 (#3104) 
* chore(release): prepare 1.0.0-beta.6

* ci(nix): harden flaky crate fetch handling

* ci(nix): drop magic cache and force fallback

* ci(nix): set explicit user-agent for crate fetch

* ci(nix): adopt determinate nix workflow stack

* ci(nix): add nix user-agent suffix for fetches

* ci(nix): add flakehub cache and align determinate actions

* ci(nix): pin determinate actions to release tags

* ci(nix): disable flakehub auth path in CI cache

* ci(nix): restore stable magic cache baseline

* ci(nix): trust local magic cache substituter

* ci(nix): stop forcing Node24 for JS actions

* ci(nix): drop manual localhost cache config

* ci(nix): adopt latest determinate flakehub stack

* ci(nix): record latest determinate workflow state
 2026-05-28 09:21:16 +00:00
houseme 4648de9e62 chore(deploy): refine systemd and nixos service docs (#3096) 
* fix(deploy): simplify systemd service startup

* docs(deploy): add nixos systemd service example

* docs(deploy): add nixos service guide

* chore(deps): update pulsar and pyroscope

* docs(deploy): refine nixos service guidance

* fix(deploy): use explicit rustfs server entrypoint
 2026-05-27 14:47:47 +00:00
houseme 909f0d57fb feat: improve degraded readiness reporting and shutdown handling (#3089) 
* fix(server): unify runtime readiness and shutdown flow

* refactor(server): decouple readiness shared types

* refactor(server): keep readiness types crate-private

* refactor(server): await protocol shutdown handles

* fix(server): bypass cached startup readiness

* feat(health): expose degraded readiness reasons

* fix(health): preserve raw IAM readiness in degraded reports
 2026-05-27 03:25:28 +00:00
houseme f1207a9e28 docs(skills): quote release bump description (#3077) 
* docs(skills): quote release bump description

* chore(deps): bump lapin and rumqttc-next
 2026-05-24 19:59:29 +00:00
houseme d74e6eb042 refactor(tls): centralize runtime foundation (#3065) 
* refactor(targets): move notify net helpers from utils

* refactor(tls): centralize runtime foundation

* refactor(targets): move notify net helpers from utils

* refactor(tls): centralize runtime foundation

* feat(tls-runtime): add TLS debug state and admin handler

* refactor(tls-runtime): unify TLS debug consumer status view

* fix(tls): address PR3065 review feedback

* refactor(tls): align debug status payload types

* refactor(targets): harden TLS hot reload paths

* fix(targets): resolve review-4348251652 findings

* fix(targets): finalize tls runtime review follow-ups

* fix(targets): harden tls reload and review follow-ups

* fix(targets): align tls reload handling across targets

* fix(targets): finalize tls reload state and metrics updates

* chore(deps): trim unused TLS deps

* style(targets): normalize TLS reload formatting

* refactor(targets): introduce tls runtime adapter path

* chore: update workspace manifests for tls refactor

* fix(tls): stabilize material reload and audit workflow

* fix(targets): refresh tls fingerprint flow across sinks

* fix(tls): align runtime coordinator and http reader updates

* fix(sftp): simplify protocol error mapping

* fix(tls): harmonize material loading behavior

* fix(server): finalize tls material wiring in startup flow

* fix(protos): tighten tls generation cache and deps
 2026-05-24 06:41:15 +00:00
houseme 6264be437c fix(storage): add scoped timeout policy and startup fs guardrail (#3056) 
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

* style: fix cargo fmt formatting in disk_store.rs

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).

Closes #2790

* feat(startup): add unsupported filesystem policy guardrail

* chore(deps): refresh lockfile and dependency pins

* feat(ecstore): add scoped timeout health-action policy

* docs(config): document drive timeout health-action policy

* refactor(ecstore): cache timeout health policy per disk wrapper

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends (#2838)

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

* style: fix cargo fmt formatting in disk_store.rs

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).

Closes #2790

* fix(utils): map verified Linux filesystem magic values (#3051)

* fix(utils): cover sha256 checksum validation (#3052)

* fix(utils): cover sha256 checksum validation

* docs: clarify sha256 checksum validation

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>

* refactor(config): replace network mount mode with timeout profile preset

* fix(review): align fallback defaults and extend fs-type detection

* fix(review): cache timeout profile and restore probe timeout semantics

* refactor(ecstore): cache timeout health policy lookup

* perf(ecstore): cache active probe timeout per monitor task

---------

Co-authored-by: mistik <mistiklord4@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-05-22 06:37:30 +00:00
houseme 503f89bf0e chore(deps): bump aws sdk and config (#3035) 2026-05-21 01:47:34 +00:00
houseme dcbffb084f chore(deps): refresh workspace deps and linux fs_type gating (#3030) 
* chore(deps): refresh workspace deps and linux fs_type gating

- refresh workspace dependency pins and lockfile updates

- remove now-unused crate dependency entries in multiple Cargo.toml files

- enable profiling export defaults in config and scripts/run.sh

- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings

* fix(utils): simplify fs_type linux gating

- keep fs_type module-level linux cfg in os::mod

- remove redundant linux cfg on get_fs_type and test module

* chore(deps): bump s3s git revision

- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5

- refresh Cargo.lock source entry for s3s

* chore(dev): allow mysql_async git source and env overrides

- allow mysql_async git source in deny.toml allow-git list

- make scripts/run.sh core env vars overrideable via existing shell env

* fix(utils): import get_fs_type in fs_type tests

- add explicit super::get_fs_type import in fs_type test module

- fix Linux E0425 unresolved function errors in unit tests

* chore(dev): tune run script observability defaults

- make profiling export env overrideable in scripts/run.sh

- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0

- update allow-git review window comments in deny.toml

* test(obs): stabilize profiling env alias tests
 2026-05-20 15:03:07 +00:00
houseme b2dfdf85de chore(release): prepare 1.0.0-beta.4 (#3032) 
* chore(release): prepare 1.0.0-beta.4

* docs(skill): refine rustfs spec changelog rule

* docs(skill): optimize rustfs release bump workflow
 2026-05-20 13:56:43 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012) 
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
 2026-05-19 12:50:25 +00:00
houseme c4248078d6 chore: update dependencies (#2890) 
* chore: update dependencies

* build(deps): bump the dependencies group with 5 updates

* build(deps): bump the dependencies group with 6 updates (#2908)

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>

* fix(ecstore): narrow Windows URL drive path rewrite

* chore(deps): bump starshard to 1.2.0

* revert(ecstore): restore endpoint Windows path behavior

* up

* up

* up

* perf(notify): use cached snapshot mode for scans

* fix

* chore(deps): bump workspace dependency versions

* chore(deps): refresh pinned dependency references

* chore(ci): align profiling and decommission tooling updates

- enable tokio_unstable cfg in performance profiling build flags\n- bump Rust base image from 1.93 to 1.95 in source and decommission Dockerfiles\n- remove obsolete compose version key from docker-compose-simple.yml\n- add standard Apache-2.0 license header to docker-compose.decommission.yml

* chore(deps): bump the dependencies group across 1 directory with 7 updates (#2994)

Bumps the dependencies group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo) | `0.39.1` | `0.39.2` |
| [dial9-tokio-telemetry](https://github.com/dial9-rs/dial9-tokio-telemetry) | `0.3.9` | `0.3.10` |
| [opentelemetry](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-otlp](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.1` | `0.32.0` |
| [opentelemetry_sdk](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-semantic-conventions](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-stdout](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |



Updates `sysinfo` from 0.39.1 to 0.39.2
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/main/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.39.1...v0.39.2)

Updates `dial9-tokio-telemetry` from 0.3.9 to 0.3.10
- [Release notes](https://github.com/dial9-rs/dial9-tokio-telemetry/releases)
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dial9-rs/dial9-tokio-telemetry/compare/dial9-tokio-telemetry-v0.3.9...dial9-tokio-telemetry-v0.3.10)

Updates `opentelemetry` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/opentelemetry-prometheus-0.31.0...opentelemetry-0.32.0)

Updates `opentelemetry-otlp` from 0.31.1 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/opentelemetry-otlp-0.31.1...opentelemetry-otlp-0.32.0)

Updates `opentelemetry_sdk` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry_sdk-0.32.0)

Updates `opentelemetry-semantic-conventions` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry-semantic-conventions-0.32.0)

Updates `opentelemetry-stdout` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry-stdout-0.32.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-version: 0.39.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: dial9-tokio-telemetry
  dependency-version: 0.3.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: opentelemetry
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-otlp
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry_sdk
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-semantic-conventions
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-stdout
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

* chore(deps): bump workspace dependency versions

* chore(deps): refresh lockfile windows crate graph

* chore(bench): align snapshot mode labels with coverage

* chore(bench): clarify tested snapshot modes

* fix(review): address PR2890 Copilot comments

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-19 04:58:42 +00:00
houseme 6e12289339 fix(runtime): finalize issue 2941 profiling cleanup (#2983) 
* perf(runtime): narrow profiling support and upgrade starshard

* style(notify): normalize starshard imports

* perf(ecstore): reduce list_path_raw coordination overhead

* docs(scripts): add issue 2941 perf capture workflow

* fix(runtime): finalize issue 2941 profiling cleanup

* build(deps): bump quick-xml to 0.40.0

* chore(scripts): untrack local perf capture guide

* fix(scripts): honor label in perf capture output
 2026-05-16 11:09:04 +00:00
houseme cd2cd74314 ci: force Node24 in Nix workflows with pinned actions (#2966) 2026-05-14 11:13:42 +00:00
houseme bdb98598d2 chore(release): prepare 1.0.0-beta.3 (#2957) 2026-05-14 04:27:26 +00:00
houseme 53ec1b95d8 keep sftp e2e tests buildable (#2897) 2026-05-10 11:52:15 +00:00
escapecode 96b293bf8a feat(sftp): add SFTPv3 protocol support (#2875) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-10 03:48:42 +00:00