gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 16:28:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,625 Commits 21 Branches 117 Tags
7bdb25ae9d2d4781bc1d33ec8316b67d71e652cc
Commit Graph

580 Commits

Author SHA1 Message Date
houseme 46775ae019 feat: harden runtime capability snapshots (#3784) 
* feat(admin): expose runtime capability snapshots

* feat(runtime): refine workload admission snapshots

* test(ci): align architecture migration checks

* build(deps): bump memmap2 for RUSTSEC-2026-0186

* build(deps): refresh cargo deny lockfile
 2026-06-23 18:10:49 +08:00
houseme 2f85ef4455 fix(security): extend outbound egress guard (#3744) 2026-06-22 19:20:31 +08:00
dependabot[bot] 5a601869f6 chore(deps): bump sysinfo from 0.39.3 to 0.39.4 in the dependencies group (#3712) 2026-06-22 09:53:57 +08:00
houseme 9e6d0c7292 chore(deps): refresh workflow actions and crate pins (#3696) 2026-06-21 20:29:17 +08:00
安正超 08b8f58e64 refactor: prune notify ecstore object alias (#3620) 2026-06-19 20:11:56 +08:00
houseme 8cf3c0bfbd fix(ecstore): support MinIO DARE fixture compatibility (#3590) 2026-06-19 10:13:39 +08:00
houseme 53ff19dd2d fix(kms): harden local master key storage (#3589) 2026-06-19 08:50:35 +08:00
安正超 b1c6578df1 refactor: narrow test harness compatibility surfaces (#3592) 2026-06-19 07:10:52 +08:00
安正超 c28fee0013 refactor: continue storage api contract cleanup (#3580) 
* refactor: move delete object contracts to storage api

* refactor: narrow store api compatibility exports

* refactor: route table catalog test through storage compat
 2026-06-18 22:42:02 +08:00
houseme 8cf11af649 fix(security): add shared outbound egress guard (#3567) 
* fix(security): block unsafe outbound webhook targets

* chore: keep issue-3557 plan local only
 2026-06-18 15:30:09 +08:00
安正超 54bbd05b25 refactor: move object list helper contracts (#3546) 2026-06-18 06:09:33 +08:00
houseme 87a3d107d6 test(property): add path validator invariants (#3541) 
test(property): add path validator invariants for #3525
 2026-06-18 02:14:24 +08:00
houseme bc769948ab test(property): add erasure recoverability invariants (#3540) 2026-06-18 01:34:07 +08:00
安正超 53337a6f71 refactor: move HTTP range helper contracts (#3533) 2026-06-18 00:47:23 +08:00
houseme cc84d9065e test(property): add filemeta invariants (#3536) 
* test(property): add filemeta invariants for #3523

* docs: relocate issue-3518 property testing plan

* docs: keep issue-3518 property plan local only
 2026-06-18 00:36:55 +08:00
houseme 841f86fe90 test(property): expand parser invariants (#3534) 
* test(property): harden parser invariants for #3522

* test(property): satisfy clippy for #3522
 2026-06-17 23:52:30 +08:00
安正超 919deeb816 refactor: move object option helper contracts (#3521) 2026-06-17 22:56:10 +08:00
Henry Guo 65c724c786 feat(table-catalog): add manifest reachability cleanup (#3484) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-15 22:29:23 +08:00
houseme adec195486 chore(scanner): enable pulsar telemetry feature (#3481) 2026-06-15 18:43:14 +08:00
dependabot[bot] fbf1f9e5b8 build(deps): bump brotli from 8.0.3 to 8.0.4 in the dependencies group (#3458) 
* build(deps): bump brotli from 8.0.3 to 8.0.4 in the dependencies group

Bumps the dependencies group with 1 update: [brotli](https://github.com/dropbox/rust-brotli).


Updates `brotli` from 8.0.3 to 8.0.4
- [Release notes](https://github.com/dropbox/rust-brotli/releases)
- [Commits](https://github.com/dropbox/rust-brotli/compare/8.0.3...8.0.4)

---
updated-dependencies:
- dependency-name: brotli
  dependency-version: 8.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump s3s from cf4c3346 to c59b62f7ba in the dependencies group

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-15 12:06:01 +08:00
houseme 036741cb1c refactor(request-id): align contracts and lock field names (#3454) 
* refactor(request-id): align header log and trace contracts

* test(contract): lock external request-id field names

* chore(deps): drop unused rustfs-ecstore links

Co-Authored-By: heihutu <heihutu@gmail.com>
 2026-06-15 01:59:11 +08:00
cxymds 046d5386ba feat: stream object zip downloads (#3380) 
* feat: stream object zip downloads

* fix: stream zip downloads page by page

* fix: prepare zip downloads before streaming

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-14 14:26:44 +08:00
houseme e8012bd1ba refactor(logging): normalize admin telemetry and error messages (#3430) 2026-06-14 13:27:10 +08:00
houseme 0d68851f7a refactor(logging): standardize rustfs runtime events (#3396) 
* fix(rpc): adjust log levels and reduce noise in storage RPC layer

Issue #682: pool rebalance normal flow logs were using warn! instead
of info!. Additionally, several high-frequency RPC entry points (ping,
write_stream, read_at, walk_dir) were logging at info! level, causing
unnecessary log noise in production.

Changes:
- load_rebalance_meta: warn! → info! for normal flow (3 occurrences)
- ping body decode: info! → debug! (fires on every health check)
- write_stream/read_at/walk_dir entry: info! → debug!
- metrics.rs: lowercase error messages, add structured error field
- http_service.rs: add structured error field to walk_dir failure
- Add tracing::instrument fields for start_rebalance context

* refactor(logging): standardize rustfs runtime events

* build(deps): bump workspace dependency versions

* build(deps): pin time to 0.3.47

* build(deps): update postgres client versions
 2026-06-13 08:39:47 +08:00
安正超 434b71e569 feat(extension): add admin views (#3386) 2026-06-12 15:54:42 +08:00
安正超 1e7c376586 feat(targets): add extension schema adapter (#3385) 2026-06-12 15:27:55 +08:00
安正超 7d6d56a547 feat(extension): add schema contracts (#3384) 2026-06-12 15:07:17 +08:00
安正超 7146c893cb refactor(storage-api): remove legacy bucket DTO re-export (#3364) 2026-06-11 21:24:56 +08:00
安正超 2205991180 refactor(config): extract server config model (#3351) 2026-06-11 16:11:17 +08:00
houseme 7d38b0cf90 refactor(obs): unify local and otlp log output (#3352) 
wip(obs): start lg-002 output unification
 2026-06-11 16:10:54 +08:00
houseme 8d337c4e87 feat(obs): add logging redaction guard rails (#3349) 
* feat(obs): add logging redaction guard rails

* chore(docs): keep logging plans out of lg-001 pr

* test(obs): guard against unmasked access key logs

* test(obs): enforce masked access key log patterns
 2026-06-11 15:24:32 +08:00
houseme ee96e194a3 build(deps): bump s3s for header parsing fix (#3344) 2026-06-11 04:28:36 +00:00
安正超 87968275a7 refactor(storage): route maintenance inventory reads (#3337) 2026-06-11 10:32:48 +08:00
安正超 94c53af264 refactor(storage): use admin API for observability reads (#3335) 2026-06-11 08:13:15 +08:00
安正超 8a90bda16a refactor(admin): use storage admin drive counts (#3332) 2026-06-11 06:36:18 +08:00
安正超 d9ddd1bedc feat(storage-api): add disk inventory contract (#3330) 2026-06-10 16:33:31 +00:00
安正超 bb5d9565a6 feat(storage-api): add bucket DTO contract (#3314) 
* feat(storage-api): add bucket DTO contract

* ci(build): increase workflow timeout to 90 minutes

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-10 07:16:14 +00:00
houseme b9c924a6ed build(ci): update macOS runner and bump regex/uuid (#3318) 
* ci(build): use macos-26-intel for x86_64 job

* build(deps): bump regex and uuid

* ci(build): disable cross for macos x86_64 job

---------

Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
 2026-06-10 04:42:10 +00:00
houseme 68400933b5 chore(release): prepare 1.0.0-beta.8 (#3317) 
* chore(release): prepare 1.0.0-beta.8

* chore(release): align release assets for 1.0.0-beta.8
 2026-06-10 04:34:39 +00:00
安正超 3795d44b86 feat(storage-api): add error code contract (#3313) 2026-06-10 09:03:29 +08:00
安正超 45391ac776 feat(storage-api): add contract crate (#3310) 2026-06-09 20:06:15 +00:00
安正超 03eb10b07f feat: add KMS config redaction safeguards (#3303) 2026-06-09 14:08:58 +00:00
安正超 b9a3a2245a feat: add admin route policy inventory (#3286) 
* feat: add admin route policy inventory

* fix: align table route policy actions

* fix: align load table route policy action
 2026-06-09 02:17:50 +00:00
houseme 9354b939ca refactor(zip): simplify archive extraction path (#3290) 
* refactor(zip): simplify archive extraction path

* build(deps): align datafusion and http updates

* fix(test): restore pre-commit compatibility
 2026-06-08 20:03:02 +00:00
唐小鸭 f7724d223b feat(rio): rio_v2 is compatible with minio for storing data. (#3115) 
* Set up a compatibility layer for replacing old Rio components with new ones.

* fix(rio). compress range

* feat(rio). Add the experimental feature rio_v2 to support minio data at the binary level.

* feat(rio_v2): add sse-c test

* test compression component

* simple fix

* fix minlz encode

* fix metadata

* fix kms key cache error

* Update launch.json

* ci: set nix crate download user agent

* fix: gate obs pyroscope backend

* ignore minio test

* fix encrypt check

* fix

* fix

* fix

* Update object_usecase.rs

* Update ci.yml

* fix

* ci add rio-v2 test

* fix

* ci fix

* fix

* Reconstructed into a more reasonable compatibility mode

* fix

* fix

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-06-08 11:59:14 +00:00
houseme c479f3d0cb fix(ecstore): gate rustix fs diagnostics on windows (#3267) 
* fix(ecstore): gate rustix fs diagnostics on windows

* fix(ecstore): improve windows disk validation diagnostics

* build(deps): bump workspace dependency versions
 2026-06-08 00:05:55 +00:00
安正超 7a9bf707ee feat: add security governance contract types (#3270) 2026-06-07 23:14:11 +00:00
安正超 3bd89944c2 perf(erasure): remove UUID from clone + increase encode inflight budget (#3212) 
* perf(erasure): remove UUID from clone + increase encode inflight budget

Two targeted optimizations for the erasure encoding hot path:

1. Erasure::clone() no longer generates Uuid::new_v4() per clone.
   The _id field is unused in the hot path; reusing the original ID
   eliminates a CSPRNG call per block encode (100 calls for a 100MB
   object with 1MB blocks).

2. Default RUSTFS_ERASURE_ENCODE_MAX_INFLIGHT_BYTES raised from 8MB
   to 32MB. This increases the encode pipeline depth from ~5 to ~20
   blocks, allowing more read-ahead between the encoder and disk
   writer stages. The per-request memory bound is still controlled
   by the 8-block hard cap and the env var override.

3. Added encode_data_owned() utility method for zero-copy encoding
   when the caller already owns a heap buffer (Vec<u8> → BytesMut
   via Bytes::try_into_mut). Not used in the hot path yet but
   available for future callers.

All 1157 ecstore tests pass. Criterion micro-benchmarks show no
regression (< 2% variance). Single-machine warp E2E tests were
inconclusive due to high variance; a dedicated multi-disk test
environment is needed for reliable E2E comparison.

Ref: https://github.com/rustfs/backlog/issues/659

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: update Cargo.lock

* fix(erasure): align encode inflight cap and tests

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-04 13:49:36 +00:00
houseme 66b8699927 chore(release): prepare 1.0.0-beta.7 (#3184) 
* chore(release): prepare 1.0.0-beta.7

* chore(release): align release assets for 1.0.0-beta.7
 2026-06-03 04:18:50 +00:00
dependabot[bot] cbb6f9c76f chore(deps): bump the dependencies group with 6 updates (#3151) 
* chore(deps): bump the dependencies group with 6 updates

Bumps the dependencies group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [hyper](https://github.com/hyperium/hyper) | `1.10.0` | `1.10.1` |
| [serial_test](https://github.com/palfrey/serial_test) | `3.4.0` | `3.5.0` |
| [snafu](https://github.com/shepmaster/snafu) | `0.9.0` | `0.9.1` |
| [uuid](https://github.com/uuid-rs/uuid) | `1.23.1` | `1.23.2` |
| [dial9-tokio-telemetry](https://github.com/dial9-rs/dial9-tokio-telemetry) | `0.3.12` | `0.3.13` |
| [pyroscope](https://github.com/grafana/pyroscope-rs) | `2.0.5` | `2.0.6` |


Updates `hyper` from 1.10.0 to 1.10.1
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v1.10.0...v1.10.1)

Updates `serial_test` from 3.4.0 to 3.5.0
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v3.4.0...v3.5.0)

Updates `snafu` from 0.9.0 to 0.9.1
- [Changelog](https://github.com/shepmaster/snafu/blob/main/CHANGELOG.md)
- [Commits](https://github.com/shepmaster/snafu/compare/0.9.0...0.9.1)

Updates `uuid` from 1.23.1 to 1.23.2
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.23.1...v1.23.2)

Updates `dial9-tokio-telemetry` from 0.3.12 to 0.3.13
- [Release notes](https://github.com/dial9-rs/dial9-tokio-telemetry/releases)
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dial9-rs/dial9-tokio-telemetry/compare/dial9-tokio-telemetry-v0.3.12...dial9-tokio-telemetry-v0.3.13)

Updates `pyroscope` from 2.0.5 to 2.0.6
- [Release notes](https://github.com/grafana/pyroscope-rs/releases)
- [Changelog](https://github.com/grafana/pyroscope-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/grafana/pyroscope-rs/compare/lib-2.0.5...lib-2.0.6)

---
updated-dependencies:
- dependency-name: hyper
  dependency-version: 1.10.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: serial_test
  dependency-version: 3.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: snafu
  dependency-version: 0.9.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: uuid
  dependency-version: 1.23.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: dial9-tokio-telemetry
  dependency-version: 0.3.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: pyroscope
  dependency-version: 2.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix(profiling): pin pyroscope to 2.0.5, add jemalloc memory profiling, unify platform gates

Pin pyroscope to =2.0.5 to avoid duplicate `perf_signal_handler` symbol
caused by 2.0.6 vendoring pprof-rs internally. 2.0.5 depends on external
pprof-pyroscope-fork (v0.1500.4), which Cargo unifies with the workspace's
pprof-pyroscope-fork — a single symbol instance, no linker collision.

Add jemalloc_backend for continuous memory profiling export to Pyroscope,
alongside the existing pprof-based CPU profiling. Both CPU and memory
profiling now work on linux and macos.

Unify platform gates from `cfg(all(target_os = "linux", target_env = "gnu",
target_arch = "x86_64"))` to `cfg(any(target_os = "linux",
target_os = "macos"))`, switching macOS global allocator from mimalloc to
jemalloc.

* style: format cfg attributes in allocator_reclaim.rs

* chore(deps): ignore pyroscope 2.x in dependabot, add profile_type tag

Add pyroscope 2.x to dependabot ignore list to prevent auto-upgrade
past 2.0.5 (>=2.0.6 vendors pprof-rs, causing duplicate symbol conflict).

Add profile_type=cpu tag to CPU profiling agent for differentiation
from memory profiling agent in Pyroscope UI.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-01 07:21:47 +00:00