623fc801f1
feat(replication): support custom TLS for bucket targets ( #3825 )
...
* feat(replication): support insecure https bucket targets
* feat(replication): support custom ca bucket targets
* test(replication): satisfy e2e clippy for TLS helpers
* fix(replication): avoid native root panic for custom trust stores
* test(replication): decouple private IP target test from TLS roots
* test(replication): use target TLS client in private IP unit test
2026-06-24 20:23:20 +08:00
0d68851f7a
refactor(logging): standardize rustfs runtime events ( #3396 )
...
* fix(rpc): adjust log levels and reduce noise in storage RPC layer
Issue #682 : pool rebalance normal flow logs were using warn! instead
of info!. Additionally, several high-frequency RPC entry points (ping,
write_stream, read_at, walk_dir) were logging at info! level, causing
unnecessary log noise in production.
Changes:
- load_rebalance_meta: warn! → info! for normal flow (3 occurrences)
- ping body decode: info! → debug! (fires on every health check)
- write_stream/read_at/walk_dir entry: info! → debug!
- metrics.rs: lowercase error messages, add structured error field
- http_service.rs: add structured error field to walk_dir failure
- Add tracing::instrument fields for start_rebalance context
* refactor(logging): standardize rustfs runtime events
* build(deps): bump workspace dependency versions
* build(deps): pin time to 0.3.47
* build(deps): update postgres client versions
2026-06-13 08:39:47 +08:00
dcbffb084f
chore(deps): refresh workspace deps and linux fs_type gating ( #3030 )
...
* chore(deps): refresh workspace deps and linux fs_type gating
- refresh workspace dependency pins and lockfile updates
- remove now-unused crate dependency entries in multiple Cargo.toml files
- enable profiling export defaults in config and scripts/run.sh
- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings
* fix(utils): simplify fs_type linux gating
- keep fs_type module-level linux cfg in os::mod
- remove redundant linux cfg on get_fs_type and test module
* chore(deps): bump s3s git revision
- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5
- refresh Cargo.lock source entry for s3s
* chore(dev): allow mysql_async git source and env overrides
- allow mysql_async git source in deny.toml allow-git list
- make scripts/run.sh core env vars overrideable via existing shell env
* fix(utils): import get_fs_type in fs_type tests
- add explicit super::get_fs_type import in fs_type test module
- fix Linux E0425 unresolved function errors in unit tests
* chore(dev): tune run script observability defaults
- make profiling export env overrideable in scripts/run.sh
- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0
- update allow-git review window comments in deny.toml
* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
53ec1b95d8
keep sftp e2e tests buildable ( #2897 )
2026-05-10 11:52:15 +00:00
96b293bf8a
feat(sftp): add SFTPv3 protocol support ( #2875 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-10 03:48:42 +00:00
d949d4e794
fix: avoid sending HEAD bodies over TLS HTTP/2 ( #2648 )
...
Signed-off-by: 唐小鸭 <tangtang1251@qq.com >
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-04-25 01:33:08 +00:00
478720d2ee
Centralize lifecycle state updates and fix systemd running status ( #2567 )
2026-04-17 03:20:11 +00:00
68d3dba9fc
fix: revert standalone #2351 artifacts (phase 5) ( #2535 )
2026-04-14 22:35:05 +08:00
c9c2428373
fix(download): preserve archive download integrity ( #2415 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
2026-04-07 16:25:34 +08:00
32bf8f5bf3
feat(storage): add direct chunk GET fast path ( #2351 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: cxymds <Cxymds@qq.com >
2026-04-07 08:33:46 +08:00
2681731443
fix(checksum): align multipart CRC64NVME with full object ( #2286 )
2026-03-25 12:44:46 +08:00
28f57b228c
feat(s3): advance parity coverage ( #2278 )
2026-03-24 17:29:33 +08:00
d17d2083d4
feat(targets): enhance webhook TLS support with custom CA and skip-verify ( #1994 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: heihutu <heihutu@gmail.com >
2026-02-27 21:24:49 +08:00
5c2eda356e
feat: migrate FTP/SFTP to protocols crate and update dependencies ( #1580 )
...
Signed-off-by: yxrxy <yxrxytrigger@gmail.com >
Signed-off-by: houseme <housemecn@gmail.com >
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-02-06 20:58:42 +08:00
90ed75a3dc
refactor(utils/os): Optimize Windows OS utilities and add safety comments ( #1671 )
...
Co-authored-by: weisd <weishidavip@163.com >
2026-01-30 16:13:42 +08:00
dce117840c
refactor: NamespaceLock (nslock), AHM→Heal Crate, and Lock/Clippy Fixes ( #1664 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: weisd <2057561+weisd@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-01-30 13:13:41 +08:00
1d6c8750e7
build(deps): bump the dependencies group with 2 updates ( #1383 )
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com >
2026-01-05 15:33:57 +08:00
cf53a9d84a
chore: replace native-tls with pure rustls for FTPS/SFTP e2e tests ( #1334 )
...
Signed-off-by: yxrxy <1532529704@qq.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-02 11:08:28 +08:00
b8aa8214e2
Feat/ftps&sftp ( #1308 )
...
[feat] ftp / sftp
2025-12-31 09:01:15 +08:00
fba201df3d
fix: harden data usage aggregation and cache handling ( #1102 )
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: loverustfs <hello@rustfs.com >
2025-12-11 09:55:25 +08:00
42d3645d6f
fix(targets): make target removal and reload transactional; prevent reappearing entries ( #662 )
...
* feat: improve code for notify
* upgrade starshard version
* upgrade version
* Fix ETag format to comply with HTTP standards by wrapping with quotes (#592 )
* Initial plan
* Fix ETag format to comply with HTTP standards by wrapping with quotes
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
* bufigx
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
* Improve lock (#596 )
* improve lock
Signed-off-by: Mu junxiang <1948535941@qq.com >
* feat(tests): add wait_for_object_absence helper and improve lifecycle test reliability
Signed-off-by: Mu junxiang <1948535941@qq.com >
* chore: remove dirty docs
Signed-off-by: Mu junxiang <1948535941@qq.com >
---------
Signed-off-by: Mu junxiang <1948535941@qq.com >
* feat(append): implement object append operations with state tracking (#599 )
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <1948535941@qq.com >
* chore: rebase
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
* build(deps): upgrade s3s (#595 )
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
* fix: validate mqtt broker
* improve code for `import`
* fix
* improve
* remove logger from `rustfs-obs` crate
* remove code for config Observability
* fix
* improve code
* fix comment
* up
* up
* upgrade version
* fix
* fmt
* upgrade tokio version to 1.48.0
* upgrade `datafusion` and `reed-solomon-simd` version
* fix
* fmt
* improve code for notify webhook example
* improve code
* fix
* fix
* fmt
---------
Signed-off-by: Mu junxiang <1948535941@qq.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com >
Co-authored-by: Nugine <nugine@foxmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-10-17 15:34:53 +08:00
639bf0c233
Revert "feat(append): implement object append operations with state tracking ( #599 )" ( #646 )
...
This reverts commit 4f73760a45
2025-10-12 23:47:51 +08:00
4f73760a45
feat(append): implement object append operations with state tracking ( #599 )
...
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <1948535941@qq.com >
* chore: rebase
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-09-27 20:06:26 -07:00
9b7f4d477a
Fix Tokio Runtime Initialization: Remove Private API Usage and Ensure IO Enabled ( #587 )
...
* fix: remove code
* improve code for tokio runtime config
* improve code for main
* fix: add tokio enable_all
* upgrade version
* improve for Cargo.toml
2025-09-24 22:23:31 +08:00
29b0935be7
RustFS rustfs-audit Complete Implementation with Enterprise Observability ( #557 )
...
* Initial plan
* Implement core audit system with multi-target fan-out and configuration management
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Changes before error encountered
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Complete audit system with comprehensive observability and test coverage
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* improve code
* fix
* improve code
* fix test
* fix test
* fix
* add `rustfs-audit` to `rustfs`
* upgrade crate version
* fmt
* fmt
* fix
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-09-24 08:23:46 +08:00
9ddf6a011d
feature: support kms && encryt ( #573 )
...
* feat(kms): implement key management service with local and vault backends
Signed-off-by: junxiang Mu <1948535941@qq.com >
* feat(kms): enhance security with zeroize for sensitive data and improve key management
Signed-off-by: junxiang Mu <1948535941@qq.com >
* remove Hashi word
Signed-off-by: junxiang Mu <1948535941@qq.com >
* refactor: remove unused request structs from kms handlers
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-09-22 17:53:05 +08:00
c5df1f92c2
refactor: replace lazy_static with LazyLock and notify crate registry create_targets_from_config ( #311 )
...
* improve code for notify
* improve code for logger and fix typo (#272 )
* Add GNU to build.yml (#275 )
* fix unzip error
* fix url change error
fix url change error
* Simplify user experience and integrate console and endpoint
Simplify user experience and integrate console and endpoint
* Add gnu to build.yml
* upgrade version
* feat: add `cargo clippy --fix --allow-dirty` to pre-commit command (#282 )
Resolves #277
- Add --fix flag to automatically fix clippy warnings
- Add --allow-dirty flag to run on dirty Git trees
- Improves code quality in pre-commit workflow
* fix: the issue where preview fails when the path length exceeds 255 characters (#280 )
* fix
* fix: improve Windows build support and CI/CD workflow (#283 )
- Fix Windows zip command issue by using PowerShell Compress-Archive
- Add Windows support for OSS upload with ossutil
- Replace Chinese comments with English in build.yml
- Fix bash syntax error in package_zip function
- Improve code formatting and consistency
- Update various configuration files for better cross-platform support
Resolves Windows build failures in GitHub Actions.
* fix: update link in README.md leading to a 404 error (#285 )
* add rustfs.spec for rustfs (#103 )
add support on loongarch64
* improve cargo.lock
* build(deps): bump the dependencies group with 5 updates (#289 )
Bumps the dependencies group with 5 updates:
| Package | From | To |
| --- | --- | --- |
| [hyper-util](https://github.com/hyperium/hyper-util ) | `0.1.15` | `0.1.16` |
| [rand](https://github.com/rust-random/rand ) | `0.9.1` | `0.9.2` |
| [serde_json](https://github.com/serde-rs/json ) | `1.0.140` | `1.0.141` |
| [strum](https://github.com/Peternator7/strum ) | `0.27.1` | `0.27.2` |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo ) | `0.36.0` | `0.36.1` |
Updates `hyper-util` from 0.1.15 to 0.1.16
- [Release notes](https://github.com/hyperium/hyper-util/releases )
- [Changelog](https://github.com/hyperium/hyper-util/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper-util/compare/v0.1.15...v0.1.16 )
Updates `rand` from 0.9.1 to 0.9.2
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.9.1...rand_core-0.9.2 )
Updates `serde_json` from 1.0.140 to 1.0.141
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.140...v1.0.141 )
Updates `strum` from 0.27.1 to 0.27.2
- [Release notes](https://github.com/Peternator7/strum/releases )
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Peternator7/strum/compare/v0.27.1...v0.27.2 )
Updates `sysinfo` from 0.36.0 to 0.36.1
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.36.0...v0.36.1 )
---
updated-dependencies:
- dependency-name: hyper-util
dependency-version: 0.1.16
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: rand
dependency-version: 0.9.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: serde_json
dependency-version: 1.0.141
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: strum
dependency-version: 0.27.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: sysinfo
dependency-version: 0.36.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* improve code for logger
* improve
* upgrade
* refactor: 优化构建工作流，统一 latest 文件处理和简化制品上传 (#293 )
* Refactor: DatabaseManagerSystem as global
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix: fmt
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Test: add e2e_test for s3select
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Test: add test script for e2e
Signed-off-by: junxiang Mu <1948535941@qq.com >
* improve code for registry and intergation
* improve code for registry `create_targets_from_config`
* fix
* Feature up/ilm (#305 )
* fix
* fix
* fix
* fix delete-marker expiration. add api_restore.
* fix
* time retry object upload
* lock file
* make fmt
* fix
* restore object
* fix
* fix
* serde-rs-xml -> quick-xml
* fix
* checksum
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* transfer lang to english
* upgrade clap version from 4.5.41 to 4.5.42
* refactor: replace `lazy_static` with `LazyLock`
* add router
* fix: modify comment
* improve code
* fix typos
* fix
* fix: modify name and fmt
* improve code for registry
* fix test
---------
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: shiro.lee <69624924+shiroleeee@users.noreply.github.com >
Co-authored-by: Marco Orlandin <mipnamic@mipnamic.net >
Co-authored-by: zhangwenlong <zhangwenlong@loongson.cn >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: likewu <likewu@126.com >
2025-07-30 19:02:10 +08:00
de905a878c
Cargo: use workspace dependence
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 11:02:40 +08:00
f3252f989b
Test: Add e2e test case for lock transactional
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 11:00:10 +08:00
612404c47f
Test: add e2e_test for s3select
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-25 15:07:44 +08:00
cad005bc21
refactor(lock): unify NamespaceLock client model and LockRequest API
...
- Refactor NamespaceLock to use a unified client vector and quorum mechanism,
removing legacy local/distributed lock split and related code.
- Update LockRequest to split timeout into acquire_timeout and ttl, and add
builder methods for both.
- Adjust all batch lock APIs to accept ttl and use new LockRequest fields.
- Update all affected tests and documentation for the new API.
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 11:52:57 +08:00
5826396cd0
refactor: Restructure project layout and clean up dependencies ( #30 )
...
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
2025-07-02 19:33:12 +08:00
houseme