01a75b5f58
Add env variable alias compatibility warnings ( #2044 )
2026-03-02 15:34:19 +08:00
d17d2083d4
feat(targets): enhance webhook TLS support with custom CA and skip-verify ( #1994 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: heihutu <heihutu@gmail.com >
2026-02-27 21:24:49 +08:00
9d2b8822cf
refactor: stabilize heal format recovery integration tests ( #1984 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-02-27 15:26:19 +08:00
4413878739
fix(compress): downgrade non-compressible log level and expand excluded file types ( #1780 )
2026-02-16 03:36:22 +08:00
f4e9ef2edc
fix(replication): avoid re-replication loop in Active-Active replication ( #1751 )
...
Co-authored-by: loverustfs <hello@rustfs.com >
2026-02-09 14:11:30 +08:00
0b870d6301
build(deps): bump the dependencies group with 19 updates ( #1745 )
2026-02-07 12:22:14 +08:00
087f58b7c8
fix(lifecycle): lifecycle fixes ( #1625 )
...
Signed-off-by: likewu <likewu@126.com >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-01-31 19:14:47 +08:00
90ed75a3dc
refactor(utils/os): Optimize Windows OS utilities and add safety comments ( #1671 )
...
Co-authored-by: weisd <weishidavip@163.com >
2026-01-30 16:13:42 +08:00
2ee81496b0
fix: deduplicate disks in capacity calculation to prevent inflation ( #1656 )
2026-01-30 00:03:21 +08:00
022e3dfc21
fix:s3 tests fix ( #1652 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-01-29 19:35:03 +08:00
74759b6e99
fix: missing object.key in S3 event notifications for multipart uploads #1609 ( #1624 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-01-27 13:32:56 +08:00
2108c4ad28
fix: remove plaintext credential logging ( #1619 )
2026-01-27 01:47:39 +08:00
d251b9fb35
fix: unify path handling to use S3-standard forward slashes on all platforms ( #1555 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Signed-off-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com >
Co-authored-by: LeonWang0735 <wlywly0735@126.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-26 18:49:21 +08:00
4890fb25c1
fix: listobjects v2 pagination ( #1607 )
2026-01-25 09:30:08 +08:00
3b5f7fb3ff
Fix/correctly handle object lock ( #1556 )
...
Signed-off-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-22 00:15:21 +08:00
a9f499282c
fix: Increase lock acquire timeout for network storage reliability ( #1548 )
2026-01-19 01:14:36 +08:00
109ca7a100
perf(utils): optimize User-Agent generation and platform detection ( #1504 )
2026-01-14 18:08:02 +08:00
68c5c0b834
Use POSIX statvfs, since statfs is not designed to be portable ( #1495 )
2026-01-14 16:03:32 +08:00
760cb1d734
Fix Windows Path Separator Handling in rustfs_utils ( #1464 )
...
Co-authored-by: reatang <tangtang1251@qq.com >
2026-01-11 19:53:51 +08:00
04f441361e
replace winapi to windows crate ( #1455 )
2026-01-10 02:15:08 +08:00
359c9d2d26
Enhance Object Version Management and Replication Status Handling ( #1413 )
2026-01-07 10:44:35 +08:00
5f19eef945
fix: OpenBSD does not support TCPKeepalive intervals ( #1382 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-06 00:41:39 +08:00
ab752458ce
Fix Path Traversal and Enhance Object Validation ( #1387 )
2026-01-05 15:57:15 +08:00
d06397cf4a
fix: try casting available blocks to a u64 on FreeBSD and OpenBSD ( #1360 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-01-04 11:06:14 +08:00
8d7cd4cb1b
chore: upgrade dependencies and migrate to aws-lc-rs ( #1333 )
2026-01-02 00:02:34 +08:00
2924b4e463
Restore globals and add unified TLS/mTLS loading from RUSTFS_TLS_PATH ( #1309 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
2025-12-30 21:55:43 +08:00
eb33e82b56
fix: Prevent panic in GetMetrics gRPC handler on invalid input ( #1291 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
2025-12-29 03:10:23 +08:00
136db7e0c9
feat: add function to extract user-defined metadata keys and integrat… ( #1281 )
...
Signed-off-by: 0xdx2 <xuedamon2@gmail.com >
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-12-27 22:18:16 +08:00
80cfb4feab
Add Disk Timeout and Health Check Functionality ( #1196 )
...
Signed-off-by: weisd <im@weisd.in >
Co-authored-by: loverustfs <hello@rustfs.com >
2025-12-22 17:15:19 +08:00
f3a1431fa5
fix: resolve TLS handshake failure in inter-node communication ( #1201 ) ( #1222 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2025-12-21 16:11:55 +08:00
8dd3e8b534
fix: decode form-urlencoded object names in webhook/mqtt Key field ( #1210 )
...
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-12-20 01:31:09 +08:00
834025d9e3
docs: fix some dead link ( #1053 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
2025-12-08 11:23:24 +08:00
7c6cbaf837
feat: enhance error handling and add precondition checks for object o… ( #1008 )
2025-12-06 20:39:03 +08:00
277d80de13
Fix: Implement priority-based heal queue with comprehensive diagnostic logging ( #884 )
...
* Initial plan
* Implement priority-based heal queue with deduplication
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Apply cargo fmt formatting fixes
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Add queue monitoring, better error handling, and adaptive processing
- Add priority-based queue statistics tracking
- Implement queue capacity warnings (>80% full)
- Process multiple tasks per cycle when capacity allows
- Add proper error logging for failed heal request submissions
- Add Hash trait to HealPriority for HashMap support
- Improve observability with detailed queue status logs
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Add comprehensive diagnostic logging to heal flow
- Add detailed logging for shard availability checks during reads
- Log successful heal request submissions
- Add detailed logging in heal_object for disk checking
- Log each disk's heal decision with reason and endpoint
- Add logging for heal writing and renaming phases
- Improve variable naming (nil_count -> available_shards)
- Add success confirmation messages for renamed healed data
These logs will help diagnose where the heal flow is failing.
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* fix
* feat(heal): enhance robustness, safety, and observability of heal process
- **Logging & Observability**:
- Upgrade critical heal logs from [info](/crates/e2e_test/src/reliant/node_interact_test.rs:196:0-213:1) to `warn` for better visibility.
- Implement structured logging with `tracing` fields for machine readability.
- Add `#[tracing::instrument]` to [HealTask](c/crates/ahm/src/heal/task.rs:182:0-205:1) and [SetDisks](/crates/ecstore/src/set_disk.rs:120:0-131:1) methods for automatic context propagation.
- **Robustness**:
- Add exponential backoff retry (3 attempts) for acquiring write locks in [heal_object](/crates/ahm/src/heal/storage.rs:438:4-460:5) to handle contention.
- Handle [rename_data](/crates/ecstore/src/set_disk.rs:392:4-516:5) failures gracefully by preserving temporary files instead of forcing deletion, preventing potential data loss.
- **Data Safety**:
- Fix [object_exists](/crates/ahm/src/heal/storage.rs:395:4-412:5) to propagate IO errors instead of treating them as "object not found".
- Update [ErasureSetHealer](/crates/ahm/src/heal/erasure_healer.rs:28:0-33:1) to mark objects as failed rather than skipped when existence checks error, ensuring they are tracked for retry.
* fix
* fmt
* improve code for heal_object
* fix
* fix
* fix
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-11-20 00:36:25 +08:00
55d44622ed
list object include deleted support ( #882 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2025-11-18 21:51:10 +08:00
601f3456bc
Fix large file upload freeze with adaptive buffer sizing ( #869 )
...
* Initial plan
* Fix large file upload freeze by increasing StreamReader buffer size
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Add comprehensive documentation for large file upload freeze fix
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* upgrade s3s version
* Fix compilation error: use BufReader instead of non-existent StreamReader::with_capacity
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Update documentation with correct BufReader implementation
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* add tokio feature `io-util`
* Implement adaptive buffer sizing based on file size
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Constants are managed uniformly and fmt code
* fix
* Fix: Trigger self-heal on read when shards missing from rejoined nodes (#871 )
* Initial plan
* Fix: Trigger self-heal when missing shards detected during read
- Added proactive heal detection in get_object_with_fileinfo
- When reading an object, now checks if any shards are missing even if read succeeds
- Sends low-priority heal request to reconstruct missing shards on rejoined nodes
- This fixes the issue where data written during node outage is not healed when node rejoins
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* fix
* Unify CRC implementations to crc-fast (#873 )
* Initial plan
* Replace CRC libraries with unified crc-fast implementation
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* fix
* fix: replace low to Normal
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-11-17 23:15:20 +08:00
b7964081ce
Fix KMS configuration synchronization across cluster nodes ( #855 )
...
* Initial plan
* Add KMS configuration persistence to cluster storage
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Apply code formatting to KMS configuration changes
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* add comment
* fix fmt
* fix
* Fix overlapping dependabot cargo configurations
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* improve code for comment and replace `Once_Cell` to `std::sync::OnceLock`
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-11-16 00:05:03 +08:00
7e1a9e2ede
🔒 Upgrade Cryptography Libraries to Latest RC Versions ( #837 )
...
* fix
* chore: upgrade cryptography libraries to RC versions
- Upgrade aes-gcm to 0.11.0-rc.2 with rand_core support
- Upgrade chacha20poly1305 to 0.11.0-rc.2
- Upgrade argon2 to 0.6.0-rc.2 with std features
- Upgrade hmac to 0.13.0-rc.3
- Upgrade pbkdf2 to 0.13.0-rc.2
- Upgrade rsa to 0.10.0-rc.10
- Upgrade sha1 and sha2 to 0.11.0-rc.3
- Upgrade md-5 to 0.11.0-rc.3
These upgrades provide enhanced security features and performance
improvements while maintaining backward compatibility with existing
encryption workflows.
* add
* improve code
* fix
2025-11-11 21:10:03 +08:00
8a020ec4d9
wip ( #830 )
2025-11-11 09:34:58 +08:00
29056a767a
Refactor Telemetry Initialization and Environment Utilities ( #811 )
...
* improve code for metrics
* improve code for metrics
* fix
* fix
* Refactor telemetry initialization and environment functions ordering
- Reorder functions in envs.rs by type size (8-bit to 64-bit, signed before unsigned) and add missing variants like get_env_opt_u16.
- Optimize init_telemetry to support three modes: stdout logging (default error level with span tracing), file rolling logs (size-based with retention), and HTTP-based observability with sub-endpoints (trace, metric, log) falling back to unified endpoint.
- Fix stdout logging issue by retaining WorkerGuard in OtelGuard to prevent premature release of async writer threads.
- Enhance observability mode with HTTP protocol, compression, and proper resource management.
- Update OtelGuard to include tracing_guard for stdout and flexi_logger_handles for file logging.
- Improve error handling and configuration extraction in OtelConfig.
* fix
* up
* fix
* fix
* improve code for obs
* fix
* fix
2025-11-07 20:01:54 +08:00
dd47fcf2a8
fix: restore localized samples in tests ( #749 )
...
* fix: restore required localized examples
* style: fix formatting issues
2025-10-29 13:16:31 +08:00
2aca1f77af
Fix/ilm ( #721 )
...
* fix tip remote tier error
* fix transitioned_object
* fix filemeta
* add GCS R2
* add aliyun tencent huaweicloud azure gcs r2 backend tier
* fix signer
* change azure to s3
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-10-27 20:23:50 +08:00
e22b24684f
chore: bump dependencies, add metrics support, remove DNS resolver ( #699 )
...
* upgrade version
* add metrics
* remove dns resolver
* add metrics counter for create bucket
* fix
* fix
* fix
2025-10-24 00:16:17 +08:00
416d3ad5b7
Refactor: Add observability enable flag, improve comments, remove unused config params, and enhance run function error logging. ( #689 )
...
* improve code for dns log
* fix
* Improve comments, remove unused parameters in config.rs (opt), add observability enable flag, and enhance error logging in run function execution.
2025-10-23 13:59:57 +08:00
a65856bdf4
Fix CRC32C Checksum Implementation and Enhance Authentication System ( #678 )
...
* fix: get_condition_values
* fix checksum crc32c
* fix clippy
2025-10-21 21:28:00 +08:00
cd1e244c68
Refactor: Introduce content checksums and improve multipart/object metadata handling ( #671 )
...
* feat: adapt to s3s typed etag support
* refactor: move replication struct to rustfs_filemeta, fix filemeta transition bug
* add head_object checksum, filter object metadata output
* fix multipart checksum
* fix multipart checksum
* add content md5,sha256 check
* fix test
* fix cargo
---------
Co-authored-by: overtrue <anzhengchao@gmail.com >
2025-10-20 23:46:13 +08:00
42d3645d6f
fix(targets): make target removal and reload transactional; prevent reappearing entries ( #662 )
...
* feat: improve code for notify
* upgrade starshard version
* upgrade version
* Fix ETag format to comply with HTTP standards by wrapping with quotes (#592 )
* Initial plan
* Fix ETag format to comply with HTTP standards by wrapping with quotes
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
* bufigx
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
* Improve lock (#596 )
* improve lock
Signed-off-by: Mu junxiang <1948535941@qq.com >
* feat(tests): add wait_for_object_absence helper and improve lifecycle test reliability
Signed-off-by: Mu junxiang <1948535941@qq.com >
* chore: remove dirty docs
Signed-off-by: Mu junxiang <1948535941@qq.com >
---------
Signed-off-by: Mu junxiang <1948535941@qq.com >
* feat(append): implement object append operations with state tracking (#599 )
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <1948535941@qq.com >
* chore: rebase
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
* build(deps): upgrade s3s (#595 )
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
* fix: validate mqtt broker
* improve code for `import`
* fix
* improve
* remove logger from `rustfs-obs` crate
* remove code for config Observability
* fix
* improve code
* fix comment
* up
* up
* upgrade version
* fix
* fmt
* upgrade tokio version to 1.48.0
* upgrade `datafusion` and `reed-solomon-simd` version
* fix
* fmt
* improve code for notify webhook example
* improve code
* fix
* fix
* fmt
---------
Signed-off-by: Mu junxiang <1948535941@qq.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com >
Co-authored-by: Nugine <nugine@foxmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-10-17 15:34:53 +08:00
aae768f446
feat: Simple OPA support ( #644 )
...
* opa-feature
* Update crates/policy/src/policy/opa.rs
* add the content related to 'Copyright'
---------
Co-authored-by: root <root@debian.localdomain >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-10-16 10:35:26 +08:00
8f310cd4a8
test: allow mocking dns resolver ( #656 )
2025-10-15 21:24:03 +08:00
639bf0c233
Revert "feat(append): implement object append operations with state tracking ( #599 )" ( #646 )
...
This reverts commit 4f73760a45
2025-10-12 23:47:51 +08:00
安正超