gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-13 08:36:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,733 Commits 30 Branches 121 Tags
458086dc8075b553a4dd596af3f881836407a785
Commit Graph

93 Commits

Author SHA1 Message Date
安正超 458086dc80 fix(get): remove GET chunk fast path (#2507) 2026-04-12 23:12:23 +08:00
houseme 064e21062d fix(capacity): harden scope registry, scan symlink guard, and test temp dir cleanup (#2432) 
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2026-04-08 20:58:17 +08:00
weisd 0b99e02891 feat(ecstore): add stale multipart upload cleanup (#2416) 2026-04-07 18:57:07 +08:00
houseme 32bf8f5bf3 feat(storage): add direct chunk GET fast path (#2351) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
 2026-04-07 08:33:46 +08:00
weisd 25512e2635 perf(ecstore): batch delete object lock acquisition (#2374) 
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-04-03 21:46:51 +08:00
weisd 5d302febb7 fix(rio): preserve reader capabilities and crypto safety (#2363) 2026-04-03 13:57:42 +08:00
weisd e86c6b726f fix(lock): split distributed read and write quorum (#2355) 2026-03-31 16:16:13 +08:00
houseme 16db18216d fix: populate tagging notification principalId and object metadata (#2342) 2026-03-30 14:32:39 +08:00
安正超 a236b0d01d feat(ecstore): implement decommission and rebalance (#2281) 
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-26 11:44:02 +08:00
weisd 41dcebda44 fix(tier): sweep transitioned copies from delete handlers (#2287) 2026-03-25 16:06:36 +08:00
weisd 28f57b228c feat(s3): advance parity coverage (#2278) 2026-03-24 17:29:33 +08:00
cxymds 5ea6d8a7e6 fix(ecstore): preserve transition object metadata (#2263) 
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-03-24 14:06:29 +08:00
cxymds 236142a682 fix(ecstore): repair lifecycle transition and restore flows (#2240) 
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: weisd <im@weisd.in>
 2026-03-23 12:29:13 +08:00
houseme 6cb094e30a fix(object): Fix concurrent request hang issue in S3 range read workloads (#2251) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-03-23 09:55:08 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
evan slack 4b480727d6 feat(perf): Add configurable bitrot skip for reads (#2110) 
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-03-11 10:59:00 +08:00
houseme 60aa47bf61 feat(storage): integrate S3Operation into OperationHelper for unified metrics and audit (#2103) 2026-03-08 17:57:33 +08:00
evan slack d52a10c5fb chore(obs): Improve tracing instrumentation (#2086) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-03-07 20:03:20 +08:00
安正超 af6c32efac refactor: improve code quality with safer error handling, trait decomposition, and dead code cleanup (#1997) 2026-02-28 01:19:47 +08:00
evan slack dcbc67eb91 perf(lock): Use global lock manager, instead of one per request (#1848) 
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-02-27 22:26:32 +08:00
evan slack ae6eacd7e3 fix(typo): change dang_ling to dangling (#1968) 2026-02-26 13:03:36 +08:00
安正超 7f132a290c refactor(ecstore): split set_disk.rs into submodules (#1945) 2026-02-25 07:41:08 +08:00
LeonWang0735 c7211c9df7 fix:correctly handle replicate delete (#1850) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-02-19 13:23:29 +08:00
LeonWang0735 f4e9ef2edc fix(replication): avoid re-replication loop in Active-Active replication (#1751) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-02-09 14:11:30 +08:00
weisd 4d19b069c3 fix: replication delete (#1714) 2026-02-04 13:39:35 +08:00
LeonWang0735 ec4458f846 Fix/correctly handle terraform s3 backend with versioned bucket (#1686) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-02-02 12:43:07 +08:00
likewu 087f58b7c8 fix(lifecycle): lifecycle fixes (#1625) 
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-01-31 19:14:47 +08:00
houseme 90ed75a3dc refactor(utils/os): Optimize Windows OS utilities and add safety comments (#1671) 
Co-authored-by: weisd <weishidavip@163.com>
 2026-01-30 16:13:42 +08:00
weisd dce117840c refactor: NamespaceLock (nslock), AHM→Heal Crate, and Lock/Clippy Fixes (#1664) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: weisd <2057561+weisd@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-30 13:13:41 +08:00
weisd 6ab7b75fd4 fix bug (#1615) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-27 09:54:13 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555) 
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-26 18:49:21 +08:00
安正超 461ba3aeba fix: handle duplicate part numbers in CompleteMultipartUpload (#1584) 
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-23 22:41:46 +08:00
houseme e5284a85ed fix: Fixed detection warnings in rust v1.93.0 (#1591) 2026-01-23 13:09:41 +08:00
weisd 6631407416 feat: Add RustFS Scanner Module and Multiple Bug Fixes (#1579) 2026-01-22 13:39:38 +08:00
houseme 7c8fd8518f feat(admin): make capacity calculation resilient when backend info is missing (#1560) 
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Signed-off-by: majinghe <42570491+majinghe@users.noreply.github.com>
Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
 2026-01-20 13:03:09 +08:00
yxrxy 29d86036b1 feat: implement bucket quota system (#1461) 
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-01-12 11:42:07 +08:00
houseme 760cb1d734 Fix Windows Path Separator Handling in rustfs_utils (#1464) 
Co-authored-by: reatang <tangtang1251@qq.com>
 2026-01-11 19:53:51 +08:00
weisd 00f3275603 rm online check (#1416) 2026-01-07 13:42:03 +08:00
weisd 359c9d2d26 Enhance Object Version Management and Replication Status Handling (#1413) 2026-01-07 10:44:35 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
weisd 80cfb4feab Add Disk Timeout and Health Check Functionality (#1196) 
Signed-off-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-22 17:15:19 +08:00
houseme 5b0a3a0764 upgrade crate version and improve heal config (#963) 2025-12-03 18:49:11 +08:00
weisd a8b7b28fd0 Fix Admin Heal API and Add Pagination Support for Large Buckets (#933) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-03 18:10:46 +08:00
weisd 4d7bf98c82 add logs (#962) 2025-12-03 13:17:47 +08:00
Copilot 277d80de13 Fix: Implement priority-based heal queue with comprehensive diagnostic logging (#884) 
* Initial plan

* Implement priority-based heal queue with deduplication

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Apply cargo fmt formatting fixes

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add queue monitoring, better error handling, and adaptive processing

- Add priority-based queue statistics tracking
- Implement queue capacity warnings (>80% full)
- Process multiple tasks per cycle when capacity allows
- Add proper error logging for failed heal request submissions
- Add Hash trait to HealPriority for HashMap support
- Improve observability with detailed queue status logs

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive diagnostic logging to heal flow

- Add detailed logging for shard availability checks during reads
- Log successful heal request submissions
- Add detailed logging in heal_object for disk checking
- Log each disk's heal decision with reason and endpoint
- Add logging for heal writing and renaming phases
- Improve variable naming (nil_count -> available_shards)
- Add success confirmation messages for renamed healed data

These logs will help diagnose where the heal flow is failing.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* feat(heal): enhance robustness, safety, and observability of heal process

- **Logging & Observability**:
  - Upgrade critical heal logs from [info](/crates/e2e_test/src/reliant/node_interact_test.rs:196:0-213:1) to `warn` for better visibility.
  - Implement structured logging with `tracing` fields for machine readability.
  - Add `#[tracing::instrument]` to [HealTask](c/crates/ahm/src/heal/task.rs:182:0-205:1) and [SetDisks](/crates/ecstore/src/set_disk.rs:120:0-131:1) methods for automatic context propagation.

- **Robustness**:
  - Add exponential backoff retry (3 attempts) for acquiring write locks in [heal_object](/crates/ahm/src/heal/storage.rs:438:4-460:5) to handle contention.
  - Handle [rename_data](/crates/ecstore/src/set_disk.rs:392:4-516:5) failures gracefully by preserving temporary files instead of forcing deletion, preventing potential data loss.

- **Data Safety**:
  - Fix [object_exists](/crates/ahm/src/heal/storage.rs:395:4-412:5) to propagate IO errors instead of treating them as "object not found".
  - Update [ErasureSetHealer](/crates/ahm/src/heal/erasure_healer.rs:28:0-33:1) to mark objects as failed rather than skipped when existence checks error, ensuring they are tracked for retry.

* fix

* fmt

* improve code for heal_object

* fix

* fix

* fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-20 00:36:25 +08:00
weisd 55d44622ed list object include deleted support (#882) 
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-18 21:51:10 +08:00
weisd 85bc0ce2d5 fix: filemeta version handling and delete operations (#879) 
* fix filemeta version

* fix clippy

* fix delete version

* fix clippy/test
 2025-11-18 09:24:22 +08:00
Copilot 601f3456bc Fix large file upload freeze with adaptive buffer sizing (#869) 
* Initial plan

* Fix large file upload freeze by increasing StreamReader buffer size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive documentation for large file upload freeze fix

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* upgrade s3s version

* Fix compilation error: use BufReader instead of non-existent StreamReader::with_capacity

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Update documentation with correct BufReader implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add tokio feature `io-util`

* Implement adaptive buffer sizing based on file size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Constants are managed uniformly and fmt code

* fix

* Fix: Trigger self-heal on read when shards missing from rejoined nodes (#871)

* Initial plan

* Fix: Trigger self-heal when missing shards detected during read

- Added proactive heal detection in get_object_with_fileinfo
- When reading an object, now checks if any shards are missing even if read succeeds
- Sends low-priority heal request to reconstruct missing shards on rejoined nodes
- This fixes the issue where data written during node outage is not healed when node rejoins

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* Unify CRC implementations to crc-fast (#873)

* Initial plan

* Replace CRC libraries with unified crc-fast implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix: replace low to Normal

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-17 23:15:20 +08:00
weisd acdefb6703 fix read lock (#866) 2025-11-16 11:44:13 +08:00
houseme 7e1a9e2ede 🔒 Upgrade Cryptography Libraries to Latest RC Versions (#837) 
* fix

* chore: upgrade cryptography libraries to RC versions

- Upgrade aes-gcm to 0.11.0-rc.2 with rand_core support
- Upgrade chacha20poly1305 to 0.11.0-rc.2
- Upgrade argon2 to 0.6.0-rc.2 with std features
- Upgrade hmac to 0.13.0-rc.3
- Upgrade pbkdf2 to 0.13.0-rc.2
- Upgrade rsa to 0.10.0-rc.10
- Upgrade sha1 and sha2 to 0.11.0-rc.3
- Upgrade md-5 to 0.11.0-rc.3

These upgrades provide enhanced security features and performance
improvements while maintaining backward compatibility with existing
encryption workflows.

* add

* improve code

* fix
 2025-11-11 21:10:03 +08:00