gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-27 16:48:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,207 Commits 15 Branches 117 Tags
d578707f95e489b122b86f1c02e2f2fe76fa01ac
Commit Graph

156 Commits

Author SHA1 Message Date
weisd 0927f937a7 fix: Fix BitrotWriter encode writer implementation (#1531) 2026-01-16 17:11:54 +08:00
LeonWang0735 18b22eedd9 Fix:correctly handle versioning obj (#1521) 2026-01-16 08:12:05 +08:00
GatewayJ 55e4cdec5d feat: add Cors (#1496) 
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-15 20:03:26 +08:00
houseme 2d58eea702 fix: exclude matching key from ListObjects results when using marker/startAfter (#1506) 2026-01-14 19:21:51 +08:00
houseme f795299d53 Optimization and collation of dependencies introduction processing (#1493) 2026-01-13 15:02:54 +08:00
yxrxy 29d86036b1 feat: implement bucket quota system (#1461) 
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-01-12 11:42:07 +08:00
houseme 760cb1d734 Fix Windows Path Separator Handling in rustfs_utils (#1464) 
Co-authored-by: reatang <tangtang1251@qq.com>
 2026-01-11 19:53:51 +08:00
houseme 6b2eebee1d fix: Remove secret and signature from the log (#1466) 2026-01-11 17:45:16 +08:00
weisd 00f3275603 rm online check (#1416) 2026-01-07 13:42:03 +08:00
weisd 359c9d2d26 Enhance Object Version Management and Replication Status Handling (#1413) 2026-01-07 10:44:35 +08:00
weisd 3ce99939a3 fix: improve memory ordering for disk health tracker (#1412) 2026-01-06 23:59:08 +08:00
houseme 40ad2a6ea9 Remove unused crates (#1394) 2026-01-05 23:18:08 +08:00
weisd b142563127 fix rpc client (#1393) 2026-01-05 21:52:04 +08:00
weisd 5660208e89 Refactor RPC Authentication System for Improved Maintainability (#1391) 2026-01-05 19:51:51 +08:00
安正超 60103f0f72 fix: s3 api compatibility (#1370) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-05 16:54:16 +08:00
weisd ab752458ce Fix Path Traversal and Enhance Object Validation (#1387) 2026-01-05 15:57:15 +08:00
houseme 75b144b7d4 Fixing URL output format in IPv6 environments #1343 and Incorrect time in UI #1350 (#1363) 2026-01-04 14:56:54 +08:00
yxrxy cf53a9d84a chore: replace native-tls with pure rustls for FTPS/SFTP e2e tests (#1334) 
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-02 11:08:28 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
安正超 61b3100260 fix: s3 list object versions next marker (#1328) 2026-01-01 23:26:32 +08:00
0xdx2 b19e8070a2 fix(tagging): fix e2e test_object_tagging failure (#1327) 2026-01-01 17:38:37 +08:00
houseme 2924b4e463 Restore globals and add unified TLS/mTLS loading from RUSTFS_TLS_PATH (#1309) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2025-12-30 21:55:43 +08:00
houseme eb33e82b56 fix: Prevent panic in GetMetrics gRPC handler on invalid input (#1291) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2025-12-29 03:10:23 +08:00
0xdx2 136db7e0c9 feat: add function to extract user-defined metadata keys and integrat… (#1281) 
Signed-off-by: 0xdx2 <xuedamon2@gmail.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-27 22:18:16 +08:00
houseme 82a6e78845 Inject GlobalReadiness into HTTP server pipeline and gate traffic until FullReady (#1255) 2025-12-25 00:19:03 +08:00
houseme 7e75c9b1f5 remove unlinked file (#1258) 2025-12-24 23:37:43 +08:00
weisd 80cfb4feab Add Disk Timeout and Health Check Functionality (#1196) 
Signed-off-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-22 17:15:19 +08:00
houseme 4abfc9f554 Fix/fix event 1216 (#1191) 
Signed-off-by: loverustfs <hello@rustfs.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-19 12:07:07 +08:00
yxrxy 8821fcc1e7 feat: Replace LRU cache with Moka async cache in policy variables (#1166) 
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-17 00:19:31 +08:00
houseme 9e2fa148ee Fix type errors in ecfs.rs and apply clippy fixes for Rust 1.92.0 (#1121) 2025-12-12 00:49:21 +08:00
dependabot[bot] 0da943a6a4 build(deps): bump s3s from 0.12.0-rc.4 to 0.12.0-rc.5 in the s3s group (#1046) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2025-12-11 15:20:36 +08:00
guojidan fba201df3d fix: harden data usage aggregation and cache handling (#1102) 
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-11 09:55:25 +08:00
yxrxy ccbab3232b fix: ListObjectsV2 correctly handles repeated folder names in prefixes (#1104) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-11 09:38:52 +08:00
tennisleng 978845b555 fix(lifecycle): Fix ObjectInfo fields and mod_time error handling (#1088) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-11 07:17:35 +08:00
Jacob 53c126d678 fix: decode percent-encoded paths in get_file_path() (#1072) 
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-10 22:30:02 +08:00
Jitter 76d25d9a20 Fix/issue #1001 dead node detection (#1054) 
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: Jitterx69 <mohit@example.com>
 2025-12-08 12:29:46 +08:00
0xdx2 7c6cbaf837 feat: enhance error handling and add precondition checks for object o… (#1008) 2025-12-06 20:39:03 +08:00
weisd 030d3c9426 fix filemeta nil versionid (#1002) 2025-12-05 20:30:08 +08:00
weisd b3c80ae362 fix: listdir rpc (#979) 
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-04 16:12:10 +08:00
houseme 5b0a3a0764 upgrade crate version and improve heal config (#963) 2025-12-03 18:49:11 +08:00
weisd a8b7b28fd0 Fix Admin Heal API and Add Pagination Support for Large Buckets (#933) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-03 18:10:46 +08:00
weisd 4d7bf98c82 add logs (#962) 2025-12-03 13:17:47 +08:00
唐小鸭 701960dd81 fix out of range for slice (#931) 2025-11-27 15:57:38 +08:00
Shyim ee04cc77a0 remove debug (#912) 
* remove debug

* Refactor get_global_encryption_service function

* Refactor get_global_encryption_service function

---------

Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-26 11:56:01 +08:00
Copilot 277d80de13 Fix: Implement priority-based heal queue with comprehensive diagnostic logging (#884) 
* Initial plan

* Implement priority-based heal queue with deduplication

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Apply cargo fmt formatting fixes

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add queue monitoring, better error handling, and adaptive processing

- Add priority-based queue statistics tracking
- Implement queue capacity warnings (>80% full)
- Process multiple tasks per cycle when capacity allows
- Add proper error logging for failed heal request submissions
- Add Hash trait to HealPriority for HashMap support
- Improve observability with detailed queue status logs

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive diagnostic logging to heal flow

- Add detailed logging for shard availability checks during reads
- Log successful heal request submissions
- Add detailed logging in heal_object for disk checking
- Log each disk's heal decision with reason and endpoint
- Add logging for heal writing and renaming phases
- Improve variable naming (nil_count -> available_shards)
- Add success confirmation messages for renamed healed data

These logs will help diagnose where the heal flow is failing.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* feat(heal): enhance robustness, safety, and observability of heal process

- **Logging & Observability**:
  - Upgrade critical heal logs from [info](/crates/e2e_test/src/reliant/node_interact_test.rs:196:0-213:1) to `warn` for better visibility.
  - Implement structured logging with `tracing` fields for machine readability.
  - Add `#[tracing::instrument]` to [HealTask](c/crates/ahm/src/heal/task.rs:182:0-205:1) and [SetDisks](/crates/ecstore/src/set_disk.rs:120:0-131:1) methods for automatic context propagation.

- **Robustness**:
  - Add exponential backoff retry (3 attempts) for acquiring write locks in [heal_object](/crates/ahm/src/heal/storage.rs:438:4-460:5) to handle contention.
  - Handle [rename_data](/crates/ecstore/src/set_disk.rs:392:4-516:5) failures gracefully by preserving temporary files instead of forcing deletion, preventing potential data loss.

- **Data Safety**:
  - Fix [object_exists](/crates/ahm/src/heal/storage.rs:395:4-412:5) to propagate IO errors instead of treating them as "object not found".
  - Update [ErasureSetHealer](/crates/ahm/src/heal/erasure_healer.rs:28:0-33:1) to mark objects as failed rather than skipped when existence checks error, ensuring they are tracked for retry.

* fix

* fmt

* improve code for heal_object

* fix

* fix

* fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-20 00:36:25 +08:00
weisd 55d44622ed list object include deleted support (#882) 
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-18 21:51:10 +08:00
weisd 85bc0ce2d5 fix: filemeta version handling and delete operations (#879) 
* fix filemeta version

* fix clippy

* fix delete version

* fix clippy/test
 2025-11-18 09:24:22 +08:00
Copilot 601f3456bc Fix large file upload freeze with adaptive buffer sizing (#869) 
* Initial plan

* Fix large file upload freeze by increasing StreamReader buffer size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive documentation for large file upload freeze fix

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* upgrade s3s version

* Fix compilation error: use BufReader instead of non-existent StreamReader::with_capacity

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Update documentation with correct BufReader implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add tokio feature `io-util`

* Implement adaptive buffer sizing based on file size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Constants are managed uniformly and fmt code

* fix

* Fix: Trigger self-heal on read when shards missing from rejoined nodes (#871)

* Initial plan

* Fix: Trigger self-heal when missing shards detected during read

- Added proactive heal detection in get_object_with_fileinfo
- When reading an object, now checks if any shards are missing even if read succeeds
- Sends low-priority heal request to reconstruct missing shards on rejoined nodes
- This fixes the issue where data written during node outage is not healed when node rejoins

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* Unify CRC implementations to crc-fast (#873)

* Initial plan

* Replace CRC libraries with unified crc-fast implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix: replace low to Normal

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-17 23:15:20 +08:00
weisd 1279baa72b fix replication (#875) 2025-11-17 17:37:41 +08:00
weisd acdefb6703 fix read lock (#866) 2025-11-16 11:44:13 +08:00