gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,838 Commits 23 Branches 117 Tags
main
Commit Graph

193 Commits

Author SHA1 Message Date
安正超 f833cd9cbe fix(metrics): keep S3 op counts when chains are disabled (#2675) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-04-25 01:24:53 +00:00
安正超 ce291ab610 test(utils): cover scanner env alias mapping (#2574) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-04-17 13:24:16 +00:00
houseme f77ccd5b23 chore: move mcp crate to standalone repository (#2578) 2026-04-17 12:42:26 +00:00
houseme 6ce24f3b63 feat(lifecycle): improve ILM compatibility and scanner runtime config (#2534) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
 2026-04-16 10:28:03 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
houseme 979626c370 refactor(utils): decouple config deps and move sys helpers (#2520) 
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2026-04-13 21:05:03 +08:00
Henry Guo 0f7e7f35e9 test(utils): make retry timer assertions deterministic (#2468) 2026-04-11 12:02:38 +08:00
安正超 939a69f9c2 test(s3): complete snowball auto-extract compatibility (#2324) 2026-03-29 19:19:45 +08:00
安正超 f98664ea4a test(s3): complete snowball auto-extract coverage (#2313) 2026-03-28 22:56:42 +08:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-03-20 18:52:33 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
安正超 83fb530609 refactor(config): normalize scanner env naming (#2129) 2026-03-11 22:41:41 +08:00
安正超 7d7e0b2654 fix(utils): harden panic-prone paths (#2113) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
 2026-03-11 15:16:03 +08:00
安正超 01a75b5f58 Add env variable alias compatibility warnings (#2044) 2026-03-02 15:34:19 +08:00
houseme d17d2083d4 feat(targets): enhance webhook TLS support with custom CA and skip-verify (#1994) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-02-27 21:24:49 +08:00
安正超 9d2b8822cf refactor: stabilize heal format recovery integration tests (#1984) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-02-27 15:26:19 +08:00
唐小鸭 4413878739 fix(compress): downgrade non-compressible log level and expand excluded file types (#1780) 2026-02-16 03:36:22 +08:00
LeonWang0735 f4e9ef2edc fix(replication): avoid re-replication loop in Active-Active replication (#1751) 
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-02-09 14:11:30 +08:00
houseme 0b870d6301 build(deps): bump the dependencies group with 19 updates (#1745) 2026-02-07 12:22:14 +08:00
likewu 087f58b7c8 fix(lifecycle): lifecycle fixes (#1625) 
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-01-31 19:14:47 +08:00
houseme 90ed75a3dc refactor(utils/os): Optimize Windows OS utilities and add safety comments (#1671) 
Co-authored-by: weisd <weishidavip@163.com>
 2026-01-30 16:13:42 +08:00
houseme 2ee81496b0 fix: deduplicate disks in capacity calculation to prevent inflation (#1656) 2026-01-30 00:03:21 +08:00
LeonWang0735 022e3dfc21 fix:s3 tests fix (#1652) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-29 19:35:03 +08:00
heihutu 74759b6e99 fix: missing object.key in S3 event notifications for multipart uploads #1609 (#1624) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-27 13:32:56 +08:00
houseme 2108c4ad28 fix: remove plaintext credential logging (#1619) 2026-01-27 01:47:39 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555) 
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-26 18:49:21 +08:00
安正超 4890fb25c1 fix: listobjects v2 pagination (#1607) 2026-01-25 09:30:08 +08:00
LeonWang0735 3b5f7fb3ff Fix/correctly handle object lock (#1556) 
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-22 00:15:21 +08:00
houseme a9f499282c fix: Increase lock acquire timeout for network storage reliability (#1548) 2026-01-19 01:14:36 +08:00
houseme 109ca7a100 perf(utils): optimize User-Agent generation and platform detection (#1504) 2026-01-14 18:08:02 +08:00
Jan S 68c5c0b834 Use POSIX statvfs, since statfs is not designed to be portable (#1495) 2026-01-14 16:03:32 +08:00
houseme 760cb1d734 Fix Windows Path Separator Handling in rustfs_utils (#1464) 
Co-authored-by: reatang <tangtang1251@qq.com>
 2026-01-11 19:53:51 +08:00
houseme 04f441361e replace winapi to windows crate (#1455) 2026-01-10 02:15:08 +08:00
weisd 359c9d2d26 Enhance Object Version Management and Replication Status Handling (#1413) 2026-01-07 10:44:35 +08:00
Jan S 5f19eef945 fix: OpenBSD does not support TCPKeepalive intervals (#1382) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-06 00:41:39 +08:00
weisd ab752458ce Fix Path Traversal and Enhance Object Validation (#1387) 2026-01-05 15:57:15 +08:00
Jan S d06397cf4a fix: try casting available blocks to a u64 on FreeBSD and OpenBSD (#1360) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-01-04 11:06:14 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
houseme 2924b4e463 Restore globals and add unified TLS/mTLS loading from RUSTFS_TLS_PATH (#1309) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2025-12-30 21:55:43 +08:00
houseme eb33e82b56 fix: Prevent panic in GetMetrics gRPC handler on invalid input (#1291) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2025-12-29 03:10:23 +08:00
0xdx2 136db7e0c9 feat: add function to extract user-defined metadata keys and integrat… (#1281) 
Signed-off-by: 0xdx2 <xuedamon2@gmail.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-27 22:18:16 +08:00
weisd 80cfb4feab Add Disk Timeout and Health Check Functionality (#1196) 
Signed-off-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
 2025-12-22 17:15:19 +08:00
loverustfs f3a1431fa5 fix: resolve TLS handshake failure in inter-node communication (#1201) (#1222) 
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-21 16:11:55 +08:00
Copilot 8dd3e8b534 fix: decode form-urlencoded object names in webhook/mqtt Key field (#1210) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-12-20 01:31:09 +08:00
yihong 834025d9e3 docs: fix some dead link (#1053) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-12-08 11:23:24 +08:00
0xdx2 7c6cbaf837 feat: enhance error handling and add precondition checks for object o… (#1008) 2025-12-06 20:39:03 +08:00
Copilot 277d80de13 Fix: Implement priority-based heal queue with comprehensive diagnostic logging (#884) 
* Initial plan

* Implement priority-based heal queue with deduplication

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Apply cargo fmt formatting fixes

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add queue monitoring, better error handling, and adaptive processing

- Add priority-based queue statistics tracking
- Implement queue capacity warnings (>80% full)
- Process multiple tasks per cycle when capacity allows
- Add proper error logging for failed heal request submissions
- Add Hash trait to HealPriority for HashMap support
- Improve observability with detailed queue status logs

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive diagnostic logging to heal flow

- Add detailed logging for shard availability checks during reads
- Log successful heal request submissions
- Add detailed logging in heal_object for disk checking
- Log each disk's heal decision with reason and endpoint
- Add logging for heal writing and renaming phases
- Improve variable naming (nil_count -> available_shards)
- Add success confirmation messages for renamed healed data

These logs will help diagnose where the heal flow is failing.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* feat(heal): enhance robustness, safety, and observability of heal process

- **Logging & Observability**:
  - Upgrade critical heal logs from [info](/crates/e2e_test/src/reliant/node_interact_test.rs:196:0-213:1) to `warn` for better visibility.
  - Implement structured logging with `tracing` fields for machine readability.
  - Add `#[tracing::instrument]` to [HealTask](c/crates/ahm/src/heal/task.rs:182:0-205:1) and [SetDisks](/crates/ecstore/src/set_disk.rs:120:0-131:1) methods for automatic context propagation.

- **Robustness**:
  - Add exponential backoff retry (3 attempts) for acquiring write locks in [heal_object](/crates/ahm/src/heal/storage.rs:438:4-460:5) to handle contention.
  - Handle [rename_data](/crates/ecstore/src/set_disk.rs:392:4-516:5) failures gracefully by preserving temporary files instead of forcing deletion, preventing potential data loss.

- **Data Safety**:
  - Fix [object_exists](/crates/ahm/src/heal/storage.rs:395:4-412:5) to propagate IO errors instead of treating them as "object not found".
  - Update [ErasureSetHealer](/crates/ahm/src/heal/erasure_healer.rs:28:0-33:1) to mark objects as failed rather than skipped when existence checks error, ensuring they are tracked for retry.

* fix

* fmt

* improve code for heal_object

* fix

* fix

* fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-20 00:36:25 +08:00
weisd 55d44622ed list object include deleted support (#882) 
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-18 21:51:10 +08:00
Copilot 601f3456bc Fix large file upload freeze with adaptive buffer sizing (#869) 
* Initial plan

* Fix large file upload freeze by increasing StreamReader buffer size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive documentation for large file upload freeze fix

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* upgrade s3s version

* Fix compilation error: use BufReader instead of non-existent StreamReader::with_capacity

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Update documentation with correct BufReader implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add tokio feature `io-util`

* Implement adaptive buffer sizing based on file size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Constants are managed uniformly and fmt code

* fix

* Fix: Trigger self-heal on read when shards missing from rejoined nodes (#871)

* Initial plan

* Fix: Trigger self-heal when missing shards detected during read

- Added proactive heal detection in get_object_with_fileinfo
- When reading an object, now checks if any shards are missing even if read succeeds
- Sends low-priority heal request to reconstruct missing shards on rejoined nodes
- This fixes the issue where data written during node outage is not healed when node rejoins

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* Unify CRC implementations to crc-fast (#873)

* Initial plan

* Replace CRC libraries with unified crc-fast implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix: replace low to Normal

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2025-11-17 23:15:20 +08:00
Copilot b7964081ce Fix KMS configuration synchronization across cluster nodes (#855) 
* Initial plan

* Add KMS configuration persistence to cluster storage

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Apply code formatting to KMS configuration changes

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add comment

* fix fmt

* fix

* Fix overlapping dependabot cargo configurations

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* improve code for comment and replace  `Once_Cell` to `std::sync::OnceLock`

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
 2025-11-16 00:05:03 +08:00