Commit Graph

72 Commits

Author SHA1 Message Date
cxymds 236142a682 fix(ecstore): repair lifecycle transition and restore flows (#2240)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: weisd <im@weisd.in>
2026-03-23 12:29:13 +08:00
马登山 f11c307aec fix(admin): avoid tier stats panic on missing tier (#2238)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-20 20:57:33 +08:00
weisd 35e1f28f23 fix: support legacy bucket metadata decoding (#2227) 2026-03-19 12:43:53 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
马登山 237c933f38 test(lifecycle): add prefix regression and zero-day checks (#2201) 2026-03-18 13:20:00 +08:00
安正超 c20b3c7f19 fix(ecstore): handle EODM rules without due date (#2198) 2026-03-18 12:54:41 +08:00
马登山 c66c6d97ec fix(lifecycle): respect Filter.Prefix and safe delete marker expiry (#2185)
Signed-off-by: likewu <likewu@126.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: likewu <likewu@126.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-17 18:45:38 +08:00
LeonWang0735 7f1cdaedad feat(replication): add bandwidth-aware reporting for bucket replication metrics (#2141) 2026-03-15 09:03:10 +08:00
houseme f83bf95b04 feat(ecstore): Skip rustls provider install if already present (#2145)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-12 18:02:19 +08:00
GatewayJ 16946c5a54 fix: allow root to bypass bucket policy deny for policy management APIs (#2102)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-09 20:36:29 +08:00
houseme 60aa47bf61 feat(storage): integrate S3Operation into OperationHelper for unified metrics and audit (#2103) 2026-03-08 17:57:33 +08:00
evan slack d52a10c5fb chore(obs): Improve tracing instrumentation (#2086)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-07 20:03:20 +08:00
LeonWang0735 a0503168d4 fix(heal):heal failed replication via must_replicate instead of check replicate_delete (#2072) 2026-03-05 15:47:36 +08:00
安正超 b4a633ebc6 fix(ecstore): set expiration header for put object via lifecycle prediction (#2003) 2026-02-28 16:21:58 +08:00
安正超 3f5ccb20fc fix(s3): normalize GetObjectAttributes ETag XML response (#2002) 2026-02-28 14:53:53 +08:00
安正超 af6c32efac refactor: improve code quality with safer error handling, trait decomposition, and dead code cleanup (#1997) 2026-02-28 01:19:47 +08:00
Brayan Jules 7ce23c6b54 fix(ecstore): allow trailing slash in object names to match S3 behavior (#1996)
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 22:46:42 +08:00
安正超 9d2b8822cf refactor: stabilize heal format recovery integration tests (#1984)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-27 15:26:19 +08:00
LeonWang0735 10140be6d8 fix(replication): handle TLS CA trust and force-delete replication edge cases (#1983) 2026-02-27 08:40:39 +08:00
安正超 09aa6d9f6f refactor(app): remove remaining global access in main init (#1978) 2026-02-26 22:14:51 +08:00
安正超 49579129c1 refactor(app): decouple AppContext adapters from GLOBAL statics (#1970) 2026-02-26 14:54:45 +08:00
LeonWang0735 06d12a8ec8 feat(replication):add replication bandwidth throttle monitor and reader (#1885)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-24 15:21:45 +08:00
安正超 4211652991 refactor(app): add application layer module entry (#1907) 2026-02-22 22:15:37 +08:00
LeonWang0735 f31cd4b716 fix(replication): replicate delete all versions to targets (#1898)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-21 20:12:05 +08:00
evan slack 9786d9b004 metrics(scanner): Add metrics to scanner (#1823)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-15 18:36:40 +08:00
LeonWang0735 f4e9ef2edc fix(replication): avoid re-replication loop in Active-Active replication (#1751)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-09 14:11:30 +08:00
thorntonmc 927f3a57d7 perf(quota): Skip expensive usage checks when no quota configured (#1749)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-08 22:37:53 +08:00
LeonWang0735 60793c17d7 fix: persist replication status and timestamp after replicate_object (#1747)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-07 12:50:37 +08:00
houseme 0b870d6301 build(deps): bump the dependencies group with 19 updates (#1745) 2026-02-07 12:22:14 +08:00
houseme 6bba41f11f Fix/fix issues #1564 (#1708)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-05 13:45:14 +08:00
weisd 4d19b069c3 fix: replication delete (#1714) 2026-02-04 13:39:35 +08:00
loverustfs a4563f7b41 fix: return null versionId when suspended (#1066) (#1709) 2026-02-04 09:22:52 +08:00
LeonWang0735 36f14acbe9 fix: object lock compliance mode allows deletion (#1687)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-03 17:06:24 +08:00
likewu 087f58b7c8 fix(lifecycle): lifecycle fixes (#1625)
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-31 19:14:47 +08:00
evan slack 51e8a4820f fix: map unversioned destination replication error correctly (#1645) 2026-01-29 10:13:36 +08:00
LeonWang0735 8edb1affc0 Fix:s3 compatibility (#1617)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-01-27 16:56:31 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555)
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-26 18:49:21 +08:00
安正超 173dad27d1 fix: preserve exact JSON format in bucket policy GET response (#1598)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-24 23:02:01 +08:00
GatewayJ 9285acba06 feat: object retention (#1589) 2026-01-24 22:12:45 +08:00
weisd 6631407416 feat: Add RustFS Scanner Module and Multiple Bug Fixes (#1579) 2026-01-22 13:39:38 +08:00
GatewayJ 55e4cdec5d feat: add Cors (#1496)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-15 20:03:26 +08:00
houseme f795299d53 Optimization and collation of dependencies introduction processing (#1493) 2026-01-13 15:02:54 +08:00
yxrxy 29d86036b1 feat: implement bucket quota system (#1461)
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-12 11:42:07 +08:00
houseme 760cb1d734 Fix Windows Path Separator Handling in rustfs_utils (#1464)
Co-authored-by: reatang <tangtang1251@qq.com>
2026-01-11 19:53:51 +08:00
weisd ab752458ce Fix Path Traversal and Enhance Object Validation (#1387) 2026-01-05 15:57:15 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
yxrxy 8821fcc1e7 feat: Replace LRU cache with Moka async cache in policy variables (#1166)
Co-authored-by: houseme <housemecn@gmail.com>
2025-12-17 00:19:31 +08:00
tennisleng 978845b555 fix(lifecycle): Fix ObjectInfo fields and mod_time error handling (#1088)
Co-authored-by: loverustfs <hello@rustfs.com>
2025-12-11 07:17:35 +08:00
Copilot 277d80de13 Fix: Implement priority-based heal queue with comprehensive diagnostic logging (#884)
* Initial plan

* Implement priority-based heal queue with deduplication

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Apply cargo fmt formatting fixes

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add queue monitoring, better error handling, and adaptive processing

- Add priority-based queue statistics tracking
- Implement queue capacity warnings (>80% full)
- Process multiple tasks per cycle when capacity allows
- Add proper error logging for failed heal request submissions
- Add Hash trait to HealPriority for HashMap support
- Improve observability with detailed queue status logs

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive diagnostic logging to heal flow

- Add detailed logging for shard availability checks during reads
- Log successful heal request submissions
- Add detailed logging in heal_object for disk checking
- Log each disk's heal decision with reason and endpoint
- Add logging for heal writing and renaming phases
- Improve variable naming (nil_count -> available_shards)
- Add success confirmation messages for renamed healed data

These logs will help diagnose where the heal flow is failing.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* feat(heal): enhance robustness, safety, and observability of heal process

- **Logging & Observability**:
  - Upgrade critical heal logs from [info](/crates/e2e_test/src/reliant/node_interact_test.rs:196:0-213:1) to `warn` for better visibility.
  - Implement structured logging with `tracing` fields for machine readability.
  - Add `#[tracing::instrument]` to [HealTask](c/crates/ahm/src/heal/task.rs:182:0-205:1) and [SetDisks](/crates/ecstore/src/set_disk.rs:120:0-131:1) methods for automatic context propagation.

- **Robustness**:
  - Add exponential backoff retry (3 attempts) for acquiring write locks in [heal_object](/crates/ahm/src/heal/storage.rs:438:4-460:5) to handle contention.
  - Handle [rename_data](/crates/ecstore/src/set_disk.rs:392:4-516:5) failures gracefully by preserving temporary files instead of forcing deletion, preventing potential data loss.

- **Data Safety**:
  - Fix [object_exists](/crates/ahm/src/heal/storage.rs:395:4-412:5) to propagate IO errors instead of treating them as "object not found".
  - Update [ErasureSetHealer](/crates/ahm/src/heal/erasure_healer.rs:28:0-33:1) to mark objects as failed rather than skipped when existence checks error, ensuring they are tracked for retry.

* fix

* fmt

* improve code for heal_object

* fix

* fix

* fix

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2025-11-20 00:36:25 +08:00
weisd 85bc0ce2d5 fix: filemeta version handling and delete operations (#879)
* fix filemeta version

* fix clippy

* fix delete version

* fix clippy/test
2025-11-18 09:24:22 +08:00