Commit Graph

32 Commits

Author SHA1 Message Date
Zhengchao An ea49d5686b fix(replication): mark failed targets offline (#4405) 2026-07-08 12:08:22 +08:00
Zhengchao An 0fad356450 fix(replication): send versionId on version-purge deletes to generic S3 targets (#4401)
Replication deletes dropped the S3 `versionId` query parameter for every
replication request, conveying the version only via the internal
x-*-source-version-id header. A generic (non-MinIO/RustFS) S3 target ignores
that header, so a version purge degenerated into a delete-marker creation while
the source still stamped VersionPurgeStatus=Complete — a silent divergence.

Only omit the query `versionId` when propagating a delete-marker CREATION
(the target must mint its own marker); version purges, delete-marker purges and
force deletes now address the exact version. Extracted into
resolve_delete_api_version_id with unit coverage.

Fixes rustfs/backlog#857
2026-07-08 09:29:58 +08:00
Zhengchao An 6a81445a96 refactor(replication): isolate ecstore owner contracts (#4239) 2026-07-03 21:42:45 +08:00
Zhengchao An dd6b5525e4 fix(ecstore): remove reachable panics in tiering, replication, and heal paths (#4205)
* fix(ecstore): remove reachable panics in tiering, replication, and heal paths

- Parse x-amz-expiration leniently in tier PUT responses; any lifecycle
  rule on the remote tier bucket returns an RFC1123 date that the previous
  ISO8601 unwrap turned into a panic of the ILM transition worker
- Skip invalid user-metadata header values (with a warning) when building
  tier and replication PUT headers instead of panicking on non-ASCII input
- Heal: tolerate absent data_dir for delete markers and remote objects
- transition_object: don't unwrap version_id on unversioned buckets when
  recording partial writes for offline disks
- Admin server info: use port_or_known_default() so default-port (80/443)
  endpoints don't panic is_server_resolvable
- Tier ListObjectsV2 client: decode response body with from_utf8_lossy
- walk_internal: log merge_entry_channels errors instead of dropping them

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ecstore): fail heal explicitly when data_dir is missing

Address review feedback: unwrap_or_default() silently substituted a nil
UUID when latest metadata lacked data_dir. Delete markers and remote
objects legitimately have no data_dir and skip the data-heal block, but
for a regular object a missing data_dir means corrupt metadata — return
FileCorrupt with a descriptive log instead of building part paths under
a nil UUID directory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:14:11 +08:00
cxymds c7a0178994 fix: honor replication status metadata (#4178)
Co-authored-by: overtrue <anzhengchao@gmail.com>
2026-07-02 22:33:41 +08:00
Zhengchao An 18d89d1e31 refactor(replication): route metadata contracts through crate (#4160) 2026-07-02 08:48:42 +08:00
Zhengchao An 22cbd41ee9 refactor(ecstore): add replication owner bridges (#4141)
* refactor(ecstore): add replication owner bridges

* fix(ecstore): tighten replication owner guards
2026-07-01 21:51:26 +08:00
cxymds 6e5c58ca3f fix(replication): harden bucket replication correctness (#4116) 2026-07-01 09:21:30 +08:00
Zhengchao An 6e72dc3076 fix(ecstore): replace unsafe k.unwrap() in bucket_target_sys (#729) (#3982)
fix(ecstore): replace k.unwrap() with safe pattern in bucket_target_sys

Replace unsafe k.unwrap().as_str() with if let Some(key_str) pattern
in 5 locations where HeaderMap iterator yields (Option<HeaderName>, Value).

This prevents potential panics if header names are invalid.

Refs https://github.com/rustfs/backlog/issues/729
2026-06-28 10:42:56 +08:00
Zhengchao An 0a5b1b1b3a refactor: consolidate ecstore owner module layout (#3934)
* refactor: shrink ecstore root owner facades

* refactor: remove ecstore core store root shims

* refactor: move ecstore erasure owner modules

* refactor: remove ecstore root rpc facade

* refactor: move ecstore services domain modules
2026-06-27 09:03:20 +08:00
houseme 623fc801f1 feat(replication): support custom TLS for bucket targets (#3825)
* feat(replication): support insecure https bucket targets

* feat(replication): support custom ca bucket targets

* test(replication): satisfy e2e clippy for TLS helpers

* fix(replication): avoid native root panic for custom trust stores

* test(replication): decouple private IP target test from TLS roots

* test(replication): use target TLS client in private IP unit test
2026-06-24 20:23:20 +08:00
Zhengchao An 1c04b86494 refactor: centralize ecstore bucket runtime sources (#3806) 2026-06-24 08:00:04 +08:00
houseme 2f85ef4455 fix(security): extend outbound egress guard (#3744) 2026-06-22 19:20:31 +08:00
houseme 64aef06c60 fix(replication): harden site-repl versioning sync (#3739) 2026-06-22 19:18:42 +08:00
weisd 334184b005 fix(replication): prevent target state loss across buckets (#2704) 2026-04-27 09:27:33 +00:00
houseme 6ce24f3b63 feat(lifecycle): improve ILM compatibility and scanner runtime config (#2534)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-16 10:28:03 +00:00
Tunglies 579b124726 lint: clippy rules or_fun_call (#2561)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:48:39 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
weisd 28f57b228c feat(s3): advance parity coverage (#2278) 2026-03-24 17:29:33 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
houseme f83bf95b04 feat(ecstore): Skip rustls provider install if already present (#2145)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-12 18:02:19 +08:00
LeonWang0735 a0503168d4 fix(heal):heal failed replication via must_replicate instead of check replicate_delete (#2072) 2026-03-05 15:47:36 +08:00
LeonWang0735 10140be6d8 fix(replication): handle TLS CA trust and force-delete replication edge cases (#1983) 2026-02-27 08:40:39 +08:00
LeonWang0735 06d12a8ec8 feat(replication):add replication bandwidth throttle monitor and reader (#1885)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-24 15:21:45 +08:00
LeonWang0735 f4e9ef2edc fix(replication): avoid re-replication loop in Active-Active replication (#1751)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-09 14:11:30 +08:00
evan slack 51e8a4820f fix: map unversioned destination replication error correctly (#1645) 2026-01-29 10:13:36 +08:00
weisd 6631407416 feat: Add RustFS Scanner Module and Multiple Bug Fixes (#1579) 2026-01-22 13:39:38 +08:00
houseme f795299d53 Optimization and collation of dependencies introduction processing (#1493) 2026-01-13 15:02:54 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
weisd 85bc0ce2d5 fix: filemeta version handling and delete operations (#879)
* fix filemeta version

* fix clippy

* fix delete version

* fix clippy/test
2025-11-18 09:24:22 +08:00
weisd 1279baa72b fix replication (#875) 2025-11-17 17:37:41 +08:00
weisd 90f21a9102 refactor: Reimplement bucket replication system with enhanced architecture (#590)
* feat:refactor replication

* use aws sdk for replication client

* refactor/replication

* merge main

* fix lifecycle test
2025-09-26 14:27:53 +08:00