380ec74ece
fix(replication): persist force-delete handoff state ( #5641 )
...
* fix(replication): persist force-delete handoff state
* fix(arch): route force-delete config access through boundary
* style: format force-delete imports
---------
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-08-03 01:44:28 +00:00
035ce5d784
feat(obs): add bounded metrics dimensions ( #5645 )
...
* feat(obs): add drive topology detail metrics
Expose additive drive info, topology, state, and per-drive API metrics while preserving the existing drive metric label sets.
Backlog: rustfs/backlog#1655
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): preserve suspect drive runtime state
Keep suspect as a bounded drive runtime state and avoid all-zero runtime_state samples for that storage health state.
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): skip unknown drive inode samples
Avoid exporting zero inode gauges for missing or stale drive snapshots and ignore zero-count API latency buckets.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add scanner source work detail metrics
Expose additive scanner source and cycle work metrics with bounded server/source/state labels while leaving the existing aggregate scanner metrics unchanged.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add ilm action detail metrics
Expose additive ILM action/state task metrics with a server label while preserving the existing aggregate ILM series.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add delivery target server metrics
Expose additive audit and notification delivery target metrics with server labels and extend removed-target tombstones for the server-aware series.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add replication target flow metrics
Expose additive bucket replication target sent and failed-flow metrics while preserving existing bucket aggregates and target backlog series.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add request server metrics
Expose additive API request metrics with server labels while preserving the existing request and traffic metric label sets.
Co-Authored-By: heihutu <heihutu@gmail.com >
* style(obs): apply rustfmt to metrics changes
Apply rustfmt output to the metrics dimension changes without altering behavior.
Co-Authored-By: heihutu <heihutu@gmail.com >
* style(obs): reuse audit target label constant
Use the exported audit target_id label constant for legacy audit target metrics.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): populate drive disk metrics
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add scanner bucket drive result metrics
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(obs): add replication proxy server metrics
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): address metric liveness review
Use checked division for drive API latency aggregation and keep recovered drive, scanner current-cycle, replication flow, audit target, and notification target series from retaining stale values.
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): address metric dimension review
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): address additional metric review
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): count drive calls at start
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(obs): address metrics dimension review
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(metrics): address dimension review gaps
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(metrics): address scanner review follow-ups
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(metrics): address runtime review follow-ups
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(metrics): reduce disk metric contention
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(metrics): address runtime review follow-ups
Co-Authored-By: heihutu <heihutu@gmail.com >
* fix(metrics): retire stale dimension series
Co-Authored-By: heihutu <heihutu@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-03 09:03:34 +08:00
114b2420a2
feat(admin): expose versioned replication capabilities ( #5631 )
...
* feat(admin): expose replication capabilities
* fix(admin): route replication capabilities through facades
2026-08-02 22:54:38 +08:00
2cc7443067
feat(replication): bound DeleteObjects queue admission ( #5637 )
...
feat(replication): batch DeleteObjects queue admission
2026-08-02 21:06:54 +08:00
921ddef2c7
fix(lifecycle): bind delete replication admission ( #5621 )
2026-08-02 19:27:39 +08:00
c1955a8498
fix(replication): harden live delete admission ( #5599 )
2026-08-02 12:52:11 +08:00
fbec33bd29
Expose target-scoped durable MRF backlog metrics ( #5584 )
...
* feat(replication): expose target durable mrf backlog
Add target ARN attribution to durable MRF entries and surface target-scoped durable backlog metrics without changing existing bucket-only metric labels.
Keep legacy MRF files bucket-only by defaulting missing targetARNs to an empty list, and expose target snapshots through an additive API so existing DurableMrfBacklogSummary callers remain source-compatible.
Co-Authored-By: heihutu <heihutu@gmail.com >
* feat(replication): expose runtime target backlog (#5586 )
Track runtime replication backlog by target ARN for regular, large, delete, and MRF admission paths while preserving the existing bucket-level backlog semantics.
Add target-scoped current backlog metrics and merge them with durable target backlog snapshots for observability.
Co-authored-by: heihutu <heihutu@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
2026-08-01 17:31:50 +00:00
c8016cbcdb
fix(replication): enforce bucket replication switches ( #5449 )
...
* fix(replication): enforce bucket replication switches
* fix(replication): satisfy delete admission clippy lint
* fix(replication): restore MinIO tag filter behavior
---------
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
Co-authored-by: cxymds <cxymds@gmail.com >
2026-08-01 15:03:57 +00:00
7320d7fab2
fix(replication): make resync starts atomic ( #5215 )
2026-07-25 08:58:06 +08:00
45b675c641
fix(replication): report authoritative backlog metrics ( #5209 )
2026-07-25 01:32:50 +08:00
13f8768e7f
feat(ecstore): make replication timing intervals env-overridable for tests ( #4680 )
...
Introduce RUSTFS_REPL_HEALTH_CHECK_INTERVAL_MS, RUSTFS_REPL_MRF_FLUSH_INTERVAL_MS
and RUSTFS_REPL_RESYNC_POLL_MAX_MS so tests and operators can shorten the
replication background loops. Defaults are unchanged; invalid values fall
back with a warn and values below 10ms are clamped to avoid busy-spin.
Add replication_fast_env() e2e helper (backlog#1147 repl-4).
2026-07-10 21:57:43 +08:00
3d4fb532e5
refactor(replication): own filemeta wire contracts ( #4254 )
2026-07-04 06:39:54 +08:00
8a0617865b
refactor(replication): route app storage through ecstore ( #4251 )
2026-07-04 02:00:28 +08:00
4883d3eb50
refactor(replication): route runtime facades through ecstore ( #4248 )
2026-07-03 23:49:24 +08:00
7cc470cb08
refactor(replication): isolate ecstore boundary imports ( #4247 )
2026-07-03 23:18:27 +08:00
769549dd2c
refactor(replication): isolate status contract boundaries ( #4236 )
2026-07-03 19:40:49 +08:00
7001e53546
refactor(replication): isolate stats and app contract boundaries ( #4235 )
...
* refactor(replication): isolate stats boundary adapters
* refactor(replication): route app contracts through storage boundary
2026-07-03 18:48:26 +08:00
aecac5c0ae
refactor(replication): isolate object decision contracts ( #4219 )
2026-07-03 11:36:14 +08:00
c260a2f20f
refactor(replication): isolate queue boundary adapters ( #4216 )
2026-07-03 09:43:31 +08:00
48b70f6e4f
refactor(replication): isolate resync boundary adapters ( #4215 )
2026-07-03 08:59:42 +08:00
f0adabbe90
refactor(replication): move object config bridge ( #4188 )
...
* refactor(replication): move object config bridge
* refactor(replication): centralize log constants
2026-07-02 16:32:02 +08:00
473132f36b
refactor(replication): move stats contracts into crate ( #4168 )
2026-07-02 12:14:36 +08:00
8ce0f824cf
refactor(replication): move queue contracts into crate ( #4167 )
2026-07-02 11:40:49 +08:00
2cf856a51d
Move replication delete worker contract into crate ( #4166 )
...
refactor(replication): move delete worker contract into crate
2026-07-02 11:03:24 +08:00
df6ce41acd
Move replication operation contracts into crate ( #4165 )
...
refactor(replication): move operation contracts into crate
2026-07-02 10:48:02 +08:00
b57820a486
refactor(replication): move config contracts into crate ( #4162 )
2026-07-02 10:17:48 +08:00
e80391f1aa
refactor(replication): extract resync contracts crate ( #4154 )
2026-07-02 01:30:01 +08:00
22cbd41ee9
refactor(ecstore): add replication owner bridges ( #4141 )
...
* refactor(ecstore): add replication owner bridges
* fix(ecstore): tighten replication owner guards
2026-07-01 21:51:26 +08:00
c0dfc260ca
refactor(ecstore): add replication object bridge ( #4140 )
2026-07-01 19:24:20 +08:00
e5ab6ee8e4
refactor(ecstore): make replication facade explicit ( #4139 )
2026-07-01 18:54:05 +08:00
fc806679fb
refactor(ecstore): add replication work bridges ( #4137 )
2026-07-01 18:37:01 +08:00
722c0805ce
refactor(ecstore): route replication errors ( #4127 )
2026-07-01 08:13:57 +08:00
e917f1f4be
refactor(ecstore): route replication filemeta contracts ( #4122 )
2026-07-01 01:01:01 +08:00
da7f421f69
refactor(ecstore): route replication storage contracts ( #4118 )
2026-06-30 20:12:06 +08:00
20919f0b97
refactor(ecstore): route replication bandwidth wrapping ( #4113 )
2026-06-30 15:42:04 +08:00
fb7bcfb541
refactor(replication): add tagging boundary ( #4081 )
2026-06-30 03:39:17 +08:00
d3a84395dc
refactor(replication): add msgp boundary ( #4076 )
2026-06-30 02:09:14 +08:00
c17df6e3d3
refactor(replication): add lock boundary ( #4075 )
2026-06-30 00:18:08 +08:00
50ca1c2cbe
refactor(replication): add config store boundary ( #4073 )
2026-06-30 00:03:45 +08:00
f8c70e017d
refactor(replication): add versioning boundary ( #4071 )
2026-06-29 23:52:36 +08:00
04e8c1bb6b
refactor(replication): add metadata boundary ( #4070 )
2026-06-29 23:24:39 +08:00
f1ec0bd765
refactor(replication): add target boundary ( #4069 )
2026-06-29 22:59:10 +08:00
c94f2981d7
refactor(replication): add event sink boundary ( #4068 )
2026-06-29 22:46:06 +08:00
13474ec1de
refactor(replication): add runtime boundary ( #4067 )
2026-06-29 22:34:10 +08:00
a0e7957e92
refactor: prune admin app compat re-exports ( #3697 )
2026-06-21 20:28:43 +08:00
7f1cdaedad
feat(replication): add bandwidth-aware reporting for bucket replication metrics ( #2141 )
2026-03-15 09:03:10 +08:00
cd1e244c68
Refactor: Introduce content checksums and improve multipart/object metadata handling ( #671 )
...
* feat: adapt to s3s typed etag support
* refactor: move replication struct to rustfs_filemeta, fix filemeta transition bug
* add head_object checksum, filter object metadata output
* fix multipart checksum
* fix multipart checksum
* add content md5,sha256 check
* fix test
* fix cargo
---------
Co-authored-by: overtrue <anzhengchao@gmail.com >
2025-10-20 23:46:13 +08:00
90f21a9102
refactor: Reimplement bucket replication system with enhanced architecture ( #590 )
...
* feat:refactor replication
* use aws sdk for replication client
* refactor/replication
* merge main
* fix lifecycle test
2025-09-26 14:27:53 +08:00
5826396cd0
refactor: Restructure project layout and clean up dependencies ( #30 )
...
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
2025-07-02 19:33:12 +08:00
cxymds