* test(admin): pin minio-go Metrics/MetricsV2 wire contract for replication metrics
Red-light evidence for backlog#1675 P1-11: ?replication-metrics[=2]
serializes the internal snake_case BucketStats family straight onto the
wire, while minio-go's replication.Metrics/MetricsV2 expect camelCase
tags (currStats/queueStats/replicaCount/queued/...). Go's decoder is
case-insensitive but does not ignore underscores, so 'mc replicate
status' shows all zeros without any error. The rewritten snapshot tests
assert the minio-go tags (plus a synthesized queueStats node — the
aggregation path leaves queue_stats.nodes empty today) and fail against
the current pass-through serialization.
* fix(admin): serialize replication metrics in minio-go wire shapes
?replication-metrics[=2] and the admin replicationmetrics endpoint
serialized the internal snake_case BucketStats family straight onto the
wire, so 'mc replicate status' decoded all zeros without any error
(backlog#1675 P1-11). The internal structs cannot be renamed: they are
the intra-cluster peer-RPC wire format (rmp_serde to_vec_named in
node_service.rs), pinned by a new regression test.
- New admin/replication_metrics_wire.rs: Serialize-only projections onto
minio-go replication.Metrics (v1 body, currStats) and MetricsV2
(uptime/currStats/queueStats/downtimeInfo) with the exact json tags;
per-target failed becomes the TimedErrStats envelope fed from the
FailStats rolling window; the queue peak is dual-emitted as max
(MinIO server tag) and peak (minio-go tag).
- queueStats synthesizes one node from the bucket queue snapshot — the
aggregation path leaves queue_stats.nodes empty, and mc treats an
empty node list as 'no data' — and carries transfer summaries
(Large/Small/Total) derived from the per-target xfer rates.
- Both endpoints share the DTOs; source-health extension keys
(provider_available/cluster_complete/...) ride along and are ignored
by Go decoders.
- Widen the ecstore replication_stats_boundary re-exports
(BucketReplicationStat/InQueueMetric/XferStats) so the admin facade
chain can name the projected types.
* fix(replication): carry failure rolling windows through cluster aggregation
Review: both metrics endpoints aggregate first, and FailStats::merge
dropped the process-local samples (which also never cross the peer-RPC
wire — serde-skipped), so lastMinute/lastHour serialized as zero right
after a failure while totals was nonzero.
- FailStats gains serializable last_minute/last_hour window snapshots
(serde default: old nodes read zeros, new fields are ignored by old
decoders), recomputed on every add_size and re-stamped at the
per-node collection point (get_latest_replication_stats), and summed
by merge.
- The wire DTO takes the component-wise max of the live samples and the
snapshot, so both the single-node and the aggregated path report the
window.
- Regression test drives a stat through rmp round trip + merge before
serialization, as requested.
Also restore the #[allow(dead_code)] attribute to route_policy — the
new module declaration had been inserted between the attribute and its
item, which broke the -D warnings CI lanes.
* fix(replication): bin transfer summaries at 128 MiB and keep window refresh off the hot path
Second review round:
- update_xfer_rate split at 1 MiB while the minio-go transferSummary
labels (and RustFS's own worker-pool split) mean >= 128 MiB for
Large, so a 2 MiB replication reported under Large with Small stuck
at zero. The producer now bins on MIN_LARGE_OBJ_SIZE; a MetricsV2
assertion covers 2 MiB / 127 MiB / exactly 128 MiB.
- add_size no longer recomputes the rolling windows: two full
one-hour-deque scans per failure under the bucket-stats write lock
made failure bursts quadratic (30k events ~2.1s). The windows are
stamped only at the collection point (get_latest_replication_stats,
which serves both the local leg and the peer RPC); the aggregation
regression now drives that path explicitly before the RPC round trip
and merge.
* fix(replication): average transfer summaries
---------
Co-authored-by: overtrue <anzhengchao@gmail.com>
唐小鸭