fix(replication): madmin reset/diff wire compat and config validation (#5799)

* fix(admin): align replication-reset responses with madmin ResyncTargetsInfo shape The replication-reset and replication-reset-status responses serialized their shell as "Targets" and per-target fields in PascalCase, while madmin-go ResyncTargetsInfo/ResyncTarget expect the "target" shell key and lowercase field tags (arn/resetid/resyncStatus/replicationCount/ completedReplicationSize/failedReplicationCount/failedReplicationSize). Go json decoding is case-insensitive per field, but Targets vs target, Status vs resyncStatus and the size/count key names cannot match, so mc replicate resync decoded empty results. Rename the serde tags to the exact madmin wire shape, keep the ResetBeforeDate/Error RustFS extension keys (unknown keys are ignored by Go decoders), pin the shape with a snapshot unit test, and update the e2e client DTO to decode the madmin shape. * fix(admin): stream bare madmin DiffInfo documents from replication diff POST /v3/replication/diff returned a single enveloped object ({Entries, IsTruncated, ScannedVersions}) while madmin-go BucketReplicationDiff decodes the body with a json.Decoder loop over bare DiffInfo documents. The envelope decoded as exactly one DiffInfo with an empty object, so mc replicate diff printed a phantom empty row instead of the real backlog. Emit one DiffInfo JSON document per line by default, using the exact madmin json tags (object/versionId/rStatus/deletemarker/lastModified; Size stays as a RustFS extension key that Go decoders ignore). The enveloped shape moves to the opt-in ?aggregate=true RustFS extension, which remains the only carrier of scan-coverage metadata; a truncated default-mode scan is surfaced via a warn tracing event instead of in-stream. Pin both shapes with unit tests and tighten the e2e helper to reject any envelope in the stream. * feat(replication): validate replication config structure before persisting PutBucketReplication accepted structurally invalid configurations that MinIO's replication.Config.Validate rejects: empty or oversized rule lists, duplicate or negative rule priorities, over-long rule IDs, filters carrying more than one of Prefix/Tag/And, and delete marker replication enabled on tag-filtered rules. Such configs persisted silently and later produced undefined routing (e.g. ambiguous priority ties) instead of failing the PUT. Add validate_replication_config_structure as a pure function in rustfs-replication (limits documented as constants), surface it through the ecstore api facade, and run it first in the PUT capability gate so defects are named before any metadata write. Missing Priority counts as zero for the uniqueness check, matching Go's zero-value semantics. The self-target rejection deliberately stays at set-remote-target, where the endpoint is known; a config can never reference a self-pointing ARN. Document the rule-level Destination.StorageClass contract (use the remote target's storage_class instead) and renumber the acceptance matrix e2e to unique priorities, which MinIO would also require. * test(replication): pin duplicated wire types with boundary reconciliation tests rustfs-filemeta (xl.meta disk format) and rustfs-replication (MRF/resync persistence format) deliberately each own ReplicationStatusType, VersionPurgeStatusType and ReplicationState; the boundary converts between them via as_str(), whose From<&str> impls fall back to Empty on unknown tokens — a variant added on one side silently degrades to Empty on the other. Add reconciliation tests in replication_filemeta_boundary: exhaustive matches with no wildcard arm on both sides of both enums (a new variant fails compilation until the mapping is reconsidered), string-token round-trip asserts (a token the other side does not recognize fails instead of quietly becoming Empty), and a full-field ReplicationState round-trip. Cross-reference the tests from both type definitions. Struct drift was already compile-guarded by the exhaustive struct literals in the conversion functions. * docs(replication): define split completion criteria and milestone sequence The ecstore replication split plan had no completion measure — the boundary scaffolding risked ossifying because nothing said when the migration counts as done. Record the criteria in the module inventory: done means the Required Contracts table's 'Current dependency to remove' column is empty; the end state moves pool/resyncer/state into crates/replication, with the boundary micro-files dissolving as code crosses the crate line (batch-merging them beforehand is explicitly rejected — the guard scripts anchor on their file names, so merging is churn with zero functional gain; only datatypes.rs can retire early). Sequence the remaining work as M2 (resyncer pure decision logic, after the oversized function splits) → M3 (worker runtime, highest risk, last) → M4 (retire boundaries and guard entries). Refresh the stale first-step text — the event sink / runtime contracts already landed — and update the split-plan status table accordingly. * fix(replication): align structural validator with MinIO semantics after adversarial review Three interop corrections found by adversarial review of the new structural validator, plus review fallout fixes: - Delete-marker replication is now rejected only for a direct Filter.Tag, not for tags inside Filter.And — MinIO's validator only inspects the direct tag, and mc replicate add --tags "k1=v1&k2=v2" (delete-marker replication on by default) puts multiple tags into And.Tags, so the stricter check rejected mc-generated configs MinIO accepts. - Rule ID length is measured in bytes (Go len semantics), not chars — a 255-char multibyte ID must not round-trip into a config MinIO rejects. - An empty <Tag/> element (no key) counts as absent, matching MinIO's Tag.IsEmpty(); console form serializers emit empty tags, which would otherwise trip the exactly-one-of and delete-marker checks. Also: repair the store-uninitialized PUT test whose empty-rules fixture now (correctly) fails structural validation before reaching the store lookup; pin the previously untested startTime madmin key in the reset-status shape test; and signal a truncated default-mode diff scan via the x-rustfs-replication-diff-truncated response header — the bare madmin stream has no envelope, so a truncated scan was otherwise indistinguishable from a complete healthy one (madmin/mc ignore unknown headers). * test(e2e): activate SSE-S3 replication contract and pin resync fail-closed path The SSE-S3 replication contract e2e was ignored under backlog#1291 (silent plaintext replication); the fail-closed gate in replication_target_boundary.rs closed that hole, so the ignore reason expired. Un-ignore the test — it now pins the current fail-closed contract (FAILED status, failure event, readable encrypted source, stable absence of all target versions), verified green. Add test_bucket_replication_sse_s3_resync_stays_fail_closed: drives the existing-object resync path (PUT ?replication-reset) over a FAILED SSE-S3 object and asserts the resync generation reaches a terminal state without ever materializing a target version, with the stays-absent window also spanning fast-scanner heal cycles. The new start_bucket_replication_reset helper doubles as the madmin ResyncTargetsInfo shape assertion (target[0].arn/resetid) for the reset-start response. Refresh the stale nextest count commentary (the module is at 20 fast + 36 nightly = 56 tests by cargo nextest list; the SSE-S3-ignored note no longer holds).
2026-08-16 09:58:21 +00:00 · 2026-08-07 22:30:12 +08:00
parent 7553715f62
commit 3792fed827
16 changed files with 861 additions and 97 deletions
@@ -886,30 +886,37 @@ const REPLICATION_DIFF_MAX_SCAN: usize = 10_000;
 const REPLICATION_DIFF_PAGE_SIZE: i32 = 1_000;

 /// A single object version whose replication is not yet complete, reported by
-/// `POST /v3/replication/diff`. Field names mirror MinIO's `madmin.DiffInfo`
-/// so MinIO-compatible admin clients can parse the response.
+/// `POST /v3/replication/diff`. Field names are the exact json tags of
+/// madmin-go `DiffInfo` (replication-api.go), which `mc replicate diff`
+/// decodes one JSON document at a time from the response body. `Size` is a
+/// RustFS extension key with no madmin counterpart; Go decoders ignore
+/// unknown keys.
 #[derive(Debug, Serialize)]
 struct ReplicationDiffEntry {
-    #[serde(rename = "Object")]
+    #[serde(rename = "object")]
    object: String,
-    #[serde(rename = "VersionID", skip_serializing_if = "Option::is_none")]
+    #[serde(rename = "versionId", skip_serializing_if = "Option::is_none")]
    version_id: Option<String>,
    #[serde(rename = "Size")]
    size: i64,
-    #[serde(rename = "IsDeleteMarker")]
+    #[serde(rename = "deletemarker")]
    is_delete_marker: bool,
-    #[serde(rename = "ReplicationStatus")]
+    #[serde(rename = "rStatus")]
    replication_status: String,
-    #[serde(rename = "LastModified", skip_serializing_if = "Option::is_none")]
+    #[serde(rename = "lastModified", skip_serializing_if = "Option::is_none")]
    last_modified: Option<String>,
 }

-/// Response body for `POST /v3/replication/diff`.
+/// Aggregate response body for `POST /v3/replication/diff?aggregate=true`.
 ///
-/// `entries` lists object versions with a `PENDING` or `FAILED` replication
-/// status. `is_truncated` indicates the on-demand scan hit
-/// [`REPLICATION_DIFF_MAX_SCAN`] before reaching the end of the bucket, so the
-/// diff is partial and should be re-run with a narrower prefix.
+/// This shell is a deliberate RustFS extension: madmin streams bare
+/// `DiffInfo` documents with no envelope (see [`render_replication_diff`]),
+/// so the scan-coverage metadata (`is_truncated`, `scanned_versions`) is only
+/// representable in this opt-in aggregate shape. `entries` lists object
+/// versions with a `PENDING` or `FAILED` replication status. `is_truncated`
+/// indicates the on-demand scan hit [`REPLICATION_DIFF_MAX_SCAN`] before
+/// reaching the end of the bucket, so the diff is partial and should be
+/// re-run with a narrower prefix.
 #[derive(Debug, Serialize)]
 struct ReplicationDiffResponse {
    #[serde(rename = "Entries")]
@@ -920,6 +927,39 @@ struct ReplicationDiffResponse {
    scanned_versions: usize,
 }

+/// Render the diff scan result as a response body.
+///
+/// Default (madmin-compatible) mode emits one `DiffInfo` JSON document per
+/// line with no envelope — madmin's `BucketReplicationDiff` reads the body
+/// with a `json.Decoder` loop, so any envelope object would decode as a
+/// single entry with an empty `object` (a phantom row in `mc replicate
+/// diff`). Scan-truncation info is not representable in that stream and is
+/// reported via the tracing event in the handler instead.
+///
+/// `aggregate=true` (RustFS extension) keeps the enveloped shape including
+/// `IsTruncated`/`ScannedVersions`.
+fn render_replication_diff(
+    entries: Vec<ReplicationDiffEntry>,
+    is_truncated: bool,
+    scanned_versions: usize,
+    aggregate: bool,
+) -> Result<Vec<u8>, serde_json::Error> {
+    if aggregate {
+        return serde_json::to_vec(&ReplicationDiffResponse {
+            entries,
+            is_truncated,
+            scanned_versions,
+        });
+    }
+
+    let mut data = Vec::new();
+    for entry in &entries {
+        serde_json::to_writer(&mut data, entry)?;
+        data.push(b'\n');
+    }
+    Ok(data)
+}
+
 #[derive(Debug, Default, Deserialize)]
 #[serde(deny_unknown_fields)]
 struct ReplicationDiffRequest {
@@ -1036,16 +1076,31 @@ impl Operation for ReplicationDiffHandler {
            truncated = is_truncated,
            "computed replication diff"
        );
+        let aggregate = queries.get("aggregate").map(String::as_str) == Some("true");
+        if is_truncated && !aggregate {
+            // The madmin stream has no envelope to carry truncation info, so
+            // surface the partial-scan condition here instead.
+            tracing::warn!(
+                bucket = %bucket,
+                prefix = %prefix,
+                scanned = scanned_versions,
+                max_scan = REPLICATION_DIFF_MAX_SCAN,
+                "replication diff scan truncated; stream response is partial — re-run with a narrower prefix or use aggregate=true"
+            );
+        }

-        let response = ReplicationDiffResponse {
-            entries,
-            is_truncated,
-            scanned_versions,
-        };
-        let data = serde_json::to_vec(&response)
+        let data = render_replication_diff(entries, is_truncated, scanned_versions, aggregate)
            .map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, format!("serialize failed: {e}")))?;
        let mut headers = HeaderMap::new();
        headers.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
+        // The madmin stream has no envelope to carry truncation info; a
+        // truncated scan would otherwise be indistinguishable from a complete
+        // one (an empty diff on a >MAX_SCAN bucket reads as "healthy"). Signal
+        // it out-of-band so RustFS-aware clients can detect the partial scan;
+        // madmin/mc ignore unknown headers.
+        if is_truncated {
+            headers.insert("x-rustfs-replication-diff-truncated", HeaderValue::from_static("true"));
+        }
        Ok(S3Response::with_headers((StatusCode::OK, Body::from(data)), headers))
    }
 }
@@ -1247,8 +1302,8 @@ impl Operation for ReplicationMrfHandler {
 mod tests {
    use super::{
        REMOTE_TARGET_UNSUPPORTED_FIELDS, REMOTE_TARGET_WRITABLE_FIELDS, RemoteTargetCredentialsRequest, RemoteTargetRequest,
-        SUPPORTED_REMOTE_TARGET_API, TargetUpdateOp, build_mrf_response, extract_query_params, parse_remote_target_update_ops,
-        unique_replication_peers, validate_remote_target_tls_settings,
+        ReplicationDiffEntry, SUPPORTED_REMOTE_TARGET_API, TargetUpdateOp, build_mrf_response, extract_query_params,
+        parse_remote_target_update_ops, render_replication_diff, unique_replication_peers, validate_remote_target_tls_settings,
    };
    use crate::admin::storage_api::bucket::target::{BucketTarget, LatencyStat};
    use crate::admin::storage_api::replication::{BucketStats, DurableMrfBacklog, MrfOpKind, MrfReplicateEntry};
@@ -1950,4 +2005,71 @@ mod tests {
        assert!(target.secure);
        assert_eq!(target.credentials.expect("credentials should be present").access_key, "access");
    }
+
+    fn sample_diff_entries() -> Vec<ReplicationDiffEntry> {
+        vec![
+            ReplicationDiffEntry {
+                object: "a.txt".to_string(),
+                version_id: Some("v1".to_string()),
+                size: 42,
+                is_delete_marker: false,
+                replication_status: "PENDING".to_string(),
+                last_modified: Some("2026-01-01T00:00:00Z".to_string()),
+            },
+            ReplicationDiffEntry {
+                object: "b.txt".to_string(),
+                version_id: None,
+                size: 0,
+                is_delete_marker: true,
+                replication_status: "FAILED".to_string(),
+                last_modified: None,
+            },
+        ]
+    }
+
+    #[test]
+    fn replication_diff_stream_emits_bare_madmin_diff_info_documents() {
+        let data = render_replication_diff(sample_diff_entries(), true, 2, false).expect("stream must render");
+        let text = std::str::from_utf8(&data).expect("stream must be utf-8");
+
+        // madmin decodes the body with a json.Decoder loop over DiffInfo — one
+        // bare document per entry, no envelope keys anywhere in the stream.
+        let lines: Vec<serde_json::Value> = text
+            .lines()
+            .map(|line| serde_json::from_str(line).expect("each line must be a JSON document"))
+            .collect();
+        assert_eq!(lines.len(), 2);
+        assert_eq!(lines[0]["object"], "a.txt");
+        assert_eq!(lines[0]["versionId"], "v1");
+        assert_eq!(lines[0]["rStatus"], "PENDING");
+        assert_eq!(lines[0]["deletemarker"], false);
+        assert_eq!(lines[0]["lastModified"], "2026-01-01T00:00:00Z");
+        assert_eq!(lines[1]["object"], "b.txt");
+        assert_eq!(lines[1]["deletemarker"], true);
+        assert_eq!(lines[1]["rStatus"], "FAILED");
+        for line in &lines {
+            assert!(line.get("Entries").is_none());
+            assert!(line.get("IsTruncated").is_none());
+            assert!(line.get("ScannedVersions").is_none());
+        }
+    }
+
+    #[test]
+    fn replication_diff_stream_renders_empty_body_for_no_entries() {
+        let data = render_replication_diff(Vec::new(), false, 0, false).expect("stream must render");
+        // An empty body makes madmin's json.Decoder loop end immediately with
+        // io.EOF — zero rows, not one phantom empty row.
+        assert!(data.is_empty());
+    }
+
+    #[test]
+    fn replication_diff_aggregate_keeps_enveloped_extension_shape() {
+        let data = render_replication_diff(sample_diff_entries(), true, 7, true).expect("aggregate must render");
+        let payload: serde_json::Value = serde_json::from_slice(&data).expect("aggregate must be one JSON object");
+
+        assert_eq!(payload["Entries"].as_array().map(Vec::len), Some(2));
+        assert_eq!(payload["Entries"][0]["object"], "a.txt");
+        assert_eq!(payload["IsTruncated"], true);
+        assert_eq!(payload["ScannedVersions"], 7);
+    }
 }
@@ -123,17 +123,20 @@ enum MiscExtRoute {
    ListenNotification { bucket: Option<String> },
 }

+// Wire shape mirrors madmin-go `ResyncTargetsInfo`/`ResyncTarget` json tags so
+// `mc replicate resync` can decode the response (Go json decoding is
+// case-insensitive per field, but the `target` shell key must match exactly).
 #[derive(Debug, Clone, serde::Serialize, Default)]
 struct ReplicationResetResponse {
-    #[serde(rename = "Targets")]
+    #[serde(rename = "target")]
    targets: Vec<ReplicationResetTarget>,
 }

 #[derive(Debug, Clone, serde::Serialize, Default)]
 struct ReplicationResetTarget {
-    #[serde(rename = "Arn")]
+    #[serde(rename = "arn")]
    arn: String,
-    #[serde(rename = "ResetID")]
+    #[serde(rename = "resetid")]
    reset_id: String,
 }

@@ -149,17 +152,21 @@ struct ReplicationResetStatusRequest {
    arn: Option<String>,
 }

+// Wire shape mirrors madmin-go `ResyncTargetsInfo`/`ResyncTarget` json tags
+// (see `ReplicationResetResponse`). `ResetBeforeDate` and `Error` are RustFS
+// extension keys with no madmin counterpart; Go decoders ignore unknown keys,
+// so they coexist with madmin/mc clients at zero cost.
 #[derive(Debug, Clone, serde::Serialize, Default)]
 struct ReplicationResetStatusResponse {
-    #[serde(rename = "Targets")]
+    #[serde(rename = "target")]
    targets: Vec<ReplicationResetStatusTarget>,
 }

 #[derive(Debug, Clone, serde::Serialize, Default)]
 struct ReplicationResetStatusTarget {
-    #[serde(rename = "Arn")]
+    #[serde(rename = "arn")]
    arn: String,
-    #[serde(rename = "ResetID")]
+    #[serde(rename = "resetid")]
    reset_id: String,
    #[serde(
        rename = "ResetBeforeDate",
@@ -168,30 +175,30 @@ struct ReplicationResetStatusTarget {
    )]
    reset_before_date: Option<OffsetDateTime>,
    #[serde(
-        rename = "StartTime",
+        rename = "startTime",
        with = "time::serde::rfc3339::option",
        skip_serializing_if = "Option::is_none"
    )]
    start_time: Option<OffsetDateTime>,
    #[serde(
-        rename = "EndTime",
+        rename = "endTime",
        with = "time::serde::rfc3339::option",
        skip_serializing_if = "Option::is_none"
    )]
    end_time: Option<OffsetDateTime>,
-    #[serde(rename = "Status")]
+    #[serde(rename = "resyncStatus")]
    status: String,
-    #[serde(rename = "ReplicatedCount")]
+    #[serde(rename = "replicationCount")]
    replicated_count: i64,
-    #[serde(rename = "ReplicatedSize")]
+    #[serde(rename = "completedReplicationSize")]
    replicated_size: i64,
-    #[serde(rename = "FailedCount")]
+    #[serde(rename = "failedReplicationCount")]
    failed_count: i64,
-    #[serde(rename = "FailedSize")]
+    #[serde(rename = "failedReplicationSize")]
    failed_size: i64,
-    #[serde(rename = "Bucket", skip_serializing_if = "String::is_empty")]
+    #[serde(rename = "bucket", skip_serializing_if = "String::is_empty")]
    bucket: String,
-    #[serde(rename = "Object", skip_serializing_if = "String::is_empty")]
+    #[serde(rename = "object", skip_serializing_if = "String::is_empty")]
    object: String,
    #[serde(rename = "Error", skip_serializing_if = "Option::is_none")]
    error: Option<String>,
@@ -3205,6 +3212,22 @@ mod tests {
        assert!(error.message().unwrap_or_default().contains("run-active"));
    }

+    #[test]
+    fn replication_reset_response_matches_madmin_resync_targets_info_shape() {
+        let payload = serde_json::to_value(ReplicationResetResponse {
+            targets: vec![ReplicationResetTarget {
+                arn: "arn:minio:replication::depl:bucket".to_string(),
+                reset_id: "rid-1".to_string(),
+            }],
+        })
+        .expect("reset response must serialize");
+
+        // madmin-go `ResyncTargetsInfo` json tags: shell `target`, fields `arn`/`resetid`.
+        assert_eq!(payload["target"][0]["arn"], "arn:minio:replication::depl:bucket");
+        assert_eq!(payload["target"][0]["resetid"], "rid-1");
+        assert!(payload.get("Targets").is_none());
+    }
+
    #[test]
    fn build_replication_reset_status_response_serializes_sorted_targets() {
        let mut status = BucketReplicationResyncStatus::new();
@@ -3225,6 +3248,7 @@ mod tests {
            "arn:a".to_string(),
            crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus {
                resync_id: "rid-a".to_string(),
+                start_time: Some(datetime!(2025-01-01 00:00 UTC)),
                last_update: Some(datetime!(2025-01-02 00:00 UTC)),
                resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncCompleted,
                replicated_count: 3,
@@ -3240,15 +3264,21 @@ mod tests {
            .to_bytes();
        let payload: serde_json::Value = serde_json::from_slice(&bytes).expect("response must be json");

-        assert_eq!(payload["Targets"][0]["Arn"], "arn:a");
-        assert_eq!(payload["Targets"][0]["Bucket"], "bucket-a");
-        assert_eq!(payload["Targets"][0]["Status"], "Completed");
-        assert_eq!(payload["Targets"][0]["EndTime"], "2025-01-02T00:00:00Z");
-        assert_eq!(payload["Targets"][1]["Arn"], "arn:z");
-        assert_eq!(payload["Targets"][1]["Bucket"], "bucket-z");
-        assert_eq!(payload["Targets"][1]["Status"], "Failed");
-        assert_eq!(payload["Targets"][1]["EndTime"], "2025-01-03T00:00:00Z");
-        assert_eq!(payload["Targets"][1]["Error"], "boom");
+        assert_eq!(payload["target"][0]["arn"], "arn:a");
+        assert_eq!(payload["target"][0]["resetid"], "rid-a");
+        assert_eq!(payload["target"][0]["bucket"], "bucket-a");
+        assert_eq!(payload["target"][0]["resyncStatus"], "Completed");
+        assert_eq!(payload["target"][0]["startTime"], "2025-01-01T00:00:00Z");
+        assert_eq!(payload["target"][0]["endTime"], "2025-01-02T00:00:00Z");
+        assert_eq!(payload["target"][0]["replicationCount"], 3);
+        assert_eq!(payload["target"][0]["completedReplicationSize"], 9);
+        assert_eq!(payload["target"][1]["arn"], "arn:z");
+        assert_eq!(payload["target"][1]["bucket"], "bucket-z");
+        assert_eq!(payload["target"][1]["resyncStatus"], "Failed");
+        assert_eq!(payload["target"][1]["endTime"], "2025-01-03T00:00:00Z");
+        assert_eq!(payload["target"][1]["failedReplicationCount"], 2);
+        assert_eq!(payload["target"][1]["failedReplicationSize"], 4);
+        assert_eq!(payload["target"][1]["Error"], "boom");
    }

    #[test]
@@ -3286,12 +3316,12 @@ mod tests {
            .to_bytes();
        let payload: serde_json::Value = serde_json::from_slice(&bytes).expect("response must be json");

-        assert_eq!(payload["Targets"].as_array().map(Vec::len), Some(1));
-        assert_eq!(payload["Targets"][0]["Arn"], "arn:z");
-        assert_eq!(payload["Targets"][0]["Bucket"], "bucket-z");
-        assert_eq!(payload["Targets"][0]["Status"], "Failed");
-        assert_eq!(payload["Targets"][0]["EndTime"], "2025-02-03T00:00:00Z");
-        assert_eq!(payload["Targets"][0]["Error"], "boom");
+        assert_eq!(payload["target"].as_array().map(Vec::len), Some(1));
+        assert_eq!(payload["target"][0]["arn"], "arn:z");
+        assert_eq!(payload["target"][0]["bucket"], "bucket-z");
+        assert_eq!(payload["target"][0]["resyncStatus"], "Failed");
+        assert_eq!(payload["target"][0]["endTime"], "2025-02-03T00:00:00Z");
+        assert_eq!(payload["target"][0]["Error"], "boom");
    }

    fn replication_check_target(arn: &str, status: &str, error: Option<&str>) -> ReplicationCheckTargetStatus {
@@ -39,7 +39,8 @@ use super::storage_api::bucket_usecase::bucket::{
    policy_sys::PolicySys,
    replication::{
        ReplicationTargetValidationError, invalid_replication_config_status_field, replication_target_arns,
-        should_remove_replication_target, unsupported_replication_config_field, validate_replication_config_target_arns,
+        should_remove_replication_target, unsupported_replication_config_field, validate_replication_config_structure,
+        validate_replication_config_target_arns,
    },
    target::{BucketTargetType, BucketTargets},
    utils::serialize,
@@ -584,6 +585,9 @@ fn validate_replication_config_targets(targets: &BucketTargets, config: &Replica
 }

 fn validate_replication_config_capabilities(config: &ReplicationConfiguration) -> S3Result<()> {
+    if let Err(err) = validate_replication_config_structure(config) {
+        return Err(S3Error::with_message(S3ErrorCode::InvalidRequest, err.message()));
+    }
    if let Some(field) = invalid_replication_config_status_field(config) {
        return Err(S3Error::with_message(
            S3ErrorCode::InvalidRequest,
@@ -3175,6 +3179,24 @@ mod tests {
        assert!(!err.to_string().contains(destination_key_id));
    }

+    #[test]
+    fn validate_replication_config_capabilities_rejects_structural_defects_before_write() {
+        let mut first = replication_rule_for_target("arn:rustfs:replication:us-east-1:target:bucket");
+        first.priority = Some(1);
+        let mut second = replication_rule_for_target("arn:rustfs:replication:us-east-1:target:bucket");
+        second.priority = Some(1);
+        let config = ReplicationConfiguration {
+            role: String::new(),
+            rules: vec![first, second],
+        };
+
+        let err = validate_replication_config_capabilities(&config)
+            .expect_err("duplicate rule priorities must be rejected before persistence");
+
+        assert_eq!(err.code(), &S3ErrorCode::InvalidRequest);
+        assert!(err.to_string().contains("Priority must be unique"));
+    }
+
    #[test]
    fn validate_replication_config_capabilities_rejects_invalid_status_before_write() {
        let mut rule = replication_rule_for_target("arn:rustfs:replication:us-east-1:target:bucket");
@@ -4448,11 +4470,13 @@ mod tests {

    #[tokio::test]
    async fn execute_put_bucket_replication_returns_internal_error_when_store_uninitialized() {
+        // The config must clear the structural/capability validators so the
+        // request actually reaches the store lookup this test pins.
        let input = PutBucketReplicationInput::builder()
            .bucket("test-bucket".to_string())
            .replication_configuration(ReplicationConfiguration {
                role: "arn:aws:iam::123456789012:role/test".to_string(),
-                rules: vec![],
+                rules: vec![replication_rule_for_target("arn:rustfs:replication:us-east-1:target:bucket")],
            })
            .build()
            .unwrap();
@@ -896,6 +896,14 @@ pub(crate) mod bucket {
        ) -> Option<&'static str> {
            replication_contracts::invalid_replication_config_status_field(config)
        }
+
+        pub(crate) type ReplicationConfigStructureError = replication_contracts::ReplicationConfigStructureError;
+
+        pub(crate) fn validate_replication_config_structure(
+            config: &s3s::dto::ReplicationConfiguration,
+        ) -> Result<(), ReplicationConfigStructureError> {
+            replication_contracts::validate_replication_config_structure(config)
+        }
    }

    pub(crate) mod tagging {