feat(replication): replicate managed-SSE objects via target re-encryption (#5885)

Open the managed-SSE replication gate (backlog#1783, PR-B of 3, after
#5872): the replication reader already decrypts through the injected
object-encryption resolver, so the source sends plaintext plus an
encryption intent header (AES256 / aws:kms, never the source key id) and
the target re-encrypts on its normal PUT path with its own KMS. No DEK
crosses sites.

- replication_put_object_options: fail closed only on Unsupported;
  insert the SSE intent after the strip loop.
- TargetClient::create_multipart_upload sends the full opts.header()
  set, fixing multipart replicas losing content-type/user metadata
  (plaintext included).
- Preserve source ETag and mtime on replicas (authorized replication
  only): receiver wires x-rustfs-source-etag into preserve_etag for PUT
  and CompleteMultipartUpload, resolve_complete_etag consumes it, and
  complete options carry source_etag/source_mtime (absent mtime
  degrades to epoch, not now_utc). Without this every replication HEAD
  comparison re-drives re-encrypted objects forever.
- e2e: managed SSE contracts flip to success on an independent-KMS
  dual-process pair (byte-identical plain GET proves target-owned
  envelopes; ETag/mtime preserved; version stable across scanner
  cycles; resync converges; multipart keeps structure and metadata);
  new target-without-KMS fail-closed contract; SSE-C stays FAILED.

Co-authored-by: houseme <housemecn@gmail.com>
This commit is contained in:
唐小鸭
2026-08-09 18:21:35 +08:00
committed by GitHub
parent a71726ef49
commit 73e4ef4dd4
8 changed files with 408 additions and 125 deletions
+40 -7
View File
@@ -1919,13 +1919,7 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
}
// etag
let etag = {
if let Some(etag) = opts.user_defined.get("etag") {
etag.clone()
} else {
get_complete_multipart_md5(&uploaded_parts)
}
};
let etag = resolve_complete_etag(opts, &uploaded_parts);
fi.metadata.insert("etag".to_owned(), etag);
@@ -2167,6 +2161,21 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
}
}
/// Final ETag for a completed multipart object. An authorized replication
/// request preserves the source ETag so the replication HEAD comparison
/// converges even when the source ETag is not derivable from the uploaded
/// parts (foreign-origin objects, ciphertext-derived ETags); the internal
/// metadata override comes next; otherwise the ETag is computed from parts.
fn resolve_complete_etag(opts: &ObjectOptions, uploaded_parts: &[CompletePart]) -> String {
if let Some(etag) = opts.preserve_etag.as_ref().filter(|etag| !etag.is_empty()) {
return etag.clone();
}
if let Some(etag) = opts.user_defined.get("etag") {
return etag.clone();
}
get_complete_multipart_md5(uploaded_parts)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -5190,4 +5199,28 @@ mod tests {
assert_eq!(body_after, new, "reclaiming the leftover upload must not disturb the committed object");
}
}
#[test]
fn resolve_complete_etag_prefers_preserved_source_etag() {
// A replication-preserved ETag that no part combination can derive
// (foreign-origin object) must win over the computed md5-of-parts.
let foreign_etag = "11111111111111111111111111111111-7".to_string();
let opts = ObjectOptions {
preserve_etag: Some(foreign_etag.clone()),
..Default::default()
};
assert_eq!(resolve_complete_etag(&opts, &[]), foreign_etag);
// Empty preserve value degrades to the next source.
let opts_empty = ObjectOptions {
preserve_etag: Some(String::new()),
user_defined: std::collections::HashMap::from([("etag".to_string(), "override-etag".to_string())]),
..Default::default()
};
assert_eq!(resolve_complete_etag(&opts_empty, &[]), "override-etag");
// Without either source the ETag is computed from the parts.
let computed = resolve_complete_etag(&ObjectOptions::default(), &[]);
assert_eq!(computed, get_complete_multipart_md5(&[]));
}
}