fix(sse): read MinIO single-part encrypted objects

Two independent defects kept them unreadable, both reached only once the fixture harness could load an inlined object at all.

The stream was keyed as if the object were multipart. is_multipart_encrypted_object falls back to an ETag length test when there is a single part, and MinIO stores an *encrypted* ETag for SSE objects — 96 characters for a single-part upload, not the 32 of a plain MD5 — so every such object read as multipart and had a per-part key derived for a stream sealed with the object key itself. MinIO records the answer outright in X-Minio-Internal-Encrypted-Multipart, so that marker is now consulted first, and on a MinIO-written object its absence is treated as just as informative as its presence.

The size was then reported as the ciphertext's. MinIO writes an explicit plaintext size only for multipart uploads, deriving it from the DARE stream otherwise, so both the read plan and ObjectInfo::decrypted_size fell back to the physical size — the reader waited for the encoding overhead as if it were payload, and a client would have seen a Content-Length overstated by exactly that much. dare_v2_decrypted_size inverts the package layout the way MinIO's sio.DecryptedSize does. It lives in rustfs-utils so the read plan and the size accessor cannot drift apart, and it returns None for a length no DARE stream can have rather than assigning a malformed object a plausible one.

The interop suite is 9/9 with the two single-part cases added, and reverting the classification to the ETag heuristic turns both red. rustfs-ecstore's object_api tests and rustfs-utils are unaffected.

Refs rustfs/backlog#1638.

crates/ecstore/src/object_api/readers.rs

@@ -630,7 +630,7 @@ impl ReadPlan {
             let material = resolved;
             #[cfg(feature = "rio-v2")]
             let uses_legacy_encryption = matches!(material.mode, ReadEncryptionMode::Direct { .. });
-            let is_multipart = is_multipart_encrypted_object(&oi.parts, oi.etag.as_deref());
+            let is_multipart = is_multipart_encrypted_object(&oi.parts, oi.etag.as_deref(), &oi.user_defined);
             let recorded_plaintext_size = oi.encryption_original_size()?;
             let plaintext_size = encrypted_plaintext_size(oi, is_multipart, is_compressed, recorded_plaintext_size)?;
             let full_plaintext_size =
@@ -1285,14 +1285,59 @@ fn encrypted_plaintext_size(
             .unwrap_or(oi.size));
     }
 
-    Ok(recorded_plaintext_size.unwrap_or(oi.size))
+    if let Some(recorded) = recorded_plaintext_size {
+        return Ok(recorded);
+    }
+
+    // A MinIO single-part object records no plaintext size: MinIO writes
+    // `X-Minio-Internal-actual-size` only for multipart uploads and otherwise
+    // derives the size from the DARE stream itself. Falling back to `oi.size`
+    // hands back the *physical* size, so the reader waits for the encoding
+    // overhead as if it were payload and the body ends short by exactly that
+    // much (rustfs/backlog#1638).
+    if rustfs_utils::http::has_minio_internal_sse_metadata(&oi.user_defined)
+        && let Some(plaintext) = rustfs_utils::http::dare_v2_decrypted_size(oi.size)
+    {
+        return Ok(plaintext);
+    }
+
+    Ok(oi.size)
 }
 
-fn is_multipart_encrypted_object(parts: &[ObjectPartInfo], etag: Option<&str>) -> bool {
+/// MinIO's explicit multipart marker, and the internal SSE prefix that
+/// identifies an object as MinIO-written in the first place.
+const MINIO_INTERNAL_ENCRYPTED_MULTIPART_KEY: &str = "X-Minio-Internal-Encrypted-Multipart";
+const MINIO_INTERNAL_SSE_PREFIX: &str = "X-Minio-Internal-Server-Side-Encryption-";
+
+/// Whether an encrypted object's stream is keyed per part.
+///
+/// `user_defined` is consulted before the ETag because MinIO records the answer
+/// outright, in `X-Minio-Internal-Encrypted-Multipart`. The ETag heuristic
+/// cannot stand in for it: MinIO stores an *encrypted* ETag for SSE objects —
+/// 96 characters for a single-part upload, not the 32 of a plain MD5 — so the
+/// length test reads such an object as multipart and derives a per-part key for
+/// a stream that was sealed with the object key itself, which then fails
+/// authentication (rustfs/backlog#1638).
+fn is_multipart_encrypted_object(parts: &[ObjectPartInfo], etag: Option<&str>, user_defined: &HashMap<String, String>) -> bool {
     if parts.len() > 1 {
         return true;
     }
 
+    if user_defined
+        .keys()
+        .any(|key| key.eq_ignore_ascii_case(MINIO_INTERNAL_ENCRYPTED_MULTIPART_KEY))
+    {
+        return true;
+    }
+    // On a MinIO-written object the marker's absence is as informative as its
+    // presence, so the ETag is not consulted at all.
+    if user_defined
+        .keys()
+        .any(|key| rustfs_utils::http::starts_with_ignore_ascii_case(key, MINIO_INTERNAL_SSE_PREFIX))
+    {
+        return false;
+    }
+
     etag.map(|etag| etag.trim_matches('"').len() != 32).unwrap_or(false)
 }
 

crates/ecstore/src/object_api/types.rs

@@ -614,7 +614,20 @@ impl ObjectInfo {
     }
 
     pub fn decrypted_size(&self) -> std::io::Result<i64> {
-        Ok(self.encryption_original_size()?.unwrap_or(self.size))
+        if let Some(recorded) = self.encryption_original_size()? {
+            return Ok(recorded);
+        }
+        // A MinIO single-part object records no plaintext size — MinIO writes one
+        // only for multipart uploads — so `self.size` here is the *physical*
+        // size, encoding overhead included. Reporting that as the object's
+        // length overstates it by exactly that overhead, which is what a client
+        // sees as Content-Length (rustfs/backlog#1638).
+        if rustfs_utils::http::has_minio_internal_sse_metadata(&self.user_defined)
+            && let Some(plaintext) = rustfs_utils::http::dare_v2_decrypted_size(self.size)
+        {
+            return Ok(plaintext);
+        }
+        Ok(self.size)
     }
 
     pub fn get_actual_size(&self) -> std::io::Result<i64> {

crates/utils/src/http/object_encryption_keys.rs

@@ -53,6 +53,50 @@ pub const MINIO_INTERNAL_ENCRYPTION_KMS_KEY_ID_HEADER: &str = "X-Minio-Internal-
 pub const MINIO_INTERNAL_ENCRYPTION_KMS_DATA_KEY_HEADER: &str = "X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key";
 pub const MINIO_INTERNAL_ENCRYPTION_KMS_CONTEXT_HEADER: &str = "X-Minio-Internal-Server-Side-Encryption-Context";
 
+/// Plaintext length of a DARE v2 stream of `ciphertext_size` bytes.
+///
+/// The stream is a sequence of packages, each a 16-byte header, up to 64 KiB of
+/// payload, and a 16-byte tag; only the last may be short. This is the same
+/// arithmetic as MinIO's `sio.DecryptedSize`, and it is the only way to size a
+/// MinIO single-part object: MinIO writes an explicit plaintext size only for
+/// multipart uploads and otherwise derives it from the stream.
+///
+/// Returns `None` for a size no DARE stream can have — a final package carrying
+/// overhead but no payload — so a malformed object is not silently assigned a
+/// plausible length.
+pub fn dare_v2_decrypted_size(ciphertext_size: i64) -> Option<i64> {
+    const HEADER_LEN: i64 = 16;
+    const TAG_LEN: i64 = 16;
+    const MAX_PAYLOAD: i64 = 64 * 1024;
+    const PACKAGE_LEN: i64 = HEADER_LEN + MAX_PAYLOAD + TAG_LEN;
+
+    if ciphertext_size < 0 {
+        return None;
+    }
+    if ciphertext_size == 0 {
+        return Some(0);
+    }
+
+    let full_packages = ciphertext_size / PACKAGE_LEN;
+    let remainder = ciphertext_size % PACKAGE_LEN;
+    if remainder == 0 {
+        return Some(full_packages * MAX_PAYLOAD);
+    }
+    if remainder <= HEADER_LEN + TAG_LEN {
+        return None;
+    }
+    Some(full_packages * MAX_PAYLOAD + remainder - HEADER_LEN - TAG_LEN)
+}
+
+/// True when the metadata was written by MinIO's SSE path.
+pub fn has_minio_internal_sse_metadata<S: std::hash::BuildHasher>(
+    metadata: &std::collections::HashMap<String, String, S>,
+) -> bool {
+    metadata
+        .keys()
+        .any(|key| super::starts_with_ignore_ascii_case(key, "x-minio-internal-server-side-encryption-"))
+}
+
 /// Reserved RustFS-branded twin of the MinIO-internal SSE key family.
 ///
 /// No RustFS writer emits these keys today — the SSE writer persists the
@@ -323,6 +367,31 @@ mod tests {
         assert!(!format!("{projected:?}").contains("secret-key"));
     }
 
+    #[test]
+    fn dare_v2_size_inverts_the_package_layout() {
+        const PACKAGE: i64 = 16 + 64 * 1024 + 16;
+
+        // Exactly one full package, and exactly two.
+        assert_eq!(dare_v2_decrypted_size(PACKAGE), Some(64 * 1024));
+        assert_eq!(dare_v2_decrypted_size(2 * PACKAGE), Some(128 * 1024));
+
+        // The shape that motivated this: a 64 KiB object stored as 65568 bytes.
+        assert_eq!(dare_v2_decrypted_size(65568), Some(65536));
+
+        // A short trailing package carries its own header and tag.
+        assert_eq!(dare_v2_decrypted_size(PACKAGE + 16 + 1 + 16), Some(64 * 1024 + 1));
+        assert_eq!(dare_v2_decrypted_size(16 + 1 + 16), Some(1));
+
+        assert_eq!(dare_v2_decrypted_size(0), Some(0));
+
+        // Sizes no DARE stream can have: overhead with no payload behind it.
+        // Refused rather than rounded into a plausible length.
+        assert_eq!(dare_v2_decrypted_size(1), None);
+        assert_eq!(dare_v2_decrypted_size(32), None);
+        assert_eq!(dare_v2_decrypted_size(PACKAGE + 32), None);
+        assert_eq!(dare_v2_decrypted_size(-1), None);
+    }
+
     #[test]
     fn transport_metadata_roundtrip_restores_stored_keys() {
         let mut headers = http::HeaderMap::new();

docs/operations/kms-backend-security.md

@@ -25,14 +25,16 @@ Support is stated per shape below because that is how far it has been *measured*
 | SSE-S3, multipart | **Yes** | `reads_minio_generated_sse_s3_multipart_fixture` |
 | SSE-KMS, multipart | **Yes** | `reads_minio_generated_sse_kms_multipart_fixture` |
 | SSE-C, multipart | **Yes** | `reads_minio_generated_sse_c_multipart_fixture` |
-| SSE-S3 / SSE-KMS / SSE-C, single-part | **Unverified** | No fixture coverage — see below |
+| SSE-S3, single-part | **Yes** | `reads_minio_generated_sse_s3_singlepart_fixture` |
+| SSE-KMS, single-part | **Yes** | `reads_minio_generated_sse_kms_singlepart_fixture` |
+| SSE-C, single-part | **Unverified** | No fixture coverage |
 | Sealed by KES, a KMS plugin, or MinKMS | **No**, and not planned | Re-encrypt at the source before migrating |
 
 SSE-C needs no KMS at all: the customer supplies the key on each request, exactly as against MinIO. Note that a MinIO SSE-C object stores no customer-key MD5, so the usual early "these parameters do not match" rejection cannot fire for it — a wrong key is refused by the decryption itself instead, which is a different error but the same outcome.
 
 Reading a supported *managed* object (SSE-S3, SSE-KMS) requires RustFS to hold the same master key MinIO used, supplied through `RUSTFS_SSE_S3_MASTER_KEY` (the production entry point, exercised by `reads_minio_generated_sse_s3_fixture_through_production_master_key_env`). MinIO's builtin KMS derives a per-ciphertext sealing key from that master secret, so the *same* secret is required — not merely an equivalently configured backend.
 
-**"Unverified" means unknown, not broken.** Single-part objects below MinIO's small-file threshold carry their data inline in `xl.meta`, sharded across disks, and the interop fixture harness cannot yet load that shape — so those objects have never been read in a test either way. Do not read the table's "Yes" rows as covering them.
+**"Unverified" means unknown, not broken.** The remaining row has no fixture coverage, so it has never been read in a test either way. Do not read the table's "Yes" rows as covering it.
 
 Whatever the table says, verify before you commit: **read a sample of encrypted objects, not just their listings.** A read that is not supported fails closed — ciphertext is never served as plaintext — but two properties still make it easy to discover late:
 

rustfs/src/storage/minio_generated_read_test.rs

@@ -374,6 +374,21 @@ async fn reads_minio_generated_sse_s3_fixture_through_production_master_key_env(
     assert_eq!(sha256_hex(&plaintext), expected_sha256);
 }
 
+/// Objects small enough that MinIO inlined them into xl.meta instead of writing
+/// a part file — the ordinary shape for everyday small objects, and the one
+/// whose encrypted ETag misleads the multipart heuristic.
+#[tokio::test]
+#[ignore = "requires generated MinIO fixture data and a local static KMS key"]
+async fn reads_minio_generated_sse_s3_singlepart_fixture() {
+    assert_fixture_round_trip("sse-s3-singlepart-64k", 64 * 1024).await;
+}
+
+#[tokio::test]
+#[ignore = "requires generated MinIO fixture data and a local static KMS key"]
+async fn reads_minio_generated_sse_kms_singlepart_fixture() {
+    assert_fixture_round_trip("sse-kms-singlepart-64k", 64 * 1024).await;
+}
+
 /// SSE-C is the one managed shape needing no KMS: the customer supplies the key
 /// on every request, so this measures the read path alone.
 #[tokio::test]