Compare commits

...

9 Commits

Author SHA1 Message Date
overtrue 1d305aeb5b docs(sse): record SSE-C as measured-readable in the migration matrix
The SSE-C row moves from unverified to supported now that a MinIO-generated SSE-C object reads end to end, with the test that establishes it. Single-part stays unverified for all three schemes — the fixture harness still cannot load objects MinIO inlined into xl.meta.

Also notes the one behavioural difference an operator will observe: a MinIO SSE-C object stores no customer-key MD5, so a wrong key is refused by the decryption rather than by the earlier parameter-mismatch check. Different error, same outcome.

Refs rustfs/backlog#1638.
2026-08-19 09:25:10 +08:00
overtrue fd26567cd7 fix(sse): read MinIO SSE-C objects, and pin the customer-key check
MinIO SSE-C objects were unreadable for the same reason its managed objects were: the dispatch demanded a header MinIO never persists. A MinIO SSE-C object stores exactly three keys — the internal sealed key, IV, and seal algorithm — and keeps the customer algorithm on the request, returning it on the response. Recognizing the internal sealed-key slot as well is the whole fix on that side; the customer key still has to be supplied by the caller.

The stored customer-key MD5 needed narrower handling. MinIO stores none, so comparing against it made every migrated object fail. The comparison is now skipped only when there is nothing to compare — no stored MD5 *and* the object carries MinIO's SSE-C slot — which does not weaken what the check buys: it is an early, friendlier rejection, while the key itself is proven by the object-key unseal, whose AEAD fails on a wrong key. A negative test holds that line by reading the fixture with a well-formed but wrong customer key.

Writing that test surfaced a gap worth closing on its own: disabling the stored-MD5 comparison for *every* object left all 115 tests in this file green, so nothing guarded it for RustFS-written objects either, and a later widening of the skip would have gone unnoticed. ssec_stored_md5_mismatch_is_refused_when_an_md5_is_stored now fails when that happens.

Verified against fixtures from a real MinIO server: the interop suite is 7/7, including SSE-C multipart, and both mutations — widening the MD5 skip, and the earlier slot-vs-key-id inference — turn it red.

Refs rustfs/backlog#1638.
2026-08-19 09:24:30 +08:00
overtrue 035a6f431a docs(sse): state MinIO interop by measured shape, and pin the production key path
The migration warning said RustFS cannot read MinIO-encrypted objects at all. That is no longer true for the shapes this branch fixes, and a blanket 'no' costs a migrating operator a full decrypt-and-recopy they may not need. It is replaced by a table that states support per object shape, with the test that establishes each row.

The table's honesty depends on one caveat worth stating in code rather than prose: every interop reader test injected its master key through __RUSTFS_SSE_SIMPLE_CMK, which is #[cfg(test)]-only and never reaches LocalSseDekProvider::new_from_env. Those tests going green therefore said nothing about a deployment. A new case reads a fixture through RUSTFS_SSE_S3_MASTER_KEY alone — the sole production entry point — so the claim now rests on the path operators actually run.

Single-part and SSE-C objects are listed as unverified rather than unsupported, because the fixture harness cannot yet load them (small objects live inline in xl.meta, sharded across disks) and they have never been read in a test either way. Conflating 'untested' with 'broken' in the other direction would be the same failure the old warning made.

The reverse direction gets an explicit ruling: MinIO cannot read RustFS-written objects, and the MinIO-branded headers RustFS writes are RustFS-internal rather than a compatibility promise — no coexistence plan should assume two-way reads.

StaticConfig's doc comment is corrected on the same basis: it described MinIO's ciphertext as the legacy JSON encoding, and claimed MinIO-written objects cannot be read at all, which now belongs to the object read path rather than to this backend.

Refs rustfs/backlog#1638.
2026-08-19 08:36:18 +08:00
overtrue 0e4953aeea Merge remote-tracking branch 'upstream/overtrue/kms-1638-d2-minio-sse-read' into overtrue/kms-1638-d2-minio-sse-read 2026-08-19 07:11:23 +08:00
overtrue 99ec65247a fix(sse): gate the MinIO data-key trait method behind rio-v2
The method's only call site sits in the rio-v2 branch of the managed read path, so a build without that feature carried a trait method nothing could reach — a warning under default features and, with -D warnings, a hard failure of the sftp lane. The declaration now carries the same gate its implementation and its sibling decrypt_legacy_sse_dek already had.

Verified against the lane that caught it (cargo clippy -p rustfs --features sftp --all-targets -- -D warnings, clean), plus the default build and the rio-v2 interop suite (4 passed).

Refs rustfs/backlog#1638.
2026-08-19 07:11:10 +08:00
Zhengchao An 6feb573f74 Merge branch 'main' into overtrue/kms-1638-d2-minio-sse-read 2026-08-19 07:09:40 +08:00
houseme d6814af2bc Merge branch 'main' into overtrue/kms-1638-d2-minio-sse-read 2026-08-18 23:54:31 +08:00
houseme dbef072bfe Merge branch 'main' into overtrue/kms-1638-d2-minio-sse-read 2026-08-18 12:46:12 +08:00
overtrue 420bfa859b fix(sse): read objects that MinIO encrypted
RustFS could not read a single MinIO-encrypted object. Two independent blockers, and backlog#1638 could only argue them statically because the fixtures the interop tests consume are generated, not checked in — so those tests had never once run. With the fixture lab working, both are now measured, fixed and covered.

The detection gate required `x-amz-server-side-encryption` to be present. MinIO never persists it: `crypto.S3.CreateMetadata` writes only the `X-Minio-Internal-*` family and the public header is synthesized onto the response by `DecryptObjectInfo`. Every MinIO object therefore fell out of the managed path and failed with "encrypted object metadata is incomplete". The scheme is now inferred from which sealed-key slot is present, which is self-consistent by construction: the slot decides both which header the unseal reads and which domain string the sealing key is derived under, so an inference that disagreed with the slot could not silently derive a wrong key. Inferring from the KMS key id would NOT be safe — MinIO writes `-S3-Kms-Key-Id` on SSE-S3 objects too, which the fixtures show and a mutation test pins.

Past the gate, the data key itself could not be unwrapped. Its wire format is `sealed_bytes || iv[16] || nonce[12]` — the randomness trails the ciphertext rather than leading it — with a per-ciphertext sealing key of `HMAC-SHA256(master, iv)` and the encryption context bound as associated data (`internal/kms/secret-key.go`). Note this is not the `{"aead":...}` JSON that backlog#1638's analysis described: current MinIO writes the raw layout and treats JSON only as a legacy encoding, normalizing it into the same byte order. Both are decoded here, in a decoder of their own — `LocalSseDekEnvelope`'s `deny_unknown_fields` is untouched, since loosening it to admit MinIO's shape would also admit malformed RustFS envelopes that backlog#1567 requires to keep failing closed.

Routing between the two decoders cannot key on metadata: RustFS's own writer fills MinIO's slots while storing a RustFS envelope in them, so neither the slot nor the header name distinguishes writers. It keys on the data key's own shape instead, recognizing the two strict RustFS JSON shapes positively and leaving only the remainder to MinIO — so neither decoder is ever handed the other's format. Three round-trip tests caught an earlier slot-based attempt doing exactly that.

Fail-closed is preserved throughout: a scheme that cannot be established still returns None, and the read plan independently classifies the object as encrypted from its markers and refuses to serve it without material, so no path degrades into returning ciphertext as plaintext.

The interop harness also gets a provider reset. The DEK provider is cached process-wide, so a case that ran earlier kept serving its master key to every later case — which silently made the wrong-key negative test unable to fail. It fails correctly now, and the whole suite is meaningful for the first time.

Refs rustfs/backlog#1638.
2026-08-18 09:32:29 +08:00
5 changed files with 539 additions and 26 deletions
+12 -5
View File
@@ -324,11 +324,18 @@ impl Default for LocalConfig {
/// wraps data encryption keys — there is no HMAC-SHA256 derivation step — and
/// each wrapped DEK is serialized as a RustFS `DataKeyEnvelope` JSON blob.
///
/// This mirrors the *concept* of MinIO's builtin/static single-key KMS, but is
/// not wire-compatible with it: MinIO wraps DEKs in a different (`{"aead": ...}`)
/// blob that this backend neither produces nor accepts, so KMS ciphertext
/// written by MinIO cannot be opened here. Reading MinIO-written SSE objects is
/// tracked separately in rustfs/backlog#1638.
/// This mirrors the *concept* of MinIO's builtin/static single-key KMS but is
/// not wire-compatible with it. MinIO seals a DEK as `sealed || iv[16] ||
/// nonce[12]` under a per-ciphertext key derived from the master secret, with
/// a legacy JSON encoding of the same layout; this backend neither produces
/// nor accepts either, so pointing it at MinIO's master key does **not** make
/// MinIO-written objects readable through it.
///
/// Reading MinIO-written SSE objects is a property of the object read path, not
/// of this backend: that path decodes MinIO's format directly, keyed by
/// `RUSTFS_SSE_S3_MASTER_KEY`. See the migration section of
/// `docs/operations/kms-backend-security.md` for which object shapes are
/// covered, and rustfs/backlog#1638 for the remainder.
#[derive(Clone, Default, Serialize, Deserialize)]
pub struct StaticConfig {
/// Key identifier (name) for the single configured key
+4
View File
@@ -255,6 +255,10 @@ pub use cache::KmsCacheStats;
pub use config::*;
pub use deletion_worker::DeletionReferenceChecker;
pub use encryption::is_data_key_envelope;
// Re-exported so the object layer binds encryption context exactly the way the
// KMS backends do. A second canonicalization is how the object layer once
// serialized a HashMap directly while the Static backend already sorted keys.
pub use encryption::context_aad;
pub use error::{KmsError, KmsUnavailableError, Result};
pub use key_impact::{KeyImpactReport, KeyReference, KeyReferenceKind, ReferenceCompleteness, ReferenceCoverage, ReferenceScope};
pub use manager::KmsManager;
+24 -7
View File
@@ -14,24 +14,41 @@ For how the Vault backends authenticate (static token, AppRole, Kubernetes, Vaul
| Vault Transit | `VaultTransit` | Key-encryption keys never leave Vault; only Transit ciphertext is visible outside | Vault Transit engine (cryptographic isolation) | Delegated to Vault storage | Via Vault Transit key versioning | Deployments that need key material to be unreadable through storage APIs |
| AWS KMS | `AWS` (alias `AwsKms`) | Key material never leaves AWS KMS; RustFS mirrors no key state | AWS KMS (cryptographic isolation) + IAM | Delegated to AWS | On-demand `RotateKeyOnDemand`; prior backing keys stay usable for decryption | Deployments already rooted in AWS IAM that want AWS as the cryptographic root — read [AWS KMS: deviations from the shared backend contract](#aws-kms-deviations-from-the-shared-backend-contract) first |
## Migrating from MinIO: encrypted objects do not carry over
## Migrating from MinIO: what carries over, and what does not
> **Warning: RustFS does not currently support reading objects that MinIO encrypted.**
> This applies to SSE-S3, SSE-KMS, and SSE-C, in every released binary and container image, and it holds regardless of which KMS backend you configure. Configuring the `Static` backend with the same key material MinIO used does **not** make those objects readable — MinIO wraps data keys in a different envelope format that no RustFS backend produces or accepts (`crates/kms/src/config.rs:304-308`). Plan for this **before** moving data. Tracked in rustfs/backlog#1638.
> **Read this before moving data.** Some MinIO-encrypted objects are readable by RustFS and some are not, and the boundary is not where you would guess. Verify against a sample of your own objects rather than assuming either answer. Tracked in rustfs/backlog#1638.
The read does fail closed — ciphertext is never served as plaintext. MinIO's internal encryption headers mark the object as encrypted (`crates/utils/src/http/header_compat.rs:50-67`), so the read path demands encryption material and refuses when none resolves (`crates/ecstore/src/object_api/readers.rs:559-568`). Two properties still make the problem easy to discover late:
Support is stated per shape below because that is how far it has been *measured* — against fixtures captured from a real MinIO server (`minio/minio:RELEASE.2025-09-07T16-13-09Z`), not inferred from the code:
| MinIO object | RustFS read | Evidence |
| --- | --- | --- |
| SSE-S3, multipart | **Yes** | `reads_minio_generated_sse_s3_multipart_fixture` |
| SSE-KMS, multipart | **Yes** | `reads_minio_generated_sse_kms_multipart_fixture` |
| SSE-C, multipart | **Yes** | `reads_minio_generated_sse_c_multipart_fixture` |
| SSE-S3 / SSE-KMS / SSE-C, single-part | **Unverified** | No fixture coverage — see below |
| Sealed by KES, a KMS plugin, or MinKMS | **No**, and not planned | Re-encrypt at the source before migrating |
SSE-C needs no KMS at all: the customer supplies the key on each request, exactly as against MinIO. Note that a MinIO SSE-C object stores no customer-key MD5, so the usual early "these parameters do not match" rejection cannot fire for it — a wrong key is refused by the decryption itself instead, which is a different error but the same outcome.
Reading a supported *managed* object (SSE-S3, SSE-KMS) requires RustFS to hold the same master key MinIO used, supplied through `RUSTFS_SSE_S3_MASTER_KEY` (the production entry point, exercised by `reads_minio_generated_sse_s3_fixture_through_production_master_key_env`). MinIO's builtin KMS derives a per-ciphertext sealing key from that master secret, so the *same* secret is required — not merely an equivalently configured backend.
**"Unverified" means unknown, not broken.** Single-part objects below MinIO's small-file threshold carry their data inline in `xl.meta`, sharded across disks, and the interop fixture harness cannot yet load that shape — so those objects have never been read in a test either way. Do not read the table's "Yes" rows as covering them.
Whatever the table says, verify before you commit: **read a sample of encrypted objects, not just their listings.** A read that is not supported fails closed — ciphertext is never served as plaintext — but two properties still make it easy to discover late:
- **The error does not say what happened.** It surfaces as a 500 `InternalError`, which reads as a RustFS fault rather than "another implementation encrypted this object".
- **Surrounding metadata migrates fine.** The object's `xl.meta` parses, so encrypted objects list and HEAD normally and report plausible sizes. The failure appears only when something reads the payload.
Read a sample of encrypted objects, not just their listings, before decommissioning the MinIO deployment.
Current options for a migration whose source contains encrypted objects:
For any shape that does not read, the options are unchanged:
- Decrypt on the MinIO side first, migrate plaintext, then let RustFS re-encrypt with its own KMS.
- Copy through the S3 API rather than moving drives — MinIO decrypts on read, and RustFS encrypts on write. This re-encrypts rather than preserving ciphertext and costs a full data transfer.
- Leave encrypted objects on MinIO and migrate only unencrypted data.
### The reverse direction does not work
MinIO cannot read objects RustFS encrypted, and that is a deliberate, documented position rather than a gap awaiting a fix. RustFS fills MinIO's metadata slots — the sealed-key and IV headers are MinIO-shaped — but the data key in `X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key` is a RustFS envelope, which MinIO's KMS cannot open. **Treat the MinIO-branded headers on a RustFS-written object as RustFS-internal.** Their presence is not a statement that MinIO can read the object, and no coexistence plan should assume two-way reads.
Inventory the source before choosing: bucket default-encryption settings mean objects can be encrypted without any client having sent SSE headers.
The same limitation applies in reverse — objects RustFS encrypts are not readable by MinIO. For the code-level breakdown of which seams block each SSE mode, see [MinIO file-format interoperability, Part C](../architecture/minio-file-format-compat.md#part-c--server-side-encryption-sse).
+169 -1
View File
@@ -4,7 +4,7 @@ use std::fs;
use std::io::Cursor;
use std::path::{Path, PathBuf};
use super::sse::SseObjectEncryptionResolver;
use super::sse::{SseObjectEncryptionResolver, reset_sse_dek_provider};
use super::storage_api::ecstore_test_support::{
DiskAPI as _, DiskOption, Endpoint, Erasure, GetObjectReader, ObjectInfo, ObjectOptions, create_bitrot_reader, new_disk,
};
@@ -131,6 +131,13 @@ async fn load_fixture_reader_input(case_id: &str) -> (ObjectInfo, Vec<u8>, Strin
async fn read_fixture_plaintext(encrypted: Vec<u8>, object_info: ObjectInfo, kms_key_b64: String) -> Result<Vec<u8>, String> {
let object_size = object_info.size;
// The DEK provider is cached process-wide once built, so without this reset
// a case that ran earlier in the same binary keeps serving its master key to
// every later case — which silently turned the wrong-key negative below into
// a test that could not fail. Reset before each read so the provider is
// built from the key this case actually configured.
reset_sse_dek_provider();
async_with_vars(
[
("__RUSTFS_SSE_SIMPLE_CMK", Some(kms_key_b64)),
@@ -253,6 +260,118 @@ async fn reads_minio_generated_sse_kms_multipart_fixture() {
assert_fixture_round_trip("sse-kms-multipart-8m", 8 * 1024 * 1024).await;
}
/// Read an SSE-C fixture, supplying the customer key the way a client does.
///
/// SSE-C needs no KMS at all — the key arrives on the request — so this path
/// shares nothing with the managed-SSE reads above beyond the fixture loader.
async fn read_ssec_fixture_plaintext(
encrypted: Vec<u8>,
object_info: ObjectInfo,
customer_key_b64: &str,
customer_key_md5_b64: &str,
) -> Result<Vec<u8>, String> {
let object_size = object_info.size;
reset_sse_dek_provider();
let mut headers = http::HeaderMap::new();
headers.insert(
http::HeaderName::from_static("x-amz-server-side-encryption-customer-algorithm"),
http::HeaderValue::from_static("AES256"),
);
headers.insert(
http::HeaderName::from_static("x-amz-server-side-encryption-customer-key"),
http::HeaderValue::from_str(customer_key_b64).expect("fixture customer key is a header value"),
);
headers.insert(
http::HeaderName::from_static("x-amz-server-side-encryption-customer-key-md5"),
http::HeaderValue::from_str(customer_key_md5_b64).expect("fixture customer key md5 is a header value"),
);
let resolver = SseObjectEncryptionResolver;
let (mut reader, offset, length) = GetObjectReader::new_with_resolver(
Box::new(Cursor::new(encrypted)),
None,
&object_info,
&ObjectOptions::default(),
&headers,
Some(&resolver),
)
.await
.map_err(|err| format!("construct GetObjectReader from MinIO SSE-C fixture: {err:?}"))?;
if offset != 0 || length != object_size {
return Err(format!("unexpected fixture range offset={offset} length={length} size={object_size}"));
}
let mut plaintext = Vec::new();
reader
.read_to_end(&mut plaintext)
.await
.map_err(|err| format!("read plaintext from MinIO SSE-C fixture: {err}"))?;
Ok(plaintext)
}
/// The interop claim must hold on the production key entry point, not only on
/// the test-only injection channel every other case here uses.
#[tokio::test]
#[ignore = "requires generated MinIO fixture data and a local static KMS key"]
async fn reads_minio_generated_sse_s3_fixture_through_production_master_key_env() {
let (object_info, encrypted, expected_sha256) = load_fixture_reader_input("sse-s3-multipart-8m").await;
let plaintext = read_fixture_plaintext_via_production_env(encrypted, object_info, minio_static_kms_key_b64())
.await
.expect("fixture must restore through RUSTFS_SSE_S3_MASTER_KEY");
assert_eq!(sha256_hex(&plaintext), expected_sha256);
}
/// SSE-C is the one managed shape needing no KMS: the customer supplies the key
/// on every request, so this measures the read path alone.
#[tokio::test]
#[ignore = "requires generated MinIO fixture data"]
async fn reads_minio_generated_sse_c_multipart_fixture() {
// The fixture lab's fixed SSE-C key; recorded in the case's request.json.
const SSEC_KEY_B64: &str = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=";
const SSEC_KEY_MD5_B64: &str = "tP/LI3N87DFaSk0aoqYgzg==";
let (object_info, encrypted, expected_sha256) = load_fixture_reader_input("sse-c-multipart-8m").await;
let plaintext = read_ssec_fixture_plaintext(encrypted, object_info, SSEC_KEY_B64, SSEC_KEY_MD5_B64)
.await
.expect("MinIO SSE-C fixture must restore with the customer key");
assert_eq!(sha256_hex(&plaintext), expected_sha256);
}
/// The read path skips the stored-MD5 comparison for MinIO SSE-C objects,
/// which store no MD5. This holds the line that made that safe: the customer
/// key is still proven by the object-key unseal, so a wrong key must fail even
/// with nothing to compare it against.
#[tokio::test]
#[ignore = "requires generated MinIO fixture data"]
async fn sse_c_wrong_customer_key_still_fails_without_a_stored_md5() {
// A well-formed 32-byte key that is not the one the fixture was sealed
// with, sent with its own correct MD5 so the request itself is valid.
const WRONG_KEY_B64: &str = "AQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQE=";
const WRONG_KEY_MD5_B64: &str = "0YB4bMPPCf9SNlqiKmM0uQ==";
let (object_info, encrypted, expected_sha256) = load_fixture_reader_input("sse-c-multipart-8m").await;
let result = read_ssec_fixture_plaintext(encrypted, object_info, WRONG_KEY_B64, WRONG_KEY_MD5_B64).await;
match result {
Err(_) => {}
// Never reached today, and asserted rather than assumed: if a future
// change let a wrong key through, returning the real plaintext would be
// the worst possible outcome.
Ok(plaintext) => assert_ne!(
sha256_hex(&plaintext),
expected_sha256,
"a wrong SSE-C customer key must never restore the original plaintext"
),
}
}
#[tokio::test]
#[ignore = "requires generated MinIO fixture data and a local static KMS key"]
async fn rejects_minio_generated_sse_s3_fixture_with_wrong_kms_key() {
@@ -281,6 +400,55 @@ async fn rejects_minio_generated_sse_s3_fixture_with_truncated_ciphertext() {
}
}
/// Read a fixture through the **production** provider selection.
///
/// [`read_fixture_plaintext`] injects the master key through
/// `__RUSTFS_SSE_SIMPLE_CMK`, which is `#[cfg(test)]`-only, so on its own it
/// proves nothing about a deployment: it never reaches
/// `LocalSseDekProvider::new_from_env`. This variant sets only
/// `RUSTFS_SSE_S3_MASTER_KEY` — the sole production entry point — so the
/// interop claim rests on the path operators actually run (backlog#1638).
async fn read_fixture_plaintext_via_production_env(
encrypted: Vec<u8>,
object_info: ObjectInfo,
master_key_b64: String,
) -> Result<Vec<u8>, String> {
let object_size = object_info.size;
reset_sse_dek_provider();
async_with_vars(
[
("__RUSTFS_SSE_SIMPLE_CMK", None::<String>),
("RUSTFS_SSE_S3_MASTER_KEY", Some(master_key_b64)),
],
async move {
let resolver = SseObjectEncryptionResolver;
let (mut reader, offset, length) = GetObjectReader::new_with_resolver(
Box::new(Cursor::new(encrypted)),
None,
&object_info,
&ObjectOptions::default(),
&http::HeaderMap::new(),
Some(&resolver),
)
.await
.map_err(|err| format!("construct GetObjectReader from MinIO raw fixture: {err:?}"))?;
if offset != 0 || length != object_size {
return Err(format!("unexpected fixture range offset={offset} length={length} size={object_size}"));
}
let mut plaintext = Vec::new();
reader
.read_to_end(&mut plaintext)
.await
.map_err(|err| format!("read plaintext from MinIO raw fixture: {err}"))?;
Ok(plaintext)
},
)
.await
}
async fn assert_fixture_round_trip(case_id: &str, expected_size: i64) {
let (object_info, encrypted, expected_sha256) = load_fixture_reader_input(case_id).await;
// `ObjectInfo.size` is the on-disk size. For SSE objects that is the
+330 -13
View File
@@ -1460,6 +1460,16 @@ fn managed_sse_domain(sse_type: SSEType) -> &'static str {
}
}
/// The public `x-amz-server-side-encryption` value a managed scheme reports.
fn managed_sse_public_header(sse_type: SSEType) -> &'static str {
match sse_type {
SSEType::SseKms => ServerSideEncryption::AWS_KMS,
// SSE-C never reaches the managed path; reporting AES256 keeps this
// total without inventing a third public value.
SSEType::SseS3 | SSEType::SseC => ServerSideEncryption::AES256,
}
}
fn canonical_kms_bucket_path(bucket: &str, key: &str) -> String {
path_join_buf(&[bucket, key])
}
@@ -2051,10 +2061,20 @@ pub async fn sse_prepare_encryption(request: PrepareEncryptionRequest<'_>) -> Re
/// }
/// ```
pub async fn sse_decryption(request: DecryptionRequest<'_>) -> Result<Option<DecryptionMaterial>, ApiError> {
// Check for SSE-C encryption
// Check for SSE-C encryption.
//
// The stored customer-algorithm marker is what RustFS writes, but a
// MinIO-written object has only the internal sealed-key slot: MinIO keeps
// the customer algorithm on the request and synthesizes it back onto the
// response, never persisting it. Recognizing that slot as well is what lets
// a migrated SSE-C object be read at all; the customer key still has to be
// supplied, and is still checked against the stored MD5 below.
if request
.metadata
.contains_key("x-amz-server-side-encryption-customer-algorithm")
|| request
.metadata
.contains_key(MINIO_INTERNAL_ENCRYPTION_SSEC_SEALED_KEY_HEADER)
{
let (key, key_md5) = match (request.sse_customer_key, request.sse_customer_key_md5) {
(Some(k), Some(md5)) => (k, md5),
@@ -2068,7 +2088,21 @@ pub async fn sse_decryption(request: DecryptionRequest<'_>) -> Result<Option<Dec
// Verify that the provided key MD5 matches the stored MD5 for security
let stored_md5 = request.metadata.get("x-amz-server-side-encryption-customer-key-md5");
verify_ssec_key_match(key_md5, stored_md5)?;
// MinIO stores no customer-key MD5 — it keeps that header on the request
// and returns it on the response — so requiring one would make every
// migrated SSE-C object unreadable. Skipping the comparison when there is
// nothing to compare against does not weaken the check it performs: the
// stored MD5 is an early, friendlier rejection, while the key itself is
// proven by the object-key unseal below, whose AEAD fails on a wrong key.
// `sse_c_wrong_customer_key_still_fails_without_a_stored_md5` holds that
// line. Objects that *do* carry a stored MD5 are unaffected.
let minio_ssec_without_stored_md5 = stored_md5.is_none()
&& request
.metadata
.contains_key(MINIO_INTERNAL_ENCRYPTION_SSEC_SEALED_KEY_HEADER);
if !minio_ssec_without_stored_md5 {
verify_ssec_key_match(key_md5, stored_md5)?;
}
let mut material = apply_ssec_decryption_material(request.bucket, request.key, request.metadata, key, key_md5).await?;
material.customer_key_md5 = Some(key_md5.clone());
@@ -2445,20 +2479,42 @@ async fn apply_managed_decryption_material_inner(
) -> Result<Option<DecryptionMaterial>, ApiError> {
#[cfg(not(feature = "rio-v2"))]
let _ = (bucket, key);
if !contains_managed_encryption_metadata(metadata) || !metadata.contains_key("x-amz-server-side-encryption") {
if !contains_managed_encryption_metadata(metadata) {
return Ok(None);
}
// Safe: presence is guaranteed by the contains_key check above.
let server_side_encryption = metadata.get("x-amz-server-side-encryption").cloned().unwrap_or_default();
let normalized_metadata = normalize_managed_metadata(metadata, Some(recode_minio_kms_context));
let encryption_type = match server_side_encryption.as_str() {
ServerSideEncryption::AES256 => SSEType::SseS3,
ServerSideEncryption::AWS_KMS => SSEType::SseKms,
_ => SSEType::SseS3,
let encryption_type = match metadata.get("x-amz-server-side-encryption").map(String::as_str) {
Some(ServerSideEncryption::AWS_KMS) => SSEType::SseKms,
Some(_) => SSEType::SseS3,
// MinIO never persists the public scheme header: `crypto.S3.CreateMetadata`
// writes only the `X-Minio-Internal-*` family and the public header is
// synthesized onto the response by `DecryptObjectInfo`. Requiring it here
// is what made every MinIO-encrypted object unreadable (backlog#1638).
//
// Inferring from the sealed-key slot is self-consistent by construction:
// the slot decides which header the unseal reads AND which domain string
// the sealing key is derived under, so a scheme that disagrees with the
// slot cannot silently derive a wrong key — it finds no key at all.
// Inferring from the KMS key id would NOT be safe: MinIO writes
// `-S3-Kms-Key-Id` on SSE-S3 objects too.
#[cfg(feature = "rio-v2")]
None => match infer_minio_managed_sse_type(metadata) {
Some(sse_type) => sse_type,
// Still fail-closed, and deliberately not an error raised here: the
// read plan independently classifies the object as encrypted from
// its markers and refuses to serve it without material, so an
// object whose scheme cannot be established never degrades into a
// plaintext read.
None => return Ok(None),
},
// Without the rio-v2 reader there is no MinIO-format read path to serve
// such an object with, so it stays on the fail-closed branch.
#[cfg(not(feature = "rio-v2"))]
None => return Ok(None),
};
let normalized_metadata = normalize_managed_metadata(metadata, Some(recode_minio_kms_context));
// Extract KMS key ID from metadata (optional, used for provider context)
let kms_key_id = normalized_metadata
.get(INTERNAL_ENCRYPTION_KEY_ID_HEADER)
@@ -2556,8 +2612,19 @@ async fn apply_managed_decryption_material_inner(
} else {
get_local_sse_dek_provider().await?
};
// A MinIO sealed key alone does not mean MinIO wrote the object: RustFS's own
// writer fills MinIO's metadata slots too, while still storing a RustFS
// envelope in them, so neither the slot nor the header name distinguishes the
// two. The data key's own shape does. RustFS envelopes are strictly-parsed
// JSON; MinIO's builtin-KMS ciphertext is opaque bytes that match neither, so
// recognizing RustFS positively — and treating only the remainder as MinIO —
// keeps a RustFS envelope from ever reaching MinIO's decoder.
#[cfg(feature = "rio-v2")]
let decrypted_data_key = if is_legacy_rustfs_managed_metadata(&normalized_metadata) {
let decrypted_data_key = if minio_sealed_key.is_some() && !is_rustfs_managed_data_key(&encrypted_data_key) {
provider
.decrypt_minio_sse_dek(&encrypted_data_key, &kms_key_id, &object_context)
.await
} else if is_legacy_rustfs_managed_metadata(&normalized_metadata) {
provider
.decrypt_legacy_sse_dek(&encrypted_data_key, &kms_key_id, &object_context)
.await
@@ -2592,7 +2659,11 @@ async fn apply_managed_decryption_material_inner(
Ok(Some(DecryptionMaterial {
sse_type: encryption_type,
server_side_encryption: ServerSideEncryption::from(server_side_encryption),
// Synthesized from the resolved scheme rather than read back from
// metadata: a MinIO-written object has no stored scheme header, which is
// exactly why the gate above had to infer it. MinIO synthesizes the same
// header onto its own responses.
server_side_encryption: ServerSideEncryption::from(managed_sse_public_header(encryption_type).to_string()),
kms_key_id: Some(SSEKMSKeyId::from(kms_key_id)),
algorithm,
customer_key_md5: None,
@@ -2659,6 +2730,30 @@ pub trait SseDekProvider: Send + Sync {
) -> Result<[u8; 32], ApiError> {
self.decrypt_sse_dek(encrypted_dek, kms_key_id, context).await
}
/// Unwrap a data key that MinIO's builtin KMS sealed.
///
/// A separate entry point rather than a shape sniff inside
/// [`Self::decrypt_sse_dek`]: the caller already knows the object carries a
/// MinIO sealed key, and MinIO's raw ciphertext is unstructured bytes that
/// no parser can reliably tell apart from anything else. Routing on the
/// caller's knowledge keeps a RustFS envelope from ever reaching MinIO's
/// decoder, and vice versa.
///
/// Defaults to refusing: only a provider holding the MinIO master secret
/// can serve these, and a provider that cannot must fail rather than fall
/// back to a decoder that would misread the bytes.
#[cfg(feature = "rio-v2")]
async fn decrypt_minio_sse_dek(
&self,
_encrypted_dek: &[u8],
_kms_key_id: &str,
_context: &ObjectEncryptionContext,
) -> Result<[u8; 32], ApiError> {
Err(ApiError::from(StorageError::other(
"This KMS provider cannot unwrap a data key sealed by MinIO's builtin KMS",
)))
}
}
// ============================================================================
@@ -2797,6 +2892,163 @@ pub(crate) struct LocalSseDekProvider {
const LOCAL_SSE_DEK_FORMAT_VERSION: u8 = 1;
#[cfg(feature = "rio-v2")]
/// Returns true when a managed-SSE data key is one RustFS itself wrote.
///
/// Both RustFS envelope shapes are strict JSON — the KMS envelope
/// ([`rustfs_kms::is_data_key_envelope`]) and the local provider's
/// [`LocalSseDekEnvelope`], whose `deny_unknown_fields` keeps it from accepting
/// anything else. Recognition is deliberately positive: an unrecognized payload
/// is left to MinIO's decoder rather than guessed at, and neither decoder is
/// ever handed the other's format.
fn is_rustfs_managed_data_key(encrypted_dek: &[u8]) -> bool {
if rustfs_kms::is_data_key_envelope(encrypted_dek) {
return true;
}
std::str::from_utf8(encrypted_dek)
.ok()
.is_some_and(|text| serde_json::from_str::<LocalSseDekEnvelope<'_>>(text).is_ok())
}
#[cfg(feature = "rio-v2")]
/// Associated data MinIO binds when sealing a data key.
///
/// MinIO passes the object's encryption context as the AEAD's associated data,
/// serialized as canonical JSON with sorted keys — the same canonicalization
/// [`rustfs_kms::context_aad`] performs, which is why the context RustFS
/// already rebuilds for the read can be reused verbatim. For SSE-S3 that
/// context is `{bucket: "bucket/object"}`; for SSE-KMS it is whatever the
/// request supplied, recovered from the stored MinIO context header.
fn minio_kms_associated_data(context: &ObjectEncryptionContext) -> Result<Vec<u8>, ApiError> {
let mut ctx = context.encryption_context.clone();
ctx.entry(context.bucket.clone())
.or_insert_with(|| canonical_kms_bucket_path(&context.bucket, &context.object_key));
rustfs_kms::context_aad(&ctx)
.map_err(|e| ApiError::from(StorageError::other(format!("Failed to canonicalize MinIO KMS context: {e}"))))
}
#[cfg(feature = "rio-v2")]
/// MinIO's builtin-KMS ciphertext in its JSON encoding.
///
/// Deliberately its own type rather than a relaxation of
/// [`LocalSseDekEnvelope`]: widening that envelope's `deny_unknown_fields`
/// to admit this shape would also admit malformed RustFS envelopes, which
/// backlog#1567 requires to keep failing closed.
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct MinioKmsCiphertextJson {
aead: String,
#[allow(
dead_code,
reason = "present in MinIO's encoding; the key is identified by metadata instead"
)]
#[serde(default)]
id: String,
iv: String,
nonce: String,
bytes: String,
}
/// Bytes of trailing randomness every MinIO builtin-KMS ciphertext carries:
/// a 16-byte IV followed by a 12-byte nonce, *after* the sealed bytes.
#[cfg(feature = "rio-v2")]
const MINIO_KMS_RANDOM_LEN: usize = 28;
#[cfg(feature = "rio-v2")]
const MINIO_KMS_IV_LEN: usize = 16;
#[cfg(feature = "rio-v2")]
const MINIO_KMS_AEAD_AES_GCM: &str = "AES-256-GCM-HMAC-SHA-256";
#[cfg(feature = "rio-v2")]
const MINIO_KMS_AEAD_CHACHA20: &str = "ChaCha20Poly1305";
#[cfg(feature = "rio-v2")]
/// Unwrap a data key sealed by MinIO's builtin (static-secret) KMS.
///
/// The wire format is `sealed_bytes || iv[16] || nonce[12]` — the randomness
/// trails the ciphertext rather than leading it, and MinIO's own decoder
/// normalizes its legacy JSON encoding into exactly that byte order before
/// opening it (`internal/kms/secret-key.go`, `parseCiphertext`). A raw
/// (non-JSON) ciphertext is AES-256-GCM by definition there; the JSON form
/// names its algorithm.
///
/// The sealing key is derived per ciphertext rather than being the master key:
/// `HMAC-SHA256(master, iv)` for AES-256-GCM, `HChaCha20(master, iv)` for
/// ChaCha20-Poly1305. The encryption context is bound as associated data.
fn decrypt_minio_kms_data_key(encrypted_dek: &[u8], master_key: &[u8; 32], aad: &[u8]) -> Result<[u8; 32], ApiError> {
let (body, algorithm) = match std::str::from_utf8(encrypted_dek) {
// MinIO only treats a payload as JSON when it both starts and ends like
// an object, and falls back to the raw layout when it does not parse —
// mirrored here so a ciphertext that merely looks like JSON is not
// rejected outright.
Ok(text)
if text.starts_with('{')
&& text.ends_with('}')
&& let Ok(json) = serde_json::from_str::<MinioKmsCiphertextJson>(text) =>
{
let decode = |what: &str, value: &str| -> Result<Vec<u8>, ApiError> {
BASE64_STANDARD
.decode(value)
.map_err(|e| ApiError::from(StorageError::other(format!("Invalid MinIO KMS {what}: {e}"))))
};
let mut body = decode("ciphertext", &json.bytes)?;
body.extend_from_slice(&decode("iv", &json.iv)?);
body.extend_from_slice(&decode("nonce", &json.nonce)?);
(body, json.aead)
}
_ => (encrypted_dek.to_vec(), MINIO_KMS_AEAD_AES_GCM.to_string()),
};
if body.len() <= MINIO_KMS_RANDOM_LEN {
return Err(ApiError::from(StorageError::other(
"MinIO KMS ciphertext is too short to carry its IV and nonce",
)));
}
let (sealed, random) = body.split_at(body.len() - MINIO_KMS_RANDOM_LEN);
let (iv, nonce) = random.split_at(MINIO_KMS_IV_LEN);
let plaintext = match algorithm.as_str() {
MINIO_KMS_AEAD_AES_GCM => {
use aes_gcm::{Aes256Gcm, KeyInit, aead::Aead};
let mut mac = HmacSha256::new_from_slice(master_key)
.map_err(|_| ApiError::from(StorageError::other("MinIO KMS sealing key derivation failed")))?;
mac.update(iv);
let sealing_key: [u8; 32] = mac.finalize().into_bytes().into();
let cipher = Aes256Gcm::new_from_slice(&sealing_key)
.map_err(|_| ApiError::from(StorageError::other("MinIO KMS sealing key is not a valid AES-256 key")))?;
let nonce = aes_gcm::Nonce::try_from(nonce)
.map_err(|_| ApiError::from(StorageError::other("MinIO KMS nonce is not 12 bytes")))?;
cipher.decrypt(&nonce, aes_gcm::aead::Payload { msg: sealed, aad })
}
MINIO_KMS_AEAD_CHACHA20 => {
use chacha20poly1305::{KeyInit, XChaCha20Poly1305, aead::Aead};
// MinIO derives this branch's key with HChaCha20 over the 16-byte
// IV, which is exactly XChaCha20-Poly1305's own construction, so the
// extended-nonce cipher does the derivation rather than hand-rolling it.
let mut extended = Vec::with_capacity(MINIO_KMS_IV_LEN + nonce.len());
extended.extend_from_slice(iv);
extended.extend_from_slice(nonce);
let cipher = XChaCha20Poly1305::new_from_slice(master_key)
.map_err(|_| ApiError::from(StorageError::other("MinIO KMS master key is not a valid ChaCha20 key")))?;
let nonce = chacha20poly1305::XNonce::try_from(extended.as_slice())
.map_err(|_| ApiError::from(StorageError::other("MinIO KMS extended nonce is not 24 bytes")))?;
cipher.decrypt(&nonce, chacha20poly1305::aead::Payload { msg: sealed, aad })
}
other => {
return Err(ApiError::from(StorageError::other(format!(
"Unsupported MinIO KMS AEAD algorithm: {other}"
))));
}
}
// An AEAD failure here is authentication, not a decode slip: a wrong master
// key, a tampered ciphertext, and an encryption context that does not match
// what sealed it all land here and must all fail closed.
.map_err(|_| ApiError::from(StorageError::other("MinIO KMS data key failed authentication")))?;
plaintext.try_into().map_err(|value: Vec<u8>| {
ApiError::from(StorageError::other(format!("MinIO KMS data key must be 32 bytes, got {}", value.len())))
})
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct LocalSseDekEnvelope<'a> {
@@ -3013,6 +3265,17 @@ impl SseDekProvider for LocalSseDekProvider {
let dek = Self::decrypt_dek(encrypted_dek_str, self.master_key)?;
Ok(dek)
}
#[cfg(feature = "rio-v2")]
async fn decrypt_minio_sse_dek(
&self,
encrypted_dek: &[u8],
_kms_key_id: &str,
context: &ObjectEncryptionContext,
) -> Result<[u8; 32], ApiError> {
let aad = minio_kms_associated_data(context)?;
decrypt_minio_kms_data_key(encrypted_dek, &self.master_key, &aad)
}
}
// ============================================================================
@@ -3201,6 +3464,23 @@ fn is_legacy_rustfs_managed_metadata(metadata: &HashMap<String, String>) -> bool
&& !metadata.contains_key(MINIO_INTERNAL_ENCRYPTION_KMS_SEALED_KEY_HEADER)
}
#[cfg(feature = "rio-v2")]
#[cfg(feature = "rio-v2")]
/// Infer the managed SSE scheme from the MinIO sealed-key slot that is present.
///
/// Returns `None` when no managed MinIO slot is present, which keeps callers on
/// their fail-closed path. SSE-C is not a managed scheme and is handled by the
/// SSE-C read path, so its slot is not considered here.
fn infer_minio_managed_sse_type(metadata: &HashMap<String, String>) -> Option<SSEType> {
if metadata.contains_key(MINIO_INTERNAL_ENCRYPTION_S3_SEALED_KEY_HEADER) {
Some(SSEType::SseS3)
} else if metadata.contains_key(MINIO_INTERNAL_ENCRYPTION_KMS_SEALED_KEY_HEADER) {
Some(SSEType::SseKms)
} else {
None
}
}
#[cfg(feature = "rio-v2")]
fn parse_minio_managed_sealed_key(
metadata: &HashMap<String, String>,
@@ -4483,6 +4763,43 @@ mod tests {
}
#[cfg(feature = "rio-v2")]
/// A stored customer-key MD5 must still be compared against the one the
/// request presents.
///
/// The read path skips that comparison for MinIO SSE-C objects, which store
/// no MD5. Nothing pinned the check for objects that *do* store one —
/// disabling it outright left this file's 115 tests green — so a later
/// widening of that skip would have gone unnoticed. The mismatch has to be
/// refused here, at the request boundary, rather than surfacing later as a
/// decryption failure.
#[tokio::test]
async fn ssec_stored_md5_mismatch_is_refused_when_an_md5_is_stored() {
let key = SSECustomerKey::from(BASE64_STANDARD.encode([0x11u8; 32]));
let provided_md5 = SSECustomerKeyMD5::from(md5_base64(&[0x11u8; 32]));
let metadata = HashMap::from([
(SSEC_ALGORITHM_HEADER.to_string(), "AES256".to_string()),
// A stored MD5 that belongs to a different key.
("x-amz-server-side-encryption-customer-key-md5".to_string(), md5_base64(&[0x22u8; 32])),
]);
let error = sse_decryption(DecryptionRequest {
bucket: "bucket",
key: "object",
metadata: &metadata,
sse_customer_key: Some(&key),
sse_customer_key_md5: Some(&provided_md5),
principal: None,
})
.await
.expect_err("a stored MD5 that does not match the request must be refused");
assert!(
format!("{error:?}").contains("did not match"),
"expected the parameter-mismatch refusal, got {error:?}"
);
}
#[tokio::test]
async fn test_sse_kms_roundtrip_persists_and_uses_minio_context() {
use rustfs_kms::types::{CreateKeyRequest, KeyUsage};