fix(kms): refuse to rotate a Vault key whose baseline version was erased (#5578)

`VaultKeyData::baseline_version` pins the master key version that every
pre-versioning DEK envelope (one with no `master_key_version`) resolves to.
Builds released before versioned rotation do not know the field, and every KV2
lifecycle write — enable, disable, schedule/cancel deletion, tag, untag —
rewrites the whole record, so a single lifecycle call from an older node during
a rolling upgrade silently drops the baseline. Serde attributes cannot prevent
this: the code doing the dropping already shipped.

The loss is only latent until the next rotation. With the baseline gone,
rotation takes the first-rotation path again and freezes a *new* baseline at the
current version, so every legacy envelope permanently resolves to material that
never wrapped it. That is the point of no return, and it is the one this commit
blocks: rotation now lists the key's immutable version records first and refuses
when records exist while the record carries no baseline. Those two are created
by the same commit, so the combination can only mean the baseline was erased
afterwards. The refusal is decided from reads alone, before any write, and names
the version to restore — the oldest recorded version *is* the lost baseline,
since version records start at the baseline the first rotation froze.

Reads are diagnosed rather than blocked. A version-less envelope on a key with
no baseline resolves to the current version, which AES-256-GCM refuses to
unwrap when it is the wrong one, so no wrong plaintext can be returned. Only
after that failure does decrypt list the version records and re-report the
failure as the lost baseline. Refusing up front on the same evidence would break
reads that work today: an older node writes version-less envelopes wrapped with
whatever material is current, and those still decrypt. This also keeps the extra
listing off every read of pre-versioning data.

Self-healing (writing back `baseline_version = min(recorded)`) is deliberately
not done: during the mixed-version window that caused the loss, an older node
can erase it again on the next lifecycle call, so healing would mask an
unfinished upgrade instead of surfacing it. Moving the baseline to a KV path
older builds cannot rewrite is the real fix and is scheduled for GA.

Refs rustfs/backlog#1581 (part of rustfs/backlog#1562)
This commit is contained in:
Zhengchao An
2026-08-01 22:02:52 +08:00
committed by GitHub
parent 860ad68afd
commit 02aa383598
3 changed files with 454 additions and 20 deletions
+19
View File
@@ -137,6 +137,13 @@ pub enum KmsError {
/// fail closed instead of being sent with credentials that may lapse mid-flight
#[error("KMS credentials unavailable: {message}")]
CredentialsUnavailable { message: String },
/// Key has master key version records but no baseline version, so envelopes
/// written before versioned rotation can no longer be resolved
#[error(
"Baseline version lost for key {key_id}: master key version records exist (oldest {oldest_version}) but the key record carries no baseline version, so data keys written before versioned rotation can no longer be resolved to the master key version that wrapped them. A node older than versioned rotation rewrote the key record and dropped the field. Finish upgrading every node, restore baseline_version to {oldest_version} on the key record, then retry"
)]
BaselineVersionLost { key_id: String, oldest_version: u32 },
}
impl KmsError {
@@ -291,6 +298,18 @@ impl KmsError {
pub fn credentials_unavailable<S: Into<String>>(message: S) -> Self {
Self::CredentialsUnavailable { message: message.into() }
}
/// Create a baseline version lost error
///
/// `oldest_version` is the lowest master key version that still has a
/// material record; it is exactly the baseline that was dropped, because
/// version records start at the baseline the first rotation froze.
pub fn baseline_version_lost<S: Into<String>>(key_id: S, oldest_version: u32) -> Self {
Self::BaselineVersionLost {
key_id: key_id.into(),
oldest_version,
}
}
}
/// Convert from standard library errors