fix(kms): fail closed on Local key records that cannot be interpreted (#5606)

* fix(kms): fail closed on Local key records this build cannot interpret

The Local backend's protection marker is the only version discriminator its
key records have, and three readers walked past it.

`ensure_missing_salt_can_be_generated` skipped every record it could not read
or parse, so a directory whose protection state is unknown still got a fresh
salt published before startup validation failed. That write is the
irreversible step: the next startup finds a salt file, never re-enters the
guard, and the evidence that the real salt was lost is gone. Every record the
guard now rejects already failed startup key validation a few lines later, so
no directory that initializes today stops initializing.

Backup export and restore folded an unknown marker into "material corrupt" /
"bundle corrupted". The record is intact and a newer build reads it fine, so
the operator response is a version change, not a disaster recovery. Both now
classify the marker before their schema parse, sharing one probe with the
backend reader.

`list_keys` dropped any record it could not decode from the page. Concurrent
removal stays a skip; anything else fails the listing rather than answering
"these are your keys" with a set that silently omits one.

* test(kms): cover every fail-closed path around the Local protection marker

Each test fails on the pre-fix code in the way the fix is about: the salt
cases because a replacement salt is published before startup validation
fails, the export and restore cases because the verdict comes back as
corruption, and the listing case because the record is edited out of the page.

The restore commit marker's unknown-version branch had no test at all,
unlike its Vault counterpart; it is now driven from the decoder, from the
restore entry point, and from backend startup.

Also states the widened salt guard in the Local backend operations doc,
including the operator recovery path for an unrecognized record.

* fix(kms): say 'not a readable JSON object' when the marker probe cannot parse

The probe now fails on any input that is not a JSON object, not only on
malformed JSON, so the message must cover both.

* test(kms): assert the salt file before the error variant

The replacement salt is written before the error the guard reports, so the
file assertion is the one that fails on a regression.

* test(kms): guard the new backup error variant's display string
This commit is contained in:
Zhengchao An
2026-08-02 13:43:38 +08:00
committed by GitHub
parent 9644064e57
commit c147afd19c
6 changed files with 343 additions and 29 deletions
+37 -2
View File
@@ -48,7 +48,7 @@
//! published. A crash at any earlier point leaves a bundle without a
//! manifest, which decodes as an incomplete bundle and can never be restored.
use crate::backends::local::{LocalKmsClient, StoredKeyProtection};
use crate::backends::local::{LocalKmsClient, StoredKeyProtection, unknown_protection_marker};
use crate::backup::capability::{AtRestProtection, BackupBackendKind, BackupResponsibility};
use crate::backup::error::BackupError;
use crate::backup::manifest::{
@@ -376,7 +376,16 @@ async fn collect_snapshot(client: &LocalKmsClient) -> Result<CollectedSnapshot>
let raw = Zeroizing::new(fs::read(&path).await?);
// Any unreadable record aborts the export: a bundle silently missing
// one key is worse than no bundle at all.
// one key is worse than no bundle at all. The protection marker is
// classified first so a record from a newer build keeps its own
// verdict — an operator who reads "material corrupt" starts a
// disaster recovery for what is only a version mismatch.
let unknown_marker = unknown_protection_marker(&raw).map_err(|error| {
KmsError::material_corrupt(&stem, format!("stored key record is not a readable JSON object: {error}"))
})?;
if let Some(version) = unknown_marker {
return Err(KmsError::unsupported_format_version(&stem, version));
}
let probe: StoredRecordProbe = serde_json::from_slice(&raw)
.map_err(|error| KmsError::material_corrupt(&stem, format!("stored key record does not deserialize: {error}")))?;
if probe.key_id != stem {
@@ -1115,4 +1124,30 @@ mod tests {
.expect_err("identity mismatch must abort the export");
assert!(matches!(error, KmsError::InvalidKey { .. }), "got {error:?}");
}
/// A record written by a newer build must abort the export as an
/// unsupported format, not as corrupt material: the two verdicts send the
/// operator down completely different runbooks, and only one of them is
/// true here.
#[tokio::test]
async fn record_from_a_newer_build_aborts_export_as_unsupported_format() {
let (client, _key_dir) = encrypted_client().await;
client.create_key("alpha", "AES_256", None).await.expect("create key");
let record_path = client.key_directory().join("alpha.key");
let mut record: serde_json::Value =
serde_json::from_slice(&std::fs::read(&record_path).expect("read record")).expect("decode record");
record["at_rest_protection"] = serde_json::json!("post-quantum-v2");
std::fs::write(&record_path, serde_json::to_vec_pretty(&record).expect("encode record")).expect("write record");
let bundle = TempDir::new().expect("bundle dir");
let error = export_local_backup(&client, &test_kek(), &export_request(bundle.path().join("bundle")))
.await
.expect_err("an uninterpretable record must abort the export");
assert!(
matches!(&error, KmsError::UnsupportedFormatVersion { key_id, version }
if key_id == "alpha" && version == "post-quantum-v2"),
"got {error:?}"
);
}
}