mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-26 05:56:50 +00:00
feat(kms): report the configuration references that block a key deletion (#5598)
* feat(kms): report configuration references that block a key deletion Adds a KeyImpactReport that states which configuration still points at a key, how exhaustively the sources were read, and which sources were not consulted at all. The report deliberately carries no in-use or safe-to-delete claim: it covers the configuration layer only, so an empty reference list means nothing was found in the scanned sources, never that the key is unreferenced. Immediate deletion destroys key material without ever reaching the deletion worker, so it never passed the worker's reference gate. The manager now consults the same checker on that path and refuses with a typed KeyStillReferenced error. This only ever adds a refusal; the scheduled deletion path and the worker's blocking behaviour are unchanged. * test(kms): cover the immediate-deletion reference refusal * feat(kms): surface configuration references on the admin key endpoints DeleteKey and DescribeKey now return an impact section listing the configuration that points at the key, so an operator scheduling a deletion sees what will refuse to destroy the material instead of learning it from a server-side log once the window has run out. The section is reported, never acted on: scheduling still succeeds while references exist, and the deletion worker's gate remains the only thing that decides whether material is destroyed. An immediate deletion that the manager refuses for an outstanding reference now answers 409. * test(kms): pin the impact wire shape and the unreferenced force-delete path * fix(kms): make the DescribeKey impact section opt-in Collecting the section lists every bucket, and DescribeKey is polled, so carrying that fan-out on the default read path trades a hot path's cost for a diagnostic. It is now collected only for impact=true; without the parameter the endpoint does exactly the work it did before and returns no impact field. A value that is neither true nor false is refused rather than read as off, so a typo cannot answer a request for the section with a response that merely lacks one. DeleteKey still reports unconditionally: that is the request whose consequences the caller cannot otherwise see, and it is not polled. * fix(kms): box the query-parse refusal now that responses carry impact The delete response grew an impact section, which pushed it past the size clippy accepts inline in a Result. It is a full response body rather than an error code, so it is boxed at the one place that returns it as an error; the wire shape and the public field type are unchanged.
This commit is contained in:
@@ -156,6 +156,7 @@ pub fn error_class(error: &KmsError) -> &'static str {
|
||||
KmsError::UnsupportedCapability { .. } => "unsupported_capability",
|
||||
KmsError::CredentialsUnavailable { .. } => "credentials_unavailable",
|
||||
KmsError::BaselineVersionLost { .. } => "baseline_version_lost",
|
||||
KmsError::KeyStillReferenced { .. } => "key_still_referenced",
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -162,6 +162,17 @@ fn record_sweep(report: &SweepReport, census: Option<KeyCensus>) {
|
||||
/// referencing configuration lives (for example bucket encryption settings in
|
||||
/// the server) and are injected via
|
||||
/// [`crate::service_manager::KmsServiceManager::set_deletion_reference_checker`].
|
||||
///
|
||||
/// # Blocks only
|
||||
///
|
||||
/// This gate is one-directional and must stay that way: a checker may add a
|
||||
/// reason to keep material, never a reason to destroy it. Nothing that
|
||||
/// enumerates key usage — least of all a scan whose coverage depends on how
|
||||
/// far a background sweep got — may ever be wired in as a condition that
|
||||
/// releases a removal this gate is holding. One incomplete scan would then be
|
||||
/// enough to destroy the only copy of a key that still has data behind it,
|
||||
/// which is why [`crate::key_impact::KeyImpactReport`] exposes no clearance
|
||||
/// and is consumed only where a refusal is being decided.
|
||||
#[async_trait]
|
||||
pub trait DeletionReferenceChecker: Send + Sync {
|
||||
/// Identifiers of configuration still referencing `key_id` (bucket names,
|
||||
|
||||
@@ -144,6 +144,16 @@ pub enum KmsError {
|
||||
"Baseline version lost for key {key_id}: master key version records exist (oldest {oldest_version}) but the key record carries no baseline version, so data keys written before versioned rotation can no longer be resolved to the master key version that wrapped them. A node older than versioned rotation rewrote the key record and dropped the field. Finish upgrading every node, restore baseline_version to {oldest_version} on the key record, then retry"
|
||||
)]
|
||||
BaselineVersionLost { key_id: String, oldest_version: u32 },
|
||||
|
||||
/// Configuration still points at the key, so its material must not be
|
||||
/// destroyed. Distinct from the generic invalid-operation errors so that
|
||||
/// callers can tell "this key is still wired into the deployment" apart
|
||||
/// from a malformed request and act on the listed references.
|
||||
#[error(
|
||||
"Key {key_id} is still referenced by configuration and its material must not be destroyed: {}. Remove or repoint the listed configuration, then retry",
|
||||
.references.join(", ")
|
||||
)]
|
||||
KeyStillReferenced { key_id: String, references: Vec<String> },
|
||||
}
|
||||
|
||||
impl KmsError {
|
||||
@@ -252,6 +262,14 @@ impl KmsError {
|
||||
Self::OperationCancelled { message: message.into() }
|
||||
}
|
||||
|
||||
/// Create a still-referenced error
|
||||
pub fn key_still_referenced<S: Into<String>>(key_id: S, references: Vec<String>) -> Self {
|
||||
Self::KeyStillReferenced {
|
||||
key_id: key_id.into(),
|
||||
references,
|
||||
}
|
||||
}
|
||||
|
||||
/// Create a material missing error
|
||||
pub fn material_missing<S: Into<String>>(key_id: S) -> Self {
|
||||
Self::MaterialMissing { key_id: key_id.into() }
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
// Copyright 2024 RustFS Team
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! What still points at a KMS key, as far as the server can prove it.
|
||||
//!
|
||||
//! # This report never says a key is unused
|
||||
//!
|
||||
//! There is deliberately no `in_use`, no `unreferenced`, and no
|
||||
//! `safe_to_delete`: the report states which sources were consulted
|
||||
//! ([`ReferenceCoverage`]), how exhaustively they could be read
|
||||
//! ([`ReferenceCompleteness`]), and what was found. An empty
|
||||
//! [`KeyImpactReport::references`] therefore means "nothing was found in the
|
||||
//! scanned sources", never "nothing references this key" — object envelopes
|
||||
//! written under a key are not, and cannot cheaply be, enumerated here.
|
||||
//! Collapsing that distinction into a boolean would hand callers a green
|
||||
//! checkmark backed by an unscanned half of the problem, so the distinction
|
||||
//! lives in the type rather than in prose a UI can skip.
|
||||
//!
|
||||
//! # It may only ever add a reason to refuse
|
||||
//!
|
||||
//! A report is an input to refusing destruction, never to permitting it.
|
||||
//! [`KeyImpactReport::blocks_destruction`] is phrased so its `false` case
|
||||
//! carries no authority: it means this report found no reason to refuse, not
|
||||
//! that any other gate has been satisfied. The deletion worker's own
|
||||
//! [`crate::DeletionReferenceChecker`] stays the gate that decides whether
|
||||
//! expired material is destroyed.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A place where references to a key can live.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub enum ReferenceScope {
|
||||
/// A bucket's default server-side encryption configuration.
|
||||
BucketDefaultEncryption,
|
||||
/// The KMS service's configured default key.
|
||||
ServiceDefaultKey,
|
||||
/// Data-key envelopes stored on object versions.
|
||||
ObjectEnvelopes,
|
||||
/// Session envelopes of multipart uploads that have not completed yet.
|
||||
InProgressMultipartUploads,
|
||||
}
|
||||
|
||||
/// Why an entry appears in [`KeyImpactReport::references`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub enum KeyReferenceKind {
|
||||
/// A bucket's default encryption configuration names the key.
|
||||
BucketDefaultEncryption,
|
||||
/// The key is the KMS service's configured default key.
|
||||
ServiceDefaultKey,
|
||||
/// A whole source in [`ReferenceCoverage::scanned`] could not be
|
||||
/// enumerated. Reported as a reference, not as an absence: a source that
|
||||
/// cannot be read may hold references, and destroying material on the
|
||||
/// strength of an unanswered question is the one outcome that cannot be
|
||||
/// undone.
|
||||
UnreadableSource,
|
||||
/// One resource inside an otherwise readable source could not be
|
||||
/// inspected. Reported for the same reason as [`Self::UnreadableSource`].
|
||||
UnreadableResource,
|
||||
}
|
||||
|
||||
/// One thing that points at a key, or one place that could not be checked.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct KeyReference {
|
||||
/// Machine-readable category.
|
||||
pub kind: KeyReferenceKind,
|
||||
/// Identifier of the referencing resource: a bucket name for bucket
|
||||
/// configuration, the key id for the service default key, the affected
|
||||
/// source or resource for the unreadable kinds. Never key material.
|
||||
pub id: String,
|
||||
/// Human-readable detail. Identifiers only; never secrets or material.
|
||||
pub detail: String,
|
||||
}
|
||||
|
||||
/// Which sources a report consulted, and which it did not look at at all.
|
||||
///
|
||||
/// `not_scanned` is mandatory rather than implied: a caller reading an empty
|
||||
/// reference list has to be able to see, from the report alone, what was left
|
||||
/// out of it.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct ReferenceCoverage {
|
||||
/// Sources this report enumerated.
|
||||
pub scanned: Vec<ReferenceScope>,
|
||||
/// Sources this report does not cover at all.
|
||||
pub not_scanned: Vec<ReferenceScope>,
|
||||
}
|
||||
|
||||
/// How far a report's reference list can be trusted.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub enum ReferenceCompleteness {
|
||||
/// Every reference within [`ReferenceCoverage::scanned`] was enumerated.
|
||||
/// Reserved for configuration-layer facts, which are finite, cheap to
|
||||
/// read, and therefore exhaustively decidable.
|
||||
Exact,
|
||||
/// The list holds what a snapshot happened to observe within the scanned
|
||||
/// scopes. Absence of a reference is not evidence that none exists.
|
||||
ObservedOnly,
|
||||
/// At least one scanned source could not be read, so the list is not a
|
||||
/// statement about the key at all.
|
||||
Unavailable,
|
||||
}
|
||||
|
||||
/// What the server can currently say about who points at a key.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct KeyImpactReport {
|
||||
/// Key the report is about.
|
||||
pub key_id: String,
|
||||
/// How far [`Self::references`] can be trusted.
|
||||
pub completeness: ReferenceCompleteness,
|
||||
/// Which sources were consulted and which were not.
|
||||
pub coverage: ReferenceCoverage,
|
||||
/// References found, plus one entry per source that could not be read.
|
||||
pub references: Vec<KeyReference>,
|
||||
}
|
||||
|
||||
impl KeyImpactReport {
|
||||
/// An empty report over the configuration layer: bucket default encryption
|
||||
/// settings and the service default key.
|
||||
///
|
||||
/// Both are finite and cheap to enumerate, so a report that reads them all
|
||||
/// stays [`ReferenceCompleteness::Exact`]; object-level scopes are
|
||||
/// declared as not scanned and stay that way.
|
||||
pub fn configuration_layer(key_id: impl Into<String>) -> Self {
|
||||
Self {
|
||||
key_id: key_id.into(),
|
||||
completeness: ReferenceCompleteness::Exact,
|
||||
coverage: ReferenceCoverage {
|
||||
scanned: vec![ReferenceScope::BucketDefaultEncryption, ReferenceScope::ServiceDefaultKey],
|
||||
not_scanned: vec![ReferenceScope::ObjectEnvelopes, ReferenceScope::InProgressMultipartUploads],
|
||||
},
|
||||
references: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Record one reference.
|
||||
///
|
||||
/// An unreadable source or resource downgrades the report to
|
||||
/// [`ReferenceCompleteness::Unavailable`] here rather than at the call
|
||||
/// site, so no producer can report a partially read source as `Exact`.
|
||||
pub fn push_reference(&mut self, reference: KeyReference) {
|
||||
if matches!(reference.kind, KeyReferenceKind::UnreadableSource | KeyReferenceKind::UnreadableResource) {
|
||||
self.completeness = ReferenceCompleteness::Unavailable;
|
||||
}
|
||||
self.references.push(reference);
|
||||
}
|
||||
|
||||
/// Whether this report on its own is reason to refuse destroying the key's
|
||||
/// material right now.
|
||||
///
|
||||
/// The two answers are not symmetric. `true` is a decision: something
|
||||
/// points at the key, or a source that might could not be read. `false`
|
||||
/// only means this report contributes no objection — it is never a
|
||||
/// clearance, and must not be used to skip, shorten, or satisfy any other
|
||||
/// check on the deletion path.
|
||||
pub fn blocks_destruction(&self) -> bool {
|
||||
// The completeness test is redundant while every producer records an
|
||||
// unreadable source as a reference, and stays here so that a future
|
||||
// producer which forgets to cannot turn an unanswered question into a
|
||||
// silent clearance.
|
||||
!self.references.is_empty() || matches!(self.completeness, ReferenceCompleteness::Unavailable)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn bucket_reference(bucket: &str) -> KeyReference {
|
||||
KeyReference {
|
||||
kind: KeyReferenceKind::BucketDefaultEncryption,
|
||||
id: bucket.to_string(),
|
||||
detail: format!("bucket {bucket} encrypts new objects with this key by default"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fresh_configuration_report_declares_the_object_layer_unscanned() {
|
||||
let report = KeyImpactReport::configuration_layer("kms-key-1");
|
||||
|
||||
assert_eq!(report.completeness, ReferenceCompleteness::Exact);
|
||||
assert!(report.references.is_empty());
|
||||
assert_eq!(
|
||||
report.coverage.not_scanned,
|
||||
vec![ReferenceScope::ObjectEnvelopes, ReferenceScope::InProgressMultipartUploads],
|
||||
"an empty reference list is only readable next to what was left unscanned"
|
||||
);
|
||||
assert!(!report.blocks_destruction());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn any_reference_blocks_destruction() {
|
||||
let mut report = KeyImpactReport::configuration_layer("kms-key-1");
|
||||
report.push_reference(bucket_reference("sse-bucket"));
|
||||
|
||||
assert!(report.blocks_destruction());
|
||||
assert_eq!(
|
||||
report.completeness,
|
||||
ReferenceCompleteness::Exact,
|
||||
"a fully readable configuration layer stays exact even when it holds references"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unreadable_source_is_never_reported_as_an_absence() {
|
||||
for kind in [KeyReferenceKind::UnreadableSource, KeyReferenceKind::UnreadableResource] {
|
||||
let mut report = KeyImpactReport::configuration_layer("kms-key-1");
|
||||
report.push_reference(KeyReference {
|
||||
kind,
|
||||
id: "bucket-default-encryption".to_string(),
|
||||
detail: "configuration could not be read".to_string(),
|
||||
});
|
||||
|
||||
assert_eq!(
|
||||
report.completeness,
|
||||
ReferenceCompleteness::Unavailable,
|
||||
"{kind:?} must downgrade completeness"
|
||||
);
|
||||
assert!(report.blocks_destruction(), "{kind:?} must block destruction");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unavailable_completeness_blocks_even_without_references() {
|
||||
// Guards the fail-closed fallback for a producer that marks a report
|
||||
// unavailable without recording why.
|
||||
let report = KeyImpactReport {
|
||||
completeness: ReferenceCompleteness::Unavailable,
|
||||
..KeyImpactReport::configuration_layer("kms-key-1")
|
||||
};
|
||||
|
||||
assert!(report.references.is_empty());
|
||||
assert!(report.blocks_destruction());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn report_round_trips_through_json() {
|
||||
let mut report = KeyImpactReport::configuration_layer("kms-key-1");
|
||||
report.push_reference(bucket_reference("sse-bucket"));
|
||||
report.push_reference(KeyReference {
|
||||
kind: KeyReferenceKind::ServiceDefaultKey,
|
||||
id: "kms-key-1".to_string(),
|
||||
detail: "configured as the KMS service default key".to_string(),
|
||||
});
|
||||
|
||||
let json = serde_json::to_string(&report).expect("serialization should succeed");
|
||||
let decoded: KeyImpactReport = serde_json::from_str(&json).expect("deserialization should succeed");
|
||||
assert_eq!(decoded, report);
|
||||
}
|
||||
|
||||
/// The wire shape is the contract callers build UIs on: a boolean that
|
||||
/// reads as "this key is unused" must never appear in it, however the
|
||||
/// report is populated.
|
||||
#[test]
|
||||
fn the_wire_shape_asserts_nothing_about_absence_of_use() {
|
||||
let mut referenced = KeyImpactReport::configuration_layer("kms-key-1");
|
||||
referenced.push_reference(bucket_reference("sse-bucket"));
|
||||
|
||||
for report in [KeyImpactReport::configuration_layer("kms-key-1"), referenced] {
|
||||
let json = serde_json::to_value(&report).expect("serialization should succeed");
|
||||
let mut fields: Vec<&str> = json
|
||||
.as_object()
|
||||
.expect("report is a JSON object")
|
||||
.keys()
|
||||
.map(String::as_str)
|
||||
.collect();
|
||||
fields.sort_unstable();
|
||||
|
||||
assert_eq!(fields, vec!["completeness", "coverage", "key_id", "references"]);
|
||||
for forbidden in ["in_use", "unused", "unreferenced", "safe_to_delete", "deletable"] {
|
||||
assert!(!json.to_string().contains(forbidden), "report must not carry a `{forbidden}` claim");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -73,6 +73,7 @@ pub mod config;
|
||||
pub mod deletion_worker;
|
||||
mod encryption;
|
||||
mod error;
|
||||
pub mod key_impact;
|
||||
pub mod manager;
|
||||
mod policy;
|
||||
pub mod probe;
|
||||
@@ -94,6 +95,7 @@ pub use config::*;
|
||||
pub use deletion_worker::DeletionReferenceChecker;
|
||||
pub use encryption::is_data_key_envelope;
|
||||
pub use error::{KmsError, KmsUnavailableError, Result};
|
||||
pub use key_impact::{KeyImpactReport, KeyReference, KeyReferenceKind, ReferenceCompleteness, ReferenceCoverage, ReferenceScope};
|
||||
pub use manager::KmsManager;
|
||||
pub use probe::{ProbeFailureKind, ProbeResult, ProbeStatus};
|
||||
pub use service::{DataKey, ObjectEncryptionService};
|
||||
|
||||
@@ -18,6 +18,7 @@ use crate::audit::{KmsAuditOperation, KmsAuditRecord, KmsAuditSink};
|
||||
use crate::backends::KmsBackend;
|
||||
use crate::cache::{KmsCache, KmsCacheStats};
|
||||
use crate::config::{ENV_KMS_ALLOW_IMMEDIATE_DELETION, KmsConfig};
|
||||
use crate::deletion_worker::DeletionReferenceChecker;
|
||||
use crate::error::{KmsError, Result};
|
||||
use crate::types::{
|
||||
CancelKeyDeletionRequest, CancelKeyDeletionResponse, CreateKeyRequest, CreateKeyResponse,
|
||||
@@ -41,6 +42,7 @@ pub struct KmsManager {
|
||||
backend_kind: &'static str,
|
||||
audit_sink: Option<Arc<dyn KmsAuditSink>>,
|
||||
allow_immediate_deletion: bool,
|
||||
reference_checker: Option<Arc<dyn DeletionReferenceChecker>>,
|
||||
}
|
||||
|
||||
impl KmsManager {
|
||||
@@ -60,9 +62,21 @@ impl KmsManager {
|
||||
backend_kind: config.backend.as_str(),
|
||||
audit_sink: None,
|
||||
allow_immediate_deletion: config.allow_immediate_deletion,
|
||||
reference_checker: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Consult `checker` before immediate deletion destroys key material.
|
||||
///
|
||||
/// This is the same checker the deletion worker consults before it removes
|
||||
/// an expired key; installing it here extends that gate to the one
|
||||
/// deletion path that never reaches the worker. It can only add a refusal:
|
||||
/// without a checker the manager behaves exactly as before.
|
||||
pub fn with_deletion_reference_checker(mut self, checker: Option<Arc<dyn DeletionReferenceChecker>>) -> Self {
|
||||
self.reference_checker = checker;
|
||||
self
|
||||
}
|
||||
|
||||
/// Send an audit record for every management operation to `sink`.
|
||||
///
|
||||
/// Without a sink the manager builds no records at all, so a deployment
|
||||
@@ -262,6 +276,9 @@ impl KmsManager {
|
||||
/// defensive assertion for callers that hold a backend handle directly.
|
||||
async fn delete_key_inner(&self, request: DeleteKeyRequest) -> Result<DeleteKeyResponse> {
|
||||
self.check_deletion_request(&request)?;
|
||||
if request.force_immediate.unwrap_or(false) {
|
||||
self.refuse_referenced_immediate_deletion(&request.key_id).await?;
|
||||
}
|
||||
|
||||
let response = self.backend.delete_key(request).await?;
|
||||
|
||||
@@ -312,6 +329,41 @@ impl KmsManager {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Refuse an immediate deletion while configuration still points at the
|
||||
/// key.
|
||||
///
|
||||
/// A scheduled deletion is re-checked against these same references by the
|
||||
/// deletion worker before it destroys anything, and stays cancellable
|
||||
/// until then. Immediate deletion has neither property: it destroys
|
||||
/// material on the spot and never reaches the worker, so the check has to
|
||||
/// happen here or not at all.
|
||||
///
|
||||
/// Only ever a refusal. An empty reference set is not a clearance — it
|
||||
/// means the sources consulted here raised no objection, while the caller
|
||||
/// still had to pass the server-side opt-in and the key-id confirmation to
|
||||
/// get this far. With no checker installed the manager has no
|
||||
/// configuration source to consult and behaves as it did before, matching
|
||||
/// the deletion worker, which also skips a checker it was not given.
|
||||
async fn refuse_referenced_immediate_deletion(&self, key_id: &str) -> Result<()> {
|
||||
let mut references = Vec::new();
|
||||
if self.default_key_id.as_deref() == Some(key_id) {
|
||||
references.push("kms-service-default-key".to_string());
|
||||
}
|
||||
if let Some(checker) = &self.reference_checker {
|
||||
references.extend(checker.references(key_id).await);
|
||||
}
|
||||
if references.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
warn!(
|
||||
key_id,
|
||||
?references,
|
||||
"immediate KMS key deletion refused; configuration still references the key"
|
||||
);
|
||||
Err(KmsError::key_still_referenced(key_id, references))
|
||||
}
|
||||
|
||||
/// Cancel key deletion
|
||||
///
|
||||
/// Audited as an internal operation; callers serving an authenticated
|
||||
@@ -1256,6 +1308,191 @@ mod tests {
|
||||
assert!(matches!(error, KmsError::KeyNotFound { .. }), "expected KeyNotFound, got {error:?}");
|
||||
}
|
||||
|
||||
/// Reference checker whose answer is fixed, standing in for the server's
|
||||
/// bucket-configuration gate.
|
||||
struct StaticReferences(Vec<String>);
|
||||
|
||||
#[async_trait]
|
||||
impl DeletionReferenceChecker for StaticReferences {
|
||||
async fn references(&self, _key_id: &str) -> Vec<String> {
|
||||
self.0.clone()
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn immediate_deletion_is_refused_while_configuration_references_the_key() {
|
||||
let temp_dir = tempdir().expect("Failed to create temp dir");
|
||||
let manager = deletion_manager(&temp_dir, true)
|
||||
.await
|
||||
.with_deletion_reference_checker(Some(Arc::new(StaticReferences(vec!["bucket:sse-bucket".to_string()]))));
|
||||
let key_id = create_named_key(&manager, "referenced-force-delete").await;
|
||||
let probe = data_key_probe(&manager, &key_id).await;
|
||||
|
||||
// Server opt-in granted and the confirmation exact: the reference is
|
||||
// the only thing left to refuse this.
|
||||
let error = manager
|
||||
.delete_key(DeleteKeyRequest {
|
||||
key_id: key_id.clone(),
|
||||
force_immediate: Some(true),
|
||||
confirm_key_id: Some(key_id.clone()),
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect_err("immediate deletion must be refused while configuration references the key");
|
||||
|
||||
match error {
|
||||
KmsError::KeyStillReferenced {
|
||||
key_id: refused,
|
||||
references,
|
||||
} => {
|
||||
assert_eq!(refused, key_id);
|
||||
assert_eq!(references, vec!["bucket:sse-bucket".to_string()], "the caller must learn what refused it");
|
||||
}
|
||||
other => panic!("expected KeyStillReferenced, got {other:?}"),
|
||||
}
|
||||
|
||||
assert_key_material_intact(&manager, &key_id, &probe).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn immediate_deletion_of_the_service_default_key_is_refused() {
|
||||
let temp_dir = tempdir().expect("Failed to create temp dir");
|
||||
let mut config = KmsConfig::local(temp_dir.path().to_path_buf()).with_insecure_development_defaults();
|
||||
config.allow_immediate_deletion = true;
|
||||
let backend = Arc::new(LocalKmsBackend::new(config.clone()).await.expect("Failed to create backend"));
|
||||
let key_id = KmsManager::new(backend.clone(), config.clone())
|
||||
.create_key(CreateKeyRequest {
|
||||
key_name: Some("default-key-force-delete".to_string()),
|
||||
key_usage: KeyUsage::EncryptDecrypt,
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect("Failed to create key")
|
||||
.key_id;
|
||||
|
||||
config.default_key_id = Some(key_id.clone());
|
||||
let manager = KmsManager::new(backend, config);
|
||||
let probe = data_key_probe(&manager, &key_id).await;
|
||||
|
||||
let error = manager
|
||||
.delete_key(DeleteKeyRequest {
|
||||
key_id: key_id.clone(),
|
||||
force_immediate: Some(true),
|
||||
confirm_key_id: Some(key_id.clone()),
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect_err("the service default key must not be destroyed out from under the deployment");
|
||||
assert!(
|
||||
matches!(error, KmsError::KeyStillReferenced { .. }),
|
||||
"expected KeyStillReferenced, got {error:?}"
|
||||
);
|
||||
|
||||
assert_key_material_intact(&manager, &key_id, &probe).await;
|
||||
}
|
||||
|
||||
/// A checker that reports nothing must not become a shortcut around the
|
||||
/// gates that were already there: it is not a clearance, only the absence
|
||||
/// of one more objection.
|
||||
#[tokio::test]
|
||||
async fn an_empty_reference_set_grants_no_deletion_on_its_own() {
|
||||
let temp_dir = tempdir().expect("Failed to create temp dir");
|
||||
let manager = deletion_manager(&temp_dir, false)
|
||||
.await
|
||||
.with_deletion_reference_checker(Some(Arc::new(StaticReferences(Vec::new()))));
|
||||
let key_id = create_named_key(&manager, "unreferenced-force-delete").await;
|
||||
let probe = data_key_probe(&manager, &key_id).await;
|
||||
|
||||
// Server opt-in withheld, then confirmation missing: both still refuse.
|
||||
let error = manager
|
||||
.delete_key(DeleteKeyRequest {
|
||||
key_id: key_id.clone(),
|
||||
force_immediate: Some(true),
|
||||
confirm_key_id: Some(key_id.clone()),
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect_err("an unreferenced key still needs the server-side opt-in");
|
||||
assert!(
|
||||
matches!(error, KmsError::InvalidOperation { .. }),
|
||||
"expected InvalidOperation, got {error:?}"
|
||||
);
|
||||
|
||||
let allowed = deletion_manager(&temp_dir, true)
|
||||
.await
|
||||
.with_deletion_reference_checker(Some(Arc::new(StaticReferences(Vec::new()))));
|
||||
let error = allowed
|
||||
.delete_key(DeleteKeyRequest {
|
||||
key_id: key_id.clone(),
|
||||
force_immediate: Some(true),
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect_err("an unreferenced key still needs the key-id confirmation");
|
||||
assert!(
|
||||
matches!(error, KmsError::InvalidOperation { .. }),
|
||||
"expected InvalidOperation, got {error:?}"
|
||||
);
|
||||
|
||||
assert_key_material_intact(&manager, &key_id, &probe).await;
|
||||
}
|
||||
|
||||
/// The refusal is the only thing the checker adds: a fully authorized
|
||||
/// immediate deletion of a key nothing points at still goes through.
|
||||
#[tokio::test]
|
||||
async fn immediate_deletion_still_succeeds_when_nothing_references_the_key() {
|
||||
let temp_dir = tempdir().expect("Failed to create temp dir");
|
||||
let manager = deletion_manager(&temp_dir, true)
|
||||
.await
|
||||
.with_deletion_reference_checker(Some(Arc::new(StaticReferences(Vec::new()))));
|
||||
let key_id = create_named_key(&manager, "unreferenced-confirmed-force-delete").await;
|
||||
|
||||
manager
|
||||
.delete_key(DeleteKeyRequest {
|
||||
key_id: key_id.clone(),
|
||||
force_immediate: Some(true),
|
||||
confirm_key_id: Some(key_id.clone()),
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect("a confirmed immediate deletion must still be allowed when nothing references the key");
|
||||
|
||||
let error = manager
|
||||
.describe_key(DescribeKeyRequest { key_id: key_id.clone() })
|
||||
.await
|
||||
.expect_err("an immediately deleted key must be gone");
|
||||
assert!(matches!(error, KmsError::KeyNotFound { .. }), "expected KeyNotFound, got {error:?}");
|
||||
}
|
||||
|
||||
/// Scheduling stays a schedule: it destroys nothing, stays cancellable,
|
||||
/// and is re-checked against the same references by the deletion worker
|
||||
/// before any material goes away. Turning references into an up-front
|
||||
/// refusal here would let one unreadable bucket block routine operations.
|
||||
#[tokio::test]
|
||||
async fn scheduled_deletion_is_unaffected_by_references() {
|
||||
let temp_dir = tempdir().expect("Failed to create temp dir");
|
||||
let manager = deletion_manager(&temp_dir, false)
|
||||
.await
|
||||
.with_deletion_reference_checker(Some(Arc::new(StaticReferences(vec!["bucket:sse-bucket".to_string()]))));
|
||||
let key_id = create_named_key(&manager, "referenced-schedule").await;
|
||||
|
||||
manager
|
||||
.delete_key(DeleteKeyRequest {
|
||||
key_id: key_id.clone(),
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.expect("a scheduled deletion must still be accepted while configuration references the key");
|
||||
|
||||
let state = manager
|
||||
.describe_key(DescribeKeyRequest { key_id: key_id.clone() })
|
||||
.await
|
||||
.expect("a scheduled key must still be describable")
|
||||
.key_metadata
|
||||
.key_state;
|
||||
assert_eq!(state, KeyState::PendingDeletion);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn pending_window_outside_the_supported_range_is_refused() {
|
||||
let temp_dir = tempdir().expect("Failed to create temp dir");
|
||||
|
||||
@@ -634,7 +634,13 @@ impl KmsServiceManager {
|
||||
};
|
||||
|
||||
// Create KMS manager
|
||||
let mut kms_manager = KmsManager::new(backend, config.clone());
|
||||
//
|
||||
// The deletion reference checker is handed to the manager as well as to
|
||||
// the worker: immediate deletion destroys material without ever
|
||||
// reaching the worker, so that path has to consult the same gate
|
||||
// itself.
|
||||
let mut kms_manager =
|
||||
KmsManager::new(backend, config.clone()).with_deletion_reference_checker(self.deletion_reference_checker());
|
||||
if let Some(sink) = self.audit_sink() {
|
||||
kms_manager = kms_manager.with_audit_sink(sink);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user