feat(admin): add KMS key lifecycle endpoints and deletion reference gate (#5496)

* feat(kms): add key lifecycle operations to the backend contract

Add enable_key/disable_key/rotate_key to KmsBackend with conservative
defaults returning the typed UnsupportedCapability error, mirroring
remove_expired_key. KmsManager gains matching pass-through methods and
drops cached key metadata after every successful state mutation so the
next describe observes backend truth. The local backend overrides
enable/disable, delegating to its state-machine-gated client methods;
rotation stays rejected, matching its advertised capabilities. New
dedicated policy actions kms:EnableKey and kms:DisableKey complete the
KMS action taxonomy alongside the existing kms:RotateKey.

* feat(admin): add KMS key enable/disable/rotate endpoints

POST /v3/kms/keys/enable, /v3/kms/keys/disable and /v3/kms/keys/rotate,
following the existing /v3/kms/keys handler conventions: key_id body
with keyId query fallback, {success, message, key_id, key_metadata}
responses, and 503 JSON while the KMS service is absent. Error mapping
keeps InvalidOperation/ValidationError at 400 like the sibling handlers
and surfaces UnsupportedCapability as 501 so a backend capability gap is
never mistaken for a missing key. Existing /v3/kms/keys handlers are
untouched apart from a visibility change on a private query helper.

* feat(kms): gate scheduled key deletion on bucket encryption references

Implement the DeletionReferenceChecker seam left by the deletion worker:
before any material is destroyed, every bucket's SSE configuration is
checked for a default KMS key reference and a hit blocks the removal.
The gate fails closed - an unpublished object store, a failed bucket
listing or an unreadable per-bucket encryption config all report a
blocking reference - because destroying key material is irreversible
while a blocked removal is simply retried on the next sweep. Registered
during init_kms_system before the service can start, so every worker
spawn observes it. Storage access goes through a new kms section of the
root storage facade.
This commit is contained in:
Zhengchao An
2026-07-31 09:37:07 +08:00
committed by GitHub
parent cad8246ffb
commit a2fe5d7d88
14 changed files with 780 additions and 2 deletions
+8
View File
@@ -1533,6 +1533,14 @@ impl KmsBackend for LocalKmsBackend {
})
}
async fn enable_key(&self, key_id: &str) -> Result<()> {
self.client.enable_key(key_id, None).await
}
async fn disable_key(&self, key_id: &str) -> Result<()> {
self.client.disable_key(key_id, None).await
}
async fn health_check(&self) -> Result<bool> {
self.client.health_check().await.map(|_| true)
}
+43
View File
@@ -256,6 +256,34 @@ pub trait KmsBackend: Send + Sync {
/// Cancel key deletion
async fn cancel_key_deletion(&self, request: CancelKeyDeletionRequest) -> Result<CancelKeyDeletionResponse>;
/// Enable a disabled key so it can be used for cryptographic operations
/// again.
///
/// Backends that advertise [`BackendCapabilities::enable_disable`] must
/// override this method; the default rejects the operation.
async fn enable_key(&self, _key_id: &str) -> Result<()> {
Err(KmsError::unsupported_capability("backend without enable/disable support", "enable_key"))
}
/// Disable a key, rejecting new cryptographic use while existing data
/// remains decryptable.
///
/// Backends that advertise [`BackendCapabilities::enable_disable`] must
/// override this method; the default rejects the operation.
async fn disable_key(&self, _key_id: &str) -> Result<()> {
Err(KmsError::unsupported_capability("backend without enable/disable support", "disable_key"))
}
/// Rotate a key to a new version while prior versions remain available
/// for decryption.
///
/// Only backends that advertise [`BackendCapabilities::rotate`] (that is,
/// backends with retained version history) may override this method; the
/// default rejects the operation.
async fn rotate_key(&self, _key_id: &str) -> Result<()> {
Err(KmsError::unsupported_capability("backend without rotation support", "rotate_key"))
}
/// Health check
async fn health_check(&self) -> Result<bool>;
@@ -523,6 +551,21 @@ mod tests {
assert!(!capabilities.physical_delete);
}
#[tokio::test]
async fn default_lifecycle_operations_are_unsupported() {
for (operation, result) in [
("enable_key", MinimalBackend.enable_key("any-key").await),
("disable_key", MinimalBackend.disable_key("any-key").await),
("rotate_key", MinimalBackend.rotate_key("any-key").await),
] {
let error = result.expect_err("backends must opt in to lifecycle operations by overriding them");
assert!(
matches!(error, KmsError::UnsupportedCapability { .. }),
"expected UnsupportedCapability for {operation}, got {error:?}"
);
}
}
#[tokio::test]
async fn default_remove_expired_key_is_unsupported() {
let error = MinimalBackend
+76
View File
@@ -155,6 +155,36 @@ impl KmsManager {
Ok(response)
}
/// Enable a disabled key
pub async fn enable_key(&self, key_id: &str) -> Result<()> {
self.backend.enable_key(key_id).await?;
self.invalidate_cached_metadata(key_id).await;
Ok(())
}
/// Disable a key; existing data remains decryptable
pub async fn disable_key(&self, key_id: &str) -> Result<()> {
self.backend.disable_key(key_id).await?;
self.invalidate_cached_metadata(key_id).await;
Ok(())
}
/// Rotate a key to a new version
pub async fn rotate_key(&self, key_id: &str) -> Result<()> {
self.backend.rotate_key(key_id).await?;
self.invalidate_cached_metadata(key_id).await;
Ok(())
}
/// Drop cached metadata after a state mutation so the next describe
/// observes backend truth instead of the pre-mutation snapshot.
async fn invalidate_cached_metadata(&self, key_id: &str) {
if self.enable_cache {
let mut cache = self.cache.write().await;
cache.remove_key_metadata(key_id).await;
}
}
/// Perform health check on the KMS backend
pub async fn health_check(&self) -> Result<bool> {
self.backend.health_check().await
@@ -230,6 +260,52 @@ mod tests {
assert!(health);
}
#[tokio::test]
async fn lifecycle_round_trip_invalidates_cached_metadata() {
let temp_dir = tempdir().expect("Failed to create temp dir");
let config = KmsConfig::local(temp_dir.path().to_path_buf()).with_insecure_development_defaults();
let backend = Arc::new(LocalKmsBackend::new(config.clone()).await.expect("Failed to create backend"));
let manager = KmsManager::new(backend, config);
let key_id = manager
.create_key(CreateKeyRequest {
key_name: Some("lifecycle-round-trip".to_string()),
..Default::default()
})
.await
.expect("Failed to create key")
.key_id;
let describe = |key_id: String| {
let manager = manager.clone();
async move {
manager
.describe_key(DescribeKeyRequest { key_id })
.await
.expect("describe should succeed")
.key_metadata
.key_state
}
};
// Warm the metadata cache, then flip states; each describe must see
// the post-mutation state, proving the cache entry was dropped.
assert_eq!(describe(key_id.clone()).await, KeyState::Enabled);
manager.disable_key(&key_id).await.expect("disable should succeed");
assert_eq!(describe(key_id.clone()).await, KeyState::Disabled);
manager.enable_key(&key_id).await.expect("enable should succeed");
assert_eq!(describe(key_id.clone()).await, KeyState::Enabled);
// The local backend does not retain version history, so rotation is
// reported as a capability gap rather than a missing key.
let error = manager.rotate_key(&key_id).await.expect_err("local rotate must be rejected");
assert!(
matches!(error, crate::error::KmsError::UnsupportedCapability { .. }),
"expected UnsupportedCapability, got {error:?}"
);
}
#[tokio::test]
async fn generate_data_key_does_not_reuse_context_bound_ciphertext() {
let temp_dir = tempdir().expect("Failed to create temp dir");
+6
View File
@@ -718,6 +718,10 @@ pub enum KmsAction {
GenerateDataKeyAction,
#[strum(serialize = "kms:DeleteKey")]
DeleteKeyAction,
#[strum(serialize = "kms:EnableKey")]
EnableKeyAction,
#[strum(serialize = "kms:DisableKey")]
DisableKeyAction,
#[strum(serialize = "kms:RotateKey")]
RotateKeyAction,
#[strum(serialize = "kms:ListKeys")]
@@ -755,6 +759,8 @@ mod tests {
("kms:ClearCache", KmsAction::ClearCacheAction),
("kms:GenerateDataKey", KmsAction::GenerateDataKeyAction),
("kms:DeleteKey", KmsAction::DeleteKeyAction),
("kms:EnableKey", KmsAction::EnableKeyAction),
("kms:DisableKey", KmsAction::DisableKeyAction),
("kms:RotateKey", KmsAction::RotateKeyAction),
("kms:ListKeys", KmsAction::ListKeysAction),
("kms:DescribeKey", KmsAction::DescribeKeyAction),
+2 -1
View File
@@ -14,12 +14,13 @@
//! KMS admin handlers for HTTP API
use super::{kms_dynamic, kms_keys, kms_management};
use super::{kms_dynamic, kms_key_lifecycle, kms_keys, kms_management};
use crate::admin::router::{AdminOperation, S3Router};
pub fn register_kms_route(r: &mut S3Router<AdminOperation>) -> std::io::Result<()> {
kms_management::register_kms_management_route(r)?;
kms_dynamic::register_kms_dynamic_route(r)?;
kms_keys::register_kms_key_route(r)?;
kms_key_lifecycle::register_kms_key_lifecycle_route(r)?;
Ok(())
}
@@ -0,0 +1,456 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! KMS key lifecycle admin API handlers: enable, disable and rotate.
use super::kms_keys::extract_query_params;
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::current_kms_runtime_service_manager;
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use hyper::{HeaderMap, Method, StatusCode};
use matchit::Params;
use rustfs_config::MAX_ADMIN_REQUEST_BODY_SIZE;
use rustfs_kms::{
KmsError, KmsManager,
types::{DescribeKeyRequest, KeyMetadata},
};
use rustfs_policy::policy::action::{Action, KmsAction};
use s3s::header::CONTENT_TYPE;
use s3s::{Body, S3Request, S3Response, S3Result, s3_error};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
use tracing::{error, info, warn};
const LOG_COMPONENT_ADMIN: &str = "admin";
const LOG_SUBSYSTEM_KMS_KEYS: &str = "kms_key_lifecycle";
const EVENT_ADMIN_KMS_KEYS_STATE: &str = "admin_kms_keys_state";
#[derive(Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct KmsKeyLifecycleRequest {
pub key_id: String,
}
#[derive(Debug, Serialize, Deserialize)]
pub struct KmsKeyLifecycleResponse {
pub success: bool,
pub message: String,
pub key_id: String,
pub key_metadata: Option<KeyMetadata>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum LifecycleOperation {
Enable,
Disable,
Rotate,
}
impl LifecycleOperation {
fn action(self) -> &'static str {
match self {
Self::Enable => "enable_key",
Self::Disable => "disable_key",
Self::Rotate => "rotate_key",
}
}
fn verb(self) -> &'static str {
match self {
Self::Enable => "enable",
Self::Disable => "disable",
Self::Rotate => "rotate",
}
}
fn past_tense(self) -> &'static str {
match self {
Self::Enable => "enabled",
Self::Disable => "disabled",
Self::Rotate => "rotated",
}
}
}
fn kms_enable_key_actions() -> Vec<Action> {
vec![Action::KmsAction(KmsAction::EnableKeyAction)]
}
fn kms_disable_key_actions() -> Vec<Action> {
vec![Action::KmsAction(KmsAction::DisableKeyAction)]
}
fn kms_rotate_key_actions() -> Vec<Action> {
vec![Action::KmsAction(KmsAction::RotateKeyAction)]
}
pub fn register_kms_key_lifecycle_route(r: &mut S3Router<AdminOperation>) -> std::io::Result<()> {
r.insert(
Method::POST,
format!("{}{}", ADMIN_PREFIX, "/v3/kms/keys/enable").as_str(),
AdminOperation(&EnableKmsKeyHandler {}),
)?;
r.insert(
Method::POST,
format!("{}{}", ADMIN_PREFIX, "/v3/kms/keys/disable").as_str(),
AdminOperation(&DisableKmsKeyHandler {}),
)?;
r.insert(
Method::POST,
format!("{}{}", ADMIN_PREFIX, "/v3/kms/keys/rotate").as_str(),
AdminOperation(&RotateKmsKeyHandler {}),
)?;
Ok(())
}
/// Map a lifecycle failure to an HTTP status.
///
/// `UnsupportedCapability` is a permanent gap of the configured backend, not a
/// missing resource: it must surface as 501, never 404. State-machine
/// rejections (`InvalidOperation`) keep the 400 mapping used by the other
/// `/v3/kms/keys` handlers.
fn lifecycle_error_status(error: &KmsError) -> StatusCode {
match error {
KmsError::KeyNotFound { .. } => StatusCode::NOT_FOUND,
KmsError::UnsupportedCapability { .. } => StatusCode::NOT_IMPLEMENTED,
KmsError::InvalidOperation { .. } | KmsError::ValidationError { .. } => StatusCode::BAD_REQUEST,
// Damaged or missing key material is an integrity fault of an existing
// key: it must surface as a server error, never as NOT_FOUND (the key
// exists) and never as a retryable backend outage.
KmsError::MaterialMissing { .. }
| KmsError::MaterialCorrupt { .. }
| KmsError::MaterialAuthenticationFailed { .. }
| KmsError::UnsupportedFormatVersion { .. } => StatusCode::INTERNAL_SERVER_ERROR,
_ => StatusCode::INTERNAL_SERVER_ERROR,
}
}
/// Run one lifecycle operation against the manager and build the HTTP reply.
/// Split from the handlers so the status/response contract is unit-testable
/// without the admin auth stack.
async fn execute_lifecycle(
manager: &KmsManager,
key_id: &str,
operation: LifecycleOperation,
) -> (StatusCode, KmsKeyLifecycleResponse) {
let result = match operation {
LifecycleOperation::Enable => manager.enable_key(key_id).await,
LifecycleOperation::Disable => manager.disable_key(key_id).await,
LifecycleOperation::Rotate => manager.rotate_key(key_id).await,
};
match result {
Ok(()) => {
info!(
event = EVENT_ADMIN_KMS_KEYS_STATE,
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_KMS_KEYS,
action = operation.action(),
key_id = %key_id,
state = "completed",
"admin kms keys state"
);
// Best effort: the state change already succeeded, so a failing
// describe only drops the metadata echo from the response.
let key_metadata = match manager
.describe_key(DescribeKeyRequest {
key_id: key_id.to_string(),
})
.await
{
Ok(response) => Some(response.key_metadata),
Err(error) => {
warn!(
event = EVENT_ADMIN_KMS_KEYS_STATE,
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_KMS_KEYS,
action = operation.action(),
key_id = %key_id,
error = %error,
"key lifecycle change succeeded but describe failed"
);
None
}
};
(
StatusCode::OK,
KmsKeyLifecycleResponse {
success: true,
message: format!("key {} successfully", operation.past_tense()),
key_id: key_id.to_string(),
key_metadata,
},
)
}
Err(error) => {
error!(
event = EVENT_ADMIN_KMS_KEYS_STATE,
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_KMS_KEYS,
action = operation.action(),
key_id = %key_id,
result = "failed",
error = %error,
"admin kms keys state"
);
(
lifecycle_error_status(&error),
KmsKeyLifecycleResponse {
success: false,
message: format!("Failed to {} key: {error}", operation.verb()),
key_id: key_id.to_string(),
key_metadata: None,
},
)
}
}
}
fn json_response(status: StatusCode, response: &KmsKeyLifecycleResponse) -> S3Result<S3Response<(StatusCode, Body)>> {
let data = serde_json::to_vec(response).map_err(|e| s3_error!(InternalError, "failed to serialize response: {}", e))?;
let mut headers = HeaderMap::new();
headers.insert(CONTENT_TYPE, "application/json".parse().expect("static content type should parse"));
Ok(S3Response::with_headers((status, Body::from(data)), headers))
}
fn unavailable_response(message: &str, key_id: String) -> S3Result<S3Response<(StatusCode, Body)>> {
json_response(
StatusCode::SERVICE_UNAVAILABLE,
&KmsKeyLifecycleResponse {
success: false,
message: message.to_string(),
key_id,
key_metadata: None,
},
)
}
async fn handle_lifecycle_request(
mut req: S3Request<Body>,
actions: Vec<Action>,
operation: LifecycleOperation,
) -> S3Result<S3Response<(StatusCode, Body)>> {
let Some(cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "authentication required"));
};
let (cred, owner) = check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
validate_admin_request(
&req.headers,
&cred,
owner,
false,
actions,
req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
)
.await?;
let body = req
.input
.store_all_limited(MAX_ADMIN_REQUEST_BODY_SIZE)
.await
.map_err(|e| s3_error!(InvalidRequest, "failed to read request body: {}", e))?;
let request: KmsKeyLifecycleRequest = if body.is_empty() {
let query_params = extract_query_params(&req.uri);
let Some(key_id) = query_params.get("keyId") else {
return json_response(
StatusCode::BAD_REQUEST,
&KmsKeyLifecycleResponse {
success: false,
message: "missing required parameter: 'keyId'".to_string(),
key_id: "".to_string(),
key_metadata: None,
},
);
};
KmsKeyLifecycleRequest { key_id: key_id.clone() }
} else {
serde_json::from_slice(&body).map_err(|e| s3_error!(InvalidRequest, "invalid JSON: {}", e))?
};
let Some(service_manager) = kms_service_manager_from_context() else {
return unavailable_response("kms service manager is not initialized", request.key_id);
};
let Some(manager) = service_manager.get_manager().await else {
return unavailable_response("kms service is not running", request.key_id);
};
let (status, response) = execute_lifecycle(&manager, &request.key_id, operation).await;
json_response(status, &response)
}
fn kms_service_manager_from_context() -> Option<Arc<rustfs_kms::KmsServiceManager>> {
current_kms_runtime_service_manager()
}
/// Enable a disabled KMS key
pub struct EnableKmsKeyHandler;
#[async_trait::async_trait]
impl Operation for EnableKmsKeyHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
handle_lifecycle_request(req, kms_enable_key_actions(), LifecycleOperation::Enable).await
}
}
/// Disable a KMS key; existing data remains decryptable
pub struct DisableKmsKeyHandler;
#[async_trait::async_trait]
impl Operation for DisableKmsKeyHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
handle_lifecycle_request(req, kms_disable_key_actions(), LifecycleOperation::Disable).await
}
}
/// Rotate a KMS key to a new version
pub struct RotateKmsKeyHandler;
#[async_trait::async_trait]
impl Operation for RotateKmsKeyHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
handle_lifecycle_request(req, kms_rotate_key_actions(), LifecycleOperation::Rotate).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use rustfs_kms::backends::local::LocalKmsBackend;
use rustfs_kms::config::KmsConfig;
use rustfs_kms::types::{CreateKeyRequest, KeyState};
use rustfs_policy::policy::action::AdminAction;
async fn local_manager(temp_dir: &tempfile::TempDir) -> KmsManager {
let config = KmsConfig::local(temp_dir.path().to_path_buf()).with_insecure_development_defaults();
let backend = Arc::new(
LocalKmsBackend::new(config.clone())
.await
.expect("local backend should build"),
);
KmsManager::new(backend, config)
}
async fn create_key(manager: &KmsManager, key_name: &str) -> String {
manager
.create_key(CreateKeyRequest {
key_name: Some(key_name.to_string()),
..Default::default()
})
.await
.expect("key should be created")
.key_id
}
fn key_state(response: &KmsKeyLifecycleResponse) -> KeyState {
response
.key_metadata
.as_ref()
.expect("lifecycle response should echo key metadata")
.key_state
.clone()
}
#[tokio::test]
async fn enable_disable_enable_round_trip() {
let temp_dir = tempfile::tempdir().expect("temp dir");
let manager = local_manager(&temp_dir).await;
let key_id = create_key(&manager, "lifecycle-round-trip").await;
let (status, response) = execute_lifecycle(&manager, &key_id, LifecycleOperation::Disable).await;
assert_eq!(status, StatusCode::OK);
assert!(response.success);
assert_eq!(key_state(&response), KeyState::Disabled);
let (status, response) = execute_lifecycle(&manager, &key_id, LifecycleOperation::Enable).await;
assert_eq!(status, StatusCode::OK);
assert!(response.success);
assert_eq!(key_state(&response), KeyState::Enabled);
// Enabling an already enabled key stays idempotent.
let (status, response) = execute_lifecycle(&manager, &key_id, LifecycleOperation::Enable).await;
assert_eq!(status, StatusCode::OK);
assert_eq!(key_state(&response), KeyState::Enabled);
}
#[tokio::test]
async fn local_rotate_is_unsupported_capability_not_missing_key() {
let temp_dir = tempfile::tempdir().expect("temp dir");
let manager = local_manager(&temp_dir).await;
let key_id = create_key(&manager, "rotate-unsupported").await;
let (status, response) = execute_lifecycle(&manager, &key_id, LifecycleOperation::Rotate).await;
assert_eq!(status, StatusCode::NOT_IMPLEMENTED);
assert_ne!(status, StatusCode::NOT_FOUND);
assert!(!response.success);
assert!(
response.message.contains("not supported"),
"message should name the capability gap: {}",
response.message
);
// A disabled key must not change the picture: rotation stays rejected.
let (status, _) = execute_lifecycle(&manager, &key_id, LifecycleOperation::Disable).await;
assert_eq!(status, StatusCode::OK);
let (status, response) = execute_lifecycle(&manager, &key_id, LifecycleOperation::Rotate).await;
assert_ne!(status, StatusCode::OK);
assert!(!response.success);
}
#[tokio::test]
async fn lifecycle_on_missing_key_maps_to_not_found() {
let temp_dir = tempfile::tempdir().expect("temp dir");
let manager = local_manager(&temp_dir).await;
let (status, response) = execute_lifecycle(&manager, "no-such-key", LifecycleOperation::Enable).await;
assert_eq!(status, StatusCode::NOT_FOUND);
assert!(!response.success);
}
#[test]
fn lifecycle_error_status_contract() {
let unsupported = KmsError::unsupported_capability("local", "rotate_key");
assert_eq!(lifecycle_error_status(&unsupported), StatusCode::NOT_IMPLEMENTED);
assert_ne!(lifecycle_error_status(&unsupported), StatusCode::NOT_FOUND);
assert_eq!(lifecycle_error_status(&KmsError::invalid_key_state("disabled")), StatusCode::BAD_REQUEST);
assert_eq!(lifecycle_error_status(&KmsError::key_not_found("gone")), StatusCode::NOT_FOUND);
}
#[test]
fn lifecycle_auth_actions_use_dedicated_kms_actions() {
assert_eq!(kms_enable_key_actions(), vec![Action::KmsAction(KmsAction::EnableKeyAction)]);
assert_eq!(kms_disable_key_actions(), vec![Action::KmsAction(KmsAction::DisableKeyAction)]);
assert_eq!(kms_rotate_key_actions(), vec![Action::KmsAction(KmsAction::RotateKeyAction)]);
for actions in [kms_enable_key_actions(), kms_disable_key_actions(), kms_rotate_key_actions()] {
assert!(!actions.contains(&Action::AdminAction(AdminAction::ServerInfoAdminAction)));
}
}
#[test]
fn lifecycle_request_rejects_unknown_fields() {
let error = serde_json::from_str::<KmsKeyLifecycleRequest>(r#"{"key_id":"key","unexpected_field":true}"#)
.expect_err("unknown lifecycle request field should fail");
assert!(error.to_string().contains("unknown field"));
}
}
+1 -1
View File
@@ -94,7 +94,7 @@ pub struct GenerateDataKeyApiResponse {
pub ciphertext_blob: String, // Base64 encoded
}
fn extract_query_params(uri: &hyper::Uri) -> HashMap<String, String> {
pub(super) fn extract_query_params(uri: &hyper::Uri) -> HashMap<String, String> {
let mut params = HashMap::new();
if let Some(query) = uri.query() {
query.split('&').for_each(|pair| {
+1
View File
@@ -33,6 +33,7 @@ pub mod inspect_archive;
pub mod is_admin;
pub mod kms;
pub mod kms_dynamic;
pub mod kms_key_lifecycle;
pub mod kms_keys;
pub mod kms_management;
pub mod metrics;
+11
View File
@@ -58,8 +58,11 @@ const KMS_CLEAR_CACHE: AdminActionRef = AdminActionRef::new("kms:ClearCache");
const KMS_CONFIGURE: AdminActionRef = AdminActionRef::new("kms:Configure");
const KMS_DELETE_KEY: AdminActionRef = AdminActionRef::new("kms:DeleteKey");
const KMS_DESCRIBE_KEY: AdminActionRef = AdminActionRef::new("kms:DescribeKey");
const KMS_DISABLE_KEY: AdminActionRef = AdminActionRef::new("kms:DisableKey");
const KMS_ENABLE_KEY: AdminActionRef = AdminActionRef::new("kms:EnableKey");
const KMS_GENERATE_DATA_KEY: AdminActionRef = AdminActionRef::new("kms:GenerateDataKey");
const KMS_LIST_KEYS: AdminActionRef = AdminActionRef::new("kms:ListKeys");
const KMS_ROTATE_KEY: AdminActionRef = AdminActionRef::new("kms:RotateKey");
const KMS_SERVICE_CONTROL: AdminActionRef = AdminActionRef::new("kms:ServiceControl");
const LIST_GROUPS: AdminActionRef = AdminActionRef::new("ListGroupsAdminAction");
const LIST_TEMPORARY_ACCOUNTS: AdminActionRef = AdminActionRef::new("ListTemporaryAccountsAdminAction");
@@ -777,6 +780,14 @@ pub const ADMIN_ROUTE_POLICY_SPECS: &[AdminRouteSpec] = &[
KMS_DESCRIBE_KEY,
RouteRiskLevel::Sensitive,
),
admin(HttpMethod::Post, "/rustfs/admin/v3/kms/keys/enable", KMS_ENABLE_KEY, RouteRiskLevel::High),
admin(
HttpMethod::Post,
"/rustfs/admin/v3/kms/keys/disable",
KMS_DISABLE_KEY,
RouteRiskLevel::High,
),
admin(HttpMethod::Post, "/rustfs/admin/v3/kms/keys/rotate", KMS_ROTATE_KEY, RouteRiskLevel::High),
public(
HttpMethod::Get,
"/rustfs/admin/v3/oidc/providers",
@@ -340,6 +340,9 @@ fn expected_admin_route_matrix() -> Vec<RouteMatrixEntry> {
admin_route(Method::POST, "/v3/kms/keys/cancel-deletion"),
admin_route(Method::GET, "/v3/kms/keys"),
admin_route_sample(Method::GET, "/v3/kms/keys/{key_id}", "/v3/kms/keys/test-key"),
admin_route(Method::POST, "/v3/kms/keys/enable"),
admin_route(Method::POST, "/v3/kms/keys/disable"),
admin_route(Method::POST, "/v3/kms/keys/rotate"),
admin_route(Method::GET, "/v3/oidc/providers"),
admin_route_sample(Method::GET, "/v3/oidc/authorize/{provider_id}", "/v3/oidc/authorize/default"),
admin_route_sample(Method::GET, "/v3/oidc/callback/{provider_id}", "/v3/oidc/callback/default"),
+7
View File
@@ -468,6 +468,13 @@ pub async fn init_kms_system(config: &config::Config) -> std::io::Result<()> {
// Initialize global KMS service manager (starts in NotConfigured state)
let service_manager = startup_runtime_sources::init_kms_service_manager();
// A key referenced by any bucket's encryption configuration must never be
// deleted. Register the gate before the service can start so every
// deletion-worker spawn observes it; the gate fails closed while the
// object store is not ready.
service_manager
.set_deletion_reference_checker(std::sync::Arc::new(crate::kms_deletion_gate::BucketEncryptionReferenceChecker));
// If KMS is enabled in configuration, configure and start the service
if config.kms_enable {
info!(
+147
View File
@@ -0,0 +1,147 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Reference gate consulted before a scheduled KMS key deletion destroys
//! material: a key that any bucket's encryption configuration still points at
//! must not be removed. Object-level references (existing envelopes) are out
//! of scope here; those objects stay decryptable only until the key is gone,
//! which is why the pending-deletion window exists.
use crate::runtime_sources::current_object_store_handle;
use crate::storage_api::kms::contract::bucket::{BucketOperations, BucketOptions};
use crate::storage_api::kms::{ECStore, StorageError, get_bucket_sse_config};
use async_trait::async_trait;
use rustfs_kms::DeletionReferenceChecker;
use s3s::dto::ServerSideEncryptionConfiguration;
use std::sync::Arc;
use tracing::warn;
/// Reference reported when bucket configuration cannot be inspected at all.
const BUCKET_CONFIG_UNAVAILABLE: &str = "bucket-encryption-config:unavailable";
/// Blocks deletion of keys referenced by any bucket's SSE configuration
/// (default bucket KMS key). Registered on the KMS service manager at startup
/// and consulted by the background deletion worker.
pub(crate) struct BucketEncryptionReferenceChecker;
#[async_trait]
impl DeletionReferenceChecker for BucketEncryptionReferenceChecker {
async fn references(&self, key_id: &str) -> Vec<String> {
collect_references(key_id, current_object_store_handle()).await
}
}
async fn collect_references(key_id: &str, store: Option<Arc<ECStore>>) -> Vec<String> {
// Fail closed: destroying key material is irreversible while the deletion
// worker retries every sweep, so an uninspectable configuration must block
// the removal rather than wave it through. This also covers the worker
// racing server startup, before the object store is published.
let Some(store) = store else {
warn!(key_id, "KMS deletion reference check: object store not ready; blocking removal");
return vec![BUCKET_CONFIG_UNAVAILABLE.to_string()];
};
let buckets = match store.list_bucket(&BucketOptions::default()).await {
Ok(buckets) => buckets,
Err(error) => {
warn!(key_id, %error, "KMS deletion reference check: listing buckets failed; blocking removal");
return vec![BUCKET_CONFIG_UNAVAILABLE.to_string()];
}
};
let mut references = Vec::new();
for bucket in &buckets {
let lookup = get_bucket_sse_config(&bucket.name).await;
references.extend(bucket_reference(&bucket.name, lookup, key_id));
}
references
}
/// `Some(reference)` when the bucket's encryption configuration references
/// `key_id` or cannot be read (fail closed); `None` when the bucket provably
/// does not reference the key.
fn bucket_reference(
bucket: &str,
lookup: Result<ServerSideEncryptionConfiguration, StorageError>,
key_id: &str,
) -> Option<String> {
match lookup {
Ok(config) if sse_config_references_key(&config, key_id) => Some(format!("bucket:{bucket}")),
Ok(_) => None,
Err(StorageError::ConfigNotFound) => None,
Err(error) => {
warn!(
bucket,
key_id,
%error,
"KMS deletion reference check: unreadable bucket encryption config; blocking removal"
);
Some(format!("bucket:{bucket}:encryption-config-unreadable"))
}
}
}
fn sse_config_references_key(config: &ServerSideEncryptionConfiguration, key_id: &str) -> bool {
config.rules.iter().any(|rule| {
rule.apply_server_side_encryption_by_default
.as_ref()
.and_then(|sse| sse.kms_master_key_id.as_deref())
.is_some_and(|configured| configured == key_id)
})
}
#[cfg(test)]
mod tests {
use super::*;
use s3s::dto::{ServerSideEncryption, ServerSideEncryptionByDefault, ServerSideEncryptionRule};
fn sse_kms_config(key_id: Option<&str>) -> ServerSideEncryptionConfiguration {
ServerSideEncryptionConfiguration {
rules: vec![ServerSideEncryptionRule {
apply_server_side_encryption_by_default: Some(ServerSideEncryptionByDefault {
sse_algorithm: ServerSideEncryption::from_static(ServerSideEncryption::AWS_KMS),
kms_master_key_id: key_id.map(str::to_string),
}),
bucket_key_enabled: None,
}],
}
}
#[test]
fn referencing_bucket_blocks_deletion() {
assert_eq!(
bucket_reference("sse-bucket", Ok(sse_kms_config(Some("kms-key-1"))), "kms-key-1"),
Some("bucket:sse-bucket".to_string())
);
}
#[test]
fn non_referencing_buckets_allow_deletion() {
// Different key, no configured key, and no SSE configuration at all.
assert_eq!(bucket_reference("other-key", Ok(sse_kms_config(Some("kms-key-2"))), "kms-key-1"), None);
assert_eq!(bucket_reference("no-key", Ok(sse_kms_config(None)), "kms-key-1"), None);
assert_eq!(bucket_reference("plain", Err(StorageError::ConfigNotFound), "kms-key-1"), None);
}
#[test]
fn unreadable_config_blocks_deletion() {
let reference = bucket_reference("broken", Err(StorageError::FaultyDisk), "kms-key-1");
assert_eq!(reference, Some("bucket:broken:encryption-config-unreadable".to_string()));
}
#[tokio::test]
async fn missing_object_store_blocks_deletion() {
let references = collect_references("kms-key-1", None).await;
assert_eq!(references, vec![BUCKET_CONFIG_UNAVAILABLE.to_string()]);
}
}
+1
View File
@@ -82,6 +82,7 @@ pub mod diagnose;
pub mod embedded;
pub mod error;
pub mod init;
pub(crate) mod kms_deletion_gate;
pub mod license;
pub mod memory_observability;
pub mod profiling;
+18
View File
@@ -72,6 +72,24 @@ pub(crate) mod error {
pub(crate) use crate::storage::storage_api::{QuotaError, StorageError};
}
pub(crate) mod kms {
pub(crate) mod contract {
pub(crate) mod bucket {
pub(crate) use super::super::super::storage_contracts::{BucketOperations, BucketOptions};
}
}
pub(crate) use crate::storage::storage_api::{ECStore, StorageError};
/// Bucket SSE configuration for the KMS deletion reference gate;
/// `Err(StorageError::ConfigNotFound)` when the bucket has none.
pub(crate) async fn get_bucket_sse_config(bucket: &str) -> Result<s3s::dto::ServerSideEncryptionConfiguration, StorageError> {
crate::storage::storage_api::ecstore_bucket::metadata_sys::get_sse_config(bucket)
.await
.map(|(config, _)| config)
}
}
pub(crate) mod protocols {
pub(crate) mod client {
pub(crate) use crate::storage::storage_api::access_consumer::ReqInfo;