mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-11 07:36:53 +00:00
Zhengchao An
db8039dece
* feat(kms): export local backend key material as sealed backup bundles Adds the producer side of the Local backup series on top of the #5483 contract: a directory-wide export fence gives the snapshot a single consistent generation, every artifact is AEAD-wrapped under a caller-supplied backup KEK that is separate from the business trust hierarchy, and the sealed manifest with completeness marker is written last so an interrupted export can never be mistaken for a restorable bundle. Restore and the admin API land in follow-up changes. * fix(kms): verify manifest digest against raw bytes, not re-serialized fields The decode path recomputed the digest by re-serializing the parsed manifest, which silently assumes every field's stored spelling survives a parse-and-reprint round trip. Timestamps do not guarantee that: the time zone annotation jiff emits depends on the host (IANA name, POSIX TZ string, Etc/Unknown), and the legacy-compat parser rewrites bracket-less spellings to +00:00[UTC]. On CI this made freshly written bundles fail digest verification while passing locally. Digest verification now operates on the raw stored bytes, normalized only through the JSON value layer with the digest slot emptied in place; parsed typed fields are never re-serialized on the decode path. Sealing uses the same value-layer canonical form, and the export additionally pins created_at to UTC so bundles are host-independent. A regression test seals a manifest whose created_at spelling cannot round-trip and proves decoding still verifies. One behavior sharpens: inserting an explicit null reserved slot after sealing is now rejected as a digest mismatch instead of being tolerated. * fix(kms): make manifest digest canonicalization independent of map ordering The canonical digest form serialized serde_json values directly, which inherits the key order of serde_json's map type: sorted by default, but insertion-ordered when any crate in the unified build graph enables the preserve_order feature. The workspace-wide CI build unified that feature while a per-crate local build did not, so the frozen fixture digest matched in one environment and not the other — and a bundle sealed by one build flavor would fail verification in the other. Canonicalization now rebuilds every JSON object with bytewise-sorted keys (array order preserved) before hashing, so the digest bytes are identical regardless of feature unification. Reproduced by enabling preserve_order in dev-dependencies (fixture test red), then verified green with the fix under both map flavors.
69 lines
3.0 KiB
Rust
69 lines
3.0 KiB
Rust
// Copyright 2024 RustFS Team
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
//! Backup/restore contracts and backup production for KMS state.
|
|
//!
|
|
//! The contract side defines the versioned backup manifest, the per-backend
|
|
//! responsibility matrix, typed failure modes, and the restore dry-run
|
|
//! report. [`local_export`] implements the producer side for the Local
|
|
//! backend as a crate-internal API; restore orchestration and the admin API
|
|
//! build on these pieces in follow-up changes.
|
|
//!
|
|
//! # Bundle model
|
|
//!
|
|
//! A backup bundle is a set of AEAD-encrypted artifacts described by a single
|
|
//! [`BackupManifest`]. All state in a bundle belongs to one snapshot
|
|
//! generation — there is no partially consistent bundle. The bundle is
|
|
//! protected by a backup KEK that is deliberately outside the business KMS
|
|
//! trust hierarchy, and the manifest is sealed with a completeness marker and
|
|
//! a final digest; a bundle that never reached its marker is permanently
|
|
//! non-restorable.
|
|
//!
|
|
//! # Restore ordering
|
|
//!
|
|
//! Restore implementations must follow this order: re-establish the external
|
|
//! trust root first (Vault/HSM native restore where one exists), then
|
|
//! material and version records into staging, then metadata and
|
|
//! configuration, then verification, and only then an explicit atomic
|
|
//! cutover. A dry-run ([`RestoreDryRunReport`]) performs zero writes.
|
|
//!
|
|
//! # Deliberately unfrozen
|
|
//!
|
|
//! Fields whose shape depends on contracts still in flight are reserved
|
|
//! rather than guessed (see [`ReservedSlot`]): the per-key version inventory
|
|
//! (backlog#1565) and capability discovery (backlog#1571). Alias and policy
|
|
//! artifacts are reserved names for features that do not exist yet. Reserved
|
|
//! slots reject data in format version 1 and become real types in a later
|
|
//! format version.
|
|
|
|
mod capability;
|
|
mod dry_run;
|
|
mod error;
|
|
pub mod local_export;
|
|
mod manifest;
|
|
|
|
pub use capability::{AtRestProtection, BackupBackendKind, BackupResponsibility};
|
|
pub use dry_run::{
|
|
ExternalDependencyMismatch, RestoreBlocker, RestoreBlockerCode, RestoreConflict, RestoreConflictKind, RestoreDryRunReport,
|
|
};
|
|
pub use error::BackupError;
|
|
pub use local_export::{
|
|
BackupKek, LOCAL_BUNDLE_MANIFEST_FILE, LocalBackupExportRequest, decrypt_bundle_artifact, export_local_backup,
|
|
read_local_bundle_manifest,
|
|
};
|
|
pub use manifest::{
|
|
AeadAlgorithm, ArtifactDescriptor, ArtifactKind, BackupKekDescriptor, BackupManifest, CompletenessState, ContentDigest,
|
|
DigestAlgorithm, LocalKdfDescriptor, LocalKeyDerivation, ReservedSlot,
|
|
};
|