mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-01 17:58:22 +00:00
fix(storage): resolve erasure parity per pool (#4977)
* fix(filemeta): add state-aware file info validation
* fix(filemeta): validate shard arithmetic and delete paths
* fix(ecstore): add fallible erasure construction
* fix(ecstore): resolve storage parity per pool
* fix(storage): report heterogeneous erasure layouts
* fix(admin): publish prepared storage config atomically
* fix(storage): harden per-pool parity boundaries
* fix(storage): address pre-PR validation findings
* test(ci): fix strict-topology validation fixtures
* fix(heal): preserve delete markers during repair
* refactor(filemeta): drop unused ValidatedFileInfo witness
ValidatedFileInfo wrapped an unread `_file_info` reference alongside an `Option<ValidatedErasureLayout>`, but only the layout was ever consumed. Return the layout directly from `FileInfo::validate` so the sole production consumer (`LocalDisk::check_parts`) and the two unit tests read it without the extra witness type and lifetime.
No behavior change.
* fix(filemeta): keep compressed and MinIO-migrated tiered objects readable
The new decode-path validation rejected several legitimate on-disk shapes that older RustFS and MinIO-migrated data carry, turning readable objects into FileCorrupt:
- Compressed objects written with an unknown upload size persist a negative per-part actual_size (the documented "unknown size" sentinel that ObjectInfo::get_actual_size already tolerates). validate_collection_contents rejected it via usize::try_from; now a negative actual_size skips shard validation and only real, non-negative sizes are checked.
- MinIO-migrated objects transitioned to a versioned remote tier store the tier version id as a UUID string, not 16 raw bytes. MetaObject::into_fileinfo returned FileCorrupt (main tolerated it as None), making all versions of the object unreadable; MetaDeleteMarker free-version records took a Some(nil) sentinel path with the same effect, which also breaks free-version expiry (remote-tier leak). Both now decode through a shared transitioned_version_id_from_meta_sys helper: 16 raw bytes or a UUID string are accepted, anything else is tolerated as None instead of failing the read.
Regression tests updated to assert the readable/compat behavior, with new tests covering MinIO string-form recovery.
* fix(scanner): build the delete-marker test fixture without erasure geometry
get_size_counts_delete_markers_separately_from_versions built its delete marker with `FileInfo::new(object, 1, 1)`, which attaches erasure geometry (data=1/parity=1/distribution). This PR classifies versions by shape via `is_storage_delete_marker()` (no geometry) rather than the raw `deleted` flag, so a geometry-bearing "delete marker" is correctly serialized as a purge-pending payload Object and counted as a version — CI saw summary.versions=3, expected 2.
Real delete markers carry no erasure geometry (delete paths build them as `FileInfo { deleted: true, ..Default::default() }`), so construct the fixture the same way. It then classifies as a storage delete marker and the counts (versions=2, delete_markers=1) hold. This keeps the PR's more-correct classification, which prevents a purge-pending object's geometry from being dropped when serialized as a bare delete marker.
* docs(changelog): note per-pool parity fix and storage-class startup upgrade caveat
Records the #4801 per-pool erasure parity fix under Fixed, and documents the upgrade behavior where a persisted storage class that a small or heterogeneous pool cannot satisfy now fails startup — with the RUSTFS_STORAGE_CLASS_STANDARD recovery steps. Docs-only; covers R4 from the on-disk compatibility audit.
* fix(heal): report parity from erasure geometry, not is_valid()
heal_object set HealResultItem.parity_blocks via `if lfi.is_valid()`, which was missed by the migration of the other quorum/metadata predicates. With the new `is_valid()` semantics (full payload validation; delete markers now return false), a delete marker or a geometry-bearing version with a benign collection quirk would misreport parity as the pool default instead of its own. Use `has_valid_erasure_geometry()` — the narrow "does this carry erasure geometry" predicate the rest of the migration uses — so reporting matches the object's actual layout. Reporting-only; no data-path change.
* fix(filemeta): do not silently serialize a non-canonical deleted FileInfo as an Object
`From<FileInfo> for FileMetaVersion` classifies by `is_storage_delete_marker()` (shape), which correctly routes canonical delete markers to Delete and purge-pending payloads (deleted=true with real erasure geometry) to Object. But a `deleted` FileInfo that is neither a canonical marker nor a valid erasure payload would silently serialize as a zero-geometry MetaObject that later fails `validate_for_metadata_read`. Write paths validate first (`validate_for_erasure_write` / `validate_for_metadata_read`), so this is a caller bug; `From` is infallible, so surface it with a structured `warn!` on the malformed branch instead of writing corrupt metadata silently. Legitimate purge-pending objects (valid geometry) are unaffected — the guard only fires for `deleted && !has_valid_erasure_geometry()`.
* test(filemeta): assert real historical xl.meta versions pass metadata-read validation
Empirical companion to the code-reasoned decode-tolerance invariants (docs/architecture/erasure-coding.md §11) and the rolling-upgrade / MinIO-migration compatibility concern: the tightened `validate_for_metadata_read` runs on every local disk read and peer-RPC-decoded FileInfo, so it must accept every version of real historically-written xl.meta, never reject it as FileCorrupt.
Loads five real fixtures — MinIO small-inline, MinIO versioned (two object versions + a delete marker), MinIO large multipart, a legacy V1 (xl.json-derived) object, and a legacy meta_ver 2 object — decodes every version with parts materialized, and asserts validate_for_metadata_read() is Ok for each. Reverting the tolerant handling (delete-marker shape, legacy per-part checksums, string/short transitioned-versionID, negative actual_size) turns this red.
* fix(ci): remove duplicate storage test re-exports
---------
Co-authored-by: overtrue <anzhengchao@gmail.com>
This commit is contained in:
cxymds
@@ -29,10 +29,10 @@ use super::*;
|
||||
use crate::ChecksumInfo;
|
||||
use rustfs_utils::HashAlgorithm;
|
||||
use rustfs_utils::http::{
|
||||
SUFFIX_CRC, SUFFIX_FREE_VERSION, SUFFIX_INLINE_DATA, SUFFIX_PURGESTATUS, SUFFIX_TIER_FV_ID, SUFFIX_TIER_FV_MARKER,
|
||||
SUFFIX_TRANSITION_STATUS, SUFFIX_TRANSITION_TIER, SUFFIX_TRANSITION_TIER_DESTINATION_ID, SUFFIX_TRANSITIONED_OBJECTNAME,
|
||||
SUFFIX_TRANSITIONED_VERSION_ID, contains_key_bytes, get_bytes, get_consistent_bytes, get_str, has_internal_suffix,
|
||||
insert_bytes, is_internal_key, remove_bytes, strip_internal_prefix,
|
||||
RUSTFS_INTERNAL_PREFIX, SUFFIX_CRC, SUFFIX_FREE_VERSION, SUFFIX_INLINE_DATA, SUFFIX_PURGESTATUS, SUFFIX_TIER_FV_ID,
|
||||
SUFFIX_TIER_FV_MARKER, SUFFIX_TRANSITION_STATUS, SUFFIX_TRANSITION_TIER, SUFFIX_TRANSITION_TIER_DESTINATION_ID,
|
||||
SUFFIX_TRANSITIONED_OBJECTNAME, SUFFIX_TRANSITIONED_VERSION_ID, contains_key_bytes, get_bytes, get_consistent_bytes, get_str,
|
||||
has_internal_suffix, insert_bytes, is_internal_key, remove_bytes, strip_internal_prefix,
|
||||
};
|
||||
|
||||
const MSGPACK_EXT8: u8 = 0xc7;
|
||||
@@ -249,6 +249,27 @@ fn parse_legacy_uuid_bytes(bytes: &[u8], field: &str) -> Result<Option<Uuid>> {
|
||||
Ok((!id.is_nil()).then_some(id))
|
||||
}
|
||||
|
||||
/// Decode a stored transitioned-version-id from a version's `meta_sys`.
|
||||
///
|
||||
/// RustFS writes it as 16 raw UUID bytes; MinIO-migrated tiered objects store
|
||||
/// the remote tier's version id as a UUID *string*. Accept both, and treat any
|
||||
/// absent / nil / otherwise-unparseable value as "no tier version" (matching the
|
||||
/// tolerant pre-hardening behavior) rather than failing the whole object read —
|
||||
/// a malformed tier id must not make an otherwise-readable object unreadable.
|
||||
fn transitioned_version_id_from_meta_sys(meta_sys: &HashMap<String, Vec<u8>>) -> Option<Uuid> {
|
||||
let value = get_bytes(meta_sys, SUFFIX_TRANSITIONED_VERSION_ID)?;
|
||||
if value.is_empty() {
|
||||
return None;
|
||||
}
|
||||
if let Ok(id) = Uuid::from_slice(&value) {
|
||||
return (!id.is_nil()).then_some(id);
|
||||
}
|
||||
std::str::from_utf8(&value)
|
||||
.ok()
|
||||
.and_then(|s| Uuid::parse_str(s.trim()).ok())
|
||||
.filter(|id| !id.is_nil())
|
||||
}
|
||||
|
||||
fn parse_legacy_erasure_algo(value: &str) -> ErasureAlgo {
|
||||
match value {
|
||||
"ReedSolomon" => ErasureAlgo::ReedSolomon,
|
||||
@@ -807,7 +828,7 @@ impl TryFrom<LegacyMetaV2Version> for FileMetaVersion {
|
||||
|
||||
impl From<FileInfo> for FileMetaVersion {
|
||||
fn from(value: FileInfo) -> Self {
|
||||
if value.deleted {
|
||||
if value.is_storage_delete_marker() {
|
||||
FileMetaVersion {
|
||||
version_type: VersionType::Delete,
|
||||
legacy_object: None,
|
||||
@@ -817,6 +838,22 @@ impl From<FileInfo> for FileMetaVersion {
|
||||
uses_legacy_checksum: false,
|
||||
}
|
||||
} else {
|
||||
// A `deleted` FileInfo that is not a canonical delete marker is only
|
||||
// legitimate as a purge-pending payload, which carries real erasure
|
||||
// geometry and is intentionally serialized as an Object. A `deleted`
|
||||
// FileInfo with neither a canonical-marker shape nor valid erasure
|
||||
// geometry would silently serialize as a zero-geometry MetaObject that
|
||||
// later fails `validate_for_metadata_read`. Write paths validate first
|
||||
// (`validate_for_erasure_write` / `validate_for_metadata_read`), so this
|
||||
// is a caller bug — surface it rather than writing malformed metadata
|
||||
// silently. (`From` is infallible, so this cannot return an error.)
|
||||
if value.deleted && !value.has_valid_erasure_geometry() {
|
||||
tracing::warn!(
|
||||
event = "filemeta_non_canonical_deleted_fileinfo_as_object",
|
||||
component = "filemeta",
|
||||
"serializing a deleted FileInfo that is neither a canonical delete marker nor a valid erasure payload as an Object version; upstream validation should have rejected it"
|
||||
);
|
||||
}
|
||||
FileMetaVersion {
|
||||
version_type: VersionType::Object,
|
||||
legacy_object: None,
|
||||
@@ -2361,9 +2398,7 @@ impl MetaObject {
|
||||
let transitioned_objname = get_bytes(&self.meta_sys, SUFFIX_TRANSITIONED_OBJECTNAME)
|
||||
.map(|v| String::from_utf8_lossy(&v).to_string())
|
||||
.unwrap_or_default();
|
||||
let transition_version_id = get_bytes(&self.meta_sys, SUFFIX_TRANSITIONED_VERSION_ID)
|
||||
.and_then(|v| Uuid::from_slice(v.as_slice()).ok())
|
||||
.filter(|u| !u.is_nil());
|
||||
let transition_version_id = transitioned_version_id_from_meta_sys(&self.meta_sys);
|
||||
let transition_tier = get_bytes(&self.meta_sys, SUFFIX_TRANSITION_TIER)
|
||||
.map(|v| String::from_utf8_lossy(&v).to_string())
|
||||
.unwrap_or_default();
|
||||
@@ -2671,9 +2706,7 @@ impl MetaDeleteMarker {
|
||||
.map(|v| String::from_utf8_lossy(&v).to_string())
|
||||
.unwrap_or_default();
|
||||
|
||||
fi.transition_version_id = get_bytes(&self.meta_sys, SUFFIX_TRANSITIONED_VERSION_ID)
|
||||
.and_then(|v| Uuid::from_slice(v.as_slice()).ok())
|
||||
.filter(|u| !u.is_nil());
|
||||
fi.transition_version_id = transitioned_version_id_from_meta_sys(&self.meta_sys);
|
||||
}
|
||||
|
||||
fi
|
||||
@@ -2788,10 +2821,54 @@ impl MetaDeleteMarker {
|
||||
|
||||
impl From<FileInfo> for MetaDeleteMarker {
|
||||
fn from(value: FileInfo) -> Self {
|
||||
let mut meta_sys = HashMap::new();
|
||||
let mut durable_metadata: HashMap<String, (&str, bool)> = HashMap::new();
|
||||
for (key, metadata_value) in &value.metadata {
|
||||
if !is_internal_key(key) || is_skip_meta_key(key) {
|
||||
continue;
|
||||
}
|
||||
let Some(suffix) = strip_internal_prefix(key) else {
|
||||
continue;
|
||||
};
|
||||
let rustfs_preferred = key
|
||||
.get(..RUSTFS_INTERNAL_PREFIX.len())
|
||||
.is_some_and(|prefix| prefix.eq_ignore_ascii_case(RUSTFS_INTERNAL_PREFIX));
|
||||
match durable_metadata.entry(suffix) {
|
||||
std::collections::hash_map::Entry::Vacant(entry) => {
|
||||
entry.insert((metadata_value, rustfs_preferred));
|
||||
}
|
||||
std::collections::hash_map::Entry::Occupied(mut entry) if rustfs_preferred && !entry.get().1 => {
|
||||
entry.insert((metadata_value, true));
|
||||
}
|
||||
std::collections::hash_map::Entry::Occupied(_) => {}
|
||||
}
|
||||
}
|
||||
for (suffix, (metadata_value, _)) in durable_metadata {
|
||||
insert_bytes(&mut meta_sys, &suffix, metadata_value.as_bytes().to_vec());
|
||||
}
|
||||
if value.tier_free_version() {
|
||||
insert_bytes(&mut meta_sys, SUFFIX_FREE_VERSION, vec![]);
|
||||
}
|
||||
if !value.transition_status.is_empty() {
|
||||
insert_bytes(&mut meta_sys, SUFFIX_TRANSITION_STATUS, value.transition_status.as_bytes().to_vec());
|
||||
}
|
||||
if !value.transitioned_objname.is_empty() {
|
||||
insert_bytes(
|
||||
&mut meta_sys,
|
||||
SUFFIX_TRANSITIONED_OBJECTNAME,
|
||||
value.transitioned_objname.as_bytes().to_vec(),
|
||||
);
|
||||
}
|
||||
if let Some(version_id) = value.transition_version_id {
|
||||
insert_bytes(&mut meta_sys, SUFFIX_TRANSITIONED_VERSION_ID, version_id.as_bytes().to_vec());
|
||||
}
|
||||
if !value.transition_tier.is_empty() {
|
||||
insert_bytes(&mut meta_sys, SUFFIX_TRANSITION_TIER, value.transition_tier.as_bytes().to_vec());
|
||||
}
|
||||
Self {
|
||||
version_id: value.version_id,
|
||||
mod_time: value.mod_time,
|
||||
meta_sys: HashMap::new(),
|
||||
meta_sys,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -3324,6 +3401,31 @@ mod tests {
|
||||
use super::*;
|
||||
use serde::Serialize;
|
||||
|
||||
#[test]
|
||||
fn delete_marker_conversion_preserves_only_durable_internal_metadata() {
|
||||
let mut marker = FileInfo::default();
|
||||
marker
|
||||
.metadata
|
||||
.insert("x-minio-internal-purgestatus".to_string(), "pending".to_string());
|
||||
marker
|
||||
.metadata
|
||||
.insert("x-rustfs-internal-healing".to_string(), "true".to_string());
|
||||
marker.metadata.insert("content-type".to_string(), "text/plain".to_string());
|
||||
let remote_version_id = Uuid::new_v4();
|
||||
marker.transition_version_id = Some(remote_version_id);
|
||||
|
||||
let converted = MetaDeleteMarker::from(marker);
|
||||
|
||||
assert_eq!(converted.meta_sys.get("x-rustfs-internal-purgestatus"), Some(&b"pending".to_vec()));
|
||||
assert_eq!(converted.meta_sys.get("x-minio-internal-purgestatus"), Some(&b"pending".to_vec()));
|
||||
assert_eq!(
|
||||
get_bytes(&converted.meta_sys, SUFFIX_TRANSITIONED_VERSION_ID),
|
||||
Some(remote_version_id.as_bytes().to_vec())
|
||||
);
|
||||
assert!(!converted.meta_sys.contains_key("x-rustfs-internal-healing"));
|
||||
assert!(!converted.meta_sys.contains_key("content-type"));
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
enum LegacyDeleteVersionTypeFixture {
|
||||
#[serde(rename = "DeleteMarker")]
|
||||
@@ -3997,6 +4099,32 @@ mod tests {
|
||||
assert_eq!(fi.transition_version_id, Some(id));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn meta_object_transition_version_id_unparseable_stays_readable_as_none() {
|
||||
// A non-UUID / non-16-byte tier version id must NOT make the object
|
||||
// unreadable; it is tolerated as "no tier version" (compat with
|
||||
// pre-hardening behavior and foreign/edge metadata).
|
||||
let mut sys = HashMap::new();
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_VERSION_ID, b"not-a-uuid".to_vec());
|
||||
let fi = make_meta_object_with_sys(sys)
|
||||
.into_fileinfo("b", "k", false)
|
||||
.expect("unparseable transition version id must not fail the object read");
|
||||
assert_eq!(fi.transition_version_id, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn meta_object_transition_version_id_minio_string_form_is_recovered() {
|
||||
// MinIO-migrated tiered objects store the remote tier's version id as a
|
||||
// UUID string, not 16 raw bytes; recover it instead of dropping it.
|
||||
let id = sample_version_id();
|
||||
let mut sys = HashMap::new();
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_VERSION_ID, id.to_string().into_bytes());
|
||||
let fi = make_meta_object_with_sys(sys)
|
||||
.into_fileinfo("b", "k", false)
|
||||
.expect("string-form transition version id must decode");
|
||||
assert_eq!(fi.transition_version_id, Some(id));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_marker_free_version_transition_version_id_nil_uuid_yields_none() {
|
||||
let mut sys = HashMap::new();
|
||||
@@ -4026,6 +4154,47 @@ mod tests {
|
||||
assert_eq!(fi.transition_version_id, Some(id));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_marker_free_version_transition_version_id_unparseable_stays_readable() {
|
||||
// A malformed tier version id must not make a free-version record corrupt:
|
||||
// it decodes to None and stays readable. Otherwise free-version expiry
|
||||
// fails and the remote-tier object leaks.
|
||||
let mut sys = HashMap::new();
|
||||
insert_bytes(&mut sys, SUFFIX_FREE_VERSION, vec![]);
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_VERSION_ID, b"not-a-uuid".to_vec());
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITION_TIER, b"WARM".to_vec());
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_OBJECTNAME, b"remote-object".to_vec());
|
||||
let fi = MetaDeleteMarker {
|
||||
version_id: Some(sample_version_id()),
|
||||
mod_time: Some(sample_mod_time()),
|
||||
meta_sys: sys,
|
||||
}
|
||||
.into_fileinfo("b", "k", false);
|
||||
|
||||
assert_eq!(fi.transition_version_id, None);
|
||||
fi.validate_for_metadata_read()
|
||||
.expect("free-version record with an unparseable tier id must remain readable");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_marker_free_version_transition_version_id_minio_string_form_is_recovered() {
|
||||
// MinIO stores the tier version id as a UUID string; recover it.
|
||||
let id = sample_version_id();
|
||||
let mut sys = HashMap::new();
|
||||
insert_bytes(&mut sys, SUFFIX_FREE_VERSION, vec![]);
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_VERSION_ID, id.to_string().into_bytes());
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITION_TIER, b"WARM".to_vec());
|
||||
insert_bytes(&mut sys, SUFFIX_TRANSITIONED_OBJECTNAME, b"remote-object".to_vec());
|
||||
let fi = MetaDeleteMarker {
|
||||
version_id: Some(sample_version_id()),
|
||||
mod_time: Some(sample_mod_time()),
|
||||
meta_sys: sys,
|
||||
}
|
||||
.into_fileinfo("b", "k", false);
|
||||
|
||||
assert_eq!(fi.transition_version_id, Some(id));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn version_header_sorts_before_prefers_object_over_delete_marker_on_equal_mod_time() {
|
||||
let object = FileMetaVersionHeader {
|
||||
|
||||
Reference in New Issue
Block a user