mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-31 01:09:23 +00:00
fix(storage): resolve erasure parity per pool (#4977)
* fix(filemeta): add state-aware file info validation
* fix(filemeta): validate shard arithmetic and delete paths
* fix(ecstore): add fallible erasure construction
* fix(ecstore): resolve storage parity per pool
* fix(storage): report heterogeneous erasure layouts
* fix(admin): publish prepared storage config atomically
* fix(storage): harden per-pool parity boundaries
* fix(storage): address pre-PR validation findings
* test(ci): fix strict-topology validation fixtures
* fix(heal): preserve delete markers during repair
* refactor(filemeta): drop unused ValidatedFileInfo witness
ValidatedFileInfo wrapped an unread `_file_info` reference alongside an `Option<ValidatedErasureLayout>`, but only the layout was ever consumed. Return the layout directly from `FileInfo::validate` so the sole production consumer (`LocalDisk::check_parts`) and the two unit tests read it without the extra witness type and lifetime.
No behavior change.
* fix(filemeta): keep compressed and MinIO-migrated tiered objects readable
The new decode-path validation rejected several legitimate on-disk shapes that older RustFS and MinIO-migrated data carry, turning readable objects into FileCorrupt:
- Compressed objects written with an unknown upload size persist a negative per-part actual_size (the documented "unknown size" sentinel that ObjectInfo::get_actual_size already tolerates). validate_collection_contents rejected it via usize::try_from; now a negative actual_size skips shard validation and only real, non-negative sizes are checked.
- MinIO-migrated objects transitioned to a versioned remote tier store the tier version id as a UUID string, not 16 raw bytes. MetaObject::into_fileinfo returned FileCorrupt (main tolerated it as None), making all versions of the object unreadable; MetaDeleteMarker free-version records took a Some(nil) sentinel path with the same effect, which also breaks free-version expiry (remote-tier leak). Both now decode through a shared transitioned_version_id_from_meta_sys helper: 16 raw bytes or a UUID string are accepted, anything else is tolerated as None instead of failing the read.
Regression tests updated to assert the readable/compat behavior, with new tests covering MinIO string-form recovery.
* fix(scanner): build the delete-marker test fixture without erasure geometry
get_size_counts_delete_markers_separately_from_versions built its delete marker with `FileInfo::new(object, 1, 1)`, which attaches erasure geometry (data=1/parity=1/distribution). This PR classifies versions by shape via `is_storage_delete_marker()` (no geometry) rather than the raw `deleted` flag, so a geometry-bearing "delete marker" is correctly serialized as a purge-pending payload Object and counted as a version — CI saw summary.versions=3, expected 2.
Real delete markers carry no erasure geometry (delete paths build them as `FileInfo { deleted: true, ..Default::default() }`), so construct the fixture the same way. It then classifies as a storage delete marker and the counts (versions=2, delete_markers=1) hold. This keeps the PR's more-correct classification, which prevents a purge-pending object's geometry from being dropped when serialized as a bare delete marker.
* docs(changelog): note per-pool parity fix and storage-class startup upgrade caveat
Records the #4801 per-pool erasure parity fix under Fixed, and documents the upgrade behavior where a persisted storage class that a small or heterogeneous pool cannot satisfy now fails startup — with the RUSTFS_STORAGE_CLASS_STANDARD recovery steps. Docs-only; covers R4 from the on-disk compatibility audit.
* fix(heal): report parity from erasure geometry, not is_valid()
heal_object set HealResultItem.parity_blocks via `if lfi.is_valid()`, which was missed by the migration of the other quorum/metadata predicates. With the new `is_valid()` semantics (full payload validation; delete markers now return false), a delete marker or a geometry-bearing version with a benign collection quirk would misreport parity as the pool default instead of its own. Use `has_valid_erasure_geometry()` — the narrow "does this carry erasure geometry" predicate the rest of the migration uses — so reporting matches the object's actual layout. Reporting-only; no data-path change.
* fix(filemeta): do not silently serialize a non-canonical deleted FileInfo as an Object
`From<FileInfo> for FileMetaVersion` classifies by `is_storage_delete_marker()` (shape), which correctly routes canonical delete markers to Delete and purge-pending payloads (deleted=true with real erasure geometry) to Object. But a `deleted` FileInfo that is neither a canonical marker nor a valid erasure payload would silently serialize as a zero-geometry MetaObject that later fails `validate_for_metadata_read`. Write paths validate first (`validate_for_erasure_write` / `validate_for_metadata_read`), so this is a caller bug; `From` is infallible, so surface it with a structured `warn!` on the malformed branch instead of writing corrupt metadata silently. Legitimate purge-pending objects (valid geometry) are unaffected — the guard only fires for `deleted && !has_valid_erasure_geometry()`.
* test(filemeta): assert real historical xl.meta versions pass metadata-read validation
Empirical companion to the code-reasoned decode-tolerance invariants (docs/architecture/erasure-coding.md §11) and the rolling-upgrade / MinIO-migration compatibility concern: the tightened `validate_for_metadata_read` runs on every local disk read and peer-RPC-decoded FileInfo, so it must accept every version of real historically-written xl.meta, never reject it as FileCorrupt.
Loads five real fixtures — MinIO small-inline, MinIO versioned (two object versions + a delete marker), MinIO large multipart, a legacy V1 (xl.json-derived) object, and a legacy meta_ver 2 object — decodes every version with parts materialized, and asserts validate_for_metadata_read() is Ok for each. Reverting the tolerant handling (delete-marker shape, legacy per-part checksums, string/short transitioned-versionID, negative actual_size) turns this red.
* fix(ci): remove duplicate storage test re-exports
---------
Co-authored-by: overtrue <anzhengchao@gmail.com>
This commit is contained in:
cxymds
@@ -121,7 +121,7 @@ impl SetDisks {
|
||||
online_disks: &[Option<DiskStore>],
|
||||
read_quorum: usize,
|
||||
) {
|
||||
if fi.deleted || !fi.is_valid() {
|
||||
if fi.deleted || !fi.has_valid_erasure_geometry() {
|
||||
return;
|
||||
}
|
||||
let (bucket, object) = identity;
|
||||
@@ -3192,10 +3192,13 @@ mod tests {
|
||||
let mut historical = metadata_fanout_test_fileinfo("object");
|
||||
historical.version_id = Some(Uuid::parse_str("00000000-0000-0000-0000-000000000002").expect("static uuid should parse"));
|
||||
|
||||
let mut delete_marker = metadata_fanout_test_fileinfo("object");
|
||||
delete_marker.deleted = true;
|
||||
delete_marker.version_id =
|
||||
Some(Uuid::parse_str("00000000-0000-0000-0000-000000000003").expect("static uuid should parse"));
|
||||
let delete_marker = FileInfo {
|
||||
name: "object".to_string(),
|
||||
deleted: true,
|
||||
version_id: Some(Uuid::parse_str("00000000-0000-0000-0000-000000000003").expect("static uuid should parse")),
|
||||
mod_time: Some(OffsetDateTime::from_unix_timestamp(3).expect("static timestamp should parse")),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let diagnostics = MetadataFanoutDiagnostics::new(
|
||||
Duration::from_millis(9),
|
||||
@@ -3296,6 +3299,47 @@ mod tests {
|
||||
assert_eq!(accumulator.final_miss_reason(), GET_METADATA_EARLY_STOP_REASON_CONFLICTING_METADATA);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn metadata_quorum_accumulator_rejects_semantic_field_splits() {
|
||||
type FileInfoMutation = fn(&mut FileInfo);
|
||||
|
||||
let mutations: &[(&str, FileInfoMutation)] = &[
|
||||
("transition_status", |fi| fi.transition_status = "complete".to_string()),
|
||||
("transitioned_objname", |fi| fi.transitioned_objname = "remote-object".to_string()),
|
||||
("transition_tier", |fi| fi.transition_tier = "WARM".to_string()),
|
||||
("transition_version_id", |fi| fi.transition_version_id = Some(Uuid::from_u128(10))),
|
||||
("expire_restored", |fi| fi.expire_restored = true),
|
||||
("written_by_version", |fi| fi.written_by_version = Some(1)),
|
||||
("replication_state_internal", |fi| {
|
||||
fi.replication_state_internal = Some(Default::default())
|
||||
}),
|
||||
("num_versions", |fi| fi.num_versions = 2),
|
||||
("successor_mod_time", |fi| {
|
||||
fi.successor_mod_time = Some(OffsetDateTime::from_unix_timestamp(10).expect("static timestamp should parse"));
|
||||
}),
|
||||
];
|
||||
|
||||
for (field, mutate) in mutations {
|
||||
let mut accumulator = metadata_early_stop_accumulator();
|
||||
let first = metadata_early_stop_candidate("object", 1);
|
||||
let mut second = metadata_early_stop_candidate("object", 2);
|
||||
let mut third = metadata_early_stop_candidate("object", 3);
|
||||
mutate(&mut second);
|
||||
mutate(&mut third);
|
||||
|
||||
accumulator.observe_file_info(&first);
|
||||
accumulator.observe_file_info(&second);
|
||||
accumulator.observe_file_info(&third);
|
||||
|
||||
assert!(accumulator.conflicting_metadata, "split {field} must block metadata early-stop");
|
||||
assert!(
|
||||
accumulator.early_stop_decision().is_none(),
|
||||
"split {field} must not be mistaken for three matching votes"
|
||||
);
|
||||
assert_eq!(accumulator.final_miss_reason(), GET_METADATA_EARLY_STOP_REASON_CONFLICTING_METADATA);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn metadata_quorum_accumulator_falls_back_on_split_data_dir() {
|
||||
let mut accumulator = metadata_early_stop_accumulator();
|
||||
@@ -3325,8 +3369,15 @@ mod tests {
|
||||
#[test]
|
||||
fn metadata_quorum_accumulator_hits_delete_marker_quorum_early_stop() {
|
||||
let mut accumulator = metadata_early_stop_accumulator();
|
||||
let mut deleted = metadata_early_stop_candidate("object", 1);
|
||||
deleted.deleted = true;
|
||||
let deleted = FileInfo {
|
||||
name: "object".to_string(),
|
||||
deleted: true,
|
||||
version_id: Some(Uuid::new_v4()),
|
||||
mod_time: Some(OffsetDateTime::now_utc()),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
assert!(!deleted.is_valid(), "real delete markers do not carry erasure geometry");
|
||||
|
||||
accumulator.observe_file_info(&deleted);
|
||||
accumulator.observe_file_info(&deleted);
|
||||
@@ -3344,8 +3395,13 @@ mod tests {
|
||||
#[test]
|
||||
fn metadata_quorum_accumulator_falls_back_on_delete_marker_below_quorum() {
|
||||
let mut accumulator = metadata_early_stop_accumulator();
|
||||
let mut deleted = metadata_early_stop_candidate("object", 1);
|
||||
deleted.deleted = true;
|
||||
let deleted = FileInfo {
|
||||
name: "object".to_string(),
|
||||
deleted: true,
|
||||
version_id: Some(Uuid::parse_str("00000000-0000-0000-0000-000000000004").expect("static uuid should parse")),
|
||||
mod_time: Some(OffsetDateTime::from_unix_timestamp(4).expect("static timestamp should parse")),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
accumulator.observe_file_info(&deleted);
|
||||
|
||||
@@ -3353,6 +3409,49 @@ mod tests {
|
||||
assert_eq!(accumulator.final_miss_reason(), GET_METADATA_EARLY_STOP_REASON_DELETE_MARKER);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn metadata_quorum_accumulator_does_not_treat_purge_pending_payload_as_delete_marker() {
|
||||
let mut accumulator = MetadataQuorumAccumulator::new(6, 3, true);
|
||||
let version_id = Uuid::parse_str("00000000-0000-0000-0000-000000000005").expect("static uuid should parse");
|
||||
|
||||
for disk_index in 1..=4 {
|
||||
let mut purge_pending = FileInfo::new("object", 5, 1);
|
||||
purge_pending.name = "object".to_string();
|
||||
purge_pending.version_id = Some(version_id);
|
||||
purge_pending.mod_time = Some(OffsetDateTime::from_unix_timestamp(5).expect("static timestamp should parse"));
|
||||
purge_pending.size = 1;
|
||||
purge_pending.deleted = true;
|
||||
purge_pending.erasure.index = disk_index;
|
||||
purge_pending.add_object_part(1, "part-etag-1".to_string(), 1, None, 1, None, None);
|
||||
|
||||
accumulator.observe_file_info(&purge_pending);
|
||||
}
|
||||
|
||||
assert!(!accumulator.delete_marker_seen);
|
||||
assert_eq!(accumulator.candidate_votes, 4);
|
||||
assert!(
|
||||
accumulator.early_stop_decision().is_none(),
|
||||
"EC:1 purge-pending payload on six disks still requires five matching payload votes"
|
||||
);
|
||||
|
||||
let mut fifth = FileInfo::new("object", 5, 1);
|
||||
fifth.name = "object".to_string();
|
||||
fifth.version_id = Some(version_id);
|
||||
fifth.mod_time = Some(OffsetDateTime::from_unix_timestamp(5).expect("static timestamp should parse"));
|
||||
fifth.size = 1;
|
||||
fifth.deleted = true;
|
||||
fifth.erasure.index = 5;
|
||||
fifth.add_object_part(1, "part-etag-1".to_string(), 1, None, 1, None, None);
|
||||
accumulator.observe_file_info(&fifth);
|
||||
|
||||
assert_eq!(
|
||||
accumulator.early_stop_decision(),
|
||||
Some(MetadataEarlyStopDecision {
|
||||
reason: GET_METADATA_EARLY_STOP_REASON_VALID_QUORUM
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn metadata_quorum_accumulator_falls_back_on_object_not_found_quorum() {
|
||||
let mut accumulator = metadata_early_stop_accumulator();
|
||||
|
||||
Reference in New Issue
Block a user