mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-20 19:42:17 +00:00
docs(ecstore): pin streaming-only bitrot layout invariant (ECA-18) (#4553)
bitrot_shard_file_size only counts per-block checksum bytes for the two streaming Highway variants, while BitrotWriter::write interleaves a hash for any hash_algo.size() > 0 and bitrot_verify's read loop assumes an interleaved hash per block. The three disagree for non-streaming algorithms (SHA256/HighwayHash256/BLAKE2b512/Md5), but the divergence is unreachable in production: every write path hardcodes HighwayHash256S and ErasureInfo::get_checksum_info defaults to HighwayHash256S. Per the audit decision (backlog#959), do NOT change the size formula: it is a byte-for-byte port of MinIO's bitrotShardFileSize and its bare return for non-streaming algorithms is correct for MinIO whole-file bitrot; changing it would break legacy interop. Instead, document the per-algorithm layout contract at bitrot_shard_file_size, BitrotWriter, and bitrot_verify, and add regression tests that pin the invariants: get_checksum_info defaults to HighwayHash256S, and the size formula counts per-block hash bytes for streaming variants only while returning the bare size for non-streaming ones. No disk layout or formula change. Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
@@ -715,6 +715,38 @@ mod tests {
|
||||
use proptest::collection::{hash_map, vec};
|
||||
use proptest::prelude::*;
|
||||
|
||||
// backlog#959 / ECA-18: the interleaved per-block bitrot subsystem in
|
||||
// rustfs-ecstore (BitrotWriter / bitrot_verify / bitrot_shard_file_size) is
|
||||
// only self-consistent for the streaming Highway variants. That safety rests
|
||||
// on production always resolving a streaming checksum algorithm, so pin the
|
||||
// default here: a part with no explicit ChecksumInfo must resolve to
|
||||
// HighwayHash256S. If this default ever changes, the ecstore bitrot layout
|
||||
// assumptions must be revisited in lockstep.
|
||||
#[test]
|
||||
fn get_checksum_info_defaults_to_highwayhash256s() {
|
||||
let ei = ErasureInfo::default();
|
||||
assert!(ei.checksums.is_empty(), "default ErasureInfo carries no checksums");
|
||||
let info = ei.get_checksum_info(1);
|
||||
assert_eq!(
|
||||
info.algorithm,
|
||||
HashAlgorithm::HighwayHash256S,
|
||||
"missing ChecksumInfo must default to the streaming HighwayHash256S algorithm"
|
||||
);
|
||||
|
||||
// A present entry is returned as-is; the default only applies on miss.
|
||||
let ei = ErasureInfo {
|
||||
checksums: vec![ChecksumInfo {
|
||||
part_number: 2,
|
||||
algorithm: HashAlgorithm::SHA256,
|
||||
hash: Bytes::new(),
|
||||
}],
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(ei.get_checksum_info(2).algorithm, HashAlgorithm::SHA256);
|
||||
// A part_number with no entry still falls back to the streaming default.
|
||||
assert_eq!(ei.get_checksum_info(99).algorithm, HashAlgorithm::HighwayHash256S);
|
||||
}
|
||||
|
||||
// backlog#949: distribution range/permutation validation.
|
||||
#[test]
|
||||
fn is_valid_distribution_accepts_permutation() {
|
||||
|
||||
Reference in New Issue
Block a user