mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-13 16:46:55 +00:00
fix(ecstore): reject short-read shards and advance ParallelReader per stripe (backlog#799 B2) (#4327)
Two compounding defects let a truncated shard corrupt a GET (ranged GET could return HTTP 200 with wrong bytes): - `BitrotReader::read` returned `Ok(short_len)` when the shard stream hit EOF before filling the caller's buffer. With `skip_verify` / `HashAlgorithm::None` / parity=0 there is no hash to catch it, so the short shard was accepted and every downstream byte shifted. - `ParallelReader.offset` was set once and never advanced per stripe, so every stripe after the first reused the first stripe's geometry and the last-stripe length clamp was wrong. Fix both (they are mutually required): - `BitrotReader::read` now errors (`UnexpectedEof`) on a short read, before and independent of the bitrot hash check, so it fires under skip-verify/no-hash too. The caller sizes the buffer to the expected per-stripe shard length, so "buffer not filled" == "shard truncated". A short read routes through the existing `errs[i]` path, dropping that reader from the stripe so parity reconstruction engages; with parity=0 the stripe fails read quorum and the GET errors loudly instead of streaming shifted bytes. Mirrors MinIO's `parallelReader.Read` (`n != shardSize` -> reader failed). - `ParallelReader::read` / `read_lockstep` advance `self.offset += shard_size` per stripe so the per-stripe expected length (incl. the shorter final stripe) is exact, matching the correct pattern already used by heal. Updates three bitrot tests that used an obsolete oversized-buffer pattern to size per-stripe (as the real decode/heal paths do), and adds a regression test that a truncated shard errors under None/HighwayHash + skip_verify. Refs backlog#799 (B2), issue rustfs/backlog#851. Design converged by two independent expert reviews referencing MinIO cmd/erasure-decode.go.
This commit is contained in:
@@ -629,6 +629,14 @@ where
|
||||
return (vec![None; num_readers], vec![None; num_readers]);
|
||||
}
|
||||
|
||||
// Advance to the next stripe so the following read() computes the correct
|
||||
// (possibly shorter, final) stripe length. `offset` previously never
|
||||
// advanced, so every stripe after the first reused the first stripe's
|
||||
// geometry; combined with BitrotReader now rejecting short reads, the
|
||||
// per-stripe expected length must be exact (backlog#799 B2). `self.offset`
|
||||
// is only read above to derive `shard_size`, so advancing here is safe.
|
||||
self.offset += shard_size;
|
||||
|
||||
let mut shards: Vec<Option<Vec<u8>>> = vec![None; num_readers];
|
||||
let mut errs = vec![None; num_readers];
|
||||
let read_costs = self.read_costs.as_slice();
|
||||
@@ -1001,6 +1009,10 @@ where
|
||||
return (shards, errs);
|
||||
}
|
||||
|
||||
// Advance to the next stripe (see the matching note in `read`); the
|
||||
// lockstep path must track stripe geometry identically (backlog#799 B2).
|
||||
self.offset += shard_size;
|
||||
|
||||
self.buffers.ensure_slots(num_readers);
|
||||
// Pre-claim per-slot buffers so the `self.readers` borrow below stays
|
||||
// disjoint from `self.buffers`.
|
||||
|
||||
Reference in New Issue
Block a user