fix(ecstore): reject short-read shards and advance ParallelReader per stripe (backlog#799 B2) (#4327)

Two compounding defects let a truncated shard corrupt a GET (ranged GET could
return HTTP 200 with wrong bytes):

- `BitrotReader::read` returned `Ok(short_len)` when the shard stream hit EOF
  before filling the caller's buffer. With `skip_verify` /
  `HashAlgorithm::None` / parity=0 there is no hash to catch it, so the short
  shard was accepted and every downstream byte shifted.
- `ParallelReader.offset` was set once and never advanced per stripe, so every
  stripe after the first reused the first stripe's geometry and the last-stripe
  length clamp was wrong.

Fix both (they are mutually required):

- `BitrotReader::read` now errors (`UnexpectedEof`) on a short read, before and
  independent of the bitrot hash check, so it fires under skip-verify/no-hash
  too. The caller sizes the buffer to the expected per-stripe shard length, so
  "buffer not filled" == "shard truncated". A short read routes through the
  existing `errs[i]` path, dropping that reader from the stripe so parity
  reconstruction engages; with parity=0 the stripe fails read quorum and the GET
  errors loudly instead of streaming shifted bytes. Mirrors MinIO's
  `parallelReader.Read` (`n != shardSize` -> reader failed).
- `ParallelReader::read` / `read_lockstep` advance `self.offset += shard_size`
  per stripe so the per-stripe expected length (incl. the shorter final stripe)
  is exact, matching the correct pattern already used by heal.

Updates three bitrot tests that used an obsolete oversized-buffer pattern to
size per-stripe (as the real decode/heal paths do), and adds a regression test
that a truncated shard errors under None/HighwayHash + skip_verify.

Refs backlog#799 (B2), issue rustfs/backlog#851. Design converged by two
independent expert reviews referencing MinIO cmd/erasure-decode.go.
This commit is contained in:
Zhengchao An
2026-07-06 23:25:54 +08:00
committed by GitHub
parent 6297d112c3
commit 655c5cb403
2 changed files with 77 additions and 3 deletions
@@ -629,6 +629,14 @@ where
return (vec![None; num_readers], vec![None; num_readers]);
}
// Advance to the next stripe so the following read() computes the correct
// (possibly shorter, final) stripe length. `offset` previously never
// advanced, so every stripe after the first reused the first stripe's
// geometry; combined with BitrotReader now rejecting short reads, the
// per-stripe expected length must be exact (backlog#799 B2). `self.offset`
// is only read above to derive `shard_size`, so advancing here is safe.
self.offset += shard_size;
let mut shards: Vec<Option<Vec<u8>>> = vec![None; num_readers];
let mut errs = vec![None; num_readers];
let read_costs = self.read_costs.as_slice();
@@ -1001,6 +1009,10 @@ where
return (shards, errs);
}
// Advance to the next stripe (see the matching note in `read`); the
// lockstep path must track stripe geometry identically (backlog#799 B2).
self.offset += shard_size;
self.buffers.ensure_slots(num_readers);
// Pre-claim per-slot buffers so the `self.readers` borrow below stays
// disjoint from `self.buffers`.