mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-06 05:17:42 +00:00
houseme
d232a46b4d
perf(ecstore): wire legacy decode stripe prefetch / bitrot-decode overlap (backlog#930 HP-9 step 2) The legacy GET decode duplex loop (default GET path + every Range request) was strictly serial: read a stripe, reconstruct it, emit it, and only then begin reading the next stripe. Two switches introduced by PR#3972 to overlap this work — RUSTFS_GET_DECODE_STRIPE_PREFETCH_COUNT and RUSTFS_GET_BITROT_DECODE_OVERLAP_ENABLE — were config-only with zero call sites since introduction. Wire both into the loop as a depth-1 stripe prefetch: while the current stripe is reconstructed and emitted, the next stripe's shard reads (including bitrot verification) run concurrently, hiding read latency under the emit / duplex-backpressure stage. ParallelReader::read is inherently serial (it takes &mut self and advances a shared stripe cursor), so at most one read can be in flight; a prefetch count above 1 therefore collapses to the same single-stripe-ahead pipeline rather than reading several stripes ahead. Both switches feed one gate, legacy_stripe_prefetch_enabled(). Default (count == 1, overlap == false) keeps the loop on the pre-existing strictly-serial path, byte for byte: the prefetch pipeline is a separate branch and the serial branch is unchanged. The reconstruct/emit body is factored into a shared emit_decoded_stripe helper used by both branches so error attribution, read-quorum handling, reconstruction verification and stage metrics cannot drift between them. Correctness guarantees preserved under prefetch: - A speculatively prefetched read for stripe N+1 is only consumed when the loop reaches N+1; if stripe N stops the loop, the in-flight read is awaited and dropped, so its errors never surface against stripe N. - Bitrot (HighwayHash) verification runs inside each stripe read and is not bypassed or reordered; a corrupt shard is still rejected and, when unrecoverable, the read fails without emitting garbage. - Shard buffers are recycled only after the overlapping next read has claimed its own — one extra stripe of memory (double buffering) with buffer reuse preserved at a one-stripe lag. - Per-stripe exact length advance (backlog#799 B2), lockstep reconstruction verification (backlog#832) and the hash_size == 0 pass-through are unchanged. Adds regression tests exercising serial-default, count>1 and overlap configs: full/range reads byte-exact, degraded (missing-shard) reconstruction, corrupt shard rejected-but-recovered, unrecoverable corruption erroring with no output, late-stripe failure attributed correctly with stripe 0 still emitted, hash_size == 0 pass-through, and the gate defaulting off. Co-authored-by: heihutu <heihutu@gmail.com>