mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-27 16:48:58 +00:00
Ramakrishna Chilaka
956bd19417
* perf(ecstore): reuse erasure-decode shard buffers across stripes ParallelReader::read allocated and zero-filled a fresh `vec![0u8; shard_size]` per shard on every erasure stripe. Erasure::decode builds one ParallelReader and loops stripes, so those buffers can be reused: BitrotReader::read overwrites buf[..n] and the caller truncates to n, so leftover bytes from the prior stripe are never observed. The decode loop now hands each stripe's shard buffers back to the reader, which reuses them (resized to shard_size) on the next stripe. The heal path, which consumes its buffers into Bytes, never hands them back and is unchanged. This path runs on every object GET. Streaming-decode benchmark (Erasure::decode end-to-end, median): 4+2 16MiB 1MiB-block: -9.6% verify-on, -23.8% verify-off 6+3 24MiB 1MiB-block: -9.7% verify-on, -22.5% verify-off 4+2 4MiB 64KiB-block: -5.5% verify-on, -16.3% verify-off Adds a streaming-decode benchmark covering the ParallelReader path and a regression test that decodes a multi-stripe object with missing data shards (so reconstructed buffers are recycled) with bitrot verification both on and off, asserting byte-exact output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * perf(ecstore): tighten decode buffer reuse and bench harness Address PR review feedback on the erasure-decode buffer-reuse change: - Only claim a recycled shard buffer inside the `Some(reader)` branch of `ParallelReader::read`, so the take is co-located with the read that uses it and missing shards no longer touch the recycle slot. - Move per-shard `BitrotReader` construction into Criterion `iter_batched` setup so reader/UUID construction stays out of the timed decode path, and assert decode success plus full output length each iteration. Re-measured with the cleaner harness (median, before -> after, all p < 0.05): verify-on -5.3%..-11.7%, verify-off -15.8%..-25.3%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * perf(ecstore): drop dead recycle slots and tidy decode bench Address the second PR review pass: - In `Erasure::decode`, clear recycled slots for missing-shard (None-reader) indices before handing buffers back. `ParallelReader::read` only reuses slots with an active reader, and `decode_data` always re-allocates the reconstructed shard, so retaining those buffers only held memory that could never be reused in degraded reads. - Move the output sink into the benchmark's `iter_batched` setup and validate decode success/output length once per config instead of inside the timed loop, so neither sink construction nor assertions touch the measurement. Re-measured with the final harness (median, before -> after, all p < 0.05): verify-on -5.2%..-9.8%, verify-off -19.4%..-25.8%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>