* perf(ecstore): reuse erasure-decode shard buffers across stripes
ParallelReader::read allocated and zero-filled a fresh
`vec![0u8; shard_size]` per shard on every erasure stripe. Erasure::decode
builds one ParallelReader and loops stripes, so those buffers can be
reused: BitrotReader::read overwrites buf[..n] and the caller truncates to
n, so leftover bytes from the prior stripe are never observed. The decode
loop now hands each stripe's shard buffers back to the reader, which
reuses them (resized to shard_size) on the next stripe. The heal path,
which consumes its buffers into Bytes, never hands them back and is
unchanged.
This path runs on every object GET. Streaming-decode benchmark
(Erasure::decode end-to-end, median):
4+2 16MiB 1MiB-block: -9.6% verify-on, -23.8% verify-off
6+3 24MiB 1MiB-block: -9.7% verify-on, -22.5% verify-off
4+2 4MiB 64KiB-block: -5.5% verify-on, -16.3% verify-off
Adds a streaming-decode benchmark covering the ParallelReader path and a
regression test that decodes a multi-stripe object with missing data
shards (so reconstructed buffers are recycled) with bitrot verification
both on and off, asserting byte-exact output.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* perf(ecstore): tighten decode buffer reuse and bench harness
Address PR review feedback on the erasure-decode buffer-reuse change:
- Only claim a recycled shard buffer inside the `Some(reader)` branch of
`ParallelReader::read`, so the take is co-located with the read that
uses it and missing shards no longer touch the recycle slot.
- Move per-shard `BitrotReader` construction into Criterion `iter_batched`
setup so reader/UUID construction stays out of the timed decode path,
and assert decode success plus full output length each iteration.
Re-measured with the cleaner harness (median, before -> after, all
p < 0.05): verify-on -5.3%..-11.7%, verify-off -15.8%..-25.3%.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* perf(ecstore): drop dead recycle slots and tidy decode bench
Address the second PR review pass:
- In `Erasure::decode`, clear recycled slots for missing-shard (None-reader)
indices before handing buffers back. `ParallelReader::read` only reuses
slots with an active reader, and `decode_data` always re-allocates the
reconstructed shard, so retaining those buffers only held memory that could
never be reused in degraded reads.
- Move the output sink into the benchmark's `iter_batched` setup and validate
decode success/output length once per config instead of inside the timed
loop, so neither sink construction nor assertions touch the measurement.
Re-measured with the final harness (median, before -> after, all p < 0.05):
verify-on -5.2%..-9.8%, verify-off -19.4%..-25.8%.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
Zhengchao An