feat(rio): add bounded tee reader for one-read two-consumer paths (#7066)

Add `tee_reader` / `tee_reader_with_options` in `rustfs-rio`: a
`TeePrimary` that drives the source and a `TeeSecondary` that observes
an identical copy of every chunk through a byte-bounded queue. The
primary returns `Pending` when the queue is full, so both sides advance
at the pace of the slowest consumer; it is meant for small objects only.

Termination: source EOF and errors propagate to the secondary with the
same `io::ErrorKind`; dropping the secondary turns the primary into a
pass-through; dropping the primary early fails the secondary with
`BrokenPipe` by default, or hands the remaining source to a background
drain task bounded by `max_drain_bytes` when
`TeeOptions::drain_on_primary_drop` is set. `TeeSecondary::into_stream`
exposes the queued `Bytes` chunks without an extra copy.

Includes a proptest equivalence test, backpressure, error, drop,
drain-limit and cancel-safety tests, and a criterion bench comparing
tee throughput against a direct read (64 MiB in 1 MiB chunks).
This commit is contained in:
Zhengchao An
2026-09-02 21:59:46 +08:00
committed by GitHub
parent 01db1f6644
commit 1dd81cf276
5 changed files with 1176 additions and 0 deletions
+6
View File
@@ -118,6 +118,12 @@ pub use hardlimit_reader::HardLimitReader;
mod hash_reader;
pub use hash_reader::*;
mod tee_reader;
pub use tee_reader::{
DEFAULT_TEE_MAX_DRAIN_BYTES, TeeDrainLimitExceeded, TeeOptions, TeePrimary, TeeSecondary, TeeStream, tee_reader,
tee_reader_with_options,
};
mod checksum;
pub use checksum::*;
File diff suppressed because it is too large Load Diff