mirror of https://github.com/rustfs/rustfs.git synced 2026-10-27 06:18:43 +00:00

Files

T

History

Jim Chen 2f90fb19f5 fix(rio): ZigZag-encode signed delta varints and widen scratch buffer in compress_index (#8893 )

* fix(rio): ZigZag-encode signed delta varints and widen scratch buffer in compress_index

Index::append_to passes signed delta residuals (c_off and u_off, which are negative whenever a compressed block is smaller than est_block_uncomp / 2) to write_varint(buf, v: i64), which checks while v >= 0x80 without ZigZag encoding and truncates every negative delta to a single byte, while tmp = [0u8; 8] panics out-of-bounds on 9-10 byte varints.

Root Cause:
In crates/rio/src/compress_index.rs, Index::append_to delta-encodes u_off and c_off (c_off -= prev.compressed_offset + c_predict, where c_predict starts at est_block_uncomp / 2) as signed i64 residuals and passes them to write_varint(buf, v: i64). However, write_varint checks while v >= 0x80 on the signed i64 without ZigZag encoding (unlike Go binary.PutVarint / binary.Varint in the S2 index format): for any negative v, v >= 0x80 is false, so write_varint writes a single truncated byte v as u8 without continuation bits, either silently corrupting compressed_offset on Index::load (when (v as u8) < 0x80) or desynchronizing the varint stream (when (v as u8) >= 0x80). Additionally, tmp = [0u8; 8] at line 223 panics with an out-of-bounds index when encoding 9-10 byte varints, and read_varint lacks an overflow check on 10-byte inputs.

Fix:
Use ZigZag encoding and decoding in write_varint and read_varint, reject 64-bit varint overflows in read_varint, and widen the scratch buffer tmp in Index::append_to from [0u8; 8] to [0u8; 10].

Signed-off-by: czcbangkai <czcbangkai@google.com>

* test(rio): use est_block_uncomp field and compare IndexInfo fields in compress_index test

---------

Signed-off-by: czcbangkai <czcbangkai@google.com>

2026-10-25 03:58:33 +08:00

benches

feat(rio): add bounded tee reader for one-read two-consumer paths (#7066 )

2026-09-02 21:59:46 +08:00

src

fix(rio): ZigZag-encode signed delta varints and widen scratch buffer in compress_index (#8893 )

2026-10-25 03:58:33 +08:00

Cargo.toml

fix: isolate internode reads and migrate crypto dependencies (#8872 )

2026-10-24 17:41:55 +08:00

README.md

docs: fix some dead link (#1053 )

2025-12-08 11:23:24 +08:00

README.md

RustFS Rio - High-Performance I/O

High-performance asynchronous I/O operations for RustFS distributed object storage

📖 Documentation · 🐛 Bug Reports · 💬 Discussions

📖 Overview

RustFS Rio provides high-performance asynchronous I/O operations for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features

Zero-copy streaming I/O operations
Hardware-accelerated encryption/decryption
Multi-algorithm compression support
Efficient buffer management and pooling
Vectored I/O for improved throughput
Real-time data integrity verification

📚 Documentation

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.