mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-09 14:49:25 +00:00
Zhengchao An
3e4a7c1f6a
fix(ecstore): lockstep stripe read to stop EC GET desync truncation On the reconstruction-verifying GET path, ParallelReader::read used a data-first schedule: it read only `data_shards` readers per stripe and pulled in a parity reader as a substitute on demand. Because the shard readers are streaming (advanced only by being read, no seek), a parity reader first used mid-object was still positioned at its stream start (block 0) and returned an earlier stripe than the surviving data shards. Every shard passed its own bitrot hash, yet the set was mutually misaligned, so decode_data_with_reconstruction_verification correctly rejected it with "inconsistent read source shards" and the large-object GET truncated mid-stream (client "unexpected EOF"). Add read_lockstep(): on the verify_reconstruction path, read every live shard reader once per stripe and wait for all of them, so all readers advance one block per stripe and stay mutually aligned; any reader that errors is retired for the rest of the object (a stream that failed mid-block can no longer be trusted to be aligned). The adaptive data-first path is unchanged for non-verifying callers (e.g. heal). Adds a regression test reproducing a data shard that dies partway through a multi-stripe object; it must still reconstruct byte-exact output. Refs backlog#832.
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team