mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-05 21:07:43 +00:00
Zhengchao An
cf056b39e3
* fix(core-storage): fix critical correctness defects from core-storage audit Fixes verified defects found in a deep audit of the core storage path (erasure coding, disk persistence, quorum, heal, replication resync): - ecstore/disk: rewrite live xl.meta atomically (temp+rename) in delete_versions_internal and write_metadata instead of in-place truncate, which exposed torn metadata to concurrent readers and crashes on the DeleteObjects hot path - ecstore/erasure: allow heal to reconstruct from exactly data_shards bitrot-verified sources; requiring data_shards+1 made objects permanently unhealable after losing parity_shards disks - ecstore/set_disk: direct-memory inline GET applied the erasure distribution permutation twice (shuffled inputs re-indexed through distribution), concatenating wrong shards into the response body in degraded reads; collect from canonical disk-ordered inputs - ecstore/set_disk: heal now preserves the committed inline layout instead of recomputing it with a hardcoded unversioned threshold, which split quorum identity of healed replicas and caused endless re-heal churn - ecstore/replication: resync results channel switched from broadcast(1) to mpsc; a lagged broadcast receiver ended the stats collector and every subsequent failure went uncounted, letting failed resyncs be marked completed - ecstore/replication: ignore an empty persisted resync checkpoint; resuming with one skipped every object and marked the resync completed without replicating anything - ecstore/replication: fix inverted not-found error classification in replicate_object/replicate_delete logging paths - ecstore/erasure: guard decode paths against zero block_size or data_shards from corrupt on-disk metadata (divide-by-zero panic) - ecstore/disk: os::read_dir no longer consumes the entry limit on entries it does not return (is_empty_dir misjudgment); create_file opens with O_TRUNC to avoid stale trailing bytes - filemeta: treat Some(nil) version id as a null version in matches_not_strict; disk-loaded headers never store None, so the mod_time quorum guard for unversioned overwrites never fired and an interrupted overwrite could displace the committed version in merge - filemeta: fix msgpack skip lengths for fixext (missed the ext type byte) and ext16/32 (over-skipped) unknown fields - filemeta: return FileCorrupt instead of usize underflow when xl.meta is truncated inside the CRC trailer - filemeta: surface delete-marker insertion failure in delete_version instead of reporting success when the data dir is shared Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(replication): drop duplicate cfg(test) etag import from boundary module The test module already imports content_matches_by_etag locally, so the top-level cfg(test) import is unused under -D warnings and fails clippy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team