mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-08 06:13:14 +00:00
houseme
608ab14d7d
Sub-change A only: the two local-disk metadata read paths (`read_metadata_with_dmtime`, `read_all_data_with_dmtime` in crates/ecstore/src/disk/local.rs) previously dispatched open, fstat and the xl.meta read as three separate async fs hops (three spawn_blocking round-trips under tokio::fs). Each is now folded into a single tokio::task::spawn_blocking closure using std::fs, cutting per-metadata-read dispatch from 3 to 1. This does NOT touch sub-change B (removing the entry-point access() call). Correctness first: this is the hottest metadata read path, so the refactor preserves byte-for-byte-equivalent Results. Every error mapping is kept identical: - open failure -> to_file_error - is_dir -> Error::FileNotFound (not to_file_error(EISDIR)) - metadata failure -> to_file_error - xl.meta parse -> propagated verbatim from the parser - try_reserve -> Error::other - read_to_end -> to_file_error For read_all_data_with_dmtime the async NotFound -> access(volume_dir) -> VolumeNotFound fallback (and its warn! event) is preserved on the async side: the closure returns the raw open error unmapped; only open() can yield ENOENT once the fd is valid, so gating the fallback on the open error is equivalent to the original open-arm-only fallback. To run the parser inside a blocking closure, filemeta gains a synchronous twin `read_xl_meta_no_data_sync` (+ `read_more_sync`) in crates/filemeta/src/filemeta/version.rs that line-for-line mirrors the async version, differing only in std vs tokio read_exact. A new equivalence test (`read_xl_meta_sync_equivalence_tests`) feeds identical buffers to both the async and sync readers and asserts equal Ok bytes / equal Err variants across: v1.0; v1.1/v1.2/v1.3; large meta triggering read_more; header truncation -> UnexpectedEof; CRC-trailer truncation -> FileCorrupt; unknown major/minor -> InvalidData; and want boundaries (exact fit, inline-data drop, read_more EOF). Co-authored-by: heihutu <heihutu@gmail.com>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team