mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-06 13:27:43 +00:00
houseme
5533e080b0
* refactor(ecstore): extract LocalIoBackend trait behind LocalDisk Model the LocalDisk per-file I/O hot path as a LocalIoBackend trait (pread_bytes / open_read_stream / open_full_read / open_write) and move the existing method bodies verbatim into a default StdBackend. LocalDisk holds Arc<dyn LocalIoBackend> and the DiskAPI methods forward to it. This is a behavior-preserving refactor: no logic, error-mapping, metrics, or cfg-branch changes. It creates the seam for an alternative runtime-probed io_uring backend (rustfs/backlog#894) without touching DiskAPI callers. Commit-point durability (fdatasync -> rename -> fsync-dir in rename_data) deliberately stays outside the trait. Add a differential test asserting all four read shapes return identical bytes across page-boundary and file-tail ranges. Tracking: rustfs/backlog#891 (parent rustfs/backlog#897) Co-Authored-By: heihutu <heihutu@gmail.com> * feat(ecstore): true O_DIRECT read path with per-disk graceful fallback (#4366) * feat(ecstore): true O_DIRECT read path with per-disk graceful fallback Implement a real O_DIRECT positioned read inside StdBackend::pread_bytes (Linux only) and wire up the previously dead RUSTFS_OBJECT_DIRECT_IO_READ_* knobs, which had zero call sites. Open with rustix OFlags::DIRECT, probe the DIO alignment once per disk via statx STATX_DIOALIGN (4096 fallback), read the aligned superset into an alignment-allocated bounce buffer with a short-read loop, and slice out the exact logical range so padding never reaches BitrotReader. Any O_DIRECT failure falls back to the buffered read methods; EINVAL or EOPNOTSUPP (tmpfs, overlayfs, 9p) latches the path off per disk with one warning. O_DIRECT errors never surface: EINVAL maps to FileNotFound in to_file_error and would trigger spurious EC rebuilds. Default behavior is unchanged (knob off). macOS keeps F_NOCACHE. Tracking: rustfs/backlog#892 (parent rustfs/backlog#897) Co-Authored-By: heihutu <heihutu@gmail.com> * test(ecstore): add P1.5 benchmark gate harness for O_DIRECT reads (#4369) * test(ecstore): add P1.5 benchmark gate harness for O_DIRECT reads Add an ignored, Linux-only release-mode test (direct_read_bench_gate) that measures DiskAPI::read_file_mmap_copy through a real LocalDisk with cold-cache enforcement (fadvise DONTNEED between rounds), reporting p50/p95/p99/mean latency, wall time, process CPU time, and throughput as one JSON line for A/B diffing between the buffered baseline and the O_DIRECT candidate selected by the production env knobs. Content is verified byte-for-byte before any timing starts. Tracking: rustfs/backlog#893 (parent rustfs/backlog#897) Co-Authored-By: heihutu <heihutu@gmail.com> * test(ecstore): add concurrency dimension to P1.5 bench harness Model the EC GET shape (FuturesUnordered over concurrent shard reads) via RUSTFS_BENCH_CONCURRENCY (default 1, sequential as before). Needed to evaluate blocking-pool pressure for the io_uring gate decision. Tracking: rustfs/backlog#893 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> * ci: retrigger after cancelled required check Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): use is_multiple_of in O_DIRECT bench cache-drop check clippy's manual_is_multiple_of (rust 1.96) fails -D warnings on the benchmark helper's `done % file_count == 0`. Verification: - cargo clippy -p rustfs-ecstore --all-targets Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team