mirror of https://github.com/rustfs/rustfs.git synced 2026-08-13 16:46:55 +00:00

Files

T

houseme 8d34b4d101 perf(get): cache read descriptors in StdBackend (#1801 ) (#5965 )

StdBackend opened, stat'd, and access-checked the shard file on every
positioned read, so each small-object GET paid N x (open + access + fstat)
syscalls even for hot shards. UringBackend already caches descriptors
behind a generation-guarded, rlimit-bounded moka cache with full
rename/delete/heal invalidation; StdBackend had no equivalent.

Port that cache to the default backend:

- StdBackend gains an `fd_cache: Option<FdCache>` (Linux only, mirroring
  UringBackend), built in `new()` behind `RUSTFS_LOCAL_FD_CACHE` (default
  on) and the same `rlimit_allows_fd_cache` guard.
- pread_bytes consults the cache on the buffered path: a hit reuses the
  descriptor via `dup` (one syscall, no path resolution or permission
  re-check) and skips volume access; a miss opens as before, snapshots the
  invalidation generation, and hands the freshly opened descriptor back
  for `insert_if_fresh`, which refuses to cache if a heal/delete bumped the
  generation mid-open (rustfs/backlog#1176). O_DIRECT reads keep opening
  their own aligned descriptors.
- The DirectReadCopy branch switches from seek+read_exact to
  `FileExt::read_exact_at`: a `dup`'d cached descriptor shares the source
  descriptor's open-file offset, so a positioned read (like the mmap path's
  offset argument) keeps concurrent cache hits on the same shard correct.
- The four `LocalIoBackend` invalidation methods now drop stale entries on
  StdBackend. LocalDisk already calls them on rename_data/rename_file/
  delete/delete_volume/close, so no new call sites are needed.
- Two tests mirror the io_uring ones: a heal rename must be hidden until
  invalidate_cached_fds_under runs, and a repeated read caches exactly one
  descriptor that prefix invalidation drops.

Behavior is byte-for-byte unchanged on a miss and on non-Linux; the cache
is auto-disabled only when RLIMIT_NOFILE is too low. macOS cargo check
--lib and --tests pass; Linux compile deferred to CI.

Co-authored-by: heihutu <heihutu@gmail.com>

2026-08-12 01:38:07 +00:00

benches

refactor: segment residual storage api boundaries (#3905 )

2026-06-26 15:56:38 +08:00

docs

refactor: use canonical Rust module paths (#4269 )

2026-07-05 03:01:14 +08:00

src

perf(get): cache read descriptors in StdBackend (#1801 ) (#5965 )

2026-08-12 01:38:07 +00:00

tests

fix(replication): harden live delete admission (#5599 )

2026-08-02 12:52:11 +08:00

AGENTS.md

chore(agents): add Rust code quality rules and skill (#3144 )

2026-05-31 22:19:36 +08:00

build.rs

refactor: Restructure project layout and clean up dependencies (#30 )

2025-07-02 19:33:12 +08:00

Cargo.toml

fix(ecstore): anchor Windows rename publication (#5677 )

2026-08-09 22:56:37 +08:00

README.md

fix: revert standalone #2351 artifacts (phase 5) (#2535 )

2026-04-14 22:35:05 +08:00

run_benchmarks.sh

fix(get): remove GET chunk fast path (#2507 )

2026-04-12 23:12:23 +08:00

README.md

RustFS ECStore - Erasure Coding Storage

High-performance erasure coding storage engine for RustFS distributed object storage

📖 Documentation · 🐛 Bug Reports · 💬 Discussions

📖 Overview

RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features

Reed-Solomon erasure coding implementation
Configurable redundancy levels (N+K schemes)
Automatic data healing and reconstruction
Multi-drive support with intelligent placement
Parallel encoding/decoding for performance
Efficient disk space utilization

📚 Documentation

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Copyright 2024 RustFS Team

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.

Made with ❤️ by the RustFS Storage Team