mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00

Files

T

houseme 608ab14d7d perf(ecstore): fold metadata read open+fstat+read into a single spawn_blocking (HP-12 item 1) (#4554 )

Sub-change A only: the two local-disk metadata read paths
(`read_metadata_with_dmtime`, `read_all_data_with_dmtime` in
crates/ecstore/src/disk/local.rs) previously dispatched open, fstat and the
xl.meta read as three separate async fs hops (three spawn_blocking round-trips
under tokio::fs). Each is now folded into a single tokio::task::spawn_blocking
closure using std::fs, cutting per-metadata-read dispatch from 3 to 1.

This does NOT touch sub-change B (removing the entry-point access() call).

Correctness first: this is the hottest metadata read path, so the refactor
preserves byte-for-byte-equivalent Results. Every error mapping is kept
identical:
  - open failure     -> to_file_error
  - is_dir           -> Error::FileNotFound (not to_file_error(EISDIR))
  - metadata failure -> to_file_error
  - xl.meta parse    -> propagated verbatim from the parser
  - try_reserve      -> Error::other
  - read_to_end      -> to_file_error
For read_all_data_with_dmtime the async NotFound -> access(volume_dir) ->
VolumeNotFound fallback (and its warn! event) is preserved on the async side:
the closure returns the raw open error unmapped; only open() can yield ENOENT
once the fd is valid, so gating the fallback on the open error is equivalent to
the original open-arm-only fallback.

To run the parser inside a blocking closure, filemeta gains a synchronous twin
`read_xl_meta_no_data_sync` (+ `read_more_sync`) in
crates/filemeta/src/filemeta/version.rs that line-for-line mirrors the async
version, differing only in std vs tokio read_exact. A new equivalence test
(`read_xl_meta_sync_equivalence_tests`) feeds identical buffers to both the
async and sync readers and asserts equal Ok bytes / equal Err variants across:
v1.0; v1.1/v1.2/v1.3; large meta triggering read_more; header truncation ->
UnexpectedEof; CRC-trailer truncation -> FileCorrupt; unknown major/minor ->
InvalidData; and want boundaries (exact fit, inline-data drop, read_more EOF).

Co-authored-by: heihutu <heihutu@gmail.com>

2026-07-08 18:28:57 +00:00

benches

perf(metadata): drop per-object heap allocs in internal-key lookups (#4260 )

2026-07-04 22:14:07 +08:00

examples

feat(rio): rio_v2 is compatible with minio for storing data. (#3115 )

2026-06-08 11:59:14 +00:00

src

perf(ecstore): fold metadata read open+fstat+read into a single spawn_blocking (HP-12 item 1) (#4554 )

2026-07-08 18:28:57 +00:00

tests/fixtures

test(interop): add real MinIO read and migration parity tests (#4377 )

2026-07-08 01:12:49 +08:00

Cargo.toml

feat(observability): feature-gated hotpath instrumentation for the data path (#4394 )

2026-07-08 07:39:47 +08:00

README.md

fix(sse). Resolving Nonce Overwriting Issues in Multi-Package Scenarios (#2582 )

2026-04-18 14:00:14 +00:00

README.md

RustFS FileMeta - File Metadata Management

Advanced file metadata management and indexing module for RustFS distributed object storage

📖 Documentation · 🐛 Bug Reports · 💬 Discussions

📖 Overview

RustFS FileMeta provides advanced file metadata management and indexing capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

Quick Use

cargo run -p rustfs-filemeta --example dump_fileinfo -- "/path/to/file/xl.meta"

✨ Features

High-performance metadata storage and retrieval
Advanced indexing with full-text search capabilities
File attribute management and custom metadata
Version tracking and history management
Distributed metadata replication
Real-time metadata synchronization

📚 Documentation

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.