mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 16:28:15 +00:00

Files

T

Zhengchao An df9cbc4ed1 fix(ecstore): validate erasure distribution values to avoid shuffle index panic (#4427 )

fix(ecstore): validate erasure distribution values to avoid shuffle index panic (backlog#949)

The element values of `erasure.distribution` read from `xl.meta` were never
range-checked. `FileInfo::is_valid()` and `MetaObjectV1::valid()` only verified
`distribution.len()` and the `erasure.index` bound, not that each distribution
value is a valid 1-based slot in `[1, N]`. The metadata shuffle helpers then use
these values directly as `distribution[k] - 1` indices, so a corrupt or
adversarial `xl.meta` carrying a `0` (usize underflow) or a value greater than N
(out-of-bounds) triggers a panic in the shuffle path, turning bad-disk metadata
that erasure coding is meant to tolerate into a request/task crash.

Fix, two layers:
- Validate distribution values at metadata acceptance: `is_valid_distribution`
  now requires the distribution to be a permutation of `1..=N` (correct length,
  every value in range, no duplicates). `FileInfo::is_valid()` and
  `MetaObjectV1::valid()` use it, so `find_file_info_in_quorum` rejects corrupt
  metadata and it surfaces as a clean `ErasureReadQuorum` error instead of an
  index path.
- Defensive indexing in the shuffle helpers (`shuffle_disks_and_parts_metadata`,
  `_by_index`, `_by_index_owned`, `shuffle_parts_metadata`, `shuffle_disks`,
  `shuffle_check_parts`): out-of-range distribution values are skipped via
  `checked_sub(1)` + bounds-checked slot access instead of a bare `idx - 1`
  index, matching the existing pattern in
  `collect_inline_data_shard_fileinfos_by_index`.

Regression tests: `is_valid_distribution`/`is_valid`/`valid` reject distributions
containing `0`, values greater than N, duplicates, and wrong length while
accepting valid permutations; the shuffle helpers no longer panic on corrupt
distributions and preserve output length.

Refs: https://github.com/rustfs/backlog/issues/949

2026-07-08 15:02:16 +08:00

benches

perf(metadata): drop per-object heap allocs in internal-key lookups (#4260 )

2026-07-04 22:14:07 +08:00

examples

feat(rio): rio_v2 is compatible with minio for storing data. (#3115 )

2026-06-08 11:59:14 +00:00

src

fix(ecstore): validate erasure distribution values to avoid shuffle index panic (#4427 )

2026-07-08 15:02:16 +08:00

tests/fixtures

test(interop): add real MinIO read and migration parity tests (#4377 )

2026-07-08 01:12:49 +08:00

Cargo.toml

feat(observability): feature-gated hotpath instrumentation for the data path (#4394 )

2026-07-08 07:39:47 +08:00

README.md

fix(sse). Resolving Nonce Overwriting Issues in Multi-Package Scenarios (#2582 )

2026-04-18 14:00:14 +00:00

README.md

RustFS FileMeta - File Metadata Management

Advanced file metadata management and indexing module for RustFS distributed object storage

📖 Documentation · 🐛 Bug Reports · 💬 Discussions

📖 Overview

RustFS FileMeta provides advanced file metadata management and indexing capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

Quick Use

cargo run -p rustfs-filemeta --example dump_fileinfo -- "/path/to/file/xl.meta"

✨ Features

High-performance metadata storage and retrieval
Advanced indexing with full-text search capabilities
File attribute management and custom metadata
Version tracking and history management
Distributed metadata replication
Real-time metadata synchronization

📚 Documentation

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.