mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-05 19:55:37 +00:00
Sergey Shlukov
1747ed0292
* fix(ecstore): stop scan_dir emitting entries past a limit hit inside a subdirectory scan_dir's flush loop recurses into a pending subdirectory when the current sibling entry's page limit is reached mid-recursion, but kept writing the current (later-sorting) entry regardless. gather_results then builds the next page's continuation marker from that later entry, which permanently skips the still-unscanned tail of the subdirectory on resume instead of just deferring it to the next page. Add a limit re-check right after the flush loop, before the current entry is written, so scan_dir stops cleanly at the true last-written key. Reproduces and fixes the rc.5 recursive ListObjectsV2 data-loss report (7826/7881 keys, contiguous 55-key block silently dropped). Adds scan_dir_does_not_emit_entries_past_a_limit_hit_inside_a_subdirectory. * fix(ecstore): re-check the page limit on every dir_stack flush iteration The flush loop that drains dir_stack can pop and recurse into more than one pending subdirectory per outer iteration (whenever more than one stack entry sorts below the current sibling entry). The limit re-check added in the previous commit only ran once, after the whole flush loop exited - so if the first recursive scan_dir call already exhausted the page limit, the loop's next pop+recurse still went ahead and scanned (and emitted entries for) another subdirectory beyond where the page was supposed to stop. Confirmed against production data: a bucket with ~1.17M objects under one prefix still cut a recursive ListObjectsV2 listing short (825 of an expected much larger next page, IsTruncated=false) even with the first fix deployed, at a two-level-nested subdirectory. Move the check inside the while loop so it runs before every pop, not just once after. --------- Co-authored-by: Claude Agent <agent@local>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team