mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 16:28:15 +00:00
houseme
2df315baf0
* fix(ecstore): fsync new object's ancestor dirs to close a power-loss gap reliable_mkdir_all creates an object's directory (and any missing prefix dirs) with plain mkdir and never fsyncs the parent chain. The commit-point fsync in rename_data persists the object dir's *contents* (its xl.meta and data dir), but not the object dir's own entry in the bucket/prefix directory. So on the first PUT of an object, a power loss after the write is acknowledged could drop the whole object directory even though its contents were durable — an acknowledged write silently lost (rustfs/backlog#922 step 4). For a new object (no prior xl.meta) under a durability tier that syncs commit metadata, fsync the ancestor chain from the object dir's parent up to and including the bucket after the commit rename, so the newly created directory entries survive power loss. A starts_with guard bounds the walk to the bucket subtree. Overwrites already have a durable object dir and are unaffected; relaxed/none accept the wider window like the existing commit fsync. Durability regressions are invisible to ordinary behavior tests, so the new tests assert directly (via the fsync_dir recorder) that a first PUT under a new prefix fsyncs both the prefix and bucket dirs, and that relaxed does not. Scope: the non-inline (erasure) commit path. The inline branch has the same gap and is a separate follow-up. Refs: rustfs/backlog#922 (HP-1 step 4), rustfs/backlog#936 Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): fsync new inline object's ancestor dirs too Extend the backlog#922 step 4 mkdir-gap fix to the inline commit branch. Like the non-inline path, a first PUT of an inline object creates its directory (and any missing prefix dirs) whose entry in the parent chain reliable_mkdir_all never fsynced; the commit fsync persists the object dir's contents, not its own entry. For a new inline object under a commit-metadata-syncing tier, fsync the ancestor chain up to and including the bucket after the commit rename, using the same starts_with-bounded walk (via the synchronous os::fsync_dir_std inside the inline spawn_blocking closure). Adds a test asserting a new inline object under a new prefix fsyncs both the prefix and bucket dirs. rename_data now closes the ack'd-write power-loss gap on both the erasure and inline paths. Refs: rustfs/backlog#922 (HP-1 step 4), rustfs/backlog#936 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team