mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-02 11:29:17 +00:00
Zhengchao An
3bc8d79fe5
fix(multipart): serialize the part commit per uploadId to prevent mixed-generation shards (backlog#853) Concurrently re-transmitting the same part could land two shard generations across different disks: each shard is individually bitrot-valid, so the corruption surfaces only as a silent mix at read time. The hazard is confined to rename_part, where two temp parts are moved cross-disk onto the SAME final part path — interleaving there can leave shards from two generations. Each concurrent stream already writes to its own unique temp dir, so the encode/stream phase never conflicts and must stay lock-free: holding a lock across it would serialize slow re-transmits of the same part, break the S3 'last finisher wins' semantics, and cause UploadPart lock-acquire timeouts (ServiceUnavailable). Scope a write lock to the uploadId namespace around only the rename_part commit, so each commit is atomic across disks and the last committer wins consistently. Mirrors MinIO's per-uploadID NS lock; disjoint from the object lock held by complete_multipart_upload (no lock-ordering cycle). Honors opts.no_lock. The pre-delete-before-rename half of backlog#853 was already fixed in #4316; this completes the issue. Adds an end-to-end regression test: six concurrent resends of the same part must all succeed (no lock-acquire timeout), exactly one generation stays visible, and the reassembled object equals one intact generation. Closes rustfs/backlog#853
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team