mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-28 16:07:05 +00:00
houseme
59fd318192
* feat(mimalloc): add arena diagnostics and configuration Based on mimalloc maintainer feedback (microsoft/mimalloc#1372), add diagnostics to check mimalloc arena configuration at runtime. Changes: - Add rustfs-mimalloc-sys to workspace dependencies - Add log_mimalloc_diagnostics() function to check: - arena_max_object_size - pagemap_commit status - mimalloc version - Add memory_observability module with mimalloc diagnostics This helps diagnose why allocations might be going outside arenas, which is the suspected root cause of futex contention. Ref: rustfs/backlog#2005 Ref: microsoft/mimalloc#1372 Co-Authored-By: heihutu <heihutu@gmail.com> * perf(ecstore): add Vec<u8> buffer pool for EC operations Add a general-purpose buffer pool to reduce Vec<u8> allocations in hot paths like EC encoding/decoding. Changes: - Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs - Thread-safe pool with capacity-based bucketing (power-of-two) - Global EC_BUFFER_POOL instance with 16 buffers per bucket - Add buffer_pool module to codec/mod.rs Expected impact: - Reduce heap allocations in EC encode/decode paths - Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool) - Reduce mimalloc lock contention Note: Main bottleneck remains mimalloc internal synchronization (futex 98.64% time). Buffer pool provides modest improvement (+2-5%). Ref: rustfs/backlog#2005 Co-Authored-By: heihutu <heihutu@gmail.com> * style: apply cargo fmt to buffer pool and related files Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): add #[allow(dead_code)] to buffer pool The BufferPool infrastructure is ready but not yet integrated into the EC hot paths. Add #[allow(dead_code)] with clear documentation about integration status. Co-Authored-By: heihutu <heihutu@gmail.com> * perf(ecstore): integrate BufferPool into bitrot verify path Replace vec![0; shard_size] with get_ec_buffer() in the bitrot verification hot path to reduce heap allocations and avoid memzero. Co-Authored-By: heihutu <heihutu@gmail.com> * style: apply cargo fmt to buffer pool and bitrot changes Co-Authored-By: heihutu <heihutu@gmail.com> * refactor(ecstore): clean up buffer pool code - Remove unnecessary #[allow(dead_code)] attributes - Update module documentation to reflect current integration status - Simplify code structure Co-Authored-By: heihutu <heihutu@gmail.com> * perf(runtime): cap default worker threads at 16 Testing showed 16 worker threads outperforms 32+ for 1KiB PUT workloads due to reduced mimalloc lock contention. A/B test results (testing 4-node cluster, c=64): - worker_threads=32: 740 obj/s (baseline) - worker_threads=16: 785 obj/s (+6.1%) The default was detect_cores() which returned 32 on our testing nodes. Cap at 16 for optimal small-object performance. Ref: rustfs/backlog#2005 Co-Authored-By: heihutu <heihutu@gmail.com> * style: apply cargo fmt to buffer pool and runtime changes Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): remove unused BufferPool::new() function The new() function was never used since EC_BUFFER_POOL initializes directly with with_limits(16). Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): update buffer_pool tests to use with_limits Replace BufferPool::new() with BufferPool::with_limits(16) in tests since new() was removed in favor of with_limits(). Co-Authored-By: heihutu <heihutu@gmail.com> * perf(ecstore): optimize opts.clone() and FileInfo clone patterns ## Changes 1. ObjectOptions helper methods: - add as_commit_opts(): creates commit options with no_lock=true, metadata_cache_safe=false, include_part_checksums=true - add as_read_opts(): creates read options with include_part_checksums=true - add with_no_lock(): creates options with modified no_lock field 2. Replace opts.clone() in hot paths: - commit_opts = opts.as_commit_opts() (was 4-line manual clone) - read_opts = opts.as_read_opts() (was 2-line manual clone) 3. Optimize FileInfo clone in rename path: - avoid double clone: clone once and modify erasure.index in place - pass &file_info reference to rename_data_borrowed_with_fence ## A/B Results (4-node cluster, c=64) | Size | main | optimized | Change | |------|------|-----------|--------| | 1KiB | 892 obj/s | 920-976 obj/s | +3%~+9% | | 4KiB | 957 obj/s | 903 obj/s | -5.7% | | 16KiB | 922 obj/s | 855 obj/s | -7.3% | Note: 1KiB improvement is consistent. 4KiB/16KiB variance likely due to test noise; needs more rounds to confirm. Ref: rustfs/backlog#2005 Co-Authored-By: heihutu <heihutu@gmail.com> * perf(ecstore): add BytesMut buffer pool to EC encoding path Pre-allocate a Vec<BytesMut> pool in the EC encoding loop to avoid repeated heap allocations for ingest buffers. Changes: - Pre-allocate buffer pool with capacity 4 - Reuse buffers from pool after encoding - Return buffers to pool when capacity is sufficient Expected impact: +10-20% in EC encoding path by reducing BytesMut allocation overhead. Ref: rustfs/backlog#2005 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: hector <hetor@rustfs.com> Co-authored-by: heihutu <heihutu@gmail.com>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team