mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-28 00:58:59 +00:00
houseme
f73880f354
* fix(server): make startup readiness wait configurable and raise default (#4264) The startup runtime-readiness wait was a hardcoded 30s constant with no env override. On slow multi-node cold starts (Docker/K8s/Synology NAS) this window is shorter than the internal startup budgets it depends on — the endpoint DNS-retry window (~90s) and the format-load retry loop (~100s worst case) — so readiness times out and the node exits with `startup readiness timed out after 30s: storage_ready=false, lock_quorum_ready=false` before storage/lock quorum can converge, feeding the restart storm reported in the issue. - Add `RUSTFS_STARTUP_READINESS_MAX_WAIT_SECS` (default 120s), documented in rustfs-config health constants. - Resolve the wait at runtime via `startup_runtime_readiness_max_wait()`; a value of `0` falls back to the default instead of timing out instantly. - Repoint `STARTUP_RUNTIME_READINESS_MAX_WAIT` at the shared config default so there is a single source of truth, and cover the getter with unit tests. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): stop peer/disk background monitors on graceful shutdown (#4264) Long-lived peer health/recovery and remote-disk monitors are detached `tokio::spawn` tasks that each hold a `tracing::Span` via `.instrument(..)` for their whole lifetime. Nothing cancelled them at shutdown, so on the normal return path the Tokio runtime was dropped while they were still alive and their `Span`s were dropped during worker-thread thread-local-storage (TLS) destruction. At that point `tracing-subscriber`'s fmt `on_close` can touch an already-destroyed TLS slot and panic with `cannot access a Thread Local Storage value during or after destruction`, which escalates to a panic-during-panic abort (SIGILL / exit 132) — the crash reported on Synology in issue #4264, amplified by the restart storm. - Add `cluster::rpc::background_monitor` with a process-global shutdown token, `spawn_background_monitor()` (races the monitor future against that token so its span drops while the runtime is alive), and public `shutdown_background_monitors()`. - Route every span-holding peer_s3 / peer_rest / remote_disk monitor spawn through `spawn_background_monitor` instead of `tokio::spawn(..).instrument()`. - Expose `rustfs_ecstore::shutdown_background_monitors()` and call it from the graceful shutdown sequence (right after `ctx.cancel()`, before runtime teardown) via the `storage_api` compatibility boundary. Existing recovery-probe span-context tests still pass, confirming log correlation is preserved. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
RustFS ECStore - Erasure Coding Storage
High-performance erasure coding storage engine for RustFS distributed object storage
📖 Documentation
· 🐛 Bug Reports
· 💬 Discussions
📖 Overview
RustFS ECStore provides erasure coding storage capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.
✨ Features
- Reed-Solomon erasure coding implementation
- Configurable redundancy levels (N+K schemes)
- Automatic data healing and reconstruction
- Multi-drive support with intelligent placement
- Parallel encoding/decoding for performance
- Efficient disk space utilization
📚 Documentation
For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.
📄 License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 RustFS Team
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
RustFS is a trademark of RustFS, Inc.
All other trademarks are the property of their respective owners.
Made with ❤️ by the RustFS Storage Team