mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 16:28:15 +00:00
refactor: extract optional runtime sidecar boundary (#3637)
This commit is contained in:
@@ -5,16 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
## Current Context
|
||||
|
||||
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
|
||||
- Branch: `overtrue/arch-startup-service-components-boundary`
|
||||
- Baseline: `overtrue/arch-startup-ready-lifecycle-boundary`
|
||||
(`a518f40489515c1ec37c3cbc5738e04955a942f2`).
|
||||
- Stacked on: rustfs/rustfs#3635.
|
||||
- Branch: `overtrue/arch-protocol-runtime-sidecar-boundary`
|
||||
- Baseline: `origin/main`
|
||||
(`aa5de1c908d6ffb778533c895c200f3c60ca1ed5`).
|
||||
- Stacked on: rustfs/rustfs#3636.
|
||||
- PR type for this branch: `pure-move`
|
||||
- Runtime behavior changes: none.
|
||||
- Rust code changes: move startup service component initialization helpers from
|
||||
`startup_services` into a dedicated startup service component boundary.
|
||||
- Rust code changes: move optional runtime sidecar ownership and protocol
|
||||
shutdown planning from `startup_optional_runtimes` into
|
||||
`startup_optional_runtime_sidecars`.
|
||||
- CI/script changes: none.
|
||||
- Docs changes: record the R-025 startup service component boundary slice.
|
||||
- Docs changes: record the R-026 optional runtime sidecar boundary slice.
|
||||
|
||||
## Phase 0 Tasks
|
||||
|
||||
@@ -2094,10 +2095,25 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
risk scan, branch freshness check, pre-commit quality gate, and
|
||||
three-expert review.
|
||||
|
||||
- [x] `R-026` Extract optional runtime sidecar boundary.
|
||||
- Do: add `startup_optional_runtime_sidecars` and move optional runtime
|
||||
sidecar ownership, shutdown planning, shutdown execution, and protocol
|
||||
shutdown order tests out of `startup_optional_runtimes`.
|
||||
- Acceptance: optional protocol startup still happens after KMS and before
|
||||
buffer profiling, while shutdown planning still records FTP, FTPS, WebDAV,
|
||||
then SFTP handles before later shutdown signaling.
|
||||
- Must preserve: feature-gated protocol startup behavior, disabled-protocol
|
||||
handling, protocol shutdown ordering, HTTP shutdown before optional protocol
|
||||
shutdown signaling, and the compatibility `startup_optional_runtimes` API.
|
||||
- Verification: focused optional runtime sidecar/runtime/shutdown tests,
|
||||
RustFS lib check, migration/layer guards, formatting, diff hygiene, Rust
|
||||
risk scan, branch freshness check, pre-commit quality gate, and
|
||||
three-expert review.
|
||||
|
||||
## Next PRs
|
||||
|
||||
1. `pure-move`: continue larger lifecycle hook slices for optional runtime
|
||||
sidecars while preserving startup and shutdown ordering.
|
||||
1. `pure-move`: continue larger lifecycle hook slices for startup profiling and
|
||||
diagnostics while preserving startup and shutdown ordering.
|
||||
2. `contract`: continue extension contract coverage for future diagnostics and
|
||||
profiler handoff surfaces after runtime owners are stable.
|
||||
|
||||
@@ -2105,9 +2121,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
|
||||
| Expert | Status | Notes |
|
||||
|---|---|---|
|
||||
| Quality/architecture | passed | R-025 keeps `startup_services` as the orchestrator and moves component helpers into `startup_service_components`. |
|
||||
| Migration preservation | passed | Runtime service startup order and embedded helper reuse remain unchanged. |
|
||||
| Testing/verification | passed | Focused component, service, lifecycle, final hygiene, and full pre-commit checks passed. |
|
||||
| Quality/architecture | passed | R-026 moves optional runtime sidecar ownership into `startup_optional_runtime_sidecars` while keeping the old module path as a compatibility handoff. |
|
||||
| Migration preservation | passed | Protocol startup order, shutdown planning order, and HTTP-before-optional-runtime shutdown ordering remain unchanged. |
|
||||
| Testing/verification | passed | Focused optional runtime sidecar/runtime/shutdown checks, guards, final hygiene, and full pre-commit passed. |
|
||||
|
||||
## Verification Notes
|
||||
|
||||
@@ -2203,6 +2219,21 @@ Passed before push:
|
||||
- `make pre-commit`: passed.
|
||||
- Three-expert review: passed.
|
||||
|
||||
- Issue #660 R-026 current slice:
|
||||
- `cargo test -p rustfs --lib startup_optional_runtime_sidecars -- --nocapture`:
|
||||
passed.
|
||||
- `cargo test -p rustfs --lib startup_optional_runtimes -- --nocapture`:
|
||||
passed.
|
||||
- `cargo test -p rustfs --lib startup_shutdown -- --nocapture`: passed.
|
||||
- `cargo check -p rustfs --lib`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- Rust risk scan on changed Rust files: passed.
|
||||
- `make pre-commit`: passed.
|
||||
- Three-expert review: passed.
|
||||
|
||||
- Issue #660 R-020 current slice:
|
||||
- `cargo test -p rustfs --lib startup_profiling -- --nocapture`: passed.
|
||||
- `cargo check -p rustfs --lib`: passed.
|
||||
|
||||
@@ -57,11 +57,13 @@ these helpers.
|
||||
|
||||
## Optional Runtime Boundary
|
||||
|
||||
`startup_optional_runtimes` owns startup and shutdown handoff for optional
|
||||
runtime services that are not readiness boundaries. The current owner set is
|
||||
protocol servers only. Future optional sidecars must enter this boundary with
|
||||
explicit shutdown handles and status snapshots instead of adding ad hoc startup
|
||||
or shutdown work to `startup_services`.
|
||||
`startup_optional_runtime_sidecars` owns startup handles, shutdown planning, and
|
||||
shutdown execution for optional runtime services that are not readiness
|
||||
boundaries. `startup_optional_runtimes` remains a compatibility handoff for the
|
||||
old module path. The current owner set is protocol servers only. Future optional
|
||||
sidecars must enter this boundary with explicit shutdown handles and status
|
||||
snapshots instead of adding ad hoc startup or shutdown work to
|
||||
`startup_services`.
|
||||
|
||||
## AppContext Foundation
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ new startup semantics.
|
||||
| `RUN-011` | `rustfs/src/startup_storage.rs` | Create cancellation token and initialize `ECStore`. | Creates the runtime cancellation token and storage engine. | Fatal if `ECStore::new` fails. | None |
|
||||
| `RUN-012` | `rustfs/src/startup_storage.rs` | Initialize ECStore config and global config system. | Initializes ECStore config, attempts server-config migration, then retries global config init up to 15 times. | Migration attempt is non-fatal in this path; global config init becomes fatal after retries. | Marks the `GlobalReadiness` `StorageReady` stage after global config init succeeds; later runtime readiness still rechecks storage, IAM, and lock quorum before `FullReady` |
|
||||
| `RUN-013` | `rustfs/src/startup_storage.rs` and `rustfs/src/startup_services.rs` | Start replication and KMS systems. | Starts background replication pool, then initializes KMS from startup services. | Replication init is non-fatal in this path; KMS init is fatal on error. | `StorageReady` stage is already marked; dynamic runtime storage readiness is still checked before `FullReady` |
|
||||
| `RUN-014` | `rustfs/src/startup_optional_runtimes.rs` and `rustfs/src/startup_protocols.rs` | Initialize optional protocol servers. | Starts FTP/FTPS/WebDAV/SFTP when feature-enabled and configured, collecting shutdown handles. | Feature-enabled protocol init is fatal on error; disabled protocols are non-fatal. | None |
|
||||
| `RUN-014` | `rustfs/src/startup_optional_runtime_sidecars.rs` and `rustfs/src/startup_protocols.rs` | Initialize optional protocol servers. | Starts FTP/FTPS/WebDAV/SFTP when feature-enabled and configured, collecting shutdown handles. | Feature-enabled protocol init is fatal on error; disabled protocols are non-fatal. | None |
|
||||
| `RUN-015` | `rustfs/src/startup_services.rs` and `rustfs/src/startup_service_components.rs` | Initialize buffer profiling, event notifier, audit, and deadlock detector. | Starts buffer profile system, event notifier, audit system, and optional deadlock detector. | Audit startup failure is logged and non-fatal; the others are non-fatal in this path. | None |
|
||||
| `RUN-016` | `rustfs/src/startup_service_components.rs` | List buckets and run bucket/replication/IAM metadata migrations. | Reads bucket names, migrates bucket metadata, initializes replication resync, migrates IAM config, and initializes bucket metadata system. | Bucket list and replication resync are fatal on error; metadata migration calls are non-fatal in this path. | Storage remains ready; IAM not yet ready |
|
||||
| `RUN-017` | `rustfs/src/startup_service_components.rs` and `rustfs/src/startup_iam.rs` | Bootstrap IAM inline or defer recovery. | Initializes IAM when possible; otherwise starts the deferred IAM recovery path through `startup_iam`. | Fatal only when `bootstrap_or_defer_iam_init` returns an unrecoverable error. | Inline success marks `IamReady`; deferred mode publishes `IamReady` later from the recovery task |
|
||||
@@ -71,9 +71,9 @@ new startup semantics.
|
||||
|---|---|---|---|---|---|
|
||||
| `STOP-001` | `rustfs/src/startup_shutdown.rs` | Cancel runtime token and move service state to `Stopping`. | Notifies cancellation-aware background tasks. | Non-fatal. | Service state moves to `Stopping`; readiness stages are not cleared here |
|
||||
| `STOP-002` | `rustfs/src/startup_shutdown.rs` | Stop scanner/background services and AHM services according to enable flags. | Calls ECStore background shutdown and heal/scanner shutdown helpers. | Non-fatal in this path. | No readiness-stage change |
|
||||
| `STOP-003` | `rustfs/src/startup_optional_runtimes.rs` | Plan optional runtime shutdown and log stopping state for FTP/FTPS/WebDAV/SFTP protocol servers. | Collects protocol shutdown handles. | Non-fatal in this path. | No readiness-stage change |
|
||||
| `STOP-003` | `rustfs/src/startup_optional_runtime_sidecars.rs` | Plan optional runtime shutdown and log stopping state for FTP/FTPS/WebDAV/SFTP protocol servers. | Collects protocol shutdown handles. | Non-fatal in this path. | No readiness-stage change |
|
||||
| `STOP-004` | `rustfs/src/startup_shutdown.rs` and `rustfs/src/startup_profiling.rs` | Stop event notifier, audit system, and profiling tasks. | Stops notifier and profiling tasks; audit stop failures are logged. | Non-fatal in this path. | No readiness-stage change |
|
||||
| `STOP-005` | `rustfs/src/startup_shutdown.rs` and `rustfs/src/startup_optional_runtimes.rs` | Stop S3 and console HTTP servers, signal and wait for optional protocol shutdowns, then mark service state `Stopped`. | HTTP shutdown happens after notifier/audit/profiling shutdown in current order. | Join failures are logged by shutdown handles; this path does not return errors. | Service state moves to `Stopped`; readiness stages are not cleared here |
|
||||
| `STOP-005` | `rustfs/src/startup_shutdown.rs` and `rustfs/src/startup_optional_runtime_sidecars.rs` | Stop S3 and console HTTP servers, signal and wait for optional protocol shutdowns, then mark service state `Stopped`. | HTTP shutdown happens after notifier/audit/profiling shutdown in current order. | Join failures are logged by shutdown handles; this path does not return errors. | Service state moves to `Stopped`; readiness stages are not cleared here |
|
||||
|
||||
## Migration Rules
|
||||
|
||||
|
||||
@@ -72,6 +72,7 @@ pub mod startup_entrypoint;
|
||||
pub mod startup_fs_guard;
|
||||
pub mod startup_iam;
|
||||
pub mod startup_lifecycle;
|
||||
pub mod startup_optional_runtime_sidecars;
|
||||
pub mod startup_optional_runtimes;
|
||||
pub mod startup_preflight;
|
||||
pub mod startup_profiling;
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
// Copyright 2024 RustFS Team
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::{
|
||||
server::ShutdownHandle,
|
||||
startup_protocols::{ProtocolShutdownSenders, init_protocol_shutdown_senders},
|
||||
};
|
||||
use futures_util::future::join_all;
|
||||
use std::io::Result;
|
||||
use tracing::info;
|
||||
|
||||
const LOG_COMPONENT_MAIN: &str = "main";
|
||||
const LOG_SUBSYSTEM_STARTUP: &str = "startup";
|
||||
const EVENT_PROTOCOL_SYSTEM_STATE: &str = "protocol_system_state";
|
||||
|
||||
pub struct OptionalRuntimeServices {
|
||||
pub protocols: ProtocolShutdownSenders,
|
||||
}
|
||||
|
||||
impl OptionalRuntimeServices {
|
||||
pub fn new(protocols: ProtocolShutdownSenders) -> Self {
|
||||
Self { protocols }
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn init_optional_runtime_services() -> Result<OptionalRuntimeServices> {
|
||||
let protocols = init_protocol_shutdown_senders().await?;
|
||||
Ok(OptionalRuntimeServices::new(protocols))
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum OptionalRuntimeShutdownStep {
|
||||
Ftp,
|
||||
Ftps,
|
||||
Webdav,
|
||||
Sftp,
|
||||
}
|
||||
|
||||
impl OptionalRuntimeShutdownStep {
|
||||
const fn protocol(self) -> &'static str {
|
||||
match self {
|
||||
Self::Ftp => "ftp",
|
||||
Self::Ftps => "ftps",
|
||||
Self::Webdav => "webdav",
|
||||
Self::Sftp => "sftp",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn optional_runtime_shutdown_steps(protocols: &ProtocolShutdownSenders) -> Vec<OptionalRuntimeShutdownStep> {
|
||||
let mut steps = Vec::with_capacity(4);
|
||||
if protocols.ftp.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Ftp);
|
||||
}
|
||||
if protocols.ftps.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Ftps);
|
||||
}
|
||||
if protocols.webdav.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Webdav);
|
||||
}
|
||||
if protocols.sftp.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Sftp);
|
||||
}
|
||||
steps
|
||||
}
|
||||
|
||||
pub fn prepare_optional_runtime_shutdowns(optional_runtimes: OptionalRuntimeServices) -> Vec<ShutdownHandle> {
|
||||
let ProtocolShutdownSenders { ftp, ftps, webdav, sftp } = optional_runtimes.protocols;
|
||||
|
||||
let mut protocol_shutdowns = Vec::new();
|
||||
if let Some(handle) = ftp {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Ftp);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
if let Some(handle) = ftps {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Ftps);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
if let Some(handle) = webdav {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Webdav);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
if let Some(handle) = sftp {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Sftp);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
|
||||
protocol_shutdowns
|
||||
}
|
||||
|
||||
pub async fn shutdown_optional_runtime_services(protocol_shutdowns: Vec<ShutdownHandle>) {
|
||||
join_all(protocol_shutdowns.into_iter().map(ShutdownHandle::shutdown)).await;
|
||||
}
|
||||
|
||||
fn log_protocol_stopping(step: OptionalRuntimeShutdownStep) {
|
||||
info!(
|
||||
target: "rustfs::main::handle_shutdown",
|
||||
event = EVENT_PROTOCOL_SYSTEM_STATE,
|
||||
component = LOG_COMPONENT_MAIN,
|
||||
subsystem = LOG_SUBSYSTEM_STARTUP,
|
||||
protocol = step.protocol(),
|
||||
state = "stopping",
|
||||
"Protocol runtime stopping"
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
OptionalRuntimeServices, OptionalRuntimeShutdownStep, optional_runtime_shutdown_steps,
|
||||
prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services,
|
||||
};
|
||||
use crate::{server::ShutdownHandle, startup_protocols::ProtocolShutdownSenders};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use tokio::sync::broadcast;
|
||||
|
||||
fn completed_shutdown_handle() -> ShutdownHandle {
|
||||
let (tx, _rx) = broadcast::channel(1);
|
||||
ShutdownHandle::new(tx, tokio::spawn(async {}))
|
||||
}
|
||||
|
||||
fn observed_shutdown_handle(label: &'static str, events: Arc<Mutex<Vec<&'static str>>>) -> ShutdownHandle {
|
||||
let (tx, mut rx) = broadcast::channel(1);
|
||||
let task_handle = tokio::spawn(async move {
|
||||
let _ = rx.recv().await;
|
||||
events.lock().unwrap_or_else(|err| err.into_inner()).push(label);
|
||||
});
|
||||
ShutdownHandle::new(tx, task_handle)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_runtime_shutdown_plan_preserves_protocol_order() {
|
||||
let protocols = ProtocolShutdownSenders {
|
||||
ftp: Some(completed_shutdown_handle()),
|
||||
ftps: Some(completed_shutdown_handle()),
|
||||
webdav: Some(completed_shutdown_handle()),
|
||||
sftp: Some(completed_shutdown_handle()),
|
||||
};
|
||||
|
||||
assert_eq!(
|
||||
optional_runtime_shutdown_steps(&protocols),
|
||||
vec![
|
||||
OptionalRuntimeShutdownStep::Ftp,
|
||||
OptionalRuntimeShutdownStep::Ftps,
|
||||
OptionalRuntimeShutdownStep::Webdav,
|
||||
OptionalRuntimeShutdownStep::Sftp,
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_runtime_shutdown_signals_registered_protocols() {
|
||||
let events = Arc::new(Mutex::new(Vec::new()));
|
||||
let protocols = ProtocolShutdownSenders {
|
||||
ftp: Some(observed_shutdown_handle("ftp", events.clone())),
|
||||
ftps: None,
|
||||
webdav: Some(observed_shutdown_handle("webdav", events.clone())),
|
||||
sftp: None,
|
||||
};
|
||||
|
||||
let protocol_shutdowns = prepare_optional_runtime_shutdowns(OptionalRuntimeServices::new(protocols));
|
||||
tokio::task::yield_now().await;
|
||||
assert!(events.lock().unwrap_or_else(|err| err.into_inner()).is_empty());
|
||||
|
||||
shutdown_optional_runtime_services(protocol_shutdowns).await;
|
||||
|
||||
let mut events = events.lock().unwrap_or_else(|err| err.into_inner()).clone();
|
||||
events.sort_unstable();
|
||||
assert_eq!(events, ["ftp", "webdav"]);
|
||||
}
|
||||
}
|
||||
@@ -12,172 +12,7 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::{
|
||||
server::ShutdownHandle,
|
||||
startup_protocols::{ProtocolShutdownSenders, init_protocol_shutdown_senders},
|
||||
pub use crate::startup_optional_runtime_sidecars::{
|
||||
OptionalRuntimeServices, init_optional_runtime_services, prepare_optional_runtime_shutdowns,
|
||||
shutdown_optional_runtime_services,
|
||||
};
|
||||
use futures_util::future::join_all;
|
||||
use std::io::Result;
|
||||
use tracing::info;
|
||||
|
||||
const LOG_COMPONENT_MAIN: &str = "main";
|
||||
const LOG_SUBSYSTEM_STARTUP: &str = "startup";
|
||||
const EVENT_PROTOCOL_SYSTEM_STATE: &str = "protocol_system_state";
|
||||
|
||||
pub struct OptionalRuntimeServices {
|
||||
pub protocols: ProtocolShutdownSenders,
|
||||
}
|
||||
|
||||
impl OptionalRuntimeServices {
|
||||
pub fn new(protocols: ProtocolShutdownSenders) -> Self {
|
||||
Self { protocols }
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn init_optional_runtime_services() -> Result<OptionalRuntimeServices> {
|
||||
let protocols = init_protocol_shutdown_senders().await?;
|
||||
Ok(OptionalRuntimeServices::new(protocols))
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum OptionalRuntimeShutdownStep {
|
||||
Ftp,
|
||||
Ftps,
|
||||
Webdav,
|
||||
Sftp,
|
||||
}
|
||||
|
||||
impl OptionalRuntimeShutdownStep {
|
||||
const fn protocol(self) -> &'static str {
|
||||
match self {
|
||||
Self::Ftp => "ftp",
|
||||
Self::Ftps => "ftps",
|
||||
Self::Webdav => "webdav",
|
||||
Self::Sftp => "sftp",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn optional_runtime_shutdown_steps(protocols: &ProtocolShutdownSenders) -> Vec<OptionalRuntimeShutdownStep> {
|
||||
let mut steps = Vec::with_capacity(4);
|
||||
if protocols.ftp.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Ftp);
|
||||
}
|
||||
if protocols.ftps.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Ftps);
|
||||
}
|
||||
if protocols.webdav.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Webdav);
|
||||
}
|
||||
if protocols.sftp.is_some() {
|
||||
steps.push(OptionalRuntimeShutdownStep::Sftp);
|
||||
}
|
||||
steps
|
||||
}
|
||||
|
||||
pub fn prepare_optional_runtime_shutdowns(optional_runtimes: OptionalRuntimeServices) -> Vec<ShutdownHandle> {
|
||||
let ProtocolShutdownSenders { ftp, ftps, webdav, sftp } = optional_runtimes.protocols;
|
||||
|
||||
let mut protocol_shutdowns = Vec::new();
|
||||
if let Some(handle) = ftp {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Ftp);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
if let Some(handle) = ftps {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Ftps);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
if let Some(handle) = webdav {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Webdav);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
if let Some(handle) = sftp {
|
||||
log_protocol_stopping(OptionalRuntimeShutdownStep::Sftp);
|
||||
protocol_shutdowns.push(handle);
|
||||
}
|
||||
|
||||
protocol_shutdowns
|
||||
}
|
||||
|
||||
pub async fn shutdown_optional_runtime_services(protocol_shutdowns: Vec<ShutdownHandle>) {
|
||||
join_all(protocol_shutdowns.into_iter().map(ShutdownHandle::shutdown)).await;
|
||||
}
|
||||
|
||||
fn log_protocol_stopping(step: OptionalRuntimeShutdownStep) {
|
||||
info!(
|
||||
target: "rustfs::main::handle_shutdown",
|
||||
event = EVENT_PROTOCOL_SYSTEM_STATE,
|
||||
component = LOG_COMPONENT_MAIN,
|
||||
subsystem = LOG_SUBSYSTEM_STARTUP,
|
||||
protocol = step.protocol(),
|
||||
state = "stopping",
|
||||
"Protocol runtime stopping"
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
OptionalRuntimeServices, OptionalRuntimeShutdownStep, optional_runtime_shutdown_steps,
|
||||
prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services,
|
||||
};
|
||||
use crate::{server::ShutdownHandle, startup_protocols::ProtocolShutdownSenders};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use tokio::sync::broadcast;
|
||||
|
||||
fn completed_shutdown_handle() -> ShutdownHandle {
|
||||
let (tx, _rx) = broadcast::channel(1);
|
||||
ShutdownHandle::new(tx, tokio::spawn(async {}))
|
||||
}
|
||||
|
||||
fn observed_shutdown_handle(label: &'static str, events: Arc<Mutex<Vec<&'static str>>>) -> ShutdownHandle {
|
||||
let (tx, mut rx) = broadcast::channel(1);
|
||||
let task_handle = tokio::spawn(async move {
|
||||
let _ = rx.recv().await;
|
||||
events.lock().unwrap_or_else(|err| err.into_inner()).push(label);
|
||||
});
|
||||
ShutdownHandle::new(tx, task_handle)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_runtime_shutdown_plan_preserves_protocol_order() {
|
||||
let protocols = ProtocolShutdownSenders {
|
||||
ftp: Some(completed_shutdown_handle()),
|
||||
ftps: Some(completed_shutdown_handle()),
|
||||
webdav: Some(completed_shutdown_handle()),
|
||||
sftp: Some(completed_shutdown_handle()),
|
||||
};
|
||||
|
||||
assert_eq!(
|
||||
optional_runtime_shutdown_steps(&protocols),
|
||||
vec![
|
||||
OptionalRuntimeShutdownStep::Ftp,
|
||||
OptionalRuntimeShutdownStep::Ftps,
|
||||
OptionalRuntimeShutdownStep::Webdav,
|
||||
OptionalRuntimeShutdownStep::Sftp,
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_runtime_shutdown_signals_registered_protocols() {
|
||||
let events = Arc::new(Mutex::new(Vec::new()));
|
||||
let protocols = ProtocolShutdownSenders {
|
||||
ftp: Some(observed_shutdown_handle("ftp", events.clone())),
|
||||
ftps: None,
|
||||
webdav: Some(observed_shutdown_handle("webdav", events.clone())),
|
||||
sftp: None,
|
||||
};
|
||||
|
||||
let protocol_shutdowns = prepare_optional_runtime_shutdowns(OptionalRuntimeServices::new(protocols));
|
||||
tokio::task::yield_now().await;
|
||||
assert!(events.lock().unwrap_or_else(|err| err.into_inner()).is_empty());
|
||||
|
||||
shutdown_optional_runtime_services(protocol_shutdowns).await;
|
||||
|
||||
let mut events = events.lock().unwrap_or_else(|err| err.into_inner()).clone();
|
||||
events.sort_unstable();
|
||||
assert_eq!(events, ["ftp", "webdav"]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -18,7 +18,7 @@ use crate::{
|
||||
init::{init_buffer_profile_system, init_kms_system},
|
||||
server::ServiceStateManager,
|
||||
startup_iam::IamBootstrapDisposition,
|
||||
startup_optional_runtimes::{OptionalRuntimeServices, init_optional_runtime_services},
|
||||
startup_optional_runtime_sidecars::{OptionalRuntimeServices, init_optional_runtime_services},
|
||||
startup_service_components::{
|
||||
init_audit_runtime, init_auth_integrations, init_background_service_runtime, init_bucket_metadata_runtime,
|
||||
init_deadlock_detector_runtime, init_iam_runtime, init_notification_runtime, init_observability_runtime,
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
|
||||
use crate::{
|
||||
server::{ServiceState, ServiceStateManager, ShutdownHandle, ShutdownSignal, shutdown_event_notifier, stop_audit_system},
|
||||
startup_optional_runtimes::{
|
||||
startup_optional_runtime_sidecars::{
|
||||
OptionalRuntimeServices, prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services,
|
||||
},
|
||||
storage_compat::shutdown_background_services,
|
||||
|
||||
Reference in New Issue
Block a user