refactor: extract startup runtime hook boundary (#3638)

docs/architecture/migration-progress.md

@@ -5,17 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
 ## Current Context
 
 - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
-- Branch: `overtrue/arch-protocol-runtime-sidecar-boundary`
+- Branch: `overtrue/arch-startup-runtime-hooks-boundary`
 - Baseline: `origin/main`
-  (`aa5de1c908d6ffb778533c895c200f3c60ca1ed5`).
-- Stacked on: rustfs/rustfs#3636.
+  (`5e96151d9c57cc8764f187cc0838dc111ddf0636`).
+- Stacked on: rustfs/rustfs#3637.
 - PR type for this branch: `pure-move`
 - Runtime behavior changes: none.
-- Rust code changes: move optional runtime sidecar ownership and protocol
-  shutdown planning from `startup_optional_runtimes` into
-  `startup_optional_runtime_sidecars`.
+- Rust code changes: add `startup_runtime_hooks` and move runtime diagnostics,
+  profiling hook dispatch, and crypto provider installation out of
+  `startup_runtime` and `startup_profiling`.
 - CI/script changes: none.
-- Docs changes: record the R-026 optional runtime sidecar boundary slice.
+- Docs changes: record the R-027 startup runtime hook boundary slice.
 
 ## Phase 0 Tasks
 
@@ -2110,6 +2110,20 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
     risk scan, branch freshness check, pre-commit quality gate, and
     three-expert review.
 
+- [x] `R-027` Extract startup runtime hook boundary.
+  - Do: add `startup_runtime_hooks` and move startup runtime diagnostics,
+    profiling hook dispatch, shutdown profiling dispatch, and default crypto
+    provider installation out of `startup_runtime` and `startup_profiling`.
+  - Acceptance: BOOT-006 keeps diagnostics, profiling init, trusted proxy init,
+    provider install, and outbound TLS material load in the same order, while
+    STOP-004 still stops profiling through the existing compatibility path.
+  - Must preserve: startup logo and telemetry/license log behavior, profiling
+    hook dispatch behavior, rustls provider install behavior, trusted proxy init
+    order, outbound TLS fatal boundary, and profiling shutdown call path.
+  - Verification: focused runtime hook/profiling/runtime/shutdown tests, RustFS
+    lib check, migration/layer guards, formatting, diff hygiene, Rust risk scan,
+    branch freshness check, pre-commit quality gate, and three-expert review.
+
 ## Next PRs
 
 1. `pure-move`: continue larger lifecycle hook slices for startup profiling and
@@ -2121,9 +2135,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
 
 | Expert | Status | Notes |
 |---|---|---|
-| Quality/architecture | passed | R-026 moves optional runtime sidecar ownership into `startup_optional_runtime_sidecars` while keeping the old module path as a compatibility handoff. |
-| Migration preservation | passed | Protocol startup order, shutdown planning order, and HTTP-before-optional-runtime shutdown ordering remain unchanged. |
-| Testing/verification | passed | Focused optional runtime sidecar/runtime/shutdown checks, guards, final hygiene, and full pre-commit passed. |
+| Quality/architecture | passed | R-027 moves runtime hook side effects into `startup_runtime_hooks` while keeping old profiling entrypoints as compatibility handoffs. |
+| Migration preservation | passed | BOOT-006 and STOP-004 ordering remain unchanged, including trusted proxy and outbound TLS ordering. |
+| Testing/verification | passed | Focused runtime hook/profiling/runtime/shutdown checks, guards, final hygiene, and full pre-commit passed. |
 
 ## Verification Notes
 
@@ -2234,6 +2248,21 @@ Passed before push:
   - `make pre-commit`: passed.
   - Three-expert review: passed.
 
+- Issue #660 R-027 current slice:
+  - `cargo test -p rustfs --lib startup_runtime_hooks -- --nocapture`:
+    passed.
+  - `cargo test -p rustfs --lib startup_profiling -- --nocapture`: passed.
+  - `cargo test -p rustfs --lib startup_runtime -- --nocapture`: passed.
+  - `cargo test -p rustfs --lib startup_shutdown -- --nocapture`: passed.
+  - `cargo check -p rustfs --lib`: passed.
+  - `./scripts/check_architecture_migration_rules.sh`: passed.
+  - `./scripts/check_layer_dependencies.sh`: passed.
+  - `cargo fmt --all --check`: passed.
+  - `git diff --check`: passed.
+  - Rust risk scan on changed Rust files: passed.
+  - `make pre-commit`: passed.
+  - Three-expert review: passed.
+
 - Issue #660 R-020 current slice:
   - `cargo test -p rustfs --lib startup_profiling -- --nocapture`: passed.
   - `cargo check -p rustfs --lib`: passed.

docs/architecture/runtime-lifecycle.md

@@ -65,6 +65,15 @@ sidecars must enter this boundary with explicit shutdown handles and status
 snapshots instead of adding ad hoc startup or shutdown work to
 `startup_services`.
 
+## Startup Runtime Hook Boundary
+
+`startup_runtime_hooks` owns runtime hook side effects that wrap the startup
+foundation but are not TLS material loading: startup diagnostics, profiling
+hook dispatch, and default crypto provider installation. `startup_runtime`
+preserves BOOT-006 orchestration and outbound TLS fatal behavior, while
+`startup_profiling` remains a compatibility handoff for the old profiling hook
+path.
+
 ## AppContext Foundation
 
 Early AppContext work should split resolver files and add compatibility tests before

docs/architecture/startup-timeline.md

@@ -23,7 +23,7 @@ new startup semantics.
 | `BOOT-003` | `rustfs/src/startup_entrypoint.rs` | Parse CLI command and dispatch non-server commands. | `info` and `tls` commands execute and return without server startup. | Command parse exits process with code 1; TLS command errors propagate. | None |
 | `BOOT-004` | `rustfs/src/startup_preflight.rs` | Initialize config snapshot and license state. | Publishes config snapshot for later readers and initializes runtime license state. | License init is non-fallible in this path. | None |
 | `BOOT-005` | `rustfs/src/startup_preflight.rs` | Initialize observability and store the global guard. | Initializes tracing/observability, stores the guard globally, and logs license/runtime telemetry status. | Fatal if observability init or guard publication fails. | None |
-| `BOOT-006` | `rustfs/src/startup_runtime.rs` and `rustfs/src/startup_profiling.rs` | Log startup logo, initialize profiling, trusted proxies, rustls provider, and outbound TLS material. | Starts optional profiling tasks, trusted proxy config, default rustls provider, outbound TLS global state, TLS generation metric, and TLS metrics when enabled. | Profiling/proxy/provider setup is non-fatal; configured TLS material load is fatal on error. | None |
+| `BOOT-006` | `rustfs/src/startup_runtime.rs`, `rustfs/src/startup_runtime_hooks.rs`, and `rustfs/src/startup_profiling.rs` | Log startup logo, initialize profiling, trusted proxies, rustls provider, and outbound TLS material. | Starts optional profiling tasks, trusted proxy config, default rustls provider, outbound TLS global state, TLS generation metric, and TLS metrics when enabled. | Profiling/proxy/provider setup is non-fatal; configured TLS material load is fatal on error. | None |
 | `RUN-001` | `rustfs/src/startup_server.rs` | Enter startup run orchestration and create `GlobalReadiness`. | Allocates the readiness tracker shared with HTTP readiness gates. | Non-fatal. | Initial readiness state is not ready |
 | `RUN-002` | `rustfs/src/startup_server.rs` | Parse and publish the configured region. | Updates ECStore global region when configured. | Fatal if the configured region is invalid. | None |
 | `RUN-003` | `rustfs/src/startup_server.rs` | Resolve server address and warn on default credentials. | Computes server port/address and emits production credential warning when defaults are used. | Address parse is fatal; default credentials warning is non-fatal. | None |
@@ -72,7 +72,7 @@ new startup semantics.
 | `STOP-001` | `rustfs/src/startup_shutdown.rs` | Cancel runtime token and move service state to `Stopping`. | Notifies cancellation-aware background tasks. | Non-fatal. | Service state moves to `Stopping`; readiness stages are not cleared here |
 | `STOP-002` | `rustfs/src/startup_shutdown.rs` | Stop scanner/background services and AHM services according to enable flags. | Calls ECStore background shutdown and heal/scanner shutdown helpers. | Non-fatal in this path. | No readiness-stage change |
 | `STOP-003` | `rustfs/src/startup_optional_runtime_sidecars.rs` | Plan optional runtime shutdown and log stopping state for FTP/FTPS/WebDAV/SFTP protocol servers. | Collects protocol shutdown handles. | Non-fatal in this path. | No readiness-stage change |
-| `STOP-004` | `rustfs/src/startup_shutdown.rs` and `rustfs/src/startup_profiling.rs` | Stop event notifier, audit system, and profiling tasks. | Stops notifier and profiling tasks; audit stop failures are logged. | Non-fatal in this path. | No readiness-stage change |
+| `STOP-004` | `rustfs/src/startup_shutdown.rs`, `rustfs/src/startup_runtime_hooks.rs`, and `rustfs/src/startup_profiling.rs` | Stop event notifier, audit system, and profiling tasks. | Stops notifier and profiling tasks; audit stop failures are logged. | Non-fatal in this path. | No readiness-stage change |
 | `STOP-005` | `rustfs/src/startup_shutdown.rs` and `rustfs/src/startup_optional_runtime_sidecars.rs` | Stop S3 and console HTTP servers, signal and wait for optional protocol shutdowns, then mark service state `Stopped`. | HTTP shutdown happens after notifier/audit/profiling shutdown in current order. | Join failures are logged by shutdown handles; this path does not return errors. | Service state moves to `Stopped`; readiness stages are not cleared here |
 
 ## Migration Rules

rustfs/src/lib.rs

@@ -78,6 +78,7 @@ pub mod startup_preflight;
 pub mod startup_profiling;
 pub mod startup_protocols;
 pub mod startup_runtime;
+pub mod startup_runtime_hooks;
 pub mod startup_server;
 pub mod startup_service_components;
 pub mod startup_services;

rustfs/src/startup_profiling.rs

@@ -12,60 +12,4 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-use std::future::Future;
-
-pub async fn init_profiling_runtime() {
-    init_profiling_runtime_with(crate::profiling::init_from_env).await;
-}
-
-async fn init_profiling_runtime_with<InitFn, InitFuture>(init: InitFn)
-where
-    InitFn: FnOnce() -> InitFuture,
-    InitFuture: Future<Output = ()>,
-{
-    init().await;
-}
-
-pub fn shutdown_profiling_runtime() {
-    shutdown_profiling_runtime_with(crate::profiling::shutdown_profiling);
-}
-
-fn shutdown_profiling_runtime_with<ShutdownFn>(shutdown: ShutdownFn)
-where
-    ShutdownFn: FnOnce(),
-{
-    shutdown();
-}
-
-#[cfg(test)]
-mod tests {
-    use super::{init_profiling_runtime_with, shutdown_profiling_runtime_with};
-    use std::sync::{
-        Arc,
-        atomic::{AtomicBool, Ordering},
-    };
-
-    #[tokio::test]
-    async fn init_profiling_runtime_invokes_registered_hook() {
-        let called = Arc::new(AtomicBool::new(false));
-        let hook_called = called.clone();
-
-        init_profiling_runtime_with(move || async move {
-            hook_called.store(true, Ordering::SeqCst);
-        })
-        .await;
-
-        assert!(called.load(Ordering::SeqCst));
-    }
-
-    #[test]
-    fn shutdown_profiling_runtime_invokes_registered_hook() {
-        let called = AtomicBool::new(false);
-
-        shutdown_profiling_runtime_with(|| {
-            called.store(true, Ordering::SeqCst);
-        });
-
-        assert!(called.load(Ordering::SeqCst));
-    }
-}
+pub use crate::startup_runtime_hooks::{init_profiling_runtime, shutdown_profiling_runtime};

rustfs/src/startup_runtime.rs

@@ -12,80 +12,27 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-use crate::{config::Config, license::license_status};
-use rustls::crypto::aws_lc_rs::default_provider;
+use crate::{
+    config::Config,
+    startup_runtime_hooks::{init_profiling_runtime, install_default_crypto_provider, log_startup_runtime_diagnostics},
+};
 use std::io::{Error, Result};
-use tracing::{debug, error, info};
+use tracing::{error, info};
 
 const LOG_COMPONENT_MAIN: &str = "main";
-const LOG_SUBSYSTEM_LICENSE: &str = "license";
 const LOG_SUBSYSTEM_STARTUP: &str = "startup";
-const EVENT_CRYPTO_PROVIDER_STATE: &str = "crypto_provider_state";
-const EVENT_DIAL9_RUNTIME_STATUS: &str = "dial9_runtime_status";
-const EVENT_RUNTIME_LICENSE_STATUS: &str = "runtime_license_status";
 const EVENT_TLS_OUTBOUND_INITIALIZED: &str = "tls_outbound_initialized";
 const EVENT_TLS_OUTBOUND_INITIALIZATION_FAILED: &str = "tls_outbound_initialization_failed";
 const TLS_STARTUP_GENERATION_CONSUMER: &str = "rustfs_server_startup";
 
 pub async fn init_startup_runtime_foundation(config: &Config) -> Result<()> {
-    log_dial9_runtime_status();
-    log_runtime_license_status();
-
-    debug!("{}", crate::server::LOGO);
-
-    crate::startup_profiling::init_profiling_runtime().await;
+    log_startup_runtime_diagnostics();
+    init_profiling_runtime().await;
     rustfs_trusted_proxies::init();
     install_default_crypto_provider();
     init_outbound_tls_material(config).await
 }
 
-fn log_dial9_runtime_status() {
-    if rustfs_obs::dial9::is_enabled() {
-        info!(
-            target: "rustfs::main",
-            event = EVENT_DIAL9_RUNTIME_STATUS,
-            component = LOG_COMPONENT_MAIN,
-            subsystem = LOG_SUBSYSTEM_STARTUP,
-            enabled = true,
-            "Dial9 Tokio runtime telemetry is enabled"
-        );
-    } else {
-        debug!(
-            target: "rustfs::main",
-            event = EVENT_DIAL9_RUNTIME_STATUS,
-            component = LOG_COMPONENT_MAIN,
-            subsystem = LOG_SUBSYSTEM_STARTUP,
-            enabled = false,
-            "Dial9 Tokio runtime telemetry is disabled"
-        );
-    }
-}
-
-fn log_runtime_license_status() {
-    info!(
-        target: "rustfs::main",
-        event = EVENT_RUNTIME_LICENSE_STATUS,
-        component = LOG_COMPONENT_MAIN,
-        subsystem = LOG_SUBSYSTEM_LICENSE,
-        license_status = %license_status(),
-        "Initialized runtime license state"
-    );
-}
-
-fn install_default_crypto_provider() {
-    if default_provider().install_default().is_err() {
-        debug!(
-            target: "rustfs::main",
-            event = EVENT_CRYPTO_PROVIDER_STATE,
-            component = LOG_COMPONENT_MAIN,
-            subsystem = LOG_SUBSYSTEM_STARTUP,
-            provider = "aws_lc_rs",
-            state = "already_installed",
-            "Rustls crypto provider state checked"
-        );
-    }
-}
-
 async fn init_outbound_tls_material(config: &Config) -> Result<()> {
     if let Some(tls_path) = normalized_tls_path(config.tls_path.as_deref()) {
         match crate::server::tls_material::load_tls_material(tls_path).await {

rustfs/src/startup_runtime_hooks.rs

@@ -0,0 +1,134 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use crate::license::license_status;
+use rustls::crypto::aws_lc_rs::default_provider;
+use std::future::Future;
+use tracing::{debug, info};
+
+const LOG_COMPONENT_MAIN: &str = "main";
+const LOG_SUBSYSTEM_LICENSE: &str = "license";
+const LOG_SUBSYSTEM_STARTUP: &str = "startup";
+const EVENT_CRYPTO_PROVIDER_STATE: &str = "crypto_provider_state";
+const EVENT_DIAL9_RUNTIME_STATUS: &str = "dial9_runtime_status";
+const EVENT_RUNTIME_LICENSE_STATUS: &str = "runtime_license_status";
+
+pub fn log_startup_runtime_diagnostics() {
+    log_dial9_runtime_status();
+    log_runtime_license_status();
+    debug!("{}", crate::server::LOGO);
+}
+
+fn log_dial9_runtime_status() {
+    if rustfs_obs::dial9::is_enabled() {
+        info!(
+            target: "rustfs::main",
+            event = EVENT_DIAL9_RUNTIME_STATUS,
+            component = LOG_COMPONENT_MAIN,
+            subsystem = LOG_SUBSYSTEM_STARTUP,
+            enabled = true,
+            "Dial9 Tokio runtime telemetry is enabled"
+        );
+    } else {
+        debug!(
+            target: "rustfs::main",
+            event = EVENT_DIAL9_RUNTIME_STATUS,
+            component = LOG_COMPONENT_MAIN,
+            subsystem = LOG_SUBSYSTEM_STARTUP,
+            enabled = false,
+            "Dial9 Tokio runtime telemetry is disabled"
+        );
+    }
+}
+
+fn log_runtime_license_status() {
+    info!(
+        target: "rustfs::main",
+        event = EVENT_RUNTIME_LICENSE_STATUS,
+        component = LOG_COMPONENT_MAIN,
+        subsystem = LOG_SUBSYSTEM_LICENSE,
+        license_status = %license_status(),
+        "Initialized runtime license state"
+    );
+}
+
+pub async fn init_profiling_runtime() {
+    init_profiling_runtime_with(crate::profiling::init_from_env).await;
+}
+
+async fn init_profiling_runtime_with<InitFn, InitFuture>(init: InitFn)
+where
+    InitFn: FnOnce() -> InitFuture,
+    InitFuture: Future<Output = ()>,
+{
+    init().await;
+}
+
+pub fn shutdown_profiling_runtime() {
+    shutdown_profiling_runtime_with(crate::profiling::shutdown_profiling);
+}
+
+fn shutdown_profiling_runtime_with<ShutdownFn>(shutdown: ShutdownFn)
+where
+    ShutdownFn: FnOnce(),
+{
+    shutdown();
+}
+
+pub fn install_default_crypto_provider() {
+    if default_provider().install_default().is_err() {
+        debug!(
+            target: "rustfs::main",
+            event = EVENT_CRYPTO_PROVIDER_STATE,
+            component = LOG_COMPONENT_MAIN,
+            subsystem = LOG_SUBSYSTEM_STARTUP,
+            provider = "aws_lc_rs",
+            state = "already_installed",
+            "Rustls crypto provider state checked"
+        );
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{init_profiling_runtime_with, shutdown_profiling_runtime_with};
+    use std::sync::{
+        Arc,
+        atomic::{AtomicBool, Ordering},
+    };
+
+    #[tokio::test]
+    async fn init_profiling_runtime_invokes_registered_hook() {
+        let called = Arc::new(AtomicBool::new(false));
+        let hook_called = called.clone();
+
+        init_profiling_runtime_with(move || async move {
+            hook_called.store(true, Ordering::SeqCst);
+        })
+        .await;
+
+        assert!(called.load(Ordering::SeqCst));
+    }
+
+    #[test]
+    fn shutdown_profiling_runtime_invokes_registered_hook() {
+        let called = AtomicBool::new(false);
+
+        shutdown_profiling_runtime_with(|| {
+            called.store(true, Ordering::SeqCst);
+        });
+
+        assert!(called.load(Ordering::SeqCst));
+    }
+}