refactor: extract startup tls material boundary (#3639)

docs/architecture/migration-progress.md

@@ -5,17 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
 ## Current Context
 
 - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
-- Branch: `overtrue/arch-startup-runtime-hooks-boundary`
+- Branch: `overtrue/arch-startup-tls-material-boundary`
 - Baseline: `origin/main`
-  (`5e96151d9c57cc8764f187cc0838dc111ddf0636`).
-- Stacked on: rustfs/rustfs#3637.
+  (`46a02b6d03e2d8274222e7ef63ac2eb5d9124c69`).
+- Stacked on: rustfs/rustfs#3638.
 - PR type for this branch: `pure-move`
 - Runtime behavior changes: none.
-- Rust code changes: add `startup_runtime_hooks` and move runtime diagnostics,
-  profiling hook dispatch, and crypto provider installation out of
-  `startup_runtime` and `startup_profiling`.
+- Rust code changes: add `startup_tls_material` and move outbound TLS material
+  loading, generation recording, and TLS metrics initialization out of
+  `startup_runtime`.
 - CI/script changes: none.
-- Docs changes: record the R-027 startup runtime hook boundary slice.
+- Docs changes: record the R-028 startup TLS material boundary slice.
 
 ## Phase 0 Tasks
 
@@ -2124,6 +2124,20 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
     lib check, migration/layer guards, formatting, diff hygiene, Rust risk scan,
     branch freshness check, pre-commit quality gate, and three-expert review.
 
+- [x] `R-028` Extract startup TLS material boundary.
+  - Do: add `startup_tls_material` and move outbound TLS material loading,
+    global TLS publication, generation recording, TLS metrics initialization,
+    and existing TLS path/generation tests out of `startup_runtime`.
+  - Acceptance: BOOT-006 keeps diagnostics, profiling init, trusted proxy init,
+    provider install, and outbound TLS material load in the same order.
+  - Must preserve: configured TLS material fatal behavior, TLS path trimming,
+    saturating TLS generation behavior, outbound TLS global state publication,
+    generation metric recording, and metrics initialization when observability
+    metrics are enabled.
+  - Verification: focused TLS material/runtime tests, RustFS lib check,
+    migration/layer guards, formatting, diff hygiene, Rust risk scan, branch
+    freshness check, pre-commit quality gate, and three-expert review.
+
 ## Next PRs
 
 1. `pure-move`: continue larger lifecycle hook slices for startup profiling and
@@ -2135,14 +2149,27 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
 
 | Expert | Status | Notes |
 |---|---|---|
-| Quality/architecture | passed | R-027 moves runtime hook side effects into `startup_runtime_hooks` while keeping old profiling entrypoints as compatibility handoffs. |
-| Migration preservation | passed | BOOT-006 and STOP-004 ordering remain unchanged, including trusted proxy and outbound TLS ordering. |
-| Testing/verification | passed | Focused runtime hook/profiling/runtime/shutdown checks, guards, final hygiene, and full pre-commit passed. |
+| Quality/architecture | passed | R-028 moves outbound TLS material ownership into `startup_tls_material` while keeping `startup_runtime` as the BOOT-006 orchestrator. |
+| Migration preservation | passed | BOOT-006 ordering, configured TLS fatal behavior, path trimming, generation saturation, publication, and metrics behavior remain unchanged. |
+| Testing/verification | passed | Focused TLS material/runtime checks, guards, final hygiene, Rust risk scan, and full pre-commit passed. |
 
 ## Verification Notes
 
 Passed before push:
 
+- Issue #660 R-028 current slice:
+  - `cargo test -p rustfs --lib startup_tls_material -- --nocapture`: passed.
+  - `cargo test -p rustfs --lib startup_runtime -- --nocapture`: passed.
+  - `cargo check -p rustfs --lib`: passed.
+  - `./scripts/check_architecture_migration_rules.sh`: passed.
+  - `./scripts/check_layer_dependencies.sh`: passed.
+  - `cargo fmt --all --check`: passed.
+  - `git diff --check`: passed.
+  - Rust risk scan on changed Rust files: passed; only existing doc-example
+    `println!` in `rustfs/src/lib.rs`.
+  - `make pre-commit`: passed; 6304 tests passed, 111 skipped, doctests
+    passed.
+
 - Issue #660 X-012 current slice:
   - `cargo test -p rustfs-extension-schema`: passed.
   - `cargo check -p rustfs-extension-schema`: passed.

docs/architecture/runtime-lifecycle.md

@@ -74,6 +74,13 @@ preserves BOOT-006 orchestration and outbound TLS fatal behavior, while
 `startup_profiling` remains a compatibility handoff for the old profiling hook
 path.
 
+## Startup TLS Material Boundary
+
+`startup_tls_material` owns configured outbound TLS material loading, global
+outbound TLS publication, generation recording, and TLS metrics initialization.
+`startup_runtime` still owns BOOT-006 ordering and must preserve the fatal
+boundary when configured TLS material fails to load.
+
 ## AppContext Foundation
 
 Early AppContext work should split resolver files and add compatibility tests before

docs/architecture/startup-timeline.md

@@ -23,7 +23,7 @@ new startup semantics.
 | `BOOT-003` | `rustfs/src/startup_entrypoint.rs` | Parse CLI command and dispatch non-server commands. | `info` and `tls` commands execute and return without server startup. | Command parse exits process with code 1; TLS command errors propagate. | None |
 | `BOOT-004` | `rustfs/src/startup_preflight.rs` | Initialize config snapshot and license state. | Publishes config snapshot for later readers and initializes runtime license state. | License init is non-fallible in this path. | None |
 | `BOOT-005` | `rustfs/src/startup_preflight.rs` | Initialize observability and store the global guard. | Initializes tracing/observability, stores the guard globally, and logs license/runtime telemetry status. | Fatal if observability init or guard publication fails. | None |
-| `BOOT-006` | `rustfs/src/startup_runtime.rs`, `rustfs/src/startup_runtime_hooks.rs`, and `rustfs/src/startup_profiling.rs` | Log startup logo, initialize profiling, trusted proxies, rustls provider, and outbound TLS material. | Starts optional profiling tasks, trusted proxy config, default rustls provider, outbound TLS global state, TLS generation metric, and TLS metrics when enabled. | Profiling/proxy/provider setup is non-fatal; configured TLS material load is fatal on error. | None |
+| `BOOT-006` | `rustfs/src/startup_runtime.rs`, `rustfs/src/startup_runtime_hooks.rs`, `rustfs/src/startup_tls_material.rs`, and `rustfs/src/startup_profiling.rs` | Log startup logo, initialize profiling, trusted proxies, rustls provider, and outbound TLS material. | Starts optional profiling tasks, trusted proxy config, default rustls provider, outbound TLS global state, TLS generation metric, and TLS metrics when enabled. | Profiling/proxy/provider setup is non-fatal; configured TLS material load is fatal on error. | None |
 | `RUN-001` | `rustfs/src/startup_server.rs` | Enter startup run orchestration and create `GlobalReadiness`. | Allocates the readiness tracker shared with HTTP readiness gates. | Non-fatal. | Initial readiness state is not ready |
 | `RUN-002` | `rustfs/src/startup_server.rs` | Parse and publish the configured region. | Updates ECStore global region when configured. | Fatal if the configured region is invalid. | None |
 | `RUN-003` | `rustfs/src/startup_server.rs` | Resolve server address and warn on default credentials. | Computes server port/address and emits production credential warning when defaults are used. | Address parse is fatal; default credentials warning is non-fatal. | None |

rustfs/src/lib.rs

@@ -84,6 +84,7 @@ pub mod startup_service_components;
 pub mod startup_services;
 pub mod startup_shutdown;
 pub mod startup_storage;
+pub mod startup_tls_material;
 pub mod storage;
 pub(crate) mod storage_compat;
 pub(crate) mod table_catalog;

rustfs/src/startup_runtime.rs

@@ -15,15 +15,9 @@
 use crate::{
     config::Config,
     startup_runtime_hooks::{init_profiling_runtime, install_default_crypto_provider, log_startup_runtime_diagnostics},
+    startup_tls_material::init_outbound_tls_material,
 };
-use std::io::{Error, Result};
-use tracing::{error, info};
-
-const LOG_COMPONENT_MAIN: &str = "main";
-const LOG_SUBSYSTEM_STARTUP: &str = "startup";
-const EVENT_TLS_OUTBOUND_INITIALIZED: &str = "tls_outbound_initialized";
-const EVENT_TLS_OUTBOUND_INITIALIZATION_FAILED: &str = "tls_outbound_initialization_failed";
-const TLS_STARTUP_GENERATION_CONSUMER: &str = "rustfs_server_startup";
+use std::io::Result;
 
 pub async fn init_startup_runtime_foundation(config: &Config) -> Result<()> {
     log_startup_runtime_diagnostics();
@@ -32,74 +26,3 @@ pub async fn init_startup_runtime_foundation(config: &Config) -> Result<()> {
     install_default_crypto_provider();
     init_outbound_tls_material(config).await
 }
-
-async fn init_outbound_tls_material(config: &Config) -> Result<()> {
-    if let Some(tls_path) = normalized_tls_path(config.tls_path.as_deref()) {
-        match crate::server::tls_material::load_tls_material(tls_path).await {
-            Ok(snapshot) => {
-                use rustfs_tls_runtime::{publish_global_outbound_tls_state, record_tls_generation};
-
-                let generation = next_tls_generation(rustfs_common::get_global_outbound_tls_generation());
-                publish_global_outbound_tls_state(generation, &snapshot.outbound).await;
-                record_tls_generation(TLS_STARTUP_GENERATION_CONSUMER, generation.0);
-                info!(
-                    target: "rustfs::main",
-                    event = EVENT_TLS_OUTBOUND_INITIALIZED,
-                    component = LOG_COMPONENT_MAIN,
-                    subsystem = LOG_SUBSYSTEM_STARTUP,
-                    tls_path,
-                    generation = generation.0,
-                    "Initialized TLS outbound material"
-                );
-            }
-            Err(err) => {
-                error!(
-                    target: "rustfs::main",
-                    event = EVENT_TLS_OUTBOUND_INITIALIZATION_FAILED,
-                    component = LOG_COMPONENT_MAIN,
-                    subsystem = LOG_SUBSYSTEM_STARTUP,
-                    tls_path,
-                    error = %err,
-                    "Failed to initialize TLS outbound material"
-                );
-                return Err(Error::other(err.to_string()));
-            }
-        }
-        if rustfs_obs::observability_metric_enabled() {
-            rustfs_tls_runtime::init_tls_metrics();
-        }
-    }
-
-    Ok(())
-}
-
-fn normalized_tls_path(path: Option<&str>) -> Option<&str> {
-    path.map(str::trim).filter(|value| !value.is_empty())
-}
-
-fn next_tls_generation(current: u64) -> rustfs_tls_runtime::TlsGeneration {
-    rustfs_tls_runtime::TlsGeneration(current.saturating_add(1))
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn normalized_tls_path_ignores_empty_values() {
-        assert_eq!(normalized_tls_path(None), None);
-        assert_eq!(normalized_tls_path(Some("")), None);
-        assert_eq!(normalized_tls_path(Some("   ")), None);
-    }
-
-    #[test]
-    fn normalized_tls_path_trims_configured_path() {
-        assert_eq!(normalized_tls_path(Some(" /tmp/rustfs-tls ")), Some("/tmp/rustfs-tls"));
-    }
-
-    #[test]
-    fn next_tls_generation_saturates() {
-        assert_eq!(next_tls_generation(0).0, 1);
-        assert_eq!(next_tls_generation(u64::MAX).0, u64::MAX);
-    }
-}

rustfs/src/startup_tls_material.rs

@@ -0,0 +1,94 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use crate::config::Config;
+use std::io::{Error, Result};
+use tracing::{error, info};
+
+const LOG_COMPONENT_MAIN: &str = "main";
+const LOG_SUBSYSTEM_STARTUP: &str = "startup";
+const EVENT_TLS_OUTBOUND_INITIALIZED: &str = "tls_outbound_initialized";
+const EVENT_TLS_OUTBOUND_INITIALIZATION_FAILED: &str = "tls_outbound_initialization_failed";
+const TLS_STARTUP_GENERATION_CONSUMER: &str = "rustfs_server_startup";
+
+pub(crate) async fn init_outbound_tls_material(config: &Config) -> Result<()> {
+    if let Some(tls_path) = normalized_tls_path(config.tls_path.as_deref()) {
+        match crate::server::tls_material::load_tls_material(tls_path).await {
+            Ok(snapshot) => {
+                use rustfs_tls_runtime::{publish_global_outbound_tls_state, record_tls_generation};
+
+                let generation = next_tls_generation(rustfs_common::get_global_outbound_tls_generation());
+                publish_global_outbound_tls_state(generation, &snapshot.outbound).await;
+                record_tls_generation(TLS_STARTUP_GENERATION_CONSUMER, generation.0);
+                info!(
+                    target: "rustfs::main",
+                    event = EVENT_TLS_OUTBOUND_INITIALIZED,
+                    component = LOG_COMPONENT_MAIN,
+                    subsystem = LOG_SUBSYSTEM_STARTUP,
+                    tls_path,
+                    generation = generation.0,
+                    "Initialized TLS outbound material"
+                );
+            }
+            Err(err) => {
+                error!(
+                    target: "rustfs::main",
+                    event = EVENT_TLS_OUTBOUND_INITIALIZATION_FAILED,
+                    component = LOG_COMPONENT_MAIN,
+                    subsystem = LOG_SUBSYSTEM_STARTUP,
+                    tls_path,
+                    error = %err,
+                    "Failed to initialize TLS outbound material"
+                );
+                return Err(Error::other(err.to_string()));
+            }
+        }
+        if rustfs_obs::observability_metric_enabled() {
+            rustfs_tls_runtime::init_tls_metrics();
+        }
+    }
+
+    Ok(())
+}
+
+fn normalized_tls_path(path: Option<&str>) -> Option<&str> {
+    path.map(str::trim).filter(|value| !value.is_empty())
+}
+
+fn next_tls_generation(current: u64) -> rustfs_tls_runtime::TlsGeneration {
+    rustfs_tls_runtime::TlsGeneration(current.saturating_add(1))
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn normalized_tls_path_ignores_empty_values() {
+        assert_eq!(normalized_tls_path(None), None);
+        assert_eq!(normalized_tls_path(Some("")), None);
+        assert_eq!(normalized_tls_path(Some("   ")), None);
+    }
+
+    #[test]
+    fn normalized_tls_path_trims_configured_path() {
+        assert_eq!(normalized_tls_path(Some(" /tmp/rustfs-tls ")), Some("/tmp/rustfs-tls"));
+    }
+
+    #[test]
+    fn next_tls_generation_saturates() {
+        assert_eq!(next_tls_generation(0).0, 1);
+        assert_eq!(next_tls_generation(u64::MAX).0, u64::MAX);
+    }
+}