refactor: extract optional runtime startup boundary (#3633)

docs/architecture/migration-progress.md

@@ -5,15 +5,15 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
 ## Current Context
 
 - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
-- Branch: `overtrue/arch-optional-runtime-sidecar-boundary`
+- Branch: `overtrue/arch-optional-runtime-start-boundary`
 - Baseline: latest `origin/main`
-  (`e6391598f0b62a66ca6eac405c26b53a9a77d5f1`).
+  (`74fcfddd7fd98d69358e02908617261d962725d1`).
 - PR type for this branch: `pure-move`
 - Runtime behavior changes: none.
-- Rust code changes: move optional protocol shutdown ownership from
+- Rust code changes: move optional protocol startup and shutdown ownership from
   `startup_services` into the optional runtime boundary.
 - CI/script changes: none.
-- Docs changes: record the R-021 optional runtime shutdown boundary slice.
+- Docs changes: record the R-022 optional runtime startup boundary slice.
 
 ## Phase 0 Tasks
 
@@ -2030,6 +2030,22 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
     check, migration/layer guards, formatting, diff hygiene, Rust risk scan,
     branch freshness check, pre-commit quality gate, and three-expert review.
 
+- [x] `R-022` Extract optional runtime startup boundary.
+  - Do: add `init_optional_runtime_services` so optional protocol startup is
+    owned by `startup_optional_runtimes`, while `startup_protocols` remains the
+    protocol implementation adapter.
+  - Acceptance: optional protocol startup order stays FTP, FTPS, WebDAV, SFTP;
+    KMS initialization still happens before optional protocol startup; buffer
+    profiling, audit, deadlock detection, metadata, IAM, notification, scanner,
+    heal, and observability startup remain after optional protocol startup.
+  - Must preserve: protocol feature gates, disabled protocol behavior,
+    protocol startup error mapping, fatal boundary on protocol startup errors,
+    startup order, shutdown order, readiness state, and runtime behavior.
+  - Verification: focused optional runtime/protocol/startup service tests,
+    RustFS lib check, migration/layer guards, formatting, diff hygiene, Rust
+    risk scan, branch freshness check, pre-commit quality gate, and
+    three-expert review.
+
 ## Next PRs
 
 1. `pure-move`: continue larger lifecycle hook slices for optional runtime
@@ -2041,9 +2057,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
 
 | Expert | Status | Notes |
 |---|---|---|
-| Quality/architecture | passed | R-021 isolates optional runtime shutdown ownership in `startup_optional_runtimes` and keeps `startup_services` as the startup/shutdown orchestrator. |
-| Migration preservation | passed | Protocol startup, shutdown order, readiness, HTTP shutdown, event notifier, audit, and profiling behavior remain unchanged. |
-| Testing/verification | passed | Focused optional runtime and startup service tests pass; remaining checks and final pre-commit are tracked below before push. |
+| Quality/architecture | passed | R-022 keeps protocol implementation details in `startup_protocols` and moves optional runtime startup ownership into `startup_optional_runtimes`. |
+| Migration preservation | passed | KMS-before-protocol ordering, protocol fatal boundary, protocol startup order, shutdown order, readiness, and HTTP shutdown behavior remain unchanged. |
+| Testing/verification | passed | Focused optional runtime, protocol, and startup service tests plus final checks are tracked below before push. |
 
 ## Verification Notes
 
@@ -2084,6 +2100,20 @@ Passed before push:
   - `make pre-commit`: passed.
   - Three-expert review: passed.
 
+- Issue #660 R-022 current slice:
+  - `cargo test -p rustfs --lib startup_optional_runtimes -- --nocapture`:
+    passed.
+  - `cargo test -p rustfs --lib startup_protocols -- --nocapture`: passed.
+  - `cargo test -p rustfs --lib startup_services -- --nocapture`: passed.
+  - `cargo check -p rustfs --lib`: passed.
+  - `./scripts/check_architecture_migration_rules.sh`: passed.
+  - `./scripts/check_layer_dependencies.sh`: passed.
+  - `cargo fmt --all --check`: passed.
+  - `git diff --check`: passed.
+  - Rust risk scan on changed Rust files: passed.
+  - `make pre-commit`: passed.
+  - Three-expert review: passed.
+
 - Issue #660 R-020 current slice:
   - `cargo test -p rustfs --lib startup_profiling -- --nocapture`: passed.
   - `cargo check -p rustfs --lib`: passed.

docs/architecture/profiling-numa-capability-inventory.md

@@ -86,3 +86,10 @@ Future sidecars for profiling, eBPF, or NUMA must preserve these invariants:
   runtime boundary with explicit shutdown ownership.
 - Preserve current service order, KMS/audit/notification fatal boundaries, and
   scanner/heal startup semantics.
+
+`R-022`:
+
+- Keep optional runtime startup handoff in `startup_optional_runtimes` while
+  leaving concrete protocol adapters in `startup_protocols`.
+- Preserve KMS-before-protocol startup ordering and disabled protocol no-op
+  behavior.

docs/architecture/runtime-lifecycle.md

@@ -33,11 +33,11 @@ Disallowed responsibilities:
 
 ## Optional Runtime Boundary
 
-`startup_optional_runtimes` owns shutdown for optional runtime services that are
-not readiness or fatal startup boundaries. The current owner set is protocol
-servers only. Future optional sidecars must enter this boundary with explicit
-shutdown handles and status snapshots instead of adding ad hoc shutdown work to
-`startup_services`.
+`startup_optional_runtimes` owns startup and shutdown handoff for optional
+runtime services that are not readiness boundaries. The current owner set is
+protocol servers only. Future optional sidecars must enter this boundary with
+explicit shutdown handles and status snapshots instead of adding ad hoc startup
+or shutdown work to `startup_services`.
 
 ## AppContext Foundation
 

docs/architecture/startup-timeline.md

@@ -37,7 +37,7 @@ new startup semantics.
 | `RUN-011` | `rustfs/src/startup_storage.rs` | Create cancellation token and initialize `ECStore`. | Creates the runtime cancellation token and storage engine. | Fatal if `ECStore::new` fails. | None |
 | `RUN-012` | `rustfs/src/startup_storage.rs` | Initialize ECStore config and global config system. | Initializes ECStore config, attempts server-config migration, then retries global config init up to 15 times. | Migration attempt is non-fatal in this path; global config init becomes fatal after retries. | Marks the `GlobalReadiness` `StorageReady` stage after global config init succeeds; later runtime readiness still rechecks storage, IAM, and lock quorum before `FullReady` |
 | `RUN-013` | `rustfs/src/startup_storage.rs` and `rustfs/src/startup_services.rs` | Start replication and KMS systems. | Starts background replication pool, then initializes KMS from startup services. | Replication init is non-fatal in this path; KMS init is fatal on error. | `StorageReady` stage is already marked; dynamic runtime storage readiness is still checked before `FullReady` |
-| `RUN-014` | `rustfs/src/startup_services.rs` | Initialize optional protocol servers. | Starts FTP/FTPS/WebDAV/SFTP when feature-enabled and configured, collecting shutdown handles. | Feature-enabled protocol init is fatal on error; disabled protocols are non-fatal. | None |
+| `RUN-014` | `rustfs/src/startup_optional_runtimes.rs` and `rustfs/src/startup_protocols.rs` | Initialize optional protocol servers. | Starts FTP/FTPS/WebDAV/SFTP when feature-enabled and configured, collecting shutdown handles. | Feature-enabled protocol init is fatal on error; disabled protocols are non-fatal. | None |
 | `RUN-015` | `rustfs/src/startup_services.rs` | Initialize buffer profiling, event notifier, audit, and deadlock detector. | Starts buffer profile system, event notifier, audit system, and optional deadlock detector. | Audit startup failure is logged and non-fatal; the others are non-fatal in this path. | None |
 | `RUN-016` | `rustfs/src/startup_services.rs` | List buckets and run bucket/replication/IAM metadata migrations. | Reads bucket names, migrates bucket metadata, initializes replication resync, migrates IAM config, and initializes bucket metadata system. | Bucket list and replication resync are fatal on error; metadata migration calls are non-fatal in this path. | Storage remains ready; IAM not yet ready |
 | `RUN-017` | `rustfs/src/startup_services.rs` and `rustfs/src/startup_iam.rs` | Bootstrap IAM inline or defer recovery. | Initializes IAM when possible; otherwise starts the deferred IAM recovery path through `startup_iam`. | Fatal only when `bootstrap_or_defer_iam_init` returns an unrecoverable error. | Inline success marks `IamReady`; deferred mode publishes `IamReady` later from the recovery task |

rustfs/src/startup_optional_runtimes.rs

@@ -12,8 +12,12 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-use crate::{server::ShutdownHandle, startup_protocols::ProtocolShutdownSenders};
+use crate::{
+    server::ShutdownHandle,
+    startup_protocols::{ProtocolShutdownSenders, init_protocol_shutdown_senders},
+};
 use futures_util::future::join_all;
+use std::io::Result;
 use tracing::info;
 
 const LOG_COMPONENT_MAIN: &str = "main";
@@ -30,6 +34,11 @@ impl OptionalRuntimeServices {
     }
 }
 
+pub async fn init_optional_runtime_services() -> Result<OptionalRuntimeServices> {
+    let protocols = init_protocol_shutdown_senders().await?;
+    Ok(OptionalRuntimeServices::new(protocols))
+}
+
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 enum OptionalRuntimeShutdownStep {
     Ftp,

rustfs/src/startup_services.rs

@@ -28,9 +28,9 @@ use crate::{
     },
     startup_iam::{IamBootstrapDisposition, bootstrap_or_defer_iam_init, publish_ready_for_iam_bootstrap},
     startup_optional_runtimes::{
-        OptionalRuntimeServices, prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services,
+        OptionalRuntimeServices, init_optional_runtime_services, prepare_optional_runtime_shutdowns,
+        shutdown_optional_runtime_services,
     },
-    startup_protocols::init_protocol_shutdown_senders,
 };
 use rustfs_audit::AuditResult;
 use rustfs_common::GlobalReadiness;
@@ -98,8 +98,7 @@ pub async fn init_startup_runtime_services(
 ) -> Result<StartupServiceRuntime> {
     init_kms_system(config).await?;
 
-    let protocol_shutdowns = init_protocol_shutdown_senders().await?;
-    let optional_runtimes = OptionalRuntimeServices::new(protocol_shutdowns);
+    let optional_runtimes = init_optional_runtime_services().await?;
 
     init_buffer_profile_system(config);
     init_audit_runtime().await;