diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index 9e1c59b5e..bbd7e8dc9 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,16 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-embedded-startup-control` -- Baseline: completed `R-036/R-037`. -- Stacked on: embedded startup server helper extraction slices. +- Branch: `overtrue/arch-embedded-build-orchestration` +- Baseline: completed `R-038/R-039`. +- Stacked on: embedded startup control helper extraction slice. - PR type for this branch: `pure-move` - Runtime behavior changes: none. -- Rust code changes: move the embedded process-global startup guard and - post-HTTP startup failure shutdown signal into startup lifecycle/shutdown - helpers while preserving embedded builder behavior. +- Rust code changes: move the embedded build startup sequence into a crate-only + startup owner while preserving the public embedded builder and server handle + behavior. - CI/script changes: none. -- Docs changes: record the embedded startup control owner slices. +- Docs changes: record the embedded build orchestration owner slices. ## Phase 0 Tasks @@ -2292,6 +2292,34 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block check, migration/layer guards, formatting, diff hygiene, Rust risk scan, branch freshness check, pre-commit quality gate, and three-expert review. +- [x] `R-040` Extract embedded build orchestration owner. + - Do: move the embedded build sequence behind a crate-only startup embedded + helper. + - Acceptance: embedded startup still runs config preparation, runtime hooks, + listen context, process-global guard, storage foundation, HTTP startup, + storage runtime, runtime services, and readiness publication in the same + order. + - Must preserve: retry-before-global-init behavior, temp-dir guard lifetime, + post-HTTP startup failure shutdown signaling, readiness publication error + text, and no public embedded API behavior changes. + - Verification: focused embedded checks, RustFS lib check, migration/layer + guards, formatting, diff hygiene, Rust risk scan, branch freshness check, + pre-commit quality gate, and three-expert review. + +- [x] `R-041` Keep embedded public API as handle assembly. + - Do: keep `embedded.rs` focused on public builder inputs, `RustFSServer` + handle construction, endpoint reporting, shutdown, and drop cleanup. + - Acceptance: builder defaults and fluent setters still feed the same startup + fields, server accessors still return the configured credentials and + region, endpoint normalization stays in the public handle, and shutdown/drop + cleanup remains unchanged. + - Must preserve: `ServerError` variants and messages, `Io` versus `Init` + error mapping, endpoint URL shape, shutdown handle ownership, cancellation + token ownership, and temp-dir cleanup path. + - Verification: focused embedded checks, RustFS lib check, migration/layer + guards, formatting, diff hygiene, Rust risk scan, branch freshness check, + pre-commit quality gate, and three-expert review. + - [x] `E-001/E-SET-001` Add ECStore layout skeleton and set-layout boundary. - Do: create the ECStore internal layout ownership buckets and pin static set layout versus runtime `Sets`/`SetDisks` orchestration boundaries before any @@ -2534,21 +2562,33 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Next PRs -1. `pure-move`: continue pruning residual embedded startup-only orchestration - after the startup control helpers land. +1. `pure-move`: continue pruning residual embedded shutdown/handle-only + boundaries after the embedded build orchestration helper lands. ## Pre-Push Review Log | Expert | Status | Notes | |---|---|---| -| Quality/architecture | passed | R-038 and R-039 move embedded startup guard and startup-failure shutdown signaling into startup owner helpers with crate-only visibility. | -| Migration preservation | passed | Retry-before-global-init, once-only process guard, `AlreadyStarted` mapping, post-HTTP signal/cancel behavior, and `Init` error mapping stay unchanged. | -| Testing/verification | passed | Focused startup lifecycle/shutdown/embedded checks, RustFS lib check, architecture/layer guards, formatting, diff hygiene, Rust risk scan, and pre-commit gate passed. | +| Quality/architecture | passed | R-040 and R-041 move embedded startup orchestration into a crate-only startup owner while keeping `embedded.rs` focused on public API and handle behavior. | +| Migration preservation | passed | Initialization order, startup guard timing, temp-dir lifetime, post-HTTP shutdown signaling, error mapping, endpoint normalization, and shutdown/drop behavior stay unchanged. | +| Testing/verification | passed | Focused embedded checks, RustFS lib check, architecture/layer guards, formatting, diff hygiene, Rust risk scan, and pre-commit gate passed. | ## Verification Notes Passed before push: +- Issue #660 R-040/R-041 current slice: + - `cargo test -p rustfs --lib embedded -- --nocapture`: passed. + - `cargo check -p rustfs --lib`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - Rust risk scan on changed Rust files: passed; newly added risky-token + matches were empty, and the changed-file scan only matched the existing + embedded `Drop` cleanup path. + - `make pre-commit`: passed. + - Issue #660 R-038/R-039 current slice: - `cargo test -p rustfs --lib startup_lifecycle -- --nocapture`: passed. - `cargo test -p rustfs --lib startup_shutdown -- --nocapture`: passed. diff --git a/rustfs/src/embedded.rs b/rustfs/src/embedded.rs index c19426e68..1161f78dd 100644 --- a/rustfs/src/embedded.rs +++ b/rustfs/src/embedded.rs @@ -47,14 +47,9 @@ //! start a second server will return an error. use crate::server::ShutdownHandle; -use crate::startup_lifecycle::{EmbeddedStartupGuard, log_embedded_server_ready, publish_embedded_startup_ready}; -use crate::startup_runtime_hooks::init_embedded_runtime_hooks; -use crate::startup_server::{ - EmbeddedStartupConfig, init_embedded_startup_listen_context, prepare_embedded_startup_config, start_embedded_http_server, -}; -use crate::startup_services::init_embedded_startup_runtime_services; -use crate::startup_shutdown::{run_embedded_server_shutdown, signal_embedded_startup_shutdown}; -use crate::startup_storage::{init_embedded_startup_storage_foundation, init_embedded_startup_storage_runtime}; +use crate::startup_embedded::{EmbeddedStartupArgs, EmbeddedStartupError, run_embedded_startup}; +use crate::startup_lifecycle::log_embedded_server_ready; +use crate::startup_shutdown::run_embedded_server_shutdown; use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr}; use std::path::PathBuf; use tokio_util::sync::CancellationToken; @@ -99,6 +94,16 @@ impl From for ServerError { } } +impl From for ServerError { + fn from(err: EmbeddedStartupError) -> Self { + match err { + EmbeddedStartupError::AlreadyStarted => ServerError::AlreadyStarted, + EmbeddedStartupError::Init(message) => ServerError::Init(message), + EmbeddedStartupError::Io(err) => ServerError::Io(err), + } + } +} + /// Builder for configuring and starting an embedded RustFS server. /// /// # Examples @@ -213,85 +218,24 @@ impl RustFSServerBuilder { self.do_build().await } - /// Inner build implementation. Separated from [`build`] so the outer - /// method can enforce the one-shot process-global startup guard. async fn do_build(&mut self) -> Result { - // Build is allowed to fail before irreversible global initialization - // (for example on temporary I/O or directory setup errors), and in that - // case callers can retry. - let mut startup_guard = EmbeddedStartupGuard::new(); - - let EmbeddedStartupConfig { config, temp_dir_guard } = prepare_embedded_startup_config( - self.address.clone(), - self.access_key.clone(), - self.secret_key.clone(), - self.volumes.clone(), - self.region.clone(), - ) - .await - .map_err(|e| ServerError::Init(e.to_string()))?; - - // --- Initialization sequence (mirrors main.rs::run) --- - - init_embedded_runtime_hooks(config.obs_endpoint.clone()) - .await - .map_err(|e| ServerError::Init(e.to_string()))?; - - let listen_context = init_embedded_startup_listen_context(&config) - .await - .map_err(|e| ServerError::Init(e.to_string()))?; - - startup_guard - .mark_global_init_started() - .map_err(|_| ServerError::AlreadyStarted)?; - - let endpoint_pools = init_embedded_startup_storage_foundation(&listen_context.server_address, &config.volumes) - .await - .map_err(|e| ServerError::Init(e.to_string()))?; - - let http_server = start_embedded_http_server(&config, listen_context.readiness.clone()).await?; - let shutdown_handle = http_server.shutdown_handle; - let bound_addr = http_server.bound_addr; - let ctx = CancellationToken::new(); - let storage_runtime = match init_embedded_startup_storage_runtime( - listen_context.server_addr, - &endpoint_pools, - listen_context.readiness.clone(), - ctx.clone(), - ) - .await - { - Ok(runtime) => runtime, - Err(e) => { - signal_embedded_startup_shutdown(&shutdown_handle, &ctx); - return Err(ServerError::Init(e.to_string())); - } - }; - let store = storage_runtime.store; - - let service_runtime = - init_embedded_startup_runtime_services(&config, endpoint_pools, store, ctx.clone(), listen_context.readiness.clone()) - .await - .map_err(|e| { - signal_embedded_startup_shutdown(&shutdown_handle, &ctx); - ServerError::Init(e.to_string()) - })?; - - publish_embedded_startup_ready(service_runtime.iam_bootstrap, listen_context.readiness.as_ref()) - .await - .map_err(|e| { - signal_embedded_startup_shutdown(&shutdown_handle, &ctx); - ServerError::Init(format!("runtime readiness: {e}")) - })?; + let started = run_embedded_startup(EmbeddedStartupArgs { + address: self.address.clone(), + access_key: self.access_key.clone(), + secret_key: self.secret_key.clone(), + volumes: self.volumes.clone(), + region: self.region.clone(), + }) + .await?; let server = RustFSServer { - address: bound_addr, + address: started.bound_addr, access_key: self.access_key.clone(), secret_key: self.secret_key.clone(), region: self.region.clone(), - shutdown_handle: Some(shutdown_handle), - cancel_token: ctx, - temp_dir: temp_dir_guard.map(|g| g.keep()), + shutdown_handle: Some(started.shutdown_handle), + cancel_token: started.cancel_token, + temp_dir: started.temp_dir, }; log_embedded_server_ready(server.endpoint_address()); diff --git a/rustfs/src/lib.rs b/rustfs/src/lib.rs index a4e0169fe..b602277c4 100644 --- a/rustfs/src/lib.rs +++ b/rustfs/src/lib.rs @@ -68,6 +68,7 @@ pub mod profiling; pub mod protocols; pub mod runtime_capabilities; pub mod server; +pub(crate) mod startup_embedded; pub mod startup_entrypoint; pub mod startup_fs_guard; pub mod startup_iam; diff --git a/rustfs/src/startup_embedded.rs b/rustfs/src/startup_embedded.rs new file mode 100644 index 000000000..d06d1e82b --- /dev/null +++ b/rustfs/src/startup_embedded.rs @@ -0,0 +1,132 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use crate::{ + server::ShutdownHandle, + startup_lifecycle::{EmbeddedStartupGuard, publish_embedded_startup_ready}, + startup_runtime_hooks::init_embedded_runtime_hooks, + startup_server::{ + EmbeddedStartupConfig, init_embedded_startup_listen_context, prepare_embedded_startup_config, start_embedded_http_server, + }, + startup_services::init_embedded_startup_runtime_services, + startup_shutdown::signal_embedded_startup_shutdown, + startup_storage::{init_embedded_startup_storage_foundation, init_embedded_startup_storage_runtime}, +}; +use std::{io, net::SocketAddr, path::PathBuf}; +use tokio_util::sync::CancellationToken; + +pub(crate) struct EmbeddedStartupArgs { + pub address: String, + pub access_key: String, + pub secret_key: String, + pub volumes: Vec, + pub region: String, +} + +pub(crate) struct EmbeddedStartedServer { + pub bound_addr: SocketAddr, + pub shutdown_handle: ShutdownHandle, + pub cancel_token: CancellationToken, + pub temp_dir: Option, +} + +#[derive(Debug)] +pub(crate) enum EmbeddedStartupError { + AlreadyStarted, + Init(String), + Io(io::Error), +} + +impl From for EmbeddedStartupError { + fn from(err: io::Error) -> Self { + Self::Io(err) + } +} + +pub(crate) async fn run_embedded_startup(args: EmbeddedStartupArgs) -> Result { + // Build is allowed to fail before irreversible global initialization + // (for example on temporary I/O or directory setup errors), and in that + // case callers can retry. + let mut startup_guard = EmbeddedStartupGuard::new(); + + let EmbeddedStartupConfig { config, temp_dir_guard } = + prepare_embedded_startup_config(args.address, args.access_key, args.secret_key, args.volumes, args.region) + .await + .map_err(init_error)?; + + init_embedded_runtime_hooks(config.obs_endpoint.clone()) + .await + .map_err(init_error)?; + + let listen_context = init_embedded_startup_listen_context(&config).await.map_err(init_error)?; + + startup_guard + .mark_global_init_started() + .map_err(|_| EmbeddedStartupError::AlreadyStarted)?; + + let endpoint_pools = init_embedded_startup_storage_foundation(&listen_context.server_address, &config.volumes) + .await + .map_err(init_error)?; + + let http_server = start_embedded_http_server(&config, listen_context.readiness.clone()).await?; + let shutdown_handle = http_server.shutdown_handle; + let bound_addr = http_server.bound_addr; + let cancel_token = CancellationToken::new(); + + let storage_runtime = match init_embedded_startup_storage_runtime( + listen_context.server_addr, + &endpoint_pools, + listen_context.readiness.clone(), + cancel_token.clone(), + ) + .await + { + Ok(runtime) => runtime, + Err(err) => { + signal_embedded_startup_shutdown(&shutdown_handle, &cancel_token); + return Err(init_error(err)); + } + }; + + let service_runtime = init_embedded_startup_runtime_services( + &config, + endpoint_pools, + storage_runtime.store, + cancel_token.clone(), + listen_context.readiness.clone(), + ) + .await + .map_err(|err| { + signal_embedded_startup_shutdown(&shutdown_handle, &cancel_token); + init_error(err) + })?; + + publish_embedded_startup_ready(service_runtime.iam_bootstrap, listen_context.readiness.as_ref()) + .await + .map_err(|err| { + signal_embedded_startup_shutdown(&shutdown_handle, &cancel_token); + EmbeddedStartupError::Init(format!("runtime readiness: {err}")) + })?; + + Ok(EmbeddedStartedServer { + bound_addr, + shutdown_handle, + cancel_token, + temp_dir: temp_dir_guard.map(|guard| guard.keep()), + }) +} + +fn init_error(err: impl std::fmt::Display) -> EmbeddedStartupError { + EmbeddedStartupError::Init(err.to_string()) +}