mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 16:28:15 +00:00
refactor: route runtime consumers through app context (#3785)
This commit is contained in:
@@ -5,13 +5,15 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
## Current Context
|
||||
|
||||
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
|
||||
- Branch: `overtrue/arch-auth-credentials-context`
|
||||
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170`.
|
||||
- Based on: API-169 PR #3782 after merge.
|
||||
- Branch: `overtrue/arch-runtime-context-consumer-batch`
|
||||
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177`.
|
||||
- Based on: API-170 merged in PR #3783; this branch batches API-171 through
|
||||
API-177 on top of latest `main`.
|
||||
- PR type for this branch: `consumer-migration`
|
||||
- Runtime behavior changes: none.
|
||||
- Rust code changes: route auth, protocol storage client, and storage helper
|
||||
action-credential reads through the AppContext action-credential resolver.
|
||||
- Rust code changes: route replication pool, outbound TLS generation, runtime
|
||||
region, KMS encryption service, runtime support handles, S3 Select DB, and
|
||||
internode RPC metrics through AppContext-first resolvers.
|
||||
- CI/script changes: lock completed owner and test/fuzz boundaries against
|
||||
bare/glob imports, scattered raw ECStore facade subpaths, and startup
|
||||
runtime/root-server/table/S3/app shared/app bucket/app ECStore/admin facade
|
||||
@@ -20,7 +22,7 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
and storage owner thin bridge regressions, plus app context and notify
|
||||
event-bridge thin module regressions; accept the reviewed AppContext resolver
|
||||
reverse dependencies in the layer baseline.
|
||||
- Docs changes: record the API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170 owner facade cleanup.
|
||||
- Docs changes: record the API-136 through API-177 owner facade cleanup.
|
||||
|
||||
## Phase 0 Tasks
|
||||
|
||||
@@ -4460,6 +4462,100 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
credential scan, Rust risk scan, branch freshness check, and three-expert
|
||||
review.
|
||||
|
||||
- [x] `API-171` Route runtime replication pool reads through AppContext.
|
||||
- Do: route bucket metadata startup resync and workload admission
|
||||
replication pool reads through the AppContext replication-pool resolver.
|
||||
- Acceptance: production startup/workload admission consumers no longer read
|
||||
the replication pool directly from the storage global facade, while the
|
||||
AppContext default adapter preserves the legacy global fallback.
|
||||
- Must preserve: bucket metadata resync initialization, replication admission
|
||||
active worker counts, queued replication counts, and unknown-runtime
|
||||
reporting.
|
||||
- Verification: RustFS compile coverage, targeted context resolver tests,
|
||||
migration guard, layer guard, formatting, diff hygiene, residual
|
||||
replication-pool scan, Rust risk scan, branch freshness check, and
|
||||
three-expert review.
|
||||
|
||||
- [x] `API-172` Route outbound TLS generation reads through AppContext.
|
||||
- Do: route startup TLS material initialization and TLS reload loop
|
||||
generation reads through the AppContext outbound TLS generation resolver.
|
||||
- Acceptance: production startup/reload paths no longer read outbound TLS
|
||||
generation directly from `rustfs_common`, while the AppContext default
|
||||
adapter preserves the legacy global fallback.
|
||||
- Must preserve: generation increment semantics, outbound TLS state publish,
|
||||
TLS generation metrics, reload-loop enrichment, and TLS acceptor rebuilds.
|
||||
- Verification: RustFS compile coverage, targeted context resolver tests,
|
||||
migration guard, layer guard, formatting, diff hygiene, residual outbound
|
||||
TLS generation scan, Rust risk scan, branch freshness check, and
|
||||
three-expert review.
|
||||
|
||||
- [x] `API-173` Route runtime region reads through AppContext.
|
||||
- Do: route bucket notification setup and S3 request context region reads
|
||||
through the AppContext region resolver.
|
||||
- Acceptance: production init/storage request paths no longer read region
|
||||
directly from the storage global facade, while the AppContext default
|
||||
adapter preserves the legacy global fallback.
|
||||
- Must preserve: notification ARN target mapping fallback region behavior,
|
||||
request context region propagation, auth/policy request construction, and
|
||||
existing startup region setters.
|
||||
- Verification: RustFS compile coverage, targeted context resolver tests,
|
||||
migration guard, layer guard, formatting, diff hygiene, residual region
|
||||
scan, Rust risk scan, branch freshness check, and three-expert review.
|
||||
|
||||
- [x] `API-174` Route KMS encryption service reads through AppContext.
|
||||
- Do: route app bucket encryption defaults and storage SSE managed
|
||||
encryption/decryption provider reads through an AppContext-first KMS
|
||||
encryption service resolver.
|
||||
- Acceptance: production app/storage paths no longer read the KMS encryption
|
||||
service directly from the global service manager, while the resolver
|
||||
preserves the legacy global fallback.
|
||||
- Must preserve: default SSE-KMS key population, managed SSE encryption and
|
||||
decryption metadata handling, DEK provider selection, and KMS service
|
||||
initialization fallback semantics.
|
||||
- Verification: RustFS compile coverage, targeted SSE/KMS tests, migration
|
||||
guard, layer guard, formatting, diff hygiene, residual encryption-service
|
||||
scan, Rust risk scan, branch freshness check, and three-expert review.
|
||||
|
||||
- [x] `API-175` Route runtime support reads through AppContext.
|
||||
- Do: route runtime readiness lock-client collections, storage concurrency
|
||||
performance metrics, and config-info buffer profile reads through
|
||||
AppContext-first resolvers.
|
||||
- Acceptance: production readiness, storage concurrency, and config-info paths
|
||||
no longer read those runtime globals directly, while default adapters
|
||||
preserve the legacy global fallbacks.
|
||||
- Must preserve: distributed readiness lock quorum aggregation, performance
|
||||
metric singleton sharing, workload profile display output, and existing
|
||||
buffer-profile enablement behavior.
|
||||
- Verification: RustFS compile coverage, targeted context resolver tests,
|
||||
migration guard, layer guard, formatting, diff hygiene, residual global-read
|
||||
scans, Rust risk scan, branch freshness check, and three-expert review.
|
||||
|
||||
- [x] `API-176` Route S3 Select DB factory reads through AppContext.
|
||||
- Do: route S3 Select object execution database creation through an
|
||||
AppContext-first S3 Select DB resolver.
|
||||
- Acceptance: production S3 Select object execution no longer reads the
|
||||
S3 Select DB factory directly, while the default adapter preserves the
|
||||
cached global component behavior.
|
||||
- Must preserve: request validation, preflight object metadata checks,
|
||||
DataFusion execution flow, output event streaming, and cached S3 Select
|
||||
component reuse.
|
||||
- Verification: RustFS compile coverage, targeted context resolver tests,
|
||||
migration guard, layer guard, formatting, diff hygiene, residual S3 Select
|
||||
DB scan, Rust risk scan, branch freshness check, and three-expert review.
|
||||
|
||||
- [x] `API-177` Route internode RPC metrics through AppContext.
|
||||
- Do: route HTTP and gRPC internode RPC metric recording through an
|
||||
AppContext-first internode metrics resolver.
|
||||
- Acceptance: production internode HTTP and disk RPC paths no longer read
|
||||
the internode metrics singleton directly, while the default adapter
|
||||
preserves the shared global metrics instance.
|
||||
- Must preserve: HTTP read/write/walk counters, gRPC read/write counters,
|
||||
byte accounting, classified transport backend labels, and error recording.
|
||||
- Verification: RustFS compile coverage, targeted context resolver tests,
|
||||
migration guard, layer guard, formatting, diff hygiene, residual internode
|
||||
metrics scan, Rust risk scan, branch freshness check, and three-expert
|
||||
review.
|
||||
|
||||
## Next PRs
|
||||
|
||||
1. `consumer-migration`: continue reducing direct global reads behind AppContext resolver boundaries.
|
||||
@@ -4526,11 +4622,146 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
| Quality/architecture | pass | API-170 keeps action-credential reads behind the AppContext action-credential resolver across auth, protocols, and storage helper paths. |
|
||||
| Migration preservation | pass | Owner checks, claim validation, policy principal classification, protocol metadata, and audit access-key enrichment preserve existing fallback behavior. |
|
||||
| Testing/verification | pass | RustFS focused compile, targeted context tests, formatting, migration/layer guards, diff hygiene, residual action credential scan, and Rust risk scan passed for API-170. |
|
||||
| Quality/architecture | pass | API-171 keeps replication pool reads behind the AppContext replication-pool resolver in startup and workload admission paths. |
|
||||
| Migration preservation | pass | Bucket metadata resync, replication worker counts, queue counts, and unknown-runtime reporting preserve existing fallback behavior. |
|
||||
| Testing/verification | pass | RustFS focused compile, workload admission tests, targeted context tests, formatting, migration/layer guards, diff hygiene, residual replication-pool scan, and Rust risk scan passed for API-171. |
|
||||
| Quality/architecture | pass | API-172 keeps outbound TLS generation reads behind the AppContext outbound TLS generation resolver in startup and reload paths. |
|
||||
| Migration preservation | pass | Generation increments, outbound TLS publication, generation metrics, reload enrichment, and acceptor rebuild behavior preserve existing semantics. |
|
||||
| Testing/verification | pass | RustFS focused compile, TLS generation test, targeted context test, formatting, migration/layer guards, diff hygiene, residual outbound TLS generation scan, and Rust risk scan passed for API-172. |
|
||||
| Quality/architecture | pass | API-173 keeps region reads behind the AppContext region resolver in notification setup and storage request context paths. |
|
||||
| Migration preservation | pass | Notification fallback region, request context propagation, auth/policy request construction, and startup setters preserve existing behavior. |
|
||||
| Testing/verification | pass | RustFS focused compile, targeted context test, formatting, migration/layer guards, diff hygiene, residual region scan, and Rust risk scan passed for API-173. |
|
||||
| Quality/architecture | pass | API-174 keeps app/storage KMS encryption service reads behind an AppContext-first resolver using the existing KMS runtime manager boundary. |
|
||||
| Migration preservation | pass | Default SSE-KMS key population, managed SSE metadata handling, DEK provider selection, and legacy global fallback behavior are preserved. |
|
||||
| Testing/verification | pass | RustFS focused compile, targeted SSE/KMS and bucket encryption tests, formatting, migration/layer guards, diff hygiene, residual encryption-service scan, and Rust risk scan passed for API-174. |
|
||||
| Quality/architecture | pass | API-175 keeps readiness lock clients, storage performance metrics, and config-info buffer profile reads behind AppContext-first resolvers. |
|
||||
| Migration preservation | pass | Lock quorum aggregation, performance metrics sharing, workload profile display, and legacy global fallback behavior are preserved. |
|
||||
| Testing/verification | pass | RustFS focused compile, targeted context resolver test, formatting, migration/layer guards, diff hygiene, residual global-read scans, and Rust risk scan passed for API-175. |
|
||||
| Quality/architecture | pass | API-176 keeps S3 Select DB factory access behind an AppContext-first resolver in object select execution. |
|
||||
| Migration preservation | pass | Request validation, object preflight, query execution, event streaming, and cached component fallback behavior are preserved. |
|
||||
| Testing/verification | pass | RustFS focused compile, targeted context resolver test, formatting, migration/layer guards, diff hygiene, residual S3 Select DB scan, and Rust risk scan passed for API-176. |
|
||||
| Quality/architecture | pass | API-177 keeps internode RPC metrics behind an AppContext-first resolver across HTTP and gRPC RPC paths. |
|
||||
| Migration preservation | pass | Request counters, byte accounting, transport backend labels, and error metrics preserve existing global fallback behavior. |
|
||||
| Testing/verification | pass | RustFS focused compile, targeted context resolver test, formatting, migration/layer guards, diff hygiene, residual internode metrics scan, and Rust risk scan passed for API-177. |
|
||||
|
||||
## Verification Notes
|
||||
|
||||
Passed before push:
|
||||
|
||||
- Issue #660 API-176 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all`: passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- S3 Select DB scan: passed; direct production `get_global_db` reads are
|
||||
removed from S3 Select object execution.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-177 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all`: passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- Internode metrics scan: passed; direct production
|
||||
`global_internode_metrics` reads are removed from RustFS HTTP and disk RPC
|
||||
paths.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-175 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all`: passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- Runtime support scan: passed; direct production lock-client collection,
|
||||
performance metrics, and config-info buffer profile reads now go through
|
||||
AppContext resolvers.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-174 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs test_kms_sse_dek_provider_uses_latest_reconfigured_service --lib`:
|
||||
passed.
|
||||
- `cargo test -p rustfs test_sse_encryption_fails_closed_without_local_sse_master_key --lib`:
|
||||
passed.
|
||||
- `cargo test -p rustfs execute_put_bucket_encryption_returns_internal_error_when_store_uninitialized --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- AppContext KMS encryption service scan: passed; direct production
|
||||
`get_global_encryption_service` reads are removed from app bucket
|
||||
encryption and storage SSE paths.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-173 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all`: passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- AppContext region scan: passed; direct production `get_global_region`
|
||||
reads are removed from notification setup and storage request context
|
||||
paths.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-172 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs next_tls_generation --lib`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- AppContext outbound TLS generation scan: passed; direct production
|
||||
`get_global_outbound_tls_generation` reads are removed from startup TLS
|
||||
material and TLS reload paths.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-171 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs workload_admission --lib`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- AppContext replication pool scan: passed; direct production
|
||||
`get_global_replication_pool` reads are removed from bucket metadata
|
||||
startup and workload admission paths.
|
||||
- Rust risk scan: no new production unwrap/expect, panic/todo/unsafe, or cast
|
||||
risks added.
|
||||
|
||||
- Issue #660 API-170 current slice:
|
||||
- `cargo check --tests -p rustfs`: passed.
|
||||
- `cargo test -p rustfs resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`:
|
||||
|
||||
@@ -39,8 +39,8 @@ use crate::admin::handlers::site_replication::{
|
||||
site_replication_bucket_meta_hook, site_replication_delete_bucket_hook, site_replication_make_bucket_hook,
|
||||
};
|
||||
use crate::app::context::{
|
||||
AppContext, get_global_app_context, resolve_notification_system, resolve_notify_interface_for_context,
|
||||
resolve_object_store_handle_for_context,
|
||||
AppContext, get_global_app_context, resolve_encryption_service, resolve_notification_system,
|
||||
resolve_notify_interface_for_context, resolve_object_store_handle_for_context,
|
||||
};
|
||||
use crate::auth::get_condition_values_with_client_info;
|
||||
use crate::error::ApiError;
|
||||
@@ -1560,7 +1560,7 @@ impl DefaultBucketUsecase {
|
||||
&& by_default.sse_algorithm.as_str() == ServerSideEncryption::AWS_KMS
|
||||
&& by_default.kms_master_key_id.as_deref().is_none_or(str::is_empty)
|
||||
{
|
||||
let service = rustfs_kms::service_manager::get_global_encryption_service()
|
||||
let service = resolve_encryption_service()
|
||||
.await
|
||||
.ok_or_else(|| S3Error::with_message(S3ErrorCode::InternalError, "KMS service not initialized".to_string()))?;
|
||||
let default_key = service
|
||||
|
||||
+256
-5
@@ -38,10 +38,13 @@ use crate::config::RustFSBufferConfig;
|
||||
use rustfs_config::server_config::Config;
|
||||
use rustfs_credentials::Credentials;
|
||||
use rustfs_iam::{store::object::ObjectStore, sys::IamSys};
|
||||
use rustfs_kms::{KmsServiceManager, init_global_kms_service_manager};
|
||||
use rustfs_io_metrics::{PerformanceMetrics, internode_metrics::InternodeMetrics};
|
||||
use rustfs_kms::{KmsServiceManager, ObjectEncryptionService, init_global_kms_service_manager};
|
||||
use rustfs_lock::LockClient;
|
||||
use rustfs_s3select_api::{QueryResult, server::dbms::DatabaseManagerSystem};
|
||||
use rustfs_tls_runtime::{GlobalPublishedOutboundTlsState, TlsGeneration};
|
||||
use std::{future::Future, sync::Arc, time::SystemTime};
|
||||
use s3s::dto::SelectObjectContentInput;
|
||||
use std::{collections::HashMap, future::Future, sync::Arc, time::SystemTime};
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
/// Resolve KMS runtime service manager using AppContext-first precedence.
|
||||
@@ -54,6 +57,11 @@ pub fn resolve_or_init_kms_runtime_service_manager() -> Arc<KmsServiceManager> {
|
||||
resolve_or_init_kms_runtime_service_manager_with(get_global_app_context(), init_global_kms_service_manager)
|
||||
}
|
||||
|
||||
/// Resolve KMS encryption service using AppContext-first precedence.
|
||||
pub async fn resolve_encryption_service() -> Option<Arc<ObjectEncryptionService>> {
|
||||
resolve_encryption_service_with(get_global_app_context(), rustfs_kms::get_global_encryption_service).await
|
||||
}
|
||||
|
||||
/// Resolve outbound TLS generation using AppContext-first precedence.
|
||||
pub fn resolve_outbound_tls_generation() -> TlsGeneration {
|
||||
resolve_outbound_tls_generation_with(get_global_app_context(), || default_outbound_tls_runtime_interface().generation())
|
||||
@@ -155,6 +163,32 @@ pub fn resolve_lock_client() -> Option<Arc<dyn LockClient>> {
|
||||
resolve_lock_client_with(get_global_app_context(), || default_lock_client_interface().handle())
|
||||
}
|
||||
|
||||
/// Resolve lock clients using AppContext-first precedence.
|
||||
pub fn resolve_lock_clients_handle() -> Option<HashMap<String, Arc<dyn LockClient>>> {
|
||||
resolve_lock_clients_handle_with(get_global_app_context(), || default_lock_clients_interface().handle())
|
||||
}
|
||||
|
||||
/// Resolve performance metrics using AppContext-first precedence.
|
||||
pub fn resolve_performance_metrics() -> Arc<PerformanceMetrics> {
|
||||
resolve_performance_metrics_with(get_global_app_context(), || default_performance_metrics_interface().handle())
|
||||
}
|
||||
|
||||
/// Resolve internode metrics using AppContext-first precedence.
|
||||
pub fn resolve_internode_metrics() -> Arc<InternodeMetrics> {
|
||||
resolve_internode_metrics_with(get_global_app_context(), || default_internode_metrics_interface().handle())
|
||||
}
|
||||
|
||||
/// Resolve S3 Select database using AppContext-first precedence.
|
||||
pub async fn resolve_s3select_db(
|
||||
input: SelectObjectContentInput,
|
||||
enable_debug: bool,
|
||||
) -> QueryResult<Arc<dyn DatabaseManagerSystem + Send + Sync>> {
|
||||
resolve_s3select_db_with(get_global_app_context(), input, enable_debug, |input, enable_debug| async move {
|
||||
default_s3select_db_interface().get(input, enable_debug).await
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
/// Resolve local node name using AppContext-first precedence.
|
||||
pub async fn resolve_local_node_name() -> String {
|
||||
resolve_local_node_name_with(get_global_app_context(), rustfs_common::get_global_local_node_name).await
|
||||
@@ -215,6 +249,21 @@ fn resolve_or_init_kms_runtime_service_manager_with(
|
||||
.unwrap_or_else(fallback)
|
||||
}
|
||||
|
||||
async fn resolve_encryption_service_with<F, Fut>(
|
||||
context: Option<Arc<AppContext>>,
|
||||
fallback: F,
|
||||
) -> Option<Arc<ObjectEncryptionService>>
|
||||
where
|
||||
F: FnOnce() -> Fut,
|
||||
Fut: Future<Output = Option<Arc<ObjectEncryptionService>>>,
|
||||
{
|
||||
if let Some(manager) = context.and_then(|context| context.kms_runtime().service_manager()) {
|
||||
return manager.get_encryption_service().await;
|
||||
}
|
||||
|
||||
fallback().await
|
||||
}
|
||||
|
||||
fn resolve_outbound_tls_generation_with(
|
||||
context: Option<Arc<AppContext>>,
|
||||
fallback: impl FnOnce() -> TlsGeneration,
|
||||
@@ -339,6 +388,44 @@ fn resolve_lock_client_with(
|
||||
context.and_then(|context| context.lock_client().handle()).or_else(fallback)
|
||||
}
|
||||
|
||||
fn resolve_lock_clients_handle_with(
|
||||
context: Option<Arc<AppContext>>,
|
||||
fallback: impl FnOnce() -> Option<HashMap<String, Arc<dyn LockClient>>>,
|
||||
) -> Option<HashMap<String, Arc<dyn LockClient>>> {
|
||||
context.and_then(|context| context.lock_clients().handle()).or_else(fallback)
|
||||
}
|
||||
|
||||
fn resolve_performance_metrics_with(
|
||||
context: Option<Arc<AppContext>>,
|
||||
fallback: impl FnOnce() -> Arc<PerformanceMetrics>,
|
||||
) -> Arc<PerformanceMetrics> {
|
||||
context.map_or_else(fallback, |context| context.performance_metrics().handle())
|
||||
}
|
||||
|
||||
fn resolve_internode_metrics_with(
|
||||
context: Option<Arc<AppContext>>,
|
||||
fallback: impl FnOnce() -> Arc<InternodeMetrics>,
|
||||
) -> Arc<InternodeMetrics> {
|
||||
context.map_or_else(fallback, |context| context.internode_metrics().handle())
|
||||
}
|
||||
|
||||
async fn resolve_s3select_db_with<F, Fut>(
|
||||
context: Option<Arc<AppContext>>,
|
||||
input: SelectObjectContentInput,
|
||||
enable_debug: bool,
|
||||
fallback: F,
|
||||
) -> QueryResult<Arc<dyn DatabaseManagerSystem + Send + Sync>>
|
||||
where
|
||||
F: FnOnce(SelectObjectContentInput, bool) -> Fut,
|
||||
Fut: Future<Output = QueryResult<Arc<dyn DatabaseManagerSystem + Send + Sync>>>,
|
||||
{
|
||||
if let Some(context) = context {
|
||||
return context.s3select_db().get(input, enable_debug).await;
|
||||
}
|
||||
|
||||
fallback(input, enable_debug).await
|
||||
}
|
||||
|
||||
async fn resolve_local_node_name_with<F, Fut>(context: Option<Arc<AppContext>>, fallback: F) -> String
|
||||
where
|
||||
F: FnOnce() -> Fut,
|
||||
@@ -419,14 +506,21 @@ mod tests {
|
||||
};
|
||||
use crate::app::context::interfaces::{
|
||||
ActionCredentialInterface, BootTimeInterface, BucketMetadataInterface, BufferConfigInterface, DeploymentIdInterface,
|
||||
EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface, LocalNodeNameInterface, LockClientInterface,
|
||||
OutboundTlsRuntimeInterface, RegionInterface, ReplicationStatsInterface, RuntimePortInterface, ScannerMetricsInterface,
|
||||
ServerConfigInterface, StorageClassInterface, TierConfigInterface, TierStatsInterface,
|
||||
EndpointsInterface, IamInterface, InternodeMetricsInterface, KmsInterface, KmsRuntimeInterface, LocalNodeNameInterface,
|
||||
LockClientInterface, LockClientsInterface, OutboundTlsRuntimeInterface, PerformanceMetricsInterface, RegionInterface,
|
||||
ReplicationStatsInterface, RuntimePortInterface, S3SelectDbInterface, ScannerMetricsInterface, ServerConfigInterface,
|
||||
StorageClassInterface, TierConfigInterface, TierStatsInterface,
|
||||
};
|
||||
use crate::config::{RustFSBufferConfig, WorkloadProfile};
|
||||
use async_trait::async_trait;
|
||||
use rustfs_iam::{store::object::ObjectStore, sys::IamSys};
|
||||
use rustfs_io_metrics::{PerformanceMetrics, internode_metrics::InternodeMetrics};
|
||||
use rustfs_lock::{LocalClient, LockClient};
|
||||
use rustfs_s3select_api::{
|
||||
QueryResult,
|
||||
query::{Query, execution::QueryStateMachineRef, logical_planner::Plan},
|
||||
server::dbms::{DatabaseManagerSystem, QueryHandle},
|
||||
};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
use std::time::{Duration, SystemTime};
|
||||
@@ -573,6 +667,76 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
struct TestLockClientsInterface {
|
||||
clients: Option<HashMap<String, Arc<dyn LockClient>>>,
|
||||
}
|
||||
|
||||
impl LockClientsInterface for TestLockClientsInterface {
|
||||
fn handle(&self) -> Option<HashMap<String, Arc<dyn LockClient>>> {
|
||||
self.clients.clone()
|
||||
}
|
||||
}
|
||||
|
||||
struct TestPerformanceMetricsInterface {
|
||||
metrics: Arc<PerformanceMetrics>,
|
||||
}
|
||||
|
||||
impl PerformanceMetricsInterface for TestPerformanceMetricsInterface {
|
||||
fn handle(&self) -> Arc<PerformanceMetrics> {
|
||||
self.metrics.clone()
|
||||
}
|
||||
}
|
||||
|
||||
struct TestInternodeMetricsInterface {
|
||||
metrics: Arc<InternodeMetrics>,
|
||||
}
|
||||
|
||||
impl InternodeMetricsInterface for TestInternodeMetricsInterface {
|
||||
fn handle(&self) -> Arc<InternodeMetrics> {
|
||||
self.metrics.clone()
|
||||
}
|
||||
}
|
||||
|
||||
struct TestS3SelectDbInterface {
|
||||
db: Arc<dyn DatabaseManagerSystem + Send + Sync>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl S3SelectDbInterface for TestS3SelectDbInterface {
|
||||
async fn get(
|
||||
&self,
|
||||
_input: SelectObjectContentInput,
|
||||
_enable_debug: bool,
|
||||
) -> QueryResult<Arc<dyn DatabaseManagerSystem + Send + Sync>> {
|
||||
Ok(self.db.clone())
|
||||
}
|
||||
}
|
||||
|
||||
struct TestS3SelectDb;
|
||||
|
||||
#[async_trait]
|
||||
impl DatabaseManagerSystem for TestS3SelectDb {
|
||||
async fn execute(&self, _query: &Query) -> QueryResult<QueryHandle> {
|
||||
unreachable!("resolver tests only compare database handles")
|
||||
}
|
||||
|
||||
async fn build_query_state_machine(&self, _query: Query) -> QueryResult<QueryStateMachineRef> {
|
||||
unreachable!("resolver tests only compare database handles")
|
||||
}
|
||||
|
||||
async fn build_logical_plan(&self, _query_state_machine: QueryStateMachineRef) -> QueryResult<Option<Plan>> {
|
||||
unreachable!("resolver tests only compare database handles")
|
||||
}
|
||||
|
||||
async fn execute_logical_plan(
|
||||
&self,
|
||||
_logical_plan: Plan,
|
||||
_query_state_machine: QueryStateMachineRef,
|
||||
) -> QueryResult<QueryHandle> {
|
||||
unreachable!("resolver tests only compare database handles")
|
||||
}
|
||||
}
|
||||
|
||||
struct TestLocalNodeNameInterface {
|
||||
name: String,
|
||||
}
|
||||
@@ -699,6 +863,25 @@ mod tests {
|
||||
(temp_dir, store, endpoint_pools)
|
||||
}
|
||||
|
||||
fn test_select_input() -> SelectObjectContentInput {
|
||||
SelectObjectContentInput {
|
||||
bucket: "test-bucket".to_string(),
|
||||
expected_bucket_owner: None,
|
||||
key: "test.csv".to_string(),
|
||||
sse_customer_algorithm: None,
|
||||
sse_customer_key: None,
|
||||
sse_customer_key_md5: None,
|
||||
request: s3s::dto::SelectObjectContentRequest {
|
||||
expression: "SELECT * FROM S3Object".to_string(),
|
||||
expression_type: s3s::dto::ExpressionType::from_static("SQL"),
|
||||
input_serialization: s3s::dto::InputSerialization::default(),
|
||||
output_serialization: s3s::dto::OutputSerialization::default(),
|
||||
request_progress: None,
|
||||
scan_range: None,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolver_helpers_are_context_first_and_fallback_when_context_is_absent() {
|
||||
let (_temp_dir, object_store, endpoints) = test_store().await;
|
||||
@@ -736,6 +919,16 @@ mod tests {
|
||||
let fallback_deployment_id = "fallback-deployment".to_string();
|
||||
let context_runtime_port = 19000;
|
||||
let fallback_runtime_port = 29000;
|
||||
let mut context_lock_clients = HashMap::new();
|
||||
context_lock_clients.insert("context-node:9000".to_string(), context_lock_client.clone());
|
||||
let mut fallback_lock_clients = HashMap::new();
|
||||
fallback_lock_clients.insert("fallback-node:9000".to_string(), fallback_lock_client.clone());
|
||||
let context_performance_metrics = Arc::new(PerformanceMetrics::new());
|
||||
let fallback_performance_metrics = Arc::new(PerformanceMetrics::new());
|
||||
let context_internode_metrics = Arc::new(InternodeMetrics::default());
|
||||
let fallback_internode_metrics = Arc::new(InternodeMetrics::default());
|
||||
let context_s3select_db: Arc<dyn DatabaseManagerSystem + Send + Sync> = Arc::new(TestS3SelectDb);
|
||||
let fallback_s3select_db: Arc<dyn DatabaseManagerSystem + Send + Sync> = Arc::new(TestS3SelectDb);
|
||||
let context_outbound_tls_state = GlobalPublishedOutboundTlsState {
|
||||
generation: TlsGeneration(41),
|
||||
root_ca_pem: Some(b"context-root-ca".to_vec()),
|
||||
@@ -796,6 +989,18 @@ mod tests {
|
||||
lock_client: Arc::new(TestLockClientInterface {
|
||||
client: Some(context_lock_client.clone()),
|
||||
}),
|
||||
lock_clients: Arc::new(TestLockClientsInterface {
|
||||
clients: Some(context_lock_clients.clone()),
|
||||
}),
|
||||
performance_metrics: Arc::new(TestPerformanceMetricsInterface {
|
||||
metrics: context_performance_metrics.clone(),
|
||||
}),
|
||||
internode_metrics: Arc::new(TestInternodeMetricsInterface {
|
||||
metrics: context_internode_metrics.clone(),
|
||||
}),
|
||||
s3select_db: Arc::new(TestS3SelectDbInterface {
|
||||
db: context_s3select_db.clone(),
|
||||
}),
|
||||
local_node_name: Arc::new(TestLocalNodeNameInterface {
|
||||
name: context_node_name.clone(),
|
||||
}),
|
||||
@@ -882,6 +1087,29 @@ mod tests {
|
||||
&resolve_lock_client_with(Some(context.clone()), || None).expect("context lock client"),
|
||||
&context_lock_client
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
resolve_lock_clients_handle_with(Some(context.clone()), || None)
|
||||
.expect("context lock clients")
|
||||
.get("context-node:9000")
|
||||
.expect("context lock client entry"),
|
||||
&context_lock_client
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
&resolve_performance_metrics_with(Some(context.clone()), || fallback_performance_metrics.clone()),
|
||||
&context_performance_metrics
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
&resolve_internode_metrics_with(Some(context.clone()), || fallback_internode_metrics.clone()),
|
||||
&context_internode_metrics
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
&resolve_s3select_db_with(Some(context.clone()), test_select_input(), false, |_input, _enable_debug| async {
|
||||
Ok(fallback_s3select_db.clone())
|
||||
})
|
||||
.await
|
||||
.expect("context S3 Select DB"),
|
||||
&context_s3select_db
|
||||
));
|
||||
assert_eq!(
|
||||
resolve_local_node_name_with(Some(context.clone()), || async { fallback_node_name.clone() }).await,
|
||||
context_node_name
|
||||
@@ -972,6 +1200,29 @@ mod tests {
|
||||
&resolve_lock_client_with(None, || Some(fallback_lock_client.clone())).expect("fallback lock client"),
|
||||
&fallback_lock_client
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
resolve_lock_clients_handle_with(None, || Some(fallback_lock_clients.clone()))
|
||||
.expect("fallback lock clients")
|
||||
.get("fallback-node:9000")
|
||||
.expect("fallback lock client entry"),
|
||||
&fallback_lock_client
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
&resolve_performance_metrics_with(None, || fallback_performance_metrics.clone()),
|
||||
&fallback_performance_metrics
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
&resolve_internode_metrics_with(None, || fallback_internode_metrics.clone()),
|
||||
&fallback_internode_metrics
|
||||
));
|
||||
assert!(Arc::ptr_eq(
|
||||
&resolve_s3select_db_with(None, test_select_input(), false, |_input, _enable_debug| async {
|
||||
Ok(fallback_s3select_db.clone())
|
||||
})
|
||||
.await
|
||||
.expect("fallback S3 Select DB"),
|
||||
&fallback_s3select_db
|
||||
));
|
||||
assert_eq!(
|
||||
resolve_local_node_name_with(None, || async { fallback_node_name.clone() }).await,
|
||||
fallback_node_name
|
||||
|
||||
@@ -16,17 +16,20 @@ use super::super::{ECStore, set_object_store_resolver};
|
||||
use super::handles::{
|
||||
IamHandle, KmsHandle, default_action_credential_interface, default_boot_time_interface, default_bucket_metadata_interface,
|
||||
default_bucket_monitor_interface, default_buffer_config_interface, default_deployment_id_interface,
|
||||
default_endpoints_interface, default_kms_runtime_interface, default_local_node_name_interface, default_lock_client_interface,
|
||||
default_endpoints_interface, default_internode_metrics_interface, default_kms_runtime_interface,
|
||||
default_local_node_name_interface, default_lock_client_interface, default_lock_clients_interface,
|
||||
default_notification_system_interface, default_notify_interface, default_outbound_tls_runtime_interface,
|
||||
default_region_interface, default_replication_pool_interface, default_replication_stats_interface,
|
||||
default_runtime_port_interface, default_scanner_metrics_interface, default_server_config_interface,
|
||||
default_storage_class_interface, default_tier_config_interface, default_tier_stats_interface,
|
||||
default_performance_metrics_interface, default_region_interface, default_replication_pool_interface,
|
||||
default_replication_stats_interface, default_runtime_port_interface, default_s3select_db_interface,
|
||||
default_scanner_metrics_interface, default_server_config_interface, default_storage_class_interface,
|
||||
default_tier_config_interface, default_tier_stats_interface,
|
||||
};
|
||||
use super::interfaces::{
|
||||
ActionCredentialInterface, BootTimeInterface, BucketMetadataInterface, BucketMonitorInterface, BufferConfigInterface,
|
||||
DeploymentIdInterface, EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface, LocalNodeNameInterface,
|
||||
LockClientInterface, NotificationSystemInterface, NotifyInterface, OutboundTlsRuntimeInterface, RegionInterface,
|
||||
ReplicationPoolInterface, ReplicationStatsInterface, RuntimePortInterface, ScannerMetricsInterface, ServerConfigInterface,
|
||||
DeploymentIdInterface, EndpointsInterface, IamInterface, InternodeMetricsInterface, KmsInterface, KmsRuntimeInterface,
|
||||
LocalNodeNameInterface, LockClientInterface, LockClientsInterface, NotificationSystemInterface, NotifyInterface,
|
||||
OutboundTlsRuntimeInterface, PerformanceMetricsInterface, RegionInterface, ReplicationPoolInterface,
|
||||
ReplicationStatsInterface, RuntimePortInterface, S3SelectDbInterface, ScannerMetricsInterface, ServerConfigInterface,
|
||||
StorageClassInterface, TierConfigInterface, TierStatsInterface,
|
||||
};
|
||||
use rustfs_iam::{store::object::ObjectStore, sys::IamSys};
|
||||
@@ -55,6 +58,10 @@ pub struct AppContext {
|
||||
deployment_id: Arc<dyn DeploymentIdInterface>,
|
||||
runtime_port: Arc<dyn RuntimePortInterface>,
|
||||
lock_client: Arc<dyn LockClientInterface>,
|
||||
lock_clients: Arc<dyn LockClientsInterface>,
|
||||
performance_metrics: Arc<dyn PerformanceMetricsInterface>,
|
||||
internode_metrics: Arc<dyn InternodeMetricsInterface>,
|
||||
s3select_db: Arc<dyn S3SelectDbInterface>,
|
||||
local_node_name: Arc<dyn LocalNodeNameInterface>,
|
||||
action_credentials: Arc<dyn ActionCredentialInterface>,
|
||||
region: Arc<dyn RegionInterface>,
|
||||
@@ -85,6 +92,10 @@ impl AppContext {
|
||||
deployment_id: default_deployment_id_interface(),
|
||||
runtime_port: default_runtime_port_interface(),
|
||||
lock_client: default_lock_client_interface(),
|
||||
lock_clients: default_lock_clients_interface(),
|
||||
performance_metrics: default_performance_metrics_interface(),
|
||||
internode_metrics: default_internode_metrics_interface(),
|
||||
s3select_db: default_s3select_db_interface(),
|
||||
local_node_name: default_local_node_name_interface(),
|
||||
action_credentials: default_action_credential_interface(),
|
||||
region: default_region_interface(),
|
||||
@@ -176,6 +187,22 @@ impl AppContext {
|
||||
self.lock_client.clone()
|
||||
}
|
||||
|
||||
pub fn lock_clients(&self) -> Arc<dyn LockClientsInterface> {
|
||||
self.lock_clients.clone()
|
||||
}
|
||||
|
||||
pub fn performance_metrics(&self) -> Arc<dyn PerformanceMetricsInterface> {
|
||||
self.performance_metrics.clone()
|
||||
}
|
||||
|
||||
pub fn internode_metrics(&self) -> Arc<dyn InternodeMetricsInterface> {
|
||||
self.internode_metrics.clone()
|
||||
}
|
||||
|
||||
pub fn s3select_db(&self) -> Arc<dyn S3SelectDbInterface> {
|
||||
self.s3select_db.clone()
|
||||
}
|
||||
|
||||
pub fn local_node_name(&self) -> Arc<dyn LocalNodeNameInterface> {
|
||||
self.local_node_name.clone()
|
||||
}
|
||||
@@ -224,6 +251,10 @@ pub(super) struct AppContextTestInterfaces {
|
||||
pub(super) deployment_id: Arc<dyn DeploymentIdInterface>,
|
||||
pub(super) runtime_port: Arc<dyn RuntimePortInterface>,
|
||||
pub(super) lock_client: Arc<dyn LockClientInterface>,
|
||||
pub(super) lock_clients: Arc<dyn LockClientsInterface>,
|
||||
pub(super) performance_metrics: Arc<dyn PerformanceMetricsInterface>,
|
||||
pub(super) internode_metrics: Arc<dyn InternodeMetricsInterface>,
|
||||
pub(super) s3select_db: Arc<dyn S3SelectDbInterface>,
|
||||
pub(super) local_node_name: Arc<dyn LocalNodeNameInterface>,
|
||||
pub(super) action_credentials: Arc<dyn ActionCredentialInterface>,
|
||||
pub(super) region: Arc<dyn RegionInterface>,
|
||||
@@ -255,6 +286,10 @@ impl AppContext {
|
||||
deployment_id: interfaces.deployment_id,
|
||||
runtime_port: interfaces.runtime_port,
|
||||
lock_client: interfaces.lock_client,
|
||||
lock_clients: interfaces.lock_clients,
|
||||
performance_metrics: interfaces.performance_metrics,
|
||||
internode_metrics: interfaces.internode_metrics,
|
||||
s3select_db: interfaces.s3select_db,
|
||||
local_node_name: interfaces.local_node_name,
|
||||
action_credentials: interfaces.action_credentials,
|
||||
region: interfaces.region,
|
||||
|
||||
@@ -18,15 +18,16 @@ use super::super::TierConfigMgr;
|
||||
use super::super::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys};
|
||||
use super::super::{
|
||||
collect_scanner_metrics_report, get_daily_all_tier_stats, get_global_boot_time, get_global_bucket_monitor,
|
||||
get_global_deployment_id, get_global_endpoints_opt, get_global_lock_client, get_global_notification_sys, get_global_region,
|
||||
get_global_replication_pool, get_global_replication_stats, get_global_tier_config_mgr, global_rustfs_port,
|
||||
set_global_storage_class,
|
||||
get_global_deployment_id, get_global_endpoints_opt, get_global_lock_client, get_global_lock_clients,
|
||||
get_global_notification_sys, get_global_region, get_global_replication_pool, get_global_replication_stats,
|
||||
get_global_tier_config_mgr, global_rustfs_port, set_global_storage_class,
|
||||
};
|
||||
use super::interfaces::{
|
||||
ActionCredentialInterface, BootTimeInterface, BucketMetadataInterface, BucketMonitorInterface, BufferConfigInterface,
|
||||
DeploymentIdInterface, EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface, LocalNodeNameInterface,
|
||||
LockClientInterface, NotificationSystemInterface, NotifyInterface, OutboundTlsRuntimeInterface, RegionInterface,
|
||||
ReplicationPoolInterface, ReplicationStatsInterface, RuntimePortInterface, ScannerMetricsInterface, ServerConfigInterface,
|
||||
DeploymentIdInterface, EndpointsInterface, IamInterface, InternodeMetricsInterface, KmsInterface, KmsRuntimeInterface,
|
||||
LocalNodeNameInterface, LockClientInterface, LockClientsInterface, NotificationSystemInterface, NotifyInterface,
|
||||
OutboundTlsRuntimeInterface, PerformanceMetricsInterface, RegionInterface, ReplicationPoolInterface,
|
||||
ReplicationStatsInterface, RuntimePortInterface, S3SelectDbInterface, ScannerMetricsInterface, ServerConfigInterface,
|
||||
StorageClassInterface, TierConfigInterface, TierStatsInterface,
|
||||
};
|
||||
use crate::config::{RustFSBufferConfig, get_global_buffer_config};
|
||||
@@ -36,14 +37,21 @@ use rustfs_config::server_config::Config;
|
||||
use rustfs_config::server_config::{get_global_server_config, set_global_server_config};
|
||||
use rustfs_credentials::{Credentials, get_global_action_cred};
|
||||
use rustfs_iam::{store::object::ObjectStore, sys::IamSys};
|
||||
use rustfs_io_metrics::{
|
||||
PerformanceMetrics,
|
||||
global_metrics::get_global_metrics,
|
||||
internode_metrics::{InternodeMetrics, global_internode_metrics},
|
||||
};
|
||||
use rustfs_kms::{KmsServiceManager, get_global_kms_service_manager};
|
||||
use rustfs_lock::LockClient;
|
||||
use rustfs_notify::{EventArgs, NotificationError, notifier_global};
|
||||
use rustfs_s3select_api::{QueryResult, server::dbms::DatabaseManagerSystem};
|
||||
use rustfs_targets::{EventName, arn::TargetID};
|
||||
use rustfs_tls_runtime::{
|
||||
GlobalPublishedOutboundTlsState, TlsGeneration, load_global_outbound_tls_generation, load_global_outbound_tls_state,
|
||||
};
|
||||
use std::{sync::Arc, time::SystemTime};
|
||||
use s3s::dto::SelectObjectContentInput;
|
||||
use std::{collections::HashMap, sync::Arc, time::SystemTime};
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
/// Default IAM interface adapter.
|
||||
@@ -256,6 +264,51 @@ impl LockClientInterface for LockClientHandle {
|
||||
}
|
||||
}
|
||||
|
||||
/// Default lock clients interface adapter.
|
||||
#[derive(Default)]
|
||||
pub struct LockClientsHandle;
|
||||
|
||||
impl LockClientsInterface for LockClientsHandle {
|
||||
fn handle(&self) -> Option<HashMap<String, Arc<dyn LockClient>>> {
|
||||
get_global_lock_clients().cloned()
|
||||
}
|
||||
}
|
||||
|
||||
/// Default performance metrics interface adapter.
|
||||
#[derive(Default)]
|
||||
pub struct PerformanceMetricsHandle;
|
||||
|
||||
impl PerformanceMetricsInterface for PerformanceMetricsHandle {
|
||||
fn handle(&self) -> Arc<PerformanceMetrics> {
|
||||
get_global_metrics()
|
||||
}
|
||||
}
|
||||
|
||||
/// Default internode metrics interface adapter.
|
||||
#[derive(Default)]
|
||||
pub struct InternodeMetricsHandle;
|
||||
|
||||
impl InternodeMetricsInterface for InternodeMetricsHandle {
|
||||
fn handle(&self) -> Arc<InternodeMetrics> {
|
||||
global_internode_metrics().clone()
|
||||
}
|
||||
}
|
||||
|
||||
/// Default S3 Select database interface adapter.
|
||||
#[derive(Default)]
|
||||
pub struct S3SelectDbHandle;
|
||||
|
||||
#[async_trait]
|
||||
impl S3SelectDbInterface for S3SelectDbHandle {
|
||||
async fn get(
|
||||
&self,
|
||||
input: SelectObjectContentInput,
|
||||
enable_debug: bool,
|
||||
) -> QueryResult<Arc<dyn DatabaseManagerSystem + Send + Sync>> {
|
||||
rustfs_s3select_query::get_global_db(input, enable_debug).await
|
||||
}
|
||||
}
|
||||
|
||||
/// Default local node name interface adapter.
|
||||
#[derive(Default)]
|
||||
pub struct LocalNodeNameHandle;
|
||||
@@ -391,6 +444,22 @@ pub fn default_lock_client_interface() -> Arc<dyn LockClientInterface> {
|
||||
Arc::new(LockClientHandle)
|
||||
}
|
||||
|
||||
pub fn default_lock_clients_interface() -> Arc<dyn LockClientsInterface> {
|
||||
Arc::new(LockClientsHandle)
|
||||
}
|
||||
|
||||
pub fn default_performance_metrics_interface() -> Arc<dyn PerformanceMetricsInterface> {
|
||||
Arc::new(PerformanceMetricsHandle)
|
||||
}
|
||||
|
||||
pub fn default_internode_metrics_interface() -> Arc<dyn InternodeMetricsInterface> {
|
||||
Arc::new(InternodeMetricsHandle)
|
||||
}
|
||||
|
||||
pub fn default_s3select_db_interface() -> Arc<dyn S3SelectDbInterface> {
|
||||
Arc::new(S3SelectDbHandle)
|
||||
}
|
||||
|
||||
pub fn default_local_node_name_interface() -> Arc<dyn LocalNodeNameInterface> {
|
||||
Arc::new(LocalNodeNameHandle)
|
||||
}
|
||||
|
||||
@@ -24,12 +24,15 @@ use async_trait::async_trait;
|
||||
use rustfs_config::server_config::Config;
|
||||
use rustfs_credentials::Credentials;
|
||||
use rustfs_iam::{store::object::ObjectStore, sys::IamSys};
|
||||
use rustfs_io_metrics::{PerformanceMetrics, internode_metrics::InternodeMetrics};
|
||||
use rustfs_kms::KmsServiceManager;
|
||||
use rustfs_lock::LockClient;
|
||||
use rustfs_notify::{EventArgs, NotificationError};
|
||||
use rustfs_s3select_api::{QueryResult, server::dbms::DatabaseManagerSystem};
|
||||
use rustfs_targets::{EventName, arn::TargetID};
|
||||
use rustfs_tls_runtime::{GlobalPublishedOutboundTlsState, TlsGeneration};
|
||||
use std::{sync::Arc, time::SystemTime};
|
||||
use s3s::dto::SelectObjectContentInput;
|
||||
use std::{collections::HashMap, sync::Arc, time::SystemTime};
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
/// IAM interface for application-layer use-cases.
|
||||
@@ -133,6 +136,31 @@ pub trait LockClientInterface: Send + Sync {
|
||||
fn handle(&self) -> Option<Arc<dyn LockClient>>;
|
||||
}
|
||||
|
||||
/// Lock clients interface for runtime readiness integration.
|
||||
pub trait LockClientsInterface: Send + Sync {
|
||||
fn handle(&self) -> Option<HashMap<String, Arc<dyn LockClient>>>;
|
||||
}
|
||||
|
||||
/// Performance metrics interface for storage runtime integration.
|
||||
pub trait PerformanceMetricsInterface: Send + Sync {
|
||||
fn handle(&self) -> Arc<PerformanceMetrics>;
|
||||
}
|
||||
|
||||
/// Internode metrics interface for RPC runtime integration.
|
||||
pub trait InternodeMetricsInterface: Send + Sync {
|
||||
fn handle(&self) -> Arc<InternodeMetrics>;
|
||||
}
|
||||
|
||||
/// S3 Select database interface for object query execution.
|
||||
#[async_trait]
|
||||
pub trait S3SelectDbInterface: Send + Sync {
|
||||
async fn get(
|
||||
&self,
|
||||
input: SelectObjectContentInput,
|
||||
enable_debug: bool,
|
||||
) -> QueryResult<Arc<dyn DatabaseManagerSystem + Send + Sync>>;
|
||||
}
|
||||
|
||||
/// Local node name interface for application-layer use-cases.
|
||||
#[async_trait]
|
||||
pub trait LocalNodeNameInterface: Send + Sync {
|
||||
|
||||
@@ -41,7 +41,7 @@ mod lifecycle_transition_api_test;
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::{collections::HashMap, sync::Arc};
|
||||
|
||||
mod ecstore_admin {
|
||||
pub(crate) use crate::storage::ecstore_admin::get_server_info;
|
||||
@@ -646,6 +646,10 @@ pub(crate) fn get_global_lock_client() -> Option<Arc<dyn rustfs_lock::client::Lo
|
||||
crate::storage::get_global_lock_client()
|
||||
}
|
||||
|
||||
pub(crate) fn get_global_lock_clients() -> Option<&'static HashMap<String, Arc<dyn rustfs_lock::client::LockClient>>> {
|
||||
crate::storage::get_global_lock_clients()
|
||||
}
|
||||
|
||||
pub(crate) fn get_global_region() -> Option<s3s::region::Region> {
|
||||
crate::storage::get_global_region()
|
||||
}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
use crate::app::context::resolve_s3select_db;
|
||||
use crate::error::ApiError;
|
||||
use crate::storage::options::get_opts;
|
||||
use crate::storage::request_context::spawn_traced;
|
||||
@@ -15,7 +16,6 @@ use rustfs_s3select_api::{
|
||||
object_store::{INVALID_SCAN_RANGE_MESSAGE, validate_scan_range_bounds},
|
||||
query::{Context, Query},
|
||||
};
|
||||
use rustfs_s3select_query::get_global_db;
|
||||
use rustfs_storage_api::ObjectOperations as _;
|
||||
use s3s::dto::*;
|
||||
use s3s::{S3Error, S3ErrorCode, S3Request, S3Response, S3Result, s3_error};
|
||||
@@ -55,7 +55,9 @@ pub async fn execute_select_object_content(
|
||||
validate_scan_range_for_object_size(&input.request, metadata.size)?;
|
||||
|
||||
let input = Arc::new(input);
|
||||
let db = get_global_db((*input).clone(), false).await.map_err(map_query_error_to_s3)?;
|
||||
let db = resolve_s3select_db((*input).clone(), false)
|
||||
.await
|
||||
.map_err(map_query_error_to_s3)?;
|
||||
let query = Query::new(Context { input: input.clone() }, input.request.expression.clone());
|
||||
let output = db
|
||||
.execute(&query)
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
//! configuration, and dependencies.
|
||||
|
||||
use super::{InfoOpts, InfoType};
|
||||
use crate::app::context::resolve_buffer_config;
|
||||
use crate::version::build;
|
||||
use rustfs_credentials::Masked;
|
||||
use serde::Serialize;
|
||||
@@ -543,9 +544,9 @@ fn collect_config_info_json() -> ConfigInfoJson {
|
||||
|
||||
// Get workload profile info
|
||||
let workload_profile = {
|
||||
use super::workload_profiles::{get_global_buffer_config, is_buffer_profile_enabled};
|
||||
use super::workload_profiles::is_buffer_profile_enabled;
|
||||
if is_buffer_profile_enabled() {
|
||||
let config = get_global_buffer_config();
|
||||
let config = resolve_buffer_config();
|
||||
let profile = config.workload_profile();
|
||||
let buffer_config = profile.config();
|
||||
Some(WorkloadProfileJson {
|
||||
@@ -863,13 +864,13 @@ fn format_protocol_config_info() -> String {
|
||||
|
||||
/// Get workload profile information from global buffer config
|
||||
fn get_workload_profile_info() -> String {
|
||||
use super::workload_profiles::{get_global_buffer_config, is_buffer_profile_enabled};
|
||||
use super::workload_profiles::is_buffer_profile_enabled;
|
||||
|
||||
if !is_buffer_profile_enabled() {
|
||||
return "| Workload Profile | (disabled) |".to_string();
|
||||
}
|
||||
|
||||
let config = get_global_buffer_config();
|
||||
let config = resolve_buffer_config();
|
||||
let profile = config.workload_profile();
|
||||
let name = config.workload_name();
|
||||
let buffer_config = profile.config();
|
||||
|
||||
+3
-3
@@ -12,10 +12,10 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::app::context::resolve_region;
|
||||
use crate::server::ShutdownHandle;
|
||||
use crate::storage::{
|
||||
get_bucket_notification_config, get_global_region, process_lambda_configurations, process_queue_configurations,
|
||||
process_topic_configurations,
|
||||
get_bucket_notification_config, process_lambda_configurations, process_queue_configurations, process_topic_configurations,
|
||||
};
|
||||
use crate::{admin, config, version};
|
||||
use rustfs_config::{
|
||||
@@ -159,7 +159,7 @@ fn arn_to_target_id(arn_str: &str) -> Result<rustfs_targets::arn::TargetID, Targ
|
||||
/// * `buckets` - A vector of bucket names to process
|
||||
#[instrument(skip_all)]
|
||||
pub async fn add_bucket_notification_configuration(buckets: Vec<String>) {
|
||||
let global_region = get_global_region();
|
||||
let global_region = resolve_region();
|
||||
let region = global_region
|
||||
.as_ref()
|
||||
.filter(|r| !r.as_str().is_empty())
|
||||
|
||||
@@ -12,10 +12,10 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::app::context::{resolve_endpoints_handle, resolve_iam_ready};
|
||||
use crate::app::context::{resolve_endpoints_handle, resolve_iam_ready, resolve_lock_clients_handle};
|
||||
use crate::server::{ServiceState, ServiceStateManager};
|
||||
use crate::server::{has_path_prefix, is_table_catalog_path};
|
||||
use crate::storage::{Endpoint, EndpointServerPools, get_global_lock_clients, is_dist_erasure, resolve_object_store_handle};
|
||||
use crate::storage::{Endpoint, EndpointServerPools, is_dist_erasure, resolve_object_store_handle};
|
||||
#[cfg(test)]
|
||||
use crate::storage::{Endpoints, PoolEndpoints};
|
||||
use bytes::Bytes;
|
||||
@@ -584,7 +584,7 @@ async fn collect_lock_quorum_status_uncached() -> LockQuorumStatus {
|
||||
let Some(pool_endpoints) = resolve_endpoints_handle() else {
|
||||
return LockQuorumStatus::default();
|
||||
};
|
||||
let Some(lock_clients) = get_global_lock_clients() else {
|
||||
let Some(lock_clients) = resolve_lock_clients_handle() else {
|
||||
return LockQuorumStatus::default();
|
||||
};
|
||||
|
||||
|
||||
@@ -20,7 +20,8 @@
|
||||
//! (system roots, leaf-as-CA, mTLS env-var path overrides) and builds the server
|
||||
//! TLS acceptor directly from the pre-loaded server material — no double reads.
|
||||
|
||||
use rustfs_common::{MtlsIdentityPem, get_global_outbound_tls_generation};
|
||||
use crate::app::context::resolve_outbound_tls_generation;
|
||||
use rustfs_common::MtlsIdentityPem;
|
||||
use rustfs_config::{
|
||||
DEFAULT_SERVER_MTLS_ENABLE, DEFAULT_TLS_KEYLOG, DEFAULT_TLS_RELOAD_ENABLE, DEFAULT_TLS_RELOAD_INTERVAL,
|
||||
DEFAULT_TRUST_LEAF_CERT_AS_CA, DEFAULT_TRUST_SYSTEM_CA, ENV_MTLS_CLIENT_CERT, ENV_MTLS_CLIENT_KEY, ENV_SERVER_MTLS_ENABLE,
|
||||
@@ -552,7 +553,7 @@ pub(crate) fn spawn_reload_loop(tls_path: String, holder: Arc<TlsAcceptorHolder>
|
||||
continue;
|
||||
}
|
||||
|
||||
let generation = get_global_outbound_tls_generation().saturating_add(1);
|
||||
let generation = resolve_outbound_tls_generation().0.saturating_add(1);
|
||||
publish_global_outbound_tls_state(TlsGeneration(generation), &snapshot.outbound).await;
|
||||
record_tls_generation("rustfs_server_reload_loop", generation);
|
||||
if !snapshot.outbound.root_ca_pem.is_empty() {
|
||||
|
||||
@@ -12,9 +12,8 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::storage::{
|
||||
ECStore, get_global_replication_pool, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config,
|
||||
};
|
||||
use crate::app::context::resolve_replication_pool_handle;
|
||||
use crate::storage::{ECStore, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config};
|
||||
use rustfs_storage_api::{BucketOperations, BucketOptions};
|
||||
use std::{
|
||||
io::{Error, Result},
|
||||
@@ -53,7 +52,7 @@ pub(crate) async fn init_bucket_metadata_runtime(store: Arc<ECStore>, ctx: Cance
|
||||
|
||||
try_migrate_bucket_metadata(store.clone()).await;
|
||||
|
||||
if let Some(pool) = get_global_replication_pool() {
|
||||
if let Some(pool) = resolve_replication_pool_handle() {
|
||||
pool.init_resync(ctx, buckets.clone()).await?;
|
||||
}
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::app::context::resolve_outbound_tls_generation;
|
||||
use crate::config::Config;
|
||||
use std::io::{Error, Result};
|
||||
use tracing::{error, info};
|
||||
@@ -28,7 +29,7 @@ pub(crate) async fn init_outbound_tls_material(config: &Config) -> Result<()> {
|
||||
Ok(snapshot) => {
|
||||
use rustfs_tls_runtime::{publish_global_outbound_tls_state, record_tls_generation};
|
||||
|
||||
let generation = next_tls_generation(rustfs_common::get_global_outbound_tls_generation());
|
||||
let generation = next_tls_generation(resolve_outbound_tls_generation().0);
|
||||
publish_global_outbound_tls_state(generation, &snapshot.outbound).await;
|
||||
record_tls_generation(TLS_STARTUP_GENERATION_CONSUMER, generation.0);
|
||||
info!(
|
||||
|
||||
@@ -18,6 +18,7 @@ use super::resolve_object_store_handle;
|
||||
use super::{
|
||||
PolicySys, StorageError, get_bucket_metadata, get_bucket_policy_raw, get_public_access_block_config, is_err_bucket_not_found,
|
||||
};
|
||||
use crate::app::context::resolve_region;
|
||||
use crate::auth::{check_key_valid, get_condition_values_with_query_and_client_info, get_session_token};
|
||||
use crate::error::ApiError;
|
||||
use crate::license::license_check;
|
||||
@@ -930,7 +931,7 @@ impl S3Access for FS {
|
||||
let req_info = ReqInfo {
|
||||
cred,
|
||||
is_owner,
|
||||
region: super::get_global_region(),
|
||||
region: resolve_region(),
|
||||
request_context,
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
@@ -19,6 +19,7 @@ use super::io_schedule::{
|
||||
get_advanced_buffer_size,
|
||||
};
|
||||
use super::request_guard::GetObjectGuard;
|
||||
use crate::app::context::resolve_performance_metrics;
|
||||
use rustfs_concurrency::{
|
||||
AdmissionState, GetObjectQueueSnapshot, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot,
|
||||
WorkloadAdmissionSnapshotProvider, WorkloadClass,
|
||||
@@ -27,7 +28,6 @@ use rustfs_config::{KI_B, MI_B};
|
||||
use rustfs_io_core::BytesPool;
|
||||
use rustfs_io_core::io_profile::{AccessPattern, IoPatternDetector, StorageMedia, detect_storage_media};
|
||||
use rustfs_io_metrics::bandwidth::{BandwidthMonitor, BandwidthSnapshot};
|
||||
use rustfs_io_metrics::global_metrics::get_global_metrics;
|
||||
use rustfs_io_metrics::{MetricsCollector, PerformanceMetrics};
|
||||
use std::sync::{Arc, LazyLock, Mutex};
|
||||
use std::time::Duration;
|
||||
@@ -118,7 +118,7 @@ impl ConcurrencyManager {
|
||||
|
||||
// Use global performance metrics instance for consistent metrics tracking
|
||||
// This allows AutoTuner and other components to access the same metrics data
|
||||
let performance_metrics = get_global_metrics();
|
||||
let performance_metrics = resolve_performance_metrics();
|
||||
|
||||
// Initialize metrics collector for I/O latency tracking
|
||||
// Keep 1000 samples for P95/P99 calculation
|
||||
@@ -231,7 +231,7 @@ impl ConcurrencyManager {
|
||||
///
|
||||
/// Arc-wrapped PerformanceMetrics instance
|
||||
pub fn performance_metrics(&self) -> Arc<PerformanceMetrics> {
|
||||
get_global_metrics()
|
||||
resolve_performance_metrics()
|
||||
}
|
||||
|
||||
/// Calculate an adaptive I/O strategy based on disk permit wait time.
|
||||
|
||||
@@ -13,9 +13,9 @@
|
||||
// limitations under the License.
|
||||
|
||||
use super::*;
|
||||
use crate::app::context::resolve_internode_metrics;
|
||||
use rustfs_io_metrics::internode_metrics::{
|
||||
INTERNODE_OPERATION_GRPC_READ_ALL, INTERNODE_OPERATION_GRPC_WRITE_ALL, INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
global_internode_metrics,
|
||||
};
|
||||
use serde::de::DeserializeOwned;
|
||||
use std::io::Cursor;
|
||||
@@ -934,11 +934,12 @@ impl NodeService {
|
||||
pub(super) async fn handle_write_all(&self, request: Request<WriteAllRequest>) -> Result<Response<WriteAllResponse>, Status> {
|
||||
let request = request.into_inner();
|
||||
let data_len = request.data.len();
|
||||
global_internode_metrics().record_incoming_request_for_operation_and_backend(
|
||||
let metrics = resolve_internode_metrics();
|
||||
metrics.record_incoming_request_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_GRPC_WRITE_ALL,
|
||||
INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
);
|
||||
global_internode_metrics().record_recv_bytes_for_operation_and_backend(
|
||||
metrics.record_recv_bytes_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_GRPC_WRITE_ALL,
|
||||
INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
data_len,
|
||||
@@ -950,7 +951,7 @@ impl NodeService {
|
||||
error: None,
|
||||
})),
|
||||
Err(err) => {
|
||||
global_internode_metrics().record_error_for_operation_and_backend(
|
||||
metrics.record_error_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_GRPC_WRITE_ALL,
|
||||
INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
);
|
||||
@@ -961,8 +962,7 @@ impl NodeService {
|
||||
}
|
||||
}
|
||||
} else {
|
||||
global_internode_metrics()
|
||||
.record_error_for_operation_and_backend(INTERNODE_OPERATION_GRPC_WRITE_ALL, INTERNODE_TRANSPORT_BACKEND_GRPC);
|
||||
metrics.record_error_for_operation_and_backend(INTERNODE_OPERATION_GRPC_WRITE_ALL, INTERNODE_TRANSPORT_BACKEND_GRPC);
|
||||
Ok(Response::new(WriteAllResponse {
|
||||
success: false,
|
||||
error: Some(DiskError::other("can not find disk".to_string()).into()),
|
||||
@@ -974,14 +974,15 @@ impl NodeService {
|
||||
debug!("read all");
|
||||
|
||||
let request = request.into_inner();
|
||||
global_internode_metrics().record_incoming_request_for_operation_and_backend(
|
||||
let metrics = resolve_internode_metrics();
|
||||
metrics.record_incoming_request_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_GRPC_READ_ALL,
|
||||
INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
);
|
||||
if let Some(disk) = self.find_disk(&request.disk).await {
|
||||
match disk.read_all(&request.volume, &request.path).await {
|
||||
Ok(data) => {
|
||||
global_internode_metrics().record_sent_bytes_for_operation_and_backend(
|
||||
metrics.record_sent_bytes_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_GRPC_READ_ALL,
|
||||
INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
data.len(),
|
||||
@@ -993,7 +994,7 @@ impl NodeService {
|
||||
}))
|
||||
}
|
||||
Err(err) => {
|
||||
global_internode_metrics().record_error_for_operation_and_backend(
|
||||
metrics.record_error_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_GRPC_READ_ALL,
|
||||
INTERNODE_TRANSPORT_BACKEND_GRPC,
|
||||
);
|
||||
@@ -1005,8 +1006,7 @@ impl NodeService {
|
||||
}
|
||||
}
|
||||
} else {
|
||||
global_internode_metrics()
|
||||
.record_error_for_operation_and_backend(INTERNODE_OPERATION_GRPC_READ_ALL, INTERNODE_TRANSPORT_BACKEND_GRPC);
|
||||
metrics.record_error_for_operation_and_backend(INTERNODE_OPERATION_GRPC_READ_ALL, INTERNODE_TRANSPORT_BACKEND_GRPC);
|
||||
Ok(Response::new(ReadAllResponse {
|
||||
success: false,
|
||||
data: Bytes::new(),
|
||||
|
||||
@@ -17,6 +17,7 @@ use super::super::StorageDiskRpcExt as _;
|
||||
use super::super::WalkDirOptions;
|
||||
use super::super::find_local_disk_by_ref;
|
||||
use super::super::verify_rpc_signature;
|
||||
use crate::app::context::resolve_internode_metrics;
|
||||
use crate::server::RPC_PREFIX;
|
||||
use crate::storage::request_context::spawn_traced;
|
||||
use bytes::{Bytes, BytesMut};
|
||||
@@ -27,7 +28,7 @@ use hyper::body::Incoming;
|
||||
use rustfs_config::MAX_ADMIN_REQUEST_BODY_SIZE;
|
||||
use rustfs_io_metrics::internode_metrics::{
|
||||
INTERNODE_OPERATION_PUT_FILE_STREAM, INTERNODE_OPERATION_READ_FILE_STREAM, INTERNODE_OPERATION_WALK_DIR,
|
||||
INTERNODE_TRANSPORT_BACKEND_TCP_HTTP, global_internode_metrics,
|
||||
INTERNODE_TRANSPORT_BACKEND_TCP_HTTP,
|
||||
};
|
||||
use rustfs_utils::net::bytes_stream;
|
||||
use s3s::Body;
|
||||
@@ -311,11 +312,10 @@ fn internode_http_operation(path: &str) -> Option<&'static str> {
|
||||
}
|
||||
|
||||
fn record_internode_rpc_error(operation: Option<&'static str>) {
|
||||
let metrics = resolve_internode_metrics();
|
||||
match operation {
|
||||
Some(operation) => {
|
||||
global_internode_metrics().record_error_for_operation_and_backend(operation, INTERNODE_TRANSPORT_BACKEND_TCP_HTTP)
|
||||
}
|
||||
None => global_internode_metrics().record_error(),
|
||||
Some(operation) => metrics.record_error_for_operation_and_backend(operation, INTERNODE_TRANSPORT_BACKEND_TCP_HTTP),
|
||||
None => metrics.record_error(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -400,7 +400,7 @@ async fn handle_read_file(req: Request<Incoming>) -> Response<Body> {
|
||||
}
|
||||
};
|
||||
|
||||
global_internode_metrics().record_incoming_request_for_operation_and_backend(
|
||||
resolve_internode_metrics().record_incoming_request_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_READ_FILE_STREAM,
|
||||
INTERNODE_TRANSPORT_BACKEND_TCP_HTTP,
|
||||
);
|
||||
@@ -420,7 +420,7 @@ fn read_file_body_stream<R>(
|
||||
where
|
||||
R: tokio::io::AsyncRead + Unpin + Send + Sync + 'static,
|
||||
{
|
||||
let metrics = global_internode_metrics().clone();
|
||||
let metrics = resolve_internode_metrics();
|
||||
let stream = ReaderStream::with_capacity(reader, DEFAULT_READ_BUFFER_SIZE).map_ok(move |bytes| {
|
||||
metrics.record_sent_bytes_for_operation_and_backend(operation, INTERNODE_TRANSPORT_BACKEND_TCP_HTTP, bytes.len());
|
||||
bytes
|
||||
@@ -535,9 +535,9 @@ async fn handle_walk_dir(req: Request<Incoming>) -> Response<Body> {
|
||||
}
|
||||
});
|
||||
|
||||
global_internode_metrics()
|
||||
resolve_internode_metrics()
|
||||
.record_incoming_request_for_operation_and_backend(INTERNODE_OPERATION_WALK_DIR, INTERNODE_TRANSPORT_BACKEND_TCP_HTTP);
|
||||
let metrics = global_internode_metrics().clone();
|
||||
let metrics = resolve_internode_metrics();
|
||||
let stream = ReaderStream::with_capacity(rd, DEFAULT_READ_BUFFER_SIZE).map_ok(move |bytes| {
|
||||
metrics.record_sent_bytes_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_WALK_DIR,
|
||||
@@ -604,14 +604,15 @@ async fn handle_put_file(req: Request<Incoming>) -> Response<Body> {
|
||||
}
|
||||
};
|
||||
|
||||
global_internode_metrics().record_incoming_request_for_operation_and_backend(
|
||||
let metrics = resolve_internode_metrics();
|
||||
metrics.record_incoming_request_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_PUT_FILE_STREAM,
|
||||
INTERNODE_TRANSPORT_BACKEND_TCP_HTTP,
|
||||
);
|
||||
global_internode_metrics().record_recv_bytes_for_operation_and_backend(
|
||||
metrics.record_recv_bytes_for_operation_and_backend(
|
||||
INTERNODE_OPERATION_PUT_FILE_STREAM,
|
||||
INTERNODE_TRANSPORT_BACKEND_TCP_HTTP,
|
||||
copied as usize,
|
||||
usize::try_from(copied).unwrap_or(usize::MAX),
|
||||
);
|
||||
|
||||
if let Err(e) = file.flush().await {
|
||||
@@ -634,7 +635,7 @@ where
|
||||
let mut pending = BytesMut::with_capacity(DEFAULT_READ_BUFFER_SIZE);
|
||||
|
||||
while let Some(bytes) = body.try_next().await.map_err(io::Error::other)? {
|
||||
copied += bytes.len() as u64;
|
||||
copied = copied.saturating_add(u64::try_from(bytes.len()).unwrap_or(u64::MAX));
|
||||
pending.extend_from_slice(&bytes);
|
||||
|
||||
if pending.len() >= DEFAULT_READ_BUFFER_SIZE {
|
||||
|
||||
@@ -70,6 +70,7 @@
|
||||
//! ```
|
||||
|
||||
use super::StorageError;
|
||||
use crate::app::context::resolve_encryption_service;
|
||||
#[cfg(feature = "rio-v2")]
|
||||
use aes_gcm::aead::Payload;
|
||||
use aes_gcm::{
|
||||
@@ -86,7 +87,7 @@ use http::{HeaderMap, HeaderValue};
|
||||
use rand::Rng;
|
||||
#[cfg(feature = "rio-v2")]
|
||||
use rand::RngExt;
|
||||
use rustfs_kms::{DataKey, service_manager::get_global_encryption_service, types::ObjectEncryptionContext};
|
||||
use rustfs_kms::{DataKey, types::ObjectEncryptionContext};
|
||||
use rustfs_utils::get_env_opt_str;
|
||||
use s3s::S3ErrorCode;
|
||||
use s3s::dto::ServerSideEncryption;
|
||||
@@ -1532,7 +1533,7 @@ async fn apply_managed_encryption_material(
|
||||
let mut kms_key_candidate = kms_key_id.clone();
|
||||
if kms_key_candidate.is_none() {
|
||||
// Try to get default key from KMS service (if available)
|
||||
if let Some(service) = get_global_encryption_service().await {
|
||||
if let Some(service) = resolve_encryption_service().await {
|
||||
kms_key_candidate = service.get_default_key_id().cloned();
|
||||
}
|
||||
}
|
||||
@@ -1625,7 +1626,7 @@ async fn apply_managed_decryption_material(
|
||||
.cloned()
|
||||
.unwrap_or_else(|| "AES256".to_string()),
|
||||
)
|
||||
} else if let Some(service) = get_global_encryption_service().await {
|
||||
} else if let Some(service) = resolve_encryption_service().await {
|
||||
// Production mode: use service for metadata parsing
|
||||
let parsed = service
|
||||
.headers_to_metadata(&normalized_metadata)
|
||||
@@ -1767,7 +1768,7 @@ pub trait SseDekProvider: Send + Sync {
|
||||
// ============================================================================
|
||||
|
||||
/// Production KMS-backed DEK provider
|
||||
/// Resolves the latest global ObjectEncryptionService on each call.
|
||||
/// Resolves the latest ObjectEncryptionService on each call.
|
||||
struct KmsSseDekProvider;
|
||||
|
||||
impl KmsSseDekProvider {
|
||||
@@ -1780,7 +1781,7 @@ impl KmsSseDekProvider {
|
||||
}
|
||||
|
||||
async fn current_service() -> Option<Arc<rustfs_kms::service::ObjectEncryptionService>> {
|
||||
get_global_encryption_service().await
|
||||
resolve_encryption_service().await
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2040,7 +2041,7 @@ static GLOBAL_SSE_DEK_PROVIDER: LazyLock<RwLock<Option<Arc<dyn SseDekProvider>>>
|
||||
/// .await?;
|
||||
/// ```
|
||||
pub async fn get_sse_dek_provider() -> Result<Arc<dyn SseDekProvider>, ApiError> {
|
||||
if get_global_encryption_service().await.is_some() {
|
||||
if resolve_encryption_service().await.is_some() {
|
||||
debug!("Using KmsSseDekProvider (KMS configured)");
|
||||
return Ok(Arc::new(KmsSseDekProvider::new().await?));
|
||||
}
|
||||
|
||||
@@ -12,7 +12,8 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use crate::storage::{bucket_metadata_runtime_initialized, get_global_replication_pool, replication_queue_current_count};
|
||||
use crate::app::context::resolve_replication_pool_handle;
|
||||
use crate::storage::{bucket_metadata_runtime_initialized, replication_queue_current_count};
|
||||
use rustfs_concurrency::{
|
||||
AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
|
||||
WorkloadClass,
|
||||
@@ -151,7 +152,7 @@ fn repair_workload_admission_snapshot_from_counts(
|
||||
}
|
||||
|
||||
pub fn replication_workload_admission_snapshot() -> WorkloadAdmissionSnapshot {
|
||||
let Some(pool) = get_global_replication_pool() else {
|
||||
let Some(pool) = resolve_replication_pool_handle() else {
|
||||
return replication_workload_admission_snapshot_from_counts(false, None, None);
|
||||
};
|
||||
|
||||
|
||||
@@ -40,6 +40,8 @@ dep|rustfs/src/app/multipart_usecase.rs|app->interface|crate::storage::s3_api::m
|
||||
dep|rustfs/src/app/object_usecase.rs|app->interface|crate::storage::ecfs
|
||||
dep|rustfs/src/app/object_usecase.rs|app->interface|crate::storage::s3_api::multipart::parse_list_parts_params
|
||||
dep|rustfs/src/auth.rs|infra->app|crate::app::context::resolve_action_credentials
|
||||
dep|rustfs/src/config/info.rs|infra->app|crate::app::context::resolve_buffer_config
|
||||
dep|rustfs/src/init.rs|infra->app|crate::app::context::resolve_region
|
||||
dep|rustfs/src/init.rs|infra->interface|crate::admin
|
||||
dep|rustfs/src/protocols/client.rs|infra->app|crate::app::context::resolve_action_credentials
|
||||
dep|rustfs/src/protocols/client.rs|infra->interface|crate::storage::ecfs::FS
|
||||
@@ -52,6 +54,10 @@ dep|rustfs/src/server/layer.rs|infra->interface|crate::admin::handlers::health::
|
||||
dep|rustfs/src/server/layer.rs|infra->interface|crate::admin::handlers::health::build_health_response_parts
|
||||
dep|rustfs/src/server/readiness.rs|infra->app|crate::app::context::resolve_endpoints_handle
|
||||
dep|rustfs/src/server/readiness.rs|infra->app|crate::app::context::resolve_iam_ready
|
||||
dep|rustfs/src/server/readiness.rs|infra->app|crate::app::context::resolve_lock_clients_handle
|
||||
dep|rustfs/src/server/tls_material.rs|infra->app|crate::app::context::resolve_outbound_tls_generation
|
||||
dep|rustfs/src/startup_bucket_metadata.rs|infra->app|crate::app::context::resolve_replication_pool_handle
|
||||
dep|rustfs/src/startup_tls_material.rs|infra->app|crate::app::context::resolve_outbound_tls_generation
|
||||
dep|rustfs/src/startup_iam.rs|infra->app|crate::app::context::AppContext
|
||||
dep|rustfs/src/storage/ecfs_extend.rs|infra->app|crate::app::context::resolve_buffer_config
|
||||
dep|rustfs/src/storage/ecfs_extend.rs|infra->interface|crate::storage::ecfs::ListObjectUnorderedQuery
|
||||
@@ -59,10 +65,16 @@ dep|rustfs/src/storage/ecfs_test.rs|infra->interface|crate::storage::ecfs::FS
|
||||
dep|rustfs/src/storage/ecfs_test.rs|infra->interface|crate::storage::ecfs::validate_object_lock_configuration_input
|
||||
dep|rustfs/src/storage/ecfs_test.rs|infra->interface|crate::storage::s3_api::common::rustfs_initiator
|
||||
dep|rustfs/src/storage/ecfs_test.rs|infra->interface|crate::storage::s3_api::common::rustfs_owner
|
||||
dep|rustfs/src/storage/access.rs|infra->app|crate::app::context::resolve_region
|
||||
dep|rustfs/src/storage/concurrency/manager.rs|infra->app|crate::app::context::resolve_performance_metrics
|
||||
dep|rustfs/src/storage/sse.rs|infra->app|crate::app::context::resolve_encryption_service
|
||||
dep|rustfs/src/storage/helper.rs|infra->app|crate::app::context::resolve_action_credentials
|
||||
dep|rustfs/src/storage/rpc/disk.rs|infra->app|crate::app::context::resolve_internode_metrics
|
||||
dep|rustfs/src/storage/rpc/health.rs|infra->app|crate::app::context::resolve_local_node_name
|
||||
dep|rustfs/src/storage/rpc/http_service.rs|infra->app|crate::app::context::resolve_internode_metrics
|
||||
dep|rustfs/src/storage/rpc/node_service.rs|infra->app|crate::app::context::resolve_iam_handle
|
||||
dep|rustfs/src/storage/rpc/node_service.rs|infra->app|crate::app::context::resolve_lock_client
|
||||
dep|rustfs/src/storage/rpc/node_service.rs|infra->interface|crate::admin::service::config::reload_dynamic_config_runtime_state
|
||||
dep|rustfs/src/storage/rpc/node_service.rs|infra->interface|crate::admin::service::config::reload_runtime_config_snapshot
|
||||
dep|rustfs/src/storage/rpc/node_service.rs|infra->interface|crate::admin::service::site_replication::reload_site_replication_runtime_state
|
||||
dep|rustfs/src/workload_admission.rs|infra->app|crate::app::context::resolve_replication_pool_handle
|
||||
|
||||
Reference in New Issue
Block a user