mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
docs(obs): inventory ecstore dependency boundary (#4023)
This commit is contained in:
@@ -41,7 +41,8 @@ rustfs-config = { workspace = true, features = ["constants", "observability"] }
|
||||
# The obs crate imports types from ecstore for metrics collection.
|
||||
# Breaking this dependency would require defining traits in obs and
|
||||
# implementing them in ecstore, which is a significant refactoring.
|
||||
# See https://github.com/rustfs/backlog/issues/735 for discussion.
|
||||
# See docs/architecture/obs-ecstore-dependency-inventory.md and
|
||||
# https://github.com/rustfs/backlog/issues/735 for discussion.
|
||||
rustfs-ecstore = { workspace = true }
|
||||
rustfs-iam = { workspace = true }
|
||||
rustfs-io-metrics = { workspace = true }
|
||||
|
||||
@@ -243,6 +243,9 @@ boundary files after the external runtime, test, and fuzz consumers have been
|
||||
narrowed. IAM, heal, scanner, notify, observability, Swift, S3 Select, e2e, and
|
||||
fuzz code must not reintroduce direct `rustfs_ecstore::api::...` references
|
||||
outside those boundary files.
|
||||
The observability ECStore dependency baseline is tracked in
|
||||
[`obs-ecstore-dependency-inventory.md`](obs-ecstore-dependency-inventory.md);
|
||||
future observability extraction PRs must update that inventory with the guard.
|
||||
|
||||
ECStore ClusterControlPlane read models must stay owned by the crate-private
|
||||
`cluster` module. Public access goes through `rustfs_ecstore::api::cluster` so
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# Observability ECStore Dependency Inventory
|
||||
|
||||
This inventory closes the first `rustfs/backlog#735` step: make every
|
||||
observability dependency on ECStore visible before introducing traits or moving
|
||||
dependency direction.
|
||||
|
||||
No behavior or crate movement is planned in inventory PRs. The current boundary
|
||||
is `crates/obs/src/metrics/storage_api.rs`; all direct `rustfs_ecstore` and
|
||||
`rustfs_storage_api` source references in `rustfs-obs` must stay in that file
|
||||
until the contracts below are extracted.
|
||||
|
||||
## Dependency Inventory
|
||||
|
||||
| Current symbol in `crates/obs/src/metrics/storage_api.rs` | Consumed by | Classification | Purpose |
|
||||
|---|---|---|---|
|
||||
| `rustfs_ecstore::api::storage::ECStore` as `ObsStore` | `stats_collector.rs` | Type dependency | Concrete object-store handle used to call storage admin methods and data-usage loaders. |
|
||||
| `rustfs_storage_api::{BucketOperations, BucketOptions, StorageAdminApi}` | `stats_collector.rs` | Type and trait dependency | Method-resolution and associated type contracts for bucket listing, backend info, and storage info. |
|
||||
| `rustfs_ecstore::api::runtime::object_store_handle` | `stats_collector.rs` | Runtime dependency | Resolves the currently published object-store handle for metric collection. |
|
||||
| `rustfs_ecstore::api::data_usage::load_data_usage_from_backend` | `stats_collector.rs` | Behavior dependency | Loads bucket/object usage and is projected into obs-local DTOs before collectors consume it. |
|
||||
| `rustfs_ecstore::api::capacity::{get_total_usable_capacity, get_total_usable_capacity_free}` | `stats_collector.rs` | Behavior dependency | Computes usable and free capacity from ECStore storage info. |
|
||||
| `rustfs_ecstore::api::bucket::metadata_sys::get_quota_config` | `stats_collector.rs` | Behavior dependency | Reads per-bucket quota limits used in bucket usage metrics. |
|
||||
| `rustfs_ecstore::api::bucket::bandwidth::monitor::Monitor` | `runtime_sources.rs`, `stats_collector.rs` | Type and runtime dependency | Reads replication bandwidth reports from the global bucket monitor handle. |
|
||||
| `rustfs_ecstore::api::runtime::bucket_monitor` | `runtime_sources.rs` | Runtime dependency | Resolves the global bucket bandwidth monitor for metric collection. |
|
||||
| `rustfs_ecstore::api::bucket::replication::{GLOBAL_REPLICATION_STATS, ReplicationStats}` | `runtime_sources.rs`, `stats_collector.rs` | Type and runtime dependency | Reads replication status, transfer, failure, and site-replication stats. |
|
||||
| `rustfs_ecstore::api::bucket::lifecycle::bucket_lifecycle_ops::{GLOBAL_ExpiryState, GLOBAL_TransitionState}` | `runtime_sources.rs`, `stats_collector.rs` | Runtime dependency | Reads lifecycle expiry and transition queue counters. |
|
||||
| `rustfs_ecstore::api::error::Result` as `ObsEcstoreResult` | `stats_collector.rs` | Type dependency | Preserves ECStore error propagation while data-usage behavior remains ECStore-owned. |
|
||||
|
||||
## Classification
|
||||
|
||||
The remaining coupling is not just a dependency declaration problem:
|
||||
|
||||
- type coupling: `ObsStore`, `ObsEcstoreResult`, `ObsReplicationStats`,
|
||||
`ObsBucketBandwidthMonitor`, `StorageAdminApi`, `BucketOperations`, and
|
||||
`BucketOptions`;
|
||||
- runtime handle coupling: object-store handle, bucket monitor, replication
|
||||
stats, expiry state, and transition state;
|
||||
- behavior coupling: data-usage loading, quota lookup, and capacity math.
|
||||
|
||||
Removing `rustfs-ecstore` from `crates/obs/Cargo.toml` is unsafe until those
|
||||
three categories have replacement contracts and compile coverage.
|
||||
|
||||
## Extraction Plan
|
||||
|
||||
1. Keep all direct ECStore and storage-api imports centralized in
|
||||
`crates/obs/src/metrics/storage_api.rs`.
|
||||
2. Keep projecting ECStore data-usage output into obs-local DTOs before
|
||||
collectors consume it.
|
||||
3. Introduce obs-owned provider traits for storage info, bucket info, quota,
|
||||
data usage, replication, bandwidth, and lifecycle queue snapshots.
|
||||
4. Implement those traits in ECStore or an ECStore-owned adapter crate after the
|
||||
trait shapes are covered by focused tests.
|
||||
5. Remove the `rustfs-ecstore` dependency from `rustfs-obs` only after metrics
|
||||
behavior is unchanged through the provider traits.
|
||||
|
||||
## Guardrails
|
||||
|
||||
The architecture guard enforces this inventory boundary:
|
||||
|
||||
- `crates/obs/src/metrics/storage_api.rs` is the only `rustfs-obs` source file
|
||||
allowed to reference `rustfs_ecstore` or `rustfs_storage_api`;
|
||||
- `rustfs-obs` must not add `storage_compat.rs` or `ecstore_compat.rs`
|
||||
passthrough bridges;
|
||||
- future extraction PRs must update this inventory and the guard in the same
|
||||
reviewed change when a dependency category is removed.
|
||||
@@ -38,6 +38,9 @@ hot-path behavior must not drift during this migration.
|
||||
- [`global-state-crate-split-plan.md`](global-state-crate-split-plan.md): late
|
||||
global-state cleanup, runtime-source boundaries, fallback removal rules, and
|
||||
crate-split evaluation criteria.
|
||||
- [`obs-ecstore-dependency-inventory.md`](obs-ecstore-dependency-inventory.md):
|
||||
observability-to-ECStore dependency inventory, classification, and extraction
|
||||
guardrails.
|
||||
- [`ecstore-config-consumer-inventory.md`](ecstore-config-consumer-inventory.md):
|
||||
current `ecstore::config::{Config, KV, KVS}` definitions, consumers,
|
||||
migration risks, and do-not-change contract.
|
||||
|
||||
@@ -67,6 +67,9 @@ require_source_contains "docs/architecture/global-state-crate-split-plan.md" "##
|
||||
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Fallback Removal Plan" "global state plan fallback section"
|
||||
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Crate Split Evaluation" "global state plan crate split section"
|
||||
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "rustfs_ecstore::api::global" "global state plan facade boundary"
|
||||
require_source_contains "docs/architecture/obs-ecstore-dependency-inventory.md" "## Dependency Inventory" "observability ECStore dependency inventory section"
|
||||
require_source_contains "docs/architecture/obs-ecstore-dependency-inventory.md" "## Extraction Plan" "observability ECStore extraction plan section"
|
||||
require_source_contains "docs/architecture/obs-ecstore-dependency-inventory.md" "crates/obs/src/metrics/storage_api.rs" "observability ECStore storage_api boundary"
|
||||
|
||||
TMP_DIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP_DIR"' EXIT
|
||||
@@ -199,6 +202,7 @@ SCANNER_STORAGE_API_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/scanner_storage_api_sour
|
||||
SCANNER_STORAGE_API_TEST_BYPASS_HITS_FILE="${TMP_DIR}/scanner_storage_api_test_bypass_hits.txt"
|
||||
HEAL_STORAGE_API_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/heal_storage_api_source_bypass_hits.txt"
|
||||
HEAL_STORAGE_API_TEST_BYPASS_HITS_FILE="${TMP_DIR}/heal_storage_api_test_bypass_hits.txt"
|
||||
OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/obs_storage_api_source_bypass_hits.txt"
|
||||
NOTIFY_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/notify_storage_compat_module_hits.txt"
|
||||
OBS_STORAGE_COMPAT_PASSTHROUGH_HITS_FILE="${TMP_DIR}/obs_storage_compat_passthrough_hits.txt"
|
||||
E2E_STORAGE_COMPAT_RPC_PASSTHROUGH_HITS_FILE="${TMP_DIR}/e2e_storage_compat_rpc_passthrough_hits.txt"
|
||||
@@ -2725,6 +2729,18 @@ if [[ -s "$HEAL_STORAGE_API_TEST_BYPASS_HITS_FILE" ]]; then
|
||||
report_failure "heal tests must route ECStore and storage-api symbols through heal test storage_api: $(paste -sd '; ' "$HEAL_STORAGE_API_TEST_BYPASS_HITS_FILE")"
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
rg -n --with-filename 'rustfs_ecstore::|^use rustfs_storage_api|rustfs_storage_api::' \
|
||||
crates/obs/src \
|
||||
--glob '*.rs' |
|
||||
rg -v '^crates/obs/src/metrics/storage_api\.rs:' || true
|
||||
) >"$OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE"
|
||||
|
||||
if [[ -s "$OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE" ]]; then
|
||||
report_failure "obs source files must route ECStore and storage-api symbols through obs metrics storage_api boundary: $(paste -sd '; ' "$OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE")"
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
if [[ -f crates/notify/src/storage_compat.rs ]]; then
|
||||
|
||||
Reference in New Issue
Block a user