docs(obs): inventory ecstore dependency boundary (#4023)

This commit is contained in:
Zhengchao An
2026-06-29 07:07:47 +08:00
committed by GitHub
parent 8284e6a75c
commit 1104d630d1
5 changed files with 88 additions and 1 deletions
+2 -1
View File
@@ -41,7 +41,8 @@ rustfs-config = { workspace = true, features = ["constants", "observability"] }
# The obs crate imports types from ecstore for metrics collection.
# Breaking this dependency would require defining traits in obs and
# implementing them in ecstore, which is a significant refactoring.
# See https://github.com/rustfs/backlog/issues/735 for discussion.
# See docs/architecture/obs-ecstore-dependency-inventory.md and
# https://github.com/rustfs/backlog/issues/735 for discussion.
rustfs-ecstore = { workspace = true }
rustfs-iam = { workspace = true }
rustfs-io-metrics = { workspace = true }
+3
View File
@@ -243,6 +243,9 @@ boundary files after the external runtime, test, and fuzz consumers have been
narrowed. IAM, heal, scanner, notify, observability, Swift, S3 Select, e2e, and
fuzz code must not reintroduce direct `rustfs_ecstore::api::...` references
outside those boundary files.
The observability ECStore dependency baseline is tracked in
[`obs-ecstore-dependency-inventory.md`](obs-ecstore-dependency-inventory.md);
future observability extraction PRs must update that inventory with the guard.
ECStore ClusterControlPlane read models must stay owned by the crate-private
`cluster` module. Public access goes through `rustfs_ecstore::api::cluster` so
@@ -0,0 +1,64 @@
# Observability ECStore Dependency Inventory
This inventory closes the first `rustfs/backlog#735` step: make every
observability dependency on ECStore visible before introducing traits or moving
dependency direction.
No behavior or crate movement is planned in inventory PRs. The current boundary
is `crates/obs/src/metrics/storage_api.rs`; all direct `rustfs_ecstore` and
`rustfs_storage_api` source references in `rustfs-obs` must stay in that file
until the contracts below are extracted.
## Dependency Inventory
| Current symbol in `crates/obs/src/metrics/storage_api.rs` | Consumed by | Classification | Purpose |
|---|---|---|---|
| `rustfs_ecstore::api::storage::ECStore` as `ObsStore` | `stats_collector.rs` | Type dependency | Concrete object-store handle used to call storage admin methods and data-usage loaders. |
| `rustfs_storage_api::{BucketOperations, BucketOptions, StorageAdminApi}` | `stats_collector.rs` | Type and trait dependency | Method-resolution and associated type contracts for bucket listing, backend info, and storage info. |
| `rustfs_ecstore::api::runtime::object_store_handle` | `stats_collector.rs` | Runtime dependency | Resolves the currently published object-store handle for metric collection. |
| `rustfs_ecstore::api::data_usage::load_data_usage_from_backend` | `stats_collector.rs` | Behavior dependency | Loads bucket/object usage and is projected into obs-local DTOs before collectors consume it. |
| `rustfs_ecstore::api::capacity::{get_total_usable_capacity, get_total_usable_capacity_free}` | `stats_collector.rs` | Behavior dependency | Computes usable and free capacity from ECStore storage info. |
| `rustfs_ecstore::api::bucket::metadata_sys::get_quota_config` | `stats_collector.rs` | Behavior dependency | Reads per-bucket quota limits used in bucket usage metrics. |
| `rustfs_ecstore::api::bucket::bandwidth::monitor::Monitor` | `runtime_sources.rs`, `stats_collector.rs` | Type and runtime dependency | Reads replication bandwidth reports from the global bucket monitor handle. |
| `rustfs_ecstore::api::runtime::bucket_monitor` | `runtime_sources.rs` | Runtime dependency | Resolves the global bucket bandwidth monitor for metric collection. |
| `rustfs_ecstore::api::bucket::replication::{GLOBAL_REPLICATION_STATS, ReplicationStats}` | `runtime_sources.rs`, `stats_collector.rs` | Type and runtime dependency | Reads replication status, transfer, failure, and site-replication stats. |
| `rustfs_ecstore::api::bucket::lifecycle::bucket_lifecycle_ops::{GLOBAL_ExpiryState, GLOBAL_TransitionState}` | `runtime_sources.rs`, `stats_collector.rs` | Runtime dependency | Reads lifecycle expiry and transition queue counters. |
| `rustfs_ecstore::api::error::Result` as `ObsEcstoreResult` | `stats_collector.rs` | Type dependency | Preserves ECStore error propagation while data-usage behavior remains ECStore-owned. |
## Classification
The remaining coupling is not just a dependency declaration problem:
- type coupling: `ObsStore`, `ObsEcstoreResult`, `ObsReplicationStats`,
`ObsBucketBandwidthMonitor`, `StorageAdminApi`, `BucketOperations`, and
`BucketOptions`;
- runtime handle coupling: object-store handle, bucket monitor, replication
stats, expiry state, and transition state;
- behavior coupling: data-usage loading, quota lookup, and capacity math.
Removing `rustfs-ecstore` from `crates/obs/Cargo.toml` is unsafe until those
three categories have replacement contracts and compile coverage.
## Extraction Plan
1. Keep all direct ECStore and storage-api imports centralized in
`crates/obs/src/metrics/storage_api.rs`.
2. Keep projecting ECStore data-usage output into obs-local DTOs before
collectors consume it.
3. Introduce obs-owned provider traits for storage info, bucket info, quota,
data usage, replication, bandwidth, and lifecycle queue snapshots.
4. Implement those traits in ECStore or an ECStore-owned adapter crate after the
trait shapes are covered by focused tests.
5. Remove the `rustfs-ecstore` dependency from `rustfs-obs` only after metrics
behavior is unchanged through the provider traits.
## Guardrails
The architecture guard enforces this inventory boundary:
- `crates/obs/src/metrics/storage_api.rs` is the only `rustfs-obs` source file
allowed to reference `rustfs_ecstore` or `rustfs_storage_api`;
- `rustfs-obs` must not add `storage_compat.rs` or `ecstore_compat.rs`
passthrough bridges;
- future extraction PRs must update this inventory and the guard in the same
reviewed change when a dependency category is removed.
+3
View File
@@ -38,6 +38,9 @@ hot-path behavior must not drift during this migration.
- [`global-state-crate-split-plan.md`](global-state-crate-split-plan.md): late
global-state cleanup, runtime-source boundaries, fallback removal rules, and
crate-split evaluation criteria.
- [`obs-ecstore-dependency-inventory.md`](obs-ecstore-dependency-inventory.md):
observability-to-ECStore dependency inventory, classification, and extraction
guardrails.
- [`ecstore-config-consumer-inventory.md`](ecstore-config-consumer-inventory.md):
current `ecstore::config::{Config, KV, KVS}` definitions, consumers,
migration risks, and do-not-change contract.
@@ -67,6 +67,9 @@ require_source_contains "docs/architecture/global-state-crate-split-plan.md" "##
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Fallback Removal Plan" "global state plan fallback section"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Crate Split Evaluation" "global state plan crate split section"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "rustfs_ecstore::api::global" "global state plan facade boundary"
require_source_contains "docs/architecture/obs-ecstore-dependency-inventory.md" "## Dependency Inventory" "observability ECStore dependency inventory section"
require_source_contains "docs/architecture/obs-ecstore-dependency-inventory.md" "## Extraction Plan" "observability ECStore extraction plan section"
require_source_contains "docs/architecture/obs-ecstore-dependency-inventory.md" "crates/obs/src/metrics/storage_api.rs" "observability ECStore storage_api boundary"
TMP_DIR="$(mktemp -d)"
trap 'rm -rf "$TMP_DIR"' EXIT
@@ -199,6 +202,7 @@ SCANNER_STORAGE_API_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/scanner_storage_api_sour
SCANNER_STORAGE_API_TEST_BYPASS_HITS_FILE="${TMP_DIR}/scanner_storage_api_test_bypass_hits.txt"
HEAL_STORAGE_API_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/heal_storage_api_source_bypass_hits.txt"
HEAL_STORAGE_API_TEST_BYPASS_HITS_FILE="${TMP_DIR}/heal_storage_api_test_bypass_hits.txt"
OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/obs_storage_api_source_bypass_hits.txt"
NOTIFY_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/notify_storage_compat_module_hits.txt"
OBS_STORAGE_COMPAT_PASSTHROUGH_HITS_FILE="${TMP_DIR}/obs_storage_compat_passthrough_hits.txt"
E2E_STORAGE_COMPAT_RPC_PASSTHROUGH_HITS_FILE="${TMP_DIR}/e2e_storage_compat_rpc_passthrough_hits.txt"
@@ -2725,6 +2729,18 @@ if [[ -s "$HEAL_STORAGE_API_TEST_BYPASS_HITS_FILE" ]]; then
report_failure "heal tests must route ECStore and storage-api symbols through heal test storage_api: $(paste -sd '; ' "$HEAL_STORAGE_API_TEST_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'rustfs_ecstore::|^use rustfs_storage_api|rustfs_storage_api::' \
crates/obs/src \
--glob '*.rs' |
rg -v '^crates/obs/src/metrics/storage_api\.rs:' || true
) >"$OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE"
if [[ -s "$OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE" ]]; then
report_failure "obs source files must route ECStore and storage-api symbols through obs metrics storage_api boundary: $(paste -sd '; ' "$OBS_STORAGE_API_SOURCE_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
if [[ -f crates/notify/src/storage_compat.rs ]]; then