feat(ecstore): run bucket operations on the store's own instance context (#4642)

backlog#1052 S7 — the final piece: full bucket-namespace isolation
between embedded servers in one process.

Server B's requests resolved B's own ECStore (per-server dispatch landed
earlier), but the store's bucket operations still went through ambient
process facades, so both servers effectively operated on the FIRST
server's disks and metadata:

- LocalPeerS3Client::local_disks_for_pools() called all_local_disk()
  (the ambient disk registry = the first published store's context), so
  list/make/delete/heal bucket scanned and wrote the wrong volumes.
- BucketMetadata::save() persisted through the ambient object handle, so
  a second server's bucket metadata landed in the first server's
  .rustfs.sys; set/remove/get/created_at all used the ambient metadata
  system.

Now the whole chain is bound to the owning store's InstanceContext:

- S3PeerSys/LocalPeerS3Client gain *_with_instance_ctx constructors and
  operate on that context's registered disks; ECStore::new (and the test
  store builder) pass the store's context. The legacy constructors keep
  the bootstrap default.
- BucketMetadata::save_with_store persists through an explicit store;
  BucketMetadataSys::persist_and_set uses the system's own api handle.
- metadata_sys gains instance-scoped variants (get_in / created_at_in /
  set_bucket_metadata_in / remove_bucket_metadata_in) that resolve the
  context's metadata system and fall back to the ambient default before
  the instance cell is initialized (early startup, unchanged behavior).
- The store's bucket handlers (make/get_info/list/delete + the
  table-bucket delete guard and emptiness check) use the per-context
  variants and this instance's disks.

Acceptance (e2e): two embedded servers with different credentials are now
isolated end to end — each authenticates only its own key, neither sees
the other's buckets or objects, and both data planes stay intact. The
embedded module doc drops the shared-IAM caveat.

579 ecstore bucket/metadata/peer regressions plus the embedded basic and
deferred-IAM e2e stay green.
This commit is contained in:
Zhengchao An
2026-07-10 10:52:51 +08:00
committed by GitHub
parent e6e4aef45b
commit dc3099bf0f
9 changed files with 192 additions and 44 deletions
+8
View File
@@ -813,6 +813,14 @@ impl BucketMetadata {
return Err(Error::other("errServerNotInitialized"));
};
self.save_with_store(store).await
}
/// Persist this metadata through an explicit store (backlog#1052 S7): the
/// owning instance's metadata system passes its own store so a second
/// server's bucket metadata lands in that server's `.rustfs.sys`, not the
/// ambient (first) one. [`BucketMetadata::save`] keeps the ambient default.
pub async fn save_with_store(&mut self, store: std::sync::Arc<crate::store::ECStore>) -> Result<()> {
self.parse_all_configs()?;
let mut buf: Vec<u8> = vec![0; 4];
+46 -1
View File
@@ -181,6 +181,44 @@ pub async fn get(bucket: &str) -> Result<Arc<BucketMetadata>> {
lock.get(bucket).await
}
// ---- Instance-scoped variants (backlog#1052 S7) ----
//
// A store's own bucket operations resolve the metadata system of *their*
// instance context so two servers in one process stay isolated; when the
// instance cell is not initialized yet (early startup) they fall back to the
// ambient default — the single-instance legacy behavior.
fn bucket_metadata_sys_of(ctx: &crate::runtime::instance::InstanceContext) -> Result<Arc<RwLock<BucketMetadataSys>>> {
if let Some(sys) = ctx.bucket_metadata_sys() {
return Ok(sys);
}
get_bucket_metadata_sys()
}
pub(crate) async fn get_in(ctx: &crate::runtime::instance::InstanceContext, bucket: &str) -> Result<Arc<BucketMetadata>> {
let sys = bucket_metadata_sys_of(ctx)?;
let lock = sys.read().await;
lock.get(bucket).await
}
pub(crate) async fn created_at_in(ctx: &crate::runtime::instance::InstanceContext, bucket: &str) -> Result<OffsetDateTime> {
let sys = bucket_metadata_sys_of(ctx)?;
let lock = sys.read().await;
lock.created_at(bucket).await
}
pub(crate) async fn set_bucket_metadata_in(ctx: &crate::runtime::instance::InstanceContext, bm: BucketMetadata) -> Result<()> {
let sys = bucket_metadata_sys_of(ctx)?;
let lock = sys.read().await;
lock.persist_and_set(bm).await
}
pub(crate) async fn remove_bucket_metadata_in(ctx: &crate::runtime::instance::InstanceContext, bucket: &str) -> Result<bool> {
let sys = bucket_metadata_sys_of(ctx)?;
let lock = sys.read().await;
Ok(lock.remove(bucket).await)
}
pub async fn update(bucket: &str, config_file: &str, data: Vec<u8>) -> Result<OffsetDateTime> {
let bucket_meta_sys_lock = get_bucket_metadata_sys()?;
let mut bucket_meta_sys = bucket_meta_sys_lock.write().await;
@@ -531,9 +569,16 @@ impl BucketMetadataSys {
return Err(Error::other("errInvalidArgument"));
}
self.persist_and_set(bm).await
}
/// Persist metadata through this system's own store and cache it here
/// (backlog#1052 S7). The store-scoped bucket path uses this so a second
/// server's metadata never leaks into the ambient (first) instance.
pub(crate) async fn persist_and_set(&self, bm: BucketMetadata) -> Result<()> {
let mut bm = bm;
bm.save().await?;
bm.save_with_store(self.api.clone()).await?;
self.set(bm.name.clone(), Arc::new(bm)).await;