mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-06 05:17:42 +00:00
Zhengchao An
957080bea5
Swift container and account metadata handlers cloned the cached BucketMetadata, set the tagging fields, and called set_bucket_metadata, which only updates the in-memory cache map. Nothing reached .metadata.bin, so every Swift metadata POST was lost on restart and silently overwritten by the next disk-truth reload (a peer LoadBucketMetadata notification or the 15-minute refresh loop) — while the client had already been told 2xx. Route these writes through a new metadata_sys::update_config_with: a read-modify-write that loads the on-disk metadata and persists the result under the same write guard metadata_sys::update uses, so the rewrite merges against disk truth instead of a possibly stale cache and cannot clobber a concurrent update to another config file. Peers are notified afterwards, matching the S3 config handlers. Persisting these writes required hardening the paths that now produce durable state: - Account metadata writes validate account ownership. This metadata holds the account's TempURL signing key, so an unauthenticated write for someone else's account would have become a durable, cluster-wide takeover of that account's pre-signed URLs. Reads stay open because TempURL signature validation runs before credentials exist. - disable_versioning verifies the container exists. Without it the metadata loader's "no metadata on disk" default would be persisted, creating an orphan metadata file and caching a fabricated default as authoritative. - Container and account metadata are size- and count-limited, reusing the Swift limits object metadata already enforces; these tags land in the bucket metadata file that every later config write rewrites whole. - A rewrite refuses to run when the persisted tagging config is unreadable, instead of merging onto an empty set and wiping the container ACL and versioning tags. It reports 409 naming the remedy. - Storage errors are logged in full and reported generically, since they now carry real disk and quorum detail. The tagging arm of BucketMetadata::update_config also clears the parsed config, as the lifecycle arm does: parse_all_configs skips empty XML rather than clearing, so a cleared config kept serving the old tags. Tagging is serialized with the S3 XML serializer the loader can parse back, not quick_xml, whose output was never round-trippable.
111 lines
3.7 KiB
Rust
111 lines
3.7 KiB
Rust
// Copyright 2024 RustFS Team
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
//! OpenStack Swift API implementation
|
|
//!
|
|
//! This module provides support for the OpenStack Swift object storage API,
|
|
//! enabling RustFS to serve as a Swift-compatible storage backend while
|
|
//! reusing the existing S3 storage layer.
|
|
//!
|
|
//! # Architecture
|
|
//!
|
|
//! Swift requests follow the pattern: `/v1/{account}/{container}/{object}`
|
|
//! where:
|
|
//! - `account`: Tenant identifier (e.g., `AUTH_{project_id}`)
|
|
//! - `container`: Swift container (maps to S3 bucket)
|
|
//! - `object`: Object key (maps to S3 object key)
|
|
//!
|
|
//! # Authentication
|
|
//!
|
|
//! Swift API uses Keystone token-based authentication via the existing
|
|
//! `KeystoneAuthMiddleware`. The middleware validates X-Auth-Token headers
|
|
//! and stores credentials in task-local storage, which Swift handlers access
|
|
//! to enforce tenant isolation.
|
|
|
|
pub mod account;
|
|
pub mod acl;
|
|
pub mod bulk;
|
|
pub mod container;
|
|
pub mod cors;
|
|
pub mod dlo;
|
|
pub mod errors;
|
|
pub mod expiration;
|
|
pub mod expiration_worker;
|
|
pub mod formpost;
|
|
pub mod handler;
|
|
pub mod object;
|
|
pub mod quota;
|
|
pub mod router;
|
|
pub mod slo;
|
|
pub mod staticweb;
|
|
mod storage_api;
|
|
pub mod symlink;
|
|
pub mod sync;
|
|
pub mod tempurl;
|
|
pub mod types;
|
|
pub mod versioning;
|
|
|
|
pub use errors::{SwiftError, SwiftResult};
|
|
pub use router::{SwiftRoute, SwiftRouter};
|
|
|
|
/// Maximum number of metadata headers allowed per resource (Swift standard)
|
|
pub(crate) const MAX_METADATA_COUNT: usize = 90;
|
|
|
|
/// Maximum size in bytes for a single metadata value (Swift standard)
|
|
pub(crate) const MAX_METADATA_VALUE_SIZE: usize = 256;
|
|
|
|
/// Validate metadata against Swift limits
|
|
///
|
|
/// Checks that:
|
|
/// - Total number of metadata entries doesn't exceed MAX_METADATA_COUNT
|
|
/// - Individual metadata values don't exceed MAX_METADATA_VALUE_SIZE
|
|
///
|
|
/// Applies to object, container and account metadata alike: all three are
|
|
/// persisted, and container/account metadata additionally lands in the
|
|
/// bucket metadata file that every later config write rewrites in full.
|
|
///
|
|
/// Returns error if limits are exceeded.
|
|
pub(crate) fn validate_metadata(metadata: &std::collections::HashMap<String, String>) -> SwiftResult<()> {
|
|
// Check total metadata count
|
|
if metadata.len() > MAX_METADATA_COUNT {
|
|
return Err(SwiftError::BadRequest(format!(
|
|
"Too many metadata headers: {} (max: {})",
|
|
metadata.len(),
|
|
MAX_METADATA_COUNT
|
|
)));
|
|
}
|
|
|
|
// Check individual value sizes
|
|
for (key, value) in metadata.iter() {
|
|
if value.len() > MAX_METADATA_VALUE_SIZE {
|
|
return Err(SwiftError::BadRequest(format!(
|
|
"Metadata value for '{}' too large: {} bytes (max: {} bytes)",
|
|
key,
|
|
value.len(),
|
|
MAX_METADATA_VALUE_SIZE
|
|
)));
|
|
}
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
// Note: Container, Object, and SwiftMetadata types used by Swift implementation
|
|
pub use storage_api::public_api::{SwiftGetObjectReader, SwiftObjectInfo, SwiftObjectOptions, SwiftPutObjReader};
|
|
pub(crate) use storage_api::public_api::{
|
|
get_swift_bucket_metadata, get_swift_bucket_usage, resolve_swift_object_store_handle, update_swift_bucket_tagging,
|
|
};
|
|
#[allow(unused_imports)]
|
|
pub use types::{Container, Object, SwiftMetadata};
|