Commit Graph

21 Commits

Author SHA1 Message Date
Zhengchao An 957080bea5 fix(swift): persist container and account metadata writes (#5398)
Swift container and account metadata handlers cloned the cached
BucketMetadata, set the tagging fields, and called set_bucket_metadata,
which only updates the in-memory cache map. Nothing reached
.metadata.bin, so every Swift metadata POST was lost on restart and
silently overwritten by the next disk-truth reload (a peer
LoadBucketMetadata notification or the 15-minute refresh loop) — while
the client had already been told 2xx.

Route these writes through a new metadata_sys::update_config_with: a
read-modify-write that loads the on-disk metadata and persists the
result under the same write guard metadata_sys::update uses, so the
rewrite merges against disk truth instead of a possibly stale cache and
cannot clobber a concurrent update to another config file. Peers are
notified afterwards, matching the S3 config handlers.

Persisting these writes required hardening the paths that now produce
durable state:

- Account metadata writes validate account ownership. This metadata
  holds the account's TempURL signing key, so an unauthenticated write
  for someone else's account would have become a durable, cluster-wide
  takeover of that account's pre-signed URLs. Reads stay open because
  TempURL signature validation runs before credentials exist.
- disable_versioning verifies the container exists. Without it the
  metadata loader's "no metadata on disk" default would be persisted,
  creating an orphan metadata file and caching a fabricated default as
  authoritative.
- Container and account metadata are size- and count-limited, reusing
  the Swift limits object metadata already enforces; these tags land in
  the bucket metadata file that every later config write rewrites whole.
- A rewrite refuses to run when the persisted tagging config is
  unreadable, instead of merging onto an empty set and wiping the
  container ACL and versioning tags. It reports 409 naming the remedy.
- Storage errors are logged in full and reported generically, since they
  now carry real disk and quorum detail.

The tagging arm of BucketMetadata::update_config also clears the parsed
config, as the lifecycle arm does: parse_all_configs skips empty XML
rather than clearing, so a cleared config kept serving the old tags.
Tagging is serialized with the S3 XML serializer the loader can parse
back, not quick_xml, whose output was never round-trippable.
2026-07-29 02:31:11 +00:00
Zhengchao An 7cd7c84e71 feat(swift): track object expiration (#4409)
fix(swift): wire object expiration tracking
2026-07-08 14:30:46 +08:00
Zhengchao An 1f7e159388 refactor: segment external storage api boundaries (#3903) 2026-06-26 15:10:49 +08:00
Zhengchao An e37e367390 refactor: route remaining external storage boundaries (#3889) 2026-06-26 06:22:51 +08:00
Zhengchao An 539c68778d refactor: remove standalone compat bridge modules (#3740) 2026-06-22 17:35:58 +08:00
安正超 c76e4dc6bc refactor: clean Swift ECStore boundaries (#3570) 2026-06-18 15:18:55 +08:00
安正超 0e7e2660bb refactor: clean remaining storage DTO imports (#3568) 2026-06-18 14:44:46 +08:00
安正超 acdf439371 refactor: clean external DTO consumers (#3566)
* refactor: clean external DTO consumers

* fix: handle empty erasure shard recovery
2026-06-18 14:22:44 +08:00
安正超 e5cad7ed20 refactor: move object operation contracts (#3559) 2026-06-18 09:48:13 +08:00
安正超 53337a6f71 refactor: move HTTP range helper contracts (#3533) 2026-06-18 00:47:23 +08:00
houseme 841f86fe90 test(property): expand parser invariants (#3534)
* test(property): harden parser invariants for #3522

* test(property): satisfy clippy for #3522
2026-06-17 23:52:30 +08:00
安正超 ed55857bca refactor: move bucket operations contract (#3507) 2026-06-17 09:10:38 +08:00
安正超 84f7027e90 refactor: add object store resolver for standalone crates (#3437) 2026-06-14 18:02:39 +08:00
houseme efa89a98ed refactor(logging): standardize protocol and observability events (#3419)
* refactor(logging): standardize object capacity events

* refactor(logging): standardize protocol server events

* refactor(logging): standardize swift protocol events

* refactor(logging): standardize observability events

* refactor(logging): move masking helper and extend guardrails
2026-06-14 07:14:45 +08:00
安正超 7146c893cb refactor(storage-api): remove legacy bucket DTO re-export (#3364) 2026-06-11 21:24:56 +08:00
安正超 97dd825468 fix(ecstore): optimize ObjectInfo clone and fix critical TODOs (#3149)
* fix(ecstore): optimize ObjectInfo clone and fix critical TODOs

## ObjectInfo hot-path clone optimization (issue #653 item 2)
- Wrap user_defined (HashMap), user_tags (String), parts (Vec) in Arc
- Clone cost reduced from ~20 heap allocations to O(1) ref count bump
- Updated 11 downstream access sites with explicit deref where needed

## Critical TODO/FIXME fixes (issue #653 item 7)
- rpc/peer_s3_client.rs: descriptive error message for empty peer response
- set_disk/write.rs: concurrent rollback deletes via tokio::spawn + join_all
- store_list_objects.rs: resolved FIXME with explanation + 4 regression tests
- store/bucket.rs: namespace write locks for make_bucket and delete_bucket

Skipped TODOs (too risky without broader context):
- multipart.rs:30 nslock — causes lock timeout in existing tests
- bucket.rs:88 cached list_bucket — needs cache invalidation strategy
- bucket.rs:149 replication delete — needs replication subsystem integration

1134 tests pass (1 flaky under parallel execution, passes individually).

* fix: address PR #3149 review comments

- Preserve NamespaceLockQuorumUnavailable variant in bucket create/delete
- Fix cancellation test to actually test cancellation (keep senders open)
- Update ObjectInfo consumers in app/ for Arc-backed fields
- Update bucket_usecase.rs user_tags to Arc<String>

* fix: update ObjectInfo Arc consumers

* fix: address ecstore merge review comments

* fix: cancel blocked merge output sends

* fix(ecstore): satisfy ObjectInfo clone clippy
2026-06-01 00:44:58 +00:00
安正超 68d3dba9fc fix: revert standalone #2351 artifacts (phase 5) (#2535) 2026-04-14 22:35:05 +08:00
安正超 2a3d127a86 revert: restore PutObjReader and remove PUT zero-copy chunk path (#2351 phase 2) (#2532) 2026-04-14 16:35:03 +08:00
houseme 32bf8f5bf3 feat(storage): add direct chunk GET fast path (#2351)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-07 08:33:46 +08:00
weisd 5d302febb7 fix(rio): preserve reader capabilities and crypto safety (#2363) 2026-04-03 13:57:42 +08:00
Senol Colak b07383760f Add OpenStack Swift API Support (#2066)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <noreply@github.com>
2026-03-08 01:11:35 +08:00