mirror of
https://github.com/rustfs/rustfs.git
synced 2026-10-31 00:06:13 +00:00
codex/plugin-runner-integration
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Hauser
|
7130f25af5 |
fix(ecstore): unblock delete replay and parallelize preparation (#8896)
* perf(ecstore): avoid head-of-line blocking in delete recovery Recover independent keys through a bounded rolling window while preserving per-key hint generations, admission budgets, cancellation retry, and the existing transaction decision engine. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(ecstore): bound durable delete preparation and bootstrap recovery Prepare independent Linux delete transactions with bounded concurrency while retaining per-key result order and serial fallback for overlapping, rollback, and legacy requests. Preserve startup discovery until bucket metadata is initialized, and add release performance and process-crash acceptance runners. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): verify clean exits in recovery acceptance Reject forced or OOM exits when measuring clean startup and verify original ETags across mixed-layout crash recovery and a second restart. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): distinguish prepared and decided batch crashes Observe real preparation and receipt states separately, retain non-vacuous acknowledged-delete sentinels, and require read-only physical convergence before S3 reads can trigger cleanup. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): move final delete recovery options Move the options value into the batch-ordering regression's final case instead of cloning it. Keep the earlier clone needed for reuse. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): replay delete receipts after interrupted proof GC Retire only the exact empty transaction leaf before journal activation. Preserve occupied directories and prefix children, and cover partial cross-disk proof GC without a fresh delete. Tighten crash evidence to distinguish directory-only remnants from retained content. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
911ea265c5 |
fix(storage): apply and index explicit delete-marker removals (#8854)
* fix(ecstore): purge every explicit delete-marker version Keep explicit marker removals on the canonical version-delete path so same-key batches are not truncated and pending replication does not retain source markers. Preserve marker identity and timestamps in the response. Add real four-disk regressions for mixed version cleanup, duplicate marker IDs, pending replication, and null/UUID marker removal. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(filemeta): index explicit delete-marker batches Reuse the batch index and final compaction for ordinary marker removals. Keep small marker-only histories on the scalar path and preserve fallback for free versions and complex mutations. Add scalar-equivalence, boundary, corruption and deterministic scan-count coverage, a real-disk mixed-history regression, and reproducible marker benchmark cases. Preserve existing benchmark defaults while honoring CLI sampling overrides. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(filemeta): remove redundant marker fixture clone Move the original metadata into its final test fixture after the last borrow, avoiding the redundant clone reported by strict Clippy checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): validate explicit marker deletion and recovery Exercise UUID/null and duplicate explicit marker removals over real HTTP with retries disabled, including immediate four-node visibility and exact retained payloads. Cover all-drive PREPARED/APPLIED/DECISION process termination, rollback/commit recovery, and old-ID retries across a second restart without deleting a fresh marker. Use flat sampling and fixed per-case fixture batches in the metadata benchmark. Update profile selection digests for the added regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(heal): synchronize crash fixture startup and flush readiness Give disk setup and the durable flush boundary separate readiness gates, keeping each phase bounded to ten seconds. Preserve the real SIGKILL and restart assertions while reporting early child exits and retaining output. Reap fixture children on assertion failure and cover early-exit diagnostics. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
8c8c74cbc3 |
fix(ecstore): unify bounded delete recovery across upgrades (#8594)
* fix(ecstore): complete delete recovery across upgrades and restarts Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): expect typed quorum error after delete rollback Assert the exact InsufficientWriteQuorum bucket and object instead of the removed Io wrapper, retaining the rollback and cleanup ownership checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
4166b9503f |
fix(ecstore): recover interrupted delete transactions (#8468)
* fix(ecstore): recover crash-interrupted delete retries Retain set-quorum cleanup proof and restore metadata before dropping rollback markers. Reclaim eligible exact-key residue during explicit delete retries, with bucket identity, version, retention and namespace fencing guards. Preserve prefix children and snapshot ownership through batched physical cleanup, and cover interrupted transactions and failure boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): route recovery test imports through contracts Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): preserve missing-key deletes at quorum * feat(ecstore): add certified set-wide delete recovery Atomically prepare deletion journals and exact targets, publish durable set receipts, and recover through one fenced disk/RPC core used by startup and scoped administrator healing. Preserve returning generations, ordinary missing-key quorum semantics, and legacy rollback wire compatibility. Retain namespace and volume ownership through late physical operations, order related transactions causally, and keep proof until full convergence. Remove the superseded local-only recovery path and add storage, protocol, RPC, durability and isolated Docker regression coverage. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): reclaim completed multipart delete sidecars * fix(ecstore): recover multipart sidecars with verified proofs Recognize native multipart sidecars at the shared recovery boundary and restore them without treating metadata or proof files as shard payloads. Validate required proof generations and bind every present live/staged shard copy to its metadata-bound Merkle proof before restoring metadata. Cover protected native MPU rollback, split source layouts, missing or corrupt proofs, and payload corruption with recomputed bitrot checksums. Add real multipart Docker regressions in both integrity modes. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): settle superseded committed version deletes Allow a validated committed decision to finish its own data cleanup after a later decision removes current metadata. Prepared copies need an inspected same-transaction receipt matching immutable local intent before promotion, and retain their local journal without importing peer hashes. Reuse the authority matcher for dry-run and settlement. Cover chained explicit-version deletions, partial receipt publication, mismatch rejection and no-resurrection, while retaining the no-decision absence protection. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): clarify delete recovery case label Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): restore recovery CI guard coverage Keep production error sites visible to the ratchet by placing test modules last. Preserve native encoding errors, group preparation identity and ownership, and reduce RPC enum size without changing the wire payload. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(admin): preserve owned OIDC validation errors Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): unify recovery authority and snapshot work Use one user-bucket legacy settlement boundary and one adoption source predicate for preview and execution. Reuse immutable snapshot digests, consume owned generation metadata, index version membership, and prepare empty witnesses through the canonical journal path. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): preserve delete recovery invariants --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |