mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-06 03:59:14 +00:00
fix(storage): harden ODM and scanner publication (#7187)
* fix(storage): harden ODM and scanner publication * fix(app): simplify absent SSE configuration matching * test(heal): settle PUT rename tails before disk-wipe fixtures * fix(ecstore): remove duplicate local rename implementation Keep the canonical commit module after concurrent storage changes merged. The control-write and rollback changes are already present there. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ci): satisfy new clippy lints * style(scanner): order merged test imports * fix(scanner): invalidate bucket work after namespace completion * fix(scanner): fence cached snapshots by scan execution --------- Co-authored-by: houseme <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com>
This commit is contained in:
@@ -23,6 +23,30 @@ therefore has three identities:
|
||||
If any identity changes before commit, the result is a candidate for retry or
|
||||
observation, not an authoritative baseline.
|
||||
|
||||
Ordinary PUT rename fanouts also track instance-scoped in-flight work. A quorum
|
||||
ACK does not release it: the actual disk tasks retain ownership until their
|
||||
rename work ends, including when the request caller is cancelled. Scan admission
|
||||
remains movement-only so sustained PUTs do not stop namespace walks and
|
||||
scanner-driven lifecycle discovery. The post-walk local publication check and
|
||||
remote publication leases reject pending fanouts. Begin/end namespace generations
|
||||
invalidate scans and cached plans across the fanout; after acquiring remote
|
||||
leases, the coordinator rechecks the full activity digest before publishing an
|
||||
authoritative aggregate. This catches a tail that finishes between the scan's
|
||||
last probe and lease acquisition.
|
||||
|
||||
This adds no namespace or movement lock. An already-verified older snapshot may
|
||||
still precede a newly started write. Sustained or stalled PUT tails can delay
|
||||
authoritative usage publication, which resumes through the existing retry
|
||||
schedule rather than a new immediate-wakeup protocol. Intermediate per-set and
|
||||
prefix cache readers retain their existing approximate-cache semantics. A
|
||||
prolonged pending tail with no generation changes can also delay cycle advancement
|
||||
and fresh rescans of already-current caches; this is not a guarantee of lifecycle
|
||||
progress under indefinitely stalled storage I/O.
|
||||
|
||||
This PUT-tail protection requires every writer node to be upgraded. It does not
|
||||
prove that a failed tail replica has healed, and it does not extend the same
|
||||
in-flight tracking to multipart or other namespace mutation paths.
|
||||
|
||||
## Fences
|
||||
|
||||
The protocol uses separate fences because they exclude different stale inputs.
|
||||
@@ -33,7 +57,7 @@ They must not be collapsed unless the replacement proves the same exclusions.
|
||||
| Scanner leadership claim | scanner | competing scanner leaders and stale cycle writers |
|
||||
| Storage publication epoch | ECStore | usage computed across rebalance, decommission, or other data-movement generations |
|
||||
| Publication lease | scanner peers through ECStore-facing activity probes | remote dirty-usage or maintenance state that has not acknowledged the candidate |
|
||||
| CAS revision | backing config object store | lost updates to `.usage.v2.json`, `.usage.json`, or cycle-state objects |
|
||||
| CAS revision | backing config object store | lost updates to usage snapshots, scanner caches, or cycle-state objects |
|
||||
| Per-set freshness | scanner aggregation | a merged usage snapshot that combines stale and current set results |
|
||||
| Tier registry generation | scanner tier accounting | bytes classified against a different warm-tier registry |
|
||||
| Usage floor identity | scanner publication and ECStore quota fallback | empty or legacy values becoming plausible authoritative quota input |
|
||||
@@ -42,6 +66,28 @@ A reader that cannot prove the required fence for its surface must fail closed
|
||||
or use the documented observed path below. It must not synthesize an empty usage
|
||||
snapshot for a missing or corrupt authoritative object.
|
||||
|
||||
## Cache Execution Identity
|
||||
|
||||
The structural scan-plan digest can remain stable across ordinary bucket writes
|
||||
so a scoped scan can retain unaffected baseline buckets. It is not sufficient
|
||||
proof for reusing a completed result within the same cycle. Bucket work uses an
|
||||
execution digest combining the structural plan and the full activity snapshot,
|
||||
with the bucket's dirty generation included in its cache identity. Completed set
|
||||
caches carry the same execution digest separately from their structural plan.
|
||||
The persisted set-root fast path requires equal execution identities as well as
|
||||
the existing source, cycle, leader, tier, and cache-structure checks.
|
||||
|
||||
A set scan also captures its starting cache revisions. When the persisted
|
||||
execution differs, replacement requires those revisions to remain unchanged;
|
||||
otherwise a slow scan could overwrite a newer completed result. The existing
|
||||
cache lock, conditional save, and movement admission still fence the commit.
|
||||
|
||||
The optional `scan_execution_digest` field is appended to the map-encoded cache
|
||||
metadata. Legacy caches remain readable but cannot satisfy same-cycle set-root
|
||||
reuse without this identity. Older readers can ignore the added map key, but
|
||||
older writers do not enforce its fence; readability is not a mixed-version
|
||||
publication-safety guarantee.
|
||||
|
||||
## Persisted Objects
|
||||
|
||||
The persisted objects are part of the compatibility contract. Removing one
|
||||
|
||||
Reference in New Issue
Block a user