Commit Graph

10 Commits

Author SHA1 Message Date
Zhengchao An d22cb5d07a fix: resolve release-blocking integration failures (#6320)
* fix: resolve release-blocking integration failures

* fix: satisfy stable clippy lints

* fix: satisfy Rust 1.98 CI lints
2026-08-21 06:39:29 +08:00
Zhengchao An d6c62b9601 chore(ecstore): drop the services dead_code blanket (#6103)
Removing the blanket exposes twenty-five items across tier, notification and rebalance. Only eight are deleted — the lowest ratio of this burn-down so far, and the reason is that these subsystems carry heavy test coverage, so the blanket was mostly hiding test-only seams rather than dead weight.

Deleted:

- crates/ecstore/src/services/tier/warm_backend_s3sdk.rs entirely (200 lines). Its WarmBackendS3 is never constructed; the type of the same name in warm_backend_s3.rs is the live one, wrapped by the Azure backend. Two implementations of one S3 warm backend, one of them never wired.
- TierConfigMgr::begin_publish_transition and publish_candidate_inner, thin wrappers whose _with_allowed_mutation_blocks siblings carry every real caller, plus retire_driver.
- The GCS backend's MAX_MULTIPART_PUT_OBJECT_SIZE, MAX_PARTS_COUNT and MIN_PART_SIZE, and its write-only storage_class field.
- mark_started_rebalance_pools_stopped and the RStats alias.

Two deletions were withdrawn after a per-name grep, both because of an inference rather than a check:

AsyncBatchProcessor::new was deleted on the strength of grepping only BATCH_PROCESSOR_OPERATION_CUSTOM, whose two hits are its definition and its use inside new. That looked like a self-contained dead pair; new in fact has seven test callers. The warning listed both items, and only one of them was actually checked.

Deleting the two dead publish wrappers then revealed a second layer — publish_candidate_owned, remove_and_save_with, clear_and_save_with, save_tiering_config_if_current. These are not dead: publish_candidate, their caller, is #[cfg(test)], so a callee that lives in the main body has no caller in the lib build and a live one in the test build. rustc reports the roots of a dead subgraph, and the next layer down can have a different character, so each layer needs its own grep.

Kept with allows: the tier mutation-intent record helpers (asserted by store::init tests), affected_targets, tier_object_blocks_target_rebind, the rebalance snapshot and retry-wait helpers, notification_sys's tier_config_reload_worker_active and call_peer_with_timeout, and active_operation_lease_count, whose only caller sits behind #[cfg(feature = "test-util")].

Also kept, with a module note rather than removal: the ecstore-side EventNotifier. All four of its methods are unreachable and init_bucket_targets logs that it is a no-op in this build; the working stack is rustfs-notify, whose own EventNotifier drives bucket configuration. Removing it means also retiring the InstanceContext slot that holds it (backlog#939 Phase 5), which belongs in its own PR.

Worth a separate issue: MAX_MULTIPART_PUT_OBJECT_SIZE, MAX_PARTS_COUNT and MIN_PART_SIZE are declared independently in eight warm-backend files plus client/constants.rs. Only the GCS copies were dead; the other seven backends each use their own.

Verification, four lanes warning-free: default, --tests, --features rio-v2 --tests, --features test-util --tests. cargo nextest run -p rustfs-ecstore 4041 passed; clippy --lib --tests -D warnings clean; make pre-commit exit 0.

Ref rustfs/backlog#1823 (step 2).
2026-08-14 21:56:11 +08:00
houseme 7ab0955f8b test(e2e): stabilize tier and storage class checks (#5228)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-25 14:21:52 +00:00
houseme 8e83087ba4 fix(tier): handle mutation intent CAS races (#5227)
* test(scripts): expand internode grpc ab env coverage

* fix(tier): handle mutation intent CAS races

Make tier mutation intent advancement retry idempotently when an If-Match CAS loses to a matching committed peer update.

Expand four-node inline fallback E2E coverage for mixed msgpack controls, transition readiness evidence, and per-node environment overrides.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-25 11:10:13 +00:00
houseme 4607c3be53 fix(tier): recover coordinator mutation intents (#5114)
Persist coordinator tier mutation intents before config CAS, recover prepared coordinator intents whose candidate digest already matches the saved config, and clean finished coordinator records after local publish.

Refs rustfs/backlog#1357

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-22 07:35:29 +00:00
houseme 3f60cc743e fix(tier): replay committed mutation intents (#5107)
Keep recovered tier mutation blocks installed until the local publish transition has atomically established draining for the affected tiers, so old-generation leases cannot slip in after peer commit replay and before local publish.

Delay committed intent cleanup until local publish succeeds. If peer replay succeeds but local publish fails, the durable committed intent remains available for retry and the runtime block stays fail-closed.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-22 03:06:55 +00:00
houseme 937b311316 fix(tier): lock tier config mutations (#5080)
* fix(tier): lock tier config mutations

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(tier): add mutation RPC auth contract (#5082)

Co-authored-by: heihutu <heihutu@gmail.com>

* fix(tier): add peer mutation handler core (#5084)

Co-authored-by: heihutu <heihutu@gmail.com>

* fix(tier): add mutation control rpc service (#5087)

Co-authored-by: heihutu <heihutu@gmail.com>

* fix(tier): recover prepared mutation drains (#5093)

Recover prepared tier mutation intent records into the local tier runtime so a restarted peer fails closed before issuing new remote-tier operation leases or conflicting admin publishes.

Reconcile the recovered block map on each scan so committed, aborted, or removed intents clear stale local blocks instead of wedging the peer until process restart.

Co-authored-by: heihutu <heihutu@gmail.com>

* fix(tier): prove zero references before tier removal (#5092)

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>

* fix(tier): clear peer mutation runtime blocks (#5094)

Install prepared mutation runtime blocks when peer prepare requests are applied or replayed so followers fail closed immediately before restart recovery.

Clear the in-memory block once peer commit or abort reaches a durable terminal state, including delayed duplicate prepare requests that observe a committed or aborted record.

Co-authored-by: heihutu <heihutu@gmail.com>

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-21 23:16:12 +08:00
houseme 7f569b67cb fix(tier): add mutation intent CAS advance (#5078)
Add idempotent terminal transition handling for tier mutation intents and a compare-and-swap record update helper backed by config-object ETags. Cover commit and abort retries, conflicting terminal updates, stale ETag rejection, and exact-limit scan pagination.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-21 08:10:31 +00:00
houseme 9f25858b05 fix(tier): add mutation intent record store (#5075)
Add canonical record-object naming and crate-private save, load, and idempotent delete helpers for tier mutation intents. Cover malformed record keys, mismatched persisted mutation IDs, and an ECStore config-object round trip under test-util.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-21 07:26:23 +00:00
houseme a4e7dd70a6 fix(tier): add mutation intent foundation (#5074)
Add an internal durable tier mutation intent model for the #1357 distributed fencing work. The new model validates schema, checksum, canonical targets, mutation-specific target identity shape, config ETags, expiry, and prepared-to-terminal state transitions without changing the production mutation path yet.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-21 06:24:57 +00:00