gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-19 11:06:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(heal): incremental status cursors and typed overlap policy (HS-06) (#6206)

Browse Source 
* feat(heal): incremental heal status cursors and typed overlap policy (HS-06)

Incremental results: every retained result item now carries a monotonic
sequence number. The status query accepts a client cursor (sinceSeq on
the admin wire, Option<u64> internally) and returns only newer items,
plus nextSeq (the next cursor) and minSeq (the oldest retained
sequence). A cursor that fell behind the 1024-item retention window is
flagged through the existing truncated signal together with minSeq so
the client can restart from it. Sequencing survives task completion:
the completion archive stores the seq-stamped window. None keeps the
exact legacy full-snapshot behavior, so existing clients see no change.

Typed overlap handling for admin starts: RUSTFS_HEAL_OVERLAP_POLICY
(merge default | minio_error). Under minio_error, an admin start whose
path overlaps an active or queued task rejects with typed
already-running / overlapping-paths admission reasons (surfaced through
reason_label in the admin error body, sharing the existing
OperationAborted site because the s3s footprint ratchet forbids new
s3_error! sites); an exact duplicate start rejects with
already-running instead of silently merging. Scanner/autoheal/
read-repair sources never take the rejection path.

forceStart semantics now match MinIO for admin requests: an admin
forceStart first cancels the overlapping active admin task, then
admits the replacement.

Wire: the heal-control Query command grows an optional sinceSeq
(defaulted and skipped when absent, so older peers stay compatible);
the admin handler accepts the sinceSeq query parameter; the local
channel query gains the same cursor.

Tests: seq monotonicity and incremental slicing, window slide moving
minSeq with lagging-cursor flags, overlap matrix (same/containing/
contained/disjoint x policy x source), forceStart cancel-then-admit,
and the completion-archive window handoff.

Co-Authored-By: heihutu <heihutu@gmail.com>

* style: fmt after main merge

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
This commit is contained in:
houseme
2026-08-18 16:09:30 +08:00
committed by GitHub
parent a4ea36b298
commit a5800033bd
11 changed files with 807 additions and 96 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rustfs/src/admin/handlers/heal.rs
+23 -1
View File
@@ -66,6 +66,9 @@ struct HealInitParams {
client_token: String,
force_start: bool,
force_stop: bool,
/// Incremental result cursor (HS-06): only result items with a sequence
/// greater than this are returned; absent means full snapshot.
since_seq: Option<u64>,
}
fn extract_heal_init_params(body: &Bytes, uri: &Uri, params: Params<'_, '_>) -> S3Result<HealInitParams> {
@@ -98,6 +101,16 @@ fn extract_heal_init_params(body: &Bytes, uri: &Uri, params: Params<'_, '_>) ->
}
hip.force_stop = parse_heal_query_bool(value.as_ref())?;
}
"sinceSeq" => {
if !seen.insert("sinceSeq") {
return Err(s3_error!(InvalidArgument, "duplicate heal query parameter"));
}
hip.since_seq = Some(
value
.parse::<u64>()
.map_err(|_| s3_error!(InvalidArgument, "sinceSeq must be a non-negative integer"))?,
);
}
_ => return Err(s3_error!(InvalidArgument, "unknown heal query parameter")),
}
}
@@ -978,7 +991,15 @@ fn reject_heal_admission(result: rustfs_common::heal_channel::HealAdmissionResul
result.result_label(),
result.reason_label()
),
HealAdmissionResult::Dropped(HealAdmissionDropReason::PolicyDropped) => s3_error!(
// Overlap rejections (HS-06) share this arm: the s3s footprint
// ratchet forbids new s3_error! sites, and the typed reason is
// preserved through reason_label() ("already_running" /
// "overlapping_paths") so madmin-style clients can distinguish.
HealAdmissionResult::Dropped(
HealAdmissionDropReason::PolicyDropped
| HealAdmissionDropReason::AlreadyRunning
| HealAdmissionDropReason::OverlappingPaths,
) => s3_error!(
OperationAborted,
"heal request not admitted: admission={}, reason={}",
result.result_label(),
@@ -1403,6 +1424,7 @@ impl Operation for HealHandler {
new_heal_control_metadata(&route)?,
heal_path,
client_token.clone(),
hip.since_seq,
)
.map_err(|err| s3_error!(InternalError, "encode heal control query failed: {err}"))?;
let response = submit_cluster_heal_channel_command(context, route, envelope, &request_id, client_token).await?;
rustfs/src/storage/rpc/node_service.rs
+8 -2
View File
@@ -571,8 +571,12 @@ async fn execute_heal_control_envelope_with_manager(
admission: receipt.result.into(),
}
}
rustfs_protos::heal_control::ExecutableCommand::Query { heal_path, client_token } => {
let response = timeout(remaining, processor.execute_query_request(heal_path, client_token))
rustfs_protos::heal_control::ExecutableCommand::Query {
heal_path,
client_token,
since_seq,
} => {
let response = timeout(remaining, processor.execute_query_request_since(heal_path, client_token, since_seq))
.await
.map_err(|_| Status::deadline_exceeded("heal control query expired before execution"))?
.map_err(|_| Status::internal("heal control query failed"))?;
@@ -2517,6 +2521,7 @@ mod tests {
metadata(),
"bucket/prefix".to_string(),
canonical_token.clone(),
None,
)
.unwrap();
let query_result = execute_heal_control_envelope_with_manager(query, coordinator_epoch, Some(Arc::clone(&manager)))
@@ -2555,6 +2560,7 @@ mod tests {
metadata(),
"bucket/prefix".to_string(),
canonical_token,
None,
)
.unwrap();
let stopped_result = execute_heal_control_envelope_with_manager(stopped_query, coordinator_epoch, Some(manager))