mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-30 16:59:52 +00:00
fix(admin): make server_info retry re-dial and match peer disks by resolved host (#4618)
Two follow-up fixes to #4607 (both verified real bugs, each with a regression test): 1. The in-call server_info retry did not re-dial on network errors. A network-like first failure runs through `finalize_result`, which sets the client's `offline` gate; the retry then called `evict_connection` and re-invoked `server_info`, but `get_client` short-circuits on that gate and returns "temporarily offline" without dialing. Only the async recovery monitor would clear it. So the retry re-dialed only in the timeout branch and was a no-op in the transport/half-open branch it was meant to cover. Add `PeerRestClient::prepare_retry` (evict + clear the offline gate) and use it before the retry. 2. Synthesized/degraded drive lists went empty on hostname deployments. `PeerRestClient::host` is an `XHost` that `hosts_sorted` builds via `XHost::try_from` -> `to_socket_addrs`, so it is the resolved `IP:port`; but `synthesized_disks`/`peer_disk_health` compared it against `Endpoint::host_port()`, which is the raw `hostname:port`. On hostname clusters the compare missed, the drive list came back empty, and `unknownDisks` stayed 0 — reproducing the "drives vanish from the summary" regression #4607 fixed (also affected the pre-existing offline path). Compare through a shared `endpoint_host_matches` helper that canonicalizes the endpoint side through the same `XHost` resolution. Tests: `peer_rest_client_prepare_retry_clears_offline_gate`, `endpoint_host_matches_direct_and_canonicalized` (uses localhost, no external DNS). Follow-up to rustfs/rustfs#4607; tracked in rustfs/backlog#1049. Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
@@ -147,6 +147,20 @@ impl PeerRestClient {
|
||||
evict_failed_connection(&self.grid_host).await;
|
||||
}
|
||||
|
||||
/// Prepare this client for an immediate fresh-connection retry.
|
||||
///
|
||||
/// On a network-like failure `finalize_result` both evicts the channel and
|
||||
/// sets the offline gate, after which `get_client` fast-fails with
|
||||
/// "temporarily offline" and only the async background recovery monitor
|
||||
/// would clear the gate (not within this call). So a plain `evict_connection`
|
||||
/// is not enough to make an in-call retry actually re-dial: the gate still
|
||||
/// short-circuits it. This drops the cached channel AND clears the gate so
|
||||
/// the very next `get_client` re-dials. See rustfs/backlog#1049 (P1-B).
|
||||
pub async fn prepare_retry(&self) {
|
||||
self.evict_connection().await;
|
||||
self.offline.store(false, Ordering::Release);
|
||||
}
|
||||
|
||||
fn is_network_like_error(err: &Error) -> bool {
|
||||
let message = err.to_string().to_ascii_lowercase();
|
||||
[
|
||||
@@ -1163,6 +1177,22 @@ mod tests {
|
||||
assert!(err.to_string().contains("temporarily offline"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn peer_rest_client_prepare_retry_clears_offline_gate() {
|
||||
// finalize_result sets the offline gate on a network error; without
|
||||
// clearing it, an in-call retry would fast-fail on the gate instead of
|
||||
// re-dialing (rustfs/backlog#1049 P1-B). prepare_retry must clear it.
|
||||
let client = test_peer_client();
|
||||
client.offline.store(true, Ordering::Release);
|
||||
|
||||
client.prepare_retry().await;
|
||||
|
||||
assert!(
|
||||
!client.offline.load(Ordering::Acquire),
|
||||
"prepare_retry must clear the offline gate so the next get_client re-dials"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn peer_rest_client_finalize_result_marks_offline_for_network_errors() {
|
||||
let client = test_peer_client();
|
||||
|
||||
Reference in New Issue
Block a user