known_addrs in PeerInfoInternal is append-only — addresses accumulate via add_addr() and PeerList gossip but are never removed. In dynamic environments (k8s pod restarts, DHCP, NAT traversal), this list grows unboundedly with stale addresses. Combined with sequential iteration in try_connect() and no TCP connect timeout in netapp.rs, each unreachable address blocks reconnection for the kernel's TCP SYN timeout (75-130s on Linux). With 10+ stale addresses, worst-case reconnection exceeds 750s — a full outage for replication_factor=3 clusters. This commit contains the two following changes: 1. Address failure tracking and pruning (peering.rs): Track consecutive connection failures per address in PeerInfoInternal. After 3 failures, prune from known_addrs. Reset count when address is re-advertised via gossip or incoming connection. Prevents unbounded list growth. 2. Shuffle before connecting (peering.rs): Randomize address order in try_connect() so the valid address (often appended last) gets a fair chance instead of always trying stale addresses first.
Garage 
[ Website and documentation | Binary releases | Git repository | Matrix channel ]
Garage is an S3-compatible distributed object storage service designed for self-hosting at a small-to-medium scale.
Garage is designed for storage clusters composed of nodes running at different physical locations, in order to easily provide a storage service that replicates data at these different locations and stays available even when some servers are unreachable. Garage also focuses on being lightweight, easy to operate, and highly resilient to machine failures.
Garage is built by Deuxfleurs, an experimental small-scale self hosted service provider, which has been using it in production since its first release in 2020.
Learn more on our dedicated documentation pages:
Garage is entirely free software released under the terms of the AGPLv3.