mirror of
https://github.com/openziti/ziti.git
synced 2026-09-10 08:45:41 +00:00
Paul Lorenz
53a231fa60
Connect setup runs off the accept path, so a router is already connected by the time it is reached and a setup that has to wait waits rather than being refused. Refusing would discard a completed handshake and have it redone, turning a reconnect storm into a larger one. But the wait had no bound and no visibility: waiters could pile up indefinitely, and the pool metrics that would have shown it went away when the pool became a semaphore. - gives up after routerConnectSetupTimeout, two minutes by default, and closes the connection so the router redials. Skipping the setup instead would leave a router that is connected, believes it is, and has no links built, which nothing reports. Two minutes is a stuck detector rather than load shedding: setup is sub-second and runs 200 at a time, so reaching it needs tens of thousands of connects queued, and waiters behind a wedge were never going to drain - reports under pool.router.connect.*, matching the convention the gossip pools use so a dashboard built for one transfers. queue_size for connects waiting, wait_timer for how long they waited, work_timer for the setup itself, timeouts for those that gave up - splits the wait from the work because their remedies differ: a high wait wants more concurrency, a high work time wants a look at what the setup does. A combined timer reports that something is slow and hides which half - keeps the gauge alongside the timers, which only record on completion. A wedged setup makes them go quiet exactly when something is wrong; the queue depth is the only one of these that still moves The changelog described a routerConnectPool that rejected connects when full. That pool was never implemented under that name, and the semaphore that replaced it waits instead, so the config key, the sample yaml and the monitoring guidance are corrected to match. The metrics line becomes accurate rather than removed, since the names above are the ones it already documented.