Commit Graph

53 Commits

Author SHA1 Message Date
Paul Lorenz 444999bc5b Break the SDK terminator removal retry storm. Fixes #4156
- adds RemoveTerminatorsV2, an asynchronous control-channel removal, so a router
  no longer holds a request slot open waiting for a synchronous reply; gated on a
  new capability so routers fall back to the synchronous path against controllers
  that do not advertise it
- paces establish and remove retries with a per-terminator exponential backoff
  after a controller rejection, so a rejected operation is not re-sent on the next
  event-loop tick, which is what let a rate-limited removal sustain itself
- routes v2 removes to the leader and skips the ordered delete for a terminator the
  router confirms it created that no longer exists, so a storm of no-op retries
  cannot consume the command rate limiter
- gates that skip on the leader having applied everything committed when it took
  leadership, since a leader reports itself elected before those applies finish and
  would otherwise read a committed create as absent and drop its delete
- guards the leader term and every write to the caught-up flag with a dedicated
  lock, so a barrier completing alongside a leadership change cannot record a result
  for a term that has already ended
- scopes v2 removals to the requesting router, so a router can only remove
  terminators it owns on the asynchronous path as well as the synchronous one
- tracks in-flight removal batches on the router, resolving or requeueing them when
  the response arrives; lost responses self-heal via rate-limiter expiry and the
  retry scan
- waits for an outstanding operation before sending a terminator delete, and checks
  that before dequeuing, so a delete neither races a create the controller has
  committed but not applied nor falls out of the queue until the retry scan finds it
  minutes later
- matches a create response to the attempt that produced it, by control channel and
  reply sequence, since attempts overlap once one overruns the establishment timeout
  and every attempt carries the same terminator id; only the outstanding attempt's
  response may confirm the create or report the terminator idle
- records the attempt as the sender assigns its sequence, so a response cannot be
  processed before the request it answers is known, and withdraws an attempt whose
  send never queued
- reports a removal that succeeds past the establishment timeout as congestion
  rather than success, matching how the establish path classifies its own latency,
  since both resolve into the same limiter
- extracts the transient-error test both remove handlers share, so the v2 path
  treats a leaderless cluster as busy like v1 does instead of as a permanent
  failure, and the two cannot drift again
- applies the same delete sequencing and response correlation to the router-hosted
  (ER/T) registry, which cleared the same in-flight marker unguarded and dropped a
  deferred delete from its queue rather than leaving it there
- bounds the v2 response send with a timeout, so a router that stops draining its
  send queue cannot accumulate blocked responder goroutines on the controller
2026-09-24 13:24:51 -04:00
Paul Lorenz 48e4d4a224 Manage router link configuration via the controller. Fixes #4004
- adds a link ConfigHandler (router/link FactoryRegistry) that applies router.link.v1 config: Apply rebuilds the listener/dialer set wholesale, and established Xlinks survive because Listener.Close() only closes the accept loop
- translates local link: YAML into router.link.v1 JSON and pushes it through the managed-config registry at startup
- adds the UpdateLinkListeners ctrl message so the router republishes its listener set to the controller on change; the controller re-fans via the existing PeerStateChange path
- re-evaluates dialers on link group and listener changes via RescanForDialOpportunities

For #3743.
2026-08-13 15:37:24 -04:00
Paul Lorenz ae806045b5 Convert self-describing receive handlers to channel/v5. For #3983
- channel.TypedReceiveHandler -> channel.ContentTypeReceiver
- binding.AddTypedReceiveHandler(h) -> channel.AddReceiveHandlers(binding, h)

channel/v5 repurposes TypedReceiveHandler for the senders-typed handler and replaces the
self-describing pattern with ContentTypeReceiver plus the AddReceiveHandlers free function
(openziti/channel#262). Mechanical conversion; does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz 1c122af490 Rewrite channel/v4 imports to channel/v5. For #3983
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5

This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz b5eaeefdb3 Prep for channel v5: bind handler invocation, send priorities. Fixes #3942
- invokes bind handlers via h.BindChannel(binding) instead of binding.Bind(h), which channel v5 removes from the Binding interface
- removes WithPriority from edge dial and state message sends; priority was already a no-op on grouped channels and channel v5 removes the priority API
2026-06-05 15:24:45 -04:00
Paul Lorenz 588cb2350c Fix data race on heartbeat lastResponse and peer logger label
- changes lastResponse from plain int64 to atomic.Int64 in both router
  and peer heartbeat callbacks, fixing a data race between the heartbeat
  response handler and the heartbeat check ticker
- fixes peer heartbeat logger channelType from "router" to "peer"
2026-04-03 17:42:18 -04:00
Paul Lorenz 2ffc6e1151 Update ziti version to 2.0 2026-01-28 12:34:10 -05:00
Paul Lorenz 08359a9e2b Remove legacy link management code from the controller. Fixes #3512 2026-01-22 15:26:42 -05:00
Paul Lorenz c18a594b56 Clustering coordination fixes
* Allow routers to request current cluster membership information. Fixes #3503
* Get cluster membership information from raft directly, rather than trying to cache it in the DB. Fixes #3501
* Set a router data model timeline when initializing a new HA setup, rather than letting it stay blank. Fixes #3500
* Reduce router data model full state updates. Fixes #3504
2026-01-17 02:03:31 -05:00
Paul Lorenz 653063767e Add alert events. Fixes #3264 2025-10-15 14:21:07 -04:00
Paul Lorenz e5fb69935e Support multi-underlay links. Fixes #3134 2025-07-11 20:56:11 -04:00
Paul Lorenz cc7eefaf24 Add support for tracking network interfaces on routers and identities. Fixes #3082. Add network discover to router. Fixes #3083 2025-06-17 09:32:51 -04:00
Paul Lorenz e647d67325 Update to channel/v4 2025-04-02 15:28:59 -04:00
Paul Lorenz cccf0c06af Update to channel/v3. Fixes #2390 2024-09-09 12:23:25 -04:00
Paul Lorenz c3b43133d1 Merge fabric and controller model code. Fixes #2205 2024-07-09 16:11:01 -04:00
Paul Lorenz fab81e4cd4 Add terminator chaos testing and fix issues found. Fixes #1794 Fixes #1369 2024-03-12 10:31:36 -04:00
Paul Lorenz 9ddd1b941d Add service max idle time. Fixes #1496. Add decomission router support. Fixes #1402 2023-12-08 12:48:56 -05:00
Paul Lorenz f3d67b7f49 Update fabric imports 2023-09-28 23:34:28 -04:00
Paul Lorenz 700ec34177 Make link heartbeats configurable. Add unresponsive close timeout. Fixes openziti/fabric#796 2023-09-18 22:40:19 -04:00
Paul Lorenz a631f48643 Do some forwarder option cleaning. Remove unused latency options 2023-08-22 20:37:59 -04:00
Paul Lorenz a7fe59e184 Do some configuration tidying 2023-08-22 20:37:25 -04:00
Paul Lorenz 509364d908 Remove trace events from dispatcher. Move trace to common 2023-08-16 12:39:42 -04:00
Paul Lorenz 2ee9539f9c Move link calculation to routers. Fixes #692, fixes #749, fixes openziti/fabric#343 2023-08-03 17:07:48 -04:00
Paul Lorenz 8149b69614 Add support for quiescing and dequiescing routers. Fixes openziti/fabric#538
Quiesce marks all terminators as failed and stores the previous state.
Dequiesce restores all terminators with failed state and stored previous
state to their previous state
2023-06-05 17:46:13 -04:00
Paul Lorenz d1d0fe781b Update for storage API changes. 2023-04-25 11:32:33 -04:00
Paul Lorenz cc09f32c34 Implement batched terminator deletes 2023-03-28 13:18:59 -04:00
Paul Lorenz 820e6391a4 Add heartbeat config and controller peer heartbeats. Fixes openziti/fabric#507
Adds config to controller for heartbeat config
Add config to routers for heartbeat config
Adds configurable heartbeats to controller <-> controller connections
2023-02-28 09:47:53 -05:00
Paul Lorenz 519574497e Update to channel v2 2022-09-26 17:00:34 -04:00
Paul Lorenz 9aa41409b3 Cleanup trace controller loop. Refactor for multiple control channels: openziti/fabric#469 2022-09-22 11:05:35 -04:00
Paul Lorenz 3d49022f4a Update for foundation changes 2022-07-01 14:29:12 -04:00
Paul Lorenz b26fcf8909 Fix copyright 2022-06-29 16:59:46 -04:00
Paul Lorenz 2cdf3f469c Verify router instead of link when getting new link dial 2022-05-10 14:26:55 -04:00
Cam Otts 113c37024e Added heartbeat logging/metrics to control channel (#341)
* Added heartbeat logging/metrics to control channel

* pr comments

* misunderstood last pr comment

* moved channel closing to happen in either case
2022-04-06 11:10:58 -05:00
Paul Lorenz 2b7db10913 Allow routers more control over link management (#318)
* Allow routers more control over link management
2022-03-07 12:22:17 -05:00
Paul Lorenz 84ae5576e1 Update to use channel library 2022-01-27 16:46:21 -05:00
Paul Lorenz e781e3c713 Update control channel to new channel API 2022-01-19 08:15:33 -05:00
Paul Lorenz 6ba7264ff3 Do additional cert chain and verify links with controller 2021-11-12 15:36:05 -05:00
Paul Lorenz 58e159236d Use string id instead of identity where we don't need certs 2021-09-03 14:10:28 -04:00
Paul Lorenz 21c261fb19 Rename session -> circuit 2021-08-04 13:17:39 -04:00
Paul Lorenz cb70e33fa0 Improve link logging. Add destination id to xlink 2021-07-30 10:18:45 -04:00
Paul Lorenz 375dfd03b4 Add configurable router health check 2021-06-17 12:32:16 -04:00
Michael Quigley eb904f8b4c Controller cleanup side. (#173) 2021-03-05 14:14:26 -05:00
Michael Quigley 765e7f30bb Implement non-acting session confirmation receive hander, to prevent future errors with newer routers (#173, openziti/fabric#174) 2021-02-08 14:08:35 -05:00
Michael Quigley 2195476448 Skeleton for route result message handler. 2021-02-05 13:12:10 -05:00
Paul Lorenz e8ff9b7799 Add latency and message metrics to router control channels 2021-02-02 16:28:12 -05:00
Paul Lorenz a9117f4426 Convert metrics, traces and session streams to new events framework. Fixes GH-106 2020-08-11 12:55:43 -04:00
Michael Quigley 10e6a7ef93 netfoundry/ziti-fabric -> openziti/fabric 2020-05-26 17:14:16 -04:00
Paul Lorenz 414cb52953 Implement update terminator for ctrl channel. Lots of xt fixes. Log level tidying 2020-05-06 14:29:10 -04:00
Michael Quigley 2902c4942b Move the X... frameworks underneath controller/ or router/. 2020-03-20 13:57:59 -04:00
Paul Lorenz c32144d668 Remove copyright date 2020-03-19 13:26:09 -04:00