149 Commits

Author SHA1 Message Date
Paul Lorenz 588cb2350c Fix data race on heartbeat lastResponse and peer logger label
- changes lastResponse from plain int64 to atomic.Int64 in both router
  and peer heartbeat callbacks, fixing a data race between the heartbeat
  response handler and the heartbeat check ticker
- fixes peer heartbeat logger channelType from "router" to "peer"
2026-04-03 17:42:18 -04:00
Paul Lorenz 7b4ae12c05 Add CreateCircuitV3 for RDM-authorized circuit creation. Fixes #3721
- adds CreateCircuitV3 message type and handler for routers that have
  already authorized dials locally via RDM, bypassing service session
  tokens in favor of identity ID, service ID, and pre-assigned circuit ID
- renames CreateCircuitRequest/Response to CreateCircuitV2Request/V2Response
  for clarity now that V3 exists
- adds CircuitManager.Reserve to atomically claim circuit IDs before routing,
  preventing collisions on pre-assigned IDs
- extends CreateCircuitParams with GetCircuitId so V3 can supply a
  pre-assigned circuit ID (falls back to UUID generation when empty)
- fixes IsDialableByIdentity which was incorrectly calling IsBindableByIdentity
- extracts V2 handler into its own file create_circuit_v2.go
- adds CreateCircuitV3RequestType/ResponseType (20222/20223) to edge_ctrl protobuf
- registers V3 handler in controller server
2026-03-27 14:41:28 -04:00
Paul Lorenz e1638173cd Fixes for SDK terminator management. Add support for ziti sdk inspection. Fixes #3609
- removes legacy v1 terminator code path; all terminators now use v2 flow
- refactors edgeTerminator.close() to decouple SDK notification from control plane notification
- adds pending SDK close notification queue with retry when channel is busy
- adds post-create inspect mechanism that verifies SDK still holds the bind after terminator creation
- queues second post-create inspect when establishment takes >30s to catch SDK timeout races
- detects and discards stale reordered binds on the same connection by comparing connIds
- re-establishes replacement terminators when a delete/create race is detected
- eliminates IsEntityPresent pre-filter in removeTerminatorsHandler to prevent raft ordering races
- fixes ValidateTerminators to query identities from the correct manager with the correct filter field
- adds postCreate flag to ValidateTerminatorsV2Request so routers skip redundant SDK inspect
- returns retry-later (nil result) from router validation when inspect is temporarily unavailable
- blocks SyncAllSubscribers until completion and guards RouterDataModel replacement with in-progress flag
- fixes InheritLocalData to enable service access tracking for all subscribed identities
- adds Services.Has check in GetServiceAccessPolicies to prevent false policy grants
- validates policy-to-identity associations in ValidateServicePolicies
- adds `ziti agent tunnel dump-sdk` command for SDK context inspection via IPC agent
- adds `ziti fabric inspect sdk` command to query SDK context through routers
- fixes --expected-per-host CLI flag binding in validate terminators command
- changes bind-access-lost retry hint from NotRetriable to RetryStartOver
- moves trace route response and xgress close handling off channel handler goroutine
- fixes listTerminators test helper to URL-encode filter parameter
- improves sdk-hosting-test validation resilience with login and query retries
- adds terminator_create_flow.md documenting the full lifecycle across SDK, router, and controller
- adds detailed logging for data model sync, service access tracking, and subscriber change detection
2026-03-11 13:10:28 -04:00
Paul Lorenz 6b869cea9d Add support for ctrlChanListener on router to the model. Fixes #3635 2026-03-07 00:00:51 -05:00
Paul Lorenz 8acf90cdf6 Multi-underlay control channel doesn't correctly handle lack of group secret on non-grouped underlays. Fixes #3624 2026-03-02 13:52:11 -05:00
Paul Lorenz ab51fa8217 Fix the control channel header conflicts with channel headers. Use single strategy for sharing capabilities with bit mask. 2026-02-24 15:50:05 -05:00
Paul Lorenz 90112219a3 Support multi-underlay control channels. Fixes #3550 2026-02-11 14:20:29 -05:00
Paul Lorenz da9ef76d19 Optimize imports 2026-01-28 15:01:35 -05:00
Paul Lorenz 2ffc6e1151 Update ziti version to 2.0 2026-01-28 12:34:10 -05:00
Paul Lorenz 08359a9e2b Remove legacy link management code from the controller. Fixes #3512 2026-01-22 15:26:42 -05:00
Paul Lorenz c18a594b56 Clustering coordination fixes
* Allow routers to request current cluster membership information. Fixes #3503
* Get cluster membership information from raft directly, rather than trying to cache it in the DB. Fixes #3501
* Set a router data model timeline when initializing a new HA setup, rather than letting it stay blank. Fixes #3500
* Reduce router data model full state updates. Fixes #3504
2026-01-17 02:03:31 -05:00
Paul Lorenz 3cfc8b5cf5 Controller should clear links not in full link sync from router. Fixes #3492 2026-01-16 14:42:39 -05:00
Paul Lorenz a4b4e1dc16 Clean up some excessive logging 2025-10-29 15:30:31 -04:00
Paul Lorenz 8bedd3af1d Clean up connect events tests and remove global XG registry. Fixes #3345 2025-10-29 14:46:52 -04:00
Paul Lorenz 653063767e Add alert events. Fixes #3264 2025-10-15 14:21:07 -04:00
Paul Lorenz e5fb69935e Support multi-underlay links. Fixes #3134 2025-07-11 20:56:11 -04:00
Paul Lorenz cc7eefaf24 Add support for tracking network interfaces on routers and identities. Fixes #3082. Add network discover to router. Fixes #3083 2025-06-17 09:32:51 -04:00
Paul Lorenz 981df8269d Support xgress flow control from the SDK. Fixes #2986 2025-04-30 10:27:43 -04:00
Paul Lorenz e647d67325 Update to channel/v4 2025-04-02 15:28:59 -04:00
Paul Lorenz 8dbf6e5d85 Update errorz.MultipleErrors users to errors.Join 2025-03-22 10:14:02 -04:00
Paul Lorenz ec57c80ff7 ER/T Hosting HA chaos test and fixes (#2806)
* Add ERT hosting chaos test. Also add ert terminator validation utility. Fixes #2288

* Rework ER/T terminator management based on SDK terminator management code

* Update deps

* Make sdk/ert-terminators into a constant
2025-02-25 17:02:21 -05:00
Paul Lorenz 32eddd61ca HA SDK terminators test. Fixes #2217. Fixes #2533 2024-11-12 18:45:08 -05:00
Paul Lorenz cccf0c06af Update to channel/v3. Fixes #2390 2024-09-09 12:23:25 -04:00
Paul Lorenz c3b43133d1 Merge fabric and controller model code. Fixes #2205 2024-07-09 16:11:01 -04:00
Paul Lorenz 8d11af3d97 Optimize network run loop. Fixes #1897
* When routers change (connect/disconnect), we re-check the mesh. We are passing in the routers to the channel and they can build up, which is unnecessary. We only need to signal once for all the routers that change since the last time we checked, that the mesh needs to be checked.
* We pass link changes through the run method, which is unnecessary since we pass it to a new goroutine. This inefficiency should be fixed.
* We're evaluating all faulted links for rerouting, even if they were still pending. Only reroute connected links.
* We're using time.After in each for loop, which can accumulate timers. Use a single Ticker instead.
2024-04-04 18:46:50 -04:00
Paul Lorenz fab81e4cd4 Add terminator chaos testing and fix issues found. Fixes #1794 Fixes #1369 2024-03-12 10:31:36 -04:00
Paul Lorenz 8423a8dd6a Fix link management race conditions found by chaos testing. Fixes #1709 2024-01-29 17:23:51 -05:00
Paul Lorenz c03c3c2851 Additional fixes and tweaks 2024-01-19 17:35:25 -05:00
Paul Lorenz 70b32603ad Start on chaos testing 2024-01-19 17:34:15 -05:00
Paul Lorenz c4abcae949 Link management fixes. Fixes #1692 Fixes #1693 2024-01-19 17:33:21 -05:00
Paul Lorenz df2ab68ad5 Make sure link accepts are not single threaded. Fixes #1669 2023-12-29 10:18:50 -05:00
Paul Lorenz 9ddd1b941d Add service max idle time. Fixes #1496. Add decomission router support. Fixes #1402 2023-12-08 12:48:56 -05:00
Paul Lorenz b270d18faf Add additional logging. Remove unnecessary router lookups. May address #1460 2023-10-31 14:57:06 -04:00
Paul Lorenz 3523b34b2c Fix panic when router version info isn't set by ensuring it is set and making sure we get the right version. Fixes #1423 2023-10-11 13:55:50 -04:00
Paul Lorenz f3d67b7f49 Update fabric imports 2023-09-28 23:34:28 -04:00
Paul Lorenz 700ec34177 Make link heartbeats configurable. Add unresponsive close timeout. Fixes openziti/fabric#796 2023-09-18 22:40:19 -04:00
Paul Lorenz a631f48643 Do some forwarder option cleaning. Remove unused latency options 2023-08-22 20:37:59 -04:00
Paul Lorenz a7fe59e184 Do some configuration tidying 2023-08-22 20:37:25 -04:00
Paul Lorenz 509364d908 Remove trace events from dispatcher. Move trace to common 2023-08-16 12:39:42 -04:00
Paul Lorenz cdfd037da9 Move protobuf definitions to common 2023-08-10 19:38:48 -04:00
Paul Lorenz 00a39c9900 Move logcontext package to common 2023-08-10 18:07:00 -04:00
Paul Lorenz 368d6e5568 Move config, ctrl_msg and handler_common to common/ 2023-08-10 17:29:08 -04:00
Paul Lorenz 8dbc759d45 move event and events packages to controller 2023-08-10 17:10:17 -04:00
Paul Lorenz f15b8bef23 Add some more config value validation and guardrails 2023-08-03 17:07:49 -04:00
Paul Lorenz 2ee9539f9c Move link calculation to routers. Fixes #692, fixes #749, fixes openziti/fabric#343 2023-08-03 17:07:48 -04:00
Paul Lorenz 8149b69614 Add support for quiescing and dequiescing routers. Fixes openziti/fabric#538
Quiesce marks all terminators as failed and stores the previous state.
Dequiesce restores all terminators with failed state and stored previous
state to their previous state
2023-06-05 17:46:13 -04:00
Paul Lorenz ef555a536e Notify controller of forward faults on links. Fixes #724. If reroute fails, circuit should be torn down. Fixes openziti/fabric#725 2023-05-25 15:05:28 -04:00
Paul Lorenz bc9bac6f0d Add config support for link groups. Doesn't use group config yet. 2023-05-11 09:34:33 -04:00
Paul Lorenz a19a2ab2fe Annotate circuit events with circuit tags, similar to usage. Fixes openziti/fabric#684 2023-04-25 22:54:22 -04:00
Paul Lorenz 23110fccb2 Implement entity change events. Fixes openziti/fabric#562 2023-04-25 11:38:03 -04:00