144 Commits

Author SHA1 Message Date
Paul Lorenz 187aa11f24 Own the metrics wire format in ziti. Fixes #4036
- adds a common/servermetrics package that owns the metrics MetricsMessage wire
  format and the reporting/usage subsystem (message builder, usage registry,
  interval and usage counters), wrapping the openziti/metrics Registry for
  metric collection
- moves the controllers metrics reporter into the router package and removes it
  from the shared metrics package, breaking a common -> router/env import cycle
- repoints controller and router consumers to common/servermetrics; base metric
  collection stays on openziti/metrics
- keeps the proto field numbers and the metrics content-type identical so the
  encoding is byte-compatible across the move, and uses a distinct proto package
  name so ziti's and the library's messages coexist without a global proto
  registry clash
- adds a round-trip test asserting wire compatibility with the library's
  MetricsMessage
- leaves openziti/metrics unchanged, so sdk-golang and the shared xgress data
  plane are unaffected
2026-06-29 22:37:25 -04:00
Paul Lorenz 86092a8640 Migrate to the sdk-golang v2 module path. For #3884
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.

- Rewrites `github.com/openziti/sdk-golang/...` imports to
  `github.com/openziti/sdk-golang/v2/...` across the main and zititest
  modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
  which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
  `RemoteAddr().String()` in the loop4 traffic-test logging.

For openziti/sdk-golang#936.
2026-06-23 15:43:39 -04:00
Paul Lorenz d481224c9b Decompose multi-underlay channels onto the channel/v5 API. Fixes #3983
- decomposes the ctrlchan, xlink and edge-listener channels onto the v5 Senders, MessageSourceProvider and UnderlayEventListener interfaces, replacing the v4 UnderlayHandler god-interface
- replaces the hand-rolled dial/grouping/backoff machinery with channel.BackoffDialPolicy and declarative Constraints; ctrl keeps survive-to-zero (Min: 0) with MinStableDuration: 0 for prompt reconnect, while xlink and edge default underlays keep Min: 1 so loss closes the channel
- builds grouped channels via channel.NewChannel(*Config) and moves handler retrieval to GetSenders()
- records the channel via InitChannel from each bind handler, before underlay events fire, so handlers registered during bind do not dereference a nil channel
- generates a group secret for ungrouped inbound ctrl underlays on the router accept path, matching the controller, since NewChannel requires one
- preserves link-id-as-channel-id (the link dial policy wraps the cloned link-id identity dialer) and adds a test asserting dialed underlays present the link id
- registers the latency handler explicitly, as it is no longer a self-describing receiver in v5
2026-06-18 12:51:02 -04:00
Paul Lorenz ee8ad78e3b Rename MultiChannel to the unified Channel for channel/v5. For #3983
- channel.MultiChannel -> channel.Channel
- channel.MultiChannelConfig -> channel.Config
- channel.NewMultiChannel(...) -> channel.NewChannel(...)

channel/v5 unifies Channel and MultiChannel into a single Channel abstraction. This is the
mechanical token rename; the Config field changes and handler retrieval that the unification
requires land in the following commit. The two channel.go files that are fully rewritten for
v5 (common/ctrlchan, router/xlink_transport) are excluded here and rewritten in that commit.
Does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz ae806045b5 Convert self-describing receive handlers to channel/v5. For #3983
- channel.TypedReceiveHandler -> channel.ContentTypeReceiver
- binding.AddTypedReceiveHandler(h) -> channel.AddReceiveHandlers(binding, h)

channel/v5 repurposes TypedReceiveHandler for the senders-typed handler and replaces the
self-describing pattern with ContentTypeReceiver plus the AddReceiveHandlers free function
(openziti/channel#262). Mechanical conversion; does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz 1c122af490 Rewrite channel/v4 imports to channel/v5. For #3983
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5

This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz b5eaeefdb3 Prep for channel v5: bind handler invocation, send priorities. Fixes #3942
- invokes bind handlers via h.BindChannel(binding) instead of binding.Bind(h), which channel v5 removes from the Binding interface
- removes WithPriority from edge dial and state message sends; priority was already a no-op on grouped channels and channel v5 removes the priority API
2026-06-05 15:24:45 -04:00
Paul Lorenz c44350b3a7 Support ziti agent inspect so ziti processes can be inspected via IPC. Fixes #3824. Fixes #2049
- adds `ziti agent inspect <value>...` CLI that sends an InspectRequest directly to a ziti process over its agent IPC channel and pretty-prints JSON values in the response
- accepts app id 0 on controller and router agent channels so a single inspect command works against any process type
- adds Controller.agentOpInspect, backed by a new InspectionsManager.InspectLocal that runs inspect processing on the local controller only, without fanning out to routers or peer controllers
- extracts the router inspect handler into a new router/inspect package and stores a single shared instance on Router, reused by both the control channel and the agent IPC channel
- adds RouterEnv.GetInspectHandler and RouterEnv.GetXgressListeners so the control channel bind pulls the shared handler from env
- removes the now-redundant InspectRouterEnv interface from handler_ctrl/bind.go
- adds tunnel HandleAgentAsyncOp with support for stackdump and sdk inspect keys
2026-04-24 12:12:02 -04:00
Paul Lorenz 90f32affda Address review comments 2026-04-14 16:09:19 -04:00
Paul Lorenz 538623ceb6 Add specific circuit failure error codes. Fixes #3717, fixes #3543, fixes #3364, fixes #2888, fixes #2859, fixes #1580
- adds ErrorType constants for rejected-by-application, DNS resolution failed,
  port not allowed, invalid link destination, and resources not available
- adds corresponding CircuitFailureCause strings reported in circuit events
- extracts classifyDialError() in route handler to map dial errors to specific
  error codes using typed errors, syscall constants, and string matching
- detects DNS errors via *net.DNSError and string fallback for ER/T hosted
  services where errors are serialized through the SDK message protocol
- detects resource exhaustion via EMFILE, ENFILE, ENOBUFS syscall errors
- introduces InvalidLinkDestinationError typed error in forwarder package
- adds unit tests covering all 16 classification cases
- adds integration tests for rejected-by-application (SDK host),
  DNS resolution failed, connection refused, and port not allowed
  (ER/T host mode) with circuit event verification
- adds CreateEnrollAndStartTunnelerEdgeRouterWithCfgTweaks to test context
2026-04-14 15:52:49 -04:00
Paul Lorenz 6d59475b35 Allow binding specific inspects to pass through to xgress listener implementations. Fixes #3719 2026-03-25 14:34:48 -04:00
Paul Lorenz e1638173cd Fixes for SDK terminator management. Add support for ziti sdk inspection. Fixes #3609
- removes legacy v1 terminator code path; all terminators now use v2 flow
- refactors edgeTerminator.close() to decouple SDK notification from control plane notification
- adds pending SDK close notification queue with retry when channel is busy
- adds post-create inspect mechanism that verifies SDK still holds the bind after terminator creation
- queues second post-create inspect when establishment takes >30s to catch SDK timeout races
- detects and discards stale reordered binds on the same connection by comparing connIds
- re-establishes replacement terminators when a delete/create race is detected
- eliminates IsEntityPresent pre-filter in removeTerminatorsHandler to prevent raft ordering races
- fixes ValidateTerminators to query identities from the correct manager with the correct filter field
- adds postCreate flag to ValidateTerminatorsV2Request so routers skip redundant SDK inspect
- returns retry-later (nil result) from router validation when inspect is temporarily unavailable
- blocks SyncAllSubscribers until completion and guards RouterDataModel replacement with in-progress flag
- fixes InheritLocalData to enable service access tracking for all subscribed identities
- adds Services.Has check in GetServiceAccessPolicies to prevent false policy grants
- validates policy-to-identity associations in ValidateServicePolicies
- adds `ziti agent tunnel dump-sdk` command for SDK context inspection via IPC agent
- adds `ziti fabric inspect sdk` command to query SDK context through routers
- fixes --expected-per-host CLI flag binding in validate terminators command
- changes bind-access-lost retry hint from NotRetriable to RetryStartOver
- moves trace route response and xgress close handling off channel handler goroutine
- fixes listTerminators test helper to URL-encode filter parameter
- improves sdk-hosting-test validation resilience with login and query retries
- adds terminator_create_flow.md documenting the full lifecycle across SDK, router, and controller
- adds detailed logging for data model sync, service access tracking, and subscriber change detection
2026-03-11 13:10:28 -04:00
Paul Lorenz 6b869cea9d Add support for ctrlChanListener on router to the model. Fixes #3635 2026-03-07 00:00:51 -05:00
Paul Lorenz 560c44b81c Address review comments 2026-02-24 16:15:35 -05:00
Paul Lorenz 84c98e2481 Version 2.0+ routers should not connect to controllers which do not support JWT formatted legacy sessions. Fixes #3569 2026-02-13 15:28:50 -05:00
Paul Lorenz 90112219a3 Support multi-underlay control channels. Fixes #3550 2026-02-11 14:20:29 -05:00
Paul Lorenz 398a402d17 Remove the legacy xgress_edge_tunnel implementation. Fixes #3535 2026-02-09 14:48:25 -05:00
Paul Lorenz da9ef76d19 Optimize imports 2026-01-28 15:01:35 -05:00
Paul Lorenz 2ffc6e1151 Update ziti version to 2.0 2026-01-28 12:34:10 -05:00
Paul Lorenz 08359a9e2b Remove legacy link management code from the controller. Fixes #3512 2026-01-22 15:26:42 -05:00
Paul Lorenz c18a594b56 Clustering coordination fixes
* Allow routers to request current cluster membership information. Fixes #3503
* Get cluster membership information from raft directly, rather than trying to cache it in the DB. Fixes #3501
* Set a router data model timeline when initializing a new HA setup, rather than letting it stay blank. Fixes #3500
* Reduce router data model full state updates. Fixes #3504
2026-01-17 02:03:31 -05:00
Paul Lorenz 3bca4801b2 Fix router ctrl channel init race condition. Fixes #3484. ert-hosting-test fixes 2026-01-06 16:37:21 -05:00
Paul Lorenz d37c721c83 Optimize router data model subscription code. Add additional events. Fixes #3359 2025-12-02 11:37:38 -05:00
Paul Lorenz 8bedd3af1d Clean up connect events tests and remove global XG registry. Fixes #3345 2025-10-29 14:46:52 -04:00
Paul Lorenz 2096535abe Add missing shutdown logic 2025-09-18 15:33:45 -04:00
Paul Lorenz 9d4ff055b5 Update for SDK flow control changes 2025-07-18 12:42:45 -04:00
Paul Lorenz 79c660903c Add API for xgress router factories allowing router env injection. Fixes #3111 2025-06-17 17:38:35 -04:00
Paul Lorenz b21db0faa1 Implement circuit validation API and CLI. Fixes #3098 2025-06-16 13:57:42 -04:00
Paul Lorenz 981df8269d Support xgress flow control from the SDK. Fixes #2986 2025-04-30 10:27:43 -04:00
Paul Lorenz e647d67325 Update to channel/v4 2025-04-02 15:28:59 -04:00
Paul Lorenz 88ee16ec21 Move some router specific functionality to xgress_router package. Fixes #2950 2025-03-28 09:59:46 -04:00
Paul Lorenz 22887a40af Make xgress acker configurable Fixes #2920 2025-03-28 09:48:05 -04:00
Paul Lorenz ec57c80ff7 ER/T Hosting HA chaos test and fixes (#2806)
* Add ERT hosting chaos test. Also add ert terminator validation utility. Fixes #2288

* Rework ER/T terminator management based on SDK terminator management code

* Update deps

* Make sdk/ert-terminators into a constant
2025-02-25 17:02:21 -05:00
Paul Lorenz 03870760c3 Ensure restores and migrations work properly with RDM. Add restore from db for HA Clusters. Fixes #2549. Fixes #2649. Fixes #2707 2025-01-31 13:57:43 -05:00
Paul Lorenz b96ae50b08 Always use RDM when there are multiple controllers. Fixes #2694 2025-01-27 09:51:41 -05:00
Paul Lorenz 57a367b48c Fix empty controller list on upgrade. Fixes #2692 2025-01-24 23:32:17 -05:00
Paul Lorenz 5429ce5064 Separate leader updates from controller cluster member updates 2025-01-09 16:18:34 -05:00
Paul Lorenz 37a80da89b Add RDM config, including a ctrl side disable flag. Fixes #2596 2025-01-07 19:22:47 -05:00
Paul Lorenz a34478bb06 Fix controller endpoint updates and update deps. Test RDM with HA. 2025-01-07 19:22:46 -05:00
Paul Lorenz e09c56867c Remove HA config from router. Fixes #2566 2025-01-07 19:21:53 -05:00
Paul Lorenz 32eddd61ca HA SDK terminators test. Fixes #2217. Fixes #2533 2024-11-12 18:45:08 -05:00
Paul Lorenz 6e1983991d Fixes from testing 2024-11-01 15:21:01 -04:00
Paul Lorenz 16f0a858e6 Add controller connect events. Fixes #1835. Fixes #2234 2024-11-01 15:21:01 -04:00
Paul Lorenz cccf0c06af Update to channel/v3. Fixes #2390 2024-09-09 12:23:25 -04:00
Paul Lorenz ef9a31359d Fix responsive check. Fixes #2171. Add router-connections inspection. Fixes #2219. Improve can't load data file log message. Fixes #2195 2024-07-15 09:49:49 -04:00
Paul Lorenz 898e41ae2a Implement router data model changesets. Fixes #1966 2024-04-23 11:56:10 -04:00
Paul Lorenz ad779ed118 Improve link mgmt backwards compatibility. Fixes #1900
* We're sending peer change messages to older routers that don't understand them.
* We should probably use the older controller link management code for routers without the link management stability fixes
* Includes a few minor cleanups
2024-04-04 19:07:24 -04:00
Paul Lorenz c64df3a7b3 Merge pull request #1788 from openziti/terminator-chaos
Terminator chaos
2024-03-12 15:31:59 -04:00
Paul Lorenz fab81e4cd4 Add terminator chaos testing and fix issues found. Fixes #1794 Fixes #1369 2024-03-12 10:31:36 -04:00
Paul Lorenz 37b2fe5a0e Fix check for timeout error. Fixes #1791 2024-03-01 14:06:36 -05:00