- closes a dialed link whose destination or link state is absent from the registry, where it was previously only logged, so a link this router can never report, fault or close does not survive
- skips dialing a queued link state whose destination is no longer registered, since removing a destination leaves its linkMap intact and the existing liveness check cannot see the removal
- leaves an accepted link alone, as the listener side holds no state for one, and leaves the existing fault path for a closed detached link unchanged
- logs each link pairing detached during listener reconciliation, with the reporting controller and whether a peer update or local rescan caused it
- passes the reporting controller id through UpdateLinkDest from the peer state handler
- tests the close, the accepted-link exemption, and the removed destination not being dialed
- adds a structured xlink.LinkKey ({DialerBinding, Protocol, DestId,
ListenerBinding}) used by all staleness and GC checks, replacing the
pre-rendered key string each side used to pass around
- adds the CheckStaleLinks ctrl message and the `ziti ops verify stale-links`
CLI for operator-driven, two-sided staleness verification with optional --gc,
which acts only when both endpoints agree
- implements auto-GC via the router.link.v1 gcMode (preserve/orphaned/changed):
the router walks its xlinks after each Apply that mutates listeners, dialers,
or gcMode and closes one-sided-stale entries under the configured mode
- sweeps a removal under orphaned whatever the removed config said, including
preserve, since a removal leaves no listeners and no dialers and every
remaining link is one the router can never re-establish
- reports staleness as a three-state verdict (unknown/notStale/stale) so a side
that cannot judge a link abstains instead of guessing; a destination absent
from the registry's listener snapshot means "listeners unknown", not "peer has
none", which keeps a briefly unhealthy peer from losing its live links
- rebuilds the link surface only when a listener or dialer definition actually
moves, so a gcMode-only change no longer rebinds every listen socket
- gates CheckStaleLinks on a new RouterStaleLinkCheck capability and explains
each unqueryable endpoint, including one left unasked by the filter, so a
partial verdict says which of the three causes applies
- detects a peer re-advertising the address a link was dialed to from the
dialing side, not just from the peer's own listener side
- pins every verdict to the link iteration it was computed against, faults that
iteration rather than the current one, and takes the conditional removal's
result as the decision, so a link re-dialed while the sweep was running is
neither closed nor reported as collected
- resolves which side a report speaks for from the sending router against the
controller's link map rather than from the side the report names, so one
router cannot fill both sides and satisfy the both-agree rule --gc depends on
- bounds the query fan-out with a semaphore, matching the other validation
sweeps, since each query carries a minute-long reply timeout
- fails the CLI when the result stream ends before every link is reported, and
drains results already buffered before declaring any missing
- rejects an unrecognized match mode rather than resolving it to changed, the
broadest removal criterion
- consolidates the controller-managed router configuration changelog into one
section, replacing the status that said routers do not yet consume configs at
runtime, and documents the per-config-type opt-in
- restores the Current Beta Features section, dropped when the 2.0 changelog was
archived, and lists controller-managed router configuration in it
- adds unparseable to the codespell ignore list, a spelling variant the
dictionary flags as a typo
For #3743.
- adds a common/servermetrics package that owns the metrics MetricsMessage wire
format and the reporting/usage subsystem (message builder, usage registry,
interval and usage counters), wrapping the openziti/metrics Registry for
metric collection
- moves the controllers metrics reporter into the router package and removes it
from the shared metrics package, breaking a common -> router/env import cycle
- repoints controller and router consumers to common/servermetrics; base metric
collection stays on openziti/metrics
- keeps the proto field numbers and the metrics content-type identical so the
encoding is byte-compatible across the move, and uses a distinct proto package
name so ziti's and the library's messages coexist without a global proto
registry clash
- adds a round-trip test asserting wire compatibility with the library's
MetricsMessage
- leaves openziti/metrics unchanged, so sdk-golang and the shared xgress data
plane are unaffected
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.
- Rewrites `github.com/openziti/sdk-golang/...` imports to
`github.com/openziti/sdk-golang/v2/...` across the main and zititest
modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
`RemoteAddr().String()` in the loop4 traffic-test logging.
For openziti/sdk-golang#936.
- decomposes the ctrlchan, xlink and edge-listener channels onto the v5 Senders, MessageSourceProvider and UnderlayEventListener interfaces, replacing the v4 UnderlayHandler god-interface
- replaces the hand-rolled dial/grouping/backoff machinery with channel.BackoffDialPolicy and declarative Constraints; ctrl keeps survive-to-zero (Min: 0) with MinStableDuration: 0 for prompt reconnect, while xlink and edge default underlays keep Min: 1 so loss closes the channel
- builds grouped channels via channel.NewChannel(*Config) and moves handler retrieval to GetSenders()
- records the channel via InitChannel from each bind handler, before underlay events fire, so handlers registered during bind do not dereference a nil channel
- generates a group secret for ungrouped inbound ctrl underlays on the router accept path, matching the controller, since NewChannel requires one
- preserves link-id-as-channel-id (the link dial policy wraps the cloned link-id identity dialer) and adds a test asserting dialed underlays present the link id
- registers the latency handler explicitly, as it is no longer a self-describing receiver in v5
- channel.MultiChannel -> channel.Channel
- channel.MultiChannelConfig -> channel.Config
- channel.NewMultiChannel(...) -> channel.NewChannel(...)
channel/v5 unifies Channel and MultiChannel into a single Channel abstraction. This is the
mechanical token rename; the Config field changes and handler retrieval that the unification
requires land in the following commit. The two channel.go files that are fully rewritten for
v5 (common/ctrlchan, router/xlink_transport) are excluded here and rewritten in that commit.
Does not build on its own.
- channel.TypedReceiveHandler -> channel.ContentTypeReceiver
- binding.AddTypedReceiveHandler(h) -> channel.AddReceiveHandlers(binding, h)
channel/v5 repurposes TypedReceiveHandler for the senders-typed handler and replaces the
self-describing pattern with ContentTypeReceiver plus the AddReceiveHandlers free function
(openziti/channel#262). Mechanical conversion; does not build on its own.
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5
This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
- invokes bind handlers via h.BindChannel(binding) instead of binding.Bind(h), which channel v5 removes from the Binding interface
- removes WithPriority from edge dial and state message sends; priority was already a no-op on grouped channels and channel v5 removes the priority API
- adds `ziti agent inspect <value>...` CLI that sends an InspectRequest directly to a ziti process over its agent IPC channel and pretty-prints JSON values in the response
- accepts app id 0 on controller and router agent channels so a single inspect command works against any process type
- adds Controller.agentOpInspect, backed by a new InspectionsManager.InspectLocal that runs inspect processing on the local controller only, without fanning out to routers or peer controllers
- extracts the router inspect handler into a new router/inspect package and stores a single shared instance on Router, reused by both the control channel and the agent IPC channel
- adds RouterEnv.GetInspectHandler and RouterEnv.GetXgressListeners so the control channel bind pulls the shared handler from env
- removes the now-redundant InspectRouterEnv interface from handler_ctrl/bind.go
- adds tunnel HandleAgentAsyncOp with support for stackdump and sdk inspect keys
- adds ErrorType constants for rejected-by-application, DNS resolution failed,
port not allowed, invalid link destination, and resources not available
- adds corresponding CircuitFailureCause strings reported in circuit events
- extracts classifyDialError() in route handler to map dial errors to specific
error codes using typed errors, syscall constants, and string matching
- detects DNS errors via *net.DNSError and string fallback for ER/T hosted
services where errors are serialized through the SDK message protocol
- detects resource exhaustion via EMFILE, ENFILE, ENOBUFS syscall errors
- introduces InvalidLinkDestinationError typed error in forwarder package
- adds unit tests covering all 16 classification cases
- adds integration tests for rejected-by-application (SDK host),
DNS resolution failed, connection refused, and port not allowed
(ER/T host mode) with circuit event verification
- adds CreateEnrollAndStartTunnelerEdgeRouterWithCfgTweaks to test context
- removes legacy v1 terminator code path; all terminators now use v2 flow
- refactors edgeTerminator.close() to decouple SDK notification from control plane notification
- adds pending SDK close notification queue with retry when channel is busy
- adds post-create inspect mechanism that verifies SDK still holds the bind after terminator creation
- queues second post-create inspect when establishment takes >30s to catch SDK timeout races
- detects and discards stale reordered binds on the same connection by comparing connIds
- re-establishes replacement terminators when a delete/create race is detected
- eliminates IsEntityPresent pre-filter in removeTerminatorsHandler to prevent raft ordering races
- fixes ValidateTerminators to query identities from the correct manager with the correct filter field
- adds postCreate flag to ValidateTerminatorsV2Request so routers skip redundant SDK inspect
- returns retry-later (nil result) from router validation when inspect is temporarily unavailable
- blocks SyncAllSubscribers until completion and guards RouterDataModel replacement with in-progress flag
- fixes InheritLocalData to enable service access tracking for all subscribed identities
- adds Services.Has check in GetServiceAccessPolicies to prevent false policy grants
- validates policy-to-identity associations in ValidateServicePolicies
- adds `ziti agent tunnel dump-sdk` command for SDK context inspection via IPC agent
- adds `ziti fabric inspect sdk` command to query SDK context through routers
- fixes --expected-per-host CLI flag binding in validate terminators command
- changes bind-access-lost retry hint from NotRetriable to RetryStartOver
- moves trace route response and xgress close handling off channel handler goroutine
- fixes listTerminators test helper to URL-encode filter parameter
- improves sdk-hosting-test validation resilience with login and query retries
- adds terminator_create_flow.md documenting the full lifecycle across SDK, router, and controller
- adds detailed logging for data model sync, service access tracking, and subscriber change detection
* Allow routers to request current cluster membership information. Fixes#3503
* Get cluster membership information from raft directly, rather than trying to cache it in the DB. Fixes#3501
* Set a router data model timeline when initializing a new HA setup, rather than letting it stay blank. Fixes#3500
* Reduce router data model full state updates. Fixes#3504
* We're sending peer change messages to older routers that don't understand them.
* We should probably use the older controller link management code for routers without the link management stability fixes
* Includes a few minor cleanups
Paul Lorenz