Commit Graph

147 Commits

Author SHA1 Message Date
Paul Lorenz d7076430c9 Make ER/T terminator create failures diagnosable. Fixes #4193
- reports edge router policy denials with an access-denied error naming the missing policy,
  replacing the session error reused on the sessionless ER/T and create-circuit-v3 paths
- adds EdgeRouterManager.GetEdgeRouterAccess, which reports which of the two required policy
  links (identity-to-edge-router, service-to-edge-router) is absent, and removes the boolean
  IsAccessToEdgeRouterAllowed it replaces
- logs the controller's rejection on the router at warn level, since it is recoverable and
  retried by the periodic scan; the router previously discarded the error code and message
- delays a new terminator's first create attempt by a fixed 2s so config applied in quick
  succession settles before the router asks, avoiding a 2-3 minute wait for the retry scan; the
  delay is a deliberate stopgap until edge router policy visibility lands in the router data model
- propagates the controller's error code and retry hint to SDK clients on the dial paths, which
  dropped the code and left every refusal classified as unknown
- adds the retry hint header to controller error replies, grouped with the other error-reply
  headers rather than the create-circuit-v3 request headers
- notes that the sync strategy headers alias the edge namespace's 1013-1015 ids and stay
  disjoint only by message content type
- tests the per-policy denial reporting, the error code carried with and without a retry hint,
  the controller-to-SDK error code mapping at both dial relay sites, and the terminator settle
  gate
- waits for terminator establishment in the tunneler dataflow tests instead of a fixed sleep, so
  they no longer race the settle delay
- restores the tproxy multiple-lanIf and multiple-resolver changelog entries with keep markers,
  which regeneration drops because their commits reference pull requests rather than issues
2026-08-03 15:43:30 -04:00
Paul Lorenz 187aa11f24 Own the metrics wire format in ziti. Fixes #4036
- adds a common/servermetrics package that owns the metrics MetricsMessage wire
  format and the reporting/usage subsystem (message builder, usage registry,
  interval and usage counters), wrapping the openziti/metrics Registry for
  metric collection
- moves the controllers metrics reporter into the router package and removes it
  from the shared metrics package, breaking a common -> router/env import cycle
- repoints controller and router consumers to common/servermetrics; base metric
  collection stays on openziti/metrics
- keeps the proto field numbers and the metrics content-type identical so the
  encoding is byte-compatible across the move, and uses a distinct proto package
  name so ziti's and the library's messages coexist without a global proto
  registry clash
- adds a round-trip test asserting wire compatibility with the library's
  MetricsMessage
- leaves openziti/metrics unchanged, so sdk-golang and the shared xgress data
  plane are unaffected
2026-06-29 22:37:25 -04:00
Paul Lorenz 67eba590e2 Implement connect-v2. Fixes #3884
Implements the router-side Connect-V2 sessionless dial path. Dials are
authorized locally via the RouterDataModel instead of a controller-issued
service session token; circuit creation flows through the existing
`CreateCircuitV3` controller endpoint (#3721). Builds on the sdk-golang
v2 migration.

- Adds `processConnectV2` on `edgeClientConn`: resolves the service by id
  or name via the RouterDataModel, checks dial access, and dispatches to
  the controller via `sendCreateCircuitV3Msg`. Supports both
  `xgEdgeForwarder` (SDK xgress) and `nonXgConnectHandler` flow-control
  modes, selected by the SDK's `UseXgressToSdkHeader`.
- Makes `CircuitId` optional in `DecodeCreateCircuitV3Request`. The V2
  router path does not pre-assign a circuit ID; the controller generates
  it as V1/V2 already do. Without this the decoder rejected the empty
  header and every V2 dial hung until timeout. Adds a regression test.
- Splits `checkAccess` to close a posture-check bypass on the V2 path.
  The old single `checkAccess` short-circuited to nil for non-OIDC
  sessions (V1 ran posture at the controller during `CreateSession`); V2
  has no such step, so posture would have been skipped. `checkAccess` now
  always runs the RDM `HasAccess` (policy + posture) check;
  `checkAccessIfOidc` keeps the OIDC-only gate for the V1 and bind paths.
- Sends the V2 `state_connected` on the default (data) sender rather than
  the control sender. On multi-underlay channels the two senders are
  independently ordered, so an early terminator payload on the data
  sender could beat `state_connected` to the SDK and be dropped (channel/v5
  has no message-priority API).
- Updates `xgEdgeForwarder.lastRx` on every forward path, including the
  fast `timeout == 0` `TrySend` branch used for normal payload dispatch.
  The old code only updated it on the `timeout > 0` path, so active V2
  circuits looked idle and could be unrouted prematurely.
- Adds `state.ConnState.ServiceId`, populated by the connect handlers from
  the service session token (V1) or the request header (V2). The
  non-xgress V2 path previously left this empty, so `handleDialAccessLost`
  could not identify and close V2 non-xgress circuits when dial access was
  revoked.
- Skips conns with no `ServiceSessionToken` in `RemoveLegacyServiceSession`;
  a sessionless V2 conn's token is nil and the cleanup loop previously
  dereferenced it unconditionally, which would panic the router.
- Advertises Connect-V2 via the `RouterCapabilityConnectV2` bit in the
  listener hello so SDKs can detect V2 support.
- Wires `ContentTypeConnectV2` and `ContentTypeXgControl` handlers in
  `Acceptor.BindChannel`, and adds `handleXgControl` for SDK-side xgress
  control messages, preserving `ControlUserVal` so trace-route responses
  correlate back to the initiator's `SendForReply` waiter.
- Adds `RouterDataModel.serviceNameIndex` for O(1) name->id lookup in the
  V2 dial path, maintained with rename safety at the `HandleServiceEvent`
  mutation points.
- Adds `tests/connect_v2_test.go` covering end-to-end V2 dataflow and the
  V1 fallback (`ForceConnectV1`), asserting the dial path via the SDK
  `DialEvent`.

- Propagates a V2 initiator's graceful half-close to legacy hosts via
  `edgeXgressConn.FlowFromFabricToXgressClosed`, which emits an edge FIN
  when the fabric->app half of the circuit closes. The SDK signals
  half-close to its router xgress peer with the native xgress EOF flag;
  without translating that to an edge FIN, a legacy host reading to EOF
  stalled until teardown.
- Records the dialing identity id as the circuit `ClientId` for
  sessionless V2 dials, since there is no dial session to key on; updates
  `Test_OidcEvents` to match.
- Adds `tests/connect_v2_teardown_test.go` covering client- and
  host-initiated close propagation on both the V2 and forced-V1 paths.
- Polls for the asynchronous conn close in the SDK posture-check tests
  (`awaitClientConnClosed`): revocation tears the circuit down out of
  band, so checking `IsClosed` immediately after the first read error was
  racy.
- Temporarily pins sdk-golang/v2 to the openziti/sdk-golang#959 commit,
  which carries the matching xgress conn-close-on-teardown fix the V2
  posture tests depend on; to be repointed at the next sdk-golang
  pre-release before merge.

For openziti/sdk-golang#936.
2026-06-24 16:40:40 -04:00
Paul Lorenz 86092a8640 Migrate to the sdk-golang v2 module path. For #3884
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.

- Rewrites `github.com/openziti/sdk-golang/...` imports to
  `github.com/openziti/sdk-golang/v2/...` across the main and zititest
  modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
  which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
  `RemoteAddr().String()` in the loop4 traffic-test logging.

For openziti/sdk-golang#936.
2026-06-23 15:43:39 -04:00
Paul Lorenz ae806045b5 Convert self-describing receive handlers to channel/v5. For #3983
- channel.TypedReceiveHandler -> channel.ContentTypeReceiver
- binding.AddTypedReceiveHandler(h) -> channel.AddReceiveHandlers(binding, h)

channel/v5 repurposes TypedReceiveHandler for the senders-typed handler and replaces the
self-describing pattern with ContentTypeReceiver plus the AddReceiveHandlers free function
(openziti/channel#262). Mechanical conversion; does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz 1c122af490 Rewrite channel/v4 imports to channel/v5. For #3983
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5

This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz f18ce1b580 Collapse fabric and edge services at the db level. Fixes #3934
- collapses the edge service boltz child store into the unified service store:
  EdgeService becomes a type alias for Service, and RoleAttributes, Configs, and
  EncryptionRequired become top-level Service fields
- adds an IsFabricOnly discriminator separating pure-fabric ("management")
  services from edge services
- adds a forward-only, irreversible migration that moves edge service data out of
  the edge child bucket, classifies pre-existing services (fabric -> IsFabricOnly
  with encryptionRequired=false; edge -> IsFabricOnly=false), deletes the old edge
  bucket and its index, and rebuilds the role-attributes index via CheckIntegrity
- guards every edge-facing path on the unified store against fabric-only services
  so they stay invisible to the edge API: by-id Read/ReadByName/Delete, list/query
  and the detail lister, ReadForIdentity, identity service-config overrides,
  service policy / SERP @id and #all/role denormalization, the association-list
  routes (terminators, policies, configs), and the router data-model sync
- rejects edge-surface updates of fabric-only services, and preserves edge fields
  on fabric-surface updates of edge services
- marks the fabric-only guards TEMPORARY(fabric-edge-collapse): they exist only to
  keep seldom-used management services hidden, and are removed when the fabric/edge
  distinction is erased
- adds in-CI migration tests (forward plus a round-trip asserting FK sub-bucket
  relocation and forward/reverse refcount preservation), durable fabric-only
  behavior tests, and store-level policy-exclusion tests
- adds one-time authentic-migration gate tooling under zititest/migration-test
  (create-model.sh, query, verify) and its operator runbook
2026-06-16 13:04:13 -04:00
Paul Lorenz 82ac2e8060 Rework connect event handling to ensure serialized-per-router handling 2026-04-24 13:22:02 -04:00
Paul Lorenz 4df6ae564d Update package paths for newly imported storage and ziti-db-explorer packages 2026-04-16 09:18:55 -04:00
Paul Lorenz f2db0e147d Fix connect events handler goroutine leak. Fixes #3746
- replaces the per-handler goroutine in connectEventsHandler with a shared,
  bounded goroutine pool on AppEnv
- the old design spawned a processEvents goroutine per router connection that
  only exited on application shutdown, leaking a goroutine on every reconnect
- adds ConnectEventsConfig to the controller config with pool tuning options
  (queueSize, minWorkers, maxWorkers, idleTime)
- defaults: queue 16, 0-16 workers, 30s idle timeout
2026-04-02 23:44:43 -04:00
Paul Lorenz 7b4ae12c05 Add CreateCircuitV3 for RDM-authorized circuit creation. Fixes #3721
- adds CreateCircuitV3 message type and handler for routers that have
  already authorized dials locally via RDM, bypassing service session
  tokens in favor of identity ID, service ID, and pre-assigned circuit ID
- renames CreateCircuitRequest/Response to CreateCircuitV2Request/V2Response
  for clarity now that V3 exists
- adds CircuitManager.Reserve to atomically claim circuit IDs before routing,
  preventing collisions on pre-assigned IDs
- extends CreateCircuitParams with GetCircuitId so V3 can supply a
  pre-assigned circuit ID (falls back to UUID generation when empty)
- fixes IsDialableByIdentity which was incorrectly calling IsBindableByIdentity
- extracts V2 handler into its own file create_circuit_v2.go
- adds CreateCircuitV3RequestType/ResponseType (20222/20223) to edge_ctrl protobuf
- registers V3 handler in controller server
2026-03-27 14:41:28 -04:00
Andrew Martinez fb2034245d fixes openziti/ziti#3356 adds www-authenticate headers (#3561)
* fixes openziti/ziti#3356 adds www-authenticate headers

- www-authenticate headers are returned on 401s from API requests
- www-authenticate headers are returned during authentication to signal
  addtional JWT bearer tokens needed (secondary ext jwt)
- adds support for additional headers on API errors
- adds SecurityTokenCtx for centralized security header processing
  (legacy, jwt, etc.)
- adds SecurityCtx for centralized identity, auth policy, MFA handling
- refactors existing JWT authentication methods (oidc, legacy) to use
  centralized processing where possible
2026-02-24 10:01:36 -05:00
Paul Lorenz 90112219a3 Support multi-underlay control channels. Fixes #3550 2026-02-11 14:20:29 -05:00
Paul Lorenz 45bf527441 Provide more error context to SDKs for terminator errors. Fixes #3524 2026-02-04 14:36:21 -05:00
Paul Lorenz da9ef76d19 Optimize imports 2026-01-28 15:01:35 -05:00
Paul Lorenz 2ffc6e1151 Update ziti version to 2.0 2026-01-28 12:34:10 -05:00
Paul Lorenz 284a08b1af Remove support for create terminator v1. Fixes #3516 2026-01-26 10:25:39 -05:00
Andrew Martinez 25dda7f994 adds error response for token update if API Session IDs do not match (#3440)
fixes #3437 adds error response for token update if API Session IDs do not match
- updates all token updates to add structured errors
- added an upfront API Session ID check on token updates in order to
  provide an error

adds fix for #3444 ensure api session types are checked to avoid nil ref

Identities may have mixed authentication modes if the credentials are
being shared. While not recommended, it is possible. This can cause
situations where API Session are not of a uniform type. During token
updates for OIDC, legacy API Sessions must be ignored.

- ignores legacy API Sessions during token update for a specific
  identity
- adds error messages for unlikely scenarios that indicate systemic
  failures
2025-12-16 09:26:46 -05:00
Paul Lorenz ffe6e16b76 Separate raft command submission limiting from in-flight limiting. Fix some create terminator error handling. Fixes #3318 2025-12-03 17:01:28 -05:00
Andrew Martinez d02d25c031 fix openziti/ziti#3354 sdk/env details not distributed (#3394)
* fix openziti/ziti#3354 sdk/env details not distributed
2025-11-26 10:54:10 -05:00
Paul Lorenz 8bedd3af1d Clean up connect events tests and remove global XG registry. Fixes #3345 2025-10-29 14:46:52 -04:00
Andrew Martinez bb5cc51a97 adds posture cache, instance, instance data, and event hooks (#3267)
* adds posture cache, instance, instance data, and event hooks

- reworks router security
- abstracts api session and service sessions
- use connection tracker instead of various maps
- doc
- adds service sessions as always JWTs
- addes token logging fields
- removes multiple connection tracking fields
- adds support for router specific posture data/respones
- adds support for router posture check evaluation
- defer dial/bind posture checking for legacy to controller
- fix double session event emitting
- fix session event missing id
2025-10-15 14:01:56 -04:00
Andrew Martinez 423dff3d92 fix #3236 SDK/ENV info is not saved on OIDC auth (#3137) 2025-08-05 09:51:00 -04:00
Paul Lorenz 10f4ad046d Fix race condition on terminator create 2025-07-23 15:00:25 -04:00
Paul Lorenz 5902631861 ids used by circuits and ingress/egress can conflict in an HA setup. Fixes #3124 2025-06-18 11:24:16 -04:00
Paul Lorenz 981df8269d Support xgress flow control from the SDK. Fixes #2986 2025-04-30 10:27:43 -04:00
Andrew Martinez 0e61edb2f9 fix #2984 identity/router enrollment do not return full chains (#2990)
* fix #2984 identity/router enrollment do not return full chains

- router extend via REST/ctrl now return full chains
- identity extend via REST now return full chains
- updates tests to allow for chain lengths
- modifies verification to look at the first cert in the chain (leaf)
- modifies edge routers to not start extension if new certs can't be
  saved
- adds network-jwt tests
2025-04-11 11:27:57 -04:00
Paul Lorenz e647d67325 Update to channel/v4 2025-04-02 15:28:59 -04:00
Andrew Martinez 78605a0955 fix #2865 remove fingerprint checks from controller (#2866)
* fix #2865 remove fingerprint checks from controller

- on their own fingerpint checking does not provide any additional
  security as the underlying certificate is not verified
- fingerprint based checking will not work with spiffeid generated
  certificates or x509 claims
- routers still support sending fingerprints for old controller support
2025-03-04 10:21:38 -05:00
Paul Lorenz ec57c80ff7 ER/T Hosting HA chaos test and fixes (#2806)
* Add ERT hosting chaos test. Also add ert terminator validation utility. Fixes #2288

* Rework ER/T terminator management based on SDK terminator management code

* Update deps

* Make sdk/ert-terminators into a constant
2025-02-25 17:02:21 -05:00
Paul Lorenz 3c9861ca3b Update deps and changelog. Update for SDK changes 2025-02-10 15:35:17 -05:00
Paul Lorenz d84292cfd9 Fix tests 2025-01-07 19:22:47 -05:00
Paul Lorenz 6517a7c22f Move routers to a subscription model where they subcribe to model data from one controller for a limited time. Fixes #2599 2025-01-07 19:21:53 -05:00
Paul Lorenz 32eddd61ca HA SDK terminators test. Fixes #2217. Fixes #2533 2024-11-12 18:45:08 -05:00
Paul Lorenz 16f0a858e6 Add controller connect events. Fixes #1835. Fixes #2234 2024-11-01 15:21:01 -04:00
Paul Lorenz cccf0c06af Update to channel/v3. Fixes #2390 2024-09-09 12:23:25 -04:00
Paul Lorenz 0945427320 Migrated edge router tunneler code to use the router data model. Fixes #2121 2024-07-26 17:06:01 -04:00
Paul Lorenz c3b43133d1 Merge fabric and controller model code. Fixes #2205 2024-07-09 16:11:01 -04:00
Paul Lorenz a0245e5cd3 Allow ER/T to intercept services w/out sessions when in HA mode. Fixes #1942 2024-04-16 15:11:00 -04:00
Andrew Martinez 36ced919ae Merge branch 'main' into ha-staging 2024-03-14 08:38:57 -04:00
Paul Lorenz fab81e4cd4 Add terminator chaos testing and fix issues found. Fixes #1794 Fixes #1369 2024-03-12 10:31:36 -04:00
Andrew Martinez 09542c7728 consolidated ha changes 2024-02-29 09:27:56 -05:00
Andrew Martinez 935c89bb7d fixes 1354 adds hostname/domain to env info 2024-02-22 14:32:00 -05:00
Paul Lorenz 744ae68bbd Fix controller crash. Fixes #1736 2024-02-08 21:55:48 -05:00
Paul Lorenz 3aace85a97 Refactor terminator state machine to be more like links state machine. Fixes #1733 Fixes #1734 2024-02-08 21:55:48 -05:00
Paul Lorenz 26c6263fa3 Fix terminator id race condition. Fixes #1685 2024-01-19 11:31:32 -05:00
Paul Lorenz a84369a6e9 Consolidate fabric and edge persistence code. Fixes #1555 2023-12-06 17:36:37 -05:00
Paul Lorenz db6554ee06 Implement idempotency for sdk terminators. Fixes #1446 2023-12-01 15:44:05 -05:00
Paul Lorenz b270d18faf Add additional logging. Remove unnecessary router lookups. May address #1460 2023-10-31 14:57:06 -04:00
Paul Lorenz 3523b34b2c Fix panic when router version info isn't set by ensuring it is set and making sure we get the right version. Fixes #1423 2023-10-11 13:55:50 -04:00