Commit Graph

488 Commits

Author SHA1 Message Date
Andrew Martinez 948735d86c fixes openziti/ziti#4094 accept first-party certs issued by a separat… (#4140)
* fixes openziti/ziti#4094 accept first-party certs issued by a separate edge signing CA

- adds FirstPartyX509CertValidation and ThirdPartyX509CertValidation usages and an
  intermediates field to the router data model public keys, deprecating
  ClientX509CertValidation
- publishes config CA bundle roots as first-party anchors with their intermediates and
  Ca store entries as third-party anchors; controller certs carry JWT validation only,
  since a controller identity is never a CA and anchors no client cert chains
- builds router first-party and client cert trust pools from the published usages,
  falling back to the deprecated usage against older controllers
- propagates the full signing cert chain between controllers via a new mesh
  SigningCertChainHeader and persists whole chains in controller records
- removes the orphaned InstantStrategy.AddPublicKey, dead since public key sync moved
  to controller list data
- gives each command dispatcher its own decoder registry so multiple in-process
  controllers no longer decode into the last-started controller's managers
- adds a three-controller in-process HA test harness with a split signing PKI,
  cluster formation and first-party cert integration tests
- trusts the edge signing CA when verifying router control channel certs
- adds a variadic additionalRoots parameter to VerifyLeafCertChain, applied to a
  clone of the caller's pool so an identity's live tls.Configs are unaffected
- passes the edge enrollment signing CA bundle as additional roots when admitting
  a router control channel connection, so a deployment whose signing CA sits
  outside the controller's own trust bundle no longer has every router refused
- leaves the fingerprint check bound to the verified leaf, so the wider anchor set
  changes which chains verify, not which routers are admitted
- covers the split-root case and the caller-pool guarantee in common/cert tests
2026-08-26 14:07:31 -04:00
Christopher Britton c639518724 Replicate admin MFA removal across HA cluster members (#4284)
* Run admin MFA removal as a single raft command so a lagging follower cannot serve a partial delete
2026-08-24 14:46:51 -04:00
Paul Lorenz e38970803c Merge commit from fork
fixes GHSA-whjr-3j94-gw3c constrain external jwt signer JWKS fetching
2026-08-20 15:27:56 -04:00
Andrew Martinez 9cf98a1dc9 fixes GHSA-4h58-w989-xgg4 enforce issuer and audience in ext-jwt token enrollment
- enforces issuer and audience claims inside TokenIssuerExtJwt.VerifyToken, which
  previously verified only the token signature and resolved the signing key by kid
- closes the ziti-token-issuer-id header enrollment path accepting a validly-signed
  token minted for a different audience or issuer that shares signing keys
- mirrors the runtime ext-jwt authentication and by-inspection enrollment paths so
  all paths validate claims consistently
- adds a controller/model unit test covering foreign-audience, foreign-issuer, and
  missing-audience rejection
2026-08-20 15:15:26 -04:00
Andrew Martinez f64b6879cf fixes GHSA-whjr-3j94-gw3c constrain external jwt signer JWKS fetching
- adds HardenedJwksResolver, a jwks.Resolver that fetches an external jwt signer's
  jwksEndpoint with an http/https-only scheme check, a total timeout and a redirect cap
- adds JwksFetchPolicy, gating a fetch on both the URL hostname and the address being
  connected to, applied to the first request and to every redirect hop
- hostname gate: deniedHostnames blocks, allowedHostnames is exclusive when set; entries
  are an exact hostname or a '*.suffix' wildcard that matches subdomains at any depth but
  never the suffix itself, normalized to lower case punycode without a trailing dot
- address gate: built-in blocked (metadata, link-local, link-local multicast,
  unspecified), then deniedIPs, then allowedIPs, then blockPrivateAddresses,
  first-match-wins with deny over allow
- keeps the gates independent, so neither can authorize what the other refuses; the
  address check runs in the dialer against the resolved address, so a hostname that
  resolves to a blocked address is refused
- adds the [edge.externalJwtSigners.jwksFetch] config section with compatible defaults:
  empty hostname lists, blockPrivateAddresses false, timeout 5s, maxRedirects 5
- takes IP lists as a flat address or a CIDR block, hostname lists as names only, and
  rejects an entry belonging to the other list at startup
- rejects a jwksEndpoint the policy refuses when an external jwt signer is created or
  updated, and logs an existing signer whose endpoint the configuration now refuses
  when the token issuer cache loads
- documents both gates, their deny-wins precedence, the accepted entry forms and the
  wildcard matching rules in etc/ctrl.with.edge.yml and CHANGELOG.md
2026-08-20 15:12:56 -04:00
Paul Lorenz 647c4daa1e Serialize router control-channel connect/disconnect. Fixes #4196
The controller decided which of two racing connections for a router was current
by comparing router instances, but loaded one per connect by evicting the router
cache and reading back through it. Two connects could both evict, and whichever
read second was handed the instance the first had just published. A shared
instance makes the two connections indistinguishable: the connect path cannot
reject the second into an occupied slot, and when either channel dies the
disconnect path finds itself current and tears down the registration the other is
still using. The surviving channel is never re-bound, so the router stays
connected at the transport layer while absent from the model, unable to recover.

Connect and disconnect were also unserialized, so a stale or superseded
disconnect could interleave with a live connection and take its links with it.

- serializes a router's connect and disconnect with a per-router striped lock
- keeps at most one connection per router: a connect into an occupied slot is
  rejected via an error from ConnectRouter, so the bind fails and NewChannel
  closes it without starting rx or registering it, and the occupant is displaced;
  the router redials into the freed slot
- displaces an occupant by closing its channel and also invoking the teardown
  directly, since a channel that is already closed never fires its close handler
  again; without this a dead but still registered connection holds the slot
  forever and every redial is rejected against a slot nothing can free
- refuses a connect whose control channel is already closed rather than
  registering it, so a connection no disconnect could ever remove is never
  published
- gives every connection its own router instance via RouterManager.NewCtrlChanRouter,
  read through readUncached so the cache neither supplies nor receives it, which
  is what makes comparing instances meaningful
- moves recording the channel and connect time out of the accept path, so a
  caller cannot attach the wrong channel or forget to attach one
- serializes link publication with that teardown on the same per-router stripe.
  Validating currency and then publishing without it is a check-then-act: a
  report can find the connection current and, by the time it reaches the link
  manager, the teardown has already snapshotted and cleared the router's links,
  so the link is recreated after everything that would have removed it. It is
  then absent from the router's own index while still in the link table with a
  disconnected source, and a reconnect reporting the same iteration can adopt
  that stale source instead of rebuilding the link
- guards the entire DisconnectRouter teardown by connection currency, all or
  nothing, and clears the connected flag and link index only when the
  registration was actually given up, with the flag cleared under the same shard
  lock as the map removal so the two cannot be observed disagreeing; the connected flag decides whether the
  controller accepts a router's link reports, so clearing it for the wrong
  connection silences a router that is up and reporting
- reduces MarkConnected to publishing the connection; the takeover-close moves
  into ConnectRouter's reject path
- makes the per-router unlock idempotent so callers can defer it as a leak-safety
  net and still unlock early before closing a channel outside the lock
- stops the replaced RouterSender in routerTxMap.Add so a takeover does not leak
  the old sender's goroutine when the broker's asynchronous RouterDisconnected
  loses the race to the redial's RouterConnected
- discards pending peer state changes for a router whose channel has closed,
  since sending on one fails immediately and the failed send is retried as soon
  as the event loop turns, spinning the loop and flooding the log
- queues the peer-state send-done event on every path, so a missing channel can
  no longer leave sendInProgress set and stall that router's updates permanently
- resolves a router's version from its connected instance when validating link
  conn info, since the version arrives in the hello and so is absent from an
  instance loaded from the database
- normalizes both endpoints to the connected instance in shortestPath, which is
  keyed and compared by pointer and so treated an endpoint held as any other
  instance of the same router as absent from the graph, reporting a router as
  unroutable from itself. That worked before only because the connect path
  published its instance into the router cache, so a cache read and the connected
  map returned the same object; nothing stated the requirement
- configures test logging once per package in TestMain, so a test no longer
  writes global logger state while a previous test's shutdown logging reads it
2026-08-13 23:54:40 -04:00
Paul Lorenz 22c0493d2d Stop linking the testing package via the model test context
- swaps testing.TB for require.TestingT in model.NewTestContext, so this
  build-included file no longer imports testing
- documents why the narrower interface is used, so it isn't reverted
- leaves hashicorp/raft as the only remaining importer of testing in the
  ziti binary's dependency graph
2026-08-13 15:37:25 -04:00
Paul Lorenz 48e4d4a224 Manage router link configuration via the controller. Fixes #4004
- adds a link ConfigHandler (router/link FactoryRegistry) that applies router.link.v1 config: Apply rebuilds the listener/dialer set wholesale, and established Xlinks survive because Listener.Close() only closes the accept loop
- translates local link: YAML into router.link.v1 JSON and pushes it through the managed-config registry at startup
- adds the UpdateLinkListeners ctrl message so the router republishes its listener set to the controller on change; the controller re-fans via the existing PeerStateChange path
- re-evaluates dialers on link group and listener changes via RescanForDialOpportunities

For #3743.
2026-08-13 15:37:24 -04:00
Paul Lorenz ea92d78f2d Merge pull request #4216 from openziti/router-link-costtags-removal
Remove unused link costTags support
2026-08-12 00:54:33 -04:00
Andrew Martinez 9abb347bb2 fixes openziti/ziti#4118 disambiguate overlapping ext-jwt-signer kids… (#4120)
* fixes openziti/ziti#4118 disambiguate overlapping ext-jwt-signer kids by issuer

- binds external JWT tokens to signers by exact issuer claim rather than by key ID, so signers drawing from a shared signing-key pool resolve deterministically
- binds controller-issued tokens by key ID first, preserving controller token resolution and preventing an external signer configured with a controller's issuer from capturing controller access tokens
- removes the external key-ID fallback so a token whose issuer matches no configured signer is not bound to an unrelated signer that happens to share its kid
- adds GetControllerIssuerByKid to the TokenIssuerCache interface and implementation
- skips disabled external signers in GetIssuerByKid so a disabled signer sharing a kid cannot poison resolution for an enabled one
- adds an integration test with two HTTPS JWKS providers sharing a key and kid, covering the enabled-collision and disabled-poison cases
- documents that controller issuers are keyed by controller id and that a controller issuer's key ID is the fingerprint of its TLS certificate
- documents that an external kid match is ambiguous because signers can share a signing-key pool, and that a definitive binding requires resolving by issuer claim
- clarifies the overlapping-kid test comment covering why issuer-claim binding is required when a disabled signer shares a kid
2026-08-06 10:55:46 -04:00
Andrew Martinez e1a431b6b4 fixes openziti/ziti#4063 enroll with empty roles when ext-jwt attribu… (#4064)
* fixes openziti/ziti#4063 enroll with empty roles when ext-jwt attribute claim is absent

- treats an absent enrollment attribute claim as no role attributes rather than
  rejecting the token; a failed jsonPointer.Get means the claim is not present,
  since pointer syntax is already validated at jsonpointer.New
- matches the existing unset-selector and empty-claim cases, which already
  enroll with an empty attribute set; a present-but-wrong-type claim still errors
- adds unit coverage for resolveStringSliceClaimProperty
- moves the two ext-jwt enrollment integration subtests that asserted the old
  failure behavior to success cases asserting an empty role-attribute set

* fixes openziti/ziti#4063 treat a null attribute claim as no role attributes

- treats an attribute claim sent as JSON null as no role attributes, logging a
  warning, matching the behavior of an absent claim
- logs at debug when the attribute claim selector does not resolve
- prints the offending value in the wrong-type error rather than the always-nil
  result of the failed array assertion
- corrects the godoc and inline comment: a pointer can fail to resolve through
  traversal as well as absence, and the malformed-pointer error is unreachable
- adds unit coverage for null and nested null claims, and asserts the wrong-type
  error names the value
- adds enrollment tests for a null attribute claim enrolling with no role
  attributes, and for a null name claim rejecting enrollment
2026-08-06 10:42:32 -04:00
Paul Lorenz a003f18792 Remove unused link costTags support
- removes the costTags option from router link listeners, which was parsed
  from config, advertised to the controller, and stored on the router model
  but never used for path selection or any other behavior
- drops GetLinkCostTags from the xlink.Listener interface and its transport
  implementation
- reserves the corresponding ctrl_pb.Listener.costTags and
  RouterLinks.RouterLink.linkCostTags protobuf fields and regenerates ctrl.pb.go
- documents the removal in the changelog
2026-08-03 18:03:26 -04:00
Paul Lorenz 4f7ff884f5 Merge pull request #4209 from openziti/fix-connection-tracker-deadlock
Fix lock order inversion in ConnectionTracker
2026-08-03 16:08:58 -04:00
Paul Lorenz d7076430c9 Make ER/T terminator create failures diagnosable. Fixes #4193
- reports edge router policy denials with an access-denied error naming the missing policy,
  replacing the session error reused on the sessionless ER/T and create-circuit-v3 paths
- adds EdgeRouterManager.GetEdgeRouterAccess, which reports which of the two required policy
  links (identity-to-edge-router, service-to-edge-router) is absent, and removes the boolean
  IsAccessToEdgeRouterAllowed it replaces
- logs the controller's rejection on the router at warn level, since it is recoverable and
  retried by the periodic scan; the router previously discarded the error code and message
- delays a new terminator's first create attempt by a fixed 2s so config applied in quick
  succession settles before the router asks, avoiding a 2-3 minute wait for the retry scan; the
  delay is a deliberate stopgap until edge router policy visibility lands in the router data model
- propagates the controller's error code and retry hint to SDK clients on the dial paths, which
  dropped the code and left every refusal classified as unknown
- adds the retry hint header to controller error replies, grouped with the other error-reply
  headers rather than the create-circuit-v3 request headers
- notes that the sync strategy headers alias the edge namespace's 1013-1015 ids and stay
  disjoint only by message content type
- tests the per-policy denial reporting, the error code carried with and without a retry hint,
  the controller-to-SDK error code mapping at both dial relay sites, and the terminator settle
  gate
- waits for terminator establishment in the tunneler dataflow tests instead of a fixed sleep, so
  they no longer race the settle delay
- restores the tproxy multiple-lanIf and multiple-resolver changelog entries with keep markers,
  which regeneration drops because their commits reference pull requests rather than issues
2026-08-03 15:43:30 -04:00
Paul Lorenz 0451ca7ebc Fix lock order inversion in ConnectionTracker. Fixes #4206
- releases the per-identity lock before acquiring the cmap shard lock when the scan
  loop reaps an entry, establishing a single shard-lock-then-value-lock order
- extracts the entry removal into removeIfEmpty, which decides based on the value
  currently in the map, so an identity that reconnects after the scan decided to
  remove it is left alone
- takes the per-identity lock in the removal check, fixing an unsynchronized read of
  the router map
- documents the lock ordering invariant on identityConnections
- adds tests covering concurrent scanning and connect/disconnect handling, entry
  reaping, and reconnection between the scan's decision and the removal
2026-07-31 14:08:08 -04:00
Andrew Martinez e7d23ef0ae fixes openziti/ziti#3990 push service and posture changes to subscrib… (#4057)
* fixes openziti/ziti#3990 push service and posture state to subscribed SDKs

- pushes indexed atomic ServiceChangeSet envelopes to subscribed SDK connections: a full snapshot on subscribe, incremental service changes per RDM scan pass, posture check definition changes as their own entries, and identity-resolved config bodies, all serialized so envelopes hit the wire in index order
- pushes per-connection PostureStateChange state (monotonic seq, resync on request) for posture pass/fail, including flips caused by definition edits that mutate no posture data
- registers pending RDM identity subscriptions for identities not yet synced to the router and sends an authoritative full sync plus full posture state when the identity arrives; an active push subscription pins the connection's RDM listener
- advertises service subscriptions and router data model support on the control-channel capability bitmask; the controller persists each router's capabilities mask and version on the EdgeRouter entity via raft and renders them on the edge APIs, so SDKs can select capable routers before connecting
- submits posture per router and corrects MFA posture semantics: pushed expiry is the earliest of timeout and pending wake/unlock grace deadlines, wake/unlock re-pass satisfies the re-prompt, api session tokens whose amr attests TOTP seed the MFA baseline from auth_time only (never iat), and token exchange carries the subject token's auth_time
- sends structured denials on dial and bind refusals: posture failures carry the failing check ids, no-policy denials are access denied, unknown services are invalid service, and session token failures are invalid session; the denial's cause no longer rides the wire as an unserializable error
- hard-closes accepted SDK connections on edge listener shutdown so clients observe a router going away immediately
- adds integration coverage: subscription snapshots and change delivery, poll and push reconciliation as capable routers come and go, posture state and definition-change push, router views over the public SDK API, typed dial errors, MFA baseline seeding, and OIDC token-exchange auth_time preservation
- removed RDM capability from SDK, router/controller only
2026-07-22 11:52:39 -04:00
Andrew Martinez 45b5046f52 fixes openziti/ziti#4071 unify router capabilities into one shared namespace (#4073)
- keys router capability bits off the sdk-golang RouterCapability enum
- adds a generic capabilities.Mask[T ~int] bitmask, centralizing capability
  set/check behind one value-to-bit translation
- adds capabilities.RouterCapability and ControllerCapability types, with
  RouterCapabilityMask/ControllerCapabilityMask aliases, so masks and checks are
  typed per namespace
- supports control-plane-only router capabilities as negative values that index
  down from the top of the mask, collision-free with the SDK's upward-numbered
  bits and invisible to the SDK and edge-api
- references the sdk-golang RouterCapability enum as the source of truth for
  shared router capability bits
- holds the router's advertised capability mask as an instance on the router env
  rather than a global, so in-process test routers do not share state
- advertises PostureChecks and BindSuccess as capability bits on both channels
  while still sending the legacy boolean edge headers for backwards compatibility
- routes GetCapabilities/IsCapable and the controller's Router.Capabilities field
  through the typed mask
- adds a provenance test that verifies, via go/packages, that every positive
  router capability is SDK-sourced, every negative is control-plane-only, and no
  two resolve to the same bit
- update sdk to v2.0.0-pre2
2026-07-02 17:02:34 -04:00
Paul Lorenz 187aa11f24 Own the metrics wire format in ziti. Fixes #4036
- adds a common/servermetrics package that owns the metrics MetricsMessage wire
  format and the reporting/usage subsystem (message builder, usage registry,
  interval and usage counters), wrapping the openziti/metrics Registry for
  metric collection
- moves the controllers metrics reporter into the router package and removes it
  from the shared metrics package, breaking a common -> router/env import cycle
- repoints controller and router consumers to common/servermetrics; base metric
  collection stays on openziti/metrics
- keeps the proto field numbers and the metrics content-type identical so the
  encoding is byte-compatible across the move, and uses a distinct proto package
  name so ziti's and the library's messages coexist without a global proto
  registry clash
- adds a round-trip test asserting wire compatibility with the library's
  MetricsMessage
- leaves openziti/metrics unchanged, so sdk-golang and the shared xgress data
  plane are unaffected
2026-06-29 22:37:25 -04:00
Paul Lorenz 2799c00191 Merge pull request #4042 from openziti/link-manager-striped-locking
Add striped per-link locking and cache-before-txn router reads
2026-06-29 13:12:36 -04:00
Paul Lorenz 863f6c90fe Merge pull request #4029 from openziti/sdk-golang-v2
Migrate to sdk-golang v2 module path
2026-06-29 13:06:36 -04:00
Paul Lorenz 474420d5c6 Merge pull request #4030 from openziti/issue-3929-revocation-max-token-duration
Use MaxTokenDuration for identity revocation lifetime
2026-06-29 13:05:56 -04:00
Andrew Martinez 1b180d14e8 fix #3933 add controller to enrollment response (#3947)
* fix #3933 add controller to enrollment response

- adds the cluster's controllers to ott, ottca, updb, and token enrollment
  responses with client and OIDC API addresses only
- synthesizes the running controller with its API addresses in non-HA mode
  so the list is never empty
- adds --not-before to ziti pki create for backdated test CAs
- replaces the test PKI with a SPIFFE-capable, ziti pki generated and managed
  one and rewires the config sets
- tests the controller list across ott/ottca/updb/token, non-HA, and raft

* fix missing wildcard cert from new PKI

* go mod tidy
2026-06-29 11:44:23 -04:00
Paul Lorenz 2ee646c1c4 Add striped per-link locking and cache-before-txn router reads. Fixes #4045
- adds a striped id locker in common/concurrency for sharded per-entity locking
- switches the link manager to striped per-link locking to reduce contention
- adds a cache-before-transaction read path to the router manager
- adds a RouterReportedLink regression test covering same-link serialization and stale/same-iteration no-op behavior
2026-06-26 11:51:00 -04:00
Paul Lorenz e168a50d2a Use MaxTokenDuration for identity revocation lifetime. For #3929
- uses Oidc.MaxTokenDuration() instead of RefreshTokenDuration so the
  revocation lifetime always covers the longest-lived token
2026-06-24 15:39:30 -04:00
Paul Lorenz 86092a8640 Migrate to the sdk-golang v2 module path. For #3884
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.

- Rewrites `github.com/openziti/sdk-golang/...` imports to
  `github.com/openziti/sdk-golang/v2/...` across the main and zititest
  modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
  which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
  `RemoteAddr().String()` in the loop4 traffic-test logging.

For openziti/sdk-golang#936.
2026-06-23 15:43:39 -04:00
Paul Lorenz 1c122af490 Rewrite channel/v4 imports to channel/v5. For #3983
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5

This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz f18ce1b580 Collapse fabric and edge services at the db level. Fixes #3934
- collapses the edge service boltz child store into the unified service store:
  EdgeService becomes a type alias for Service, and RoleAttributes, Configs, and
  EncryptionRequired become top-level Service fields
- adds an IsFabricOnly discriminator separating pure-fabric ("management")
  services from edge services
- adds a forward-only, irreversible migration that moves edge service data out of
  the edge child bucket, classifies pre-existing services (fabric -> IsFabricOnly
  with encryptionRequired=false; edge -> IsFabricOnly=false), deletes the old edge
  bucket and its index, and rebuilds the role-attributes index via CheckIntegrity
- guards every edge-facing path on the unified store against fabric-only services
  so they stay invisible to the edge API: by-id Read/ReadByName/Delete, list/query
  and the detail lister, ReadForIdentity, identity service-config overrides,
  service policy / SERP @id and #all/role denormalization, the association-list
  routes (terminators, policies, configs), and the router data-model sync
- rejects edge-surface updates of fabric-only services, and preserves edge fields
  on fabric-surface updates of edge services
- marks the fabric-only guards TEMPORARY(fabric-edge-collapse): they exist only to
  keep seldom-used management services hidden, and are removed when the fabric/edge
  distinction is erased
- adds in-CI migration tests (forward plus a round-trip asserting FK sub-bucket
  relocation and forward/reverse refcount preservation), durable fabric-only
  behavior tests, and store-level policy-exclusion tests
- adds one-time authentic-migration gate tooling under zititest/migration-test
  (create-model.sh, query, verify) and its operator runbook
2026-06-16 13:04:13 -04:00
Paul Lorenz ff619272ba Enforce api-session and identity revocations on the router. Fixes #3927
The router's CheckConnections reaper enforced only JWT expiry, so a revoked OIDC
api-session, or a disabled/deleted identity, kept its live circuits and hosted
terminators until the access token expired. The router now enforces the
RouterDataModel revocations directly, tightening access-loss propagation to the
reaper interval.

- adds a Type field to DataState_Revocation and the raft Revocation command
  proto, mirroring rest_model.RevocationTypeEnum (API_SESSION/IDENTITY/JTI) so the
  management API, OIDC producers, sync, and router enforcement share one
  vocabulary; the common.RevocationType* constants are compile-time bound to the
  enum to prevent drift
- adds an IssuedBefore cutoff so an identity revocation invalidates only sessions
  issued before it; a session re-authenticated after the cutoff survives the
  still-lingering revocation. Persists IssuedBefore on the db and model Revocation
  and carries it (plus the Type) through the single and batched raft marshalling
- adds RouterDataModel.IsApiSessionRevoked and IsIdentityRevoked and enforces both
  in CheckConnections, closing a revoked session's connections
- revokes a deleted or disabled identity's live OIDC sessions via an
  IdentityRevocationConstraint in the db package, run as a store pre-commit
  constraint so the revocation is written in the same transaction as the identity
  change and cannot be skipped (self-contained OIDC JWTs aren't otherwise
  reachable). NewIdentityManager installs it with the revocation type and lifetime
- has the OIDC end-session (TerminateSessionFromRequest) revoke the specific
  api-session named by the z_asid claim, with an identity-scoped fallback; sets
  IssuedBefore on the identity fallback and the management revocation API; adds
  RevocationManager.CreateOrReplace, routed through by both the OIDC paths and the
  management revocation API, so a repeat logout/termination/revocation refreshes
  the cutoff rather than colliding on the reused id. Expiry derives from the
  longest configured token duration via a shared common.MaxTokenDuration helper
- adds tests/revocation_enforcement_oidc_test.go covering api-session revocation
  (and a fresh session staying unaffected), identity disable and delete, and the
  identity cutoff (a post-cutoff session surviving the lingering revocation)
2026-06-13 01:36:12 -04:00
Andrew Martinez 9470c1f126 fixes #3952 reject invalid externalIdClaim and stop enrollment panic (#3953)
* fixes #3952 reject invalid externalIdClaim and stop enrollment panic

- moves the error check before the locator assignment in Ca.GetExternalId
  so an unsupported matcher/parser no longer nil-derefs and returns HTTP 500
- guards the claim index against negative and out-of-range values and errors
  when a matched claim resolves to an empty string, so an empty externalId is
  never silently accepted nor mapped to an identity
- adds validateExternalIdClaim, rejecting unsupported locations, unsupported
  matcher/parser combos, missing matcher/parser criteria, and negative indexes
  with HTTP 400 at CA create and update time
- applies the validation to both CA create and update, which previously had
  no externalIdClaim validation on the update path
- tests GetExternalId across the matcher/parser matrix for no-panic and
  correct error/value behavior
- tests CA create and update reject invalid externalIdClaim configurations
- tests that an externalIdClaim resolving to empty fails enrollment cleanly

* address pr review

- treats an externalIdClaim with no location as no claim in GetExternalId so an
  empty stored claim (e.g. the empty bucket older CLIs leave behind) falls back to
  fingerprint enrollment instead of erroring
- validates the merged result on CA update rather than the raw request: a partial
  patch overlays only its supplied subfields onto the stored claim, an empty {}
  object preserves the stored claim, and a full replace validates as-is
- skips externalIdClaim validation on update when no claim is supplied, fixing a
  spurious rejection of updates to CAs that have no claim
- reports the offending location/matcher/parser in the externalIdClaim validation
  error instead of the raw struct
- adds a --clear-external-id-claim flag to update ca and only sends the claim object
  when a claim flag changed, so existing updates preserve the claim and clearing is
  explicit
- tests externalIdClaim patch merge, preserve, and clear behavior, and that an
  unconfigured claim reads as no claim
2026-06-12 13:52:54 -04:00
Paul Lorenz cdc5d89885 Add configs field to routers. Fixes #3780 2026-06-11 13:45:58 -04:00
Paul Lorenz a582fe5d0f Add role-attribute usage queries. Fixes #1593
- adds management API endpoints listing role-attribute usage for identities,
  edge routers, services, and posture checks, reporting per-source counts
  (and optionally ids via withIds) across home-entity collections and the
  policies that reference each attribute
- adds RoleAttributeUsage model with QueryRoleAttributeUsage, mapping each
  RoleAttributeKind to its contributing sources, with all reads in a single
  transaction so counts stay consistent with the attribute list; returns an
  empty result (not a panic) for kinds with no attributes and an error for
  unknown kinds
- adds boltz.SetIndexValueTransform and AddSetIndexWithTransform, letting a
  SetIndex filter and rewrite symbol values without persisting a derived field
- adds derived role-attribute set indexes to the service policy, edge router
  policy, and service edge router policy stores, built from existing role
  fields via a role-attribute-only transform that excludes the #all wildcard
- adds a db migration that backfills the new indexes via each index's own
  CheckIntegrity(fix=true), scoped to the role-attribute indexes only, with a
  per-index summary log, stopping at the first error
- tests the new endpoints via the typed edge-api management client
2026-06-10 12:39:26 -04:00
Paul Lorenz 2bf9808df6 Add target field to config types. Fixes #3744 2026-05-27 13:17:06 -04:00
Andrew Martinez 949de99ee4 fixes #3809 support CSR submission during OIDC authentication (#3840)
* fixes #3809 support CSR submission during OIDC authentication

- accepts an optional CSR during OIDC login (all auth methods) and
  signs it into a session-bound certificate with a SPIFFE ID derived
  from the identity and API session
- returns the signed certificate PEM as a top-level "session_cert"
  field in the token endpoint JSON response (CodeExchange, RefreshToken,
  TokenExchange)
- adds cert-binding verification on RefreshToken and TokenExchange:
  if z_cfs is present the peer cert fingerprint must match (strict),
  otherwise falls back to SPIFFE ID verification
- supports cert rotation via csr_pem form parameter on refresh and
  token exchange; replaces the session cert fingerprint while
  preserving the authenticating cert fingerprint
- adds AuthCertFingerprints (z_acfs) claim to track permanent auth
  cert fingerprints separately from rotatable session cert fingerprints
- only the leaf certificate fingerprint is added to z_cfs and z_acfs,
  intermediates are never included
- invalid CSR returns 400 Bad Request in OIDC error format
- propagates updated CustomClaims (including CertFingerprints) from
  access token to renewed refresh token so rotated fingerprints are
  enforced on subsequent refreshes
- adds CertGenerated field to ApiSessionEvent
- advertises OIDC_AUTH_WITH_CSR controller capability when OIDC is
  enabled
- adds unit tests for verifyCertBinding (fingerprint, SPIFFE ID,
  edge cases) and CsrPem field parsing
- adds integration tests for initial CSR auth (updb, cert, ext-jwt),
  cert-binding on refresh/exchange, CSR rotation with cert auth,
  SPIFFE fallback and z_cfs transition, and CSR property forging
  resistance

* address pr concerns

* add z_cfs len tests on junk chain certs

* strip csr subject info, replace w/ santized values

* fix csr rotation rejection/paths during token exchange/refresh
2026-05-07 13:56:48 -04:00
Andrew Martinez 8297a817b7 fixes #3734 enforce client certificate proof-of-possession for OIDC sessions (#3805)
* fixes #3734 enforce client certificate proof-of-possession for OIDC sessions

- adds verifyCertProofOfPossession() in resolveOidcSession() to require
  TLS client cert matching a z_cfs fingerprint or SPIFFE ID when the
  OIDC token was issued with cert bindings
- adds z_cae (CertAllowExpired) claim from auth policy, propagated
  through token issuance and refresh
- adds TrustCache.VerifyClientCert() with tiered pool matching
  (first-party roots, trust anchors, third-party) and TTL cache
- adds WrapIdentityWithCertValidation() on the router to verify client
  certs at the TLS level against RDM PublicKeys
- adds IsFirstPartyCert() on the router to gate SPIFFE ID matching by
  checking whether the cert chains to the controller root CA
- adds VerifySpiffeId() in common/spiffehlp with SpiffeMatchApiSession,
  SpiffeMatchIdentity, and SpiffeMatchNone return types
- adds SPIFFE IDs to OTT and token enrollment certs (/identity/<id>)
- enforces cert expiry by match type: API session certs must be valid,
  fingerprint-matched certs respect z_cae, legacy sessions skip checks
- shallow-copies leaf certs before overriding time fields in all cert
  verification paths to avoid races on shared x509.Certificate pointers
- fixes controllerRootCache setting inited=true before the ctrl channel
  is available, which permanently cached the failure
2026-04-23 12:00:46 -04:00
Paul Lorenz 0042f6a345 Merge pull request #3810 from openziti/db-explorer-import-pr
Import the storage and ziti-db-explorer repos into the ziti repo
2026-04-21 09:54:19 -04:00
Paul Lorenz 9ac9a4f17a Filter service policies by type="Dial"/type="Bind" in queries. Fixes #3818
- registers the service policy type symbol as a string, so queries match the
  API's "Dial" and "Bind" names rather than the internal int32 ids
- adds a symbol mapper that converts the stored int32 to its PolicyType name
  at query eval time
- updates the posture-checks lookup in EdgeServiceManager, the only direct
  caller that went through GetSymbol, to read the mapped string form
- adds a store test covering type = "Dial" and type = "Bind" filtering
- notes the breaking removal of the undocumented type = 1/type = 2 form in
  the 2.0 deprecation cleanup list
2026-04-20 15:25:37 -04:00
Paul Lorenz 4df6ae564d Update package paths for newly imported storage and ziti-db-explorer packages 2026-04-16 09:18:55 -04:00
Andrew Martinez 6515e3615c fixes #3680 add revocation management API, CLI, and enforcement (#3789)
* fixes #3680 add revocation management API, CLI, and enforcement

- adds Management API endpoints for revocations (POST, GET, LIST) with
  type-aware validation (JTI/API_SESSION require UUID, IDENTITY requires
  existing identity)
- adds CLI commands: ziti edge create revocation identity|api-session|jti
- adds revocation checks to resolveOidcSession in security_ctx.go so the
  REST API returns 401 for revoked OIDC tokens. Previously only
  ValidateAccessToken (router ctrl channel path) checked revocations,
  so revoked tokens still received 200 OK from the management and client
  HTTP APIs
- adds api-session revocation check to ValidateAccessToken, which only
  checked JTI and identity revocations
- adds Type field to Revocation model, store, and protobuf message
- adds integration tests covering CRUD, input validation, and token
  enforcement for all three revocation types
- use release edge-api@v0.28.1
2026-04-14 15:53:03 -04:00
Paul Lorenz 6a3292e804 Set empty path when reserving circuit, to avoid panics. Fixes #3777 2026-04-07 09:25:55 -04:00
Paul Lorenz 7b4ae12c05 Add CreateCircuitV3 for RDM-authorized circuit creation. Fixes #3721
- adds CreateCircuitV3 message type and handler for routers that have
  already authorized dials locally via RDM, bypassing service session
  tokens in favor of identity ID, service ID, and pre-assigned circuit ID
- renames CreateCircuitRequest/Response to CreateCircuitV2Request/V2Response
  for clarity now that V3 exists
- adds CircuitManager.Reserve to atomically claim circuit IDs before routing,
  preventing collisions on pre-assigned IDs
- extends CreateCircuitParams with GetCircuitId so V3 can supply a
  pre-assigned circuit ID (falls back to UUID generation when empty)
- fixes IsDialableByIdentity which was incorrectly calling IsBindableByIdentity
- extracts V2 handler into its own file create_circuit_v2.go
- adds CreateCircuitV3RequestType/ResponseType (20222/20223) to edge_ctrl protobuf
- registers V3 handler in controller server
2026-03-27 14:41:28 -04:00
Paul Lorenz 62e29169c2 Coalesce OIDC JWT revocations to reduce controller write pressure. Fixes #3681
- adds DeleteRevocationsBatchCommand so expired-revocation cleanup goes
  through raft as a single log entry per batch
- adds CreateRevocationsBatchCommand for batched revocation creation
  through raft
- moves refresh-token revocations from synchronous inline creation to a
  background batcher that flushes on a configurable interval, removing
  the database and raft as a bottleneck on token refreshes
- skips revocation creation for tokens expiring within a configurable
  threshold (revocationMinTokenLifetime), since they become invalid on
  their own
- validates that revocationMinTokenLifetime is less than 50% of the
  configured refresh token lifetime
- makes the revocation enforcer frequency configurable and restricts it
  to run only on the raft leader
- adds tests for multi-batch delete, batched create with router RDM
  propagation, and skip-threshold behavior
- adds new configuration tunables under edge.oidc: revocationBucketInterval,
  revocationMinTokenLifetime, revocationBucketMaxSize, revocationMaxQueued,
  revocationEnforcerFrequency
2026-03-19 08:35:00 -04:00
Andrew Martinez dbeb5c9f46 fixes openziti/ziti#3673 purge expired revocations, fix revocation bugs (#3679)
- fixes RevokeToken double-write: adds return after JWTID-keyed revocation
    save, preventing fallthrough write with raw JWT string as unreachable key
  - fixes TerminateSession key mismatch: stores revocation by identityId alone,
    matching the Subject-based lookup in ValidateAccessToken
  - fixes RevocationDelete sync action: passes DataState_Delete instead of
    DataState_Create so routers evict the entry from their data model
  - adds RevocationManager.DeleteExpired: batch-deletes expired revocations in
    batches of 500 until none remain
  - adds RevocationEnforcer: periodic policy runner (every 1 minute) that calls
    DeleteExpired and records metrics

before test fixes
2026-03-11 20:33:38 -04:00
Paul Lorenz e1638173cd Fixes for SDK terminator management. Add support for ziti sdk inspection. Fixes #3609
- removes legacy v1 terminator code path; all terminators now use v2 flow
- refactors edgeTerminator.close() to decouple SDK notification from control plane notification
- adds pending SDK close notification queue with retry when channel is busy
- adds post-create inspect mechanism that verifies SDK still holds the bind after terminator creation
- queues second post-create inspect when establishment takes >30s to catch SDK timeout races
- detects and discards stale reordered binds on the same connection by comparing connIds
- re-establishes replacement terminators when a delete/create race is detected
- eliminates IsEntityPresent pre-filter in removeTerminatorsHandler to prevent raft ordering races
- fixes ValidateTerminators to query identities from the correct manager with the correct filter field
- adds postCreate flag to ValidateTerminatorsV2Request so routers skip redundant SDK inspect
- returns retry-later (nil result) from router validation when inspect is temporarily unavailable
- blocks SyncAllSubscribers until completion and guards RouterDataModel replacement with in-progress flag
- fixes InheritLocalData to enable service access tracking for all subscribed identities
- adds Services.Has check in GetServiceAccessPolicies to prevent false policy grants
- validates policy-to-identity associations in ValidateServicePolicies
- adds `ziti agent tunnel dump-sdk` command for SDK context inspection via IPC agent
- adds `ziti fabric inspect sdk` command to query SDK context through routers
- fixes --expected-per-host CLI flag binding in validate terminators command
- changes bind-access-lost retry hint from NotRetriable to RetryStartOver
- moves trace route response and xgress close handling off channel handler goroutine
- fixes listTerminators test helper to URL-encode filter parameter
- improves sdk-hosting-test validation resilience with login and query retries
- adds terminator_create_flow.md documenting the full lifecycle across SDK, router, and controller
- adds detailed logging for data model sync, service access tracking, and subscriber change detection
2026-03-11 13:10:28 -04:00
Paul Lorenz 6b869cea9d Add support for ctrlChanListener on router to the model. Fixes #3635 2026-03-07 00:00:51 -05:00
Paul Lorenz d1300de040 Add preferredLeader flag to raft configuration. Fixes #3600 2026-03-05 17:24:43 -05:00
Paul Lorenz 7a4967f634 Transit router disabled flag not passed through raft command structure Fixes #3617 2026-02-26 19:03:08 -05:00
Andrew Martinez 7cae68be7a Merge pull request #3604 from openziti/fix.openziti.ziti.3333.updb.auth.attempts.fix
fix.openziti.ziti.3333.updb.auth.attempts.fix
2026-02-26 10:53:20 -05:00
Andrew Martinez ce2ad6dbd0 fixes two issues
- fixes race condition on valid/invalid mixture of auths resulting in
  clear/disabled
- removes 65s wait on lock removal test
2026-02-25 09:34:59 -05:00
Andrew Martinez f0c2592916 fixes openziti/ziti#3333 based on the work from Jan Starkl <jan.starkl@tttech-digital.com>
- thanks to Jan Starkl <jan.starkl@tttech-digital.com> for the issue
- thanks to Jan Starkl <jan.starkl@tttech-digital.com> for the fix
- rebases work to top of main
- reworks tests to fix new testing model, helpers, callers
- unfurls nested tests
2026-02-25 09:13:07 -05:00
Paul Lorenz ab51fa8217 Fix the control channel header conflicts with channel headers. Use single strategy for sharing capabilities with bit mask. 2026-02-24 15:50:05 -05:00