Commit Graph

96 Commits

Author SHA1 Message Date
Andrew Martinez f64b6879cf fixes GHSA-whjr-3j94-gw3c constrain external jwt signer JWKS fetching
- adds HardenedJwksResolver, a jwks.Resolver that fetches an external jwt signer's
  jwksEndpoint with an http/https-only scheme check, a total timeout and a redirect cap
- adds JwksFetchPolicy, gating a fetch on both the URL hostname and the address being
  connected to, applied to the first request and to every redirect hop
- hostname gate: deniedHostnames blocks, allowedHostnames is exclusive when set; entries
  are an exact hostname or a '*.suffix' wildcard that matches subdomains at any depth but
  never the suffix itself, normalized to lower case punycode without a trailing dot
- address gate: built-in blocked (metadata, link-local, link-local multicast,
  unspecified), then deniedIPs, then allowedIPs, then blockPrivateAddresses,
  first-match-wins with deny over allow
- keeps the gates independent, so neither can authorize what the other refuses; the
  address check runs in the dialer against the resolved address, so a hostname that
  resolves to a blocked address is refused
- adds the [edge.externalJwtSigners.jwksFetch] config section with compatible defaults:
  empty hostname lists, blockPrivateAddresses false, timeout 5s, maxRedirects 5
- takes IP lists as a flat address or a CIDR block, hostname lists as names only, and
  rejects an entry belonging to the other list at startup
- rejects a jwksEndpoint the policy refuses when an external jwt signer is created or
  updated, and logs an existing signer whose endpoint the configuration now refuses
  when the token issuer cache loads
- documents both gates, their deny-wins precedence, the accepted entry forms and the
  wildcard matching rules in etc/ctrl.with.edge.yml and CHANGELOG.md
2026-08-20 15:12:56 -04:00
Paul Lorenz ecbdb92ecb Document and test what the router connect churn limit now guards
routerConnectChurnLimit predates the per-router connect lock. It was added
alongside the ability for a new control channel to take over from an established
one, as the guard on how often that may happen, and it was also the only thing
keeping two connections for one router out of the connected map.

That second job is gone: at most one connection per router is now enforced under
the per-router lock, where the decision is atomic. The check in the accept path
runs against the connected map with no lock held, so it can only refuse a
connection early that would be refused there anyway.

Its first job remains, and is now the only thing doing it. ConnectRouter always
displaces an occupant it does not recognise, so without the limit a spurious
first-connection hello would tear down a healthy control channel and make the
router redial. Nothing said so, and the field carried no godoc at all.

- documents on the option what it protects, that it is churn policy rather than
  the uniqueness guarantee, and that zero always allows takeover
- extracts the decision so it can be tested without standing up a network, and
  tests it: protected when just established, protected part way through the
  window, displaceable once past it, and never protected at zero

The struct's field alignment shifts because a comment ends gofmt's alignment
group; that part of the diff is whitespace only.

Behaviour is unchanged. Worth noting for readers of the option: past the window,
the established connection is now displaced and the connect refused, so the
router redials into the freed slot, where previously the arriving connection took
over directly. Same end state, one extra round trip, and nothing unvetted is
registered on the way.
2026-08-13 23:54:41 -04:00
Paul Lorenz 86092a8640 Migrate to the sdk-golang v2 module path. For #3884
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.

- Rewrites `github.com/openziti/sdk-golang/...` imports to
  `github.com/openziti/sdk-golang/v2/...` across the main and zititest
  modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
  which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
  `RemoteAddr().String()` in the loop4 traffic-test logging.

For openziti/sdk-golang#936.
2026-06-23 15:43:39 -04:00
Paul Lorenz 1c122af490 Rewrite channel/v4 imports to channel/v5. For #3983
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5

This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
2026-06-18 12:51:02 -04:00
Paul Lorenz ff619272ba Enforce api-session and identity revocations on the router. Fixes #3927
The router's CheckConnections reaper enforced only JWT expiry, so a revoked OIDC
api-session, or a disabled/deleted identity, kept its live circuits and hosted
terminators until the access token expired. The router now enforces the
RouterDataModel revocations directly, tightening access-loss propagation to the
reaper interval.

- adds a Type field to DataState_Revocation and the raft Revocation command
  proto, mirroring rest_model.RevocationTypeEnum (API_SESSION/IDENTITY/JTI) so the
  management API, OIDC producers, sync, and router enforcement share one
  vocabulary; the common.RevocationType* constants are compile-time bound to the
  enum to prevent drift
- adds an IssuedBefore cutoff so an identity revocation invalidates only sessions
  issued before it; a session re-authenticated after the cutoff survives the
  still-lingering revocation. Persists IssuedBefore on the db and model Revocation
  and carries it (plus the Type) through the single and batched raft marshalling
- adds RouterDataModel.IsApiSessionRevoked and IsIdentityRevoked and enforces both
  in CheckConnections, closing a revoked session's connections
- revokes a deleted or disabled identity's live OIDC sessions via an
  IdentityRevocationConstraint in the db package, run as a store pre-commit
  constraint so the revocation is written in the same transaction as the identity
  change and cannot be skipped (self-contained OIDC JWTs aren't otherwise
  reachable). NewIdentityManager installs it with the revocation type and lifetime
- has the OIDC end-session (TerminateSessionFromRequest) revoke the specific
  api-session named by the z_asid claim, with an identity-scoped fallback; sets
  IssuedBefore on the identity fallback and the management revocation API; adds
  RevocationManager.CreateOrReplace, routed through by both the OIDC paths and the
  management revocation API, so a repeat logout/termination/revocation refreshes
  the cutoff rather than colliding on the reused id. Expiry derives from the
  longest configured token duration via a shared common.MaxTokenDuration helper
- adds tests/revocation_enforcement_oidc_test.go covering api-session revocation
  (and a fresh session staying unaffected), identity disable and delete, and the
  identity cutoff (a post-cutoff session surviving the lingering revocation)
2026-06-13 01:36:12 -04:00
Paul Lorenz ba809e4d26 Keep controller mesh fully connected, as much as possible. Fixes #3684 2026-05-27 10:21:34 -04:00
Paul Lorenz 975a23e5f6 Add a recover mechanism for when a controller cluster can't form a quorum. Fixes #3849
- adds 'ziti ops cluster recover <controller-config>', an offline CLI
  that opens a stopped controller's data directory, forces the raft
  configuration to a single local node via raft.RecoverCluster, and
  aligns the FSM-tracked member list in ctrl-ha.db so stale peers don't
  leak through IsPeerMember or CtrlAddresses on restart
- removes the previous in-process recovery path: the cluster.recover
  config flag and the corresponding RaftConfig.Recover field are gone,
  along with the os.Exit branch in Controller.Init that consumed them
- adds BoltDbFsm.OverwriteServers and GetCachedServers so offline
  tooling can update and inspect the FSM-side server list without a
  live raft instance; OverwriteServers runs before raft.RecoverCluster
  so the snapshot it produces captures the corrected configuration
- updates Broker.AcceptClusterEvent to call DeleteRemovedPeers on every
  ClusterLeadershipGained, making the controllers entity table
  self-healing for any membership change a non-leader missed (offline
  recovery, or a 'cluster remove' applied while another node was leader)
- wires the new subcommand into both V1 and V2 CLI roots so it's
  reachable regardless of ZITI_CLI_LAYOUT
- switches filesystem path joins in the raft package and the recover
  command from path.Join to filepath.Join
- tests bootstrap a two-node configuration, run recoverDataDir, then
  verify the post-recovery snapshot, the FSM-cached server list, and
  Fsm.GetCurrentState (after starting a real raft instance) all report
  the survivor only
2026-05-27 09:23:01 -04:00
Paul Lorenz 82ac2e8060 Rework connect event handling to ensure serialized-per-router handling 2026-04-24 13:22:02 -04:00
Paul Lorenz 4df6ae564d Update package paths for newly imported storage and ziti-db-explorer packages 2026-04-16 09:18:55 -04:00
Paul Lorenz 0766e79526 Fix minor config parsing bugs
- uses already-parsed AdvertiseAddress for cert validation instead of
  re-reading from the raw map, which panicked when the key was absent
- replaces errors.Wrapf(err, ...) with fmt.Errorf where err is nil, so
  routerDataModel.listenerBufferSize validation errors are returned
  instead of silently swallowed
- fixes pfxlog.Logger().Warn() call using %w verb, which is not
  interpolated outside fmt.Errorf
2026-04-03 17:42:18 -04:00
Paul Lorenz afa3121469 Fix TLS rate limiter timeout check reading from wrong scope. Fixes #3756
- checks for the timeout key in the rateLimiter submap instead of the
  parent tls cfgmap, so a user-specified timeout is no longer silently
  overwritten by the default
2026-04-03 15:22:52 -04:00
Paul Lorenz 438c507a63 Fix commandHandler config read from wrong scope. Fixes #3755
- reads commandHandler from the cluster submap instead of the top-level
  cfgmap, matching all surrounding config reads in the cluster block
2026-04-03 15:22:49 -04:00
Paul Lorenz 33e9115182 Fix swapped HasPrefix args in SPIFFE trust domain check. Fixes #3753
- swaps arguments to strings.HasPrefix so it checks whether the trust
  domain starts with "spiffe://" rather than whether "spiffe://" starts
  with the trust domain
- fixes both the trustDomain and additionalTrustDomains code paths
2026-04-03 15:22:41 -04:00
Paul Lorenz f2db0e147d Fix connect events handler goroutine leak. Fixes #3746
- replaces the per-handler goroutine in connectEventsHandler with a shared,
  bounded goroutine pool on AppEnv
- the old design spawned a processEvents goroutine per router connection that
  only exited on application shutdown, leaking a goroutine on every reconnect
- adds ConnectEventsConfig to the controller config with pool tuning options
  (queueSize, minWorkers, maxWorkers, idleTime)
- defaults: queue 16, 0-16 workers, 30s idle timeout
2026-04-02 23:44:43 -04:00
Paul Lorenz 62e29169c2 Coalesce OIDC JWT revocations to reduce controller write pressure. Fixes #3681
- adds DeleteRevocationsBatchCommand so expired-revocation cleanup goes
  through raft as a single log entry per batch
- adds CreateRevocationsBatchCommand for batched revocation creation
  through raft
- moves refresh-token revocations from synchronous inline creation to a
  background batcher that flushes on a configurable interval, removing
  the database and raft as a bottleneck on token refreshes
- skips revocation creation for tokens expiring within a configurable
  threshold (revocationMinTokenLifetime), since they become invalid on
  their own
- validates that revocationMinTokenLifetime is less than 50% of the
  configured refresh token lifetime
- makes the revocation enforcer frequency configurable and restricts it
  to run only on the raft leader
- adds tests for multi-batch delete, batched create with router RDM
  propagation, and skip-threshold behavior
- adds new configuration tunables under edge.oidc: revocationBucketInterval,
  revocationMinTokenLifetime, revocationBucketMaxSize, revocationMaxQueued,
  revocationEnforcerFrequency
2026-03-19 08:35:00 -04:00
Paul Lorenz 9513fc902d Fix changelog errors. Fix config parsing bug 2026-03-10 13:08:22 -04:00
Paul Lorenz 6b869cea9d Add support for ctrlChanListener on router to the model. Fixes #3635 2026-03-07 00:00:51 -05:00
Paul Lorenz 46cd641c71 Use success rate for adaptive rate limiter window sizing. Fixes #3636
- replaces queue-position-based window adjustment with an exponentially
  decaying success rate histogram to drive grow/shrink decisions
- introduces AdaptiveRateLimitTrackerConfig with configurable
  successThreshold, increaseFactor, decreaseFactor,
  increaseCheckInterval, and decreaseCheckInterval
- grows window by increaseFactor when success rate exceeds threshold,
  shrinks by decreaseFactor when it falls below
- renames LoadAdaptiveRateLimiterConfig to a Load method on the config
- adds currentSize and currentWindow to Success/Backoff/Failed debug logs
- updates controller TLS handshake, raft, and router ctrl rate limiters
  to use the new AdaptiveRateLimitTrackerConfig
2026-03-05 18:35:57 -05:00
Paul Lorenz d1300de040 Add preferredLeader flag to raft configuration. Fixes #3600 2026-03-05 17:24:43 -05:00
Andrew Martinez a9dd2fc8f8 fix.openziti.ziti.3597.OIDC.by.default (#3605)
* fixes #3597, enable OIDC by default

- adds ability to have different sets of environment configs for tests
- adds ConfigSet struct as a configuration device for integration tests
  - allows entire environments to be defined as needed
  - original ats config set at "default-ats"
  - tests that do not specify a config set, use "default-ats" as before
- standardizes configuration location, naming, etc.
- adds README.md for the above
- updates testContext to now be config set aware
- add config value to disable, update tests, changelog
- add defense against cached version data for tests
2026-03-04 10:03:14 -05:00
Paul Lorenz 15ae909c07 Remove option to disable the router data model in the controller. Fixes #3541 2026-02-04 16:23:28 -05:00
Paul Lorenz 35f4d5c363 Enforce policy on the router for oidc sessions, by closing open circuits and terminators when service access is lost. Fixes #3509 2026-02-04 12:19:56 -05:00
Paul Lorenz da9ef76d19 Optimize imports 2026-01-28 15:01:35 -05:00
Paul Lorenz 2ffc6e1151 Update ziti version to 2.0 2026-01-28 12:34:10 -05:00
Paul Lorenz 08359a9e2b Remove legacy link management code from the controller. Fixes #3512 2026-01-22 15:26:42 -05:00
Paul Lorenz 7b94e65282 Revert tls handshake rate limited to disabled for now 2026-01-16 14:42:39 -05:00
Paul Lorenz ee69f8edde Update deps and changelog. Change background queue drops on full config to false by default 2026-01-07 14:13:58 -05:00
Paul Lorenz d9f08ed1e4 Create generic validation workflow 2025-12-29 17:13:41 -05:00
Paul Lorenz f3ad7b65ae Have identity env updates and system authenticator updates happen in the background. Fixes #3477 2025-12-29 12:08:16 -05:00
Paul Lorenz d4f927b79b Enable tls handshake rate limiter by default. Update default values. Fixes #3473. Break out old changelogs. 2025-12-23 12:55:52 -05:00
eugene 24ea80181d Fix OIDC configuration parsing for token durations 2025-12-09 12:57:05 -05:00
Paul Lorenz ffe6e16b76 Separate raft command submission limiting from in-flight limiting. Fix some create terminator error handling. Fixes #3318 2025-12-03 17:01:28 -05:00
Paul Lorenz d37c721c83 Optimize router data model subscription code. Add additional events. Fixes #3359 2025-12-02 11:37:38 -05:00
Paul Lorenz a70218a11c Allow disabling posture checks. Fixes #3063 2025-05-29 12:40:00 -04:00
Andrew Martinez 1743e9c50e fixes #2614 adds token expiration configuration for OIDC (#3022) 2025-05-27 08:29:56 -04:00
Andrew Martinez 0ca892060f fix #2996 add ability to flag cert authenticators for extension/key roll (#3025)
* fix #2996 add ability to flag cert authenticators for extension/key roll
2025-05-01 17:41:00 -04:00
Paul Lorenz 981df8269d Support xgress flow control from the SDK. Fixes #2986 2025-04-30 10:27:43 -04:00
Paul Lorenz e647d67325 Update to channel/v4 2025-04-02 15:28:59 -04:00
Paul Lorenz e28c32122d Update Raft Configuration Defaults. Fixes #2829 2025-03-10 09:48:56 -04:00
Paul Lorenz c5f67f6a1a Cluster operations should return 503 if there's no leader. Also add 503s for other operations where appropriate. Fixes #2738 2025-02-14 11:56:10 -05:00
dovholuknf 60ca94e9d5 attempts to probe advertise address on startup to ensure the SANS is correct (#2723)
* fixes #2723 - check router and controller advertised addresses compared to configured certs on start

* fixes #2723 - don't verify cert/addresses when loadIdentity=false

* fixes #2723 - don't check ctrl endpoint using router certs

* fixes #2723 - advertiese -> advertise

* fixes #2722 - cleanup/rework to use xweb

* fixes #2722 - actually remove unused code

* placating linter
2025-02-06 16:21:09 -05:00
Paul Lorenz 6e7c5bf4ac Cluster config/cli tweaks
* Remove initialMembers config. Fixes #2745
* Consolidate agent cluster commands. Fixes #2746
* Start removing HA doc, now that it's being added to doc site
2025-02-05 16:33:05 -05:00
Paul Lorenz 6a01db9e4c Rename raft config to cluster. Fixes #2731 2025-02-04 13:31:15 -05:00
Paul Lorenz 03870760c3 Ensure restores and migrations work properly with RDM. Add restore from db for HA Clusters. Fixes #2549. Fixes #2649. Fixes #2707 2025-01-31 13:57:43 -05:00
Paul Lorenz e77ebaf4f8 Add warning message when cluster is leaderless longer than configured threshold. Fixes #2548 2025-01-09 18:07:51 -05:00
Paul Lorenz 5429ce5064 Separate leader updates from controller cluster member updates 2025-01-09 16:18:34 -05:00
Paul Lorenz d84292cfd9 Fix tests 2025-01-07 19:22:47 -05:00
Paul Lorenz 37a80da89b Add RDM config, including a ctrl side disable flag. Fixes #2596 2025-01-07 19:22:47 -05:00
Paul Lorenz 32eddd61ca HA SDK terminators test. Fixes #2217. Fixes #2533 2024-11-12 18:45:08 -05:00
Paul Lorenz 74386da333 Update deps and changelog. Test fixes and spiffe id check 2024-11-01 15:21:24 -04:00