* fixesopenziti/ziti#4321 compare process hashes and fingerprints case-insensitively
- normalizes configured hashes and signer fingerprints to lowercase unseparated hex in the
PROCESS and PROCESS_MULTI stores, on load as well as on save, so values stored before this
change read back normalized without a migration
- replaces the PROCESS store's lowercase-only pass, PROCESS_MULTI stored its values verbatim
- adds store tests for both types covering configured values and values written straight to bolt
- updates the process multi patch round-trip test, the management API now returns normalized
values rather than the submitted strings
- removes the trailing newline from the signer fingerprint literal in the process and
process-multi OIDC posture tests
- adds Test_ServerConnCloseWritePropagationXgressTerminator, binding the host with SDK flow
control and forcing the dial onto ConnectV1 so the router bridges the two modes
- asserts the client's read ends at io.EOF while its own write side stays open, the case a
default dial does not reach because both ends then run xgress
- adds an oidc-auth-test fablab model that exercises OIDC authentication end to end, with an event-forwarder component, an oidc-test-client, a ziti-prox-c component, and OIDC event/gossip/traffic validations
- moves zitirest out of the public API into separate test-only shims: an integration-test shim (tests/restclient.go) and a fablab shim (zititest/zitirest), and repoints upgrade-test at the fablab shim
- adds region-isolation network partitioning and packet capture on all interfaces to the chaos toolkit
- queues oidc-test-client results reporting on a single sender goroutine with a bounded write deadline, so a stalled results circuit can't block the traffic loops
- counts results events dropped when that queue fills and reports them to the collector as errors, so a reporting outage fails validation instead of looking like a clean run
- bounds the event forwarder's event and keepalive writes, and checks for shutdown on every send attempt, so an unreadable destination can't park the forward loop holding its lock
- logs read errors on collector connections instead of ending collection for that client silently
- bounds the oidc-auth-test debug server's header and body reads
- matches ziti-prox-c process matching to the version-agnostic binary name, so a version change still finds the running process
- points upgrade-test's per-service terminator count at the shared validations helper
- adds a fablab design summary doc
Test_ServerCloseListenerPropagation never received from errC, so it was the only test
in the file that did not wait for its hosting goroutine. Reading the last reply only
proves the data reached the wire: the SDK writes with SendAndWaitForWire, so the write
is still waiting to be told its buffer went out. The test body returning ran the
deferred conn, listener, context and context-teardown closes under it, which resolved
that wait as "channel closed" and failed a write whose data the client already had.
- waits on errC before returning, matching the other six tests in the file
The window is between the data hitting the wire and the sender being notified, so it
does not reproduce on an idle machine; it showed up on a loaded CI runner as a single
failure with no client-side error, which is what identified the ordering.
- withholds an enrollment's token and jwt from a non-admin caller when the owning
identity is an admin, in the identity detail mapper and the enrollment mapper
- scopes the identity enrollments subresource so a non-admin does not see
enrollments of admin identities, matching the top level /enrollments list
- threads the request context into MapIdentityToRestModel and
MapEnrollmentToRestModel so the mappers can see the caller's permissions
- adds an integration test over the identity detail, identity list, and identity
enrollments routes for every enrollment method
- adds the advisory to the 2.1.0 changelog
- removes a stray assignment that overwrote isCertExtendable with a literal true on every OIDC certificate authentication
- reports isCertExtendable as false over OIDC for cert authenticators not issued by the network, matching legacy cert auth
- adds coverage asserting the z_ice claim and current-api-session both report a 3rd party CA certificate as not extendable
* fixesopenziti/ziti#4094 accept first-party certs issued by a separate edge signing CA
- adds FirstPartyX509CertValidation and ThirdPartyX509CertValidation usages and an
intermediates field to the router data model public keys, deprecating
ClientX509CertValidation
- publishes config CA bundle roots as first-party anchors with their intermediates and
Ca store entries as third-party anchors; controller certs carry JWT validation only,
since a controller identity is never a CA and anchors no client cert chains
- builds router first-party and client cert trust pools from the published usages,
falling back to the deprecated usage against older controllers
- propagates the full signing cert chain between controllers via a new mesh
SigningCertChainHeader and persists whole chains in controller records
- removes the orphaned InstantStrategy.AddPublicKey, dead since public key sync moved
to controller list data
- gives each command dispatcher its own decoder registry so multiple in-process
controllers no longer decode into the last-started controller's managers
- adds a three-controller in-process HA test harness with a split signing PKI,
cluster formation and first-party cert integration tests
- trusts the edge signing CA when verifying router control channel certs
- adds a variadic additionalRoots parameter to VerifyLeafCertChain, applied to a
clone of the caller's pool so an identity's live tls.Configs are unaffected
- passes the edge enrollment signing CA bundle as additional roots when admitting
a router control channel connection, so a deployment whose signing CA sits
outside the controller's own trust bundle no longer has every router refused
- leaves the fingerprint check bound to the verified leaf, so the wider anchor set
changes which chains verify, not which routers are admitted
- covers the split-root case and the caller-pool guarantee in common/cert tests
- serves the overlapping-kid JWKS providers over loopback HTTP instead of HTTPS: the hardened
JWKS resolver (GHSA-whjr) builds its own http.Transport and validates TLS against the system
roots, so a CA injected into http.DefaultTransport is no longer honored and the HTTPS JWKS
never resolved, leaving primary auth unable to succeed
- removes the now-dead default-transport TLS trust setup and the unused newTlsJwksServer helper
- transport is orthogonal to the kid/issuer disambiguation the test exercises
- requires an API session token on CreateCircuitV3 requests and validates it,
covering signature, audience, token type, and revocation by token id,
identity, and api session
- takes the dialing identity from the validated token claims rather than the
router-supplied identity id, and rejects a request whose asserted identity
does not match the token subject
- adds the api session id to the log context, matching the V1 and V2 paths
- adds tests for a missing token, an invalid token, and a token belonging to a
different identity than the one asserted
- notes the advisory in the 2.1.0 release notes
- adds HardenedJwksResolver, a jwks.Resolver that fetches an external jwt signer's
jwksEndpoint with an http/https-only scheme check, a total timeout and a redirect cap
- adds JwksFetchPolicy, gating a fetch on both the URL hostname and the address being
connected to, applied to the first request and to every redirect hop
- hostname gate: deniedHostnames blocks, allowedHostnames is exclusive when set; entries
are an exact hostname or a '*.suffix' wildcard that matches subdomains at any depth but
never the suffix itself, normalized to lower case punycode without a trailing dot
- address gate: built-in blocked (metadata, link-local, link-local multicast,
unspecified), then deniedIPs, then allowedIPs, then blockPrivateAddresses,
first-match-wins with deny over allow
- keeps the gates independent, so neither can authorize what the other refuses; the
address check runs in the dialer against the resolved address, so a hostname that
resolves to a blocked address is refused
- adds the [edge.externalJwtSigners.jwksFetch] config section with compatible defaults:
empty hostname lists, blockPrivateAddresses false, timeout 5s, maxRedirects 5
- takes IP lists as a flat address or a CIDR block, hostname lists as names only, and
rejects an entry belonging to the other list at startup
- rejects a jwksEndpoint the policy refuses when an external jwt signer is created or
updated, and logs an existing signer whose endpoint the configuration now refuses
when the token issuer cache loads
- documents both gates, their deny-wins precedence, the accepted entry forms and the
wildcard matching rules in etc/ctrl.with.edge.yml and CHANGELOG.md
- makes the sdk/env info poll a RequireIdentitySdkInfoUpdated method on
ManagementHelperClient, alongside the client's other helpers, rather than a
package-level function in the test file
- drops the context and helper parameters, since the helper already carries the
test context it needs to assert through
- waits until a whole second after an identity revocation before authenticating the
session that is expected to survive it, so the fresh token's issued-at is
unambiguously later than the cutoff
- a token's iat carries whole seconds while the revocation cutoff keeps full
precision, so enforcement cannot distinguish a session issued just before the
cutoff from one issued just after it within the same second, and rejects both;
the test passed only when the reaper's latency happened to push the
re-authentication into the next second
- returns immediately when the clock has already moved on, which is the usual case,
so the wait costs nothing outside the race
- replaces the read-once identity lookups in Test_Authenticate_OIDC_Auth with a helper that polls
the management API until the identity reports the expected sdk info
- removes the fixed one second sleep that previously stood in for that wait on the updb path
- adds a link ConfigHandler (router/link FactoryRegistry) that applies router.link.v1 config: Apply rebuilds the listener/dialer set wholesale, and established Xlinks survive because Listener.Close() only closes the accept loop
- translates local link: YAML into router.link.v1 JSON and pushes it through the managed-config registry at startup
- adds the UpdateLinkListeners ctrl message so the router republishes its listener set to the controller on change; the controller re-fans via the existing PeerStateChange path
- re-evaluates dialers on link group and listener changes via RescanForDialOpportunities
For #3743.
- passes appEnv into the legacy v1 handler's request context, so token
validation no longer makes a nil-interface call and panics the controller
- adds a regression test driving the v1 handler over the control channel with
JWT-prefixed, opaque, and empty tokens, asserting an error reply comes back
and the controller keeps serving
- rejects a remove or update request whose terminator is owned by a different
router, on the fabric control channel handlers
- drops ids the requesting router does not own from batch removals, keeping
absent ids so a delete racing a not-yet-applied create is still ordered after
it
- adds unit tests for the ownership filter and for the single-terminator check
- adds an end-to-end test that drives the fabric control channel from an
enrolled router against a second router's terminator, covering single remove,
batch remove, and re-weight, plus a control that a router can still remove its
own
* fixesopenziti/ziti#4118 disambiguate overlapping ext-jwt-signer kids by issuer
- binds external JWT tokens to signers by exact issuer claim rather than by key ID, so signers drawing from a shared signing-key pool resolve deterministically
- binds controller-issued tokens by key ID first, preserving controller token resolution and preventing an external signer configured with a controller's issuer from capturing controller access tokens
- removes the external key-ID fallback so a token whose issuer matches no configured signer is not bound to an unrelated signer that happens to share its kid
- adds GetControllerIssuerByKid to the TokenIssuerCache interface and implementation
- skips disabled external signers in GetIssuerByKid so a disabled signer sharing a kid cannot poison resolution for an enabled one
- adds an integration test with two HTTPS JWKS providers sharing a key and kid, covering the enabled-collision and disabled-poison cases
- documents that controller issuers are keyed by controller id and that a controller issuer's key ID is the fingerprint of its TLS certificate
- documents that an external kid match is ambiguous because signers can share a signing-key pool, and that a definitive binding requires resolving by issuer claim
- clarifies the overlapping-kid test comment covering why issuer-claim binding is required when a disabled signer shares a kid
* fixesopenziti/ziti#4063 enroll with empty roles when ext-jwt attribute claim is absent
- treats an absent enrollment attribute claim as no role attributes rather than
rejecting the token; a failed jsonPointer.Get means the claim is not present,
since pointer syntax is already validated at jsonpointer.New
- matches the existing unset-selector and empty-claim cases, which already
enroll with an empty attribute set; a present-but-wrong-type claim still errors
- adds unit coverage for resolveStringSliceClaimProperty
- moves the two ext-jwt enrollment integration subtests that asserted the old
failure behavior to success cases asserting an empty role-attribute set
* fixesopenziti/ziti#4063 treat a null attribute claim as no role attributes
- treats an attribute claim sent as JSON null as no role attributes, logging a
warning, matching the behavior of an absent claim
- logs at debug when the attribute claim selector does not resolve
- prints the offending value in the wrong-type error rather than the always-nil
result of the failed array assertion
- corrects the godoc and inline comment: a pointer can fail to resolve through
traversal as well as absence, and the malformed-pointer error is unreachable
- adds unit coverage for null and nested null claims, and asserts the wrong-type
error names the value
- adds enrollment tests for a null attribute claim enrolling with no role
attributes, and for a null name claim rejecting enrollment
- preserves tags when creating external JWT signers, transit routers and authenticators, all of
which previously discarded them while update and patch honored them
- switches the external JWT signer update and patch mappers to TagsOrDefault, matching the
pattern used by the other entity mappers
- adds a unit test for TagsOrDefault covering nil, nil sub-tags, empty and populated inputs
- tests that create persists tags for external JWT signers, transit routers and authenticators
- tests that patching an external JWT signer leaves existing tags untouched, and that patching
tags replaces them
- adds a changelog entry
- reports edge router policy denials with an access-denied error naming the missing policy,
replacing the session error reused on the sessionless ER/T and create-circuit-v3 paths
- adds EdgeRouterManager.GetEdgeRouterAccess, which reports which of the two required policy
links (identity-to-edge-router, service-to-edge-router) is absent, and removes the boolean
IsAccessToEdgeRouterAllowed it replaces
- logs the controller's rejection on the router at warn level, since it is recoverable and
retried by the periodic scan; the router previously discarded the error code and message
- delays a new terminator's first create attempt by a fixed 2s so config applied in quick
succession settles before the router asks, avoiding a 2-3 minute wait for the retry scan; the
delay is a deliberate stopgap until edge router policy visibility lands in the router data model
- propagates the controller's error code and retry hint to SDK clients on the dial paths, which
dropped the code and left every refusal classified as unknown
- adds the retry hint header to controller error replies, grouped with the other error-reply
headers rather than the create-circuit-v3 request headers
- notes that the sync strategy headers alias the edge namespace's 1013-1015 ids and stay
disjoint only by message content type
- tests the per-policy denial reporting, the error code carried with and without a retry hint,
the controller-to-SDK error code mapping at both dial relay sites, and the terminator settle
gate
- waits for terminator establishment in the tunneler dataflow tests instead of a fixed sleep, so
they no longer race the settle delay
- restores the tproxy multiple-lanIf and multiple-resolver changelog entries with keep markers,
which regeneration drops because their commits reference pull requests rather than issues
- asserts the current api session certificate list excludes another identity's certificates
- asserts scoping is per api session, so a second api session of the same identity is excluded
- adds CreateCurrentApiSessionCertificate and ListCurrentApiSessionCertificates client helpers
- caps buffered HTTP request bodies at 1 MiB across the client, management, fabric management, and OIDC web APIs
- rejects oversized bodies with 413 REQUEST_ENTITY_TOO_LARGE before authentication
- surfaces request body read errors instead of ignoring them
- returns after writing the request context error response in the client API handler
- adds integration coverage for oversized bodies with and without Content-Length
* fixesopenziti/ziti#3990 push service and posture state to subscribed SDKs
- pushes indexed atomic ServiceChangeSet envelopes to subscribed SDK connections: a full snapshot on subscribe, incremental service changes per RDM scan pass, posture check definition changes as their own entries, and identity-resolved config bodies, all serialized so envelopes hit the wire in index order
- pushes per-connection PostureStateChange state (monotonic seq, resync on request) for posture pass/fail, including flips caused by definition edits that mutate no posture data
- registers pending RDM identity subscriptions for identities not yet synced to the router and sends an authoritative full sync plus full posture state when the identity arrives; an active push subscription pins the connection's RDM listener
- advertises service subscriptions and router data model support on the control-channel capability bitmask; the controller persists each router's capabilities mask and version on the EdgeRouter entity via raft and renders them on the edge APIs, so SDKs can select capable routers before connecting
- submits posture per router and corrects MFA posture semantics: pushed expiry is the earliest of timeout and pending wake/unlock grace deadlines, wake/unlock re-pass satisfies the re-prompt, api session tokens whose amr attests TOTP seed the MFA baseline from auth_time only (never iat), and token exchange carries the subject token's auth_time
- sends structured denials on dial and bind refusals: posture failures carry the failing check ids, no-policy denials are access denied, unknown services are invalid service, and session token failures are invalid session; the denial's cause no longer rides the wire as an unserializable error
- hard-closes accepted SDK connections on edge listener shutdown so clients observe a router going away immediately
- adds integration coverage: subscription snapshots and change delivery, poll and push reconciliation as capable routers come and go, posture state and definition-change push, router views over the public SDK API, typed dial errors, MFA baseline seeding, and OIDC token-exchange auth_time preservation
- removed RDM capability from SDK, router/controller only
- reorders the edge client create-session flow so Session.Create resolves the
session id (via dedup or persist) before CreateJwt signs the token, so the
JWT jti always matches the stored session record
- prevents a permanent invalid-session retry storm for legacy-authenticated
clients when a create dedups to an existing session, seen after expanding a
single-node controller into a multi-node HA cluster
- preserves delete-as-revoke semantics: the session remains a durable record,
the controller still loads it via Session.Read
- adds an apitests regression test asserting the returned token id matches the
persisted session id, including on a deduped create
- Widens the TOTP-vs-access-token iat tolerance from 2ms to one second. Both
claims are truncated to one-second precision (jwt's default TimePrecision), so
their delta is either 0 or a whole number of seconds; the 2ms bound only passed
when both issuances fell in the same wall-clock second and flaked on a
second-boundary straddle
- keys router capability bits off the sdk-golang RouterCapability enum
- adds a generic capabilities.Mask[T ~int] bitmask, centralizing capability
set/check behind one value-to-bit translation
- adds capabilities.RouterCapability and ControllerCapability types, with
RouterCapabilityMask/ControllerCapabilityMask aliases, so masks and checks are
typed per namespace
- supports control-plane-only router capabilities as negative values that index
down from the top of the mask, collision-free with the SDK's upward-numbered
bits and invisible to the SDK and edge-api
- references the sdk-golang RouterCapability enum as the source of truth for
shared router capability bits
- holds the router's advertised capability mask as an instance on the router env
rather than a global, so in-process test routers do not share state
- advertises PostureChecks and BindSuccess as capability bits on both channels
while still sending the legacy boolean edge headers for backwards compatibility
- routes GetCapabilities/IsCapable and the controller's Router.Capabilities field
through the typed mask
- adds a provenance test that verifies, via go/packages, that every positive
router capability is SDK-sourced, every negative is control-plane-only, and no
two resolve to the same bit
- update sdk to v2.0.0-pre2
- adds a common/servermetrics package that owns the metrics MetricsMessage wire
format and the reporting/usage subsystem (message builder, usage registry,
interval and usage counters), wrapping the openziti/metrics Registry for
metric collection
- moves the controllers metrics reporter into the router package and removes it
from the shared metrics package, breaking a common -> router/env import cycle
- repoints controller and router consumers to common/servermetrics; base metric
collection stays on openziti/metrics
- keeps the proto field numbers and the metrics content-type identical so the
encoding is byte-compatible across the move, and uses a distinct proto package
name so ziti's and the library's messages coexist without a global proto
registry clash
- adds a round-trip test asserting wire compatibility with the library's
MetricsMessage
- leaves openziti/metrics unchanged, so sdk-golang and the shared xgress data
plane are unaffected
- repoints the controller identity and enrollment signingCert to the ziti
pki generated testdata/pki tree, replacing the removed
testdata/ca/intermediate key/cert paths
- sets trustDomain to ziti.test to match the regenerated PKI
- adds Test_EnrollmentToken_ToToken subtest asserting a valid bearer token
presented to /authenticate before enrolling is rejected and creates no identity
- confirms the auth path never provisions an identity as a side effect;
provisioning happens only via token enrollment
- adds ListIdentitiesByFilter helper to ManagementHelperClient
* fix#3933 add controller to enrollment response
- adds the cluster's controllers to ott, ottca, updb, and token enrollment
responses with client and OIDC API addresses only
- synthesizes the running controller with its API addresses in non-HA mode
so the list is never empty
- adds --not-before to ziti pki create for backdated test CAs
- replaces the test PKI with a SPIFFE-capable, ziti pki generated and managed
one and rewires the config sets
- tests the controller list across ott/ottca/updb/token, non-HA, and raft
* fix missing wildcard cert from new PKI
* go mod tidy
- updates openziti/xweb/v3 for actual fix
- adds Test_OidcListener_BindFailure_DoesNotPanic and the
oidc-listener-bind-failure config set, asserting the controller keeps
the other web servers running when an edge-oidc listener cannot bind
Implements the router-side Connect-V2 sessionless dial path. Dials are
authorized locally via the RouterDataModel instead of a controller-issued
service session token; circuit creation flows through the existing
`CreateCircuitV3` controller endpoint (#3721). Builds on the sdk-golang
v2 migration.
- Adds `processConnectV2` on `edgeClientConn`: resolves the service by id
or name via the RouterDataModel, checks dial access, and dispatches to
the controller via `sendCreateCircuitV3Msg`. Supports both
`xgEdgeForwarder` (SDK xgress) and `nonXgConnectHandler` flow-control
modes, selected by the SDK's `UseXgressToSdkHeader`.
- Makes `CircuitId` optional in `DecodeCreateCircuitV3Request`. The V2
router path does not pre-assign a circuit ID; the controller generates
it as V1/V2 already do. Without this the decoder rejected the empty
header and every V2 dial hung until timeout. Adds a regression test.
- Splits `checkAccess` to close a posture-check bypass on the V2 path.
The old single `checkAccess` short-circuited to nil for non-OIDC
sessions (V1 ran posture at the controller during `CreateSession`); V2
has no such step, so posture would have been skipped. `checkAccess` now
always runs the RDM `HasAccess` (policy + posture) check;
`checkAccessIfOidc` keeps the OIDC-only gate for the V1 and bind paths.
- Sends the V2 `state_connected` on the default (data) sender rather than
the control sender. On multi-underlay channels the two senders are
independently ordered, so an early terminator payload on the data
sender could beat `state_connected` to the SDK and be dropped (channel/v5
has no message-priority API).
- Updates `xgEdgeForwarder.lastRx` on every forward path, including the
fast `timeout == 0` `TrySend` branch used for normal payload dispatch.
The old code only updated it on the `timeout > 0` path, so active V2
circuits looked idle and could be unrouted prematurely.
- Adds `state.ConnState.ServiceId`, populated by the connect handlers from
the service session token (V1) or the request header (V2). The
non-xgress V2 path previously left this empty, so `handleDialAccessLost`
could not identify and close V2 non-xgress circuits when dial access was
revoked.
- Skips conns with no `ServiceSessionToken` in `RemoveLegacyServiceSession`;
a sessionless V2 conn's token is nil and the cleanup loop previously
dereferenced it unconditionally, which would panic the router.
- Advertises Connect-V2 via the `RouterCapabilityConnectV2` bit in the
listener hello so SDKs can detect V2 support.
- Wires `ContentTypeConnectV2` and `ContentTypeXgControl` handlers in
`Acceptor.BindChannel`, and adds `handleXgControl` for SDK-side xgress
control messages, preserving `ControlUserVal` so trace-route responses
correlate back to the initiator's `SendForReply` waiter.
- Adds `RouterDataModel.serviceNameIndex` for O(1) name->id lookup in the
V2 dial path, maintained with rename safety at the `HandleServiceEvent`
mutation points.
- Adds `tests/connect_v2_test.go` covering end-to-end V2 dataflow and the
V1 fallback (`ForceConnectV1`), asserting the dial path via the SDK
`DialEvent`.
- Propagates a V2 initiator's graceful half-close to legacy hosts via
`edgeXgressConn.FlowFromFabricToXgressClosed`, which emits an edge FIN
when the fabric->app half of the circuit closes. The SDK signals
half-close to its router xgress peer with the native xgress EOF flag;
without translating that to an edge FIN, a legacy host reading to EOF
stalled until teardown.
- Records the dialing identity id as the circuit `ClientId` for
sessionless V2 dials, since there is no dial session to key on; updates
`Test_OidcEvents` to match.
- Adds `tests/connect_v2_teardown_test.go` covering client- and
host-initiated close propagation on both the V2 and forced-V1 paths.
- Polls for the asynchronous conn close in the SDK posture-check tests
(`awaitClientConnClosed`): revocation tears the circuit down out of
band, so checking `IsClosed` immediately after the first read error was
racy.
- Temporarily pins sdk-golang/v2 to the openziti/sdk-golang#959 commit,
which carries the matching xgress conn-close-on-teardown fix the V2
posture tests depend on; to be repointed at the next sdk-golang
pre-release before merge.
For openziti/sdk-golang#936.
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.
- Rewrites `github.com/openziti/sdk-golang/...` imports to
`github.com/openziti/sdk-golang/v2/...` across the main and zititest
modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
`RemoteAddr().String()` in the loop4 traffic-test logging.
For openziti/sdk-golang#936.
Andrew Martinez