- adds a common/servermetrics package that owns the metrics MetricsMessage wire
format and the reporting/usage subsystem (message builder, usage registry,
interval and usage counters), wrapping the openziti/metrics Registry for
metric collection
- moves the controllers metrics reporter into the router package and removes it
from the shared metrics package, breaking a common -> router/env import cycle
- repoints controller and router consumers to common/servermetrics; base metric
collection stays on openziti/metrics
- keeps the proto field numbers and the metrics content-type identical so the
encoding is byte-compatible across the move, and uses a distinct proto package
name so ziti's and the library's messages coexist without a global proto
registry clash
- adds a round-trip test asserting wire compatibility with the library's
MetricsMessage
- leaves openziti/metrics unchanged, so sdk-golang and the shared xgress data
plane are unaffected
* fix#3933 add controller to enrollment response
- adds the cluster's controllers to ott, ottca, updb, and token enrollment
responses with client and OIDC API addresses only
- synthesizes the running controller with its API addresses in non-HA mode
so the list is never empty
- adds --not-before to ziti pki create for backdated test CAs
- replaces the test PKI with a SPIFFE-capable, ziti pki generated and managed
one and rewires the config sets
- tests the controller list across ott/ottca/updb/token, non-HA, and raft
* fix missing wildcard cert from new PKI
* go mod tidy
- adds a striped id locker in common/concurrency for sharded per-entity locking
- switches the link manager to striped per-link locking to reduce contention
- adds a cache-before-transaction read path to the router manager
- adds a RouterReportedLink regression test covering same-link serialization and stale/same-iteration no-op behavior
Bumps the sdk-golang dependency from v1 to the v2 module
(`github.com/openziti/sdk-golang/v2` at v2.0.0-pre1) and updates all
import paths. This is a no-behavior-change precursor that isolates the
dependency migration from the Connect-V2 feature work in #3884.
- Rewrites `github.com/openziti/sdk-golang/...` imports to
`github.com/openziti/sdk-golang/v2/...` across the main and zititest
modules.
- Pins both modules to `github.com/openziti/sdk-golang/v2 v2.0.0-pre1`.
- Adapts `edgeXgressConn.AcceptMessage` to the v2 `MsgSink` signature,
which now takes an `edge.SdkChannel` argument.
- Replaces the removed `edge.Conn.GetRouterId()` with
`RemoteAddr().String()` in the loop4 traffic-test logging.
For openziti/sdk-golang#936.
- moves the channel dependency to channel/v5 v5.0.10 and sdk-golang to v1.9.0 in the root and zititest modules
- mechanically rewrites every channel/v4 import path to channel/v5
This is the import-path-only step; the API-level changes the switch requires land in the following commit. This commit does not build on its own.
- collapses the edge service boltz child store into the unified service store:
EdgeService becomes a type alias for Service, and RoleAttributes, Configs, and
EncryptionRequired become top-level Service fields
- adds an IsFabricOnly discriminator separating pure-fabric ("management")
services from edge services
- adds a forward-only, irreversible migration that moves edge service data out of
the edge child bucket, classifies pre-existing services (fabric -> IsFabricOnly
with encryptionRequired=false; edge -> IsFabricOnly=false), deletes the old edge
bucket and its index, and rebuilds the role-attributes index via CheckIntegrity
- guards every edge-facing path on the unified store against fabric-only services
so they stay invisible to the edge API: by-id Read/ReadByName/Delete, list/query
and the detail lister, ReadForIdentity, identity service-config overrides,
service policy / SERP @id and #all/role denormalization, the association-list
routes (terminators, policies, configs), and the router data-model sync
- rejects edge-surface updates of fabric-only services, and preserves edge fields
on fabric-surface updates of edge services
- marks the fabric-only guards TEMPORARY(fabric-edge-collapse): they exist only to
keep seldom-used management services hidden, and are removed when the fabric/edge
distinction is erased
- adds in-CI migration tests (forward plus a round-trip asserting FK sub-bucket
relocation and forward/reverse refcount preservation), durable fabric-only
behavior tests, and store-level policy-exclusion tests
- adds one-time authentic-migration gate tooling under zititest/migration-test
(create-model.sh, query, verify) and its operator runbook
The router's CheckConnections reaper enforced only JWT expiry, so a revoked OIDC
api-session, or a disabled/deleted identity, kept its live circuits and hosted
terminators until the access token expired. The router now enforces the
RouterDataModel revocations directly, tightening access-loss propagation to the
reaper interval.
- adds a Type field to DataState_Revocation and the raft Revocation command
proto, mirroring rest_model.RevocationTypeEnum (API_SESSION/IDENTITY/JTI) so the
management API, OIDC producers, sync, and router enforcement share one
vocabulary; the common.RevocationType* constants are compile-time bound to the
enum to prevent drift
- adds an IssuedBefore cutoff so an identity revocation invalidates only sessions
issued before it; a session re-authenticated after the cutoff survives the
still-lingering revocation. Persists IssuedBefore on the db and model Revocation
and carries it (plus the Type) through the single and batched raft marshalling
- adds RouterDataModel.IsApiSessionRevoked and IsIdentityRevoked and enforces both
in CheckConnections, closing a revoked session's connections
- revokes a deleted or disabled identity's live OIDC sessions via an
IdentityRevocationConstraint in the db package, run as a store pre-commit
constraint so the revocation is written in the same transaction as the identity
change and cannot be skipped (self-contained OIDC JWTs aren't otherwise
reachable). NewIdentityManager installs it with the revocation type and lifetime
- has the OIDC end-session (TerminateSessionFromRequest) revoke the specific
api-session named by the z_asid claim, with an identity-scoped fallback; sets
IssuedBefore on the identity fallback and the management revocation API; adds
RevocationManager.CreateOrReplace, routed through by both the OIDC paths and the
management revocation API, so a repeat logout/termination/revocation refreshes
the cutoff rather than colliding on the reused id. Expiry derives from the
longest configured token duration via a shared common.MaxTokenDuration helper
- adds tests/revocation_enforcement_oidc_test.go covering api-session revocation
(and a fresh session staying unaffected), identity disable and delete, and the
identity cutoff (a post-cutoff session surviving the lingering revocation)
* fixes#3952 reject invalid externalIdClaim and stop enrollment panic
- moves the error check before the locator assignment in Ca.GetExternalId
so an unsupported matcher/parser no longer nil-derefs and returns HTTP 500
- guards the claim index against negative and out-of-range values and errors
when a matched claim resolves to an empty string, so an empty externalId is
never silently accepted nor mapped to an identity
- adds validateExternalIdClaim, rejecting unsupported locations, unsupported
matcher/parser combos, missing matcher/parser criteria, and negative indexes
with HTTP 400 at CA create and update time
- applies the validation to both CA create and update, which previously had
no externalIdClaim validation on the update path
- tests GetExternalId across the matcher/parser matrix for no-panic and
correct error/value behavior
- tests CA create and update reject invalid externalIdClaim configurations
- tests that an externalIdClaim resolving to empty fails enrollment cleanly
* address pr review
- treats an externalIdClaim with no location as no claim in GetExternalId so an
empty stored claim (e.g. the empty bucket older CLIs leave behind) falls back to
fingerprint enrollment instead of erroring
- validates the merged result on CA update rather than the raw request: a partial
patch overlays only its supplied subfields onto the stored claim, an empty {}
object preserves the stored claim, and a full replace validates as-is
- skips externalIdClaim validation on update when no claim is supplied, fixing a
spurious rejection of updates to CAs that have no claim
- reports the offending location/matcher/parser in the externalIdClaim validation
error instead of the raw struct
- adds a --clear-external-id-claim flag to update ca and only sends the claim object
when a claim flag changed, so existing updates preserve the claim and clearing is
explicit
- tests externalIdClaim patch merge, preserve, and clear behavior, and that an
unconfigured claim reads as no claim
- adds management API endpoints listing role-attribute usage for identities,
edge routers, services, and posture checks, reporting per-source counts
(and optionally ids via withIds) across home-entity collections and the
policies that reference each attribute
- adds RoleAttributeUsage model with QueryRoleAttributeUsage, mapping each
RoleAttributeKind to its contributing sources, with all reads in a single
transaction so counts stay consistent with the attribute list; returns an
empty result (not a panic) for kinds with no attributes and an error for
unknown kinds
- adds boltz.SetIndexValueTransform and AddSetIndexWithTransform, letting a
SetIndex filter and rewrite symbol values without persisting a derived field
- adds derived role-attribute set indexes to the service policy, edge router
policy, and service edge router policy stores, built from existing role
fields via a role-attribute-only transform that excludes the #all wildcard
- adds a db migration that backfills the new indexes via each index's own
CheckIntegrity(fix=true), scoped to the role-attribute indexes only, with a
per-index summary log, stopping at the first error
- tests the new endpoints via the typed edge-api management client
* fixes#3809 support CSR submission during OIDC authentication
- accepts an optional CSR during OIDC login (all auth methods) and
signs it into a session-bound certificate with a SPIFFE ID derived
from the identity and API session
- returns the signed certificate PEM as a top-level "session_cert"
field in the token endpoint JSON response (CodeExchange, RefreshToken,
TokenExchange)
- adds cert-binding verification on RefreshToken and TokenExchange:
if z_cfs is present the peer cert fingerprint must match (strict),
otherwise falls back to SPIFFE ID verification
- supports cert rotation via csr_pem form parameter on refresh and
token exchange; replaces the session cert fingerprint while
preserving the authenticating cert fingerprint
- adds AuthCertFingerprints (z_acfs) claim to track permanent auth
cert fingerprints separately from rotatable session cert fingerprints
- only the leaf certificate fingerprint is added to z_cfs and z_acfs,
intermediates are never included
- invalid CSR returns 400 Bad Request in OIDC error format
- propagates updated CustomClaims (including CertFingerprints) from
access token to renewed refresh token so rotated fingerprints are
enforced on subsequent refreshes
- adds CertGenerated field to ApiSessionEvent
- advertises OIDC_AUTH_WITH_CSR controller capability when OIDC is
enabled
- adds unit tests for verifyCertBinding (fingerprint, SPIFFE ID,
edge cases) and CsrPem field parsing
- adds integration tests for initial CSR auth (updb, cert, ext-jwt),
cert-binding on refresh/exchange, CSR rotation with cert auth,
SPIFFE fallback and z_cfs transition, and CSR property forging
resistance
* address pr concerns
* add z_cfs len tests on junk chain certs
* strip csr subject info, replace w/ santized values
* fix csr rotation rejection/paths during token exchange/refresh
* fixes#3734 enforce client certificate proof-of-possession for OIDC sessions
- adds verifyCertProofOfPossession() in resolveOidcSession() to require
TLS client cert matching a z_cfs fingerprint or SPIFFE ID when the
OIDC token was issued with cert bindings
- adds z_cae (CertAllowExpired) claim from auth policy, propagated
through token issuance and refresh
- adds TrustCache.VerifyClientCert() with tiered pool matching
(first-party roots, trust anchors, third-party) and TTL cache
- adds WrapIdentityWithCertValidation() on the router to verify client
certs at the TLS level against RDM PublicKeys
- adds IsFirstPartyCert() on the router to gate SPIFFE ID matching by
checking whether the cert chains to the controller root CA
- adds VerifySpiffeId() in common/spiffehlp with SpiffeMatchApiSession,
SpiffeMatchIdentity, and SpiffeMatchNone return types
- adds SPIFFE IDs to OTT and token enrollment certs (/identity/<id>)
- enforces cert expiry by match type: API session certs must be valid,
fingerprint-matched certs respect z_cae, legacy sessions skip checks
- shallow-copies leaf certs before overriding time fields in all cert
verification paths to avoid races on shared x509.Certificate pointers
- fixes controllerRootCache setting inited=true before the ctrl channel
is available, which permanently cached the failure
- registers the service policy type symbol as a string, so queries match the
API's "Dial" and "Bind" names rather than the internal int32 ids
- adds a symbol mapper that converts the stored int32 to its PolicyType name
at query eval time
- updates the posture-checks lookup in EdgeServiceManager, the only direct
caller that went through GetSymbol, to read the mapped string form
- adds a store test covering type = "Dial" and type = "Bind" filtering
- notes the breaking removal of the undocumented type = 1/type = 2 form in
the 2.0 deprecation cleanup list
* fixes#3680 add revocation management API, CLI, and enforcement
- adds Management API endpoints for revocations (POST, GET, LIST) with
type-aware validation (JTI/API_SESSION require UUID, IDENTITY requires
existing identity)
- adds CLI commands: ziti edge create revocation identity|api-session|jti
- adds revocation checks to resolveOidcSession in security_ctx.go so the
REST API returns 401 for revoked OIDC tokens. Previously only
ValidateAccessToken (router ctrl channel path) checked revocations,
so revoked tokens still received 200 OK from the management and client
HTTP APIs
- adds api-session revocation check to ValidateAccessToken, which only
checked JTI and identity revocations
- adds Type field to Revocation model, store, and protobuf message
- adds integration tests covering CRUD, input validation, and token
enforcement for all three revocation types
- use release edge-api@v0.28.1
- adds CreateCircuitV3 message type and handler for routers that have
already authorized dials locally via RDM, bypassing service session
tokens in favor of identity ID, service ID, and pre-assigned circuit ID
- renames CreateCircuitRequest/Response to CreateCircuitV2Request/V2Response
for clarity now that V3 exists
- adds CircuitManager.Reserve to atomically claim circuit IDs before routing,
preventing collisions on pre-assigned IDs
- extends CreateCircuitParams with GetCircuitId so V3 can supply a
pre-assigned circuit ID (falls back to UUID generation when empty)
- fixes IsDialableByIdentity which was incorrectly calling IsBindableByIdentity
- extracts V2 handler into its own file create_circuit_v2.go
- adds CreateCircuitV3RequestType/ResponseType (20222/20223) to edge_ctrl protobuf
- registers V3 handler in controller server
- adds DeleteRevocationsBatchCommand so expired-revocation cleanup goes
through raft as a single log entry per batch
- adds CreateRevocationsBatchCommand for batched revocation creation
through raft
- moves refresh-token revocations from synchronous inline creation to a
background batcher that flushes on a configurable interval, removing
the database and raft as a bottleneck on token refreshes
- skips revocation creation for tokens expiring within a configurable
threshold (revocationMinTokenLifetime), since they become invalid on
their own
- validates that revocationMinTokenLifetime is less than 50% of the
configured refresh token lifetime
- makes the revocation enforcer frequency configurable and restricts it
to run only on the raft leader
- adds tests for multi-batch delete, batched create with router RDM
propagation, and skip-threshold behavior
- adds new configuration tunables under edge.oidc: revocationBucketInterval,
revocationMinTokenLifetime, revocationBucketMaxSize, revocationMaxQueued,
revocationEnforcerFrequency
- fixes RevokeToken double-write: adds return after JWTID-keyed revocation
save, preventing fallthrough write with raw JWT string as unreachable key
- fixes TerminateSession key mismatch: stores revocation by identityId alone,
matching the Subject-based lookup in ValidateAccessToken
- fixes RevocationDelete sync action: passes DataState_Delete instead of
DataState_Create so routers evict the entry from their data model
- adds RevocationManager.DeleteExpired: batch-deletes expired revocations in
batches of 500 until none remain
- adds RevocationEnforcer: periodic policy runner (every 1 minute) that calls
DeleteExpired and records metrics
before test fixes
- removes legacy v1 terminator code path; all terminators now use v2 flow
- refactors edgeTerminator.close() to decouple SDK notification from control plane notification
- adds pending SDK close notification queue with retry when channel is busy
- adds post-create inspect mechanism that verifies SDK still holds the bind after terminator creation
- queues second post-create inspect when establishment takes >30s to catch SDK timeout races
- detects and discards stale reordered binds on the same connection by comparing connIds
- re-establishes replacement terminators when a delete/create race is detected
- eliminates IsEntityPresent pre-filter in removeTerminatorsHandler to prevent raft ordering races
- fixes ValidateTerminators to query identities from the correct manager with the correct filter field
- adds postCreate flag to ValidateTerminatorsV2Request so routers skip redundant SDK inspect
- returns retry-later (nil result) from router validation when inspect is temporarily unavailable
- blocks SyncAllSubscribers until completion and guards RouterDataModel replacement with in-progress flag
- fixes InheritLocalData to enable service access tracking for all subscribed identities
- adds Services.Has check in GetServiceAccessPolicies to prevent false policy grants
- validates policy-to-identity associations in ValidateServicePolicies
- adds `ziti agent tunnel dump-sdk` command for SDK context inspection via IPC agent
- adds `ziti fabric inspect sdk` command to query SDK context through routers
- fixes --expected-per-host CLI flag binding in validate terminators command
- changes bind-access-lost retry hint from NotRetriable to RetryStartOver
- moves trace route response and xgress close handling off channel handler goroutine
- fixes listTerminators test helper to URL-encode filter parameter
- improves sdk-hosting-test validation resilience with login and query retries
- adds terminator_create_flow.md documenting the full lifecycle across SDK, router, and controller
- adds detailed logging for data model sync, service access tracking, and subscriber change detection
- thanks to Jan Starkl <jan.starkl@tttech-digital.com> for the issue
- thanks to Jan Starkl <jan.starkl@tttech-digital.com> for the fix
- rebases work to top of main
- reworks tests to fix new testing model, helpers, callers
- unfurls nested tests
* fixesopenziti/ziti#3356 adds www-authenticate headers
- www-authenticate headers are returned on 401s from API requests
- www-authenticate headers are returned during authentication to signal
addtional JWT bearer tokens needed (secondary ext jwt)
- adds support for additional headers on API errors
- adds SecurityTokenCtx for centralized security header processing
(legacy, jwt, etc.)
- adds SecurityCtx for centralized identity, auth policy, MFA handling
- refactors existing JWT authentication methods (oidc, legacy) to use
centralized processing where possible
* Allow routers to request current cluster membership information. Fixes#3503
* Get cluster membership information from raft directly, rather than trying to cache it in the DB. Fixes#3501
* Set a router data model timeline when initializing a new HA setup, rather than letting it stay blank. Fixes#3500
* Reduce router data model full state updates. Fixes#3504
Paul Lorenz