Commit Graph

8870 Commits

Author SHA1 Message Date
Christopher Britton aa71cd768c Add ZET integration tests to release workflow for pre/rc tags (#3997) 2026-06-22 10:00:39 -04:00
Paul Lorenz c2e6c8ee3d Merge pull request #3917 from openziti/convert-router-forwarder
Convert router/forwarder to slog
2026-06-17 18:52:50 -04:00
Paul Lorenz d1b132c4da Convert router/forwarder to slog. Fixes #3916
Three sub-channels chosen to match operational triage needs:

- router.forwarder.route (forwarder.go) covers circuit routing and
  per-payload / per-acknowledgement / per-control forwarding. The
  payload and acknowledgement hot paths are gated by an explicit
  routeLog.Enabled(Debug) check before the structured emit, so the
  cost when below Debug is a single level comparison.
- router.forwarder.scan (scanner.go) covers idle-circuit scanning;
  dominant log-volume contributor in real traffic, separated so
  operators can silence it without losing the data-plane channels.
- router.forwarder.fault (faulter.go) covers circuit and link fault
  reporting between the router data-plane and the controllers.

WithField chains rewritten as variadic key/value pairs; scoped
With() loggers used where multiple emits share attrs (e.g.,
faulter's per-ctrl loop). Levels preserved across the conversion.

Each file documents its channel name in the var godoc so a contributor
or operator can find it without grep.
2026-06-17 18:25:30 -04:00
Paul Lorenz 0ee288ecbf Distribute routers and router-target configs through the RDM. Fixes #3976 (#3977)
- adds a `DataState.Router` event variant (id, name, fingerprint,
  configs) to the RDM protobuf and the matching ConfigType.Target field
  flows
- loads routers into `RouterDataModelSender` at startup and registers
  entity-change listeners so router create/update/delete and config
  reassignment emit RDM events
- filters `Config` events per-router at `RouterSender`: each router
  sees the full `Router` set but only its own router-target Configs
- receiver-side: parses and stores `Router` entities; GCs router-target
  Configs locally when they drop off the router's `Configs` list, so
  no synthetic remove events are needed on the wire
- extends the `router-data-model-test` fablab model with router-config
  distribution scenarios (assignment, reassignment, controller
  restart, RDM cache miss)
- updates `validate-router-data-model` to recognize the new event
  shape
- refreshes the design doc to describe per-router filtering and the
  change-notification flow
2026-06-16 22:31:43 -04:00
Paul Lorenz 1d6333a6c0 Add router.link.v1 config type. Fixes #3974 (#3975)
* Add router.link.v1 config type. Fixes #3974

- adds the built-in `router.link.v1` ConfigType with a JSON schema
  covering listeners, dialers, heartbeats, payload/ack sender queue
  sizes, and `gcMode` for auto-GC of stale links
- targets routers (`Target=router`) via the field added in #3743
- registers it for new databases via `createConfigType` in
  `initialize`, and for existing databases via a migration step that
  bumps the schema version 46 -> 47
- adds config-type-store tests covering registration, the router
  target, and migration-driven creation on existing databases
- documents the Phase 1c step in the controller-managed router
  configuration design doc

* Address review feedback on router.link.v1 config type. For #3974

- broadens the duration schema pattern to accept compound and fractional values (e.g. 1h30m, 1.5h) matching time.ParseDuration
- adds the gcMode schema property (enum preserve/orphaned/changed) to the built-in config type so its definition is complete where the type is created
- adds duration-format and gcMode validation test cases
- removes the now-unnecessary per-iteration loop variable copy in the reject-cases test
2026-06-16 16:38:13 -04:00
Clint Dovholuk a5b0fce362 Merge pull request #3995 from openziti/issue-3994-add-support-for-multiple-sans
add support to bootstrap.bash for multiple addressees to be supplied
2026-06-16 16:00:17 -04:00
Paul Lorenz 28cd3d746b Merge pull request #3935 from openziti/collapse-fabric-and-edge-services-in-db
Collapse fabric and edge services at the db level. Fixes #3934
2026-06-16 13:26:51 -04:00
Paul Lorenz f18ce1b580 Collapse fabric and edge services at the db level. Fixes #3934
- collapses the edge service boltz child store into the unified service store:
  EdgeService becomes a type alias for Service, and RoleAttributes, Configs, and
  EncryptionRequired become top-level Service fields
- adds an IsFabricOnly discriminator separating pure-fabric ("management")
  services from edge services
- adds a forward-only, irreversible migration that moves edge service data out of
  the edge child bucket, classifies pre-existing services (fabric -> IsFabricOnly
  with encryptionRequired=false; edge -> IsFabricOnly=false), deletes the old edge
  bucket and its index, and rebuilds the role-attributes index via CheckIntegrity
- guards every edge-facing path on the unified store against fabric-only services
  so they stay invisible to the edge API: by-id Read/ReadByName/Delete, list/query
  and the detail lister, ReadForIdentity, identity service-config overrides,
  service policy / SERP @id and #all/role denormalization, the association-list
  routes (terminators, policies, configs), and the router data-model sync
- rejects edge-surface updates of fabric-only services, and preserves edge fields
  on fabric-surface updates of edge services
- marks the fabric-only guards TEMPORARY(fabric-edge-collapse): they exist only to
  keep seldom-used management services hidden, and are removed when the fabric/edge
  distinction is erased
- adds in-CI migration tests (forward plus a round-trip asserting FK sub-bucket
  relocation and forward/reverse refcount preservation), durable fabric-only
  behavior tests, and store-level policy-exclusion tests
- adds one-time authentic-migration gate tooling under zititest/migration-test
  (create-model.sh, query, verify) and its operator runbook
2026-06-16 13:04:13 -04:00
dovholuknf c213b11068 remove ZITI_CTRL_EDGE_ALT_ADVERTISED_ADDRESS from the equation and make the comment better 2026-06-16 12:13:09 -04:00
dovholuknf 5410074cac add support to bootstrap.bash for multiple addressees to be supplied 2026-06-16 11:49:47 -04:00
Paul Lorenz 7ce75078b9 Merge pull request #3911 from openziti/slog-ziti-integration
Install slog and route agent log-level callbacks through common/logging
2026-06-15 16:27:05 -04:00
Paul Lorenz 0aefe599c9 Install slog and route agent log-level callbacks through common/logging. Fixes #3910
- adds BuildPrettyHandler and BuildHandlerForFormat in common/logging:
  pretty output wraps the hand-rolled logging.PrettyHandler (a direct
  port of pfxlog's; df/dl was dropped after review found level-label
  gaps, so there is no github.com/michaelquigley/df dependency) in the
  AsyncHandler chain; the format-aware builder picks pretty / json /
  text by --log-formatter so default look matches pre-slog
- adds BuildTextHandler so --log-formatter=text emits logrus-TextFormatter-
  style key=value output (level=info msg=...) via a slog TextHandler rather
  than the colored pretty handler, restoring the pre-slog meaning of text
- adds logging.Fatal: a slog-world fatal (slog provides none) that emits
  at LevelFatal durably via SyncEmit, then exits, so hard-exit paths do
  not lose the record to the async queue; converts the controller and
  router startup hard-exit sites from Error+os.Exit / Error+panic to it,
  dropping the router's startup panic
- rewires agentlog.DefaultLogLevelCallbacks onto common/logging:
  SetLogLevel drives logging.SetGlobalLevel (lockstep slog + logrus),
  SetChannelLogLevel and ClearChannelLogLevel drive SetNamedLevel /
  ClearNamedLevel; per-channel overrides become slog-only per design
- adds agentToSlog mapping across the seven canonical levels with an
  Info fallback for forward-compat
- ziti/run Options.PreRun and ziti/tunnel rootPreRun build the slog
  handler chain via logging.BuildHandlerForFormat and call Install;
  --verbose seeds the initial level instead of mutating logrus
  directly; AsyncOptions flags exposed via logging.AddFlags on each
  persistent flag set
- hardens the run command's logging flags: PreRun reads --verbose /
  --log-formatter across the command chain so they are honored at either
  the alias-parent (ziti controller run) or child position, and the
  ziti controller / ziti router alias parents skip their legacy
  pfxlog/logrus PersistentPreRun setup for the run subcommand (which
  installs the slog chain itself), keeping it for sibling subcommands
- adds Phase 7 acceptance tests in common/agentlog: TestInstallInvariant
  covers Out=io.Discard, noop formatter, ReportCaller, and the
  lockstep level mirror after Install; TestEndToEnd_AgentSetLogLevel
  walks the agent set-log-level path end to end across bridged-logrus
  and direct-slog routes; TestPerChannelOverride_AppliesToSlogOnly_NotPfxlog
  confirms the design's slog-only channel semantics
- adds Fatal/Panic durability subprocess tests in common/logging that
  fork the test binary, Install the production handler chain, then
  call logrus.Fatal / logrus.Panic and assert the records reach stderr
  before exit/panic; proves the bridge's SyncEmit path flushes before
  os.Exit
- adds doc/logging.md developer note covering how to write a slog
  line, channel-naming convention, the no-Warn/Error-in-hot-paths
  rule, the operator surface, the migration checklist, AsyncOptions
  tunables, and what's deliberately deferred
- adds doc/design/slog-conversion-plan.md with the code-grounded
  per-package channel inventory, the sdk-golang embedder-injection
  pattern, conversion order with deep analysis for the first four
  chunks, and cross-repo coordination notes
2026-06-15 15:34:31 -04:00
Edward Moscardini 81930be53f Merge pull request #3989 from openziti/feature/resolver-multi-interface
Feature/resolver multi interface
2026-06-15 20:28:21 +02:00
Edward Moscardini 71508d831c merge main; resolve conflicts 2026-06-15 13:21:20 -04:00
Edward Moscardini 3f09e263d4 Merge pull request #3973 from openziti/feature/lanif-multi-interface
Feature/lanif multi interface
2026-06-15 19:16:50 +02:00
Edward Moscardini 57cec2f095 updated to error instead of timeout on server failure 2026-06-15 12:49:20 -04:00
Edward Moscardini ae16461c4b update change log 2026-06-15 12:11:37 -04:00
Edward Moscardini 84c44c7de5 expand resolver to support a list of addresses in tproxy mode
resolver previously accepted only a single udp:// address. This change makes it accept either a bare
  string (backwards-compatible) or a YAML list of addresses. A single shared resolver instance handles
  all listeners so hostname mappings remain consistent across interfaces.
2026-06-15 12:05:25 -04:00
Paul Lorenz c7f23fecf0 Merge pull request #3930 from openziti/revocation-enforcement
Enforce api-session and identity revocations on the router
2026-06-13 02:15:22 -04:00
Paul Lorenz f4b9798352 Merge pull request #3907 from openziti/slog-named-loggers
Add named-logger registry, logrus bridge, and pfxlog-shape JSON
2026-06-13 01:36:29 -04:00
Paul Lorenz ff619272ba Enforce api-session and identity revocations on the router. Fixes #3927
The router's CheckConnections reaper enforced only JWT expiry, so a revoked OIDC
api-session, or a disabled/deleted identity, kept its live circuits and hosted
terminators until the access token expired. The router now enforces the
RouterDataModel revocations directly, tightening access-loss propagation to the
reaper interval.

- adds a Type field to DataState_Revocation and the raft Revocation command
  proto, mirroring rest_model.RevocationTypeEnum (API_SESSION/IDENTITY/JTI) so the
  management API, OIDC producers, sync, and router enforcement share one
  vocabulary; the common.RevocationType* constants are compile-time bound to the
  enum to prevent drift
- adds an IssuedBefore cutoff so an identity revocation invalidates only sessions
  issued before it; a session re-authenticated after the cutoff survives the
  still-lingering revocation. Persists IssuedBefore on the db and model Revocation
  and carries it (plus the Type) through the single and batched raft marshalling
- adds RouterDataModel.IsApiSessionRevoked and IsIdentityRevoked and enforces both
  in CheckConnections, closing a revoked session's connections
- revokes a deleted or disabled identity's live OIDC sessions via an
  IdentityRevocationConstraint in the db package, run as a store pre-commit
  constraint so the revocation is written in the same transaction as the identity
  change and cannot be skipped (self-contained OIDC JWTs aren't otherwise
  reachable). NewIdentityManager installs it with the revocation type and lifetime
- has the OIDC end-session (TerminateSessionFromRequest) revoke the specific
  api-session named by the z_asid claim, with an identity-scoped fallback; sets
  IssuedBefore on the identity fallback and the management revocation API; adds
  RevocationManager.CreateOrReplace, routed through by both the OIDC paths and the
  management revocation API, so a repeat logout/termination/revocation refreshes
  the cutoff rather than colliding on the reused id. Expiry derives from the
  longest configured token duration via a shared common.MaxTokenDuration helper
- adds tests/revocation_enforcement_oidc_test.go covering api-session revocation
  (and a fresh session staying unaffected), identity disable and delete, and the
  identity cutoff (a post-cutoff session surviving the lingering revocation)
2026-06-13 01:36:12 -04:00
Paul Lorenz 53a7dea7f3 Add named-logger registry, logrus bridge, and pfxlog-shape JSON. Fixes #3906
- adds Registry (exported, for tests) tracking the global *slog.LevelVar, a
  per-name override map of *slog.LevelVar, a logger cache, and the root
  handler; For(name) returns a cached *slog.Logger whose handler chain binds
  channel: name as the first attr (panics on empty); SetGlobalLevel /
  SetNamedLevel / ClearNamedLevel mutate the registry under a write lock;
  ClearNamedLevel deliberately tracks the live global rather than
  snapshotting, so a later global change still reaches the previously-
  overridden name
- adds namedHandler, the chain node that gates Enabled on the registry-
  resolved level (override or live global) and forwards Handle to the
  registry's root; WithAttrs / WithGroup compose with the existing
  boundHandler / groupedHandler so For(name).WithGroup(...).With(...) works
- adds a package-level default Registry behind an atomic.Pointer; Configure
  swaps the default and clears the logger cache (the new Registry starts
  empty); the package-level For / SetGlobalLevel / etc. panic if Configure
  has not been called
- adds slogBridge, the logrus.Hook that copies each logrus.Entry into a
  slog.Record and dispatches via RootHandler.Handle; Fatal and Panic levels
  route through SyncEmit instead so they're durable before logrus exits the
  process
- adds Install (and the testable InstallTo) which Configure's the default
  Registry, sets the global slog level, redirects logrus output to
  io.Discard, replaces its formatter with noopFormatter, sets its level to
  the mapped equivalent, and registers slogBridge as a hook
- adds bidirectional level mappings between logrus.Level and slog.Level for
  the seven canonical names; non-canonical slog levels bucket into the
  canonical level whose value they most recently exceeded
- adds SyncEmit, the package-level entry point that type-asserts the root to
  *AsyncHandler and routes through its SyncEmit when possible, falling back
  to Handle (already synchronous for non-async handlers)
- adds ReplaceAttr, the slog.HandlerOptions.ReplaceAttr callback that
  coerces JSON output into the pfxlog shape: lowercase level via LevelName,
  nested source attr suppressed at the top level
- adds sourceFlattener, the chain wrapper that decodes the record's PC into
  flat file and func attrs; bridged records arrive with PC == 0 and the
  bridge derives their file/func attrs from logrus's already-resolved
  Entry.Caller (its symbolized PC does not re-decode reliably, so the bridge
  does not forward it), and sourceFlattener passes those records through
- adds BuildHandler, which constructs the production chain (AsyncHandler ->
  sourceFlattener -> JSONHandler) over a caller-supplied io.Writer
- exposes Registry.Root so the package-level RootHandler / SyncEmit can
  reach the underlying handler without exporting the field
- covers everything under -race: For panics on empty, caches loggers,
  GlobalLevel gates Debug below Info, per-name override lets Debug through
  one logger while a peer still filters, clear reverts to the live global
  (and reflects a later global change), level changes affect previously-
  created loggers, the composed
  For("router.link").WithGroup("g").With("k","v").Info(...) produces
  {msg, channel:"router.link", g:{k:"v", x:1}} through the async queue,
  Configure replaces the default with a fresh cache, concurrent For +
  SetNamedLevel + ClearNamedLevel + SetGlobalLevel don't race, level
  round-trip and non-canonical bucketing, bridge async-for-non-fatal vs
  synchronous-for-fatal-and-panic, InstallTo invariants and pre-filter for
  below-level records, SyncEmit fallback for a non-async root, ReplaceAttr
  level rename + source suppression + group isolation, sourceFlattener PC=0
  pass-through, BuildHandler end-to-end pfxlog-shape JSON, and custom-level
  rendering
2026-06-13 00:05:27 -04:00
Paul Lorenz ca08e1e7de Merge pull request #3905 from openziti/slog-async-handler
Add common/logging async slog handler and chain composition
2026-06-12 23:37:26 -04:00
Paul Lorenz ff8ae7cd45 Add common/logging package: AsyncHandler, level helpers, and chain composition. Fixes #3904
- adds a new common/logging package with custom slog.Level constants for
  Trace (-8), Fatal (12), and Panic (16) extending slog's four standard
  levels, plus LevelName and ParseLevel as the single source of truth for
  canonical lowercase wire names (warn and warning both accepted)
- adds AsyncOptions (QueueSize, BlockThreshold, SummaryInterval) with
  Validate, defaults of 4096 / Warn / 5s, and AddFlags / OptionsFromFlags
  bindings so the package can wire into cobra via spf13/pflag alone
- adds AsyncHandler, a bounded async slog.Handler that hands records to a
  single drain goroutine and onto a downstream handler under a shared
  mutex; records at or above the block threshold block (with a closeNotify
  escape so shutdown cannot deadlock), records below it drop when the
  queue is full and bump a per-level atomic counter
- the drain emits a drop-summary record on each SummaryInterval tick when
  any per-level counter is non-zero, and also counts downstream errors in
  a drain_errors counter that appears in the same summary line; downstream
  errors are also logged once to os.Stderr to avoid slog recursion
- Close signals shutdown and returns immediately; the drain final-flushes
  records that beat the close, emits a final summary if drops occurred,
  and closes drainDone for tests
- SyncEmit bypasses the queue and writes through the downstream handler
  synchronously under the same downstreamMu the drain uses, so fatal/panic
  records are durable before the process exits
- adds boundHandler, which prepends bound attrs to every record flowing
  through it before delegating to its parent; WithAttrs returns a new
  boundHandler whose parent is the receiver's parent (not the receiver
  itself), so a chain of slog.Logger.With calls produces sibling
  boundHandlers at the same chain depth rather than stacking
  wrapper-on-wrapper
- adds groupedHandler, which wraps record attrs in slog.Group(name, ...)
  before delegating; a subsequent WithAttrs creates a boundHandler whose
  parent is the groupedHandler, so the attrs land inside the group
- AsyncHandler.WithAttrs and WithGroup are the real chain entry points;
  empty attrs and empty group names return the receiver so no-op
  slog.Logger.With() and WithGroup("") allocate nothing
- covers the lot with -race tests: level round-trip and offset fallback,
  defaults validity and bad-value rejection, flag round-trip, async normal
  flow, drop-on-full with summary attrs, block at the threshold, Close
  idempotent + non-blocking + unblocks Handle, Handle racing Close never
  panics, SyncEmit synchronous and serialized with the drain, drain-error
  counting, the four worked examples from the design doc for the chain
  (with-then-group, group-then-with, nested groups, basic with-attrs),
  no-nesting on repeated WithAttrs, sibling-loggers-do-not-leak-attrs,
  WithGroup("") and WithAttrs(nil) as no-ops on all three handler types,
  and Enabled delegation through the chain
2026-06-12 19:34:17 -04:00
Paul Lorenz d35af13964 Merge pull request #3903 from openziti/agent-capabilities
Add agent IPC capability discovery and channel-based log-level commands
2026-06-12 16:46:04 -04:00
Paul Lorenz c92de09624 Document the agent capability model and cover commands end-to-end
- adds an in-process end-to-end test that starts the agent, registers recording
  callbacks, and exercises AppInfoV2, the three v2 channel commands, and the
  framed-through-callback path
- adds a developer note (common/agent/README.md) explaining the two-tier
  capability model and how to add an agent capability, register an app
  capability, add a v2 channel command, and preserve wire compatibility
2026-06-12 16:18:28 -04:00
Paul Lorenz 32dae7f424 Merge pull request #3895 from openziti/agent-prep
Import agent library into common/agent and consolidate channel-upgrade code
2026-06-12 16:12:07 -04:00
Paul Lorenz c45ce84d6f Wire ziti registration and CLI v2 selection. Fixes #3902
- adds common/agentlog.DefaultLogLevelCallbacks, the one place mapping
  agent.LogLevel onto logrus (global level) and pfxlog (per-channel overrides)
- controller, router, and tunnel register the callbacks before starting the
  agent listener; this activates logging.slog-levels and binds the v2 handlers
- the set-log-level, set-channel-log-level, and clear-channel-log-level CLI
  commands send the v2 channel command when the target advertises
  logging.slog-levels, falling back to the framed command otherwise
- the clear-channel-log-level two-arg positional-address form stays framed
- the CLI parses log-level args via agent.ParseLogLevel, so "warn" is accepted
  (previously rejected because logrus.WarnLevel.String() returns "warning")
2026-06-12 16:04:27 -04:00
Paul Lorenz 7e43a0a731 Add agent IPC capabilities and v2 log-level channel commands
- adds the agent-local LogLevel enum and a capability registry
  (AgentLoggingSlogLevels, the names table, mask / string-list / bit-from-string
  helpers, RegisterAppCapabilities) that freezes on Listen and panics on late
  registration
- adds AppInfoV2 (framed op 0x16) returning identity plus the two capability
  lists, with a client-side reader that falls back cleanly to the legacy
  AppInfo when an old server closes on the unknown op
- carries the agent capability bitmask in the channel hello, with
  CapabilitiesFromHeaders to decode it
- adds three v2 channel commands (SetLogLevelV2, SetChannelLogLevelV2,
  ClearChannelLogLevelV2) in the agent-reserved 30000+ content-type band,
  carrying parameters as channel string headers
- routes both the v2 channel handlers and the legacy framed handlers through a
  registered LogLevelCallbacks, falling back to today's logrus / pfxlog
  behavior when no callback is registered
- HandleChannelConnection auto-binds the v2 handlers on every agent channel
  when callbacks are registered
- the v2 handlers emit the same pfxlog audit Info lines as the framed handlers,
  so server-side audit output is identical across both paths
2026-06-12 16:04:27 -04:00
Paul Lorenz 18671a2593 Hold listener in handler to fix gops listen-goroutine race
The agent listen goroutine read the package-global listener for both
Accept and the deferred Close. Tests that cleared the global after
calling Close (so subsequent Listen calls work) raced with the
goroutine and could trigger a nil-pointer panic in the deferred close.

The goroutine now keeps its own *net.Listener captured at launch
time. Close (and any test cleanup) can manipulate the global without
the goroutine ever seeing the change; the deferred Close on the
already-closed listener is a benign no-op that just logs.

Caught by TestLogLevelCommandsEndToEnd on the agent-capabilities
stack; the racy code was inherited from upstream openziti/agent
during the absorption commit.
2026-06-12 15:47:03 -04:00
Paul Lorenz 9483c39bbd Consolidate agent channel plumbing into common/agent. Fixes #3894
- adds HandleChannelConnection (server-side conn-to-channel upgrade) and the
  client-side NewChannel/ConnToChannel/MakeChannelRequest helpers to common/agent;
  these import channel/v4 + identity, which are already root-module deps, so there
  is no go.mod or Go-version change
- replaces the duplicated agent channel-upgrade handlers in controller, router,
  tunnel, and demo with one-line delegations to common/agent.HandleChannelConnection
- moves the CLI-side channel dialer (was NewAgentChannel/connToChannelMapper/
  MakeAgentChannelRequest in agentcli) into common/agent and repoints the agentcli
  and demo callers
- normalizes the demo echo-server to accept the AppIdAny wildcard, matching the
  other agent servers; previously it rejected it
- preserves the wire protocol (leading app-id byte, "agent" channel, 1s connect
  timeout), so existing channel-based agent commands are unchanged
2026-06-12 15:47:03 -04:00
Paul Lorenz 92348ddd96 Absorb openziti/agent into common/agent. Fixes #3893
- moves the openziti/agent library source (5 files) into common/agent and drops
  the external github.com/openziti/agent dependency from the root and zititest modules
- re-points all 21 importers to the in-tree path; the package name stays agent, so
  call sites are unchanged, and the wire protocol (Magic bytes, op constants, gops
  socket prefix) is identical
- promotes go-ps to a direct dependency (now imported by common/agent); go mod tidy
  also reclassifies golang.org/x/exp as indirect, a pre-existing staleness fix
2026-06-12 15:47:01 -04:00
Paul Lorenz 16e8dc2244 Add logging-refactor design proposal
doc/design/logging-refactor.md captures the design for moving the
codebase to log/slog without forcing a whole-tree migration.

- slog as the API contract at every call site
- logrus.Hook bridge so ~3000 pfxlog/logrus call sites benefit
  without code changes
- common/logging.AsyncHandler: bounded queue, level-aware drop
  policy, periodic summary emission, never-close-the-queue shutdown
- dl.NewPrettyHandler adopted for dev/console; JSONHandler with
  ReplaceAttr for production JSON shape compatibility
- named-logger overrides via logging.For(name); existing
  set-channel-log-level IPC dispatches to both pfxlog and slog
  registries
- PC-based method/file overrides deferred to a follow-up branch
- one proof-of-pattern call-site conversion in this branch
2026-06-12 15:46:32 -04:00
Andrew Martinez 9470c1f126 fixes #3952 reject invalid externalIdClaim and stop enrollment panic (#3953)
* fixes #3952 reject invalid externalIdClaim and stop enrollment panic

- moves the error check before the locator assignment in Ca.GetExternalId
  so an unsupported matcher/parser no longer nil-derefs and returns HTTP 500
- guards the claim index against negative and out-of-range values and errors
  when a matched claim resolves to an empty string, so an empty externalId is
  never silently accepted nor mapped to an identity
- adds validateExternalIdClaim, rejecting unsupported locations, unsupported
  matcher/parser combos, missing matcher/parser criteria, and negative indexes
  with HTTP 400 at CA create and update time
- applies the validation to both CA create and update, which previously had
  no externalIdClaim validation on the update path
- tests GetExternalId across the matcher/parser matrix for no-panic and
  correct error/value behavior
- tests CA create and update reject invalid externalIdClaim configurations
- tests that an externalIdClaim resolving to empty fails enrollment cleanly

* address pr review

- treats an externalIdClaim with no location as no claim in GetExternalId so an
  empty stored claim (e.g. the empty bucket older CLIs leave behind) falls back to
  fingerprint enrollment instead of erroring
- validates the merged result on CA update rather than the raw request: a partial
  patch overlays only its supplied subfields onto the stored claim, an empty {}
  object preserves the stored claim, and a full replace validates as-is
- skips externalIdClaim validation on update when no claim is supplied, fixing a
  spurious rejection of updates to CAs that have no claim
- reports the offending location/matcher/parser in the externalIdClaim validation
  error instead of the raw struct
- adds a --clear-external-id-claim flag to update ca and only sends the claim object
  when a claim flag changed, so existing updates preserve the claim and clearing is
  explicit
- tests externalIdClaim patch merge, preserve, and clear behavior, and that an
  unconfigured claim reads as no claim
2026-06-12 13:52:54 -04:00
Paul Lorenz 27790c8cbc Merge pull request #3922 from openziti/posture-revalidation
Revalidate dial circuits and bind terminators on posture changes
2026-06-12 12:22:05 -04:00
Paul Lorenz eccdf7f420 Add posture revalidation integration tests; fix gaps they exposed. For #3908
- adds tests/posture_revalidation_oidc_test.go: two positive tests (a hosted bind
  terminator is revoked and its active circuit torn down when the host's posture
  data goes invalid; an active dial circuit is revalidated and revoked when a
  posture check is added to its dial policy) and two negative controls (a
  posture-data change that stays compliant retains the terminator and circuit)
- fixes an inverted guard in revalidatePostureAccess: it skipped OIDC sessions
  when it should skip legacy ones. Legacy posture is enforced controller-side via
  session invalidation; only OIDC needs router-side revalidation, so the
  requirement-change path was a no-op for the very sessions it protects
- makes the access-loss paths branch on the circuit's originator (dial vs host
  side) instead of relying on serving conns having no service id. Host-side conns
  are now stamped with their service at creation and expose IsHostSide(); dial
  circuits are re-evaluated against dial access, host-side circuits against bind
  access. This fixes the over-revocation surfaced by the negative control, where a
  benign posture change tore down a host's serving circuits via a spurious
  "service not found" dial denial
- makes bind-access revocation assertive: CloseForBindAccessLoss now tears the
  active circuits served through a revoked terminator, found by iterating the
  conn's circuits for host-side ones on that service. This covers both mux-sink
  and SDK-xgress serving conns (xgEdgeForwarder already carries its originator
  and service id), with no per-terminator bookkeeping, so a host that loses bind
  access drops its established circuits promptly instead of draining
- renames the now-both-sided abstractions to match: state.DialCircuit ->
  EdgeCircuit and ConnProvider.IterateDialCircuits -> IterateEdgeCircuits, the
  per-circuit predicate to IsHostSide(), and the circuit close from
  CloseForDialAccessLoss to CloseForAccessLoss(reason)
2026-06-12 10:25:45 -04:00
Edward Moscardini 7826fb8881 update changelog 2026-06-11 17:12:40 -04:00
Edward Moscardini 93583ed27b expand lanIf to support a list of interfaces in tproxy mode
lanIf previously accepted only a single string. This change makes it accept either a bare string (backwards-compatible) or a YAML list,
  inserting one iptables ACCEPT rule per interface per intercepted service address. The CLI --lanIf flag now accepts comma-separated values or
  repeated flags.
2026-06-11 16:14:12 -04:00
Paul Lorenz ab1c513b8b Merge pull request #3879 from openziti/router-configs
Add configs field to routers. Fixes #3780
2026-06-11 14:34:04 -04:00
Paul Lorenz cdc5d89885 Add configs field to routers. Fixes #3780 2026-06-11 13:45:58 -04:00
Christopher Britton 292543a68a Add ref build and LTS line selectors to setup-cli action (#3962)
* Add main-branch source build to setup-cli action

* Add active-lts/maint-lts selectors to setup-cli action

* Build from source via dedicated ref input instead of version: main for  setup-cli action
2026-06-10 16:07:10 -04:00
Clint Dovholuk e8507ce217 Merge pull request #3963 from openziti/issue-3961-router-panic-on-posture-check
fix router panic with process multi posture check
2026-06-10 13:50:37 -04:00
Andrew Martinez 556a16df6e add test-writing guide for the tests package (#3966)
- adds tests/README.md documenting how to write new tests: integration/
  black-box/gray-box scope, build tags and how to run, file layout, test
  skeleton, Arrange/Act/Assert, the no-inline-closures rule, the two helper
  flavors (fixture-setup vs asserting) and where each lives, assertions,
  typed API clients, raw-HTTP wire-format tests, fixtures and data isolation,
  dataflow patterns, and a consolidated anti-pattern list
- updates CONTRIBUTING.md to point contributors at local module README.md
  files for development guidance, citing tests/README.md as the example
- fixes a <issuer-number> -> <issue-number> typo in CONTRIBUTING.md
2026-06-10 13:22:23 -04:00
Paul Lorenz 0ba057e1bf Merge pull request #3918 from openziti/extend-attribute-search
Add role-attribute usage queries. Fixes #1593
2026-06-10 13:00:40 -04:00
Paul Lorenz a582fe5d0f Add role-attribute usage queries. Fixes #1593
- adds management API endpoints listing role-attribute usage for identities,
  edge routers, services, and posture checks, reporting per-source counts
  (and optionally ids via withIds) across home-entity collections and the
  policies that reference each attribute
- adds RoleAttributeUsage model with QueryRoleAttributeUsage, mapping each
  RoleAttributeKind to its contributing sources, with all reads in a single
  transaction so counts stay consistent with the attribute list; returns an
  empty result (not a panic) for kinds with no attributes and an error for
  unknown kinds
- adds boltz.SetIndexValueTransform and AddSetIndexWithTransform, letting a
  SetIndex filter and rewrite symbol values without persisting a derived field
- adds derived role-attribute set indexes to the service policy, edge router
  policy, and service edge router policy stores, built from existing role
  fields via a role-attribute-only transform that excludes the #all wildcard
- adds a db migration that backfills the new indexes via each index's own
  CheckIntegrity(fix=true), scoped to the role-attribute indexes only, with a
  per-index summary log, stopping at the first error
- tests the new endpoints via the typed edge-api management client
2026-06-10 12:39:26 -04:00
dovholuknf 5d628b0b09 don't mutate the passed in variable, use protobuf getter instead 2026-06-10 12:05:22 -04:00
dovholuknf c6b5c98443 terser comments 2026-06-10 11:56:13 -04:00
dovholuknf 362330a4e6 fix router panic with process multi posture check 2026-06-10 11:44:03 -04:00
Paul Lorenz df2054e003 Merge pull request #3950 from openziti/fix-delete-trigger-err-return
Return event-firing error in DeleteById instead of swallowing it
2026-06-09 23:38:03 -04:00
Paul Lorenz 328304e04f Refcount shared intercept hostnames so iptables rules are installed for every service (#3869)
* Refcount shared intercept hostnames so iptables rules are installed for every service. Fixes #3867

- modifies getDnsIp to invoke addrCB and register a cleanup action when the hostname already has an allocated IP, so each service sharing the hostname gets its per-service iptables rule installed
- adds a reference count on hostname allocations so the resolver entry is removed and the CGNAT IP recycled only when the last service using the hostname is cleaned up
- canonicalizes the refcount key by lowercasing so case variants of the same hostname share one allocation, matching the resolver's case-insensitive view
- adds tests covering shared-hostname rule installation, refcounted cleanup, out-of-order cleanup, and case-insensitive sharing

* Address review comments

* Refcount wildcard-allocated intercept hostnames. Fixes #3867

- moves hostname registration into allocateDnsIp, under the allocation mutex, so every allocation (fresh or reused) takes one reference through the refcounting resolver and closes the lookup/registration race between wildcard DNS queries and service updates
- removes the direct AddHostname call in getAddress, which bypassed the refcounting layer and let one service's cleanup remove a hostname still used by an overlapping wildcard or literal intercept
- documents the AddDomain callback contract: the callback registers the hostname mapping itself
- updates tests to match the production wiring and adds coverage for overlapping wildcard/literal intercepts in both removal orders
- adds a dns package test pinning the getAddress callback contract
- adds local ai tooling files to .gitignore
2026-06-09 16:35:32 -04:00