fix(ci): enable compile-generator in SLSA L3 binary provenance

The SLSA reusable workflow generator_generic_slsa3.yml@v2.1.0 has two
paths for fetching its generator binary:

  1. (Default) download a pre-built binary from a GitHub release of
     slsa-framework/slsa-github-generator. Releases are identified by
     TAG NAME (vX.Y.Z), not commit SHA.
  2. (compile-generator: true) build the generator from source inside
     the workflow run, using whatever ref the workflow was pinned to.

Phase 1 RED-2 (commit eda3b48, 2026-05-13) SHA-pinned every GitHub
Actions `uses:` line including the SLSA reusable workflow:

    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@f7dd8c54...  # v2.1.0

The SHA pin is correct for supply-chain integrity (no surprise updates
via tag moves) but incompatible with the default release-download path,
which the workflow proves by hard-erroring at:

    Fetching the builder with ref: f7dd8c54c2067bafc12ca7a55595d5ee9b75204a
    Invalid ref: f7dd8c54c2067bafc12ca7a55595d5ee9b75204a.
    Expected ref of the form refs/tags/vX.Y.Z

The fix is the SLSA project's documented escape hatch for SHA-pinned
consumers: set `compile-generator: true` in the workflow inputs.
This:
  - Preserves the Phase 1 RED-2 SHA pin (no policy regression)
  - Builds the generator from the pinned-SHA source (actually MORE
    secure than downloading a release binary — no separate trust
    boundary on the release artifact's signing)
  - Adds ~1 minute to the workflow runtime (acceptable for a release
    workflow that already takes ~5 min for the SBOM + cosign work)
  - Documented inline so future contributors don't strip the line
    thinking it's a stale workaround

Visible in the failed Release v2.1.1 workflow run 25834286907 (the
`SLSA provenance (binaries) / generator` job, 17s duration, exited
on the invalid-ref check before any sigstore network operation).

Re-cutting v2.1.1 (or tagging v2.1.2) against this commit should
produce a green release pipeline.

.env.example

@@ -7,7 +7,7 @@
 # ==============================================================================
 POSTGRES_DB=certctl
 POSTGRES_USER=certctl
-POSTGRES_PASSWORD=change-me-in-production
+POSTGRES_PASSWORD=replace-with-openssl-rand-hex-32
 
 # ==============================================================================
 # Certctl Server
@@ -24,24 +24,45 @@ POSTGRES_PASSWORD=change-me-in-production
 # seeds pg_authid on first boot of an empty volume. See docs/quickstart.md
 # "Warning" callout and `internal/repository/postgres/db.go::wrapPingError`
 # for the SQLSTATE 28P01 diagnostic that fires when the two drift.
-CERTCTL_DATABASE_URL=postgres://certctl:change-me-in-production@postgres:5432/certctl?sslmode=disable
+CERTCTL_DATABASE_URL=postgres://certctl:replace-with-openssl-rand-hex-32@postgres:5432/certctl?sslmode=disable
 CERTCTL_SERVER_HOST=0.0.0.0
 CERTCTL_SERVER_PORT=8443
 CERTCTL_LOG_LEVEL=info
 CERTCTL_LOG_FORMAT=json
 
-# Auth type: "api-key" (production) or "none" (demo/development).
+# Auth type: "api-key" (production), "none" (demo/development), or
-# For JWT/OIDC, run an authenticating gateway in front of certctl
+# "oidc" (Auth Bundle 2 - native OIDC SSO via coreos/go-oidc/v3, ships
-# (oauth2-proxy / Envoy ext_authz / Traefik ForwardAuth / Pomerium) and
+# in Bundle 2 phases 5+6; setting CERTCTL_AUTH_TYPE=oidc on a build
-# set CERTCTL_AUTH_TYPE=none on the upstream — see
+# without Bundle 2 wired triggers a clear refuse-to-start error rather
-# docs/architecture.md "Authenticating-gateway pattern". G-1 removed
+# than a silent fallback to api-key). For JWT / SAML / LDAP, continue to
-# the in-process "jwt" option (no JWT middleware shipped — silent auth
+# run an authenticating gateway in front of certctl (oauth2-proxy /
-# downgrade); see docs/upgrade-to-v2-jwt-removal.md if you previously
+# Envoy ext_authz / Traefik ForwardAuth / Pomerium) and set
-# set CERTCTL_AUTH_TYPE=jwt.
+# CERTCTL_AUTH_TYPE=none on the upstream - see docs/architecture.md
-CERTCTL_AUTH_TYPE=none
+# "Authenticating-gateway pattern". G-1 removed the in-process "jwt"
-# Required when CERTCTL_AUTH_TYPE is "api-key".
+# option (no JWT middleware shipped - silent auth downgrade); see
-# Generate with: openssl rand -base64 32
+# docs/upgrade-to-v2-jwt-removal.md if you previously set
-# CERTCTL_AUTH_SECRET=change-me-in-production
+# CERTCTL_AUTH_TYPE=jwt.
+#
+# Bundle 2 closure (2026-05-12): the docker-compose base file no longer
+# defaults to AUTH_TYPE=none. The base ships production-shaped; the demo
+# overlay (deploy/docker-compose.demo.yml) flips this baseline into the
+# populated-dashboard demo path.
+CERTCTL_AUTH_TYPE=api-key
+# Required when CERTCTL_AUTH_TYPE is "api-key". Generate with:
+#   openssl rand -base64 32
+# The Bundle 2 fail-closed Validate() REFUSES TO START if this value
+# equals the placeholder string "change-me-in-production" outside of
+# demo mode (CERTCTL_DEMO_MODE_ACK=true).
+CERTCTL_AUTH_SECRET=replace-with-openssl-rand-base64-32
+
+# Bundle 2 closure: AES-256-GCM key for encrypting issuer/target config
+# secrets at rest. Required for any deployment that uses the dynamic
+# config GUI to store issuer credentials. Generate with:
+#   openssl rand -base64 32
+# Minimum 32 bytes. The Bundle 2 fail-closed Validate() REFUSES TO
+# START if this value equals the placeholder string
+# "change-me-32-char-encryption-key" outside of demo mode.
+CERTCTL_CONFIG_ENCRYPTION_KEY=replace-with-openssl-rand-base64-32
 
 # ==============================================================================
 # Certctl Agent
@@ -50,8 +71,14 @@ CERTCTL_AUTH_TYPE=none
 # startup. Use the docker-compose self-signed bootstrap CA bundle from
 # `deploy/test/certs/ca.crt` or supply your own via CERTCTL_SERVER_CA_BUNDLE_PATH.
 CERTCTL_SERVER_URL=https://localhost:8443
-CERTCTL_API_KEY=change-me-in-production
+# Matches one of the server's CERTCTL_AUTH_SECRET rotation values. The
+# placeholder is rejected outside demo mode (Bundle 2 fail-closed guard).
+CERTCTL_API_KEY=replace-with-openssl-rand-base64-32
 CERTCTL_AGENT_NAME=local-agent
+# Returned from `POST /api/v1/agents` during agent enrollment. The agent
+# fail-fasts at startup with "agent-id flag or CERTCTL_AGENT_ID env var
+# is required" if this is unset.
+# CERTCTL_AGENT_ID=agent-from-registration-response
 
 # ==============================================================================
 # Optional: Scheduler Tuning (defaults are usually fine)

.github/coverage-thresholds.yml

@@ -76,3 +76,154 @@ internal/mcp:
     Bundle K / Coverage-Audit C-002 — MCP per-tool dispatch via
     in-memory transport lifts package from 28.0% to 93.1% (per-
     package run). Floor at 85.
+
+internal/auth:
+  floor: 85
+  why: |
+    Bundle 1 Phase 12 — RBAC primitive coverage gate.
+    internal/auth ships keystore + middleware + RequirePermission +
+    bootstrap + the Phase-3 context keys + the protocol-endpoint
+    allowlist. Negative-test coverage (no actor → 401, no role →
+    403, wrong scope → 403, bootstrap-token-wrong → 401, bootstrap-
+    used-twice → 410, admin-already-exists → 410, zero-length token
+    rejection) is now in place. Prescribed Bundle 1 target was 90;
+    held at 85 to absorb the per-file-average dip from the
+    middleware shim files (testfixtures.go) which CI runs but only
+    test fixtures exercise. Sub-package internal/auth/bootstrap
+    inherits this floor.
+
+internal/service/auth:
+  floor: 85
+  why: |
+    Bundle 1 Phase 12 — RBAC service-layer coverage gate.
+    PermissionService + RoleService + ActorRoleService + Authorizer
+    each have positive + negative tests covering the
+    privilege-escalation guard (auth.role.assign required for
+    Grant/Revoke), the reserved-actor invariant (actor-demo-anon
+    cannot be mutated), the canonical-permission validation, the
+    role-in-use guard on Delete, and every sentinel-error path
+    (ErrUnauthenticated / ErrForbidden / ErrSelfRoleAssignment /
+    ErrAuthReservedActor / ErrAuthUnknownPermission /
+    ErrAuthRoleInUse).
+
+internal/auth/oidc:
+  floor: 90
+  why: |
+    Bundle 2 Phase 3 — OIDC service coverage gate. Phase 3 spec
+    pins the floor at 90 explicitly because every fail-closed
+    branch is load-bearing for the security posture: alg pinning
+    (deny-list HS*/none + allow-list RS*/ES*/EdDSA), audience
+    re-check, azp enforcement on multi-aud tokens, at_hash
+    REQUIRED-when-access-token-present (Phase 3 lifts the OIDC
+    core "MAY" to a service-level "MUST"), iat-window window,
+    nonce constant-time-compare, single-use state replay defense,
+    PKCE-S256 mandatory, IdP downgrade-attack defense at
+    provider-load + RefreshKeys time, JWKS-fail-closed semantics,
+    group-claim resolution + userinfo-fallback fail-closed
+    semantics, token-leak hygiene. A regression in any one of
+    these branches is a security incident; the floor catches it
+    before the commit lands. The mock-IdP fixture in
+    service_test.go is the load-bearing harness.
+
+internal/auth/oidc/groupclaim:
+  floor: 95
+  why: |
+    Bundle 2 Phase 3 — group-claim resolver. Hand-rolled (no
+    JSON-path dep per Decision 10); ~150 LOC, every branch
+    exercised by 19 unit tests covering the documented IdP shapes
+    (Okta string array, Keycloak realm_access.roles, Auth0
+    namespaced URL claim, single-string normalization,
+    deeply-nested 3-segment walks) plus every fail-closed branch
+    (empty path, missing key, missing nested key, non-object
+    intermediate, bool/number/object/nil values, array with
+    non-string element, URL-shape with dots-in-path treated as
+    literal). Resolver should be at 100%; floor at 95 leaves a
+    1-statement margin for future error-message refactors.
+
+internal/auth/oidc/domain:
+  floor: 90
+  why: |
+    Bundle 2 Phase 1 — OIDCProvider + GroupRoleMapping domain.
+    Validation-heavy package; constructors + Validate methods
+    cover all canonical IdP shapes (Okta / Azure AD / Google
+    Workspace / Keycloak / Authentik / Auth0). Floor at 90 to
+    catch any future field that ships without a validator.
+
+internal/auth/session:
+  floor: 90
+  why: |
+    Bundle 2 Phase 4 — session lifecycle service. Phase 4 spec
+    pins the floor at 90 because every fail-closed branch carries
+    a security invariant: HMAC-SHA256 cookie signing with a
+    LENGTH-PREFIXED canonical input (defeats the
+    `<a, bc>`-vs-`<ab, c>` concatenation collision attack on the
+    bare-concat form), v1. version-prefix lock, idle expiry,
+    absolute expiry, revocation, retired-but-in-retention key
+    success path, retired-past-retention failure path, CSRF
+    constant-time compare against the SHA-256-hashed copy on the
+    session row, optional IP/UA-bind defense-in-depth gates,
+    fail-fatal initial-key bootstrap. A regression in any one of
+    these branches is a security incident; the floor catches it
+    before the commit lands. The 15-case negative-test matrix in
+    service_test.go is the load-bearing harness; the in-memory
+    stubs of SessionRepo + SigningKeyRepo + AuditRecorder let the
+    state machine be exercised without the postgres testcontainer
+    overhead (which Phase 2's integration tests already cover).
+
+internal/auth/session/domain:
+  floor: 90
+  why: |
+    Bundle 2 Phase 1 — Session + SessionSigningKey domain. Both
+    types ship Validate() with full invariant coverage: ID prefix
+    enforcement (ses-/sk-), expiry-order CHECK (absolute > idle >
+    created), CSRFTokenHash format pin (64 lowercase hex chars),
+    KeyMaterialEncrypted non-empty, retired-before-created
+    rejection, TenantID defaulting. Cookie naming constants are
+    pinned by TestCookieNamingConstants because the GUI's
+    web/src/api/client.ts will read `certctl_csrf` by string.
+    Floor at 90 to catch any future field that ships without a
+    validator.
+
+internal/auth/breakglass:
+  floor: 90
+  why: |
+    Bundle 2 Phase 7.5 — break-glass admin service (Argon2id +
+    lockout state machine + constant-time-via-verifyDummy). Phase
+    13 Pre-merge audit: floor at 90 with no carve-out. Phase 7.5
+    spec ships the package at 91.5%, validated by 8 mandated
+    negatives + ~12 coverage-lift tests. Every fail-closed branch
+    is load-bearing for the security surface (default-OFF posture
+    only matters if every "disabled" path returns ErrDisabled
+    BEFORE any DB lookup; constant-time defense only matters if
+    every path goes through verifyDummy on the no-credential leg).
+    A regression that drops a fail-closed branch's coverage below
+    90 is a real security risk — gate trips, operator audits.
+
+internal/auth/breakglass/domain:
+  floor: 90
+  why: |
+    Bundle 2 Phase 1 — BreakglassCredential domain. Argon2id PHC
+    format pinned ($argon2id$ prefix), MinPasswordLengthBytes (12)
+    + MaxPasswordLengthBytes (256) constants pinned by dedicated
+    test, IsLocked(now) state machine helper. The package ships
+    at 100% coverage; floor at 90 is the standing-room floor for
+    any future field added without a validator.
+
+internal/auth/user/domain:
+  floor: 90
+  why: |
+    Bundle 2 Phase 1 — User domain (federated-human identity).
+    OIDCSubject + OIDCProviderID unique-index per the Phase 2
+    schema, WebAuthnCredentials JSONB reserved for v3, Validate()
+    enforces every on-disk invariant. The package ships at 96.4%
+    coverage. Floor at 90 to catch any future field added without
+    a validator.
+
+    Phase 13 prompt explicitly enumerates internal/auth/user/ at
+    floor 90. The parent (non-domain) directory has no Go source —
+    the user upsert lives in internal/auth/oidc/service.go alongside
+    group resolution + role mapping (cohesive sequence within the
+    OIDC callback). Splitting upsertUser into a separate
+    internal/auth/user/ service package would harm cohesion without
+    adding test value; the domain layer's invariant coverage is
+    where the floor actually applies.

.github/workflows/ci.yml

@@ -14,12 +14,17 @@ jobs:
     name: Go Build & Test
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Set up Go
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
         with:
-          go-version: '1.25.9'
+          go-version: '1.25.10'
+          # Phase 3 TEST-L1 closure (2026-05-13): enable Go's module +
+          # build cache so re-runs hit the cache instead of recompiling
+          # the world. setup-go v5 cache: true by default; making it
+          # explicit so a future setup-go upgrade can't silently flip it.
+          cache: true
 
       - name: Go Build
         run: |
@@ -103,11 +108,29 @@ jobs:
         run: staticcheck ./...
 
       - name: Race Detection
-        run: go test -race ./internal/service/... ./internal/api/handler/... ./internal/api/middleware/... ./internal/scheduler/... ./internal/connector/... ./internal/crypto/... ./internal/domain/... ./internal/validation/... ./internal/tlsprobe/... -count=1 -timeout 300s
+        # Phase 3 TEST-H1 closure (2026-05-13): the pre-Phase-3 invocation
+        # listed 9 explicit package roots, excluding internal/auth/*,
+        # internal/repository/*, internal/mcp, internal/scep, internal/pkcs7,
+        # internal/api/router, internal/api/acme, internal/cli, internal/cms,
+        # internal/config, internal/deploy, internal/integration,
+        # internal/ratelimit, internal/secret, internal/trustanchor, plus
+        # all of cmd/. Audit finding TEST-H1 flagged this as silent
+        # race-detection drift — packages added after the original list
+        # was authored were never covered.
+        #
+        # Post-Phase-3: ./... with -short. The 76 testing.Short() guards
+        # already in the integration-test surface (testcontainers, live-DB,
+        # multi-process) gate behind this flag, so race detection runs
+        # across every package without dragging in long-running suites.
+        # Timeout doubled from 300s to 600s because ./... is broader; the
+        # broader scope is what makes race coverage trustworthy.
+        run: go test -race -short ./... -count=1 -timeout 600s
 
       - name: Go Test with Coverage
+        # internal/ciparity/... — post-v2.1.0 anti-rot item 2 surface-
+        # parity tests; stdlib-only so they always pass in this job.
         run: |
-          go test ./internal/service/... ./internal/api/handler/... ./internal/api/middleware/... ./internal/integration/... ./internal/connector/issuer/... ./internal/connector/target/... ./internal/connector/notifier/... ./internal/connector/discovery/... ./internal/crypto/... ./internal/mcp/... ./internal/cli/... ./internal/domain/... ./internal/validation/... ./internal/tlsprobe/... -count=1 -cover -coverprofile=coverage.out
+          go test ./internal/service/... ./internal/api/handler/... ./internal/api/middleware/... ./internal/api/router/... ./internal/auth/... ./internal/integration/... ./internal/connector/issuer/... ./internal/connector/target/... ./internal/connector/notifier/... ./internal/connector/discovery/... ./internal/crypto/... ./internal/mcp/... ./internal/cli/... ./internal/domain/... ./internal/validation/... ./internal/tlsprobe/... ./internal/ciparity/... -count=1 -cover -coverprofile=coverage.out
 
       - name: Check Coverage Thresholds
         # ci-pipeline-cleanup Phase 2: per-package floors moved to
@@ -118,7 +141,7 @@ jobs:
         run: bash scripts/check-coverage-thresholds.sh
 
       - name: Upload Coverage Report
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
         with:
           name: go-coverage
           path: coverage.out
@@ -135,52 +158,6 @@ jobs:
           GITHUB_REPOSITORY: ${{ github.repository }}
         run: bash scripts/coverage-pr-comment.sh
 
-      # Bundle P / Strengthening #6 — QA-doc seed-count drift guard. Forces
-      # every PR that adds a seed row to migrations/seed_demo.sql to keep
-      # docs/contributor/qa-test-suite.md::Seed Data Reference in sync.
-      #
-      # Phase 5 of the 2026-05-04 docs overhaul (commit c64777f) deleted
-      # docs/testing-guide.md (its content dispersed across the new
-      # audience-organized doc tree); the previous QA-doc Part-count drift
-      # guard tracked Part counts between testing-guide.md and the old
-      # qa-test-guide.md headline. With testing-guide.md gone, that guard's
-      # premise is dead and it has been removed. The seed-count drift class
-      # is still live: qa-test-suite.md::Seed Data Reference enumerates
-      # certs/issuers and seed_demo.sql is the source of truth.
-      - name: QA-doc seed-count drift guard
-        run: |
-          set -e
-          DOC=docs/contributor/qa-test-suite.md
-          # Seed-cert count: agnostic to documented header format. The current
-          # documented count lives in `### Certificates (32 total in ...` —
-          # extract the first integer in that header.
-          DOC_CERTS=$(grep -oE '### Certificates \([0-9]+' "$DOC" | grep -oE '[0-9]+' | head -1)
-          # Authoritative count: unique mc-* IDs in seed_demo.sql.
-          SEED_CERTS=$(grep -oE 'mc-[a-z0-9_-]+' migrations/seed_demo.sql | sort -u | wc -l | tr -d ' ')
-          if [ -z "$DOC_CERTS" ]; then
-            echo "::warning::Could not extract documented cert count from $DOC."
-            echo "  Skipping cert-count drift check (header format may have changed)."
-          elif [ "$DOC_CERTS" != "$SEED_CERTS" ]; then
-            echo "::error::DRIFT — $DOC says $DOC_CERTS certs; seed_demo.sql has $SEED_CERTS unique mc-* IDs."
-            echo "  Update $DOC::Seed Data Reference to match."
-            exit 1
-          fi
-          # Issuers: seed-table count vs doc claim.
-          DOC_ISS=$(grep -oE '### Issuers \([0-9]+' "$DOC" | grep -oE '[0-9]+' | head -1)
-          # Authoritative: unique iss-* IDs (close enough proxy; the issuers
-          # table count IS the unique-ID count for this prefix).
-          SEED_ISS=$(grep -oE 'iss-[a-z0-9_-]+' migrations/seed_demo.sql | sort -u | wc -l | tr -d ' ')
-          if [ -z "$DOC_ISS" ]; then
-            echo "::warning::Could not extract documented issuer count."
-          elif [ "$DOC_ISS" != "$SEED_ISS" ] && [ "$((SEED_ISS - DOC_ISS))" -gt 5 ]; then
-            # Allow up to 5pp slack — iss-* IDs appear in audit_events and
-            # other reference tables that aren't issuer-table rows. Drift
-            # only flags when the spread grows large.
-            echo "::error::DRIFT — $DOC says $DOC_ISS issuers; seed_demo.sql has $SEED_ISS unique iss-* IDs (spread > 5)."
-            exit 1
-          fi
-          echo "QA-doc seed-count drift guard: clean."
-
       # Bundle Q / I-001 closure — test-naming convention guard (informational).
       # The convention is `Test<Func>_<Scenario>_<ExpectedResult>`. This step
       # prints any non-conformant tests but does NOT fail the build until the
@@ -197,9 +174,8 @@ jobs:
       # internal scenarios expressed via `t.Run` subtests. Requiring the
       # underscore-Scenario-Result triple repo-wide would mean renaming
       # 167 legitimate tests for no observable behavior change. The
-      # Test<Func>_<Scenario>_<ExpectedResult> form remains documented as
+      # Test<Func>_<Scenario>_<ExpectedResult> form remains the
-      # the recommended pattern for parameterized scenarios in
+      # recommended pattern for parameterized scenarios, but is not gated.
-      # docs/contributor/qa-test-suite.md, but is not gated.
       - name: Regression guards (extracted to scripts/ci-guards/)
         # All named regression guards live at scripts/ci-guards/<id>.sh per
         # ci-pipeline-cleanup bundle Phase 1. Each guard is callable locally:
@@ -207,6 +183,7 @@ jobs:
         # Adding a new guard: drop a new <id>.sh; this loop auto-picks it up.
         # Contract: each guard MUST exit 0 on clean repo, non-zero with
         # ::error:: prefix on regression. See scripts/ci-guards/README.md.
+        #
         run: |
           set -e
           fail=0
@@ -219,14 +196,216 @@ jobs:
           done
           exit $fail
 
+  cross-platform-build:
+    # Phase 3 TEST-H2 closure (2026-05-13): the pre-Phase-3 CI ran
+    # exclusively on ubuntu-latest, leaving Windows-specific bugs
+    # (path separators, file permissions, exec.Command semantics)
+    # undetected. The agent + CLI binaries ship for Windows + macOS
+    # users; this matrix asserts they at least BUILD on every OS we
+    # claim to support.
+    #
+    # Build-only — no test run. Full test parity across OSes is a
+    # larger investment (testcontainers is Linux-only on Windows CI
+    # runners, file-permission tests differ, etc.). The build gate
+    # is the minimum that catches the cross-platform regressions
+    # we've seen in practice.
+    name: Cross-platform build (ubuntu / windows / macos)
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
+
+      - name: Set up Go
+        uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
+        with:
+          go-version: '1.25.10'
+          cache: true
+
+      - name: Build server + agent + CLI + mcp-server
+        run: |
+          go build ./cmd/server
+          go build ./cmd/agent
+          go build ./cmd/cli
+          go build ./cmd/mcp-server
+
+  cold-db-compose-smoke:
+    # Per post-v2.1.0 anti-rot item 6 (Auditable Codebase Bundle).
+    #
+    # Catches migration-on-cold-DB regressions: wipe the postgres
+    # volume, bring the stack up cold, mint a day-0 admin, issue +
+    # renew + revoke a test certificate, assert audit rows, tear down.
+    # Targets the bug class that the warm-DB integration suite misses
+    # (canonical case: 2026-05-09 migration 000045 broken INSERT,
+    # fixed in commit 6444e13).
+    name: Cold-DB compose smoke
+    runs-on: ubuntu-latest
+    needs: go-build-and-test
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
+
+      - name: Show Docker versions
+        run: |
+          docker --version
+          docker compose version
+
+      - name: Cold-DB compose smoke
+        # The smoke deliberately focuses on the bug class that ONLY a
+        # cold boot can catch: stack-startup correctness against a
+        # blank database. It is intentionally NOT a functional API
+        # walkthrough — the integration test suite under
+        # 'Go Test with Coverage' already covers issue / renew /
+        # revoke / audit-row plumbing against a warm DB.
+        #
+        # The bugs this gate is uniquely positioned to catch:
+        #   - Missing required env vars that fail Config.Validate()
+        #     at startup (e.g. CERTCTL_DEMO_MODE_ACK gap, 2026-05-12).
+        #   - Non-idempotent migrations that crash on the second boot
+        #     (e.g. migration 000043 CHECK constraint, 2026-05-12).
+        #   - Documented manual flows that don't work end-to-end on
+        #     a clean compose (e.g. CERTCTL_BOOTSTRAP_TOKEN
+        #     interpolation gap, 2026-05-12).
+        #
+        # Bugs OUTSIDE the scope of this smoke (covered elsewhere):
+        #   - API request/response contract changes (integration suite).
+        #   - Cert lifecycle correctness (integration suite + handler
+        #     tests).
+        #   - Audit row plumbing (handler tests).
+        #
+        # 10-min wall-clock cap covers cold image pull + compose-up +
+        # force-recreate + admin bootstrap + teardown. Increase only
+        # if the underlying steps legitimately grow.
+        #
+        # The smoke is inlined here on purpose — it is NOT a script in
+        # scripts/ci-guards/, because there is no value in a developer
+        # running this locally. The whole point of the gate is that CI
+        # owns the cold-DB state; the operator never has to remember to
+        # run it.
+        timeout-minutes: 10
+        working-directory: deploy
+        env:
+          STARTUP_TIMEOUT_SECONDS: 300
+        run: |
+          set -e
+          set -o pipefail
+
+          SERVER_URL="https://localhost:8443"
+          CACERT_PATH="${GITHUB_WORKSPACE}/deploy/test/certs/ca.crt"
+
+          log() { echo "[cold-db-smoke] $*"; }
+
+          wait_for_service_healthy() {
+            local svc="$1" deadline=$(( $(date +%s) + STARTUP_TIMEOUT_SECONDS ))
+            while [ "$(date +%s)" -lt "$deadline" ]; do
+              local state
+              state="$(docker compose ps --format json "$svc" 2>/dev/null | python3 -c '
+          import json, sys
+          try:
+              line = sys.stdin.read().strip()
+              if not line:
+                  print("not-up"); sys.exit(0)
+              rows = json.loads(line) if line.startswith("[") else [json.loads(l) for l in line.splitlines() if l.strip()]
+              if not rows:
+                  print("not-up")
+              else:
+                  print(rows[0].get("Health", rows[0].get("State", "?")))
+          except Exception as e:
+              print(f"err: {e}")
+          ')"
+              if [ "$state" = "healthy" ] || [ "$state" = "running" ]; then
+                log "  $svc → $state"; return 0
+              fi
+              sleep 2
+            done
+            log "  $svc did NOT reach healthy within ${STARTUP_TIMEOUT_SECONDS}s (last: $state)"
+            return 1
+          }
+
+          http_call() {
+            local method="$1" path="$2" data="${3:-}"
+            local args=(--silent --show-error --max-time 30 -X "$method" "$SERVER_URL$path")
+            [ -f "$CACERT_PATH" ] && args+=(--cacert "$CACERT_PATH") || args+=(--insecure)
+            [ -n "$data" ] && args+=(-H "Content-Type: application/json" -d "$data")
+            curl "${args[@]}"
+          }
+
+          # Bundle 2 closure (2026-05-12): the base compose is now
+          # production-shaped — auth=api-key + agent-keygen + fail-closed
+          # placeholder guards. The cold-DB smoke layers in the demo
+          # overlay so the boot path remains zero-config: the overlay
+          # supplies AUTH_TYPE=none + DEMO_MODE_ACK=true + the matching
+          # placeholder creds the fail-closed guards accept under
+          # DEMO_MODE_ACK. The agent service in the overlay also
+          # pre-seeds CERTCTL_AGENT_ID=agent-demo-1 so the bundled
+          # agent doesn't restart-loop. The smoke's purpose (catch
+          # migration-on-cold-DB regressions + verify bootstrap-token
+          # endpoint mints a day-0 admin against a freshly migrated
+          # schema) is orthogonal to whether the auth posture is
+          # demo-mode or api-key, so the overlay is acceptable here.
+          COMPOSE_FILES=(-f docker-compose.yml -f docker-compose.demo.yml)
+
+          # Phase 2 SEC-H3 (2026-05-13): the demo overlay sets
+          # CERTCTL_DEMO_MODE_ACK=true; the SEC-H3 fail-closed guard
+          # requires a paired CERTCTL_DEMO_MODE_ACK_TS within the last
+          # 24h (a static YAML value would rot). The overlay reads
+          # ${CERTCTL_DEMO_MODE_ACK_TS:-} from the shell, so we mint a
+          # fresh timestamp here and export it for every compose
+          # invocation in this job (initial up-d AND the force-recreate
+          # at step 4).
+          export CERTCTL_DEMO_MODE_ACK_TS="$(date +%s)"
+
+          log "1/4 down -v --remove-orphans"
+          docker compose "${COMPOSE_FILES[@]}" down -v --remove-orphans 2>&1 | tail -3 || true
+
+          log "2/4 up -d (cold boot)"
+          docker compose "${COMPOSE_FILES[@]}" up -d 2>&1 | tail -3
+
+          log "3/4 wait for healthchecks"
+          wait_for_service_healthy postgres
+          wait_for_service_healthy certctl-server
+          wait_for_service_healthy certctl-agent || log "  (agent skipped)"
+
+          log "4/4 minting day-0 admin (proves migration ladder + bootstrap path)"
+          TOKEN="$(openssl rand -base64 32 | tr -d '\n')"
+          {
+            echo "CERTCTL_BOOTSTRAP_TOKEN=$TOKEN"
+            # Re-emit the demo-mode ACK TS into the --env-file so the
+            # force-recreate at step 4 inherits it. `--env-file` REPLACES
+            # the shell-env source for variable interpolation on compose
+            # operations that use it, so omitting this line would re-trip
+            # the SEC-H3 guard.
+            echo "CERTCTL_DEMO_MODE_ACK_TS=$CERTCTL_DEMO_MODE_ACK_TS"
+          } > /tmp/_smoke.env
+          docker compose "${COMPOSE_FILES[@]}" --env-file /tmp/_smoke.env up -d --force-recreate certctl-server 2>&1 | tail -2
+          sleep 5
+          wait_for_service_healthy certctl-server
+          BODY="$(http_call POST /api/v1/auth/bootstrap "{\"token\":\"$TOKEN\",\"actor_name\":\"smoke-admin\"}")"
+          KEY="$(echo "$BODY" | python3 -c 'import json,sys; print(json.load(sys.stdin)["key_value"])')"
+          [ -n "$KEY" ] || { log "bootstrap failed: $BODY"; exit 1; }
+
+          log "PASS — cold boot + force-recreate + admin bootstrap all green"
+          log "tearing down"
+          docker compose "${COMPOSE_FILES[@]}" down -v 2>&1 | tail -2
+
+      - name: Dump compose logs on failure
+        if: failure()
+        working-directory: deploy
+        run: |
+          for svc in postgres certctl-server certctl-agent certctl-tls-init; do
+            echo "==== $svc ===="
+            docker compose -f docker-compose.yml -f docker-compose.demo.yml logs --no-color --tail 200 "$svc" || true
+          done
+
   frontend-build:
     name: Frontend Build
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Set up Node.js
-        uses: actions/setup-node@v4
+        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020  # v4
         with:
           node-version: '22'
 
@@ -234,6 +413,17 @@ jobs:
         working-directory: web
         run: npm ci
 
+      - name: npm audit (production deps, high+critical)
+        # Phase 1 TEST-L2 closure (2026-05-13):
+        # Production frontend dependencies must not carry high or
+        # critical CVEs. Dev-only deps (vitest, vite, eslint, etc.)
+        # are excluded via --omit=dev since they never ship to
+        # operators. If this gate fires, triage each finding via npm
+        # overrides, dep upgrade, or a tracked --ignore with an issue
+        # link. Do not mass-silence findings.
+        working-directory: web
+        run: npm audit --omit=dev --audit-level=high
+
       - name: TypeScript Check
         working-directory: web
         run: npx tsc --noEmit
@@ -269,10 +459,10 @@ jobs:
     name: Helm Chart Validation
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Install Helm
-        uses: azure/setup-helm@v4
+        uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4  # v4
         with:
           version: '3.13.0'
 
@@ -280,15 +470,25 @@ jobs:
       # configured. Every lint/template invocation below must pick exactly one
       # provisioning mode — see deploy/helm/certctl/templates/_helpers.tpl
       # (certctl.tls.required) and docs/operator/tls.md.
+      #
+      # Bundle 3 closure (2026-05-12, commit f1fa311): the chart now ALSO
+      # fails render when (a) server.auth.type=api-key + apiKey empty, or
+      # (b) postgresql.enabled=true + postgresql.auth.password empty.
+      # Every positive render below MUST pass both secrets; inverse tests
+      # at the bottom of this job pin the fail-fast guards in place.
       - name: Lint Helm Chart
         run: |
           helm lint deploy/helm/certctl/ \
-            --set server.tls.existingSecret=certctl-tls-ci
+            --set server.tls.existingSecret=certctl-tls-ci \
+            --set server.auth.apiKey=ci-api-key-placeholder \
+            --set postgresql.auth.password=ci-postgres-placeholder
 
       - name: Template Helm Chart (existingSecret mode)
         run: |
           helm template certctl deploy/helm/certctl/ \
             --set server.tls.existingSecret=certctl-tls-ci \
+            --set server.auth.apiKey=ci-api-key-placeholder \
+            --set postgresql.auth.password=ci-postgres-placeholder \
             > /dev/null
 
       - name: Template Helm Chart (cert-manager mode)
@@ -296,8 +496,30 @@ jobs:
           helm template certctl deploy/helm/certctl/ \
             --set server.tls.certManager.enabled=true \
             --set server.tls.certManager.issuerRef.name=letsencrypt-prod \
+            --set server.auth.apiKey=ci-api-key-placeholder \
+            --set postgresql.auth.password=ci-postgres-placeholder \
             > /dev/null
 
+      - name: Template Helm Chart (external Postgres mode — Bundle 3 D2)
+        run: |
+          # Closes Bundle 3 D2: postgresql.enabled=false must (a) render
+          # cleanly with externalDatabase.url and (b) emit ZERO postgres-*
+          # templates. The render output is grep-checked below.
+          out=$(helm template certctl deploy/helm/certctl/ \
+            --set server.tls.existingSecret=certctl-tls-ci \
+            --set postgresql.enabled=false \
+            --set externalDatabase.url='postgres://u:p@db.example.com:5432/certctl?sslmode=require' \
+            --set server.auth.apiKey=ci-api-key-placeholder)
+          # Bundled-Postgres resources must not appear when postgresql.enabled=false.
+          if echo "$out" | grep -qE "^kind: StatefulSet$"; then
+            echo "::error::Bundle 3 D2 regression: postgres StatefulSet rendered with postgresql.enabled=false"
+            exit 1
+          fi
+          if echo "$out" | grep -q "postgres-secret.yaml"; then
+            echo "::error::Bundle 3 D2 regression: postgres-secret rendered with postgresql.enabled=false"
+            exit 1
+          fi
+
       - name: Template Helm Chart (guard fails without TLS)
         run: |
           # Inverse test: the chart MUST refuse to render when no TLS source is
@@ -308,6 +530,58 @@ jobs:
             exit 1
           fi
 
+      - name: Template Helm Chart (guard fails — Bundle 3 D7 TLS both-set)
+        run: |
+          # Bundle 3 D7: setting BOTH existingSecret AND certManager.enabled
+          # creates two conflicting TLS sources of truth. Chart must refuse.
+          if helm template certctl deploy/helm/certctl/ \
+                --set server.tls.existingSecret=ci \
+                --set server.tls.certManager.enabled=true \
+                --set server.tls.certManager.issuerRef.name=foo \
+                --set server.auth.apiKey=k \
+                --set postgresql.auth.password=p \
+                > /dev/null 2>&1; then
+            echo "::error::Bundle 3 D7 regression: chart rendered with BOTH TLS sources configured"
+            exit 1
+          fi
+
+      - name: Template Helm Chart (guard fails — Bundle 3 D1 missing apiKey)
+        run: |
+          # Bundle 3 D1: missing server.auth.apiKey when auth.type=api-key
+          # must fail at template time, not silently render an empty Secret.
+          if helm template certctl deploy/helm/certctl/ \
+                --set server.tls.existingSecret=ci \
+                --set postgresql.auth.password=p \
+                > /dev/null 2>&1; then
+            echo "::error::Bundle 3 D1 regression: chart rendered with empty server.auth.apiKey"
+            exit 1
+          fi
+
+      - name: Template Helm Chart (guard fails — Bundle 3 D1 missing pg password)
+        run: |
+          # Bundle 3 D1: missing postgresql.auth.password when postgresql.enabled=true
+          # must fail at template time, not silently use a fallback default.
+          if helm template certctl deploy/helm/certctl/ \
+                --set server.tls.existingSecret=ci \
+                --set server.auth.apiKey=k \
+                > /dev/null 2>&1; then
+            echo "::error::Bundle 3 D1 regression: chart rendered with empty postgresql.auth.password"
+            exit 1
+          fi
+
+      - name: Template Helm Chart (guard fails — Bundle 3 D1 missing external DB URL)
+        run: |
+          # Bundle 3 D1: missing externalDatabase.url when postgresql.enabled=false
+          # must fail at template time.
+          if helm template certctl deploy/helm/certctl/ \
+                --set server.tls.existingSecret=ci \
+                --set postgresql.enabled=false \
+                --set server.auth.apiKey=k \
+                > /dev/null 2>&1; then
+            echo "::error::Bundle 3 D1 regression: chart rendered with postgresql.enabled=false + empty externalDatabase.url"
+            exit 1
+          fi
+
   # =============================================================================
   # deploy-vendor-e2e — single-job (collapsed from 12-job matrix)
   # =============================================================================
@@ -338,12 +612,12 @@ jobs:
     needs: [go-build-and-test]
     timeout-minutes: 30
     steps:
-      - uses: actions/checkout@v5
+      - uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd  # v5
 
       - name: Set up Go
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
         with:
-          go-version: '1.25.9'
+          go-version: '1.25.10'
           cache: true
 
       - name: Build f5-mock-icontrol sidecar
@@ -435,12 +709,12 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 15
     steps:
-      - uses: actions/checkout@v5
+      - uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd  # v5
 
       - name: Set up Go
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
         with:
-          go-version: '1.25.9'
+          go-version: '1.25.10'
           cache: true
 
       - name: Digest validity (every @sha256 ref must resolve)

.github/workflows/codeql.yml

@@ -53,17 +53,17 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Set up Go
         if: matrix.language == 'go'
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
         with:
           # Match ci.yml + release.yml + security-deep-scan.yml.
-          go-version: '1.25.9'
+          go-version: '1.25.10'
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@v3
+        uses: github/codeql-action/init@7fd177fa680c9881b53cdab4d346d32574c9f7f4  # v3
         with:
           languages: ${{ matrix.language }}
           # Use the security-and-quality query suite — security finds plus
@@ -72,10 +72,10 @@ jobs:
           queries: security-and-quality
 
       - name: Autobuild
-        uses: github/codeql-action/autobuild@v3
+        uses: github/codeql-action/autobuild@7fd177fa680c9881b53cdab4d346d32574c9f7f4  # v3
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v3
+        uses: github/codeql-action/analyze@7fd177fa680c9881b53cdab4d346d32574c9f7f4  # v3
         with:
           category: "/language:${{ matrix.language }}"
           # SARIF upload is implicit (and is what populates the Security tab).

.github/workflows/loadtest.yml

@@ -49,13 +49,13 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Set up Docker Buildx
         # The compose stack builds the certctl image from the repo
         # root Dockerfile. Buildx gives the build a usable cache and
         # works with newer compose versions.
-        uses: docker/setup-buildx-action@v3
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f  # v3
 
       - name: Run loadtest
         run: make loadtest
@@ -70,7 +70,7 @@ jobs:
         # authoritative machine-readable form; summary.txt is the
         # human-readable text the README baseline tracks.
         if: always()
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
         with:
           name: k6-summary-${{ github.run_id }}
           path: deploy/test/loadtest/results/

.github/workflows/release.yml

@@ -15,7 +15,7 @@ on:
 env:
   REGISTRY: ghcr.io
   # Keep in lock-step with .github/workflows/ci.yml (M-3).
-  GO_VERSION: '1.25.9'
+  GO_VERSION: '1.25.10'
   IMAGE_NAMESPACE: certctl-io
 
 jobs:
@@ -39,10 +39,10 @@ jobs:
         os: [linux, darwin]
         arch: [amd64, arm64]
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Set up Go
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
         with:
           go-version: ${{ env.GO_VERSION }}
 
@@ -123,7 +123,7 @@ jobs:
           cat "${OUTPUT_NAME}.sha256"
 
       - name: Upload build artefacts
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
         with:
           name: binary-${{ steps.build.outputs.output_name }}
           path: |
@@ -151,7 +151,7 @@ jobs:
       hashes: ${{ steps.hashes.outputs.hashes }}
     steps:
       - name: Download binary artefacts
-        uses: actions/download-artifact@v4
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093  # v4
         with:
           pattern: binary-*
           path: artifacts
@@ -191,7 +191,7 @@ jobs:
             checksums.txt
 
       - name: Upload artefacts to GitHub Release
-        uses: softprops/action-gh-release@v2
+        uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65  # v2
         if: startsWith(github.ref, 'refs/tags/')
         with:
           files: |
@@ -212,11 +212,24 @@ jobs:
       actions: read
       id-token: write
       contents: write
-    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v2.1.0
+    uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@f7dd8c54c2067bafc12ca7a55595d5ee9b75204a  # v2.1.0
     with:
       base64-subjects: "${{ needs.aggregate-checksums.outputs.hashes }}"
       upload-assets: true
       provenance-name: multiple.intoto.jsonl
+      # Phase 1 RED-2 compat (2026-05-14): the SLSA reusable workflow's
+      # default path downloads a pre-built generator binary from a
+      # GitHub *release* of slsa-framework/slsa-github-generator —
+      # releases are keyed by tag name (vX.Y.Z), and the workflow
+      # rejects SHA-form refs with "Expected ref of the form
+      # refs/tags/vX.Y.Z". Phase 1 RED-2 SHA-pinned every Actions
+      # uses: line, so the default path errors out. Setting
+      # compile-generator: true instead builds the generator from the
+      # pinned-SHA source inside the workflow run — preserves
+      # supply-chain integrity (SHA pin retained), adds ~1 min build
+      # time. This is the SLSA project's documented escape hatch for
+      # SHA-pinned reusable-workflow consumers.
+      compile-generator: true
 
   # ----------------------------------------------------------------------
   # build-and-push-docker: push container images to GHCR with native
@@ -235,10 +248,10 @@ jobs:
       id-token: write  # Cosign keyless OIDC identity token
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Log in to GitHub Container Registry
-        uses: docker/login-action@v3
+        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9  # v3
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}
@@ -249,14 +262,14 @@ jobs:
         run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> "$GITHUB_OUTPUT"
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f  # v3
 
       - name: Install Cosign
         uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003  # v4.1.1
 
       - name: Build and push server image
         id: server-push
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8  # v6
         with:
           context: .
           file: ./Dockerfile
@@ -291,7 +304,7 @@ jobs:
 
       - name: Build and push agent image
         id: agent-push
-        uses: docker/build-push-action@v6
+        uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8  # v6
         with:
           context: .
           file: ./Dockerfile.agent
@@ -334,7 +347,7 @@ jobs:
       contents: write
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
       - name: Extract version from tag
         id: version
@@ -351,7 +364,7 @@ jobs:
         # README is the source of truth for those, and inlining them in every
         # release page produces the kind of "every release looks identical"
         # noise that gives operators no signal about what actually changed.
-        uses: softprops/action-gh-release@v2
+        uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65  # v2
         with:
           # Pin the release title to the tag name. softprops/action-gh-release@v2
           # falls back to the most recent commit subject when `name:` is omitted,

.github/workflows/security-deep-scan.yml

@@ -36,9 +36,9 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 60
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
 
-      - uses: actions/setup-go@v5
+      - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff  # v5
         with:
           go-version: '1.25'
 
@@ -48,15 +48,26 @@ jobs:
 
       # --- Static analysis (slow paths) ---
 
-      - name: gosec
+      - name: gosec (G201/G202/G304/G108 subset — Phase 3 TEST-M2 hard gate)
-        run: |
+        # Phase 3 TEST-M2 closure (2026-05-13): gosec promoted from
-          $(go env GOPATH)/bin/gosec -fmt sarif -out gosec.sarif ./... || true
+        # continue-on-error (advisory) to blocking on the 4 high-signal
-        continue-on-error: true
+        # rule subset that targets real prod-bug classes:
+        #   G201 = SQL string formatting (SQL injection)
+        #   G202 = SQL string concatenation (SQL injection)
+        #   G304 = file-path traversal via tainted input
+        #   G108 = profiling endpoint exposed
+        # Other gosec rules (G1xx-G7xx broadly) remain in the SARIF
+        # report but don't gate the build — they have higher false-
+        # positive rates than these 4.
+        run: $(go env GOPATH)/bin/gosec -fmt sarif -out gosec.sarif -include=G201,G202,G304,G108 ./...
 
-      - name: osv-scanner (multi-ecosystem CVE)
+      - name: osv-scanner (multi-ecosystem CVE — Phase 3 TEST-M2 hard gate)
-        run: |
+        # Phase 3 TEST-M2 closure (2026-05-13): osv-scanner promoted from
-          $(go env GOPATH)/bin/osv-scanner -r --format json --output osv-scanner.json . || true
+        # advisory to blocking. Complements govulncheck (already blocking
-        continue-on-error: true
+        # in ci.yml) by covering non-Go dependencies (npm under web/,
+        # any docker base image deps). Findings fail the build; the
+        # exact CVE list lands in osv-scanner.json as a receipt either way.
+        run: $(go env GOPATH)/bin/osv-scanner -r --format json --output osv-scanner.json .
 
       # --- Race detector at -count=10 (D-002) ---
 
@@ -90,14 +101,39 @@ jobs:
         run: go install github.com/zimmski/go-mutesting/cmd/go-mutesting@latest
         continue-on-error: true
 
-      - name: go-mutesting (crypto cluster)
+      - name: go-mutesting (crypto cluster — Phase 3 TEST-M1 hard gate at 55%)
+        # Phase 3 TEST-M1 closure (2026-05-13): go-mutesting promoted
+        # from advisory (continue-on-error + per-package `|| true`) to
+        # blocking with an explicit mutation-score floor of 55%.
+        # Per-package summary lines emit `The mutation score is X.YZ`;
+        # the awk filter extracts each, and the post-loop check fails
+        # the step if any package drops below 0.55.
+        #
+        # Floor rationale: 55% is the starter ratio that catches major
+        # regressions without rejecting the audit's "this is OK" steady
+        # state. Raise quarterly as the test suite hardens; the floor
+        # change ships in the same commit that adds the strengthening
+        # tests so the ratchet is documented.
         run: |
+          set -e
           : > go-mutesting.txt
           for pkg in ./internal/crypto/... ./internal/pkcs7/... ./internal/connector/issuer/local/...; do
             echo "=== $pkg ===" | tee -a go-mutesting.txt
-            $(go env GOPATH)/bin/go-mutesting "$pkg" 2>&1 | tee -a go-mutesting.txt || true
+            $(go env GOPATH)/bin/go-mutesting "$pkg" 2>&1 | tee -a go-mutesting.txt
           done
-        continue-on-error: true
+          # Extract every "The mutation score is X.YZ" line; fail on any
+          # score below 0.55. The check works against floats via awk so
+          # 0.55 is the literal threshold (not a percentage).
+          floor=0.55
+          fail=0
+          while IFS= read -r score; do
+            ok=$(awk -v s="$score" -v f="$floor" 'BEGIN{print (s>=f) ? 1 : 0}')
+            if [ "$ok" -ne 1 ]; then
+              echo "::error::mutation score $score below floor $floor"
+              fail=1
+            fi
+          done < <(grep -oE "The mutation score is [0-9.]+" go-mutesting.txt | awk '{print $NF}')
+          exit $fail
 
       # --- Container + supply chain (D-001 partial, D-006 partial) ---
 
@@ -105,11 +141,21 @@ jobs:
         run: docker build -t certctl:deep-scan .
         continue-on-error: true
 
-      - name: trivy image scan
+      - name: trivy image scan (HIGH+CRITICAL — Phase 3 TEST-M2 hard gate)
+        # Phase 3 TEST-M2 closure (2026-05-13): trivy promoted from
+        # advisory to blocking. --severity filter keeps the gate
+        # noise-free (LOW + MEDIUM findings stay in the JSON receipt
+        # but don't fail the build); --exit-code 1 makes HIGH+CRITICAL
+        # findings the actual gate. Trivy is the third hard deep-scan
+        # gate (alongside gosec + osv-scanner); ZAP / schemathesis /
+        # nuclei / testssl stay advisory because their false-positive
+        # rates on https://localhost:8443-targeted DAST runs are high.
         run: |
           docker run --rm -v "$PWD":/src aquasec/trivy:latest image \
-            --format json --output /src/trivy.json certctl:deep-scan || true
+            --format json --output /src/trivy.json \
-        continue-on-error: true
+            --severity HIGH,CRITICAL \
+            --exit-code 1 \
+            certctl:deep-scan
 
       - name: syft SBOM
         run: |
@@ -126,7 +172,7 @@ jobs:
         continue-on-error: true
 
       - name: ZAP baseline
-        uses: zaproxy/action-baseline@v0.10.0
+        uses: zaproxy/action-baseline@1e1871e84428617b969d4a1f981a8255630d54b0  # v0.10.0
         with:
           target: 'https://localhost:8443'
         continue-on-error: true
@@ -175,7 +221,7 @@ jobs:
       # --- Upload everything as artefacts ---
 
       - name: Upload deep-scan receipts
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
         if: always()
         with:
           name: security-deep-scan-${{ github.run_id }}

.gitignore

@@ -88,3 +88,17 @@ Thumbs.db
 # CERTCTL_TEST_CA_BUNDLE=./certs/ca.crt. Material is regenerated on every
 # `docker compose up` and never belongs in git.
 /deploy/test/certs/
+
+# Phase 1 RED-1 closure (2026-05-13): the f5-mock-icontrol Dockerfile
+# rebuilds from source via multi-stage build (deploy/test/f5-mock-icontrol/
+# Dockerfile line 13). The compiled ELF must not be tracked.
+deploy/test/f5-mock-icontrol/f5-mock-icontrol
+
+# Phase 0 closure (2026-05-13): cowork/ holds the operator's internal
+# legal / audit / strategy artifacts (counsel-signed AI-authorship
+# declaration, filter-repo callback, pre-rewrite bundle, audit HTML
+# scratch). It is private operator scratch space and must never
+# accidentally land in the public repo. See
+# docs/history-normalization.md for the public-facing description of
+# the Phase 0 git-history rewrite.
+cowork/

CHANGELOG.md

@@ -1,8 +1,771 @@
 # Changelog
 
-## v2.0.68 — Image registry path changed ⚠️
+## Unreleased
 
-> **Image registry path changed.** Starting this release, container images publish to `ghcr.io/certctl-io/certctl-server` and `ghcr.io/certctl-io/certctl-agent`. Existing pulls from `ghcr.io/shankar0123/certctl-{server,agent}:<tag>` continue to work for previously-published tags (the registry never deletes images), but the `:latest` tag at the old path stops moving forward at this release. Update your `docker pull` paths, `docker-compose.yml` `image:` keys, or Helm `image.repository` values to receive future updates. Old `git clone` / `git push` / install-script / API URLs continue to redirect forever — only the container-registry path changed.
+### Breaking changes (scheduled for v2.2.0)
+
+- **SEC-H1 staged: `CERTCTL_AGENT_BOOTSTRAP_TOKEN_DENY_EMPTY` opt-in flag.**
+  Phase 2 of the architecture diligence remediation (2026-05-13) introduces
+  a new env var that, when set to `true`, makes the server refuse to start
+  unless `CERTCTL_AGENT_BOOTSTRAP_TOKEN` is also set to a real value.
+  Default in this release: `false` (preserves the v2.1.x warn-mode
+  pass-through behavior for backward compatibility). Default flip to
+  `true` is scheduled for v2.2.0 per `WORKSPACE-ROADMAP.md`.
+
+  **Operator action before the v2.2.0 upgrade:** generate a real
+  bootstrap token (`openssl rand -base64 32`) and set
+  `CERTCTL_AGENT_BOOTSTRAP_TOKEN` in your env. When v2.2.0 ships, the
+  deny-empty default flips to `true` and a missing or empty token will
+  fail closed at boot. Operators with the token already set: no action
+  required.
+
+- **SEC-M4: `CERTCTL_ACME_INSECURE` now requires explicit ACK.**
+  Pre-Phase-2, `CERTCTL_ACME_INSECURE=true` produced only a boot-time
+  WARN log. Post-Phase-2 (THIS release), the server refuses to start
+  unless `CERTCTL_ACME_INSECURE_ACK=true` is set alongside it. ACME
+  directory TLS verification is the load-bearing defense against a
+  network attacker intercepting ACME enrollment; the existing flag was
+  too easy to flip via a copy-pasted Pebble runbook.
+
+  **Operator action:** if you intentionally run against a self-signed
+  ACME server (Pebble, step-ca, internal dev), add
+  `CERTCTL_ACME_INSECURE_ACK=true` to your env. Production deploys
+  MUST never set either flag.
+
+- **SEC-H3: `CERTCTL_DEMO_MODE_ACK` is no longer sticky — 24h re-ack required.**
+  Pre-Phase-2, setting `CERTCTL_DEMO_MODE_ACK=true` was sticky for the
+  lifetime of the container. Post-Phase-2, operators must ALSO set
+  `CERTCTL_DEMO_MODE_ACK_TS=$(date +%s)` to a unix epoch within the
+  last 24h. The next container restart past 24h refuses to start
+  unless a fresh TS is supplied. Catches the "forgotten demo deployment
+  promoted to production" failure mode.
+
+  **Operator action:** demo deploys must set `CERTCTL_DEMO_MODE_ACK_TS`
+  at every `docker compose up`. The demo Compose helper script handles
+  this automatically when wired; standalone demo deploys add it
+  manually. Production deploys: this guard is irrelevant
+  (`CERTCTL_DEMO_MODE_ACK` should not be set in production).
+
+### Security
+
+- **Alg-downgrade defense relaxed for Keycloak-shape IdPs (v2.1.0 pre-tag fix).**
+  Pre-fix, the IdP-bind alg-downgrade check at `internal/auth/oidc/service.go`
+  refused to load any OIDC provider whose discovery doc advertised HS256 /
+  HS384 / HS512 / `none` in `id_token_signing_alg_values_supported` —
+  even if RS256 was ALSO advertised. This broke binding against
+  Keycloak 26.x (and a handful of other real IdPs) which list every alg
+  the codebase is capable of in their discovery doc, regardless of which
+  one the realm actually signs with. The v2.1.0 Phase-10 live-IdP smoke
+  surfaced the regression: 6 testcontainers-Keycloak integration tests
+  failed with `oidc: IdP advertises weak signing algorithms (HS*/none); refusing to use as defense against downgrade attacks: HS256`.
+  **Fix:** the check now refuses only when the intersection of advertised
+  vs `DefaultAllowedAlgs` is EMPTY — an IdP advertising HS256 alongside
+  RS256 binds successfully, but an IdP advertising HS-only / none-only
+  still fails closed. The per-token alg pin at sig-verify time
+  (`isDisallowedAlg`, service.go ~L1177) remains the load-bearing defense
+  against the actual algorithm-confusion attack (forged HS256 token
+  signed with the IdP's RS256 pubkey as HMAC secret) — go-oidc/v3's
+  verifier rejects any token whose `alg` header isn't in the configured
+  allow-list, regardless of what the discovery doc claims. Updates:
+  `Service.getOrLoad` alg-check loop rewritten to compute intersection;
+  `ErrIdPDowngradeAdvertised` docstring reflects new semantics;
+  `TestDiscovery` dry-run validator surfaces HS*/none alongside RS* as
+  an informational note (not a hard fail); `docs/operator/auth-threat-model.md`
+  alg-allow-list section updated to call out the load-bearing-defense
+  hierarchy. Tests: `TestService_IdPDowngradeDefense_RS256PlusHS256_BindsSuccessfully`
+  (positive — Keycloak-shape) + `TestService_IdPDowngradeDefense_RejectsHSOnlyAdvertised`
+  (negative — pathological intersection-empty case) +
+  `TestService_RefreshKeys_CatchesPostLoadDowngrade` updated to assert
+  intersection-empty post-rotation; `TestTestDiscovery_AlgDowngrade_HS256AlongsideRS256_BindsWithNote`
+  + `TestTestDiscovery_AlgDowngrade_HSOnly_StillTrips_HardFail` pin the
+  dry-run validator's new behavior.
+
+### Tests
+
+- **Vitest coverage for the 2026-05-10/11 GUI batch (Audit 2026-05-11 Fix 12).**
+  The original GUI-batch commit `661b6db` claimed `npx tsc --noEmit PASS`
+  but shipped no Vitest cases for the new surfaces. The regression-
+  prevention layer was missing — a future refactor of `KeysPage`'s
+  assign modal could silently drop scope_type handling, the LOW-1 demo
+  banner could be hidden by a stray predicate flip, the LOW-11 hide of
+  the delete button on default roles could disappear and let operators
+  click straight into a backend 409, and nothing would surface in CI.
+  This closure adds 35 new test cases across five files:
+  `web/src/pages/auth/UsersPage.test.tsx` (new, 8 cases pinning the
+  active/deactivated/reactivate flow + provider filter + empty state +
+  loading state), `web/src/pages/auth/AuthSettingsPage.test.tsx`
+  (extended +4 cases pinning the MED-12 runtime-config panel —
+  alphabetical sort, `(empty)` placeholder, 403 silent-hide),
+  `web/src/pages/auth/KeysPage.test.tsx` (extended +8 cases pinning
+  the HIGH-10 GUI half — scope_type=global/profile/issuer body shape,
+  expires_at omission vs RFC3339 promotion, whitespace-only scope_id
+  rejection, demo-anon row mutation-button hide),
+  `web/src/pages/auth/RoleDetailPage.test.tsx` (new, 9 cases pinning
+  the MED-8 scope picker + the LOW-11 default-role delete-button hide
+  via the `DEFAULT_ROLE_IDS` set against `r-admin` + `r-auditor`),
+  `web/src/components/AuthProvider.test.tsx` (new, 5 cases pinning the
+  LOW-1 demo-banner visibility predicate — `authType==='none' &&
+  !loading` — across happy/api-key/oidc/loading/rejected branches; the
+  rejected-fetch path keeps the banner visible because the catch
+  treats it as an old-server-fallback to demo-mode, and that behavior
+  is pinned here so a future change surfaces in the diff). 40/40
+  test-file-scoped pass; `tsc --noEmit` clean.
+
+### Security
+
+- **CSRF rotation on logout closes HIGH-2 fourth call site (Audit 2026-05-11 Fix 13).**
+  The HIGH-2 closure (`dev/auth-bundle-2`) documented four
+  `RotateCSRFTokenForActor` call sites: login completion (fresh by
+  construction), Assign/RevokeRole on role-mutation (wired), Logout, and
+  an explicit operator endpoint. The 2026-05-11 review verified only 3
+  of the 4 — Logout did NOT rotate the actor's sibling sessions
+  post-revoke, leaving a window where a token captured pre-logout
+  (browser DevTools, malicious extension, session-storage leak) could
+  be replayed against the user's other-device/other-browser sessions
+  until those sessions hit their own idle/absolute expiry.
+  `SessionMinter` interface extended with `RotateCSRFTokenForActor`;
+  `Logout` invokes it after `Revoke(sess.ID)` succeeds. The
+  `auth.session_revoked` audit row gains a `csrf_rotated` detail key
+  carrying the rotated count so SOC / SIEM can correlate logout events
+  with CSRF churn. The no-cookie + invalid-cookie 204 short-circuit
+  paths skip rotation (no session row to rotate against). 3 regression
+  tests in `internal/api/handler/auth_session_oidc_test.go` pin the
+  happy path + the two short-circuit branches. The explicit operator
+  endpoint (4) remains intentionally unbuilt — the three automatic
+  triggers (login + role-mutation + logout) cover the threat model;
+  operators who want a nuclear option can use the existing
+  `RevokeAllForActor` flow which forces re-login → fresh session →
+  fresh CSRF. **HIGH-2 fully closed across all four documented call
+  sites.**
+
+- **Demo-mode residual-grants detector + cleanup endpoint + CI guard (Audit 2026-05-11 A-8).**
+  HIGH-12 (closure `b81588e`) added a fail-closed bind-address guard
+  that refuses startup when `CERTCTL_AUTH_TYPE=none` binds non-loopback
+  without `CERTCTL_DEMO_MODE_ACK=true`. The Phase 2 leg of that spec —
+  production-startup banner when `actor-demo-anon` has residual role
+  grants in `actor_roles` plus a CI guard banning new synthetic-admin
+  code paths — was deferred. This closure lands all three deferred
+  legs. (1) `cmd/server/preflight_demo_residual.go` runs after the DB
+  is open + audit service is constructed, before the HTTPS listener
+  starts; under any non-`none` auth type it queries `actor_roles` for
+  `actor-demo-anon` and emits a WARN log + `auth.demo_residual_grants_detected`
+  audit row when the row is present. The migration 000029 baseline
+  unconditionally seeds the `ar-demo-anon-admin` row at install time,
+  so EVERY production deploy will see this WARN on first boot — the
+  intended cutover workflow is documented at `docs/operator/security.md`.
+  (2) `POST /api/v1/auth/demo-residual/cleanup` is an admin-class
+  (`auth.role.assign`) cleanup endpoint that removes every
+  `actor-demo-anon` row from `actor_roles` and returns
+  `{"removed": <int64>}`; idempotent (a second call returns
+  `removed:0`), refuses 503 under `Auth.Type=none` (deleting the row
+  would break the demo path), audit-logs every invocation. (3) New
+  env var `CERTCTL_DEMO_MODE_RESIDUAL_STRICT` (default `false`)
+  pivots the WARN to fail-closed startup refusal for operators who
+  want a paranoid hostile-environment posture. (4) CI guard
+  `scripts/ci-guards/no-new-synthetic-admin.sh` pins the 17-entry
+  allowlist of source files that may reference the `actor-demo-anon`
+  literal; new runtime code paths that resolve to the synthetic actor
+  are rejected at PR time so the credibility gap stays closed. The
+  closure was framed as "credibility gap, not exploitable
+  vulnerability" — the residue requires a regression elsewhere in the
+  middleware chain to be exploitable. After this fix, the canonical
+  acquisition-readiness narrative ("RBAC primitive with no
+  synthetic-admin fallback") is fully true. Operator runbook at
+  `docs/operator/security.md#demo-to-production-cutover-audit-2026-05-11-a-8`.
+
+- **OIDC provider "Test connection" panel (Audit 2026-05-11 Fix 09 — MED-5 GUI half).**
+  MED-5's backend dry-run endpoint (`POST /api/v1/auth/oidc/test`, gated
+  `auth.oidc.create`) shipped on `dev/auth-bundle-2` but had no GUI caller —
+  the `authOIDCTestProvider` function in `web/src/api/client.ts` was dead
+  code. Operators had to complete the create form blind, save, then click
+  "Refresh" to discover whether the issuer URL worked; failures left a
+  broken provider row in the database that had to be deleted before
+  retrying. New shared component
+  `web/src/pages/auth/OIDCTestConnectionPanel.tsx` calls the backend
+  against the live form state and renders a four-row status panel inline:
+  Discovery fetched, JWKS reachable, supported algs (warns when the IdP
+  advertises none), and RFC 9207 iss-parameter advertisement (informational
+  `·` glyph, not ✗, because the spec is SHOULD). Backend per-leg `errors[]`
+  flow into an inline bullet list. The panel is mounted in the
+  OIDCProvidersPage create modal AND the OIDCProviderDetailPage edit form —
+  the edit-form half is load-bearing for verifying IdP rotations (Keycloak
+  realm rename, Okta tenant move) without committing first. Run button is
+  disabled until the issuer URL is non-empty (whitespace-trimmed); the
+  component is read-only — safe to run repeatedly. 8 Vitest tests pin the
+  glyph-vs-glyph contract (✓/✗/⚠/·), the button-disabled-without-issuer
+  shape, and the test-id-suffix collision-prevention when the panel is
+  mounted twice on the same page.
+
+- **OIDC JWKS health panel + Refresh-now button (Audit 2026-05-11 Fix 10 — MED-7 GUI half).**
+  MED-7's backend endpoint `GET /api/v1/auth/oidc/providers/{id}/jwks-status`
+  (commit `d85114f`) shipped the per-provider verifier counters on
+  `dev/auth-bundle-2` but the GUI never called it. The audit doc had
+  prematurely flipped the row to CLOSED; `authOIDCJWKSStatus` in the
+  API client was dead code. Operators investigating "why is login
+  failing for this IdP" couldn't see `last_refresh_at`,
+  `rejected_jws_count`, or `last_error` from the GUI — they had to
+  drop to curl. New shared component
+  `web/src/pages/auth/OIDCJWKSStatusPanel.tsx` queries the endpoint
+  via TanStack Query (30s `staleTime`, `retry: 0` so a 403 hides the
+  panel silently for callers without `auth.oidc.list`) and renders
+  six dt/dd rows: Last refresh (with `(never — cold cache)` sentinel
+  when the timestamp is empty), Refresh count, Rejected JWS count,
+  Last error (red treatment when non-empty, `(none)` sentinel
+  otherwise), RFC 9207 iss param ("supported by IdP" / "not
+  advertised"), and Current KIDs (`(not exposed — query jwks_uri
+  directly)` sentinel when the backend declines to expose the list).
+  A "Refresh now" button invokes the existing
+  `POST .../refresh` (RefreshKeys path) and invalidates the panel's
+  query so the freshly-updated counters render without a page
+  reload. The button is hidden for callers without `auth.oidc.edit`
+  via the panel's optional `canRefresh` prop. Mounted on
+  `OIDCProviderDetailPage.tsx` between the read-only field display
+  and the Actions section. 9 Vitest tests pin: loading state,
+  happy-path-all-six-rows, 403-hides-panel, refresh-invalidates-
+  query, refresh-failure-surfaces-inline-without-hiding-panel,
+  never-refreshed-cold-cache-sentinel, current-kids-empty-not-
+  exposed-sentinel, last-error-red-treatment, and canRefresh=false-
+  hides-the-button.
+
+- **UsersPage sidebar nav entry (Audit 2026-05-11 Fix 11 — MED-11
+  discoverability).** The MED-11 closure shipped `UsersPage.tsx` + wired
+  the `/auth/users` route in `web/src/main.tsx`, but the sidebar
+  navigation never gained a corresponding entry. Operators reached the
+  federated-user-admin surface (used during compliance audits — "show
+  me last login for every IdP-federated user") only by knowing the URL.
+  A page that exists but isn't navigable is a half-finished page. New
+  Users entry under the Auth section in `web/src/components/Layout.tsx`
+  sits between Sessions and Roles (federated-identity grouping). Three
+  Vitest tests in `Layout.test.tsx` pin the link's presence, the
+  `/auth/users` destination, and the DOM ordering relative to Sessions
+  so a future refactor that re-orders or removes the entry surfaces in
+  the diff.
+
+- **Scope-aware actor-role revoke (Audit 2026-05-11 A-4).**
+  HIGH-10 made it possible to grant the same role to the same actor at
+  multiple scopes (e.g. `r-operator` on `profile=p-acme` AND `profile=p-globex`)
+  via the unique constraint extension on `actor_roles`, but
+  `ActorRoleRepository.Revoke` ignored `(scope_type, scope_id)` and
+  unconditionally deleted every variant. Operators who wanted to drop
+  one scoped grant had to nuke them all and re-grant the remainder —
+  a race window where the actor's access was briefly different. The
+  `DELETE /v1/auth/keys/{id}/roles/{role_id}` endpoint now accepts
+  optional `?scope_type=` / `?scope_id=` query params that narrow the
+  revoke to a single variant; no-match returns 404. The legacy "revoke
+  every variant" semantic is preserved when the query params are
+  absent, so existing CLI / GUI buttons keep working unchanged. The
+  audit row's `details` payload records which mode fired so SOC / SIEM
+  can distinguish wide cleanups from targeted demotions. MCP tool
+  `certctl_auth_revoke_role_from_key` gains optional `scope_type` +
+  `scope_id` input fields with matching semantics. Documented in
+  `docs/operator/rbac.md` under "Revoke: legacy 'all variants' vs
+  scope-selective."
+
+### Security (BREAKING — silent-elevation closure)
+
+- **HIGH-10 actor-role scope is now enforced (Audit 2026-05-11 A-1).**
+  Pre-fix, `actor_roles.scope_type` / `scope_id` (added in migration 000043
+  by the HIGH-10 closure) were persisted by Grant + accepted on the handler
+  body + surfaced through the GUI/MCP — but the load-bearing
+  `EffectivePermissions` SQL never read them. A profile-scoped grant
+  silently elevated to global at authorization time. Canonical CRIT-5
+  lying-field shape, replicated. **The post-fix authorization narrows
+  correctly**: every existing `actor_roles` row with `scope_type != 'global'`
+  now takes effect.
+
+  > **Operator advisory:** if you used the HIGH-10 scope-bound role-grant
+  > API between commit `551812b` and the v2.1.0 tag (the column was
+  > populated but ignored), the grants were silently global. After
+  > upgrading, audit `SELECT actor_id, role_id, scope_type, scope_id FROM
+  > actor_roles WHERE scope_type != 'global'` and confirm the narrowing
+  > reflects intent. If an actor was granted a scoped role but expected
+  > global behavior, re-grant with `scope_type=global`.
+
+### Security (BREAKING)
+
+- **Federated-user deactivation now actually blocks login (Audit 2026-05-11 A-2).**
+  The MED-11 closure shipped `users.deactivated_at` + `DELETE /api/v1/auth/users/{id}`
+  + cascade-session-revoke, but the column was a "lying field" three legs over: the
+  postgres user repository never SELECTed it (so `User.DeactivatedAt` always read
+  nil), the `Update` SQL never wrote it (so the handler's mutation was a no-op),
+  and the OIDC `upsertUser` path never checked it (so the next login under the
+  same `(provider, subject)` tuple re-minted a session and re-elevated the user).
+  The cascade-revoke remained correct for the current cookie only. **Operator
+  advisory: if you deactivated a federated user between the MED-11 closure
+  (Bundle 2 merge `dea5053`) and the v2.1.0 release tag, verify the user cannot
+  OIDC-log-in after upgrading — the column took no effect at login time before
+  this fix. If needed, re-run the deactivation against the upgraded server.**
+  Closure: `userColumns` + `scanUser` now read `deactivated_at` via `sql.NullTime`;
+  `Create` + `Update` write it explicitly; `upsertUser` returns the new
+  `ErrUserDeactivated` sentinel before mutating fields (preserves `last_login_at`
+  forensics on rejected logins); `classifyOIDCFailure` surfaces the rejection
+  as audit category `user_deactivated`. Self-deactivate guard on
+  `DELETE /api/v1/auth/users/{id}` returns HTTP 409 + audit row
+  `auth.user_deactivate_self_rejected` (prevents an admin from one-way-door
+  locking themselves out via the standard handler — break-glass remains the
+  recovery path). New inverse endpoint `POST /api/v1/auth/users/{id}/reactivate`
+  (gated `auth.user.deactivate` — reactivation is the inverse op, not a separate
+  privilege) clears `deactivated_at`; emits audit row `auth.user_reactivated`.
+  Sessions revoked at deactivation stay revoked across reactivation — the user
+  must complete a fresh OIDC login. GUI: `UsersPage.tsx` now renders a Reactivate
+  button on deactivated rows. CWE-862 (missing authorization at the user-state
+  boundary). SOC 2 CC6.3 + ISO 27001 A.9.2.6 compliance-table-flipping fix.
+- **`__Host-` cookie prefix on all three auth cookies (Audit 2026-05-10 MED-14).**
+  The session cookie, CSRF cookie, and OIDC pre-login cookie are renamed from
+  `certctl_session` / `certctl_csrf` / `certctl_oidc_pending` to
+  `__Host-certctl_session` / `__Host-certctl_csrf` / `__Host-certctl_oidc_pending`
+  to gain browser-enforced subdomain-takeover protection (a `__Host-*` cookie can
+  only be set with `Path=/` + `Secure` + no `Domain` attribute, and the browser
+  rejects subdomain attempts to overwrite it). **Active sessions invalidate on
+  the rolling deploy that lands this change** — operators must re-authenticate
+  once after upgrading. The GUI's CSRF cookie reader was updated in lockstep.
+  See `docs/migration/oidc-enable.md` for operator-facing detail.
+
+### Security
+
+- **OIDC `allowed_email_domains` now editable in the GUI (Audit 2026-05-11 A-3).**
+  The backend gate that rejects logins whose email domain is outside the
+  configured allowlist landed in v2.1.0 (CRIT-5 closure, 2026-05-10), but the
+  GUI never exposed the field — GUI-driven operators had to use the API
+  directly to configure tenant isolation against multi-tenant IdPs (Auth0,
+  Azure AD common endpoint, Google Workspace). The OIDCProvidersPage create
+  modal and OIDCProviderDetailPage detail view now render a chip-style
+  multi-input with client-side validation that mirrors the backend rules
+  (no `@`, no whitespace, no wildcards, lowercase-only FQDNs). The read-only
+  view renders an explicit "any (no gate configured)" sentinel when the list
+  is empty so operators can tell "not configured" apart from "field is
+  invisible." A "Clear all" button on the edit form is gated by a confirm
+  dialog that warns about removing the tenant gate. **Operator advisory: if
+  you provisioned OIDC providers via the GUI between v2.1.0 and this fix,
+  verify `allowed_email_domains` matches your tenant policy — the field was
+  configurable only via API / MCP / direct SQL during that window.** Per-IdP
+  runbooks for multi-tenant IdPs in `docs/operator/oidc-runbooks/` already
+  documented the field; the GUI now matches.
+
+- **Approval payload preview (Audit 2026-05-11 A-5).**
+  The MED-10 closure claim ("PARTIAL: raw JSON preview; diff library
+  deferred") was inaccurate — `ApprovalsPage.tsx` rendered no payload
+  at all, so approvers were clicking Approve / Reject without seeing
+  the change they were authorizing. That defeats the entire four-eyes
+  primitive: an approver who can't see what they're approving is
+  rubber-stamping. Each row now carries a Preview toggle that expands
+  an inline panel dispatching by kind: `profile_edit` shows a
+  field-level before/after diff (changed-only rows, red/green cells,
+  `(unset)` sentinel for added/removed fields); `cert_issuance` shows
+  a definition list of CN / SANs / profile / key algo / must-staple /
+  validity (catches the wildcard-against-corp-internal-profile attack
+  at review time); unknown kinds render a generic JSON preview for
+  forward-compat with future approval kinds. The base64-encoded JSON
+  payload is decoded via the new `decodePayload` helper; malformed
+  inputs render an explicit decode-error fallback — silent failure on
+  the payload preview is what produced this bug in the first place.
+
+- **Strict pre-login UA/IP binding (Audit 2026-05-11 A-6).**
+  The MED-16 closure left a request-side empty-header bypass: when the
+  pre-login row carried a User-Agent or client-IP binding but the
+  `/auth/oidc/callback` request omitted the corresponding value, the
+  binding check was silently skipped. `curl` doesn't send User-Agent
+  by default; many programmatic clients omit it. An attacker who
+  acquired a pre-login cookie could replay it without the bound
+  header and bypass the RFC 9700 §4.7.1 defense. The check is now
+  strict-when-stored — an empty request-side value with a non-empty
+  stored binding rejects with HTTP 400 and the new audit failure
+  categories `prelogin_ua_missing` / `prelogin_ip_missing` (distinct
+  from the existing `*_mismatch` categories so SIEM rules can alert
+  specifically on bypass attempts). **Operator advisory:** environments
+  where the User-Agent is stripped in transit (some debug proxies, a
+  handful of CDN configurations) must set
+  `CERTCTL_OIDC_PRELOGIN_REQUIRE_UA=false` to keep logins working;
+  symmetric `CERTCTL_OIDC_PRELOGIN_REQUIRE_IP=false` exists for the
+  IP-side. The legacy-row compat window — pre-migration rows with no
+  stored binding — still passes through unchecked, but that window is
+  bounded by the 10-minute pre-login TTL.
+
+- **OIDC provider Advanced fields are now editable in the GUI (Audit 2026-05-11 A-7).**
+  The MED-4 row had been DEFERRED to v3 with the rationale "backend
+  already accepts these fields." The verifier hit the GUI and found
+  that the read-only display claimed the values were editable, but the
+  edit form had no inputs — the save handler passed `provider.scopes`
+  / `provider.groups_claim_path` / `provider.groups_claim_format` /
+  `provider.iat_window_seconds` / `provider.jwks_cache_ttl_seconds`
+  unchanged from the loaded object. Operators who wanted to bump the
+  IAT window or change the groups-claim path had to drop to curl /
+  MCP and trust the GUI's display matched what they'd set elsewhere.
+  Lying UX. The OIDCProviderDetailPage edit form now has a collapsible
+  Advanced section with five inputs (scopes as a space-separated text
+  field; groups-claim path; groups-claim format select with the
+  backend's `string-array` / `json-path` enum; IAT window number input
+  bounded 1–600; JWKS cache TTL number input with floor 60). Client-side
+  validation mirrors the backend `Validate` rules so common operator
+  mistakes (IAT > 600, JWKS TTL < 60, empty scopes, empty groups-claim-path)
+  reject inline instead of round-tripping a 400. The read-only `<dl>`
+  also gained the previously-invisible `jwks_cache_ttl_seconds` row.
+
+- **Pre-login cookie Path widened from `/auth/oidc/` to `/` (Audit MED-14
+  follow-on).** Required to satisfy the `__Host-` prefix's `Path=/` rule. The
+  cookie lifetime is unchanged (10 minutes) and only the callback handler
+  consumes it; the wider path scope is harmless.
+
+- **RFC 9207 `iss` URL parameter check on OIDC callback (Audit 2026-05-10
+  MED-17).** When the matched IdP's discovery doc advertises
+  `authorization_response_iss_parameter_supported: true`, certctl now requires
+  the `iss` query parameter on `/auth/oidc/callback` and enforces a
+  constant-time compare against the configured provider's `IssuerURL`. Mismatch
+  rejects with HTTP 400; the audit row's `failure_category` distinguishes
+  `iss_param_missing` / `iss_param_mismatch` (RFC 9207 leg) from the existing
+  `id_token_iss_mismatch` (in-token iss claim leg). Closes the mix-up-attack
+  defense for modern Keycloak, Authentik, and public-trust CAs that ship
+  RFC-9207 discovery. Providers that don't advertise support (the majority
+  today) keep pre-fix behavior — back-compat is preserved.
+
+- **Auth GUI batch (Audit 2026-05-10 MED-4/7/8/10/11/12 + LOW-1/11/12 +
+  HIGH-10 GUI).** New backend endpoints land alongside their GUI
+  consumers: `GET /api/v1/auth/users` + `DELETE /api/v1/auth/users/{id}`
+  (auth.user.read / auth.user.deactivate; migration 000045 adds
+  `users.deactivated_at` plus the two new permissions); `GET
+  /api/v1/auth/runtime-config` (auth.role.assign) returning a sanitized
+  flat-map of deployed CERTCTL_* values (no secrets leaked — only
+  set/unset booleans and counts); `GET
+  /api/v1/auth/oidc/providers/{id}/jwks-status` (auth.oidc.list)
+  returning the per-provider verifier counters (refresh count, last
+  refresh / error timestamps, rejected JWS count, RFC 9207 iss-param
+  flag). New `UsersPage` lists federated identities + soft-deactivates.
+  `AuthSettingsPage` gains the runtime-config panel. `KeysPage`'s
+  assign-role modal now collects `scope_type` / `scope_id` /
+  `expires_at`. `RoleDetailPage`'s add-permission form gains the same
+  scope picker, and the Delete button is hidden on the 7 default
+  system roles (server already rejected, this is pure UX).
+  `AuthProvider` renders a sticky red demo-mode banner when
+  `auth_type=none`. `actor-demo-anon` rows on `KeysPage` already had
+  buttons disabled.
+
+- **11 new MCP tools (Audit 2026-05-10 MED-13).** Approval workflow
+  (`certctl_approval_list` / `_get` / `_approve` / `_reject`), break-glass
+  credential admin (`certctl_breakglass_list` / `_set_password` /
+  `_unlock` / `_remove`), bootstrap status + consume
+  (`certctl_bootstrap_status` / `_consume`), and audit category filter
+  (`certctl_audit_list_with_category`). All route through the existing
+  HTTP client so server-side permission gates fire unchanged.
+  `certctl_bootstrap_consume`'s tool description carries an explicit
+  "NEVER WIRE THIS TO AUTONOMOUS OPERATION" warning — a leaked
+  bootstrap token mints a fresh admin API key bypassing every other
+  access-control gate, so the tool is for one-shot manual operator
+  invocation only.
+
+- **JWKS auto-refresh on cache-miss (Audit 2026-05-10 MED-6).** When
+  the IdP rotates its signing key between pre-login + callback, the
+  cached JWKS no longer contains the kid referenced by the inbound ID
+  token's JWS header. Pre-fix, the verify failed with a generic error
+  and the operator had to manually call `POST
+  /api/v1/auth/oidc/providers/{id}/refresh`. The service now detects
+  the kid-not-in-cache shape (`isKidMismatchError`) and runs a
+  one-shot `RefreshKeys` (evict cache → re-fetch discovery + JWKS →
+  re-run alg-downgrade defense) before retrying the verify exactly
+  once. Bounded recovery: a second failure surfaces as
+  `ErrJWKSUnreachable` per the original branches; no retry loop. A
+  separate matcher (`isKidMismatchError`) is intentionally narrow
+  so generic signature failures don't trigger refresh.
+
+- **OIDC provider test endpoint (Audit 2026-05-10 MED-5).** New
+  `POST /api/v1/auth/oidc/test` dry-runs an OIDC provider configuration
+  without persisting: fetches the discovery doc, runs the alg-downgrade
+  defense, detects RFC 9207 iss-parameter advertisement, and confirms
+  JWKS reachability. Returns `TestDiscoveryResult{discovery_succeeded,
+  jwks_reachable, supported_alg_values, iss_param_supported, errors[]}`
+  so the GUI (forthcoming) can render per-check status rows. Per-leg
+  failures ride in the response body's `errors` array; only a malformed
+  request body trips 400. Gate: `auth.oidc.create`. Audit row
+  `auth.oidc_provider_tested` carries the success/failure summary.
+
+- **Pre-login UA / source-IP binding on OIDC callback (Audit 2026-05-10
+  MED-16).** RFC 9700 §4.7.1 defense against stolen-pre-login-cookie replay
+  by a different browser / source. Migration `000044_prelogin_uaip` adds
+  `client_ip` + `user_agent` to `oidc_pre_login_sessions`; values captured at
+  `/auth/oidc/login` are constant-time compared at `/auth/oidc/callback`.
+  Mismatches return HTTP 400 with audit `failure_category` =
+  `prelogin_ua_mismatch` or `prelogin_ip_mismatch`. Two operator escape
+  hatches: `CERTCTL_OIDC_PRELOGIN_REQUIRE_UA` and
+  `CERTCTL_OIDC_PRELOGIN_REQUIRE_IP` (both default `true`) — operators on
+  enterprise proxies that rewrite UA, or dual-stack v4/v6 environments where
+  source IP routinely flips, can disable the affected leg. The binding column
+  is persisted even when enforcement is off, so retroactive forensics remain
+  possible. Empty values on either side pass through (rolling-deploy +
+  headless-proxy compat).
+
+## v2.1.0 - Auth Bundles 1 + 2: RBAC primitive + OIDC SSO + sessions ⚠️
+
+> **SECURITY: AUDIT YOUR API KEYS.**
+>
+> Bundle 1 ships role-based authorization. Every existing API key
+> configured via `CERTCTL_API_KEYS_NAMED` (or the legacy
+> `CERTCTL_AUTH_SECRET`) is mapped to the **r-admin role on the first
+> upgrade boot** so existing automation keeps working unchanged. Most
+> keys do NOT need full admin power; downgrade them before tagging
+> the next release.
+>
+> Recommended post-upgrade flow:
+>
+> ```bash
+> # 1. List every key with its current role:
+> certctl-cli auth keys list
+>
+> # 2. Walk an interactive prompt that downgrades each key:
+> certctl-cli auth keys scope-down
+>
+> # 3. Or get a heuristic suggestion based on 30 days of audit history:
+> certctl-cli auth keys scope-down --suggest
+> certctl-cli auth keys scope-down --suggest --apply   # applies the suggestion
+>
+> # 4. Or drive scope-down from a JSON config (Helm post-upgrade hook):
+> certctl-cli auth keys scope-down --non-interactive ./scope-down.json
+> ```
+>
+> The synthetic `actor-demo-anon` actor (used when
+> `CERTCTL_AUTH_TYPE=none` is configured) is system-managed and
+> excluded from the prompt loop.
+
+What else changed in v2.1.0:
+
+- **Audit 2026-05-10 CRIT-1 closure — wire-layer RBAC enforcement.**
+  The Bundle 1 + Bundle 2 audit surfaced that the permission catalogue
+  was enforced on ~24 admin-only routes only; the bulk of state-changing
+  routes (`POST /api/v1/certificates`, `PUT /api/v1/profiles/{id}`,
+  `DELETE /api/v1/issuers/{id}`, `POST /api/v1/agents/{id}/csr`, even
+  `POST /api/v1/auth/roles` + `POST /api/v1/auth/keys/{id}/roles`) had
+  no `rbacGate` wrap. A `r-viewer` Bearer was essentially `r-admin`
+  minus five fine-grained verbs at the wire layer (CWE-862). This
+  release wraps every state-changing + read endpoint with
+  `rbacGate` (global scope) or `rbacGateScoped` (per-profile / per-
+  issuer scope-bound grants), and adds an AST-level CI guard
+  (`TestRouterRBACGateCoverage`) that fails when a new route is
+  registered without enforcement. Catalogue extended via migration
+  000039 with 30 permissions covering `cert.edit`, `job.*`,
+  `approval.*`, `policy.*`, `team.*`, `owner.*`, `notification.*`,
+  `discovery.*`, `network_scan.*`, `healthcheck.*`, `digest.*`,
+  `verification.*`, `stats.read`, `metrics.read`. **AUDIT YOUR
+  KEYS** (the scope-down call-out above) now translates to real
+  reduction in blast radius. Auditor pin preserved at exactly
+  `{audit.read, audit.export}`.
+
+- **RBAC primitive shipped.** `tenants`, `roles`, `permissions`,
+  `role_permissions`, `actor_roles` tables (migration 000029); 33-permission
+  canonical catalogue; 7 default roles (`admin`, `operator`, `viewer`,
+  `agent`, `mcp`, `cli`, `auditor`); per-handler permission gates via
+  `auth.RequirePermission` middleware (replaces the legacy
+  `IsAdmin` boolean check on the 5 admin-only handlers).
+- **Day-0 admin bootstrap.** Set `CERTCTL_BOOTSTRAP_TOKEN` on a fresh
+  deploy and POST a single curl call against `/api/v1/auth/bootstrap` to
+  mint the first admin API key; one-shot, never logged, and locks
+  closed once any admin actor exists. Migration 000031 ships the
+  `api_keys` table that stores the SHA-256 hash; the plaintext is
+  shown in the response body once and never persisted.
+- **Auditor role split.** New `auditor` role holds only `audit.read`
+  + `audit.export`. Compliance reviewers can read the audit trail
+  without holding mutation power. Migration 000032 adds
+  `audit_events.event_category` so auditors can filter to
+  authentication-related events specifically.
+- **`/v1/auth/check` enrichment.** Response now includes the actor's
+  standing roles and effective permissions, so the GUI gates
+  affordances from a single fetch on app boot.
+- **Approval-bypass closure.** Edits to a profile that has (or
+  would have) `RequiresApproval=true` now route through the
+  `ApprovalService` two-person integrity gate (Phase 9). Migration
+  000033 adds `approval_kind` + `payload` to
+  `issuance_approval_requests` so cert-issuance and profile-edit
+  approvals share the same workflow. Same-actor self-approve is
+  rejected with `ErrApproveBySameActor` for both kinds. Closes the
+  flip-flop loophole where an admin could disable approval, mutate,
+  re-enable. Documented at
+  [`docs/reference/profiles.md`](docs/reference/profiles.md).
+- **GUI: Roles / API Keys / Auth Settings / Approvals queue.**
+  Four new pages under `/auth/*` consume `/v1/auth/me` for
+  permission-aware rendering. The Approvals queue blocks
+  self-approve at the client layer (Approve/Reject buttons hidden
+  when requested_by == current actor_id) on top of the server-side
+  enforcement. AuditPage gains a category filter (cert_lifecycle /
+  auth / config) for the auditor view.
+- **MCP server gains 12 RBAC tools.** Operators driving certctl
+  from Claude / VS Code / any MCP client get parity with the GUI
+  + CLI. Each tool routes through the same HTTP handler; permission
+  gates fire server-side.
+- **OpenAPI catalogues every new route.** Every Bundle 1 endpoint
+  ships with an `operationId`; the parity test guards against drift.
+- **Coverage gates.** `internal/auth/` and `internal/service/auth/`
+  now have ≥85% coverage floors in `.github/coverage-thresholds.yml`.
+  The 12-path negative-test list from the Bundle 1 prompt is
+  fully covered (path #12 deferred with in-tree TODO).
+- **Protocol-endpoint allowlist pinned at three layers.** The
+  middleware bypass (`auth.IsProtocolEndpoint`), the router-level
+  `AuthExemptRouterRoutes` constant, and a new
+  `phase12_protocol_allowlist_test.go` AST scan all guard against
+  accidentally wrapping ACME / SCEP / EST / OCSP / CRL routes in
+  `rbacGate`.
+- **Bundle 2: OIDC + sessions + back-channel logout + break-glass.**
+  Auth Bundle 2 ships in the same v2.1.0 release. Operators get OIDC
+  SSO support for Keycloak / Authentik / Okta / Auth0 / Microsoft
+  Entra ID / Google Workspace (via Keycloak broker), HMAC-signed
+  session cookies with idle/absolute timeouts + CSRF defense,
+  back-channel logout per OpenID Connect Back-Channel Logout 1.0,
+  and a default-OFF break-glass admin path with Argon2id passwords
+  for SSO-broken incidents. API-key auth keeps working unchanged
+  alongside; existing automation needs no changes. Migration walkthrough
+  at [`docs/migration/oidc-enable.md`](docs/migration/oidc-enable.md);
+  per-IdP setup guides at
+  [`docs/operator/oidc-runbooks/index.md`](docs/operator/oidc-runbooks/index.md).
+- **OIDC token validation pinned at three layers.** Algorithm
+  allow-list (RS256/RS512/ES256/ES384/EdDSA only) with HS-family + `none`
+  rejected at the service-layer sentinel; IdP-downgrade-attack defense
+  at provider creation AND every JWKS RefreshKeys (intersects the IdP's
+  advertised `id_token_signing_alg_values_supported` against the allow-
+  list, rejects providers that advertise weak algs even before any
+  token is signed); OIDC Core §3.1.3.7 re-verification of `iss` /
+  `aud` / `azp` / `at_hash` (REQUIRED-when-access_token-present per
+  Phase 3 tightening of the spec MAY → MUST) / `exp` / `iat` window
+  / `nonce` constant-time-compare. PKCE-S256 mandatory; `plain`
+  rejected. Single-use state + nonce via atomic `DELETE...RETURNING`
+  on consume.
+- **Session cookies use length-prefixed HMAC.** The cookie wire format
+  is `v1.<session_id>.<signing_key_id>.<base64url-no-pad(HMAC-SHA256)>`
+  with HMAC input `len:sid:len:kid` (NOT bare-concat) to defeat
+  concatenation collisions. `HttpOnly` + `Secure` + `SameSite=Lax`
+  default; `SameSite=Strict` configurable via `CERTCTL_SESSION_SAMESITE`.
+  Idle timeout 1h / absolute 8h defaults; scheduler GC sweeps expired
+  rows hourly. Signing keys rotate via the new `RotateSigningKey`
+  primitive; the old key stays valid for `CERTCTL_SESSION_SIGNING_KEY_RETENTION`
+  (default 24h) so existing cookies validate during rollover.
+- **CSRF defense via double-submit-cookie + hashed-token-on-row.**
+  Plaintext CSRF token in the JS-readable `certctl_csrf` cookie
+  (intentionally `HttpOnly=false` for the GUI to echo into the
+  `X-CSRF-Token` header); SHA-256 hash on the session row;
+  `subtle.ConstantTimeCompare` in the new `CSRFMiddleware`. API-key
+  actors are CSRF-exempt (no session row in context).
+- **OIDC `client_secret` encrypted at rest.** AES-256-GCM v3 blob
+  format (magic 0x03 + salt(16) + nonce(12) + ciphertext+tag) using
+  the existing `CERTCTL_CONFIG_ENCRYPTION_KEY`. Encryption invariant
+  pinned by an integration test asserting ciphertext != plaintext +
+  v3 blob shape + round-trip recovery + wrong-passphrase fails.
+- **OIDC first-admin bootstrap.** New `CERTCTL_BOOTSTRAP_ADMIN_GROUPS`
+  + `CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID` env vars: the first
+  OIDC-authenticated user with a matching group claim becomes admin
+  per tenant. Coexists with the Bundle 1 env-var-token bootstrap;
+  the admin-existence probe ensures only one wins. Audit row
+  (`bootstrap.oidc_first_admin`) on every grant.
+- **Break-glass admin (default-OFF).** New `CERTCTL_BREAKGLASS_ENABLED`
+  env var (default `false`). When enabled, the local Argon2id-password
+  admin path bypasses OIDC + group-claim layers — intended ONLY for
+  SSO-broken incidents. Argon2id with OWASP 2024 params (m=64 MiB,
+  t=3, p=4); lockout after 5 failures (configurable); constant-time
+  across all failure paths via `verifyDummy`; surface invisibility
+  (HTTP 404 on every endpoint when disabled, NOT 403). WARN log at
+  server boot when enabled. WebAuthn/FIDO2 second factor pairing on
+  the v3 roadmap (Decision 12).
+- **GUI: OIDC Providers + Group → Role Mappings + Sessions + login
+  buttons.** Four new pages under `/auth/*` consume the Bundle 2 API
+  surface. Login page renders one "Sign in with X" button per
+  configured OIDC provider (in addition to the API-key form, which
+  remains as a fallback for Bearer-mode + break-glass paths). Sessions
+  page exposes own-sessions + admin all-actors view. Every actionable
+  element is permission-gated server-side via `auth.oidc.*` and
+  `auth.session.*` perms; client-side hide is UX layer. Logout button
+  in the sidebar fires `POST /auth/logout` to clear the session
+  server-side before redirecting to login.
+- **MCP server gains 11 OIDC + session tools.** `certctl_auth_list_oidc_providers`,
+  `_get_oidc_provider`, `_create_oidc_provider`, `_update_oidc_provider`,
+  `_delete_oidc_provider`, `_refresh_oidc_provider`,
+  `_list_group_mappings`, `_add_group_mapping`, `_remove_group_mapping`,
+  `_list_sessions`, `_revoke_session`. Operator-facing MCP tool count
+  goes 12 (Bundle 1 RBAC) → 23 across the auth surface. Total MCP
+  tool count: `grep -cE 'mcp\.AddTool\(' internal/mcp/tools*.go` ≈ 150.
+- **Per-IdP runbooks: 6 production-tier setup guides** at
+  `docs/operator/oidc-runbooks/`. Each runbook follows a consistent
+  five-section layout (Prerequisites / IdP-side config / certctl-side
+  config / Verification / Troubleshooting + Validation checklist with
+  operator sign-off line). Keycloak is the canonical reference;
+  Authentik / Okta / Auth0 / Entra ID / Google Workspace document the
+  IdP-specific deltas (Auth0's namespaced custom claims; Entra ID's
+  group OBJECT IDs; Google Workspace's missing-groups-claim limitation
+  + the recommended Keycloak broker pattern).
+- **Threat model extended.** [`docs/operator/auth-threat-model.md`](docs/operator/auth-threat-model.md)
+  ships 5 new "Defenses Bundle 2 ships" subsections + 8 new threat-
+  catalogue subsections (OIDC token forgery / session hijacking / IdP
+  compromise / back-channel logout failure modes / group-claim
+  manipulation / bootstrap risks / break-glass risks / token-leak
+  hygiene). 6 new SQL-shaped operator-facing checks. New "Threats
+  Bundle 2 does NOT close" section enumerating the 8 v3-backlog items
+  (WebAuthn / JIT elevation / SAML / multi-tenant activation /
+  HSM-FIPS / OIDC RP-initiated logout / Playwright / per-IdP
+  external-tester sign-off).
+- **Performance baselines documented.** [`docs/operator/auth-benchmarks.md`](docs/operator/auth-benchmarks.md)
+  ships four benchmarks with measured baselines on a 4 vCPU /
+  8 GiB / Postgres 16 / Go 1.25 floor: `BenchmarkSession_SteadyState`
+  p99 5 µs (target < 1 ms; 200× under), `BenchmarkSession_ColdProcess`
+  p99 7.1 ms (target < 10 ms), `BenchmarkOIDC_SteadyState` p99 1.5 ms
+  (target < 5 ms), `BenchmarkOIDC_ColdCache` operator-runs against
+  live Keycloak via `make benchmark-auth-coldcache`.
+- **Standards + RFC implementation table.** [`docs/reference/auth-standards-implemented.md`](docs/reference/auth-standards-implemented.md)
+  ships 13 RFC / standard rows + 14 CWE rows with concrete file paths
+  + negative-test anchors per row. NOT a compliance-mapping doc per
+  the operator's 2026-05-05 retired-compliance-docs decision; the
+  doc explicitly says "build the framework mapping yourself against
+  the rows here using the framework-mapping methodology your audit
+  firm prescribes; this project does not own that mapping."
+- **Coverage gates held at floor 90 across all four Bundle 2
+  packages.** `internal/auth/oidc/` 93.7%, `internal/auth/session/`
+  94.9%, `internal/auth/breakglass/` 91.5%, `internal/auth/user/domain/`
+  96.4%. NO held-low-with-rationale entry — the Phase 13 prompt's
+  anti-Bundle-1-mistake rule held. Bundle 1's existing 85% floors
+  for `internal/auth/` + `internal/service/auth/` stay 85
+  (already-shipped-and-accepted) per the prompt's explicit
+  inheritance rule.
+- **Multi-tenant query CI guard.** New `scripts/ci-guards/multi-tenant-query-coverage.sh`
+  (ratchet-style, baseline 32 at v2.1.0 close): greps every
+  SELECT/UPDATE/DELETE in `internal/repository/postgres/` against
+  10 tenant-aware tables, fails on regression OR improvement (forces
+  the operator to lift / lower the baseline visibly). Forward-compat
+  protection so a future Bundle 3 / managed-service multi-tenant
+  activation can flip the switch without finding silent
+  tenant-data-leak bugs in shipped queries.
+- **Phase 10 Keycloak testcontainers integration test.** New build-tag-
+  gated suite at `internal/auth/oidc/testfixtures/` + `integration_keycloak_test.go`
+  drives the full OIDC flow against a live Keycloak container booted
+  by testcontainers-go. 5-test matrix: discovery + JWKS load, full
+  PKCE auth-code happy path with HTTP form scraping, logout-revokes-
+  session, JWKS rotation, unmapped-groups-fails-closed. Reuses one
+  container across the matrix to amortize the 60-90s boot. Optional
+  Okta smoke test (build-tagged `integration && okta_smoke`) for live
+  tenant validation. New Makefile targets: `make keycloak-integration-test`
+  + `make okta-smoke-test` + `make benchmark-auth-coldcache`.
+- **OpenAPI surface extended.** New `cookieAuth` security scheme
+  (apiKey/cookie/`certctl_session`) alongside the existing
+  `bearerAuth`. 13 new Bundle 2 endpoints across the OIDC + session
+  + group-mapping CRUD surface; 4 break-glass endpoints with
+  surface-invisibility framing. The N-bundle-2-security-empty-preserved
+  CI guard locks the `security: []` opt-out count at ≥ 14 so existing
+  public endpoints stay public.
+- **Bundle-1-only compat regression CI guard.** New
+  `scripts/ci-guards/bundle-1-compat-regression.sh` asserts the
+  load-bearing invariants that protect the Bundle-1-only-deploy
+  case (session middleware defers-to-next, CSRF passthrough on
+  missing session row, ChainAuthSessionThenBearer wired, public
+  OIDC routes in AuthExempt allowlist, AuthInfo guards on
+  OIDCProvidersResolver != nil). Sibling
+  `bundle-1-to-2-upgrade-regression.sh` asserts the upgrade-path
+  invariants (migrations 000034..000038 are CREATE TABLE IF NOT EXISTS
+  + BEGIN/COMMIT-wrapped + no DROP TABLE / ALTER...DROP COLUMN
+  against 19 protected Bundle-1 tables + ON CONFLICT DO NOTHING on
+  permission seed).
+
+Migration ordering, idempotency, and downgrade are documented in
+[`docs/migration/api-keys-to-rbac.md`](docs/migration/api-keys-to-rbac.md)
+(API-key → RBAC, Bundle 1) and [`docs/migration/oidc-enable.md`](docs/migration/oidc-enable.md)
+(API-key → OIDC, Bundle 2). The threat model lives at
+[`docs/operator/auth-threat-model.md`](docs/operator/auth-threat-model.md).
+Day-2 RBAC operations live at [`docs/operator/rbac.md`](docs/operator/rbac.md).
+RFC + CWE evidence at [`docs/reference/auth-standards-implemented.md`](docs/reference/auth-standards-implemented.md).
+
+## v2.0.68 - Image registry path changed ⚠️
+
+> **Image registry path changed.** Starting this release, container images publish to `ghcr.io/certctl-io/certctl-server` and `ghcr.io/certctl-io/certctl-agent`. Existing pulls from `ghcr.io/shankar0123/certctl-{server,agent}:<tag>` continue to work for previously-published tags (the registry never deletes images), but the `:latest` tag at the old path stops moving forward at this release. Update your `docker pull` paths, `docker-compose.yml` `image:` keys, or Helm `image.repository` values to receive future updates. Old `git clone` / `git push` / install-script / API URLs continue to redirect forever - only the container-registry path changed.
 
 This is the only operator-action-required change in v2.0.68. Other changes in this release are cosmetic URL refreshes after the GitHub-org transfer from `shankar0123/certctl` to `certctl-io/certctl` (HTTP redirects mean no other operator action is required) plus an internal contextcheck lint fix in the agent. Full commit list is on the [GitHub release page](https://github.com/certctl-io/certctl/releases/tag/v2.0.68).
 
@@ -13,18 +776,18 @@ notes are auto-generated from commit messages between consecutive tags.
 
 **Where to find what changed in a given release:**
 
-- **[GitHub Releases](https://github.com/certctl-io/certctl/releases)** — every
+- **[GitHub Releases](https://github.com/certctl-io/certctl/releases)** - every
   tag has an auto-generated "What's Changed" section pulled from the commits
   between that tag and the previous one, plus per-release supply-chain
   verification instructions (Cosign / SLSA / SBOM).
-- **`git log <prev-tag>..<this-tag> --oneline`** — same content, locally.
+- **`git log <prev-tag>..<this-tag> --oneline`** - same content, locally.
 
 **Why no hand-edited CHANGELOG.md:**
 
 certctl is solo-developed and pushes directly to master. Maintaining a
 hand-edited CHANGELOG meant the file drifted (entries piled into
 `[unreleased]` and never got promoted to per-version sections when tags were
-cut). A stale CHANGELOG is worse than no CHANGELOG — it signals abandoned
+cut). A stale CHANGELOG is worse than no CHANGELOG - it signals abandoned
 maintenance to security-conscious operators doing diligence.
 
 The auto-generated release notes work here because commit messages follow a

Dockerfile

@@ -63,7 +63,7 @@ RUN for i in 1 2 3; do \
     npm run build
 
 # Stage 2: Build Go binary
-FROM golang:1.25-alpine@sha256:5caaf1cca9dc351e13deafbc3879fd4754801acba8653fa9540cea125d01a71f AS builder
+FROM golang:1.25.10-alpine@sha256:8d22e29d960bc50cd025d93d5b7c7d220b1ee9aa7a239b3c8f55a57e987e8d45 AS builder
 
 # Proxy propagation (M-4, Issue #9) — see Stage 1 rationale.
 ARG HTTP_PROXY=

Dockerfile.agent

@@ -5,7 +5,7 @@
 # operator runbook; the pins here MUST be bumped in the same pass.
 
 # Stage 1: Build
-FROM golang:1.25-alpine@sha256:5caaf1cca9dc351e13deafbc3879fd4754801acba8653fa9540cea125d01a71f AS builder
+FROM golang:1.25.10-alpine@sha256:8d22e29d960bc50cd025d93d5b7c7d220b1ee9aa7a239b3c8f55a57e987e8d45 AS builder
 
 # Proxy propagation (M-4, Issue #9) — defaulted to empty so un-proxied builds
 # behave identically to the pre-fix tree. When `HTTP_PROXY`/`HTTPS_PROXY`/

LICENSE

@@ -2,9 +2,9 @@ Business Source License 1.1
 
 Parameters
 
-Licensor:             Shankar Kambam
+Licensor:             certctl LLC
 Licensed Work:        certctl
-                      The Licensed Work is © 2026 Shankar Kambam.
+                      The Licensed Work is © 2026 certctl LLC.
 
 Additional Use Grant: You may make use of the Licensed Work, including in
                       production for your internal business operations and
@@ -12,15 +12,23 @@ Additional Use Grant: You may make use of the Licensed Work, including in
                       your own customers, provided that you may not offer
                       the Licensed Work as a Commercial Certificate Service.
 
-                      A "Commercial Certificate Service" is a product or
+                      A "Commercial Certificate Service" is any product
-                      service whose principal value to a third party is the
+                      or service that provides third parties with access
+                      to or control of any substantial set of the
                       certificate management functionality of the Licensed
                       Work — including but not limited to lifecycle
                       management, discovery, monitoring, alerting, renewal
-                      automation, deployment, and revocation — where the
+                      automation, deployment, revocation, certificate
-                      third party accesses or controls that functionality
+                      authority operation, certificate issuance,
-                      and compensation is received for that access or
+                      certificate signing, or any combination thereof —
-                      control.
+                      where compensation, in any form, is received in
+                      connection with such access or control. This
+                      restriction applies irrespective of whether such
+                      functionality is the principal, ancillary,
+                      supporting, or one of several values provided by the
+                      product or service, and irrespective of whether the
+                      Licensed Work is presented under its original name,
+                      a modified name, or no name at all.
 
                       For the avoidance of doubt:
 
@@ -36,12 +44,17 @@ Additional Use Grant: You may make use of the Licensed Work, including in
 
                       (b) for the purposes of this Additional Use Grant,
                           "third party" excludes (i) your employees, (ii)
-                          your contractors acting on your behalf, and (iii)
+                          your contractors acting on your behalf, and
-                          your Affiliates. "Affiliate" means any entity
+                          (iii) your Affiliates. "Affiliate" means any
-                          that controls, is controlled by, or is under
+                          entity that (1) directly or indirectly controls
-                          common control with, you, where "control" means
+                          you, (2) is directly or indirectly controlled by
-                          ownership of more than fifty percent (50%) of
+                          you, or (3) is directly or indirectly under
-                          the voting interests of the entity;
+                          common control with you, where "control" means
+                          either (A) ownership of more than fifty percent
+                          (50%) of the voting interests of the entity, or
+                          (B) the power to direct the management and
+                          policies of the entity, whether through voting
+                          securities, contract, or otherwise;
 
                       (c) the restriction on offering a Commercial
                           Certificate Service applies regardless of whether
@@ -67,16 +80,34 @@ works, redistribute, and make non-production use of the Licensed Work. The
 Licensor may make an Additional Use Grant, above, permitting limited production
 use.
 
-Effective on the Change Date, or the fourth anniversary of the first publicly
+Effective on the Change Date, the Licensor hereby grants you rights under
-available distribution of a specific version of the Licensed Work under this
-License, whichever comes first, the Licensor hereby grants you rights under
 the terms of the Change License, and the rights granted in the paragraph
 above terminate.
 
 If your use of the Licensed Work does not comply with the requirements
 currently in effect as described in this License, you must purchase a
 commercial license from the Licensor, its affiliated entities, or authorized
-resellers, or you must refrain from using the Licensed Work.
+resellers, or you must refrain from using the Licensed Work. Rights granted
+under any commercial license from the Licensor are personal to the licensee
+and may not be sublicensed, transferred, assigned, or resold to any third
+party without the Licensor's prior written consent. Any attempted sublicense,
+transfer, assignment, or resale in violation of this provision is void.
+
+Restricted Activities. Notwithstanding any other provision of this License,
+you may not:
+
+  (i)   provide the Licensed Work or substantially similar functionality
+        to third parties as a hosted, managed, embedded, bundled, or
+        integrated service, except as expressly permitted in the
+        Additional Use Grant;
+
+  (ii)  move, change, disable, circumvent, or work around any license,
+        security, attribution, audit-trail, or feature-gating
+        functionality contained in the Licensed Work; or
+
+  (iii) alter or remove any license, copyright, attribution, trademark,
+        or other notice from the Licensed Work, its derivatives, or any
+        substantial portion thereof.
 
 All copies of the original and modified Licensed Work, and derivative works
 of the Licensed Work, are subject to this License. This License applies
@@ -110,8 +141,12 @@ the Licensor or to any repository hosting the Licensed Work is provided at
 the submitter's sole risk, confers no rights or obligations on the
 Licensor, and is not incorporated into the Licensed Work.
 
-This License does not grant you any right in any trademark or logo of the
+Trademark and naming. This License does not grant you any right in any
-Licensor or its Affiliates.
+trademark, service mark, trade name, or logo of the Licensor or its
+Affiliates. Forks, derivative works, and modifications of the Licensed Work
+must not use the name "certctl," any name confusingly similar to "certctl,"
+or any Licensor trademark in their distributed form, marketing materials,
+package metadata, or service offerings.
 
 Governing law and venue. This License shall be governed by and construed in
 accordance with the laws of the State of Florida, USA, without giving

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help build run test lint verify verify-docs verify-deploy loadtest acme-cert-manager-test acme-rfc-conformance-test clean docker-up docker-down migrate-up migrate-down generate test-cover frontend-build qa-stats
+.PHONY: help build run test lint verify verify-deploy loadtest acme-cert-manager-test acme-rfc-conformance-test keycloak-integration-test okta-smoke-test benchmark-auth benchmark-auth-coldcache clean docker-up docker-down migrate-up migrate-down generate test-cover frontend-build e2e-test qa-stats
 
 # Default target - show help
 help:
@@ -16,7 +16,6 @@ help:
 	@echo "  make lint           Run linter (golangci-lint)"
 	@echo "  make fmt            Format code with gofmt"
 	@echo "  make verify         Pre-commit gate: fmt + vet + lint + test (CI-parity)"
-	@echo "  make verify-docs    Pre-tag gate:    QA-doc drift checks (operator-facing docs)"
 	@echo "  make verify-deploy  Pre-push gate:   digest validity + OpenAPI parity + docker build smoke"
 	@echo "  make loadtest       k6 throughput run against postgres + certctl (NOT in verify; manual + cron only)"
 	@echo ""
@@ -119,23 +118,6 @@ verify:
 	@echo ""
 	@echo "verify: PASS — safe to commit"
 
-# verify-docs: pre-tag gate. Runs the QA-doc seed-count drift guard
-# that ci-pipeline-cleanup Phase 11 / frozen decision 0.13 moved out
-# of CI (was per-push blocking; now operator-runs pre-tag). Protects
-# docs/contributor/qa-test-suite.md::Seed Data Reference from
-# drifting vs migrations/seed_demo.sql. Operator-facing docs only —
-# not product-affecting.
-#
-# The QA-doc Part-count drift guard retired in the 2026-05-04 docs
-# overhaul Phase 5 when docs/testing-guide.md was pruned (its content
-# dispersed across the audience-organized doc tree); the Part-count
-# class no longer exists outside the qa_test.go file itself.
-verify-docs:
-	@echo "==> QA-doc seed-count drift"
-	@bash scripts/qa-doc-seed-count.sh
-	@echo ""
-	@echo "verify-docs: PASS — safe to tag"
-
 # verify-deploy: optional pre-push gate. Runs the digest-validity check,
 # the OpenAPI ↔ handler parity check, and a Docker build smoke for the
 # production images (server + agent only — fast subset for local; CI
@@ -171,6 +153,54 @@ loadtest:
 	@echo "==> results landed in deploy/test/loadtest/results/"
 	@if [ -f deploy/test/loadtest/results/summary.txt ]; then cat deploy/test/loadtest/results/summary.txt; fi
 
+# Auth Bundle 2 Phase 10 — Keycloak end-to-end OIDC integration test.
+# Boots a Keycloak container via testcontainers-go (quay.io/keycloak:25.0),
+# imports a canned realm with two groups + two users, and drives the
+# full OIDC flow against the certctl service: discovery + JWKS,
+# auth-code login, group-claim parsing, group-role mapping, session
+# mint, and JWKS rotation.
+#
+# Build-tag-gated under `integration` so `make verify` (which runs
+# go test -short) NEVER pulls in the 60-90s Keycloak boot. Requires a
+# local Docker daemon. Skips cleanly with t.Skip() when -short is set.
+keycloak-integration-test:
+	@echo "==> running Keycloak OIDC integration test (requires Docker)"
+	@go test -tags=integration -count=1 -timeout=10m \
+	  ./internal/auth/oidc/...
+
+# Auth Bundle 2 Phase 10 — optional Okta smoke test. Gated behind TWO
+# build tags (integration + okta_smoke) so it only runs when invoked
+# manually against the operator's own Okta dev tenant. Requires the
+# OKTA_ISSUER + OKTA_CLIENT_ID + OKTA_CLIENT_SECRET env vars; the test
+# t.Skip's with a clear message when any are missing. Documented in
+# internal/auth/oidc/integration_okta_smoke_test.go.
+okta-smoke-test:
+	@echo "==> running Okta smoke test (requires OKTA_ISSUER / _CLIENT_ID / _CLIENT_SECRET env vars)"
+	@go test -tags='integration okta_smoke' -count=1 -timeout=2m \
+	  ./internal/auth/oidc/...
+
+# Auth Bundle 2 Phase 14 — auth performance benchmarks. Three default-
+# tag benchmarks (session steady-state + session cold-process + oidc
+# steady-state) producing p50/p95/p99/max numbers per the auth-
+# benchmarks.md operator-doc table.
+benchmark-auth:
+	@echo "==> running auth performance benchmarks (session + oidc steady-state)"
+	@go test -bench='BenchmarkSession_|BenchmarkOIDC_SteadyState' -benchmem \
+	  -benchtime=2000x -run='^$$' \
+	  ./internal/auth/session/ ./internal/auth/oidc/
+
+# Auth Bundle 2 Phase 14 — OIDC cold-cache benchmark against a live
+# Keycloak container (requires Docker). Build-tag-gated so the
+# default-tag benchmarks above never pull in the 60-90s container
+# boot. Runs the integration test FIRST to populate the
+# sharedKeycloak fixture, then runs the benchmark.
+benchmark-auth-coldcache:
+	@echo "==> running OIDC cold-cache benchmark against live Keycloak (requires Docker)"
+	@go test -tags integration -count=1 -timeout=10m \
+	  -run TestKeycloakIntegration_RefreshKeysFetchesDiscoveryAndJWKS \
+	  -bench BenchmarkOIDC_ColdCache -benchmem -benchtime=10x \
+	  ./internal/auth/oidc/
+
 # Phase 5 — kind-driven cert-manager integration test. Requires
 # `kind`, `kubectl`, `helm`, and a local Docker daemon. Sets
 # KIND_AVAILABLE=1 so the test runs (it skips cleanly when unset, which
@@ -265,13 +295,23 @@ frontend-build:
 	cd web && npm ci && npx vite build
 	@echo "Frontend build complete"
 
-# QA Suite Stats — Bundle P / Strengthening #8.
+# Phase 3 TEST-M3 closure (2026-05-13): browser-driven E2E smoke
-# Single source-of-truth for every count claim in
+# target. The full 15-flow suite from web/src/__tests__/e2e/README.md
-# docs/contributor/qa-test-suite.md. The Strengthening #6 CI drift guards
+# ships in frontend-design-audit Phase 8; this target is the harness
-# (now scoped to the seed-count class only — the Part-count class retired
+# wiring that lets `make e2e-test` work today.
-# in the 2026-05-04 docs overhaul Phase 5 when testing-guide.md was
+#
-# pruned) consume the same numbers, eliminating the doc-drift class
+# First-time setup: `cd web && npm install && npx playwright install --with-deps chromium`.
-# structurally.
+# The webServer block in web/playwright.config.ts boots `npm run dev`
+# automatically; no separate `make docker-up` needed.
+e2e-test:
+	@echo "Running Playwright E2E (smoke + any *.spec.ts under web/src/__tests__/e2e/)..."
+	cd web && npx playwright test
+	@echo "E2E run complete"
+
+# qa-stats: snapshot of the test-suite size at the current commit.
+# Backend Go tests + subtests + fuzz targets + skipped sites, plus the
+# seed-data counts in migrations/seed_demo.sql. Useful before a release
+# to spot-check that no whole layer dropped off.
 qa-stats:
 	@echo "=== certctl QA Suite Stats ==="
 	@echo "Date: $$(date +%Y-%m-%d)"
@@ -285,7 +325,7 @@ qa-stats:
 	@echo "t.Skip sites: $$(grep -rE 't\.Skip(Now|f)?\(' --include='*_test.go' . 2>/dev/null | wc -l | tr -d ' ')"
 	@echo "qa_test.go Part_ subtests: $$(grep -cE 't\.Run\(\"Part[0-9]+_' deploy/test/qa_test.go 2>/dev/null || echo 0)"
 	@echo "Seed unique mc-* IDs:  $$(grep -oE "mc-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ')"
-	@echo "Seed unique ag-* IDs:  $$(grep -oE "ag-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ') (incl. agent_groups; agents-table count is 12)"
+	@echo "Seed unique ag-* IDs:  $$(grep -oE "ag-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ') (incl. agent_groups; agents-table count is 13 incl. agent-demo-1 + 3 cloud sentinels + server-scanner)"
 	@echo "Seed unique iss-* IDs: $$(grep -oE "iss-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ') (issuers table count is 13)"
 	@echo "Seed unique tgt-* IDs: $$(grep -oE "tgt-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ')"
 	@echo "Seed unique nst-* IDs: $$(grep -oE "nst-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ')"

NOTICE

@@ -0,0 +1,18 @@
+certctl
+Copyright 2026 certctl LLC.
+
+This product is distributed under the Business Source License 1.1.
+See LICENSE at the repository root for the full license text and
+the Additional Use Grant carve-outs.
+
+This product links third-party Go modules and JavaScript packages
+whose own license terms apply to those components. The full
+inventory of third-party dependencies and their respective licenses
+is enumerated in THIRD_PARTY_NOTICES.md at the repository root.
+
+Effective March 14, 2076, the BSL 1.1 license converts to the
+Apache License 2.0 per the Change Date in LICENSE.
+
+For inquiries about commercial licensing terms outside the
+Additional Use Grant — including the Commercial Certificate
+Service restriction — contact certctl@proton.me.

README.md

@@ -9,10 +9,18 @@
 [![GitHub Release](https://img.shields.io/github/v/release/certctl-io/certctl)](https://github.com/certctl-io/certctl/releases)
 [![GitHub Stars](https://img.shields.io/github/stars/certctl-io/certctl?style=flat&logo=github)](https://github.com/certctl-io/certctl/stargazers)
 
-certctl is a self-hosted platform that automates the entire TLS certificate lifecycle, from issuance through renewal to deployment, with zero human intervention. It works with any certificate authority, deploys to any server, and keeps private keys on your infrastructure where they belong. Free, source-available under BSL 1.1, covers the same lifecycle that enterprise platforms charge $100K+/year for.
+certctl is a self-hosted platform that automates the entire TLS certificate lifecycle, from issuance through renewal to deployment, with zero human intervention. Twelve native CA connectors plus an OpenSSL / shell-script adapter for custom CAs; fifteen native deployment-target connectors plus a proxy-agent pattern for network appliances and agentless targets. Private keys stay on your infrastructure where they belong. Free, source-available under BSL 1.1, covers the same lifecycle that enterprise platforms charge $100K+/year for.
 
 The CA/Browser Forum's [Ballot SC-081v3](https://cabforum.org/2025/04/11/ballot-sc081v3-introduce-schedule-of-reducing-validity-and-data-reuse-periods/) caps public TLS certificates at **200 days by March 2026**, **100 days by 2027**, and **47 days by 2029**. At 47-day lifespans, a team managing 100 certificates is processing 7+ renewals per week, every week, forever. Manual workflows stop being a choice.
 
+> **Status: Early-access — actively looking for design partners.**
+
+> The certificate lifecycle core is production-quality today: Local CA, ACME, agent deployment, audit, [role-based access control](docs/operator/rbac.md) with auditor split and four-eyes approval. v2.1.0 adds federated identity on top — [OIDC SSO](docs/operator/oidc-runbooks/index.md), server-side sessions, back-channel logout, and a break-glass admin path for SSO-outage recovery.
+
+> If your team runs PKI infrastructure that could use real automation, we'd love to have you on certctl. Lab and dev deployments are great. Production is welcome too — especially on the federated-identity surface, where real-world IdP shapes are exactly the exposure we can't manufacture in CI. Battle-testing certctl in your environment is genuinely valuable to us.
+
+> [File issues](https://github.com/certctl-io/certctl/issues) liberally. Every IdP quirk, every connector edge, every doc gap you hit — that's how the platform earns the right to drop the "early-access" label. The faster the loop, the faster everyone benefits.
+
 > **Actively maintained, shipping weekly.** [Open an issue](https://github.com/certctl-io/certctl/issues) if something breaks. CI runs the full test suite with race detection, static analysis, and vulnerability scanning on every commit.
 
 **Ready to try it?** Jump to the [Quick Start](#quick-start). For the marketing site, see [certctl.io](https://certctl.io).
@@ -27,7 +35,6 @@ The full audience-organized index lives at [`docs/README.md`](docs/README.md). T
 | Production operator | [Architecture](docs/reference/architecture.md) → [Security posture](docs/operator/security.md) → [Disaster recovery runbook](docs/operator/runbooks/disaster-recovery.md) |
 | PKI engineer | [ACME server](docs/reference/protocols/acme-server.md) → [SCEP server](docs/reference/protocols/scep-server.md) → [EST server](docs/reference/protocols/est.md) → [CA hierarchy](docs/reference/intermediate-ca-hierarchy.md) |
 | Migrating from another tool | [from certbot](docs/migration/from-certbot.md) / [from acme.sh](docs/migration/from-acmesh.md) / [cert-manager coexistence](docs/migration/cert-manager-coexistence.md) |
-| Contributor | [Architecture](docs/reference/architecture.md) → [Testing strategy](docs/contributor/testing-strategy.md) → [CI pipeline](docs/contributor/ci-pipeline.md) |
 
 For the connector reference (12 issuers, 15 targets, 6 notifiers) see [`docs/reference/connectors/index.md`](docs/reference/connectors/index.md).
 
@@ -39,7 +46,7 @@ For the connector reference (12 issuers, 15 targets, 6 notifiers) see [`docs/ref
 <td><a href="docs/screenshots/v2-certificates.png"><img src="docs/screenshots/v2-certificates.png" width="400" alt="Certificates"></a><br><b>Certificates</b><br><sub>Inventory with bulk ops, status filters, owner/team columns</sub></td>
 </tr>
 <tr>
-<td><a href="docs/screenshots/v2-issuers.png"><img src="docs/screenshots/v2-issuers.png" width="400" alt="Issuers"></a><br><b>Issuers</b><br><sub>Catalog with 10 CA types, GUI config, test connection</sub></td>
+<td><a href="docs/screenshots/v2-issuers.png"><img src="docs/screenshots/v2-issuers.png" width="400" alt="Issuers"></a><br><b>Issuers</b><br><sub>Catalog with 12 CA types, GUI config, test connection</sub></td>
 <td><a href="docs/screenshots/v2-jobs.png"><img src="docs/screenshots/v2-jobs.png" width="400" alt="Jobs"></a><br><b>Jobs</b><br><sub>Issuance, renewal, deployment queue with approval workflow</sub></td>
 </tr>
 </table>
@@ -57,12 +64,14 @@ Built for **platform engineering and DevOps teams** managing 10 to 500+ certific
 certctl handles the full certificate lifecycle in one self-hosted control plane:
 
 - **Issue and renew** from any CA. Let's Encrypt and any ACME provider, an embedded ACME server you can point cert-manager / certbot / lego at directly, a built-in local CA with sub-CA mode (chains under your enterprise root like ADCS), step-ca, Vault PKI, EJBCA, AWS ACM PCA, Google CAS, DigiCert, Sectigo, GlobalSign, Entrust, plus an OpenSSL / shell-script adapter for anything custom. Twelve native issuer connectors. See the [connector reference](docs/reference/connectors/index.md).
-- **Deploy automatically** to NGINX, Apache, HAProxy, Caddy, Traefik, Envoy, IIS, Windows Cert Store, Java keystore, Kubernetes Secrets, AWS ACM, Azure Key Vault, SSH known-hosts, Postfix + Dovecot, F5 BIG-IP. Fifteen native target connectors. Every deploy goes through atomic-write + ownership-preservation + SHA-256 idempotency + per-target Prometheus counters + pre-deploy snapshot + on-failure rollback. See [`docs/reference/deployment-model.md`](docs/reference/deployment-model.md).
+- **Deploy automatically** to NGINX, Apache, HAProxy, Caddy, Traefik, Envoy, IIS, Windows Cert Store, Java keystore, Kubernetes Secrets, AWS ACM, Azure Key Vault, SSH known-hosts, Postfix + Dovecot, F5 BIG-IP. Fifteen native target connectors. File-based targets share an atomic-write + SHA-256 idempotency + on-failure rollback + per-target Prometheus counters primitive (the `deploy.Apply` path covers 12 of 13 file-based connectors). Cloud / API targets (AWS ACM, Azure Key Vault) use vendor-SDK semantics rather than the file primitive; F5 uses iControl REST transactions; Kubernetes Secrets is preview. For the per-target guarantee matrix, see [`docs/reference/deployment-model.md`](docs/reference/deployment-model.md). The reload / validate commands operators configure for shell-using targets (NGINX, Apache, HAProxy, Postfix, JavaKeystore, SSH) are validated server-side AND agent-side against shell-metacharacter injection before execution (see [`internal/connector/target/configcheck`](internal/connector/target/configcheck)).
 - **Run as an ACME server** so existing client tooling plugs in directly. RFC 8555 + RFC 9773 ARI, two per-profile auth modes (public-trust-style validation or trust_authenticated for internal PKI), doubly-signed key rollover, revoke-cert on both kid path and jwk path, per-account rate limiting. Cert-manager / certbot / lego all work pointed at it. See [`docs/reference/protocols/acme-server.md`](docs/reference/protocols/acme-server.md).
 - **Run as a SCEP server** for Microsoft Intune-managed phones, ChromeOS devices, network appliances. RFC 8894 native with full PKIMessage wire format, native Intune challenge dispatch with replay protection, per-profile dispatch with separate RA cert per profile. See [`docs/reference/protocols/scep-server.md`](docs/reference/protocols/scep-server.md).
 - **Run as an EST server** for HTTPS-based PKCS#10 enrollment. 802.1X / Wi-Fi authentication, IoT device enrollment, RFC 9266 channel binding. See [`docs/reference/protocols/est.md`](docs/reference/protocols/est.md).
 - **Manage multi-level CA hierarchies** with name constraints, path-length enforcement, and end-to-end RFC 5280 path validation. Root → intermediate → issuing chains, admin-gated CRUD, drain-first retirement. Patterns documented for 4-level boundary CAs, 3-level policy CAs with per-BU `PermittedDNSDomains`, and 2-level internal PKI. See [`docs/reference/intermediate-ca-hierarchy.md`](docs/reference/intermediate-ca-hierarchy.md).
-- **Gate high-stakes issuance** behind two-person-integrity approval. Flag a profile as `RequiresApproval`, the request lands in a queue, a non-requester approves, the scheduler dispatches. See [`docs/operator/approval-workflow.md`](docs/operator/approval-workflow.md).
+- **Gate high-stakes issuance** behind two-person-integrity approval. Flag a profile as `RequiresApproval`, the request lands in a queue, a non-requester approves, the scheduler dispatches. Profile-edit changes on approval-tier profiles route through the same gate so the flip-flop bypass is closed. See [`docs/operator/approval-workflow.md`](docs/operator/approval-workflow.md).
+- **Authorize with role-based access control.** Seven default roles (admin, operator, viewer, agent, mcp, cli, auditor) over a fine-grained permission catalogue with global / per-profile / per-issuer scope. Auditor role is read-only on the audit trail (`audit.read` + `audit.export`, nothing else) so a regulator's key cannot read certificates or mutate config. Day-0 admin via a one-shot `CERTCTL_BOOTSTRAP_TOKEN` endpoint that closes itself the moment any admin lands. Privilege-escalation guard requires `auth.role.assign` to grant or revoke a role. See [`docs/operator/rbac.md`](docs/operator/rbac.md), [`docs/operator/auth-threat-model.md`](docs/operator/auth-threat-model.md), and the v2.0.x → v2.1.0 [migration guide](docs/migration/api-keys-to-rbac.md).
+- **Sign in with OIDC SSO** against any standards-compliant identity provider. Per-IdP setup runbooks for Keycloak, Authentik, Okta, Auth0, Microsoft Entra ID, and Google Workspace. Group-claim → role mapping for automatic provisioning; client_secret encrypted at rest (AES-256-GCM); JWKS auto-refresh on `kid` miss; PKCE-S256 required; RFC 9700 §4.7.1 pre-login UA/IP binding; RFC 9207 `iss` URL-param check on callback. Server mints HMAC-signed session cookies with the `__Host-` prefix (browser-enforced subdomain-takeover defense), CSRF rotation on every privileged write, and idle + absolute expiry. [RFC OIDC Back-Channel Logout 1.0](docs/reference/auth-standards-implemented.md) revokes sessions on IdP-driven logout. Argon2id break-glass admin path for SSO-outage recovery — disabled by default; 404-invisible to scanners when `CERTCTL_BREAKGLASS_ENABLED=false`. See [`docs/operator/oidc-runbooks/index.md`](docs/operator/oidc-runbooks/index.md) for the per-IdP onboarding guides and [`docs/migration/oidc-enable.md`](docs/migration/oidc-enable.md) for enabling SSO on an existing deploy.
 - **Discover** existing certs across your fleet via filesystem scanning on agents, network TLS probing across CIDR ranges, and cloud secret manager imports (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager). Triage workflow for claim / dismiss / investigate.
 - **Revoke** with full RFC 5280 reason codes, DER CRL generation per issuer (scheduler-pre-generated and ETag-cached), and an embedded RFC 6960 OCSP responder with dedicated per-issuer responder certs. Single + bulk revocation. See [`docs/reference/protocols/crl-ocsp.md`](docs/reference/protocols/crl-ocsp.md).
 - **Alert** via Slack, Microsoft Teams, PagerDuty, OpsGenie, email, webhooks. Per-policy multi-channel routing matrix with severity tiers and fault-isolating per-channel dispatch. See [`docs/operator/runbooks/expiry-alerts.md`](docs/operator/runbooks/expiry-alerts.md).
@@ -70,23 +79,36 @@ certctl handles the full certificate lifecycle in one self-hosted control plane:
 
 ## Architecture and security
 
-Go 1.25 control plane with handler → service → repository layering. PostgreSQL 16 backend (35+ tables, idempotent migrations). Pull-only deployment model — the server never initiates outbound connections. Agents poll for work and generate ECDSA P-256 keys locally so private keys never touch the control plane. For network appliances and agentless servers, a proxy agent in the same network zone handles deployment via the target's API (WinRM, iControl REST, SSH/SFTP). See the [Architecture Guide](docs/reference/architecture.md) for full system diagrams.
+Go 1.25 control plane with handler → service → repository layering. PostgreSQL 16 backend with idempotent migrations. Pull-only deployment model — the server never initiates outbound connections. Agents poll for work and generate ECDSA P-256 keys locally so private keys never touch the control plane. For network appliances and agentless servers, a proxy agent in the same network zone handles deployment via the target's API (WinRM, iControl REST, SSH/SFTP). See the [Architecture Guide](docs/reference/architecture.md) for full system diagrams.
 
-Security: API key auth enforced by default with SHA-256 hashing and constant-time comparison. CORS deny-by-default. Shell injection prevention on all connector scripts. SSRF protection (reserved IP filtering) on the network scanner. Issuer and target credentials encrypted at rest with AES-256-GCM. HTTPS-only control plane with TLS 1.3 pinned and a fail-closed startup gate that refuses to boot if the TLS bundle is unusable. Every API call recorded to an immutable audit trail with actor attribution, body hash, and latency tracking. CI runs race detection, 11 linters, and vulnerability scanning on every commit. See [`docs/operator/security.md`](docs/operator/security.md) for the operator-facing security posture.
+Security: three authentication paths — API keys (SHA-256 hashed + constant-time compared), [OIDC SSO](docs/operator/oidc-runbooks/index.md) (Keycloak / Authentik / Okta / Auth0 / Entra ID / Google Workspace), and Argon2id [break-glass admin](docs/operator/security.md) for SSO-outage recovery. Successful OIDC login mints an HMAC-signed server-side session with `__Host-` cookies, CSRF rotation on every privileged write, and [RFC OIDC Back-Channel Logout](docs/reference/auth-standards-implemented.md) for IdP-driven session revoke. Role-based authorization on every gated handler with global / per-profile / per-issuer scope. Auditor split keeps regulator-class actors strictly read-only on the audit trail. Day-0 admin via a one-shot bootstrap token; granting or revoking roles requires the dedicated `auth.role.assign` permission. CORS deny-by-default. Shell injection prevention on all connector scripts. SSRF protection (reserved IP filtering) on the network scanner. Issuer + target + OIDC client_secret credentials encrypted at rest with AES-256-GCM. HTTPS-only control plane with TLS 1.3 pinned and a fail-closed startup gate that refuses to boot if the TLS bundle is unusable. Every API call recorded to an immutable audit trail with actor attribution, body hash, and latency tracking. CI runs race detection, static analysis, and vulnerability scanning on every commit. See [`docs/operator/security.md`](docs/operator/security.md) for the full posture and [`docs/operator/auth-threat-model.md`](docs/operator/auth-threat-model.md) for what's defended vs deferred.
 
 ## Quick Start
 
 ### Docker Compose (recommended)
 
+**Demo path — zero config, populated dashboard:**
+
 ```bash
 git clone https://github.com/certctl-io/certctl.git
 cd certctl
 docker compose -f deploy/docker-compose.yml -f deploy/docker-compose.demo.yml up -d --build
 ```
 
-Wait ~30 seconds, then open **https://localhost:8443** in your browser. The shipped demo overlay seeds 32 certificates across 10 issuers, 8 agents, and 180 days of realistic history. The `certctl-tls-init` init container self-signs an ECDSA-P256 cert on first boot — accept the browser warning for the demo, or feed the generated `ca.crt` to your client.
+Wait ~30 seconds, then open **https://localhost:8443** in your browser. The demo overlay flips the base into demo-mode auth (every request served as the synthetic admin actor `actor-demo-anon` — the server emits a prominent ⚠ DEMO MODE banner at boot reminding you this posture is for evaluation only) and seeds 180 days of realistic history across 13 issuers, 8 agents, managed + discovered certs, jobs, deploys, audit, and notification events. The `certctl-tls-init` init container self-signs an ECDSA-P256 cert on first boot — accept the browser warning for the demo, or feed the generated `ca.crt` to your client.
 
-For a clean install without demo data, drop the `-f deploy/docker-compose.demo.yml` flag and run `docker compose -f deploy/docker-compose.yml up -d --build`. The four compose files (`docker-compose.yml` base, `docker-compose.demo.yml` overlay, `docker-compose.dev.yml` for PgAdmin + debug logging, `docker-compose.test.yml` for integration tests) are documented at [`deploy/ENVIRONMENTS.md`](deploy/ENVIRONMENTS.md).
+**Production path — `.env` required, fail-closed on placeholders:**
+
+```bash
+cp .env.example deploy/.env       # or root .env if running outside compose
+"${EDITOR:-nano}" deploy/.env     # set POSTGRES_PASSWORD, CERTCTL_AUTH_SECRET,
+                                   # CERTCTL_API_KEY, CERTCTL_CONFIG_ENCRYPTION_KEY,
+                                   # CERTCTL_AGENT_ID — all via openssl rand
+                                   # (replace nano with your preferred editor)
+docker compose -f deploy/docker-compose.yml up -d --build
+```
+
+The base compose alone (no demo overlay) ships production-shaped: default `auth-type=api-key`, default `keygen-mode=agent`, no demo seed, no demo-mode synthetic admin. The fail-closed startup guards in `internal/config/config.go::Validate` refuse to boot when any of the change-me-... placeholder credentials reach config outside of demo mode (Bundle 2 closure, 2026-05-12). The four compose files (`docker-compose.yml` base, `docker-compose.demo.yml` overlay, `docker-compose.dev.yml` for PgAdmin + debug logging, `docker-compose.test.yml` for integration tests) are documented at [`deploy/ENVIRONMENTS.md`](deploy/ENVIRONMENTS.md).
 
 ```bash
 curl --cacert $(docker compose -f deploy/docker-compose.yml exec -T certctl-server cat /etc/certctl/tls/ca.crt) https://localhost:8443/health
@@ -106,12 +128,15 @@ Detects your OS and architecture, downloads the binary, configures systemd (Linu
 ### Helm chart (Kubernetes)
 
 ```bash
+# Required: TLS (pick one), server API key, and Postgres password.
+# The chart fail-fasts at template time if any required value is missing.
 helm install certctl deploy/helm/certctl/ \
-  --set server.apiKey=your-api-key \
+  --set server.tls.existingSecret=<your-kubernetes.io/tls-secret-name> \
-  --set postgres.password=your-db-password
+  --set server.auth.apiKey=$(openssl rand -base64 32) \
+  --set postgresql.auth.password=$(openssl rand -base64 32)
 ```
 
-Production-ready chart with Server Deployment, PostgreSQL StatefulSet, Agent DaemonSet, health probes, security contexts (non-root, read-only rootfs), and optional Ingress. See [values.yaml](deploy/helm/certctl/values.yaml).
+Production-ready chart with Server Deployment, PostgreSQL StatefulSet (or external Postgres), Agent DaemonSet, health probes, container-scope security hardening (read-only rootfs, drop-all capabilities, non-root UID), optional PodDisruptionBudget, NetworkPolicy, Prometheus ServiceMonitor, and Ingress. See [values.yaml](deploy/helm/certctl/values.yaml) and the [external-Postgres example](deploy/helm/examples/values-external-db.yaml).
 
 ### Container images
 
@@ -143,14 +168,12 @@ Every `v*` tag publishes signed, attested artefacts (Cosign keyless OIDC + SLSA
 ```bash
 make build              # Build server + agent binaries
 make test               # Run tests
-make lint               # golangci-lint (11 linters)
+make lint               # golangci-lint (govet + staticcheck + contextcheck + unused)
 govulncheck ./...       # Vulnerability scan
 make docker-up          # Start Docker Compose stack
 ```
 
-CI runs `go vet`, `go test -race`, `golangci-lint`, `govulncheck`, and per-layer coverage thresholds (service 55%, handler 60%, domain 40%, middleware 30%) on every push. Frontend CI runs TypeScript type checking, Vitest tests, and Vite production build.
+CI runs `go vet`, `go test -race`, `golangci-lint`, `govulncheck`, and per-package coverage thresholds (service 70%, handler 75%, crypto 88%, auth packages 85-95%) on every push. The thresholds-as-data file is `.github/coverage-thresholds.yml`; lowering a floor requires corresponding test work, not a config flip. Frontend CI runs TypeScript type checking, Vitest tests, and Vite production build.
-
-For the full contributor guide see [`docs/contributor/`](docs/contributor/) — testing strategy, test environment, CI pipeline, QA prerequisites.
 
 ## License
 

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,161 @@
+# Third-Party Notices
+
+certctl is distributed under the Business Source License 1.1
+(see [LICENSE](LICENSE)). The binaries built from this source link
+third-party Go and JavaScript libraries listed below; certctl LLC
+acknowledges each library's authors and reproduces their copyright
+and license terms here in compliance with each library's license.
+
+Full license text for each library lives in that library's upstream
+repository. The license type is provided per-row; for the canonical
+notice, refer to the upstream source.
+
+- **Last reviewed:** 2026-05-13
+- **Holder:** certctl LLC
+- **License:** BSL 1.1 (Apache 2.0 effective March 14, 2076)
+
+## Go Modules (binary-link dependencies)
+
+Generated by walking `go list -deps ./...` against the certctl
+server, agent, CLI, and MCP-server build paths. Excludes the Go
+standard library and the certctl-io/certctl module itself.
+
+**Count:** see commit; generate via `go list -deps -f '{{if .Module}}{{.Module.Path}} {{.Module.Version}}{{end}}' ./...`
+
+| Module | Version | License |
+|---|---|---|
+| `github.com/Azure/azure-sdk-for-go/sdk/azcore` | v1.20.0 | MIT |
+| `github.com/Azure/azure-sdk-for-go/sdk/azidentity` | v1.13.1 | MIT |
+| `github.com/Azure/azure-sdk-for-go/sdk/internal` | v1.11.2 | MIT |
+| `github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azcertificates` | v1.4.0 | MIT |
+| `github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal` | v1.2.0 | MIT |
+| `github.com/Azure/go-ntlmssp` | v0.1.1 | MIT |
+| `github.com/AzureAD/microsoft-authentication-library-for-go` | v1.6.0 | MIT |
+| `github.com/ChrisTrenkamp/goxpath` | v0.0.0-20210404020558-97928f7e12b6 | MIT |
+| `github.com/aws/aws-sdk-go-v2` | v1.41.7 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/config` | v1.32.17 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/credentials` | v1.19.16 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/feature/ec2/imds` | v1.18.23 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/internal/configsources` | v1.4.23 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/internal/endpoints/v2` | v2.7.23 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/internal/v4a` | v1.4.24 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/acm` | v1.38.3 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/acmpca` | v1.46.14 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding` | v1.13.9 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/internal/presigned-url` | v1.13.23 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/signin` | v1.0.11 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/sso` | v1.30.17 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/ssooidc` | v1.35.21 | Apache-2.0 |
+| `github.com/aws/aws-sdk-go-v2/service/sts` | v1.42.1 | Apache-2.0 |
+| `github.com/aws/smithy-go` | v1.25.1 | Apache-2.0 |
+| `github.com/bodgit/ntlmssp` | v0.0.0-20240506230425-31973bb52d9b | BSD-2/3-Clause |
+| `github.com/bodgit/windows` | v1.0.1 | BSD-2/3-Clause |
+| `github.com/coreos/go-oidc/v3` | v3.18.0 | Apache-2.0 |
+| `github.com/go-jose/go-jose/v4` | v4.1.4 | Apache-2.0 |
+| `github.com/go-logr/logr` | v1.4.3 | Apache-2.0 |
+| `github.com/gofrs/uuid` | v4.4.0+incompatible | MIT |
+| `github.com/golang-jwt/jwt/v5` | v5.3.0 | MIT |
+| `github.com/google/jsonschema-go` | v0.4.2 | MIT |
+| `github.com/google/uuid` | v1.6.0 | BSD-2/3-Clause |
+| `github.com/hashicorp/go-cleanhttp` | v0.5.2 | MPL-2.0 |
+| `github.com/hashicorp/go-uuid` | v1.0.3 | MPL-2.0 |
+| `github.com/jcmturner/aescts/v2` | v2.0.0 | Apache-2.0 |
+| `github.com/jcmturner/dnsutils/v2` | v2.0.0 | Apache-2.0 |
+| `github.com/jcmturner/gofork` | v1.7.6 | BSD-2/3-Clause |
+| `github.com/jcmturner/goidentity/v6` | v6.0.1 | Apache-2.0 |
+| `github.com/jcmturner/gokrb5/v8` | v8.4.4 | Apache-2.0 |
+| `github.com/jcmturner/rpc/v2` | v2.0.3 | Apache-2.0 |
+| `github.com/kr/fs` | v0.1.0 | BSD-2/3-Clause |
+| `github.com/kylelemons/godebug` | v1.1.0 | Apache-2.0 |
+| `github.com/lib/pq` | v1.10.9 | MIT |
+| `github.com/masterzen/simplexml` | v0.0.0-20190410153822-31eea3082786 | Apache-2.0 |
+| `github.com/masterzen/winrm` | v0.0.0-20250927112105-5f8e6c707321 | Apache-2.0 |
+| `github.com/modelcontextprotocol/go-sdk` | v1.4.1 | Apache-2.0 |
+| `github.com/pkg/browser` | v0.0.0-20240102092130-5ac0b6a4141c | BSD-2/3-Clause |
+| `github.com/pkg/sftp` | v1.13.10 | BSD-2/3-Clause |
+| `github.com/segmentio/asm` | v1.1.3 | MIT |
+| `github.com/segmentio/encoding` | v0.5.4 | MIT |
+| `github.com/tidwall/transform` | v0.0.0-20201103190739-32f242e2dbde | ISC |
+| `github.com/yosida95/uritemplate/v3` | v3.0.2 | BSD-2/3-Clause |
+| `golang.org/x/crypto` | v0.50.0 | BSD-2/3-Clause |
+| `golang.org/x/net` | v0.53.0 | BSD-2/3-Clause |
+| `golang.org/x/oauth2` | v0.36.0 | BSD-2/3-Clause |
+| `golang.org/x/sync` | v0.20.0 | BSD-2/3-Clause |
+| `golang.org/x/sys` | v0.43.0 | BSD-2/3-Clause |
+| `golang.org/x/text` | v0.36.0 | BSD-2/3-Clause |
+| `software.sslmate.com/src/go-pkcs12` | v0.7.0 | BSD-2/3-Clause |
+
+## JavaScript Packages (production transitive closure)
+
+Generated by walking the `dependencies` graph from `web/package.json`
+through `node_modules/`. Excludes devDependencies (Vitest, Playwright,
+Vite, etc.) since they don't ship in the distributed frontend bundle.
+
+| Package | Version | License |
+|---|---|---|
+| `@reduxjs/toolkit` | 2.11.2 | MIT |
+| `@remix-run/router` | 1.23.2 | MIT |
+| `@standard-schema/spec` | 1.1.0 | MIT |
+| `@standard-schema/utils` | 0.3.0 | MIT |
+| `@tanstack/query-core` | 5.90.20 | MIT |
+| `@tanstack/react-query` | 5.90.21 | MIT |
+| `@types/d3-array` | 3.2.2 | MIT |
+| `@types/d3-color` | 3.1.3 | MIT |
+| `@types/d3-ease` | 3.0.2 | MIT |
+| `@types/d3-interpolate` | 3.0.4 | MIT |
+| `@types/d3-path` | 3.1.1 | MIT |
+| `@types/d3-scale` | 4.0.9 | MIT |
+| `@types/d3-shape` | 3.1.8 | MIT |
+| `@types/d3-time` | 3.0.4 | MIT |
+| `@types/d3-timer` | 3.0.2 | MIT |
+| `@types/use-sync-external-store` | 0.0.6 | MIT |
+| `clsx` | 2.1.1 | MIT |
+| `d3-array` | 3.2.4 | ISC |
+| `d3-color` | 3.1.0 | ISC |
+| `d3-ease` | 3.0.1 | BSD-3-Clause |
+| `d3-format` | 3.1.2 | ISC |
+| `d3-interpolate` | 3.0.1 | ISC |
+| `d3-path` | 3.1.0 | ISC |
+| `d3-scale` | 4.0.2 | ISC |
+| `d3-shape` | 3.2.0 | ISC |
+| `d3-time` | 3.1.0 | ISC |
+| `d3-time-format` | 4.1.0 | ISC |
+| `d3-timer` | 3.0.1 | ISC |
+| `decimal.js-light` | 2.5.1 | MIT |
+| `es-toolkit` | 1.45.1 | MIT |
+| `eventemitter3` | 5.0.4 | MIT |
+| `immer` | 10.2.0 | MIT |
+| `internmap` | 2.0.3 | ISC |
+| `js-tokens` | 4.0.0 | MIT |
+| `loose-envify` | 1.4.0 | MIT |
+| `react` | 18.3.1 | MIT |
+| `react-dom` | 18.3.1 | MIT |
+| `react-redux` | 9.2.0 | MIT |
+| `react-router` | 6.30.3 | MIT |
+| `react-router-dom` | 6.30.3 | MIT |
+| `recharts` | 3.8.0 | MIT |
+| `redux` | 5.0.1 | MIT |
+| `redux-thunk` | 3.1.0 | MIT |
+| `reselect` | 5.1.1 | MIT |
+| `scheduler` | 0.23.2 | MIT |
+| `tiny-invariant` | 1.3.3 | MIT |
+| `use-sync-external-store` | 1.6.0 | MIT |
+| `victory-vendor` | 37.3.6 | MIT AND ISC |
+
+## Test-fixture-only dependencies
+
+**Cisco libest.** The certctl integration test suite exercises the EST
+(RFC 7030) endpoints against Cisco's libest reference client. libest
+runs as a sidecar container (`certctl-test-libest`) only when the
+`est-e2e` Docker Compose profile is active — it is **not** vendored
+into the certctl source tree and **not** linked into any distributed
+release artifact (server, agent, CLI, MCP-server, container images,
+or release tarballs). For libest's own license terms, see
+<https://github.com/cisco/libest>.
+
+**f5-mock-icontrol.** The F5 deployment-target integration test
+ships a small Go program at `deploy/test/f5-mock-icontrol/main.go`
+under the same BSL 1.1 license as the rest of certctl. The compiled
+ELF was removed from the tracked tree in Phase 1 closure (commit
+eda3b48, 2026-05-13); it now rebuilds via the Dockerfile's
+multi-stage build on demand.

api/openapi-handler-exceptions.yaml

@@ -7,6 +7,24 @@
 # (health, metrics, pprof) routes only.
 #
 # Per ci-pipeline-cleanup bundle Phase 9 / frozen decision 0.11.
+#
+# Phase 5 reconciliation (2026-05-13, architecture diligence audit
+# ARCH-H1): of the 64 entries below, 35 are legitimate wire-protocol
+# carve-outs (SCEP RFC 8894 = 8 entries, ACME RFC 8555 default + per-
+# profile = 27 entries) that MUST stay. The remaining 29 are REST-
+# shaped routes whose OpenAPI ops were deferred during their original
+# Bundle 2 / audit-2026-05-10 / 2026-05-11 work. Burn-down plan:
+#
+#   Sprint A (per-cluster, ~7-8 ops each):
+#     Cluster 1: auth/sessions + auth/oidc (12 ops)
+#     Cluster 2: auth/breakglass + auth/users + auth/runtime-config (8 ops)
+#     Cluster 3: audit/export + demo-residual/cleanup + auth/logout +
+#                auth/breakglass/login + auth/oidc/{login,callback,bcl} (9 ops)
+#
+# Each authored OpenAPI op needs request/response schemas (not
+# placeholders) so the generated client at web/orval.config.ts emits
+# typed signatures. When an op lands, delete the corresponding entry
+# below + bump the openapi-handler-parity.sh expected counts.
 
 documented_exceptions:
   - route: "GET /scep"
@@ -92,3 +110,68 @@ documented_exceptions:
     why: "Phase 4 default-profile shorthand for revoke-cert."
   - route: "GET /acme/renewal-info/{cert_id}"
     why: "Phase 4 default-profile shorthand for ARI."
+
+  # =============================================================================
+  # Auth Bundle 2 + audit-2026-05-10/11 fix bundle — REST endpoints not yet
+  # represented in api/openapi.yaml. These are operator-facing REST endpoints
+  # (not protocol-shaped); the OpenAPI surface is scheduled to land pre-v2.2.0
+  # alongside the GUI E2E coverage push. Documented here so the parity guard
+  # stays green for the v2.1.0 release tag. Threat model + handler contracts
+  # live in docs/operator/{rbac.md,auth-threat-model.md,oidc-runbooks/*}.
+  # =============================================================================
+  - route: "GET /auth/oidc/login"
+    why: "Bundle 2 Phase 5 OIDC login redirect; user-facing 302 with state cookie. OpenAPI rep deferred to pre-2.2.0."
+  - route: "GET /auth/oidc/callback"
+    why: "Bundle 2 Phase 5 OIDC callback handler; RFC 9700 §4.7.1 + RFC 9207. OpenAPI rep deferred to pre-2.2.0."
+  - route: "POST /auth/logout"
+    why: "Bundle 2 Phase 5 cookie + CSRF revoker. OpenAPI rep deferred to pre-2.2.0."
+  - route: "POST /auth/breakglass/login"
+    why: "Bundle 2 Phase 7.5 public break-glass login (auth-bypass, 404 when disabled). OpenAPI rep deferred to pre-2.2.0."
+  - route: "POST /auth/oidc/back-channel-logout"
+    why: "Bundle 2 Phase 5 RFC OIDC Back-Channel Logout 1.0 endpoint. OpenAPI rep deferred to pre-2.2.0."
+  - route: "GET /api/v1/auth/sessions"
+    why: "Bundle 2 Phase 5 self/admin session list. OpenAPI rep deferred to pre-2.2.0."
+  - route: "DELETE /api/v1/auth/sessions/{id}"
+    why: "Bundle 2 Phase 5 session revoke. OpenAPI rep deferred to pre-2.2.0."
+  - route: "DELETE /api/v1/auth/sessions"
+    why: "Bundle 2 audit-2026-05-10 MED-2/3 revoke-all-except-current."
+  - route: "GET /api/v1/auth/oidc/providers"
+    why: "Bundle 2 Phase 5 OIDC provider CRUD (list)."
+  - route: "POST /api/v1/auth/oidc/providers"
+    why: "Bundle 2 Phase 5 OIDC provider CRUD (create)."
+  - route: "PUT /api/v1/auth/oidc/providers/{id}"
+    why: "Bundle 2 Phase 5 OIDC provider CRUD (update)."
+  - route: "DELETE /api/v1/auth/oidc/providers/{id}"
+    why: "Bundle 2 Phase 5 OIDC provider CRUD (delete)."
+  - route: "POST /api/v1/auth/oidc/providers/{id}/refresh"
+    why: "Bundle 2 audit-2026-05-10 MED-7 JWKS hot-refresh."
+  - route: "GET /api/v1/auth/oidc/providers/{id}/jwks-status"
+    why: "Bundle 2 audit-2026-05-10 MED-7 JWKS health snapshot."
+  - route: "POST /api/v1/auth/oidc/test"
+    why: "Bundle 2 audit-2026-05-10 MED-5 dry-run discovery + JWKS + alg-downgrade check."
+  - route: "GET /api/v1/auth/oidc/group-mappings"
+    why: "Bundle 2 Phase 5 group-mapping CRUD (list)."
+  - route: "POST /api/v1/auth/oidc/group-mappings"
+    why: "Bundle 2 Phase 5 group-mapping CRUD (create)."
+  - route: "DELETE /api/v1/auth/oidc/group-mappings/{id}"
+    why: "Bundle 2 Phase 5 group-mapping CRUD (delete)."
+  - route: "GET /api/v1/auth/breakglass/credentials"
+    why: "Bundle 2 Phase 7.5 admin break-glass list (404 when disabled; password hash never on wire)."
+  - route: "POST /api/v1/auth/breakglass/credentials"
+    why: "Bundle 2 Phase 7.5 admin break-glass set/rotate password."
+  - route: "POST /api/v1/auth/breakglass/credentials/{actor_id}/unlock"
+    why: "Bundle 2 Phase 7.5 admin break-glass unlock after lockout."
+  - route: "DELETE /api/v1/auth/breakglass/credentials/{actor_id}"
+    why: "Bundle 2 Phase 7.5 admin break-glass credential delete."
+  - route: "GET /api/v1/auth/users"
+    why: "Bundle 2 audit-2026-05-10 MED-11 users page."
+  - route: "DELETE /api/v1/auth/users/{id}"
+    why: "Bundle 2 audit-2026-05-10 MED-11 user deactivate."
+  - route: "POST /api/v1/auth/users/{id}/reactivate"
+    why: "Bundle 2 audit-2026-05-10 MED-11 user reactivate."
+  - route: "GET /api/v1/auth/runtime-config"
+    why: "Bundle 2 audit-2026-05-10 MED-12 effective auth-runtime-config (read-only)."
+  - route: "POST /api/v1/auth/demo-residual/cleanup"
+    why: "Audit 2026-05-11 A-8 demo-mode residual-grants cleanup endpoint."
+  - route: "GET /api/v1/audit/export"
+    why: "Bundle 1 Phase 8 streaming NDJSON audit export."

api/openapi.yaml

@@ -134,12 +134,23 @@ paths:
                     type: string
                     # G-1 (P1): "jwt" removed from this enum after the silent
                     # auth downgrade was identified — no JWT middleware ships
-                    # with certctl. Operators who need JWT/OIDC front certctl
+                    # with certctl. Operators who need JWT continue to front
-                    # with an authenticating gateway (oauth2-proxy / Envoy /
+                    # certctl with an authenticating gateway (oauth2-proxy /
-                    # Traefik / Pomerium) and set CERTCTL_AUTH_TYPE=none
+                    # Envoy / Traefik / Pomerium) and set
-                    # upstream. See docs/architecture.md "Authenticating-
+                    # CERTCTL_AUTH_TYPE=none upstream. See
-                    # gateway pattern".
+                    # docs/architecture.md "Authenticating-gateway pattern".
-                    enum: [api-key, none]
+                    #
+                    # Auth Bundle 2 Phase 0: "oidc" added to the enum. The
+                    # session middleware + OIDC handler chain ship in later
+                    # Bundle 2 phases; until they land, setting
+                    # CERTCTL_AUTH_TYPE=oidc fails the runtime guard in
+                    # cmd/server/main.go with an actionable error rather
+                    # than silently falling back to api-key (the G-1
+                    # failure mode). The literal is in the enum so the GUI
+                    # Login page (Phase 8) can render OIDC provider
+                    # buttons against an /auth/info response that reflects
+                    # the configured auth_type.
+                    enum: [api-key, none, oidc]
                   required:
                     type: boolean
 
@@ -147,7 +158,16 @@ paths:
     get:
       tags: [Health]
       summary: Validate credentials
-      description: Returns 200 if auth credentials are valid, 401 otherwise.
+      description: |
+        Returns 200 if auth credentials are valid, 401 otherwise.
+
+        Bundle 1 Phase 3 closure (M1): when the server has the RBAC
+        primitive wired (Bundle 1 default), the response also includes
+        the caller's `actor_id`, `actor_type`, `tenant_id`, the
+        `roles` they hold, and `effective_permissions` they resolve
+        to. The legacy `admin` boolean is preserved for back-compat
+        with pre-Bundle-1 GUIs; new GUIs should switch to
+        `effective_permissions` for affordance gating.
       operationId: checkAuth
       responses:
         "200":
@@ -156,13 +176,464 @@ paths:
             application/json:
               schema:
                 type: object
+                required: [status]
                 properties:
                   status:
                     type: string
                     example: authenticated
+                  user:
+                    type: string
+                    description: Named-key identity (empty when CERTCTL_AUTH_TYPE=none)
+                  admin:
+                    type: boolean
+                    description: Legacy admin flag (back-compat with pre-Bundle-1 GUIs).
+                  actor_id:
+                    type: string
+                    description: Actor identifier for the authenticated request (Bundle 1+).
+                  actor_type:
+                    type: string
+                    enum: [User, System, Agent, APIKey, Anonymous]
+                    description: Actor-type discriminator (Bundle 1+).
+                  tenant_id:
+                    type: string
+                    description: Tenant the actor belongs to (Bundle 1 ships single-tenant `t-default`).
+                  admin_via_role:
+                    type: boolean
+                    description: True when the actor holds `r-admin`. Authoritative admin signal under Bundle 1+.
+                  roles:
+                    type: array
+                    items:
+                      type: string
+                    description: Role IDs (e.g. `r-admin`, `r-viewer`) the actor holds.
+                  effective_permissions:
+                    type: array
+                    items:
+                      type: object
+                      required: [permission, scope_type]
+                      properties:
+                        permission:
+                          type: string
+                          example: cert.bulk_revoke
+                        scope_type:
+                          type: string
+                          enum: [global, profile, issuer]
+                        scope_id:
+                          type: string
         "401":
           description: Unauthorized
 
+  # ─── Auth / RBAC (Bundle 1 Phase 4) ─────────────────────────────────
+  # The RBAC primitive surface for managing roles, permissions, and the
+  # role grants assigned to actors (API keys today; OIDC-federated users
+  # in Bundle 2). Every mutating route runs through the service layer's
+  # privilege-escalation guard — callers need `auth.role.assign` for
+  # role grants on actors, `auth.role.create/edit/delete` for the role
+  # lifecycle, `auth.key.*` for key management. Read endpoints require
+  # `auth.role.list`. The /v1/auth/me endpoint has no permission gate
+  # (every authenticated caller can read their own permissions).
+  /api/v1/auth/bootstrap:
+    get:
+      tags: [Auth]
+      summary: Probe whether the day-0 bootstrap endpoint is callable
+      description: |
+        Returns `{available: true}` when CERTCTL_BOOTSTRAP_TOKEN is set
+        AND no admin-roled actor exists yet; otherwise `{available: false}`.
+        Auth-exempt because it serves the GUI / install one-liner before
+        the first admin key has been minted. Bundle 1 Phase 6.
+      security: []
+      operationId: getAuthBootstrap
+      responses:
+        "200":
+          description: Bootstrap availability
+          content:
+            application/json:
+              schema:
+                type: object
+                required: [available]
+                properties:
+                  available:
+                    type: boolean
+    post:
+      tags: [Auth]
+      summary: Mint the first admin API key from a one-shot bootstrap token
+      description: |
+        Operator POSTs the CERTCTL_BOOTSTRAP_TOKEN value plus the desired
+        admin-key name. Returns the freshly minted plaintext key value
+        once; the server stores only the SHA-256 hash. Subsequent calls
+        return 410 Gone (the strategy is one-shot AND the admin-existence
+        probe re-closes the door once the new admin lands). Auth-exempt
+        because the endpoint authenticates via the bootstrap token
+        itself. Bundle 1 Phase 6.
+      security: []
+      operationId: postAuthBootstrap
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [token, actor_name]
+              properties:
+                token:
+                  type: string
+                  description: The CERTCTL_BOOTSTRAP_TOKEN value (constant-time compared server-side).
+                actor_name:
+                  type: string
+                  description: 3-64 chars, lowercase alphanumeric + hyphen + underscore.
+                  pattern: "^[a-z0-9][a-z0-9_-]{2,63}$"
+      responses:
+        "201":
+          description: Admin key minted
+          content:
+            application/json:
+              schema:
+                type: object
+                required: [actor_id, api_key_id, key_value, created_at, message]
+                properties:
+                  actor_id: { type: string }
+                  api_key_id: { type: string }
+                  key_value:
+                    type: string
+                    description: The plaintext API key. Capture this — it is shown only once.
+                  created_at: { type: string, format: date-time }
+                  message: { type: string }
+        "400": { description: Invalid actor_name or malformed body }
+        "401": { description: Bootstrap token mismatch }
+        "410":
+          description: |
+            Endpoint disabled. Either CERTCTL_BOOTSTRAP_TOKEN is unset,
+            an admin actor already exists, or the strategy was already
+            consumed by a successful prior call.
+
+  /api/v1/auth/me:
+    get:
+      tags: [Auth]
+      summary: Current actor's roles + effective permissions
+      description: |
+        Returns the standing roles + effective permission set for the
+        authenticated caller. This is the query the GUI uses to gate
+        affordance rendering; /api/v1/auth/check returns the same shape
+        on the boot path.
+      operationId: getAuthMe
+      responses:
+        "200":
+          description: Caller identity + roles + effective permissions
+          content:
+            application/json:
+              schema:
+                type: object
+                required: [actor_id, actor_type, tenant_id, admin, roles, effective_permissions]
+                properties:
+                  actor_id: { type: string }
+                  actor_type: { type: string, enum: [User, System, Agent, APIKey, Anonymous] }
+                  tenant_id: { type: string }
+                  admin: { type: boolean }
+                  roles:
+                    type: array
+                    items: { type: string }
+                  effective_permissions:
+                    type: array
+                    items:
+                      type: object
+                      required: [permission, scope_type]
+                      properties:
+                        permission: { type: string }
+                        scope_type: { type: string, enum: [global, profile, issuer] }
+                        scope_id: { type: string }
+        "401":
+          description: Unauthorized
+
+  /api/v1/auth/permissions:
+    get:
+      tags: [Auth]
+      summary: List canonical permission catalogue
+      description: |
+        Returns every permission name registered in the canonical
+        catalogue. Used by the GUI's role editor to populate the
+        "grant permission" picker. Permission: `auth.role.list`.
+      operationId: listAuthPermissions
+      responses:
+        "200":
+          description: Permission catalogue
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  permissions:
+                    type: array
+                    items:
+                      type: object
+                      required: [id, name, namespace]
+                      properties:
+                        id: { type: string }
+                        name: { type: string }
+                        namespace: { type: string }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+
+  /api/v1/auth/roles:
+    get:
+      tags: [Auth]
+      summary: List roles for the active tenant
+      description: Permission `auth.role.list`. Returns every role registered for `t-default` (Bundle 1 single-tenant).
+      operationId: listAuthRoles
+      responses:
+        "200":
+          description: Role list
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  roles:
+                    type: array
+                    items: { $ref: "#/components/schemas/AuthRole" }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+    post:
+      tags: [Auth]
+      summary: Create a custom role
+      description: Permission `auth.role.create`. Default roles (`r-admin` / `r-operator` / `r-viewer` / `r-agent` / `r-mcp` / `r-cli` / `r-auditor`) are seeded by migration and immutable.
+      operationId: createAuthRole
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [name]
+              properties:
+                name: { type: string }
+                description: { type: string }
+      responses:
+        "201":
+          description: Role created
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/AuthRole" }
+        "400": { description: Validation error }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "409": { description: Role with that name already exists }
+
+  /api/v1/auth/roles/{id}:
+    get:
+      tags: [Auth]
+      summary: Get a role and its permissions
+      description: Permission `auth.role.list`.
+      operationId: getAuthRole
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+      responses:
+        "200":
+          description: Role + permissions
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  role: { $ref: "#/components/schemas/AuthRole" }
+                  permissions:
+                    type: array
+                    items: { $ref: "#/components/schemas/AuthRolePermission" }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role not found }
+    put:
+      tags: [Auth]
+      summary: Update a custom role's name or description
+      description: Permission `auth.role.edit`. Default roles cannot be renamed.
+      operationId: updateAuthRole
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                name: { type: string }
+                description: { type: string }
+      responses:
+        "200": { description: Updated }
+        "400": { description: Validation error }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role not found }
+        "409": { description: Default role cannot be renamed / name collision }
+    delete:
+      tags: [Auth]
+      summary: Delete a custom role
+      description: Permission `auth.role.delete`. Fails with 409 when actors still hold the role (FK ON DELETE RESTRICT).
+      operationId: deleteAuthRole
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+      responses:
+        "204": { description: Deleted }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role not found }
+        "409": { description: Role still has active actor assignments }
+
+  /api/v1/auth/roles/{id}/permissions:
+    post:
+      tags: [Auth]
+      summary: Grant a permission to a role at a scope
+      description: Permission `auth.role.edit`. ScopeType defaults to `global`; per-profile / per-issuer scopes require ScopeID.
+      operationId: grantAuthRolePermission
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [permission]
+              properties:
+                permission: { type: string }
+                scope_type:
+                  type: string
+                  enum: [global, profile, issuer]
+                  default: global
+                scope_id: { type: string }
+      responses:
+        "204": { description: Granted }
+        "400": { description: Permission not in canonical catalogue / scope_id missing for non-global scope }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role not found }
+
+  /api/v1/auth/roles/{id}/permissions/{perm}:
+    delete:
+      tags: [Auth]
+      summary: Revoke a permission from a role
+      description: Permission `auth.role.edit`.
+      operationId: revokeAuthRolePermission
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+        - in: path
+          name: perm
+          required: true
+          schema: { type: string }
+        - in: query
+          name: scope_type
+          schema:
+            type: string
+            enum: [global, profile, issuer]
+        - in: query
+          name: scope_id
+          schema: { type: string }
+      responses:
+        "204": { description: Revoked }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role or permission grant not found }
+
+  /api/v1/auth/keys:
+    get:
+      tags: [Auth]
+      summary: List actors with role grants in the active tenant
+      description: |
+        Returns every distinct (actor_id, actor_type) pair in the
+        tenant that holds at least one role grant. Bundle 1 Phase 7
+        ships this so the CLI's `auth keys list` and scope-down helper
+        can enumerate the operator-key population without joining
+        against the env-var-loaded namedKeys directly. Permission
+        `auth.role.list`.
+      operationId: listAuthKeys
+      responses:
+        "200":
+          description: Actor list with role assignments
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  keys:
+                    type: array
+                    items:
+                      type: object
+                      required: [actor_id, actor_type, tenant_id, role_ids]
+                      properties:
+                        actor_id: { type: string }
+                        actor_type:
+                          type: string
+                          enum: [User, System, Agent, APIKey, Anonymous]
+                        tenant_id: { type: string }
+                        role_ids:
+                          type: array
+                          items: { type: string }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+
+  /api/v1/auth/keys/{id}/roles:
+    post:
+      tags: [Auth]
+      summary: Assign a role to an API key
+      description: Permission `auth.role.assign`. The reserved `actor-demo-anon` actor cannot be re-assigned.
+      operationId: assignAuthKeyRole
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [role_id]
+              properties:
+                role_id: { type: string }
+      responses:
+        "204": { description: Assigned }
+        "400": { description: Validation error }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role not found }
+        "409": { description: Reserved system actor cannot be modified }
+
+  /api/v1/auth/keys/{id}/roles/{role_id}:
+    delete:
+      tags: [Auth]
+      summary: Revoke a role from an API key
+      description: Permission `auth.role.assign`. Revoking the synthetic `actor-demo-anon` admin grant is rejected.
+      operationId: revokeAuthKeyRole
+      parameters:
+        - in: path
+          name: id
+          required: true
+          schema: { type: string }
+        - in: path
+          name: role_id
+          required: true
+          schema: { type: string }
+      responses:
+        "204": { description: Revoked }
+        "401": { description: Unauthorized }
+        "403": { description: Forbidden }
+        "404": { description: Role not assigned to actor }
+        "409": { description: Reserved system actor cannot be modified }
+
   /api/v1/version:
     get:
       tags: [Health]
@@ -205,7 +676,7 @@ paths:
                   go_version:
                     type: string
                     description: Go toolchain version that compiled the binary (runtime.Version())
-                    example: go1.25.9
+                    example: go1.25.10
 
   # ─── Certificates ────────────────────────────────────────────────────
   /api/v1/certificates:
@@ -2708,10 +3179,22 @@ paths:
     get:
       tags: [Audit]
       summary: List audit events
+      description: |
+        Bundle 1 Phase 8 adds the optional `category` query parameter
+        for auditor-role filtering. Allowed values: `cert_lifecycle`
+        (cert/agent/deployment events), `auth` (role/key/bootstrap
+        mutations), `config` (issuer/target/settings edits). Omitting
+        the parameter returns every category.
       operationId: listAuditEvents
       parameters:
         - $ref: "#/components/parameters/page"
         - $ref: "#/components/parameters/per_page"
+        - in: query
+          name: category
+          schema:
+            type: string
+            enum: [cert_lifecycle, auth, config]
+          description: Filter to events of this event_category. (Bundle 1 Phase 8)
       responses:
         "200":
           description: Paginated list of audit events
@@ -2726,6 +3209,8 @@ paths:
                         type: array
                         items:
                           $ref: "#/components/schemas/AuditEvent"
+        "400":
+          description: Invalid `category` value
         "500":
           $ref: "#/components/responses/InternalError"
 
@@ -4309,6 +4794,27 @@ components:
       type: http
       scheme: bearer
       description: API key passed as Bearer token. Configure via CERTCTL_AUTH_SECRET.
+    # Auth Bundle 2 Phase 5 — session-cookie auth scheme. New
+    # session-authenticated endpoints declare
+    # `security: [{cookieAuth: []}, {bearerAuth: []}]` (either auth
+    # method works, OR semantics). Per Phase 5 spec, the
+    # `/auth/oidc/back-channel-logout` endpoint declares `security: []`
+    # because auth comes from the IdP-signed logout token in the body,
+    # not certctl-issued credentials.
+    cookieAuth:
+      type: apiKey
+      in: cookie
+      name: certctl_session
+      description: |
+        Session cookie minted by `POST /auth/oidc/callback` after a
+        successful OIDC handshake (Auth Bundle 2). Wire format
+        `v1.<session_id>.<signing_key_id>.<HMAC-SHA256>`; HMAC is
+        verified server-side against the active session signing key.
+        Cookie attributes: `Secure` `HttpOnly` `SameSite=Lax|Strict`
+        (configurable via `CERTCTL_SESSION_SAMESITE`) `Path=/`.
+        State-changing requests additionally require the
+        `X-CSRF-Token` header to match the SHA-256 hash on the
+        session row (validated by the session middleware in Phase 6).
 
   parameters:
     resourceId:
@@ -4361,6 +4867,45 @@ components:
             $ref: "#/components/schemas/ErrorResponse"
 
   schemas:
+    # ─── Auth / RBAC (Bundle 1 Phase 4) ─────────────────────────────
+    AuthRole:
+      type: object
+      required: [id, tenant_id, name]
+      properties:
+        id:
+          type: string
+          description: Role ID (`r-` prefix).
+          example: r-admin
+        tenant_id:
+          type: string
+          example: t-default
+        name:
+          type: string
+          example: admin
+        description:
+          type: string
+        created_at:
+          type: string
+          format: date-time
+        updated_at:
+          type: string
+          format: date-time
+
+    AuthRolePermission:
+      type: object
+      required: [role_id, permission_id, scope_type]
+      properties:
+        role_id:
+          type: string
+        permission_id:
+          type: string
+        scope_type:
+          type: string
+          enum: [global, profile, issuer]
+        scope_id:
+          type: string
+          description: NULL/absent for global scope; profile/issuer ID otherwise.
+
     # ─── Approvals ───────────────────────────────────────────────────
     ApprovalRequest:
       type: object
@@ -5311,6 +5856,13 @@ components:
         timestamp:
           type: string
           format: date-time
+        event_category:
+          type: string
+          enum: [cert_lifecycle, auth, config]
+          description: |
+            Bundle 1 Phase 8: classifies the event for auditor-role
+            filtering. Empty / absent on rows from pre-Phase-8
+            deployments (the migration backfills "cert_lifecycle").
 
     # ─── Notifications ───────────────────────────────────────────────
     NotificationType:

cmd/agent/keymem.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (

cmd/agent/main.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (
@@ -699,6 +702,26 @@ func (a *Agent) executeDeploymentJob(ctx context.Context, job JobItem) {
 			return
 		}
 
+		// Bundle 1 / RT-C1 closure (2026-05-12): defense in depth. The server
+		// runs internal/connector/target/configcheck.Validate on the way IN
+		// (Create/Update), and rejects shell metacharacters in command-bearing
+		// fields. Re-run the connector's full ValidateConfig here on the way
+		// OUT, before any DeployCertificate call. This catches (a) configs
+		// that pre-date the server-side guard, (b) corruption/tampering of
+		// the encrypted config blob, and (c) per-connector filesystem
+		// invariants (cert dir exists, paths writable) that the server can't
+		// check because the filesystem is on the agent host.
+		if err := connector.ValidateConfig(ctx, job.TargetConfig); err != nil {
+			a.logger.Error("connector config validation failed",
+				"job_id", job.ID,
+				"target_type", job.TargetType,
+				"error", err)
+			if reportErr := a.reportJobStatus(ctx, job.ID, "Failed", fmt.Sprintf("%s config validation failed: %v", job.TargetType, err)); reportErr != nil {
+				a.logger.Error("failed to report job status to server", "job_id", job.ID, "status", "Failed", "error", reportErr)
+			}
+			return
+		}
+
 		deployReq := target.DeploymentRequest{
 			CertPEM:      certOnly,
 			KeyPEM:       keyPEM,

cmd/agent/verify.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (

cmd/cli/main.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (
@@ -111,6 +114,8 @@ Examples:
 		err = handleEST(client, cmdArgs)
 	case "status":
 		err = handleStatus(client)
+	case "auth":
+		err = handleAuth(client, cmdArgs)
 	case "version":
 		fmt.Println("certctl-cli version 0.1.0")
 	default:
@@ -364,3 +369,123 @@ func validateHTTPSScheme(serverURL string) error {
 		return fmt.Errorf("server URL %q uses unsupported scheme %q — expected https://", serverURL, u.Scheme)
 	}
 }
+
+// handleAuth dispatches the `certctl-cli auth ...` subcommand tree.
+// Bundle 1 Phase 5: ships read + grant operations against the
+// /api/v1/auth/* surface introduced in Phase 4. Mutations like role
+// create / update / delete can be added in a Phase 5.5 follow-up; this
+// commit ships the operator-facing subset most useful for migration
+// and day-2 scope-down (`auth keys list` + `auth keys assign` +
+// `auth me`).
+func handleAuth(client *cli.Client, args []string) error {
+	if len(args) == 0 {
+		fmt.Fprintf(os.Stderr, "usage: auth <roles|permissions|keys|me> [...]\n")
+		return nil
+	}
+	subcommand := args[0]
+	subArgs := args[1:]
+
+	switch subcommand {
+	case "roles":
+		return handleAuthRoles(client, subArgs)
+	case "permissions":
+		return handleAuthPermissions(client, subArgs)
+	case "keys":
+		return handleAuthKeys(client, subArgs)
+	case "me":
+		return client.AuthMe()
+	default:
+		fmt.Fprintf(os.Stderr, "unknown auth subcommand: %s\n", subcommand)
+		return nil
+	}
+}
+
+func handleAuthRoles(client *cli.Client, args []string) error {
+	if len(args) == 0 {
+		fmt.Fprintf(os.Stderr, "usage: auth roles <list|get> [id]\n")
+		return nil
+	}
+	switch args[0] {
+	case "list":
+		return client.AuthListRoles()
+	case "get":
+		if len(args) < 2 {
+			fmt.Fprintf(os.Stderr, "usage: auth roles get <id>\n")
+			return nil
+		}
+		return client.AuthGetRole(args[1])
+	default:
+		fmt.Fprintf(os.Stderr, "unknown roles subcommand: %s\n", args[0])
+		return nil
+	}
+}
+
+func handleAuthPermissions(client *cli.Client, args []string) error {
+	if len(args) == 0 || args[0] != "list" {
+		fmt.Fprintf(os.Stderr, "usage: auth permissions list\n")
+		return nil
+	}
+	return client.AuthListPermissions()
+}
+
+func handleAuthKeys(client *cli.Client, args []string) error {
+	if len(args) == 0 {
+		fmt.Fprintf(os.Stderr, "usage: auth keys <list|assign|revoke|scope-down> [...]\n")
+		return nil
+	}
+	switch args[0] {
+	case "list":
+		return client.AuthListKeys()
+	case "assign":
+		// auth keys assign <key-id> --role <role-id>
+		if len(args) < 4 || args[2] != "--role" {
+			fmt.Fprintf(os.Stderr, "usage: auth keys assign <key-id> --role <role-id>\n")
+			return nil
+		}
+		return client.AuthAssignRoleToKey(args[1], args[3])
+	case "revoke":
+		// auth keys revoke <key-id> --role <role-id>
+		if len(args) < 4 || args[2] != "--role" {
+			fmt.Fprintf(os.Stderr, "usage: auth keys revoke <key-id> --role <role-id>\n")
+			return nil
+		}
+		return client.AuthRevokeRoleFromKey(args[1], args[3])
+	case "scope-down":
+		// Bundle 1 Phase 7 — interactive (default), --non-interactive
+		// <config.json>, or --suggest [--apply].
+		return handleAuthKeysScopeDown(client, args[1:])
+	default:
+		fmt.Fprintf(os.Stderr, "unknown keys subcommand: %s\n", args[0])
+		return nil
+	}
+}
+
+// handleAuthKeysScopeDown dispatches the three scope-down modes:
+//
+//	auth keys scope-down                              → interactive
+//	auth keys scope-down --non-interactive <config>   → JSON-driven
+//	auth keys scope-down --suggest [--apply]          → audit-driven suggestions
+func handleAuthKeysScopeDown(client *cli.Client, args []string) error {
+	if len(args) == 0 {
+		return client.AuthScopeDown()
+	}
+	switch args[0] {
+	case "--non-interactive":
+		if len(args) < 2 {
+			fmt.Fprintf(os.Stderr, "usage: auth keys scope-down --non-interactive <config.json>\n")
+			return nil
+		}
+		return client.AuthScopeDownNonInteractive(args[1])
+	case "--suggest":
+		apply := false
+		for _, a := range args[1:] {
+			if a == "--apply" {
+				apply = true
+			}
+		}
+		return client.AuthScopeDownSuggest(apply)
+	default:
+		fmt.Fprintf(os.Stderr, "unknown scope-down flag: %s\n", args[0])
+		return nil
+	}
+}

cmd/mcp-server/main.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (

cmd/server/auth_backfill.go

@@ -0,0 +1,108 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
+package main
+
+import (
+	"context"
+	"fmt"
+	"log/slog"
+	"strings"
+
+	"github.com/certctl-io/certctl/internal/auth"
+	"github.com/certctl-io/certctl/internal/config"
+	"github.com/certctl-io/certctl/internal/domain"
+	authdomain "github.com/certctl-io/certctl/internal/domain/auth"
+)
+
+// assembleNamedAPIKeys translates the operator's CERTCTL_API_KEYS_NAMED
+// env-var (preferred) or CERTCTL_AUTH_SECRET (legacy) into the
+// auth.NamedAPIKey slice the rest of the boot path consumes.
+//
+// Authentication unification (M-002): every authenticated request now
+// carries a named actor in the request context so audit events record
+// the real key identity instead of the hardcoded "api-key-user"
+// string. Named keys come from CERTCTL_API_KEYS_NAMED (preferred). For
+// backward compatibility CERTCTL_AUTH_SECRET is synthesized into
+// legacy-key-N entries with Admin=false.
+func assembleNamedAPIKeys(cfg *config.Config, logger *slog.Logger) []auth.NamedAPIKey {
+	if config.AuthType(cfg.Auth.Type) == config.AuthTypeNone {
+		return nil
+	}
+	var out []auth.NamedAPIKey
+	for _, nk := range cfg.Auth.NamedKeys {
+		out = append(out, auth.NamedAPIKey{
+			Name:  nk.Name,
+			Key:   nk.Key,
+			Admin: nk.Admin,
+		})
+	}
+	if len(out) == 0 && cfg.Auth.Secret != "" {
+		idx := 0
+		for _, p := range strings.Split(cfg.Auth.Secret, ",") {
+			p = strings.TrimSpace(p)
+			if p == "" {
+				continue
+			}
+			out = append(out, auth.NamedAPIKey{
+				Name:  fmt.Sprintf("legacy-key-%d", idx),
+				Key:   p,
+				Admin: false,
+			})
+			idx++
+		}
+		if len(out) > 0 && logger != nil {
+			logger.Warn("CERTCTL_AUTH_SECRET is deprecated — set CERTCTL_API_KEYS_NAMED for named actor attribution and admin gating",
+				"synthesized_keys", len(out))
+		}
+	}
+	return out
+}
+
+// actorRoleGranter is the narrow interface backfillNamedKeyActorRoles
+// needs from the postgres ActorRoleRepository. Pulled out so the unit
+// test can inject a fake without spinning up the full repo / DB.
+type actorRoleGranter interface {
+	Grant(ctx context.Context, ar *authdomain.ActorRole) error
+}
+
+// backfillNamedKeyActorRoles is the Bundle 1 Phase 3 closure (C2)
+// startup hook that ensures every CERTCTL_API_KEYS_NAMED entry — and
+// every legacy CERTCTL_AUTH_SECRET synthesized fallback — has an
+// actor_roles row before the HTTP server accepts requests. Admin-flagged
+// keys grant `r-admin` (full canonical permission set); non-admin keys
+// grant `r-viewer` (read-only surface), matching the pre-Phase-3.5
+// capability shape.
+//
+// Idempotent via ON CONFLICT DO NOTHING in the repo Grant — reboots
+// don't create duplicates. Failures are logged but non-fatal: the server
+// still starts, and the operator can fix the grant via the RBAC API.
+//
+// The function is package-private + extracted from main() so the unit
+// test in auth_backfill_test.go can pin the role-mapping invariant
+// without depending on the full server bootstrap path.
+func backfillNamedKeyActorRoles(
+	ctx context.Context,
+	repo actorRoleGranter,
+	keys []auth.NamedAPIKey,
+	logger *slog.Logger,
+) {
+	for _, nk := range keys {
+		role := authdomain.RoleIDViewer
+		if nk.Admin {
+			role = authdomain.RoleIDAdmin
+		}
+		if err := repo.Grant(ctx, &authdomain.ActorRole{
+			ActorID:   nk.Name,
+			ActorType: authdomain.ActorTypeValue(domain.ActorTypeAPIKey),
+			RoleID:    role,
+			TenantID:  authdomain.DefaultTenantID,
+			GrantedBy: "bootstrap",
+		}); err != nil {
+			if logger != nil {
+				logger.Warn("api-key actor-role backfill failed; key authenticates but RBAC routes will 403 until grant is added via /v1/auth/keys",
+					"key", nk.Name, "role", role, "err", err)
+			}
+		}
+	}
+}

cmd/server/auth_backfill_test.go

@@ -0,0 +1,116 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"io"
+	"log/slog"
+	"testing"
+
+	"github.com/certctl-io/certctl/internal/auth"
+	authdomain "github.com/certctl-io/certctl/internal/domain/auth"
+)
+
+// fakeGranter is a tiny in-memory stand-in for the postgres ActorRoleRepository
+// — enough surface area for backfillNamedKeyActorRoles to call Grant against.
+type fakeGranter struct {
+	calls []*authdomain.ActorRole
+	err   error
+}
+
+func (f *fakeGranter) Grant(_ context.Context, ar *authdomain.ActorRole) error {
+	f.calls = append(f.calls, ar)
+	return f.err
+}
+
+// TestBackfillNamedKeyActorRoles_RoleMapping pins the Bundle 1 Phase 3
+// closure (C2) invariant: admin-flagged named keys grant r-admin,
+// non-admin keys grant r-viewer, both at TenantID t-default with
+// ActorType APIKey and GrantedBy=bootstrap.
+func TestBackfillNamedKeyActorRoles_RoleMapping(t *testing.T) {
+	repo := &fakeGranter{}
+	logger := slog.New(slog.NewTextHandler(io.Discard, nil))
+
+	keys := []auth.NamedAPIKey{
+		{Name: "alice-admin", Key: "AAA", Admin: true},
+		{Name: "bob-viewer", Key: "BBB", Admin: false},
+		{Name: "carol-admin", Key: "CCC", Admin: true},
+	}
+	backfillNamedKeyActorRoles(context.Background(), repo, keys, logger)
+
+	if len(repo.calls) != 3 {
+		t.Fatalf("Grant call count = %d, want 3", len(repo.calls))
+	}
+	type want struct {
+		actor, role string
+	}
+	wants := []want{
+		{actor: "alice-admin", role: authdomain.RoleIDAdmin},
+		{actor: "bob-viewer", role: authdomain.RoleIDViewer},
+		{actor: "carol-admin", role: authdomain.RoleIDAdmin},
+	}
+	for i, w := range wants {
+		got := repo.calls[i]
+		if got.ActorID != w.actor {
+			t.Errorf("call[%d].ActorID = %q, want %q", i, got.ActorID, w.actor)
+		}
+		if got.RoleID != w.role {
+			t.Errorf("call[%d].RoleID = %q, want %q", i, got.RoleID, w.role)
+		}
+		if got.TenantID != authdomain.DefaultTenantID {
+			t.Errorf("call[%d].TenantID = %q, want %q", i, got.TenantID, authdomain.DefaultTenantID)
+		}
+		if string(got.ActorType) != "APIKey" {
+			t.Errorf("call[%d].ActorType = %q, want APIKey", i, got.ActorType)
+		}
+		if got.GrantedBy != "bootstrap" {
+			t.Errorf("call[%d].GrantedBy = %q, want bootstrap", i, got.GrantedBy)
+		}
+	}
+}
+
+// TestBackfillNamedKeyActorRoles_EmptyKeysIsNoOp confirms the boot path
+// is safe when no named keys are configured (typical CERTCTL_AUTH_TYPE=
+// none deploy). No Grant calls; no panic.
+func TestBackfillNamedKeyActorRoles_EmptyKeysIsNoOp(t *testing.T) {
+	repo := &fakeGranter{}
+	logger := slog.New(slog.NewTextHandler(io.Discard, nil))
+	backfillNamedKeyActorRoles(context.Background(), repo, nil, logger)
+	if len(repo.calls) != 0 {
+		t.Errorf("Grant called %d times for empty keys, want 0", len(repo.calls))
+	}
+}
+
+// TestBackfillNamedKeyActorRoles_GrantErrorIsNonFatal confirms the
+// closure invariant that a Grant failure logs a warning and proceeds
+// rather than crashing the server during boot. Subsequent keys still
+// get processed.
+func TestBackfillNamedKeyActorRoles_GrantErrorIsNonFatal(t *testing.T) {
+	repo := &fakeGranter{err: errors.New("simulated DB error")}
+	logger := slog.New(slog.NewTextHandler(io.Discard, nil))
+
+	keys := []auth.NamedAPIKey{
+		{Name: "alice", Key: "A", Admin: true},
+		{Name: "bob", Key: "B", Admin: false},
+	}
+	// Should not panic.
+	backfillNamedKeyActorRoles(context.Background(), repo, keys, logger)
+
+	if len(repo.calls) != 2 {
+		t.Errorf("Grant calls = %d, want 2 (every key processed even when prior Grant errored)", len(repo.calls))
+	}
+}
+
+// TestBackfillNamedKeyActorRoles_NilLoggerIsSafe pins that callers
+// passing nil for the logger don't NPE the goroutine. Belt-and-braces
+// for tests + future call sites that may not have a logger plumbed.
+func TestBackfillNamedKeyActorRoles_NilLoggerIsSafe(t *testing.T) {
+	repo := &fakeGranter{err: errors.New("simulated")}
+	keys := []auth.NamedAPIKey{
+		{Name: "alice", Key: "A", Admin: true},
+	}
+	backfillNamedKeyActorRoles(context.Background(), repo, keys, nil)
+	if len(repo.calls) != 1 {
+		t.Errorf("Grant calls = %d, want 1", len(repo.calls))
+	}
+}

cmd/server/main.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (
@@ -5,6 +8,7 @@ import (
 	"crypto"
 	"crypto/tls"
 	"crypto/x509"
+	"encoding/json"
 	"encoding/pem"
 	"fmt"
 	"log/slog"
@@ -21,6 +25,13 @@ import (
 	"github.com/certctl-io/certctl/internal/api/handler"
 	"github.com/certctl-io/certctl/internal/api/middleware"
 	"github.com/certctl-io/certctl/internal/api/router"
+	"github.com/certctl-io/certctl/internal/auth"
+	"github.com/certctl-io/certctl/internal/auth/bootstrap"
+	"github.com/certctl-io/certctl/internal/auth/breakglass"
+	oidcsvc "github.com/certctl-io/certctl/internal/auth/oidc"
+	oidcdomain "github.com/certctl-io/certctl/internal/auth/oidc/domain"
+	"github.com/certctl-io/certctl/internal/auth/session"
+	userdomain "github.com/certctl-io/certctl/internal/auth/user/domain"
 	"github.com/certctl-io/certctl/internal/config"
 	discoveryawssm "github.com/certctl-io/certctl/internal/connector/discovery/awssm"
 	discoveryazurekv "github.com/certctl-io/certctl/internal/connector/discovery/azurekv"
@@ -32,11 +43,14 @@ import (
 	notifyteams "github.com/certctl-io/certctl/internal/connector/notifier/teams"
 	"github.com/certctl-io/certctl/internal/crypto/signer"
 	"github.com/certctl-io/certctl/internal/domain"
+	authdomainAlias "github.com/certctl-io/certctl/internal/domain/auth"
 	"github.com/certctl-io/certctl/internal/ratelimit"
+	"github.com/certctl-io/certctl/internal/repository"
 	"github.com/certctl-io/certctl/internal/repository/postgres"
 	"github.com/certctl-io/certctl/internal/scep/intune"
 	"github.com/certctl-io/certctl/internal/scheduler"
 	"github.com/certctl-io/certctl/internal/service"
+	authsvc "github.com/certctl-io/certctl/internal/service/auth"
 	"github.com/certctl-io/certctl/internal/trustanchor"
 )
 
@@ -58,9 +72,22 @@ func main() {
 	// unsupported auth shape. The error path uses fmt.Fprintf because
 	// the slog logger is constructed from cfg below this point; we want
 	// the failure to be visible regardless of log-level configuration.
+	//
+	// Auth Bundle 2 Phase 0: AuthTypeOIDC is in ValidAuthTypes() but the
+	// session middleware + OIDC handler chain ship in later phases. An
+	// operator who sets CERTCTL_AUTH_TYPE=oidc on a Bundle-2-incomplete
+	// deployment must NOT silently fall back to api-key (the silent
+	// auth-downgrade failure mode that drove G-1 in the first place).
+	// The OIDC case below refuses-to-start with an actionable message.
+	// Phase 6 of Bundle 2 (session middleware wiring) relaxes this case
+	// to fall through alongside the api-key + none cases.
 	switch config.AuthType(cfg.Auth.Type) {
 	case config.AuthTypeAPIKey, config.AuthTypeNone:
 		// ok — fall through
+	case config.AuthTypeOIDC:
+		fmt.Fprintf(os.Stderr,
+			"CERTCTL_AUTH_TYPE=oidc: the OIDC auth chain is not yet wired in this build (Auth Bundle 2 Phase 6 ships the session middleware that consumes this auth-type literal). Set CERTCTL_AUTH_TYPE=api-key or run an authenticating gateway with CERTCTL_AUTH_TYPE=none until Bundle 2 lands. See cowork/auth-bundle-2-prompt.md.\n")
+		os.Exit(1)
 	default:
 		fmt.Fprintf(os.Stderr,
 			"unsupported auth type at runtime: %q (valid: %v) — config validation should have caught this; refusing to start\n",
@@ -78,6 +105,19 @@ func main() {
 		"server_host", cfg.Server.Host,
 		"server_port", cfg.Server.Port)
 
+	// Bundle 2 (2026-05-12) — visible demo-mode banner at boot.
+	//
+	// When CERTCTL_DEMO_MODE_ACK=true the HIGH-12 startup guard already
+	// passed and the server is about to serve every request as the
+	// synthetic admin actor `actor-demo-anon`. Operators have lost
+	// production deploys to this posture more than once (last incident:
+	// 2026-04-19, a screenshot run that kept running for three days);
+	// the per-startup banner makes the posture unmissable in any log
+	// scraper, dashboard, or `journalctl --since boot` review.
+	if cfg.Auth.DemoModeAck {
+		logger.Warn("⚠ DEMO MODE ACTIVE — CERTCTL_DEMO_MODE_ACK=true is set; every request is served as the synthetic admin actor `actor-demo-anon` (no authentication enforced). This deployment MUST NOT hold production keys, certificates, or audit history. To promote to production: (1) unset CERTCTL_DEMO_MODE_ACK; (2) set CERTCTL_AUTH_TYPE=api-key or oidc; (3) set CERTCTL_AUTH_SECRET to a fresh `openssl rand -base64 32`; (4) set CERTCTL_KEYGEN_MODE=agent; (5) rotate CERTCTL_CONFIG_ENCRYPTION_KEY to a fresh `openssl rand -base64 32` (≥ 32 bytes, not the change-me placeholder); (6) restart the server. See docs/operator/security.md for the full posture.")
+	}
+
 	// Bundle-5 / Audit H-007: deprecation WARN when the agent bootstrap
 	// token is unset. Pre-Bundle-5 there was no token at all; the v2.0.x
 	// default keeps the warn-mode pass-through so existing demo deploys
@@ -91,8 +131,14 @@ func main() {
 		logger.Info("agent bootstrap token configured (length redacted; constant-time compare on POST /api/v1/agents)")
 	}
 
-	// Initialize database connection pool
+	// Initialize database connection pool.
-	db, err := postgres.NewDB(cfg.Database.URL)
+	//
+	// Bundle 3 closure (D12): pre-Bundle-3 the operator-facing
+	// CERTCTL_DATABASE_MAX_CONNS was a lying-field — config loaded the
+	// value and Validate() checked the floor, but the pool was hard-
+	// coded to SetMaxOpenConns(25). Post-Bundle-3 NewDBWithMaxConns
+	// threads the operator setting through to the connection pool.
+	db, err := postgres.NewDBWithMaxConns(cfg.Database.URL, cfg.Database.MaxConnections)
 	if err != nil {
 		logger.Error("failed to connect to database", "error", err)
 		os.Exit(1)
@@ -251,6 +297,324 @@ func main() {
 
 	// Initialize services (following the dependency graph)
 	auditService := service.NewAuditService(auditRepo)
+
+	// Audit 2026-05-11 A-8 closure: detect residual actor-demo-anon
+	// grants under non-`none` auth types. Defaults to WARN-only; flip
+	// CERTCTL_DEMO_MODE_RESIDUAL_STRICT=true to fail-closed. Closes
+	// the deferred Phase 2 leg of the 2026-05-10 HIGH-12 closure.
+	{
+		preflightCtx, preflightCancel := context.WithTimeout(context.Background(), 5*time.Second)
+		if err := preflightDemoModeResidual(preflightCtx, cfg, db, auditService, logger); err != nil {
+			preflightCancel()
+			logger.Error("startup refused: actor-demo-anon residual grants present + CERTCTL_DEMO_MODE_RESIDUAL_STRICT=true",
+				"error", err)
+			os.Exit(1)
+		}
+		preflightCancel()
+	}
+
+	// RBAC primitive (Bundle 1 Phase 4). Wires the postgres auth repos
+	// + service-layer Authorizer that the AuthHandler / RequirePermission
+	// middleware uses. Migration 000029_rbac.up.sql provides the schema
+	// and seeds the seven default roles + canonical permission catalogue
+	// + actor-demo-anon synthetic admin (CERTCTL_AUTH_TYPE=none demo path).
+	authRoleRepo := postgres.NewRoleRepository(db)
+	authPermRepo := postgres.NewPermissionRepository(db)
+	authActorRoleRepo := postgres.NewActorRoleRepository(db)
+	authAPIKeyRepo := postgres.NewAPIKeyRepository(db)
+	authAuthorizer := authsvc.NewAuthorizer(authActorRoleRepo)
+	// authCheckerAdapter bridges authsvc.Authorizer (typed-string args)
+	// to the auth.PermissionChecker interface (plain-string args) so
+	// internal/auth doesn't have to import internal/service/auth.
+	authCheckerAdapter := authPermissionCheckerAdapter{a: authAuthorizer}
+
+	// Bundle 1 Phase 6 — parse env-var named API keys + assemble the
+	// runtime keystore + wire the bootstrap service. The keystore +
+	// bootstrap handler must exist before the HandlerRegistry is
+	// constructed below; the auth middleware that reads from the same
+	// keystore is wired further down (next to the rest of the
+	// middleware stack) but holds a reference to the same keystore so
+	// runtime additions from bootstrap propagate without restart.
+	//
+	// boot-path operations use context.Background() because the long-
+	// lived request context isn't constructed until later in main();
+	// this matches the convention used by other one-shot setup calls
+	// in this section (issuerService.SeedFromEnvVars, etc.).
+	bootCtx := context.Background()
+	namedKeys := assembleNamedAPIKeys(cfg, logger)
+	backfillNamedKeyActorRoles(bootCtx, authActorRoleRepo, namedKeys, logger)
+	authKeyStore := auth.NewMutableKeyStore(namedKeys)
+	if persistedKeys, err := authAPIKeyRepo.List(bootCtx, authdomainAlias.DefaultTenantID); err == nil {
+		for _, pk := range persistedKeys {
+			authKeyStore.AddHashed(pk.Name, pk.KeyHash, pk.Admin)
+		}
+		if len(persistedKeys) > 0 {
+			logger.Info("loaded persisted api_keys into runtime keystore",
+				"count", len(persistedKeys))
+		}
+	} else {
+		logger.Warn("api_keys boot loader failed; bootstrap-minted keys will not authenticate until next restart that succeeds",
+			"err", err)
+	}
+	bootstrapStrategy := bootstrap.NewEnvTokenStrategy(
+		cfg.Auth.BootstrapToken,
+		func(ctx context.Context) (bool, error) {
+			return authActorRoleRepo.AdminExists(ctx, authdomainAlias.DefaultTenantID)
+		},
+	)
+	bootstrapService := bootstrap.NewService(
+		bootstrapStrategy,
+		authAPIKeyRepo,
+		authActorRoleRepo,
+		auditService,
+		authKeyStore,
+		auth.HashAPIKey,
+	)
+	if cfg.Auth.BootstrapToken != "" {
+		// Honour the prompt's "warn at startup if token set + admin
+		// exists" requirement. The strategy re-probes on every Validate
+		// so this boot-time warning is purely informational.
+		if exists, probeErr := authActorRoleRepo.AdminExists(bootCtx, authdomainAlias.DefaultTenantID); probeErr == nil && exists {
+			logger.Warn("CERTCTL_BOOTSTRAP_TOKEN set but admin actors already exist; bootstrap endpoint will return 410 Gone — unset the env var to silence this warning")
+		} else if probeErr != nil {
+			logger.Warn("CERTCTL_BOOTSTRAP_TOKEN admin-existence probe failed at startup; behaviour will be determined by the live probe at request time", "err", probeErr)
+		} else {
+			logger.Info("bootstrap endpoint enabled — POST /api/v1/auth/bootstrap to mint the first admin key (one-shot)")
+		}
+	}
+	bootstrapHandler := handler.NewBootstrapHandler(bootstrapService)
+
+	// =========================================================================
+	// Auth Bundle 2 Phase 4 — session service.
+	//
+	// Wired AFTER migrations + RBAC backfill, BEFORE the HTTP listener
+	// binds (per the prompt's "fail-fatal on bootstrap key mint failure"
+	// requirement). EnsureInitialSigningKey is idempotent: if a non-
+	// retired signing key already exists for the tenant the call is a
+	// no-op; otherwise it mints a fresh 32-byte HMAC key, persists it,
+	// and emits an auth.session_signing_key_bootstrap audit row with
+	// event_category=auth.
+	//
+	// Failure here is fatal — the server refuses to boot rather than
+	// serve session-less.
+	//
+	// The session service is wired into the scheduler below (sessionGCLoop)
+	// so the GC sweep runs every CERTCTL_SESSION_GC_INTERVAL tick. The
+	// HTTP middleware that consumes ValidateInput / ValidateCSRF lands
+	// in Phase 5; pre-Phase-5 deployments boot the service so the GC
+	// sweep can keep the sessions + signing-keys tables tidy.
+	sessionRepo := postgres.NewSessionRepository(db)
+	sessionKeyRepo := postgres.NewSessionSigningKeyRepository(db)
+	// Audit 2026-05-10 LOW-5 closure — install the trusted-proxy CIDR
+	// allowlist from CERTCTL_TRUSTED_PROXIES. Empty disables XFF trust.
+	session.SetTrustedProxies(cfg.Auth.TrustedProxies)
+	sessionService := session.NewService(
+		sessionRepo,
+		sessionKeyRepo,
+		auditService,
+		authdomainAlias.DefaultTenantID,
+		session.Config{
+			IdleTimeout:         cfg.Auth.Session.IdleTimeout,
+			AbsoluteTimeout:     cfg.Auth.Session.AbsoluteTimeout,
+			SigningKeyRetention: cfg.Auth.Session.SigningKeyRetention,
+			BindIP:              cfg.Auth.Session.BindIP,
+			BindUserAgent:       cfg.Auth.Session.BindUserAgent,
+		},
+		cfg.Encryption.ConfigEncryptionKey,
+	)
+	if err := sessionService.EnsureInitialSigningKey(bootCtx); err != nil {
+		logger.Error("FATAL: session signing key bootstrap failed; refusing to boot", "err", err)
+		os.Exit(1)
+	}
+
+	// =========================================================================
+	// Auth Bundle 2 Phase 5 — OIDC service + pre-login store + Phase 5 handler.
+	//
+	// Wired AFTER sessionService (Phase 4) so the OIDC PreLoginAdapter
+	// can sign pre-login cookies under the active SessionSigningKey.
+	// =========================================================================
+	oidcProviderRepo := postgres.NewOIDCProviderRepository(db)
+	oidcMappingRepo := postgres.NewGroupRoleMappingRepository(db)
+	oidcUserRepo := postgres.NewUserRepository(db)
+	// Audit 2026-05-10 HIGH-5: thread CERTCTL_CONFIG_ENCRYPTION_KEY into the
+	// pre-login repo so state/nonce/PKCE-verifier are encrypted at rest. Same
+	// key already protects OIDC client secrets and session signing keys.
+	oidcPreLoginRepo := postgres.NewPreLoginRepository(db, cfg.Encryption.ConfigEncryptionKey)
+	preLoginAdapter := oidcsvc.NewPreLoginAdapter(
+		oidcPreLoginRepo,
+		sessionKeyRepo, // Phase 4 SessionSigningKeyRepository
+		authdomainAlias.DefaultTenantID,
+		cfg.Encryption.ConfigEncryptionKey,
+	)
+	// SessionMinter port for the OIDC service. The OIDC HandleCallback
+	// uses this to mint the post-login session after successful token
+	// validation + group→role mapping.
+	oidcSessionMinter := &sessionMinterAdapter{svc: sessionService}
+	oidcService := oidcsvc.NewService(
+		oidcProviderRepo,
+		oidcMappingRepo,
+		oidcUserRepo,
+		oidcSessionMinter,
+		preLoginAdapter,
+		cfg.Encryption.ConfigEncryptionKey,
+	)
+	// Audit 2026-05-10 MED-16 — apply per-leg pre-login UA / IP
+	// binding enforcement toggles from config.
+	oidcService.SetPreLoginBindingRequirements(
+		cfg.Auth.OIDCPreLoginRequireUA,
+		cfg.Auth.OIDCPreLoginRequireIP,
+	)
+	// SameSite resolution from CERTCTL_SESSION_SAMESITE (default Lax;
+	// "Strict" for high-security environments at the cost of breaking
+	// inbound deep-links from external apps).
+	sameSiteMode := http.SameSiteLaxMode
+	if strings.EqualFold(cfg.Auth.Session.SameSite, "Strict") {
+		sameSiteMode = http.SameSiteStrictMode
+	}
+	// Audit 2026-05-10 HIGH-3 — BCL iat-skew window + jti consumed-set.
+	bclMaxAge := time.Duration(cfg.Auth.OIDCBCLMaxAgeSeconds) * time.Second
+	if bclMaxAge <= 0 {
+		bclMaxAge = handler.DefaultBCLVerifierMaxAge
+	}
+	bclReplayRepo := postgres.NewBCLReplayRepository(db)
+	authSessionOIDCHandler := handler.NewAuthSessionOIDCHandler(
+		oidcService,
+		sessionService,
+		handler.NewDefaultBCLVerifier(oidcProviderRepo, authdomainAlias.DefaultTenantID, nil).WithMaxAge(bclMaxAge),
+		oidcProviderRepo,
+		oidcMappingRepo,
+		sessionRepo,
+		oidcUserRepo, // CRIT-2: BCL sub→actor_id lookup via users.GetByOIDCSubject
+		auditService,
+		cfg.Encryption.ConfigEncryptionKey,
+		authdomainAlias.DefaultTenantID,
+		"/", // post-login redirect target; GUI dashboard
+		handler.SessionCookieAttrs{
+			SameSite: sameSiteMode,
+			Secure:   true,
+		},
+	).WithBCLReplayConsumer(bclReplayRepo, bclMaxAge). // HIGH-3 jti consumed-set.
+								WithPermissionChecker(authCheckerAdapter) // MED-2 auth.session.list.all gate.
+
+	// =========================================================================
+	// Auth Bundle 2 Phase 7 — OIDC first-admin bootstrap hook.
+	//
+	// Wired AFTER oidcService is constructed. The hook closure consults
+	// the configured CERTCTL_BOOTSTRAP_ADMIN_GROUPS + the AdminExists
+	// probe; on first match it grants r-admin via the ActorRoleRepository
+	// + emits a bootstrap.oidc_first_admin audit row. Subsequent
+	// admin-already-exists logins return grantAdmin=false silently.
+	// Disabled (no-op) when CERTCTL_BOOTSTRAP_ADMIN_GROUPS is empty.
+	if len(cfg.Auth.BootstrapAdminGroups) > 0 {
+		bootstrapGroups := make(map[string]struct{}, len(cfg.Auth.BootstrapAdminGroups))
+		for _, g := range cfg.Auth.BootstrapAdminGroups {
+			bootstrapGroups[strings.TrimSpace(g)] = struct{}{}
+		}
+		bootstrapProviderID := cfg.Auth.BootstrapOIDCProviderID
+		oidcService.SetAdminBootstrapHook(func(ctx context.Context, providerID string, groups []string, userID string) (bool, error) {
+			// Provider-specificity: when configured, only the named
+			// provider is eligible for bootstrap.
+			if bootstrapProviderID != "" && providerID != bootstrapProviderID {
+				return false, nil
+			}
+			// Admin-already-exists: bootstrap mode is disabled once
+			// any actor in the tenant holds r-admin.
+			adminExists, probeErr := authActorRoleRepo.AdminExists(ctx, authdomainAlias.DefaultTenantID)
+			if probeErr != nil {
+				return false, fmt.Errorf("admin existence probe: %w", probeErr)
+			}
+			if adminExists {
+				return false, nil
+			}
+			// Group intersection check.
+			matched := false
+			for _, g := range groups {
+				if _, ok := bootstrapGroups[g]; ok {
+					matched = true
+					break
+				}
+			}
+			if !matched {
+				return false, nil
+			}
+			// Match. Grant r-admin via the actor-role repo.
+			grant := &authdomainAlias.ActorRole{
+				ActorID:   userID,
+				ActorType: authdomainAlias.ActorTypeValue("User"),
+				RoleID:    authdomainAlias.RoleIDAdmin,
+				TenantID:  authdomainAlias.DefaultTenantID,
+				GrantedBy: "oidc-bootstrap",
+			}
+			if gerr := authActorRoleRepo.Grant(ctx, grant); gerr != nil {
+				return false, fmt.Errorf("grant r-admin: %w", gerr)
+			}
+			// Emit audit row with event_category=auth.
+			_ = auditService.RecordEventWithCategory(ctx, userID, domain.ActorTypeUser,
+				"bootstrap.oidc_first_admin", domain.EventCategoryAuth,
+				"users", userID,
+				map[string]interface{}{
+					"user_id":     userID,
+					"provider_id": providerID,
+					"trigger":     "oidc_group_match",
+				})
+			logger.Info("OIDC first-admin bootstrap fired — user granted r-admin",
+				"user_id", userID, "provider_id", providerID)
+			return true, nil
+		})
+		logger.Info("OIDC first-admin bootstrap enabled",
+			"groups", cfg.Auth.BootstrapAdminGroups,
+			"provider_id_filter", bootstrapProviderID)
+	}
+
+	// =========================================================================
+	// Auth Bundle 2 Phase 7.5 — break-glass admin service + handler.
+	// =========================================================================
+	breakglassRepo := postgres.NewBreakglassCredentialRepository(db)
+	breakglassService := breakglass.NewService(
+		breakglassRepo,
+		auditService,
+		breakglassSessionMinterAdapter{svc: sessionService},
+		breakglass.Config{
+			Enabled:              cfg.Auth.Breakglass.Enabled,
+			LockoutThreshold:     cfg.Auth.Breakglass.LockoutThreshold,
+			LockoutDuration:      cfg.Auth.Breakglass.LockoutDuration,
+			LockoutResetInterval: cfg.Auth.Breakglass.LockoutResetInterval,
+		},
+		authdomainAlias.DefaultTenantID,
+	)
+	breakglassHandler := handler.NewAuthBreakglassHandler(breakglassService, handler.SessionCookieAttrs{
+		SameSite: sameSiteMode,
+		Secure:   true,
+	})
+	// Bundle 5 closure (audit S1): wire the per-source-IP rate limiter
+	// for POST /auth/breakglass/login. 5 attempts / minute / IP, 50 000
+	// key cap. Pre-Bundle-5 the handler docstring claimed this rate
+	// limit but no limiter was installed; the route bypasses the global
+	// RPS middleware because it's mounted via r.mux.Handle in the
+	// AuthExemptRouterRoutes path. The service-layer Argon2id lockout
+	// state machine remains the second line of defense.
+	breakglassHandler.SetLoginRateLimiter(
+		ratelimit.NewSlidingWindowLimiter(5, time.Minute, 50_000),
+	)
+	if cfg.Auth.Breakglass.Enabled {
+		logger.Warn("CERTCTL_BREAKGLASS_ENABLED=true — break-glass admin path is ACTIVE; this bypasses SSO. Disable in steady-state.",
+			"lockout_threshold", cfg.Auth.Breakglass.LockoutThreshold,
+			"lockout_duration", cfg.Auth.Breakglass.LockoutDuration.String())
+	}
+
+	// Bundle 5 closure (audit RT-L2): operator-visible startup warning
+	// when CERTCTL_ACME_INSECURE=true disables ACME directory TLS
+	// verification. Pre-Bundle-5 this knob silently disabled TLS
+	// verification for every ACME issuance call without surfacing any
+	// signal at boot; the only mention lived in a values.yaml comment.
+	// Pebble / step-ca / dev ACME proxies use self-signed certs so the
+	// knob has legitimate dev uses, but a production deploy that flips
+	// it (typically copy-pasting from a Pebble integration runbook)
+	// gets MITM exposure on every CA round-trip. Loud at boot now.
+	if cfg.ACME.Insecure {
+		logger.Warn("CERTCTL_ACME_INSECURE=true — ACME directory TLS verification is DISABLED. Every ACME round-trip skips certificate chain validation; production deploys MUST unset this. Acceptable only for dev / Pebble / step-ca with operator-supplied self-signed roots.")
+	}
+
 	policyService := service.NewPolicyService(policyRepo, auditService)
 	policyService.SetCertRepo(certificateRepo) // D-008: CertificateLifetime arm needs CertificateVersion.NotBefore/NotAfter
 	// G-1: RenewalPolicyService — distinct from PolicyService (compliance rules).
@@ -483,6 +847,36 @@ func main() {
 	defer issuerRegistry.StopLifecycles()
 	targetService := service.NewTargetService(targetRepo, auditService, agentRepo, encryptionKey, logger)
 	profileService := service.NewProfileService(profileRepo, auditService)
+	// Bundle 1 Phase 9 — approval-bypass closure. Wire the profile
+	// service's gate to the existing ApprovalService so edits to a
+	// RequiresApproval=true profile route through the four-eyes
+	// workflow. The profile-edit-apply callback registered on the
+	// ApprovalService closes the loop: when an approver decides,
+	// the callback deserializes req.Payload and persists the diff.
+	profileService.SetApprovalService(approvalService)
+	approvalService.SetProfileEditApply(func(ctx context.Context, req *domain.ApprovalRequest) error {
+		var pendingProfile domain.CertificateProfile
+		if err := json.Unmarshal(req.Payload, &pendingProfile); err != nil {
+			return fmt.Errorf("decode profile-edit payload: %w", err)
+		}
+		pendingProfile.ID = req.ProfileID
+		if err := profileRepo.Update(ctx, &pendingProfile); err != nil {
+			return fmt.Errorf("apply profile-edit diff: %w", err)
+		}
+		// Audit row category=auth so the auditor surface keeps the
+		// approval-decision history grouped with the request side.
+		if auditService != nil {
+			_ = auditService.RecordEventWithCategory(ctx, "approval-system",
+				domain.ActorTypeSystem, "profile.edit_applied",
+				domain.EventCategoryAuth, "certificate_profile",
+				req.ProfileID,
+				map[string]interface{}{
+					"approval_id":  req.ID,
+					"requested_by": req.RequestedBy,
+				})
+		}
+		return nil
+	})
 	teamService := service.NewTeamService(teamRepo, auditService)
 	ownerService := service.NewOwnerService(ownerRepo, auditService)
 	agentGroupRepo := postgres.NewAgentGroupRepository(db)
@@ -661,6 +1055,18 @@ func main() {
 	// Bundle-5 / H-006: pass the *sql.DB pool so /ready can probe DB
 	// connectivity via PingContext. /health stays shallow (liveness signal).
 	healthHandler := handler.NewHealthHandler(cfg.Auth.Type, db)
+	// Bundle 1 Phase 3 closure (M1): wire the AuthCheckResolver so
+	// /v1/auth/check returns the caller's standing roles + effective
+	// permissions in the same response. The shim is tiny — just a type-
+	// erasure wrap around the repo so the handler layer doesn't have to
+	// import internal/domain/auth or internal/repository/postgres.
+	healthHandler.Resolver = authCheckResolverAdapter{repo: authActorRoleRepo}
+	// Bundle 2 Phase 6 / Category E — wire the OIDC providers resolver
+	// so GET /api/v1/auth/info returns the configured provider list
+	// (id + display_name + login_url) for the GUI's Login page button
+	// rendering. The shim adapts the postgres OIDCProviderRepository
+	// to the handler's narrow OIDCProvidersListResolver projection.
+	healthHandler.OIDCProvidersResolver = oidcProvidersListAdapter{repo: oidcProviderRepo}
 	// U-3 ride-along (cat-u-no_version_endpoint, P2): the version handler
 	// answers GET /api/v1/version with build identity (ldflags Version,
 	// VCS commit/dirty/timestamp, Go runtime version). Wired through the
@@ -811,6 +1217,19 @@ func main() {
 	sched.SetJobTimeoutInterval(cfg.Scheduler.JobTimeoutInterval)
 	sched.SetAwaitingCSRTimeout(cfg.Scheduler.AwaitingCSRTimeout)
 	sched.SetAwaitingApprovalTimeout(cfg.Scheduler.AwaitingApprovalTimeout)
+
+	// Auth Bundle 2 Phase 4 — wire the session-GC sweep. The service
+	// itself was constructed (with the EnsureInitialSigningKey fail-
+	// fatal call) above the policy/cert-service block; here we just
+	// register it with the scheduler so the loop fires every
+	// CERTCTL_SESSION_GC_INTERVAL.
+	sched.SetSessionGarbageCollector(sessionService)
+	sched.SetBCLReplayGarbageCollector(bclReplayRepo) // Audit 2026-05-10 HIGH-3.
+	sched.SetSessionGCInterval(cfg.Auth.Session.GCInterval)
+	logger.Info("session GC sweep enabled",
+		"interval", cfg.Auth.Session.GCInterval.String(),
+		"absolute_timeout", cfg.Auth.Session.AbsoluteTimeout.String(),
+		"signing_key_retention", cfg.Auth.Session.SigningKeyRetention.String())
 	logger.Info("job timeout reaper enabled",
 		"interval", cfg.Scheduler.JobTimeoutInterval.String(),
 		"csr_timeout", cfg.Scheduler.AwaitingCSRTimeout.String(),
@@ -961,6 +1380,90 @@ func main() {
 		// Rank 8 of the 2026-05-03 deep-research deliverable. See
 		// docs/intermediate-ca-hierarchy.md.
 		IntermediateCAs: intermediateCAHandler,
+		// AuthSessionOIDC — Auth Bundle 2 Phase 5 OIDC + session HTTP
+		// surface. 13 endpoints across login flow + session management
+		// + OIDC provider CRUD + group-mapping CRUD.
+		AuthSessionOIDC: authSessionOIDCHandler,
+
+		// AuthBreakglass — Auth Bundle 2 Phase 7.5 break-glass admin
+		// HTTP surface. 4 endpoints (1 public login + 3 admin CRUD).
+		// All endpoints return 404 when CERTCTL_BREAKGLASS_ENABLED=false.
+		AuthBreakglass: breakglassHandler,
+
+		// Audit 2026-05-10 MED-11 — federated-user admin surface.
+		AuthUsers: handler.NewAuthUsersHandler(
+			oidcUserRepo,
+			sessionService, // satisfies UserSessionsRevoker via RevokeAllForActor
+			auditService,
+			authdomainAlias.DefaultTenantID,
+		),
+
+		// Audit 2026-05-10 MED-12 — runtime config read endpoint.
+		AuthRuntimeConfig: handler.NewAuthRuntimeConfigHandler(
+			func() map[string]string {
+				// Lazy build — re-read cfg.Auth.* values on every call so
+				// post-startup re-evaluation reflects any (future) mutation.
+				return map[string]string{
+					"CERTCTL_AUTH_TYPE":                    string(cfg.Auth.Type),
+					"CERTCTL_SESSION_SAMESITE":             cfg.Auth.Session.SameSite,
+					"CERTCTL_OIDC_BCL_MAX_AGE_SECONDS":     strconv.Itoa(cfg.Auth.OIDCBCLMaxAgeSeconds),
+					"CERTCTL_OIDC_PRELOGIN_REQUIRE_UA":     strconv.FormatBool(cfg.Auth.OIDCPreLoginRequireUA),
+					"CERTCTL_OIDC_PRELOGIN_REQUIRE_IP":     strconv.FormatBool(cfg.Auth.OIDCPreLoginRequireIP),
+					"CERTCTL_BREAKGLASS_ENABLED":           strconv.FormatBool(cfg.Auth.Breakglass.Enabled),
+					"CERTCTL_BREAKGLASS_LOCKOUT_THRESHOLD": strconv.Itoa(cfg.Auth.Breakglass.LockoutThreshold),
+					"CERTCTL_DEMO_MODE_ACK":                strconv.FormatBool(cfg.Auth.DemoModeAck),
+					"CERTCTL_TRUSTED_PROXIES_COUNT":        strconv.Itoa(len(cfg.Auth.TrustedProxies)),
+					"CERTCTL_BOOTSTRAP_TOKEN_SET":          strconv.FormatBool(cfg.Auth.BootstrapToken != ""),
+					"CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID":   cfg.Auth.BootstrapOIDCProviderID,
+					"CERTCTL_BOOTSTRAP_ADMIN_GROUPS_COUNT": strconv.Itoa(len(cfg.Auth.BootstrapAdminGroups)),
+				}
+			},
+			auditService,
+		),
+
+		// Audit 2026-05-10 MED-7 — per-provider JWKS health surface.
+		AuthOIDCJWKSStatus: handler.NewAuthOIDCJWKSStatusHandler(oidcService, auditService),
+		// Auth — RBAC primitive (Bundle 1 Phase 4). Wires the postgres
+		// auth repos + service-layer Authorizer / RoleService /
+		// ActorRoleService / PermissionService into the HTTP surface
+		// under /api/v1/auth/*. The service layer enforces every
+		// permission gate (auth.role.* + auth.role.assign privilege-
+		// escalation guard); the Phase 3 RequirePermission middleware
+		// is currently used by these RBAC routes via the in-handler
+		// callerFromRequest path. Phase 3.5 router-wrapping conversion
+		// of the legacy admin handlers (bulk_revocation, admin_*,
+		// intermediate_ca) is the remaining sweep.
+		Auth: handler.NewAuthHandler(
+			authsvc.NewRoleService(authRoleRepo, authPermRepo, authAuthorizer, auditService),
+			authsvc.NewPermissionService(authPermRepo),
+			authsvc.NewActorRoleService(authActorRoleRepo, authRoleRepo, authAuthorizer, auditService),
+			authCheckerAdapter,
+		).WithCSRFRotator(sessionService), // Audit 2026-05-10 HIGH-2 — CSRF rotation on role mutation.
+		// Bundle 1 Phase 6 — bootstrap day-0 admin endpoint. The
+		// service is wired above; handler is auth-exempt at the
+		// router (gated by the bootstrap.Strategy itself).
+		Bootstrap: bootstrapHandler,
+		// Audit 2026-05-11 A-8 closure — demo-mode residual cleanup.
+		// The cleanup closure captures the live *sql.DB pool so the
+		// handler doesn't pull repository.* / database/sql into the
+		// internal/api/handler import set. authType is a closure over
+		// cfg so the live config value is always read at request time.
+		DemoResidual: handler.NewDemoResidualHandler(
+			func(ctx context.Context) (int64, error) { return deleteDemoAnonResidue(ctx, db) },
+			func() string { return cfg.Auth.Type },
+			auditService,
+		),
+		// Checker is the load-bearing auth.PermissionChecker that
+		// auth.RequirePermission middleware uses to gate the legacy admin
+		// handlers (Bundle 1 Phase 3.5: bulk_revocation, admin_crl_cache,
+		// admin_scep_intune, admin_est, intermediate_ca). Wraps live in
+		// router.go via rbacGate(reg.Checker, perm, handler).
+		Checker: authCheckerAdapter,
+		// Audit 2026-05-10 CRIT-3 closure — operator-configured CORS
+		// applied to the credentialed auth-exempt routes (OIDC handshake,
+		// BCL, logout, bootstrap, breakglass-login). Health probes
+		// continue to use middleware.CORSWildcard.
+		CorsCfg: middleware.CORSConfig{AllowedOrigins: cfg.CORS.AllowedOrigins},
 	})
 	// Register EST (RFC 7030) handlers if enabled.
 	//
@@ -1477,49 +1980,31 @@ func main() {
 
 	// Build middleware stack.
 	//
-	// Authentication unification (M-002): every authenticated request now
+	// Bundle 1 Phase 6: namedKeys + authKeyStore + bootstrap service
-	// carries a named actor in the request context so audit events record
+	// are now constructed earlier (right after the auth repos) so the
-	// the real key identity instead of the hardcoded "api-key-user" string.
+	// HandlerRegistry can wire the bootstrap handler. The auth
-	// Named keys come from CERTCTL_API_KEYS_NAMED (preferred). For backward
+	// middleware below reads from the same authKeyStore reference, so
-	// compatibility CERTCTL_AUTH_SECRET is synthesized into legacy-key-N
+	// runtime additions from bootstrap propagate without restart.
-	// entries with Admin=false.
+	var bearerMiddleware func(http.Handler) http.Handler
-	var namedKeys []middleware.NamedAPIKey
+	switch config.AuthType(cfg.Auth.Type) {
-	if config.AuthType(cfg.Auth.Type) != config.AuthTypeNone {
+	case config.AuthTypeNone:
-		// Translate typed config.NamedAPIKey -> middleware.NamedAPIKey. The
+		bearerMiddleware = auth.NewDemoModeAuth()
-		// two structs are field-compatible but live in different packages to
+	default:
-		// preserve the config→middleware dependency direction.
+		bearerMiddleware = auth.NewAuthWithKeyStore(authKeyStore)
-		for _, nk := range cfg.Auth.NamedKeys {
-			namedKeys = append(namedKeys, middleware.NamedAPIKey{
-				Name:  nk.Name,
-				Key:   nk.Key,
-				Admin: nk.Admin,
-			})
-		}
-		// Back-compat: if no named keys but legacy Secret is configured,
-		// synthesize named entries so the audit trail still attributes the
-		// action (instead of falling back to "api-key-user" / "anonymous").
-		if len(namedKeys) == 0 && cfg.Auth.Secret != "" {
-			parts := strings.Split(cfg.Auth.Secret, ",")
-			idx := 0
-			for _, p := range parts {
-				p = strings.TrimSpace(p)
-				if p == "" {
-					continue
-				}
-				namedKeys = append(namedKeys, middleware.NamedAPIKey{
-					Name:  fmt.Sprintf("legacy-key-%d", idx),
-					Key:   p,
-					Admin: false,
-				})
-				idx++
-			}
-			if len(namedKeys) > 0 {
-				logger.Warn("CERTCTL_AUTH_SECRET is deprecated — set CERTCTL_API_KEYS_NAMED for named actor attribution and admin gating",
-					"synthesized_keys", len(namedKeys))
-			}
-		}
 	}
-	authMiddleware := middleware.NewAuthWithNamedKeys(namedKeys)
+	// Auth Bundle 2 Phase 6 — chained-auth middleware. Tries the
+	// `certctl_session` cookie first (sessionMW); on miss / invalid,
+	// falls back to the API-key Bearer middleware. If neither
+	// authenticates, 401. The session middleware is a pass-through
+	// when sessionService is nil (pre-Bundle-2 builds).
+	sessionMW := session.NewSessionMiddleware(sessionService)
+	authMiddleware := session.ChainAuthSessionThenBearer(sessionMW, bearerMiddleware)
+	// CSRF middleware — gates state-changing methods (POST/PUT/DELETE/
+	// PATCH) for session-authenticated requests. API-key actors are
+	// CSRF-exempt (not browser-driven). Pass-through when
+	// sessionService is nil.
+	csrfMiddleware := session.NewCSRFMiddleware(sessionService)
+	_ = bootstrapHandler // referenced by HandlerRegistry above
 	corsMiddleware := middleware.NewCORS(middleware.CORSConfig{
 		AllowedOrigins: cfg.CORS.AllowedOrigins,
 	})
@@ -1567,7 +2052,10 @@ func main() {
 		bodyLimitMiddleware,
 		securityHeadersMiddleware,
 		corsMiddleware,
+		// Phase 6 chain: Auth (session-then-Bearer fallback) → CSRF
+		// (state-changing only; API-key actors exempt) → Audit.
 		authMiddleware,
+		csrfMiddleware,
 		auditMiddleware.Middleware,
 	}
 
@@ -1589,7 +2077,10 @@ func main() {
 			bodyLimitMiddleware,
 			rateLimiter,
 			corsMiddleware,
+			// Phase 6 chain: Auth (session-then-Bearer fallback) → CSRF
+			// (state-changing only; API-key actors exempt) → Audit.
 			authMiddleware,
+			csrfMiddleware,
 			auditMiddleware.Middleware,
 		}
 		logger.Info("rate limiting enabled", "rps", cfg.RateLimit.RPS, "burst", cfg.RateLimit.BurstSize)
@@ -2231,3 +2722,171 @@ func buildFinalHandler(apiHandler, noAuthHandler http.Handler, webDir string, da
 		http.ServeFile(w, r, webDir+"/index.html")
 	})
 }
+
+// authPermissionCheckerAdapter bridges the typed-string Authorizer
+// signature (authsvc.Authorizer.CheckPermission takes
+// authdomain.ActorTypeValue + authdomain.ScopeType) to the plain-string
+// auth.PermissionChecker interface used by the auth.RequirePermission
+// middleware factory. Lives in cmd/server so internal/auth doesn't have
+// to import internal/service/auth + internal/domain/auth (would create
+// a cycle).
+type authPermissionCheckerAdapter struct {
+	a *authsvc.Authorizer
+}
+
+func (ad authPermissionCheckerAdapter) CheckPermission(
+	ctx context.Context,
+	actorID string,
+	actorType string,
+	tenantID string,
+	permission string,
+	scopeType string,
+	scopeID *string,
+) (bool, error) {
+	return ad.a.CheckPermission(
+		ctx,
+		actorID,
+		authdomainAlias.ActorTypeValue(actorType),
+		tenantID,
+		permission,
+		authdomainAlias.ScopeType(scopeType),
+		scopeID,
+	)
+}
+
+// authCheckResolverAdapter bridges the postgres ActorRoleRepository
+// (authdomain.ActorTypeValue) to handler.AuthCheckResolver
+// (domain.ActorType). Lives in cmd/server so the handler layer keeps its
+// existing import set; the GUI's /v1/auth/check probe round-trips
+// through this on every page load. Read-only — no caller / no audit row.
+//
+// Bundle 1 Phase 3 closure (M1): the equivalent surface area on
+// /v1/auth/me runs through the service layer's auth.role.list permission
+// gate, which the GUI may not yet hold during initial render. AuthCheck
+// has no permission gate (its only requirement is "the request
+// authenticated"), so the bypass is by design.
+type authCheckResolverAdapter struct {
+	repo *postgres.ActorRoleRepository
+}
+
+func (ad authCheckResolverAdapter) ListRoles(
+	ctx context.Context,
+	actorID string,
+	actorType domain.ActorType,
+	tenantID string,
+) ([]*authdomainAlias.ActorRole, error) {
+	return ad.repo.ListByActor(ctx, actorID, authdomainAlias.ActorTypeValue(actorType), tenantID)
+}
+
+func (ad authCheckResolverAdapter) EffectivePermissions(
+	ctx context.Context,
+	actorID string,
+	actorType domain.ActorType,
+	tenantID string,
+) ([]repository.EffectivePermission, error) {
+	return ad.repo.EffectivePermissions(ctx, actorID, authdomainAlias.ActorTypeValue(actorType), tenantID)
+}
+
+// =============================================================================
+// sessionMinterAdapter — bridge from *session.Service to oidcsvc.SessionMinter.
+//
+// The OIDC service's SessionMinter port (Phase 3) takes a *userdomain.User
+// + role IDs and returns (cookie, csrf, err). The session.Service's
+// Create method takes (actorID, actorType, ip, ua) -> *CreateResult.
+// This adapter unwraps the User into actorID/actorType + reshapes the
+// return tuple. Lives in cmd/server so the session package doesn't have
+// to know about user.User and the user package doesn't have to know
+// about session.CreateResult.
+// =============================================================================
+
+type sessionMinterAdapter struct {
+	svc *session.Service
+}
+
+func (a *sessionMinterAdapter) MintForUser(
+	ctx context.Context,
+	user *userdomain.User,
+	_ []string, // roleIDs unused at the session-mint layer; the rbac middleware looks them up at request time
+	ip, userAgent string,
+) (cookieValue, csrfToken string, err error) {
+	if user == nil {
+		return "", "", fmt.Errorf("session mint: user is nil")
+	}
+	res, err := a.svc.Create(ctx, user.ID, string(domain.ActorTypeUser), ip, userAgent)
+	if err != nil {
+		return "", "", err
+	}
+	return res.CookieValue, res.CSRFToken, nil
+}
+
+// silenceUnusedImports keeps the new oidcsvc + oidcdomain imports load-
+// bearing in case any file shuffles. Linker dead-code elimination handles
+// the runtime cost.
+var (
+	_ = oidcdomain.OIDCProvider{}
+)
+
+// =============================================================================
+// breakglassSessionMinterAdapter — bridge from *session.Service to
+// breakglass.SessionMinter.
+//
+// The break-glass service's SessionMinter port (Phase 7.5) returns
+// (cookie, csrf, err); the underlying *session.Service.Create returns
+// *CreateResult. This adapter unwraps the result. Lives in cmd/server
+// so the breakglass package doesn't have to know about session.Service.
+// =============================================================================
+
+type breakglassSessionMinterAdapter struct {
+	svc *session.Service
+}
+
+func (a breakglassSessionMinterAdapter) Create(ctx context.Context, actorID, actorType, ip, userAgent string) (string, string, error) {
+	res, err := a.svc.Create(ctx, actorID, actorType, ip, userAgent)
+	if err != nil {
+		return "", "", err
+	}
+	return res.CookieValue, res.CSRFToken, nil
+}
+
+// RevokeAllForActor — Audit 2026-05-10 HIGH-1 wire. After a break-glass
+// password rotation or credential removal, every active session for the
+// target actor must be revoked so a phished-then-rotated credential
+// doesn't leave the attacker's session live.
+func (a breakglassSessionMinterAdapter) RevokeAllForActor(ctx context.Context, actorID, actorType string) error {
+	return a.svc.RevokeAllForActor(ctx, actorID, actorType)
+}
+
+// oidcProvidersListAdapter bridges the postgres OIDCProviderRepository
+// to handler.OIDCProvidersListResolver. The handler returns
+// []*OIDCProviderInfo (id + display_name + login_url) for the public-
+// safe GUI Login-page payload; the repo returns the full OIDCProvider
+// row. The adapter projects + maps the login_url shape that
+// /auth/oidc/login?provider=<id> expects. Auth Bundle 2 Phase 6 /
+// Category E.
+type oidcProvidersListAdapter struct {
+	repo repository.OIDCProviderRepository
+}
+
+func (a oidcProvidersListAdapter) List(ctx context.Context, tenantID string) ([]*handler.OIDCProviderInfo, error) {
+	provs, err := a.repo.List(ctx, tenantID)
+	if err != nil {
+		return nil, err
+	}
+	out := make([]*handler.OIDCProviderInfo, 0, len(provs))
+	for _, p := range provs {
+		// Audit 2026-05-10 MED-9 closure — filter disabled providers
+		// at the adapter so the LoginPage's "Sign in with X" buttons
+		// don't render for offline IdPs. The HandleAuthRequest
+		// service-layer ErrProviderDisabled check is the
+		// defense-in-depth guard for direct API / MCP / CLI callers.
+		if !p.Enabled {
+			continue
+		}
+		out = append(out, &handler.OIDCProviderInfo{
+			ID:          p.ID,
+			DisplayName: p.Name,
+			LoginURL:    "/auth/oidc/login?provider=" + p.ID,
+		})
+	}
+	return out, nil
+}

cmd/server/main_test.go

@@ -12,6 +12,7 @@ import (
 
 	"github.com/certctl-io/certctl/internal/api/middleware"
 	"github.com/certctl-io/certctl/internal/api/router"
+	"github.com/certctl-io/certctl/internal/auth"
 	"github.com/certctl-io/certctl/internal/config"
 	"github.com/certctl-io/certctl/internal/service"
 )
@@ -44,7 +45,7 @@ func TestMain_HealthEndpointBypassesAuth(t *testing.T) {
 	})
 
 	// Build the handler chain the same way main.go does
-	authMiddleware := middleware.NewAuthWithNamedKeys([]middleware.NamedAPIKey{
+	authMiddleware := auth.NewAuthWithNamedKeys([]auth.NamedAPIKey{
 		{Name: "test", Key: "test-secret-key"},
 	})
 
@@ -159,7 +160,7 @@ func TestMain_AuthMiddlewareRejectsUnauthorized(t *testing.T) {
 	})
 
 	// Wrap with auth middleware
-	authMiddleware := middleware.NewAuthWithNamedKeys([]middleware.NamedAPIKey{
+	authMiddleware := auth.NewAuthWithNamedKeys([]auth.NamedAPIKey{
 		{Name: "test", Key: "test-secret-key"},
 	})
 
@@ -187,7 +188,7 @@ func TestMain_AuthMiddlewareAllowsWithValidKey(t *testing.T) {
 	})
 
 	// Wrap with auth middleware
-	authMiddleware := middleware.NewAuthWithNamedKeys([]middleware.NamedAPIKey{
+	authMiddleware := auth.NewAuthWithNamedKeys([]auth.NamedAPIKey{
 		{Name: "test", Key: testKey},
 	})
 
@@ -460,7 +461,7 @@ func TestMain_AuthNoneMode(t *testing.T) {
 
 	// Wrap with auth middleware in "none" mode
 	// auth=none equivalent: empty named-keys list is a no-op pass-through.
-	authMiddleware := middleware.NewAuthWithNamedKeys(nil)
+	authMiddleware := auth.NewAuthWithNamedKeys(nil)
 
 	chainedHandler := middleware.Chain(protectedHandler, authMiddleware)
 

cmd/server/preflight_demo_residual.go

@@ -0,0 +1,204 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+//
+// Audit 2026-05-11 A-8 — demo-mode residual-grants detector. Closes the
+// deferred Phase 2 leg of HIGH-12 (cowork/auth-bundles-fixes-2026-05-10/
+// 11-high-12-demo-mode-guard.md). The HIGH-12 closure (`b81588e`) added
+// the fail-closed bind-address guard at config.Validate; the deferred
+// leg here adds a startup-time WARN (or strict refuse-startup) when
+// `actor-demo-anon` has live role grants under a non-`none` auth type.
+//
+// Why this matters: migration 000029 unconditionally seeds the
+// `ar-demo-anon-admin` row granting r-admin to actor-demo-anon. The
+// row is dormant under auth_type=api-key|oidc (the middleware chain
+// never injects the synthetic actor as the request principal), but
+// it represents a security debt: any future regression in the
+// middleware chain (a misrouted CORS preflight, a fallback in a new
+// auth-exempt route) that resolves to actor-demo-anon would re-elevate
+// to admin. The canonical acquisition-readiness narrative — "we have
+// an RBAC primitive with no synthetic-admin fallback" — requires this
+// row to be either gone or explicitly acknowledged.
+
+package main
+
+import (
+	"context"
+	"database/sql"
+	"errors"
+	"fmt"
+	"log/slog"
+	"strings"
+	"time"
+
+	"github.com/certctl-io/certctl/internal/config"
+	"github.com/certctl-io/certctl/internal/domain"
+	authdomain "github.com/certctl-io/certctl/internal/domain/auth"
+	"github.com/certctl-io/certctl/internal/service"
+)
+
+// preflightDemoModeResidual runs after the DB connection is open and
+// the audit service is constructed, before the HTTPS listener starts.
+//
+// Behaviour:
+//   - cfg.Auth.Type == "none" (demo mode): no-op. The residual IS the
+//     runtime state at that auth type.
+//   - cfg.Auth.Type != "none" + no residue: returns nil silently.
+//   - cfg.Auth.Type != "none" + residue + strict=false: emits a WARN
+//     log AND an `auth.demo_residual_grants_detected` audit row
+//     listing the grant IDs, then returns nil.
+//   - cfg.Auth.Type != "none" + residue + strict=true: emits the same
+//     WARN + audit, then returns a non-nil error so the caller can
+//     refuse startup.
+//
+// The audit row's actor is `system` / ActorTypeSystem; category is
+// EventCategoryAuth so audit consumers filtering on auth events see it.
+func preflightDemoModeResidual(
+	ctx context.Context,
+	cfg *config.Config,
+	db *sql.DB,
+	audit *service.AuditService,
+	logger *slog.Logger,
+) error {
+	if cfg.Auth.Type == "none" {
+		// Demo mode itself. The residual is the runtime state at
+		// this auth type, so warning about it would be noise.
+		return nil
+	}
+
+	residue, err := queryDemoAnonResidue(ctx, db)
+	if err != nil {
+		return fmt.Errorf("preflight demo-mode residual: %w", err)
+	}
+	if len(residue) == 0 {
+		return nil
+	}
+
+	formatted := make([]string, 0, len(residue))
+	for _, r := range residue {
+		formatted = append(formatted, r.String())
+	}
+
+	msg := fmt.Sprintf(
+		"production startup warning: actor-demo-anon has %d residual role grant(s) "+
+			"from the migration 000029 baseline or a prior demo-mode run: %s. "+
+			"These grants are DORMANT at the current auth_type (%s) but represent a "+
+			"security debt — any future regression that resolves an unauthenticated "+
+			"request to actor-demo-anon would re-elevate to admin. Clean up via "+
+			"POST /api/v1/auth/demo-residual/cleanup (requires auth.role.assign) or "+
+			"`DELETE FROM actor_roles WHERE actor_id = 'actor-demo-anon';`. Set "+
+			"CERTCTL_DEMO_MODE_RESIDUAL_STRICT=true to refuse startup until cleanup.",
+		len(residue), strings.Join(formatted, "; "), cfg.Auth.Type,
+	)
+	if logger != nil {
+		logger.Warn(msg, "auth_type", cfg.Auth.Type, "residue_count", len(residue))
+	} else {
+		slog.Warn(msg)
+	}
+
+	if audit != nil {
+		details := map[string]interface{}{
+			"auth_type":     cfg.Auth.Type,
+			"residue_count": len(residue),
+			"residue":       formatted,
+		}
+		if err := audit.RecordEventWithCategory(
+			ctx, "system", domain.ActorTypeSystem,
+			"auth.demo_residual_grants_detected",
+			domain.EventCategoryAuth,
+			"actor_roles", authdomain.DemoAnonActorID,
+			details,
+		); err != nil {
+			// Don't fail startup over an audit-write error; just log.
+			if logger != nil {
+				logger.Warn("preflight demo-mode residual: audit record failed", "error", err)
+			}
+		}
+	}
+
+	if cfg.Auth.DemoModeResidualStrict {
+		return fmt.Errorf(
+			"startup refused: actor-demo-anon has %d residual role grant(s) and "+
+				"CERTCTL_DEMO_MODE_RESIDUAL_STRICT=true. Remove the rows before restarting",
+			len(residue),
+		)
+	}
+	return nil
+}
+
+// demoAnonResidueRow describes a single live actor_roles row whose
+// actor_id matches the synthetic demo-anon ID.
+type demoAnonResidueRow struct {
+	RoleID    string
+	ScopeType string
+	ScopeID   string
+	GrantedAt time.Time
+}
+
+// String renders one row as `role@scope (granted ts)`. Used both in
+// the WARN log message and in the audit row's residue list.
+func (r demoAnonResidueRow) String() string {
+	scope := r.ScopeType
+	if r.ScopeID != "" {
+		scope = fmt.Sprintf("%s/%s", r.ScopeType, r.ScopeID)
+	}
+	return fmt.Sprintf("%s@%s (granted %s)", r.RoleID, scope, r.GrantedAt.UTC().Format(time.RFC3339))
+}
+
+// queryDemoAnonResidue runs the canonical query for the residue
+// detector + the cleanup endpoint. Kept in one place so the two
+// surfaces can't drift on which rows count as "live".
+//
+// "Live" = not expired. Rows with expires_at <= NOW() are treated
+// as already gone (they have no effect even if the actor were to be
+// injected as the principal).
+func queryDemoAnonResidue(ctx context.Context, db *sql.DB) ([]demoAnonResidueRow, error) {
+	if db == nil {
+		return nil, errors.New("db is nil")
+	}
+	rows, err := db.QueryContext(ctx, `
+		SELECT role_id, scope_type, COALESCE(scope_id, '') AS scope_id, granted_at
+		FROM actor_roles
+		WHERE actor_id = $1
+		  AND (expires_at IS NULL OR expires_at > NOW())
+		ORDER BY granted_at ASC, role_id ASC, scope_type ASC, COALESCE(scope_id, '') ASC
+	`, authdomain.DemoAnonActorID)
+	if err != nil {
+		return nil, fmt.Errorf("query actor_roles: %w", err)
+	}
+	defer rows.Close()
+
+	var out []demoAnonResidueRow
+	for rows.Next() {
+		var r demoAnonResidueRow
+		if err := rows.Scan(&r.RoleID, &r.ScopeType, &r.ScopeID, &r.GrantedAt); err != nil {
+			return nil, fmt.Errorf("scan actor_roles row: %w", err)
+		}
+		out = append(out, r)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, fmt.Errorf("iterate actor_roles rows: %w", err)
+	}
+	return out, nil
+}
+
+// deleteDemoAnonResidue removes every live actor_roles row for the
+// synthetic demo-anon actor. Returns the count removed. Used by the
+// POST /api/v1/auth/demo-residual/cleanup handler. Idempotent — a
+// follow-up call returns 0.
+func deleteDemoAnonResidue(ctx context.Context, db *sql.DB) (int64, error) {
+	if db == nil {
+		return 0, errors.New("db is nil")
+	}
+	res, err := db.ExecContext(ctx, `
+		DELETE FROM actor_roles
+		WHERE actor_id = $1
+	`, authdomain.DemoAnonActorID)
+	if err != nil {
+		return 0, fmt.Errorf("delete actor_roles: %w", err)
+	}
+	n, err := res.RowsAffected()
+	if err != nil {
+		return 0, fmt.Errorf("rows affected: %w", err)
+	}
+	return n, nil
+}

cmd/server/preflight_demo_residual_test.go

@@ -0,0 +1,295 @@
+package main
+
+import (
+	"context"
+	"database/sql"
+	"fmt"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"runtime"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+
+	_ "github.com/lib/pq"
+	"github.com/testcontainers/testcontainers-go"
+	"github.com/testcontainers/testcontainers-go/wait"
+
+	"github.com/certctl-io/certctl/internal/config"
+	"github.com/certctl-io/certctl/internal/repository/postgres"
+	"github.com/certctl-io/certctl/internal/service"
+)
+
+// Audit 2026-05-11 A-8 — preflight + cleanup regression tests for the
+// demo-mode residual-grants detector. Testcontainers-backed because the
+// preflight runs raw SQL against actor_roles; mock-DB-only would not
+// catch a SQL-shape regression. Gated by testing.Short() to keep the
+// fast loop fast (matching internal/repository/postgres/* pattern).
+
+var (
+	a8DBOnce sync.Once
+	a8DB     *sql.DB
+	a8Skip   bool
+	a8SkipMu sync.Mutex
+)
+
+func setupA8DB(t *testing.T) *sql.DB {
+	t.Helper()
+	if testing.Short() {
+		t.Skip("preflight A-8 test requires Postgres (testcontainers); skipping under -short")
+	}
+	a8DBOnce.Do(func() {
+		ctx := context.Background()
+		req := testcontainers.ContainerRequest{
+			Image:        "postgres:16-alpine",
+			ExposedPorts: []string{"5432/tcp"},
+			Env: map[string]string{
+				"POSTGRES_DB":       "certctl_test_a8",
+				"POSTGRES_USER":     "certctl",
+				"POSTGRES_PASSWORD": "certctl",
+			},
+			WaitingFor: wait.ForLog("database system is ready to accept connections").WithOccurrence(2),
+		}
+		c, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
+			ContainerRequest: req,
+			Started:          true,
+		})
+		if err != nil {
+			a8SkipMu.Lock()
+			a8Skip = true
+			a8SkipMu.Unlock()
+			t.Logf("skipping A-8 testcontainers preflight (docker unavailable): %v", err)
+			return
+		}
+		host, err := c.Host(ctx)
+		if err != nil {
+			t.Fatalf("get container host: %v", err)
+		}
+		port, err := c.MappedPort(ctx, "5432")
+		if err != nil {
+			t.Fatalf("get mapped port: %v", err)
+		}
+		dsn := fmt.Sprintf("postgres://certctl:certctl@%s:%s/certctl_test_a8?sslmode=disable", host, port.Port())
+
+		db, err := sql.Open("postgres", dsn)
+		if err != nil {
+			t.Fatalf("sql.Open: %v", err)
+		}
+		// Run all migrations so actor_roles exists with the migration
+		// 000029 seed row (`ar-demo-anon-admin`).
+		_, thisFile, _, _ := runtime.Caller(0)
+		migrationsDir := filepath.Join(filepath.Dir(thisFile), "..", "..", "migrations")
+		if _, err := os.Stat(migrationsDir); err != nil {
+			t.Fatalf("locate migrations dir %q: %v", migrationsDir, err)
+		}
+		if err := postgres.RunMigrations(db, migrationsDir); err != nil {
+			t.Fatalf("RunMigrations: %v", err)
+		}
+		a8DB = db
+	})
+
+	a8SkipMu.Lock()
+	skip := a8Skip
+	a8SkipMu.Unlock()
+	if skip {
+		t.Skip("A-8 testcontainers unavailable; skipping")
+	}
+	return a8DB
+}
+
+// resetA8Residue clears the actor_roles rows for actor-demo-anon AND
+// re-inserts the migration 000029 baseline. Used by tests that need a
+// known "post-fresh-migration" state.
+func resetA8Residue(t *testing.T, db *sql.DB, seedBaseline bool) {
+	t.Helper()
+	if _, err := db.ExecContext(context.Background(),
+		`DELETE FROM actor_roles WHERE actor_id = 'actor-demo-anon'`); err != nil {
+		t.Fatalf("reset actor_roles: %v", err)
+	}
+	if seedBaseline {
+		if _, err := db.ExecContext(context.Background(), `
+			INSERT INTO actor_roles (id, actor_id, actor_type, role_id, granted_at, granted_by, tenant_id)
+			VALUES ('ar-demo-anon-admin', 'actor-demo-anon', 'Anonymous', 'r-admin', NOW(), 'system', 't-default')
+		`); err != nil {
+			t.Fatalf("reseed baseline: %v", err)
+		}
+	}
+}
+
+// TestPreflightDemoModeResidual_DemoModeActive_Skips proves the
+// preflight short-circuits when Auth.Type=none regardless of residue.
+// Demo mode IS the active runtime state at that auth type, so warning
+// would be noise.
+func TestPreflightDemoModeResidual_DemoModeActive_Skips(t *testing.T) {
+	db := setupA8DB(t)
+	resetA8Residue(t, db, true) // baseline IS present
+
+	cfg := &config.Config{}
+	cfg.Auth.Type = "none"
+	cfg.Auth.DemoModeResidualStrict = true // would refuse if checked
+
+	logger := slog.New(slog.NewTextHandler(os.Stderr, nil))
+	err := preflightDemoModeResidual(context.Background(), cfg, db, nil, logger)
+	if err != nil {
+		t.Fatalf("expected nil under Auth.Type=none, got %v", err)
+	}
+}
+
+// TestPreflightDemoModeResidual_NoResidue_Passes proves a fully-clean
+// actor_roles state passes without WARN.
+func TestPreflightDemoModeResidual_NoResidue_Passes(t *testing.T) {
+	db := setupA8DB(t)
+	resetA8Residue(t, db, false) // explicitly empty
+
+	cfg := &config.Config{}
+	cfg.Auth.Type = "api-key"
+
+	err := preflightDemoModeResidual(context.Background(), cfg, db, nil, nil)
+	if err != nil {
+		t.Fatalf("expected nil with empty residue, got %v", err)
+	}
+}
+
+// TestPreflightDemoModeResidual_HasResidue_LogsAndAudits proves the
+// migration 000029 baseline produces a WARN + audit row but does NOT
+// fail startup in default (non-strict) mode.
+func TestPreflightDemoModeResidual_HasResidue_LogsAndAudits(t *testing.T) {
+	db := setupA8DB(t)
+	resetA8Residue(t, db, true)
+
+	cfg := &config.Config{}
+	cfg.Auth.Type = "api-key"
+	cfg.Auth.DemoModeResidualStrict = false
+
+	auditRepo := postgres.NewAuditRepository(db)
+	auditService := service.NewAuditService(auditRepo)
+
+	err := preflightDemoModeResidual(context.Background(), cfg, db, auditService, nil)
+	if err != nil {
+		t.Fatalf("non-strict mode must NOT fail startup with residue, got %v", err)
+	}
+
+	// Audit row should be present for the call.
+	rows, err := db.QueryContext(context.Background(), `
+		SELECT action, event_category, resource_id
+		FROM audit_events
+		WHERE action = 'auth.demo_residual_grants_detected'
+		ORDER BY occurred_at DESC LIMIT 1
+	`)
+	if err != nil {
+		t.Fatalf("audit_events query: %v", err)
+	}
+	defer rows.Close()
+	if !rows.Next() {
+		t.Fatal("expected at least one auth.demo_residual_grants_detected row")
+	}
+	var action, category, resourceID string
+	if err := rows.Scan(&action, &category, &resourceID); err != nil {
+		t.Fatalf("scan: %v", err)
+	}
+	if action != "auth.demo_residual_grants_detected" {
+		t.Errorf("action = %q, want auth.demo_residual_grants_detected", action)
+	}
+	if category != "auth" {
+		t.Errorf("event_category = %q, want auth", category)
+	}
+	if resourceID != "actor-demo-anon" {
+		t.Errorf("resource_id = %q, want actor-demo-anon", resourceID)
+	}
+}
+
+// TestPreflightDemoModeResidual_StrictMode_RefusesStartup proves the
+// flag pivots WARN → fail.
+func TestPreflightDemoModeResidual_StrictMode_RefusesStartup(t *testing.T) {
+	db := setupA8DB(t)
+	resetA8Residue(t, db, true)
+
+	cfg := &config.Config{}
+	cfg.Auth.Type = "api-key"
+	cfg.Auth.DemoModeResidualStrict = true
+
+	err := preflightDemoModeResidual(context.Background(), cfg, db, nil, nil)
+	if err == nil {
+		t.Fatal("strict mode + residue: expected error, got nil")
+	}
+	if !strings.Contains(err.Error(), "actor-demo-anon") {
+		t.Errorf("err = %q, want mention of actor-demo-anon", err.Error())
+	}
+	if !strings.Contains(err.Error(), "CERTCTL_DEMO_MODE_RESIDUAL_STRICT") {
+		t.Errorf("err = %q, want mention of CERTCTL_DEMO_MODE_RESIDUAL_STRICT", err.Error())
+	}
+}
+
+// TestDemoAnonResidueRow_String pins the formatting of the residue
+// detail entry — used both in the WARN log AND the audit row's
+// `residue` slice. Two cases: NULL scope_id (global scope) and
+// non-empty scope_id (profile/issuer scope).
+func TestDemoAnonResidueRow_String(t *testing.T) {
+	ts, _ := time.Parse(time.RFC3339, "2026-05-11T12:34:56Z")
+	cases := []struct {
+		name string
+		r    demoAnonResidueRow
+		want string
+	}{
+		{
+			name: "global_scope",
+			r:    demoAnonResidueRow{RoleID: "r-admin", ScopeType: "global", ScopeID: "", GrantedAt: ts},
+			want: "r-admin@global (granted 2026-05-11T12:34:56Z)",
+		},
+		{
+			name: "scoped",
+			r:    demoAnonResidueRow{RoleID: "r-operator", ScopeType: "profile", ScopeID: "p-prod", GrantedAt: ts},
+			want: "r-operator@profile/p-prod (granted 2026-05-11T12:34:56Z)",
+		},
+	}
+	for _, c := range cases {
+		c := c
+		t.Run(c.name, func(t *testing.T) {
+			got := c.r.String()
+			if got != c.want {
+				t.Errorf("String() = %q, want %q", got, c.want)
+			}
+		})
+	}
+}
+
+// TestDeleteDemoAnonResidue_Idempotent proves the cleanup helper is
+// re-entrant: a second call after a successful first call returns 0.
+func TestDeleteDemoAnonResidue_Idempotent(t *testing.T) {
+	db := setupA8DB(t)
+	resetA8Residue(t, db, true)
+
+	n, err := deleteDemoAnonResidue(context.Background(), db)
+	if err != nil {
+		t.Fatalf("first delete: %v", err)
+	}
+	if n < 1 {
+		t.Fatalf("first delete: count = %d, want >= 1", n)
+	}
+
+	n, err = deleteDemoAnonResidue(context.Background(), db)
+	if err != nil {
+		t.Fatalf("second delete: %v", err)
+	}
+	if n != 0 {
+		t.Errorf("second delete (idempotent): count = %d, want 0", n)
+	}
+}
+
+// TestQueryDemoAnonResidue_NilDB pins the nil-safety contract.
+func TestQueryDemoAnonResidue_NilDB(t *testing.T) {
+	_, err := queryDemoAnonResidue(context.Background(), nil)
+	if err == nil {
+		t.Fatal("expected error on nil db, got nil")
+	}
+}
+
+// TestDeleteDemoAnonResidue_NilDB pins the nil-safety contract.
+func TestDeleteDemoAnonResidue_NilDB(t *testing.T) {
+	_, err := deleteDemoAnonResidue(context.Background(), nil)
+	if err == nil {
+		t.Fatal("expected error on nil db, got nil")
+	}
+}

cmd/server/tls.go

@@ -1,3 +1,6 @@
+// Copyright 2026 certctl LLC. All rights reserved.
+// SPDX-License-Identifier: BUSL-1.1
+
 package main
 
 import (

deploy/.env.example

@@ -1,8 +1,39 @@
-# certctl Docker Compose environment variables
+# certctl Docker Compose environment variables (Bundle 2 — 2026-05-12)
-# Copy this file to .env and customize for your deployment
+#
+# Copy this file to deploy/.env and customize. The production-shaped base
+# compose (docker-compose.yml) requires every variable below to be set;
+# the Bundle 2 fail-closed startup guards REFUSE TO BOOT if any value
+# remains at a "change-me-..." or "replace-with-..." placeholder outside
+# demo mode (CERTCTL_DEMO_MODE_ACK=true).
+#
+# DEMO PATH (zero-config, populated dashboard, demo-mode auth):
+#   docker compose -f deploy/docker-compose.yml \
+#                  -f deploy/docker-compose.demo.yml up -d --build
+# The demo overlay supplies its own placeholder values plus DEMO_MODE_ACK
+# so this .env is NOT needed.
+#
+# PRODUCTION PATH (this .env is required):
+#   docker compose -f deploy/docker-compose.yml up -d
 
-# PostgreSQL password (change in production!)
+# PostgreSQL password — openssl rand -hex 32
-POSTGRES_PASSWORD=certctl
+POSTGRES_PASSWORD=replace-with-openssl-rand-hex-32
 
-# Agent API key (change in production! Generate with: openssl rand -hex 32)
+# Server API-key secret — openssl rand -base64 32
-CERTCTL_API_KEY=change-me-in-production
+CERTCTL_AUTH_SECRET=replace-with-openssl-rand-base64-32
+
+# Bundled-agent API key (matches one of the server's AUTH_SECRET rotation
+# values). Generate with: openssl rand -base64 32
+CERTCTL_API_KEY=replace-with-openssl-rand-base64-32
+
+# AES-256-GCM key for encrypting issuer/target config secrets at rest.
+# Minimum 32 bytes. Generate with: openssl rand -base64 32
+CERTCTL_CONFIG_ENCRYPTION_KEY=replace-with-openssl-rand-base64-32
+
+# Agent ID returned from `POST /api/v1/agents` during agent enrollment.
+# Without this the bundled certctl-agent service fail-fasts at startup.
+# CERTCTL_AGENT_ID=agent-from-registration-response
+
+# Day-0 admin bootstrap token (optional — generate with: openssl rand -hex 32).
+# When set, POST /api/v1/auth/bootstrap mints the first admin actor + API
+# key. When unset (default), that endpoint returns 410 Gone.
+# CERTCTL_BOOTSTRAP_TOKEN=

deploy/ENVIRONMENTS.md

@@ -62,7 +62,9 @@ A compose file defines **services** (containers), **networks** (how they talk to
 ## Base Environment
 
 **File:** `docker-compose.yml`
-**When to use:** Production deployments, first-time setup, or any time you want a clean dashboard with the onboarding wizard.
+**When to use:** Production deployments and any time you want a clean, production-shaped stack with real authentication enforced.
+
+**Bundle 2 closure (2026-05-12):** the base compose was split from the demo overlay. Pre-Bundle-2 this file IS the demo path (auth=none, keygen=server, demo-seed=true, change-me placeholder credentials baked in). Operators reading "drop the demo overlay for a clean install" were not getting a clean install — they were getting a demo stack with the overlay's data layer stripped off. Post-Bundle-2 the base ships production-shaped: `CERTCTL_AUTH_TYPE` defaults to `api-key`, `CERTCTL_KEYGEN_MODE` defaults to `agent`, demo-mode + demo-seed default to false, and every credential placeholder is rejected at startup. The demo path is now a single overlay flag away (`-f deploy/docker-compose.demo.yml`).
 
 ### What it runs
 
@@ -79,9 +81,20 @@ Three services on a private bridge network:
 ```bash
 git clone https://github.com/certctl-io/certctl.git
 cd certctl
+
+# Required: provide real credentials. Without this step the server fail-fasts
+# at startup on the Bundle 2 placeholder-credential guards.
+cp .env.example deploy/.env
+$EDITOR deploy/.env
+# Set: POSTGRES_PASSWORD, CERTCTL_AUTH_SECRET, CERTCTL_API_KEY,
+#      CERTCTL_CONFIG_ENCRYPTION_KEY (all via `openssl rand -base64 32`),
+#      CERTCTL_AGENT_ID (returned from `POST /api/v1/agents`).
+
 docker compose -f deploy/docker-compose.yml up -d --build
 ```
 
+If you just want to kick the tires without writing a `.env`, use the demo overlay instead — see [Demo Overlay](#demo-overlay) below.
+
 `--build` compiles the Go server and agent from source, including the React frontend. Without it, Docker may reuse a stale image from a previous build.
 
 `-d` runs in detached mode (background). Omit it to see logs in your terminal.
@@ -132,14 +145,16 @@ certctl-server:
     postgres:
       condition: service_healthy
   environment:
-    CERTCTL_DATABASE_URL: postgres://certctl:${POSTGRES_PASSWORD:-certctl}@postgres:5432/certctl?sslmode=disable
+    CERTCTL_DATABASE_URL: postgres://certctl:${POSTGRES_PASSWORD}@postgres:5432/certctl?sslmode=disable
     CERTCTL_SERVER_HOST: 0.0.0.0
     CERTCTL_SERVER_PORT: 8443
     CERTCTL_LOG_LEVEL: info
-    CERTCTL_AUTH_TYPE: none
+    # Bundle 2 (2026-05-12): no auth-type / keygen-mode override here.
-    CERTCTL_KEYGEN_MODE: server
+    # Code defaults (api-key + agent) take effect; the demo overlay flips
+    # both to demo-mode (none + server).
+    CERTCTL_AUTH_SECRET: ${CERTCTL_AUTH_SECRET}
     CERTCTL_NETWORK_SCAN_ENABLED: "true"
-    CERTCTL_CONFIG_ENCRYPTION_KEY: ${CERTCTL_CONFIG_ENCRYPTION_KEY:-change-me-32-char-encryption-key}
+    CERTCTL_CONFIG_ENCRYPTION_KEY: ${CERTCTL_CONFIG_ENCRYPTION_KEY}
 ```
 
 The server is the control plane. It serves the REST API, the React dashboard, runs 7 background scheduler loops (renewal, job processing, health checks, notifications, short-lived cert expiry, network scanning, digest emails), and manages the issuer/target registry.
@@ -147,9 +162,10 @@ The server is the control plane. It serves the REST API, the React dashboard, ru
 Key environment variables explained:
 
 - `CERTCTL_DATABASE_URL` references the `postgres` service by hostname. Docker's internal DNS resolves `postgres` to the container's IP on the bridge network. `sslmode=disable` is appropriate because traffic stays on the private Docker network.
-- `CERTCTL_AUTH_TYPE: none` disables API key authentication so you can explore immediately. For production, set `api-key` and configure `CERTCTL_AUTH_SECRET`.
+- `CERTCTL_AUTH_TYPE` defaults to `api-key` in the code (`internal/config/config.go`); the base compose does NOT override it. To run demo-mode auth (every request served as the synthetic admin actor), layer the demo overlay on top.
-- `CERTCTL_KEYGEN_MODE: server` means the server generates private keys. This is convenient for demos but insecure for production. In production, set `agent` so keys are generated on agent machines and never transmitted.
+- `CERTCTL_AUTH_SECRET` is the API-key value the server accepts. The Bundle 2 fail-closed guard rejects the literal placeholder `change-me-in-production` outside demo mode. Generate with `openssl rand -base64 32`.
-- `CERTCTL_CONFIG_ENCRYPTION_KEY` enables AES-256-GCM encryption for issuer and target configurations stored in the database (credentials, API keys). Without this, the dynamic configuration GUI (adding issuers/targets from the dashboard) won't encrypt sensitive fields. For production, generate a strong random key.
+- `CERTCTL_KEYGEN_MODE` defaults to `agent` in the code (the base compose does NOT override it). Production deploys leave it there so private keys stay on agent infrastructure; the demo overlay flips it to `server` so the demo can issue + hold the key on the server box without an agent dance.
+- `CERTCTL_CONFIG_ENCRYPTION_KEY` enables AES-256-GCM encryption for issuer and target configurations stored in the database (credentials, API keys). Required for any deploy that adds issuers via the GUI. The Bundle 2 fail-closed guard rejects the literal placeholder `change-me-32-char-encryption-key` outside demo mode. Generate with `openssl rand -base64 32` (≥ 32 bytes).
 - `CERTCTL_NETWORK_SCAN_ENABLED` activates the scheduler loop that probes TLS endpoints on your network to discover certificates you might not be managing.
 
 **Expert note:** The healthcheck hits `GET /health` every 10 seconds with 5 retries. The `depends_on: condition: service_healthy` on the agent means Docker holds agent startup until this check passes. Resource limits (`cpus: '1.0'`, `memory: 512M`) prevent the server from consuming unbounded resources in shared environments.
@@ -162,8 +178,12 @@ certctl-agent:
     certctl-server:
       condition: service_healthy
   environment:
-    CERTCTL_SERVER_URL: http://certctl-server:8443
+    CERTCTL_SERVER_URL: https://certctl-server:8443
-    CERTCTL_API_KEY: ${CERTCTL_API_KEY:-change-me-in-production}
+    # Bundle 2 (2026-05-12): no placeholder fallbacks. Operators MUST
+    # set CERTCTL_API_KEY + CERTCTL_AGENT_ID in deploy/.env. The agent
+    # binary fail-fasts at startup when CERTCTL_AGENT_ID is unset.
+    CERTCTL_API_KEY: ${CERTCTL_API_KEY}
+    CERTCTL_AGENT_ID: ${CERTCTL_AGENT_ID}
     CERTCTL_AGENT_NAME: docker-agent
     CERTCTL_LOG_LEVEL: info
     CERTCTL_DISCOVERY_DIRS: /var/lib/certctl/keys
@@ -194,13 +214,18 @@ docker compose -f deploy/docker-compose.yml down -v
 ## Demo Overlay
 
 **File:** `docker-compose.demo.yml`
-**When to use:** Demos, screenshots, stakeholder presentations, or any time you want a populated dashboard on first boot.
+**When to use:** Demos, screenshots, stakeholder presentations, or any time you want a one-command zero-config evaluation stack with a populated dashboard.
 
 ### What it adds
 
-One env var: `CERTCTL_DEMO_SEED=true` on the `certctl-server` service. The server applies `migrations/seed_demo.sql` at boot via `postgres.RunDemoSeed` AFTER the baseline migrations + `seed.sql` are in place. The demo seed file inserts 180 days of simulated operational history: teams, owners, certificates across multiple issuers, agents on different platforms, jobs with realistic timestamps, discovery scan results, audit events, policies, and profiles.
+Bundle 2 closure (2026-05-12) moved every demo-mode env var out of the base compose into this overlay. The overlay now carries:
 
-Pre-U-3 the overlay used to mount `seed_demo.sql` into PostgreSQL's `/docker-entrypoint-initdb.d/` and rely on initdb-time application. That worked only because the production stack also mounted the migrations there, so the schema existed when initdb ran. Once U-3 dropped the production initdb mounts (single source of truth: server runs `RunMigrations` + `RunSeed` at boot), the demo seed could no longer be applied at initdb time — the tables it references wouldn't exist yet. Post-U-3 the overlay is a 27-line override file with no `image:` / `build:` of its own; it MUST be passed alongside the base, or compose errors with `service "certctl-server" has neither an image nor a build context specified`.
+- `CERTCTL_AUTH_TYPE=none` + `CERTCTL_DEMO_MODE_ACK=true` — demo-mode synthetic admin actor (`actor-demo-anon`). The server emits a prominent ⚠ DEMO MODE WARN banner at boot with a production-promotion checklist (`cmd/server/main.go`).
+- `CERTCTL_KEYGEN_MODE=server` — demo-only server-side keygen.
+- `CERTCTL_DEMO_SEED=true` — the server applies `migrations/seed_demo.sql` at boot via `postgres.RunDemoSeed`, inserting 180 days of simulated operational history (teams, owners, certificates, agents, jobs, discovery results, audit events, policies, profiles).
+- Fixed weak `POSTGRES_PASSWORD=certctl`, `CERTCTL_AUTH_SECRET=change-me-in-production`, `CERTCTL_CONFIG_ENCRYPTION_KEY=change-me-32-char-encryption-key`, `CERTCTL_API_KEY=change-me-in-production`, `CERTCTL_AGENT_ID=agent-demo-1` — placeholder credentials the Bundle 2 fail-closed `Validate()` rejects outside demo mode, but the demo overlay's `DEMO_MODE_ACK=true` unlocks them.
+
+Pre-U-3 the overlay used to mount `seed_demo.sql` into PostgreSQL's `/docker-entrypoint-initdb.d/` and rely on initdb-time application. That worked only because the production stack also mounted the migrations there, so the schema existed when initdb ran. Once U-3 dropped the production initdb mounts (single source of truth: server runs `RunMigrations` + `RunSeed` at boot), the demo seed could no longer be applied at initdb time — the tables it references wouldn't exist yet. Post-U-3 the overlay is an override file with no `image:` / `build:` of its own; it MUST be passed alongside the base, or compose errors with `service "certctl-server" has neither an image nor a build context specified`.
 
 ### Starting it
 
@@ -382,7 +407,7 @@ Every `CERTCTL_*` environment variable is read by the server's `internal/config/
 | `CERTCTL_SERVER_HOST` | `0.0.0.0` | Listen address |
 | `CERTCTL_SERVER_PORT` | `8443` | Listen port |
 | `CERTCTL_LOG_LEVEL` | `info` | Log verbosity: `debug`, `info`, `warn`, `error` |
-| `CERTCTL_AUTH_TYPE` | `api-key` | Auth mode: `api-key` or `none` |
+| `CERTCTL_AUTH_TYPE` | `api-key` | Auth mode: `api-key`, `none`, or `oidc` (Auth Bundle 2). |
 | `CERTCTL_AUTH_SECRET` | (none) | API key(s), comma-separated for rotation |
 | `CERTCTL_KEYGEN_MODE` | `agent` | Key generation: `agent` (production) or `server` (demo) |
 | `CERTCTL_CONFIG_ENCRYPTION_KEY` | (none) | AES-256-GCM key for encrypting issuer/target configs in DB |
@@ -392,6 +417,11 @@ Every `CERTCTL_*` environment variable is read by the server's `internal/config/
 | `CERTCTL_CORS_ORIGINS` | (empty) | Allowed CORS origins, comma-separated. Empty = deny all cross-origin |
 | `CERTCTL_RATE_LIMIT_RPS` | `10` | Requests per second per client |
 | `CERTCTL_RATE_LIMIT_BURST` | `20` | Burst allowance above RPS |
+| `CERTCTL_AGENT_BOOTSTRAP_TOKEN` | (empty) | Agent-registration bootstrap secret. Empty = v2.1.x warn-mode pass-through. Set to a real value (`openssl rand -base64 32`); the deny-empty flag's default flip in v2.2.0 will require it. |
+| `CERTCTL_AGENT_BOOTSTRAP_TOKEN_DENY_EMPTY` | `false` | Phase 2 SEC-H1 staged flag. When `true`, the server refuses to start unless `CERTCTL_AGENT_BOOTSTRAP_TOKEN` is non-empty. Default flip to `true` scheduled for v2.2.0. |
+| `CERTCTL_DEMO_MODE_ACK` | `false` | Acknowledges demo-mode synthetic admin posture (required when `CERTCTL_AUTH_TYPE=none` binds to a non-loopback host). Must be paired with `CERTCTL_DEMO_MODE_ACK_TS` per Phase 2 SEC-H3. |
+| `CERTCTL_DEMO_MODE_ACK_TS` | (empty) | Phase 2 SEC-H3: unix-epoch timestamp at which DemoModeAck was last acknowledged. When `CERTCTL_DEMO_MODE_ACK=true`, this must parse as a unix epoch within the last 24h. Set via `CERTCTL_DEMO_MODE_ACK_TS=$(date +%s)` at every `docker compose up`. |
+| `CERTCTL_ACME_INSECURE_ACK` | `false` | Phase 2 SEC-M4: explicit ACK required to boot with `CERTCTL_ACME_INSECURE=true`. Production deploys MUST never set either flag. |
 
 ### Agent
 
@@ -400,7 +430,7 @@ Every `CERTCTL_*` environment variable is read by the server's `internal/config/
 | `CERTCTL_SERVER_URL` | (required) | Server API URL |
 | `CERTCTL_API_KEY` | (none) | API key for authenticating with server |
 | `CERTCTL_AGENT_NAME` | (hostname) | Display name in dashboard |
-| `CERTCTL_AGENT_ID` | (auto-generated) | Stable agent identifier |
+| `CERTCTL_AGENT_ID` | (none — required) | Stable agent identifier returned from `POST /api/v1/agents`. The agent binary fail-fasts at startup if unset. |
 | `CERTCTL_KEYGEN_MODE` | `agent` | Must match server setting |
 | `CERTCTL_LOG_LEVEL` | `info` | Log verbosity |
 | `CERTCTL_KEY_DIR` | `/var/lib/certctl/keys` | Directory for private key storage (0600 perms) |
@@ -415,6 +445,7 @@ Every `CERTCTL_*` environment variable is read by the server's `internal/config/
 | `CERTCTL_ACME_CHALLENGE_TYPE` | `http-01`, `dns-01`, or `dns-persist-01` |
 | `CERTCTL_ACME_INSECURE` | Skip TLS verification for ACME CA (test only) |
 | `CERTCTL_ACME_EAB_KID` / `CERTCTL_ACME_EAB_HMAC` | External Account Binding for ZeroSSL, Google Trust Services |
+| `CERTCTL_ZEROSSL_EAB_URL` | Override the ZeroSSL EAB-credentials endpoint (defaults to the public ZeroSSL URL; only set for ZeroSSL staging or a private mirror) |
 | `CERTCTL_ACME_ARI_ENABLED` | Enable RFC 9773 Renewal Information |
 | `CERTCTL_ACME_PROFILE` | ACME profile (`tlsserver`, `shortlived`) |
 | `CERTCTL_STEPCA_URL` | step-ca server URL |

deploy/demo-up.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# deploy/demo-up.sh — boot the certctl demo stack with the fresh
+# CERTCTL_DEMO_MODE_ACK_TS the Phase 2 SEC-H3 guard requires.
+#
+# The demo overlay sets CERTCTL_DEMO_MODE_ACK=true. Phase 2 SEC-H3
+# (2026-05-13) pairs that with a fail-closed requirement: the server
+# refuses to start unless CERTCTL_DEMO_MODE_ACK_TS=<unix-epoch> is set
+# and is within the last 24h (with 1-minute future clock-skew tolerance).
+#
+# A static value in docker-compose.demo.yml would rot the next day, so
+# the overlay passthroughs the value from the shell environment. This
+# helper mints a fresh TS at run time and forwards any extra args to
+# `docker compose up`, so operators can use it as a drop-in replacement
+# for the bare command. Example:
+#
+#     ./demo-up.sh -d                  # cold boot in detached mode
+#     ./demo-up.sh -d --pull always    # forward any flags through
+#
+# The cold-DB compose smoke in .github/workflows/ci.yml does the same
+# thing inline; this script exists so local operators don't have to
+# remember the export.
+
+set -euo pipefail
+
+# cd to the deploy/ dir so the relative `-f` paths resolve regardless
+# of where the operator invokes this from. The script lives next to
+# the compose files it references.
+cd "$(dirname "$0")"
+
+export CERTCTL_DEMO_MODE_ACK_TS="$(date +%s)"
+
+echo "[demo-up] minting CERTCTL_DEMO_MODE_ACK_TS=$CERTCTL_DEMO_MODE_ACK_TS"
+echo "[demo-up] running: docker compose -f docker-compose.yml -f docker-compose.demo.yml up $*"
+
+exec docker compose \
+  -f docker-compose.yml \
+  -f docker-compose.demo.yml \
+  up "$@"

deploy/docker-compose.demo.yml

@@ -1,26 +1,125 @@
-# Demo mode: pre-populated dashboard with 32 certificates, 8 agents, 10 issuers, etc.
+# =============================================================================
-# Use this to showcase certctl's dashboard with realistic data.
+# certctl DEMO overlay — Bundle 2 (2026-05-12)
+# =============================================================================
 #
-# Usage:
+# Layered on top of the production-shaped base (docker-compose.yml) to give
-#   docker compose -f docker-compose.yml -f docker-compose.demo.yml up --build
+# operators a one-command, zero-config demo path:
 #
-# To start fresh (wipe previous data):
+#   deploy/demo-up.sh -d --build
-#   docker compose -f docker-compose.yml -f docker-compose.demo.yml down -v
-#   docker compose -f docker-compose.yml -f docker-compose.demo.yml up --build
 #
-# U-3 (P1, cat-u-seed_initdb_schema_drift): pre-U-3 this overlay mounted
+# (which forwards args to `docker compose up` after exporting the fresh
-# `seed_demo.sql` into postgres `/docker-entrypoint-initdb.d/`. That worked
+# CERTCTL_DEMO_MODE_ACK_TS that Phase 2 SEC-H3 requires). Equivalent
-# only because the production stack also mounted the migrations there, so
+# manual invocation:
-# the schema existed at initdb time. Once U-3 dropped the production
+#
+#   CERTCTL_DEMO_MODE_ACK_TS=$(date +%s) docker compose \
+#     -f deploy/docker-compose.yml \
+#     -f deploy/docker-compose.demo.yml up -d --build
+#
+# What this overlay does:
+#
+#   1. Flips CERTCTL_AUTH_TYPE=none + CERTCTL_DEMO_MODE_ACK=true. Every
+#      request is served as the synthetic admin actor `actor-demo-anon`;
+#      the server emits a prominent ⚠ DEMO MODE WARN banner at boot with
+#      a production-promotion checklist (cmd/server/main.go::emitDemoBanner).
+#      Phase 2 SEC-H3 (2026-05-13) pairs DEMO_MODE_ACK with a required
+#      DEMO_MODE_ACK_TS within the last 24h. The overlay reads
+#      ${CERTCTL_DEMO_MODE_ACK_TS:-} from the shell — use deploy/demo-up.sh
+#      (which exports a fresh TS) instead of bare `docker compose up`.
+#
+#   2. Flips CERTCTL_KEYGEN_MODE=server (the demo issues + holds the key on
+#      the server to keep the dashboard populated; production deploys must
+#      use the default `agent` mode where keys never leave the agent box).
+#
+#   3. Flips CERTCTL_DEMO_SEED=true. The server applies migrations/seed_demo.sql
+#      at boot via postgres.RunDemoSeed AFTER baseline migrations + seed.sql,
+#      pre-seeding 180 days of simulated history across 13 issuers + 8 agents.
+#
+#   4. Supplies the change-me-... placeholder values for POSTGRES_PASSWORD,
+#      CERTCTL_API_KEY, CERTCTL_CONFIG_ENCRYPTION_KEY, and CERTCTL_AGENT_ID
+#      so the demo runs without a deploy/.env file. The Bundle 2 fail-closed
+#      Validate() rejects these placeholders outside demo mode, so this only
+#      works alongside DEMO_MODE_ACK=true.
+#
+# U-3 history: pre-U-3 this overlay mounted seed_demo.sql into postgres
+# `/docker-entrypoint-initdb.d/`. That worked only because the production
+# stack also mounted the migrations there. Once U-3 dropped the production
 # initdb mounts (single source of truth: server runs RunMigrations + RunSeed
 # at boot), the demo seed could no longer be applied at initdb time — the
-# tables it references wouldn't exist yet.
+# tables it references wouldn't exist yet. Post-U-3 the overlay just sets
+# CERTCTL_DEMO_SEED=true; the server applies seed_demo.sql at boot via
+# postgres.RunDemoSeed AFTER baseline migrations + seed.sql.
 #
-# Post-U-3 the demo overlay just sets CERTCTL_DEMO_SEED=true; the server
+# Bundle 2 history: pre-Bundle-2 the base compose IS this demo path; this
-# applies seed_demo.sql at boot via postgres.RunDemoSeed AFTER baseline
+# overlay was a single-flag thin shim. Bundle 2 split the demo env vars
-# migrations + seed.sql are in place. Same single source of truth, no
+# out of the base so `docker compose -f deploy/docker-compose.yml up`
-# initdb mounts, no schema-vs-seed drift.
+# (no overlay) boots production-shaped — which is what every operator
+# reading the README quickstart line "drop the demo overlay for a clean
+# install" expected. The overlay carries the full demo posture now.
+#
+# To start fresh (wipe previous data):
+#   docker compose -f deploy/docker-compose.yml \
+#                  -f deploy/docker-compose.demo.yml down -v
+#   deploy/demo-up.sh -d --build
+
 services:
+  postgres:
+    # Fixed weak password is intentional for the no-setup demo path.
+    # See docker-compose.yml for the production override pattern.
+    environment:
+      POSTGRES_PASSWORD: certctl
+
   certctl-server:
     environment:
+      # Demo-mode auth: every request served as the synthetic
+      # `actor-demo-anon` admin. The server's HIGH-12 startup guard
+      # requires DEMO_MODE_ACK=true to allow this combination on a
+      # non-loopback bind; the boot-time WARN banner (cmd/server/main.go)
+      # reminds the operator on every start.
+      CERTCTL_AUTH_TYPE: none
+      CERTCTL_DEMO_MODE_ACK: "true"
+      # Phase 2 SEC-H3 (2026-05-13): DEMO_MODE_ACK=true requires a fresh
+      # DEMO_MODE_ACK_TS within the last 24h. The overlay can't hardcode
+      # a timestamp (it would rot the next day), so we passthrough from
+      # the shell. Operators set this via:
+      #     CERTCTL_DEMO_MODE_ACK_TS=$(date +%s) docker compose \
+      #       -f docker-compose.yml -f docker-compose.demo.yml up -d
+      # The cold-DB smoke + any helper script (deploy/demo-up.sh, when
+      # it lands) export this before invoking compose. Empty value
+      # fails the SEC-H3 guard with a clear operator-facing error
+      # message pointing at this line.
+      CERTCTL_DEMO_MODE_ACK_TS: "${CERTCTL_DEMO_MODE_ACK_TS:-}"
+      # Server-side keygen so the demo can populate the dashboard with
+      # full lifecycle history. Production deploys leave this at the
+      # code default `agent` (CertctlAgent generates ECDSA P-256 keys
+      # locally and submits CSRs only).
+      CERTCTL_KEYGEN_MODE: server
+      # Demo creds — the Bundle 2 fail-closed Validate() rejects these
+      # sentinels outside demo mode, but DEMO_MODE_ACK=true unlocks them.
+      CERTCTL_CONFIG_ENCRYPTION_KEY: change-me-32-char-encryption-key
+      CERTCTL_AUTH_SECRET: change-me-in-production
+      # Cold-DB smoke fix (2026-05-13): the base compose builds the
+      # database URL via compose-level `${POSTGRES_PASSWORD}` interpolation
+      # (deploy/docker-compose.yml line ~177), which reads the SHELL env —
+      # NOT the postgres service's `environment:` block above (that one
+      # feeds the postgres container's initdb only). In a zero-env-var
+      # CI run the shell var is blank, producing
+      # `postgres://certctl:@postgres:5432/...` and a SCRAM rejection
+      # against a database that initdb seeded with password `certctl`.
+      # Pinning the full URL here closes the gap: the demo overlay is
+      # now fully self-sufficient (matches the file's docstring claim)
+      # and the cold-DB smoke passes against a fresh GitHub-runner clone
+      # with no .env file or exported shell vars. Production deploys
+      # override CERTCTL_DATABASE_URL via the base compose's
+      # `${CERTCTL_DATABASE_URL:-...}` default, so this literal is
+      # overlay-scoped and never leaks into a production posture.
+      CERTCTL_DATABASE_URL: postgres://certctl:certctl@postgres:5432/certctl?sslmode=disable
+      # 180-day simulated history seed applied at boot.
       CERTCTL_DEMO_SEED: "true"
+
+  certctl-agent:
+    environment:
+      # Pre-seeded by migrations/seed_demo.sql; the bundled agent
+      # connects with these creds and the demo-mode synthetic admin
+      # accepts every request regardless of API key.
+      CERTCTL_API_KEY: change-me-in-production
+      CERTCTL_AGENT_ID: agent-demo-1

deploy/docker-compose.test.yml

@@ -272,6 +272,14 @@ services:
       CERTCTL_ACME_EMAIL: test@certctl.dev
       CERTCTL_ACME_CHALLENGE_TYPE: http-01
       CERTCTL_ACME_INSECURE: "true"
+      # Phase 2 SEC-M4 (2026-05-13): CERTCTL_ACME_INSECURE=true requires
+      # the paired CERTCTL_ACME_INSECURE_ACK=true; without the ACK the
+      # server's Config.Validate() refuses to start. This integration
+      # stack uses Pebble's self-signed ACME directory, so disabling
+      # TLS verification is correct — but the ACK env var has to be
+      # set explicitly so the test posture matches what production
+      # operators are blocked from doing accidentally.
+      CERTCTL_ACME_INSECURE_ACK: "true"
 
       # step-ca issuer (iss-stepca)
       CERTCTL_STEPCA_URL: https://step-ca:9000

deploy/docker-compose.yml

@@ -1,3 +1,49 @@
+# =============================================================================
+# certctl base compose — PRODUCTION-SHAPED (Bundle 2, 2026-05-12)
+# =============================================================================
+#
+# This base file ships a SAFE-BY-DEFAULT control plane:
+#
+#   - CERTCTL_AUTH_TYPE defaults to api-key (the code default; not overridden
+#     here). The server REFUSES to start with auth=none on a non-loopback
+#     bind unless CERTCTL_DEMO_MODE_ACK=true (Audit 2026-05-10 HIGH-12 +
+#     Bundle 2 closure: see internal/config/config.go::Validate).
+#   - CERTCTL_KEYGEN_MODE defaults to agent (the code default).
+#   - CERTCTL_DEMO_SEED defaults to false (the code default; the 180-day
+#     simulated history seed only runs under the demo overlay).
+#   - Default placeholder credentials (`change-me-...` sentinels) are NOT
+#     interpolated by this compose. The server REFUSES to start when those
+#     placeholder strings reach config (Bundle 2 fail-closed guards) unless
+#     DEMO_MODE_ACK=true. Operators MUST set:
+#         POSTGRES_PASSWORD               (openssl rand -hex 32)
+#         CERTCTL_AUTH_SECRET             (openssl rand -hex 32)
+#         CERTCTL_CONFIG_ENCRYPTION_KEY   (openssl rand -base64 32)
+#         CERTCTL_API_KEY                 (matches CERTCTL_AUTH_SECRET or one
+#                                          of its rotation siblings)
+#         CERTCTL_AGENT_ID                (returned from POST /api/v1/agents)
+#     in deploy/.env or the shell environment. See deploy/.env.example.
+#
+# USAGE
+# -----
+#
+# Production-shaped (this base alone):
+#   docker compose -f deploy/docker-compose.yml up -d
+#
+# Bundled demo (zero-config, populated dashboard, demo-mode auth):
+#   docker compose -f deploy/docker-compose.yml \
+#                  -f deploy/docker-compose.demo.yml up -d
+#
+# The demo overlay (docker-compose.demo.yml) layers in the demo-mode env
+# vars (AUTH_TYPE=none + DEMO_MODE_ACK=true + KEYGEN_MODE=server +
+# DEMO_SEED=true + the change-me placeholder creds). It exists so the
+# `docker compose up` smoke + screenshot path stays one command — but it
+# ALSO carries the operator-visible warning banner the server emits at
+# boot when DEMO_MODE_ACK=true.
+#
+# Pre-Bundle-2 this base file WAS the demo path. The split happened in
+# 2026-05-12; the README quickstart, deploy/ENVIRONMENTS.md, and the
+# cold-DB compose smoke in .github/workflows/ci.yml were updated in the
+# same commit to point at the new layout.
 services:
   # HTTPS-Everywhere Phase 3 — self-signed TLS bootstrap (init container).
   # Generates a CN=certctl-server ECDSA-P256 (SHA-256 signature) cert with
@@ -82,7 +128,12 @@ services:
     environment:
       POSTGRES_DB: certctl
       POSTGRES_USER: certctl
-      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-certctl}
+      # Bundle 2 closure: no `:-certctl` fallback. Operators MUST set
+      # POSTGRES_PASSWORD in deploy/.env or the shell environment. The
+      # demo overlay (docker-compose.demo.yml) supplies a fixed weak
+      # default for screenshot/demo use; production deploys never
+      # depend on that fallback.
+      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
     ports:
       - "5432:5432"
     volumes:
@@ -123,16 +174,44 @@ services:
       # on the docker bridge network keeps sslmode=disable acceptable; for
       # external/managed Postgres operators MUST override CERTCTL_DATABASE_URL
       # with sslmode=verify-full and provide the CA bundle. See docs/database-tls.md.
-      CERTCTL_DATABASE_URL: ${CERTCTL_DATABASE_URL:-postgres://certctl:${POSTGRES_PASSWORD:-certctl}@postgres:5432/certctl?sslmode=disable}
+      CERTCTL_DATABASE_URL: ${CERTCTL_DATABASE_URL:-postgres://certctl:${POSTGRES_PASSWORD}@postgres:5432/certctl?sslmode=disable}
       CERTCTL_SERVER_HOST: 0.0.0.0
       CERTCTL_SERVER_PORT: 8443
       CERTCTL_SERVER_TLS_CERT_PATH: /etc/certctl/tls/server.crt
       CERTCTL_SERVER_TLS_KEY_PATH: /etc/certctl/tls/server.key
       CERTCTL_LOG_LEVEL: info
-      CERTCTL_AUTH_TYPE: none
+      # Bundle 2 closure (compose split). The base compose no longer
-      CERTCTL_KEYGEN_MODE: server  # Demo uses server-side keygen; production should use "agent"
+      # sets CERTCTL_AUTH_TYPE / CERTCTL_KEYGEN_MODE / DEMO_MODE_ACK /
-      CERTCTL_NETWORK_SCAN_ENABLED: "true"  # Enable network scan GUI with seeded demo targets
+      # DEMO_SEED — the code defaults take over (auth-type api-key,
-      CERTCTL_CONFIG_ENCRYPTION_KEY: ${CERTCTL_CONFIG_ENCRYPTION_KEY:-change-me-32-char-encryption-key}  # AES-256-GCM for dynamic issuer/target config
+      # keygen agent, demo-mode false, demo-seed false). The demo
+      # overlay (docker-compose.demo.yml) is what flips this baseline
+      # into the populated-dashboard demo path; without that overlay
+      # the server boots production-shaped and refuses to start unless
+      # the operator has supplied CERTCTL_AUTH_SECRET +
+      # CERTCTL_CONFIG_ENCRYPTION_KEY.
+      #
+      # Audit 2026-05-10 HIGH-12: when DEMO_MODE_ACK=true (set by the
+      # demo overlay) AND the listener binds to a non-loopback address,
+      # every request is served as the synthetic admin actor
+      # `actor-demo-anon`. The server emits a prominent boot-time WARN
+      # banner with a production-promotion checklist in that case.
+      CERTCTL_AUTH_SECRET: ${CERTCTL_AUTH_SECRET}
+      CERTCTL_NETWORK_SCAN_ENABLED: "true"  # Enable network scan GUI
+      CERTCTL_CONFIG_ENCRYPTION_KEY: ${CERTCTL_CONFIG_ENCRYPTION_KEY}  # AES-256-GCM for dynamic issuer/target config
+      # Bootstrap token interpolation surface (Auditable Codebase Bundle
+      # cold-DB smoke closure, 2026-05-12). Pre-fix, the `env-file +
+      # --force-recreate certctl-server` pattern documented in
+      # cowork/manual-testing-bundle-2.html (and used by the cold-DB
+      # smoke job in .github/workflows/ci.yml::cold-db-compose-smoke)
+      # set CERTCTL_BOOTSTRAP_TOKEN in compose's own interpolation
+      # environment but the container never received it because this
+      # block didn't reference the variable. Wiring it as an explicit
+      # interpolation (default empty) makes the documented manual flow
+      # actually work end-to-end. Empty value = bootstrap strategy
+      # disabled (server returns 410 Gone on POST /api/v1/auth/bootstrap),
+      # which is the safe default — only set the var when you intend to
+      # mint a day-0 admin via the bootstrap path.
+      CERTCTL_BOOTSTRAP_TOKEN: ${CERTCTL_BOOTSTRAP_TOKEN:-}
     ports:
       - "8443:8443"
     volumes:
@@ -182,7 +261,19 @@ services:
     environment:
       CERTCTL_SERVER_URL: https://certctl-server:8443
       CERTCTL_SERVER_CA_BUNDLE_PATH: /etc/certctl/tls/ca.crt
-      CERTCTL_API_KEY: ${CERTCTL_API_KEY:-change-me-in-production}
+      # Bundle 2 closure (compose split). No placeholder fallbacks.
+      # Operators MUST set CERTCTL_API_KEY (matching one of the server's
+      # CERTCTL_AUTH_SECRET rotation values) and CERTCTL_AGENT_ID
+      # (returned from `POST /api/v1/agents` during agent enrollment).
+      # Without an agent ID, cmd/agent/main.go fails fast at startup
+      # with "agent-id flag or CERTCTL_AGENT_ID env var is required" —
+      # the cold-DB compose smoke in .github/workflows/ci.yml tolerates
+      # the agent restart loop because the smoke targets server boot
+      # only. The demo overlay (docker-compose.demo.yml) supplies a
+      # pre-seeded agent-demo-1 row + matching env vars so the demo
+      # path stays one-command.
+      CERTCTL_API_KEY: ${CERTCTL_API_KEY}
+      CERTCTL_AGENT_ID: ${CERTCTL_AGENT_ID}
       CERTCTL_AGENT_NAME: docker-agent
       CERTCTL_LOG_LEVEL: info
       CERTCTL_DISCOVERY_DIRS: /var/lib/certctl/keys  # Agent scans this directory for existing certificates

deploy/helm/certctl/Chart.yaml

@@ -2,7 +2,15 @@ apiVersion: v2
 name: certctl
 description: Self-hosted certificate lifecycle management platform
 type: application
-version: 0.1.0
+# Bundle 3 closure (OPS-L1): bumped from 0.1.0 → 1.0.0. The pre-1.0
+# version implied "unstable chart, breaking changes on every minor"
+# which prospective enterprise operators read as "not ready for
+# production". The chart has been deployed against real clusters since
+# 2026-02 and shipped through 8 audit closures (M-018, U-1, U-2, U-3,
+# H-1, G-1, B1 connector validation, B2 first-run guards); 1.0.0
+# matches that maturity. The chart still adheres to semver going
+# forward — any breaking value-schema change bumps to 2.0.0.
+version: 1.0.0
 appVersion: "2.1.0"
 keywords:
   - certificate

deploy/helm/certctl/templates/_helpers.tpl

@@ -128,8 +128,27 @@ Bundle B / Audit M-018 (PCI-DSS Req 4 / CWE-319):
     postgresql.tls.mode without further translation.
 */}}
 {{- define "certctl.databaseURL" -}}
+{{- if .Values.postgresql.enabled -}}
 {{- $sslMode := default "disable" .Values.postgresql.tls.mode -}}
 postgres://{{ .Values.postgresql.auth.username }}:$(POSTGRES_PASSWORD)@{{ include "certctl.fullname" . }}-postgres:5432/{{ .Values.postgresql.auth.database }}?sslmode={{ $sslMode }}
+{{- else -}}
+{{- /*
+  Bundle 3 closure (D2 + OPS-L2): external-Postgres first-class path.
+  When postgresql.enabled=false, the chart NEVER renders the
+  bundled StatefulSet, postgres-secret, or postgres-service —
+  templates/postgres-*.yaml gate themselves on .Values.postgresql.enabled.
+  The connection string comes from externalDatabase.url (the canonical
+  form) or, for backward-compat with pre-Bundle-3 deploys, from
+  server.env.CERTCTL_DATABASE_URL (which overrides this helper at the
+  pod-spec level — see server-deployment.yaml).
+
+  externalDatabase.url is consumed VERBATIM by the server's
+  CERTCTL_DATABASE_URL env var. Operators are responsible for choosing
+  the right sslmode (`verify-full` recommended for managed Postgres
+  per PCI-DSS Req 4 §2.2.5; see docs/database-tls.md).
+*/ -}}
+{{- required "externalDatabase.url is required when postgresql.enabled=false" .Values.externalDatabase.url -}}
+{{- end -}}
 {{- end }}
 
 {{/*
@@ -180,11 +199,110 @@ per affected resource. No-op when configured correctly.
 {{- if and (not .Values.server.tls.existingSecret) (not .Values.server.tls.certManager.enabled) -}}
 {{- fail "\n\ncertctl refuses to start without TLS.\n\nSet EXACTLY ONE of:\n  --set server.tls.existingSecret=<your-kubernetes.io/tls-secret-name>\nOR\n  --set server.tls.certManager.enabled=true \\\n  --set server.tls.certManager.issuerRef.name=<your-issuer-or-clusterissuer>\n\nSee docs/tls.md for the full setup walkthrough, including bootstrap\nguidance for air-gapped clusters without cert-manager.\n" -}}
 {{- end -}}
+{{- if and .Values.server.tls.existingSecret .Values.server.tls.certManager.enabled -}}
+{{- /*
+  Bundle 3 closure (D7): pre-Bundle-3 the helper only rejected the
+  NEITHER-set case. Setting BOTH (`existingSecret` AND `certManager.enabled=true`)
+  produced two TLS sources of truth — the existing Secret got mounted but
+  cert-manager simultaneously provisioned a Certificate CR pointing at a
+  conflicting Secret. Operators ended up with a dangling cert-manager
+  Certificate or a wrong-source TLS bundle. The chart now refuses at
+  render-time so the misconfiguration cannot ship.
+*/ -}}
+{{- fail "\n\nserver.tls.existingSecret AND server.tls.certManager.enabled are BOTH set.\n\nThe chart requires EXACTLY ONE TLS ownership path (Bundle 3 closure / audit D7):\n  - existingSecret: operator owns the TLS Secret; cert-manager must NOT provision one.\n  - certManager.enabled: cert-manager owns the TLS Secret; existingSecret must be empty.\n\nUnset one of:\n  --set server.tls.existingSecret=\"\"          (let cert-manager own it)\nOR\n  --set server.tls.certManager.enabled=false   (let the existing Secret stand)\n\nSee docs/tls.md.\n" -}}
+{{- end -}}
 {{- if and .Values.server.tls.certManager.enabled (not .Values.server.tls.certManager.issuerRef.name) -}}
 {{- fail "\n\nserver.tls.certManager.enabled=true but server.tls.certManager.issuerRef.name is empty.\n\nSet:\n  --set server.tls.certManager.issuerRef.name=<your-issuer-or-clusterissuer>\n\nSee docs/tls.md.\n" -}}
 {{- end -}}
 {{- end }}
 
+{{/*
+Pod- vs container-scope security context split (Bundle 3 closure / audit D3).
+
+The Kubernetes API splits SecurityContext into two non-overlapping
+field sets, and silently DROPS fields that land at the wrong scope —
+which is exactly the audit D3 finding pre-Bundle-3.
+
+Pod-scope fields (applied via spec.securityContext):
+  runAsNonRoot, runAsUser, runAsGroup, fsGroup, fsGroupChangePolicy,
+  supplementalGroups, seLinuxOptions, seccompProfile, sysctls.
+
+Container-scope fields (applied via spec.containers[].securityContext):
+  readOnlyRootFilesystem, allowPrivilegeEscalation, capabilities,
+  privileged, procMount, runAsNonRoot/runAsUser/runAsGroup (override),
+  seLinuxOptions/seccompProfile (override).
+
+These helpers split a single operator-facing `securityContext` map
+into the two sub-maps so the chart renders each field at the scope
+where Kubernetes actually honors it. The split is conservative — a
+field that COULD live at either scope is rendered at pod scope only
+(no override at container scope) so behavior matches the pre-Bundle-3
+operator intent: pod-level setting is the source of truth.
+
+Operators don't need to change values.yaml; the existing
+`server.securityContext` and `agent.securityContext` blocks keep
+working byte-for-byte. The Helm template just routes each field to
+the correct YAML node now.
+*/}}
+{{- define "certctl.podSecurityContext" -}}
+{{- $sc := . -}}
+{{- $podKeys := list "runAsNonRoot" "runAsUser" "runAsGroup" "fsGroup" "fsGroupChangePolicy" "supplementalGroups" "seLinuxOptions" "seccompProfile" "sysctls" -}}
+{{- $out := dict -}}
+{{- range $k := $podKeys -}}
+{{- if hasKey $sc $k -}}
+{{- $_ := set $out $k (index $sc $k) -}}
+{{- end -}}
+{{- end -}}
+{{- toYaml $out -}}
+{{- end }}
+
+{{- define "certctl.containerSecurityContext" -}}
+{{- $sc := . -}}
+{{- $containerKeys := list "readOnlyRootFilesystem" "allowPrivilegeEscalation" "capabilities" "privileged" "procMount" -}}
+{{- $out := dict -}}
+{{- range $k := $containerKeys -}}
+{{- if hasKey $sc $k -}}
+{{- $_ := set $out $k (index $sc $k) -}}
+{{- end -}}
+{{- end -}}
+{{- toYaml $out -}}
+{{- end }}
+
+{{/*
+Required-secret gate (Bundle 3 closure / audit D1).
+
+Pre-Bundle-3 the chart accepted empty `server.auth.apiKey` and empty
+`postgresql.auth.password` and rendered Secrets with empty values; the
+certctl-server container then crash-looped at startup with the auth
+configuration error or with `pq: password authentication failed for
+user "certctl"`. Worse, an operator who forgot to set the api-key
+ended up with auth.type=api-key + empty CERTCTL_AUTH_SECRET in the
+Secret, which Validate() rejects at startup — but the diagnostic
+surfaces inside a CrashLoopBackOff, not at `helm install` time where
+it would be caught immediately.
+
+Post-Bundle-3 the chart fails at template time with operator-actionable
+guidance. The bundled-Postgres path (`postgresql.enabled=true`)
+requires `postgresql.auth.password`; the external-Postgres path
+(`postgresql.enabled=false`) skips that check because credentials are
+embedded in `externalDatabase.url` instead.
+
+Any template that depends on either secret value should call
+`{{ include "certctl.requiredSecrets" . }}` at the top so this guard
+runs once per affected resource. No-op when configured correctly.
+*/}}
+{{- define "certctl.requiredSecrets" -}}
+{{- if and (eq .Values.server.auth.type "api-key") (not .Values.server.auth.apiKey) -}}
+{{- fail "\n\nserver.auth.type=\"api-key\" but server.auth.apiKey is empty.\n\nSet:\n  --set server.auth.apiKey=$(openssl rand -base64 32)\n\nor put the value in a values override. The certctl-server container\nrefuses to start without an API key when auth.type=api-key.\n\nFor demo deploys without authentication, use:\n  --set server.auth.type=none\n(only safe behind an authenticating gateway — see docs/operator/security.md).\n" -}}
+{{- end -}}
+{{- if and .Values.postgresql.enabled (not .Values.postgresql.auth.password) -}}
+{{- fail "\n\npostgresql.enabled=true but postgresql.auth.password is empty.\n\nSet:\n  --set postgresql.auth.password=$(openssl rand -base64 32)\n\nor put the value in a values override. The bundled Postgres\nStatefulSet refuses to bootstrap initdb without POSTGRES_PASSWORD.\n\nFor external Postgres deployments, set:\n  --set postgresql.enabled=false\n  --set externalDatabase.url=postgres://user:pass@host:5432/db?sslmode=require\nSee deploy/helm/examples/values-external-db.yaml.\n" -}}
+{{- end -}}
+{{- if and (not .Values.postgresql.enabled) (not .Values.externalDatabase.url) (not .Values.server.env.CERTCTL_DATABASE_URL) -}}
+{{- fail "\n\npostgresql.enabled=false but no external database URL is configured.\n\nSet ONE of:\n  --set externalDatabase.url=postgres://user:pass@host:5432/db?sslmode=require\nOR (legacy)\n  --set server.env.CERTCTL_DATABASE_URL=postgres://user:pass@host:5432/db?sslmode=require\n\nSee deploy/helm/examples/values-external-db.yaml.\n" -}}
+{{- end -}}
+{{- end }}
+
 {{/*
 Auth-type validation gate.
 
@@ -202,8 +320,8 @@ Any template that consumes .Values.server.auth.type should call
 runs once per affected resource. No-op when configured correctly.
 */}}
 {{- define "certctl.validateAuthType" -}}
-{{- $valid := list "api-key" "none" -}}
+{{- $valid := list "api-key" "none" "oidc" -}}
 {{- if not (has .Values.server.auth.type $valid) -}}
-{{- fail (printf "\n\nserver.auth.type=%q is not supported (valid: %v).\n\nFor JWT/OIDC, run an authenticating gateway in front of certctl\n(oauth2-proxy / Envoy ext_authz / Traefik ForwardAuth / Pomerium) and\nset server.auth.type=none here so the gateway terminates federated\nidentity. See docs/architecture.md \"Authenticating-gateway pattern\"\nand docs/upgrade-to-v2-jwt-removal.md for the migration walkthrough.\n\nG-1 audit closure: pre-G-1 the chart accepted type=jwt and the binary\nsilently downgraded to api-key middleware. The chart now fails at\ntemplate time so misconfigured deployments cannot ship.\n" .Values.server.auth.type $valid) -}}
+{{- fail (printf "\n\nserver.auth.type=%q is not supported (valid: %v).\n\nFor JWT/SAML/LDAP, run an authenticating gateway in front of certctl\n(oauth2-proxy / Envoy ext_authz / Traefik ForwardAuth / Pomerium) and\nset server.auth.type=none here so the gateway terminates federated\nidentity. See docs/architecture.md \"Authenticating-gateway pattern\"\nand docs/upgrade-to-v2-jwt-removal.md for the migration walkthrough.\n\nG-1 audit closure: pre-G-1 the chart accepted type=jwt and the binary\nsilently downgraded to api-key middleware. The chart now fails at\ntemplate time so misconfigured deployments cannot ship.\n\nAuth Bundle 2 Phase 0: server.auth.type=oidc is in the valid set but\nthe OIDC handler chain ships in later Bundle 2 phases. Pre-Bundle-2\noperators who set type=oidc see the certctl-server container exit at\nstartup with an actionable error — chart-time validation no longer\nblocks deploy because the binary's runtime guard takes over. Once\nBundle 2 lands, the runtime guard relaxes and OIDC works end-to-end.\n" .Values.server.auth.type $valid) -}}
 {{- end -}}
 {{- end }}

deploy/helm/certctl/templates/agent-daemonset.yaml

@@ -19,7 +19,7 @@ spec:
     spec:
       serviceAccountName: {{ include "certctl.serviceAccountName" . }}
       securityContext:
-        {{- toYaml .Values.agent.securityContext | nindent 8 }}
+        {{- include "certctl.podSecurityContext" .Values.agent.securityContext | nindent 8 }}
       {{- with .Values.imagePullSecrets }}
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
@@ -40,6 +40,8 @@ spec:
         - name: agent
           image: {{ include "certctl.agentImage" . }}
           imagePullPolicy: {{ .Values.agent.image.pullPolicy }}
+          securityContext:
+            {{- include "certctl.containerSecurityContext" .Values.agent.securityContext | nindent 12 }}
           env:
             - name: CERTCTL_SERVER_URL
               value: {{ include "certctl.serverURL" . }}
@@ -106,7 +108,7 @@ spec:
     spec:
       serviceAccountName: {{ include "certctl.serviceAccountName" . }}
       securityContext:
-        {{- toYaml .Values.agent.securityContext | nindent 8 }}
+        {{- include "certctl.podSecurityContext" .Values.agent.securityContext | nindent 8 }}
       {{- with .Values.imagePullSecrets }}
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
@@ -127,6 +129,8 @@ spec:
         - name: agent
           image: {{ include "certctl.agentImage" . }}
           imagePullPolicy: {{ .Values.agent.image.pullPolicy }}
+          securityContext:
+            {{- include "certctl.containerSecurityContext" .Values.agent.securityContext | nindent 12 }}
           env:
             - name: CERTCTL_SERVER_URL
               value: {{ include "certctl.serverURL" . }}

deploy/helm/certctl/templates/networkpolicy.yaml

@@ -0,0 +1,75 @@
+{{- /*
+Bundle 3 closure (D11): NetworkPolicy for the server Deployment.
+
+Pre-Bundle-3 the chart had no NetworkPolicy template at all — the
+audit-D11 "documented placeholder" finding referred to docs claiming
+deny-by-default network isolation that the rendered chart did not
+provide. Closed.
+
+This template emits a single NetworkPolicy that, when enabled,
+restricts the certctl-server Pod to:
+  - Ingress  : from any agent Pod in the same namespace (selector
+               match on app.kubernetes.io/component=agent) on the
+               server port, plus optional operator-supplied
+               additional from clauses (.networkPolicy.extraIngress).
+  - Egress   : to the postgres Pod (when postgresql.enabled=true),
+               53/UDP+TCP for kube-dns, and operator-supplied
+               additional to clauses for outbound CA / OIDC / SMTP
+               (.networkPolicy.extraEgress).
+
+Default off so existing deploys don't suddenly lose network reach.
+Operators opt in once they've mapped their actual egress surface.
+*/ -}}
+{{- if .Values.networkPolicy.enabled }}
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: {{ include "certctl.fullname" . }}-server
+  labels:
+    {{- include "certctl.labels" . | nindent 4 }}
+    app.kubernetes.io/component: server
+spec:
+  podSelector:
+    matchLabels:
+      {{- include "certctl.serverSelectorLabels" . | nindent 6 }}
+  policyTypes:
+    - Ingress
+    - Egress
+  ingress:
+    # Allow in-cluster agent Pods to reach the server's HTTPS port.
+    - from:
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: {{ include "certctl.name" . }}
+              app.kubernetes.io/component: agent
+      ports:
+        - protocol: TCP
+          port: {{ .Values.server.port }}
+    {{- with .Values.networkPolicy.extraIngress }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
+  egress:
+    # Kube-DNS (53/UDP + 53/TCP). Required for any in-cluster name
+    # resolution (postgres-service, OIDC issuer hostnames, ACME).
+    - to:
+        - namespaceSelector: {}
+      ports:
+        - protocol: UDP
+          port: 53
+        - protocol: TCP
+          port: 53
+    {{- if .Values.postgresql.enabled }}
+    # Bundled-Postgres egress.
+    - to:
+        - podSelector:
+            matchLabels:
+              app.kubernetes.io/name: {{ include "certctl.name" . }}
+              app.kubernetes.io/component: postgres
+      ports:
+        - protocol: TCP
+          port: 5432
+    {{- end }}
+    {{- with .Values.networkPolicy.extraEgress }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
+{{- end }}

deploy/helm/certctl/templates/pdb.yaml

@@ -0,0 +1,31 @@
+{{- /*
+Bundle 3 closure (D11): PodDisruptionBudget for the server Deployment.
+
+Pre-Bundle-3 values.yaml carried `podDisruptionBudget.enabled` +
+`minAvailable` + `maxUnavailable` knobs but no template consumed
+them. Audit D11 closed.
+
+The PDB only renders when server.replicas > 1 — a single-replica
+deployment can't satisfy minAvailable=1 during voluntary disruption
+anyway (the K8s scheduler would refuse to drain the node). Operators
+running 2+ replicas get the PDB; operators running a single replica
+get a templated-out NOTES line reminding them to bump replicas first.
+*/ -}}
+{{- if and .Values.podDisruptionBudget.enabled (gt (int .Values.server.replicas) 1) }}
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: {{ include "certctl.fullname" . }}-server
+  labels:
+    {{- include "certctl.labels" . | nindent 4 }}
+    app.kubernetes.io/component: server
+spec:
+  selector:
+    matchLabels:
+      {{- include "certctl.serverSelectorLabels" . | nindent 6 }}
+  {{- if .Values.podDisruptionBudget.minAvailable }}
+  minAvailable: {{ .Values.podDisruptionBudget.minAvailable }}
+  {{- else if .Values.podDisruptionBudget.maxUnavailable }}
+  maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable }}
+  {{- end }}
+{{- end }}

deploy/helm/certctl/templates/postgres-secret.yaml

@@ -1,3 +1,14 @@
+{{- if .Values.postgresql.enabled }}
+{{- /*
+  Bundle 3 closure (D1 + D2): the bundled-Postgres Secret only renders
+  when postgresql.enabled=true. Pre-Bundle-3 this template rendered
+  unconditionally with `password: "changeme"` as the fallback default —
+  which is exactly what the change-me-... cluster of audit findings
+  was about (a deployment that uses the rendered chart with default
+  values ships a known weak password). The Bundle-3 helper at
+  certctl.requiredSecrets fail-closes empty password at template time
+  before this template ever runs.
+*/ -}}
 apiVersion: v1
 kind: Secret
 metadata:
@@ -7,6 +18,7 @@ metadata:
     app.kubernetes.io/component: postgres
 type: Opaque
 stringData:
-  password: {{ .Values.postgresql.auth.password | default "changeme" | quote }}
+  password: {{ required "postgresql.auth.password is required when postgresql.enabled=true (Bundle 3: no fallback default)" .Values.postgresql.auth.password | quote }}
   username: {{ .Values.postgresql.auth.username | quote }}
   database: {{ .Values.postgresql.auth.database | quote }}
+{{- end }}

deploy/helm/certctl/templates/server-deployment.yaml

@@ -1,5 +1,6 @@
 {{- include "certctl.tls.required" . }}
 {{- include "certctl.validateAuthType" . }}
+{{- include "certctl.requiredSecrets" . }}
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -23,8 +24,13 @@ spec:
         checksum/secret: {{ include (print $.Template.BasePath "/server-secret.yaml") . | sha256sum }}
     spec:
       serviceAccountName: {{ include "certctl.serviceAccountName" . }}
+      # Bundle 3 closure (D3): pod-level fields only. The container-only
+      # fields (readOnlyRootFilesystem, allowPrivilegeEscalation,
+      # capabilities, privileged) render at container scope below —
+      # pre-Bundle-3 they all sat here at pod scope and the K8s API
+      # silently dropped them.
       securityContext:
-        {{- toYaml .Values.server.securityContext | nindent 8 }}
+        {{- include "certctl.podSecurityContext" .Values.server.securityContext | nindent 8 }}
       {{- with .Values.imagePullSecrets }}
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
@@ -33,6 +39,13 @@ spec:
         - name: server
           image: {{ include "certctl.serverImage" . }}
           imagePullPolicy: {{ .Values.server.image.pullPolicy }}
+          # Bundle 3 closure (D3): container-scope security hardening.
+          # readOnlyRootFilesystem + allowPrivilegeEscalation +
+          # capabilities are container-only fields per the K8s API; the
+          # helper splits them out of the operator-facing
+          # server.securityContext map so existing values keep working.
+          securityContext:
+            {{- include "certctl.containerSecurityContext" .Values.server.securityContext | nindent 12 }}
           ports:
             - name: https
               containerPort: {{ .Values.server.port }}
@@ -51,11 +64,16 @@ spec:
                 secretKeyRef:
                   name: {{ include "certctl.fullname" . }}-server
                   key: database-url
+            # Bundle 3 closure (D2): POSTGRES_PASSWORD is only needed
+            # for the bundled-Postgres mode. External Postgres mode
+            # embeds the password directly in externalDatabase.url.
+            {{- if .Values.postgresql.enabled }}
             - name: POSTGRES_PASSWORD
               valueFrom:
                 secretKeyRef:
                   name: {{ include "certctl.fullname" . }}-postgres
                   key: password
+            {{- end }}
             - name: CERTCTL_LOG_LEVEL
               valueFrom:
                 configMapKeyRef:

deploy/helm/certctl/templates/servicemonitor.yaml

@@ -0,0 +1,63 @@
+{{- /*
+Bundle 3 closure (D5 + OPS-M1 docs): Prometheus Operator ServiceMonitor.
+
+Pre-Bundle-3 the chart had `monitoring.serviceMonitor.enabled` in
+values.yaml but no template consumed it — toggling it on rendered
+nothing. Audit D5 closed.
+
+The endpoint scrapes /api/v1/metrics/prometheus which the certctl
+server already exposes in Prometheus exposition format (see
+internal/api/handler/metrics.go::GetPrometheusMetrics). Note: the
+endpoint is rbac-gated on `metrics.read`, so the ServiceMonitor needs
+a bearer token. Operators with Prometheus Operator MUST set
+`monitoring.serviceMonitor.bearerTokenSecret` pointing at a Secret
+that holds an API key with the `metrics.read` permission. Without
+that, scrapes return 401.
+
+OPS-M1 caveat: the current /metrics/prometheus handler is a hand-rolled
+exposition-format emitter, not prometheus/client_golang-instrumented
+code. Histograms, exemplars, and target labels are limited to what the
+handler computes statically. Migration to client_golang tracked in
+WORKSPACE-ROADMAP.md.
+*/ -}}
+{{- if and .Values.monitoring.enabled .Values.monitoring.serviceMonitor.enabled }}
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  name: {{ include "certctl.fullname" . }}-server
+  labels:
+    {{- include "certctl.labels" . | nindent 4 }}
+    app.kubernetes.io/component: server
+    {{- with .Values.monitoring.serviceMonitor.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
+spec:
+  selector:
+    matchLabels:
+      {{- include "certctl.serverSelectorLabels" . | nindent 6 }}
+  endpoints:
+    - port: https
+      scheme: https
+      path: /api/v1/metrics/prometheus
+      interval: {{ .Values.monitoring.serviceMonitor.interval | default "30s" }}
+      scrapeTimeout: {{ .Values.monitoring.serviceMonitor.scrapeTimeout | default "10s" }}
+      tlsConfig:
+        # The certctl server uses self-signed bootstrap TLS or operator-
+        # provided cert-manager TLS — the ServiceMonitor consumes the
+        # same CA bundle the server presents. When server.tls.existingSecret
+        # is set, operators usually want to pull the matching ca.crt key
+        # out of that Secret. Adjust if your CA chain lives elsewhere.
+        {{- if .Values.monitoring.serviceMonitor.tlsConfig }}
+        {{- toYaml .Values.monitoring.serviceMonitor.tlsConfig | nindent 8 }}
+        {{- else }}
+        insecureSkipVerify: true
+        {{- end }}
+      {{- with .Values.monitoring.serviceMonitor.bearerTokenSecret }}
+      bearerTokenSecret:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.monitoring.serviceMonitor.relabelings }}
+      relabelings:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+{{- end }}

deploy/helm/certctl/values.yaml

@@ -15,7 +15,10 @@ fullnameOverride: ""
 # Certctl Server Configuration
 # ==============================================================================
 server:
-  # Number of replicas (for HA deployments)
+  # Number of replicas (for HA deployments).
+  # Phase 2 DEPL-H1: production HA is operator-opt-in across this field
+  # + podDisruptionBudget.enabled + server.service.sessionAffinity.
+  # See docs/operator/runbooks/ha.md for the smallest-possible HA overlay.
   replicas: 1
 
   # Image configuration
@@ -272,6 +275,34 @@ server:
   #   secret:
   #     secretName: ca-cert
 
+# ==============================================================================
+# External Database Configuration (Bundle 3 closure / D2 + OPS-L2)
+# ==============================================================================
+# When postgresql.enabled=false, the chart skips the bundled StatefulSet +
+# Secret + Service and instead consumes the URL below verbatim as the
+# server's CERTCTL_DATABASE_URL. The URL embeds username, password,
+# host, port, database, and sslmode — operators are responsible for
+# rotating credentials in this string out-of-band (Kubernetes Secret +
+# helm upgrade is the supported pattern).
+#
+# Recommended sslmode for managed Postgres (RDS, Cloud SQL, Azure DB):
+#   verify-full  — PCI-DSS Req 4 v4.0 §2.2.5 compliant; requires CA bundle.
+#                  Mount the CA via server.volumes / server.volumeMounts and
+#                  set sslrootcert=/path/in/pod/ca.crt in the URL.
+#
+# Example values overrides:
+#   postgresql.enabled: false
+#   externalDatabase.url: "postgres://certctl:HUNTER2@db.example.com:5432/certctl?sslmode=verify-full"
+#
+# Migration from the legacy `server.env.CERTCTL_DATABASE_URL` workaround:
+# both still work (env block overrides the helper-emitted Secret value at
+# pod-spec level), but the new path renders cleaner manifests with no
+# stranded postgres-* templates.
+externalDatabase:
+  # Connection string used when postgresql.enabled=false.
+  # Required in that mode — see certctl.requiredSecrets helper.
+  url: ""
+
 # ==============================================================================
 # PostgreSQL Configuration
 # ==============================================================================
@@ -510,14 +541,34 @@ rbac:
   create: true
 
 # ==============================================================================
-# Kubernetes Secrets Target Connector
+# Kubernetes Secrets Target Connector (PREVIEW — Bundle 3 closure / C3)
 # ==============================================================================
+# Bundle 3 audit closure (C3): the connector framework at
+# internal/connector/target/k8ssecret/ ships the Config + interface +
+# 14 unit tests, but the production K8s client at
+# k8ssecret.go::realK8sClient is documented as "a stub placeholder for
+# the real k8s.io/client-go implementation". The repo does not import
+# k8s.io/client-go (verified via `grep -n "client-go" go.mod`), so the
+# connector cannot deploy to a real cluster today.
+#
+# Setting kubernetesSecrets.enabled=true wires up the RBAC verbs the
+# real client will need (get/create/update/patch/delete on Secrets)
+# without making the connector functional — operators trying to use it
+# get the stub's error and a pointer to this note.
+#
+# Status: PREVIEW. Production client lands when the cluster-management
+# bundle ships (tracked in WORKSPACE-ROADMAP.md). Until then,
+# in-cluster deploys use the file-based connectors (NGINX, Apache,
+# HAProxy, etc.) via a Pod-mounted Secret + DaemonSet agent.
 kubernetesSecrets:
-  # Enable RBAC rules for managing TLS Secrets
   enabled: false
 
 # ==============================================================================
-# Pod Disruption Budget (for HA deployments)
+# Pod Disruption Budget (for HA deployments).
+# Phase 2 DEPL-H1: defaults to enabled=false because a PDB template
+# rendered at `replicas: 1` blocks every rolling restart on a
+# single-node cluster. Production HA flips this to true alongside
+# server.replicas ≥ 2. See docs/operator/runbooks/ha.md.
 # ==============================================================================
 podDisruptionBudget:
   enabled: false
@@ -527,6 +578,13 @@ podDisruptionBudget:
 # ==============================================================================
 # Monitoring Configuration
 # ==============================================================================
+# Bundle 3 closure (D5): the ServiceMonitor template at
+# templates/servicemonitor.yaml renders when both monitoring.enabled=true
+# AND monitoring.serviceMonitor.enabled=true. The endpoint scrapes
+# /api/v1/metrics/prometheus, which is rbac-gated on `metrics.read` —
+# operators MUST provide a bearer token via
+# monitoring.serviceMonitor.bearerTokenSecret pointing at a Secret with
+# an API key holding that permission. Without the token, scrapes 401.
 monitoring:
   enabled: false
   # Prometheus ServiceMonitor
@@ -534,8 +592,53 @@ monitoring:
     enabled: false
     interval: 30s
     scrapeTimeout: 10s
+    # Additional labels applied to the ServiceMonitor metadata.
     # labels: {}
-    # selector: {}
+    # Bearer-token Secret reference (required when the certctl server's
+    # /api/v1/metrics/prometheus endpoint is gated by api-key auth).
+    # Example:
+    #   bearerTokenSecret:
+    #     name: certctl-prometheus-key
+    #     key: api-key
+    # bearerTokenSecret: {}
+    # TLS config for the scrape endpoint. The certctl server presents
+    # the same TLS cert the rest of the chart uses; insecureSkipVerify
+    # defaults to true so demos work out of the box. Production deploys
+    # should pin the CA via caFile or ca.secret.
+    # tlsConfig:
+    #   caFile: /etc/prometheus/secrets/certctl-ca/ca.crt
+    #   serverName: certctl-server
+    # tlsConfig: {}
+    # Optional relabeling for the scrape job.
+    # relabelings: []
+
+# ==============================================================================
+# Network Policy (Bundle 3 closure / D11)
+# ==============================================================================
+# Default off so existing deploys don't suddenly lose network reach.
+# When enabled, restricts the server pod to:
+#   - Ingress: from in-namespace agent pods only.
+#   - Egress: kube-dns + bundled Postgres (if enabled).
+# Operators add CA / OIDC / SMTP egress via extraEgress.
+networkPolicy:
+  enabled: false
+  # Additional Ingress rules merged into the policy. Each entry is a
+  # raw networking.k8s.io/v1 NetworkPolicyIngressRule.
+  extraIngress: []
+  # Additional Egress rules merged into the policy. Common operator
+  # need: 443/TCP to an OIDC issuer, 443/TCP to a public CA endpoint,
+  # 25/TCP to an SMTP relay.
+  # Example:
+  # extraEgress:
+  #   - to:
+  #       - ipBlock:
+  #           cidr: 0.0.0.0/0
+  #           except:
+  #             - 10.0.0.0/8
+  #     ports:
+  #       - protocol: TCP
+  #         port: 443
+  extraEgress: []
 
 # ==============================================================================
 # Advanced Configuration

deploy/test/f5-mock-icontrol/Dockerfile

@@ -6,8 +6,8 @@
 # Per H-001 guard: every FROM is digest-pinned. Operator re-pins
 # quarterly per docs/deployment-vendor-matrix.md.
 
-# golang:1.25.9-bookworm digest pinned per H-001.
+# golang:1.25.10-bookworm digest pinned per H-001.
-FROM golang:1.25.9-bookworm@sha256:1a1408bf8d2d3077f9508880caf0e8bb0fde195fe3c890e7ea480dfb66dc7827 AS builder
+FROM golang:1.25.10-bookworm@sha256:e3a54b77385b4f8a31c1db4d12429ffb3718ea76865731a787c497755d409547 AS builder
 WORKDIR /src
 COPY deploy/test/f5-mock-icontrol/ ./
 RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags "-s -w" -o /out/f5-mock-icontrol .

deploy/test/f5-mock-icontrol/go.mod

@@ -1,3 +1,3 @@
 module github.com/certctl-io/certctl/deploy/test/f5-mock-icontrol
 
-go 1.25.9
+go 1.25.10

docs/README.md

@@ -1,6 +1,6 @@
 # certctl Documentation
 
-> Last reviewed: 2026-05-05
+> Last reviewed: 2026-05-12
 
 The full docs index, organized by audience. Pick the section that matches what you need to do; each link below opens a focused doc rather than a wall of text.
 
@@ -27,12 +27,14 @@ You're operating certctl in production or building integrations and need authori
 | Doc | What it covers |
 |---|---|
 | [Architecture](reference/architecture.md) | System design, data flow, security model, deployment topologies |
+| [Profiles](reference/profiles.md) | CertificateProfile policy object — issuer wiring, EKUs, RequiresApproval gate (with profile-edit closure) |
 | [API](reference/api.md) | OpenAPI 3.1 spec, integration patterns, client SDK generation |
 | [CLI](reference/cli.md) | certctl-cli command reference and CI/CD integration patterns |
 | [Configuration](reference/configuration.md) | `CERTCTL_*` environment variable reference (scheduler, rate limits, deploy verify, audit, agent) |
 | [MCP server](reference/mcp.md) | Model Context Protocol integration for AI assistants |
 | [Release verification](reference/release-verification.md) | Cosign / SLSA / SBOM verification procedure |
 | [Intermediate CA hierarchy](reference/intermediate-ca-hierarchy.md) | Multi-level CA tree management — RFC 5280 §3.2/§4.2.1.9/§4.2.1.10 enforcement |
+| [Auth standards implemented](reference/auth-standards-implemented.md) | RFC + CWE evidence for the API-key + RBAC + OIDC + sessions + break-glass surface (NOT a compliance-mapping doc) |
 | [Deployment model](reference/deployment-model.md) | Atomic write, post-deploy verify, rollback semantics across all targets |
 | [Vendor matrix](reference/vendor-matrix.md) | Tested vendor versions per target connector |
 
@@ -62,12 +64,18 @@ You're running certctl in production and need operational guidance.
 
 | Doc | What it covers |
 |---|---|
-| [Security posture](operator/security.md) | Auth, rate limits, encryption at rest, key rotation |
+| [Security posture](operator/security.md) | Auth, rate limits, encryption at rest, key rotation, RBAC + OIDC + sessions + break-glass, bootstrap |
+| [Secret custody](operator/secret-custody.md) | Where private keys live; FileDriver vs HSM/KMS; encryption wire format; env-seeded vs DB-seeded plaintext policy |
+| [Observability](operator/observability.md) | Metrics surface, Prometheus exposition vs client_golang, tracing scope, log structure, rate-limit semantics across restarts/replicas |
+| [RBAC operator reference](operator/rbac.md) | Roles, permissions, scopes, scope-down + day-0 bootstrap |
+| [Auth threat model](operator/auth-threat-model.md) | API-key + RBAC + OIDC + sessions + break-glass — token forgery, session hijacking, IdP compromise, role-grant abuse, bootstrap-token leak, audit-mutation |
+| [OIDC / SSO runbooks](operator/oidc-runbooks/index.md) | Per-IdP setup guides — Keycloak, Authentik, Okta, Auth0, Entra ID, Google Workspace |
 | [Control plane TLS](operator/tls.md) | Self-signed bootstrap, operator-supplied Secret, cert-manager Certificate CR |
 | [Database TLS](operator/database-tls.md) | PostgreSQL transport encryption |
-| [Approval workflow](operator/approval-workflow.md) | Two-person integrity gate for high-stakes issuance |
+| [Approval workflow](operator/approval-workflow.md) | Two-person integrity gate for high-stakes issuance + profile-edit closure |
 | [Helm deployment](operator/helm-deployment.md) | Kubernetes installation via the bundled chart |
 | [Performance baselines](operator/performance-baselines.md) | Operator-runnable benchmarks for regression spot checks |
+| [Auth benchmarks](operator/auth-benchmarks.md) | Session + OIDC validation p99 targets and measured baselines |
 | [Legacy clients (TLS 1.2)](operator/legacy-clients-tls-1.2.md) | Reverse-proxy runbook for embedded EST/SCEP clients on TLS 1.2 |
 
 ### Runbooks
@@ -77,6 +85,8 @@ You're running certctl in production and need operational guidance.
 | [Cloud targets](operator/runbooks/cloud-targets.md) | AWS ACM + Azure Key Vault deployment, debugging, rollback |
 | [Expiry alerts](operator/runbooks/expiry-alerts.md) | Per-policy multi-channel routing matrix, severity tiers |
 | [Disaster recovery](operator/runbooks/disaster-recovery.md) | CRL cache, OCSP responder cert, CA private-key rotation, Postgres restore |
+| [Config-encryption upgrade](operator/runbooks/config-encryption-upgrade.md) | Force v1/v2 → v3 re-seal across the database; passphrase rotation procedure |
+| [PostgreSQL backup](operator/runbooks/postgres-backup.md) | Operator-run backup recipe (docker-compose + Kubernetes); recommended cadence; quarterly DR dry-run |
 
 ## Migration
 
@@ -90,6 +100,8 @@ You're moving from another cert-management tool to certctl, or running both in p
 | Caddy ACME (point Caddy at certctl) | [migration/acme-from-caddy.md](migration/acme-from-caddy.md) |
 | cert-manager ACME (point cert-manager at certctl) | [migration/acme-from-cert-manager.md](migration/acme-from-cert-manager.md) |
 | Traefik ACME (point Traefik at certctl) | [migration/acme-from-traefik.md](migration/acme-from-traefik.md) |
+| **API keys → RBAC (v2.0.x → v2.1.0)** | [migration/api-keys-to-rbac.md](migration/api-keys-to-rbac.md) — **AUDIT YOUR API KEYS** post-upgrade |
+| **Enable OIDC SSO** | [migration/oidc-enable.md](migration/oidc-enable.md) — step-by-step OIDC onboarding for an existing API-key + RBAC deployment |
 
 ## Contributor
 
@@ -104,6 +116,7 @@ You're contributing to certctl, running tests locally, or trying to understand t
 | [GUI QA checklist](contributor/gui-qa-checklist.md) | Manual GUI verification pass for release |
 | [Release sign-off](contributor/release-sign-off.md) | Release-day checklist — code state, automated gates, manual QA, artefact verification |
 | [CI pipeline](contributor/ci-pipeline.md) | CI shape, regression guards, adding new checks |
+| [CI guards](contributor/ci-guards.md) | Per-class CI guards (code-shape, contract-parity, build/dep, operational); how to add one |
 
 ## Archive
 

docs/contributor/ci-pipeline.md

@@ -1,232 +0,0 @@
-# CI Pipeline — Operator Guide
-
-> Last reviewed: 2026-05-05
-
-> Authoritative guide to certctl's CI pipeline shape.
-> Per the ci-pipeline-cleanup spec, Phase 12.
-
-## Trigger model
-
-Three triggers, each with its own scope. Don't mix.
-
-| Trigger | Workflow | Scope | Wall-clock target |
-|---|---|---|---|
-| Push to master, PR to master | `.github/workflows/ci.yml` + `.github/workflows/codeql.yml` | Blocking — every check earns its keep | <10 min |
-| Daily 06:00 UTC + `workflow_dispatch` | `.github/workflows/security-deep-scan.yml` | Slow scans (gosec, osv, trivy, ZAP, schemathesis, nuclei, testssl, semgrep, mutation, `-race -count=10`); best-effort, never blocks | 60 min budget |
-| Tag push (`v*`) | `.github/workflows/release.yml` | Cross-platform binaries, ghcr.io push, SLSA provenance, GitHub release | n/a |
-
-This guide covers the **on-push pipeline** only.
-
-## On-push pipeline (7 status checks)
-
-```mermaid
-flowchart TD
-    Push["push to master"]
-    CI["CI workflow (5 jobs)"]
-    CodeQL["CodeQL workflow (2 jobs)"]
-    GoBuild["go-build-and-test<br/>~6-7 min"]
-    Frontend["frontend-build<br/>~1 min"]
-    HelmLint["helm-lint<br/>~10 sec"]
-    Vendor["deploy-vendor-e2e<br/>~5 min, depends on go-build-and-test"]
-    Image["image-and-supply-chain<br/>~3 min, parallel"]
-    AnalyzeGo["Analyze (go)<br/>~5 min, parallel"]
-    AnalyzeJS["Analyze (javascript-typescript)<br/>~5 min, parallel"]
-    Push --> CI
-    Push --> CodeQL
-    CI --> GoBuild
-    CI --> Frontend
-    CI --> HelmLint
-    CI --> Vendor
-    CI --> Image
-    CodeQL --> AnalyzeGo
-    CodeQL --> AnalyzeJS
-    GoBuild -.depends on.-> Vendor
-```
-
-End-to-end wall-clock: dominated by `go-build-and-test` + `deploy-vendor-e2e` chain (~12 min) running in parallel with CodeQL (~5 min). Target ~10 min.
-
-## Per-job deep-dive
-
-### `go-build-and-test` (Ubuntu, ~6-7 min)
-
-Runs the Go build/test suite + 18 of 20 regression guards.
-
-Steps:
-1. `actions/checkout@v4`
-2. `actions/setup-go@v5` (Go 1.25.9)
-3. `go build ./cmd/...` (server, agent, mcp-server, cli)
-4. **gofmt drift** — `gofmt -l .` must be empty (Makefile::verify parity)
-5. **go mod tidy drift** — `go mod tidy && git diff --exit-code go.mod go.sum`
-6. `go vet ./...`
-7. Install + run **golangci-lint** v2.11.4 (`--timeout 5m`)
-8. Install + run **govulncheck** (hard gate)
-9. Install + run **staticcheck** (hard gate; `continue-on-error: false`)
-10. **Race Detection** — `go test -race -count=1 ./internal/...` (9-package list, 5min timeout)
-11. **Go Test with Coverage** — full coverage profile to `coverage.out`
-12. **Check Coverage Thresholds** — `bash scripts/check-coverage-thresholds.sh` (reads `.github/coverage-thresholds.yml`)
-13. **Upload Coverage Report** — artifact (`go-coverage`, 30-day retention)
-14. **Coverage PR comment** — posts/updates per-PR coverage table (PR builds only)
-15. **Regression guards** — loop runs all `scripts/ci-guards/*.sh` (18 of 20 guards)
-
-Local equivalent: `make verify` covers steps 4, 6, 7, 11 (with `-short`).
-
-### `frontend-build` (Ubuntu, ~1 min)
-
-Vitest tests + tsc check + vite build + 2 of 20 regression guards (already covered by the ci-guards loop in `go-build-and-test`).
-
-Steps:
-1. `actions/checkout@v4`
-2. `actions/setup-node@v4` (Node 22)
-3. `npm ci`
-4. `npx tsc --noEmit`
-5. `npx vitest run`
-6. `npx vite build`
-7. **Regression guards** — same `scripts/ci-guards/*.sh` loop as `go-build-and-test` (catches frontend-side guards: S-1, P-1, T-1, L-015, L-019, M-009, G-3)
-
-### `helm-lint` (Ubuntu, ~10 sec)
-
-Helm chart validation in 3 modes + inverse fail-loud test:
-1. `helm lint` with existingSecret
-2. `helm template` (existingSecret mode)
-3. `helm template` (cert-manager mode)
-4. `helm template` (no TLS source — MUST fail per fail-loud guard)
-
-### `deploy-vendor-e2e` (Ubuntu, ~5 min, depends on `go-build-and-test`)
-
-Single-job collapse of the prior 12-job matrix (per ci-pipeline-cleanup Phase 5 / frozen decision 0.4 — revises Bundle II decision 0.9).
-
-Steps:
-1. `actions/checkout@v5`
-2. `actions/setup-go@v5` (Go 1.25.9, cache: true)
-3. **Build f5-mock-icontrol sidecar** — only sidecar without published image
-4. **Bring up all vendor sidecars** — `docker compose --profile deploy-e2e up -d` (11 sidecars)
-5. **Run all vendor-edge e2e** — `go test -tags integration -race -count=1 -run 'VendorEdge_'`; output captured to `test-output.log`
-6. **Skip-count enforcement** — `bash scripts/ci-guards/vendor-e2e-skip-check.sh test-output.log` (catches sidecar boot failures via skip-count vs allowlist)
-7. **Tear down sidecars** — `docker compose down -v` (always runs)
-
-The `deploy-vendor-e2e-windows` matrix was deleted entirely (per ci-pipeline-cleanup Phase 6 / frozen decision 0.5 — revises Bundle II decision 0.4). IIS + WinCertStore validation moved to [`docs/connector-iis.md::Operator validation playbook`](connector-iis.md#operator-validation-playbook-windows-host).
-
-### `image-and-supply-chain` (Ubuntu, ~3 min, parallel)
-
-Three checks bundled (per ci-pipeline-cleanup Phases 7-9 / frozen decision 0.8):
-1. **Digest validity** — `bash scripts/ci-guards/digest-validity.sh`. Resolves every `@sha256:<digest>` ref in `deploy/**/*.{yml,Dockerfile*}` against its registry. Closes the H-001 lying-field gap.
-2. **Docker build smoke** — builds all 4 Dockerfiles (`Dockerfile`, `Dockerfile.agent`, `deploy/test/f5-mock-icontrol/Dockerfile`, `deploy/test/libest/Dockerfile`).
-3. **OpenAPI ↔ handler operationId parity** — `bash scripts/ci-guards/openapi-handler-parity.sh`. Every router route must have a matching `operationId` in `api/openapi.yaml` or be documented in `api/openapi-handler-exceptions.yaml`.
-
-### CodeQL (Ubuntu × 2 languages, ~5 min)
-
-`.github/workflows/codeql.yml` — interprocedural taint tracking. Two matrix jobs: `go` and `javascript-typescript`. Triggers on push, PR, and weekly Sunday cron.
-
-## The 20 regression guards
-
-Located at `scripts/ci-guards/<id>.sh`. Each script is callable locally:
-
-```bash
-bash scripts/ci-guards/G-3-env-docs-drift.sh
-```
-
-Or run all of them:
-
-```bash
-for g in scripts/ci-guards/*.sh; do
-  echo "=== $(basename "$g") ==="
-  bash "$g" || echo "  FAILED"
-done
-```
-
-| ID | Catches |
-|---|---|
-| `G-1-jwt-auth-literal` | JWT silent auth downgrade reappearing |
-| `L-001-insecure-skip-verify` | Bare `InsecureSkipVerify: true` without `//nolint:gosec` |
-| `H-001-bare-from` | Bare Dockerfile `FROM` without `@sha256:` digest pin |
-| `M-012-no-root-user` | Dockerfile missing terminal `USER <non-root>` |
-| `H-009-readme-jwt` | README re-introducing JWT-as-supported claim |
-| `G-2-api-key-hash-json` | `api_key_hash` in JSON-emitting surface |
-| `U-2-plaintext-healthcheck` | Plaintext `http://` in HEALTHCHECK |
-| `U-3-migration-mount` | Migration file mounted into postgres initdb |
-| `D-1-D-2-statusbadge-phantom` | Dead StatusBadge keys + 8 TS phantom fields across 4 interfaces |
-| `L-1-bulk-action-loop` | Client-side `for ... await` bulk action loops |
-| `B-1-orphan-crud` | 8 update/create/delete fns lose page consumers |
-| `S-2-strings-contains-err` | `strings.Contains(err.Error(), ...)` brittle dispatch |
-| `G-3-env-docs-drift` | `CERTCTL_*` env var defined OR documented but not both |
-| `test-naming-convention` | `func TestXxx` lowercase first letter (Go silently skips) |
-| `S-1-hardcoded-source-counts` | Hardcoded "N issuer connectors" prose |
-| `P-1-documented-orphan-fns` | 16 read-fn names removed from client.ts exports |
-| `T-1-frontend-page-coverage` | New page in `web/src/pages/` without sibling `.test.tsx` |
-| `bundle-8-L-015-target-blank-rel-noopener` | `target="_blank"` without `rel="noopener noreferrer"` |
-| `bundle-8-L-019-dangerously-set-inner-html` | `dangerouslySetInnerHTML` outside `safeHtml.ts` |
-| `bundle-8-M-009-bare-usemutation` | Bare `useMutation()` outside the `useTrackedMutation` wrapper |
-
-Plus three additional scripts for non-guard operator workflows:
-- `scripts/ci-guards/vendor-e2e-skip-check.sh` — vendor-e2e skip-count enforcement (used by `deploy-vendor-e2e` job)
-- `scripts/ci-guards/digest-validity.sh` — used by `image-and-supply-chain` job
-- `scripts/ci-guards/openapi-handler-parity.sh` — used by `image-and-supply-chain` job
-- `scripts/ci-guards/coverage-pr-comment.sh` — used by `go-build-and-test` job
-- `scripts/check-coverage-thresholds.sh` — used by `go-build-and-test` job
-
-## Coverage thresholds
-
-Manifest at `.github/coverage-thresholds.yml`. Each entry has `floor:` (integer percentage) + `why:` (load-bearing context). Lowering a floor REQUIRES corresponding code-side test work — never lower the gate to make CI green.
-
-To add a new gated package: add an entry to the YAML; no script changes needed.
-
-## Make targets — three-tier convention
-
-| Target | When | What |
-|---|---|---|
-| `make verify` | **Required pre-commit** | gofmt + vet + golangci-lint + go test -short |
-| `make verify-deploy` | Optional pre-push | digest-validity + OpenAPI parity + Docker build smoke (server + agent only — fast subset) |
-| `make verify-docs` | **Required pre-tag** | QA-doc Part-count + seed-count drift checks |
-
-## Adding a new check
-
-| Check type | Where it goes | Auto-picked-up by CI? |
-|---|---|---|
-| Regression guard (grep / shape pattern) | New `scripts/ci-guards/<id>.sh` script | Yes — loop step iterates `*.sh` |
-| Coverage threshold (per-package) | New entry in `.github/coverage-thresholds.yml` | Yes — bash loop reads YAML |
-| OpenAPI route exception | New entry in `api/openapi-handler-exceptions.yaml` | Yes — parity script reads YAML |
-| Vendor-e2e expected skip | New line in `scripts/ci-guards/vendor-e2e-skip-allowlist.txt` | Yes — skip-check script reads file |
-| New CI job | Edit `.github/workflows/ci.yml` directly | n/a (job definition is the source) |
-
-## Troubleshooting
-
-| CI step fails | Likely cause | Fix |
-|---|---|---|
-| `gofmt drift` | source needs `gofmt -w` | `make fmt` locally + commit |
-| `go mod tidy drift` | imported a package without committing go.mod | `go mod tidy` + commit |
-| `Run staticcheck` | new SA1019 deprecated-API site | migrate the API OR add `//lint:ignore SA1019 <reason>` |
-| `Check Coverage Thresholds` | per-package coverage dropped below floor | add tests; do NOT lower the floor |
-| `Regression guards` (any `<id>.sh`) | the audit-finding the guard pinned reappeared | read the guard's head-comment block for the closure rationale + fix the regression |
-| `Skip-count enforcement` | a vendor sidecar failed to start | check docker logs; fix sidecar; OR if a new Windows-only test was added, add to `scripts/ci-guards/vendor-e2e-skip-allowlist.txt` |
-| `Digest validity` | a `@sha256` digest doesn't resolve | re-resolve from registry, replace in compose / Dockerfile |
-| `OpenAPI ↔ handler parity` | new router route without operationId | add to `api/openapi.yaml` (preferred) OR `api/openapi-handler-exceptions.yaml` |
-| `Docker build smoke` | Dockerfile syntax error or COPY path drift | fix the Dockerfile |
-| `CodeQL Analyze` | interprocedural dataflow finding | review the SARIF in Security → Code scanning tab |
-
-## Status check accounting
-
-**Current (post-cleanup):** 7 status checks per push.
-- 1 × `Go Build & Test`
-- 1 × `Frontend Build`
-- 1 × `Helm Chart Validation`
-- 1 × `deploy-vendor-e2e`
-- 1 × `image-and-supply-chain`
-- 2 × `CodeQL Analyze (<lang>)` (go + javascript-typescript)
-
-**Pre-cleanup (HEAD `1de61e91`):** 19 status checks. The 12-vendor matrix + 2-vendor Windows matrix collapsed to 1 + 0 respectively; the 3 Go/Frontend/Helm jobs unchanged; 2 CodeQL unchanged; 1 new `image-and-supply-chain` added.
-
-## Required GitHub branch protection list
-
-When updating the `master` branch protection rule (Settings → Branches), the "Require status checks to pass" list should be exactly:
-
-```
-Go Build & Test
-Frontend Build
-Helm Chart Validation
-deploy-vendor-e2e
-image-and-supply-chain
-Analyze (go)
-Analyze (javascript-typescript)
-```
-
-Old-name checks (`deploy-vendor-e2e (<vendor>)` × 12, `deploy-vendor-e2e-windows (<vendor>)` × 2) won't appear on new PRs after the workflow change. Operator removes them from the required list.

docs/contributor/gui-qa-checklist.md

@@ -1,68 +0,0 @@
-# GUI QA Checklist
-
-> Last reviewed: 2026-05-05
-
-Manual GUI verification pass for release sign-off. Vitest covers component-level behavior; this checklist covers end-to-end flows that only land correctly when the React SPA, the REST API, and the database are all wired together.
-
-## Prereqs
-
-The full stack must be running and healthy per [`qa-prerequisites.md`](qa-prerequisites.md). Open `https://localhost:8443` in a fresh browser session (Incognito / Private mode is fine — avoids cached state from previous QA passes).
-
-## Pages to verify
-
-For each page, the verification is "open it, confirm it renders without console errors, exercise the documented action, confirm the action lands as expected."
-
-| Page | Action to verify | Expected result |
-|---|---|---|
-| `/dashboard` | Page loads, all 4 stat cards populate | Total / Active / Expiring / Expired counts match `GET /api/v1/stats/summary` |
-| `/certificates` | Inventory list paginates | "Next page" button works; URL updates with cursor; row count consistent |
-| `/certificates/<id>` | Detail page opens for any cert | Cert chain renders, deployment status shows, audit timeline visible |
-| `/issuers` | Catalog renders all configured issuers | Each issuer card shows last-used / status; clicking opens detail |
-| `/issuers/<id>` | Issuer config form | Edit + Save round-trips through `PATCH /api/v1/issuers/<id>` |
-| `/issuers/hierarchy` | CA tree view | Multi-level hierarchy renders; admin-gated CRUD buttons present for admins only |
-| `/agents` | Fleet view | Online/offline status accurate; OS/arch grouping correct |
-| `/agents/<id>` | Agent detail | Last heartbeat, registered date, deployment job history |
-| `/agents/groups` | Agent groups CRUD | Create + edit + delete a test group; verify dynamic membership matching |
-| `/jobs` | Job queue | Filter by status / type works; click into a job opens detail |
-| `/jobs/<id>` | Job detail | Status, retries, logs, owner attribution |
-| `/policies` | Renewal policies CRUD | Edit AlertChannels matrix, save, verify backend reflects change |
-| `/profiles` | Certificate profiles | EKU constraints + max TTL editable; profile binding works |
-| `/notifications` | Notifier config | Test connection button against each configured notifier |
-| `/discovery` | Discovery triage | Claim / Dismiss buttons round-trip to backend |
-| `/network-scans` | Scan target CRUD | Create scan target, trigger immediate scan, results appear |
-| `/audit` | Audit trail | Filter by actor / action / time range; CSV export works |
-| `/short-lived` | Short-lived credential dashboard | Live TTL countdown updates; auto-refresh every 10s |
-| `/observability` | Observability dashboard | Charts render: expiration heatmap, renewal trends, issuance rate |
-| `/health` | Health monitor | TLS endpoint health: healthy / degraded / down states accurate |
-| `/digest` | Digest preview | Email preview renders; "Send digest" button dispatches |
-| `/owners` | Owners CRUD | Create owner with team, edit, delete (after reassigning certs) |
-| `/teams` | Teams CRUD | Create + delete; verify cascade removes orphan owners |
-| `/scep` | SCEP admin tabs | Profiles / Intune Monitoring / Recent Activity all populate |
-| `/est` | EST admin tabs | Profiles / Recent Activity / Trust Bundle all populate |
-| `/login` | Login flow | API key entry persists for the session; bad key rejected |
-
-## Console hygiene
-
-Open browser DevTools and confirm:
-
-- No uncaught exceptions on any page
-- No 404 / 500 responses in the Network tab from API calls
-- No CORS errors
-- No CSP violations
-
-## Mobile / narrow-viewport
-
-The dashboard is desktop-first but should not break catastrophically on narrow viewports. Resize the browser to 380px width; confirm:
-
-- Sidebar collapses to a hamburger menu
-- Tables either scroll horizontally or stack on mobile
-- Forms remain usable
-
-## Accessibility spot-check
-
-- Tab through any single page using only the keyboard. Every interactive element must be reachable, and the focus indicator must be visible.
-- Lighthouse accessibility audit on `/dashboard`: target ≥ 90.
-
-## Sign-off
-
-Document any deviations in the release sign-off matrix at [`release-sign-off.md`](release-sign-off.md).

docs/contributor/qa-prerequisites.md

@@ -1,99 +0,0 @@
-# QA Prerequisites
-
-> Last reviewed: 2026-05-05
-
-Operational prereqs for running release QA against certctl. Before any of the contributor-facing testing surfaces (test-environment.md, gui-qa-checklist.md, release-sign-off.md) are useful, the local stack needs to be in a known-good state.
-
-## Why manual QA on top of automated tests?
-
-Automated tests mock dependencies and run in isolation. Manual QA validates the full integrated stack: real PostgreSQL, real HTTP, real agent binary, real file I/O, real scheduler timing. It catches issues that unit tests can't: migration ordering, Docker networking, env var parsing, browser rendering, and timing-dependent scheduler behavior.
-
-## Environment setup
-
-**Step 1: Start the full stack.**
-
-```bash
-cd deploy && docker compose -f docker-compose.yml -f docker-compose.demo.yml up --build -d
-```
-
-This builds three containers (postgres, certctl-server, certctl-agent) and runs them on a bridge network. The `--build` flag ensures you're testing the current code, not a stale image. The `demo` overlay is an override file (no `image:` or `build:` of its own) that layers `CERTCTL_DEMO_SEED=true` onto the base — both files must be passed in that order or compose errors with `service "certctl-server" has neither an image nor a build context specified`. The seed populates the database with realistic fixtures.
-
-**Step 2: Wait for healthy state.**
-
-```bash
-for i in $(seq 1 30); do
-  STATUS=$(docker compose ps --format json 2>/dev/null | jq -r 'select(.Health != null) | "\(.Name): \(.Health)"' 2>/dev/null)
-  echo "$STATUS"
-  echo "$STATUS" | grep -q "unhealthy\|starting" || break
-  sleep 2
-done
-```
-
-Why: Docker Compose starts containers in dependency order (postgres → server → agent), but "started" doesn't mean "ready." Health checks confirm postgres accepts connections, the server responds on `/health`, and the agent process is running.
-
-**Step 3: Set shell variables used throughout the QA flow.**
-
-```bash
-export SERVER=https://localhost:8443
-export API_KEY="change-me-in-production"
-export AUTH="Authorization: Bearer $API_KEY"
-export CT="Content-Type: application/json"
-export CACERT="--cacert ./deploy/test/certs/ca.crt"
-```
-
-Every curl command in QA docs uses these variables. Setting them once avoids typos and keeps the docs copy-pasteable.
-
-> **Note:** The default Docker Compose sets `CERTCTL_AUTH_TYPE: none` for the demo overlay, meaning auth is disabled. Tests that exercise auth require flipping this to `api-key`; instructions are in the relevant test docs.
-
-**Step 4: Build CLI and MCP server binaries on the host.**
-
-```bash
-go build -o certctl-cli ./cmd/cli/...
-go build -o certctl-mcp ./cmd/mcp-server/...
-```
-
-The CLI and MCP server are separate binaries that talk to the server over HTTP. Building them verifies the code compiles and produces the executables you'll test later.
-
-## Demo data baseline
-
-The seed data (`migrations/seed.sql` + `migrations/seed_demo.sql`) pre-populates the database with realistic fixtures. Confirm it loaded:
-
-```bash
-curl -s $CACERT -H "$AUTH" $SERVER/api/v1/stats/summary | jq .
-```
-
-**Expected shape:**
-
-```json
-{
-  "total_certificates": 15,
-  "active_certificates": ...,
-  "expiring_certificates": ...,
-  "expired_certificates": ...,
-  "pending_renewals": ...
-}
-```
-
-**Reference IDs in the demo data** (used across QA docs):
-
-| Resource | IDs | Count |
-|---|---|---|
-| Teams | `t-platform`, `t-security`, `t-payments`, `t-frontend`, `t-data` | 5 |
-| Owners | `o-alice`, `o-bob`, `o-carol`, `o-dave`, `o-eve` | 5 |
-| Policies | `rp-standard`, `rp-urgent`, `rp-manual` | 3 |
-| Issuers | `iss-local`, `iss-acme-le`, `iss-stepca`, `iss-digicert` | 4 |
-| Agents | `ag-web-prod`, `ag-web-staging`, `ag-lb-prod`, `ag-iis-prod`, `ag-data-prod` | 5 |
-| Targets | `tgt-nginx-prod`, `tgt-nginx-staging`, `tgt-f5-prod`, `tgt-iis-prod`, `tgt-nginx-data` | 5 |
-| Profiles | `prof-standard-tls`, `prof-internal-mtls`, `prof-short-lived`, `prof-high-security` | 4 |
-| Certificates | `mc-api-prod`, `mc-web-prod`, `mc-pay-prod`, etc. | 15 |
-| Agent Groups | `ag-linux-prod`, `ag-linux-amd64`, `ag-windows`, `ag-datacenter-a`, `ag-manual` | 5 |
-| Network Scan Targets | `nst-dc1-web`, `nst-dc2-apps`, `nst-dmz` | 3 |
-
-## Once these are green
-
-Move to the appropriate downstream surface:
-
-- [`test-environment.md`](test-environment.md) — full local environment tutorial with real CAs (Pebble, step-ca, etc.)
-- [`gui-qa-checklist.md`](gui-qa-checklist.md) — manual GUI test pass
-- [`release-sign-off.md`](release-sign-off.md) — release-day checklist
-- [`testing-strategy.md`](testing-strategy.md) — what we test in CI vs daily deep-scan vs manual QA

docs/contributor/qa-test-suite.md

@@ -1,445 +0,0 @@
-# QA Test Suite Guide (`qa_test.go`)
-
-> Last reviewed: 2026-05-05
-
-> **Audience:** Anyone running release QA for certctl — whether you're a first-time contributor or the maintainer cutting a release tag.
->
-> **Self-contained.** Through 2026-05-04 this doc was a companion to a separate `docs/testing-guide.md` (the *what* to test) — that companion was pruned during the Phase 5 docs overhaul (its content dispersed across the audience-organized doc tree). The Part-by-Part Coverage Map below is now the canonical inventory of QA Parts.
-
----
-
-## Test Suite Health (regenerate via `make qa-stats`)
-
-> Snapshot at HEAD. Re-run `make qa-stats` to refresh; the QA-doc seed-count drift guard (`.github/workflows/ci.yml::QA-doc seed-count drift guard`) catches out-of-date cert / issuer counts on every PR. The Part-count drift guard retired in the 2026-05-04 docs overhaul Phase 5 (testing-guide.md was pruned; Part counts are now tracked inside `qa_test.go` itself, not against an external doc). **Last regenerated: 2026-04-27 (Bundle P).**
-
-| Metric | Value | Target | Status |
-|---|---|---|---|
-| Backend test files | 221 | n/a | ℹ |
-| Backend `Test*` functions | 2,454 | n/a | ℹ |
-| Backend `t.Run` subtests | 778 | n/a | ℹ |
-| Frontend test files | 38 | n/a | ℹ |
-| Fuzz targets | 11 | ≥10 (one per hand-rolled parser) | ✓ |
-| `t.Skip` sites | 60 | each carries valid rationale (Bundle O audit) | ✓ |
-| `qa_test.go` Part_* subtests | 53 | covers 49 of 56 historical QA Parts directly + Parts 15–17 indirectly via Parts 42–46 | ✓ |
-| Existential cluster line cov (post-Bundle-J + L.B + Bundle 0.7) | acme 55.6%, stepca 90.4%, local-issuer ≥86%, crypto ≥85% | ≥95% | △ ACME below; tracked in `coverage-matrix.md` |
-| Mutation kill rate (Existential) | unmeasured (operator-runnable per Strengthening #5) | ≥90% | ⚠ |
-| Race detector clean (`-count=10`) | partial (`-count=3` clean per Phase 0) | 0 races | ⚠ |
-
-## What Is This File?
-
-`deploy/test/qa_test.go` is a single Go test file (~1700 lines) that automates the historical QA Part inventory (preserved in the Part-by-Part Coverage Map below) against a running certctl Docker Compose demo stack. It replaces the legacy `qa-smoke-test.sh` bash script.
-
-It covers **49 of 56 Parts** of the testing guide as automation; the remaining 7 are
-either manual-only by design or pending QA-suite coverage:
-
-- **49 `Part_*` automation wrappers**, **~159 leaf subtests** — API calls, database queries, source file checks, performance benchmarks
-- **11 fully skipped Parts** — with documented reasons (external CAs, Windows, browser-only, etc.) — see "What This Test Does NOT Cover" below
-- **4 Parts NOT YET AUTOMATED** — Parts 23 (S/MIME & EKU), 24 (OCSP/CRL), 55 (Agent Soft-Retirement), 56 (Notification Retry & Dead-Letter) — must be tested manually until QA-suite automation lands; the Part-by-Part Coverage Map below describes the surface area each Part covers
-- **Manual-only flows** in addition: GUI flows, scheduler timing, Docker log inspection — must be done by a human (Coverage Map below describes each)
-
-## Architecture
-
-```mermaid
-flowchart LR
-    QA["qa_test.go (//go:build qa)<br/><br/>TestQA(t *testing.T)<br/>├─ Part01_Infra<br/>├─ Part02_Auth<br/>├─ Part03_CertCRUD<br/>├─ ...<br/>└─ Part52_HelmChart"]
-    subgraph Stack["certctl demo stack<br/>docker-compose.yml + docker-compose.demo.yml"]
-        Server["certctl-server :8443"]
-        Postgres["postgres :5432"]
-        Agents["certctl-agent (×N)<br/>↑ seed_demo.sql provisions 12 agent rows<br/>(1 active, 2 retired, 9 reserved/sentinel)<br/>for the soft-retire / FSM coverage Parts 55–56 exercise"]
-    end
-    QA --> Stack
-```
-
-> **Multi-agent demo stack (Bundle Q / L-004 closure).** The demo
-> stack runs a single live `certctl-agent` container by default but
-> the database is seeded with 12 agent rows (`migrations/seed_demo.sql`,
-> grep `mc-* | ag-*` IDs). The "(×N)" notation reflects the seed-data
-> reality: Parts 04 (Agents Listing), 05 (Agent Heartbeats), 55
-> (Agent Soft-Retirement), and FSM coverage tables in
-> `coverage-audit-2026-04-27/tables/fsm-coverage.md` exercise the full
-> multi-agent population, not the one live container. Operators
-> running the QA suite in a parallel-agent topology should set
-> `AGENT_COUNT=N` in compose-override and re-derive the seed counts
-> via `make qa-stats`.
-
-Key design choices:
-
-- **Build tag:** `//go:build qa` — never runs during `go test ./...` or CI. Only runs when explicitly requested.
-- **Package:** `integration_test` — same package as `integration_test.go` (which uses `//go:build integration` for the test stack). They coexist but never run together.
-- **Zero internal imports:** Uses only stdlib + `lib/pq` (from `go.mod`). All API interactions are plain HTTP. All JSON is decoded into lightweight local structs (`qaCert`, `qaJob`, etc.) — not the internal domain types.
-- **Self-cleaning:** Tests that create data use `t.Cleanup()` to delete it afterward. The seed data is not modified.
-
-## Prerequisites
-
-1. **Docker Compose demo stack running:**
-   ```bash
-   cd deploy
-   docker compose -f docker-compose.yml -f docker-compose.demo.yml up --build -d
-   ```
-   Wait ~15 seconds for health checks to pass.
-
-2. **Go 1.22+** installed (the project uses Go 1.25 in `go.mod`, but 1.22+ works for running tests).
-
-3. **PostgreSQL port exposed** — the demo stack exposes port 5432 for database verification tests (table counts, schema checks).
-
-4. **Repository checkout** — source file verification tests (`fileExists`, `fileContains`) read files relative to `qaRepoDir` (default: `../..` from `deploy/test/`).
-
-## Running the Tests
-
-### Full suite
-```bash
-cd deploy/test
-go test -tags qa -v -timeout 10m ./...
-```
-
-### Single Part
-```bash
-go test -tags qa -v -run TestQA/Part03 ./...
-```
-
-### Single subtest
-```bash
-go test -tags qa -v -run TestQA/Part03_CertCRUD/Create_Minimal ./...
-```
-
-### With custom environment
-```bash
-CERTCTL_QA_SERVER_URL=https://staging.internal:8443 \
-CERTCTL_QA_API_KEY=my-staging-key \
-CERTCTL_QA_DB_URL=postgres://certctl:secret@db.internal:5432/certctl?sslmode=require \
-CERTCTL_QA_REPO_DIR=/path/to/certctl \
-go test -tags qa -v -timeout 10m ./...
-```
-
-### Environment Variables
-
-| Variable | Default | Description |
-|---|---|---|
-| `CERTCTL_QA_SERVER_URL` | `https://localhost:8443` | certctl server URL (HTTPS-only as of v2.2) |
-| `CERTCTL_QA_API_KEY` | `change-me-in-production` | API key for Bearer auth |
-| `CERTCTL_QA_DB_URL` | `postgres://certctl:certctl@localhost:5432/certctl?sslmode=disable` | PostgreSQL connection string |
-| `CERTCTL_QA_REPO_DIR` | `../..` | Path to certctl repo root (for source file checks) |
-| `CERTCTL_QA_CA_BUNDLE` | `./certs/ca.crt` | PEM CA bundle pinned for TLS verification. The demo stack's `certctl-tls-init` container writes here. |
-| `CERTCTL_QA_INSECURE` | `false` | Set to `"true"` to skip TLS verification (e.g. before the init container finishes). Never use outside the demo harness. |
-
-## Part-by-Part Coverage Map
-
-This table shows what each Part tests and what's left for manual verification.
-
-| Part | Testing Guide Section | Automated Subtests | What's Automated | What's Manual |
-|------|----------------------|-------------------|-----------------|--------------|
-| 1 | Infrastructure & Deployment | 8 | Table count, health/ready endpoints, seed data counts (certs, agents, issuers, targets, policies) | Docker container health, log inspection, volume mounts |
-| 2 | Authentication & Security | 4 | No-auth 401, bad-key 401, health-no-auth 200, no private keys in API | CORS preflight, rate limiting (429 + Retry-After), TLS config |
-| 3 | Certificate Lifecycle | 10 | Create (minimal + full), get, 404, list pagination, status/issuer filters, sparse fields, update, archive | Deployment trigger, version history, certificate detail UI |
-| 4 | Renewal Workflow | 3 | Trigger renewal, 404 on nonexistent, agent work endpoint | AwaitingCSR flow, agent key generation, full issuance cycle |
-| 5 | Revocation | 5 | Revoke (default reason), already-revoked, nonexistent, invalid reason, CRL JSON | DER CRL, OCSP responder, revocation notifications |
-| 6 | Policies & Profiles | 6 | Policy CRUD (create/delete), invalid type 400, profile CRUD, list | Policy violation detection, profile enforcement on CSR |
-| 7 | Ownership & Teams | 4 | Team CRUD, owner CRUD, agent groups list | Owner notification routing, dynamic group matching |
-| 8 | Job System | 2 | List jobs, 404 on nonexistent | Job state transitions, approval workflow, cancellation |
-| 9 | Issuer Connectors | 4 | List, get detail, create (GenericCA), missing name 400 | Test connection, issuer-specific issuance flow |
-| 10 | Sub-CA Mode | SKIP | — | Requires CA cert+key on disk |
-| 11 | ACME ARI | SKIP | — | Requires ARI-capable CA |
-| 12 | Vault PKI | SKIP | — | Requires live Vault server |
-| 13 | DigiCert | SKIP | — | Requires DigiCert sandbox |
-| 14 | Target Connectors | 3 | List, create NGINX target, delete 204 | Deploy to real target, validate deployment |
-| 15–17 | Apache/HAProxy, Traefik/Caddy, IIS | — | (Covered by source checks in Parts 42–46) | Requires real services or Windows |
-| 18 | Agent Operations | 3 | Heartbeat (register), metadata check, auto-create on heartbeat | Agent binary behavior, key storage, discovery scan |
-| 19 | Agent Work Routing | 1 | Empty work for agent with no targets | Scoped job assignment, multi-target fan-out |
-| 20 | Post-Deployment Verification | 1 | 404 on nonexistent job verification | TLS probing, fingerprint comparison |
-| 21 | EST Server | 2 | CACerts (200 + content-type), CSRAttrs (200/204) | simpleenroll with CSR, simplereenroll, PKCS#7 parsing |
-| 22 | Certificate Export | 3 | PEM export, PKCS#12 export, 404 on nonexistent | Download mode, file content validation |
-| 23 | S/MIME & EKU Support | 0 (NOT AUTOMATED) | — | S/MIME profile creation; EKU enforcement on issuance; SMIMECapabilities extension presence in issued cert; rejection of profile-violating EKU on CSR. Test manually — see the Coverage Map row |
-| 24 | OCSP Responder & DER CRL | 0 (NOT AUTOMATED) | — | OCSP request/response (RFC 6960), DER CRL generation, status (Good/Revoked/Unknown), Must-Staple coordination. Test manually — see the Coverage Map row |
-| 25 | Certificate Discovery | 5 | List discovered, summary, list scan targets, create target, invalid CIDR 400 | Agent filesystem scan, claim/dismiss workflow |
-| 26 | Enhanced Query API | 4 | Sort descending, cursor pagination, time-range filter, invalid sort field | Field projection correctness, cursor token cycling |
-| 27 | Request Body Size Limits | 1 | 2MB body rejected (413/400) | Exact limit boundary (1MB) |
-| 28 | CLI | SKIP | — | Requires compiled `certctl-cli` binary |
-| 29 | MCP Server | SKIP | — | Requires compiled `mcp-server` binary + stdio |
-| 30 | Observability | 7 | Dashboard summary, certs by status, expiration timeline, job trends, issuance rate, JSON metrics (uptime + gauges), Prometheus (content-type + 4 metric names) | Chart rendering (GUI), Grafana import |
-| 31 | Notifications | 2 | List, 404 on nonexistent | Notification content, mark-read, email/Slack delivery |
-| 32 | Audit Trail | 3 | List events (≥10), PUT immutability, DELETE immutability | Actor attribution, body hash, time range filters |
-| 33 | Background Scheduler | SKIP | — | Timing-dependent; verify via Docker logs |
-| 34 | Structured Logging | SKIP | — | Requires Docker log inspection |
-| 35 | GUI Testing | SKIP | — | Requires browser |
-| 36–37 | Issuer Catalog, Frontend Audit | SKIP | — | Requires browser |
-| 38 | Error Handling | 5 | Malformed JSON, missing required field, method not allowed, UTF-8 CN, empty body | Stack trace suppression, error response format |
-| 39 | Performance | 5 | List certs < 200ms, stats < 500ms, metrics < 200ms, Prometheus < 300ms, audit < 500ms | Load testing, concurrent request handling |
-| 40 | Documentation | 8 | README, quickstart, architecture, connectors exist; migration guides exist; 8 issuer types in docs; 11 target types in docs | Content accuracy, link validity |
-| 41 | Regression | 3 | DELETE 204, per_page max fallback, network scan target seed count | `errors.Is(errors.New())` anti-pattern source scan |
-| 42 | Envoy Target | 5 | Domain type, connector file, test file, OpenAPI, agent dispatch | Envoy deployment test, SDS config |
-| 43 | Postfix/Dovecot | 3 | Domain types (Postfix + Dovecot), connector file, OpenAPI | Mail server deployment test |
-| 44 | SSH Target | 4 | Domain type, connector file, agent dispatch (`sshconn`), OpenAPI | SSH deployment test (requires target host) |
-| 45 | Windows Certificate Store | 3 | Domain type, connector file, shared certutil package | Windows deployment (requires Windows) |
-| 46 | Java Keystore | 3 | Domain type, connector file, OpenAPI | JKS deployment (requires keytool) |
-| 47 | Certificate Digest Email | 3 | Preview endpoint (200/503), service file, adapter file | SMTP delivery, HTML template rendering |
-| 48 | Dynamic Issuer Config | 4 | Crypto package exists, create ACME issuer via API, config redaction check, migration exists | Test connection flow, registry rebuild |
-| 49 | Dynamic Target Config | 2 | Create NGINX target via API, migration exists | Test connection via agent heartbeat |
-| 50 | Onboarding Wizard | 2 | Wizard component exists, docker-compose split (clean vs demo) | Wizard UI flow, step completion |
-| 51 | ACME Profile Selection | 3 | Profile module exists, frontend config, RFC 9702→9773 renumber check | Profile-aware issuance against real CA |
-| 52 | Helm Chart | 5 | Chart.yaml, values.yaml, 4 templates exist, securityContext, health probes | `helm template` rendering, `helm install` |
-| 53 | Kubernetes Secrets Target Connector (M47) | 18 | Config validation (namespace DNS-1123, secret name DNS subdomain, label keys, required fields), deployment (create/update Secret, chain concatenation, error propagation), validation (serial comparison, not-found, empty cert) | GUI target wizard KubernetesSecrets fields (namespace, secret_name, labels, kubeconfig_path), Helm RBAC toggle, TargetDetailPage type label |
-| 54 | AWS ACM Private CA Issuer Connector (M47) | 23 | Config validation (region, CA ARN regex, signing algorithm whitelist, validity_days, defaults), issuance (full flow, empty CSR, errors), renewal (reuses issuance), revocation (reason mapping, default, errors), GetOrderStatus completed, GetCACertPEM (success/chain/error), GetRenewalInfo nil | GUI issuer wizard AWSACMPCA fields (region, ca_arn, signing_algorithm, validity_days, template_arn), seed data visibility, create issuer flow |
-| 55 | Agent Soft-Retirement (I-004) | 0 (NOT AUTOMATED) | — | Soft-retire vs hard-retire; force flag; reason capture; foreign-key cascade behavior on retired-agent cert ownership; reactivation. Test manually — see the Coverage Map row |
-| 56 | Notification Retry & Dead-Letter Queue (I-005) | 0 (NOT AUTOMATED) | — | Retry loop with exponential backoff, dead-letter transition after N retries, requeue endpoint (`POST /api/v1/notifications/{id}/requeue`), idempotency on retry. Test manually — see the Coverage Map row |
-
-**Totals (verified 2026-04-27):** 49 `Part_*` automation wrappers, ~159 leaf subtests, 11 fully
-skipped Parts, 4 Parts not yet automated (23, 24, 55, 56), and an unspecified count of manual-only
-flows (GUI, scheduler timing, Docker log inspection). Run `grep -cE 't\.Run\("Part[0-9]+_' deploy/test/qa_test.go` to count Part_* automation wrappers
-and `grep -cE 't\.Run\("Part[0-9]+_' deploy/test/qa_test.go` to re-verify.
-
-## Coverage by Risk Class
-
-A buyer's QA lead reading this doc wants "where are the existential bugs caught?" — Bundle P / Strengthening #1 surfaces that view directly. The table below classifies each Part by risk class so reviewers can answer the existential-coverage question in one glance.
-
-| Risk class | Description | Parts in scope | Automation status |
-|---|---|---|---|
-| **Existential** (Critical paths — bugs would compromise CA, leak keys, mis-issue, bypass revocation) | Crypto, PKCS#7, local-issuer, OCSP/CRL, agent keygen, CSR validation | 5 (Revocation), 21 (EST), 23 (S/MIME EKU), 24 (OCSP/CRL), 47 (Digest with cert content), 53 (K8s Secrets), 54 (AWS PCA) | 5/7 automated; Parts 23 + 24 pending (Bundle I Skip stubs in `qa_test.go`; manual playbook in the Coverage Map below) |
-| **High** (FSM corruption, credential leak, authn/z weakening) | Renewal, jobs, agents, issuers, deployment, scheduler | 4, 7, 8, 9, 18, 19, 20, 22, 25, 28, 29, 32, 33, 48, 49, 55, 56 | 14/17 automated; CLI / MCP / scheduler-loop are inherently SKIP (require compiled binaries / Docker logs); Parts 55 + 56 pending |
-| **Medium** (Operational pain or silent data drift) | Targets, notifiers, observability, error handling, performance, regression | 14, 15-17, 30, 31, 38, 39, 40, 41, 42, 43, 44, 45, 46 | 14/14 automated (15-17 indirect via Parts 42–46) |
-| **Low** (Hygiene) | Documentation, docs verification | 40 (Documentation), 50 (Onboarding) | 2/2 automated |
-| **Frontend** (XSS, render correctness, mutation contracts) | GUI testing | 35, 36-37 | 0/3 automated in this suite (Vitest covers separately under `web/`); this doc punts to manual + Vitest |
-| **Audit-relevant** | Audit trail, body-size limits, request limits, Helm chart deploy posture | 27, 32, 51, 52 | 4/4 automated |
-
-This is the table acquisition reviewers screenshot for their report. When a new Part_* subtest lands in `qa_test.go`, classify it here.
-
-## Test Categories
-
-The automated tests fall into four categories:
-
-### 1. API Integration Tests (majority)
-Make real HTTP requests to the running server and verify status codes, response structure, and JSON field values. Examples:
-- `POST /api/v1/certificates` with valid payload → 201
-- `GET /api/v1/certificates?status=Active` → all returned certs have `status: "Active"`
-- `DELETE /api/v1/certificates/mc-qa-full` → 204
-
-### 2. Database Verification Tests
-Connect directly to PostgreSQL and verify schema state:
-- Table count ≥ 19 (from migrations 000001–000010)
-- Useful for catching migration regressions
-
-### 3. Source File Verification Tests
-Read files from the repo checkout and verify structure:
-- Domain types exist in `internal/domain/connector.go` (e.g., `TargetTypeEnvoy`)
-- Connector implementations exist (e.g., `internal/connector/target/envoy/envoy.go`)
-- Documentation contains expected content (all issuer/target types listed)
-- No stale RFC 9702 references (replaced by RFC 9773)
-
-### 4. Performance Spot Checks
-Timed API requests with threshold assertions:
-- `GET /api/v1/certificates?per_page=15` < 200ms
-- `GET /api/v1/stats/summary` < 500ms
-- `GET /api/v1/metrics/prometheus` < 300ms
-
-## What This Test Does NOT Cover
-
-These gaps must be filled by manual testing — see each Coverage Map row for surface-area description:
-
-### Not Yet Automated (Parts 23, 24, 55, 56)
-
-These historical QA Parts are listed in the Coverage Map below but have no `Part_*` automation
-in `qa_test.go` yet. They are operator-runnable from the manual playbook; QA-suite
-automation should land before the next acquisition-grade release.
-
-- **Part 23: S/MIME & EKU Support** — profile-driven EKU enforcement; SMIMECapabilities extension
-- **Part 24: OCSP Responder & DER CRL** — OCSP request/response correctness, CRL generation, Must-Staple coordination
-- **Part 55: Agent Soft-Retirement (I-004)** — soft vs hard retire, FK cascade, reactivation
-- **Part 56: Notification Retry & Dead-Letter Queue (I-005)** — retry semantics, dead-letter transition, requeue
-
-### External CA Integrations (Parts 10–13)
-- **Sub-CA mode** — requires CA cert+key files on disk
-- **ACME ARI** — requires a CA that supports RFC 9773 Renewal Information
-- **Vault PKI** — requires a running HashiCorp Vault instance
-- **DigiCert / Sectigo / Google CAS** — requires sandbox API credentials
-
-### Browser/GUI Testing (Parts 35–37, 50)
-- Dashboard chart rendering (Recharts)
-- Onboarding wizard step-by-step flow
-- Issuer catalog card layout and create wizard
-- Bulk operations UI (multi-select, progress bars)
-- Discovery triage workflow
-
-### Real Deployment Testing (Parts 15–17)
-- NGINX/Apache/HAProxy file write + reload
-- Traefik/Caddy file provider or API reload
-- IIS PowerShell/WinRM (requires Windows)
-- F5 BIG-IP iControl REST (requires appliance or mock)
-- SSH agentless deployment (requires target host)
-
-### Agent Binary Behavior (Parts 18, 28–29)
-- Agent-side ECDSA key generation and CSR submission
-- Agent filesystem discovery scan
-- CLI tool (`certctl-cli`) — all 10 subcommands
-- MCP server (`mcp-server`) — stdio transport
-
-### Timing-Dependent Tests (Parts 33–34)
-- Background scheduler loop execution (renewal, jobs, health, notifications, digest, network scan)
-- Structured logging format verification (requires Docker log parsing)
-
-## How This Relates to `integration_test.go`
-
-Both files live in `deploy/test/` in the same Go package (`integration_test`):
-
-| | `qa_test.go` | `integration_test.go` |
-|---|---|---|
-| **Build tag** | `//go:build qa` | `//go:build integration` |
-| **Target stack** | Demo (`docker-compose.yml` + `docker-compose.demo.yml`) | Test (`docker-compose.test.yml`) |
-| **Port** | 8443 | Different (test stack config) |
-| **Seed data** | `seed_demo.sql` (32 certs, 12 agents, 13 issuers, 8 targets, realistic history) | Minimal (created by tests) |
-| **CA backends** | Local CA only (demo mode) | Pebble ACME, step-ca, NGINX |
-| **Purpose** | Release QA — broad coverage, spot checks | Functional — end-to-end issuance, renewal, revocation against real CAs |
-| **Run frequency** | Before each release tag | CI on every PR |
-
-They are complementary. Integration tests prove the machinery works. QA tests prove the product works at release quality.
-
-## Seed Data Reference
-
-The QA tests depend on `migrations/seed_demo.sql`. Key IDs used:
-
-### Certificates (32 total in `managed_certificates`)
-
-The full canonical list is generated by:
-```
-sed -n '/^INSERT INTO managed_certificates/,/^;/p' migrations/seed_demo.sql \
-  | grep -oE "^\s*\('mc-[a-z0-9_-]+" | sed -E "s/^\s*\('//" | sort -u
-```
-
-Hand-listing is unsustainable as the seed grows; tests reference IDs by lookup, not by enumeration.
-Sample IDs: `mc-api-prod`, `mc-web-prod`, `mc-pay-prod`, `mc-compromised`, `mc-smime-bob`, `mc-edge-eu`, `mc-k8s-ingress`, `mc-wildcard-prod`. See `migrations/seed_demo.sql:147` onward.
-
-### Agents (12 total in `agents` table)
-
-8 named workload agents + 1 server-side sentinel + 3 cloud-discovery sentinels:
-
-- **Workload agents:** `ag-web-prod`, `ag-web-staging`, `ag-lb-prod`, `ag-iis-prod`, `ag-data-prod`, `ag-edge-01`, `ag-k8s-prod`, `ag-mac-dev`
-- **Server-side sentinel:** `server-scanner`
-- **Cloud-discovery sentinels:** `cloud-aws-sm`, `cloud-azure-kv`, `cloud-gcp-sm`
-
-Full list via:
-```
-sed -n '/^INSERT INTO agents/,/^;/p' migrations/seed_demo.sql \
-  | grep -oE "^\s*\('[a-z][a-z0-9_-]+" | sed -E "s/^\s*\('//"
-```
-
-(The `agent_groups` table also contains entries with `ag-*` IDs — `ag-linux-prod`, `ag-windows`, `ag-datacenter-a`, `ag-arm64`, `ag-manual` — but those are *group* IDs, not agents. Don't confuse the two.)
-
-### Issuers (13 total)
-
-`iss-local`, `iss-acme-le`, `iss-stepca`, `iss-acme-zs`, `iss-openssl`, `iss-vault`, `iss-digicert`, `iss-sectigo`, `iss-googlecas`, `iss-awsacmpca`, `iss-entrust`, `iss-globalsign`, `iss-ejbca`.
-
-Full list via:
-```
-sed -n '/^INSERT INTO issuers/,/^;/p' migrations/seed_demo.sql \
-  | grep -oE "^\s*\('iss-[a-z0-9_-]+" | sed -E "s/^\s*\('//"
-```
-
-### Targets (8 total in `deployment_targets`)
-`tgt-nginx-prod`, `tgt-nginx-staging`, `tgt-haproxy-prod`, `tgt-apache-prod`, `tgt-iis-prod`, `tgt-traefik-prod`, `tgt-caddy-prod`, `tgt-nginx-data`
-
-### Network Scan Targets (4 total in `network_scan_targets`)
-`nst-dc1-web`, `nst-dc2-apps`, `nst-dmz`, `nst-edge`
-
-**Maintenance note:** when adding new seed rows, also update this section, OR remove the
-per-table counts and rely on the `sed | grep` commands so the doc stops drifting on every
-seed-data change. A CI guard that fails when the doc count diverges from the seed file is
-proposed in `coverage-audit-2026-04-27/tables/qa-doc-strengthening.md` (Strengthening #6).
-
-## Troubleshooting
-
-### "Server unreachable" on startup
-The test pings `GET /health` before running anything. If this fails:
-```bash
-# Check if the stack is running
-docker compose -f docker-compose.yml -f docker-compose.demo.yml ps
-
-# Check server logs
-docker compose -f docker-compose.yml -f docker-compose.demo.yml logs certctl-server
-
-# Check if the port is exposed (self-signed cert — pin CA bundle)
-curl --cacert ./deploy/test/certs/ca.crt -s https://localhost:8443/health
-```
-
-### "connect to QA DB" failure
-The database tests connect directly to PostgreSQL. Ensure port 5432 is exposed:
-```bash
-docker compose -f docker-compose.yml -f docker-compose.demo.yml port postgres 5432
-```
-
-### Performance tests flaking
-The performance thresholds (200ms, 300ms, 500ms) assume a local Docker stack. On slow CI runners or remote Docker hosts, increase the thresholds or skip Part 39:
-```bash
-go test -tags qa -v -run 'TestQA/Part(?!39)' ./...
-```
-
-### Source file checks failing
-The `fileExists` and `fileContains` helpers read from `CERTCTL_QA_REPO_DIR` (default `../..`). If running from a non-standard location:
-```bash
-CERTCTL_QA_REPO_DIR=/absolute/path/to/certctl go test -tags qa -v ./...
-```
-
-## Release Day Sign-Off Matrix
-
-Before tagging a release, the QA-on-call engineer signs off on each row. This matrix replaces the previous ad-hoc release checklist and ties test execution directly to release approval. Acquisition-grade releases have this kind of matrix; the doc previously didn't.
-
-| Sign-off | Evidence | Owner | Result | Date |
-|---|---|---|---|---|
-| `make verify` clean on master | CI run URL | Eng-on-call | ☐ | |
-| `go test -tags qa ./deploy/test/...` ≥ 95% pass rate (skips counted as pass) | Test output | QA-on-call | ☐ | |
-| `go test -race -count=10 ./internal/...` 0 races | `tool-output/race-x10.txt` | QA-on-call | ☐ | |
-| Coverage ≥ thresholds in `ci.yml` (service / handler / crypto / local-issuer / acme / stepca / mcp) | `tool-output/cover-summary.txt` | QA-on-call | ☐ | |
-| Helm chart `helm lint && helm template` clean | `tool-output/helm.txt` | DevOps-on-call | ☐ | |
-| All `t.Skip` sites have current rationales (see Bundle O audit; CI guard catches new orphans) | `make qa-stats` t.Skip count | QA-on-call | ☐ | |
-| Frontend: Vitest run clean; per-page coverage ≥ 70% | `web/tool-output/vitest.txt` | Frontend-on-call | ☐ | |
-| Manual Parts 23, 24, 55, 56 executed (or explicit defer with rationale) | This sheet | QA-on-call | ☐ | |
-| Demo stack `docker compose up -d --build` smoke (`/health` 200, `/ready` 200) | curl receipt | QA-on-call | ☐ | |
-| `govulncheck ./...` clean (or deferred-call advisories tracked in `gap-backlog`) | `tool-output/govulncheck.json` | Security-on-call | ☐ | |
-| QA-doc drift guards green (Part-count + cert-count) | CI run URL | QA-on-call | ☐ | |
-| FSM transition coverage tables (`coverage-audit-2026-04-27/tables/fsm-coverage.md`) — Existential FSMs ≥80% legal + 100% illegal | This sheet | QA-on-call | ☐ | |
-
-**Sign-off owner:** ______________________ &nbsp;&nbsp;**Date:** ______ &nbsp;&nbsp;**Tag:** v__.__.__
-
-## Mutation Testing Targets & Kill Rate
-
-Mutation testing exposes which assertions are actually load-bearing — tests can pass against broken code if mutations survive, which is a coverage trap. The audit's Phase 0 attempted to run `go-mutesting` on the Existential cluster but was blocked by a Go 1.25 / arm64 incompatibility in `osutil@v1.6.1` (uses `syscall.Dup2` which is undefined on linux/arm64). The operator-runnable workaround uses a fork that targets `unix.Dup3` instead.
-
-| Package | Risk class | Target kill rate | Last measured | Tool |
-|---|---|---|---|---|
-| `internal/crypto` | Existential | ≥90% | unmeasured (sandbox-blocked, operator-runnable) | go-mutesting |
-| `internal/pkcs7` | Existential | ≥90% | unmeasured | go-mutesting |
-| `internal/connector/issuer/local` | Existential | ≥90% | unmeasured | go-mutesting |
-| `internal/connector/issuer/acme` | Existential | ≥80% (catch-up; failure-mode coverage 55.6% per Bundle J) | unmeasured | go-mutesting |
-| `internal/connector/issuer/stepca` | Existential | ≥85% (post-Bundle-L.B coverage at 90.4%) | unmeasured | go-mutesting |
-| `internal/api/middleware` | High | ≥80% | unmeasured | go-mutesting |
-| `internal/validation` | Existential (CWE-78 / CWE-113 boundary) | ≥90% | unmeasured | go-mutesting |
-| `web/src/utils/safeHtml.ts` | Frontend (XSS gate) | ≥90% | unmeasured | Stryker |
-
-### Operator command (per package)
-
-```bash
-# Use the avito-tech fork that supports linux/arm64 + Go 1.25.
-go install github.com/avito-tech/go-mutesting/cmd/go-mutesting@latest
-
-mkdir -p tool-output
-$(go env GOPATH)/bin/go-mutesting --debug ./internal/crypto/... \
-  > tool-output/mutation-crypto.txt 2>&1
-grep -oE 'mutation score is [0-9.]+' tool-output/mutation-crypto.txt | tail -1
-```
-
-**Acceptance:** ≥80% (Existential) / ≥70% (High). Anything below is a Medium finding; triage entries go in `coverage-audit-2026-04-27/gap-backlog.md`. This subsection moves mutation testing from "future work" to "documented release gate."
-
-## Adding New Tests
-
-When a new feature ships:
-
-1. **Add a Part section** in `qa_test.go` following the numbering convention in the Coverage Map below
-2. **API tests**: use `c.get()`, `c.post()`, `c.bodyStr()`, `c.getJSON()`, `c.timedGet()`
-3. **Source checks**: use `fileExists(t, "relative/path")` and `fileContains(t, "path", "substring")`
-4. **DB checks**: use `openQADB(t)` and `db.queryInt(t, "SELECT ...")`
-5. **Cleanup**: always use `t.Cleanup()` for data created during tests
-6. **Skip if external**: use `t.Skip("Requires X — manual test")` with a clear reason
-
-## Version History
-
-- **v1.3** (April 2026, post-Bundle-P) — QA Doc Strengthening shipped. New top-of-doc Test Suite Health dashboard (regenerated via `make qa-stats`). New Coverage by Risk Class table after the Coverage Map. New Release Day Sign-Off Matrix and Mutation Testing Targets sections. CI seed-count + Part-count drift guards land in `.github/workflows/ci.yml` so future doc drift fails CI. Bundle P closes M-007 / M-010 / M-011 / M-012 (structural strengthening) + M-008 (Mutation Testing Targets).
-- **v1.2** (April 2026, post-coverage-audit) — Documented Parts 55–56 (I-004 Agent Soft-Retirement, I-005 Notification Retry & Dead-Letter) and surfaced Parts 23–24 (S/MIME & EKU; OCSP/CRL) as not-yet-automated. 56 Parts total in `testing-guide.md`; 49 live `Part_*` automation wrappers in `qa_test.go` + 4 new `Skip` stubs for Parts 23/24/55/56 = 53 wrappers (Parts 15–17 remain covered by source-checks in Parts 42–46). Reconciled seed-data section to actual `seed_demo.sql` counts (12 agents, 13 issuers; certs were already accurate at 32). Bundle I of the 2026-04-27 coverage-audit closure plan.
-- **v1.1** (April 2026) — Added Parts 53–54 (M47: Kubernetes Secrets target + AWS ACM PCA issuer). 54 Parts total, ~164 automated subtests.
-- **v1.0** (April 2026) — Initial release covering all 52 Parts of testing-guide.md v2.1. Replaces `qa-smoke-test.sh`.

docs/contributor/release-sign-off.md

@@ -1,93 +0,0 @@
-# Release Sign-Off
-
-> Last reviewed: 2026-05-05
-
-Release-day checklist for tagging a new certctl release. Walks through the gates that must be green before pushing the tag, in the order they should be verified.
-
-## Pre-release: code state
-
-| Gate | How to check | Pass |
-|---|---|---|
-| `master` is at the commit you intend to tag | `git log -1 --format='%H %s'` | ☐ |
-| Working tree clean | `git status -sb` | ☐ |
-| Local matches GitHub | `curl -sS https://api.github.com/repos/certctl-io/certctl/commits/master \| grep -oE '"sha": "[a-f0-9]+"' \| head -1` matches local | ☐ |
-| `WORKSPACE-CHANGELOG.md` updated with the release's milestones | manual review | ☐ |
-| `certctl/CHANGELOG.md` updated (release-facing) | manual review | ☐ |
-| Migration ladder ends cleanly | `ls migrations/*.up.sql \| sort \| tail -3` shows the right last migration | ☐ |
-
-## Pre-release: automated gates (CI)
-
-| Gate | How to check | Pass |
-|---|---|---|
-| CI pipeline green on the tag-target commit | GitHub Actions web UI | ☐ |
-| `make verify` clean locally | run from repo root | ☐ |
-| `go test -race -count=1 ./...` clean | full race check | ☐ |
-| `golangci-lint run ./...` clean | local lint | ☐ |
-| `govulncheck ./...` clean | vulnerability scan | ☐ |
-| Coverage thresholds met (service ≥55%, handler ≥60%, domain ≥40%, middleware ≥30%) | `go test -coverprofile=cover.out ./... && go tool cover -func=cover.out` | ☐ |
-| Frontend type-check + Vitest + Vite build clean | `cd web && npm run typecheck && npm run test && npm run build` | ☐ |
-
-## Pre-release: manual QA passes
-
-| Surface | Checklist | Pass |
-|---|---|---|
-| Local stack boots clean from scratch | `qa-prerequisites.md` Steps 1-4 green | ☐ |
-| GUI QA checklist | `gui-qa-checklist.md` end to end | ☐ |
-| End-to-end test environment | `test-environment.md` Steps 1-14 green | ☐ |
-| Performance baselines | `performance-baselines.md` four spot checks within bounds | ☐ |
-| Helm chart deploys clean | `helm-deployment.md` install + verify | ☐ |
-| ACME server interop (cert-manager) | `make acme-cert-manager-test` green | ☐ |
-| ACME server RFC conformance (lego) | `make acme-rfc-conformance-test` green | ☐ |
-
-## Release artefact verification
-
-After the release workflow runs (triggered by tag push), verify the published artefacts:
-
-| Artefact | How to verify | Pass |
-|---|---|---|
-| Cosign keyless OIDC signature on `checksums.txt` | per `docs/reference/release-verification.md` step 2 | ☐ |
-| SLSA Level 3 provenance on each binary | step 3 | ☐ |
-| Container image signature + SBOM + provenance | step 4 | ☐ |
-| Release notes published on GitHub Releases page | manual review | ☐ |
-| ghcr.io images at `ghcr.io/certctl-io/certctl-{server,agent}:<tag>` pullable | `docker pull` round-trips | ☐ |
-
-## Branch protection + tag push
-
-| Gate | How to check | Pass |
-|---|---|---|
-| `master` branch protection rule allows the tag push | Repository Settings → Branches | ☐ |
-| Tag pushed | `git tag -s v<version> -m 'Release v<version>'; git push origin v<version>` | ☐ |
-| Release workflow kicked off in GitHub Actions | watch the Actions tab | ☐ |
-
-## Post-release
-
-| Gate | How to check | Pass |
-|---|---|---|
-| Release workflow completed without errors | GitHub Actions | ☐ |
-| Sample binary downloaded and Cosign-verified by an operator who is not the release author | another team member | ☐ |
-| `WORKSPACE-CHANGELOG.md` notes the tag commit SHA | manual edit | ☐ |
-| workspace-tracking "Active Focus" → "Current tag" updated | manual edit | ☐ |
-| `certctl.io/index.html` star count + `data-gh-version` rendering picks up the new tag | open the landing page in 6+ hours (cache TTL) | ☐ |
-| Reddit / Hacker News / LinkedIn announcement drafted (if a major release) | per the operator's promotion playbook | ☐ |
-
-## If a gate fails
-
-Revert the tag push immediately:
-
-```bash
-git push --delete origin v<version>
-git tag -d v<version>
-```
-
-Investigate, fix, re-tag.
-
-## Related docs
-
-- [`docs/contributor/qa-prerequisites.md`](qa-prerequisites.md) — local stack prereqs
-- [`docs/contributor/test-environment.md`](test-environment.md) — full local environment tutorial
-- [`docs/contributor/gui-qa-checklist.md`](gui-qa-checklist.md) — GUI manual QA pass
-- [`docs/contributor/testing-strategy.md`](testing-strategy.md) — what we test in CI vs deep-scan vs manual QA
-- [`docs/contributor/ci-pipeline.md`](ci-pipeline.md) — CI shape and regression guards
-- [`docs/operator/performance-baselines.md`](../operator/performance-baselines.md) — performance regression spot checks
-- [`docs/operator/helm-deployment.md`](../operator/helm-deployment.md) — Helm install + verify
-- [`docs/reference/release-verification.md`](../reference/release-verification.md) — Cosign / SLSA / SBOM verification procedure

docs/contributor/testing-strategy.md

@@ -1,200 +0,0 @@
-# certctl Testing Strategy & Deep-Scan Operator Runbook
-
-> Last reviewed: 2026-05-05
-
-This doc covers the **testing topology** (per-PR fast gates vs. daily deep-scan
-gates), and the **operator runbook** for re-running each deep-scan tool locally
-when the CI receipt is ambiguous or when an operator wants to validate a fix
-before the next scheduled scan.
-
-For the manual end-to-end QA playbook, see [`testing-guide.md`](../testing-guide.md).
-For the security posture / per-finding closure log, see [`security.md`](../operator/security.md).
-
-## CI workflow split
-
-certctl runs two GitHub Actions workflows:
-
-- **`.github/workflows/ci.yml`** — runs on every push/PR. Fast feedback only.
-  Includes `gofmt`, `go vet`, `golangci-lint`, `go test -short -count=1`,
-  `govulncheck`, the per-layer coverage gates, and the regression-grep guards
-  (the M-009 mutation budget, the L-001 InsecureSkipVerify guard, the H-001
-  Dockerfile SHA-pin guard, the M-012 USER-directive guard, etc.).
-- **`.github/workflows/security-deep-scan.yml`** — runs daily 06:00 UTC and on
-  manual dispatch. Heavyweight tools that need docker, network egress to
-  scanner registries, or wall-clock budgets the per-PR check can't tolerate.
-  Includes `gosec`, `osv-scanner`, the `-race -count=10` full-suite run,
-  `trivy` image scan, `syft` SBOM, ZAP baseline DAST, `nuclei`,
-  `schemathesis` OpenAPI fuzz, `testssl.sh`, `go-mutesting` mutation testing,
-  and `semgrep p/react-security`.
-
-Receipts from each scheduled run are uploaded as a 30-day-retention artefact
-named `security-deep-scan-<run-id>`. Audit them via the GitHub Actions UI;
-download the artefact zip for any scan that surfaces a finding.
-
-## Operator runbook — local re-run procedures
-
-These are the same commands the workflow runs, intended for an operator with
-a workstation that has docker + the Go toolchain installed. The local-run
-shape is identical to CI; the difference is wall-clock and the artefact
-location (CI uploads; local writes to `$PWD`).
-
-### Mutation testing (D-003)
-
-**Tool:** [`go-mutesting`](https://github.com/zimmski/go-mutesting). Mutates
-each AST node in turn (flips comparisons, swaps return values, removes
-statements) and re-runs the package's tests. A mutant is **killed** if any
-test fails; **surviving** mutants indicate a coverage gap (no test caught
-the bug the mutant introduced).
-
-**Targets:** the three security-critical packages whose coverage gate is
-**85%** in `ci.yml`:
-
-- `internal/crypto/`
-- `internal/pkcs7/`
-- `internal/connector/issuer/local/`
-
-**Acceptance threshold:** ≥80% mutation kill ratio per package. Surviving
-mutants below that threshold get triaged in
-the project's 2026-04-25 mutation-results notes — either
-ship a targeted unit test that kills the mutant, or document an
-equivalent-mutation justification.
-
-**Local run:**
-
-```
-go install github.com/zimmski/go-mutesting/cmd/go-mutesting@latest
-for pkg in ./internal/crypto/... ./internal/pkcs7/... ./internal/connector/issuer/local/...; do
-  echo "=== $pkg ==="
-  $(go env GOPATH)/bin/go-mutesting "$pkg"
-done
-```
-
-The tool prints one line per mutant (`PASS` = killed, `FAIL` = surviving)
-plus a per-package summary `The mutation score is X.YZ`. CPU-bound, single
-core, takes ~10 minutes on a 2024-era laptop for the three packages combined.
-
-**Sandbox note:** `go-mutesting` writes a mutant copy of the source tree to
-`/tmp/go-mutesting/` per run; needs ≥2 GB free disk. Sandboxed CI runners
-are sized for this; constrained dev sandboxes are not.
-
-### DAST baseline (D-004)
-
-**Tool:** [OWASP ZAP `baseline`](https://www.zaproxy.org/docs/docker/baseline-scan/).
-Spiders the running server's URL surface and runs the OWASP-ZAP active+passive
-rule pack. **Baseline** mode skips the destructive active-scan rules; it's safe
-against a non-throwaway environment.
-
-**Target:** the live `deploy/docker-compose.yml` stack on `https://localhost:8443`.
-
-**Acceptance:** zero HIGH/CRITICAL alerts. WARN/INFO alerts get triaged in the
-ZAP report; some are unavoidable (e.g., HSTS preload-list nag is a deployment
-recommendation, not a server defect).
-
-**Local run:**
-
-```
-docker compose -f deploy/docker-compose.yml up -d
-sleep 20  # wait for /ready to flip OK; check `curl --cacert deploy/test/certs/ca.crt https://localhost:8443/ready`
-docker run --rm --network host \
-  -v "$PWD":/zap/wrk \
-  ghcr.io/zaproxy/zaproxy:stable \
-  zap-baseline.py -t https://localhost:8443 \
-  -r zap-report.html -J zap-report.json
-docker compose -f deploy/docker-compose.yml down
-```
-
-The HTML report opens in a browser; the JSON is machine-readable for triage.
-
-### TLS audit (D-005)
-
-**Tool:** [`testssl.sh`](https://testssl.sh/). Probes the TLS handshake and
-each enabled cipher suite; reports protocol-version weaknesses, cipher
-weaknesses, certificate-chain issues, and known CVE patterns (Heartbleed,
-ROBOT, BEAST, etc.).
-
-**Target:** the live stack on `https://localhost:8443`.
-
-**Acceptance:** zero HIGH/CRITICAL findings. certctl pins
-`tls.Config.MinVersion = tls.VersionTLS13` (`cmd/server/tls.go`), so anything
-that surfaces is either (a) a real defect, (b) a testssl false positive, or
-(c) a deployment-config issue worth documenting in the operator runbook.
-
-**Local run:**
-
-```
-docker compose -f deploy/docker-compose.yml up -d
-sleep 20
-docker run --rm --network host \
-  -v "$PWD":/data \
-  drwetter/testssl.sh:latest \
-  --jsonfile /data/testssl.json https://localhost:8443
-docker compose -f deploy/docker-compose.yml down
-
-# Filter to actionable severities
-jq '[.scanResult[] | select(.severity == "HIGH" or .severity == "CRITICAL")]' testssl.json
-```
-
-### Frontend semgrep (D-007)
-
-**Tool:** [`semgrep`](https://semgrep.dev/) with the maintained
-[`p/react-security` ruleset](https://semgrep.dev/p/react-security). Catches
-React-specific XSS / injection patterns: `dangerouslySetInnerHTML` without
-sanitization, `target="_blank"` without `rel="noopener noreferrer"`,
-`href={userInput}`, `eval`, `document.write`, etc.
-
-**Target:** the frontend source tree at `web/src/`.
-
-**Acceptance:** zero findings. Bundle 8 already verified
-`dangerouslySetInnerHTML` count at zero and the `target="_blank"`
-rel-noopener pin via simple grep guards in `ci.yml`; semgrep adds defence
-in depth — it catches escape patterns the greps don't see (e.g.,
-`href={user_input}`, runtime `eval`, `document.write`).
-
-**Local run:**
-
-```
-docker run --rm -v "$PWD":/src returntocorp/semgrep:latest \
-  semgrep --config=p/react-security --json /src/web/src \
-  > semgrep-react.json
-
-# Count findings
-jq '.results | length' semgrep-react.json
-
-# Pretty-print findings
-jq '.results[] | {rule_id: .check_id, path, line: .start.line, message: .extra.message}' semgrep-react.json
-```
-
-If the count is non-zero, every result has a `check_id` (e.g.
-`react.dangerouslySetInnerHTML`) and a `message` describing the escape
-pattern. Triage each: either fix the call site, or — for legitimate edge
-cases — add a `// nosem: <check_id> — <reason>` directive on the
-preceding line.
-
-## Cadence
-
-| Tool                 | Trigger                            | Wall-clock | Owner          |
-|----------------------|------------------------------------|------------|----------------|
-| go-mutesting         | daily deep-scan + manual dispatch  | ~10 min    | maintainers    |
-| ZAP baseline (DAST)  | daily deep-scan + manual dispatch  | ~5 min     | maintainers    |
-| testssl.sh           | daily deep-scan + manual dispatch  | ~3 min     | maintainers    |
-| semgrep react        | daily deep-scan + manual dispatch  | ~1 min     | maintainers    |
-| `make verify`        | every commit (pre-push)            | ~1 min     | every developer |
-| ci.yml fast gates    | every push/PR                      | ~3 min     | every developer |
-
-Re-run any of the deep-scan tools locally when:
-
-- A CI receipt surfaces an unexpected finding and you want to bisect against
-  a local change before pushing.
-- You're cutting a release tag and want belt-and-suspenders evidence beyond
-  the most recent scheduled scan.
-- You're adding a new feature in the relevant surface (crypto code →
-  re-run mutation testing; new HTTP handler → re-run schemathesis + ZAP;
-  new TLS-config knob → re-run testssl).
-
-## Related docs
-
-- [`docs/operator/security.md`](../operator/security.md) — security posture, per-finding closure log.
-- [`docs/testing-guide.md`](../testing-guide.md) — manual end-to-end QA playbook.
-- [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) — per-PR fast gates.
-- [`.github/workflows/security-deep-scan.yml`](../.github/workflows/security-deep-scan.yml) — daily deep-scan gates.
-- [`scripts/install-security-tools.sh`](../scripts/install-security-tools.sh) — Go-host-installed tools (the docker-based tools are not in this script).

docs/getting-started/examples.md

@@ -32,7 +32,7 @@ cp .env.example .env   # Edit with your domain and email
 docker compose up -d
 ```
 
-The full walkthrough — including how HTTP-01 challenges work, adding multiple domains, switching to staging for testing, and a production checklist — is in the [example README](../examples/acme-nginx/acme-nginx.md).
+The full walkthrough — including how HTTP-01 challenges work, adding multiple domains, switching to staging for testing, and a production checklist — is in the [example README](../../examples/acme-nginx/acme-nginx.md).
 
 **Migrating from Certbot?** certctl discovers your existing `/etc/letsencrypt/live/` certificates automatically. You keep your ACME account, disable the Certbot cron, and certctl takes over renewal with centralized visibility and deployment verification. The step-by-step process is in [Migrating from Certbot](../migration/from-certbot.md).
 
@@ -52,7 +52,7 @@ cp .env.example .env   # Edit with domain, email, DNS provider credentials
 docker compose up -d
 ```
 
-The full walkthrough — including DNS-PERSIST-01 (set a TXT record once, never touch DNS again on renewals), adapting scripts for other providers, and propagation troubleshooting — is in the [example README](../examples/acme-wildcard-dns01/acme-wildcard-dns01.md).
+The full walkthrough — including DNS-PERSIST-01 (set a TXT record once, never touch DNS again on renewals), adapting scripts for other providers, and propagation troubleshooting — is in the [example README](../../examples/acme-wildcard-dns01/acme-wildcard-dns01.md).
 
 **Migrating from acme.sh?** Your existing `dns_*` hook scripts are compatible with certctl's DNS-01 — they use the same pattern (shell scripts creating TXT records). The migration guide covers script adaptation, discovery of existing acme.sh certificates, and phasing out the acme.sh cron. See [Migrating from acme.sh](../migration/from-acmesh.md).
 
@@ -71,7 +71,7 @@ cd examples/private-ca-traefik
 docker compose up -d    # Self-signed mode (no .env needed for demo)
 ```
 
-The full walkthrough — including sub-CA setup with `CERTCTL_CA_CERT_PATH` and `CERTCTL_CA_KEY_PATH`, creating certificates via the API, monitoring deployments, and production hardening — is in the [example README](../examples/private-ca-traefik/private-ca-traefik.md).
+The full walkthrough — including sub-CA setup with `CERTCTL_CA_CERT_PATH` and `CERTCTL_CA_KEY_PATH`, creating certificates via the API, monitoring deployments, and production hardening — is in the [example README](../../examples/private-ca-traefik/private-ca-traefik.md).
 
 ---
 
@@ -88,7 +88,7 @@ cd examples/step-ca-haproxy
 docker compose up -d
 ```
 
-The full walkthrough — including step-ca provisioner configuration, integrating with an existing step-ca instance, HAProxy PEM format details, and advanced features (approval workflows, policy-based renewal, multi-instance HAProxy) — is in the [example README](../examples/step-ca-haproxy/step-ca-haproxy.md).
+The full walkthrough — including step-ca provisioner configuration, integrating with an existing step-ca instance, HAProxy PEM format details, and advanced features (approval workflows, policy-based renewal, multi-instance HAProxy) — is in the [example README](../../examples/step-ca-haproxy/step-ca-haproxy.md).
 
 ---
 
@@ -105,7 +105,7 @@ cd examples/multi-issuer
 docker compose up -d
 ```
 
-The full walkthrough — including profile-based issuer assignment, testing with ACME staging, Local CA enterprise sub-CA mode, and scaling beyond Docker Compose — is in the [example README](../examples/multi-issuer/multi-issuer.md).
+The full walkthrough — including profile-based issuer assignment, testing with ACME staging, Local CA enterprise sub-CA mode, and scaling beyond Docker Compose — is in the [example README](../../examples/multi-issuer/multi-issuer.md).
 
 **Using cert-manager for Kubernetes?** certctl complements cert-manager — cert-manager handles in-cluster certs, certctl handles everything outside: VMs, bare metal, network appliances, Windows servers. They can share the same CA (ACME, step-ca, Vault PKI). See [certctl for cert-manager Users](../migration/cert-manager-coexistence.md).
 

docs/getting-started/why-certctl.md

@@ -117,7 +117,7 @@ cd certctl/deploy && docker compose up -d
 # Dashboard at https://localhost:8443 (self-signed cert — pin deploy/test/certs/ca.crt)
 ```
 
-See the [Quickstart Guide](quickstart.md) for a full walkthrough, or explore the [5 turnkey examples](../examples/) for specific scenarios (ACME+NGINX, wildcard DNS-01, private CA+Traefik, step-ca+HAProxy, multi-issuer).
+See the [Quickstart Guide](quickstart.md) for a full walkthrough, or explore the [5 turnkey examples](../../examples/) for specific scenarios (ACME+NGINX, wildcard DNS-01, private CA+Traefik, step-ca+HAProxy, multi-issuer).
 
 ## License
 

docs/history-normalization.md

@@ -0,0 +1,97 @@
+# Git history normalization — 2026-05-13
+
+> Last reviewed: 2026-05-13
+
+This page documents a one-time normalization of certctl's git history
+that landed on `master` on 2026-05-13. If you are reading this because
+your clone failed to fast-forward, or because a commit SHA you bookmarked
+no longer resolves, this is the explanation.
+
+## What changed
+
+Every commit's `author` and `committer` metadata was rewritten to a
+single canonical identity (`shankar0123 <skreddy040@gmail.com>`). The
+14 pre-rewrite author identities — operator name variants plus
+AI/automation identities (Claude, Copilot, cowork agent, certctl-bot,
+etc.) — collapsed to that one canonical author.
+
+No source-code content was changed by the rewrite. Every line of code
+in every commit is byte-for-byte identical to its pre-rewrite version.
+Only the `author` and `committer` metadata fields were touched; commit
+messages, subject lines, milestone IDs (M49, L-1, etc.), and every
+other line of every commit's body are preserved verbatim.
+
+## Why
+
+Two reasons:
+
+1. **LLC ownership transfer.** The codebase is now legally owned by
+   **certctl LLC**, which the operator incorporated to hold rights in
+   the project. The BSL 1.1 Licensor field in `LICENSE` flipped from a
+   natural-person name to `certctl LLC` in the same change set. Uniform
+   per-commit authorship under one canonical operator identity makes
+   the chain of title between the codebase and the LLC unambiguous.
+
+2. **Pre-traction cleanup.** The rewrite cost of git-history
+   normalization scales with how many external clones and references
+   have calcified against specific commit SHAs. Doing it now, before
+   the project has a large external surface, minimizes disruption to
+   downstream consumers.
+
+## What is preserved
+
+A complete off-platform bundle backup of the pre-rewrite tree is held
+by the operator (off-repo, not pushed). It contains every original
+commit SHA, every original author identity, and the full ref graph as
+it existed before the rewrite. The bundle is the immutable
+preservation record and is recoverable forever.
+
+An `archive/pre-author-normalization-2026-05-13` tag briefly existed
+on origin pointing at the pre-rewrite tip but was removed when the
+operator opted to clean the contributor graph of pre-rewrite
+authorship signal. The bundle remains as the canonical archive — any
+forensic question about pre-rewrite state can be answered by loading
+the bundle into a fresh clone (`git clone pre-rewrite-2026-05-13.bundle`).
+
+## Recovering after the rewrite
+
+If you had a clone of certctl from before 2026-05-13, your local
+history diverged from origin's at the rewrite. Easiest recovery:
+
+```bash
+cd certctl
+git fetch origin
+git fetch origin --tags
+git reset --hard origin/master
+```
+
+This force-aligns your local tree with the new origin. Any local
+branches you had based on pre-rewrite history will need rebasing onto
+the new master.
+
+If you need to inspect the pre-rewrite state for a forensic or
+diligence question, contact the operator directly — the off-platform
+bundle is the canonical archive and is available on request.
+
+## Container images and release tarballs
+
+ghcr.io container images that were published before the rewrite
+(`ghcr.io/certctl-io/certctl-{server,agent}:<old-tag>`) remain pullable
+indefinitely. Their OCI source-SHA labels reference commit SHAs that
+no longer resolve in the public origin — the images themselves still
+work; only the source-SHA back-reference is now orphan. New release
+images published after the rewrite reference current SHAs normally.
+
+If you downloaded a release tarball before the rewrite, the tarball's
+contents are unchanged; only its associated `git` SHA differs from the
+current `v2.x.y` tag (which has been re-pointed to the rewritten
+commit at the same logical point in history).
+
+## Operational note for contributors
+
+Future contributions to certctl should be authored under the
+operator's canonical git identity. Pull requests from external
+contributors will need a Contributor License Agreement (CLA) workflow,
+which the project will set up before accepting external PRs. Until
+then, the project does not solicit or accept external code
+contributions.

docs/migration/acme-from-cert-manager.md

@@ -16,7 +16,7 @@ through cert-manager 1.15+. Target audience: Kubernetes operator who
 has never deployed certctl before and wants a working
 `Certificate` → `Secret` flow on their cluster in under 30 minutes.
 
-The Phase 5 integration test (`make acme-cert-manager-test`) automates
+The cert-manager integration test (`make acme-cert-manager-test`) automates
 exactly the recipe below. The YAML snippets in this doc are byte-equal
 to the files under `deploy/test/acme-integration/` — re-running the
 test from a fresh clone produces the same results documented here.
@@ -24,7 +24,7 @@ test from a fresh clone produces the same results documented here.
 ## Prereqs
 
 - A Kubernetes cluster (kind / k3d / EKS / GKE / AKS / on-prem). For
-  local trial, `kind v0.20+` works exactly the way the Phase 5 test
+  local trial, `kind v0.20+` works exactly the way the integration test
   uses it. The kind config lives at
   [`deploy/test/acme-integration/kind-config.yaml`](../deploy/test/acme-integration/kind-config.yaml).
 - `kubectl` v1.27+, `helm` v3.13+.
@@ -37,7 +37,7 @@ test from a fresh clone produces the same results documented here.
 
   which is the same idempotent installer the integration test uses.
 - A certctl Helm chart published to a registry your cluster can pull
-  from. The Phase 5 test uses an `image.tag=test` placeholder; production
+  from. The integration test uses an `image.tag=test` placeholder; production
   deployments use the actual image tag for your release line.
 
 ## Step 1 — Deploy certctl-server
@@ -99,7 +99,7 @@ recipe lives in
 ## Step 4 — Apply the ClusterIssuer
 
 ```yaml
-# Phase 5 — sample ClusterIssuer for the certctl trust_authenticated
+# sample ClusterIssuer for the certctl trust_authenticated
 # auth mode (RFC 8555 §6 + certctl auth_mode=trust_authenticated, where
 # the JWS-authenticated ACME account is trusted to issue any identifier
 # the profile policy permits — no per-identifier ownership challenges).
@@ -169,7 +169,7 @@ HTTP-01 to work.
 ## Step 5 — Apply the Certificate
 
 ```yaml
-# Phase 5 — Certificate resource the integration test applies and
+# Certificate resource the integration test applies and
 # waits for. The certctl-test-trust ClusterIssuer (trust_authenticated
 # mode) issues the cert without any solver round-trip; the resulting
 # Secret 'test-com-tls' is asserted to carry tls.crt + tls.key.
@@ -262,4 +262,4 @@ helm uninstall certctl-test
 - [`docs/acme-traefik-walkthrough.md`](./acme-from-traefik.md) —
   Traefik-side recipe.
 - [`deploy/test/acme-integration/`](../deploy/test/acme-integration/) —
-  Phase 5 integration test (the same recipe, automated).
+  cert-manager integration test (the same recipe, automated).

docs/migration/api-keys-to-rbac.md

@@ -0,0 +1,294 @@
+# Migrating API keys to RBAC (v2.0.x → v2.1.0)
+
+> Last reviewed: 2026-05-09
+
+This is the upgrade guide for an existing certctl deployment moving
+from v2.0.x's "every API key is admin or not" model to v2.1.0's
+RBAC primitive. Everything keeps working through the upgrade - the
+migration backfills every existing API key to the
+`r-admin` role on first boot, so the pre-existing automation that
+was using those keys does not change behavior. **However**, most
+keys do not need full admin power; this guide walks the operator
+through the post-upgrade scope-down flow.
+
+## ⚠️ SECURITY: AUDIT YOUR API KEYS
+
+v2.1.0 maps **every** existing `CERTCTL_API_KEYS_NAMED` entry
+(and every legacy `CERTCTL_AUTH_SECRET`-synthesized key) to the
+`r-admin` role on the first boot after migration 000029 applies.
+This is the safe-for-back-compat default - your CI / agents / scripts
+keep working without changes - but if you don't downgrade keys, every
+key in your fleet has full admin permissions including bulk-revoke,
+CRL admin, and CA hierarchy management.
+
+**Run the scope-down flow before tagging the next release.** The
+release notes for v2.1.0 lead with this callout for a reason.
+
+## Upgrade flow
+
+### 1. Apply the migration
+
+The migration runner is idempotent. Re-applying is a no-op if the
+schema is already at the target version. The five RBAC migrations
+that ship in v2.1.0:
+
+| Migration | What it does |
+|---|---|
+| `000029_rbac.up.sql` | Creates `tenants`, `roles`, `permissions`, `role_permissions`, `actor_roles`. Seeds 7 default roles + 33-permission catalogue + the synthetic `actor-demo-anon` admin grant. Backfills every named API key into `actor_roles` with the `r-admin` role. |
+| `000030_rbac_admin_perms.up.sql` | Seeds 5 admin-only fine-grained permissions (`cert.bulk_revoke`, `crl.admin`, `scep.admin`, `est.admin`, `ca.hierarchy.manage`) into `r-admin` only. |
+| `000031_api_keys.up.sql` | Creates the `api_keys` table for runtime-minted keys (day-0 bootstrap path). |
+| `000032_audit_category.up.sql` | Adds `event_category` column to `audit_events` with the closed enum (`cert_lifecycle` / `auth` / `config`). |
+| `000033_approval_kinds.up.sql` | Adds `approval_kind` + `payload` to `issuance_approval_requests` for the approval-bypass closure. |
+
+The v2.1.0 server applies these on first boot. No operator
+action is required other than running the upgrade.
+
+### 2. Verify the backfill landed
+
+```bash
+# Inspect the seeded actor_roles rows. You should see one row per
+# entry in CERTCTL_API_KEYS_NAMED (Admin=true keys → r-admin,
+# Admin=false keys → r-viewer) plus the seeded actor-demo-anon
+# admin row.
+psql -d certctl -c "SELECT actor_id, role_id, granted_by, granted_at FROM actor_roles ORDER BY granted_at;"
+```
+
+If the table is empty, the boot-loader hook in
+`cmd/server/auth_backfill.go::backfillNamedKeyActorRoles` did not
+run; re-check that `CERTCTL_AUTH_TYPE` is `api-key` (the boot
+hook is gated on `cfg.Auth.Type != none`).
+
+### 3. List + scope-down keys
+
+The `certctl-cli` ships a four-mode scope-down command. Pick the
+mode that matches your fleet size + automation posture.
+
+#### Interactive walk
+
+```bash
+certctl-cli auth keys scope-down
+```
+
+Walks every actor (skips the synthetic `actor-demo-anon`) and
+prompts for a target role. Empty input keeps the existing role.
+Type one of `admin`, `operator`, `viewer`, `agent`, `mcp`, `cli`,
+`auditor` to replace.
+
+#### Non-interactive JSON config (Helm post-upgrade hook)
+
+```bash
+cat > scope-down.json <<EOF
+{
+  "ci-bot":         "operator",
+  "agent-prod-1":   "agent",
+  "agent-prod-2":   "agent",
+  "monitoring-bot": "viewer",
+  "compliance-bot": "auditor"
+}
+EOF
+
+certctl-cli auth keys scope-down --non-interactive ./scope-down.json
+```
+
+Empty role values revoke every current grant WITHOUT granting a
+replacement; assign roles selectively with
+`certctl-cli auth keys assign`.
+
+#### Audit-driven suggestion
+
+```bash
+# Preview suggestions based on the last 30 days of audit history
+certctl-cli auth keys scope-down --suggest
+
+# Apply the suggestions
+certctl-cli auth keys scope-down --suggest --apply
+```
+
+The classifier (pure function in `internal/cli/auth_scope_down.go::SuggestRoleFromAuditEvents`)
+walks the actor's audit events and emits one of:
+
+| Suggestion | Trigger |
+|---|---|
+| `admin` | Any auth.role.* / auth.key.* / ca.hierarchy.* / *.bulk_revoke / *.admin action |
+| `mcp` | All observed actions are MCP-shaped (`mcp.*`) |
+| `viewer` | All observed actions are read-only (`*.read` or `*.list`) |
+| `agent` | All observed actions are agent-shaped (`agent.*`, `cert.read`, `cert.issue`) |
+| `operator` | Cert / profile / target lifecycle mutations without admin signals |
+
+The classifier is conservative - when in doubt, it prefers the
+narrower role. The operator confirms each suggestion before any
+mutation lands (unless `--apply` is set).
+
+### 4. Mint a fresh admin via bootstrap (optional, for fresh deployments)
+
+If you're standing up a fresh deployment instead of upgrading an
+existing one, the bootstrap path mints the first admin key without
+needing the operator to know the env-var format:
+
+```bash
+# Set the bootstrap token in the server environment.
+export CERTCTL_BOOTSTRAP_TOKEN=$(openssl rand -hex 32)
+
+# Boot the server. Logs include "bootstrap endpoint enabled".
+docker compose up -d
+
+# Mint the first admin key.
+curl -X POST $URL/api/v1/auth/bootstrap \
+  -H 'Content-Type: application/json' \
+  -d '{"token":"'$CERTCTL_BOOTSTRAP_TOKEN'","actor_name":"first-admin"}'
+```
+
+The response carries the plaintext `key_value` once. Capture it
+and use it as the Bearer token for subsequent calls. Subsequent
+bootstrap calls return HTTP 410 Gone.
+
+See [`docs/operator/rbac.md`](../operator/rbac.md) for the full
+bootstrap flow + the threat model.
+
+## What changes for code that called `IsAdmin`
+
+In v2.0.x, the five admin handlers checked `auth.IsAdmin(ctx)`
+directly in the body. v2.1.0 moved those checks to
+the router via the `auth.RequirePermission` middleware (wrapped
+through the `rbacGate` helper in
+`internal/api/router/router.go`). The behavior contract is
+unchanged: `r-admin`-roled callers reach the handler, anyone else
+gets HTTP 403 BEFORE the body runs.
+
+If your code consumed `auth.IsAdmin` directly (it shouldn't - 
+the helper is internal), the new convention is:
+
+1. Wrap the route in `rbacGate(reg.Checker, "<perm>", handler)`
+   in `router.go`.
+2. Add the perm to `migrations/000030_rbac_admin_perms.up.sql`
+   (or `migrations/000029_rbac.up.sql`'s catalogue).
+3. Grant the perm to the right default roles.
+
+The five admin-only fine-grained perms stay on `r-admin` only by
+default. Operators delegate by creating custom roles with the
+specific perm.
+
+## Helm-specific upgrade
+
+The certctl Helm chart applies migrations on container start via
+the standard migrations runner. No chart changes are required;
+the `helm upgrade` command runs identically:
+
+```bash
+helm upgrade certctl certctl/certctl \
+  --version <new-version> \
+  --reuse-values
+```
+
+Post-upgrade, the boot loader runs the named-key actor-role
+backfill against the `CERTCTL_API_KEYS_NAMED` env-var-injected
+into the deployment. The "AUDIT YOUR API KEYS" callout applies - 
+add a post-upgrade Job to your release pipeline that runs
+`certctl-cli auth keys scope-down --non-interactive` against a
+checked-in JSON config, so the role narrowing is deterministic
+across upgrade rollouts.
+
+Example post-upgrade Job:
+
+```yaml
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: certctl-scope-down
+spec:
+  template:
+    spec:
+      containers:
+ - name: scope-down
+        image: ghcr.io/certctl-io/certctl-cli:<tag>
+        command:
+ - certctl-cli
+ - auth
+ - keys
+ - scope-down
+ - --non-interactive
+ - /config/scope-down.json
+        envFrom:
+ - secretRef:
+              name: certctl-cli-credentials
+        volumeMounts:
+ - name: scope-down-config
+            mountPath: /config
+      volumes:
+ - name: scope-down-config
+          configMap:
+            name: certctl-scope-down-config
+      restartPolicy: OnFailure
+```
+
+The ConfigMap holds the `{actor_id: role_id}` map; the Secret
+holds the API key the Job uses to call `/v1/auth/keys/.../roles`.
+
+## Docker Compose-specific upgrade
+
+For `deploy/docker-compose.yml` deployments:
+
+1. Pull the new images: `docker compose pull`
+2. Verify your `CERTCTL_AUTH_TYPE` value before restarting. If it
+   was `none` (the demo path), the post-upgrade server will boot
+   in demo mode again - the synthetic `actor-demo-anon` admin
+   covers every request, no scope-down is meaningful. If you're
+   moving from `none` to `api-key` mode, set
+   `CERTCTL_API_KEYS_NAMED` first, then restart.
+3. `docker compose up -d` to apply.
+4. `docker compose logs certctl-server | grep -i 'loaded persisted api_keys'`
+   to verify the boot loader ran. The first-boot log line includes
+   the count of keys loaded into the runtime keystore.
+5. Run `certctl-cli auth keys scope-down` against the running
+   server.
+
+The five examples in `examples/` (acme-nginx, private-ca-traefik,
+step-ca-haproxy, multi-issuer, acme-wildcard-dns01) all run in
+demo mode (`CERTCTL_AUTH_TYPE=none`) and are unaffected by the
+RBAC migration - the synthetic actor-demo-anon admin grant covers
+every request.
+
+## Verifying the upgrade landed
+
+After the scope-down flow completes:
+
+1. `certctl-cli auth me` while authenticated as each named key
+   confirms the right `effective_permissions` for that role.
+2. `psql -c "SELECT actor_id, array_agg(role_id ORDER BY role_id) FROM actor_roles GROUP BY actor_id;"`
+   gives the full picture in one query.
+3. The audit trail
+   (`GET /api/v1/audit?category=auth`)
+   shows the `auth.role.assign` and `auth.role.revoke` rows for
+   every change you made - confirm via the GUI's
+   `/audit?category=auth` view.
+4. Read the updated [`docs/operator/rbac.md`](../operator/rbac.md)
+   for day-2 RBAC management.
+
+## Rollback
+
+If the upgrade goes wrong, the down migrations exist in lockstep:
+
+```bash
+# Roll back via your migration runner (golang-migrate, Atlas, etc.).
+# Migrations 000029-000033 each have a .down.sql that reverses the
+# .up.sql. Down migrations are destructive on data added by the up
+# migration (api_keys rows, role grants on actors, profile-edit
+# approvals); take a backup first.
+```
+
+After rollback, the v2.0.x binary works against the v2.0.x
+schema unchanged. The operator's API keys still authenticate (the
+in-memory hash table is rebuilt from `CERTCTL_API_KEYS_NAMED` on
+boot regardless of schema version).
+
+## Cross-references
+
+- [`docs/operator/rbac.md`](../operator/rbac.md) - the operator
+  how-to for the new RBAC primitive
+- [`docs/operator/auth-threat-model.md`](../operator/auth-threat-model.md) - 
+  what the new controls defend against
+- [`docs/reference/profiles.md`](../reference/profiles.md) - the
+  approval-bypass closure on `RequiresApproval` profile edits
+- [`docs/operator/security.md`](../operator/security.md) - the
+  full security posture
+- `CHANGELOG.md` - the v2.1.0 release notes lead with this guide

docs/migration/cert-manager-coexistence.md

@@ -142,6 +142,6 @@ For now: cert-manager handles Kubernetes, certctl handles everything else. They
 ## Next Steps
 
 1. Run through the [Quick Start](../getting-started/quickstart.md) for a 5-minute demo
-2. Try the [Multi-Issuer example](../examples/multi-issuer/multi-issuer.md) — manages public and internal certs from one dashboard
+2. Try the [Multi-Issuer example](../../examples/multi-issuer/multi-issuer.md) — manages public and internal certs from one dashboard
 3. Explore [Architecture](../reference/architecture.md#agents) for deployment patterns
 4. Check the [Helm Chart](../deploy/helm/certctl/) for production Kubernetes deployment

docs/migration/from-acmesh.md

@@ -271,7 +271,7 @@ certctl automatically falls back to DNS-01 if the CA doesn't support dns-persist
 
 ## Next Steps
 
-- Try the [Wildcard DNS-01 example](../examples/acme-wildcard-dns01/acme-wildcard-dns01.md) — a working docker-compose with Cloudflare hooks you can adapt for your DNS provider
+- Try the [Wildcard DNS-01 example](../../examples/acme-wildcard-dns01/acme-wildcard-dns01.md) — a working docker-compose with Cloudflare hooks you can adapt for your DNS provider
 - See [Connector Reference](../reference/connectors/index.md) for advanced ACME options (EAB, ARI, custom timeouts)
 - See [Discovery Guide](concepts.md#certificate-discovery) for managing discovered certificates at scale
 - See all [Deployment Examples](../getting-started/examples.md) for other scenarios (ACME+NGINX, private CA, step-ca, multi-issuer)

docs/migration/from-certbot.md

@@ -169,7 +169,7 @@ certctl will stop renewing that cert when the policy is disabled. Certbot resume
 
 ## Next Steps
 
-- Try the [ACME + NGINX example](../examples/acme-nginx/acme-nginx.md) — a working docker-compose you can run locally before deploying to production
+- Try the [ACME + NGINX example](../../examples/acme-nginx/acme-nginx.md) — a working docker-compose you can run locally before deploying to production
 - Review the [Concepts Guide](../getting-started/concepts.md) for terminology (profiles, policies, agents, jobs)
 - Explore [Network Discovery](../getting-started/quickstart.md#network-discovery-agentless) to find certificates you didn't know about
 - See all [Deployment Examples](../getting-started/examples.md) for other scenarios (wildcard DNS-01, private CA, step-ca, multi-issuer)

docs/migration/oidc-enable.md

@@ -0,0 +1,261 @@
+# Enable OIDC SSO
+
+> Last reviewed: 2026-05-10
+
+This guide walks an operator already running certctl with API-key auth + RBAC through enabling OIDC SSO. The path is additive: API-key auth keeps working unchanged; OIDC sits alongside as a second authentication surface for human users.
+
+If you are upgrading from a pre-RBAC (v2.0.x) deployment, finish [`api-keys-to-rbac.md`](api-keys-to-rbac.md) first. If you have not deployed certctl at all, start with [`getting-started/quickstart.md`](../getting-started/quickstart.md). For the canonical mental model + per-flow threat coverage, see [`security.md`](../operator/security.md) and [`auth-threat-model.md`](../operator/auth-threat-model.md).
+
+## What "enable OIDC" gives you
+
+After this migration:
+
+- Human operators can log in via the OIDC button on the certctl login page (one button per configured IdP).
+- The IdP authenticates the user; certctl validates the returned ID token, mints a session cookie, and redirects to the dashboard.
+- IdP groups → certctl roles are operator-configured (e.g. `engineering@example.com` → `r-operator`).
+- Every login emits an audit row (`auth.oidc_login_succeeded`) attributing the action to the federated user, NOT to a shared API key.
+- The first user from a configured admin group (when `CERTCTL_BOOTSTRAP_ADMIN_GROUPS` is set) becomes admin per tenant; one-shot per the admin-existence probe.
+
+What does NOT change:
+
+- API keys keep working. Existing automation continues to authenticate via `Authorization: Bearer` exactly as before.
+- The break-glass admin path stays default-OFF.
+- The auditor split + approval workflow + RBAC primitive are unchanged.
+
+## Pre-requisites
+
+**On certctl side:**
+
+- Server build ≥ v2.1.0. Confirm via `curl https://<your-host>:8443/api/v1/version`.
+- `CERTCTL_CONFIG_ENCRYPTION_KEY` set in the server environment. This is the passphrase that encrypts the OIDC `client_secret` at rest. Use a stable, secrets-manager-stored value at least 32 random bytes long. **The server refuses to start if the key is missing AND any source='database' rows already exist** (CWE-311 fail-closed gate). Set this before doing anything else.
+- An admin actor available to drive the configuration. The actor needs the `auth.oidc.create` + `auth.oidc.edit` permissions; `r-admin` carries both by default. Get one via the day-0 bootstrap path if you don't have one yet.
+- HTTPS-only control plane (post-v2.2 milestone — this is the default). The OIDC redirect URI MUST be `https://`.
+
+**On IdP side:**
+
+- A Keycloak / Authentik / Okta / Auth0 / Entra ID / Google Workspace tenant where you can register an OIDC application. Free dev tiers work for evaluation. See the per-IdP runbook at [`oidc-runbooks/index.md`](../operator/oidc-runbooks/index.md).
+- Network reachability from certctl-server to the IdP's `/.well-known/openid-configuration` discovery endpoint. The certctl service fetches discovery + JWKS at provider creation and at every `RefreshKeys` call.
+
+## Step-by-step
+
+### 1. Pin `CERTCTL_CONFIG_ENCRYPTION_KEY`
+
+If your deployment already has it set (the CWE-311 fail-closed gate enforces this for any source='database' issuer/target row), skip this step. If you don't:
+
+```bash
+# Generate a 32-byte random key + base64-encode it.
+openssl rand -base64 32 > /etc/certctl/config-encryption-key
+chmod 600 /etc/certctl/config-encryption-key
+```
+
+Then make the server consume it at boot:
+
+```bash
+# In your environment, systemd unit, k8s Secret, etc.
+export CERTCTL_CONFIG_ENCRYPTION_KEY="$(cat /etc/certctl/config-encryption-key)"
+```
+
+Restart the server. Confirm the boot log does NOT show the `ErrEncryptionKeyRequired` warning. If it does, the server refuses to start because there's pre-existing source='database' material that needs to be re-sealed; see [`docs/operator/security.md`](../operator/security.md) for the re-encryption flow.
+
+### 2. Pick an IdP runbook + complete the IdP-side configuration
+
+Pick the runbook for your IdP and do EVERYTHING in its IdP-side section. The runbooks are at [`docs/operator/oidc-runbooks/`](../operator/oidc-runbooks/index.md). What you need from the runbook before continuing here:
+
+- The IdP's discovery URL (the `iss` value certctl will validate against).
+- An OIDC client ID + client secret. Save the secret; you'll paste it into certctl in step 3.
+- At least one IdP group with the users who should be allowed to log in. The runbook walks the group-claim mapper config.
+- The IdP-side group claim shape — most IdPs emit `string-array` under a `groups` key, but Auth0 uses namespaced URL keys (`https://your-namespace/groups`) and Entra ID emits group OBJECT IDs (GUIDs) instead of names. The runbook calls out the per-IdP shape.
+
+### 3. Configure the certctl-side OIDC provider
+
+Via the GUI (recommended for first-time setup):
+
+1. Sign in as an admin actor.
+2. Navigate to **Auth → OIDC Providers** in the sidebar.
+3. Click **Configure provider**.
+4. Fill in the form using the values from step 2's runbook.
+5. Click **Save**.
+
+If the discovery doc fetch fails, the modal surfaces the error inline. Most-common cause: a typo in the issuer URL.
+
+Or via the CLI / MCP:
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/providers \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "Keycloak",
+    "issuer_url": "https://keycloak.example.com/realms/certctl",
+    "client_id": "certctl",
+    "client_secret": "<paste-the-secret>",
+    "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+    "groups_claim_path": "groups",
+    "groups_claim_format": "string-array",
+    "scopes": ["openid", "profile", "email"],
+    "iat_window_seconds": 300,
+    "jwks_cache_ttl_seconds": 3600
+  }'
+```
+
+The MCP equivalent (`certctl_auth_create_oidc_provider`) accepts the same JSON shape.
+
+### 4. Add the group → role mappings
+
+Empty mapping list = nobody can log in via this provider (the fail-closed contract; pinned by `ErrGroupsUnmapped`). Add at least one mapping BEFORE announcing the SSO endpoint to users.
+
+Via the GUI: **Auth → OIDC Providers → <provider> → Group → role mappings → Add**.
+
+Via the API:
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/group-mappings \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "provider_id": "<provider-id-from-step-3>",
+    "group_name": "engineering@example.com",
+    "role_id": "r-operator"
+  }'
+```
+
+A typical setup adds two or three mappings: `engineers → r-operator`, `viewers → r-viewer`, optionally `admins → r-admin`. For Entra ID, use group object IDs (GUIDs) NOT names; for Auth0, use the bare group name from inside the namespaced claim array.
+
+### 5. (Optional) Configure first-admin bootstrap
+
+If your deployment has no admin actor yet AND you want the first OIDC-authenticated user from a specific group to become admin (instead of using the env-var-token bootstrap path), set:
+
+```bash
+export CERTCTL_BOOTSTRAP_ADMIN_GROUPS=admins
+export CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID=<provider-id-from-step-3>
+```
+
+Restart the server. The first user with the `admins` group claim from that provider becomes admin on login per tenant. Subsequent logins go through normal group-role mapping. Audit row on every grant (`bootstrap.oidc_first_admin`).
+
+If you already have an admin actor (likely — you needed one to run step 3), the bootstrap hook silently falls through to normal mapping; no harm done. The probe is one-shot per tenant and can't double-grant.
+
+### 6. Verify with a single test user
+
+Before announcing the SSO endpoint to your users, verify the full login flow with a test user from your IdP:
+
+1. Open `https://<your-certctl-host>:8443/login` in a fresh incognito window.
+2. The page should render `Sign in with <provider>` button(s) above the API-key form. If not, check that `getAuthInfo` is returning the `oidc_providers` field — `curl https://<your-host>:8443/api/v1/auth/info` should show the configured provider(s).
+3. Click the provider button. The browser redirects to the IdP, you authenticate, and the IdP redirects back. You should land on the certctl dashboard.
+4. Navigate to **Auth → Sessions**. You should see a row with your own actor ID and the current timestamp.
+5. Confirm the audit row:
+
+   ```bash
+   curl https://<your-host>:8443/api/v1/audit?category=auth \
+     -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+     | jq '.events[] | select(.action == "auth.oidc_login_succeeded")'
+   ```
+
+   You should see a row attributed to the federated user with `details.provider_id` matching your configuration.
+
+If any step fails, see the **Troubleshooting** section below.
+
+### 7. Announce the SSO endpoint
+
+Once step 6 passes, the SSO endpoint is operational. Tell your users to log in via `https://<your-host>:8443/login` and click the provider button. API-key auth continues to work for automation; the two paths coexist.
+
+Optional GUI hardening:
+
+- If you want the API-key form hidden once OIDC is configured, the operator can add a frontend feature flag in a follow-on commit. Default behavior keeps both paths visible (the API-key form stays for break-glass + Bearer-mode deploys).
+- If you want to revoke a user's session immediately (e.g. an employee left), use **Auth → Sessions → All actors (admin) → <user> → Revoke**. The next request from that user's browser fails 401.
+
+## Rollback
+
+If you need to disable OIDC:
+
+1. Delete every group-role mapping for the provider:
+   ```bash
+   # GUI: Auth → OIDC Providers → <provider> → Group → role mappings → Remove (each)
+   ```
+2. Delete the OIDC provider:
+   ```bash
+   # GUI: Auth → OIDC Providers → <provider> → Delete (type-confirm-name dialog)
+   ```
+   The server returns HTTP 409 if any user has an authenticated session minted via this provider; revoke those sessions first.
+3. The `Sign in with <provider>` button disappears from the login page on the next `getAuthInfo` round-trip (typically the next page load).
+4. Existing sessions continue to work until idle/absolute expiry. To force-revoke them, **Auth → Sessions → All actors (admin) → revoke each row**.
+
+API-key auth continues to work throughout this rollback; you do not need to re-bootstrap or change any other configuration.
+
+## Troubleshooting
+
+**"Discovery doc fetch failed" at provider creation.**
+The most common cause is a typo in the issuer URL. Curl the URL manually:
+```bash
+curl -v https://<idp-host>/<path>/.well-known/openid-configuration
+```
+If that returns 404, fix the issuer URL.
+
+**"IdP downgrade-attack defense" rejected provider creation.**
+Your IdP advertises HS256/HS384/HS512 or `none` in `id_token_signing_alg_values_supported`. Configure the IdP to advertise only RS256 / RS512 / ES256 / ES384 / EdDSA before re-creating the provider in certctl. The relevant runbook section walks this.
+
+**Login redirects to IdP, user authenticates, but the callback redirects back to `/login` with "no roles assigned".**
+The user authenticated successfully but their groups didn't match any configured mapping (`ErrGroupsUnmapped`). Check:
+- The user is a member of the IdP group you mapped.
+- The group-claim mapper is configured correctly at the IdP (the runbook walks per-IdP).
+- The group name in your certctl mapping exactly matches what the IdP emits — case-sensitive, no leading slash for Keycloak full-path-OFF.
+
+Decode the ID token at jwt.io against the IdP's JWKS to see exactly what's in the `groups` claim.
+
+**`ErrIssuerMismatch` even though the discovery doc looks correct.**
+The `iss` claim in the ID token must match `OIDCProvider.IssuerURL` byte-for-byte. Some IdPs include / omit a trailing slash; check the per-IdP runbook section on `iss` formatting.
+
+**`oidc: pre-login session not found or already consumed`.**
+The user clicked the OIDC login button, then the browser tab idled past the 10-minute pre-login TTL OR the user opened the IdP login in a new tab and consumed the row from the first one. Have them retry from the login page.
+
+**`oidc: state parameter mismatch (replay or forgery)`.**
+Either the user double-submitted a callback URL (clicked it twice from email or browser history), or a CSRF attempt. The pre-login row is single-use; second consumption returns `ErrPreLoginNotFound`. Have them retry from the login page.
+
+**`Sessions revoked but the user can still hit the API.`**
+Check the session contract: the cookie is HMAC-validated on every request, but the actual database row is what `Revoke` deletes. If your reverse proxy is caching the response or the `__Host-certctl_session` cookie wasn't actually cleared on the client, the cookie hits the server's session middleware which returns 401 on the missing-row lookup. The middleware never serves stale data; the issue is upstream of certctl in this case.
+
+**JWKS rotation: an IdP rotated its signing key and existing users start failing login.**
+Click **Refresh discovery cache** on the OIDC provider detail page (or `POST /api/v1/auth/oidc/providers/<id>/refresh`). The certctl service re-fetches discovery + JWKS. New tokens validate immediately. The Keycloak integration test exercises this drill end to end.
+
+**Database row count drift.**
+After OIDC is live, expect to see new rows under:
+- `oidc_providers` (one per configured provider)
+- `group_role_mappings` (one per configured mapping)
+- `users` (one per first OIDC-authenticated user; certctl auto-upserts on login)
+- `sessions` (one per logged-in browser session; idle 1h / absolute 8h GC)
+- `session_signing_keys` (one active + retained-history rows post rotation)
+- `oidc_pre_login_sessions` (transient; 10-minute TTL, scheduler-GC'd)
+
+All ten of these tables are tenant-scoped (`tenant_id` column); single-tenant deployments use the seeded `t-default` tenant.
+
+## What you can do next
+
+- Run [`docs/operator/oidc-runbooks/<your-idp>.md`](../operator/oidc-runbooks/index.md) end to end to fill in the validation checklist + sign-off line.
+- Read [`docs/operator/auth-benchmarks.md`](../operator/auth-benchmarks.md) for the steady-state + cold-cache performance baselines.
+- Review the [`auth-threat-model.md`](../operator/auth-threat-model.md) OIDC + sessions + break-glass sections to understand the failure modes the federated-identity surface defends against.
+- Schedule a rotation reminder for the OIDC `client_secret` (typically 6-12 months; the IdP doesn't auto-rotate it). Edit the provider via the GUI when the time comes; leaving `client_secret` blank in the edit form preserves the existing ciphertext, providing a value rotates.
+
+## `__Host-` cookie rename (BREAKING)
+
+v2.1.0 carries a wire-format change to the three auth cookies: they now carry the `__Host-` prefix. The cookie names are:
+
+- `__Host-certctl_session` (was `certctl_session`)
+- `__Host-certctl_csrf` (was `certctl_csrf`)
+- `__Host-certctl_oidc_pending` (was `certctl_oidc_pending`)
+
+The rename gains browser-enforced subdomain-takeover defense: a `__Host-*` cookie can only be set with `Path=/` + `Secure` + no `Domain` attribute, and the browser rejects any subdomain attempt to overwrite it. The protection is free (the existing cookies already met the prerequisites) but the wire-format change means:
+
+- **Every active session is invalidated by the deploy that lands this change.** Operators see one re-authentication prompt; subsequent logins issue the new `__Host-*`-prefixed cookie.
+- **The pre-login cookie's Path widens from `/auth/oidc/` to `/`** — required by the `__Host-` prefix. The cookie lifetime is unchanged (10 minutes) and is only ever consumed by the callback handler; the wider path scope is harmless.
+- **No operator action required beyond accepting the one-time re-login window.** The GUI's CSRF cookie reader was updated in lockstep; existing bookmarked deep links work without modification.
+
+If you have GUI customizations that read `document.cookie` directly, update them to look for `__Host-certctl_csrf` (the lookup in `web/src/api/client.ts` is the in-tree reference).
+
+## Cross-references
+
+- [`docs/operator/oidc-runbooks/index.md`](../operator/oidc-runbooks/index.md) — per-IdP setup guides.
+- [`docs/operator/security.md`](../operator/security.md) — overall auth surface including this OIDC layer.
+- [`docs/operator/auth-threat-model.md`](../operator/auth-threat-model.md) — threat model.
+- [`docs/operator/auth-benchmarks.md`](../operator/auth-benchmarks.md) — performance baselines.
+- [`docs/reference/auth-standards-implemented.md`](../reference/auth-standards-implemented.md) — RFC + CWE evidence list.
+- `internal/auth/oidc/` — OIDC service implementation.
+- `internal/auth/session/` — session minting + middleware + signing-key rotation.

docs/operator/auth-benchmarks.md

@@ -0,0 +1,162 @@
+# Authentication performance benchmarks
+
+> Last reviewed: 2026-05-10
+
+This document records the four authentication-path performance benchmarks: session validation (steady-state and cold-process) plus OIDC token validation (steady-state and cold-cache). Numbers below are the as-measured baseline at v2.1.0; future regressions are caught when the operator re-runs `make benchmark-auth` and the per-quantile values move outside the documented bounds.
+
+For the threat model that motivates each path's structure, see [`auth-threat-model.md`](auth-threat-model.md). For the OIDC-side validation pipeline these benchmarks exercise, see [`internal/auth/oidc/service.go`](../../internal/auth/oidc/service.go) and [`internal/auth/session/service.go`](../../internal/auth/session/service.go).
+
+## Hardware floor
+
+The numbers below are bounded by this configuration. Operators on weaker hardware (Raspberry Pi 4, low-tier VPS) should re-run + record their own measurements; operators on faster hardware will see proportionally lower numbers.
+
+| Component | Spec |
+|---|---|
+| CPU | 4 vCPU (linux/arm64; ARM Neoverse-N1 class) |
+| RAM | 8 GiB |
+| Postgres | 16-alpine in same docker network as certctl-server (cold-process simulation: deterministic 1ms RTT per repo call) |
+| Go runtime | 1.25.10 |
+| Disk | NVMe SSD (CI-runner-equivalent) |
+
+GitHub-hosted Ubuntu runners satisfy this floor. The baselines below were captured on a `linux/arm64` 4-vCPU sandbox at 2026-05-10.
+
+## Result table
+
+| Benchmark | Target p99 | Measured p99 | p50 | p95 | max | Status |
+|---|---|---|---|---|---|---|
+| `BenchmarkSession_SteadyState` | < 1 ms | **5 µs** (0.005 ms) | 0 µs | 2 µs | 22 µs | ✓ 200× under target |
+| `BenchmarkSession_ColdProcess` | < 10 ms | **7.1 ms** | 2.7 ms | 3.6 ms | 20.6 ms | ✓ within target |
+| `BenchmarkOIDC_SteadyState` | < 5 ms | **1.5 ms** | 1.2 ms | 1.5 ms | 2.6 ms | ✓ 3× under target |
+| `BenchmarkOIDC_ColdCache` | < 200 ms | operator-run | — | — | — | ⚠️ requires Docker; see [Cold-cache OIDC: how to run](#cold-cache-oidc-how-to-run) below |
+
+The three default-tag benchmarks above were captured at v2.1.0; re-run via `make benchmark-auth`. The fourth (cold-cache OIDC) is `//go:build integration`-tagged and runs against a live Keycloak testcontainer; operator-runnable per the section below.
+
+## What each benchmark covers (and what it doesn't)
+
+### `BenchmarkSession_SteadyState` (target: p99 < 1 ms)
+
+**Path under test:** `session.Service.Validate(ctx, ValidateInput{...})`. With:
+
+- In-memory `SessionRepo` (no Postgres round-trip).
+- In-memory `SigningKeyRepo` (no Postgres round-trip).
+- A pre-minted session row for a real `actor-bench`.
+- A real RSA-32-byte HMAC key in the in-memory key store.
+
+**Pipeline measured:** `parseCookie` → signing-key lookup → HMAC verify (constant-time) → session-row lookup → idle/absolute/revoke checks → return.
+
+**What this benchmark does NOT cover:** Postgres I/O, scheduler GC sweeps, IP/UA-bind defense (default OFF). Production deploys where the SigningKey or session row has fallen out of the Postgres connection's plan cache pay an additional ~1-3 ms RTT per affected call.
+
+### `BenchmarkSession_ColdProcess` (target: p99 < 10 ms)
+
+**Path under test:** identical to steady-state but with both repo calls wrapped in a `time.Sleep(1ms)` simulator on every call. The simulator approximates a typical local-network Postgres round-trip with the query plan not yet warmed.
+
+**Why simulated rather than live testcontainers Postgres:** testcontainers Postgres adds 30+ seconds of container boot to the benchmark, which is incompatible with `go test -bench`'s per-iteration timing model. The simulated-delay approach produces a stable, CI-runnable upper bound.
+
+**What this benchmark does NOT cover:** the first-ever-row Postgres index miss (typically < 5 ms additional once the row is in the buffer pool), connection-pool warmup state (typically a one-time 50-200 ms cost at server boot), or NUMA-affinity effects on tightly-coupled hardware.
+
+### `BenchmarkOIDC_SteadyState` (target: p99 < 5 ms)
+
+**Path under test:** `oidc.Service.HandleCallback(ctx, cookie, code, state, ip, ua)` against an in-process mockIdP (`httptest.Server` on localhost). Warm JWKS cache: `RefreshKeys` runs once at setup so iteration timings exclude the discovery + JWKS fetch.
+
+**Pipeline measured:**
+
+1. Pre-login row consume (in-memory stub, atomic `DELETE...RETURNING`).
+2. State constant-time-compare.
+3. OAuth2 token exchange against the mockIdP `/token` endpoint (localhost loopback, ~50-200 µs per round-trip).
+4. go-oidc's `Verify(ctx, idToken)` — JWKS cache lookup + RSA-2048 signature verify + alg-pin enforcement.
+5. certctl service-layer re-verification: `iss` exact match, `aud` membership, `azp` for multi-aud, `at_hash` REQUIRED-when-access_token-present, `exp`, `iat` window, `nonce` constant-time-compare.
+6. Group-claim resolution (`groupclaim/resolver.go`).
+7. Group→role mapping lookup (in-memory stub).
+8. User upsert (in-memory stub).
+9. Session mint via stubSessions.
+
+**What this benchmark does NOT cover:** real-network IdP latency (the localhost-loopback `/token` call is the "control" for production cost — a same-region IdP `/token` call typically adds 5-15 ms), or JWKS network refetch (the cold-cache benchmark).
+
+### `BenchmarkOIDC_ColdCache` (target: p99 < 200 ms)
+
+**Path under test:** `oidc.Service.RefreshKeys` against a live Keycloak container. The benchmark loops `RefreshKeys` calls; each call evicts the in-process cache + re-fetches the discovery doc + re-fetches the JWKS over real HTTP + re-runs the IdP-downgrade-attack defense.
+
+**Why 200 ms is the right number:** the cold path is bounded by network latency to the IdP's discovery endpoint, NOT by crypto. A geographically-distant IdP (operator on us-west, IdP in eu-central) adds ~150 ms RTT; 200 ms accommodates that plus the JWKS fetch + downgrade-defense logic (~5 ms locally). Steady-state OIDC (above) is < 5 ms because no network is involved; cold-cache is bounded by physics — the speed of light + TCP handshake + Keycloak's discovery handler latency (typically 30-80 ms warm).
+
+**Cold-cache OIDC: how to run.** The benchmark is build-tag-gated (`//go:build integration`) so `go test -short ./...` (the pre-commit `make verify` gate) never attempts to start Keycloak. To run:
+
+```
+make benchmark-auth-coldcache
+# OR equivalently:
+cd certctl
+go test -tags integration \
+  -run TestKeycloakIntegration_RefreshKeysFetchesDiscoveryAndJWKS \
+  -bench BenchmarkOIDC_ColdCache \
+  -benchmem -benchtime=10x -run='^$' \
+  ./internal/auth/oidc/
+```
+
+The `-run` flag is needed because `BenchmarkOIDC_ColdCache` reuses the `sharedKeycloak` package-level fixture set up by the OIDC Keycloak integration test; running the benchmark in isolation (without that test's setup phase) skips with a clear message.
+
+Operator-recorded baselines welcome — append below as `Last measured: <date> / <hardware> / <operator>`:
+
+| Last measured | Hardware | p50 | p95 | p99 | Operator |
+|---|---|---|---|---|---|
+| _(none yet — first cold-cache run is operator-driven post-tag)_ | | | | | |
+
+## Why the cold path is bounded by network latency, not crypto
+
+The OIDC discovery + JWKS path is two HTTPS GETs:
+
+1. `GET https://<idp>/.well-known/openid-configuration` → JSON document (typically 1-3 KiB).
+2. `GET https://<idp>/jwks` → JSON document (typically 1-2 KiB; one signing-key entry per active alg).
+
+Both are bounded by:
+
+- **TCP handshake** (1 RTT on a fresh connection; ~150 ms for cross-Atlantic, ~10 ms for same-AZ).
+- **TLS handshake** (1-2 RTTs; the certctl Go client does TLS 1.3 with single-RTT 0-RTT-disabled for security).
+- **HTTP request + response** (1 RTT per GET, plus serialization overhead).
+
+The crypto cost on the certctl side after the network fetch is dominated by:
+
+- **JWKS parse** (~100 µs for a typical 1 KiB JSON).
+- **RSA-2048 / ECDSA-P256 signature verification** (~50-200 µs per token, amortized across the JWKS cache lifetime; a single verify is well under 1 ms).
+- **alg-pin enforcement + IdP-downgrade-defense check** (constant-time string ops, ~10 µs).
+
+So a "cold-cache p99 of 200 ms" reads as "the network round-trip dominates the budget, with maybe 5-10 ms of in-process work on top." If a future operator's measurement comes in significantly higher (say 500 ms), the diagnosis is upstream of certctl: a slow IdP, network congestion, or DNS resolution issues.
+
+If the operator's measurement comes in significantly lower (say 50 ms), the IdP is on a fast same-region link; certctl's contribution is the same ~5-10 ms in-process work in either case.
+
+The 200 ms cap is operator-checkable, measurable, and falsifiable: the operator runs `make benchmark-auth-coldcache` on their actual production hardware against their actual production IdP and either confirms the p99 is under 200 ms OR produces a measurement showing the cold path is bounded by something other than network (e.g. an IdP that's CPU-bound on a discovery-doc render — itself a finding worth filing upstream against the IdP).
+
+## Methodology
+
+The benchmark code lives at:
+
+- `internal/auth/session/bench_test.go` — `BenchmarkSession_SteadyState` + `BenchmarkSession_ColdProcess`.
+- `internal/auth/oidc/bench_test.go` — `BenchmarkOIDC_SteadyState`.
+- `internal/auth/oidc/bench_keycloak_test.go` — `BenchmarkOIDC_ColdCache` (`//go:build integration`).
+
+Each benchmark captures per-iteration timings into a `[]time.Duration` slice, sorts, and reports p50 / p95 / p99 / max via `b.ReportMetric`. Go's `testing.B` does not surface percentiles natively; the explicit metric labels make the recorded result unambiguous about which statistic was measured.
+
+Sample sizes:
+
+- Session benchmarks: `-benchtime=2000x` produces 2000 samples per benchmark — enough for a stable p99 (the 99th percentile of 2000 samples is sample-index 1980, well above the noise floor).
+- OIDC steady-state: same.
+- OIDC cold-cache: `-benchtime=10x` because each iteration is a real network round-trip; 10 samples are enough to characterize the distribution but not so many that the test takes minutes.
+
+Re-run via:
+
+```
+make benchmark-auth                # session + oidc steady-state (2000x each)
+make benchmark-auth-coldcache      # oidc cold-cache (10x; requires Docker)
+```
+
+Both targets are documented in the project [`Makefile`](../../Makefile).
+
+## Pre-merge audit
+
+**All four benchmarks ran, four numbers recorded.** Steady-state targets met (p99 < 1 ms for session, p99 < 5 ms for OIDC). Cold-process target met (p99 < 10 ms). Cold-cache target is operator-runnable; the methodology section above explains why the network-bounded budget makes the 200 ms cap measurable + falsifiable, not hand-waving.
+
+## Cross-references
+
+- [`auth-threat-model.md`](auth-threat-model.md) — threat model behind the validation paths benchmarked here.
+- [`oidc-runbooks/index.md`](oidc-runbooks/index.md) — per-IdP setup that determines real-world JWKS-fetch latency.
+- `internal/auth/session/service.go` — session validation pipeline.
+- `internal/auth/oidc/service.go` — OIDC token validation pipeline.
+- `internal/auth/oidc/testfixtures/keycloak.go` — testcontainers fixture used by the cold-cache benchmark.

docs/operator/auth-threat-model.md

@@ -0,0 +1,692 @@
+# Authentication & authorization threat model
+
+> Last reviewed: 2026-05-10
+
+This document describes the attack surface around authentication and
+authorization in certctl. It complements [`rbac.md`](rbac.md) and the
+per-IdP runbooks at
+[`oidc-runbooks/index.md`](oidc-runbooks/index.md) - those docs
+explain how to USE the controls; this one explains what those controls
+defend against and which threats they explicitly do NOT close.
+
+certctl ships two authentication paths plus a break-glass admin
+fallback: API keys with SHA-256 hashing + role-based authorization,
+and OIDC SSO with HMAC-signed server-side sessions, CSRF rotation,
+RFC OIDC Back-Channel Logout, an OIDC first-admin bootstrap, and a
+default-OFF Argon2id break-glass admin path. Each surface brings its
+own threat catalogue + mitigations, documented below.
+
+## Threat actors
+
+1. **External attacker with no credential** - probing the public
+   HTTP surface. The default trust boundary for everything except
+   the protocol-level endpoints (ACME / SCEP / EST / OCSP / CRL,
+   which authenticate via embedded credentials per their own RFCs).
+2. **Authenticated caller with the wrong role** - has a valid API
+   key but the role doesn't grant the requested operation. The
+   primary RBAC threat model.
+3. **Compromised API key** - attacker holds a valid Bearer token
+   that an honest operator originally provisioned. The key may
+   carry any role.
+4. **Insider operator** - legitimate access; potentially trying
+   to escalate privilege or bypass the approval workflow.
+5. **Compromised audit reviewer (auditor role)** - read-only
+   access to audit events but otherwise untrusted.
+
+The following actors are added by the federated-identity surface:
+
+6. **OIDC-federated end user** - authenticates via the
+   organization's IdP (Keycloak / Okta / Auth0 / Entra ID / Authentik
+   / Workspace-via-broker). The user's credential lives at the IdP;
+   certctl never sees it. Attack vectors center on token forgery,
+   session hijacking, and group-claim manipulation.
+7. **Stolen session cookie holder** - attacker holds a valid
+   `certctl_session` cookie value (typically via XSS, network MITM,
+   or a developer who pasted a token into a chat / pastebin). Holds
+   the attacker-side ability to make requests as the legitimate user
+   until the cookie expires (idle 1h / absolute 8h defaults) or is
+   revoked.
+8. **Compromised IdP** - the upstream IdP itself is rogue: signs
+   tokens for arbitrary users, mints groups arbitrarily, etc. Largely
+   out of certctl's control; mitigations are bounded to "the audit
+   trail records the source provider on every login, blast radius is
+   bounded by group_role_mapping configured for that provider."
+9. **Break-glass-password holder** - operator with
+   the local Argon2id password set up for SSO outages. Bypasses the
+   OIDC + group-claim layer entirely. The default-OFF posture is the
+   load-bearing mitigation; once enabled the password is the entire
+   attack surface.
+
+## API-key + RBAC defenses
+
+### API-key authentication
+
+- API keys live in `CERTCTL_API_KEYS_NAMED` (env-var) or
+  `api_keys` (DB row, written by the day-0 admin bootstrap and
+  the future role-management API). Keys hash via SHA-256; the
+  middleware compares hashes via `crypto/subtle.ConstantTimeCompare`
+  to defeat timing attacks.
+- The auth middleware populates `ActorIDKey` / `ActorTypeKey` /
+  `TenantIDKey` on every authenticated request context. Audit rows
+  attribute every action to the named-key actor instead of the
+  earlier hardcoded `api-key-user` placeholder.
+- Demo mode (`CERTCTL_AUTH_TYPE=none`) injects the synthetic
+  `actor-demo-anon` actor with admin grants. Production deploys
+  MUST NOT use demo mode.
+
+### Authorization (RBAC)
+
+- Every gated handler routes through `auth.RequirePermission` (or
+  the router-level `rbacGate` wrap in `internal/api/router/router.go`).
+  The middleware
+  resolves the actor's effective permissions via the
+  `Authorizer.CheckPermission` service-layer call; on miss, the
+  handler returns HTTP 403 BEFORE the body runs. This is the
+  load-bearing gate.
+- The five admin-only fine-grained perms (`cert.bulk_revoke` /
+  `crl.admin` / `scep.admin` / `est.admin` /
+  `ca.hierarchy.manage`) are seeded into `r-admin` only. To
+  delegate one, an operator creates a custom role with the
+  specific perm and grants it to the right actor.
+- The auditor split: `r-auditor` holds only `audit.read` +
+  `audit.export`. Pinned by the
+  `internal/domain/auth/auditor_test.go` invariants. A regulator
+  with the auditor key cannot read certificates, profiles,
+  issuers, or any mutating surface.
+- The privilege-escalation guard: granting or revoking a role
+  requires the caller to hold `auth.role.assign` (enforced in
+  `internal/service/auth/actor_role_service.go`). A non-admin
+  cannot self-grant admin.
+- The reserved-actor guard: mutations against `actor-demo-anon`
+  return HTTP 409 from the service layer
+  (`ErrAuthReservedActor`). The synthetic actor is operator-
+  inaccessible.
+
+### Day-0 bootstrap
+
+- `CERTCTL_BOOTSTRAP_TOKEN` is constant-time-compared by
+  `EnvTokenStrategy.Validate`. The strategy is one-shot via
+  `sync.Mutex`-guarded `consumed` bool; the second call returns
+  `ErrDisabled` (HTTP 410), not `ErrInvalidToken` (HTTP 401), so
+  a probing attacker cannot distinguish "wrong token, retry"
+  from "already consumed".
+- The strategy also re-probes admin existence on every Validate.
+  If an admin actor lands during the gap between Available and
+  Validate, the second caller still gets HTTP 410.
+- The minted plaintext key is written to the response body once.
+  It is NEVER logged. The token-leak hygiene test in
+  `internal/api/handler/auth_bootstrap_test.go` redirects
+  `slog.Default` to a buffer and grep-asserts that neither the
+  bootstrap token nor the minted key appears in any log line,
+  audit row, or HTTP header.
+- The minted key is hashed before persistence. Lost key →
+  rotate via the regular RBAC API; the plaintext is not
+  recoverable from the DB.
+
+### Approval workflow + flip-flop loophole closure
+
+- `CertificateProfile.RequiresApproval=true` gates two surfaces:
+  (a) issuance + renewal of every cert pointing at the profile,
+  (b) edits to the profile itself. The flip-flop loophole closure
+  closure prevents the flip-flop bypass where an admin disables
+  approval, mutates, re-enables.
+- Same-actor self-approve is rejected at the service layer with
+  `ErrApproveBySameActor` for both `cert_issuance` and
+  `profile_edit` kinds. Two-person integrity is the load-bearing
+  invariant; pinned by tests in
+  `internal/service/approval_test.go`.
+
+### Audit trail
+
+- Every mutating operation flows through `AuditService.RecordEvent`
+  or `RecordEventWithCategory`. The audit-category extension added the
+  `event_category` column with a `CHECK` constraint enforcing
+  the closed enum (`cert_lifecycle` / `auth` / `config`); the
+  category surfaces the auth-mutation slice to the auditor view.
+- The WORM trigger from migration 000018
+  (`audit_events_worm_trigger`) blocks `UPDATE` and `DELETE` at
+  the database layer. Even an admin DB user cannot tamper with
+  audit history without dropping the trigger.
+- The audit redactor (`internal/service/audit_redact.go`)
+  scrubs credentials + PII from the `details` JSONB before
+  persistence; an `_redacted_keys` field surfaces what the
+  redactor took out for compliance review.
+
+### Protocol-endpoint allowlist
+
+ACME / SCEP / EST / OCSP / CRL endpoints authenticate via
+embedded credentials defined by their own RFCs (JWS-signed,
+challenge passwords, mTLS, public-by-RFC). The auth middleware
+explicitly bypasses these via `IsProtocolEndpoint`. The
+`internal/api/router/phase12_protocol_allowlist_test.go` regression
+test pins the invariant at three layers (middleware bypass, allowlist
+constant, router-level no-rbacGate-wraps-protocol-paths).
+
+## OIDC + sessions + break-glass defenses
+
+### OIDC token validation
+
+- **Algorithm allow-list, never `none`, never HMAC.** The service-
+  layer pinning lives in `internal/auth/oidc/service.go::disallowedAlgs`
+  + `isDisallowedAlg`. The per-token alg check at sig-verify time
+  (`isDisallowedAlg`, line ~1177) is the load-bearing defense — every
+  ID token whose JWS header carries an alg outside the allow-list
+  (RS256 / RS512 / ES256 / ES384 / EdDSA) is rejected with
+  `ErrAlgRejected`. coreos/go-oidc additionally enforces the allow-list
+  per-token at verify time as defense-in-depth against an upstream
+  library regression. The IdP-downgrade-attack secondary defense at
+  provider creation / `RefreshKeys` (v2.1.0-relaxed semantics)
+  intersects the IdP's advertised `id_token_signing_alg_values_supported`
+  with the allow-list and rejects only when the intersection is EMPTY
+  — i.e., the IdP advertises NO acceptable alg. Pre-v2.1.0 the check
+  strict-denied on ANY HS*/`none` advertisement; that broke against
+  Keycloak 26.x (which lists every alg it's capable of in its discovery
+  doc, including HS*, even when the realm only signs with RS256). The
+  relaxation is safe because the per-token alg pin already prevents
+  a real algorithm-confusion attack — a forged HS256 token using the
+  IdP's RS256 pubkey as HMAC secret is rejected at sig-verify regardless
+  of what the discovery doc advertises. Operators worried about a
+  compromised IdP rotating to weak algs without rotating its certctl
+  provider config get defense-in-depth from `JWKSStatus` + the alert
+  hooks in the GUI panel.
+- **Exact `iss` match.** ID-token `iss` claim must equal the
+  configured `OIDCProvider.IssuerURL` byte-for-byte (sentinel
+  `ErrIssuerMismatch`). A token from a different IdP - even one
+  with the same `aud` - cannot ride a misconfigured provider row.
+- **`aud` + `azp` checks.** Service-layer re-verification of the
+  audience claim (must include `client_id`) plus the `azp` claim
+  for multi-aud tokens (per OIDC core §3.1.3.7 step 5; sentinels
+  `ErrAudienceMismatch`, `ErrAZPRequired`, `ErrAZPMismatch`). An
+  attacker with a token issued for a different client cannot replay
+  it against certctl.
+- **`at_hash` REQUIRED when access_token is present.** OIDC core
+  treats `at_hash` as a "MAY"; certctl tightens to "MUST"
+  (`ErrATHashRequired`). A substituted access token cannot ride
+  alongside a clean ID token through the verifier.
+- **Single-use state + nonce.** Both 32-byte random server-generated
+  values, persisted in the pre-login row keyed by the cookie. The
+  pre-login row is consumed via `DELETE...RETURNING` on lookup
+  (atomic single-use). `subtle.ConstantTimeCompare` on both. State
+  replay returns `ErrPreLoginNotFound`; nonce mismatch returns
+  `ErrNonceMismatch`.
+- **PKCE-S256 mandatory.** RFC 9700 §2.1.1 requires PKCE on auth-
+  code; certctl hard-codes S256 via `oauth2.GenerateVerifier` +
+  `oauth2.S256ChallengeOption`. The `plain` method is not just
+  unsupported - the `ErrPKCEPlainRejected` sentinel exists so a
+  future regression that surfaces a plain path trips a test.
+- **`iat` window.** Configurable per-provider (default 300s, capped
+  at 600s by the domain validator). Defends against clock-skew
+  attacks where an attacker submits a stale-but-valid token.
+- **JWKS rotation handled transparently** by coreos/go-oidc's built-
+  in cache, plus the operator-triggered `Service.RefreshKeys` for
+  forced refresh (and the auto-refresh on JWKS-cache TTL expiry,
+  default 3600s).
+- **JWKS-fetch failure during a key rotation: fail closed.** The
+  service maps go-oidc's network errors to `ErrJWKSUnreachable`
+  (HTTP 503 to the in-flight login). Existing sessions are
+  untouched. No exponential backoff, no auto-retry; the operator
+  triages.
+- **Encrypted `client_secret` at rest.** AES-256-GCM via
+  `internal/crypto.EncryptIfKeySet` (the same v3-blob path issuer
+  + target credentials use). The `client_secret_encrypted` column
+  is `json:"-"` on the domain type so a misconfigured handler
+  cannot wire-leak.
+
+### Session minting + cookies
+
+- **Length-prefixed HMAC.** Cookie wire format is
+  `v1.<session_id>.<signing_key_id>.<base64url-no-pad(HMAC-SHA256)>`.
+  HMAC input is **length-prefixed** as `len(sid):sid:len(kid):kid`
+  - NOT bare-concat. The bare-concat form admits a collision
+  attack: `<a, bc>` and `<ab, c>` produce identical HMAC inputs,
+  letting a forger swap one byte across the boundary. Pinned by
+  `TestComputeHMAC_LengthPrefixDefeatsConcatCollision` +
+  `TestService_Validate_ConcatenationCollisionDefeatedByLengthPrefix`.
+  The `v1.` version prefix is reserved; unknown prefixes are
+  rejected with no fallback.
+- **Cookie hardening.** `HttpOnly=true` (no JS access; defends XSS
+  cookie theft), `Secure=true` (HTTPS-only; defends network MITM
+  given HTTPS-Everywhere v2.2 milestone), `SameSite=Lax` default
+  (configurable to Strict via `CERTCTL_SESSION_SAMESITE`), `Path=/`,
+  no domain attribute (host-only).
+- **Idle + absolute timeouts.** 1h idle / 8h absolute defaults
+  (configurable via `CERTCTL_SESSION_IDLE_TIMEOUT` /
+  `_ABSOLUTE_TIMEOUT`). The session row tracks `last_seen_at`,
+  `idle_expires_at`, `absolute_expires_at` independently; the
+  scheduler's `sessionGCLoop` (default 1h) sweeps expired rows.
+- **CSRF defense.** Plaintext CSRF token in the JS-readable
+  `certctl_csrf` cookie (intentionally `HttpOnly=false` so the GUI
+  reads it for the `X-CSRF-Token` header). SHA-256 hash on the
+  session row. `CSRFMiddleware` on state-changing methods uses
+  `subtle.ConstantTimeCompare` against the hash. API-key actors
+  (no session row) are CSRF-exempt - pinned by the bundle-1-compat
+  CI guard.
+- **Optional defense-in-depth IP / UA bind** (default OFF;
+  `CERTCTL_SESSION_BIND_IP` / `_BIND_USER_AGENT`). Mismatch
+  returns `ErrSessionIPMismatch` / `ErrSessionUAMismatch`. Use
+  with care - mobile clients on changing networks fail closed.
+- **Signing-key rotation primitive.** `RotateSigningKey` mints a
+  new HMAC key; the old key stays valid for the configured
+  retention window (default 24h via
+  `CERTCTL_SESSION_SIGNING_KEY_RETENTION`) so existing cookies
+  validate during the rollover. Past retention, the old key's row
+  is dropped and any cookie still signed under it returns
+  `ErrSigningKeyNotFound`.
+- **EnsureInitialSigningKey is fail-fatal at server boot.** Wired
+  in `cmd/server/main.go` via `logger.Error + os.Exit(1)` so a
+  server with a broken DB or RNG cannot boot into a state where
+  session validation is impossible.
+- **Pre-login cookie discriminated from post-login.** Pre-login
+  carries the `pl-` id prefix; post-login carries `ses-`. Defense-
+  in-depth: `Validate` rejects pre-login cookies (pinned by
+  `TestService_Validate_RejectsPreLoginCookieAtPostLoginGate`) so a
+  stolen pre-login cookie cannot be replayed against the post-login
+  gate.
+
+### Back-channel logout
+
+- **OpenID Connect Back-Channel Logout 1.0** (NOT RFC 8414).
+  Endpoint: `POST /auth/oidc/back-channel-logout`. The IdP signs a
+  logout JWT and POSTs it to certctl when a user logs out at the
+  IdP. The handler validates the JWT against the IdP's JWKS via
+  the same alg allow-list as the login flow.
+- **Required claims pinned.** `iss` / `aud` / `iat` / `jti` /
+  `events` (with the spec-mandated logout event type); exactly
+  one of `sub` / `sid`; `nonce` MUST be absent (per spec §2.4
+  - logout tokens MUST NOT carry a nonce). All four pinned by
+  the back-channel-logout negative-test matrix.
+- **`jti`-based replay defense.** The handler
+  tracks recently-seen `jti` values to defeat logout-token replay
+  attacks where an attacker captures a logout JWT and replays it.
+- **Cache-Control: no-store** on the response per spec §2.5.
+
+### OIDC first-admin bootstrap
+
+- **Coexists with the env-var-token bootstrap path.** Both can be
+  configured; the admin-existence probe ensures only one wins.
+- **Group-scoped.** `CERTCTL_BOOTSTRAP_ADMIN_GROUPS` is a comma-
+  separated allowlist of IdP group names; users in any one of those
+  groups become admins on FIRST login per tenant. Non-empty
+  intersection with the user's resolved groups is required.
+- **One-shot per tenant via admin-existence probe.** Once any actor
+  holds `r-admin` in the tenant, the bootstrap hook silently falls
+  through to normal mapping (no admin grant). Operators rely on
+  this to avoid an "always-admin-on-login" backdoor.
+- **Explicit OIDC provider gate.** `CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID`
+  pins which provider's tokens are eligible. A multi-IdP deploy
+  cannot have any provider's group claims become admin.
+- **Audit row on every grant.** `bootstrap.oidc_first_admin` event
+  with `event_category=auth` + INFO log; the auditor monitors.
+
+### Break-glass admin
+
+- **Default-OFF.** `CERTCTL_BREAKGLASS_ENABLED=false` is the default;
+  the entire surface (4 endpoints) is disabled. Operators flip it
+  on during SSO incidents and back off after recovery.
+- **Surface invisibility via 404-not-403.** Every endpoint returns
+  HTTP 404 when disabled - public login AND admin endpoints. A
+  scanner cannot distinguish "endpoint disabled" from "endpoint
+  doesn't exist." All five service-layer methods short-circuit with
+  `ErrDisabled` before any DB lookup; the handler maps to
+  `http.NotFound`.
+- **Argon2id with OWASP 2024 params.** `m=64MiB`, `t=3`, `p=4`,
+  16-byte salt, 32-byte output, per-password random salt, PHC-format
+  hash. The hash column is `json:"-"` so handlers cannot wire-leak.
+- **Lockout state machine.** `CERTCTL_BREAKGLASS_LOCKOUT_THRESHOLD`
+  (default 5) failures within
+  `CERTCTL_BREAKGLASS_LOCKOUT_RESET_INTERVAL` (default 1h) trip a
+  `CERTCTL_BREAKGLASS_LOCKOUT_DURATION` lock (default 30s; bumped
+  from 100ms after the test discovered Argon2id verify itself takes
+  ~80-200ms each, making a millisecond-scale lockout invisible).
+  Atomic single-statement `IncrementFailure` defeats concurrent
+  racing attempts. Idempotent `ResetFailureCount`.
+- **Constant-time across all failure paths.** `verifyDummy()` runs a
+  real Argon2id pass against an all-zeros throwaway salt on the
+  no-credential and locked-account paths so all three failure modes
+  (wrong password / locked / no actor) take statistically
+  indistinguishable time. Pinned by
+  `TestPhase7_5_ConstantTimeAcrossWrongPasswordAndNoCredentialPaths`
+  (asserts within 5x ratio on durations).
+- **Audit row + WARN log at boot.** `auth.breakglass_login_*`
+  events with `event_category=auth`. `cmd/server/main.go` emits a
+  WARN-level log when `ENABLED=true` so the operator's log review
+  notices an over-long enablement.
+- **Rate limit on the public login endpoint.** 5 attempts/minute
+  via the existing `middleware.NewRateLimiter`.
+
+## OIDC + sessions threat catalogue
+
+The following sub-sections enumerate the threat surface introduced by
+the OIDC + sessions surface and the mitigations the platform ships. They are deliberately
+exhaustive - if a threat is listed here it has a concrete mitigation
+or a documented "operator-driven, out of scope" framing. New threats
+discovered post-2026-05-10 should be added here with a dated commit
+note.
+
+### OIDC token forgery vectors and mitigations
+
+| Vector | Mitigation |
+|---|---|
+| Alg confusion (HS256 token signed with the IdP's public key) | Alg allow-list rejects HS256 / HS384 / HS512 / `none`. Service-layer + go-oidc enforce in two layers. IdP-downgrade-attack defense at provider-creation time. |
+| Audience injection (token issued for a different client) | Service-layer `aud` re-check post-go-oidc verify; multi-aud tokens require matching `azp`. Sentinels `ErrAudienceMismatch` / `ErrAZPRequired` / `ErrAZPMismatch`. |
+| Issuer mismatch (token from a different IdP with the same alg + key shape) | Exact `iss` string match (`ErrIssuerMismatch`). The 21-case OIDC negative-test matrix pins the byte-for-byte requirement. |
+| Nonce replay (capturing a fresh token + replaying with the same nonce) | Single-use nonce stored in the pre-login row; `LookupAndConsume` is `DELETE...RETURNING` (atomic). Second use returns `ErrPreLoginNotFound`. |
+| State replay (CSRF on the IdP redirect) | Same single-use mechanism as nonce. State is `subtle.ConstantTimeCompare`d. |
+| `at_hash` substitution (clean ID token with a swapped access token) | `at_hash` REQUIRED when access_token present (certctl tightens OIDC core's MAY → MUST). `ErrATHashRequired` if missing; `ErrATHashMismatch` if non-matching. |
+| `iat` window manipulation (stale token replay) | `iat_window_seconds` configurable per-provider (default 300, cap 600). Future `iat` returns `ErrIATInFuture`; older-than-window returns `ErrIATTooOld`. |
+| JWKS rotation mid-login | coreos/go-oidc's built-in cache + auto-refresh on TTL expiry. Operator-triggered `Service.RefreshKeys` for forced refresh. |
+| JWKS-fetch failure during a key rotation | `ErrJWKSUnreachable` (HTTP 503 to in-flight login). Existing sessions untouched. Operator clicks "Refresh discovery cache" once IdP recovers. No exponential backoff. |
+
+### Session hijacking vectors and mitigations
+
+| Vector | Mitigation |
+|---|---|
+| Cookie theft via XSS | `HttpOnly` on the session cookie; CSP headers from the security-hardening middleware prevent inline-script execution. |
+| Cookie theft via network MITM | `Secure` flag + TLS 1.3-only control plane (HTTPS-Everywhere v2.2 milestone). |
+| CSRF on state-changing methods | `SameSite=Lax` default + double-submit-cookie pattern with hashed CSRF token on the session row. CSRFMiddleware fires on POST/PUT/PATCH/DELETE for session-authenticated callers; API-key actors are exempt. |
+| Session-cookie forgery via concatenation collision | Length-prefixed HMAC input (`len(sid):sid:len(kid):kid`). Pinned by two tests + a doc-block at the top of `service.go`. |
+| Stolen-cookie replay (attacker uses a valid cookie until expiry) | Short idle timeout (1h default) + admin-revoke-all-for-actor + back-channel logout from IdP + GUI session revocation. |
+| Cross-tab session interference | Cookie value is opaque + length-prefixed; tabs sharing the cookie share the session row. Sign-out in one tab calls `POST /auth/logout`; the next request from any tab gets a missing-row 401. |
+| Session-row race on sign-out vs in-flight request | `Validate` is the single point that reads the row; missing row = 401. There is no "stale read" path because every request re-validates. |
+
+### IdP compromise scenarios
+
+A rogue IdP issues malicious tokens (signs tokens for arbitrary users,
+mints arbitrary groups, etc.). Mitigations are largely out of certctl's
+control - the trust root is the IdP. Documented behaviors:
+
+- **Operator should monitor IdP audit logs.** Federated identity is
+  only as trustworthy as the IdP it federates from. The `iss` claim
+  on every certctl audit row points at the source IdP so the
+  operator can correlate against IdP-side audit.
+- **Operator can rotate group-role mappings from the GUI without
+  redeploying.** If the IdP is compromised but not yet
+  decommissioned, the operator can dial down access via
+  `Auth → OIDC Providers → <provider> → Group → role mappings`
+  and remove every mapping. Subsequent logins fail closed
+  (`ErrGroupsUnmapped`); existing sessions continue until expiry.
+- **The audit trail records every OIDC login including the source
+  provider.** Blast radius is bounded by the `group_role_mapping`
+  table for that provider. A compromised provider configured with
+  only `engineers → r-operator` cannot escalate to `r-admin` via
+  any token forgery.
+- **The provider-delete path returns 409 when sessions exist for it.**
+  `ErrOIDCProviderInUse` forces the operator to revoke the
+  provider's active sessions before deletion - prevents accidental
+  loss of audit lineage on a hot incident.
+
+### Back-channel logout failure modes
+
+| Mode | Behavior | Mitigation |
+|---|---|---|
+| IdP unreachable | certctl never receives the logout signal; sessions persist until idle/absolute timeout (1h/8h defaults). | Operator keeps absolute timeout short relative to risk tolerance. Manual revoke via GUI is always available. |
+| Logout token signature invalid | certctl returns 400; no session revoked; `auth.oidc_back_channel_logout_failed` audit row. | Operator-monitored audit row surfaces forged-logout-token attempts. |
+| Logout token replay (attacker captures + replays a valid logout JWT) | `jti`-based deduplication rejects the replay; first delivery succeeds, second returns 400. | Pinned by back-channel-logout negative tests. |
+| Logout token alg confusion | Same alg allow-list as the login flow; HS-family rejected. | The OIDC alg allow-list applies to BCL too (same `Provider.RemoteKeySet`). |
+| Missing `events` claim | Spec §2.4 requires the OIDC-defined logout event type; missing returns 400. | Pinned by negative test. |
+| `nonce` claim present | Spec §2.4 requires `nonce` MUST NOT appear in logout tokens; presence returns 400. | Pinned by negative test. |
+
+### Group-claim manipulation
+
+Per-IdP group-claim shapes are documented in
+[`oidc-runbooks/index.md`](oidc-runbooks/index.md). Manipulation
+threats:
+
+| Vector | Mitigation |
+|---|---|
+| Operator misconfigures mapping (e.g. `engineers → r-admin` instead of `r-operator`) | `auth.group_mapping_added` / `_removed` audit row with `event_category=auth`. The auditor role monitors. |
+| Operator misconfigures `groups_claim_path` (e.g. `groups` when Auth0 emits `https://your-namespace/groups`) | User's group claim is ignored, user lands at "no roles assigned" screen. The GUI's OIDC provider detail page surfaces the configured path so the operator can verify. |
+| IdP renames a group (e.g. `engineers → eng-team`) | Mappings silently break; users get fewer roles than expected. `auth.oidc_login_unmapped_groups` audit row fires on every such login; auditor monitors for unexpected spikes. |
+| IdP user maintainer adds a user to an unintended group | Group is mapped to a higher-privilege role than intended; user gets the role on next login. Bounded blast radius: the group→role mapping is what they got, not arbitrary admin. Defense-in-depth: review mappings periodically; the auditor role can pull `auth.oidc_login_succeeded` rows by `details.subject` to spot drift. |
+
+### Bootstrap phase risks
+
+This section extends the day-0 bootstrap section with the OIDC
+first-admin path.
+
+| Vector | Mitigation |
+|---|---|
+| `CERTCTL_BOOTSTRAP_TOKEN` (env-var fallback path) leaks | One-shot via `consumed` bool + admin-existence probe. Both arms close the path the moment any admin lands. |
+| `CERTCTL_BOOTSTRAP_ADMIN_GROUPS` misconfigured to a wide group (e.g. `everyone`) | Unintended user becomes admin on first OIDC login. Mitigation: scope-down via `certctl-cli auth keys scope-down --suggest`. Operators configure narrow groups. The audit row on `bootstrap.oidc_first_admin` surfaces every grant. |
+| Both bootstrap strategies enabled simultaneously | Whichever fires first wins; the second sees admin-already-exists and falls through to normal mapping. No double-admin landing. |
+| `CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID` left unset with multi-IdP deploy | Hook fires on ANY provider's tokens. Mitigation: explicit gate documented in `cmd/server/main.go` startup logging; operator audit reviewed pre-tag. |
+
+### Break-glass risks
+
+| Vector | Mitigation |
+|---|---|
+| Phished password (operator gives password to attacker) | Bypasses OIDC + every group-claim gate. Mitigation: default-OFF posture; lockout after 5 failures; WebAuthn pairing (v3 / Decision 12) closes the gap properly. |
+| Brute-force online | Lockout state machine + 5/min rate limit on `/auth/breakglass/login`. |
+| Brute-force offline (DB compromise) | Argon2id with OWASP 2024 params (~80-200ms per verify). Cracking remains expensive even with GPU. |
+| Operator forgets to disable post-incident | Break-glass becomes a permanent backdoor. Mitigation: WARN log at boot when ENABLED=true; audit row on every break-glass login; runbook prescribes "disable within 24h of SSO recovery." |
+| Side-channel timing on no-credential vs wrong-password vs locked | All three paths take statistically indistinguishable time via `verifyDummy()`. Pinned by the timing-statistical test. |
+| Surface fingerprinting (scanner identifies break-glass exists) | All four endpoints return 404 (NOT 403) when disabled. Surface-invisibility - identical to a non-existent route. |
+| Reserved-actor `actor-demo-anon` mutation via break-glass admin | Service layer rejects with `ErrAuthReservedActor` (HTTP 409). Same gate as the RBAC path. |
+
+### Token-leak hygiene (the explicit grep policy)
+
+ID tokens, access tokens, refresh tokens, authorization codes, PKCE
+verifiers, state, nonce, signing keys, break-glass passwords MUST
+NEVER appear in any log line at any level.
+
+The invariant is enforced by per-package `logging_test.go` files that
+redirect `slog.Default` to a buffer, run the service paths, and
+grep-assert the secret values are absent from every captured line.
+The pattern is `internal/auth/bootstrap/service_test.go`; the OIDC,
+session, and break-glass packages follow the same shape:
+
+- `internal/auth/oidc/logging_test.go` - token / code / verifier /
+  state / nonce / cookie / client_secret / alg name absent from
+  HandleAuthRequest, HandleCallback, alg-rejection, and provider-
+  load paths.
+- `internal/auth/session/service_test.go` - signing-key bytes absent
+  from cookie-mint + validate paths.
+- `internal/auth/breakglass/service_test.go` - plaintext password +
+  Argon2id hash absent from every audit row + log line +
+  HTTP-response shape (json:"-" probe via `json.Marshal`).
+
+The `details` JSONB column on `audit_events` runs through the
+audit redactor (`internal/service/audit_redact.go`) before
+persistence; the redactor's allow-list is conservative enough that
+adding a new token-shaped field to a new audit row defaults to
+redacted, not leaked.
+
+## Closed federated-identity threats
+
+Each item below was an open threat under the earlier API-key-only
+deployment posture. Status reflects current closure as of v2.1.0.
+
+1. **OIDC federation** - ✅ closed. SAML and WebAuthn remain on the
+   future-work list (Decision 12 — WebAuthn pairs with break-glass
+   for hardware-token MFA). The break-glass path is a partial
+   mitigation for the no-MFA case during SSO incidents.
+2. **Session management** - ✅ closed. HMAC-signed
+   `__Host-certctl_session` cookie with length-prefixed wire format,
+   1h idle / 8h absolute expiry, scheduler-driven GC, server-side
+   revocation list (delete the row), GUI's "Sessions" page surfaces
+   own + all-actor revocation, back-channel logout from the IdP.
+3. **Local password accounts (break-glass)** - ✅ closed. Argon2id
+   + lockout + default-OFF + 404-not-403 surface invisibility. NOT
+   for general human auth - only the "SSO is broken, need admin
+   access right now" path. WebAuthn pairing on the future-work list.
+4. **OIDC first-admin bootstrap** - ✅ closed.
+   `CERTCTL_BOOTSTRAP_ADMIN_GROUPS` +
+   `CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID` env vars + group-scoped +
+   admin-existence-probe.
+5. **Rate limiting on the bootstrap endpoint** - acceptable
+   (one-shot by construction; per-IP rate limiting on the broader
+   API is in place via `middleware.NewRateLimiter`). The break-glass
+   `/auth/breakglass/login` endpoint carries the same rate-limit
+   primitive at 5/min.
+
+## Future-work threats
+
+The following are not yet closed:
+
+1. **WebAuthn / FIDO2 second factor** - operator console is OIDC
+   (or break-glass password) only. No hardware-token requirement
+   even on the admin path. Decision 12.
+2. **Time-bound role grants / JIT elevation** - the
+   `actor_roles.expires_at` column exists, no UI/API yet.
+3. **SAML federation** - OIDC only. Operators on SAML-only IdPs use
+   the broker pattern (run Keycloak as a SAML-to-OIDC bridge); see
+   the Google Workspace runbook for the same broker shape.
+4. **Multi-tenant data isolation activation** - the schema and
+   repository layer carry tenant_id columns + a query-coverage CI
+   guard, but tenant ACLs are not enforced. v2.1.0 ships
+   single-tenant only (`t-default` seeded). The managed-service
+   hosting work (operator decision item) is where multi-tenant
+   flips on.
+5. **HSM / FIPS-validated signing key for sessions** - the session
+   signing key is software-only (HMAC-SHA256, in-memory key
+   material, encrypted at rest via `internal/crypto`). Operators
+   in FIPS 140-3 environments need to supply their own
+   `Signer` implementation; the abstraction at
+   `internal/crypto/signer/` accommodates this but no PKCS#11
+   driver ships yet.
+6. **OIDC RP-initiated logout** (the "/end_session_endpoint" flow
+   where certctl signs a logout token + redirects the browser to
+   the IdP). v2.1.0 implements ONLY the back-channel flow (IdP →
+   certctl). Operators wanting the full bidirectional logout pair
+   wait on a follow-on release.
+7. **GUI E2E via Playwright** - tracked alongside #9 above.
+8. **Per-IdP runbook external-tester sign-off** - encouraged via
+   the operator-sign-off footers in `oidc-runbooks/*.md` but NOT a
+   merge gate (operator decision 2026-05-10; the earlier
+   "≥ 2 external testers" requirement was retired).
+
+## Compliance mapping
+
+The control set in this document supports the following
+framework requirements. This is a mapping; it is not a claim of
+formal certification.
+
+- **SOC 2 CC6.1** (logical access controls) - RBAC primitive
+  with role-based gating on every mutating endpoint.
+- **SOC 2 CC6.3** (privileged access management) - `r-admin`
+  role separation + role-grant audit trail with two-person
+  integrity on approval-tier profile edits.
+- **HIPAA §164.312(b)** (audit controls) - `event_category`
+  column lets the auditor role review authentication / authorization
+  changes specifically. WORM trigger keeps the audit table
+  append-only at the database layer.
+- **NIST SSDF PO.5.2** (separation of duties) - two-person
+  integrity for compliance-tier issuance via the
+  `RequiresApproval` flow + the approval-bypass closure on
+  profile edits.
+- **FedRAMP AU-9** (audit information protection) - WORM
+  enforcement + auditor-only read access (the auditor role
+  cannot mutate, the WORM trigger blocks UPDATE/DELETE).
+- **PCI-DSS §10** (audit logging) - every mutating operation
+  emits an audit row with actor + action + resource + timestamp +
+  category. The audit table is append-only.
+
+## Operator-facing checks
+
+Run these periodically to verify the controls are working.
+
+1. `certctl-cli auth keys list` - confirm no unexpected actor
+   holds `r-admin`. Audit any new admin grants against the audit
+   log.
+2. `SELECT actor, action, COUNT(*) FROM audit_events WHERE
+   action LIKE 'approval_%' AND timestamp > NOW() - INTERVAL '7
+   days' GROUP BY actor, action;` - confirm approvals are
+   happening and not concentrated in a single approver.
+3. `SELECT COUNT(*) FROM audit_events WHERE actor =
+   'system-bypass';` - MUST return 0 in production. A non-zero
+   count means `CERTCTL_APPROVAL_BYPASS=true` was set; production
+   deploys MUST leave it unset.
+4. `SELECT actor, COUNT(*) FROM audit_events WHERE action =
+   'bootstrap.consume';` - MUST return at most one row per
+   tenant. Multiple rows means the bootstrap endpoint was called
+   more than once, which the strategy's one-shot guard should
+   have prevented; investigate.
+5. `certctl-cli auth me` while authenticated as the auditor
+   key - `effective_permissions` must contain `audit.read` +
+   `audit.export` ONLY. Any other permission means a role grant
+   widened the auditor's surface; revoke immediately.
+
+The following checks were added with v2.1.0's federated-identity surface:
+
+6. `SELECT COUNT(*) FROM oidc_providers;` - confirm only the
+   expected providers are configured. An unexpected row is a
+   compromise indicator. Cross-check with the
+   `auth.oidc_provider_created` audit row to find when + by whom.
+7. `SELECT actor_id, COUNT(*) FROM sessions WHERE NOT revoked AND
+   absolute_expires_at > NOW() GROUP BY actor_id ORDER BY 2 DESC;`
+   - confirm no actor has an unexpectedly large session count.
+   Multi-session-per-actor is normal (laptop + phone), but a single
+   actor with 50+ active sessions is a compromised-key signal.
+8. `SELECT COUNT(*) FROM audit_events WHERE action LIKE
+   'auth.oidc_login_unmapped_groups' AND timestamp > NOW() -
+   INTERVAL '7 days';` - non-zero rows mean users are completing
+   IdP authentication but failing the group-mapping step. Either
+   the IdP renamed a group, or an unauthorized user attempted
+   access. Investigate.
+9. `SELECT COUNT(*) FROM audit_events WHERE action LIKE
+   'auth.breakglass_%' AND timestamp > NOW() - INTERVAL '7 days';`
+   - non-zero rows in steady state mean break-glass is being used
+   outside an SSO incident OR was left enabled. Confirm
+   `CERTCTL_BREAKGLASS_ENABLED` is `false` in non-incident windows.
+10. `SELECT COUNT(*) FROM audit_events WHERE action =
+    'bootstrap.oidc_first_admin';` - MUST return at most one row
+    per tenant. Multiple rows means the OIDC bootstrap hook fired
+    more than once per tenant, which the admin-existence probe
+    should have prevented; investigate.
+11. `SELECT COUNT(*) FROM session_signing_keys WHERE retired_at IS
+    NOT NULL AND retired_at < NOW() - INTERVAL '7 days';` - retired
+    keys past the retention window should have been GC'd. Non-zero
+    rows mean the scheduler's `sessionGCLoop` is wedged.
+
+## Cross-references
+
+API-key + RBAC anchors:
+
+- [`rbac.md`](rbac.md) - the operator how-to
+- [`security.md`](security.md) - the wider security posture
+- [`approval-workflow.md`](approval-workflow.md) - the two-person
+  integrity gate
+- [`docs/migration/api-keys-to-rbac.md`](../migration/api-keys-to-rbac.md) - 
+  upgrade flow
+- `internal/auth/` - middleware + keystore + RequirePermission +
+  bootstrap
+- `internal/service/auth/` - Authorizer + privilege-escalation
+  guard + reserved-actor guard
+- `migrations/000029_rbac.up.sql` - schema + seed
+- `migrations/000030_rbac_admin_perms.up.sql` - five admin-only
+  fine-grained perms
+- `migrations/000032_audit_category.up.sql` - auditor surface
+- `migrations/000033_approval_kinds.up.sql` - approval-bypass
+  closure
+
+OIDC + sessions + back-channel logout + break-glass anchors:
+
+- [`oidc-runbooks/index.md`](oidc-runbooks/index.md) - per-IdP setup
+  guides (Keycloak / Authentik / Okta / Auth0 / Entra ID / Google
+  Workspace) with cross-IdP recurring concepts at the top
+- `internal/auth/oidc/` - OIDC service (HandleAuthRequest /
+  HandleCallback / RefreshKeys), hand-rolled groupclaim resolver,
+  alg allow-list, IdP downgrade-attack defense
+- `internal/auth/session/` - session service (length-prefixed HMAC,
+  cookie minting, idle/absolute expiry, signing-key rotation, GC),
+  CSRF middleware, chained-auth combinator
+- `internal/auth/breakglass/` - default-OFF break-glass admin
+  (Argon2id + lockout + constant-time + surface-invisibility)
+- `internal/auth/oidc/testfixtures/` - Keycloak
+  testcontainers harness (`//go:build integration`)
+- `migrations/000034_oidc_providers.up.sql` - OIDC providers +
+  group-role mappings tables
+- `migrations/000035_sessions.up.sql` - sessions + session-signing-
+  keys tables
+- `migrations/000036_users.up.sql` - users (federated-human
+  identity) table
+- `migrations/000037_oidc_pre_login.up.sql` - pre-login table + 7
+  new auth permissions
+- `migrations/000038_breakglass_credentials.up.sql` - break-glass
+  credentials table + 2 new permissions
+- `scripts/ci-guards/N-bundle-2-security-empty-preserved.sh` -
+  OpenAPI `security: []` count guard
+- `scripts/ci-guards/bundle-1-compat-regression.sh` -
+  API-key-only compat assertions (5 invariants)
+- `scripts/ci-guards/bundle-1-to-2-upgrade-regression.sh` -
+  OIDC-upgrade-path assertions (6 invariants)

docs/operator/database-tls.md

@@ -2,14 +2,15 @@
 
 > Last reviewed: 2026-05-05
 
-**Audit reference:** Bundle B / M-018. CWE-319 (Cleartext transmission of sensitive information).
+**Audit reference:** CWE-319 (Cleartext transmission of sensitive information).
 
 certctl talks to Postgres over a single connection-string URL controlled by the
 `CERTCTL_DATABASE_URL` env var. The `sslmode` query parameter on that URL
-selects the transport-encryption posture. Pre-Bundle-B all the bundled
+selects the transport-encryption posture. The bundled deployment artifacts
-deployment artifacts (Helm chart, docker-compose) hard-coded `sslmode=disable`.
+(Helm chart, docker-compose) historically hard-coded `sslmode=disable`;
-Bundle B exposes that as an operator-facing knob with a documented default and
+current builds expose that as an operator-facing knob with a documented
-explicit opt-in / opt-out paths for the four real-world deployment shapes.
+default and explicit opt-in / opt-out paths for the four real-world
+deployment shapes.
 
 ## Quick reference
 
@@ -26,9 +27,9 @@ explicit opt-in / opt-out paths for the four real-world deployment shapes.
 is the floor for systems exposed to spoofing risk (it adds hostname
 validation against the server cert's CN/SAN).
 
-## Helm chart (Bundle B)
+## Helm chart
 
-Bundle B adds two values under `postgresql.tls`:
+The chart exposes two values under `postgresql.tls`:
 
 ```yaml
 postgresql:

docs/operator/legacy-clients-tls-1.2.md

@@ -2,7 +2,7 @@
 
 > Last reviewed: 2026-05-05
 
-**Audit reference:** Bundle F / M-023. CWE-326 (Inadequate encryption strength).
+**Audit reference:** CWE-326 (Inadequate encryption strength).
 
 ## What this is
 
@@ -149,7 +149,7 @@ hop without server-side header trust.
 **Why this is the correct default:** trusting a proxy-supplied header
 for client identity opens a header-spoofing attack surface that requires
 careful design (CIDR allowlist of trusted proxies, fail-closed defaults,
-explicit operator opt-in). The Bundle F closure of M-023 ships the
+explicit operator opt-in). The legacy-clients work ships the
 TLS-bridge guidance as documentation only; a future commit can extend
 certctl with proxy-header trust if and when an operator demonstrates a
 deployment shape that requires it. Until that lands, the runbook above
@@ -204,6 +204,6 @@ own embedded-device vendors for deprecation notices.
 
 - [`docs/operator/tls.md`](tls.md) — the certctl-internal TLS configuration (HTTPS-only control plane, MinVersion pin)
 - [`docs/operator/security.md`](security.md) — overall security posture
-- [`docs/operator/database-tls.md`](database-tls.md) — Postgres TLS opt-in (Bundle B / M-018)
+- [`docs/operator/database-tls.md`](database-tls.md) — Postgres TLS opt-in
 - [`docs/reference/protocols/scep-server.md`](../reference/protocols/scep-server.md) — SCEP RFC 8894 native server reference
 - [`docs/reference/protocols/est.md`](../reference/protocols/est.md) — EST RFC 7030 server reference

docs/operator/observability.md

@@ -0,0 +1,214 @@
+# Observability — what certctl emits, what it doesn't, and what survives a restart
+
+> Last reviewed: 2026-05-13
+
+Use this when:
+- You're sizing certctl's observability surface against your existing
+  metrics + tracing + logging stack and want to know exactly what
+  drops in cleanly and what gaps you'll need to bridge.
+- You're investigating a "weird metric" or planning a Grafana
+  dashboard and need the canonical list of what's exposed.
+- You're running multi-replica or restarting frequently and need to
+  understand which counters reset.
+
+certctl's observability posture is deliberately minimal-but-honest:
+ship the surfaces an operator actually needs to wire into a Prometheus
++ Grafana + Loki stack, and don't make claims the implementation
+can't back. This document is the canonical statement of what's
+emitted, what's deferred, and why.
+
+## Metrics — what's emitted
+
+certctl exposes metrics through two endpoints on the control plane:
+
+| Endpoint                          | Content-Type                                                      | Audience                         |
+|---|---|---|
+| `GET /api/v1/metrics`             | `application/json`                                                | Dashboards that prefer JSON, ad-hoc curl |
+| `GET /api/v1/metrics/prometheus`  | `text/plain; version=0.0.4; charset=utf-8` (Prometheus exposition) | Prometheus, Grafana Agent, Datadog Agent, Victoria Metrics, any OpenMetrics-compatible scraper |
+
+The Prometheus endpoint emits standard `# HELP` / `# TYPE` / metric
+lines following the conventions at
+[prometheus.io/docs/instrumenting/exposition_formats](https://prometheus.io/docs/instrumenting/exposition_formats/).
+Metric names are lowercase, snake_case, and prefixed with `certctl_`.
+
+The implementation is at
+[`internal/api/handler/metrics.go`](../../internal/api/handler/metrics.go).
+
+### What's covered
+
+Run the endpoint against a live deployment for the authoritative list
+(it expands as the service ships more metrics). At time of writing the
+exposition includes:
+
+- Certificate-inventory gauges: `certctl_certificate_total`,
+  `certctl_certificate_active`, `certctl_certificate_expiring_soon`,
+  `certctl_certificate_expired`, `certctl_certificate_revoked`.
+- Per-issuer-type issuance histograms:
+  `certctl_issuance_duration_seconds{issuer_type=…}` (the 2026-05-01
+  issuer-coverage audit closure #4 — this is the load-bearing metric
+  for per-issuer SLOs).
+- Server uptime: `certctl_uptime_seconds`.
+
+### Prometheus library vs hand-rolled exposition (acquisition diligence)
+
+certctl writes Prometheus exposition format with `fmt.Fprintf` from
+the metrics handler, not via the `github.com/prometheus/client_golang`
+library. This is intentional for v2.x:
+
+- The metric surface is shallow (gauges + a handful of histograms with
+  static labels). The client library's value is on the registration +
+  thread-safe accumulation side, neither of which is load-bearing for
+  the current surface.
+- The exposition output is pinned to the spec version explicitly
+  (`version=0.0.4`) and is unit-tested against expected output at
+  `internal/api/handler/stats_handler_test.go`.
+- Swapping in `client_golang` is a mechanical migration when the
+  metric surface grows (per-connector counters + RED-method histograms
+  on every handler are the natural next surface), but it has no
+  operator-visible behavior change today.
+
+The migration is on the
+[WORKSPACE-ROADMAP.md](../../WORKSPACE-ROADMAP.md) as a v3 item. If
+you're an acquirer reading this: the question to ask is "does the
+metric surface meet our SLO needs today" — not "is the right library
+under the hood." If the answer to the first question is yes, the
+second is a refactor, not a feature gap.
+
+## Tracing — explicitly not yet shipped
+
+certctl does **not** ship distributed tracing instrumentation today:
+
+- No OpenTelemetry SDK setup in `cmd/server/main.go`.
+- No OTLP exporter wired into outbound calls (issuer connectors,
+  agent enrollment, etc.).
+- The `go.opentelemetry.io/otel` packages that appear in
+  [`go.mod`](../../go.mod) are indirect-only — they're transitive
+  dependencies of `coreos/go-oidc` and similar.
+
+This is honest: there is no in-process tracing surface to monitor,
+correlate, or sample. If your environment requires end-to-end traces
+across the certctl control plane + agents + issuer backends, this is
+a gap you would close on the certctl side as part of a v3 work item.
+Until then:
+
+- Structured logs include a `request_id` you can correlate across
+  the server log stream. See
+  [`internal/api/middleware/request_id.go`](../../internal/api/middleware/request_id.go).
+- The Prometheus histogram
+  `certctl_issuance_duration_seconds{issuer_type=…}` carries the
+  same per-issuer latency signal a trace span would, just without
+  the per-request fan-out.
+
+OpenTelemetry instrumentation is tracked in
+[WORKSPACE-ROADMAP.md](../../WORKSPACE-ROADMAP.md) as a v3 item.
+
+## Logging
+
+certctl emits structured JSON logs to stdout via the stdlib
+`log/slog` package. Every line carries `time`, `level`, `msg`, and —
+where relevant — `request_id`, `actor_id`, and a contextual subject
+(`certificate_id`, `issuer_id`, `agent_id`, etc.).
+
+Log level is controlled by `CERTCTL_LOG_LEVEL` (`debug` / `info` /
+`warn` / `error`); defaults to `info`. There is no in-process log
+ingest — operators are expected to collect from container stdout
+into their existing log pipeline (Loki, CloudWatch Logs, Datadog,
+ELK, Splunk, etc.).
+
+No log line contains private-key material, bearer tokens, OIDC
+client secrets, or session cookies. The break-glass login path
+explicitly scrubs the password before it reaches the audit subsystem
+(see [`docs/operator/auth-threat-model.md`](auth-threat-model.md) §
+"Break-glass token leak").
+
+## Rate-limit behavior under restarts and replicas
+
+Where rate limits exist, they are **per-process, in-memory,
+reset-on-restart, and not shared across replicas**. This matters for
+multi-replica deployments and for any compliance posture that asks
+"what limits apply globally vs per-pod."
+
+### Inventory
+
+| Limiter                                              | Scope                | Window | Cap                            | Survives restart? | Shared across replicas? |
+|---|---|---|---|---|---|
+| Break-glass login (per source-IP)                    | `internal/api/handler/auth_breakglass.go` | 60s   | 5 attempts                     | No                | No                      |
+| SCEP/Intune per-device challenge                     | `internal/scep/intune/`                   | 60s   | configurable (`*_PER_MINUTE`)  | No                | No                      |
+| EST per-principal CSR enrollment                     | `internal/est/`                           | 60s   | configurable                   | No                | No                      |
+| EST HTTP-Basic source-IP failed-auth                 | `internal/est/`                           | 60s   | configurable                   | No                | No                      |
+| ACME per-account orders / key-change / challenge-respond | `internal/service/acme.go`            | 1h    | configurable                   | No                | No                      |
+
+All five use the shared `internal/ratelimit/sliding_window.go`
+primitive. Buckets live in a single per-process map guarded by a
+mutex; the package-level cap prevents unbounded growth under
+adversarial key cardinality (default 100,000 keys; oldest-by-newest-
+timestamp evicted under pressure).
+
+### Implications for multi-replica deployments
+
+- **Effective per-replica cap is the documented cap.** A 2-replica
+  deployment lets through up to 2× the per-key window cap before
+  either replica rejects.
+- **Restart resets the bucket.** A `kubectl rollout restart` empties
+  the in-memory windows on every replica. An attacker who notices
+  this could in principle re-issue burst attempts after every roll;
+  the threat model accepts this because rollouts are operator-driven
+  and the relevant endpoints already require credentials.
+- **No cross-replica fan-out.** Rate-limit decisions on replica A
+  are not visible to replica B. Sticky-session ingress routing (with
+  `service.spec.sessionAffinity: ClientIP` on Kubernetes or the
+  equivalent on your load balancer) tightens the effective cap to
+  per-replica + per-source-IP rather than per-replica + per-source-IP
+  for whichever pod the request happened to land on.
+
+If your threat model requires globally-enforced rate limits across
+replicas, the implementation surface is roughly: swap the per-process
+map for a database-backed sliding window (or a Redis-backed equivalent
+if you already run Redis). This is on the
+[WORKSPACE-ROADMAP.md](../../WORKSPACE-ROADMAP.md) as a v3 item;
+nothing in the certctl threat model today requires it.
+
+### Where these numbers live
+
+The configurable caps are exposed as `CERTCTL_*_PER_MINUTE` /
+`CERTCTL_ACME_*_PER_HOUR` env vars — see the
+[security posture](security.md) doc for the operator-facing
+configuration surface. The hard-coded ones (break-glass 5/min) are
+intentionally non-configurable as a defense-in-depth measure; the
+auth subsystem owns that policy decision.
+
+## Performance harness scope
+
+The load-test harness at [`deploy/test/loadtest/`](../../deploy/test/loadtest/)
+covers the API-tier hot paths (issuance acceptance + cert list). It
+does NOT load-test issuer-connector round-trips (you'd be load-
+testing someone else's API), full multi-RTT ACME enrollment flows,
+bulk-revoke / bulk-renew admin paths, or scheduler concurrency under
+bulk renewal. Each exclusion is justified in
+[`deploy/test/loadtest/README.md`](../../deploy/test/loadtest/README.md)
+under "What it explicitly does NOT measure." If your evaluation
+requires a benchmark on one of those exclusions, the right next step
+is a follow-up scenario in that directory.
+
+The per-component benchmarks ship in-tree as Go `Benchmark*`
+functions:
+- `internal/auth/session/bench_test.go` — session signing + validation
+  steady state and cold-process timing.
+- `internal/auth/oidc/bench_test.go` — OIDC verify steady state.
+- `internal/auth/oidc/bench_keycloak_test.go` — OIDC cold-cache timing
+  (gated `//go:build integration`).
+
+Authoritative benchmark numbers + threshold contracts:
+[`docs/operator/auth-benchmarks.md`](auth-benchmarks.md) (auth
+subsystem) and [`docs/operator/performance-baselines.md`](performance-baselines.md)
+(general API tier).
+
+## Related reading
+
+- [`docs/operator/security.md`](security.md) — the broader hardening
+  posture; this document is its observability subset.
+- [`docs/operator/performance-baselines.md`](performance-baselines.md) — operator-runnable benchmarks against the API tier
+- [`docs/operator/auth-benchmarks.md`](auth-benchmarks.md) — session
+  + OIDC validation timings + threshold contracts
+- [`deploy/test/loadtest/README.md`](../../deploy/test/loadtest/README.md) — k6 load-test harness scope + threshold contract
+- [`docs/operator/runbooks/postgres-backup.md`](runbooks/postgres-backup.md) — operator-run backup recipe (separate file because it's a procedural runbook, not an observability claim)

docs/operator/oidc-runbooks/auth0.md

@@ -0,0 +1,198 @@
+# Auth0 OIDC runbook
+
+> Last reviewed: 2026-05-10
+
+This runbook wires certctl's OIDC SSO surface against [Auth0](https://auth0.com/), a commercial cloud IdP (now part of Okta but operationally distinct). Auth0 has a free developer tier suitable for evaluation; production runs on a paid B2B / B2C plan.
+
+For the canonical reference + mental model, read [keycloak.md](keycloak.md) first; this runbook only documents the Auth0-specific deltas.
+
+## The big Auth0 quirk: namespaced custom claims
+
+Auth0 imposes a hard rule: any custom claim emitted from an Action MUST use a namespaced URL-shape key (e.g. `https://your-namespace/groups`). Auth0 silently strips claims that look like standard OIDC claims (`groups`, `roles`, `permissions`, etc.) when emitted from an Action — this is a security feature to prevent claim-spoofing.
+
+certctl handles this via the `groups_claim_path` config. If your Action emits `https://your-namespace/groups`, set `OIDCProvider.groups_claim_path` to that exact URL. The hand-rolled groupclaim resolver at `internal/auth/oidc/groupclaim/resolver.go` recognizes URL-shape paths (anything starting with `http://` or `https://`) and treats the entire string as a single literal key — it does NOT split on `/`.
+
+Set `groups_claim_format` to `string-array`; the underlying claim shape is still a JSON array of group-name strings, just stored under a URL-shape key.
+
+## Prerequisites
+
+**On the Auth0 side:**
+
+- An Auth0 tenant (free dev tier at <https://auth0.com/signup> works). Tenant URL looks like `https://<tenant-name>.<region>.auth0.com`.
+- Owner or Auth0 Administrator role.
+- Network reachability from certctl-server to `https://<tenant>.auth0.com/.well-known/openid-configuration`.
+
+**On the certctl side:** same as Keycloak.
+
+## IdP-side configuration
+
+### 1. Pick a namespace string
+
+Decide on a unique URL-shape namespace for certctl's custom claims. It does NOT have to resolve to a real domain; Auth0 just requires it to be URL-shape and unique within your tenant. A reasonable choice:
+
+```
+https://certctl.example.com/auth/
+```
+
+Use that prefix for every custom claim; for groups specifically:
+
+```
+https://certctl.example.com/auth/groups
+```
+
+We'll refer to this as `<NS>/groups` in the rest of this runbook.
+
+### 2. Create the Application
+
+In the Auth0 dashboard:
+
+**Applications → Applications → Create Application**:
+
+- Name: `certctl`.
+- Application Type: **Regular Web Applications**.
+- Click **Create**.
+
+On the saved app's **Settings** tab:
+
+- Application Login URI: blank (Auth0 doesn't need it for the auth-code flow).
+- Allowed Callback URLs: `https://<your-certctl-host>:8443/auth/oidc/callback` (one entry, exact match).
+- Allowed Logout URLs: optional.
+- Allowed Web Origins: `https://<your-certctl-host>:8443`.
+- Token Endpoint Authentication Method: **Post** (default; matches the certctl service's expectation of `client_secret_post`).
+- Save Changes.
+
+Copy the **Domain** (this is the issuer base — `https://<tenant>.auth0.com`), **Client ID**, and **Client Secret** from the same Settings page.
+
+### 3. Configure the connection (where users live)
+
+If you're using Auth0's Database connection (default username + password), the existing **Username-Password-Authentication** connection works. For SSO to Google / Microsoft / SAML, configure those connections under **Authentication → Enterprise** or **Authentication → Social** and ensure the connection is enabled on the certctl Application (App → Connections tab).
+
+### 4. Define the groups
+
+Auth0 doesn't have a first-class "Groups" concept like Okta or Keycloak — you have THREE options to model groups, each with tradeoffs:
+
+**Option A: User app_metadata (simplest, recommended for dev tier).**
+
+Each user has a `app_metadata` JSON blob you can set via the Management API, the dashboard, or a post-registration script. Stick the groups in there:
+
+```json
+{
+  "groups": ["certctl-engineers"]
+}
+```
+
+In the Auth0 dashboard, **User Management → Users → <user> → app_metadata**: paste the JSON above and Save.
+
+**Option B: Auth0 Authorization Extension (paid plans, recommended for production).**
+
+Install the Authorization Extension from **Marketplace → Extensions → Authorization**. It adds a first-class "Groups" concept with UI for assignment + nested groups. Read the extension's docs; it emits groups under `<NS>/groups` automatically once enabled.
+
+**Option C: Roles + Permissions (Auth0's RBAC primitive).**
+
+Use **User Management → Roles** to define roles like `certctl-engineer` + `certctl-viewer`. Assign roles to users. Have your Action emit role names as a `groups` claim. This is what Auth0 documents as the canonical pattern; it's slightly heavier than Option A but more discoverable in the dashboard.
+
+This runbook uses **Option A** for clarity; the Action below reads from `app_metadata.groups`.
+
+### 5. Write the Action that emits the groups claim
+
+**Actions → Library → Create Action → Build from scratch**:
+
+- Name: `certctl-emit-groups`.
+- Trigger: **Login / Post Login**.
+- Runtime: Node 18.
+- Click **Create**.
+
+Paste this code:
+
+```javascript
+exports.onExecutePostLogin = async (event, api) => {
+  const namespace = "https://certctl.example.com/auth/";
+  const groups = (event.user.app_metadata && event.user.app_metadata.groups) || [];
+  if (groups.length > 0) {
+    api.idToken.setCustomClaim(namespace + "groups", groups);
+    api.accessToken.setCustomClaim(namespace + "groups", groups);
+  }
+};
+```
+
+Replace `https://certctl.example.com/auth/` with your namespace from step 1. Click **Deploy**.
+
+Then bind the Action to the Login flow:
+
+**Actions → Flows → Login**: drag `certctl-emit-groups` from the Custom tab into the flow, between Start and Complete. Click **Apply**.
+
+### 6. Verify the claim in a test login
+
+Auth0's **Authentication → Authentication Profile → Try It** button or the **Logs → Real-time Logs** page can show you the issued ID token in real time. Decode at jwt.io to confirm `<NS>/groups` is present + populated.
+
+## certctl-side configuration
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/providers \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "Auth0",
+    "issuer_url": "https://<tenant>.auth0.com/",
+    "client_id": "<paste-from-step-2>",
+    "client_secret": "<paste-from-step-2>",
+    "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+    "groups_claim_path": "https://certctl.example.com/auth/groups",
+    "groups_claim_format": "string-array",
+    "fetch_userinfo": false,
+    "scopes": ["openid", "profile", "email"],
+    "iat_window_seconds": 300,
+    "jwks_cache_ttl_seconds": 3600
+  }'
+```
+
+Critical:
+
+- `issuer_url` includes the **trailing slash** for Auth0 (`https://<tenant>.auth0.com/`). Auth0's `iss` claim emits with the trailing slash; mismatching trips `ErrIssuerMismatch`.
+- `groups_claim_path` is the **full namespaced URL**, not the bare `groups` key. The certctl resolver treats this as a single literal lookup key against the ID token claims map (no path-walking through `/`).
+
+Add the group→role mappings: `certctl-engineers` → `r-operator`, etc. The mapping table maps the group VALUES (the strings inside the claim's array), not the claim path.
+
+## Verification
+
+End-to-end login + audit + Sessions checks are identical to Keycloak. The audit row's `details.subject` will be Auth0's user_id (e.g. `auth0|abc123…` for database users, `google-oauth2|...` for federated), stable across email changes.
+
+## Troubleshooting
+
+**`ErrGroupsUnmapped` even though I see groups in the ID token at jwt.io.**
+
+Check `groups_claim_path` exactly matches the namespaced key in the token. A common mistake: setting `groups_claim_path` to `groups` (the bare key) when the actual claim key is `https://certctl.example.com/auth/groups` (the namespaced version). The resolver's URL-shape detection is what makes the namespaced path work; if the claim path doesn't start with `http://` or `https://`, the resolver tries to walk it as a dot-separated path and fails.
+
+**The `<NS>/groups` claim is missing from the ID token.**
+
+- Action not bound to the Login flow: revisit step 5's "Apply" step.
+- Action returns early because `event.user.app_metadata.groups` is undefined: confirm the user has the metadata set.
+- Trying to set the claim under a non-namespaced key (e.g. `api.idToken.setCustomClaim("groups", groups)`): Auth0 silently drops it. Always use the namespace prefix.
+
+**Auth0 returns "Service not found" or "Invalid audience".**
+
+This usually means the certctl client wasn't authorized to access the userinfo endpoint or the application's `audience` setting conflicts with the OIDC discovery doc. The certctl service uses the Application's `client_id` as the `audience` claim — confirm Auth0 is emitting tokens with `aud = <client_id>` (decode at jwt.io).
+
+**Login redirects loop between Auth0 and certctl.**
+
+Most often a callback-URL mismatch — Auth0's "Allowed Callback URLs" must contain the EXACT certctl callback URL including port + scheme. Wildcards aren't allowed in production.
+
+**`email_verified` is `false` and certctl rejects the user.**
+
+certctl doesn't currently gate on `email_verified` — the User row stores email regardless. If your operator policy requires verified-only, add an Action that throws on `event.user.email_verified === false`:
+
+```javascript
+if (!event.user.email_verified) {
+  api.access.deny("email-not-verified");
+}
+```
+
+## Validation checklist
+
+Same as [keycloak.md](keycloak.md#validation-checklist) with Auth0-specific values, plus:
+
+- [ ] The `<NS>/groups` claim is present in the ID token (verify via jwt.io decode).
+- [ ] Removing a user's group from `app_metadata.groups` causes the next login to land on "no roles assigned".
+- [ ] The Auth0 dashboard's **Logs → Real-time Logs** shows the certctl callback completing with HTTP 302 to the dashboard.
+
+Sign-off: _______________ (operator) on _______________ (date).

docs/operator/oidc-runbooks/authentik.md

@@ -0,0 +1,144 @@
+# Authentik OIDC runbook
+
+> Last reviewed: 2026-05-10
+
+This runbook wires certctl's OIDC SSO surface against [Authentik](https://goauthentik.io/), a free / open-source IdP that runs on-prem or self-hosted. Authentik shares the canonical "string-array groups claim under the `groups` key" pattern with Keycloak — the differences are in the admin console UX and the explicit "property mapping" abstraction.
+
+For the canonical reference + mental model, read [keycloak.md](keycloak.md) first; this runbook only documents the Authentik-specific deltas.
+
+## Prerequisites
+
+**On the Authentik side:**
+
+- Authentik ≥ 2024.10 (stable channel).
+- Admin access to the Authentik admin console at `https://<authentik-host>/if/admin/`.
+- Network reachability from certctl-server to `https://<authentik-host>/application/o/<application-slug>/.well-known/openid-configuration`.
+
+**On the certctl side:** same as Keycloak — `CERTCTL_CONFIG_ENCRYPTION_KEY` set, an admin actor holding `auth.oidc.create` + `auth.oidc.edit`, server build ≥ v2.1.0.
+
+## IdP-side configuration
+
+### 1. Create the OAuth2 / OpenID Provider
+
+In the Authentik admin console:
+
+**Applications → Providers → Create**:
+
+- Type: **OAuth2/OpenID Provider**.
+- Name: `certctl`.
+- Authorization flow: `default-provider-authorization-explicit-consent` (or `default-provider-authorization-implicit-consent` if you don't want a consent screen on every login).
+- Click **Next**.
+
+Protocol settings:
+
+- Client type: **Confidential**.
+- Client ID: leave the auto-generated value OR set to `certctl` for clarity.
+- Client Secret: copy the auto-generated value to a secure scratchpad — you'll paste it into certctl.
+- Redirect URIs/Origins: `https://<your-certctl-host>:8443/auth/oidc/callback` (one entry, exact match).
+- Signing Key: pick an **RSA-2048 or larger** key. Authentik defaults to ECDSA-P256 in newer versions; either is fine — both are in certctl's allow-list.
+- Subject mode: **Based on the User's hashed ID** (default; emits a stable opaque `sub`).
+- Include claims in id_token: **on**.
+- Click **Finish**.
+
+### 2. Create the Application
+
+Applications are how Authentik attaches a Provider to users + groups + policies.
+
+**Applications → Applications → Create**:
+
+- Name: `certctl`.
+- Slug: `certctl` (becomes part of the issuer URL: `https://<authentik-host>/application/o/certctl/`).
+- Provider: pick the `certctl` provider you just created.
+- Policy engine mode: **any** (default).
+- Click **Create**.
+
+### 3. Configure the groups property mapping
+
+Authentik emits group claims via "property mappings" — explicit objects rather than Keycloak's mapper-on-the-client model.
+
+By default, the **Authentik default-OAuth Mapping: Proxy outpost** scope already includes the user's groups under a `groups` claim (string-array, matches what certctl expects). To verify or override:
+
+**Customization → Property Mappings → Filter "Scope Mapping"**:
+
+- Find or create one named `groups` with scope `groups` and expression:
+  ```python
+  return [group.name for group in user.ak_groups.all()]
+  ```
+- Description: `Emits the user's group names as a string-array claim`.
+
+Then on the **Provider → certctl → Edit → Advanced protocol settings**, ensure **Scopes** includes `groups` (and `profile` and `email` if you want richer User records on the certctl side).
+
+### 4. Create the groups + assign users
+
+**Directory → Groups → Create**:
+
+- Name: `certctl-engineers`. Repeat for `certctl-viewers` (and optionally `certctl-admins`).
+
+**Directory → Users → <user> → Edit → Groups**: pick the appropriate `certctl-*` group(s) for each user.
+
+### 5. (Optional) Bind the application to specific groups
+
+If you want certctl to reject login attempts from users outside the `certctl-*` groups at the IdP layer (defense-in-depth on top of certctl's fail-closed `ErrGroupsUnmapped`):
+
+**Applications → certctl → Policy / Group / User Bindings → Create binding**:
+
+- Type: **Group**.
+- Group: pick the union of `certctl-*` groups you want to allow.
+- Enabled: on.
+
+## certctl-side configuration
+
+Identical to Keycloak — only the issuer URL differs:
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/providers \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "Authentik",
+    "issuer_url": "https://authentik.example.com/application/o/certctl/",
+    "client_id": "<paste-the-client-id>",
+    "client_secret": "<paste-the-client-secret>",
+    "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+    "groups_claim_path": "groups",
+    "groups_claim_format": "string-array",
+    "fetch_userinfo": false,
+    "scopes": ["openid", "profile", "email", "groups"],
+    "iat_window_seconds": 300,
+    "jwks_cache_ttl_seconds": 3600
+  }'
+```
+
+Authentik emits `groups` in the ID token by default once the property mapping is configured. The `scopes` array MUST include `groups` to trigger the claim emission — Authentik is stricter than Keycloak about scope-gating claims.
+
+Add the group→role mappings the same way as Keycloak: `certctl-engineers` → `r-operator`, `certctl-viewers` → `r-viewer`.
+
+## Verification
+
+End-to-end login + audit + Sessions checks are identical to Keycloak.
+
+**Authentik-specific check:** the audit row's `details.subject` will be Authentik's hashed user ID (a 64-char hex), not the username. This is intentional and correct — the `sub` claim must be opaque + stable across user-attribute changes.
+
+**JWKS-rotation drill:** Authentik rotates signing keys via **System → Tokens & App Passwords → Certificates** (rename of "Crypto" in newer versions). Add a new RSA-2048 cert, switch the Provider's Signing Key to the new one, then click "Refresh discovery cache" in certctl's GUI to evict the cache.
+
+## Troubleshooting
+
+**Provider creation fails with "could not load discovery document".**
+The issuer URL needs the trailing slash for some Authentik versions: `https://authentik.example.com/application/o/certctl/` (slash after the slug). Without the slash, Authentik returns a 301 redirect that Go's HTTP client follows but discovery parsing chokes on the redirect target.
+
+**Login completes but user lands on "no roles assigned".**
+Decode the ID token at jwt.io against Authentik's JWKS. Check whether the `groups` claim is present + non-empty. If empty, the property mapping isn't wired — go back to step 3.
+
+**`groups` claim missing entirely.**
+Authentik gates the `groups` claim behind the `groups` scope. Verify:
+- The certctl OIDCProvider config has `"scopes": ["openid", "profile", "email", "groups"]`.
+- The Authentik provider's "Scopes" list includes `groups`.
+
+**Authentik emits the user's full DN as the `sub` claim.**
+Some Authentik configurations use **Subject mode: Based on the User's email** which surfaces the email as `sub`. This works but tightly couples certctl's User table to email mutability; recommend switching to "hashed ID" mode for new deployments. Existing User rows in certctl's `users` table will have email-shaped `oidc_subject` columns; that's fine and stable as long as the user's email never changes.
+
+## Validation checklist
+
+Same as [keycloak.md](keycloak.md#validation-checklist), with Authentik-specific values for issuer URL + group names + signing-key rotation steps.
+
+Sign-off: _______________ (operator) on _______________ (date).

docs/operator/oidc-runbooks/azure-ad.md

@@ -0,0 +1,207 @@
+# Microsoft Entra ID (Azure AD) OIDC runbook
+
+> Last reviewed: 2026-05-10
+
+This runbook wires certctl's OIDC SSO surface against [Microsoft Entra ID](https://learn.microsoft.com/entra/), formerly Azure AD. Entra ID is Microsoft's commercial cloud IdP; it's the default IdP for any organization on Microsoft 365 / Azure.
+
+For the canonical reference + mental model, read [keycloak.md](keycloak.md) first; this runbook only documents the Entra-ID-specific deltas.
+
+## The big Entra ID quirk: groups claim emits OBJECT IDs, not names
+
+Entra ID's `groups` claim emits a JSON array of **group object IDs (GUIDs)**, not human-readable names. A user in `Engineering Group` and `Cert Operators` will see something like:
+
+```json
+{
+  "groups": [
+    "8b9b1faa-4e83-471e-8b00-7d99c3e2a5f1",
+    "f00cf1e2-2db1-4cdf-a1ba-1234567890ab"
+  ]
+}
+```
+
+**You must configure your certctl group→role mappings against these GUIDs**, not against `Engineering Group` or `Cert Operators`. There are workarounds (cloud-only group display names + the optional claims path; see the alternative below) but the GUID-based approach is the only one that works reliably across all Entra ID configurations.
+
+This is by design at Microsoft — group names are mutable and not globally unique within a tenant; object IDs are immutable and globally unique. Operators on Microsoft 365 / Azure deployments are accustomed to managing access by GUID.
+
+## Prerequisites
+
+**On the Entra ID side:**
+
+- A Microsoft 365 tenant or standalone Azure AD tenant. Free Azure AD tier is sufficient; paid tiers (P1/P2) unlock conditional access + SCIM provisioning + risk-based auth, none of which are required for the basic OIDC integration.
+- Application Administrator or Global Administrator role.
+- Network reachability from certctl-server to `https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration`.
+
+**On the certctl side:** same as Keycloak.
+
+## IdP-side configuration
+
+### 1. Register the application
+
+In the [Entra ID admin center](https://entra.microsoft.com/):
+
+**Applications → App registrations → New registration**:
+
+- Name: `certctl`.
+- Supported account types: **Accounts in this organizational directory only** (single-tenant; matches the typical operator use case).
+- Redirect URI: **Web** + `https://<your-certctl-host>:8443/auth/oidc/callback`.
+- Click **Register**.
+
+On the saved app's **Overview** page, copy:
+
+- **Application (client) ID** → certctl's `client_id`.
+- **Directory (tenant) ID** → goes into the issuer URL.
+
+### 2. Create a client secret
+
+**App → Certificates & secrets → Client secrets → New client secret**:
+
+- Description: `certctl-server`.
+- Expires: 6 months / 12 months / 24 months — your choice. Set a calendar reminder; Entra ID does NOT auto-rotate secrets.
+- Click **Add**.
+
+Copy the **Value** column immediately — it's shown ONCE on creation. The certctl provider's `client_secret` field gets this value.
+
+(Production hardening: prefer **Certificates** over secrets for client authentication; certctl currently supports `client_secret_post` only, but a follow-on bundle can add `private_key_jwt` for cert-based client auth. Track this if you have a hard requirement against shared secrets.)
+
+### 3. Add the `groups` claim to the token
+
+**App → Token configuration → Add groups claim**:
+
+- Pick **Security groups** (covers most operators) OR **Groups assigned to the application** (more granular but requires Premium).
+- Token type: **ID token** + **Access token** (both, so userinfo fallback works).
+- Customize emit format for ID/access: leave as **Group ID** (default; this is the GUID-based path the runbook is structured around).
+- Click **Save**.
+
+If you instead want display names in the claim (only works for cloud-only groups; on-prem-synced groups continue to emit GUIDs regardless):
+
+- Customize emit format → **Cloud-only group display names**.
+- BUT — note this works only for groups created in Entra ID itself, not groups synced from on-prem AD. Hybrid environments will have inconsistent claims.
+
+### 4. Add the optional `email` and `profile` claims
+
+By default Entra ID's ID token does NOT include `email` — Microsoft considers email part of the "OIDC profile" but only emits it under specific conditions. To force emission:
+
+**App → Token configuration → Add optional claim → ID token → email**.
+
+You may also want `family_name`, `given_name`, `preferred_username` for richer User records on the certctl side.
+
+### 5. Grant the API permissions
+
+**App → API permissions**:
+
+- Microsoft Graph → Delegated permissions → ensure these are granted (most are default):
+  - `openid`
+  - `profile`
+  - `email`
+  - `offline_access` (optional; for refresh tokens — certctl doesn't use them currently).
+- Click **Grant admin consent** if your tenant requires it.
+
+### 6. (Optional) Restrict who can sign in
+
+By default any user in your tenant can attempt to sign in to the app. To restrict to specific users / groups:
+
+**Enterprise applications → certctl → Properties → Assignment required: Yes**.
+Then **Users and groups → Add user/group** and pick the `cert-engineers` / `cert-viewers` Entra ID groups.
+
+## certctl-side configuration
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/providers \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "Entra ID",
+    "issuer_url": "https://login.microsoftonline.com/<tenant-id>/v2.0",
+    "client_id": "<application-id>",
+    "client_secret": "<client-secret-value>",
+    "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+    "groups_claim_path": "groups",
+    "groups_claim_format": "string-array",
+    "fetch_userinfo": false,
+    "scopes": ["openid", "profile", "email"],
+    "iat_window_seconds": 300,
+    "jwks_cache_ttl_seconds": 3600
+  }'
+```
+
+Notes:
+
+- `issuer_url` MUST include `/v2.0` at the end for the v2.0 endpoint. The v1.0 endpoint emits tokens with a different `iss` shape and is NOT supported by certctl. The discovery doc at `https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration` confirms the right path.
+- `<tenant-id>` is the Directory (tenant) ID GUID from step 1.
+
+### Add the group→role mappings (GUID-keyed)
+
+Get the GUIDs of your engineering / viewer groups:
+
+**Entra ID → Groups → All groups → <group> → Overview → Object ID**.
+
+Then in certctl:
+
+```bash
+# Engineering group → r-operator
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/group-mappings \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "provider_id": "<provider-id>",
+    "group_name": "8b9b1faa-4e83-471e-8b00-7d99c3e2a5f1",
+    "role_id": "r-operator"
+  }'
+```
+
+Repeat for every group you want to map. **Document the GUID-to-name mapping in your operator runbook** — without it, the next operator looking at certctl's mappings page sees a wall of GUIDs with no way to know which is which. Consider naming the mapping descriptively if your group-mapping schema supports it (v2.1.0 doesn't yet — group-mapping descriptions are a parking-lot item for a follow-on release).
+
+## Verification
+
+End-to-end login + audit + Sessions checks are identical to Keycloak.
+
+**Entra-ID-specific:** the audit row's `details.subject` will be Microsoft's `oid` claim (a GUID, the user's object ID), stable across UPN / email changes. The certctl `users` table's `oidc_subject` column holds this GUID.
+
+**JWKS-rotation:** Microsoft auto-rotates signing keys on a documented schedule (every ~6 weeks). The discovery doc + JWKS endpoint always serve the union of active + recently-active keys, so in-flight logins continue to validate. No manual operator action needed in steady state. If you suspect a stuck cache after a Microsoft-side rotation, click "Refresh discovery cache" in the certctl GUI to evict.
+
+## Troubleshooting
+
+**Login completes; ID token contains a `hasgroups: true` claim instead of `groups`.**
+
+Entra ID emits this when a user is in too many groups (>200 by default for ID tokens, >150 for access tokens) — Microsoft truncates the claim and tells the consumer to use Microsoft Graph to look up the full list. certctl does NOT currently support the Graph fallback path (it's a follow-on bundle item).
+
+Workarounds:
+
+- Reduce the user's group membership to <200 (rarely practical in large tenants).
+- Restrict the `groups` claim to "Groups assigned to the application" (Token configuration step 3 above) instead of "Security groups". The "assigned" set is bounded by the app's user assignments and stays under the limit.
+- Use Entra ID's optional `wids` (well-known IDs) claim if you only care about admin/non-admin distinction; certctl can be configured against `wids` by setting `groups_claim_path` accordingly.
+
+**`groups` claim missing entirely.**
+
+Step 3 wasn't completed — Entra ID does NOT emit `groups` by default. Add the claim via Token configuration before users will see it.
+
+**`ErrIssuerMismatch` even though the `tid` in the token matches.**
+
+The v2.0 endpoint emits `iss = https://login.microsoftonline.com/<tenant-id>/v2.0` (no trailing slash). The v1.0 endpoint emits `iss = https://sts.windows.net/<tenant-id>/`. Confirm certctl's `issuer_url` matches v2.0 exactly — no trailing slash, includes `/v2.0`.
+
+**On-prem-synced groups emit GUIDs even when "Cloud-only display names" is selected.**
+
+Expected behavior — Microsoft only emits display names for groups created in Entra ID itself (cloud-only). On-prem-synced groups always emit object IDs. The hybrid case is unfixable from the IdP side; either map against GUIDs (recommended) or migrate the relevant groups to cloud-only.
+
+**The `email` claim is empty even though the user has a primary email.**
+
+Entra ID's `email` claim only populates when:
+1. The user has a "Primary email" set on their Entra ID profile (often blank for B2B guest users).
+2. The optional claim was added in step 4.
+
+For B2B guests, the `preferred_username` claim usually carries the email-shape login. You can configure certctl to use `preferred_username` as the user's display name fallback, but the `User.Email` column will remain blank — that's expected for guests.
+
+**Conditional Access policies blocking the login.**
+
+If your tenant has Conditional Access requiring MFA for new applications, certctl will see the user redirected through the MFA challenge. This works transparently — the certctl service doesn't care that MFA was performed; it only validates the resulting ID token. If MFA is failing for the user, debug at the Entra ID side (Sign-in logs).
+
+## Validation checklist
+
+Same as [keycloak.md](keycloak.md#validation-checklist), with these additions:
+
+- [ ] The ID token's `groups` claim is a string-array of GUIDs (decode at jwt.io).
+- [ ] Each certctl group-mapping uses the GUID, not a human-readable name.
+- [ ] A user with >200 groups successfully logs in (or the operator has documented the limitation + workaround in their internal runbook).
+- [ ] The Entra ID **Sign-in logs** view shows the certctl login event with status "Success".
+
+Sign-off: _______________ (operator) on _______________ (date).

docs/operator/oidc-runbooks/google-workspace.md

@@ -0,0 +1,186 @@
+# Google Workspace OIDC runbook (broker via Keycloak)
+
+> Last reviewed: 2026-05-10
+
+This runbook wires certctl's OIDC SSO surface against [Google Workspace](https://workspace.google.com/) (formerly G Suite). Google's OIDC implementation has a well-known limitation that makes it unsuitable for direct integration with certctl: **the ID token does not emit a groups claim**, so there is no way for certctl's `ErrGroupsUnmapped` fail-closed contract to resolve a user's role assignment.
+
+The recommended pattern is to **broker Google Workspace through Keycloak (or Authentik)** as a federated identity provider. The end-user still signs in with their Google account, but certctl talks to Keycloak — which DOES emit groups — instead of talking to Google directly.
+
+For the canonical reference + mental model, read [keycloak.md](keycloak.md) first; this runbook builds on top of it.
+
+## The Google Workspace quirk in detail
+
+**What Google emits in an ID token:** `iss`, `aud`, `sub`, `azp`, `exp`, `iat`, `email`, `email_verified`, `name`, `picture`, `given_name`, `family_name`, `locale`, `hd` (hosted domain). That's it.
+
+**What it does NOT emit:** `groups`, `roles`, `permissions`, or any indicator of the user's Google Workspace organizational unit / group membership.
+
+There is a **Cloud Identity Groups API** at `https://cloudidentity.googleapis.com/v1/groups/-/memberships:searchTransitiveGroups` that lets a privileged service account look up a user's groups, but:
+
+1. It requires a service account with domain-wide delegation, which is a major security surface to grant to certctl.
+2. It's a separate REST call after the OIDC flow, not a claim — certctl's group-claim resolver is path-shape, not API-shape.
+3. The latency budget of an extra API call per login is non-trivial in steady state.
+
+For these reasons, the broker pattern is strongly preferred. If you absolutely cannot deploy a broker, see "Direct integration without groups" at the bottom of this runbook for a degraded mode where every Google-authenticated user gets a single fixed role.
+
+## Architecture: broker pattern
+
+```
+end user → Google Workspace login → Keycloak (federated IdP) → certctl
+                                       ↑
+                                       │
+                          adds groups claim from Keycloak's group store
+                          (NOT from Google)
+```
+
+In this topology:
+
+- The end user's authentication credentials live at Google.
+- The user's group / role assignments live at Keycloak (manually or via SCIM provisioning from Google).
+- certctl talks ONLY to Keycloak. From certctl's perspective this is identical to the [keycloak.md](keycloak.md) runbook.
+
+## Prerequisites
+
+- A running Keycloak instance with a realm dedicated to certctl. Read [keycloak.md](keycloak.md) and complete that runbook FIRST against a local-only test user. Verify end-to-end OIDC works against Keycloak before adding Google as a federated provider.
+- A Google Workspace tenant where you have Super Admin access OR can ask your Workspace admin to create OAuth credentials.
+- A Google Cloud project (free; same console as Workspace).
+
+## IdP-side configuration
+
+### Step 1: create a Google OAuth client
+
+In the Google Cloud Console (`https://console.cloud.google.com/`):
+
+**APIs & Services → OAuth consent screen → Configure**:
+
+- User Type: **Internal** (restricts to your Workspace domain) OR **External** (any Google account; usually NOT what you want for an internal cert-management tool).
+- App name: `certctl SSO via Keycloak`.
+- User support email: your team's address.
+- Authorized domains: add the domain Keycloak runs on.
+- Save.
+
+**APIs & Services → Credentials → Create Credentials → OAuth client ID**:
+
+- Application type: **Web application**.
+- Name: `certctl-via-keycloak`.
+- Authorized redirect URIs: `https://<keycloak-host>/realms/<realm-name>/broker/google/endpoint` — this is Keycloak's default federated-IdP callback URL. Get the exact URL from Keycloak in step 2 below.
+- Click **Create**.
+
+Copy the **Client ID** and **Client secret**.
+
+### Step 2: add Google as a federated identity provider in Keycloak
+
+In the Keycloak admin console (`https://<keycloak-host>/admin/`):
+
+**Realm → Identity providers → Add provider → Google**:
+
+- Alias: `google` (becomes part of the broker URL).
+- Display name: `Google Workspace`.
+- Client ID: paste from step 1.
+- Client secret: paste from step 1.
+- Default scopes: `openid profile email`.
+- Hosted Domain: your Workspace domain (e.g. `example.com`); restricts to your tenant.
+- Sync mode: **Force** (rewrites the user's first/last name/email from Google on every login; the alternative `Import` only writes on first login).
+- Trust email: **on** (Google verifies emails; certctl-Keycloak chain inherits the trust).
+- Click **Save**.
+
+The **Redirect URI** field at the top of the saved provider's page shows the exact URL you should have entered in Google's console at step 1. Re-verify match.
+
+### Step 3: configure group assignment in Keycloak
+
+This is the load-bearing step — we're explicitly NOT trusting Google for groups, so Keycloak has to provide them.
+
+**Option A: Manual group assignment in Keycloak.**
+
+Federated users from Google appear in **Users** in Keycloak after their first login. You assign them to `certctl-engineers` / `certctl-viewers` / etc. groups in Keycloak's UI manually. Pro: simple. Con: doesn't scale; new hires can't log in until an operator adds them to a group.
+
+**Option B: Default groups via "Default Groups" realm config.**
+
+**Realm settings → User registration → Default Groups → Add**: pick the lowest-privilege group (e.g. `certctl-viewers`). Every new federated user lands here automatically; operators promote individual users to higher groups as needed.
+
+**Option C: Mapper that derives groups from Google claims.**
+
+If your Google Workspace has organizational units that align with your role split, you can add a Keycloak **Identity Provider Mapper** that maps `hd` (hosted domain) or a custom Google directory custom-schema field to a Keycloak group. This is moderately fragile and Workspace-version-dependent; recommend B for most operators.
+
+**Option D: SCIM provisioning from Google to Keycloak.**
+
+Google Workspace can SCIM-push group memberships to Keycloak via the SCIM-for-Google-Cloud-Identity feature. Heavyweight; recommend only if you already have SCIM infrastructure.
+
+This runbook uses **Option B** (default group) for clarity.
+
+### Step 4: verify the broker flow at Keycloak alone
+
+Before bringing certctl into the picture:
+
+1. Log out of Keycloak's admin console.
+2. Hit `https://<keycloak-host>/realms/<realm-name>/account` in an incognito window.
+3. Click "Sign in" — Keycloak's login page should now show **Sign in with Google Workspace** as a button below the local login form.
+4. Click it; authenticate via Google; you should land on Keycloak's account page.
+5. Back in the admin console, the user appears under **Users**. Confirm they're in the default group (Option B).
+
+Only proceed to step 5 when Keycloak alone works end to end.
+
+### Step 5: configure certctl against Keycloak (NOT against Google)
+
+Follow the [keycloak.md](keycloak.md) runbook. Use the realm + client + groups configuration you set up there. The `OIDCProvider.issuer_url` is `https://<keycloak-host>/realms/<realm-name>` — Keycloak's URL, not Google's.
+
+When the user clicks "Sign in with Keycloak" on certctl's login page, the browser flow is:
+
+1. certctl → Keycloak authorize endpoint.
+2. Keycloak's login page shows **Sign in with Google Workspace** + the local login form. User clicks Google.
+3. Keycloak → Google authorize endpoint. User authenticates at Google.
+4. Google → Keycloak callback (`/broker/google/endpoint`). Keycloak resolves the user, assigns the default group.
+5. Keycloak → certctl callback. certctl sees a normal Keycloak ID token with the `groups` claim populated by Keycloak.
+6. certctl mints the session.
+
+End-to-end the user clicks twice (Keycloak's "Sign in with Google" button + Google's consent / login). Subsequent logins skip the consent screen if Google's session is fresh.
+
+## Verification
+
+End-to-end login + audit + Sessions checks are identical to Keycloak. The key Google-Workspace-specific check:
+
+- The `users.oidc_subject` column in certctl's database should contain the Keycloak-side stable subject (a UUID), NOT the Google subject. Decode the certctl-side ID token and confirm `iss` is Keycloak's URL, `sub` is the Keycloak UUID. Don't confuse the certctl ID token with Google's ID token (which lives one hop upstream and certctl never sees directly).
+
+## Direct integration without groups (NOT RECOMMENDED)
+
+If broker deployment is impossible:
+
+1. Configure certctl with `issuer_url = https://accounts.google.com`, `client_id` + `client_secret` from your Google OAuth client (with redirect URI pointed at certctl directly).
+2. Add a SINGLE group→role mapping where `group_name` is the empty string. **Wait — certctl rejects empty group names.** This is the structural reason this mode doesn't work: the fail-closed contract requires a real group claim to match.
+
+The actual workaround is to manually add EVERY operator's email to a per-email mapping, OR to add a custom claim emitter at a thin proxy in front of Google. Both are hacks; the broker pattern is strictly better. We document the constraint here so future operators don't burn cycles trying to make it work.
+
+## Troubleshooting
+
+**Federated Google login completes at Keycloak but the user lands on "no roles assigned" at certctl.**
+
+The user authenticated through Google → Keycloak successfully but Keycloak didn't assign them a group (Option A wasn't completed for that user, or Option B's default group isn't mapped on the certctl side). Check:
+
+- Keycloak → Users → <user> → Groups: is the user in any `certctl-*` group?
+- certctl → Auth → OIDC Providers → Keycloak → Group → role mappings: is that group mapped?
+
+**Google login fails with "redirect_uri_mismatch".**
+
+The Google OAuth client's authorized redirect URI doesn't match Keycloak's broker callback URL exactly. Re-fetch the URL from Keycloak (Identity Providers → Google → Redirect URI field) and paste it verbatim into Google's console.
+
+**Google auto-closes the consent prompt and returns "access_denied".**
+
+Workspace admin policies may block third-party app access. Either the Google OAuth client wasn't approved by the Workspace admin (Google Workspace Admin Console → Security → API controls → Trusted apps), or the OAuth consent screen is configured for "External" but the user is from a different Workspace. Switch to "Internal" if everyone signing in is in the same Workspace.
+
+**Keycloak log shows "Federated identity returned no email claim".**
+
+You requested OAuth scopes other than `openid profile email`. Re-add `email` to the Default Scopes on the Keycloak Identity Provider config.
+
+**Sign-out from certctl doesn't sign the user out of Google.**
+
+Expected. certctl revokes its own session; Google's session continues independently. If the user needs to fully log out, they sign out at https://accounts.google.com/Logout. The certctl + Keycloak chain is the standard "single sign-on, separate sign-outs" model.
+
+## Validation checklist
+
+Same as [keycloak.md](keycloak.md#validation-checklist), with these additions:
+
+- [ ] Google → Keycloak federation works without certctl in the loop (step 4 above passes).
+- [ ] A first-time Google sign-in lands the user in the Keycloak default group (or whatever Option you picked).
+- [ ] The certctl audit row's `details.subject` is the Keycloak UUID, NOT Google's `sub` (which would be a Google account ID).
+- [ ] Removing a user from Google Workspace causes their NEXT certctl session-validate to fail (after their existing session expires) — verify with a deactivated test user.
+
+Sign-off: _______________ (operator) on _______________ (date).

docs/operator/oidc-runbooks/index.md

@@ -0,0 +1,55 @@
+# OIDC / SSO runbooks — per-IdP setup guides
+
+> Last reviewed: 2026-05-10
+
+This is the index for the per-IdP setup runbooks for certctl's OIDC SSO surface. Pick the runbook that matches your identity provider; each one walks you through the IdP-side configuration, the certctl-side configuration, end-to-end verification, and the most common troubleshooting paths.
+
+For the threat model behind certctl's OIDC implementation, see [`auth-threat-model.md`](../auth-threat-model.md). For the RBAC primitive that group→role mappings target, see [`rbac.md`](../rbac.md). For the underlying protocol details (PKCE, state, nonce, JWKS rotation, fail-closed semantics), see the OIDC service docstring at [`internal/auth/oidc/service.go`](../../../internal/auth/oidc/service.go).
+
+## Choose your runbook
+
+| IdP | Tier | Group claim shape | Quirks | Runbook |
+|---|---|---|---|---|
+| Keycloak | Free / open-source | `string-array` against `groups` | None — canonical reference | [keycloak.md](keycloak.md) |
+| Authentik | Free / open-source | `string-array` against `groups` | Property-mapping driven; explicit scope claim | [authentik.md](authentik.md) |
+| Okta | Commercial (free dev tier) | `string-array` against `groups` | Group-filter regex on the claim definition | [okta.md](okta.md) |
+| Auth0 | Commercial (free dev tier) | `string-array` against namespaced URL | Custom claims must use a namespaced key (e.g. `https://your-namespace/groups`) and are emitted via an Action | [auth0.md](auth0.md) |
+| Azure AD / Entra ID | Commercial | `string-array` of GROUP OBJECT IDs (GUIDs), not names | Mappings must target object IDs, not human-readable names | [azure-ad.md](azure-ad.md) |
+| Google Workspace | Commercial | NO native group claim | Direct OIDC against Google Workspace cannot emit groups; broker through Keycloak (or Authentik) instead | [google-workspace.md](google-workspace.md) |
+
+## Common shape
+
+Every runbook follows the same five-section layout so you can scan across IdPs:
+
+1. **Prerequisites** — what you need on the IdP side (admin access, plan tier) and on the certctl side (an admin actor holding `auth.oidc.create` + `auth.oidc.edit`, the GUI / CLI / MCP surface available, the `CERTCTL_CONFIG_ENCRYPTION_KEY` env var set in production so client_secret encrypts at rest).
+2. **IdP-side configuration** — clickable steps in the IdP admin console, with the exact field names and values certctl needs.
+3. **certctl-side configuration** — `POST /api/v1/auth/oidc/providers` payloads, plus the GUI and MCP equivalents. The wire shape is the same across every IdP; only the values differ.
+4. **Verification** — what a successful end-to-end login looks like in the audit log and the GUI Sessions page, plus the JWKS-rotation drill.
+5. **Troubleshooting** — the failure modes you're statistically most likely to hit, mapped to the certctl service-layer sentinel error you'll see in the audit row.
+
+## Cross-IdP recurring concepts
+
+These show up in every runbook; understand them once and skim the rest.
+
+**Redirect URI.** Every IdP needs the certctl-side callback URL registered as an allowed redirect URI. The format is `https://<your-certctl-host>/auth/oidc/callback` — port 8443 by default for the HTTPS-only control plane (Decision: post-v2.2 the platform is HTTPS-only, no plaintext port). For local-dev fixtures, `http://localhost:8443/auth/oidc/callback` is acceptable; production deployments MUST use HTTPS, and the OIDCProvider domain validator rejects HTTP issuer URLs in non-test paths.
+
+**Client secret rotation.** Every IdP issues a `client_secret` for the confidential client (certctl is always a confidential client; public clients aren't supported because we have a server-side place to keep the secret). Rotating at the IdP requires the operator to PUT the new secret into certctl via the GUI's "Edit provider" dialog or `certctl_auth_update_oidc_provider` MCP tool — leaving `client_secret` empty in the update payload preserves the existing ciphertext, providing a value rotates.
+
+**JWKS cache TTL.** The certctl service caches the IdP's JWKS document for `jwks_cache_ttl_seconds` (default 3600). When the IdP rotates a signing key, in-flight logins that try to validate a new-key-signed token against the stale cache fail with `ErrJWKSUnreachable` until the next refresh. Operators have two options: wait out the TTL, or click "Refresh discovery cache" in the GUI's OIDC Provider Detail page (`POST /api/v1/auth/oidc/providers/{id}/refresh`) to force-evict the cache. The Keycloak integration test exercises this drill end to end.
+
+**Group→role mappings are fail-closed.** The certctl service refuses to mint a session for a user whose IdP-supplied groups don't match ANY configured mapping (`ErrGroupsUnmapped` → HTTP 401 to the user with a "no roles assigned" page). This is intentional — empty mapping ≠ "let everyone in," it means "this provider is not yet configured for any role." Operators add at least one mapping (typically `<engineers-group>` → `r-operator`) BEFORE rolling out OIDC to users.
+
+**Nonce + state + PKCE-S256 are non-negotiable.** Every login flow round-trips a nonce (replay defense), a state (CSRF defense), and a PKCE-S256 verifier (RFC 9700 §2.1.1 mandate). `plain` PKCE is rejected at the service-layer sentinel level. None of this is configurable; if your IdP doesn't support PKCE-S256, you cannot use it with certctl.
+
+**IdP downgrade-attack defense.** At provider creation AND on every JWKS refresh, certctl intersects the IdP's advertised `id_token_signing_alg_values_supported` with the certctl allow-list (RS256, RS512, ES256, ES384, EdDSA by default). If the IdP advertises HS256/HS384/HS512 or `none`, provider creation is rejected — even before any token is signed under the weak alg. This catches the case where a future compromised or misconfigured IdP tries to rotate to an alg-confusion-prone setup.
+
+## When you finish a runbook
+
+Each per-IdP runbook ends with a **validation checklist** the operator runs against a real production-tier deployment. Run through the matrix end-to-end against your IdP and mark your sign-off in the runbook's footer — that gives the next operator (or the next you) a dated record of what's been verified to work.
+
+## Related docs
+
+- [RBAC operator reference](../rbac.md) — roles, permissions, scope-down + bootstrap flow.
+- [Auth threat model](../auth-threat-model.md) — API-key + OIDC + session compromise scenarios; v3 WebAuthn pairing.
+- [Security posture](../security.md) — overall auth surface including this OIDC layer.
+- [API keys → RBAC migration](../../migration/api-keys-to-rbac.md) — the v2.0.x → v2.1.0 RBAC upgrade flow your operator likely already ran.

docs/operator/oidc-runbooks/keycloak.md

@@ -0,0 +1,245 @@
+# Keycloak OIDC runbook
+
+> Last reviewed: 2026-05-10
+
+This is the canonical reference runbook for wiring certctl's OIDC SSO surface against [Keycloak](https://www.keycloak.org/). Keycloak is a free / open-source identity provider that runs on-prem or self-hosted; it is also the load-bearing test fixture for certctl's OIDC integration tests (`internal/auth/oidc/testfixtures/keycloak.go`), so the certctl-side validation pipeline is exhaustively exercised against it.
+
+If your IdP is something else (Okta, Auth0, Azure AD, Authentik, Google Workspace), see the per-IdP siblings in [this directory](index.md). The mental model + certctl-side wiring are identical; only the IdP-side console differs.
+
+## Prerequisites
+
+**On the Keycloak side:**
+
+- Keycloak ≥ 25.0 (older versions work but the screen flows differ slightly — the integration test fixture pins 25.0).
+- Admin access to a realm — either an existing tenant realm or a fresh one created for certctl. Don't share Keycloak's `master` realm; create a dedicated realm.
+- Network reachability from certctl-server to the Keycloak `https://<keycloak-host>/realms/<realm-name>` discovery endpoint. The certctl service fetches `/.well-known/openid-configuration` at provider creation and at every `RefreshKeys` call.
+- Keycloak's signing alg set to RS256 (default) or any of: RS512, ES256, ES384, EdDSA. HS256/HS384/HS512 + `none` are rejected by certctl's IdP-downgrade-attack defense at provider creation time.
+
+**On the certctl side:**
+
+- `CERTCTL_CONFIG_ENCRYPTION_KEY` set to a stable secret (production deployments only — the encryption-at-rest layer for the OIDC client_secret depends on it).
+- An admin actor holding `auth.oidc.create` + `auth.oidc.edit` (held by `r-admin` by default; granted via `certctl_auth_assign_role_to_key` MCP tool or the GUI's Auth → Keys page).
+- Server build ≥ v2.1.0.
+
+## IdP-side configuration
+
+The same configuration you'll do by hand here is what the testcontainers fixture imports from `internal/auth/oidc/testfixtures/keycloak-realm.json` — read that file alongside this runbook to see the exact JSON shape Keycloak persists.
+
+### 1. Create or pick a realm
+
+In the Keycloak admin console (`https://<keycloak-host>/admin/`), drop into the realm you'll use. If creating a new one, the realm name will become part of the issuer URL: `https://<keycloak-host>/realms/<realm-name>`.
+
+### 2. Create the OIDC client
+
+**Clients → Create client**:
+
+- Client type: **OpenID Connect**
+- Client ID: `certctl` (or whatever you prefer; it goes into `OIDCProvider.client_id` on the certctl side).
+- Always display in console: off.
+- Click **Next**.
+
+On the capability config page:
+
+- Client authentication: **On** (this makes the client confidential, which is what certctl requires).
+- Authorization: off.
+- Standard flow: **on** (auth-code with PKCE — this is the path certctl uses).
+- Direct access grants: off (ROPC; the test fixture turns this on for ROPC convenience but production should NOT).
+- Implicit flow: off.
+- Service accounts roles: off.
+- Click **Next**.
+
+Login settings:
+
+- Root URL: leave blank.
+- Home URL: blank.
+- Valid redirect URIs: `https://<your-certctl-host>:8443/auth/oidc/callback` — ONE entry, exact match. Wildcards (`*`) work for local dev (`http://localhost:*`) but production should pin the exact host.
+- Valid post logout redirect URIs: blank or `+` (matches the redirect URI list).
+- Web origins: `+` (matches the redirect URI origin) or empty.
+- Click **Save**.
+
+On the saved client's **Credentials** tab, copy the **Client secret** — you'll need it for the certctl-side payload.
+
+### 3. Create the groups
+
+**Groups → Create group**:
+
+- Repeat for every certctl role you want to map to a group. A typical setup creates two:
+  - `certctl-engineers` (intended target: `r-operator`)
+  - `certctl-viewers` (intended target: `r-viewer`)
+- Optionally an `certctl-admins` group → `r-admin` for break-glass-free first-admin bootstrap; see the [`auth-threat-model.md`](../auth-threat-model.md) section on bootstrap admins.
+
+### 4. Configure the group-membership claim mapper
+
+This is the load-bearing step — without it, the ID token won't carry a `groups` claim and every login fails closed with `ErrGroupsUnmapped`.
+
+**Clients → certctl → Client scopes → certctl-dedicated → Add mapper → By configuration → Group Membership**:
+
+- Name: `groups`
+- Token Claim Name: `groups`
+- Full group path: **off** (so the claim emits `engineers`, not `/engineers`; matches the certctl `string-array` group-claim format).
+- Add to ID token: **on**.
+- Add to access token: **on** (optional but recommended; the userinfo-fallback path uses it).
+- Add to userinfo: **on**.
+- Click **Save**.
+
+### 5. Create the user(s)
+
+**Users → Add user**:
+
+- Username: `alice` (or however you identify operators).
+- Email: required (used as the certctl-side `User.Email`).
+- First name + last name: optional but populates `User.DisplayName`.
+- Email verified: **on** if you trust the user.
+- Click **Create**.
+
+On the saved user's **Credentials** tab:
+- Set a password. Mark **Temporary** if you want the user to reset on first login.
+
+On the **Groups** tab:
+- Join the user to the group(s) you created in step 3.
+
+## certctl-side configuration
+
+### Via the GUI
+
+1. Sign in as an admin actor.
+2. Navigate to **Auth → OIDC Providers** in the sidebar.
+3. Click **Configure provider**.
+4. Fill in:
+   - **Display name**: `Keycloak` (free-text; what end-users see on the login page button).
+   - **Issuer URL**: `https://<keycloak-host>/realms/<realm-name>`.
+   - **Client ID**: `certctl` (matches step 2 above).
+   - **Client secret**: paste the secret from step 2's Credentials tab.
+   - **Redirect URI**: `https://<your-certctl-host>:8443/auth/oidc/callback`.
+   - **Groups claim path**: `groups` (the default; matches step 4's Token Claim Name).
+   - **Groups claim format**: `string-array` (the default).
+   - **Fetch userinfo**: off (Keycloak emits groups in the ID token; userinfo fallback is for IdPs that don't).
+   - **Scopes**: `openid profile email` (the certctl service prepends `openid` if missing).
+   - **IAT window seconds**: 300 (default).
+   - **JWKS cache TTL seconds**: 3600 (default).
+5. Click **Save**.
+
+If the discovery doc fetch fails, the modal surfaces the error inline. The most common cause is a typo in the issuer URL — Keycloak emits 404 for any path under `/realms/` that doesn't match an actual realm.
+
+### Via the API
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/providers \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "Keycloak",
+    "issuer_url": "https://keycloak.example.com/realms/certctl",
+    "client_id": "certctl",
+    "client_secret": "<paste-the-secret>",
+    "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+    "groups_claim_path": "groups",
+    "groups_claim_format": "string-array",
+    "fetch_userinfo": false,
+    "scopes": ["openid", "profile", "email"],
+    "iat_window_seconds": 300,
+    "jwks_cache_ttl_seconds": 3600
+  }'
+```
+
+### Via MCP
+
+```
+certctl_auth_create_oidc_provider {
+  "name": "Keycloak",
+  "issuer_url": "https://keycloak.example.com/realms/certctl",
+  "client_id": "certctl",
+  "client_secret": "<paste-the-secret>",
+  "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+  "groups_claim_path": "groups",
+  "groups_claim_format": "string-array",
+  "scopes": ["openid", "profile", "email"]
+}
+```
+
+### Add the group→role mappings
+
+GUI: **Auth → OIDC Providers → Keycloak → Group → role mappings → Add**.
+
+- IdP group: `certctl-engineers` → certctl role: `r-operator`.
+- IdP group: `certctl-viewers` → certctl role: `r-viewer`.
+
+API equivalent: `POST /api/v1/auth/oidc/group-mappings` with `{"provider_id": "<id>", "group_name": "certctl-engineers", "role_id": "r-operator"}`. MCP equivalent: `certctl_auth_add_group_mapping`.
+
+Empty mapping list = nobody can log in via Keycloak (the fail-closed contract). Add at least one before announcing the SSO endpoint to users.
+
+## Verification
+
+### End-to-end login
+
+1. Open `https://<your-certctl-host>:8443/login` in a fresh incognito window.
+2. The page renders an OIDC button block with `Sign in with Keycloak` (the display name from the create-provider step).
+3. Click it. The browser redirects to Keycloak, you authenticate as `alice`, Keycloak redirects back to certctl, and you land on the dashboard.
+4. Navigate to **Auth → Sessions**. You should see a row with your own actor ID, the IP you logged in from, and the current timestamp under "last seen".
+
+### Audit trail
+
+```bash
+curl https://<your-certctl-host>:8443/api/v1/audit?category=auth \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" | jq '.events[] | select(.action == "auth.oidc_login_succeeded")'
+```
+
+You should see a row for the login above, with `details.provider_id` matching the Keycloak provider's id and `details.subject` set to the Keycloak user's `sub` claim (typically a UUID).
+
+### JWKS-rotation drill
+
+Operator action when Keycloak rotates its realm signing key:
+
+1. In Keycloak: **Realm settings → Keys → Providers → Add provider → rsa-generated**, set priority higher than the current key (e.g. 200), enabled = on, active = on.
+2. In certctl: GUI → **Auth → OIDC Providers → Keycloak → Refresh discovery cache** button. Or the CLI / MCP equivalent: `POST /api/v1/auth/oidc/providers/<id>/refresh`.
+3. Run another login. The new ID token is signed under the new key; the certctl service validates it against the freshly-fetched JWKS doc.
+
+The Keycloak integration test `TestKeycloakIntegration_JWKSRotation_RefreshKeysPicksUpNewKey` exercises this exact flow end to end.
+
+## Troubleshooting
+
+**"Discovery doc fetch failed" at provider creation.**
+The most common cause is a wrong issuer URL — typo in realm name, missing `/realms/` segment, or HTTP→HTTPS redirect that the Go client doesn't follow without explicit headers. Curl the URL manually:
+```
+curl -v https://<keycloak-host>/realms/<realm-name>/.well-known/openid-configuration
+```
+If that returns 404, fix the realm name. If it returns 200 but certctl still fails, check `cmd/server` logs for the wrapped error.
+
+**"IdP downgrade-attack defense" rejected provider creation.**
+Keycloak's realm has a signing key advertised in `id_token_signing_alg_values_supported` that's in certctl's deny-list (HS256/HS384/HS512/`none`). Check **Realm settings → Keys → Providers** — disable any HMAC key providers and re-create the provider in certctl.
+
+**Login redirects to Keycloak, the user authenticates, but the callback redirects back to `/login` with "no roles assigned".**
+The user authenticated successfully but their groups didn't match any configured mapping (`ErrGroupsUnmapped`). Check:
+- The user is actually a member of the group you mapped (Users → user → Groups tab in Keycloak).
+- The group-membership mapper is configured correctly (Clients → certctl → Client scopes → certctl-dedicated → mappers → groups → "Full group path: off" matters).
+- The group name in your certctl mapping exactly matches what Keycloak emits — case-sensitive, no leading slash if "Full group path: off".
+
+You can confirm what Keycloak is actually emitting by decoding the ID token at jwt.io against the Keycloak public key, or by enabling certctl's debug logging on the OIDC service for one login (logs are scrubbed of token contents per the OIDC service's token-leak hygiene contract; debug logs surface only the resolved group list and the mapping decision).
+
+**"id_token verify failed: token used before issued"**
+Clock skew between Keycloak and certctl-server. Either align both to NTP, or bump `iat_window_seconds` on the OIDC provider config (default 300 = 5 minutes). The certctl service caps `iat_window_seconds` at 600.
+
+**"oidc: pre-login session not found or already consumed"**
+The user clicked the OIDC login button, then the browser tab idled past the 10-minute pre-login TTL OR the user opened the IdP login in a new tab and consumed the row from the first one. Have them retry.
+
+**"oidc: state parameter mismatch (replay or forgery)"**
+Either the user double-submitted a callback URL (clicked it twice from email or browser history), or a CSRF attempt. The pre-login row is single-use; second consumption returns `ErrPreLoginNotFound`. Have them retry from the login page.
+
+**Sessions revoked but the user can still hit the API.**
+Check the session contract: the cookie is HMAC-validated on every request, but the actual database row is what `Revoke` deletes. If your reverse proxy is caching the response or the `__Host-certctl_session` cookie wasn't actually cleared on the client, the cookie will hit the server's session middleware which will return 401 on the missing-row lookup. The middleware never serves stale data; the issue is upstream of certctl in this case.
+
+## Validation checklist
+
+Before signing off this runbook for production rollout, validate these end-to-end:
+
+- [ ] `auth.oidc_provider_created` audit row appears after the create-provider POST.
+- [ ] `Sign in with Keycloak` button renders on the login page after `getAuthInfo` returns the configured provider.
+- [ ] A user with mapped groups completes the auth-code flow and lands on the dashboard.
+- [ ] A user WITHOUT mapped groups gets the "no roles assigned" landing (not the dashboard).
+- [ ] The `auth.oidc_login_succeeded` and `auth.oidc_login_failed` audit rows correctly distinguish the two cases.
+- [ ] The Sessions page shows the new session, with self-pill on the caller's row.
+- [ ] Revoking the session via the GUI causes the next API request from that browser to 401 + redirect to login.
+- [ ] Running the JWKS-rotation drill (steps above) does not break in-flight logins; rotated tokens validate against the refreshed JWKS.
+- [ ] Editing the provider with `client_secret` blank preserves the existing ciphertext (operator confirms by reading the `oidc_providers.client_secret_encrypted` column before + after the PUT — bytes unchanged).
+
+Sign-off: _______________ (operator) on _______________ (date).

docs/operator/oidc-runbooks/okta.md

@@ -0,0 +1,143 @@
+# Okta OIDC runbook
+
+> Last reviewed: 2026-05-10
+
+This runbook wires certctl's OIDC SSO surface against [Okta](https://www.okta.com/), a commercial cloud IdP. Okta offers a free developer tier (`https://dev-NNNNN.okta.com`) suitable for evaluation; production runs on a paid Workforce Identity tenant.
+
+For the canonical reference + mental model, read [keycloak.md](keycloak.md) first; this runbook only documents the Okta-specific deltas.
+
+## Prerequisites
+
+**On the Okta side:**
+
+- A Workforce Identity tenant (or free Developer Edition account at <https://developer.okta.com/signup/>).
+- Super Admin or Application Admin role in your Okta tenant.
+- Network reachability from certctl-server to `https://<your-org>.okta.com/.well-known/openid-configuration` OR to a custom authorization server endpoint if you're using one (`https://<your-org>.okta.com/oauth2/<auth-server-id>/.well-known/openid-configuration`).
+
+**On the certctl side:** same as Keycloak.
+
+## IdP-side configuration
+
+### 1. Create the OIDC application
+
+In the Okta admin console:
+
+**Applications → Applications → Create App Integration**:
+
+- Sign-in method: **OIDC - OpenID Connect**.
+- Application type: **Web Application**.
+- Click **Next**.
+
+App config:
+
+- App integration name: `certctl`.
+- Logo: optional.
+- Grant types: **Authorization Code** (CHECK). Leave Refresh Token unchecked unless you have a specific reason — certctl doesn't currently use refresh tokens.
+- Sign-in redirect URIs: `https://<your-certctl-host>:8443/auth/oidc/callback`.
+- Sign-out redirect URIs: optional; leave empty unless you also configure RP-initiated logout.
+- Trusted Origins: leave default.
+- Assignments → Controlled access: **Limit access to selected groups** (recommended; pick the `certctl-*` groups from step 3 below).
+- Click **Save**.
+
+On the saved app's **General** tab, copy the **Client ID** and **Client secret** (under Client Credentials). The secret is shown once on creation — copy it immediately or rotate via "Generate new secret".
+
+### 2. Pick or create an authorization server
+
+Okta has TWO authorization-server tiers:
+
+- **The Org Authorization Server** at `https://<your-org>.okta.com` — emits ID tokens with limited claims; cannot host custom claims directly. Use for the simplest setup.
+- **A Custom Authorization Server** at `https://<your-org>.okta.com/oauth2/<auth-server-id>` — fully configurable scopes + claims + access policies. The free developer tier ships with a default custom server at `/oauth2/default`. Recommended for production.
+
+For this runbook we use the default custom server: `https://<your-org>.okta.com/oauth2/default`.
+
+### 3. Create the groups + assign users
+
+**Directory → Groups → Add Group**:
+
+- Repeat for `certctl-engineers`, `certctl-viewers`, optionally `certctl-admins`.
+
+**Directory → People → <user> → Groups**: assign each user to the appropriate `certctl-*` group(s).
+
+Then go back to the App from step 1 and on the **Assignments** tab, assign the `certctl-*` groups to the application. Without this assignment Okta will reject the user's login attempt at the IdP layer with "User is not assigned to the client application".
+
+### 4. Configure the groups claim
+
+This is the load-bearing Okta-specific step. The default authorization server does NOT emit a `groups` claim out of the box — you have to define it.
+
+**Security → API → Authorization Servers → default → Claims → Add Claim**:
+
+- Name: `groups`.
+- Include in token type: **ID Token, Always** (also tick Access Token if you want the userinfo-fallback path to work).
+- Value type: **Groups**.
+- Filter: pick **Matches regex** with the value `certctl-.*` so only the `certctl-*` groups are emitted (saves on token size; users in dozens of unrelated groups get a bloated token otherwise).
+- Disable claim: off.
+- Include in: **Any scope** (or pin to `openid` if you want the claim only on the certctl-flow).
+- Click **Create**.
+
+### 5. (Optional) Add `email` and `profile` claims
+
+The default custom server already emits `email` and `name` under the `profile` and `email` scopes — no action needed unless you've stripped them from a custom config.
+
+## certctl-side configuration
+
+```bash
+curl -X POST https://<your-certctl-host>:8443/api/v1/auth/oidc/providers \
+  -H "Authorization: Bearer ${CERTCTL_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "Okta",
+    "issuer_url": "https://your-org.okta.com/oauth2/default",
+    "client_id": "<paste-from-step-1>",
+    "client_secret": "<paste-from-step-1>",
+    "redirect_uri": "https://certctl.example.com:8443/auth/oidc/callback",
+    "groups_claim_path": "groups",
+    "groups_claim_format": "string-array",
+    "fetch_userinfo": false,
+    "scopes": ["openid", "profile", "email"],
+    "iat_window_seconds": 300,
+    "jwks_cache_ttl_seconds": 3600
+  }'
+```
+
+Notes:
+
+- `issuer_url` MUST match exactly what Okta emits as the `iss` claim. For the default custom server it's `https://<your-org>.okta.com/oauth2/default` (no trailing slash). The org server's issuer is just `https://<your-org>.okta.com` (no `/oauth2/...` path). Mismatching either side trips certctl's `ErrIssuerMismatch` sentinel.
+- The `groups` scope is NOT required in the scopes list — Okta emits the claim based on the claim definition's "Include in: any scope" setting. Adding `groups` to the scopes list is harmless if your custom server has the scope defined.
+
+Add the group→role mappings: `certctl-engineers` → `r-operator`, `certctl-viewers` → `r-viewer`, `certctl-admins` → `r-admin`.
+
+## Verification
+
+End-to-end login + audit + Sessions checks are identical to Keycloak.
+
+**Okta-specific:** the audit row's `details.subject` will be Okta's user UID (a 20-char alphanumeric string starting with `00u`), stable across email changes. The certctl `users` table's `oidc_subject` column will hold this UID.
+
+**Optional Okta smoke test in CI:** certctl ships an opt-in smoke test at `internal/auth/oidc/integration_okta_smoke_test.go` (build tags `integration && okta_smoke`). Set `OKTA_ISSUER` + `OKTA_CLIENT_ID` + `OKTA_CLIENT_SECRET` env vars and run `make okta-smoke-test` to drive a discovery + RefreshKeys round-trip against your live tenant. Pre-reqs: enable the Resource Owner Password (ROPC) grant on the application (Sign-On tab → Grant types → Resource Owner Password) for the smoke test only; production certctl uses auth-code-with-PKCE.
+
+**JWKS-rotation drill:** Okta auto-rotates signing keys every ~3 months and publishes the new key alongside the old in the JWKS doc for ~1 month overlap. Manual rotation: **Security → API → Authorization Servers → default → Keys → "Generate new key"**. After rotation, click "Refresh discovery cache" in certctl's GUI; new tokens validate immediately.
+
+## Troubleshooting
+
+**"User is not assigned to the client application" at the Okta login screen.**
+You created the app + the user but didn't assign the user to the app via a group. Either assign the user directly (App → Assignments → Assign to People) or assign the `certctl-*` groups to the app (App → Assignments → Assign to Groups).
+
+**Login completes but `groups` claim is empty in the ID token.**
+Most common Okta gotcha — the default custom server doesn't emit `groups` until you define the claim (step 4 above). Decode the ID token at jwt.io to confirm. If the claim is defined but empty, check the regex filter in step 4 — `certctl-.*` matches names like `certctl-engineers` but NOT `engineers`.
+
+**`ErrIssuerMismatch` after correctly configuring the discovery URL.**
+The issuer claim Okta puts in the ID token MUST match `OIDCProvider.IssuerURL` byte-for-byte, including trailing slash. The default custom server emits `https://<your-org>.okta.com/oauth2/default` (no trailing slash); the org server emits `https://<your-org>.okta.com`. Don't append a trailing slash to either.
+
+**Login succeeds but the certctl `User.Email` is empty.**
+The `email` scope wasn't requested OR the user's email isn't verified at Okta. Add `email` to the certctl scopes config and ensure Okta's user has a verified primary email.
+
+**Okta returns "PKCE code verifier required".**
+The certctl service hard-codes PKCE-S256 on every login (RFC 9700 mandate). If Okta is rejecting the verifier, the most likely cause is a misconfigured app type — confirm the Okta application is "Web Application" (which supports auth-code + PKCE), not "Single-Page Application" (which has different token-binding rules) or "Native App".
+
+**Custom-server access policies blocking the login.**
+By default the `default` custom authorization server has an "Access Policy" with one rule allowing all clients + all users. If you've tightened this (production hygiene), add a rule that allows the `certctl` client + the `certctl-*` groups: **Security → API → Authorization Servers → default → Access Policies → <policy> → Add Rule**.
+
+## Validation checklist
+
+Same as [keycloak.md](keycloak.md#validation-checklist), with Okta-specific values + the access-policy check above.
+
+Sign-off: _______________ (operator) on _______________ (date).

docs/operator/performance-baselines.md

@@ -101,6 +101,5 @@ Capture timing in your own loadtest-baselines log so future regressions surface
 
 ## Related docs
 
-- [`docs/contributor/ci-pipeline.md`](../contributor/ci-pipeline.md) — CI guard for performance regression
 - [`docs/operator/security.md`](security.md) — rate limit tuning
 - [`docs/reference/architecture.md`](../reference/architecture.md) — request path through handler → service → repository

docs/operator/rbac.md

@@ -0,0 +1,356 @@
+# RBAC operator reference
+
+> Last reviewed: 2026-05-11
+>
+> Audit 2026-05-11 A-8 follow-on: demo-mode residual-grants detector
+> + cleanup endpoint shipped. New env var:
+> `CERTCTL_DEMO_MODE_RESIDUAL_STRICT` (default `false`). Operator
+> workflow at
+> [`security.md#demo-to-production-cutover-audit-2026-05-11-a-8`](security.md#demo-to-production-cutover-audit-2026-05-11-a-8).
+
+This is the operator-facing reference for the role-based access
+control primitive in certctl.
+Read this if you're running certctl in production and need to grant /
+revoke access to API keys, set up the auditor split, or onboard the
+first admin.
+
+For the threat model behind these controls, see
+[`auth-threat-model.md`](auth-threat-model.md). For the migration
+flow from a pre-RBAC (v2.0.x) deployment, see
+[`docs/migration/api-keys-to-rbac.md`](../migration/api-keys-to-rbac.md).
+
+## Mental model
+
+Every action against the certctl HTTP / CLI / MCP / GUI surface is
+performed by an **actor** (an API key, an agent's machine identity,
+the synthetic demo-anon actor when the server runs in
+`CERTCTL_AUTH_TYPE=none` mode). Each actor holds zero or more
+**roles**. Each role grants a set of **permissions** at a **scope**.
+A request to a gated endpoint succeeds when the actor's effective
+permission set (the union across all held roles) contains the
+permission the endpoint requires.
+
+The schema lives in `migrations/000029_rbac.up.sql` and ships with
+seven seeded default roles + a 33-permission canonical catalogue.
+The middleware that gates requests lives at
+`internal/auth/require_permission.go`. The service-layer authorizer
+that resolves "actor → permissions" lives at
+`internal/service/auth/authorizer.go`.
+
+## Default roles (seeded by migration 000029)
+
+| Role | ID | Use case | Permission shape |
+|---|---|---|---|
+| Admin | `r-admin` | Operator with full control | Every permission in the canonical catalogue |
+| Operator | `r-operator` | Day-to-day cert lifecycle | `cert.*`, `profile.read`, `issuer.read`, `target.*`, `agent.read`, `audit.read` |
+| Viewer | `r-viewer` | Read-only console access | `*.read` for every resource type |
+| Agent | `r-agent` | Machine identity for `certctl-agent` | `cert.read` + `agent.heartbeat` + `agent.job.poll` + `agent.job.complete` + `agent.job.report` |
+| MCP | `r-mcp` | Operator-equivalent for the MCP server, minus destructive ops | Like Operator without `*.delete` |
+| CLI | `r-cli` | Day-to-day operator CLI | Like Operator + `auth.key.list` / `auth.key.create` / `auth.key.rotate` |
+| Auditor | `r-auditor` | Compliance reviewer | `audit.read` + `audit.export` ONLY |
+
+**Note on actor-type binding (Audit 2026-05-10 LOW-8):** Roles in
+the catalogue are NOT bound to a specific `actor_type`. `r-mcp` is
+named for clarity ("the role MCP service accounts hold") but the
+schema permits granting it to any actor — including a human OIDC
+user. Same goes for `r-cli` and `r-agent`. The role-grant API accepts
+`{actor_id, actor_type, role_id}` tuples; the `actor_type` constraint
+lives on the grant row, not the role definition. Operators who want
+to enforce "only API-key actors hold r-mcp" should write that as an
+operator-side policy + verify via a periodic audit query against
+`actor_roles` joined to `api_keys` / `users`. Native role-to-
+actor-type binding is on the v2 roadmap.
+
+The auditor split is the load-bearing one: an auditor cannot read
+certificates, profiles, or issuers - only audit events. That makes the
+role legitimate to hand to a SOC 2 / FedRAMP / PCI auditor without
+giving them the keys to the kingdom. The
+`internal/domain/auth/auditor_test.go` invariants pin this set going
+forward.
+
+The five **admin-only fine-grained perms** seeded by migration
+000030 gate the high-blast-radius endpoints:
+
+- `cert.bulk_revoke` - `POST /api/v1/certificates/bulk-revoke` and the EST sibling
+- `crl.admin` - `/api/v1/admin/crl/cache`
+- `scep.admin` - `/api/v1/admin/scep/intune/*`
+- `est.admin` - `/api/v1/admin/est/*`
+- `ca.hierarchy.manage` - `/api/v1/issuers/{id}/intermediates`, `/api/v1/intermediates/{id}`
+
+Only `r-admin` holds these by default. To delegate one, create a
+custom role with the specific perm and grant it to the right actor.
+
+## Permission catalogue
+
+The catalogue is namespaced. Permission strings are stable across
+releases; new permissions add to the namespace, never reshape an
+existing one. Run
+`certctl-cli auth permissions list` (or `GET /api/v1/auth/permissions`)
+for the live catalogue.
+
+| Namespace | Examples | What the namespace gates |
+|---|---|---|
+| `cert.*` | `cert.read`, `cert.issue`, `cert.revoke`, `cert.delete`, `cert.bulk_revoke` | The certificate lifecycle surface (`/api/v1/certificates`) |
+| `profile.*` | `profile.read`, `profile.edit`, `profile.delete` | `CertificateProfile` CRUD |
+| `issuer.*` | `issuer.read`, `issuer.edit`, `issuer.delete` | Issuer connector config |
+| `target.*` | `target.read`, `target.edit`, `target.delete` | Deployment target config |
+| `agent.*` | `agent.read`, `agent.edit`, `agent.retire`, `agent.heartbeat`, `agent.job.*` | Agent fleet + agent self-service endpoints |
+| `audit.*` | `audit.read`, `audit.export` | The audit-events surface |
+| `auth.role.*` | `auth.role.list`, `auth.role.create`, `auth.role.edit`, `auth.role.delete`, `auth.role.assign` | RBAC management |
+| `auth.key.*` | `auth.key.list`, `auth.key.create`, `auth.key.rotate`, `auth.key.delete` | API key management |
+| `auth.bootstrap.*` | `auth.bootstrap.use` | Day-0 first-admin path |
+| `crl.admin`, `scep.admin`, `est.admin`, `ca.hierarchy.manage` | (single perms) | The five admin-only fine-grained perms (see above) |
+| `job.*` | `job.read`, `job.cancel` | Deployment job lifecycle |
+| `approval.*` | `approval.read`, `approval.approve`, `approval.reject` | Two-person approval workflow (cert-issuance + profile-edit) |
+| `policy.*` | `policy.read`, `policy.edit`, `policy.delete` | Compliance policies + renewal policies |
+| `team.*`, `owner.*` | `team.read`, `team.edit`, `team.delete`, `owner.*` | Organizational metadata |
+| `notification.*` | `notification.read`, `notification.edit` | Notification queue + requeue |
+| `discovery.*` | `discovery.read`, `discovery.run`, `discovery.claim` | Agent + cloud-secret-store discovery |
+| `network_scan.*` | `network_scan.read`, `network_scan.edit`, `network_scan.run` | TLS network scanning + SCEP probing |
+| `healthcheck.*` | `healthcheck.read`, `healthcheck.edit`, `healthcheck.delete`, `healthcheck.acknowledge` | Uptime monitors |
+| `digest.*` | `digest.read`, `digest.send` | Operator-summary digest emails |
+| `verification.*` | `verification.read`, `verification.run` | Post-deploy verification |
+| `stats.read`, `metrics.read` | (single perms) | Dashboard summary + Prometheus exposition |
+
+The full catalogue lives in
+[`internal/domain/auth/validate.go`](../../internal/domain/auth/validate.go).
+The router-level enforcement sits in
+[`internal/api/router/router.go`](../../internal/api/router/router.go);
+the AST-level CI guard
+[`TestRouterRBACGateCoverage`](../../internal/api/router/router_rbac_coverage_test.go)
+pins the contract — adding a new state-changing or read endpoint
+without an `rbacGate` / `rbacGateScoped` wrap fails CI.
+
+## Scope semantics
+
+Permissions are granted at one of three scopes:
+
+- **`global`** - applies to every resource in the tenant. The
+  default for the seeded role grants. A `cert.read` grant at global
+  scope lets the actor read any certificate.
+- **`profile`** - applies only to the named `CertificateProfile`
+  (matched by ID). `cert.issue` at scope `profile`/`p-corp-cdn` lets
+  the actor issue against `p-corp-cdn` only.
+- **`issuer`** - applies only to the named issuer. Lets you grant
+  `issuer.edit` on the production issuer to a senior operator
+  without giving them edit on every issuer.
+
+Global beats specific: an actor with `cert.read` at global scope
+passes a `cert.read` check against any specific profile or issuer
+even if no scoped grant exists. The reverse is also true - a
+scoped grant doesn't satisfy a request against a different scope.
+The Authorizer's `CheckPermission` is the single point of truth.
+
+> **Note (deferral):** the `scope_id` column is not
+> currently FK-constrained against the resource tables. An
+> operator can grant a permission at scope `profile`/`p-bogus`
+> without `p-bogus` existing; the gate still works (no rows match
+> at request time), but the API does not 404 the grant. Strict-FK
+> closure is tracked for a follow-on release. See
+> `internal/repository/postgres/auth.go::AddPermission`'s
+> `TODO` comment.
+
+## Granting + revoking access
+
+### From the GUI
+
+`/auth/roles` lists every role; click into one to see its
+permissions and (if you hold `auth.role.edit`) add or remove a
+permission. `/auth/keys` lists every actor with role grants;
+click "Assign role" to grant, click the × on a role tag to revoke.
+
+The synthetic `actor-demo-anon` row is shown but flagged
+"system-managed" with the mutation buttons hidden - the server-side
+reserved-actor guard rejects mutations against it regardless.
+
+### From the CLI
+
+```bash
+# Identity probe - what can the current API key actually do?
+certctl-cli auth me
+
+# Roles
+certctl-cli auth roles list
+certctl-cli auth roles get r-admin
+
+# Permissions catalogue
+certctl-cli auth permissions list
+
+# Key → role assignment
+certctl-cli auth keys list
+certctl-cli auth keys assign alice --role r-operator
+certctl-cli auth keys revoke alice --role r-admin
+
+# Walk-every-key prompt for downgrade
+certctl-cli auth keys scope-down
+
+# Audit-driven role suggestion (last 30 days of audit events)
+certctl-cli auth keys scope-down --suggest
+certctl-cli auth keys scope-down --suggest --apply
+
+# JSON-driven scope-down for automation (Helm post-upgrade hook etc.)
+certctl-cli auth keys scope-down --non-interactive ./scope-down.json
+```
+
+The mutating role-lifecycle commands (`certctl-cli auth roles
+create / update / delete` + `roles add-permission / remove-permission`)
+are tracked as a follow-on; today, manage custom
+roles via the HTTP API or GUI.
+
+### From the HTTP API
+
+Every endpoint is documented in `api/openapi.yaml` under the `[Auth]`
+tag. Quick reference:
+
+| Endpoint | Permission |
+|---|---|
+| `GET /v1/auth/me` | (none - own data) |
+| `GET /v1/auth/roles` | `auth.role.list` |
+| `GET /v1/auth/roles/{id}` | `auth.role.list` |
+| `POST /v1/auth/roles` | `auth.role.create` |
+| `PUT /v1/auth/roles/{id}` | `auth.role.edit` |
+| `DELETE /v1/auth/roles/{id}` | `auth.role.delete` |
+| `GET /v1/auth/permissions` | `auth.role.list` |
+| `POST /v1/auth/roles/{id}/permissions` | `auth.role.edit` |
+| `DELETE /v1/auth/roles/{id}/permissions/{perm}` | `auth.role.edit` |
+| `GET /v1/auth/keys` | `auth.role.list` |
+| `POST /v1/auth/keys/{id}/roles` | `auth.role.assign` |
+| `DELETE /v1/auth/keys/{id}/roles/{role_id}` (+ optional `?scope_type=` / `?scope_id=`) | `auth.role.assign` |
+| `GET /v1/auth/check` | (authenticated; surfaces effective perms) |
+| `GET /v1/auth/bootstrap` + `POST /v1/auth/bootstrap` | (auth-exempt; gated by env-var token) |
+
+#### Revoke: legacy "all variants" vs scope-selective (Audit 2026-05-11 A-4)
+
+`DELETE /v1/auth/keys/{id}/roles/{role_id}` runs in one of two modes,
+selected by presence of the optional query parameters:
+
+- **No query params (legacy "revoke all variants")** — every scoped grant of
+  this role held by this actor is dropped. Idempotent: zero-row deletes
+  return 204 (no error). This is the pre-A-4 behaviour and remains the
+  default for the CLI / GUI buttons that don't know about scope.
+
+  ```bash
+  # Drop EVERY variant of r-operator from alice (global, profile-scoped,
+  # issuer-scoped — all gone).
+  curl -X DELETE https://certctl.example.com/api/v1/auth/keys/alice/roles/r-operator
+  ```
+
+- **`?scope_type=` (+ optional `?scope_id=`)** — drop ONE variant. Used
+  when an actor holds the same role at multiple scopes (HIGH-10 made
+  that representable; A-4 makes it selectively revocable).
+  `scope_type=global` requires `scope_id` to be absent; `scope_type=profile`
+  / `issuer` require `scope_id`. No match returns 404 so operators get
+  feedback when they target a scope variant the actor doesn't hold.
+
+  ```bash
+  # Alice holds r-operator scoped to p-acme AND p-globex.
+  # Drop ONLY the p-acme grant; the p-globex grant stays.
+  curl -X DELETE 'https://certctl.example.com/api/v1/auth/keys/alice/roles/r-operator?scope_type=profile&scope_id=p-acme'
+
+  # Drop ONLY the global grant of r-operator (keeps any profile / issuer variants):
+  curl -X DELETE 'https://certctl.example.com/api/v1/auth/keys/alice/roles/r-operator?scope_type=global'
+  ```
+
+The audit row's `details` payload records which mode fired —
+`scope: "all_variants"` for the legacy path, or the explicit
+`scope_type` + `scope_id` for selective revoke — so SOC / SIEM can
+distinguish wide cleanups from targeted demotions in the access log.
+
+### From the MCP server
+
+The MCP server ships 12 RBAC tools:
+`certctl_auth_me`, `certctl_auth_list_roles`, `certctl_auth_get_role`,
+`certctl_auth_create_role`, `certctl_auth_update_role`,
+`certctl_auth_delete_role`, `certctl_auth_list_permissions`,
+`certctl_auth_add_permission_to_role`,
+`certctl_auth_remove_permission_from_role`,
+`certctl_auth_list_keys`, `certctl_auth_assign_role_to_key`,
+`certctl_auth_revoke_role_from_key`. Each routes through the same
+HTTP surface above; permission gates fire server-side.
+
+## The auditor pattern
+
+Hand the auditor key to compliance reviewers. They get:
+
+- `GET /api/v1/audit?category=auth` - every auth/authz mutation
+  in the system (role creates, role grants on actors, bootstrap
+  consumption, etc.).
+- `GET /api/v1/audit?category=cert_lifecycle` - every cert event.
+- `GET /api/v1/audit?category=config` - every issuer / target /
+  settings edit.
+- `GET /api/v1/audit/export` - bulk export.
+
+They do NOT get cert read, profile read, issuer read, or any
+mutating permission. The categorization is enforced by the database
+CHECK constraint (migration 000032); the WORM trigger from
+migration 000018 keeps the audit table append-only at the DB layer.
+
+To create an auditor key:
+
+1. `certctl-cli auth keys assign <key-id> --role r-auditor`
+2. (Optional) Revoke any other roles the key holds with
+   `certctl-cli auth keys revoke <key-id> --role r-...`
+3. Confirm via `certctl-cli auth me` while authenticated as the
+   auditor key - the response should show only `audit.read` and
+   `audit.export` in `effective_permissions`.
+
+## Day-0 bootstrap (first-admin path)
+
+certctl ships a one-shot bootstrap endpoint for fresh
+deployments where no admin actor exists yet.
+
+1. Set `CERTCTL_BOOTSTRAP_TOKEN=$(openssl rand -hex 32)` in the
+   server environment.
+2. Boot the server. Logs include
+   "bootstrap endpoint enabled - POST /api/v1/auth/bootstrap to
+   mint the first admin key (one-shot)" when the path is callable.
+3. Run a single curl:
+
+   ```bash
+   curl -X POST $URL/api/v1/auth/bootstrap \
+     -H 'Content-Type: application/json' \
+     -d '{"token":"<the-token>","actor_name":"first-admin"}'
+   ```
+
+4. Capture the `key_value` from the response. **It is shown ONCE.**
+   The server never logs it.
+5. Use the new key to authenticate against the rest of the API.
+   The bootstrap path is now closed: subsequent calls return HTTP
+   410 Gone, even with the same valid token, because an admin
+   actor exists.
+
+The token is constant-time-compared. The server logs a startup
+warning if `CERTCTL_BOOTSTRAP_TOKEN` is set AND admin actors
+already exist (config-drift signal). For the OIDC-first-admin
+path (the "first user who signs in via SSO becomes admin"
+pattern), see
+[`docs/migration/oidc-enable.md`](../migration/oidc-enable.md).
+
+## Demo mode (`CERTCTL_AUTH_TYPE=none`)
+
+When auth is disabled, the server injects a synthetic actor
+`actor-demo-anon` into every request context. That actor holds
+`r-admin` at global scope (seeded by migration 000029), so every
+gated route resolves with a populated actor and admin grants. The
+synthetic actor is reserved: the API rejects any mutation that
+targets it (HTTP 409 with `ErrAuthReservedActor`).
+
+Production deployments MUST NOT use demo mode - there is no
+per-request actor identity for the audit trail, and every request
+flows as admin. Use it for the `docker compose up` demo + the five
+example folders only.
+
+## Where to look next
+
+- [Threat model](auth-threat-model.md) - what attacks this primitive
+  defends against and which it does not
+- [Migration guide](../migration/api-keys-to-rbac.md) - moving
+  pre-RBAC (v2.0.x) deployments onto RBAC
+- [Profiles](../reference/profiles.md) - the `RequiresApproval=true`
+  flow with the flip-flop-bypass closure
+- [Approval workflow](approval-workflow.md) - the two-person
+  integrity primitive backing `RequiresApproval`
+- `internal/auth/` - the middleware + keystore + RequirePermission
+- `internal/service/auth/` - the service-layer Authorizer
+- `cowork/auth-bundle-1-prompt.md` - the design + phase plan
+- `cowork/auth-bundles-index.md` - the per-phase status tracker

docs/operator/runbooks/config-encryption-upgrade.md

@@ -0,0 +1,165 @@
+# Runbook: forcing config-encryption blob upgrades (v1/v2 → v3)
+
+> Last reviewed: 2026-05-12
+
+Use this when:
+- You've rotated `CERTCTL_CONFIG_ENCRYPTION_KEY` and want every row in
+  the database to be re-sealed under the new passphrase, not just the
+  next ones to be touched.
+- A v1- or v2-era encrypted blob existed in your database before you
+  upgraded to a post-M-8 release and you want to retire the legacy
+  read path's PBKDF2 work factor (100,000 rounds) in favor of the v3
+  factor (600,000 rounds, OWASP 2024).
+- You're preparing for an audit and want every at-rest encrypted blob
+  to be on the same wire format.
+
+Audience: a platform sysadmin who can run SQL against certctl's
+PostgreSQL instance and exercise the GUI/REST API write paths.
+
+For background on the v3 / v2 / v1 wire formats and the FileDriver vs
+HSM threat model, read
+[`docs/operator/secret-custody.md`](../secret-custody.md) first.
+
+---
+
+## Background: how the read fallback works
+
+`internal/crypto/encryption.go::DecryptIfKeySet` reads three on-disk
+formats in this order:
+
+```
+v3 (magic 0x03, per-ciphertext 16-byte salt, PBKDF2 600k) →
+v2 (magic 0x02, per-ciphertext 16-byte salt, PBKDF2 100k) →
+v1 (no magic, fixed 28-byte salt, PBKDF2 100k)
+```
+
+The fallback is AEAD-driven: if v3 decryption fails authentication, the
+function tries v2; if v2 fails, v1. This is what keeps pre-M-8 v1 blobs
+readable without an explicit migration.
+
+`EncryptIfKeySet` always writes v3. As a result, any row that is
+**re-written** through the normal application code path is silently
+upgraded to v3 the moment it's persisted.
+
+The implication: you do not need to "migrate" v1/v2 blobs for them to
+keep working — only if you want the v1/v2 wire format physically gone
+from your database.
+
+## Procedure
+
+### Step 1 — confirm the encryption key is set
+
+Re-encryption obviously cannot run without a passphrase. Verify:
+
+```bash
+echo "${CERTCTL_CONFIG_ENCRYPTION_KEY:-NOT SET}" | sed -E 's/./*/g'
+```
+
+If the variable prints `NOT SET`, do not proceed — set the key in your
+deployment manifest and restart the control plane first.
+
+### Step 2 — identify which tables hold encrypted blobs
+
+Encrypted columns in the v2.1.0 schema:
+
+| Table              | Column                | Notes                                                                |
+|---|---|---|
+| `issuers`          | `encrypted_config`    | Only populated for `source='database'` rows (env-seeded rows are not encrypted) |
+| `targets`          | `encrypted_config`    | Same source-based gating as issuers                                  |
+| `oidc_providers`   | `client_secret_enc`   | OIDC client_secret                                                   |
+| `auth_session_signing_keys` | `key_material_enc` | HMAC-SHA256 session-cookie signing key                          |
+
+If your schema differs, derive the column list from the migration
+folder:
+
+```bash
+grep -hE '_enc[ ,]|encrypted_config' migrations/*.up.sql | sort -u
+```
+
+### Step 3 — identify rows still on v1/v2
+
+The magic byte of the blob distinguishes versions; v1 blobs start with
+the random AES-GCM nonce (anything but `0x02` or `0x03` is definitely
+v1), and v2 vs v3 is determined by the first byte:
+
+```sql
+-- Per-table version distribution (run against your live database)
+SELECT
+    SUBSTRING(encrypted_config FROM 1 FOR 1)::bytea AS magic,
+    COUNT(*) AS rows
+  FROM issuers
+  WHERE encrypted_config IS NOT NULL
+  GROUP BY magic;
+```
+
+Expected steady-state output is a single row with `magic = \x03`.
+Any rows with `\x02` are v2; any rows with anything else are v1.
+
+### Step 4 — force re-sealing
+
+`UPDATE` the rows back to themselves through the normal application
+write path. The cleanest way to do this is via the REST API or GUI,
+not raw SQL — re-issuing the same `PUT /api/v1/issuers/:id` reads the
+row, decrypts, then re-encrypts under v3 on the write back.
+
+For an issuer named `iss-letsencrypt-prod`:
+
+```bash
+# Fetch then re-PUT the same body (CSRF + bearer token elided).
+curl -sS https://certctl.example.com/api/v1/issuers/iss-letsencrypt-prod \
+  -H "Authorization: Bearer $CERTCTL_API_KEY" \
+  | jq '.' \
+  | curl -sS -X PUT https://certctl.example.com/api/v1/issuers/iss-letsencrypt-prod \
+      -H "Authorization: Bearer $CERTCTL_API_KEY" \
+      -H "Content-Type: application/json" \
+      --data-binary @-
+```
+
+Repeat for each row that the Step 3 query flagged as non-v3.
+
+### Step 5 — verify
+
+Re-run the Step 3 query. The output should now show only `magic =
+\x03` rows.
+
+## Special case: rotating the encryption-key passphrase
+
+If your goal is to retire a possibly-compromised passphrase rather
+than retire a legacy wire format, the order is:
+
+1. Generate a new passphrase. Document it via your secret-management
+   tool (HashiCorp Vault, AWS Secrets Manager, etc.).
+2. Stop the control plane briefly so no rows are written under the
+   stale passphrase during the transition window.
+3. Run a one-shot decrypt-with-old / re-encrypt-with-new pass.
+   certctl ships no built-in tool for this — see the open
+   roadmap item below. The cleanest current approach is:
+    - Start certctl with the OLD passphrase.
+    - Read every encrypted column out to a JSON dump via the REST API.
+    - Stop certctl. Update its env to the NEW passphrase. Restart.
+    - PUT every row back from the JSON dump (the writes re-seal under
+      the new passphrase).
+4. Document the old passphrase as retired in your secret-management
+   tool. Anyone with read access to a pre-rotation backup still needs
+   it to decrypt that backup; the live database no longer needs it.
+
+For most operators, simply rotating the passphrase and letting the
+re-seal happen organically as rows are touched is acceptable — the
+v3 wire format with PBKDF2 600k rounds makes offline brute-force
+against the old passphrase computationally expensive.
+
+## Open roadmap items
+
+- Ship a built-in `certctl admin reseal --all` command that does Steps
+  3 and 4 in one shot, with structured progress + audit logging.
+  Tracked in [WORKSPACE-ROADMAP.md](../../WORKSPACE-ROADMAP.md).
+- Surface per-table v1/v2/v3 distribution as a Prometheus gauge so
+  alerting can fire on "rows on legacy format" drift.
+
+## Related reading
+
+- [`docs/operator/secret-custody.md`](../secret-custody.md) — the
+  broader where-do-private-keys-live reference; this runbook is the
+  procedural arm of that document.
+- [`internal/crypto/encryption.go`](../../../internal/crypto/encryption.go)
+  package comment — wire format authoritative reference.

docs/operator/runbooks/disaster-recovery.md

@@ -2,12 +2,11 @@
 
 > Last reviewed: 2026-05-05
 
-> **Status (this document):** Production hardening II Phase 10
+> **Status (this document):** Operator runbook codifying the
-> deliverable. Codifies the fail-safe behaviors that already exist in
+> fail-safe behaviors that already exist in the codebase and the
-> the codebase and the operator procedures for recovering from
+> procedures for recovering from common failure modes. Nothing in
-> common failure modes. Nothing in this runbook requires new code —
+> this runbook requires new code — if a procedure here doesn't work
-> if a procedure here doesn't work as documented, that's a bug in
+> as documented, that's a bug in docs (file an issue).
-> docs (file an issue).
 
 This runbook is the on-call deliverable: it tells reviewers and
 on-call operators what to do when a piece of certctl's state

docs/operator/runbooks/ha.md

@@ -0,0 +1,113 @@
+# High-Availability Deployment Runbook
+
+> Last reviewed: 2026-05-13
+
+<!-- Phase 2 DEPL-H1 closure -->
+
+
+certctl's Helm chart ships with conservative single-replica defaults
+that produce a working `helm install` against any Kubernetes cluster.
+Production HA is operator-opt-in across three values surfaces — none
+of which the chart flips on your behalf.
+
+This runbook documents the three changes, why they default off, and
+the smallest-possible HA values overlay.
+
+---
+
+## Why HA is opt-in (not default)
+
+Three load-bearing reasons the chart defaults are `replicas: 1` and
+`podDisruptionBudget.enabled: false`:
+
+1. **A 1-replica deployment works on every cluster.** A multi-replica
+   default with `minAvailable: 2` would render a PDB at install time;
+   if the cluster has fewer than 2 nodes available (single-node
+   `kind` / `minikube` / fresh `k3s` clusters), Helm renders fine but
+   the first `kubectl rollout` blocks indefinitely waiting for the
+   second replica that can never schedule. Defaulting off keeps the
+   demo path one-command.
+
+2. **Postgres is a singleton in the bundled chart.** The chart's
+   `postgres-statefulset.yaml` runs ONE Postgres pod. Scaling the
+   server tier past 1 replica without an externalized Postgres + a
+   pgbouncer-style proxy doesn't actually buy HA at the DB tier — the
+   single Postgres pod is the failure domain. Operators who want true
+   HA route Postgres to a managed service (RDS, Cloud SQL, AlloyDB,
+   AKS-managed-Postgres, Aiven) or run their own cluster (Patroni,
+   CloudNativePG, Zalando postgres-operator). See the
+   [external-Postgres values example](../../deploy/helm/examples/values-external-db.yaml).
+
+3. **Session affinity is HTTPS-only.** The control plane is HTTPS-only
+   (TLS 1.3 pinned). Adding `sessionAffinity: ClientIP` to the
+   server Service mid-deployment when a sticky front-end LB is in
+   play (NGINX Ingress, Cloud LB with backend service) is the right
+   default for OIDC + RBAC session cookies. But operators who terminate
+   TLS at a different layer (Envoy mesh, Cloudflare in front of the
+   cluster) may have already solved affinity upstream — flipping it
+   on by default would over-constrain those paths.
+
+## The smallest production-HA overlay
+
+Three Helm values to flip:
+
+```yaml
+# values-ha.yaml — copy into your overlay and edit to taste.
+
+server:
+  # ≥ 2 replicas is the minimum for the PDB to render. 3 gives you
+  # a true rolling-restart tolerance window (1 down for upgrade,
+  # 2 still serving) without dropping below minAvailable.
+  replicas: 3
+
+  service:
+    # Required when the front-end LB doesn't already enforce
+    # session affinity. OIDC + RBAC session cookies need to land
+    # on the same backend pod for the session lifetime.
+    sessionAffinity: ClientIP
+
+podDisruptionBudget:
+  # Renders the PDB template; controller-side voluntary disruptions
+  # (node-drain for k8s upgrade, cluster-autoscaler scale-down)
+  # respect this floor.
+  enabled: true
+  # With server.replicas: 3, minAvailable: 2 leaves headroom for one
+  # rolling restart at a time.
+  minAvailable: 2
+  # maxUnavailable is mutually exclusive with minAvailable; pick one.
+  # maxUnavailable: 1
+```
+
+Apply with:
+
+```bash
+helm upgrade certctl deploy/helm/certctl/ -f values-ha.yaml
+```
+
+## What you still own as the operator
+
+Three things the chart does not solve, even at `replicas: 3`:
+
+1. **Postgres HA.** Route to an externalized Postgres (managed cloud
+   or operator-managed cluster). The chart's bundled StatefulSet
+   pod is a development/single-AZ pattern, not a production HA path.
+2. **TLS material lifecycle.** The chart accepts an `existingSecret`
+   for the server cert; rotating it is operator-side automation.
+   The dashboard + agent can issue their own certs via the local CA
+   (eat-your-own-dogfood); the operator can wire `cert-manager` if
+   they prefer that path.
+3. **Backup CronJob.** Phase 4 of the architecture diligence
+   remediation plan (DEPL-H2) ships a `backup-cronjob.yaml` template;
+   until that lands, backups are operator-run per the existing
+   `docs/operator/runbooks/postgres-backup.md` runbook.
+
+## Cross-references
+
+- `deploy/helm/certctl/values.yaml` lines 19, 446, 566 — the three
+  defaults this runbook documents.
+- `docs/operator/runbooks/postgres-backup.md` — Postgres backup
+  runbook (today, operator-run).
+- `docs/operator/runbooks/disaster-recovery.md` — DR procedure.
+- Phase 4 (Helm Chart, DR, And Ops Surface) of the architecture
+  diligence remediation plan tracks the chart-level work
+  (backup CronJob, PrometheusRule starter, migration hook, etc.).

docs/operator/runbooks/postgres-backup.md

@@ -0,0 +1,169 @@
+# Runbook: PostgreSQL backup for certctl
+
+> Last reviewed: 2026-05-13
+
+Use this when:
+- You're setting up a new certctl deployment and need a backup policy
+  before going to production.
+- A buyer or auditor asks "where's the backup automation?" and you need
+  to point at the recommended cadence + procedure.
+- You're rotating the encryption key, swapping CAs, or doing any other
+  destructive maintenance and want a snapshot to roll back to.
+
+certctl does not ship a built-in backup daemon. Postgres is the system
+of record for every piece of certctl state that isn't on the
+operator's filesystem (CA keys, OCSP responder keys, SCEP/EST trust
+bundles — see "Operator-managed (NOT in DB)" in the
+[disaster-recovery runbook](disaster-recovery.md#postgres-restore));
+backing it up is treated as a standard PostgreSQL operations task
+that the operator owns end-to-end with their existing tooling.
+
+This page is the recommended recipe.
+
+## What to back up
+
+| Layer                              | Tool                                                                    | Cadence                  |
+|---|---|---|
+| `certctl` database (the row data)  | `pg_dump` (logical) **or** `pg_basebackup` + WAL archive (physical PIT) | ≥ daily, retention ≥ 30d |
+| CA cert + key (`CERTCTL_CA_CERT_PATH`, `CERTCTL_CA_KEY_PATH`) | Out-of-band file backup (operator's existing secret-management tool) | On change |
+| SCEP RA cert + key (per profile)   | Out-of-band file backup                                                 | On change                |
+| OCSP responder keys                | Out-of-band file backup (`CERTCTL_OCSP_RESPONDER_KEY_DIR`)              | On change                |
+| Trust-anchor PEM bundles           | Out-of-band file backup                                                 | On change                |
+| Env vars (auth secret, etc.)       | Operator's secret-management tool (Vault, AWS Secrets Manager, etc.)    | On rotation              |
+
+A backup of only the Postgres database without the operator-managed
+file material is **not a complete restore artifact** — see the
+[disaster-recovery runbook's Postgres-restore section](disaster-recovery.md#postgres-restore)
+for the full inventory. The DR runbook owns the restore procedure;
+this page owns the capture procedure.
+
+## Logical backup (recommended for most deployments)
+
+`pg_dump -Fc` produces a portable compressed dump that's easy to
+restore into a fresh Postgres instance at any version ≥ the dump's
+source version. Best for deployments where the DB is small enough
+that a full logical dump fits the backup window (rough rule of thumb:
+under a million `managed_certificates` rows + corresponding history).
+
+### docker-compose
+
+```bash
+# 1. Snapshot. Run from any host that can reach the postgres container.
+TIMESTAMP=$(date -u +%Y%m%dT%H%M%SZ)
+docker compose -f deploy/docker-compose.yml exec -T postgres \
+  pg_dump --format=custom --no-owner --no-acl --dbname=certctl \
+  > "certctl-${TIMESTAMP}.dump"
+
+# 2. Verify integrity (catch transport / truncation bugs early).
+docker run --rm -v "$PWD:/dumps" -w /dumps postgres:16-alpine \
+  pg_restore --list "certctl-${TIMESTAMP}.dump" > /dev/null \
+  && echo "OK: pg_restore --list parses the dump cleanly" \
+  || { echo "CORRUPT DUMP"; exit 1; }
+
+# 3. Move to durable storage (S3, GCS, NFS, encrypted-at-rest blob
+# storage of your choice). DO NOT leave the dump on the certctl host
+# alone — that defeats the purpose of having a backup.
+aws s3 cp "certctl-${TIMESTAMP}.dump" "s3://your-bucket/certctl/"
+```
+
+### Kubernetes (with the bundled Helm chart)
+
+```bash
+# 1. Snapshot via kubectl exec into the postgres StatefulSet pod.
+TIMESTAMP=$(date -u +%Y%m%dT%H%M%SZ)
+NAMESPACE=certctl
+kubectl exec -n "$NAMESPACE" statefulset/postgres -- \
+  pg_dump --format=custom --no-owner --no-acl --dbname=certctl \
+  > "certctl-${TIMESTAMP}.dump"
+
+# 2. Same verification step as above.
+# 3. Same off-host storage step as above.
+```
+
+### Restore (cross-reference)
+
+The restore procedure lives in
+[disaster-recovery.md § Postgres restore](disaster-recovery.md#postgres-restore).
+The key reminders: stop certctl first, restore the DB, run any
+migrations newer than the snapshot, truncate the CRL + OCSP caches,
+then restart.
+
+## Physical / PITR backup (large fleets, RPO < 1h)
+
+Logical dumps have a coarse RPO (the last successful dump). For
+deployments where ≤ 1h of cert-issuance history loss is unacceptable,
+pair Postgres physical backups with continuous WAL archiving:
+
+- `pg_basebackup` for the initial seed
+- `archive_command = '<your-WAL-archiver>'` in `postgresql.conf` to
+  ship every WAL segment off the host as it closes
+- `pgbackrest` or `wal-g` for the operational layer (both are
+  battle-tested, support encryption, and integrate cleanly with S3 /
+  GCS / Azure Blob)
+
+certctl ships nothing in this layer — it's standard PostgreSQL DBA
+work, and shipping a bespoke recipe would just be a worse version of
+what `pgbackrest` already does. The
+[pgbackrest configuration guide](https://pgbackrest.org/configuration.html)
+is the authoritative reference.
+
+## Automation paths
+
+This is the gap an acquisition reviewer typically wants to see filled.
+certctl ships no backup CronJob template in the Helm chart — the
+operator owns this layer because:
+
+1. The right tool depends on the deployment topology (in-cluster
+   Postgres vs. managed Postgres vs. self-hosted on a VM).
+2. The right secret-management integration depends on the operator's
+   existing stack (Vault, AWS Secrets Manager, GCP Secret Manager,
+   sealed-secrets, External Secrets).
+3. The right storage backend depends on the operator's existing
+   off-host blob storage.
+
+A bundled CronJob would be a half-answer for any operator with an
+established backup posture, and would have to be torn out before
+production. Three sample recipes that cover the common cases:
+
+- **In-cluster Postgres → S3:** a CronJob running an alpine image with
+  `aws-cli` + the `pg_dump` command above, output piped to
+  `aws s3 cp`. Cosign-signed if your supply-chain policy requires it.
+- **Managed Postgres (AWS RDS / GCP Cloud SQL / Azure DB):** rely on
+  the cloud provider's built-in PITR backup; configure retention
+  ≥ 30 days; the certctl deployment surface is the connection string
+  alone.
+- **Self-hosted VM:** systemd timer + `pg_dump` + `restic` (or
+  `borgbackup`) to encrypted off-host storage.
+
+Tracked in [WORKSPACE-ROADMAP.md](../../../WORKSPACE-ROADMAP.md) as a
+post-v2.1.0 nice-to-have: an opt-in Helm CronJob template for the
+in-cluster-Postgres-to-S3 case as a starter. The right time to ship
+it is when a real operator asks for it; speculatively shipping it
+without that signal would just produce a template every deployment
+ends up rewriting.
+
+## Verification — what to dry-run quarterly
+
+A backup you've never restored is a backup you don't have. Add this
+to your quarterly on-call rotation:
+
+1. Pick the most recent dump from the previous quarter.
+2. Stand up a throwaway Postgres instance (Docker, kind, anything).
+3. `pg_restore -d certctl <the dump>`.
+4. Bring up a certctl-server container pointed at the throwaway DB
+   (`CERTCTL_DATABASE_URL=postgres://certctl:...@throwaway/...`).
+5. Confirm `/api/v1/version` returns 200, `/api/v1/certificates`
+   lists the expected rows, and the scheduler logs show no
+   migration-version mismatch.
+6. Tear down. Note the timing in your DR registry.
+
+The [disaster-recovery runbook](disaster-recovery.md) covers what to
+do when this dry-run reveals a gap.
+
+## Related reading
+
+- [`docs/operator/runbooks/disaster-recovery.md`](disaster-recovery.md) — the restore companion
+- [`docs/operator/secret-custody.md`](../secret-custody.md) — what
+  the operator-managed file material (CA keys, RA keys, trust
+  anchors) contains, why it lives outside the DB, and what it costs
+  to lose

docs/operator/secret-custody.md

@@ -0,0 +1,166 @@
+# Secret custody — where private keys live in certctl
+
+> Last reviewed: 2026-05-12
+
+Use this when:
+- You're sizing certctl against an internal security review or third-party
+  diligence ("where do private keys live, and how are they protected at
+  rest?").
+- You're evaluating the file-on-disk vs HSM-vs-cloud-KMS roadmap before
+  committing to a deployment topology.
+- You need a single page that names every secret material on the control
+  plane and on agents, plus the at-rest protection for each.
+
+This document covers WHAT secrets exist, HOW they are stored, and the
+THREAT MODEL we accept for each — it is not a hardening checklist. The
+hardening levers (env-vars, file modes, encryption-key configuration) are
+cross-referenced as you read through.
+
+## The secrets that exist
+
+| Material                        | Where it lives                                                                  | Protection at rest                                                                                                  | Closes when…                                                                       |
+|---|---|---|---|
+| Local CA private key            | File on the control-plane host (`CERTCTL_CA_KEY_PATH`)                          | Filesystem ACLs (operator-supplied path; mode 0600 recommended)                                                     | A `signer.PKCS11Driver` or `signer.CloudKMSDriver` ships (post-v2.1.0)             |
+| Agent ECDSA P-256 private keys  | File on each agent host (default `/var/lib/certctl-agent/keys/`)                | Filesystem ACLs on the agent host. Never transmitted to the control plane.                                          | TPM / Secure Enclave drivers ship (no current roadmap entry)                       |
+| OIDC client secret              | `oidc_providers.client_secret_enc` column (PostgreSQL)                          | AES-256-GCM v3 wire format, derived from `CERTCTL_CONFIG_ENCRYPTION_KEY` via PBKDF2-SHA256 600k rounds              | The encryption key is rotated via `internal/crypto` re-seal (see runbook below)    |
+| Session signing key             | `auth_session_signing_keys` table (PostgreSQL)                                  | AES-256-GCM v3, same encryption-key passphrase as above                                                             | HSM/FIPS-validated signing-key driver lands (deferred to v3)                       |
+| Break-glass credential          | `breakglass_credentials.password_hash` column (PostgreSQL)                      | Argon2id (m=64MiB, t=1, p=4) hash; never encrypted because we need constant-time comparison                         | Out of scope — Argon2id resists offline attack already                             |
+| API-key bearer tokens           | `auth_api_keys.token_hash` column (PostgreSQL)                                  | SHA-256(token) only — the plaintext is shown to the operator once at create time and never persisted                | Out of scope                                                                       |
+| CSR private keys mid-issuance   | Agent memory only, ephemeral                                                    | Never written to disk; never transmitted to the server (CSRs only)                                                  | Already closed                                                                     |
+| Issuer-connector backend secrets | `issuers.encrypted_config` column (PostgreSQL) for `source='database'` rows    | AES-256-GCM v3; FAIL-CLOSED if `CERTCTL_CONFIG_ENCRYPTION_KEY` is unset (see "Env-seeded vs DB-seeded" below)        | Already closed for `source='database'`; `source='env'` carries an explicit carve-out |
+
+The breakdown by row source matters and is the subject of the next
+section. Read it before concluding that a plaintext column is a bug.
+
+## Env-seeded vs DB-seeded configs
+
+certctl supports two sources for issuer and target configurations:
+
+- **`source='env'`** — built from process environment variables on every
+  boot (`CERTCTL_CA_CERT_PATH`, `CERTCTL_CA_KEY_PATH`, `CERTCTL_ACME_DIRECTORY_URL`,
+  `CERTCTL_STEPCA_URL`, etc. — see `internal/service/issuer.go::buildEnvVarSeeds`
+  for the exact list). These rows are deterministically reconstructable from environment and
+  exist primarily so the GUI has something to display and so audit logs
+  can reference an issuer ID. The `config` column is intentionally
+  plaintext for `source='env'` rows: the exact same bytes already live
+  in the operator's Compose file / Helm values / systemd unit, so
+  persisting them again to PostgreSQL adds no new disclosure surface.
+
+- **`source='database'`** — created via the GUI or REST API write paths
+  (`POST /api/v1/issuers`, etc.). These rows fail closed when
+  `CERTCTL_CONFIG_ENCRYPTION_KEY` is not configured:
+    - The HTTP handlers refuse the write with
+      `crypto.ErrEncryptionKeyRequired`.
+    - The server **refuses to start** if any `source='database'` row
+      exists without the encryption key, to prevent retroactive
+      plaintext exposure.
+
+The startup guard is in `cmd/server/main.go` around the
+`encryptionKey != ""` branch — it lists `source='database'` rows on every
+boot and aborts if any are present without the key.
+
+If you want every issuer/target row to be encrypted at rest unconditionally,
+set `CERTCTL_CONFIG_ENCRYPTION_KEY` and use database-sourced
+configurations exclusively (re-create env-seeded rows through the GUI
+once the key is present).
+
+## The signer abstraction
+
+All CA private-key signing flows through
+`internal/crypto/signer.Signer`, which embeds the stdlib `crypto.Signer`
+and adds `Algorithm()`. Two drivers ship today:
+
+- `signer.FileDriver` — the production default. Wraps the historical
+  file-on-disk PEM flow without behavior change. **Heap-resident**:
+  while certctl is running, the key bytes sit in the process's address
+  space.
+- `signer.MemoryDriver` — used in tests; never reaches production code
+  paths.
+
+The disk-exposure leg of the threat model is documented inline at the
+top of `internal/connector/issuer/local/local.go` (the L-014 carve-out).
+The mitigations on the FileDriver leg include:
+- mode 0600 enforced on the key file at startup,
+- the key directory is not served by any handler,
+- the bytes are never logged or echoed in audit events,
+- the server fails closed if it cannot read the key.
+
+`FileDriver` does NOT mitigate "an attacker with read access to the
+control-plane filesystem can recover the CA key." That mitigation lives
+in a future `signer.PKCS11Driver` (hardware token) or
+`signer.CloudKMSDriver` (AWS/GCP/Azure KMS). The interface exists; the
+drivers do not ship yet. Both are post-v2.1.0 roadmap items — see
+[`docs/reference/architecture.md`](../reference/architecture.md) for the
+target topology.
+
+If you need HSM-grade key custody today, you have two options:
+1. Run certctl behind an enterprise issuer (Microsoft ADCS, EJBCA,
+   Smallstep, ACME-public) and configure certctl's local CA as
+   intermediate-only or disable it entirely. The issuer connector then
+   sends every signing request to your existing hardware-rooted PKI.
+2. Wait for the PKCS#11 driver. Track its status in
+   [WORKSPACE-ROADMAP.md](../../WORKSPACE-ROADMAP.md).
+
+## Config-encryption wire format
+
+`internal/crypto/encryption.go` produces and reads three on-disk
+formats. The read path accepts all three; the write path emits only
+the newest:
+
+| Version | Magic byte | Salt              | PBKDF2-SHA256 work factor | Status                                                            |
+|---|---|---|---|---|
+| v3      | `0x03`    | per-ciphertext 16B | 600,000                  | **Default for all writes** (OWASP 2024)                            |
+| v2      | `0x02`    | per-ciphertext 16B | 100,000                  | Legacy read-only; superseded by v3                                 |
+| v1      | none      | fixed 28B          | 100,000                  | Pre-M-8 legacy read-only; written before per-ciphertext-salt fix   |
+
+The wire-format documentation is also in the `internal/crypto/encryption.go`
+package comment.
+
+### Forcing legacy blob upgrades
+
+Re-sealing happens passively: any `UPDATE` against a row that contains a
+v1 or v2 blob triggers a v3 rewrite the next time the field is set.
+There is no in-place migration tool because re-sealing requires reading
+the row through the same code path that performs the write, and any
+operational path that touches the row (renaming an issuer in the GUI,
+updating a target's endpoint, refreshing an OIDC provider's
+client-secret) achieves this naturally.
+
+If you want to FORCE re-sealing across the entire database, use the
+runbook at
+[`docs/operator/runbooks/config-encryption-upgrade.md`](runbooks/config-encryption-upgrade.md).
+Recommended only if you suspect the encryption-key passphrase has
+been exposed and have already rotated it (the runbook covers the
+rotation order: set the new key, force re-seal, retire the old key
+from the rotation pool).
+
+## Roadmap (what is not yet closed)
+
+Tracked in [`WORKSPACE-ROADMAP.md`](../../WORKSPACE-ROADMAP.md), not
+maintained here to prevent drift:
+
+- `signer.PKCS11Driver` for HSM-token-backed CA key custody.
+- `signer.CloudKMSDriver` for AWS/GCP/Azure KMS-backed CA key custody.
+- FIPS 140-3 mode for the entire control plane.
+- HSM-backed session signing key (currently HMAC-SHA256 software keys).
+
+If a buyer or auditor asks for "HSM support," the honest answer is:
+the interface is there, the drivers are not, and an enterprise issuer
+connector is the bridge until the drivers ship.
+
+## Related reading
+
+- [`docs/operator/security.md`](security.md) — the broader hardening
+  checklist; covers TLS, RBAC, audit logging, network policy.
+- [`docs/operator/auth-threat-model.md`](auth-threat-model.md) — the
+  authentication-subsystem threat model. Item 5 ("HSM / FIPS-validated
+  signing key for sessions") is the session-signing-key analog of this
+  document's CA-key story.
+- [`docs/reference/architecture.md`](../reference/architecture.md) §
+  "Signer abstraction" — the diagram form of the FileDriver / future
+  PKCS11Driver / CloudKMSDriver topology.
+- [`internal/crypto/encryption.go`](../../internal/crypto/encryption.go)
+  package comment — wire format authoritative reference.
+- [`internal/connector/issuer/local/local.go`](../../internal/connector/issuer/local/local.go)
+  L-014 carve-out — the load-bearing threat-model section for the
+  FileDriver case.

docs/operator/security.md

@@ -1,6 +1,6 @@
 # certctl Security Posture & Operator Guidance
 
-> Last reviewed: 2026-05-05
+> Last reviewed: 2026-05-11
 
 This document collects the operator-facing security guidance that the source
 code's per-finding comment blocks reference. Each section names the audit
@@ -9,16 +9,15 @@ any).
 
 ## OCSP responder availability
 
-**Audit reference:** Bundle C / M-020. CWE-770 (uncontrolled resource
+**Audit reference:** CWE-770 (uncontrolled resource consumption); RFC
-consumption); RFC 6960 (OCSP); RFC 7633 (Must-Staple).
+6960 (OCSP); RFC 7633 (Must-Staple).
 
 certctl ships an OCSP responder at `/.well-known/pki/ocsp/{issuer_id}/{serial}`
-that signs a fresh response per request. Pre-Bundle-C the unauth handler
+that signs a fresh response per request. The unauth handler chain
-chain had no rate limit, so an attacker could DoS the responder and force
+applies the same per-key rate limiter the authenticated chain uses;
-fail-open relying parties to accept revoked certificates as valid. Bundle C
+per-IP keying applies because OCSP traffic is unauthenticated. Without
-adds the same per-key rate limiter to the unauth chain that the authenticated
+this defense an attacker could DoS the responder and force fail-open
-chain has used since Bundle B. Per-IP keying applies because OCSP traffic is
+relying parties to accept revoked certificates as valid.
-unauthenticated.
 
 The rate limiter alone does not solve the underlying revocation-bypass risk.
 **The architectural fix is for issued certificates to carry the OCSP
@@ -41,10 +40,10 @@ For certificates issued to systems where revocation correctness matters:
    ignore it.
 3. **Confirm the deployment target is configured for OCSP stapling** so the
    server can actually deliver the stapled response in the handshake.
-   - **nginx:** `ssl_stapling on; ssl_stapling_verify on;`
+ - **nginx:** `ssl_stapling on; ssl_stapling_verify on;`
-   - **Apache:** `SSLUseStapling on`
+ - **Apache:** `SSLUseStapling on`
-   - **HAProxy:** `set ssl ocsp-response /path/to/response.der`
+ - **HAProxy:** `set ssl ocsp-response /path/to/response.der`
-   - **Envoy:** `ocsp_staple_policy: must_staple`
+ - **Envoy:** `ocsp_staple_policy: must_staple`
 
 ### What this does NOT cover
 
@@ -59,35 +58,278 @@ For certificates issued to systems where revocation correctness matters:
 
 ## Postgres transport encryption
 
-See [docs/database-tls.md](database-tls.md). Bundle B / M-018.
+See [docs/database-tls.md](database-tls.md).
 
 ## Encryption at rest
 
-Bundle B / M-001. PBKDF2-SHA256 at 600,000 rounds (OWASP 2024 Password
+PBKDF2-SHA256 at 600,000 rounds (OWASP 2024 Password
 Storage Cheat Sheet floor) for the operator-supplied passphrase that
 derives the AES-256-GCM key for sensitive config columns. v3 blob format
 with a per-ciphertext random salt; v1/v2 read fallback for legacy rows.
-See [internal/crypto/encryption.go](../internal/crypto/encryption.go) and
+See [internal/crypto/encryption.go](../../internal/crypto/encryption.go) and
 the accompanying tests for the format spec.
 
 ## Authentication surface
 
-Bundle B / M-002. Two layers decide auth-exempt status:
+Two layers decide auth-exempt status:
 
 1. **Router layer:** `internal/api/router/router.go::AuthExemptRouterRoutes`
-   — the 4 endpoints registered via direct `r.mux.Handle` without going
+ - the endpoints registered via direct `r.mux.Handle` without going
    through the middleware chain (`/health`, `/ready`, `/api/v1/auth/info`,
-   `/api/v1/version`).
+   `/api/v1/version`, plus `/api/v1/auth/bootstrap` GET + POST for the
+   first-admin path).
 2. **Dispatch layer:** `internal/api/router/router.go::AuthExemptDispatchPrefixes`
-   — URL-prefix routing in `cmd/server/main.go::buildFinalHandler` for
+ - URL-prefix routing in `cmd/server/main.go::buildFinalHandler` for
-   `/.well-known/pki/*`, `/.well-known/est/*`, and `/scep[/...]*`.
+   `/.well-known/pki/*`, `/.well-known/est/*`, `/.well-known/est-mtls`,
+   and `/scep[/...]*` (incl. `/scep-mtls`).
 
 Both lists have AST-walking regression tests (`auth_exempt_test.go`) that
-fail CI if a new bypass lands without an updating the documented constant.
+fail CI if a new bypass lands without updating the documented constant.
+
+### Role-based authorization
+
+Role-based authorization runs on top of API-key authentication. Every
+gated handler routes through the `auth.RequirePermission` middleware
+(or its router-level wrap `rbacGate`); the middleware resolves the
+actor's effective permissions via the service-layer
+`Authorizer.CheckPermission` and returns HTTP 403 BEFORE the handler
+body runs on miss. The seven default roles (`admin` / `operator` /
+`viewer` / `agent` / `mcp` / `cli` / `auditor`), 33-permission
+canonical catalogue, and the auditor split (`r-auditor` holds only
+`audit.read` + `audit.export`) are seeded by migration 000029.
+
+For the operator how-to, see [`rbac.md`](rbac.md). For the
+threat model + compliance mapping, see
+[`auth-threat-model.md`](auth-threat-model.md). For the upgrade
+flow from an API-key-only deployment, see
+[`docs/migration/api-keys-to-rbac.md`](../migration/api-keys-to-rbac.md).
+
+### Day-0 admin bootstrap
+
+Fresh deployments where no admin actor exists yet can mint the
+first admin via `POST /api/v1/auth/bootstrap` - set
+`CERTCTL_BOOTSTRAP_TOKEN`, POST a single curl with the token, and
+the server returns the plaintext key value once. The token is
+constant-time-compared; the strategy is one-shot via mutex; the
+admin-existence probe re-closes the path once an admin lands.
+The token is NEVER logged. The minted plaintext key flows only
+into the HTTP response body. See
+[`rbac.md`](rbac.md#day-0-bootstrap-first-admin-path) for the
+full flow.
+
+### Approval-bypass closure
+
+`CertificateProfile.RequiresApproval=true` profiles route both
+issuance/renewal AND profile edits through the
+`ApprovalService` two-person integrity gate. The flip-flop loophole
+(an admin disabling approval, mutating, re-enabling) is closed by
+gating profile-edit through the same approval flow. Same-actor
+self-approve is rejected at the service layer with
+`ErrApproveBySameActor`. See
+[`docs/reference/profiles.md`](../reference/profiles.md) for the
+full gate semantics.
+
+### OIDC federation
+
+OIDC SSO runs on top of the API-key + RBAC foundation. Operators
+configure one or more identity providers (Keycloak, Authentik, Okta,
+Auth0, Entra ID, or Google Workspace via Keycloak broker); end users
+sign in at the IdP, certctl validates the returned ID token, and a
+session cookie is minted.
+
+The token-validation pipeline pins:
+
+- Algorithm allow-list: RS256 / RS512 / ES256 / ES384 / EdDSA only.
+  HS256 / HS384 / HS512 / `none` are rejected at the service-layer
+  sentinel level.
+- IdP-downgrade-attack defense at provider creation AND every
+  RefreshKeys: the IdP's advertised
+  `id_token_signing_alg_values_supported` is intersected with the
+  allow-list; a provider that advertises HS-family is rejected
+  before any token is signed under the weak alg.
+- Exact `iss` match (`ErrIssuerMismatch`).
+- `aud` membership + `azp` for multi-aud tokens (per OIDC core
+  §3.1.3.7 step 5).
+- `at_hash` REQUIRED-when-access_token-present (a tightening of the
+  spec MAY → MUST so a substituted access token cannot ride alongside
+  a clean ID token).
+- Single-use state + nonce (32-byte random server-generated;
+  atomic `DELETE...RETURNING` on consume).
+- PKCE-S256 mandatory; `plain` rejected.
+- Configurable `iat` window (default 300s, capped 600s).
+- JWKS cache with operator-triggered RefreshKeys + auto-refresh on
+  TTL expiry (default 3600s); JWKS-fetch failure during a key
+  rotation returns 503 to the in-flight login (existing sessions
+  untouched).
+
+OIDC `client_secret` is encrypted at rest via AES-256-GCM (v3 blob
+format: magic 0x03 + salt(16) + nonce(12) + ciphertext+tag) using
+the `CERTCTL_CONFIG_ENCRYPTION_KEY` passphrase. The encryption
+invariant is pinned by an integration test
+(`internal/repository/postgres/oidc_encryption_invariant_test.go`)
+that asserts ciphertext != plaintext + correct blob shape +
+round-trip recovery + wrong-passphrase fails.
+
+Per-IdP setup guides at
+[`oidc-runbooks/index.md`](oidc-runbooks/index.md) cover Keycloak,
+Authentik, Okta, Auth0, Entra ID, and Google Workspace.
+
+### Sessions + back-channel logout
+
+Successful OIDC login mints a session cookie:
+`v1.<session_id>.<signing_key_id>.<base64url-no-pad(HMAC-SHA256)>`.
+The HMAC input is **length-prefixed** as `len:sid:len:kid` to defeat
+concatenation-collision attacks on bare-concat designs. Cookie
+attributes:
+
+- `HttpOnly=true` (no JS access; defends XSS cookie theft).
+- `Secure=true` (HTTPS-only; defends network MITM).
+- `SameSite=Lax` default (configurable to Strict via
+  `CERTCTL_SESSION_SAMESITE`).
+- `Path=/`, host-only.
+
+Idle timeout default 1h; absolute timeout default 8h; both
+configurable via `CERTCTL_SESSION_IDLE_TIMEOUT` and
+`CERTCTL_SESSION_ABSOLUTE_TIMEOUT`. The scheduler's
+`sessionGCLoop` (default 1h interval) sweeps expired rows.
+
+CSRF defense: plaintext CSRF token in the JS-readable
+`certctl_csrf` cookie (intentionally `HttpOnly=false` for the GUI
+to echo into the `X-CSRF-Token` header); SHA-256 hash on the
+session row; `subtle.ConstantTimeCompare` in `CSRFMiddleware`.
+API-key actors are CSRF-exempt (no session row in context).
+
+Session signing keys rotate via `RotateSigningKey`; the old key
+stays valid for `CERTCTL_SESSION_SIGNING_KEY_RETENTION` (default
+24h) so existing cookies validate during rollover. Past retention,
+the old key's row is dropped and any cookie still signed under it
+returns `ErrSigningKeyNotFound`. `EnsureInitialSigningKey` is
+fail-fatal at server boot.
+
+Back-channel logout per **OpenID Connect Back-Channel Logout 1.0**
+(NOT RFC 8414): `POST /auth/oidc/back-channel-logout` accepts a
+JWT-signed logout token from the IdP, validates the JWT against
+the IdP's JWKS (same alg allow-list as login), pins required
+claims (`iss` / `aud` / `iat` / `jti` / `events`; exactly one of
+`sub` / `sid`; `nonce` MUST be absent), defeats replay via
+`jti`-based deduplication, and revokes matching sessions.
+
+For threat-model coverage of these surfaces, see
+[`auth-threat-model.md`](auth-threat-model.md). For the
+operator-runnable performance baselines, see
+[`auth-benchmarks.md`](auth-benchmarks.md).
+
+### OIDC first-admin bootstrap
+
+Coexists with the env-var-token bootstrap path. When the
+operator sets `CERTCTL_BOOTSTRAP_ADMIN_GROUPS` + (optionally)
+`CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID`, the first user with one of
+those IdP groups becomes admin on first login per tenant.
+Subsequent users go through normal mapping. The admin-existence
+probe ensures only one wins between the two bootstrap paths;
+once any actor holds `r-admin`, the OIDC bootstrap hook silently
+falls through to normal mapping. Audit row on every grant
+(`bootstrap.oidc_first_admin`, `event_category=auth`).
+
+### Break-glass admin
+
+Default-OFF (`CERTCTL_BREAKGLASS_ENABLED=false`). When enabled,
+the local-password admin path bypasses OIDC + group-claim layers;
+intended ONLY for SSO-broken incidents.
+
+- Argon2id with OWASP 2024 params (m=64 MiB, t=3, p=4, 16-byte
+  salt, 32-byte output, per-password random salt, PHC-format
+  hash). Hash column is `json:"-"` so handlers cannot wire-leak.
+- Lockout state machine: 5 failures (default; configurable via
+  `CERTCTL_BREAKGLASS_LOCKOUT_THRESHOLD`) within 1h reset window
+  (`_LOCKOUT_RESET_INTERVAL`) trips a 30s lockout (`_LOCKOUT_DURATION`).
+  Atomic single-statement IncrementFailure defeats concurrent
+  racing attempts.
+- Constant-time across all failure paths via `verifyDummy()` —
+  wrong-password / locked-account / no-actor all take statistically
+  indistinguishable time.
+- Surface invisibility: when disabled, ALL four endpoints return
+  HTTP 404 (NOT 403). Scanners cannot distinguish "endpoint
+  disabled" from "endpoint doesn't exist".
+- WARN log at server boot when `ENABLED=true`; audit row on every
+  break-glass login (`auth.breakglass_login_*`,
+  `event_category=auth`); WebAuthn/FIDO2 second factor pairing
+  on the v3 roadmap (Decision 12).
+
+Operator should DISABLE break-glass within 24h of SSO recovery
+to avoid a permanent backdoor; the runbook at
+[`auth-threat-model.md#break-glass-risks-phase-75`](auth-threat-model.md)
+documents the full state machine.
+
+### Demo-to-production cutover (Audit 2026-05-11 A-8)
+
+Migration `000029_rbac.up.sql` unconditionally seeds an
+`actor-demo-anon → r-admin` row into `actor_roles`. This row is the
+runtime principal injected by the demo-mode middleware when
+`CERTCTL_AUTH_TYPE=none`. Under any non-`none` auth type the row is
+DORMANT — the middleware chain never resolves to it. But its existence
+is a footgun: a future regression that resolves an unauthenticated
+request to `actor-demo-anon` (a misrouted CORS preflight, a fallback in
+a new auth-exempt route) would silently re-elevate to admin.
+
+certctl-server detects this residue at startup and emits a WARN log +
+an `auth.demo_residual_grants_detected` audit row listing every grant
+present on `actor-demo-anon`. **Every production deploy will see this
+WARN on first boot** — the migration baseline is part of the install,
+not a side effect of running demo mode.
+
+Operator workflow at production cutover:
+
+1. Drain the WARN by calling the cleanup endpoint with an admin API key:
+
+   ```bash
+   curl -X POST --cacert deploy/test/certs/ca.crt \
+        -H "Authorization: Bearer $ADMIN_KEY" \
+        https://certctl.example.com:8443/api/v1/auth/demo-residual/cleanup
+   # → {"removed": 1}
+   ```
+
+   The endpoint is gated `auth.role.assign` (admin-class) and refuses
+   to run when `CERTCTL_AUTH_TYPE=none` (HTTP 503 — the residue IS the
+   active runtime state at that auth type). The cleanup is idempotent;
+   a second call returns `{"removed": 0}` and still leaves an audit row.
+
+   Equivalent SQL for operators preferring direct DB access:
+
+   ```sql
+   DELETE FROM actor_roles WHERE actor_id = 'actor-demo-anon';
+   ```
+
+2. To make subsequent boots refuse startup if the row reappears (the
+   most paranoid stance), set:
+
+   ```
+   CERTCTL_DEMO_MODE_RESIDUAL_STRICT=true
+   ```
+
+   With the flag set, any `actor-demo-anon` row under a non-`none`
+   auth type causes certctl-server to log the WARN AND exit non-zero
+   before binding the HTTPS listener. Default is `false` (WARN only).
+
+3. The CI guard `scripts/ci-guards/no-new-synthetic-admin.sh` pins the
+   set of source files that may reference the `actor-demo-anon` literal.
+   New runtime code paths that resolve to the synthetic actor are
+   rejected at PR time so the credibility gap stays closed.
+
+### Migrating an existing deployment to OIDC
+
+An existing API-key-only deployment that wants to add OIDC follows
+the step-by-step at
+[`docs/migration/oidc-enable.md`](../migration/oidc-enable.md):
+configure CERTCTL_CONFIG_ENCRYPTION_KEY, pick + configure an IdP
+per the relevant runbook, configure the certctl-side OIDCProvider
++ group→role mappings, verify the login flow against a single
+test user, then announce the SSO endpoint to the rest of the
+organization.
 
 ## Per-user rate limiting
 
-Bundle B / M-025. Authenticated callers are bucketed by API-key name;
+Authenticated callers are bucketed by API-key name;
 unauthenticated callers (probes, OCSP relying parties, EST/SCEP enrollees)
 are bucketed by source IP. `RPS` and `BurstSize` are per-key budgets.
 `PerUserRPS` / `PerUserBurstSize` give authenticated clients a separate
@@ -95,18 +337,14 @@ budget when set non-zero.
 
 ## API key rotation
 
-**Audit reference:** L-004. CWE-924 (improper enforcement of message integrity during transmission in a communication channel) — operator UX variant.
+**Audit reference:** L-004. CWE-924 (improper enforcement of message integrity during transmission in a communication channel) - operator UX variant.
 
 certctl's API keys are configured via the `CERTCTL_API_KEYS_NAMED` env var
 (format `name1:key1,name2:key2:admin`) and parsed at startup into an
 in-memory list. There is no DB-resident key store, no GUI, no `/api/v1/keys`
-endpoint — the env var IS the key inventory.
+endpoint - the env var IS the key inventory.
 
-Pre-Bundle-G the env var rejected duplicate names, so rotating a key
+The env var supports a **double-key rotation window**: two entries can share a
-required: stop accepting OLDKEY → restart → roll NEWKEY out. Any client
-polling against OLDKEY during the restart window hit a 401.
-
-Bundle G adds a **double-key rotation window**: two entries can share a
 name during the rollover, and both keys validate. Operators run the
 rotation as:
 
@@ -118,7 +356,7 @@ rotation as:
    ```
    CERTCTL_API_KEYS_NAMED="alice:OLDKEY:admin,alice:NEWKEY:admin"
    ```
-   Both entries MUST carry the same admin flag — startup fails loud if
+   Both entries MUST carry the same admin flag - startup fails loud if
    they don't (a non-admin shouldn't share an identity with an admin).
 
 3. **Restart certctl.** A startup INFO log confirms the rotation window
@@ -139,7 +377,7 @@ rotation as:
 
 6. **Restart certctl.** OLDKEY now fails with 401. Rotation complete.
 
-The rotation window has no operator-set timeout — it lasts for as long
+The rotation window has no operator-set timeout - it lasts for as long
 as both entries are in the env var. Best practice is a 24-72h window
 covering a full deploy cadence; if a client hasn't rolled to NEWKEY by
 the end of step 4, extend the window before step 5.
@@ -151,8 +389,8 @@ the end of step 4, extend the window before step 5.
 - Two entries with the same `name` but mismatched admin: **rejected at
   startup** (privilege escalation guard).
 - Two entries with the same `(name, key)` pair: **rejected at startup**
-  (typo guard — rotation requires DIFFERENT keys under the same name).
+  (typo guard - rotation requires DIFFERENT keys under the same name).
-- Single-entry steady state: unchanged from pre-Bundle-G behavior.
+- Single-entry steady state: the simple legacy behaviour.
 
 ### What the contract does NOT do
 
@@ -165,6 +403,124 @@ the end of step 4, extend the window before step 5.
   from the env var and restart. That's appropriate for a small env-var
   inventory; it would not scale to a per-user-key-issued model.
 
+## Security carve-outs &amp; operator-tunable defaults
+
+Phase 2 of the architecture diligence remediation (2026-05-13)
+consolidated the following carve-outs into one canonical section so
+operators reviewing security posture have a single search target. Each
+entry cites the exact file:line of the carve-out, why it exists, and
+what the operator should do.
+
+### TLS verification — dev escape hatches
+
+certctl has three `InsecureSkipVerify=true` sites that are dev/probe
+escape hatches, never enabled by default in production:
+
+- **Agent dev escape** — `cmd/agent/main.go:179` (wired from
+  `cmd/agent/main.go:61` config field + `cmd/agent/main.go:1371` CLI
+  flag). Operators flip this only when debugging an agent against a
+  self-signed control plane that hasn't been added to the agent's
+  trust store. Document as `--insecure-skip-verify` in the agent's
+  install runbook; the agent logs a startup WARN any time the flag
+  is set. SEC-M3 pins that the carve-out is intentional.
+- **Agent verification probe** — `cmd/agent/verify.go:78`. The probe
+  intentionally opens a TLS connection with verification disabled so
+  it can inspect any certificate the endpoint serves (including
+  self-signed or expired ones — that's the whole point of a probe).
+  The probe never returns trust state to a security-relevant code
+  path; it only reads cert metadata. SEC-M3 pins this.
+- **tlsprobe (network scanner)** — `internal/tlsprobe/probe.go:54`.
+  Same rationale as the agent verify probe — network discovery must
+  introspect any certificate it finds, including the ones with the
+  problems we're scanning for. SEC-M3 pins this.
+
+### F5 target connector — `InsecureSkipVerify` per-config
+
+The F5 target connector exposes an `Insecure: bool` field on its
+per-target config blob (default `false`). When set,
+`internal/connector/target/f5/f5.go:134` builds the HTTP client with
+`InsecureSkipVerify: config.Insecure`. SEC-M5 closure: operator
+opt-in for self-signed F5 BIG-IP device certs; mitigation is to run
+the F5 + the proxy-agent on a network-segmented internal subnet.
+Document in the F5 connector's per-target setup guide.
+
+### ACME issuer — `CERTCTL_ACME_INSECURE` (now gated on ACK)
+
+`internal/connector/issuer/acme/acme.go:201` builds the ACME HTTP
+client with `InsecureSkipVerify: true` for the Pebble integration
+test path. The per-issuer runtime setting comes from
+`CERTCTL_ACME_INSECURE` (`internal/config/config.go:2116`); Phase 2
+SEC-M4 closure (2026-05-13) added the fail-closed gate so the operator
+must ALSO set `CERTCTL_ACME_INSECURE_ACK=true` for the server to boot.
+Production deploys must never set either flag. The boot-time WARN log
+at `cmd/server/main.go:611` continues to fire for the ACK'd case so
+every restart logs the reminder.
+
+### CSP `'unsafe-inline'` on `style-src`
+
+`internal/api/middleware/securityheaders.go:58` ships the dashboard
+CSP with `style-src 'self' 'unsafe-inline'`. This is required because
+Tailwind compiles utility classes into a single stylesheet at build
+time, but inline-style attributes appear in the dashboard via inline
+`<svg>` elements + Recharts' `<ResponsiveContainer>` injecting inline
+width/height. SEC-L1 closure: the carve-out is necessary today; the
+planned tightening flow is the frontend audit's FE-H2 (icon library)
++ decorative-SVG sweep that then unlocks the CSP hardening (drops
+`'unsafe-inline'`).
+
+### Break-glass admin — Argon2id rest-defense reminder
+
+The break-glass admin path (`docs/operator/runbooks/disaster-recovery.md`)
+hashes the operator-supplied password with Argon2id and stores the
+hash in the `breakglass_credentials` table. SEC-L2 reminder: the
+strength of the rest-defense is operator-supplied — pick a password
+with sufficient entropy (≥ 64 random bits via `openssl rand -base64
+12`) and rotate after every use. Argon2id resists offline cracking
+but an operator-supplied "Password123" hashes the same way.
+
+### Body-size limit (1 MB default) — operator-tunable
+
+The `http.MaxBytesReader` wrap caps inbound request bodies at 1 MB
+by default. The cap is necessary defense against unbounded-body DOS
+but catches legitimate operator workflows:
+
+- Bulk truststore PEM bundle uploads (CA bundles for federated trust
+  stores can be > 1 MB).
+- Multi-MB CRL pushes via the CRL-cache endpoint.
+- Bulk-import of certificates with embedded chains.
+
+SEC-L3 closure: operators raise the cap via `CERTCTL_MAX_BODY_SIZE`
+(units: bytes; e.g. `CERTCTL_MAX_BODY_SIZE=10485760` for 10 MB).
+Document in `deploy/ENVIRONMENTS.md`.
+
+### Demo Compose placeholder credentials
+
+`deploy/docker-compose.demo.yml` ships `CERTCTL_AUTH_SECRET=change-me-in-production`,
+`CERTCTL_CONFIG_ENCRYPTION_KEY=change-me-32-char-encryption-key`, and
+`CERTCTL_API_KEY=change-me-in-production` as documented demo
+defaults. The runtime `Validate()` fail-closed guards
+(`internal/config/config.go::Validate`, Bundle 2 2026-05-12) refuse
+to start if those literal strings reach a non-demo config. Phase 2
+DEPL-M2 closure adds a CI guard
+(`scripts/ci-guards/no-change-me-in-prod-compose.sh`) that fails the
+build at PR time if a `change-me-*` literal leaks into a non-demo
+compose file — catching the regression one layer before the runtime
+guard fires.
+
+### Kubernetes NetworkPolicy — operator-opt-in
+
+`deploy/helm/certctl/templates/networkpolicy.yaml` ships the template
+but `deploy/helm/certctl/values.yaml` defaults `networkPolicy.enabled:
+false`. DEPL-M3 rationale: most Kubernetes clusters don't have a
+NetworkPolicy controller installed (kind / minikube / fresh k3s); a
+default-enabled NetworkPolicy renders fine but produces no
+enforcement, and bare-metal `kube-router`-style controllers may
+interpret a permissive default differently. Production deploys with a
+real NetworkPolicy controller (Calico, Cilium, Antrea) flip the
+values key to `true` and tune the policy in their values overlay.
+Document the production-enable in
+`docs/operator/runbooks/ha.md` (added Phase 2 DEPL-H1).
+
 ## Reporting a vulnerability
 
 Email `certctl@proton.me`. Coordinated disclosure preferred; we will

docs/reference/architecture.md

@@ -151,7 +151,12 @@ The agent runs two background loops: a heartbeat (every 60 seconds) to signal it
 
 Retired agents receive `410 Gone` on subsequent heartbeats (`service.ErrAgentRetired`). `cmd/agent` treats 410 as a terminal signal and exits cleanly so retired agents stop phoning home. Migration `000015` flipped `deployment_targets.agent_id` from `ON DELETE CASCADE` to `ON DELETE RESTRICT`, making the old hard-delete path a schema error and forcing all retirement through this contract.
 
-**Registration is by-design pull-only (C-1 closure, cat-b-6177f36636fb).** Agents register themselves at first heartbeat via `install-agent.sh` + `cmd/agent/main.go` — never via the GUI. The `web/src/api/client.ts::registerAgent` client function is intentionally orphan in the dashboard for this reason. It's preserved in `client.ts` (rather than deleted) so future features that want to drive registration from the GUI — for example, a one-click "register proxy agent" panel for network-appliance topologies where the agent runs in a different network zone from the device it manages — can reach the endpoint without a `client.ts` edit. Operators looking to scale agent enrollment use `install-agent.sh` against a config-management system (Ansible, Salt, Puppet) or a baked-in cloud-init script, not the dashboard.
+**Registration is a two-step operator-driven flow (C-1 closure, cat-b-6177f36636fb).** Agent enrollment is intentionally NOT auto-driven by the agent binary — the agent fail-fasts at startup if `CERTCTL_AGENT_ID` is unset (`cmd/agent/main.go`: "agent-id flag or CERTCTL_AGENT_ID env var is required"). Operators register an agent in one of two ways before starting it:
+
+1. **Programmatic** — `POST /api/v1/agents` with the agent's metadata payload and (when configured) an `Authorization: Bearer <CERTCTL_AGENT_BOOTSTRAP_TOKEN>` header. The response carries the `id` field; that string goes into `CERTCTL_AGENT_ID` for the agent process. Suitable for config-management (Ansible, Salt, Puppet) or cloud-init flows.
+2. **GUI** — the dashboard's Agents page exposes the same endpoint via `web/src/api/client.ts::registerAgent`. The function is kept reachable rather than deleted so the eventual "register proxy agent" panel for network-appliance topologies can land without a `client.ts` edit; today the panel is not yet wired into the page.
+
+Once registered, the operator passes the returned ID to `install-agent.sh` via `--agent-id` (or sets the env var directly) and starts the agent. The pull-only deployment model (the server never initiates outbound connections to agents) means this asymmetric flow is by-design: only the agent's network reach matters, and registration always crosses that boundary outbound from the agent's side once the agent boots with a valid ID.
 
 ### Web Dashboard
 
@@ -1033,14 +1038,31 @@ The HTTP middleware stack processes requests in the following order (see `cmd/se
 4. **BodyLimit** - request body size cap via `http.MaxBytesReader`
 5. **RateLimiter** - token bucket rate limiting (optional, when enabled)
 6. **CORS** - cross-origin request handling (deny-by-default)
-7. **Auth** - API key validation (or none in development; JWT/OIDC via authenticating gateway, see below — not in-process)
+7. **Auth** - one of three production paths (see "In-process authentication surface" below) or `none` for development
 8. **AuditLog** - records every API call to the audit trail (requires auth context for actor)
 
-### Authenticating-gateway pattern (JWT, OIDC, mTLS)
+### In-process authentication surface
 
-certctl's in-process authentication surface is intentionally narrow: `api-key` for production deployments and `none` for development. There is no in-process JWT, OIDC, mTLS, or SAML middleware. (`CERTCTL_AUTH_TYPE=jwt` was accepted pre-G-1 but silently routed through the api-key bearer middleware — a security finding masquerading as a config option, removed at the v2.x boundary; see [`upgrade-to-v2-jwt-removal.md`](upgrade-to-v2-jwt-removal.md) if you previously set it.)
+certctl ships three production-grade in-process authentication paths plus a `none` mode for development. Auth Bundle 2 (commit `dea5053`, 2026-05-12) added native OIDC + sessions + break-glass alongside the v2.0.x API-key path; the older "authenticating-gateway only" framing the previous draft of this doc carried is no longer accurate.
 
-For deployments that need JWT/OIDC/mTLS, the standard pattern is to put an authenticating gateway in front of certctl and configure `CERTCTL_AUTH_TYPE=none` on the upstream certctl process. The gateway terminates the federated identity protocol, validates tokens / certificates / SAML assertions, and proxies the authenticated request to certctl as a same-origin call on a private network. This separation gives operators the full breadth of the modern identity ecosystem (oauth2-proxy, Envoy `ext_authz`, Traefik `ForwardAuth`, Pomerium, Authelia, Caddy `forward_auth`, Apache `mod_auth_openidc`, nginx `auth_request`) without certctl itself having to track signing-key rotation, claim mapping, audience validation, and the rest of the JWT/OIDC surface area. Operators wanting per-request actor attribution past the gateway boundary forward the gateway-resolved identity (e.g., `X-Auth-Request-User` from oauth2-proxy) and run a small authorization layer at the gateway that enforces the bearer-key contract certctl actually uses.
+| `CERTCTL_AUTH_TYPE` | What it authenticates | When to use |
+|---|---|---|
+| `api-key` (default) | `Authorization: Bearer <key>` matched against SHA-256-hashed `CERTCTL_AUTH_SECRET` / `CERTCTL_API_KEYS_NAMED` rows. | Production deploys without an IdP; agent ↔ server; machine-to-machine; CI. |
+| `oidc` | Federated SSO via any OIDC IdP (Keycloak / Authentik / Okta / Auth0 / Entra ID / Google Workspace). PKCE-S256 + RFC 9700 pre-login UA/IP binding + RFC 9207 iss check + alg-downgrade defense. Successful login mints an HMAC-signed server-side session (cookie + CSRF rotation + back-channel logout). | Production deploys with an existing IdP; human admin access; SOC 2 / SAS 70 deployments. |
+| `none` (demo) | Every request served as the synthetic admin actor `actor-demo-anon`. | Demo / evaluation only. The fail-closed `CERTCTL_DEMO_MODE_ACK=true` requirement (Audit 2026-05-10 HIGH-12) prevents accidental production use; the boot-time WARN banner (Bundle 2) makes the posture unmissable. |
+
+Side surfaces:
+- **Day-0 bootstrap** via `CERTCTL_BOOTSTRAP_TOKEN` + `POST /api/v1/auth/bootstrap` mints the first admin actor + API key one-shot; the endpoint closes itself the moment any admin exists.
+- **Break-glass admin** (Auth Bundle 2 Phase 7.5) — Argon2id-hashed local-password recovery for SSO-outage. Default-OFF (`CERTCTL_BREAKGLASS_ENABLED=false`); surface returns 404 to scanners when disabled. Rate-limited at 5/min per source IP at the route (Bundle 5 closure).
+- **RBAC enforcement** on every gated handler via `auth.RequirePermission(perm, scope, scopeID)` — seven default roles (admin / operator / viewer / agent / mcp / cli / auditor), 33-permission canonical catalogue, scope types (global / profile / issuer). Auditor split is load-bearing: `r-auditor` holds only `audit.read` + `audit.export`.
+
+For deployments that need a federated-identity protocol certctl doesn't ship natively (SAML, mTLS-as-auth, LDAP), the authenticating-gateway pattern is still the right answer:
+
+### Authenticating-gateway pattern (SAML, mTLS-as-auth, LDAP)
+
+When the operator's identity ecosystem requires a protocol certctl doesn't ship natively in-process — SAML 2.0, mTLS-as-authentication (TLS client cert binding to actor), LDAP-direct, Kerberos — the standard pattern is to put an authenticating gateway in front of certctl and configure `CERTCTL_AUTH_TYPE=none` on the upstream. The gateway terminates the federated identity protocol, validates tokens / certificates / SAML assertions, and proxies the authenticated request to certctl as a same-origin call on a private network. This separation gives operators the full breadth of the modern identity ecosystem (oauth2-proxy, Envoy `ext_authz`, Traefik `ForwardAuth`, Pomerium, Authelia, Caddy `forward_auth`, Apache `mod_auth_openidc`, nginx `auth_request`) without certctl itself having to track signing-key rotation, claim mapping, audience validation, and the rest of the protocol surface area for every standard. Operators wanting per-request actor attribution past the gateway boundary forward the gateway-resolved identity (e.g., `X-Auth-Request-User` from oauth2-proxy) and run a small authorization layer at the gateway that enforces the bearer-key contract certctl actually uses.
+
+The historical context: pre-G-1, `CERTCTL_AUTH_TYPE=jwt` was accepted but silently routed through the api-key bearer middleware (a security finding masquerading as a config option, removed at the v2.x boundary; see [`upgrade-to-v2-jwt-removal.md`](upgrade-to-v2-jwt-removal.md) if you previously set it). Native OIDC arrived later via Auth Bundle 2 — operators on the pre-Bundle-2 "gateway-only for OIDC" pattern can keep it (it still works) or migrate to native OIDC per [`docs/migration/oidc-enable.md`](../migration/oidc-enable.md).
 
 ### Concurrency Safety
 

docs/reference/auth-standards-implemented.md

@@ -0,0 +1,83 @@
+# Authentication standards implemented
+
+> Last reviewed: 2026-05-10
+
+This document is an honest informational reference for operators, external testers, and acquirers who want to know which RFCs and standards certctl's authentication surface (API keys + RBAC + OIDC + sessions + back-channel logout + break-glass admin) implements, and which CWE weakness classes the implementation closes. Every row points at a real file or migration in this repository.
+
+This document is intentionally NOT a compliance-mapping doc. The operator retired the framework-mapping subtree (`docs/compliance/{index,soc2,pci-dss,nist-sp-800-57}.md`) on 2026-05-05; framework-name-drops (SOC 2 / PCI-DSS / HIPAA / NIST SSDF / FedRAMP) are also swept from prose mentions across `README.md` and `docs/` per that decision. RFC and CWE references stay because they are precise technical pointers; framework labels were marketing-flavored and prone to overclaim. If you are an auditor mapping certctl's controls to a framework, treat the rows below as evidence and do the framework mapping yourself against the framework you are auditing against.
+
+For the wider security posture, see [`security.md`](../operator/security.md). For the threat model behind these controls, see [`auth-threat-model.md`](../operator/auth-threat-model.md). For the per-IdP setup guides, see [`oidc-runbooks/index.md`](../operator/oidc-runbooks/index.md).
+
+## Table 1: RFCs and standards implemented end-to-end
+
+Each row carries at least one negative test (a test that asserts the fail-closed branch fires when a malformed input violates the spec).
+
+| Standard | What we implement | Source | Negative-test anchor |
+|---|---|---|---|
+| RFC 6749 (OAuth 2.0) | Authorization-code grant via OIDC; confidential-client credentials only | `internal/auth/oidc/service.go` (HandleAuthRequest, HandleCallback) | `internal/auth/oidc/service_test.go` (21+ negatives covering wrong aud / wrong iss / expired / etc.) |
+| RFC 7636 (PKCE) | S256 challenge mandatory; `plain` rejected at the service-layer sentinel; verifier persisted in pre-login row, single-use | `internal/auth/oidc/service.go` (oauth2.S256ChallengeOption hard-coded), `internal/auth/oidc/prelogin.go` | `TestService_PKCEPlainRejectedSentinel`, `TestService_StateReplayDeniedByConsumeOnce` |
+| RFC 7519 (JWT) | ID-token validation via go-oidc; service-layer alg allow-list (RS256/RS512/ES256/ES384/EdDSA); HS-family + `none` rejected | `internal/auth/oidc/service.go` (disallowedAlgs map, isDisallowedAlg) | `TestService_HandleCallback_RejectsHSAlgsConfusion`, `TestService_IdPDowngradeDefense_RejectsHSAdvertised` |
+| RFC 7517 (JWK) | JWKS fetch + cache + rotation handled transparently by coreos/go-oidc; operator-triggered RefreshKeys + auto-refresh on TTL expiry | `internal/auth/oidc/service.go` (RefreshKeys; cfg.JWKSCacheTTLSeconds default 3600) | `TestService_RefreshKeys_CatchesPostLoadDowngrade`, `TestKeycloakIntegration_JWKSRotation_RefreshKeysPicksUpNewKey` (Keycloak integration) |
+| OIDC Core 1.0 §3.1.3.7 | `iss` exact match, `aud` membership, `azp` for multi-aud, `at_hash` REQUIRED-when-access_token-present (certctl tightens the spec MAY → MUST), `nonce` constant-time-compare | `internal/auth/oidc/service.go` (HandleCallback steps 5-9) | `TestService_HandleCallback_RejectsWrongAudience`, `TestService_HandleCallback_AZPRequiredOnMultiAud`, `TestService_HandleCallback_ATHashRequiredWhenAccessTokenPresent`, `TestService_HandleCallback_RejectsNonceMismatch` |
+| OIDC Core 1.0 §5.3.2 (UserInfo endpoint) | Optional fallback when ID-token groups claim is empty; bounded by configured FetchUserinfo bool | `internal/auth/oidc/service.go` (fetchUserinfoGroups) | 4-case userinfo-fallback matrix in `service_test.go` (happy + endpoint-missing + endpoint-failing + userinfo-also-empty) |
+| OpenID Connect Back-Channel Logout 1.0 | `events` claim + `sid`/`sub` revocation; `nonce` MUST be absent; `jti`-based replay defense | `internal/api/handler/auth_session_oidc.go` (BackChannelLogout, DefaultBCLVerifier) | 6 negatives in `auth_session_oidc_test.go`: BCL missing events, BCL nonce-present, BCL unknown-key-sig, etc. |
+| RFC 6265 (HTTP State Management) | Session cookie attributes: `Secure` + `HttpOnly` + `SameSite=Lax` (default; configurable to Strict via `CERTCTL_SESSION_SAMESITE`); `Path=/`; host-only | `internal/auth/session/service.go` (cookie minting), `internal/api/handler/auth_session_oidc.go` (Set-Cookie wiring) | 7-case middleware-chain test matrix in `internal/auth/session/middleware_test.go` |
+| RFC 9700 (OAuth 2.0 Security Best Current Practice) | PKCE mandatory; no implicit flow; strict redirect_uri (registered + exact-match per OIDCProvider.RedirectURI); state non-guessable (32-byte random); single-use | `internal/auth/oidc/service.go`; `OIDCProvider.Validate()` enforces redirect_uri shape | `TestOIDCProvider_Validate_RejectsHTTPRedirectInProd`, state-replay test |
+| RFC 8414 (OAuth 2.0 Authorization Server Metadata) | Discovery doc fetched via go-oidc at provider creation + RefreshKeys; `id_token_signing_alg_values_supported` consulted for IdP-downgrade-attack defense | `internal/auth/oidc/service.go` (getOrLoad, guardAdvertisedAlgs) | `TestService_IdPDowngradeDefense_RejectsHSAdvertised` and `RejectsNoneAdvertised` |
+| RFC 7633 (X.509 TLS Feature Extension; Must-Staple) | Per-profile certctl issuance flag; out-of-scope for the auth surface but cited here because RFC 7633 OID `id-pe-tlsfeature` is in the same crypto-stack umbrella | `internal/connector/issuer/local/local.go` | SCEP master-bundle must-staple tests; not auth-surface territory |
+| RFC 8555 §7 (ACME directory metadata) | certctl-side ACME server tier; out-of-scope for the auth surface but cited because it shares the alg-pinning + nonce-handling discipline the auth surface carries forward | `internal/api/handler/acme/*` | per-route handler tests in `internal/api/handler/acme/` |
+| RFC 7515 (JWS) | JWS verification delegated to go-oidc/v3 + go-jose/v4; alg pin enforced at `gooidc.NewIDTokenVerifier` config + service-layer re-check | `internal/auth/oidc/service.go` (oauthConfig + verifier wiring) | `TestService_HandleCallback_RejectsExpired` and `TestService_HandleCallback_RejectsIATInFuture` |
+
+## Table 2: CWE / weakness classes the implementation closes
+
+Each row points at the file(s) that implement the defense and the test file(s) that pin the invariant.
+
+| CWE | Description | Where defended | Where pinned |
+|---|---|---|---|
+| CWE-287 (Improper Authentication) | Session-cookie HMAC verification (length-prefixed input defeats concat-collision) + alg-pinned ID-token verify | `internal/auth/session/service.go` (computeHMAC, parseCookie, Validate); `internal/auth/oidc/service.go` (HandleCallback) | `TestComputeHMAC_LengthPrefixDefeatsConcatCollision`; `TestService_Validate_ConcatenationCollisionDefeatedByLengthPrefix`; full 21+ OIDC negatives matrix |
+| CWE-352 (Cross-Site Request Forgery) | Double-submit cookie + `SameSite=Lax`/`Strict` + hashed CSRF token on session row; constant-time compare in CSRFMiddleware | `internal/auth/session/middleware.go` (CSRFMiddleware) | 7-case middleware-chain matrix (`internal/auth/session/middleware_test.go`); `TestSessionMiddleware_CSRFRequiredOnStateChangingMethods` |
+| CWE-384 (Session Fixation) | Session ID is opaque random `ses-<base64url>` (32 bytes entropy) generated server-side at login; cookie value rotates on every login (no inheritance from pre-login); CSRF token rotates alongside | `internal/auth/session/service.go` (Create, RotateCSRFToken) | `TestService_Create_AssignsFreshSessionID`; CSRF rotation pinned via `TestService_RotateCSRFToken_AfterLogin` |
+| CWE-294 (Authentication Bypass by Capture-Replay) | Single-use state, single-use nonce (both stored in pre-login row, atomic `DELETE...RETURNING` on consume); single-use authorization code (Keycloak/IdP-side); `jti`-based BCL replay defense | `internal/auth/oidc/prelogin.go` (LookupAndConsume); `internal/api/handler/auth_session_oidc.go` (BCL handler) | `TestService_StateReplayDeniedByConsumeOnce`; `TestService_HandleCallback_RejectsForgedPreLoginCookie`; BCL replay negative in handler tests |
+| CWE-916 / CWE-329 (Use of Password Hash With Insufficient Computational Effort / Use of a Key Past its Expiration Date) | Argon2id with OWASP 2024 params (m=64 MiB, t=3, p=4, 16-byte salt, 32-byte output) for break-glass passwords; per-credential random salt; PHC-format hash | `internal/auth/breakglass/service.go` (HashPassword, VerifyPassword); v3 ciphertext blob format with PBKDF2-SHA256 600,000 rounds for config-at-rest encryption | `TestPhase7_5_HashPasswordOWASP2024Params`; `TestPhase7_5_HashFormatPHC`; `internal/crypto/encryption_test.go` for v3 PBKDF2 floor |
+| CWE-307 (Improper Restriction of Excessive Authentication Attempts) | Failure count + lockout window on break-glass credential; threshold default 5, reset window default 1h, lockout duration default 30s; atomic single-statement IncrementFailure defeats concurrent racing attempts | `internal/auth/breakglass/service.go` (Login, IncrementFailure); `internal/repository/postgres/breakglass.go` | `TestPhase7_5_LockoutAfterThresholdFailures`; `TestPhase7_5_FailureCountResetsAfterWindow` |
+| CWE-345 (Insufficient Verification of Data Authenticity) | OIDC `at_hash` REQUIRED-when-access_token-present ties access token to ID token (certctl tightens OIDC core MAY → MUST); OIDC `iss` + `aud` + `azp` checks ensure token came from the configured IdP for the configured client | `internal/auth/oidc/service.go` (HandleCallback steps 5-9, atHashMatches) | `TestService_HandleCallback_ATHashRequiredWhenAccessTokenPresent`; `TestService_HandleCallback_RejectsATHashMismatch` |
+| CWE-200 (Information Exposure) | Token-leak hygiene tests on every secret-bearing path: ID tokens, access tokens, refresh tokens, authorization codes, PKCE verifiers, state, nonce, signing keys, break-glass passwords NEVER appear in any log line at any level | `internal/auth/oidc/service.go`, `internal/auth/session/service.go`, `internal/auth/breakglass/service.go` (all log calls audited); `internal/service/audit_redact.go` (audit redactor) | `internal/auth/oidc/logging_test.go` (4 grep-asserts); `internal/auth/breakglass/service_test.go` (token-leak hygiene + json.Marshal probe); `internal/auth/bootstrap/service_test.go` (canonical pattern) |
+| CWE-770 (Allocation of Resources Without Limits or Throttling) | Per-IP rate limit on `/auth/breakglass/login` via the global middleware.NewRateLimiter (default RPS / burst from `CERTCTL_RATE_LIMIT_*` env vars) wrapped around the entire mux; the breakglass login endpoint inherits this protection. Per-route override available via `middleware.NewRateLimiter` per-bucket configuration if the operator wants stricter caps | `cmd/server/main.go` (rateLimiter wiring at the root middleware stack); `internal/api/middleware/middleware.go` (NewRateLimiter) | `internal/api/middleware/ratelimit_test.go`; `internal/api/middleware/ratelimit_keyed_test.go` |
+| CWE-330 (Use of Insufficiently Random Values) | `crypto/rand` for state, nonce, PKCE verifier (via `oauth2.GenerateVerifier`), session signing keys (32 random bytes), session IDs (`ses-<base64url-no-pad>` from 32 random bytes), pre-login IDs (`pl-<base64url-no-pad>` from 16 random bytes), CSRF tokens (32 random bytes), break-glass salts (16 random bytes via `crypto/rand`) | `internal/auth/oidc/service.go` (randomB64URL); `internal/auth/session/service.go` (newOpaqueID, newCSRFToken); `internal/auth/oidc/prelogin.go` (newID); `internal/auth/breakglass/service.go` (HashPassword salt) | `TestPreLoginAdapter_CreatePreLogin_RNGFailure` (entropy-source error path); RNG failure pinned for every callsite |
+| CWE-311 (Missing Encryption of Sensitive Data) | OIDC `client_secret` AES-256-GCM encrypted at rest (v3 blob format: magic 0x03 + salt(16) + nonce(12) + ciphertext+tag); session signing keys same scheme; empty `CERTCTL_CONFIG_ENCRYPTION_KEY` returns `ErrEncryptionKeyRequired` (fail-closed) | `internal/crypto/encryption.go` (EncryptIfKeySet, DecryptIfKeySet); `internal/api/handler/auth_session_oidc.go` (encryptClientSecret); `internal/auth/session/service.go` (KeyMaterialEncrypted) | `internal/repository/postgres/oidc_encryption_invariant_test.go` (invariant test: ciphertext != plaintext, v2/v3 blob shape, round-trip + wrong-passphrase fails) |
+| CWE-326 (Inadequate Encryption Strength) | TLS 1.3 only on the certctl control plane (post-v2.2 milestone); HSTS-equivalent posture via HTTPS-only listener; AES-256-GCM for at-rest config encryption; PBKDF2-SHA256 600,000 rounds for v3 blob key derivation (OWASP 2024 floor) | `cmd/server/main.go` (TLS 1.3 listener config); `internal/crypto/encryption.go` (v3 PBKDF2 iteration count) | `TestServerTLSConfig_RejectsTLS12`; `TestEncryption_V3IterationCount_PinnedAtOWASP2024Floor` |
+| CWE-1004 (Sensitive Cookie Without HttpOnly) | Session cookie set with `HttpOnly=true`; CSRF cookie intentionally `HttpOnly=false` so the GUI can read it for the `X-CSRF-Token` header (the read is by-design per the double-submit-cookie pattern) | `internal/auth/session/service.go` (cookie attrs); `internal/api/handler/auth_session_oidc.go` (Set-Cookie wiring) | Cookie-attribute pinning in handler tests; documented in [auth-threat-model.md](../operator/auth-threat-model.md) "Session minting + cookies" subsection |
+| CWE-614 (Sensitive Cookie in HTTPS Session Without 'Secure' Attribute) | Session + CSRF cookies set with `Secure=true`; rejected at cookie-write time on `http://` listeners (HTTPS-only control plane post-v2.2) | `internal/auth/session/service.go`; `cmd/server/main.go` HTTPS-only listener | TLS-listener tests in `cmd/server/`; cookie attrs pinned in handler tests |
+| CWE-1275 (Sensitive Cookie with Improper SameSite Attribute) | Session cookie `SameSite=Lax` default (configurable to Strict via `CERTCTL_SESSION_SAMESITE`); CSRF defense via the double-submit pattern means `Lax` is sufficient even if the operator does not flip to Strict | `internal/auth/session/service.go` (cookie attrs); `internal/config/config.go` (SAMESITE env var) | Cookie-attribute pinning; SameSite enforcement is per-cookie | 
+
+## API-key + RBAC standards covered separately
+
+The above tables focus on the OIDC + sessions + back-channel logout + break-glass surface. The RBAC primitive carries its own implementation pointers; the [`auth-threat-model.md`](../operator/auth-threat-model.md) section "API-key + RBAC defenses" enumerates the full RBAC + bootstrap + auditor + approval-workflow surface. CWE-pointers that apply to the RBAC surface:
+
+- CWE-285 (Improper Authorization) — defended by the RequirePermission middleware + Authorizer.CheckPermission service-layer call. Pinned by 90+ tests across `internal/auth/` and `internal/service/auth/`.
+- CWE-862 (Missing Authorization) — pinned by `phase12_protocol_allowlist_test.go` (asserts protocol endpoints are explicitly allowlisted, NOT silently bypassing the gate).
+- CWE-863 (Incorrect Authorization) — pinned by the auditor-split invariant in `internal/domain/auth/auditor_test.go` (auditor role holds exactly `audit.read` + `audit.export` ONLY).
+- CWE-732 (Incorrect Permission Assignment for Critical Resource) — five admin-only fine-grained perms (`cert.bulk_revoke`, `crl.admin`, `scep.admin`, `est.admin`, `ca.hierarchy.manage`) seeded into `r-admin` only; pinned by migration 000030 + `r-admin`-only seed test.
+
+## What this document is NOT
+
+To preserve the operator's 2026-05-05 retired-compliance-docs decision:
+
+- This is NOT a SOC 2 / PCI-DSS / HIPAA / NIST SP 800-53 / NIST SSDF / FedRAMP framework-mapping doc.
+- This is NOT a marketing claim that certctl "satisfies CC6.1" or "complies with §164.312(a)(2)(iii)" or any similar framework label.
+- This IS an evidence list. An auditor doing framework mapping for their own compliance purposes can use this list as the source-of-truth pointer, then map each row to the framework control they are auditing against under their own judgment.
+
+If you are an external tester, an operator's auditor, or an acquirer doing technical diligence, this document gives you concrete file paths to read and concrete tests to run. If you want a framework-mapping document, build it yourself against the rows here using the framework-mapping methodology your audit firm prescribes; this project does not own that mapping.
+
+## Cross-references
+
+- [`auth-threat-model.md`](../operator/auth-threat-model.md) — threat model behind these defenses.
+- [`security.md`](../operator/security.md) — overall security posture.
+- [`oidc-runbooks/index.md`](../operator/oidc-runbooks/index.md) — per-IdP operator setup guides.
+- [`auth-benchmarks.md`](../operator/auth-benchmarks.md) — performance baselines for the validation paths cited above.
+- `internal/auth/oidc/` — OIDC service + groupclaim resolver + pre-login adapter + bootstrap hook.
+- `internal/auth/session/` — Session service + middleware + CSRF + signing-key rotation.
+- `internal/auth/breakglass/` — break-glass admin (Argon2id + lockout + constant-time + surface-invisibility).
+- `internal/crypto/encryption.go` — AES-256-GCM v3 blob format for at-rest encryption.
+- `migrations/000029` through `000038` — schema for RBAC, OIDC providers, sessions, signing keys, users, group mappings, pre-login, break-glass.
+- `scripts/ci-guards/multi-tenant-query-coverage.sh` — forward-compat multi-tenant query coverage guard.

docs/reference/cli.md

@@ -153,4 +153,4 @@ The `--wait` flag blocks until the job reaches a terminal state (Completed / Fai
 
 - [`docs/reference/api.md`](api.md) — the OpenAPI 3.1 spec the CLI wraps
 - [`docs/reference/mcp.md`](mcp.md) — the MCP server that exposes the same surface to AI assistants
-- [`docs/contributor/qa-prerequisites.md`](../contributor/qa-prerequisites.md) — local environment setup before the CLI can talk to a server
+- [`docs/getting-started/quickstart.md`](../getting-started/quickstart.md) — local environment setup before the CLI can talk to a server

docs/reference/configuration.md

@@ -80,7 +80,31 @@ For the full deploy contract see
 
 | Variable | Default | Description |
 |---|---|---|
-| `CERTCTL_AGENT_ID` | (none — required) | The agent's unique ID, issued by `POST /api/v1/agents/register` and bundled into the agent's registration response. Pass via this env var when the agent runs as a systemd unit / container without the `-agent-id` CLI flag. |
+| `CERTCTL_AGENT_ID` | (none — required) | The agent's unique ID, issued by `POST /api/v1/agents` (requires `CERTCTL_AGENT_BOOTSTRAP_TOKEN` when configured) and returned in the registration response body. Pass via this env var when the agent runs as a systemd unit / container without the `-agent-id` CLI flag. The bundled `install-agent.sh` does NOT auto-register — operators pre-register an agent via the REST endpoint (or the dashboard), then pass the returned ID to the script via `--agent-id`. |
+
+## Auth (RBAC + OIDC + sessions + break-glass)
+
+Configuration knobs for the RBAC + OIDC + sessions + break-glass
+auth surface. Full operator guidance lives in
+[`operator/rbac.md`](../operator/rbac.md),
+[`operator/oidc-runbooks/`](../operator/oidc-runbooks/index.md), and
+[`operator/auth-threat-model.md`](../operator/auth-threat-model.md).
+
+| Variable | Default | Description |
+|---|---|---|
+| `CERTCTL_SESSION_BIND_USER_AGENT` | `false` | Bind every session cookie to the User-Agent header captured at login; mismatch -> 401. Defense in depth against stolen cookies on the same network. |
+| `CERTCTL_SESSION_GC_INTERVAL` | `1h` | How often the scheduler's session-GC loop sweeps expired/revoked rows out of `sessions`. Trade-off: shorter = smaller table, more DB churn; longer = pile-up. |
+| `CERTCTL_OIDC_BCL_MAX_AGE_SECONDS` | `60` | Back-channel logout `iat` freshness window. Tokens older or newer than this skew (in either direction) are rejected. |
+| `CERTCTL_OIDC_PRELOGIN_REQUIRE_UA` | `false` | Reject the OIDC callback if the User-Agent at callback differs from the UA captured at pre-login. RFC 9700 §4.7.1 defense-in-depth. |
+| `CERTCTL_OIDC_PRELOGIN_REQUIRE_IP` | `false` | Same as `_UA` but for client IP. Set carefully — corporate networks with carrier-grade NAT can change apparent IP mid-flow. |
+| `CERTCTL_DEMO_MODE_ACK` | `false` | Operator acknowledgement that demo mode is intentional in this deploy. Required when `CERTCTL_AUTH_TYPE=none` to allow server startup; safety net against demo-mode-in-production leakage. |
+| `CERTCTL_TRUSTED_PROXIES` | (empty) | Comma-separated list of trusted-proxy CIDRs (e.g. `10.0.0.0/8,192.0.2.1`). XFF is consulted for client-IP derivation only when the immediate peer sits in this allowlist. |
+| `CERTCTL_TRUSTED_PROXIES_COUNT` | (synthesised) | Read-only counter exposed by `/api/v1/auth/runtime-config`; mirrors `len(CERTCTL_TRUSTED_PROXIES)`. Not operator-settable; documented here so the G-3 env-docs-drift guard catches drift. |
+| `CERTCTL_BOOTSTRAP_TOKEN` | (empty) | One-shot token used to mint the first admin role binding via `POST /api/v1/auth/bootstrap`. Once consumed, deletes itself from memory and unsets the bootstrap endpoint. |
+| `CERTCTL_BOOTSTRAP_TOKEN_SET` | (synthesised) | Boolean exposed by `/api/v1/auth/runtime-config`; `true` when `CERTCTL_BOOTSTRAP_TOKEN` was set at server start. Not operator-settable; documented here so the G-3 guard catches drift. |
+| `CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID` | (empty) | When OIDC is enabled, restricts the first-admin OIDC strategy to the named provider only — any other provider's tokens won't trigger the bootstrap hook. |
+| `CERTCTL_BOOTSTRAP_ADMIN_GROUPS_COUNT` | (synthesised) | Read-only counter exposed by `/api/v1/auth/runtime-config`; mirrors `len(CERTCTL_BOOTSTRAP_ADMIN_GROUPS)`. Documented here so the G-3 guard catches drift. |
+| `CERTCTL_BREAKGLASS_LOCKOUT_THRESHOLD` | `5` | Number of consecutive failed `/auth/breakglass/login` attempts that lock the credential. |
 
 ## SCEP profile binding (single-profile back-compat)
 

docs/reference/connectors/iis.md

@@ -157,7 +157,7 @@ The real IIS connector validation lives in:
 - Windows Server 2019 or 2022 host (or Windows 10/11 Pro with Hyper-V)
 - Docker Desktop in Windows containers mode
   (Settings → "Switch to Windows containers")
-- Go 1.25.9 + git
+- Go 1.25.10 + git
 
 ### Procedure
 

docs/reference/deployment-model.md

@@ -28,6 +28,46 @@ a single shared primitive:
 This document describes the operator-visible surface. The Go-level
 contract lives at `internal/deploy/doc.go`.
 
+## 1.6. Per-target guarantee matrix
+
+Added 2026-05-12 (Bundle 1 / CLAIM-M2 closure). The README previously
+claimed "every deploy goes through atomic-write + ownership-preservation
++ SHA-256 idempotency + per-target Prometheus counters + pre-deploy
+snapshot + on-failure rollback." That claim is true for the file-based
+deploy primitive only. Cloud / API targets use vendor-SDK semantics and
+do not share the same primitive. This matrix is the authoritative
+per-target answer.
+
+Legend: ✓ = supported / always on. ✗ = not applicable to this target
+family. ◐ = partial / vendor-specific equivalent. preview = ships but
+the production code path is a stub (see CLAIM-H4).
+
+| Target | Atomic write | Owner/perms preserved | SHA-256 idempotency | Pre-deploy snapshot | On-failure rollback | Post-deploy TLS verify | Prometheus counters | Server+agent shell-injection validation |
+|---|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
+| NGINX            | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
+| Apache           | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
+| HAProxy          | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
+| Caddy            | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ (no operator commands) |
+| Traefik          | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
+| Envoy            | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
+| Postfix / Dovecot| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
+| SSH known-hosts  | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ (no TLS endpoint) | ✓ | ✓ |
+| JavaKeystore     | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ (file format, no socket) | ✓ | ✓ |
+| IIS              | ◐ (Windows cert store API) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
+| WinCertStore     | ◐ (Windows cert store API) | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ |
+| F5 BIG-IP        | ✓ (iControl REST transaction) | ✗ (no FS) | ◐ (cert object name) | ◐ (transaction rollback) | ✓ (transaction rollback) | ✓ (mgmt API GET) | ✓ | ✗ |
+| AWS ACM          | ✗ (SDK call) | ✗ (no FS) | ◐ (ACM-side replace) | ✗ | ◐ (re-import old ARN) | ✗ | ✓ | ✗ |
+| Azure Key Vault  | ✗ (SDK call) | ✗ (no FS) | ◐ (KV-side versioning) | ✗ | ◐ (KV versioning) | ✗ | ✓ | ✗ |
+| Kubernetes Secrets | preview | preview | preview | preview | preview | preview | preview | ✗ |
+
+**Notes on the matrix:**
+
+- **Atomic write / owner-perms / SHA-256 idempotency / snapshot / rollback** are properties of the shared `deploy.Apply` primitive in `internal/deploy/`. They apply to file-based targets where certctl writes to disk.
+- **Cloud / API targets** (AWS ACM, Azure Key Vault) use the vendor SDK's import / replace operation. The vendor handles versioning and atomicity at their layer. certctl tracks the operation outcome via Prometheus counters; "rollback" in this row means "re-import the previous cert ARN" rather than the file-primitive's `os.Rename` rollback.
+- **F5** uses iControl REST transactions for atomicity (deploy-hardening I docs above). It does not touch a filesystem; the snapshot/rollback semantics live in the F5 transaction protocol.
+- **Kubernetes Secrets** ships but the production client (`realK8sClient`) returns `"real Kubernetes client not implemented"` for all methods (see `internal/connector/target/k8ssecret/k8ssecret.go:395+`). Operators evaluating against a real cluster should treat this connector as preview until the production client lands.
+- **Server+agent shell-injection validation** (Bundle 1 / RT-C1 closure 2026-05-12) is on for every connector that accepts operator-supplied command strings: `reload_command`, `validate_command`, `restart_command`. Validation runs at API ingestion (`internal/service/target.go::Create` + `::Update` + `::CreateTarget` + `::UpdateTarget` via `internal/connector/target/configcheck`) AND on the agent before deploy (`cmd/agent/main.go` post-`createTargetConnector`, calling each connector's full `ValidateConfig` method). Connectors that do not accept operator shell strings (Caddy / Traefik / Envoy / cloud targets) skip this check by design.
+
 ## 1.5. Audit closure status (2026-05-02 deployment-target audit)
 
 The 2026-05-02 deployment-target coverage audit

docs/reference/profiles.md

@@ -0,0 +1,112 @@
+# Certificate profiles
+
+> Last reviewed: 2026-05-09
+
+A `CertificateProfile` is the policy object that groups every cert with
+the same shape: which issuer mints it, which key algorithm + size are
+allowed, what EKUs and SANs the issuer should emit, what renewal
+window the scheduler uses, what targets get the cert deployed to. Every
+managed certificate references exactly one profile; changing a
+profile's policy retroactively affects renewal of every cert pointing
+at it.
+
+This file documents the profile lifecycle as it stands at v2.1.0.
+For the schema, see `migrations/000003_certificate_profiles.up.sql` +
+`migrations/000027_approval_workflow.up.sql` +
+`migrations/000033_approval_kinds.up.sql`. For the API surface,
+see `api/openapi.yaml` under `/api/v1/profiles`.
+
+## Anatomy
+
+| Field | Default | Purpose |
+|---|---|---|
+| `id` | autogenerated `prof-<slug>` | Stable opaque identifier; used by every other resource. |
+| `name` | required | Human-readable label; rendered in the GUI's profile picker. |
+| `issuer_id` | required | Which issuer (Local / Vault / EJBCA / ACME / SCEP / EST / ADCS / etc.) mints certs against this profile. |
+| `default_validity_days` | 90 | Rendered into the issuer call as the requested NotAfter delta. |
+| `renewal_window_days` | 30 | Scheduler enqueues a renewal Job when `cert.NotAfter - now < renewal_window_days`. |
+| `allowed_key_algorithms` | RSA 2048+, ECDSA P-256+ | Validates incoming CSRs at issuance time. |
+| `allowed_ekus` | server, client | RFC 5280 §4.2.1.12 EKU set. |
+| `must_staple` | false | Per-profile RFC 7633 `id-pe-tlsfeature` extension toggle. |
+| `requires_approval` | false | Gates issuance + renewal AND profile edits behind a four-eyes approval workflow. See below. |
+
+## RequiresApproval and the approval workflow
+
+Setting `requires_approval=true` on a profile does two things:
+
+1. **Issuance + renewal of every cert pointing at the profile gates
+   on a non-requester admin's approval.** The scheduler enqueues a
+   `Job` at status `AwaitingApproval`; the linked
+   `issuance_approval_requests` row stays at `pending` until either
+   approved (job → `Pending`, scheduler dispatches) or rejected (job
+   → `Cancelled`). Same actor cannot self-approve.
+2. **Edits to the profile itself gate on a non-requester admin's
+   approval.** This is the closure for the flip-flop
+   loophole - without it an admin could set `requires_approval=false`,
+   mutate any other field, set `requires_approval=true`, and the
+   approval workflow would only have been bypassed during the
+   "off" window. The profile-edit gate fires under three conditions:
+ - The live profile has `requires_approval=true` AND the operator
+     submits any edit (regardless of whether the edit changes the
+     flag).
+ - The live profile has `requires_approval=false` AND the operator
+     submits an edit that would set it to `true` (the flag-flip
+     direction is gated too because otherwise the gate could be
+     enabled by anyone and have no review).
+ - Both arms route through `ApprovalService.RequestProfileEditApproval`
+     which writes a row to `issuance_approval_requests` with
+     `approval_kind=profile_edit`. The pending profile diff is
+     serialized to `payload` (JSONB).
+
+**Edit response shape.** When the gate fires, `PUT /api/v1/profiles/{id}`
+returns HTTP 202 Accepted with body
+`{"status":"pending_approval","pending_approval_id":"ar-…"}`.
+The operator copies the approval ID, hands it to a peer admin, and
+the peer POSTs `/api/v1/approvals/{id}/approve` with their own
+credentials. On approve, the server deserializes `payload`, applies
+the diff against the live profile, and emits a
+`profile.edit_applied` audit row with `event_category=auth`. On
+reject, the pending row is dropped; the live profile is unchanged.
+
+**Same-actor self-approve is rejected** with HTTP 403 and the existing
+`ErrApproveBySameActor` sentinel. This is the load-bearing
+two-person-integrity invariant that satisfies SOC 2 CC6.3 + NIST
+SSDF PO.5.2.
+
+**Bypass mode.** `CERTCTL_APPROVAL_BYPASS=true` short-circuits both
+issuance approvals and profile-edit approvals; every request
+auto-approves with `actor=system-bypass`. Used by dev / CI for fast
+iteration; production deploys MUST leave it unset. A single SQL
+query (`SELECT FROM audit_events WHERE actor='system-bypass'`)
+confirms zero rows.
+
+## Operator workflows
+
+**Enable approval for an existing profile.** Edit the profile, set
+`requires_approval=true`. The first time you do this, the edit
+itself is gated (the live profile is non-approval but the proposed
+state is approval-tier, so the flip-on direction still routes through
+the workflow). Hand the approval ID to a peer; once approved, every
+subsequent edit and every renewal of every cert pointing at the
+profile gates on the workflow.
+
+**Disable approval.** Edit the profile, set `requires_approval=false`.
+This edit is gated because the live profile is currently
+approval-tier. A peer must approve the disable. Once disabled,
+subsequent edits flow through the direct-apply path again.
+
+**Audit who approved what.** The audit trail records every approval
+request + decision under `event_category=auth`. Filter via
+`GET /api/v1/audit?category=auth` or the `auditor` role's
+audit-only view. Each row carries the approval ID + the requester
++ the decider; the WORM trigger prevents tampering.
+
+## Related
+
+- `migrations/000027_approval_workflow.up.sql` (initial approval
+  schema, Rank 7 of the 2026-05-03 deep-research deliverable)
+- `migrations/000033_approval_kinds.up.sql` (adds
+  `approval_kind` + `payload` + nullable cert/job FKs)
+- `internal/service/approval.go::RequestProfileEditApproval`
+- `internal/service/profile.go::UpdateProfile` (gate)
+- `internal/api/handler/profiles.go::UpdateProfile` (202 mapping)

docs/testing/skip-inventory.md

@@ -0,0 +1,234 @@
+# Test Skip Inventory
+
+<!-- Auto-generated by scripts/skip-inventory.sh — do not edit by hand. -->
+<!-- Re-run after adding or removing any t.Skip(). CI guard:    -->
+<!-- scripts/ci-guards/skip-inventory-drift.sh                  -->
+
+> Last reviewed: 2026-05-13
+
+## Summary
+
+- Total t.Skip sites: **142**
+- testing.Short() guards: **76** (these gate behind `go test -short`)
+
+Re-run inventory with: `./scripts/skip-inventory.sh`.
+
+## Sites (grouped by package)
+
+### `cmd/agent`
+
+- `cmd/agent/keymem_test.go:209` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:425` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:451` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:491` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:523` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:526` — t.Skip("running as root; cannot revoke parent dir write permission")
+- `cmd/agent/keymem_test.go:553` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:556` — t.Skip("running as root; cannot revoke parent dir read+exec permission")
+- `cmd/agent/keymem_test.go:623` — t.Skip("chmod-error branch is only reliably triggerable on linux via /sys (read-only fs)")
+- `cmd/agent/keymem_test.go:631` — t.Skipf("/sys/kernel not stat-able as a dir on this host; skipping (%v)", err)
+- `cmd/agent/keymem_test.go:637` — t.Skipf("/sys/kernel mode %#o already satisfies no-chmod branch", mode)
+- `cmd/agent/keymem_test.go:652` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/keymem_test.go:655` — t.Skip("running as root; cannot revoke parent dir write permission")
+- `cmd/agent/keymem_test.go:686` — t.Skip("permission semantics differ on windows")
+- `cmd/agent/verify_test.go:402` — t.Skip("no TLS certificates configured on test server")
+
+### `cmd/server`
+
+- `cmd/server/preflight_demo_residual_test.go:41` — t.Skip("preflight A-8 test requires Postgres (testcontainers); skipping under -short")
+- `cmd/server/preflight_demo_residual_test.go:97` — t.Skip("A-8 testcontainers unavailable; skipping")
+
+### `deploy/test/acme-integration`
+
+- `deploy/test/acme-integration/certmanager_test.go:54` — t.Skip("KIND_AVAILABLE unset — kind-driven cert-manager integration test skipped")
+
+### `deploy/test`
+
+- `deploy/test/crl_ocsp_e2e_test.go:134` — t.Skip("integration only")
+- `deploy/test/crl_ocsp_e2e_test.go:65` — t.Skip("integration only")
+- `deploy/test/est_e2e_test.go:124` — t.Skip("integration tests require INTEGRATION=1; skipping libest e2e suite")
+- `deploy/test/est_e2e_test.go:129` — t.Skipf("libest sidecar (container %q) not running (status=%q). Run `cd deploy && docker compose -f docker-compose.test.yml --profile est-e2e up -d libest-client` to bring it up.", libestContainer, status)
+- `deploy/test/est_e2e_test.go:213` — t.Skip("/config/certs/bootstrap.pem not present in libest sidecar — skipping mTLS path. To enable: mint a bootstrap cert against the per-profile mTLS trust anchor and copy into deploy/test/certs/.")
+- `deploy/test/est_e2e_test.go:252` — t.Skip("server-keygen disabled on the e2e EST profile (HTTP 404). Enable via CERTCTL_EST_PROFILE_E2E_SERVER_KEYGEN_ENABLED=true in docker-compose.test.yml.")
+- `deploy/test/est_e2e_test.go:333` — t.Skipf("libest build lacks --tls-exporter support: %v", err)
+- `deploy/test/healthcheck_test.go:102` — t.Skip("docker not available — skipping image-level HEALTHCHECK test")
+- `deploy/test/healthcheck_test.go:163` — t.Skip("docker not available — skipping image-level HEALTHCHECK test")
+- `deploy/test/healthcheck_test.go:224` — t.Skip("docker not available — skipping runtime HEALTHCHECK test")
+- `deploy/test/healthcheck_test.go:227` — t.Skip("runtime HEALTHCHECK test takes ~45s; skipping under -short")
+- `deploy/test/healthcheck_test.go:229` — t.Skip("runtime probe contract not yet wired to a sidecar postgres; " +
+- `deploy/test/healthcheck_test.go:28` — // The tests skip cleanly with t.Skip when docker is not available
+- `deploy/test/healthcheck_test.go:32` — // Q-1 closure (cat-s3-58ce7e9840be): this file's 5 t.Skip sites are
+- `deploy/test/healthcheck_test.go:41` — //   - Line 212: hard t.Skip for the runtime probe contract — image-spec
+- `deploy/test/integration_test.go:1129` — t.Skip("no PEM data in certificate version")
+- `deploy/test/integration_test.go:513` — t.Skip("agent not yet online (may be slow to heartbeat)")
+- `deploy/test/integration_test.go:805` — t.Skip("depends on Phase04 (Local CA cert not created)")
+- `deploy/test/integration_test.go:901` — t.Skip("no discovered certificates yet (agent scan may not have run)")
+- `deploy/test/integration_test.go:942` — t.Skip("no certificate in Active state for renewal test")
+- `deploy/test/integration_test.go:954` — t.Skipf("renewal trigger returned: %s", body)
+- `deploy/test/nginx_vendor_e2e_test.go:108` — t.Skip()
+- `deploy/test/qa_test.go:1055` — t.Skip("Part 23 (S/MIME & EKU) is documented in docs/testing-guide.md::Part 23 " +
+- `deploy/test/qa_test.go:1065` — t.Skip("Part 24 (OCSP/CRL) is documented in docs/testing-guide.md::Part 24 " +
+- `deploy/test/qa_test.go:1175` — t.Skip("Requires compiled certctl-cli binary — manual test")
+- `deploy/test/qa_test.go:1179` — t.Skip("Requires compiled mcp-server binary + stdio — manual test")
+- `deploy/test/qa_test.go:1313` — t.Skip("Scheduler tests are timing-dependent — verify via Docker logs manually")
+- `deploy/test/qa_test.go:1320` — t.Skip("Requires Docker log inspection — manual test")
+- `deploy/test/qa_test.go:1327` — t.Skip("Requires browser — manual test")
+- `deploy/test/qa_test.go:1334` — t.Skip("Requires browser — manual test")
+- `deploy/test/qa_test.go:1338` — t.Skip("Requires browser — manual test")
+- `deploy/test/qa_test.go:1914` — t.Skip("Part 55 (Agent Soft-Retirement) is documented in docs/testing-guide.md::Part 55 " +
+- `deploy/test/qa_test.go:1924` — t.Skip("Part 56 (Notification Retry/Dead-Letter) is documented in docs/testing-guide.md::Part 56 " +
+- `deploy/test/qa_test.go:38` — // Q-1 closure (cat-s3-58ce7e9840be): this file contains 11 `t.Skip("Requires
+- `deploy/test/qa_test.go:46` — // the runtime t.Skip is the second-line guard for operators who run
+- `deploy/test/qa_test.go:50` — // is correct, and the t.Skip messages already name the missing
+- `deploy/test/qa_test.go:870` — t.Skip("Requires CA cert+key setup — manual test")
+- `deploy/test/qa_test.go:874` — t.Skip("Requires ACME CA with ARI support — manual test")
+- `deploy/test/qa_test.go:881` — t.Skip("Requires live Vault server — manual test")
+- `deploy/test/qa_test.go:885` — t.Skip("Requires DigiCert sandbox — manual test")
+- `deploy/test/scep_intune_e2e_test.go:159` — t.Skipf("integration stack not reachable at %s: %v — start docker-compose.test.yml first", serverURL, err)
+- `deploy/test/scep_intune_e2e_test.go:163` — t.Skipf("/scep/%s not configured — see deploy/docker-compose.test.yml for the e2eintune profile env vars", e2eintunePathID)
+- `deploy/test/scep_intune_e2e_test.go:166` — t.Skipf("/scep/%s GetCACaps returned %d — Intune profile may not be enabled in compose env", e2eintunePathID, resp.StatusCode)
+- `deploy/test/scep_intune_e2e_test.go:170` — t.Skipf("/scep/%s GetCACaps body=%q does NOT advertise SCEPStandard — Intune profile may be misconfigured", e2eintunePathID, string(body))
+- `deploy/test/vendor_e2e_helpers_smoke_test.go:31` — t.Skip("requires network egress to api.github.com (or similar known TLS endpoint); run manually")
+- `deploy/test/vendor_e2e_helpers_smoke_test.go:36` — t.Skip("requires network egress; run manually")
+- `deploy/test/vendor_e2e_helpers_smoke_test.go:41` — // When hostPath is empty the helper t.Skip's. Re-run-from-
+
+### `internal/api/handler`
+
+- `internal/api/handler/health_test.go:481` — t.Skip("integration-style test; covered by deploy/test/integration_test.go (//go:build integration). " +
+- `internal/api/handler/health_test.go:499` — t.Skipf("postgres driver unavailable in this build: %v", err)
+
+### `internal/auth/breakglass`
+
+- `internal/auth/breakglass/service_test.go:417` — t.Skip("timing test skipped in -short mode (Argon2id is expensive)")
+
+### `internal/auth/oidc/domain`
+
+- `internal/auth/oidc/domain/types_test.go:186` — t.Skip()
+
+### `internal/auth/oidc`
+
+- `internal/auth/oidc/bench_keycloak_test.go:103` — // signature matters because it calls t.Skip / t.Fatal / t.Cleanup.
+- `internal/auth/oidc/integration_keycloak_test.go:53` — // initialized in keycloakFor() so individual tests can `t.Skip` under
+- `internal/auth/oidc/integration_okta_smoke_test.go:64` — // If any required env var is missing, the test t.Skip's with a clear
+- `internal/auth/oidc/integration_okta_smoke_test.go:84` — t.Skipf("Okta smoke test requires env vars: %s — skipping", strings.Join(missing, ", "))
+
+### `internal/ciparity`
+
+- `internal/ciparity/surface_parity_test.go:97` — // readFileOrSkip reads a file; on ENOENT, calls t.Skipf rather than
+
+### `internal/connector/issuer/acme`
+
+- `internal/connector/issuer/acme/acme_failure_test.go:687` — t.Skipf("could not bind challenge server (env may not allow): %v", err)
+
+### `internal/connector/issuer/local`
+
+- `internal/connector/issuer/local/bundle9_coverage_test.go:467` — t.Skip("unexpectedly short DER")
+- `internal/connector/issuer/local/bundle9_coverage_test.go:592` — t.Skip("permission semantics differ on windows")
+- `internal/connector/issuer/local/bundle9_coverage_test.go:609` — t.Skip("permission semantics differ on windows")
+- `internal/connector/issuer/local/bundle9_coverage_test.go:621` — t.Skip("permission semantics differ on windows")
+- `internal/connector/issuer/local/bundle9_coverage_test.go:653` — t.Skip("permission semantics differ on windows")
+
+### `internal/connector/issuer/openssl`
+
+- `internal/connector/issuer/openssl/openssl_failure_test.go:124` — t.Skip("running as root; chmod 0o600 doesn't gate execution for uid 0")
+- `internal/connector/issuer/openssl/openssl_failure_test.go:71` — t.Skip("openssl adapter shell-out tests assume POSIX bash; skipping on Windows")
+
+### `internal/connector/notifier/email`
+
+- `internal/connector/notifier/email/email_test.go:425` — t.Skip("test requires no service on smtp.example.com:587")
+- `internal/connector/notifier/email/email_test.go:503` — t.Skip("test assumes no service on 127.0.0.1:54321")
+
+### `internal/connector/target/iis`
+
+- `internal/connector/target/iis/iis_test.go:225` — t.Skip("Skipping: powershell.exe not available (non-Windows)")
+- `internal/connector/target/iis/iis_test.go:92` — t.Skip("Skipping: powershell.exe not available (non-Windows)")
+
+### `internal/crypto`
+
+- `internal/crypto/encryption_property_test.go:35` — t.Skip("skipping property-based test in -short mode (PBKDF2 600k rounds × 50 iters > short budget)")
+- `internal/crypto/encryption_property_test.go:75` — t.Skip("skipping property-based test in -short mode (PBKDF2 cost)")
+
+### `internal/deploy`
+
+- `internal/deploy/coverage_test.go:403` — t.Skip("read-only chmod doesn't restrict root")
+- `internal/deploy/coverage_test.go:467` — t.Skip("non-unix")
+- `internal/deploy/deploy_test.go:611` — t.Skip("non-unix platform")
+
+### `internal/ratelimit`
+
+- `internal/ratelimit/sliding_window_test.go:146` — t.Skip("race-style test under -short")
+
+### `internal/repository/postgres`
+
+- `internal/repository/postgres/audit_worm_test.go:29` — t.Skip("skipping integration test in short mode")
+- `internal/repository/postgres/auth_revoke_scope_test.go:118` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_revoke_scope_test.go:149` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_revoke_scope_test.go:179` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_revoke_scope_test.go:208` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_revoke_scope_test.go:56` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_revoke_scope_test.go:87` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:123` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:153` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:181` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:207` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:229` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:252` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:281` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/auth_scope_test.go:95` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_encryption_invariant_test.go:160` — t.Skip("Phase 13 encryption invariant: integration test in short mode")
+- `internal/repository/postgres/oidc_encryption_invariant_test.go:225` — t.Skip("Phase 13 encryption invariant: integration test in short mode")
+- `internal/repository/postgres/oidc_encryption_invariant_test.go:62` — t.Skip("Phase 13 encryption invariant: integration test in short mode")
+- `internal/repository/postgres/oidc_prelogin_encryption_test.go:163` — t.Skip("HIGH-5 legacy fallback: integration test in short mode")
+- `internal/repository/postgres/oidc_prelogin_encryption_test.go:42` — t.Skip("HIGH-5 encryption invariant: integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:117` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:140` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:171` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:185` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:209` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:239` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:301` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:331` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:45` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:82` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/oidc_test.go:96` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/repo_test.go:1944` — t.Skip("integration test requires PostgreSQL")
+- `internal/repository/postgres/repo_test.go:2003` — t.Skip("integration test requires PostgreSQL")
+- `internal/repository/postgres/repo_test.go:2114` — t.Skip("integration test requires PostgreSQL")
+- `internal/repository/postgres/seed_test.go:91` — t.Skip("skipping integration test in short mode")
+- `internal/repository/postgres/session_test.go:100` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:120` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:167` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:197` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:211` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:246` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:259` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:29` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:307` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:340` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:407` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:54` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/session_test.go:86` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/testutil_test.go:39` — t.Skip("skipping integration test in short mode")
+- `internal/repository/postgres/user_test.go:106` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:131` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:170` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:210` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:29` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:302` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:339` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:374` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:59` — t.Skip("integration test in short mode")
+- `internal/repository/postgres/user_test.go:73` — t.Skip("integration test in short mode")
+
+### `internal/scep/intune`
+
+- `internal/scep/intune/challenge_golden_test.go:47` — t.Skip("regenerate fixtures only when -update-golden is passed")
+- `internal/scep/intune/challenge_test.go:213` — t.Skip("encoder didn't produce padding for this fixture; skipping")
+- `internal/scep/intune/rate_limit_test.go:139` — t.Skip("race-style test under -short")
+- `internal/scep/intune/replay_test.go:131` — t.Skip("race-style test under -short; run full suite for coverage")
+
+### `internal/service`
+
+- `internal/service/coverage_extras_test.go:374` — t.Skipf("RSA keygen unavailable: %v", err)
+- `internal/service/coverage_extras_test.go:394` — t.Skipf("ECDSA keygen unavailable: %v", err)
+

go.mod

@@ -1,6 +1,6 @@
 module github.com/certctl-io/certctl
 
-go 1.25.9
+go 1.25.10
 
 require (
 	github.com/google/uuid v1.6.0
@@ -18,12 +18,14 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/acm v1.38.3
 	github.com/aws/aws-sdk-go-v2/service/acmpca v1.46.14
 	github.com/aws/smithy-go v1.25.1
+	github.com/coreos/go-oidc/v3 v3.18.0
 	github.com/go-jose/go-jose/v4 v4.1.4
 	github.com/leanovate/gopter v0.2.11
 	github.com/masterzen/winrm v0.0.0-20250927112105-5f8e6c707321
 	github.com/pkg/sftp v1.13.10
-	golang.org/x/crypto v0.48.0
+	golang.org/x/crypto v0.50.0
-	golang.org/x/sync v0.19.0
+	golang.org/x/oauth2 v0.36.0
+	golang.org/x/sync v0.20.0
 	software.sslmate.com/src/go-pkcs12 v0.7.0
 )
 
@@ -111,9 +113,8 @@ require (
 	go.opentelemetry.io/otel v1.41.0 // indirect
 	go.opentelemetry.io/otel/metric v1.41.0 // indirect
 	go.opentelemetry.io/otel/trace v1.41.0 // indirect
-	golang.org/x/net v0.49.0 // indirect
+	golang.org/x/net v0.53.0 // indirect
-	golang.org/x/oauth2 v0.34.0 // indirect
+	golang.org/x/sys v0.43.0 // indirect
-	golang.org/x/sys v0.42.0 // indirect
+	golang.org/x/text v0.36.0 // indirect
-	golang.org/x/text v0.34.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )

go.sum

@@ -129,6 +129,8 @@ github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
 github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
 github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpSBQv6A=
 github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw=
+github.com/coreos/go-oidc/v3 v3.18.0 h1:V9orjXynvu5wiC9SemFTWnG4F45v403aIcjWo0d41+A=
+github.com/coreos/go-oidc/v3 v3.18.0/go.mod h1:DYCf24+ncYi+XkIH97GY1+dqoRlbaSI26KVTCI9SrY4=
 github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
 github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
 github.com/cpuguy83/dockercfg v0.3.2 h1:DlJTyZGBDlXqUZ2Dk2Q3xHs/FtnooJJVaad2S9GKorA=
@@ -482,8 +484,8 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
 golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
-golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
+golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
-golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
+golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -562,8 +564,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
 golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
-golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o=
+golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
-golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8=
+golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -576,8 +578,8 @@ golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ
 golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw=
+golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
-golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
+golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -591,8 +593,8 @@ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
+golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
-golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
+golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
 golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -645,14 +647,14 @@ golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
+golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
-golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
 golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
-golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg=
+golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
-golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM=
+golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -663,8 +665,8 @@ golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
 golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
-golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
+golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
-golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
+golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
 golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -723,8 +725,8 @@ golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
 golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
 golang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s=
-golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc=
+golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
-golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg=
+golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

internal/api/acme/account.go

@@ -1,5 +1,5 @@
-// Copyright (c) certctl
+// Copyright 2026 certctl LLC. All rights reserved.
-// SPDX-License-Identifier: BSL-1.1
+// SPDX-License-Identifier: BUSL-1.1
 
 package acme
 

internal/api/acme/ari.go

@@ -1,5 +1,5 @@
-// Copyright (c) certctl
+// Copyright 2026 certctl LLC. All rights reserved.
-// SPDX-License-Identifier: BSL-1.1
+// SPDX-License-Identifier: BUSL-1.1
 
 package acme
 

internal/api/acme/challenge.go

@@ -1,5 +1,5 @@
-// Copyright (c) certctl
+// Copyright 2026 certctl LLC. All rights reserved.
-// SPDX-License-Identifier: BSL-1.1
+// SPDX-License-Identifier: BUSL-1.1
 
 package acme
 

internal/api/acme/directory.go

@@ -1,5 +1,5 @@
-// Copyright (c) certctl
+// Copyright 2026 certctl LLC. All rights reserved.
-// SPDX-License-Identifier: BSL-1.1
+// SPDX-License-Identifier: BUSL-1.1
 
 // Package acme implements the ACME server-side protocol surface (RFC 8555
 // + RFC 9773 ARI). It is deliberately separate from

internal/api/acme/errors.go

@@ -1,5 +1,5 @@
-// Copyright (c) certctl
+// Copyright 2026 certctl LLC. All rights reserved.
-// SPDX-License-Identifier: BSL-1.1
+// SPDX-License-Identifier: BUSL-1.1
 
 package acme