Closes Phase 16 of cowork/auth-bundle-2-prompt.md. Three operator-
facing docs updated, one new migration guide ships, README nav row
added.
Files
=====
docs/operator/security.md (MODIFIED, Last reviewed bumped to 2026-05-10):
* Added 5 new Bundle 2 subsections under '## Authentication
surface' after the Bundle 1 approval-bypass-closure entry:
- 'OIDC federation (Bundle 2 Phases 1-7)' — alg allow-list,
IdP-downgrade defense, iss/aud/azp/at_hash, single-use
state+nonce, PKCE-S256 mandatory, JWKS rotation handling,
encrypted client_secret at rest with the v3 blob format
pinned by an integration test, pointer to oidc-runbooks/
for per-IdP setup.
- 'Sessions + back-channel logout (Bundle 2 Phases 4-6)' —
length-prefixed HMAC cookie wire format, HttpOnly + Secure
+ SameSite cookie hardening, idle/absolute timeouts, CSRF
defense, signing-key rotation primitive, fail-fatal
EnsureInitialSigningKey at server boot, OpenID Connect
Back-Channel Logout 1.0 (NOT RFC 8414).
- 'OIDC first-admin bootstrap (Bundle 2 Phase 7)' — coexists
with Bundle 1's env-var-token bootstrap, group-scoped via
CERTCTL_BOOTSTRAP_ADMIN_GROUPS + CERTCTL_BOOTSTRAP_OIDC_PROVIDER_ID,
one-shot per tenant.
- 'Break-glass admin (Bundle 2 Phase 7.5)' — default-OFF,
surface invisibility via 404-not-403, Argon2id with OWASP
2024 params, lockout state machine, constant-time-via-
verifyDummy, WARN log at boot, runbook pointer for
operator drill.
- 'Migrating an existing deployment to OIDC' — pointer to
the new migration/oidc-enable.md walkthrough.
docs/migration/oidc-enable.md (NEW, Last reviewed 2026-05-10):
* Step-by-step migration guide for an operator on a Bundle-1-merged
deployment to enable OIDC SSO. Pre-reqs (CERTCTL_CONFIG_ENCRYPTION_KEY,
admin actor with auth.oidc.create + auth.oidc.edit, IdP tenant)
+ 7 numbered steps (pin encryption key, complete IdP-side per
runbook, configure certctl-side OIDCProvider, add group→role
mappings with fail-closed warning, optional first-admin bootstrap,
verify with single test user, announce SSO endpoint).
* Rollback section covering the 4-step disable flow + the 409
Conflict on provider-delete-while-sessions-exist + the
existing-sessions-keep-working-until-expiry semantics.
* Troubleshooting section pinning 8 most-common failure modes
(discovery doc fetch fails / IdP downgrade defense rejects /
no roles assigned / iss mismatch / pre-login expired / state
mismatch / sessions revoked but user can hit API / JWKS
rotation breaks login).
* Database row count drift documented so operators know what to
expect after OIDC is live (10 Bundle 2 tables enumerated).
* Cross-references to oidc-runbooks/ + security.md +
auth-threat-model.md + auth-benchmarks.md + auth-standards-implemented.md.
CHANGELOG.md (MODIFIED):
* v2.1.0 section title bumped from 'Auth Bundle 1: RBAC primitive'
to 'Auth Bundles 1 + 2: RBAC primitive + OIDC SSO + sessions'.
* Replaced the Bundle 1 closing-bullet ('Bundle 2 starts after
Bundle 1 lands on master') with 18 new Bundle 2 entries:
- OIDC + sessions + back-channel logout + break-glass overview.
- OIDC token validation pinned at three layers (alg allow-list,
IdP-downgrade defense, OIDC Core §3.1.3.7 re-verification).
- Length-prefixed HMAC session cookies.
- CSRF double-submit + hashed-token-on-row.
- OIDC client_secret AES-256-GCM v3 blob at rest +
integration-test invariant.
- OIDC first-admin bootstrap.
- Default-OFF break-glass admin (Argon2id + lockout +
constant-time + surface invisibility).
- GUI: 4 new pages + login-page IdP buttons + sidebar logout.
- 11 new MCP tools for OIDC + session management.
- 6 per-IdP runbooks (Keycloak / Authentik / Okta / Auth0 /
Entra ID / Google Workspace).
- Threat model extended with 5 new defense subsections + 8 new
threat-catalogue subsections.
- Performance baselines documented (4 benchmarks; 3 measured
+ 1 operator-runs).
- Standards-and-RFC implementation table (13 RFCs + 14 CWEs;
NOT a compliance-mapping doc).
- Coverage gates held at floor 90 across all 4 Bundle 2
packages (anti-Bundle-1-mistake invariant).
- Multi-tenant query CI guard (ratchet baseline 32).
- Phase 10 Keycloak testcontainers integration test + optional
Okta smoke test.
- OpenAPI cookieAuth security scheme + 13 new endpoints + 4
break-glass endpoints.
- Bundle-1-only compat regression CI guard +
Bundle-1-to-2-upgrade regression CI guard.
* Final paragraph updated to point at oidc-enable.md alongside
api-keys-to-rbac.md as the two migration walkthroughs.
docs/README.md (MODIFIED):
* Added the new oidc-enable.md migration row under '## Migration'
alongside the existing api-keys-to-rbac.md entry, with a
one-line description flagging it as the Bundle 2 OIDC
onboarding walkthrough.
Verification
============
* Last-reviewed on security.md + oidc-enable.md: 2026-05-10.
* Internal-link sweep on oidc-enable.md: 0 broken (every relative
link resolves via shell-loop verification).
* Internal-link sweep on docs/README.md: 0 broken (all .md
references resolve).
* No Go-side impact, make verify gate unchanged.
Bundle 2 documentation deliverables now complete: security.md +
auth-threat-model.md + oidc-runbooks/ + auth-benchmarks.md +
auth-standards-implemented.md + api-keys-to-rbac.md + oidc-enable.md
+ CHANGELOG.md v2.1.0. The full Bundle 2 surface is operator-
discoverable from docs/README.md root nav.
17 KiB
Changelog
v2.1.0 - Auth Bundles 1 + 2: RBAC primitive + OIDC SSO + sessions ⚠️
SECURITY: AUDIT YOUR API KEYS.
Bundle 1 ships role-based authorization. Every existing API key configured via
CERTCTL_API_KEYS_NAMED(or the legacyCERTCTL_AUTH_SECRET) is mapped to the r-admin role on the first upgrade boot so existing automation keeps working unchanged. Most keys do NOT need full admin power; downgrade them before tagging the next release.Recommended post-upgrade flow:
# 1. List every key with its current role: certctl-cli auth keys list # 2. Walk an interactive prompt that downgrades each key: certctl-cli auth keys scope-down # 3. Or get a heuristic suggestion based on 30 days of audit history: certctl-cli auth keys scope-down --suggest certctl-cli auth keys scope-down --suggest --apply # applies the suggestion # 4. Or drive scope-down from a JSON config (Helm post-upgrade hook): certctl-cli auth keys scope-down --non-interactive ./scope-down.jsonThe synthetic
actor-demo-anonactor (used whenCERTCTL_AUTH_TYPE=noneis configured) is system-managed and excluded from the prompt loop.
What else changed in v2.1.0:
- RBAC primitive shipped.
tenants,roles,permissions,role_permissions,actor_rolestables (migration 000029); 33-permission canonical catalogue; 7 default roles (admin,operator,viewer,agent,mcp,cli,auditor); per-handler permission gates viaauth.RequirePermissionmiddleware (replaces the legacyIsAdminboolean check on the 5 admin-only handlers). - Day-0 admin bootstrap. Set
CERTCTL_BOOTSTRAP_TOKENon a fresh deploy and POST a single curl call against/api/v1/auth/bootstrapto mint the first admin API key; one-shot, never logged, and locks closed once any admin actor exists. Migration 000031 ships theapi_keystable that stores the SHA-256 hash; the plaintext is shown in the response body once and never persisted. - Auditor role split. New
auditorrole holds onlyaudit.readaudit.export. Compliance reviewers can read the audit trail without holding mutation power. Migration 000032 addsaudit_events.event_categoryso auditors can filter to authentication-related events specifically.
/v1/auth/checkenrichment. Response now includes the actor's standing roles and effective permissions, so the GUI gates affordances from a single fetch on app boot.- Approval-bypass closure. Edits to a profile that has (or
would have)
RequiresApproval=truenow route through theApprovalServicetwo-person integrity gate (Phase 9). Migration 000033 addsapproval_kind+payloadtoissuance_approval_requestsso cert-issuance and profile-edit approvals share the same workflow. Same-actor self-approve is rejected withErrApproveBySameActorfor both kinds. Closes the flip-flop loophole where an admin could disable approval, mutate, re-enable. Documented atdocs/reference/profiles.md. - GUI: Roles / API Keys / Auth Settings / Approvals queue.
Four new pages under
/auth/*consume/v1/auth/mefor permission-aware rendering. The Approvals queue blocks self-approve at the client layer (Approve/Reject buttons hidden when requested_by == current actor_id) on top of the server-side enforcement. AuditPage gains a category filter (cert_lifecycle / auth / config) for the auditor view. - MCP server gains 12 RBAC tools. Operators driving certctl
from Claude / VS Code / any MCP client get parity with the GUI
- CLI. Each tool routes through the same HTTP handler; permission gates fire server-side.
- OpenAPI catalogues every new route. Every Bundle 1 endpoint
ships with an
operationId; the parity test guards against drift. - Coverage gates.
internal/auth/andinternal/service/auth/now have ≥85% coverage floors in.github/coverage-thresholds.yml. The 12-path negative-test list from the Bundle 1 prompt is fully covered (path #12 deferred with in-tree TODO). - Protocol-endpoint allowlist pinned at three layers. The
middleware bypass (
auth.IsProtocolEndpoint), the router-levelAuthExemptRouterRoutesconstant, and a newphase12_protocol_allowlist_test.goAST scan all guard against accidentally wrapping ACME / SCEP / EST / OCSP / CRL routes inrbacGate. - Bundle 2: OIDC + sessions + back-channel logout + break-glass.
Auth Bundle 2 ships in the same v2.1.0 release. Operators get OIDC
SSO support for Keycloak / Authentik / Okta / Auth0 / Microsoft
Entra ID / Google Workspace (via Keycloak broker), HMAC-signed
session cookies with idle/absolute timeouts + CSRF defense,
back-channel logout per OpenID Connect Back-Channel Logout 1.0,
and a default-OFF break-glass admin path with Argon2id passwords
for SSO-broken incidents. API-key auth keeps working unchanged
alongside; existing automation needs no changes. Migration walkthrough
at
docs/migration/oidc-enable.md; per-IdP setup guides atdocs/operator/oidc-runbooks/index.md. - OIDC token validation pinned at three layers. Algorithm
allow-list (RS256/RS512/ES256/ES384/EdDSA only) with HS-family +
nonerejected at the service-layer sentinel; IdP-downgrade-attack defense at provider creation AND every JWKS RefreshKeys (intersects the IdP's advertisedid_token_signing_alg_values_supportedagainst the allow- list, rejects providers that advertise weak algs even before any token is signed); OIDC Core §3.1.3.7 re-verification ofiss/aud/azp/at_hash(REQUIRED-when-access_token-present per Phase 3 tightening of the spec MAY → MUST) /exp/iatwindow /nonceconstant-time-compare. PKCE-S256 mandatory;plainrejected. Single-use state + nonce via atomicDELETE...RETURNINGon consume. - Session cookies use length-prefixed HMAC. The cookie wire format
is
v1.<session_id>.<signing_key_id>.<base64url-no-pad(HMAC-SHA256)>with HMAC inputlen:sid:len:kid(NOT bare-concat) to defeat concatenation collisions.HttpOnly+Secure+SameSite=Laxdefault;SameSite=Strictconfigurable viaCERTCTL_SESSION_SAMESITE. Idle timeout 1h / absolute 8h defaults; scheduler GC sweeps expired rows hourly. Signing keys rotate via the newRotateSigningKeyprimitive; the old key stays valid forCERTCTL_SESSION_SIGNING_KEY_RETENTION(default 24h) so existing cookies validate during rollover. - CSRF defense via double-submit-cookie + hashed-token-on-row.
Plaintext CSRF token in the JS-readable
certctl_csrfcookie (intentionallyHttpOnly=falsefor the GUI to echo into theX-CSRF-Tokenheader); SHA-256 hash on the session row;subtle.ConstantTimeComparein the newCSRFMiddleware. API-key actors are CSRF-exempt (no session row in context). - OIDC
client_secretencrypted at rest. AES-256-GCM v3 blob format (magic 0x03 + salt(16) + nonce(12) + ciphertext+tag) using the existingCERTCTL_CONFIG_ENCRYPTION_KEY. Encryption invariant pinned by an integration test asserting ciphertext != plaintext + v3 blob shape + round-trip recovery + wrong-passphrase fails. - OIDC first-admin bootstrap. New
CERTCTL_BOOTSTRAP_ADMIN_GROUPSCERTCTL_BOOTSTRAP_OIDC_PROVIDER_IDenv vars: the first OIDC-authenticated user with a matching group claim becomes admin per tenant. Coexists with the Bundle 1 env-var-token bootstrap; the admin-existence probe ensures only one wins. Audit row (bootstrap.oidc_first_admin) on every grant.
- Break-glass admin (default-OFF). New
CERTCTL_BREAKGLASS_ENABLEDenv var (defaultfalse). When enabled, the local Argon2id-password admin path bypasses OIDC + group-claim layers — intended ONLY for SSO-broken incidents. Argon2id with OWASP 2024 params (m=64 MiB, t=3, p=4); lockout after 5 failures (configurable); constant-time across all failure paths viaverifyDummy; surface invisibility (HTTP 404 on every endpoint when disabled, NOT 403). WARN log at server boot when enabled. WebAuthn/FIDO2 second factor pairing on the v3 roadmap (Decision 12). - GUI: OIDC Providers + Group → Role Mappings + Sessions + login
buttons. Four new pages under
/auth/*consume the Bundle 2 API surface. Login page renders one "Sign in with X" button per configured OIDC provider (in addition to the API-key form, which remains as a fallback for Bearer-mode + break-glass paths). Sessions page exposes own-sessions + admin all-actors view. Every actionable element is permission-gated server-side viaauth.oidc.*andauth.session.*perms; client-side hide is UX layer. Logout button in the sidebar firesPOST /auth/logoutto clear the session server-side before redirecting to login. - MCP server gains 11 OIDC + session tools.
certctl_auth_list_oidc_providers,_get_oidc_provider,_create_oidc_provider,_update_oidc_provider,_delete_oidc_provider,_refresh_oidc_provider,_list_group_mappings,_add_group_mapping,_remove_group_mapping,_list_sessions,_revoke_session. Operator-facing MCP tool count goes 12 (Bundle 1 RBAC) → 23 across the auth surface. Total MCP tool count:grep -cE 'mcp\.AddTool\(' internal/mcp/tools*.go≈ 150. - Per-IdP runbooks: 6 production-tier setup guides at
docs/operator/oidc-runbooks/. Each runbook follows a consistent five-section layout (Prerequisites / IdP-side config / certctl-side config / Verification / Troubleshooting + Validation checklist with operator sign-off line). Keycloak is the canonical reference; Authentik / Okta / Auth0 / Entra ID / Google Workspace document the IdP-specific deltas (Auth0's namespaced custom claims; Entra ID's group OBJECT IDs; Google Workspace's missing-groups-claim limitation- the recommended Keycloak broker pattern).
- Threat model extended.
docs/operator/auth-threat-model.mdships 5 new "Defenses Bundle 2 ships" subsections + 8 new threat- catalogue subsections (OIDC token forgery / session hijacking / IdP compromise / back-channel logout failure modes / group-claim manipulation / bootstrap risks / break-glass risks / token-leak hygiene). 6 new SQL-shaped operator-facing checks. New "Threats Bundle 2 does NOT close" section enumerating the 8 v3-backlog items (WebAuthn / JIT elevation / SAML / multi-tenant activation / HSM-FIPS / OIDC RP-initiated logout / Playwright / per-IdP external-tester sign-off). - Performance baselines documented.
docs/operator/auth-benchmarks.mdships four benchmarks with measured baselines on a 4 vCPU / 8 GiB / Postgres 16 / Go 1.25 floor:BenchmarkSession_SteadyStatep99 5 µs (target < 1 ms; 200× under),BenchmarkSession_ColdProcessp99 7.1 ms (target < 10 ms),BenchmarkOIDC_SteadyStatep99 1.5 ms (target < 5 ms),BenchmarkOIDC_ColdCacheoperator-runs against live Keycloak viamake benchmark-auth-coldcache. - Standards + RFC implementation table.
docs/reference/auth-standards-implemented.mdships 13 RFC / standard rows + 14 CWE rows with concrete file paths- negative-test anchors per row. NOT a compliance-mapping doc per the operator's 2026-05-05 retired-compliance-docs decision; the doc explicitly says "build the framework mapping yourself against the rows here using the framework-mapping methodology your audit firm prescribes; this project does not own that mapping."
- Coverage gates held at floor 90 across all four Bundle 2
packages.
internal/auth/oidc/93.7%,internal/auth/session/94.9%,internal/auth/breakglass/91.5%,internal/auth/user/domain/96.4%. NO held-low-with-rationale entry — the Phase 13 prompt's anti-Bundle-1-mistake rule held. Bundle 1's existing 85% floors forinternal/auth/+internal/service/auth/stay 85 (already-shipped-and-accepted) per the prompt's explicit inheritance rule. - Multi-tenant query CI guard. New
scripts/ci-guards/multi-tenant-query-coverage.sh(ratchet-style, baseline 32 at v2.1.0 close): greps every SELECT/UPDATE/DELETE ininternal/repository/postgres/against 10 tenant-aware tables, fails on regression OR improvement (forces the operator to lift / lower the baseline visibly). Forward-compat protection so a future Bundle 3 / managed-service multi-tenant activation can flip the switch without finding silent tenant-data-leak bugs in shipped queries. - Phase 10 Keycloak testcontainers integration test. New build-tag-
gated suite at
internal/auth/oidc/testfixtures/+integration_keycloak_test.godrives the full OIDC flow against a live Keycloak container booted by testcontainers-go. 5-test matrix: discovery + JWKS load, full PKCE auth-code happy path with HTTP form scraping, logout-revokes- session, JWKS rotation, unmapped-groups-fails-closed. Reuses one container across the matrix to amortize the 60-90s boot. Optional Okta smoke test (build-taggedintegration && okta_smoke) for live tenant validation. New Makefile targets:make keycloak-integration-testmake okta-smoke-test+make benchmark-auth-coldcache.
- OpenAPI surface extended. New
cookieAuthsecurity scheme (apiKey/cookie/certctl_session) alongside the existingbearerAuth. 13 new Bundle 2 endpoints across the OIDC + session- group-mapping CRUD surface; 4 break-glass endpoints with
surface-invisibility framing. The N-bundle-2-security-empty-preserved
CI guard locks the
security: []opt-out count at ≥ 14 so existing public endpoints stay public.
- group-mapping CRUD surface; 4 break-glass endpoints with
surface-invisibility framing. The N-bundle-2-security-empty-preserved
CI guard locks the
- Bundle-1-only compat regression CI guard. New
scripts/ci-guards/bundle-1-compat-regression.shasserts the load-bearing invariants that protect the Bundle-1-only-deploy case (session middleware defers-to-next, CSRF passthrough on missing session row, ChainAuthSessionThenBearer wired, public OIDC routes in AuthExempt allowlist, AuthInfo guards on OIDCProvidersResolver != nil). Siblingbundle-1-to-2-upgrade-regression.shasserts the upgrade-path invariants (migrations 000034..000038 are CREATE TABLE IF NOT EXISTS- BEGIN/COMMIT-wrapped + no DROP TABLE / ALTER...DROP COLUMN against 19 protected Bundle-1 tables + ON CONFLICT DO NOTHING on permission seed).
Migration ordering, idempotency, and downgrade are documented in
docs/migration/api-keys-to-rbac.md
(API-key → RBAC, Bundle 1) and docs/migration/oidc-enable.md
(API-key → OIDC, Bundle 2). The threat model lives at
docs/operator/auth-threat-model.md.
Day-2 RBAC operations live at docs/operator/rbac.md.
RFC + CWE evidence at docs/reference/auth-standards-implemented.md.
v2.0.68 - Image registry path changed ⚠️
Image registry path changed. Starting this release, container images publish to
ghcr.io/certctl-io/certctl-serverandghcr.io/certctl-io/certctl-agent. Existing pulls fromghcr.io/shankar0123/certctl-{server,agent}:<tag>continue to work for previously-published tags (the registry never deletes images), but the:latesttag at the old path stops moving forward at this release. Update yourdocker pullpaths,docker-compose.ymlimage:keys, or Helmimage.repositoryvalues to receive future updates. Oldgit clone/git push/ install-script / API URLs continue to redirect forever - only the container-registry path changed.
This is the only operator-action-required change in v2.0.68. Other changes in this release are cosmetic URL refreshes after the GitHub-org transfer from shankar0123/certctl to certctl-io/certctl (HTTP redirects mean no other operator action is required) plus an internal contextcheck lint fix in the agent. Full commit list is on the GitHub release page.
certctl no longer maintains a hand-edited per-version changelog. Per-release notes are auto-generated from commit messages between consecutive tags.
Where to find what changed in a given release:
- GitHub Releases - every tag has an auto-generated "What's Changed" section pulled from the commits between that tag and the previous one, plus per-release supply-chain verification instructions (Cosign / SLSA / SBOM).
git log <prev-tag>..<this-tag> --oneline- same content, locally.
Why no hand-edited CHANGELOG.md:
certctl is solo-developed and pushes directly to master. Maintaining a
hand-edited CHANGELOG meant the file drifted (entries piled into
[unreleased] and never got promoted to per-version sections when tags were
cut). A stale CHANGELOG is worse than no CHANGELOG - it signals abandoned
maintenance to security-conscious operators doing diligence.
The auto-generated release notes work here because commit messages follow a
descriptive convention: <area>: <summary> with a longer body for non-trivial
changes (see git log v2.0.50..HEAD for the established pattern). Anyone
reading the GitHub Releases page can see exactly what landed in each version
without depending on the author to manually update a separate file.
For the historical record: earlier versions (pre-v2.2.0 and the [2.2.0] tag itself) had a hand-edited CHANGELOG. That content is preserved in git history at the v2.2.0 tag.