certctl/docs/migration/api-keys-to-rbac.md

Migrating API keys to RBAC (v2.0.x → v2.1.0)

Last reviewed: 2026-05-09

This is the upgrade guide for an existing certctl deployment moving from v2.0.x's "every API key is admin or not" model to v2.1.0's RBAC primitive. Everything keeps working through the upgrade — the Bundle 1 migration backfills every existing API key to the r-admin role on first boot, so the pre-existing automation that was using those keys does not change behavior. However, most keys do not need full admin power; this guide walks the operator through the post-upgrade scope-down flow.

⚠️ SECURITY: AUDIT YOUR API KEYS

Bundle 1 maps every existing CERTCTL_API_KEYS_NAMED entry (and every legacy CERTCTL_AUTH_SECRET-synthesized key) to the r-admin role on the first boot after migration 000029 applies. This is the safe-for-back-compat default — your CI / agents / scripts keep working without changes — but if you don't downgrade keys, every key in your fleet has full admin permissions including bulk-revoke, CRL admin, and CA hierarchy management.

Run the scope-down flow before tagging the next release. The release notes for v2.1.0 lead with this callout for a reason.

Upgrade flow

1. Apply the migration

The migration runner is idempotent. Re-applying is a no-op if the schema is already at the target version. The Bundle 1 migrations that ship with v2.1.0:

Migration	What it does
000029_rbac.up.sql	Creates tenants, roles, permissions, role_permissions, actor_roles. Seeds 7 default roles + 33-permission catalogue + the synthetic actor-demo-anon admin grant. Backfills every named API key into actor_roles with the r-admin role.
000030_rbac_admin_perms.up.sql	Seeds 5 admin-only fine-grained permissions (cert.bulk_revoke, crl.admin, scep.admin, est.admin, ca.hierarchy.manage) into r-admin only.
000031_api_keys.up.sql	Creates the api_keys table for runtime-minted keys (Bundle 1 Phase 6 bootstrap).
000032_audit_category.up.sql	Adds event_category column to audit_events with the closed enum (cert_lifecycle / auth / config).
000033_approval_kinds.up.sql	Adds approval_kind + payload to issuance_approval_requests for the Phase 9 approval-bypass closure.

The Bundle 1 server applies these on first boot. No operator action is required other than running the upgrade.

2. Verify the backfill landed

# Inspect the seeded actor_roles rows. You should see one row per
# entry in CERTCTL_API_KEYS_NAMED (Admin=true keys → r-admin,
# Admin=false keys → r-viewer) plus the seeded actor-demo-anon
# admin row.
psql -d certctl -c "SELECT actor_id, role_id, granted_by, granted_at FROM actor_roles ORDER BY granted_at;"

If the table is empty, the boot-loader hook in cmd/server/auth_backfill.go::backfillNamedKeyActorRoles did not run; re-check that CERTCTL_AUTH_TYPE is api-key (the boot hook is gated on cfg.Auth.Type != none).

3. List + scope-down keys

The certctl-cli ships a four-mode scope-down command. Pick the mode that matches your fleet size + automation posture.

Interactive walk

certctl-cli auth keys scope-down

Walks every actor (skips the synthetic actor-demo-anon) and prompts for a target role. Empty input keeps the existing role. Type one of admin, operator, viewer, agent, mcp, cli, auditor to replace.

Non-interactive JSON config (Helm post-upgrade hook)

cat > scope-down.json <<EOF
{
  "ci-bot":         "operator",
  "agent-prod-1":   "agent",
  "agent-prod-2":   "agent",
  "monitoring-bot": "viewer",
  "compliance-bot": "auditor"
}
EOF

certctl-cli auth keys scope-down --non-interactive ./scope-down.json

Empty role values revoke every current grant WITHOUT granting a replacement; assign roles selectively with certctl-cli auth keys assign.

Audit-driven suggestion

# Preview suggestions based on the last 30 days of audit history
certctl-cli auth keys scope-down --suggest

# Apply the suggestions
certctl-cli auth keys scope-down --suggest --apply

The classifier (pure function in internal/cli/auth_scope_down.go::SuggestRoleFromAuditEvents) walks the actor's audit events and emits one of:

Suggestion	Trigger
admin	Any auth.role.* / auth.key.* / ca.hierarchy.* / *.bulk_revoke / *.admin action
mcp	All observed actions are MCP-shaped (mcp.*)
viewer	All observed actions are read-only (*.read or *.list)
agent	All observed actions are agent-shaped (agent.*, cert.read, cert.issue)
operator	Cert / profile / target lifecycle mutations without admin signals

The classifier is conservative — when in doubt, it prefers the narrower role. The operator confirms each suggestion before any mutation lands (unless --apply is set).

4. Mint a fresh admin via bootstrap (optional, for fresh deployments)

If you're standing up a fresh deployment instead of upgrading an existing one, the bootstrap path mints the first admin key without needing the operator to know the env-var format:

# Set the bootstrap token in the server environment.
export CERTCTL_BOOTSTRAP_TOKEN=$(openssl rand -hex 32)

# Boot the server. Logs include "bootstrap endpoint enabled".
docker compose up -d

# Mint the first admin key.
curl -X POST $URL/api/v1/auth/bootstrap \
  -H 'Content-Type: application/json' \
  -d '{"token":"'$CERTCTL_BOOTSTRAP_TOKEN'","actor_name":"first-admin"}'

The response carries the plaintext key_value once. Capture it and use it as the Bearer token for subsequent calls. Subsequent bootstrap calls return HTTP 410 Gone.

See docs/operator/rbac.md for the full bootstrap flow + the threat model.

What changes for code that called IsAdmin

Pre-Bundle-1, the five admin handlers checked auth.IsAdmin(ctx) directly in the body. Bundle 1 Phase 3.5 moved those checks to the router via the auth.RequirePermission middleware (wrapped through the rbacGate helper in internal/api/router/router.go). The behavior contract is unchanged: r-admin-roled callers reach the handler, anyone else gets HTTP 403 BEFORE the body runs.

If your code consumed auth.IsAdmin directly (it shouldn't — the helper is internal), the new convention is:

1. Wrap the route in rbacGate(reg.Checker, "<perm>", handler) in router.go.
2. Add the perm to migrations/000030_rbac_admin_perms.up.sql (or migrations/000029_rbac.up.sql's catalogue).
3. Grant the perm to the right default roles.

The five admin-only fine-grained perms shipped in Phase 3.5 stay on r-admin only by default. Operators delegate by creating custom roles with the specific perm.

Helm-specific upgrade

The certctl Helm chart applies migrations on container start via the standard migrations runner. No chart changes are required; the helm upgrade command runs identically:

helm upgrade certctl certctl/certctl \
  --version <new-version> \
  --reuse-values

Post-upgrade, the boot loader runs the named-key actor-role backfill against the CERTCTL_API_KEYS_NAMED env-var-injected into the deployment. The "AUDIT YOUR API KEYS" callout applies — add a post-upgrade Job to your release pipeline that runs certctl-cli auth keys scope-down --non-interactive against a checked-in JSON config, so the role narrowing is deterministic across upgrade rollouts.

Example post-upgrade Job:

apiVersion: batch/v1
kind: Job
metadata:
  name: certctl-scope-down
spec:
  template:
    spec:
      containers:
      - name: scope-down
        image: ghcr.io/certctl-io/certctl-cli:<tag>
        command:
          - certctl-cli
          - auth
          - keys
          - scope-down
          - --non-interactive
          - /config/scope-down.json
        envFrom:
          - secretRef:
              name: certctl-cli-credentials
        volumeMounts:
          - name: scope-down-config
            mountPath: /config
      volumes:
        - name: scope-down-config
          configMap:
            name: certctl-scope-down-config
      restartPolicy: OnFailure

The ConfigMap holds the {actor_id: role_id} map; the Secret holds the API key the Job uses to call /v1/auth/keys/.../roles.

Docker Compose-specific upgrade

For deploy/docker-compose.yml deployments:

1. Pull the new images: docker compose pull
2. Verify your CERTCTL_AUTH_TYPE value before restarting. If it was none (the demo path), the post-upgrade server will boot in demo mode again — the synthetic actor-demo-anon admin covers every request, no scope-down is meaningful. If you're moving from none to api-key mode, set CERTCTL_API_KEYS_NAMED first, then restart.
3. docker compose up -d to apply.
4. docker compose logs certctl-server | grep -i 'loaded persisted api_keys' to verify the boot loader ran. The first-boot log line includes the count of keys loaded into the runtime keystore.
5. Run certctl-cli auth keys scope-down against the running server.

The five examples in examples/ (acme-nginx, private-ca-traefik, step-ca-haproxy, multi-issuer, acme-wildcard-dns01) all run in demo mode (CERTCTL_AUTH_TYPE=none) and are unaffected by the RBAC migration — the synthetic actor-demo-anon admin grant covers every request.

Verifying the upgrade landed

After the scope-down flow completes:

1. certctl-cli auth me while authenticated as each named key confirms the right effective_permissions for that role.
2. psql -c "SELECT actor_id, array_agg(role_id ORDER BY role_id) FROM actor_roles GROUP BY actor_id;" gives the full picture in one query.
3. The audit trail (GET /api/v1/audit?category=auth) shows the auth.role.assign and auth.role.revoke rows for every change you made — confirm via the GUI's /audit?category=auth view.
4. Read the updated docs/operator/rbac.md for day-2 RBAC management.

Rollback

If the upgrade goes wrong, the down migrations exist in lockstep:

# Roll back via your migration runner (golang-migrate, Atlas, etc.).
# Migrations 000029-000033 each have a .down.sql that reverses the
# .up.sql. Down migrations are destructive on data added by the up
# migration (api_keys rows, role grants on actors, profile-edit
# approvals); take a backup first.

After rollback, the v2.0.x binary works against the v2.0.x schema unchanged. The operator's API keys still authenticate (the in-memory hash table is rebuilt from CERTCTL_API_KEYS_NAMED on boot regardless of schema version).

Cross-references

• docs/operator/rbac.md — the operator how-to for the new RBAC primitive
• docs/operator/auth-threat-model.md — what the new controls defend against
• docs/reference/profiles.md — the Phase 9 approval-bypass closure
• docs/operator/security.md — the full security posture
• cowork/auth-bundle-1-prompt.md — the design + phase plan
• cowork/auth-bundles-index.md — the per-phase status tracker
• CHANGELOG.md — the v2.1.0 release notes lead with this guide