certctl/docs/features.md

certctl V2 Feature Inventory

Complete reference of all features shipped in the V2 release (as of March 2026).

---

API Surface

Overview

• 84 endpoints across 17 resource domains under /api/v1/
• REST API with HTTP semantics (GET, POST, PUT, DELETE)
• All endpoints require authentication by default (configurable)
• OpenAPI 3.1 spec with full schema documentation

Authentication & Security

• API Key Authentication — SHA-256 hashed keys with constant-time comparison
• Bearer Token Flow — Authorization: Bearer {api_key} header
• Auth Configuration — Configurable via CERTCTL_AUTH_TYPE (api-key, jwt, none)
• Auth Info Endpoint — GET /api/v1/auth/info (no auth required for GUI pre-login detection)
• Auth Check Endpoint — GET /api/v1/auth/check (validate credentials)

Rate Limiting

• Token Bucket Algorithm — Configurable requests-per-second (RPS) and burst size
• 429 Responses — Rate limit exceeded with Retry-After header
• Configuration — CERTCTL_RATE_LIMIT_ENABLED, CERTCTL_RATE_LIMIT_RPS (default 50), CERTCTL_RATE_LIMIT_BURST (default 100)

CORS

• Configurable Per-Origin Allowlist — CERTCTL_CORS_ORIGINS (comma-separated or wildcard)
• Preflight Caching — Standard CORS headers

Query Features (M20)

Feature	Details
Sorting	?sort=-notAfter (8 fields: notAfter, expiresAt, createdAt, updatedAt, commonName, name, status, environment)
Pagination (Page-Based)	?page=1&per_page=50 (max 500, default 50)
Pagination (Cursor)	?cursor=base64_token&page_size=100 (keyset pagination with next_cursor in response)
Time-Range Filters	?expires_before=2026-12-31T23:59:59Z&expires_after=2026-01-01T00:00:00Z&created_after=...&updated_after=... (RFC3339 format)
Sparse Fields	?fields=id,common_name,status (reduce response size)
Additional Filters	?status=active&agent_id=a-xxx&profile_id=p-xxx&issuer_id=...&owner_id=...&team_id=...

Endpoint Breakdown by Domain

Domain	Endpoints	Key Operations
Certificates	11	List, create, get, update (archive), versions, deployments, trigger renewal, trigger deployment, revoke
CRL & OCSP	3	JSON CRL, DER CRL per issuer, OCSP responder
Issuers	6	List, create, get, update, delete, test connection
Targets	5	List, create, get, update, delete
Agents	7	List, register, get, heartbeat, CSR submit, certificate pickup, get work, report job status
Jobs	5	List, get, cancel, approve, reject
Policies	6	List, create, get, update, delete, list violations
Profiles	5	List, create, get, update, delete
Teams	5	List, create, get, update, delete
Owners	5	List, create, get, update, delete
Agent Groups	6	List, create, get, update, delete, list agents in group
Discovery	7	Submit scan results, list discovered certs, get detail, claim, dismiss, list scans, summary stats
Audit	3	List events, list by resource, export (CSV/JSON)
Notifications	3	List, get, mark as read
Stats	5	Dashboard summary, certificates by status, expiration timeline, job trends, issuance rate
Metrics	1	JSON metrics (gauges, counters, uptime)
Health	4	Health check, readiness check, auth info, auth check

---

Certificate Lifecycle

Certificate States (8 total)

• Pending — Created, awaiting issuance
• Active — Valid and deployed
• Expiring — Within configured threshold (default 30 days)
• Expired — Past NotAfter date
• RenewalInProgress — Renewal job submitted
• Failed — Issuance or renewal failed
• Revoked — Revoked via POST /api/v1/certificates/{id}/revoke
• Archived — Manually archived via DELETE endpoint

Key Generation Modes

Mode	Details
Agent-Side (Default)	ECDSA P-256 key generation on agent; private keys never touch control plane
Server-Side (Demo Only)	RSA-2048 key generation on server; requires explicit CERTCTL_KEYGEN_MODE=server with log warning

Certificate Versions

• Multiple versions per certificate (issuance, renewal)
• Each version includes: serial number, fingerprint, PEM-encoded chain
• CSR preserved for audit trail
• Version history with rollback capability in GUI

AwaitingCSR Job State

• Renewal and issuance jobs pause when CERTCTL_KEYGEN_MODE=agent
• Agent generates ECDSA P-256 key locally, creates CSR, submits via POST /api/v1/agents/{id}/csr
• Server signs and stores certificate version
• Work endpoint enriched with common_name and sans for agent CSR generation

---

Revocation Infrastructure

Revocation API

• Endpoint — POST /api/v1/certificates/{id}/revoke (RFC 5280 reason codes)
• 8 Reason Codes — unspecified, keyCompromise, caCompromise, affiliationChanged, superseded, cessationOfOperation, certificateHold, privilegeWithdrawn
• Best-Effort Issuer Notification — Issuer connector failure doesn't block revocation
• Immutable Recording — certificate_revocations table with idempotent ON CONFLICT logic

CRL (Certificate Revocation List)

• JSON CRL — GET /api/v1/crl returns entries array with serial numbers, reasons, revoked timestamps
• DER X.509 CRL — GET /api/v1/crl/{issuer_id} returns proper DER-encoded CRL signed by issuing CA
• 24-Hour Validity — CRL refreshed every 24 hours
• CA Key Required — Sub-CA or issuing CA key must be available for signing

OCSP Responder

• Endpoint — GET /api/v1/ocsp/{issuer_id}/{serial}
• Responses — good (certificate valid), revoked (in CRL), unknown (not issued by this CA)
• Signed — OCSP responses signed by issuing CA

Short-Lived Certificate Exemption

• Policy — Certificates with TTL < 1 hour (from profile) skip CRL/OCSP
• Rationale — Expiry is sufficient revocation signal for short-lived certs
• Exemption Applied — During CRL generation and OCSP response construction

Revocation Notifications

• Webhook + email notifications on revocation events
• Routed by certificate owner email via existing notifier system

---

Certificate Profiles

Profile Model

Named enrollment profiles defining certificate issuance constraints.

Field	Details
ID	Prefixed text PK (p-xxx)
Name	Human-readable profile name
Allowed Key Algorithms	RSA, ECDSA, Ed25519 with minimum key sizes (e.g., RSA 2048+, ECDSA P-256+)
Max TTL	Maximum certificate lifetime (days or duration)
Allowed EKUs	Extended key usage OIDs (serverAuth, clientAuth, etc.)
Required SANs	Mandatory Subject Alternative Names (patterns or fixed values)
Short-Lived Support	TTL < 1 hour triggers CRL/OCSP exemption

GUI Management

• Full CRUD page with profile details
• Crypto constraint badges visible in list view
• Profile assignment dropdown on certificate detail

---

Policy Engine

Policy Rules (5 types)

Rule Type	Purpose	Example
AllowedIssuers	Restrict which CAs can issue	Only LetsEncrypt or Internal CA
AllowedDomains	Domain whitelist/blacklist	Allow *.example.com, deny *.staging.example.com
RequiredMetadata	Enforce ownership, team	Require owner_id and team_id populated
AllowedEnvironments	Environment constraints	Restrict to production or staging
RenewalLeadTime	Minimum renewal window	Renew 60 days before expiry (minimum)

Violation Tracking

• Severity Levels — Warning, Error, Critical
• Per-Policy Violations — GET /api/v1/policies/{id}/violations with timestamp and violated certificate ID
• Real-Time Evaluation — Violations checked during issuance, renewal, and deployment
• Audit Trail — All violations logged to audit events table

Policy Application Scope

• Applied at renewal policy level
• Scoped to agent groups via agent_group_id foreign key
• Rule set can be enabled/disabled per policy

---

Issuer Connectors (4 Implemented)

Local CA

• Mode — Self-signed (default) or sub-CA (production)
• Sub-CA Configuration — Load CA cert+key from disk (CERTCTL_CA_CERT_PATH, CERTCTL_CA_KEY_PATH)
• Key Formats Supported — RSA, ECDSA, PKCS#8
• CRL Generation — Signed by CA, 24h validity
• OCSP Signing — Delegates to CA's private key
• Use Case — Internal PKI, enterprise trust chains

ACME v2

• Challenge Types — HTTP-01 (default) and DNS-01 (wildcard support)
• DNS-01 Script Hooks — Pluggable DNS solver for any provider (Cloudflare, Route53, Azure DNS, etc.)
• Configuration — CERTCTL_ACME_DIRECTORY_URL, CERTCTL_ACME_EMAIL, CERTCTL_ACME_CHALLENGE_TYPE=dns-01, CERTCTL_ACME_DNS_PRESENT_SCRIPT, CERTCTL_ACME_DNS_CLEANUP_SCRIPT
• DNS Propagation Wait — Configurable timeout before validation
• Use Case — Public CAs (LetsEncrypt), wildcard certs

step-ca

• Protocol — Native /sign and /revoke API (not ACME)
• Authentication — JWK provisioner with key file + password
• Configuration — CERTCTL_STEPCA_URL, CERTCTL_STEPCA_PROVISIONER_NAME, CERTCTL_STEPCA_PROVISIONER_KEY_PATH, CERTCTL_STEPCA_PROVISIONER_PASSWORD
• Operations — Issue, renew, revoke
• Use Case — Smallstep private CA, internal PKI with strong auth

OpenSSL / Custom CA

• Mechanism — Delegate signing to user-provided shell scripts
• Scripts — Sign script (CSR→cert), revoke script (serial+reason), CRL script (full CRL)
• Timeout — Configurable timeout (default 30s) with process interruption
• Configuration — CERTCTL_OPENSSL_SIGN_SCRIPT, CERTCTL_OPENSSL_REVOKE_SCRIPT, CERTCTL_OPENSSL_CRL_SCRIPT, CERTCTL_OPENSSL_TIMEOUT_SECONDS
• Use Case — PKIX-compliant external CAs, PowerShell issuers, custom workflows

---

Target Connectors (3 Implemented + 2 Stubs)

NGINX

• Deployment — Separate cert, chain, and key files
• Validation — nginx -t configuration test
• Reload — Graceful reload via SIGHUP (or nginx -s reload)
• Target Config — Certificate path, chain path, key path
• Status — Fully implemented (M10)

Apache httpd

• Deployment — Separate cert, chain, and key files
• Validation — apachectl configtest or apache2ctl configtest
• Reload — Graceful reload via apachectl graceful or apache2ctl graceful
• Target Config — Certificate path, chain path, key path
• Status — Fully implemented (M10)

HAProxy

• Deployment — Combined PEM file (cert + chain + key concatenated)
• Validation — Optional haproxy -c -f config test
• Reload — Process signal or socket-based reload (configurable)
• Target Config — Combined PEM path, optional reload command
• Status — Fully implemented (M10)

F5 BIG-IP (Stub)

• Protocol — iControl REST API via proxy agent
• Status — Interface only in V2; implementation planned for V2 or V3
• Deployment Model — Proxy agent + BIG-IP API client in same network zone
• Authentication — iControl credentials stored in target config

IIS (Stub)

• Dual-Mode Architecture — Agent-local PowerShell (primary) or proxy agent WinRM (agentless)
• Status — Interface only in V2; implementation planned for V2 or V3
• Deployment Model — Agent runs PowerShell cmdlets locally or proxy agent invokes WinRM
• Binding — Bind certificate to IIS site by hostname

---

Notifier Connectors (6 Channels)

Email

• SMTP — Standard SMTP or TLS endpoint
• Configuration — Server, port, auth credentials (env vars)
• Use Case — Owner notifications, compliance distribution lists

Webhook

• HTTP POST — Custom JSON payload to any endpoint
• Headers — Content-Type, custom auth headers (configurable)
• Use Case — Slack (via custom webhook), Microsoft Power Automate, custom platforms

Slack

• Protocol — Incoming Webhook
• Message Format — Markdown with bold subject, formatted body
• Overrides — Channel (CERTCTL_SLACK_CHANNEL), username (CERTCTL_SLACK_USERNAME), emoji
• Configuration — CERTCTL_SLACK_WEBHOOK_URL
• Use Case — Team notifications, ops channels

Microsoft Teams

• Protocol — Incoming Webhook
• Message Format — MessageCard with ThemeColor, Summary, Sections
• Markdown Support — Formatted text within sections
• Configuration — CERTCTL_TEAMS_WEBHOOK_URL
• Use Case — Team-wide alerts, cross-team visibility

PagerDuty

• Protocol — Events API v2
• Trigger Events — Alert on expiration, failure, revocation
• Severity — Configurable default (default "warning")
• Custom Details — Certificate ID, days remaining, owner, etc.
• Configuration — CERTCTL_PAGERDUTY_ROUTING_KEY, CERTCTL_PAGERDUTY_SEVERITY
• Use Case — Incident response, on-call escalations

OpsGenie

• Protocol — Alert API v2
• Priority — Configurable default (default "P3")
• Tags — Category tags (cert expiration, deployment failure, etc.)
• Responders — Optional team routing
• Configuration — CERTCTL_OPSGENIE_API_KEY, CERTCTL_OPSGENIE_PRIORITY
• Use Case — Multi-team alerting, escalation policies

Notification Types

• Expiration Alert — Certificate approaching threshold (30/14/7/0 days)
• Renewal Started — Renewal job initiated
• Renewal Completed — Certificate successfully renewed
• Deployment Completed — Certificate deployed to target
• Deployment Failed — Target deployment error
• Revocation — Certificate revoked with reason
• Policy Violation — Certificate violates renewal policy

---

Agent Fleet

Agent Registration & Heartbeat

• Registration — POST /api/v1/agents with agent name and API key
• Heartbeat — POST /api/v1/agents/{id}/heartbeat every 60 seconds
• Auto-Offline — Agents marked offline after 3 missed heartbeats (configurable)
• Last Heartbeat Timestamp — Tracked in agents table

Agent Metadata (M10)

Collected via runtime introspection and network utilities.

Field	Source	Example
OS	runtime.GOOS	linux, darwin, windows
Architecture	runtime.GOARCH	amd64, arm64
Hostname	os.Hostname()	nginx-prod-1
IP Address	net.Interface + net.IP	10.0.1.5
Version	Agent binary version (from build flags)	v2.1.0

Agent Groups (M11b)

Dynamic grouping and filtering for policy assignment and deployment targeting.

Criterion	Details	Example
OS Match	Exact string match	linux, darwin, windows
Architecture Match	Exact string match	amd64, arm64, 386
IP CIDR Match	IPv4 or IPv6 CIDR block	10.0.0.0/8, 192.168.1.0/24
Version Match	Semantic version range (optional)	>=2.0.0, <3.0.0
Manual Membership	Explicit include/exclude	Include a-xxx, exclude a-yyy
MatchesAgent()	Dynamic evaluation at job time	Criteria match→agent included

Agent Group GUI

• List with dynamic match criteria badges (color-coded)
• Enable/disable toggle per group
• Manual membership editor (include/exclude lists)
• Agent count per group (dynamic)
• Scoped to renewal policies via agent_group_id FK

Agent Capabilities

Agents report to /api/v1/agents/{id}/work with supported target types and issuers.

• Target Deployment — NGINX, Apache httpd, HAProxy, F5 BIG-IP (proxy), IIS (proxy)
• Key Management — ECDSA P-256 keygen, key storage at CERTCTL_KEY_DIR (default /var/lib/certctl/keys), 0600 file permissions
• CSR Submission — POST /api/v1/agents/{id}/csr for AwaitingCSR jobs

Fleet Overview Page

• OS/Architecture Grouping — Agents grouped by GOOS + GOARCH
• Charts — Status distribution (pie), version breakdown (bar)
• Per-Platform Listing — Expandable agent list under each OS/Arch combo
• Health Indicators — Online/offline status, last heartbeat, uptime

---

Certificate Discovery (M18b)

Overview

Agents automatically discover existing certificates in the infrastructure — on filesystem, in key stores, or elsewhere — report findings to the control plane, and operators triage them for enrollment.

Agent-Side Discovery

• Configuration — CERTCTL_DISCOVERY_DIRS env var (comma-separated list) or --discovery-dirs CLI flag
• Scan Execution — Runs on agent startup and every 6 hours in background
• Supported Formats — PEM (.pem, .crt, .cer, .cert) and DER (.der) files
• Recursive Walk — Scans directory trees to find all certificates
• File Filtering — Skips files > 1MB and obvious key files

Certificate Extraction

Each discovered certificate is parsed and its metadata extracted:

Field	Source	Example
Common Name	X.509 Subject CN	api.example.com
SANs	X.509 SubjectAltNames	api.example.com, *.api.example.com
Serial	Certificate serial number	0x123abc...
Issuer DN	X.509 Issuer	CN=Internal CA, O=Acme Inc
Subject DN	X.509 Subject	CN=api.example.com, O=Acme Inc
Not Before	Validity start	2024-01-15T00:00:00Z
Not After	Validity end	2026-01-15T00:00:00Z
Key Algorithm	Key type	RSA, ECDSA, Ed25519
Key Size	Bits	2048, 256, 4096
Is CA	CA flag in extensions	true/false
Fingerprint	SHA-256 hash (dedup key)	a1b2c3d4e5f6...

Server-Side Processing

• Deduplication — Uses fingerprint + agent ID + path as unique key; prevents duplicates
• Status Tracking — Three statuses: Unmanaged (discovered, not yet claimed), Managed (linked to control plane cert), Dismissed (operator decided not to manage)
• Audit Trail — discovery_scan_completed, discovery_cert_claimed, discovery_cert_dismissed events logged with actor and reason
• Storage — discovered_certificates and discovery_scans tables in PostgreSQL

Triage Workflow

1. Agent submits scan results via POST /api/v1/agents/{id}/discoveries
2. Server deduplicates and stores discovery records
3. Operator views GET /api/v1/discovered-certificates?status=Unmanaged
4. For each unmanaged cert:
   • Claim it — POST /api/v1/discovered-certificates/{id}/claim links to managed cert or creates new enrollment
   • Dismiss it — POST /api/v1/discovered-certificates/{id}/dismiss removes from triage queue
5. Tracking enables visibility into what's deployed vs. what's managed

Discovery API Endpoints (M18b)

Endpoint	Method	Purpose
/api/v1/agents/{id}/discoveries	POST	Agent submits scan results
/api/v1/discovered-certificates	GET	List discovered certs (with ?agent_id, ?status filters)
/api/v1/discovered-certificates/{id}	GET	Get single discovered cert detail
/api/v1/discovered-certificates/{id}/claim	POST	Link to managed cert or create enrollment
/api/v1/discovered-certificates/{id}/dismiss	POST	Dismiss from triage
/api/v1/discovery-scans	GET	List scan history with timestamps
/api/v1/discovery-summary	GET	Aggregate status counts (Unmanaged, Managed, Dismissed)

Use Cases

• Inventory Baseline — Scan production servers at deployment time to establish baseline of existing certificates
• Compliance Discovery — Find all TLS certs before renewing certificate policies
• Migration Planning — Discover unmanaged certs to plan migration from other CA/platforms
• Audit Preparation — Triage discovered certs into managed and dismissed for compliance reports
• Multi-CA Migration — Find all certs currently issued by old CA, claim them for renewal under new issuer

---

Ownership & Accountability

Teams

• Model — Team grouping for organizational structure
• Team Assignment — Certificates and policies assigned to teams
• Email Distribution — Optional team email for notifications
• Resolver Logic — Team name → member lookup via API (external resolution)
• GUI — CRUD page with member management

Owners

• Model — Individual person responsible for certificates
• Email Routing — Owner email used for notification delivery
• Team Association — Owners belong to teams
• Certificate Assignment — Certificates assigned to owner (1:1 or group)
• Notification Routing — Expiration/renewal/revocation alerts sent to owner email
• GUI — CRUD page with team picker, email validation

Interactive Renewal Approval (M11b)

• AwaitingApproval Job State — Renewal jobs pause for human approval
• Approval Flow — POST /api/v1/jobs/{id}/approve (proceed with renewal)
• Rejection Flow — POST /api/v1/jobs/{id}/reject with reason text (cancel job)
• Reason Tracking — Approval/rejection reason logged to job history and audit
• Use Case — Change control, compliance gates, sensitive certificate renewal

---

Observability

Observability Layers

Dashboard Charts (M14)

Live aggregated views of certificate and job metrics.

Chart	Type	Details
Expiration Heatmap	Stacked bar	90-day weekly buckets; per-status color bands
Renewal Success Rate	Line (30-day)	Success % trending over time
Certificate Status Distribution	Donut	Pie breakdown: Active, Expiring, Expired, Failed, Revoked, etc.
Issuance Rate	Bar (30-day)	Certs issued per day; trend line

Metrics Endpoint

• URL — GET /api/v1/metrics
• Format — JSON with timestamp
• Gauges — Certificate counts by status, agent count (online/offline), pending job count
• Counters — Total jobs completed, total jobs failed, total renewals, total issuances
• Uptime — Server uptime in seconds

Stats API (M14)

Five parameterized endpoints for dashboard data.

Endpoint	Parameters	Response
GET /api/v1/stats/summary	None	Total certs, expiring soon, renewals in progress, failed jobs, agents online
GET /api/v1/stats/certificates-by-status	None	Count per status (Active, Expiring, Expired, etc.)
GET /api/v1/stats/expiration-timeline	days (default 90)	Weekly buckets with cert counts; 90-day default
GET /api/v1/stats/job-trends	days (default 30)	Daily completed/failed job counts; line chart ready
GET /api/v1/stats/issuance-rate	days (default 30)	Certs issued per day; 30-day default

Structured Logging (M14)

• Library — Go's log/slog (structured, context-aware)
• Request ID Propagation — Per-request UUID in context; logged on all operations
• Middleware — NewLogging(logger *slog.Logger) middleware wrapping all API calls
• Log Format — JSON (default) or text; configurable via CERTCTL_LOG_FORMAT
• Log Level — debug, info, warn, error; configurable via CERTCTL_LOG_LEVEL

API Audit Middleware (M19)

Every API call recorded to immutable audit_events table.

Logged Field	Details
Method	HTTP verb (GET, POST, PUT, DELETE)
Path	Request path (e.g., /api/v1/certificates)
Actor	Authenticated user/API key (or "anonymous")
Body Hash	SHA-256 of request body (truncated first 16 chars for brevity)
Response Status	HTTP status code
Latency	Request processing time in ms
Timestamp	RFC3339 format

Immutable Audit Trail

• Table — audit_events append-only (no UPDATE/DELETE)
• Events — Issuance, renewal, deployment, revocation, policy violations, approval/rejection
• Retention — Indefinite (no expiration)
• GUI Export — CSV/JSON export with applied time-range, actor, action filters
• Query API — GET /api/v1/audit?actor=...&resource=...&action=...&before=...&after=...

Deployment Rollback Support (M14)

• Version History — Sorted by deployment timestamp
• Current Badge — Visual indicator on latest deployed version
• Rollback Button — Click to re-deploy previous version
• Versioning — Each cert version tracked (serial, fingerprint, PEM)

---

Job System

Job Types (4 total)

Type	Trigger	States	Output
Issuance	New certificate creation	Pending → AwaitingCSR/Running → Completed/Failed	Certificate version with serial
Renewal	Auto-renewal or manual trigger	Pending → AwaitingCSR/AwaitingApproval/Running → Completed/Failed	New certificate version
Deployment	Automatic or manual post-renewal	Pending → AwaitingCSR/Running → Completed/Failed	Target-specific status
Validation	Scheduled or manual	Pending → Running → Completed/Failed	Validation report (TBD V3)

Job States (7 total)

State	Meaning	Transition
Pending	Created, awaiting processing	→ AwaitingCSR or Running
AwaitingCSR	Agent needs to generate key + submit CSR	→ Running (after CSR received)
AwaitingApproval	Human approval required (renewal only)	→ Running (approve) or Cancelled (reject)
Running	Active processing (issuance, deployment, etc.)	→ Completed or Failed
Completed	Successfully finished	(terminal)
Failed	Error during processing; no retry auto-scheduled	(terminal; manual retry available)
Cancelled	Explicitly cancelled by user or system	(terminal)

Job Lifecycle Example (Agent Keygen)

1. Renewal triggered → Job created in Pending state
2. Scheduler polls → Job transitioned to AwaitingCSR
3. Work endpoint → Agent receives job with common_name and SANs
4. Agent keygen → ECDSA P-256 key created locally; CSR submitted
5. CSR received → Server signs; Job transitioned to Running
6. Deployment scheduled → New Deployment job created in Pending
7. Agent deploys → Deployment job → Running → Completed
8. Status reported → POST /api/v1/agents/{id}/jobs/{job_id}/status

Approval Flow (Interactive)

1. Renewal job created in AwaitingApproval state (if policy requires)
2. Human reviews on GUI
3. Approve → POST /api/v1/jobs/{id}/approve → Job → Running
4. Reject → POST /api/v1/jobs/{id}/reject + reason → Job → Cancelled

Background Scheduler (5 loops)

Loop	Interval	Task
Renewal Checker	1 hour	Scan policies; trigger renewals if cert expires soon
Job Processor	30 seconds	Process Pending → AwaitingCSR/Running; poll agent status
Health Checker	2 minutes	Check agent heartbeat; mark offline if >3 missed
Notification Processor	1 minute	Send queued notifications (email, Slack, webhook, etc.)
Short-Lived Cleanup	30 seconds	Audit short-lived credential expirations

All loops have configurable intervals via environment variables (CERTCTL_SCHEDULER_*_INTERVAL).

---

Web Dashboard (19 Pages)

Overview

The web dashboard is the primary operational interface for certctl. Built with Vite + React 18 + TypeScript + TanStack Query v5 + Tailwind CSS 3 + Recharts.

Page	Route	Purpose
Dashboard	/	Overview: summary cards, 4 charts (expiration, renewal rate, status, issuance), quick actions
Certificates	/certificates	List with multi-select, bulk operations (renew/revoke/reassign), new cert modal, sorting/filtering
Certificate Detail	/certificates/:id	Full cert view: deployment timeline, inline policy editor, version history, rollback, revoke, archive, renew actions
Agents	/agents	List with metadata (OS, architecture, IP, version), online status, uptime
Agent Detail	/agents/:id	Full system information, recent jobs, heartbeat graph, capabilities, metrics
Agent Fleet Overview	/fleet	OS/architecture grouping with pie charts (status, version), per-platform agent listing
Jobs	/jobs	Queue view with type filter, status filter, inline cancel/approve/reject, retry button
Notifications	/notifications	Grouped by certificate, mark-as-read toggle, filter by type (expiration, deployment, revocation)
Policies	/policies	CRUD with rule builder, enable/disable toggle, violations summary bar, violation list
Profiles	/profiles	List with crypto constraints (key algorithms, TTL, EKUs), create/edit/delete
Issuers	/issuers	List, create new issuer, test connection button, delete
Targets	/targets	List, 3-step configuration wizard (Select Type → Configure → Review), type-specific fields
Owners	/owners	List, create/edit with team picker, email field, delete
Teams	/teams	List, create/edit with member resolver, delete
Agent Groups	/agent-groups	List with dynamic match criteria badges (OS, arch, IP CIDR, version), manual membership editor
Audit Trail	/audit	Filtered view (time range, actor, action), CSV/JSON export buttons, event detail modal
Short-Lived Credentials	/short-lived	Filtered by profile with TTL < 1 hour, live countdown timer, auto-refresh every 10s, stats bar
Login	/login	API key entry, auth mode detection, redirect after successful auth
ErrorBoundary	(all pages)	Graceful crash recovery; displays user-friendly error message instead of white screen

Dashboard Features

Bulk Operations

• Multi-Select — Checkbox column in certificate list; "Select All" toggle
• Bulk Renew — Trigger renewal on selected certs; progress bar
• Bulk Revoke — Select reason codes per cert; sequential revocation; progress
• Bulk Reassign — Owner picker modal; assign to multiple certs at once

Deployment Timeline

• Visual 4-Step Timeline — Requested → Issued → Deploying → Active
• Per-Certificate Job Queries — Query jobs to get current phase
• Status Indicators — Checkmarks for completed phases; spinner for running; X for failed

Inline Policy Editor

• Edit Mode — Click edit button on cert detail
• Policy Dropdown — Select from list of policies
• Renewal Threshold Config — Inline sliders/inputs for 30/14/7/0 day thresholds
• Save/Cancel — API mutations with optimistic updates via TanStack Query

Target Configuration Wizard

• Step 1: Select Type — Radio or dropdown (NGINX, Apache, HAProxy, F5, IIS)
• Step 2: Configure — Type-specific fields (cert path, chain path, key path, etc.)
• Step 3: Review — Summary of config; confirm create
• Validation — Real-time field validation; show errors; disable Create if invalid

Auth & Session

• Auth Context — React context with API key, auth mode, session state
• Auto-Redirect — 401 response → redirect to /login
• Logout — Button in sidebar; clears context; redirects to /login
• Remember API Key — Persisted in localStorage (production should clear on logout)

Demo Mode

• Activates when API is unreachable
• Renders realistic mock data for screenshots
• Useful for offline presentations

---

Integration Interfaces

MCP Server (M18a)

Separate binary (cmd/mcp-server/) providing AI-native access to certctl via Claude, Cursor, OpenClaw.

• Transport — stdio (stdin/stdout)
• Protocol — Model Context Protocol v1
• SDK — Official modelcontextprotocol/go-sdk v1.4.1
• Tools — 76 MCP tools covering all API endpoints
• Organization — 16 resource domains (Certificates, Issuers, Targets, Agents, Jobs, etc.)
• Authentication — Bearer token via CERTCTL_API_KEY env var
• Configuration — CERTCTL_SERVER_URL (e.g., http://localhost:8080) + CERTCTL_API_KEY
• Input Types — 33 typed structs with jsonschema tags for auto-generated LLM-friendly schemas
• Stateless Design — HTTP proxy (no state held in MCP server; all logic in REST API)

CLI Tool (certctl-cli, M16b)

Lightweight command-line wrapper around REST API.

Subcommand	Usage	Output Format
list-certs	certctl-cli list-certs [--filter]	Table or JSON (--format=json)
get-cert	certctl-cli get-cert <id>	JSON cert details
renew-cert	certctl-cli renew-cert <id>	Job ID confirmation
revoke-cert	certctl-cli revoke-cert <id> [--reason]	Revocation confirmation
list-agents	certctl-cli list-agents	Table or JSON
list-jobs	certctl-cli list-jobs [--filter]	Table or JSON
health	certctl-cli health	Server status
metrics	certctl-cli metrics	JSON metrics
import	certctl-cli import <pem-file>	Bulk import cert count
help	certctl-cli help [command]	Command documentation

Implementation Details:

• Stdlib-only (flag + text/tabwriter); no Cobra dependency
• JSON + table output formatters
• PEM parser for bulk import (multi-cert PEM files)
• Environment variables: CERTCTL_SERVER_URL, CERTCTL_API_KEY
• CLI flags: --server, --api-key, --format (json/table)
• Tested with httptest mock server; all commands covered

OpenAPI 3.1 Specification

• File — api/openapi.yaml
• Scope — 78 operations (76 API endpoints + /health + /ready)
• Schemas — Complete domain models with examples
• Enums — Job types, states, policy rule types, notification types
• Pagination — Standard envelope (data, total, page, per_page)
• Security — Bearer token security scheme
• SDK Generation — Supports go-swagger, openapi-generator, etc.

---

Security Architecture

Private Key Isolation

• Agent-Side Keygen (Default) — ECDSA P-256 keys generated on agents via Go's crypto/ecdsa
• Local Key Storage — Keys written to agent's CERTCTL_KEY_DIR (default /var/lib/certctl/keys) with 0600 permissions (user-readable only)
• Server-Side Keygen (Demo Only) — RSA-2048 keygen available via CERTCTL_KEYGEN_MODE=server with explicit log warning; never used in production
• CSR Submission Only — Agents submit CSRs (public) to control plane; private keys never leave agent infrastructure
• Key Rotation — Agents can re-key without control plane involvement (local only)

Pull-Only Deployment Model

• No Outbound Initiations — Server never initiates connections to agents or targets
• Agent Polling — Agents poll GET /api/v1/agents/{id}/work every 30 seconds
• Proxy Agent Pattern — For network appliances (F5, Palo Alto) or agentless targets (Windows servers), a "proxy agent" in the same network zone executes deployments via the target's API
• Credential Scope — Proxy agent credentials limited to its zone; control plane never stores target credentials directly
• Firewall-Friendly — Control plane can be completely locked down; no inbound rules needed for agents

Sub-CA Capability

• Enterprise Integration — Local CA can operate as subordinate CA under enterprise root (e.g., ADCS)
• Disk-Based Cert+Key — CERTCTL_CA_CERT_PATH + CERTCTL_CA_KEY_PATH load pre-signed CA cert and key
• Chain Validation — Issued certs chain to enterprise root; full trust hierarchy
• Self-Signed Fallback — Default mode generates self-signed root if paths not set (development/demo)
• Key Formats — RSA, ECDSA, PKCS#8 support with auto-detection

API Authentication

• SHA-256 Hashing — API keys hashed with SHA-256 before storage
• Constant-Time Comparison — Prevents timing attacks during key validation
• Bearer Token — Authorization: Bearer {api_key} header on all authenticated endpoints
• Configurable — CERTCTL_AUTH_TYPE=api-key (default) enforced; "none" requires explicit opt-in with log warning

Rate Limiting

• Token Bucket — Smooth rate limiting with burst capacity
• RPS + Burst — Configurable CERTCTL_RATE_LIMIT_RPS (default 50) and CERTCTL_RATE_LIMIT_BURST (default 100)
• 429 Responses — Rate limit exceeded responses include Retry-After header
• Per-Client — Implemented per IP (future: per API key)

Audit & Compliance

• Immutable Audit Trail — Append-only table; no UPDATE/DELETE operations
• API Audit Middleware — Every call logged with method, path, actor, body hash, status, latency
• Event Timestamps — RFC3339 format with second precision
• Actor Tracking — API key ID or username extracted from auth context
• Compliance Export — CSV/JSON export of audit events with filtering

---

Infrastructure

Deployment Architecture

• Server — Go HTTP server (net/http stdlib) on :8080 (default) or :8443 (Docker)
• Database — PostgreSQL 16 with 18+ tables, TEXT primary keys (human-readable prefixed IDs)
• Agent — Lightweight Go binary on target infrastructure
• Dashboard — React SPA served from /web/dist/ (Vite build)

Docker Compose Deployment

• Services — PostgreSQL 16, certctl server, agent
• Health Checks — On all services (server health check, database readiness)
• Seed Data — Demo dataset with 15 certs, 5 agents, 5 targets, policies, audit events
• Credentials — Environment variables in .env file; app.key for API key

PostgreSQL Schema

• 18+ Tables — Certificates, agents, jobs, targets, issuers, policies, profiles, teams, owners, agent groups, audit events, notifications, revocations, versions, etc.
• TEXT Primary Keys — Human-readable prefixed IDs: mc-, t-, a-, j-, p-*, etc.
• Indexes — 5+ performance indexes on foreign keys, timestamps, status fields
• Migrations — Idempotent migrations with IF NOT EXISTS, ON CONFLICT, numbered sequentially
• Max Connections — Configurable via CERTCTL_DATABASE_MAX_CONNS (default 25)

CI/CD Pipeline

• GitHub Actions — .github/workflows/ci.yml
• Parallel Jobs — Go (build, vet, test+coverage, gates) and Frontend (tsc, vitest, vite build)
• Coverage Gates — Service layer ≥30%, handler layer ≥50%
• Release Workflow — Tag push → build → publish Docker images to ghcr.io
• Docker Tags — :latest, :v{version} (ghcr.io/shankar0123/certctl)

Test Suite

• Unit Tests — 625+ test functions across service, handler, middleware, domain layers
• Integration Tests — End-to-end workflows (issuance→renewal→deployment)
• Negative Tests — Malformed input, nonexistent resources, error conditions
• Frontend Tests — 86 Vitest tests (API client, utilities, stats/metrics, full endpoint coverage)
• Total Coverage — 860+ tests (Go + frontend combined)

Licensing

• License — Business Source License 1.1 (BSL 1.1)
• Conversion — Automatic conversion to Apache 2.0 on March 23, 2033 (7-year term)
• Source-Available — Code available for inspection; copying/modification restricted until conversion

---

Configuration Reference

Environment Variables (All CERTCTL_ Prefixed)

Server

Variable	Type	Default	Purpose
CERTCTL_SERVER_HOST	string	127.0.0.1	Bind address
CERTCTL_SERVER_PORT	int	8080	Listen port

Database

Variable	Type	Default	Purpose
CERTCTL_DATABASE_URL	string	postgres://localhost/certctl	PostgreSQL connection string
CERTCTL_DATABASE_MAX_CONNS	int	25	Max connection pool size
CERTCTL_DATABASE_MIGRATIONS_PATH	string	./migrations	Migration file directory

Scheduler

Variable	Type	Default	Purpose
CERTCTL_SCHEDULER_RENEWAL_CHECK_INTERVAL	duration	1h	Renewal checker loop interval
CERTCTL_SCHEDULER_JOB_PROCESSOR_INTERVAL	duration	30s	Job processor loop interval
CERTCTL_SCHEDULER_AGENT_HEALTH_CHECK_INTERVAL	duration	2m	Agent health checker loop interval
CERTCTL_SCHEDULER_NOTIFICATION_PROCESS_INTERVAL	duration	1m	Notification processor loop interval

Logging

Variable	Type	Default	Purpose
CERTCTL_LOG_LEVEL	string	info	debug, info, warn, error
CERTCTL_LOG_FORMAT	string	json	json or text

Authentication

Variable	Type	Default	Purpose
CERTCTL_AUTH_TYPE	string	api-key	api-key, jwt, or none
CERTCTL_AUTH_SECRET	string	(required)	API key or JWT secret

Rate Limiting

Variable	Type	Default	Purpose
CERTCTL_RATE_LIMIT_ENABLED	bool	true	Enable/disable rate limiting
CERTCTL_RATE_LIMIT_RPS	float	50	Requests per second
CERTCTL_RATE_LIMIT_BURST	int	100	Max burst size

CORS

Variable	Type	Default	Purpose
CERTCTL_CORS_ORIGINS	string	(empty)	Comma-separated origins or * for all

Key Generation

Variable	Type	Default	Purpose
CERTCTL_KEYGEN_MODE	string	agent	agent or server

Local CA Sub-CA Mode

Variable	Type	Default	Purpose
CERTCTL_CA_CERT_PATH	string	(empty)	Path to PEM-encoded CA cert (sub-CA mode)
CERTCTL_CA_KEY_PATH	string	(empty)	Path to PEM-encoded CA key (sub-CA mode)

ACME Issuer

Variable	Type	Default	Purpose
CERTCTL_ACME_DIRECTORY_URL	string	(empty)	ACME server directory URL
CERTCTL_ACME_EMAIL	string	(empty)	Account email for ACME registration
CERTCTL_ACME_CHALLENGE_TYPE	string	http-01	http-01 or dns-01
CERTCTL_ACME_DNS_PRESENT_SCRIPT	string	(empty)	Script path for DNS-01 present hook
CERTCTL_ACME_DNS_CLEANUP_SCRIPT	string	(empty)	Script path for DNS-01 cleanup hook

step-ca Issuer

Variable	Type	Default	Purpose
CERTCTL_STEPCA_URL	string	(empty)	step-ca server URL
CERTCTL_STEPCA_PROVISIONER_NAME	string	(empty)	JWK provisioner name
CERTCTL_STEPCA_PROVISIONER_KEY_PATH	string	(empty)	Path to provisioner JWK private key
CERTCTL_STEPCA_PROVISIONER_PASSWORD	string	(empty)	Provisioner key password (if encrypted)

OpenSSL/Custom CA Issuer

Variable	Type	Default	Purpose
CERTCTL_OPENSSL_SIGN_SCRIPT	string	(empty)	Path to sign script (CSR → cert)
CERTCTL_OPENSSL_REVOKE_SCRIPT	string	(empty)	Path to revoke script (serial+reason)
CERTCTL_OPENSSL_CRL_SCRIPT	string	(empty)	Path to CRL generation script
CERTCTL_OPENSSL_TIMEOUT_SECONDS	int	30	Script timeout in seconds

Notifiers

Variable	Type	Default	Purpose
CERTCTL_SLACK_WEBHOOK_URL	string	(empty)	Slack incoming webhook URL
CERTCTL_SLACK_CHANNEL	string	(empty)	Slack channel override
CERTCTL_SLACK_USERNAME	string	certctl	Slack username override
CERTCTL_TEAMS_WEBHOOK_URL	string	(empty)	Microsoft Teams webhook URL
CERTCTL_PAGERDUTY_ROUTING_KEY	string	(empty)	PagerDuty Events API routing key
CERTCTL_PAGERDUTY_SEVERITY	string	warning	PagerDuty event severity
CERTCTL_OPSGENIE_API_KEY	string	(empty)	OpsGenie API key
CERTCTL_OPSGENIE_PRIORITY	string	P3	OpsGenie alert priority

Agent

Variable	Type	Default	Purpose
CERTCTL_AGENT_NAME	string	(generated)	Agent display name
CERTCTL_KEY_DIR	string	/var/lib/certctl/keys	Local private key storage directory
CERTCTL_AGENT_ID	string	(env or generated)	Agent unique ID (mc-xxx prefix)

MCP Server

Variable	Type	Default	Purpose
CERTCTL_SERVER_URL	string	http://localhost:8080	Base URL of certctl server
CERTCTL_API_KEY	string	(required)	API key for authentication

---

Feature Matrix: V2 Free vs. V3 Paid (Roadmap)

Feature	V2	V3 (Paid)	Status
Certificate lifecycle (create/renew/revoke)	✓	✓	Shipped v1.0+
4 issuer connectors (Local CA, ACME, step-ca, OpenSSL)	✓	✓	Shipped
3 target connectors (NGINX, Apache, HAProxy)	✓	✓	Shipped
6 notifier channels (Email, Webhook, Slack, Teams, PagerDuty, OpsGenie)	✓	✓	Shipped
Agent fleet + metadata	✓	✓	Shipped
Agent groups (dynamic + manual)	✓	✓	Shipped
Policies + violations	✓	✓	Shipped
Profiles + crypto constraints	✓	✓	Shipped
Revocation (RFC 5280, CRL, OCSP)	✓	✓	Shipped
Dashboard + 19 pages	✓	✓	Shipped
Observability (charts, metrics, stats)	✓	✓	Shipped
REST API (77 endpoints)	✓	✓	Shipped
MCP server (76 tools)	✓	✓	Shipped v2.1
CLI tool (10 subcommands)	✓	✓	Shipped
OIDC/SSO auth	✗	✓	Planned V3
RBAC (role-based access control)	✗	✓	Planned V3
F5 BIG-IP implementation	Stub	✓	Stub in V2
IIS implementation	Stub	✓	Stub in V2
NATS event bus	✗	✓	Planned V3
Real-time updates (SSE/WebSocket)	✗	✓	Planned V3
Advanced search DSL	✗	✓	Planned V3
Bulk operations	✓	✓	M13 (free)
Bulk revocation	✗	✓	Planned V3 (paid)
Certificate health scores	✗	✓	Planned V3
Compliance scoring	✗	✓	Planned V3
DigiCert issuer	✗	✓	Planned V3
CT Log monitoring	✗	✓	Planned V3

---

Summary Statistics

Category	Count
API Endpoints	84 (under /api/v1/)
Dashboard Pages	19
Issuer Connectors	4 (Local CA, ACME, step-ca, OpenSSL)
Target Connectors	5 (3 impl: NGINX, Apache, HAProxy; 2 stubs: F5, IIS)
Notifier Channels	6 (Email, Webhook, Slack, Teams, PagerDuty, OpsGenie)
Job Types	4 (Issuance, Renewal, Deployment, Validation)
Job States	7 (Pending, AwaitingCSR, AwaitingApproval, Running, Completed, Failed, Cancelled)
Policy Rule Types	5 (AllowedIssuers, AllowedDomains, RequiredMetadata, AllowedEnvironments, RenewalLeadTime)
Certificate States	8 (Pending, Active, Expiring, Expired, RenewalInProgress, Failed, Revoked, Archived)
Revocation Reason Codes	8 (RFC 5280 compliant)
Discovery Statuses	3 (Unmanaged, Managed, Dismissed)
MCP Tools	83 (17 resource domains)
CLI Subcommands	10
Database Tables	20+
Test Suite	881+ tests
Environment Variables	41+ configuration options