feat(pre-2.1.0): demo data overhaul, examples, migration guides, install script

Pre-2.1.0 adoption polish delivering all four milestones: A) Demo Data Overhaul — seed_demo.sql rewritten with 35 certs across 5 issuers, 8 agents, 8 targets, 50+ jobs spanning 90 days, 55+ audit events, discovery scans, network scan targets, S/MIME cert. B) Examples Directory — 5 turnkey docker-compose configs: acme-nginx, acme-wildcard-dns01, private-ca-traefik, step-ca-haproxy, multi-issuer. C) Migration Guides — migrate-from-certbot.md, migrate-from-acmesh.md, certctl-for-cert-manager-users.md. D) Agent Install Script — install-agent.sh with cross-platform support (Linux systemd + macOS launchd), release.yml updated for 6-target cross-compilation. Triple-audited against codebase: 22 factual corrections applied across docs, examples, and config (env var names, CLI flags, ports, DNS hook interface, scheduler loop counts, license conversion date). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-07 14:11:31 +00:00 · 2026-03-29 18:26:58 -04:00
parent 5f81de3219
commit bcf2c3ae92
29 changed files with 4508 additions and 214 deletions
@@ -0,0 +1,358 @@
+# Private CA + Traefik Example
+
+This example demonstrates certctl managing certificates for **internal services without public CA dependency**. Ideal for enterprise environments where:
+
+- All services are internal (VPN, private networks)
+- You need unified certificate lifecycle management across multiple internal apps
+- You want automatic cert deployment to your reverse proxy
+- You may have an existing enterprise root CA (ADCS, OpenCA, etc.)
+
+## What's Included
+
+- **certctl server** with Local CA issuer (self-signed or sub-CA mode)
+- **certctl agent** that deploys certificates to Traefik
+- **Traefik** reverse proxy with file provider for dynamic cert discovery
+- **PostgreSQL** database for certificate storage and audit trail
+- Automatic certificate discovery for existing certs in Traefik
+
+## Architecture
+
+```
+┌──────────────────┐
+│   certctl-server │  (Local CA issuer)
+│    (control      │
+│     plane)       │
+└────────┬─────────┘
+         │
+         │ REST API (job polling)
+         │
+┌────────▼──────────┐
+│  certctl-agent    │  (certificate deployer)
+└────────┬──────────┘
+         │
+         │ Write cert/key files
+         │
+┌────────▼──────────────────────┐
+│   Traefik                      │
+│ (watches cert directory)       │
+└────────────────────────────────┘
+         │
+         │ TLS handshakes
+         │
+   [Internal Services]
+```
+
+## Quick Start (Self-Signed CA)
+
+The simplest way to get running in 2 minutes:
+
+```bash
+# 1. Create directory structure
+mkdir -p traefik-config ca-certs
+
+# 2. Create a minimal Traefik dynamic config
+cat > traefik-config/default.yaml << 'EOF'
+# Traefik will auto-load certificates from /etc/traefik/certs
+# Certctl deploys {cert-id}.crt and {cert-id}.key files here
+http:
+  routers:
+    api:
+      rule: "Host(`api.internal.local`)"
+      service: api-service
+      tls: {}
+  services:
+    api-service:
+      loadBalancer:
+        servers:
+          - url: "http://localhost:3000"
+EOF
+
+# 3. Start the stack
+docker compose up -d
+
+# 4. Access the dashboards
+# - certctl: http://localhost:8443 (API only, use the CLI or direct HTTP calls)
+# - Traefik dashboard: http://localhost:8080
+```
+
+The self-signed CA will be automatically generated on first startup.
+
+## Using Sub-CA Mode (Enterprise Root CA)
+
+If you have an existing enterprise CA (ADCS, OpenCA, etc.) and want issued certs to chain to your root:
+
+```bash
+# 1. Create directory structure
+mkdir -p traefik-config ca-certs
+
+# 2. Copy your enterprise CA cert and key
+cp /path/to/your/enterprise-ca.crt ca-certs/ca-cert.pem
+cp /path/to/your/enterprise-ca-key.pem ca-certs/ca-key.pem
+
+# 3. Edit docker-compose.yml and uncomment the sub-CA env vars:
+#    CERTCTL_CA_CERT_PATH: /etc/certctl/ca-cert.pem
+#    CERTCTL_CA_KEY_PATH: /etc/certctl/ca-key.pem
+
+# 4. Create the dynamic config (same as above)
+mkdir -p traefik-config
+cat > traefik-config/default.yaml << 'EOF'
+http:
+  routers:
+    api:
+      rule: "Host(`api.internal.local`)"
+      service: api-service
+      tls: {}
+  services:
+    api-service:
+      loadBalancer:
+        servers:
+          - url: "http://localhost:3000"
+EOF
+
+# 5. Start the stack
+docker compose up -d
+```
+
+**Requirements for sub-CA mode:**
+- CA certificate must have `X509v3 Basic Constraints: CA:TRUE`
+- CA certificate must have `X509v3 Key Usage: Certificate Sign`
+- Key format: RSA, ECDSA, or PKCS#8
+- Paths: must be absolute paths to mounted files
+
+## Creating a Certificate
+
+Once the stack is running:
+
+```bash
+# 1. Create a certificate profile in certctl (defines allowed key types, TTL, etc.)
+curl -X POST http://localhost:8443/api/v1/profiles \
+  -H "Content-Type: application/json" \
+  -d '{
+    "id": "prof-internal",
+    "name": "Internal Services",
+    "description": "For internal APIs and web apps",
+    "max_ttl_hours": 8760,
+    "key_types": ["rsa-2048", "ecdsa-p256"]
+  }'
+
+# 2. Create a renewal policy (defines issuer, renewal thresholds, etc.)
+curl -X POST http://localhost:8443/api/v1/policies \
+  -H "Content-Type: application/json" \
+  -d '{
+    "id": "pol-internal",
+    "name": "Internal Renewal Policy",
+    "issuer_id": "iss-local",
+    "profile_id": "prof-internal",
+    "renewal_threshold_days": 30,
+    "alert_thresholds_days": [30, 14, 7, 0]
+  }'
+
+# 3. Create a certificate (triggers issuance immediately)
+curl -X POST http://localhost:8443/api/v1/certificates \
+  -H "Content-Type: application/json" \
+  -d '{
+    "common_name": "api.internal.local",
+    "sans": ["app.internal.local", "www.internal.local"],
+    "policy_id": "pol-internal"
+  }'
+
+# 4. Create a Traefik target (agent will deploy to this)
+curl -X POST http://localhost:8443/api/v1/targets \
+  -H "Content-Type: application/json" \
+  -d '{
+    "id": "target-traefik-01",
+    "name": "Traefik Primary",
+    "type": "traefik",
+    "config": {
+      "cert_dir": "/etc/traefik/certs"
+    }
+  }'
+
+# 5. Create a deployment job (agent picks this up and deploys)
+curl -X POST http://localhost:8443/api/v1/certificates/{cert-id}/deploy \
+  -H "Content-Type: application/json" \
+  -d '{
+    "target_ids": ["target-traefik-01"]
+  }'
+```
+
+Once deployed, Traefik automatically loads the new certificate from the certs directory.
+
+## How It Works
+
+### Certificate Lifecycle
+
+1. **Issue** — certctl-server generates certificate from Local CA (self-signed or sub-CA)
+2. **Store** — certificate stored in PostgreSQL with full audit trail
+3. **Deploy** — certctl-agent writes `{cert-id}.crt` + `{cert-id}.key` to `/etc/traefik/certs`
+4. **Reload** — Traefik file provider detects new files and hot-loads them (zero downtime)
+5. **Monitor** — certctl tracks deployment status and renewal timelines
+
+### Self-Signed CA
+
+- Generated automatically on first startup if `CERTCTL_CA_CERT_PATH` and `CERTCTL_CA_KEY_PATH` are not set
+- Certificate stored in server's in-memory state (not persisted)
+- All issued certs chain to this self-signed root
+- Use this for: demos, development, internal labs
+
+### Sub-CA Mode
+
+- Requires you to provide an existing CA certificate and key
+- Issued certificates chain to your enterprise root CA
+- All issued certs are trustworthy to systems with your root CA in their trust store
+- Use this for: production internal services, compliance requirements, enterprise PKI
+
+## File Organization
+
+```
+private-ca-traefik/
+├── docker-compose.yml          # Stack definition
+├── traefik-config/             # Traefik dynamic config (you create)
+│   └── default.yaml            # Routing rules and TLS settings
+├── ca-certs/                   # CA certificate and key (for sub-CA mode)
+│   ├── ca-cert.pem            # Your enterprise CA certificate
+│   └── ca-key.pem             # Your enterprise CA private key
+└── README.md                   # This file
+```
+
+## Monitoring
+
+### certctl Dashboard
+The server provides a REST API on port 8443. Example queries:
+
+```bash
+# List all certificates
+curl http://localhost:8443/api/v1/certificates
+
+# Check certificate status
+curl http://localhost:8443/api/v1/certificates/{cert-id}
+
+# View audit trail
+curl http://localhost:8443/api/v1/audit
+
+# Check renewal policy compliance
+curl http://localhost:8443/api/v1/policies/{policy-id}
+```
+
+### Traefik Dashboard
+http://localhost:8080 shows:
+- HTTP routers and services
+- TLS certificates currently loaded
+- Request/response metrics
+
+### Logs
+```bash
+# certctl server logs
+docker compose logs certctl-server
+
+# certctl agent logs
+docker compose logs certctl-agent
+
+# Traefik logs
+docker compose logs traefik
+```
+
+## Customizing Traefik Config
+
+Edit `traefik-config/default.yaml` to add routers for your services:
+
+```yaml
+http:
+  routers:
+    # Internal API
+    api:
+      rule: "Host(`api.internal.local`)"
+      service: api-service
+      tls: {}
+
+    # Web application
+    webapp:
+      rule: "Host(`app.internal.local`)"
+      service: webapp-service
+      tls: {}
+
+  services:
+    api-service:
+      loadBalancer:
+        servers:
+          - url: "http://api-backend:3000"
+
+    webapp-service:
+      loadBalancer:
+        servers:
+          - url: "http://webapp-backend:3001"
+```
+
+Changes are picked up automatically (file watcher enabled).
+
+## Production Considerations
+
+1. **Use sub-CA mode** — chain to your enterprise root for full trust
+2. **Enable API key authentication** — set `CERTCTL_AUTH_TYPE: api-key` and `CERTCTL_API_KEY`
+3. **Use agent-side key generation** — set `CERTCTL_KEYGEN_MODE: agent` (keys never leave agents)
+4. **Back up PostgreSQL** — certificate data is authoritative; database loss means certificate loss
+5. **Monitor renewal windows** — set up alerts on policy thresholds
+6. **Rotate CA keys regularly** — plan for future CA refresh (sub-CA mode)
+7. **Audit certificate usage** — review `certctl_audit_events` for compliance
+
+## Troubleshooting
+
+### Certificates not deploying
+```bash
+# Check agent is healthy
+docker compose logs certctl-agent | grep heartbeat
+
+# Check deployment job status
+curl http://localhost:8443/api/v1/jobs | jq '.[] | select(.type == "Deployment")'
+
+# Check Traefik is watching the directory
+docker compose exec traefik ls -la /etc/traefik/certs/
+```
+
+### Traefik not reloading certs
+```bash
+# Verify file provider is enabled (check docker-compose.yml command)
+# Verify certs volume is mounted at /etc/traefik/certs
+# Check Traefik logs
+docker compose logs traefik | grep "file"
+```
+
+### CA cert not loading in sub-CA mode
+```bash
+# Verify file permissions
+docker compose exec certctl-server ls -la /etc/certctl/
+
+# Check server logs for CA loading errors
+docker compose logs certctl-server | grep -i "ca\|cert"
+
+# Verify CA certificate format
+openssl x509 -in ca-certs/ca-cert.pem -text -noout | grep -A 3 "Basic Constraints"
+```
+
+## Cleanup
+
+```bash
+# Stop all services
+docker compose down
+
+# Remove all data (certificates, database, etc.)
+docker compose down -v
+
+# Remove CA cert files (if using custom CA)
+rm -rf ca-certs/
+```
+
+## Next Steps
+
+1. **Add more services** — create additional routers and backends in `traefik-config/default.yaml`
+2. **Set up renewal automation** — configure renewal policies with thresholds
+3. **Integrate with monitoring** — expose certctl metrics to Prometheus
+4. **Enable notifications** — configure email/Slack alerts on certificate events
+5. **Scale to multiple environments** — deploy separate certctl stacks per environment (dev/staging/prod)
+
+## Related Documentation
+
+- [certctl Architecture](../../docs/architecture.md)
+- [Traefik File Provider](https://doc.traefik.io/traefik/providers/file/)
+- [Local CA Sub-CA Mode](../../docs/connectors.md#local-ca)
+- [Certificate Profiles](../../docs/quickstart.md#profiles)
@@ -0,0 +1,182 @@
+version: '3.8'
+
+services:
+  # PostgreSQL database for certctl
+  postgres:
+    image: postgres:16-alpine
+    container_name: certctl-postgres-private-ca
+    environment:
+      POSTGRES_DB: certctl
+      POSTGRES_USER: certctl
+      POSTGRES_PASSWORD: ${DB_PASSWORD:-certctl-dev-password}
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+    healthcheck:
+      test: ['CMD-SHELL', 'pg_isready -U certctl -d certctl']
+      interval: 5s
+      timeout: 5s
+      retries: 5
+    networks:
+      - certctl-network
+    restart: unless-stopped
+
+  # certctl server (control plane) with Local CA in sub-CA mode
+  certctl-server:
+    image: ghcr.io/shankar0123/certctl-server:latest
+    container_name: certctl-server-private-ca
+    environment:
+      # Database
+      DATABASE_URL: postgres://certctl:${DB_PASSWORD:-certctl-dev-password}@postgres:5432/certctl?sslmode=disable
+
+      # Server settings
+      CERTCTL_SERVER_PORT: 8443
+      CERTCTL_SERVER_HOST: 0.0.0.0
+
+      # Auth (disabled for demo; production should use API keys)
+      CERTCTL_AUTH_TYPE: none
+
+      # CORS (allow agent and Traefik communication)
+      CERTCTL_CORS_ORIGINS: '*'
+
+      # Key generation mode (agent-side in production, server-side for demo)
+      CERTCTL_KEYGEN_MODE: server
+
+      # Local CA configuration
+      # For self-signed CA (default, no paths set):
+      #   - CA generates a self-signed root certificate
+      #   - All issued certificates chain to this root
+      #
+      # For sub-CA mode (provide both paths):
+      #   - Load pre-signed CA certificate and key from these paths
+      #   - All issued certificates chain to your enterprise root CA
+      #   - Requires: CA cert must have IsCA=true and KeyUsageCertSign
+      #   - Supports: RSA, ECDSA, PKCS#8 key formats
+      #
+      # To use sub-CA mode:
+      #   1. Place your enterprise CA cert at ./ca-cert.pem
+      #   2. Place your enterprise CA key at ./ca-key.pem
+      #   3. Uncomment the two lines below
+      #   4. Restart the service
+      #
+      # CERTCTL_CA_CERT_PATH: /etc/certctl/ca-cert.pem
+      # CERTCTL_CA_KEY_PATH: /etc/certctl/ca-key.pem
+
+      # Logging
+      CERTCTL_LOG_LEVEL: info
+    ports:
+      - '${SERVER_PORT:-8443}:8443'
+    volumes:
+      # Mount directory for CA cert/key (for sub-CA mode)
+      # Copy your enterprise CA cert+key here:
+      #   cp /path/to/your/ca.pem ./ca-cert.pem
+      #   cp /path/to/your/ca-key.pem ./ca-key.pem
+      - ./ca-certs:/etc/certctl:ro
+    depends_on:
+      postgres:
+        condition: service_healthy
+    networks:
+      - certctl-network
+    healthcheck:
+      test: ['CMD-SHELL', 'curl -sf http://localhost:8443/api/v1/health || exit 1']
+      interval: 10s
+      timeout: 5s
+      retries: 3
+    restart: unless-stopped
+
+  # certctl agent (deploys certs to Traefik)
+  certctl-agent:
+    image: ghcr.io/shankar0123/certctl-agent:latest
+    container_name: certctl-agent-private-ca
+    environment:
+      # Control plane connection
+      CERTCTL_SERVER_URL: http://certctl-server:8443
+      CERTCTL_API_KEY: ${AGENT_API_KEY:-agent-demo-key}
+
+      # Key generation (agent-side keys, never sent to server)
+      CERTCTL_KEYGEN_MODE: server
+      CERTCTL_KEY_DIR: /var/lib/certctl/keys
+
+      # Discovery (scan for existing certs in Traefik's directory)
+      CERTCTL_DISCOVERY_DIRS: /etc/traefik/certs
+
+      # Heartbeat interval
+      CERTCTL_HEARTBEAT_INTERVAL: 30s
+
+      # Agent metadata (self-reported)
+      CERTCTL_AGENT_NAME: traefik-agent-01
+
+      # Logging
+      CERTCTL_LOG_LEVEL: info
+    volumes:
+      # Mount Traefik cert directory for deployment
+      - traefik_certs:/etc/traefik/certs
+      # Agent key storage (persisted across restarts)
+      - agent_keys:/var/lib/certctl/keys
+    depends_on:
+      certctl-server:
+        condition: service_healthy
+    networks:
+      - certctl-network
+    restart: unless-stopped
+
+  # Traefik reverse proxy / edge router
+  # Certificates deployed by certctl-agent are automatically loaded from the certs directory
+  traefik:
+    image: traefik:v3.0
+    container_name: certctl-traefik-private-ca
+    command:
+      # Enable dashboard and API
+      - '--api.insecure=true'
+      - '--api.dashboard=true'
+
+      # File provider: watch the certs directory for dynamic config updates
+      - '--providers.file.directory=/etc/traefik/dynamic'
+      - '--providers.file.watch=true'
+
+      # Entry points (HTTP and HTTPS)
+      - '--entrypoints.web.address=:80'
+      - '--entrypoints.websecure.address=:443'
+      - '--entrypoints.websecure.http.tls=true'
+
+      # Global TLS settings
+      - '--entryPoints.websecure.http.tls.certResolver=internal'
+
+      # Logging
+      - '--log.level=info'
+      - '--accesslog=true'
+    ports:
+      # HTTP
+      - '80:80'
+      # HTTPS
+      - '443:443'
+      # Dashboard (http://localhost:8080)
+      - '8080:8080'
+    volumes:
+      # Mount Traefik config directory
+      - ./traefik-config:/etc/traefik/dynamic:ro
+      # Mount cert directory (where certctl deploys certs)
+      - traefik_certs:/etc/traefik/certs:ro
+      # Allow Traefik to read Docker socket (optional, for container labeling)
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+    networks:
+      - certctl-network
+    depends_on:
+      - certctl-agent
+    healthcheck:
+      test: ['CMD-SHELL', 'curl -sf http://localhost:8080/ping || exit 1']
+      interval: 10s
+      timeout: 5s
+      retries: 3
+    restart: unless-stopped
+
+networks:
+  certctl-network:
+    driver: bridge
+
+volumes:
+  postgres_data:
+    driver: local
+  traefik_certs:
+    driver: local
+  agent_keys:
+    driver: local