Per Phase 1 audit at cowork/docs-overhaul-phase-1-audit-2026-05-04/
and the section-by-section plan in testing-guide-tumor.md.
testing-guide.md was 30% of all docs/ content (8268 lines) but was
integration test code written in markdown, not operator documentation.
The audit's tumor analysis disposed of every Part:
- ~65% DELETE (test cases that already exist in code)
- ~22% MOVE to inline test code
- ~8% KEEP-COMPRESSED into focused operator-runbook docs
- Title + contents + release sign-off ~5% KEEP
This commit ships the KEEP-COMPRESSED dispersal:
docs/contributor/qa-prerequisites.md (NEW, ~120 lines):
From testing-guide.md "Prerequisites" section. Stack boot procedure,
demo data baseline, reference IDs operators reuse across QA docs.
docs/contributor/gui-qa-checklist.md (NEW, ~105 lines):
From testing-guide.md "Part 35: GUI Testing". Manual GUI verification
pass for release sign-off. 25-row table covering every dashboard page.
docs/contributor/release-sign-off.md (NEW, ~130 lines):
From testing-guide.md "Release Sign-Off" section (originally 1009
lines of per-test detail tables). Compressed to a release-day
checklist organized by gate category: code state, automated gates,
manual QA passes, release artefact verification, branch protection,
post-release.
docs/operator/performance-baselines.md (NEW, ~100 lines):
From testing-guide.md "Part 39: Performance Spot Checks". Four
operator-runnable benchmarks (API request handling, inventory list
pagination, scheduler tick, bulk revoke) with baseline numbers and
when-to-re-baseline guidance.
docs/operator/helm-deployment.md (NEW, ~120 lines):
From testing-guide.md "Part 52: Helm Chart Deployment". Operator
runbook for the bundled deploy/helm/certctl/ chart: prereqs,
install, four cert-source patterns, verify, upgrade, troubleshooting.
docs/reference/cli.md (NEW, ~120 lines):
From testing-guide.md "Part 28: CLI Tool". certctl-cli command
reference with command-group breakdown, common workflows
(list/filter, renew, revoke, bulk import, EST enrollment, status),
output formats, CI/CD integration patterns.
docs/README.md navigation index updated to include the 6 new docs:
Reference section gains: cli.md, release-verification.md (was added
in Phase 13)
Operator section gains: helm-deployment.md, performance-baselines.md
Contributor section gains: qa-prerequisites.md, gui-qa-checklist.md,
release-sign-off.md
docs/testing-guide.md deleted. Git history preserves the 8268 lines —
if any specific test case is found missing from inline test code or
the destination docs during future work, lift from `git show
HEAD~1:docs/testing-guide.md`.
Net: docs/ total line count drops by ~7700 lines (28%), from 26,369
to 18,742. testing-guide.md was the single largest doc; pruning it is
the single biggest content-edit win of the entire restructure.
Phase 5 is the last major content phase. Remaining: Phase 4 follow-on
(per-connector page extractions from reference/connectors/index.md),
Phase 15 (WHAT/HOW/WHY remediation), Phase 16 (final acceptance gate).
4.8 KiB
Helm Deployment
Last reviewed: 2026-05-05
Operator runbook for deploying certctl on Kubernetes via the bundled Helm chart at deploy/helm/certctl/.
Prereqs
- Kubernetes cluster, v1.27+
kubectlconfigured and authenticatedhelmv3.13+- Storage class for the PostgreSQL StatefulSet PVC
- TLS cert source: either an operator-supplied
kubernetes.io/tlsSecret OR a cert-managerClusterIssuer/Issuer. The chart refuses to render without one. Seetls.mdfor the four cert provisioning patterns.
Install
helm install certctl deploy/helm/certctl/ \
--namespace certctl \
--create-namespace \
--set server.apiKey=$(openssl rand -hex 32) \
--set postgres.password=$(openssl rand -hex 32) \
--set server.tls.existingSecret=certctl-server-tls
server.apiKey and postgres.password should be high-entropy values. The example above generates them inline; production deployments use a secrets manager (Vault, External Secrets Operator, AWS Secrets Manager) instead.
What you get
- Server Deployment with a configurable replica count (default 1; HA needs sticky sessions on the ACME server's nonce path)
- PostgreSQL StatefulSet with PVC-backed persistence
- Agent DaemonSet with one agent per node (configurable via
agent.daemonset.enabled=falseif you don't want the in-cluster agent) - Health probes (
/healthliveness +/readyreadiness) - Security contexts: non-root, read-only root filesystem
- Optional Ingress (off by default; opt in via
ingress.enabled=true)
Cert source patterns
Pattern 1 — operator-supplied Secret (recommended for non-cert-manager shops)
kubectl create secret tls certctl-server-tls \
--cert=server.crt --key=server.key \
--namespace certctl
helm install certctl deploy/helm/certctl/ \
--namespace certctl \
--set server.tls.existingSecret=certctl-server-tls
Pattern 2 — cert-manager Certificate CR (recommended for cert-manager shops)
helm install certctl deploy/helm/certctl/ \
--namespace certctl \
--set server.tls.certManager.enabled=true \
--set server.tls.certManager.issuerRef.name=my-cluster-issuer \
--set server.tls.certManager.issuerRef.kind=ClusterIssuer
Refuses to render without one of the above
helm install certctl deploy/helm/certctl/ --namespace certctl
# Error: server.tls.existingSecret OR server.tls.certManager.enabled must be set
The render-time guard catches the missing config at helm install time, not at pod-crash-loop time.
Verify the install
kubectl wait --for=condition=Ready --timeout=3m \
-n certctl pod -l app.kubernetes.io/name=certctl-server
kubectl port-forward -n certctl svc/certctl-server 8443:8443 &
# Bundle the TLS root from the Secret to verify
kubectl get secret -n certctl certctl-server-tls -o jsonpath='{.data.ca\.crt}' \
| base64 -d > /tmp/certctl-ca.crt
curl --cacert /tmp/certctl-ca.crt https://localhost:8443/health
# {"status":"healthy"}
If the Secret has no ca.crt key (operator-supplied Secrets often don't), use tls.crt as the bundle. For a self-signed cert the two files are identical; for a chained cert distribute the root CA bundle separately via ConfigMap.
Upgrade
helm upgrade certctl deploy/helm/certctl/ \
--namespace certctl \
--reuse-values
Postgres state survives the upgrade (the PVC is retained). The server / agent images bump per the chart's image.tag. See docs/archive/upgrades/ for version-specific upgrade guidance.
Configuration reference
Every value is documented at deploy/helm/certctl/values.yaml. Common tweaks:
server.replicaCount— replica count (default 1)server.resources.{requests,limits}— pod resource boundsagent.daemonset.enabled— toggle the in-cluster agent (default true)postgres.storageSize— PVC size (default 10Gi)ingress.enabled+ingress.host— opt into Ingress
Troubleshooting
Pod crash-loops with TLS error. Cert + key in the Secret don't pair. Verify with openssl x509 -modulus -in server.crt -noout | md5 against openssl rsa -modulus -in server.key -noout | md5 — outputs must match.
Agent DaemonSet pods can't reach the server. Service DNS / NetworkPolicy issue. Confirm the agent's CERTCTL_SERVER_URL env points at the in-cluster service name (https://certctl-server.certctl.svc.cluster.local:8443).
Postgres won't start. PVC permissions. Check kubectl describe pvc -n certctl certctl-postgres and confirm the storage class supports fsGroup.
Related docs
tls.md— cert provisioning patterns + SIGHUP rotationsecurity.md— production security posturerunbooks/disaster-recovery.md— Postgres restore + recovery proceduresdocs/archive/upgrades/— version-specific upgrade procedures