mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 15:01:32 +00:00
Update all documentation to reflect M1–M9 completion
Align docs with actual codebase state post-M8 (agent-side keygen) and M9 (test hardening). Key changes: - README: V1 roadmap reflects all milestones complete, correct coverage thresholds (30%/50%), lists only remaining v1.0.0 tag items - architecture.md: ACME marked as fully implemented, security diagram corrected to ECDSA P-256, testing strategy rewritten with accurate counts (205 tests), target connector docs expanded with KeyPEM - connectors.md: DeploymentRequest struct updated with KeyPEM field, NGINX/F5/IIS sections expanded with config examples and flow details - demo-advanced.md: keygen mode notes updated for agent-side default, DeploymentRequest explanation corrected - CLAUDE.md: M9 deferred items clarified, connector test path fixed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+15
-10
@@ -399,11 +399,11 @@ type Connector interface {
|
||||
}
|
||||
```
|
||||
|
||||
Built-in issuers: **Local CA** (self-signed, for development/demos) and **ACME** (Let's Encrypt, Sectigo, etc., in progress).
|
||||
Built-in issuers: **Local CA** (self-signed, in-memory CA for development/demos using `crypto/x509`) and **ACME v2** (fully implemented with HTTP-01 challenge solving, compatible with Let's Encrypt, Sectigo, and any ACME-compliant CA). The ACME connector uses `golang.org/x/crypto/acme`, generates an ECDSA P-256 account key, handles account registration with ToS acceptance, order creation, HTTP-01 challenge solving via a built-in temporary HTTP server, order finalization, and DER-to-PEM chain conversion. Configure via `CERTCTL_ACME_DIRECTORY_URL` and `CERTCTL_ACME_EMAIL`.
|
||||
|
||||
### Target Connector
|
||||
|
||||
Deploys certificates to infrastructure. Note: the interface does NOT include private keys — agents handle keys locally.
|
||||
Deploys certificates to infrastructure. The `DeploymentRequest` includes `KeyPEM` because agents generate and hold private keys locally — the key is passed from the agent's local key store into the target connector, never from the control plane.
|
||||
|
||||
```go
|
||||
type Connector interface {
|
||||
@@ -413,7 +413,9 @@ type Connector interface {
|
||||
}
|
||||
```
|
||||
|
||||
Built-in targets: **NGINX**, **F5 BIG-IP**, **IIS**.
|
||||
The `DeploymentRequest` struct carries the full material needed by the target system: the signed certificate, the CA chain, the agent-generated private key, target-specific configuration, and arbitrary metadata. The key field is populated by the agent from its local key store (`CERTCTL_KEY_DIR`) — it never originates from the control plane.
|
||||
|
||||
Built-in targets: **NGINX** (writes cert/chain/key files, validates with `nginx -t`, reloads), **F5 BIG-IP** (REST API upload + virtual server binding), **IIS** (WinRM PFX import + site binding).
|
||||
|
||||
### Notifier Connector
|
||||
|
||||
@@ -438,7 +440,7 @@ See the [Connector Development Guide](connectors.md) for details on building cus
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph "Agent (Your Infrastructure)"
|
||||
GEN["1. GENERATE\ncrypto/rsa 2048-bit"]
|
||||
GEN["1. GENERATE\ncrypto/ecdsa P-256"]
|
||||
STORE["2. STORE\nFile perms 0600"]
|
||||
USE["3. USE\nCSR gen + deployment"]
|
||||
ROT["4. ROTATE\nDelete old after renewal"]
|
||||
@@ -558,14 +560,17 @@ For production, you would also add an ingress controller, TLS termination for th
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
certctl uses a layered testing approach aligned with the handler → service → repository architecture:
|
||||
certctl uses a layered testing approach aligned with the handler → service → repository architecture, with 170+ tests across four layers. The goal is high-confidence regression prevention at the service and handler layers, where the most complex business logic lives, combined with integration tests that exercise the full request path from HTTP to database.
|
||||
|
||||
- **Service layer unit tests** (`internal/service/*_test.go`) — 74 test functions across 7 files with mock repositories. Tests all business logic: certificate CRUD, agent lifecycle, job state machine, policy evaluation, renewal/issuance flow, notification deduplication.
|
||||
- **Handler layer tests** (`internal/api/handler/*_test.go`) — 50 test functions using `httptest`. Currently covers certificates and agents; M9 expands to all 7 handler files.
|
||||
- **Integration tests** (`internal/integration/lifecycle_test.go`) — 11 subtests covering the full lifecycle from certificate creation through issuance, deployment, and status reporting. M9 adds negative-path scenarios (issuer failure, malformed CSR, DB timeout).
|
||||
- **CI pipeline** (`.github/workflows/ci.yml`) — Parallel Go (build, vet, test with coverage) and Frontend (TypeScript check, Vite build) jobs. M9 adds coverage threshold enforcement.
|
||||
**Service layer unit tests** (`internal/service/*_test.go`) — 74 test functions across 7 files with mock repositories. These test all business logic in isolation: certificate CRUD with validation, agent lifecycle (registration, heartbeat, CSR submission with both keygen modes), job state machine (creation, processing, cancellation, retry logic), policy evaluation (all 4 rule types, violation creation), renewal and issuance flow (server-side and agent-side keygen paths), and notification deduplication (threshold tag matching, channel routing). Mock repositories are simple structs with function fields, avoiding heavy mocking frameworks — this keeps tests readable and avoids coupling to mock library APIs.
|
||||
|
||||
Remaining gaps before v1.0 (M9): handler tests for jobs/notifications/policies/issuers/targets, negative-path integration tests, scheduler loop tests, connector error handling tests, and CI coverage gates.
|
||||
**Handler layer tests** (`internal/api/handler/*_test.go`) — 119 test functions across 7 files using Go's `httptest` package. Every handler file has a corresponding test file: certificates (20 tests), agents (20 tests), jobs (14 tests), notifications (11 tests), policies (15 tests), issuers (15 tests), and targets (14 tests). Each test file follows the same pattern: a mock service struct with function fields, `httptest.NewRecorder` for capturing responses, and a shared `contextWithRequestID()` helper. Tests cover the happy path, input validation (missing fields, invalid JSON, empty IDs), error propagation from the service layer, method-not-allowed responses, and pagination parameters.
|
||||
|
||||
**Integration tests** (`internal/integration/`) — Two test files exercising the full stack from HTTP request through router, handler, service, and postgres repository layers. `lifecycle_test.go` has 11 subtests covering the complete certificate lifecycle: team/owner creation, certificate creation, issuer verification, renewal trigger, job verification, agent registration, CSR submission, deployment, and status reporting. `negative_test.go` has 12 subtests covering error paths: nonexistent resource lookups (404s), invalid request bodies (malformed JSON, missing required fields), invalid CSR submission, heartbeat for nonexistent agents, wrong HTTP methods on list endpoints, empty list responses, renewal on nonexistent certificates, and expired certificate lifecycle. Both use a shared `setupTestServer()` that builds a fully-wired server with real postgres repositories and the Local CA issuer connector.
|
||||
|
||||
**CI pipeline** (`.github/workflows/ci.yml`) — Two parallel jobs: Go (build, vet, test with `-race` and coverage, coverage threshold enforcement) and Frontend (TypeScript type check, Vite production build). The Go job runs all tests with `-coverprofile`, then enforces coverage thresholds: service layer must be at least 30% (current: ~34%) and handler layer must be at least 50% (current: ~61%). These thresholds act as regression floors — they can only go up. The service layer threshold is deliberately lower because much of the service code depends on postgres repositories and external connectors that require real infrastructure to test meaningfully. Connector tests are included via `./internal/connector/issuer/local/...` (the Local CA package, which has unit tests for certificate signing logic).
|
||||
|
||||
**What's not tested and why:** Postgres repository implementations (`internal/repository/postgres/`) require a real database and are tested only through integration tests, not unit tests. Target connectors (NGINX, F5, IIS) depend on real infrastructure or complex mocks. Scheduler loops are time-dependent and tested manually during development. The ACME connector requires a real ACME server (tested manually against Let's Encrypt staging). These are all candidates for future expansion as the test infrastructure matures.
|
||||
|
||||
## What's Next
|
||||
|
||||
|
||||
+46
-9
@@ -209,11 +209,16 @@ type Connector interface {
|
||||
}
|
||||
|
||||
type DeploymentRequest struct {
|
||||
CertPEM string // Signed certificate (PEM)
|
||||
ChainPEM string // CA chain (PEM)
|
||||
TargetConfig json.RawMessage // Target-specific config
|
||||
Metadata map[string]string
|
||||
// NOTE: No private key — agents handle keys locally
|
||||
CertPEM string // Signed certificate (PEM), from control plane
|
||||
ChainPEM string // CA chain (PEM), from control plane
|
||||
KeyPEM string // Private key (PEM), from agent's local key store
|
||||
TargetConfig json.RawMessage // Target-specific config (NGINX paths, F5 API, IIS site)
|
||||
Metadata map[string]string // Arbitrary context (cert ID, environment, etc.)
|
||||
// NOTE: KeyPEM is populated by the agent from its local key store
|
||||
// (CERTCTL_KEY_DIR). It is NEVER sent from the control plane.
|
||||
// The control plane only provides CertPEM and ChainPEM (public material).
|
||||
// The agent combines the locally-generated private key with the signed
|
||||
// certificate to create the full deployment payload.
|
||||
}
|
||||
|
||||
type DeploymentResult struct {
|
||||
@@ -244,31 +249,62 @@ type ValidationResult struct {
|
||||
|
||||
### Built-in: NGINX
|
||||
|
||||
The NGINX connector writes certificate and chain files to disk, validates the NGINX configuration, and reloads the server.
|
||||
The NGINX connector writes certificate, chain, and key files to disk, validates the NGINX configuration, and reloads the server. This is a common deployment pattern for teams running NGINX as a reverse proxy or TLS termination point.
|
||||
|
||||
Configuration:
|
||||
```json
|
||||
{
|
||||
"cert_path": "/etc/nginx/certs/cert.pem",
|
||||
"chain_path": "/etc/nginx/certs/chain.pem",
|
||||
"key_path": "/etc/nginx/certs/key.pem",
|
||||
"reload_command": "systemctl reload nginx",
|
||||
"validate_command": "nginx -t"
|
||||
}
|
||||
```
|
||||
|
||||
The connector writes cert and chain files with mode 0644, runs the validation command first (so a bad config doesn't take down NGINX), and only reloads if validation passes.
|
||||
The deployment flow is designed to be safe and atomic where possible: the connector writes cert and chain files with mode 0644 and the key file with mode 0600 (read-only by owner), runs the validation command first (so a bad config doesn't take down NGINX), and only reloads if validation passes. If the validation command fails, the connector rolls back the file writes and returns an error with the validation output — this prevents a partial deployment from breaking a running NGINX instance.
|
||||
|
||||
The `reload_command` defaults to `systemctl reload nginx` but can be overridden for custom setups (e.g., `nginx -s reload` for non-systemd environments, or `docker exec nginx nginx -s reload` for containerized NGINX).
|
||||
|
||||
Location: `internal/connector/target/nginx/nginx.go`
|
||||
|
||||
### Built-in: F5 BIG-IP
|
||||
|
||||
Deploys certificates via the F5 REST API. Uploads the certificate and key, then updates virtual server SSL profiles.
|
||||
Deploys certificates to F5 BIG-IP load balancers via the iControl REST API. This is the standard integration path for organizations using F5 for TLS offloading. The connector uploads the certificate and private key to the F5 SSL certificate store, then updates the SSL profile on the virtual server to reference the new certificate.
|
||||
|
||||
Configuration:
|
||||
```json
|
||||
{
|
||||
"host": "f5.internal.example.com",
|
||||
"username": "admin",
|
||||
"password": "...",
|
||||
"partition": "Common",
|
||||
"virtual_server": "/Common/vs_api",
|
||||
"ssl_profile": "/Common/clientssl_api"
|
||||
}
|
||||
```
|
||||
|
||||
The connector authenticates to the F5 REST API at `https://{host}/mgmt/tm/`, uploads the certificate via `POST /mgmt/tm/sys/crypto/cert`, uploads the key via `POST /mgmt/tm/sys/crypto/key`, and binds them to the specified SSL profile. The F5's native REST API handles certificate chain assembly. Agent credentials for the F5 API are stored locally on the agent, never on the control plane.
|
||||
|
||||
Location: `internal/connector/target/f5/f5.go`
|
||||
|
||||
### Built-in: IIS
|
||||
|
||||
Deploys certificates to Microsoft IIS via WinRM. Imports the certificate into the Windows certificate store and binds it to an IIS site.
|
||||
Deploys certificates to Microsoft IIS web servers via WinRM (Windows Remote Management). This connector is for organizations running Windows-based infrastructure where IIS terminates TLS. The connector executes PowerShell commands over WinRM to import a PFX certificate into the Windows certificate store and bind it to an IIS site.
|
||||
|
||||
Configuration:
|
||||
```json
|
||||
{
|
||||
"host": "iis-server.internal.example.com",
|
||||
"username": "Administrator",
|
||||
"password": "...",
|
||||
"site_name": "Default Web Site",
|
||||
"cert_store": "WebHosting",
|
||||
"use_https": true
|
||||
}
|
||||
```
|
||||
|
||||
The deployment flow: the connector combines the certificate and private key into a PFX (PKCS#12) bundle, transfers it to the Windows server via WinRM, runs `Import-PfxCertificate` to install it into the specified certificate store (typically `WebHosting` or `My`), then runs `Set-WebBinding` to bind the new certificate to the IIS site. Old certificate bindings are updated in-place so there is no downtime window.
|
||||
|
||||
Location: `internal/connector/target/iis/iis.go`
|
||||
|
||||
@@ -371,6 +407,7 @@ func TestNginxDeploy(t *testing.T) {
|
||||
result, err := connector.DeployCertificate(ctx, target.DeploymentRequest{
|
||||
CertPEM: testCertPEM,
|
||||
ChainPEM: testChainPEM,
|
||||
KeyPEM: testKeyPEM,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("deploy failed: %v", err)
|
||||
|
||||
@@ -106,7 +106,7 @@ You should see:
|
||||
|
||||
**How it works:** The issuer record was inserted during database seeding (`migrations/seed_demo.sql`). The `type` field (`GenericCA`) maps to a connector implementation. When the server starts, it registers connector instances in an `issuerRegistry` map keyed by issuer ID. When a certificate needs issuance, the service layer looks up the issuer ID in this registry to find the right connector.
|
||||
|
||||
**How the Local CA works internally:** The Local CA connector (`internal/connector/issuer/local/local.go`) generates a self-signed root CA certificate on first use using Go's `crypto/x509` and `crypto/rsa` packages. The CA key pair lives in memory only — it's regenerated each time the server restarts. When it receives an `IssuanceRequest` containing a CSR (Certificate Signing Request), it:
|
||||
**How the Local CA works internally:** The Local CA connector (`internal/connector/issuer/local/local.go`) generates a self-signed root CA certificate on first use using Go's `crypto/x509` package. The CA key pair lives in memory only — it's regenerated each time the server restarts, which means all certificates it issued become untrusted on restart (acceptable for dev/demo). When it receives an `IssuanceRequest` containing a CSR (Certificate Signing Request), it:
|
||||
|
||||
1. Parses the CSR using `x509.ParseCertificateRequest()`
|
||||
2. Generates a random serial number via `crypto/rand`
|
||||
@@ -233,7 +233,7 @@ sequenceDiagram
|
||||
S->>DB: SELECT pending jobs
|
||||
DB-->>S: [job-123: Renewal for mc-demo-api]
|
||||
|
||||
SVC->>SVC: Generate RSA-2048 key + CSR (server-side in V1)
|
||||
SVC->>SVC: Generate ECDSA P-256 key + CSR (server-side in demo mode)
|
||||
SVC->>ISS: IssueCertificate(commonName, sans, csrPEM)
|
||||
ISS-->>SVC: {cert_pem, chain_pem, serial, not_after}
|
||||
|
||||
@@ -251,7 +251,7 @@ sequenceDiagram
|
||||
A->>SVC: POST /api/v1/agents/{id}/jobs/{jobId}/status {Completed}
|
||||
```
|
||||
|
||||
**V1 note:** In V1 with the Local CA, key generation happens server-side in `RenewalService.ProcessRenewalJob`. In V2+, agents will generate keys locally and submit CSRs, ensuring private keys never touch the control plane.
|
||||
**Keygen mode note:** By default, certctl uses agent-side key generation (`CERTCTL_KEYGEN_MODE=agent`) where agents generate ECDSA P-256 keys locally and submit CSRs to the control plane — private keys never leave agent infrastructure. The Docker Compose demo stack uses server-side keygen mode (`CERTCTL_KEYGEN_MODE=server`) for simplicity, where the control plane generates keys within `RenewalService.ProcessRenewalJob`. In production, always use agent keygen mode.
|
||||
|
||||
Check the jobs list:
|
||||
|
||||
@@ -316,7 +316,7 @@ sequenceDiagram
|
||||
A->>A: Report deployment status to control plane
|
||||
```
|
||||
|
||||
Notice the `DeploymentRequest` struct intentionally omits the private key field. The agent loads the key from its own local storage and combines it with the certificate from the control plane. This is the architectural boundary that ensures zero private key exposure — the target connector interface physically cannot receive keys from the control plane because the data structure doesn't carry them.
|
||||
The `DeploymentRequest` struct includes a `KeyPEM` field, but this field is populated by the agent from its local key store (`CERTCTL_KEY_DIR`), never from the control plane. The control plane only sends the signed certificate and CA chain (public material). The agent combines the locally-generated private key with the certificate from the control plane to create the full deployment payload. This is the architectural boundary that ensures zero private key exposure — the control plane API never transmits private keys, and the agent's key store is the sole source of key material for target deployment.
|
||||
|
||||
Check for deployment jobs:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user