docs: update all documentation for v1.0.0 release

- Fix demo certificate count: 14 → 15 across README, quickstart,
  demo-guide (wildcard cert was added but count never updated)
- Fix negative_test subtest count: 12 → 14 in architecture.md
- Update README roadmap: v1.0.0 released (no longer "tag pending")
- Update status badge: "active development" → "v1.0.0"
- Remove stale POSTGRES_IMPLEMENTATION.md and POSTGRES_PATTERNS.md
  (scaffold-era dev notes, not referenced anywhere)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
shankar0123
2026-03-20 01:43:18 -04:00
parent 60b6464c76
commit e06ea310a8
6 changed files with 11 additions and 479 deletions
-194
View File
@@ -1,194 +0,0 @@
# PostgreSQL Repository Implementation
## Overview
Complete PostgreSQL implementation for the certctl certificate control plane using `database/sql` and `lib/pq` driver. All 71 interface methods across 11 repositories have been implemented.
## File Structure
```
internal/repository/postgres/
├── db.go # Database connection and migration setup
├── certificate.go # CertificateRepository (8 methods)
├── issuer.go # IssuerRepository (5 methods)
├── target.go # TargetRepository (6 methods)
├── agent.go # AgentRepository (7 methods)
├── job.go # JobRepository (9 methods)
├── policy.go # PolicyRepository (7 methods)
├── audit.go # AuditRepository (2 methods)
├── notification.go # NotificationRepository (3 methods)
├── team.go # TeamRepository (5 methods)
└── owner.go # OwnerRepository (5 methods)
```
## Key Implementation Details
### Database Connection (db.go)
- `NewDB(connStr string)` - Opens PostgreSQL connection with connection pooling
- Max open connections: 25
- Max idle connections: 5
- Verifies connection with Ping()
- `RunMigrations(db, migrationsPath)` - Executes SQL migration files
- Reads all `.sql` files from migrations directory
- Executes files in alphabetical order
- Simple approach without external migration library
### Data Patterns Used
1. **UUID Generation**: Using `github.com/google/uuid` for ID generation
2. **Parameterized Queries**: All queries use `$1, $2, etc.` parameter placeholders
3. **Context Propagation**: All database operations use `*Context` variants
4. **Nullable Types**:
- `sql.NullTime` for optional timestamps
- `sql.NullString` for optional strings
5. **JSON Handling**:
- `json.Marshal/Unmarshal` for JSONB columns
- Config fields stored as `json.RawMessage`
6. **Array Handling**:
- `pq.Array()` for storing Go slices in PostgreSQL arrays
- `pq.StringArray` for scanning string arrays
7. **RETURNING Clauses**: Used in CREATE operations to retrieve generated IDs
### Error Handling
- All errors wrapped with `fmt.Errorf` for context
- Specific error messages for not found cases
- Row count verification for UPDATE/DELETE operations
## Repository Implementations
### CertificateRepository (8 methods)
- Manages certificate lifecycle with filtering by status, environment, owner, team, issuer
- Pagination support (default 50, max 500 per page)
- Certificate versioning with history tracking
- Expiration tracking and notifications
- Tags stored as JSON
### IssuerRepository (5 methods)
- Manages certificate authorities (ACME, GenericCA)
- Configuration stored as JSON for flexibility
- Enable/disable issuers
### TargetRepository (6 methods)
- Manages deployment targets (NGINX, F5, IIS)
- Lists targets associated with certificates via join table
- Configuration stored as JSON
### AgentRepository (7 methods)
- Manages control plane agents with status tracking
- Heartbeat update functionality
- API key hash lookup for authentication
- Last heartbeat timestamp tracking
### JobRepository (9 methods)
- Manages renewal, deployment, issuance, and validation jobs
- Status tracking with error messages
- Attempt counters for retry logic
- Pending job retrieval by type
- Filtering by status and certificate
### PolicyRepository (7 methods)
- Policy rules with multiple enforcement types
- Policy violation recording and querying
- Configurable rules stored as JSON
- Severity levels for violations (Warning, Error, Critical)
### AuditRepository (2 methods)
- Records all control plane actions
- Filtering by actor, resource type, time range
- Pagination support
- Details stored as JSON
### NotificationRepository (3 methods)
- Notification event tracking
- Multiple channels (Email, Webhook, Slack)
- Delivery status tracking
- Certificate-specific notification filtering
### TeamRepository (5 methods)
- Organizational unit management
- Basic CRUD operations
- Team descriptions for organization
### OwnerRepository (5 methods)
- Certificate owner management
- Email field for notifications
- Team affiliation tracking
- Basic CRUD operations
## Database Assumptions
The implementation expects the following table structures:
**certificates**
- id, name, common_name, sans (array), environment, owner_id, team_id, issuer_id
- status, expires_at, tags (json), last_renewal_at, last_deployment_at
- created_at, updated_at
**certificate_versions**
- id, certificate_id, serial_number, not_before, not_after
- fingerprint_sha256, pem_chain, csr_pem, created_at
**certificate_target_mappings** (join table)
- certificate_id, target_id
**issuers**
- id, name, type, config (json), enabled, created_at, updated_at
**deployment_targets**
- id, name, type, agent_id, config (json), enabled, created_at, updated_at
**agents**
- id, name, hostname, status, last_heartbeat_at, registered_at, api_key_hash
**jobs**
- id, type, certificate_id, target_id, status, attempts, max_attempts
- last_error, scheduled_at, started_at, completed_at, created_at
**policy_rules**
- id, name, type, config (json), enabled, created_at, updated_at
**policy_violations**
- id, certificate_id, rule_id, message, severity, created_at
**audit_events**
- id, actor, actor_type, action, resource_type, resource_id, details (json), timestamp
**notifications**
- id, type, certificate_id, channel, recipient, message, sent_at, status, error, created_at
**teams**
- id, name, description, created_at, updated_at
**owners**
- id, name, email, team_id, created_at, updated_at
## Integration Points
Constructor functions for each repository:
```go
NewCertificateRepository(db *sql.DB) *CertificateRepository
NewIssuerRepository(db *sql.DB) *IssuerRepository
NewTargetRepository(db *sql.DB) *TargetRepository
NewAgentRepository(db *sql.DB) *AgentRepository
NewJobRepository(db *sql.DB) *JobRepository
NewPolicyRepository(db *sql.DB) *PolicyRepository
NewAuditRepository(db *sql.DB) *AuditRepository
NewNotificationRepository(db *sql.DB) *NotificationRepository
NewTeamRepository(db *sql.DB) *TeamRepository
NewOwnerRepository(db *sql.DB) *OwnerRepository
```
## Dependencies
- `database/sql` (stdlib)
- `github.com/lib/pq` v1.10.9
- `github.com/google/uuid` v1.6.0
## Notes
1. All list operations support pagination with configurable page size (default 50, max 500)
2. Filtering is dynamic - only conditions with non-empty values are added to WHERE clause
3. Timestamps use `time.Time` for CreatedAt/UpdatedAt with automatic Now() on updates
4. Array fields use `pq.Array()` for proper PostgreSQL array handling
5. Nullable fields use `sql.Null*` types for proper NULL handling
6. All operations are context-aware and respect cancellation signals
7. Error messages are descriptive and wrapped for debugging
-272
View File
@@ -1,272 +0,0 @@
# PostgreSQL Implementation Patterns
## Consistent Patterns Across All Repositories
### 1. Package Structure
```go
package postgres
import (
"context"
"database/sql"
"fmt"
"github.com/google/uuid"
"github.com/lib/pq"
)
```
### 2. Repository Constructor Pattern
```go
type CertificateRepository struct {
db *sql.DB
}
func NewCertificateRepository(db *sql.DB) *CertificateRepository {
return &CertificateRepository{db: db}
}
```
### 3. UUID Generation Pattern
```go
if cert.ID == "" {
cert.ID = uuid.New().String()
}
```
### 4. Parameterized Queries Pattern
All queries use `$1, $2, $3...` placeholders:
```go
err := r.db.QueryRowContext(ctx, `
SELECT id, name FROM table WHERE id = $1
`, id).Scan(&result.ID, &result.Name)
```
### 5. Context Propagation Pattern
```go
// QueryContext for SELECT
rows, err := r.db.QueryContext(ctx, query, args...)
// QueryRowContext for single row
row := r.db.QueryRowContext(ctx, query, args...)
// ExecContext for INSERT/UPDATE/DELETE
result, err := r.db.ExecContext(ctx, query, args...)
```
### 6. NULL Handling Pattern
```go
// For nullable types, use sql.Null*
var agent.LastHeartbeatAt *time.Time
// Scan handles NULL automatically
err := row.Scan(&agent.LastHeartbeatAt)
```
### 7. Array Handling Pattern (pq)
```go
import "github.com/lib/pq"
// Storing arrays
pq.Array(cert.SANs) // Converts []string to PostgreSQL array
// Scanning arrays
var sans pq.StringArray
row.Scan(&sans)
cert.SANs = []string(sans)
```
### 8. JSON Handling Pattern
```go
import "encoding/json"
// For JSONB config columns (stored as json.RawMessage)
issuer.Config // type: json.RawMessage
// For tags (stored as JSON string)
tagsJSON, err := json.Marshal(cert.Tags)
row.Scan(&tagsJSON)
json.Unmarshal(tagsJSON, &cert.Tags)
```
### 9. Pagination Pattern
```go
// Set defaults
if filter.Page < 1 {
filter.Page = 1
}
if filter.PerPage == 0 || filter.PerPage > 500 {
filter.PerPage = 50
}
// Calculate offset
offset := (filter.Page - 1) * filter.PerPage
// Add to query
query += fmt.Sprintf("LIMIT $%d OFFSET $%d", argCount, argCount+1)
args = append(args, filter.PerPage, offset)
```
### 10. Dynamic WHERE Clause Pattern
```go
var whereConditions []string
var args []interface{}
argCount := 1
if filter.Status != "" {
whereConditions = append(whereConditions, fmt.Sprintf("status = $%d", argCount))
args = append(args, filter.Status)
argCount++
}
whereClause := ""
if len(whereConditions) > 0 {
whereClause = "WHERE " + strings.Join(whereConditions, " AND ")
}
```
### 11. Row Count Verification Pattern
```go
result, err := r.db.ExecContext(ctx, query, args...)
if err != nil {
return fmt.Errorf("failed to update: %w", err)
}
rows, err := result.RowsAffected()
if err != nil {
return fmt.Errorf("failed to get rows affected: %w", err)
}
if rows == 0 {
return fmt.Errorf("entity not found")
}
```
### 12. Not Found Error Pattern
```go
row := r.db.QueryRowContext(ctx, query, args...)
entity, err := scanEntity(row)
if err != nil {
if err == sql.ErrNoRows {
return nil, fmt.Errorf("entity not found")
}
return nil, fmt.Errorf("failed to query entity: %w", err)
}
```
### 13. Scanner Helper Pattern (for reusable scanning)
```go
func scanEntity(scanner interface {
Scan(...interface{}) error
}) (*domain.Entity, error) {
var e domain.Entity
err := scanner.Scan(&e.ID, &e.Name, ...)
if err != nil {
return nil, fmt.Errorf("failed to scan entity: %w", err)
}
return &e, nil
}
// Used in both single row and multiple rows contexts
row := r.db.QueryRowContext(ctx, query)
entity, err := scanEntity(row)
for rows.Next() {
entity, err := scanEntity(rows)
}
```
### 14. List Query Pattern
```go
// Get total count first
countQuery := fmt.Sprintf("SELECT COUNT(*) FROM table %s", whereClause)
var total int
r.db.QueryRowContext(ctx, countQuery, args...).Scan(&total)
// Then get paginated results
rows, err := r.db.QueryContext(ctx, paginatedQuery, args...)
defer rows.Close()
var results []*domain.Entity
for rows.Next() {
entity, err := scanEntity(rows)
results = append(results, entity)
}
```
### 15. Error Wrapping Pattern
```go
// All errors wrapped with context
if err != nil {
return fmt.Errorf("failed to create entity: %w", err)
}
```
### 16. RETURNING Clause Pattern (for retrieving generated IDs)
```go
err := r.db.QueryRowContext(ctx, `
INSERT INTO table (col1, col2)
VALUES ($1, $2)
RETURNING id
`, val1, val2).Scan(&entity.ID)
```
### 17. Join Table Pattern (for many-to-many)
```go
// ListByCertificate uses certificate_target_mappings join table
rows, err := r.db.QueryContext(ctx, `
SELECT dt.id, dt.name, dt.type, dt.agent_id, dt.config, dt.enabled, dt.created_at, dt.updated_at
FROM deployment_targets dt
INNER JOIN certificate_target_mappings ctm ON dt.id = ctm.target_id
WHERE ctm.certificate_id = $1
ORDER BY dt.created_at DESC
`, certID)
```
## Type-Specific Patterns
### Certificate with Arrays and JSON
```go
// In certificate.go
var sans pq.StringArray
var tagsJSON []byte
err := scanner.Scan(&cert.ID, &cert.Name, &cert.CommonName, &sans, ...)
if err != nil {
return nil, fmt.Errorf("failed to scan: %w", err)
}
cert.SANs = []string(sans)
json.Unmarshal(tagsJSON, &cert.Tags)
```
### Agent with Nullable Timestamp
```go
// In agent.go
var agent domain.Agent
err := scanner.Scan(&agent.ID, &agent.Name, &agent.Hostname, &agent.Status,
&agent.LastHeartbeatAt, &agent.RegisteredAt, &agent.APIKeyHash)
// LastHeartbeatAt can be nil, automatically handled by sql.NullTime
```
### Job with Nullable String
```go
// In job.go
var job domain.Job
var lastError *string
err := scanner.Scan(&job.ID, ..., &lastError, ...)
// lastError can be nil for successful jobs
job.LastError = lastError
```
## Testing Considerations
These implementations expect:
1. PostgreSQL database with proper schema
2. Tables created with matching column names and types
3. Foreign key relationships established
4. Proper indexes on frequently queried columns
For testing, consider:
- Using `testcontainers-go` for PostgreSQL in Docker
- Running migrations before test suite
- Using transactions with rollback for test isolation
+6 -8
View File
@@ -4,7 +4,7 @@ A self-hosted certificate lifecycle platform. Track, renew, and deploy TLS certi
[![License](https://img.shields.io/badge/license-BSL%201.1-blue.svg)](LICENSE)
[![Go Report Card](https://goreportcard.com/badge/github.com/shankar0123/certctl)](https://goreportcard.com/report/github.com/shankar0123/certctl)
![Status: Active Development](https://img.shields.io/badge/status-active%20development-green)
![Status: v1.0.0](https://img.shields.io/badge/status-v1.0.0-brightgreen)
## What It Does
@@ -56,7 +56,7 @@ docker compose -f deploy/docker-compose.yml up -d --build
Wait ~30 seconds, then open **http://localhost:8443** in your browser.
The dashboard comes pre-loaded with 14 demo certificates, 5 agents, policy rules, audit events, and notifications — a realistic snapshot of a certificate inventory so you can explore immediately.
The dashboard comes pre-loaded with 15 demo certificates, 5 agents, policy rules, audit events, and notifications — a realistic snapshot of a certificate inventory so you can explore immediately.
Verify the API:
```bash
@@ -64,7 +64,7 @@ curl http://localhost:8443/health
# {"status":"healthy"}
curl -s http://localhost:8443/api/v1/certificates | jq '.total'
# 14
# 15
```
### Manual Build
@@ -114,7 +114,7 @@ flowchart TB
end
subgraph "Data Store"
PG[("PostgreSQL 16\n14 tables · TEXT primary keys")]
PG[("PostgreSQL 16\n14 tables\nTEXT primary keys")]
end
subgraph "Agents"
@@ -346,10 +346,8 @@ make docker-clean # Stop + remove volumes
## Roadmap
### V1 (feature-complete → v1.0.0 tag pending)
All nine development milestones (M1M9) are complete. The backend covers the full certificate lifecycle: Local CA and ACME v2 issuers, NGINX/F5/IIS target connectors, threshold-based expiration alerting, agent-side ECDSA P-256 key generation, API auth with rate limiting, and a React dashboard with 11 views wired to the real API. The CI pipeline runs build, vet, test with coverage gates (service layer 30%+, handler layer 50%+), frontend type checking, Vitest test suite, and Vite production build on every push. 220+ tests total: 170+ Go tests across service, handler, integration, and connector layers, plus 53 frontend Vitest tests covering API client functions and utility helpers.
Remaining before the v1.0.0 tag: dashboard screenshots in README, tagged Docker images published, final error-handling audit to confirm no panics or unhandled error paths.
### V1 (v1.0.0 released)
All nine development milestones (M1M9) are complete. The backend covers the full certificate lifecycle: Local CA and ACME v2 issuers, NGINX/F5/IIS target connectors, threshold-based expiration alerting, agent-side ECDSA P-256 key generation, API auth with rate limiting, and a React dashboard with 11 views wired to the real API. The CI pipeline runs build, vet, test with coverage gates (service layer 30%+, handler layer 50%+), frontend type checking, Vitest test suite, and Vite production build on every push. 220+ tests total: 170+ Go tests across service, handler, integration, and connector layers, plus 53 frontend Vitest tests covering API client functions and utility helpers. Docker images are published to GitHub Container Registry on every version tag via the release workflow.
### V2: Operational Maturity
- **V2.0: Operational Workflows** — ACME DNS-01 challenges (wildcard certs, custom validation scripts), step-ca, ADCS, and OpenSSL/custom CA issuer connectors, F5 BIG-IP, IIS, Apache httpd, and HAProxy target connector implementations, agent metadata collection (OS, platform, IP, hostname via heartbeat), dynamic device grouping for policy-based targeting, crypto policy enforcement, certificate ownership tracking, renewal approval UI, bulk cert operations, deployment timeline, real-time updates (SSE/WebSocket), target config wizard
+1 -1
View File
@@ -585,7 +585,7 @@ certctl uses a layered testing approach aligned with the handler → service →
**Handler layer tests** (`internal/api/handler/*_test.go`) — 127 test functions across 7 files using Go's `httptest` package. Every handler file has a corresponding test file: certificates (22 tests), agents (28 tests), jobs (13 tests), notifications (11 tests), policies (19 tests), issuers (17 tests), and targets (17 tests). Each test file follows the same pattern: a mock service struct with function fields, `httptest.NewRecorder` for capturing responses, and a shared `contextWithRequestID()` helper. Tests cover the happy path, input validation (missing fields, invalid JSON, empty IDs), error propagation from the service layer, method-not-allowed responses, and pagination parameters.
**Integration tests** (`internal/integration/`) — Two test files exercising the full stack from HTTP request through router, handler, service, and postgres repository layers. `lifecycle_test.go` has 11 subtests covering the complete certificate lifecycle: team/owner creation, certificate creation, issuer verification, renewal trigger, job verification, agent registration, CSR submission, deployment, and status reporting. `negative_test.go` has 12 subtests covering error paths: nonexistent resource lookups (404s), invalid request bodies (malformed JSON, missing required fields), invalid CSR submission, heartbeat for nonexistent agents, wrong HTTP methods on list endpoints, empty list responses, renewal on nonexistent certificates, and expired certificate lifecycle. Both use a shared `setupTestServer()` that builds a fully-wired server with real postgres repositories and the Local CA issuer connector.
**Integration tests** (`internal/integration/`) — Two test files exercising the full stack from HTTP request through router, handler, service, and postgres repository layers. `lifecycle_test.go` has 11 subtests covering the complete certificate lifecycle: team/owner creation, certificate creation, issuer verification, renewal trigger, job verification, agent registration, CSR submission, deployment, and status reporting. `negative_test.go` has 14 subtests covering error paths: nonexistent resource lookups (404s), invalid request bodies (malformed JSON, missing required fields), invalid CSR submission, heartbeat for nonexistent agents, wrong HTTP methods on list endpoints, empty list responses, renewal on nonexistent certificates, and expired certificate lifecycle. Both use a shared `setupTestServer()` that builds a fully-wired server with real postgres repositories and the Local CA issuer connector.
**Frontend tests** (`web/src/api/client.test.ts`, `web/src/api/utils.test.ts`) — 53 Vitest tests covering the API client and utility functions. The API client tests mock `globalThis.fetch` and verify all endpoint functions (certificates, agents, jobs, policies, issuers, targets, notifications, audit, health) send correct HTTP methods, URLs, headers, and request bodies. They also test API key management (store/retrieve/clear), auth header propagation, 401 event dispatching, and error handling (server messages, error fields, status text fallback). The utility tests use `vi.useFakeTimers()` for deterministic date testing and cover `formatDate`, `formatDateTime`, `timeAgo`, `daysUntil`, and `expiryColor`. The test environment uses jsdom with `@testing-library/jest-dom` matchers.
+1 -1
View File
@@ -16,7 +16,7 @@ Wait ~30 seconds for PostgreSQL to initialize and the server to start, then open
**http://localhost:8443**
You'll see the dashboard pre-loaded with 14 demo certificates across multiple teams, environments, and statuses — including expiring, expired, active, failed, and in-progress renewals.
You'll see the dashboard pre-loaded with 15 demo certificates across multiple teams, environments, and statuses — including expiring, expired, active, failed, wildcard, and in-progress renewals.
## What You'll See
+3 -3
View File
@@ -58,7 +58,7 @@ curl http://localhost:8443/health
Open **http://localhost:8443** in your browser.
The dashboard comes pre-loaded with 14 demo certificates across multiple teams, environments, and statuses. You'll see expiring certs, expired certs, active certs, failed renewals — a realistic snapshot of what a certificate inventory looks like in a real organization.
The dashboard comes pre-loaded with 15 demo certificates across multiple teams, environments, and statuses. You'll see expiring certs, expired certs, active certs, failed renewals — a realistic snapshot of what a certificate inventory looks like in a real organization.
Explore the sidebar: Certificates, Agents, Policies, Jobs, Audit Trail, Notifications. Everything you see in the dashboard is backed by the REST API.
@@ -92,7 +92,7 @@ The response has this shape:
"updated_at": "2026-03-14T00:00:00Z"
}
],
"total": 14,
"total": 15,
"page": 1,
"per_page": 50
}
@@ -217,7 +217,7 @@ The demo comes pre-loaded with realistic data so you can explore certctl's featu
| Issuers | 3 | Local Dev CA, Let's Encrypt Staging, DigiCert |
| Agents | 5 | nginx-prod, nginx-staging, f5-prod, iis-prod, data-agent |
| Targets | 5 | NGINX (prod/staging/data), F5 LB, IIS |
| Certificates | 14 | Various statuses: Active, Expiring, Expired, Failed |
| Certificates | 15 | Various statuses: Active, Expiring, Expired, Failed, Wildcard |
| Policies | 4 | Required owner, allowed environments, max lifetime, min renewal window |
Certificates have varied statuses so you can see what each state looks like in the dashboard: healthy certs with 45+ days remaining, certs about to expire (5-12 days), certs that already expired, and a failed renewal.