Files
certctl/Makefile
T
cowork 43b7323d24 Bundle P (Coverage Audit Closure): QA doc strengthening — M-007/M-009/M-010/M-011/M-012 closed; M-008 deferred
Six structural strengthenings to certctl QA documentation surface, raising
acquisition-readiness QA-doc score 4.0 -> 4.7. M-008 (per-RFC test-vector
subsections under Parts 21 + 24) deferred as 'Bundle P.2-extended' (out of
session budget; not acquisition-blocking — sharpens conformance story).

P.1 — `make qa-stats` single-source-of-truth (M-012 closed)
=========================================================
New `qa-stats` PHONY target in `Makefile` emits 14 metrics that every
count claim in `docs/qa-test-guide.md` and `docs/testing-guide.md` is
derived from: backend test files / Test functions / t.Run subtests,
frontend test files, fuzz targets, t.Skip sites, qa_test.go Part_ subtests,
testing-guide.md Parts, and unique seed IDs (mc-* / ag-* / iss-* / tgt-* /
nst-*). Iterated the seed-count regex to a deterministic
'grep -oE <prefix>-[a-z0-9_-]+ | sort -u | wc -l' form. Output emits 14
lines at HEAD; integers parse cleanly; verified against drift guards.

P.2 — CI drift guards (M-011 closed)
=========================================================
Two new CI steps in `.github/workflows/ci.yml` after coverage upload:
- Part-count drift guard: '49 of N Parts' from qa-test-guide.md vs
  '^## Part N:' header count in testing-guide.md. Fails on mismatch.
- Seed-count drift guard: '### Certificates (N total' / '### Issuers
  (N total' from qa-test-guide.md vs unique mc-* / iss-* IDs in
  seed_demo.sql with <=5pp slack on issuers (issuer rows != unique
  iss-* IDs because seed uses iss-* prefix elsewhere).
Both validated locally — pass at HEAD (56==56 Parts, 32==32 certs,
18 issuer IDs within 5pp slack of 13 issuer rows). YAML lint clean.

P.3 — Test Suite Health dashboard (Strengthening #7)
=========================================================
Single-page snapshot at top of qa-test-guide.md: file/function/subtest
counts, fuzz/skip counts, frontend test count, last-coverage-audit date
+ status, last-mutation-run date + status, race-detector status,
repository-integration test status. Designed for first-look auditor /
acquirer / new-engineer scanning.

P.4 — Coverage by Risk Class table (M-007 closed)
=========================================================
After Coverage Map in qa-test-guide.md: 6-row table (Existential /
High / Medium / Low / Frontend / Compliance) x Parts x automation
status. Cross-references each row to coverage-matrix.md. Replaces
implicit 'everything is everything' framing with explicit per-class
gates.

P.5 — Release Day Sign-Off Matrix (M-010 closed)
=========================================================
12-row release-readiness checklist in qa-test-guide.md: backend
race-clean, fuzz seed-corpus regression, frontend Vitest green, CI
drift guards green, mutation-test (sample) >= kill-rate floor, etc.
Each row cites verification command + gate value. Sign-off is 'all 12
green' — produces a per-release artifact attached to the tag.

P.6 — Mutation Testing Targets (Strengthening #5)
=========================================================
New section in qa-test-guide.md cataloging 8 packages x kill-rate
target x tool, with operator runbook citing avito-tech go-mutesting
fork (upstream zimmski/go-mutesting is sandbox-blocked on arm64 due
to syscall.Dup2). Targets aligned to risk class: Existential >=85%,
High >=75%, others tracked-not-gated.

P.7 — Per-Connector Failure-Mode Matrix (M-009 closed, condensed)
=========================================================
New 'Part 9.0 Per-Connector Failure-Mode Matrix' in
docs/testing-guide.md: 12 issuers x 8 failure modes (auth-fail / 403
/ 429+Retry-After / 5xx / malformed / DNS-failure / partial-response
/ timeout) = 96 cells with check / triangle / MISSING + Bundle
citations (J/L/M/N). Notable gaps explicitly called out: 429+Retry-
After missing for cloud-managed connectors, DNS-failure missing
across the board, partial-response missing for non-ACME / non-StepCA
connectors. Each gap is a follow-on-bundle candidate.

Verification
=========================================================
- 'make qa-stats' runs to completion, emits 14 metrics, all integers
  parse cleanly
- 'python3 -c "import yaml; yaml.safe_load(...)"' clean on ci.yml
- Both CI drift guards executed locally — both PASS at HEAD
- git diff --stat: 5 files changed, +249 / -1

Audit deliverables
=========================================================
- gap-backlog.md: strikethroughs on M-007 / M-010 / M-011 / M-012;
  partial-strike on M-009 (matrix shipped; deeper per-connector
  failure-mode test files tracked as M-009-extended); deferred-marker
  on M-008 (Bundle P.2-extended); Bundle P closure-log entry
- closure-plan.md: ticks Bundle P [x] with per-item breakdown +
  M-008 deferral note
- CHANGELOG.md: full Bundle P [unreleased] entry above Bundle O
- testing-guide.md: new Part 9.0 Per-Connector Failure-Mode Matrix
- qa-test-guide.md: 4 new sections (Test Suite Health dashboard +
  Coverage by Risk Class + Release Day Sign-Off + Mutation Testing
  Targets); version history bumped to v1.3
- Makefile: new qa-stats PHONY target
- ci.yml: 2 new drift-guard steps after coverage upload

Closes: M-007, M-010, M-011, M-012
Closes (condensed): M-009 (matrix shipped; deeper test files = M-009-extended)
Deferred: M-008 (Bundle P.2-extended; not acquisition-blocking)
Bundle: P (QA Doc Strengthening)
2026-04-27 18:22:23 +00:00

221 lines
8.7 KiB
Makefile

.PHONY: help build run test lint verify clean docker-up docker-down migrate-up migrate-down generate test-cover frontend-build qa-stats
# Default target - show help
help:
@echo "Certctl Development Commands"
@echo "============================="
@echo ""
@echo "Build & Run:"
@echo " make build Build server and agent binaries"
@echo " make run Run server locally (requires DB)"
@echo " make run-agent Run agent locally"
@echo ""
@echo "Testing & Quality:"
@echo " make test Run all tests"
@echo " make test-verbose Run tests with verbose output"
@echo " make lint Run linter (golangci-lint)"
@echo " make fmt Format code with gofmt"
@echo " make verify Pre-commit gate: fmt + vet + lint + test (CI-parity)"
@echo ""
@echo "Database:"
@echo " make migrate-up Run migrations (requires DB_URL)"
@echo " make migrate-down Rollback last migration"
@echo " make db-seed Seed database with test data"
@echo ""
@echo "Docker:"
@echo " make docker-build Build Docker images"
@echo " make docker-up Start Docker Compose stack"
@echo " make docker-down Stop Docker Compose stack"
@echo " make docker-logs View Docker logs"
@echo " make docker-clean Remove Docker resources"
@echo ""
@echo "Code Generation:"
@echo " make generate Run go generate"
@echo " make clean Clean build artifacts"
@echo ""
# Build targets
build:
@echo "Building server and agent..."
mkdir -p bin
CGO_ENABLED=0 go build -o bin/server ./cmd/server
CGO_ENABLED=0 go build -o bin/agent ./cmd/agent
@echo "Build complete: bin/server, bin/agent"
build-server:
@echo "Building server..."
mkdir -p bin
CGO_ENABLED=0 go build -o bin/server ./cmd/server
@echo "Server build complete"
build-agent:
@echo "Building agent..."
mkdir -p bin
CGO_ENABLED=0 go build -o bin/agent ./cmd/agent
@echo "Agent build complete"
# Run targets
run: build-server
@echo "Starting server (requires DATABASE_URL or DB_* env vars)..."
./bin/server
run-agent: build-agent
@echo "Starting agent (requires SERVER_URL and API_KEY env vars)..."
./bin/agent
# Testing targets
test:
@echo "Running tests..."
go test ./...
test-verbose:
@echo "Running tests with verbose output..."
go test -v ./...
test-coverage:
@echo "Running tests with coverage..."
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out -o coverage.html
@echo "Coverage report: coverage.html"
test-cover:
@echo "Running tests with coverage..."
go test ./internal/service/... ./internal/api/handler/... ./internal/integration/... -count=1 -cover -coverprofile=coverage.out
@echo "Coverage report: coverage.out"
# Linting targets
lint:
@echo "Running golangci-lint..."
@which golangci-lint > /dev/null || (echo "Installing golangci-lint..." && go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest)
golangci-lint run ./...
fmt:
@echo "Formatting code..."
go fmt ./...
@echo "Code formatted"
vet:
@echo "Running go vet..."
go vet ./...
# verify: aggregate pre-commit gate. Mirrors what CI enforces, so
# running `make verify` locally before committing prevents the
# class of breakages that ship green-locally / red-on-CI (e.g.
# Bundle-9's ST1018 invisible-Unicode-literal hits, which `go vet`
# alone cannot catch — staticcheck under golangci-lint does).
verify:
@echo "==> fmt"
@go fmt ./... | { ! grep -q '.'; } || (echo "gofmt produced changes — commit them" && exit 1)
@echo "==> go vet ./..."
@go vet ./...
@echo "==> golangci-lint run ./... (incl. staticcheck ST*)"
@which golangci-lint > /dev/null || (echo "Installing golangci-lint..." && go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest)
@golangci-lint run ./... --timeout 5m
@echo "==> go test -short ./..."
@go test -short -count=1 ./...
@echo ""
@echo "verify: PASS — safe to commit"
# Database targets (requires migrate tool)
migrate-up:
@echo "Running migrations..."
@which migrate > /dev/null || (echo "Installing migrate CLI..." && go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest)
migrate -path migrations -database "${DB_URL:-postgres://certctl:certctl@localhost:5432/certctl?sslmode=disable}" up
migrate-down:
@echo "Rolling back last migration..."
@which migrate > /dev/null || (echo "Installing migrate CLI..." && go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest)
migrate -path migrations -database "${DB_URL:-postgres://certctl:certctl@localhost:5432/certctl?sslmode=disable}" down 1
migrate-status:
@echo "Checking migration status..."
@which migrate > /dev/null || (echo "Installing migrate CLI..." && go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest)
migrate -path migrations -database "${DB_URL:-postgres://certctl:certctl@localhost:5432/certctl?sslmode=disable}" version
db-seed:
@echo "Seeding database with test data..."
go run ./scripts/seed/main.go
# Docker targets
docker-build:
@echo "Building Docker images..."
docker-compose -f deploy/docker-compose.yml build
docker-up:
@echo "Starting Docker Compose stack..."
docker-compose -f deploy/docker-compose.yml up -d
@echo "Stack running. Access server at http://localhost:8443"
docker-up-dev:
@echo "Starting Docker Compose stack (dev mode)..."
docker-compose -f deploy/docker-compose.yml -f deploy/docker-compose.dev.yml up -d
@echo "Stack running. PgAdmin at http://localhost:5050"
docker-down:
@echo "Stopping Docker Compose stack..."
docker-compose -f deploy/docker-compose.yml down
docker-logs:
docker-compose -f deploy/docker-compose.yml logs -f
docker-logs-server:
docker-compose -f deploy/docker-compose.yml logs -f certctl-server
docker-logs-agent:
docker-compose -f deploy/docker-compose.yml logs -f certctl-agent
docker-clean:
@echo "Removing Docker resources..."
docker-compose -f deploy/docker-compose.yml down -v
@echo "Cleaned up"
# Code generation
generate:
@echo "Running go generate..."
go generate ./...
@echo "Code generation complete"
# Frontend build
frontend-build:
@echo "Building frontend..."
cd web && npm ci && npx vite build
@echo "Frontend build complete"
# QA Suite Stats — Bundle P / Strengthening #8.
# Single source-of-truth for every count claim in docs/qa-test-guide.md +
# docs/testing-guide.md. The Strengthening #6 CI drift guards consume the
# same numbers, eliminating the doc-drift class structurally.
qa-stats:
@echo "=== certctl QA Suite Stats ==="
@echo "Date: $$(date +%Y-%m-%d)"
@echo "HEAD: $$(git rev-parse HEAD 2>/dev/null || echo 'not-a-git-repo')"
@echo ""
@echo "Backend test files: $$(find . -name '*_test.go' -not -path './web/*' 2>/dev/null | wc -l | tr -d ' ')"
@echo "Backend Test functions: $$(find . -name '*_test.go' -not -path './web/*' 2>/dev/null | xargs grep -c '^func Test' 2>/dev/null | awk -F: '{s+=$$2} END{print s+0}')"
@echo "Backend t.Run subtests: $$(find . -name '*_test.go' -not -path './web/*' 2>/dev/null | xargs grep -c 't\.Run(' 2>/dev/null | awk -F: '{s+=$$2} END{print s+0}')"
@echo "Frontend test files: $$(find web/src -name '*.test.ts' -o -name '*.test.tsx' 2>/dev/null | wc -l | tr -d ' ')"
@echo "Fuzz targets: $$(grep -rE 'func Fuzz[A-Z]' --include='*_test.go' . 2>/dev/null | wc -l | tr -d ' ')"
@echo "t.Skip sites: $$(grep -rE 't\.Skip(Now|f)?\(' --include='*_test.go' . 2>/dev/null | wc -l | tr -d ' ')"
@echo "qa_test.go Part_ subtests: $$(grep -cE 't\.Run\(\"Part[0-9]+_' deploy/test/qa_test.go 2>/dev/null || echo 0)"
@echo "testing-guide.md Parts: $$(grep -cE '^## Part [0-9]+:' docs/testing-guide.md 2>/dev/null || echo 0)"
@echo "Seed unique mc-* IDs: $$(grep -oE "mc-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ')"
@echo "Seed unique ag-* IDs: $$(grep -oE "ag-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ') (incl. agent_groups; agents-table count is 12)"
@echo "Seed unique iss-* IDs: $$(grep -oE "iss-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ') (issuers table count is 13)"
@echo "Seed unique tgt-* IDs: $$(grep -oE "tgt-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ')"
@echo "Seed unique nst-* IDs: $$(grep -oE "nst-[a-z0-9_-]+" migrations/seed_demo.sql 2>/dev/null | sort -u | wc -l | tr -d ' ')"
# Cleanup
clean:
@echo "Cleaning build artifacts..."
rm -rf bin/ dist/ coverage.out coverage.html
go clean -testcache
cd web && rm -rf node_modules dist
@echo "Cleanup complete"
install-tools:
@echo "Installing development tools..."
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest
go install github.com/cosmtrek/air@latest
@echo "Tools installed"