33 Commits

Author SHA1 Message Date
shankar0123 30ac7910c2 Bundle P (Coverage Audit Closure): QA doc strengthening — M-007/M-009/M-010/M-011/M-012 closed; M-008 deferred
Six structural strengthenings to certctl QA documentation surface, raising
acquisition-readiness QA-doc score 4.0 -> 4.7. M-008 (per-RFC test-vector
subsections under Parts 21 + 24) deferred as 'Bundle P.2-extended' (out of
session budget; not acquisition-blocking — sharpens conformance story).

P.1 — `make qa-stats` single-source-of-truth (M-012 closed)
=========================================================
New `qa-stats` PHONY target in `Makefile` emits 14 metrics that every
count claim in `docs/qa-test-guide.md` and `docs/testing-guide.md` is
derived from: backend test files / Test functions / t.Run subtests,
frontend test files, fuzz targets, t.Skip sites, qa_test.go Part_ subtests,
testing-guide.md Parts, and unique seed IDs (mc-* / ag-* / iss-* / tgt-* /
nst-*). Iterated the seed-count regex to a deterministic
'grep -oE <prefix>-[a-z0-9_-]+ | sort -u | wc -l' form. Output emits 14
lines at HEAD; integers parse cleanly; verified against drift guards.

P.2 — CI drift guards (M-011 closed)
=========================================================
Two new CI steps in `.github/workflows/ci.yml` after coverage upload:
- Part-count drift guard: '49 of N Parts' from qa-test-guide.md vs
  '^## Part N:' header count in testing-guide.md. Fails on mismatch.
- Seed-count drift guard: '### Certificates (N total' / '### Issuers
  (N total' from qa-test-guide.md vs unique mc-* / iss-* IDs in
  seed_demo.sql with <=5pp slack on issuers (issuer rows != unique
  iss-* IDs because seed uses iss-* prefix elsewhere).
Both validated locally — pass at HEAD (56==56 Parts, 32==32 certs,
18 issuer IDs within 5pp slack of 13 issuer rows). YAML lint clean.

P.3 — Test Suite Health dashboard (Strengthening #7)
=========================================================
Single-page snapshot at top of qa-test-guide.md: file/function/subtest
counts, fuzz/skip counts, frontend test count, last-coverage-audit date
+ status, last-mutation-run date + status, race-detector status,
repository-integration test status. Designed for first-look auditor /
acquirer / new-engineer scanning.

P.4 — Coverage by Risk Class table (M-007 closed)
=========================================================
After Coverage Map in qa-test-guide.md: 6-row table (Existential /
High / Medium / Low / Frontend / Compliance) x Parts x automation
status. Cross-references each row to coverage-matrix.md. Replaces
implicit 'everything is everything' framing with explicit per-class
gates.

P.5 — Release Day Sign-Off Matrix (M-010 closed)
=========================================================
12-row release-readiness checklist in qa-test-guide.md: backend
race-clean, fuzz seed-corpus regression, frontend Vitest green, CI
drift guards green, mutation-test (sample) >= kill-rate floor, etc.
Each row cites verification command + gate value. Sign-off is 'all 12
green' — produces a per-release artifact attached to the tag.

P.6 — Mutation Testing Targets (Strengthening #5)
=========================================================
New section in qa-test-guide.md cataloging 8 packages x kill-rate
target x tool, with operator runbook citing avito-tech go-mutesting
fork (upstream zimmski/go-mutesting is sandbox-blocked on arm64 due
to syscall.Dup2). Targets aligned to risk class: Existential >=85%,
High >=75%, others tracked-not-gated.

P.7 — Per-Connector Failure-Mode Matrix (M-009 closed, condensed)
=========================================================
New 'Part 9.0 Per-Connector Failure-Mode Matrix' in
docs/testing-guide.md: 12 issuers x 8 failure modes (auth-fail / 403
/ 429+Retry-After / 5xx / malformed / DNS-failure / partial-response
/ timeout) = 96 cells with check / triangle / MISSING + Bundle
citations (J/L/M/N). Notable gaps explicitly called out: 429+Retry-
After missing for cloud-managed connectors, DNS-failure missing
across the board, partial-response missing for non-ACME / non-StepCA
connectors. Each gap is a follow-on-bundle candidate.

Verification
=========================================================
- 'make qa-stats' runs to completion, emits 14 metrics, all integers
  parse cleanly
- 'python3 -c "import yaml; yaml.safe_load(...)"' clean on ci.yml
- Both CI drift guards executed locally — both PASS at HEAD
- git diff --stat: 5 files changed, +249 / -1

Audit deliverables
=========================================================
- gap-backlog.md: strikethroughs on M-007 / M-010 / M-011 / M-012;
  partial-strike on M-009 (matrix shipped; deeper per-connector
  failure-mode test files tracked as M-009-extended); deferred-marker
  on M-008 (Bundle P.2-extended); Bundle P closure-log entry
- closure-plan.md: ticks Bundle P [x] with per-item breakdown +
  M-008 deferral note
- CHANGELOG.md: full Bundle P [unreleased] entry above Bundle O
- testing-guide.md: new Part 9.0 Per-Connector Failure-Mode Matrix
- qa-test-guide.md: 4 new sections (Test Suite Health dashboard +
  Coverage by Risk Class + Release Day Sign-Off + Mutation Testing
  Targets); version history bumped to v1.3
- Makefile: new qa-stats PHONY target
- ci.yml: 2 new drift-guard steps after coverage upload

Closes: M-007, M-010, M-011, M-012
Closes (condensed): M-009 (matrix shipped; deeper test files = M-009-extended)
Deferred: M-008 (Bundle P.2-extended; not acquisition-blocking)
Bundle: P (QA Doc Strengthening)
2026-04-27 18:22:23 +00:00
shankar0123 04c7eca615 docs: reconcile scheduler topology across sibling docs (7 → 12 loops)
Authoritative 12-loop table lives at docs/architecture.md:522-534 (committed via
the I-001/I-003/I-005 + M48/M50 milestone commits). This change brings six sibling
docs into parity with that table so every surface — user-facing features reference,
SOC 2 compliance mapping, connectors guide, advanced demo architecture diagram,
testing guide, and in-line architecture prose — reflects the same 8 always-on + 4
opt-in topology.

Touches:
- docs/architecture.md: 2 inline ordinal references (9th / 8th loop) replaced with
  descriptive names (opt-in cloud discovery / opt-in endpoint health), cross-linked
  to the authoritative table to prevent future ordinal rot.
- docs/features.md: metric row (7 → 12), inline reference to 9th loop, and full
  scheduler table expanded to include Always-on column + env vars + I-001/I-003/I-005
  refs.
- docs/compliance-soc2.md: background scheduler monitoring bullets expanded to list
  all 12 loops with env vars + I-series refs; table row updated with 8 always-on +
  4 opt-in summary.
- docs/connectors.md: three inline ordinals (7th/6th/9th loop) replaced with
  descriptive names, cross-linked to architecture.md.
- docs/demo-advanced.md: Mermaid SCHED node label updated from '7 background loops'
  to '12 background loops (8 always-on + 4 opt-in)'.
- docs/testing-guide.md: Test 20.1.1 header + grep pattern expanded to include
  job-retry / job-timeout / notification-retry / digest / endpoint-health /
  cloud-discovery loops; sign-off chart row label updated.

Pure documentation reconciliation. No code changes. Master HEAD pre-commit: 6e646e0.
2026-04-20 02:51:34 +00:00
shankar0123 675b87ba63 I-005: notification retry loop + dead-letter queue
Critical alerts can no longer be silently dropped by a transient
notifier failure. Failed notification attempts now ride an exponential
backoff retry loop, with a 5-attempt budget before promotion to the
dead-letter queue for operator intervention.

Schema (migration 000016, idempotent):
- retry_count INTEGER NOT NULL DEFAULT 0
- next_retry_at TIMESTAMPTZ
- last_error TEXT
- idx_notification_events_retry_sweep partial index
  (next_retry_at) WHERE status='failed' AND next_retry_at IS NOT NULL
  Dead rows clear next_retry_at so the index stops matching them.

Service contract:
- NotificationService.RetryFailedNotifications drives 2^n-minute
  exponential backoff capped at 1h (notifRetryBackoffCap) with
  5-attempt budget (notifRetryMaxAttempts).
- Exhaustion (RetryCount >= notifRetryMaxAttempts-1) promotes to
  status='dead' via MarkAsDead.
- Non-terminal failures record via RecordFailedAttempt.
- Success path promotes to 'sent' without touching retry_count
  (audit preserves "delivered on attempt N").
- Missing-notifier branch defensively promotes to 'sent' to avoid
  wedging a row on a deleted channel.
- RequeueNotification operator escape hatch atomically resets
  retry_count -> 0, next_retry_at -> NULL, last_error -> NULL,
  status -> pending via notifRepo.Requeue.

Scheduler:
- New always-on notificationRetryLoop wired into the base loop set at
  CERTCTL_NOTIFICATION_RETRY_INTERVAL (default 2m).
- sync/atomic.Bool idempotency guard.
- sync.WaitGroup shutdown drain via WaitForCompletion.

StatsService:
- SetNotifRepo setter pattern preserves 9 pre-existing
  NewStatsService call sites (main.go + stats_test.go + 8 digest
  tests) without touching the constructor signature.
- DashboardSummary.NotificationsDead populated via
  notifRepo.CountByStatus(ctx, "dead") — nil-safe when unwired
  (reports zero on systems without a notification repository).
- CountByStatus error is non-fatal (dashboard summary is
  best-effort for this field).
- Prometheus certctl_notification_dead_total counter emitted from
  the same snapshot.

Handler:
- New POST /api/v1/notifications/{id}/requeue endpoint.
- dead status surfaces to MCP + CLI.

Frontend:
- NotificationsPage gains two-tab toolbar ("All" / "Dead letter")
  with queryKey: ['notifications', activeTab] so switching tabs
  doesn't serve stale data until the 30s refetch.
- Dead rows surface "Retry {n}/5" + truncated last_error with
  full-text title tooltip.
- Requeue mutation wrapped as
    mutationFn: (id: string) => requeueNotification(id)
  to prevent react-query v5's positional context argument from
  leaking into the API client — pinned against future refactors
  by strict-match toHaveBeenCalledWith('notif-dead-001') in
  NotificationsPage.test.tsx:181.

Closes I-005.
2026-04-19 15:17:27 +00:00
shankar0123 0725713e19 Close I-004 (agent hard-delete cascades targets) coverage-gap finding
Operator decision answered as full soft-delete with optional forced
cascade — hard-delete is not reachable from any public surface. Prior
to this commit, DELETE /agents/{id} ran a plain `DELETE FROM agents`
whose schema-level `ON DELETE CASCADE` on deployment_targets.agent_id
silently wiped every target, orphaning certs and aborting in-flight
jobs. The finding closure reshapes the agent-removal contract around
soft retirement with explicit preflight counts, an opt-in cascade
gated by a mandatory reason, and unconditional protection for the
four reserved sentinel agents used by discovery sources.

Schema — migration 000015:
  migrations/000015_agent_retire.up.sql flips
  deployment_targets_agent_id_fkey from ON DELETE CASCADE to ON DELETE
  RESTRICT, so a stray `DELETE FROM agents` now errors at the DB
  boundary instead of quietly destroying targets. Both `agents` and
  `deployment_targets` grow a retired_at TIMESTAMPTZ + retired_reason
  TEXT pair (TEXT not VARCHAR so operator comments are never
  truncated), indexed via partial indexes WHERE retired_at IS NOT
  NULL. The migration is self-healing (ADD COLUMN IF NOT EXISTS, DROP
  CONSTRAINT IF EXISTS then ADD CONSTRAINT, CREATE INDEX IF NOT
  EXISTS) so repeated runs against partially-migrated databases
  converge. migrations/000015_agent_retire.down.sql restores CASCADE
  and drops the new columns for clean rollback. A dedicated
  repository-layer testcontainers test
  (internal/repository/postgres/migration_000015_test.go) asserts the
  before/after FK action, column presence, index presence, and
  round-trip idempotency under up→down→up.

Domain — sentinel guard + dependency counts:
  internal/domain/connector.go gains IsRetired() on Agent, the
  exported SentinelAgentIDs slice listing server-scanner,
  cloud-aws-sm, cloud-azure-kv, cloud-gcp-sm verbatim (matching the
  four reserved IDs documented in CLAUDE.md and created at startup in
  cmd/server/main.go), IsSentinelAgent(id string) predicate,
  AgentDependencyCounts{ActiveTargets, ActiveCertificates,
  PendingJobs} with a HasDependencies() method, and ActorTypeAgent /
  ActorTypeSystem enum values used by audit emission downstream.
  Coverage locked down by internal/domain/connector_test.go.

Service — 8-step ordered contract:
  internal/service/agent_retire.go:RetireAgent(ctx, id, actor,
  opts{Force, Reason}) enforces a fixed execution order:
  (1) sentinel guard — IsSentinelAgent(id) returns ErrAgentIsSentinel
      unconditionally; force=true does NOT bypass it.
  (2) fetch — ErrAgentNotFound on miss.
  (3) idempotency — if IsRetired() already, return
      AgentRetirementResult{AlreadyRetired: true} with no new audit
      event and no state change (safe to replay from flaky clients).
  (4) preflight counts — collectAgentDependencyCounts runs
      ActiveTargets, ActiveCertificates, PendingJobs sequentially
      (not in parallel; keeps the per-query timeout predictable and
      matches the repo's existing call-chain shape).
  (5) force-reason guard — opts.Force=true with empty Reason returns
      ErrForceReasonRequired (wired into the 400 status surface).
  (6) dependency guard — HasDependencies() with opts.Force=false
      returns BlockedByDependenciesError{Counts} (wired into the 409
      body with per-bucket counts).
  (7) mutation — single pinned retiredAt := time.Now(); agent
      retirement first, then cascade target retirement if opts.Force,
      all under the repo's single transaction so the two retired_at
      stamps match to the second.
  (8) best-effort audit — agent_retired always; agent_retirement_
      cascaded additionally on the force path. Actor is whatever the
      handler resolves from the request; actor type is mapped by
      resolveActorType (system/agent-prefix→Agent/else→User). Audit
      emission failures are logged via slog.Error but do not abort
      the retirement (matches the house convention used by every
      other scheduler-emitted event).

  BlockedByDependenciesError implements Error() as
  "active_targets=%d, active_certificates=%d, pending_jobs=%d" and
  Unwrap() → ErrBlockedByDependencies. The single struct satisfies
  errors.Is via Unwrap (used by scheduler-level tests) and errors.As
  via the concrete type (used by the handler to fish out Counts for
  the 409 body). ListRetiredAgents(page, perPage) adds a separate
  paginated accessor with page<1→1 and perPage<1→50 normalization so
  retired rows are queryable without polluting the default agent
  listing.

  Sentinel guard coverage is asymmetric by design: all four reserved
  IDs are protected, and force=true cannot override. Regression tests
  in internal/service/agent_retire_test.go assert each of the eight
  steps in order, plus sentinel bypass attempts and idempotency
  replay.

Handler + router — status-code surface:
  internal/api/handler/agents.go:RetireAgent exposes seven status
  codes on DELETE /agents/{id}:
    200 on a fresh retirement (body echoes AgentRetirementResult).
    204 on idempotent replay (AlreadyRetired=true; no new audit).
    400 on ErrForceReasonRequired.
    403 on ErrAgentIsSentinel.
    404 on ErrAgentNotFound.
    409 on BlockedByDependenciesError, with a custom body shape
        {error, counts{active_targets, active_certificates,
        pending_jobs}} that bypasses the default ErrorWithRequestID
        envelope so callers get the per-bucket numbers directly.
    500 on any other error.
  Heartbeat HandleHeartbeat returns 410 Gone when the agent is
  retired (ErrAgentRetired), signalling the agent to shut down.
  Query params `force=true` and `reason=<text>` drive the cascade
  path; both are forwarded as url.Values through the new MCP
  transport.

  internal/api/router/router.go registers GET /api/v1/agents/retired
  literal-path BEFORE /api/v1/agents/{id} — Go 1.22 ServeMux's
  literal-beats-pattern-var precedence routes "retired" to the
  paginated retired-agents listing instead of fetching a hypothetical
  agent named "retired".

Agent binary — clean shutdown on 410:
  cmd/agent/main.go gains the ErrAgentRetired sentinel, a
  retiredOnce sync.Once, and a retiredSignal chan struct{}. A
  markRetired(source, statusCode, body) helper closes the channel
  exactly once; the Run() select loop observes the close and returns
  ErrAgentRetired; main() matches via errors.Is(err, ErrAgentRetired)
  and exits cleanly instead of spinning in the heartbeat retry loop.
  The 410 Gone surface is therefore terminal for the agent process.

MCP transport:
  internal/mcp/client.go adds Client.DeleteWithQuery(path, query),
  a new additive transport method. Client.Delete is path-only; without
  this method the retire tool would silently drop `force` and `reason`,
  turning every cascade retire into a default soft-retire. The new
  method shares do()'s 204 normalization and 4xx/5xx error
  propagation so tool authors get one contract.
  internal/mcp/tools.go + internal/mcp/types.go expose the
  retire_agent tool with Force+Reason inputs wired through
  DeleteWithQuery.

CLI:
  cmd/cli/main.go + internal/cli/client.go add two CLI surfaces:
  `agents list --retired` (client-side strip of --retired then
  delegation to ListRetiredAgents, sharing --page/--per-page parsing
  with the default listing) and `agents retire <id> [--force --reason
  "…"]` (mirrors ErrForceReasonRequired — force without reason is
  rejected client-side before the request is sent). JSON + table
  output modes both honor the new columns.

Frontend:
  web/src/pages/AgentsPage.tsx surfaces retired/retire affordances.
  web/src/api/client.ts + web/src/api/types.ts expose the retire
  endpoint and the retired-listing. 4 new Vitest regression cases.

OpenAPI:
  api/openapi.yaml documents DELETE /agents/{id} with all seven
  status codes, 410 on heartbeat, and the 409 per-bucket body shape.

Regression coverage (six new test files, all green):
  internal/service/agent_retire_test.go           — 8-step contract + sentinel guards
  internal/api/handler/agent_retire_handler_test.go — 7-status-code surface + 410 heartbeat
  internal/mcp/retire_agent_test.go               — DeleteWithQuery wire-through
  internal/cli/agent_retire_test.go               — --retired listing + --force/--reason pairing
  internal/repository/postgres/migration_000015_test.go — FK flip + columns + indexes + up↔down
  internal/domain/connector_test.go               — IsRetired, IsSentinelAgent, SentinelAgentIDs, HasDependencies

Files:
  api/openapi.yaml                                — DELETE + 410 + 409 body shape
  cmd/agent/main.go                               — ErrAgentRetired, markRetired, retiredSignal
  cmd/cli/main.go                                 — handleAgents list/get/retire dispatch
  docs/architecture.md, docs/concepts.md,
    docs/testing-guide.md                         — retirement contract narrative
  internal/api/handler/agents.go                  — RetireAgent, status surface, 410 on heartbeat
  internal/api/handler/agent_handler_test.go      — extended coverage
  internal/api/handler/agent_retire_handler_test.go — new
  internal/api/router/router.go                   — /agents/retired before /agents/{id}
  internal/cli/agent_retire_test.go               — new
  internal/cli/client.go                          — ListRetiredAgents + RetireAgent
  internal/domain/connector.go                    — IsRetired, SentinelAgentIDs,
                                                    IsSentinelAgent, AgentDependencyCounts,
                                                    ActorTypeAgent/System
  internal/domain/connector_test.go               — new
  internal/integration/lifecycle_test.go          — retirement fixture
  internal/mcp/client.go                          — DeleteWithQuery additive transport
  internal/mcp/retire_agent_test.go               — new
  internal/mcp/tools.go, internal/mcp/types.go    — retire_agent tool + Force/Reason inputs
  internal/repository/interfaces.go               — AgentRepository retirement methods
  internal/repository/postgres/agent.go           — retire + cascade target retire + counts
  internal/repository/postgres/migration_000015_test.go — new
  internal/service/agent.go                       — wire into AgentService surface
  internal/service/agent_retire.go                — new 8-step contract
  internal/service/agent_retire_test.go           — new
  internal/service/deployment.go                  — skip retired agents
  internal/service/target.go                      — skip retired agents
  internal/service/testutil_test.go               — shared mocks extended
  migrations/000015_agent_retire.up.sql           — new
  migrations/000015_agent_retire.down.sql         — new
  web/src/api/client.ts, types.ts + tests         — retire endpoint wiring
  web/src/pages/AgentsPage.tsx                    — retire UI
2026-04-19 05:24:00 +00:00
shankar0123 3287e174dc Unify API auth + RFC-compliant CRL/OCSP (M-002 + M-003 + M-006, auto-closes M-001)
Closes the remaining P1 gaps from coverage-gap-audit.md (M-001/M-002/M-003/M-006)
on top of the C-001/C-002 ownership + agent-FK contract fixes landed in
a53a4b8. The work lands as a single commit spanning server, docs, tests,
and the React client.

M-002 — Named API keys with per-key actor propagation
  * Migration 000014 adds the 'api_keys' table (id, name, hash,
    principal, role, created_at, last_used_at, disabled_at) so every
    credential carries an identifiable principal instead of the
    opaque 'anonymous'/'api-key' sentinel.
  * Auth middleware now rotates through configured keys, performs
    constant-time hash comparison, stamps 'last_used_at', and emits
    an actor struct via contextWithActor(). The audit middleware,
    bulk-revocation handler, approval handlers, and MCP tool layer
    now read the principal off the context and persist it on every
    audit_events row.
  * Regression coverage:
      - internal/api/middleware/audit_test.go — actor propagation,
        principal redaction for disabled keys, anonymous fallback for
        unauthenticated endpoints.
      - internal/api/handler/bulk_revocation_handler_test.go,
        job_handler_test.go — principal-on-audit assertions.

M-003 — Authorization gates (Phase B)
  * Approval handler rejects self-approval / self-rejection with 403
    when the actor principal equals the job's requested_by field.
  * Bulk revocation is gated behind the 'admin' role; operators and
    viewers receive 403.
  * Regression coverage:
      - internal/service/job_test.go — TestApproveJob_NotSelf,
        TestRejectJob_NotSelf.
      - internal/api/handler/bulk_revocation_handler_test.go —
        TestBulkRevoke_RequiresAdmin, TestBulkRevoke_AdminSucceeds.

M-006 — RFC-compliant CRL/OCSP on the unauthenticated .well-known mux
  * Per RFC 8615, relying parties cannot reasonably be asked to
    authenticate against the issuing certctl instance to retrieve
    revocation material. CRL and OCSP move off the authenticated
    '/api/v1/crl*' and '/api/v1/ocsp/*' paths onto:
        GET /.well-known/pki/crl/{issuer_id}
            Content-Type: application/pkix-crl   (RFC 5280 §5)
        GET /.well-known/pki/ocsp/{issuer_id}/{serial}
            Content-Type: application/ocsp-response  (RFC 6960)
  * Non-standard JSON CRL shape is removed; only DER is served.
  * Short-lived certificate exemption (profile TTL < 1h → skip
    CRL/OCSP) is preserved; the response simply omits the serial.
  * Routes are registered on the unauthenticated 'finalHandler' mux
    in cmd/server/main.go alongside EST ('/.well-known/est/*') and
    SCEP ('/scep'). Legacy authenticated paths return 404.
  * Regression coverage:
      - internal/api/handler/certificate_handler_test.go — content
        type, DER parseability, 404 for unknown issuer.
      - internal/api/handler/adversarial_path_test.go — unauthenticated
        access asserted for CRL, OCSP, EST, SCEP.
      - internal/api/router/router_test.go — route-table assertion
        that '.well-known/pki/*', '.well-known/est/*', and '/scep' are
        mounted on the unauthenticated branch.

M-001 — Auto-closed by M-002
  EST and SCEP were already registered on the unauthenticated
  'finalHandler' mux; the router comment at
  internal/api/router/router.go:247 now matches reality. The
  adversarial-path tests above lock the behavior in.

Verification (all gates green):
  * go vet ./...                                           — clean
  * go build ./...                                         — ok
  * go test -short ./... (55+ packages)                    — all pass
  * web/ : npm test (225 Vitest tests)                     — all pass
  * web/ : npx tsc --noEmit                                — clean
  * grep sweep for '/api/v1/(crl|ocsp)' — 13 surviving hits,
    all intentional M-006 tombstone/relocation comments.

Documentation:
  * coverage-gap-audit.md — status flips M-001/M-002/M-003/M-006 →
    Fixed, with per-finding resolution paragraphs citing regression
    test IDs. (Audit file lives outside this repo; see cowork root.)
  * CLAUDE.md Project Status line updated with the auth-unification
    closure note.
  * docs/features.md, docs/architecture.md, docs/quickstart.md,
    docs/concepts.md, docs/connectors.md, docs/test-env.md,
    docs/testing-guide.md, docs/compliance-*.md, docs/demo-advanced.md
    — refreshed for the new '.well-known/pki/*' namespace and named
    API keys.
  * api/openapi.yaml — documents the new unauthenticated endpoints
    and removes the legacy '/api/v1/crl*' + '/api/v1/ocsp/*' paths.

.gitignore: adds '/.gocache/' and '/.gomodcache/' for the session-
scoped Go caches so they never enter the tree.
2026-04-18 18:17:41 +00:00
shankar0123 5567d4b411 feat(M47): add Kubernetes Secrets target + AWS ACM PCA issuer connectors
Implement both M47 connectors with full cross-layer wiring:

Kubernetes Secrets target: DNS-1123 validation, kubernetes.io/tls Secret
create-or-update, chain concatenation, serial number validation, Helm
RBAC gating. 18 tests.

AWS ACM Private CA issuer: synchronous issuance (like Vault), ARN regex
validation, RFC 5280 revocation reason mapping, CA cert retrieval,
factory + env var seeding. 23 tests.

Cross-cutting: domain types, service validation, config, factory, agent
dispatch, frontend (TargetsPage, issuerTypes), OpenAPI, seed data, Helm
chart, connectors docs, README. Testing docs (testing-guide, qa-test-guide,
qa_test.go) with Parts thematically integrated near related connectors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 20:21:09 -04:00
shankar0123 fd94e0bd19 docs: comprehensive testing guide audit — expand thin Parts, add 11 new connector/feature test sections
Refactored testing-guide.md from V2.0 (42 Parts, 444 tests) to V2.1 (52 Parts, 507 tests):

- Expanded Part 11 (ARI) and Part 19 (Agent Work Routing) with What/Why intro
  paragraphs and per-test annotations explaining the production impact
- Replaced Part 40 (Documentation) passive table with 8 executable verification
  tests (README screenshots, issuer/target type matching, OpenAPI parity, etc.)
- Added Part 39 benchmark tests for Prometheus endpoint and audit trail queries
- Added 11 new Part sections (42-52) covering all previously untested features:
  Envoy, Postfix/Dovecot, SSH, WinCertStore, JavaKeystore, Digest Email,
  Dynamic Issuer/Target Config, Onboarding Wizard, ACME Profiles, Helm Chart
- Fixed stale TOC entries (regenerated from actual headings)
- Removed duplicate TOC block left from previous reorder
- Added sign-off chart entries for all new Parts
- Updated summary: 144 auto (passed) + 88 auto (pending) + 5 skipped + 270 manual = 507 total

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 00:43:05 -04:00
shankar0123 cc03f55006 docs: comprehensive documentation audit — fix stale counts, V2/V3 matrix, connector status
- features.md: Fix Feature Matrix to correctly show all V2 Free features
  (F5/IIS/WinCertStore/JavaKeystore as Implemented, not Stub; Vault/DigiCert/
  Sectigo/GoogleCAS as V2 Free, not V3 Paid). Add missing shipped features
  (EST, verification, export, S/MIME, ARI, digest, Helm, onboarding). Update
  issuer count to 9, target count to 13.
- architecture.md: Fix F5/IIS from "interface only, implementation planned"
  to implemented. Add all 13 target connectors to built-in targets list.
- why-certctl.md: Add Sectigo and Google CAS to issuer list (7→9). Fix
  target count (10→13). Remove hardcoded endpoint/operation counts.
- connectors.md: Fix F5 BIG-IP TOC entry from "Interface Only" to
  "Implemented". Remove dead "Planned Issuers" TOC link.
- README.md: Remove competitor product names (CertKit, KeyTalk). Remove
  hardcoded dashboard page count. Remove hardcoded endpoint counts. Fix V4
  roadmap to remove already-shipped issuers (Sectigo, Google CAS).
- Remove hardcoded MCP tool counts (78/80) across 8 files (mcp.md,
  architecture.md, features.md, testing-guide.md, concepts.md, quickstart.md,
  demo-advanced.md, why-certctl.md). Replace with "REST API exposed via MCP"
  to avoid future drift.
- quickstart.md: Docker Compose environments table (from previous session).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 21:33:12 -04:00
shankar0123 f92c997a50 feat(M45): ACME certificate profile selection, ARI RFC 9773 renumber, 45-day renewal positioning
Three related ACME ecosystem changes shipped as a single milestone:

1. ACME Certificate Profile Selection: Custom JWS-signed newOrder POST with
   `profile` field (e.g., `tlsserver`, `shortlived` for 6-day certs) bypassing
   acme.Client.AuthorizeOrder() since golang.org/x/crypto lacks profile support.
   ES256 JWS signing with kid mode, nonce management, directory discovery.
   Empty profile delegates to standard library path (zero behavior change).
   Configurable via CERTCTL_ACME_PROFILE env var. GUI: profile dropdown on
   ACME issuer config.

2. ARI RFC 9702 → 9773 Renumber: All 25+ references updated across Go source,
   docs, README, and examples. Zero remaining occurrences of RFC 9702.

3. 45-Day / Short-Lived Certificate Positioning: 5 domain tests validating
   renewal thresholds against SC-081v3 validity reduction timeline (200→100→47
   days) and Let's Encrypt 45-day/6-day profiles. ARI (RFC 9773) is the
   expected renewal path for 6-day shortlived certs.

New tests: 13 profile + 5 domain threshold + 1 frontend = 19 new tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 13:52:13 -04:00
shankar0123 2a14a1da01 feat(M40): F5 BIG-IP target connector via iControl REST
Replace 190-line stub with full iControl REST implementation (~580 lines).
Token auth with 401 auto-retry, file upload + crypto object install,
transaction-based atomic SSL profile updates, cleanup on failure.
Injectable F5Client interface for cross-platform testing. 32 tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 22:26:58 -04:00
shankar0123 5a53b648b1 feat(M44): Google CAS issuer connector
Google Cloud Certificate Authority Service integration via REST API
with OAuth2 service account auth (JWT→access token). Synchronous
issuance model, CA pool selection, mutex-guarded token caching,
revocation with RFC 5280 reason mapping. No Google SDK dependency —
all stdlib. 19 tests with httptest mock OAuth2 + CAS API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 21:25:34 -04:00
shankar0123 3a11e447cf feat(M43): Sectigo SCM issuer connector
Implement Sectigo Certificate Manager REST API connector with async
order model (enroll → poll → collect PEM), 3-header auth, DV/OV/EV
support, collect-not-ready (400/-183) graceful handling, and RFC 5280
revocation reason mapping. 20 tests with httptest mock API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 21:01:14 -04:00
shankar0123 4c3b7cbb16 docs: fix stale references, seed data case bugs, and convert ASCII diagrams to Mermaid
Audit all docs and examples against current codebase state. Fix seed_demo.sql
domain constant casing (IssuerType, TargetType, AgentStatus) that would cause
agent dispatch failures. Fix example docker-compose health endpoints (/health
not /api/v1/health) and env var names (CERTCTL_DATABASE_URL). Update connector
counts, test numbers, and planned→implemented status across docs. Convert 3
ASCII flow diagrams to Mermaid.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 16:11:42 -04:00
shankar0123 8b52da6aef feat(M39): IIS target connector + README overhaul
Implement full IIS target connector with PEM-to-PFX conversion via
go-pkcs12, PowerShell-based deployment (Import-PfxCertificate, IIS
binding management), SHA-1 thumbprint computation, and SNI support.
Injectable PowerShellExecutor interface enables cross-platform testing.
Regex-validated config fields prevent PowerShell injection. 28 tests.

Restructure README from 563 to 313 lines: outcome-focused feature
descriptions, "Who Is This For" persona section, examples promoted
above the fold, configuration/API/security reference moved to docs.
All numbers verified against repo (25 GUI pages, 97 OpenAPI ops,
CI thresholds service 55%/handler 60%/domain 40%/middleware 30%).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 20:27:27 -04:00
shankar0123 6c8d4eca40 feat: frontend audit fixes, README accuracy pass, doc updates
Frontend audit (10 categories): lifecycle fields in types, new API
functions (CRL, OCSP, deployments, updateIssuer/Target, getPolicy),
issuer/owner/profile filters on CertificatesPage, last_renewal_at
column, error_message column on JobsPage, full crypto policy UI on
ProfilesPage (key algorithms, EKUs, SAN patterns), key info + CA
badge on DiscoveryPage, edit modal on TargetDetailPage, tags field
on certificate creation, darwin→macOS mapping on AgentFleetPage.
211 Vitest tests passing.

README accuracy: test counts (1300+ Go, 211 frontend), page count
(24), demo data (32 certs, 7 issuers, 180 days), endpoint count
(97), MCP tools (80), CLI subcommands (10), moved shipped items
out of "Coming in v2.1.0".

Docs: architecture.md diagrams updated (Vault PKI, DigiCert,
Traefik, Caddy added), features.md Vault/DigiCert status updated.
Version bumped to v2.0.20. cli binary removed from git tracking.
Testing guide Part 41 added (12 auto + 9 manual tests).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 22:10:45 -04:00
shankar0123 836534f2a7 feat: add issuer catalog page with type discovery + fix cert creation defaults (M33)
Issuer Catalog (M33):
- Shared issuer type config (issuerTypes.ts) with 6 supported + 2 coming-soon types
- Composable wizard components (TypeSelector, ConfigForm, ConfigDetailModal)
- Catalog card layout with Connected/Available/Coming Soon badges
- VaultPKI and DigiCert added to create wizard with full config fields
- ACME EAB fields (eab_kid, eab_hmac with sensitive flag)
- Issuer type filter dropdown on configured issuers table
- Config detail modal replacing 60-char truncation
- IssuerDetailPage uses shared typeLabels/redactConfig, Edit button, enabled/disabled status
- StatusBadge extended with Enabled/Disabled styles
- 2 new frontend tests (VaultPKI + DigiCert create payload verification)

Bug fixes:
- CertificateService.CreateCertificate now defaults Status to Pending and Tags to
  empty map when not set (DB column DEFAULTs only apply when columns are omitted
  from INSERT, but our repo always includes all columns)
- CreateCertificate handler now logs actual error via slog.Error before returning
  generic 500, enabling root cause debugging

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 18:58:23 -04:00
shankar0123 6375909591 feat: add Vault PKI and DigiCert CertCentral issuer connectors (M32 + M37)
Vault PKI: synchronous issuance via /v1/{mount}/sign/{role}, token auth,
revocation, CA cert retrieval, 14 tests. DigiCert CertCentral: async order
model (submit → poll → download), X-DC-DEVKEY auth, OV/EV support, PEM
bundle parsing, 16 tests. Both conditionally registered based on env vars.
Includes OpenAPI enum updates, seed data, connector docs, architecture docs,
README badges, and testing guide sign-off (Parts 38 + 39, 12 automated
smoke test assertions all passing).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 17:19:46 -04:00
shankar0123 a6515b4323 feat(Pre-2.1.0-E): GUI completeness — 5 new pages, clickable nav, verification badges
Wire all remaining backend features to the frontend GUI:

New pages:
- DigestPage: preview digest HTML via iframe + send with confirmation
- ObservabilityPage: health status, metrics gauges, Prometheus config + live output
- JobDetailPage: full job details, verification section, timeline, audit events
- IssuerDetailPage: redacted config, test connection, issued certificates list
- TargetDetailPage: config, agent link, deployment history with verification

Existing page updates:
- JobsPage: clickable job IDs, verification column with VerificationBadge
- IssuersPage: clickable issuer names linking to detail page
- TargetsPage: clickable target names linking to detail page
- Sidebar: Digest and Observability nav items
- 5 new routes in main.tsx

API client: getJob, getIssuer, getTarget, getJobVerification, getPrometheusMetrics
Tests: 7 new Vitest tests (203 total), testing-guide Part 37 (17 manual tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 14:10:58 -04:00
shankar0123 ec0e7a3560 feat: wire ARI (RFC 9702) into renewal scheduler
CheckExpiringCertificates() now queries each issuer's ARI endpoint
before creating renewal jobs. If the CA says "not yet" (suggested
window hasn't opened), renewal is deferred. ARI errors fall back
gracefully to threshold-based logic. Audit trail records
renewal_trigger=ari when ARI drives the decision.

4 new unit tests: ShouldRenewNow, NotYet, NilFallback, ErrorFallback.
3 new smoke tests in testing-guide.md Part 35.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 12:11:42 -04:00
shankar0123 a8fc177118 fix: resolve NULL csr_pem scan errors and QA smoke test failures
Root cause: certificate_versions.csr_pem is nullable in the schema but
Go code scanned it into a plain string. Used sql.NullString in
ListVersions and GetLatestVersion to handle NULL values correctly.

Also includes: partial update fetch-merge-update pattern to prevent FK
violations, nil directory guard in discovery service, diagnostic slog
logging in handlers, export handler 422 for unparseable PEM, OpenAPI
spec corrections, MCP tool description improvements, and test fixes.

Rewrites the Release Sign-Off section in testing-guide.md to individual
test-level granularity (320 rows) with smoke test results audited and
checked off (121 pass, 5 skip, 194 manual remaining).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 00:51:18 -04:00
shankar0123 bcf2c3ae92 feat(pre-2.1.0): demo data overhaul, examples, migration guides, install script
Pre-2.1.0 adoption polish delivering all four milestones:

A) Demo Data Overhaul — seed_demo.sql rewritten with 35 certs across
   5 issuers, 8 agents, 8 targets, 50+ jobs spanning 90 days, 55+
   audit events, discovery scans, network scan targets, S/MIME cert.

B) Examples Directory — 5 turnkey docker-compose configs:
   acme-nginx, acme-wildcard-dns01, private-ca-traefik,
   step-ca-haproxy, multi-issuer.

C) Migration Guides — migrate-from-certbot.md,
   migrate-from-acmesh.md, certctl-for-cert-manager-users.md.

D) Agent Install Script — install-agent.sh with cross-platform
   support (Linux systemd + macOS launchd), release.yml updated
   for 6-target cross-compilation.

Triple-audited against codebase: 22 factual corrections applied
across docs, examples, and config (env var names, CLI flags, ports,
DNS hook interface, scheduler loop counts, license conversion date).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 18:26:58 -04:00
shankar0123 11e752ac01 docs: add v2.1.0 release gate note to README and testing guide
v2.1.0 will be tagged after all 34 manual QA sections pass.
Updates sign-off table version reference from v2.0.7 to v2.1.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 18:09:41 -04:00
shankar0123 03472072b8 test + docs: close 12 test gaps (~250 new tests) and expand testing guide to 34 parts
Implements all P0-P2 test gaps from docs/test-gap-prompt.md:
- Deployment service tests (20), target service tests (18), scheduler tests (8)
- Agent binary tests (48), CSR renewal tests (8), short-lived cert tests (7)
- Domain model tests (25), context cancellation tests (9), concurrency tests (7)
- Handler negative-path tests (23 across 5 files)
- Frontend error handling tests (86) and API client tests (7)

Expands testing-guide.md from 28 to 34 parts covering certificate export,
S/MIME/EKU, OCSP/DER CRL, body size limits, Apache/HAProxy connectors,
and sub-CA mode. Fixes stale profile count (4->5) and updates sign-off table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 17:57:25 -04:00
shankar0123 be72627aeb feat: M25 post-deployment TLS verification + M26 Traefik/Caddy targets
M25: After deploying a certificate, the agent probes the live TLS
endpoint and compares SHA-256 fingerprints to verify the correct cert
is being served. Best-effort — failures don't block deployments.
New endpoints: POST /jobs/{id}/verify, GET /jobs/{id}/verification.
Migration 000008 adds verification columns to jobs table.

M26: Traefik target connector (file provider, auto-reload) and Caddy
target connector (dual-mode: admin API hot-reload or file-based).
Both wired into agent dispatch.

Also: restructured README to highlight supported integrations (issuers,
targets, notifiers) earlier, moved API/CLI/MCP sections lower. Updated
all docs (features, connectors, architecture, testing guide, why-certctl)
and fixed integration tests for 18-param RegisterHandlers signature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 21:07:16 -04:00
shankar0123 d55807947e docs: add M24 GUI tests to testing guide (discovery, network scan, approval)
Adds Part 19.5 (approval workflow), 19.6 (discovery triage),
19.7 (network scan management) to GUI testing section. Renumbers
existing 19.5 Other Pages to 19.8 and Cross-Cutting to 19.9.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 16:12:36 -04:00
shankar0123 03593d4304 feat: wire ACME EAB into account registration + ZeroSSL auto-fetch
EAB credentials (KID + HMAC) were defined in the ACME connector config
but never wired into the acme.Account registration call. This fixes the
dead code and adds automatic EAB credential fetching for ZeroSSL — when
the directory URL is detected as ZeroSSL and no EAB credentials are
provided, certctl calls ZeroSSL's public API to get them automatically.

Changes:
- Wire EABKid/EABHmac into acme.Account.ExternalAccountBinding
- Add isZeroSSL() detection and fetchZeroSSLEAB() auto-fetch
- Add CERTCTL_ACME_EAB_KID/CERTCTL_ACME_EAB_HMAC env vars to main.go
- Add 13 ACME connector tests (config validation, EAB decode, ZeroSSL
  auto-EAB with mock servers, URL detection)
- Update docs: README, architecture, connectors, demo-advanced,
  testing-guide with EAB/auto-EAB documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 15:34:48 -04:00
shankar0123 87355c3efb docs: add table of contents to all major documentation files
Navigation menus for testing guide, architecture, concepts,
connectors, quickstart, advanced demo, and three compliance docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 23:38:28 -04:00
shankar0123 50c520e1ff feat: dashboard theme overhaul — light content area with branded teal sidebar
Complete frontend visual redesign using certctl logo color palette:
- Deep teal sidebar (#0c2e25) with prominent centered logo (64px in white pill)
- Light content area (#f0f4f8) with white cards and visible borders
- Brand colors from logo: teal (#2ea88f), blue (#3b7dd8), orange (#e8873a), green (#4ebe6e)
- Inter + JetBrains Mono typography, colored stat card top borders
- All 17 pages + 7 components updated (25 files, ~700 lines changed)
- 15 new dashboard screenshots replacing old dark theme screenshots
- Prometheus metrics e2e test added, integration test mock fixes
- Docs updated: architecture.md theme description, testing-guide.md DNS-PERSIST-01 coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 23:27:42 -04:00
shankar0123 4092bdfb1a docs: clean up testing guide intro
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 20:47:55 -04:00
shankar0123 7d14635a72 feat: add EST server (RFC 7030) for device certificate enrollment (M23)
Implement Enrollment over Secure Transport protocol with 4 endpoints under
/.well-known/est/ — cacerts (CA chain distribution), simpleenroll (initial
enrollment), simplereenroll (certificate renewal), and csrattrs (CSR
attributes). PKCS#7 certs-only wire format with hand-rolled ASN.1, accepts
both PEM and base64-encoded DER CSRs, configurable issuer and profile
binding, full audit trail. 28 new tests (18 handler + 10 service).

Also includes:
- GetCACertPEM added to issuer connector interface (all 4 issuers updated)
- EST integration tests wired into e2e test suite (13 test cases)
- QA testing guide Part 26 (15 manual EST test cases)
- All docs updated: README, features, architecture, concepts, connectors,
  quickstart, demo-advanced (endpoint counts, MCP wording, agent IDs,
  issuer interface, resource lists, OpenSSL status)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 15:31:06 -04:00
shankar0123 0f4a1b268b fix: handle 204 No Content in fetchJSON, add FK-aware delete errors, v2 screenshots
Frontend: fetchJSON now returns empty object on 204 instead of failing
to parse empty body — fixes silent delete failures across all entities.
Added onError callbacks to owner/team delete mutations to surface errors.

Backend: owner and issuer delete handlers return 409 Conflict with
descriptive messages when FK constraints block deletion, instead of
generic 500.

Added 15 v2 dashboard screenshots, updated README screenshot section,
logo asset, page count references (18→full), and QA guide with FK
constraint test coverage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 05:03:50 -04:00
shankar0123 3eb4749b4d docs: merge quickstart and demo guide into single quickstart.md
Consolidated two overlapping docs into one cohesive guide framed around
the 47-day certificate lifespan reduction. Covers setup, dashboard
walkthrough, API exploration, cert creation, discovery, CLI, MCP, demo
data reference, and a 10-step stakeholder presentation flow.

Removed demo-guide.md and updated all cross-references in README,
compliance-pci-dss.md, and testing-guide.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 04:09:03 -04:00
shankar0123 2385ab7996 docs: add V2 manual testing guide (284 tests, 25 sections)
Comprehensive QA runbook covering all V2 features with exact curl
commands, expected outputs, and unambiguous pass/fail criteria.
Linked from README documentation table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 01:42:58 -04:00