10 Commits

Author	SHA1	Message	Date
shankar0123	85e60b24ec	fix(bundle-5): Operational Liveness + Bootstrap — 4 audit findings closed ... Closes Audit-2026-04-25 H-006 (High), H-007 (High), M-011 (Medium), L-006 (Low — verified-already-closed via C-1 master closure in v2.0.54). Hardens the orchestrator-facing surface — k8s probes, agent enrollment, shutdown audit drain, scheduler config plumbing. What changed - internal/api/handler/health.go — split contract: * /health stays shallow 200 (k8s liveness — process alive) * /ready accepts *sql.DB; runs db.PingContext(2s); 503 on failure * Nil DB path returns 200 + db=not_configured (test fixtures) - internal/api/handler/agent_bootstrap.go (NEW) — verifyBootstrapToken: * empty expected = warn-mode pass-through * non-empty = `Authorization: Bearer <token>` required * crypto/subtle.ConstantTimeCompare; length-mismatch path runs dummy compare to keep timing uniform * ErrBootstrapTokenInvalid sentinel - internal/api/handler/agents.go — RegisterAgent calls verifyBootstrapToken BEFORE body parse so unauth probes don't even allocate a JSON decoder - internal/config/config.go — two new env vars: * CERTCTL_AGENT_BOOTSTRAP_TOKEN (Auth.AgentBootstrapToken) * CERTCTL_AUDIT_FLUSH_TIMEOUT_SECONDS (Server.AuditFlushTimeoutSeconds) - cmd/server/main.go — 3 changes: * pass *sql.DB into NewHealthHandler (H-006) * pass cfg.Auth.AgentBootstrapToken into NewAgentHandler (H-007) * configurable shutdown audit-flush timeout (M-011) * one-shot startup WARN when bootstrap token unset (deprecation) - new tests: agent_bootstrap_test.go (full deny/accept/warn-mode coverage, constant-time compare path, length-mismatch); health_test.go extended with /ready DB-probe failure (503), nil-DB pass-through, /health-shallow L-006 verified - cmd/server/main.go:557 already calls sched.SetShortLivedExpiryCheckInterval(cfg.Scheduler.ShortLivedExpiryCheckInterval) per the C-1 master closure in v2.0.54. Bundle 5 confirms; no code change. Threat model: TB-1 (operator/orchestrator), TB-2 (Agent↔Server). - CWE-754 (Improper Check for Unusual or Exceptional Conditions) for H-006 - CWE-306 + CWE-288 (Missing Authentication for Critical Function) for H-007 Verification - go vet ./... → clean - go build ./... → clean - go test -short -count=1 ./... → all packages pass - targeted Bundle-5 regressions → all pass - npx tsc --noEmit (web) → clean - npx vitest run (web) → in-flight (sandbox 45s ceiling exceeded; no failure markers in dot stream; no frontend changes in this bundle so no regression risk) - python3 yaml.safe_load(api/openapi.yaml) → 89 paths Backward compatibility - Bootstrap token defaults to empty (warn-mode) — existing demo deployments unaffected. Server logs deprecation WARN; v2.2.0 will require it. - Audit flush timeout default 30s preserves prior behaviour. - Helm chart already routes readiness probe to /ready (no chart change needed); now /ready actually probes the DB. Bundle 5 of the 2026-04-25 comprehensive audit.	2026-04-25 23:54:18 +00:00
shankar0123	0e29c416b1	refactor(handler,repo): replace strings.Contains error dispatch with typed sentinels (S-2) ... Closes one 2026-04-24 audit finding (P2): - cat-s6-efc7f6f6bd50: 30 strings.Contains(err.Error(), ...) sites in internal/api/handler/ — brittle to repository-layer message changes, untyped against the actual failure mode. Approach (Option B from prompt design notes): - New typed sentinels in internal/repository/errors.go: ErrNotFound, ErrForeignKeyConstraint IsForeignKeyError(err) helper (the only place substring matching at the lib/pq boundary is allowed; isolates the DB-driver string knowledge to one function). - New typed sentinel in internal/domain/errors.go: ErrValidation (reserved for future per-entity validation wrappers; not yet used by all handlers). - 49 sites in internal/repository/postgres/*.go updated to wrap sql.ErrNoRows-derived errors via fmt.Errorf("...: %w", repository.ErrNotFound). - 18 not-found handler sites + 2 FK-constraint handler sites refactored to errors.Is(err, repository.ErrNotFound) / repository.IsForeignKeyError(err). - 23 inline `fmt.Errorf("X not found")` test fixtures across handler tests rewrapped to wrap repository.ErrNotFound. - test_utils.go::ErrMockNotFound rewrapped to wrap repository.ErrNotFound; renewal_policy.go closure docblock updated to reflect the new convention. - integration test mockJobRepository.Get wraps repository.ErrNotFound. CI regression guardrail: - .github/workflows/ci.yml::"Forbidden strings.Contains(err.Error()) regression guard (S-2)" greps for the three patterns ("not found", "violates foreign key", "RESTRICT") under internal/api/handler/ and fails the build on regression. Verification: - go build ./... — clean - go vet ./... — clean - go test ./... -short -count=1 — all packages pass (handler + repository + service + integration) - golangci-lint v2.11.4 run ./... — 0 issues - S-2 guardrail dry-run on post-fix tree → empty (good) - All sibling guardrails (S-1, G-3, D-1+D-2, B-1, L-1, H-1, C-1, F-1, P-1) pass Audit findings closed: - cat-s6-efc7f6f6bd50 (P2) Deferred follow-ups: - 6 domain-specific substring patterns still inline in handlers ("cannot approve", "cannot reject", "cannot be parsed", "no certificates found", "challenge password", "invalid"/ "required" validation chains in profiles + agent_groups). Each needs its own typed sentinel, scoped per service. Documented by the S-2 CI guardrail's allowlist for closure-comments only. - Per-entity not-found sentinels (Option A — ErrCertificateNotFound, ErrAgentNotFound, etc.) deferred. Generic ErrNotFound covers the current dispatch needs; per-entity precision would let handlers return entity-aware error bodies without a domain.Type field, but not blocking.	2026-04-25 17:54:14 +00:00
shankar0123	0725713e19	Close I-004 (agent hard-delete cascades targets) coverage-gap finding ... Operator decision answered as full soft-delete with optional forced cascade — hard-delete is not reachable from any public surface. Prior to this commit, DELETE /agents/{id} ran a plain `DELETE FROM agents` whose schema-level `ON DELETE CASCADE` on deployment_targets.agent_id silently wiped every target, orphaning certs and aborting in-flight jobs. The finding closure reshapes the agent-removal contract around soft retirement with explicit preflight counts, an opt-in cascade gated by a mandatory reason, and unconditional protection for the four reserved sentinel agents used by discovery sources. Schema — migration 000015: migrations/000015_agent_retire.up.sql flips deployment_targets_agent_id_fkey from ON DELETE CASCADE to ON DELETE RESTRICT, so a stray `DELETE FROM agents` now errors at the DB boundary instead of quietly destroying targets. Both `agents` and `deployment_targets` grow a retired_at TIMESTAMPTZ + retired_reason TEXT pair (TEXT not VARCHAR so operator comments are never truncated), indexed via partial indexes WHERE retired_at IS NOT NULL. The migration is self-healing (ADD COLUMN IF NOT EXISTS, DROP CONSTRAINT IF EXISTS then ADD CONSTRAINT, CREATE INDEX IF NOT EXISTS) so repeated runs against partially-migrated databases converge. migrations/000015_agent_retire.down.sql restores CASCADE and drops the new columns for clean rollback. A dedicated repository-layer testcontainers test (internal/repository/postgres/migration_000015_test.go) asserts the before/after FK action, column presence, index presence, and round-trip idempotency under up→down→up. Domain — sentinel guard + dependency counts: internal/domain/connector.go gains IsRetired() on Agent, the exported SentinelAgentIDs slice listing server-scanner, cloud-aws-sm, cloud-azure-kv, cloud-gcp-sm verbatim (matching the four reserved IDs documented in CLAUDE.md and created at startup in cmd/server/main.go), IsSentinelAgent(id string) predicate, AgentDependencyCounts{ActiveTargets, ActiveCertificates, PendingJobs} with a HasDependencies() method, and ActorTypeAgent / ActorTypeSystem enum values used by audit emission downstream. Coverage locked down by internal/domain/connector_test.go. Service — 8-step ordered contract: internal/service/agent_retire.go:RetireAgent(ctx, id, actor, opts{Force, Reason}) enforces a fixed execution order: (1) sentinel guard — IsSentinelAgent(id) returns ErrAgentIsSentinel unconditionally; force=true does NOT bypass it. (2) fetch — ErrAgentNotFound on miss. (3) idempotency — if IsRetired() already, return AgentRetirementResult{AlreadyRetired: true} with no new audit event and no state change (safe to replay from flaky clients). (4) preflight counts — collectAgentDependencyCounts runs ActiveTargets, ActiveCertificates, PendingJobs sequentially (not in parallel; keeps the per-query timeout predictable and matches the repo's existing call-chain shape). (5) force-reason guard — opts.Force=true with empty Reason returns ErrForceReasonRequired (wired into the 400 status surface). (6) dependency guard — HasDependencies() with opts.Force=false returns BlockedByDependenciesError{Counts} (wired into the 409 body with per-bucket counts). (7) mutation — single pinned retiredAt := time.Now(); agent retirement first, then cascade target retirement if opts.Force, all under the repo's single transaction so the two retired_at stamps match to the second. (8) best-effort audit — agent_retired always; agent_retirement_ cascaded additionally on the force path. Actor is whatever the handler resolves from the request; actor type is mapped by resolveActorType (system/agent-prefix→Agent/else→User). Audit emission failures are logged via slog.Error but do not abort the retirement (matches the house convention used by every other scheduler-emitted event). BlockedByDependenciesError implements Error() as "active_targets=%d, active_certificates=%d, pending_jobs=%d" and Unwrap() → ErrBlockedByDependencies. The single struct satisfies errors.Is via Unwrap (used by scheduler-level tests) and errors.As via the concrete type (used by the handler to fish out Counts for the 409 body). ListRetiredAgents(page, perPage) adds a separate paginated accessor with page<1→1 and perPage<1→50 normalization so retired rows are queryable without polluting the default agent listing. Sentinel guard coverage is asymmetric by design: all four reserved IDs are protected, and force=true cannot override. Regression tests in internal/service/agent_retire_test.go assert each of the eight steps in order, plus sentinel bypass attempts and idempotency replay. Handler + router — status-code surface: internal/api/handler/agents.go:RetireAgent exposes seven status codes on DELETE /agents/{id}: 200 on a fresh retirement (body echoes AgentRetirementResult). 204 on idempotent replay (AlreadyRetired=true; no new audit). 400 on ErrForceReasonRequired. 403 on ErrAgentIsSentinel. 404 on ErrAgentNotFound. 409 on BlockedByDependenciesError, with a custom body shape {error, counts{active_targets, active_certificates, pending_jobs}} that bypasses the default ErrorWithRequestID envelope so callers get the per-bucket numbers directly. 500 on any other error. Heartbeat HandleHeartbeat returns 410 Gone when the agent is retired (ErrAgentRetired), signalling the agent to shut down. Query params `force=true` and `reason=<text>` drive the cascade path; both are forwarded as url.Values through the new MCP transport. internal/api/router/router.go registers GET /api/v1/agents/retired literal-path BEFORE /api/v1/agents/{id} — Go 1.22 ServeMux's literal-beats-pattern-var precedence routes "retired" to the paginated retired-agents listing instead of fetching a hypothetical agent named "retired". Agent binary — clean shutdown on 410: cmd/agent/main.go gains the ErrAgentRetired sentinel, a retiredOnce sync.Once, and a retiredSignal chan struct{}. A markRetired(source, statusCode, body) helper closes the channel exactly once; the Run() select loop observes the close and returns ErrAgentRetired; main() matches via errors.Is(err, ErrAgentRetired) and exits cleanly instead of spinning in the heartbeat retry loop. The 410 Gone surface is therefore terminal for the agent process. MCP transport: internal/mcp/client.go adds Client.DeleteWithQuery(path, query), a new additive transport method. Client.Delete is path-only; without this method the retire tool would silently drop `force` and `reason`, turning every cascade retire into a default soft-retire. The new method shares do()'s 204 normalization and 4xx/5xx error propagation so tool authors get one contract. internal/mcp/tools.go + internal/mcp/types.go expose the retire_agent tool with Force+Reason inputs wired through DeleteWithQuery. CLI: cmd/cli/main.go + internal/cli/client.go add two CLI surfaces: `agents list --retired` (client-side strip of --retired then delegation to ListRetiredAgents, sharing --page/--per-page parsing with the default listing) and `agents retire <id> [--force --reason "…"]` (mirrors ErrForceReasonRequired — force without reason is rejected client-side before the request is sent). JSON + table output modes both honor the new columns. Frontend: web/src/pages/AgentsPage.tsx surfaces retired/retire affordances. web/src/api/client.ts + web/src/api/types.ts expose the retire endpoint and the retired-listing. 4 new Vitest regression cases. OpenAPI: api/openapi.yaml documents DELETE /agents/{id} with all seven status codes, 410 on heartbeat, and the 409 per-bucket body shape. Regression coverage (six new test files, all green): internal/service/agent_retire_test.go — 8-step contract + sentinel guards internal/api/handler/agent_retire_handler_test.go — 7-status-code surface + 410 heartbeat internal/mcp/retire_agent_test.go — DeleteWithQuery wire-through internal/cli/agent_retire_test.go — --retired listing + --force/--reason pairing internal/repository/postgres/migration_000015_test.go — FK flip + columns + indexes + up↔down internal/domain/connector_test.go — IsRetired, IsSentinelAgent, SentinelAgentIDs, HasDependencies Files: api/openapi.yaml — DELETE + 410 + 409 body shape cmd/agent/main.go — ErrAgentRetired, markRetired, retiredSignal cmd/cli/main.go — handleAgents list/get/retire dispatch docs/architecture.md, docs/concepts.md, docs/testing-guide.md — retirement contract narrative internal/api/handler/agents.go — RetireAgent, status surface, 410 on heartbeat internal/api/handler/agent_handler_test.go — extended coverage internal/api/handler/agent_retire_handler_test.go — new internal/api/router/router.go — /agents/retired before /agents/{id} internal/cli/agent_retire_test.go — new internal/cli/client.go — ListRetiredAgents + RetireAgent internal/domain/connector.go — IsRetired, SentinelAgentIDs, IsSentinelAgent, AgentDependencyCounts, ActorTypeAgent/System internal/domain/connector_test.go — new internal/integration/lifecycle_test.go — retirement fixture internal/mcp/client.go — DeleteWithQuery additive transport internal/mcp/retire_agent_test.go — new internal/mcp/tools.go, internal/mcp/types.go — retire_agent tool + Force/Reason inputs internal/repository/interfaces.go — AgentRepository retirement methods internal/repository/postgres/agent.go — retire + cascade target retire + counts internal/repository/postgres/migration_000015_test.go — new internal/service/agent.go — wire into AgentService surface internal/service/agent_retire.go — new 8-step contract internal/service/agent_retire_test.go — new internal/service/deployment.go — skip retired agents internal/service/target.go — skip retired agents internal/service/testutil_test.go — shared mocks extended migrations/000015_agent_retire.up.sql — new migrations/000015_agent_retire.down.sql — new web/src/api/client.ts, types.ts + tests — retire endpoint wiring web/src/pages/AgentsPage.tsx — retire UI	2026-04-19 05:24:00 +00:00
shankar0123	b059ec930f	fix: end-to-end certificate lifecycle bugs + integration test environment ... Fixes 12 production bugs preventing the full issuance→deployment flow from working with ACME (Pebble/Let's Encrypt) and step-ca issuers: ACME connector (acme.go): - Save orderURI before WaitOrder overwrites it (Go crypto/acme bug) - Add CreateOrderCert fallback via WaitOrder+FetchCert - Remove defer-reset in ValidateConfig that caused nil pointer panic - Add Insecure TLS option for self-signed ACME servers (Pebble) step-ca connector (stepca.go, jwe.go): - Real JWE provisioner key loading + decryption (was using ephemeral keys) - Fix JWT audience (/1.0/sign), sha claim (key fingerprint), kid header - Custom root CA trust via RootCertPath config - Remove hardcoded 90-day validity default (let step-ca decide) NGINX target connector (nginx.go): - Use sh -c for validate/reload commands (shell interpretation) - Use filepath.Dir instead of fragile string slicing - Add private key file writing (agent-mode keys were never deployed) - Make chain_path write conditional Server/service layer: - TriggerRenewalWithActor now creates actual Job records (was no-op) - createDeploymentJobs falls back to DB query when cert.TargetIDs empty - ProcessPendingJobs skips agent-routed deployment jobs - Agent cert pickup path parsing: len(parts)<4 → len(parts)<3 - Health/ready/auth-info endpoints bypass auth middleware - Write timeout 15s→120s for ACME issuance - Cert fingerprint computed on CSR submission Integration test environment (deploy/test/): - 10-phase test script covering Local CA, ACME, step-ca, revocation, discovery, renewal, and API spot checks - Docker Compose with 7 containers (server, agent, postgres, nginx, pebble, challtestsrv, step-ca) on isolated network - TLS verification checks SAN (not just Subject CN) for modern CA compat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-02 17:02:20 -04:00
shankar0123	a8fc177118	fix: resolve NULL csr_pem scan errors and QA smoke test failures ... Root cause: certificate_versions.csr_pem is nullable in the schema but Go code scanned it into a plain string. Used sql.NullString in ListVersions and GetLatestVersion to handle NULL values correctly. Also includes: partial update fetch-merge-update pattern to prevent FK violations, nil directory guard in discovery service, diagnostic slog logging in handlers, export handler 422 for unparseable PEM, OpenAPI spec corrections, MCP tool description improvements, and test fixes. Rewrites the Release Sign-Off section in testing-guide.md to individual test-level granularity (320 rows) with smoke test results audited and checked off (121 pass, 5 skip, 194 manual remaining). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-30 00:51:18 -04:00
shankar0123	200bdf990f	fix(quality): TICKET-012 propagate request context instead of context.Background() ... - Updated AgentService interface to accept context.Context parameter in all methods - Replaced context.Background() calls with proper ctx parameter in agent.go - Updated AgentGroupService interface to accept context.Context parameter - Replaced context.Background() calls with proper ctx parameter in agent_group.go - Updated handler methods to pass r.Context() to service methods - Context now properly propagates through request lifecycle for timeout/cancellation - Improved request tracing and cancellation behavior	2026-03-27 21:35:22 -04:00
shankar0123	07275bf92f	feat: M10 — agent metadata collection, Apache httpd + HAProxy target connectors ... Agents now report OS, architecture, IP address, hostname, and version via heartbeat using runtime.GOOS, runtime.GOARCH, and net.Dial. New migration adds columns to agents table. Heartbeat handler, service, and repository updated to accept and persist metadata. GUI shows OS/Arch in agent list and full system info in agent detail page. Apache httpd connector: separate cert/chain/key files, apachectl configtest validation, graceful reload. HAProxy connector: combined PEM file (cert+chain+key), optional config validation, reload. Both wired into agent binary's target connector switch. 14 tests for new connectors. All existing tests updated for new Heartbeat/UpdateHeartbeat signatures. Docs updated across README, architecture, concepts, and connectors guides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-20 02:19:28 -04:00
shankar0123	9e6756d02f	Implement M5: hardening, input validation, and Vite+React+TS dashboard ... Backend hardening: - Fix 6 nginx.go non-constant format string build errors - Add validation.go with hostname, PEM, and enum validators - Apply input validation to all POST/PUT handlers (certificates, agents, CSR, policies, teams, owners, targets, issuers) - Fix unchecked JSON decode in TriggerDeployment handler Frontend (Vite + React + TypeScript): - Migrate from single-file SPA to proper build pipeline - 7 pages: Dashboard, Certificates (list+detail), Agents, Jobs, Notifications, Policies, Audit Trail - TanStack Query for server state with auto-refetch intervals - Certificate detail with version history and renewal trigger - Job cancellation, status/type filtering, expiry countdowns - Reusable components: DataTable, StatusBadge, ErrorState, PageHeader - Dark theme with Tailwind CSS, sidebar nav via React Router Server integration: - Go server serves web/dist/ (Vite output) with SPA fallback - Falls back to web/index.html for legacy mode - .gitignore updated for web/node_modules/ and web/dist/ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 01:19:19 -04:00
shankar0123	ae67b10708	Complete M1, M1.1, M2: end-to-end lifecycle, agent deployment, ACME v2 ... - Wire issuer connector end-to-end with IssuerConnectorAdapter (dependency inversion) - Renewal/issuance job processor: RSA key + CSR generation, Local CA signing, cert version storage - Agent work API (GET /agents/{id}/work) and job status API (POST /agents/{id}/jobs/{job_id}/status) - Agent-side deployment: WorkItem enrichment with target type/config, NGINX/F5/IIS connector invocation - Full ACME v2 implementation: HTTP-01 challenge solving, account registration, order lifecycle - Update all docs (README, architecture, connectors, demo-advanced, quickstart) for M1-M2 - Fix go vet warning in deployment.go (non-constant format string) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 23:49:45 -04:00
shankar0123	d395776a95	Initial scaffold: certificate control plane v0.1.0	2026-03-14 08:22:17 -04:00