11 Commits

Author	SHA1	Message	Date
shankar0123	02438ad9e1	ci: floor raise + doc drift (Phase 3 closure — TEST-H1/H2/M1/M2/M3/M4/L1, ARCH-H3/L1/L2/L3/L4) ... Twelve findings from the architecture diligence audit's Phase 3 bundle closed in one PR. All touch the CI workflows + small doc-drift fixes across the production Go tree + migration headers. CI workflow changes ==================== TEST-H1 — Race detection on ./... -short .github/workflows/ci.yml:106 was a 9-package explicit list. Audit finding TEST-H1 flagged that 25+ packages (internal/auth/*, internal/repository/*, internal/mcp, internal/scep, internal/pkcs7, internal/api/router, internal/api/acme, internal/cli, internal/cms, internal/config, internal/deploy, internal/integration, internal/ratelimit, internal/secret, internal/trustanchor, all of cmd/) silently dropped off race coverage. Post-fix: 'go test -race -short ./... -count=1 -timeout 600s'. 76 testing.Short() guards already cover testcontainers + live-DB integration suites, so -short keeps the long-running tests out. TEST-H2 — Cross-platform build matrix New 'cross-platform-build' job in ci.yml. Matrix: ubuntu-latest + windows-latest + macos-latest, fail-fast: false. Builds cmd/server + cmd/agent + cmd/cli + cmd/mcp-server on each. Catches Windows-specific regressions (path separators, file permissions, exec.Command semantics) the pre-Phase-3 Ubuntu-only CI missed. TEST-L1 — actions/setup-go cache: true (explicit) setup-go v5 defaults cache: true; making it explicit so a future setup-go upgrade can't silently flip it. Re-runs hit the Go module + build cache instead of recompiling cold. TEST-M1 — Mutation-testing floor at 55% security-deep-scan.yml::go-mutesting step rewritten. Removed continue-on-error + per-package '|| true'. New post-loop check extracts every 'The mutation score is X.YZ' line and fails the step if any package drops below 0.55. Floor rationale: starter ratio catches major regressions without rejecting the audit's 'this is OK' steady state; raise quarterly. TEST-M2 — 3 advisory deep-scan gates promoted to blocking Removed continue-on-error: true from: - gosec (filtered to G201/G202/G304/G108 high-signal rules: SQL-injection + path-traversal + pprof-exposed) - osv-scanner (multi-ecosystem CVE; complements govulncheck which is already blocking in ci.yml) - trivy image scan (--severity HIGH,CRITICAL --exit-code 1) continue-on-error count: 15 → 11. ZAP / schemathesis / nuclei / testssl stay advisory because their false-positive rates on https://localhost:8443-targeted DAST runs are high. TEST-M3 — Playwright harness stub web/package.json adds '@playwright/test' devDep + 'e2e' / 'e2e:install' npm scripts. web/playwright.config.ts ships single chromium project with webServer block pointing at 'npm run dev'. web/src/__tests__/ e2e/smoke.spec.ts proves the harness wires through. The full 15-flow suite ships in frontend-design-audit Phase 8 (TEST-H1 in THAT audit); this is the wiring + a single smoke test as the regression floor. New Makefile target: 'make e2e-test'. Doc/code drift fixes ==================== TEST-M4 + ARCH-L2 — Skip inventory artifact + CI guard scripts/skip-inventory.sh walks every t.Skip site under cmd/ + internal/ + deploy/test/ and emits docs/testing/skip-inventory.md grouped by package with file:line:expression triples. Current inventory: 142 t.Skip sites, 76 testing.Short() guards. scripts/ci-guards/skip-inventory-drift.sh regenerates and fails on diff (excluding the 'Last reviewed' timestamp line which drifts daily). The Markdown is the canonical acquisition-diligence artifact for 'what tests are being skipped and why.' ARCH-H3 — MCP catalogue floor reconciliation Audit framing was '121 vs floor 150 — doc/code drift.' Live count via the test's actual regex over all 5 tool files (tools.go + tools_audit_fix.go + tools_auth.go + tools_auth_bundle2.go + tools_est.go): 155 unique 'Name: "certctl_*"' declarations. Pre-Phase-3 audit measured tools.go in isolation (121) and missed the other 4 files (+34 unique names). The test at internal/ciparity/surface_parity_test.go::TestSurfaceParity_MCP passes today (155 ≥ 150). Added a clarifying comment near mcpBaselineFloor explaining the measurement scope so future reviewers don't repeat the audit's framing error. STATUS: stale — no code drift, just a measurement scoping error in the audit. ARCH-L1 — panic() rationale comments 5 panic sites in production Go (excluding _test.go): - internal/repository/postgres/tx.go:84 - internal/service/issuer.go:861 (mustJSON) - internal/service/est.go:728 (mustParseTime) - internal/service/acme.go:1288 (rand source failure — already documented) - internal/pkcs7/certrep.go:270 (OID marshal — already documented) Added ARCH-L1 rationale comments to the 3 sites that didn't have them. All 5 are defensible impossible-path / rethrow / hardcoded- constant guards. ARCH-L3 — Migration IF-NOT-EXISTS carve-outs 4 migrations skip the literal 'IF NOT EXISTS' token but ARE idempotent via different Postgres patterns: - 000014_policy_violation_severity_check.up.sql: ALTER TABLE ADD CONSTRAINT CHECK doesn't accept IF NOT EXISTS; idempotency via DROP CONSTRAINT IF EXISTS preamble. - 000018_audit_events_worm.up.sql: CREATE OR REPLACE FUNCTION + DROP TRIGGER IF EXISTS + CREATE TRIGGER + DO $$ pg_roles existence check. CREATE TRIGGER doesn't take IF NOT EXISTS. - 000030_rbac_admin_perms.up.sql: INSERT ... ON CONFLICT DO NOTHING. - 000039_audit_crit1_perms.up.sql: same INSERT + ON CONFLICT pattern. Added ARCH-L3 header comments to each explaining the carve-out so reviewers don't flag the missing literal token. STATUS: largely stale — migrations are already idempotent. ARCH-L4 — TODO/FIXME → see #<descriptor> 5 TODOs rewritten to the allowed 'see #<descriptor>' pattern: - internal/repository/postgres/auth.go:220 → see #bundle-2-scope-fk - internal/connector/discovery/gcpsm/gcpsm.go:547 → see #gcpsm-pagination - internal/service/audit.go:244 → see #audit-pagination-count - internal/service/job.go:295, 299 → see #validation-job-impl New CI guard scripts/ci-guards/no-todo-in-prod.sh grep-fails any new TODO/FIXME in cmd/ + internal/ (excluding _test.go); allows 'see #N' / 'see #<descriptor>' patterns. Sandbox limitation ================== The 6.1 GB certctl working tree fills the sandbox volume; go1.25.10 toolchain download fails with 'no space left on device' (sandbox has 1.25.9; go.mod requires 1.25.10). Local 'go test' / 'go build' NOT run in this commit. Operator must run 'make verify' on their workstation before push per CLAUDE.md operating rules. The smoke.spec.ts NOT executed in the sandbox (no chromium installed). Operator runs 'cd web && npm install && npx playwright install --with-deps chromium && npm run e2e' on first wire-up. All CI guards (no-todo-in-prod, skip-inventory-drift, G-3 env-docs-drift, doc-rot-detector, and every existing guard) verified clean by running each individually. Closes: cowork/certctl-architecture-diligence-audit.html#fix-TEST-H1, cowork/certctl-architecture-diligence-audit.html#fix-TEST-H2, cowork/certctl-architecture-diligence-audit.html#fix-TEST-M1, cowork/certctl-architecture-diligence-audit.html#fix-TEST-M2, cowork/certctl-architecture-diligence-audit.html#fix-TEST-M3, cowork/certctl-architecture-diligence-audit.html#fix-TEST-M4, cowork/certctl-architecture-diligence-audit.html#fix-TEST-L1, cowork/certctl-architecture-diligence-audit.html#fix-ARCH-H3, cowork/certctl-architecture-diligence-audit.html#fix-ARCH-L1, cowork/certctl-architecture-diligence-audit.html#fix-ARCH-L2, cowork/certctl-architecture-diligence-audit.html#fix-ARCH-L3, cowork/certctl-architecture-diligence-audit.html#fix-ARCH-L4	2026-05-13 20:10:08 +00:00
shankar0123	912ec3f547	fix(audit): ship streaming NDJSON audit export endpoint (HIGH-9 / HIGH-11) ... Audit 2026-05-10 HIGH-9 + HIGH-11 closure. HIGH-10 deferred to v3. HIGH-9 (verification only): Fix 01's CRIT-1 router-gate sweep already wraps every role-mgmt route with rbacGate. Verified via grep: - GET /api/v1/auth/roles → auth.role.list - POST /api/v1/auth/roles → auth.role.create - GET /api/v1/auth/roles/{id} → auth.role.list - PUT /api/v1/auth/roles/{id} → auth.role.edit - DELETE /api/v1/auth/roles/{id} → auth.role.delete - POST /api/v1/auth/roles/{id}/permissions → auth.role.edit - DELETE /api/v1/auth/roles/{id}/permissions/{perm} → auth.role.edit - POST /api/v1/auth/keys/{id}/roles → auth.role.assign - DELETE /api/v1/auth/keys/{id}/roles/{role_id} → auth.role.revoke Defense-in-depth invariant restored: privilege check fires at BOTH router and service layers; AST-level coverage is pinned by TestRouterRBACGateCoverage (Fix 01's CI guard). HIGH-11: ship GET /api/v1/audit/export — streaming NDJSON audit export gated by audit.export. Pre-fix, the permission was seeded into r-admin and r-auditor (migration 000031) but no endpoint enforced it; r-auditor's claim was misleading capability advertisement. Post-fix: - internal/api/handler/audit.go::ExportAudit emits one JSON event per line as application/x-ndjson — the de-facto compliance-archive format consumed by SIEMs (Splunk universal forwarder, Elastic Filebeat, Vector). - Required from/to (RFC3339) bounded to a 90-day max window; optional category filter (cert_lifecycle/auth/config); optional limit capped at 100k rows. - Content-Disposition: attachment; filename="certctl-audit-<from>_to_<to>.ndjson" so curl + browser downloads land with a sensible filename. - Recursively self-audits: every successful export emits an audit.export row capturing actor + range + category + row count so compliance reviewers can see who pulled which evidence and when. - Service layer: AuditService.ExportEventsByFilter reuses the existing repository.AuditFilter (From/To/EventCategory already supported); no SQL duplication. - OpenAPI parity exception added for the streaming-shape route (matches the ACME/SCEP/EST precedent at internal/api/router/openapi_parity_test.go::SpecParityExceptions). Regression matrix in audit_export_test.go (7 cases): - TestExportAudit_StreamsNDJSONLines (happy path; pins content-type + content-disposition + JSON-per-line shape + recursive self-audit) - TestExportAudit_RejectsRangeBeyond90Days (100-day window → 400) - TestExportAudit_RejectsMissingFromOrTo (3 cases) - TestExportAudit_RejectsInvalidCategory (unknown enum → 400) - TestExportAudit_AcceptsValidCategoryFilter (auth filter passes through) - TestExportAudit_RejectsNonGET (POST → 405) - TestExportAudit_RejectsToBeforeFrom (inverted range → 400) The auditor role's surface is now complete (read + export). The handler interface is extended with ExportEventsByFilter + RecordEventWithCategory; mockAuditService satisfies both with a self-audit trace (lastAuditAction / lastAuditCategory / lastAuditActor). HIGH-10 (scope + expiry on assignRoleRequest): DEFERRED to v3. Schema column already exists (ActorRole.ExpiresAt); load-bearing wire remains v3 work. Documented carve-out at HIGH-10's annotation. Refs: cowork/auth-bundles-audit-2026-05-10.md HIGH-9 HIGH-11 Spec: cowork/auth-bundles-fixes-2026-05-10/12-high-9-10-11-role-mgmt-cleanup.md	2026-05-10 21:36:01 +00:00
shankar0123	f5ba17114d	fix(audit): close silence-leg of HIGH-6; emit WARN on audit-write failure ... Audit 2026-05-10 HIGH-6 partial closure (silence leg). The audit identified two distinct gaps in the auth surface's audit-emit pattern: (1) silence — `_ = audit.RecordEventWithCategory(...)` discards the error, so a DB hiccup or connection reset between action and audit-row INSERT goes completely unnoticed. CWE-778; SOC 2 / NIST AU-9 compliance requires every authorization event to be durably logged, and 'we have an audit log' is a weaker claim than 'every authorization event is durably logged.' (2) non-transactional — the audit row uses a separate connection from the action's tx, so partial failure leaves an orphan action row that committed with no audit trail. Decision 8 of the auth-bundles-index requires action + audit row atomic. This commit closes leg (1) fully across all six audit-emit call sites in the auth surface: - internal/service/auth/actor_role_service.go::recordAudit - internal/service/auth/role_service.go::recordAudit - internal/auth/bootstrap/service.go::ValidateAndMint - internal/auth/breakglass/service.go::recordAudit - internal/auth/session/service.go::recordAudit - internal/api/handler/auth_session_oidc.go::recordAudit - internal/service/profile.go::Update (Phase 9 approval-bypass) Each `_ = ...` swallow is replaced with: if err := audit.RecordEventWithCategory(...); err != nil { slog.WarnContext(ctx, '<surface> audit write failed (action committed; audit row may be missing)', 'action', action, 'actor_id', actor, 'resource_id', resource, 'err', err) } Operators monitoring audit-write failures now see structured WARN logs with action + actor + resource attribution; missing audit rows can be cross-referenced against monitoring without manual SELECT-from- audit-table. Infrastructure for leg (2) (transactional commit) is also landed in this commit: - service.AuditService.RecordEventWithCategoryWithTx (new method; accepts repository.Querier from postgres.WithinTx — the existing helper used by the issuer-coverage audit closure) - service/auth.AuditService interface declares the new method - test stub fakeAudit.RecordEventWithCategoryWithTx satisfies the extended interface The eight per-path WithinTx-refactors documented in cowork/auth-bundles-fixes-2026-05-10/10-high-6-atomic-audit-commit.md (role grant/revoke, session revoke, breakglass set/remove, approval submit/approve/reject, OIDC provider CRUD, bootstrap consume) are deferred to a v3 follow-on bundle. Each requires reshaping the corresponding repository methods to accept *Tx variants; collectively that's ~2 days of refactor work that warrants its own bundle. The silence-leg closure is the high-impact, low-risk subset that catches the common-failure case (DB connection drops, audit-table outage). Refs: cowork/auth-bundles-audit-2026-05-10.md HIGH-6 Spec: cowork/auth-bundles-fixes-2026-05-10/10-high-6-atomic-audit-commit.md	2026-05-10 21:24:29 +00:00
shankar0123	3ef45e2ad4	auth-bundle-1 Phase 6-7-8: bootstrap path + scope-down CLI + auditor-role split ... # Phase 6 — day-0 admin bootstrap * internal/auth/bootstrap/ (new package): Strategy interface + EnvTokenStrategy with constant-time compare, one-shot consumption via sync.Mutex, optional admin-existence probe. Bundle 2's OIDC- first-admin will plug in alongside as an alternate Strategy. * BootstrapService.ValidateAndMint: validates the operator's CERTCTL_BOOTSTRAP_TOKEN, mints a 32-byte (64-hex-char) random API key value, persists the SHA-256 hash to api_keys, grants r-admin via actor_roles, AddHashed's the runtime keystore so the just- minted key authenticates the next request without restart, and records bootstrap.consume to the audit trail with category=auth. * internal/auth/keystore.go (new): KeyStore interface + StaticKeyStore (immutable env-var-only path) + MutableKeyStore (env-var keys + DB-loaded api_keys + runtime AddHashed). The auth middleware now consumes a KeyStore so the bootstrap path can extend the lookup table at runtime. * migrations/000031_api_keys.up/down.sql: api_keys table with (id, name UNIQUE, key_hash UNIQUE, tenant_id, admin, created_by, created_at, expires_at, last_used_at). Idempotent. * /v1/auth/bootstrap GET (probe) + POST (mint) — auth-exempt. Both routes documented in api/openapi.yaml + AuthExemptRouterRoutes allowlist updated. The token never leaves internal/auth/bootstrap; the minted plaintext key flows only into the HTTP response body. * Startup warning emitted when CERTCTL_BOOTSTRAP_TOKEN is set AND admin actors already exist (config drift signal). * Tests: 4 strategy invariants (empty token born disabled, wrong token=ErrInvalidToken without consumption, one-shot consumption, admin-exists closes path), 5 service tests (happy path + actor- name validation + propagation of strategy errors + nil-deps guard + 32-byte entropy budget), 8 HTTP-handler tests (status 201/410/401/400 mapping + token-leak hygiene scan of slog + audit details + Location header). Token-leak test redirects slog.Default to a buffer for the test scope. # Phase 7 — API-key migration + scope-down CLI * GET /v1/auth/keys handler + service method ListKeys backed by ActorRoleRepository.ListDistinctActors. Returns one row per (actor_id, actor_type) pair with the slice of role IDs they hold. Permission: auth.role.list. * internal/cli/auth_scope_down.go: AuthListKeys, AuthScopeDown (interactive), AuthScopeDownNonInteractive (JSON config), AuthScopeDownSuggest (--suggest with optional --apply). The synthetic actor-demo-anon is filtered out of every interactive / bulk path; non-interactive flow logs and skips it explicitly. * SuggestRoleFromAuditEvents (pure function): walks 30 days of audit events per actor and returns the narrowest matching role (admin / mcp / viewer / agent / operator) plus a one-line reason. Classification: any admin-shaped action wins; otherwise all-MCP → mcp; all-read-only → viewer; all-agent-shaped → agent; otherwise operator. Test table pins all six classifications. * CLI subcommand tree extended: 'auth keys list' + 'auth keys scope-down [--non-interactive <cfg>] [--suggest [--apply]]'. * CHANGELOG.md leads v2.1.0 with the SECURITY: AUDIT YOUR API KEYS call-out + four flow examples. # Phase 8 — auditor role + event_category column * migrations/000032_audit_category.up/down.sql: ALTER TABLE audit_events ADD COLUMN event_category TEXT NOT NULL DEFAULT 'cert_lifecycle' + CHECK constraint (cert_lifecycle/auth/config) + (event_category) and (event_category, timestamp DESC) indexes for the auditor-filter query path. WORM trigger from migration 000018 continues to enforce append-only at the DB layer (DDL is not blocked). * domain.AuditEvent gains EventCategory string (omitempty); domain.EventCategoryCertLifecycle / Auth / Config constants. * AuditService.RecordEventWithCategory sibling of RecordEvent; legacy callers stay on RecordEvent (defaults to cert_lifecycle). Auth callers (RoleService, ActorRoleService, BootstrapService) switched to RecordEventWithCategory(..., 'auth', ...). * GET /v1/audit?category=<cat>: handler accepts the optional query param, validates against the enum (400 on invalid value), dispatches through ListAuditEventsByCategory. OpenAPI updated with the new query param + AuditEvent.event_category schema. * Postgres AuditRepository.Create now writes event_category; AuditRepository.List filters on it; AuditFilter.EventCategory gates the WHERE clause. * Tests: 5 audit-category-filter HTTP tests (dispatch routing, back-compat fallback, 400 for invalid values, all 3 enum values accepted, page+category combine, JSON output surfaces the field). 3 auditor-role invariants (auditor holds exactly audit.read+audit.export, no mutating perms, disjoint from viewer except audit.read). # Cross-phase wiring * HandlerRegistry.Bootstrap field added; cmd/server/main.go wires the bootstrap service ahead of RegisterHandlers (extracted assembleNamedAPIKeys helper into auth_backfill.go, moved the keystore + bootstrap construction up alongside the auth repos). * AuthCheckResolver / AuthActorRoleService extended with ListKeys to satisfy the Phase 7 surface; existing fakes updated. * fakeAudit + mockAuditService stubs in tests gain RecordEventWithCategory + ListAuditEventsByCategory; existing tests untouched. # Verifications * gofmt -l: clean across every modified file. * go vet ./...: clean. * staticcheck across internal/auth + handler + router + cli + service + repository + cmd + domain: clean. * go test -short -count=1: green across every Bundle-1-touched package — internal/auth (incl. bootstrap), internal/api/handler, internal/api/router, internal/cli, internal/service/auth, internal/service, internal/domain/auth, internal/repository/postgres, cmd/server, cmd/cli, plus internal/scheduler, internal/api/middleware, cmd/agent, internal/mcp.	2026-05-09 20:15:43 +00:00
shankar0123	8b75e0311b	chore: rename Go module path to github.com/certctl-io/certctl ... Mechanical sed across the main go.mod's module declaration, the f5-mock-icontrol sub-module's go.mod, every Go file's import path (361 files), and a rebuild of the checked-in f5-mock-icontrol binary so its embedded build-info reflects the new module path. No behavior change. Choice B from cowork/transfer-certctl-to-org.md, executed 2026-05-04. Choice A (keep module path declared as github.com/shankar0123/certctl regardless of repo URL) shipped on the day of the org transfer (2026-05-03) since we had no external Go consumers; this commit closes that deferral. Backward-compat: GitHub HTTP redirects continue to forward github.com/shankar0123/certctl → github.com/certctl-io/certctl at the URL level, but Go's module proxy uses the path declared in go.mod as the canonical name. Pre-fix, anyone trying `go get github.com/certctl-io/certctl/...` hit a "module path mismatch" error because go.mod said github.com/shankar0123/certctl and the URL they fetched it from said certctl-io/certctl. Post-fix, the canonical name and the URL agree, so go get / go install / external Go consumers / Go-tooling integrations work cleanly via either the new path (preferred) or the old path (which redirects and Go follows the redirect for source fetch). Anyone still importing the old path inside their own code keeps working provided they update their go.mod's `require` line to match — the module path declared in their consumer's go.sum / go.mod is the authoritative import name, so a mass sed across their import statements is the migration on the consumer side. No external consumers exist today. Diff shape: 361 *.go files — import path replacement only 2 go.mod — module declaration replacement only 1 binary — deploy/test/f5-mock-icontrol/f5-mock-icontrol rebuilt so embedded build-info reflects the new path (8618965 vs 8618933 bytes; 32-byte diff is the build-info change) Total: 364 files, 730 insertions / 730 deletions, net-zero size, pure mechanical substitution. Verification: gofmt: 17 files needed re-alignment after sed (the new path is one char shorter than the old, so column-aligned import groups drifted). Applied `gofmt -w` to fix. go mod tidy: clean exit on both modules. go vet ./...: clean exit. go build ./...: clean exit. go test -short -count=1 on representative packages: all green (internal/domain, internal/validation, internal/crypto, internal/crypto/signer, cmd/agent). Test output now reads `ok github.com/certctl-io/certctl/...` confirming the module path resolves correctly. binary: f5-mock-icontrol rebuilt; `strings | grep shankar0123` returns nothing; `strings | grep certctl-io/certctl` shows the new module path embedded in build-info. Files intentionally NOT touched in this commit: README.md / CHANGELOG.md / docs/ / etc. — already swept to certctl-io URLs in commit 0729ee4 (the post-transfer URL refresh). This commit is purely the Go-tooling layer. Scarf pixels (`shankar0123.docker.scarf.sh/...`) — Scarf-account namespace, not a Go import or GitHub repo URL. Stays. This is a non-blocking, non-customer-impacting change. Operators pulling container images, running `make verify`, hitting the API, or installing the agent see no functional difference. Only Go-tooling consumers (none today) are affected, and they're enabled — not broken — by this commit.	2026-05-04 00:30:29 +00:00
shankar0123	b0efdbe2f8	repo,service: introduce WithinTx and atomic audit rows for issue/renew/revoke ... Closes the #3 acquisition-readiness blocker from the 2026-05-01 issuer coverage audit (Part 1.5 finding #1: audit row not transactional with issuance). AuditRepository.Create previously ran on the package-level *sql.DB while the certificate insert / version insert / revocation insert ran on independent connections — a failed audit INSERT after a successful operation INSERT was silently lost. SOX §404 over IT general controls, PCI-DSS §10 audit logging, HIPAA §164.312(b) audit controls, and CA/B Forum Baseline Requirements §5.4.1 audit log records all presume audit-with-operation atomicity. Design — Option A (Querier abstraction). The chosen pattern: a shared repository.Querier interface (subset of *sql.DB and *sql.Tx) plus a postgres.WithinTx helper that begins a tx, runs fn, commits on nil error, rolls back on error or panic, and returns the wrapped result. Repository methods that participate in a service-layer transaction expose a *WithTx variant taking repository.Querier; the bare methods remain for stand-alone use. A repository.Transactor abstracts the "begin tx, run fn, commit/rollback" lifecycle so service-layer code runs multi-write operations atomically without holding *sql.DB directly. Option B (UnitOfWork) was considered but adds boilerplate without behavioral benefit for the current scope. Option C (context-carried tx) was explicitly rejected — it hides the transactional boundary from the type system, reproducing the class of bug we're fixing. This commit: - Adds internal/repository/querier.go with the Querier interface (compile-time guards that *sql.DB and *sql.Tx satisfy it) and the Transactor interface for service-layer use. - Adds internal/repository/postgres/tx.go with the WithinTx helper (begin/fn/commit/rollback with panic recovery) and a transactor type that satisfies repository.Transactor. - Adds CreateWithTx variants on AuditRepository, CertificateRepository (Create + Update + CreateVersion), and RevocationRepository. Existing bare methods now delegate to the *WithTx variant using the package-level *sql.DB so existing call sites are behavior-preserving. - Updates repository/interfaces.go: AuditRepository, CertificateRepository, and RevocationRepository declare the new *WithTx methods. Adds an atomicity contract doc-comment on AuditRepository pointing at WithinTx + the audit blocker. - Adds AuditService.RecordEventWithTx, mirroring RecordEvent but routing through CreateWithTx so the audit row is part of the caller's transaction. Same redaction + marshalling contract. - Refactors three audit-emitting service paths to use Transactor.WithinTx when SetTransactor was wired, with a legacy fallback for backward compat: * CertificateService.Create — cert insert + audit row in one tx. * RevocationSvc.RevokeCertificateWithActor — cert status update + revocation row + audit row in one tx. The OCSP cache invalidate remains best-effort (out of scope per the prompt). * RenewalService CompleteServerRenewal — cert version insert + cert update + audit row in one tx. Job status update stays outside the audit-atomicity scope (job state lives outside the operator-facing audit trail). - Adds SetTransactor on CertificateService, RevocationSvc, and RenewalService. cmd/server/main.go wires a single Transactor instance shared across all three so all audit-emitting paths run their writes in transactions backed by the same *sql.DB handle. - Updates 5 mock implementations to satisfy the new interface methods: mockCertRepo (testutil_test.go), mockCertRepoWithGetError (shortlived_test.go), fakeRevocationRepo (crl_cache_test.go), intuneE2EAuditRepo (scep_intune_e2e_test.go), and the integration- test mocks (lifecycle_test.go: mockCertificateRepository, mockAuditRepository, mockRevocationRepository). All *WithTx mocks ignore the Querier and delegate to the bare method (mocks have no DB; in-memory state is shared regardless of "tx"). - Adds a service-layer test mockTransactor with BeginTxErr and CommitErr knobs so the atomic-audit tests can assert error propagation through the transactional boundary. - Adds internal/repository/postgres/tx_test.go: unit-level test that WithinTx surfaces "begin tx" wrap when BeginTx fails, and that Transactor.WithinTx delegates correctly. Real-Postgres rollback semantics are covered by the testcontainers tests in the postgres package — sandbox disk pressure prevented adding a sqlmock dep for the in-fn / commit-failure unit test, so those scenarios are exercised through atomic_audit_test.go using the mockTransactor's CommitErr / BeginTxErr fields. - Adds internal/service/atomic_audit_test.go: * TestCertificateService_Create_AtomicWithTx — asserts audit insert failure inside the tx surfaces as the operation's error (closes the blocker contract). * TestCertificateService_Create_LegacyPathLogs — pins the backward-compat behavior when SetTransactor isn't wired: audit failure is logged-not-failed, matching pre-fix. * TestCertificateService_Create_TransactorBeginFailure — BeginTx error path: operation fails, no cert insert, no audit insert. * TestCertificateService_Create_TransactorCommitFailure — Commit error after successful in-fn writes surfaces as the operation's error. Real Postgres can fail Commit on serialization conflicts; the service must report this. Out of scope (separate follow-up commits, same shape): - Issuer CRUD audit atomicity. - Target CRUD audit atomicity. - Agent retire (already transactional via RetireAgentWithCascade; verified, not changed). - Renewal-policy CRUD audit atomicity. - Owner/team/agent-group CRUD audit atomicity. - Discovery / health-check audit atomicity. Verified locally: - gofmt -l . clean - go vet ./... clean - staticcheck ./... clean - golangci-lint run --timeout 5m ./... → 0 issues - go test -short -count=1 ./internal/service/ green - go test -short -count=1 ./internal/api/handler/ green - go test -short -count=1 ./internal/integration/ green - go test -short -count=1 ./internal/repository/postgres/ green - go build ./... success Audit reference: cowork/issuer-coverage-audit-2026-05-01/RESULTS.md Top-10 fix #3 (Part 3, narrative section).	2026-05-02 00:29:09 +00:00
shankar0123	1d6c7a0552	fix(bundle-6): Audit Integrity + Privacy — 3 audit findings closed ... Closes Audit-2026-04-25 H-008 (High), M-017 (Medium), M-022 (Medium). Hardens audit-trail tamper-resistance + minimizes PII leakage in one cohesive change, with both controls applying automatically and no operator action required at install time. What changed - internal/service/audit_redact.go (NEW) — RedactDetailsForAudit: * credentialKeys deny-list (api_key, password, *_pem, eab_secret, ...) * piiKeys deny-list (email, phone, ssn, name, address, ip_address, ...) * case-insensitive key match; recurses into nested maps + arrays * mutation-free; surfaces redacted_keys array for operator visibility * nil/empty input → nil out (preserves pre-Bundle-6 behaviour) - internal/service/audit.go — RecordEvent now routes details through RedactDetailsForAudit BEFORE marshaling. No call-site changes required. - internal/service/audit_redact_test.go (NEW) — full coverage: * credential keys (~30 entries) * PII keys (~20 entries) * nested maps + arrays * case-insensitivity * mutation-free invariant * JSON round-trip (catches type-assertion regressions) * scalar pass-through (no panic on int/bool/nil) - migrations/000018_audit_events_worm.up.sql (NEW) — DB-level WORM: * BEFORE UPDATE OR DELETE trigger raises check_violation with diagnostic citing the rationale + compliance-superuser hint * REVOKE UPDATE,DELETE ON audit_events FROM certctl (defence-in-depth) * REVOKE wrapped in pg_roles existence check so test fixtures without the certctl role stay idempotent - migrations/000018_audit_events_worm.down.sql (NEW) — clean teardown for dev resets; not for production use. - internal/repository/postgres/audit_worm_test.go (NEW, testcontainers, -short gated) — INSERT succeeds; UPDATE + DELETE fail with check_violation; second INSERT after blocked modification still succeeds (no trigger-state corruption). - docs/compliance.md — new section "Audit-Trail Integrity & Privacy (Bundle 6)" with verification psql snippet, compliance-superuser pattern (NOT auto-created), redactor before/after example, and a maintenance note for adding new credential keys. Compliance mapping - H-008 (CWE-532 Insertion of Sensitive Information into Log File) - M-017 (HIPAA Technical Safeguards §164.312(b) — audit controls) - M-022 (GDPR Art. 32 — data minimization) Threat model: TB-3 (audit log tampering), TB-1 (operator/orchestrator). Verification - go vet ./... → clean - go build ./... → clean - go test -short -count=1 ./... → all packages pass - go test -count=1 -run TestRedactDetailsForAudit ./internal/service/... → all pass - (testcontainers, gated by -short) audit_worm_test.go pins WORM contract - npx tsc --noEmit (web) → clean (no frontend changes) - python3 yaml.safe_load(api/openapi.yaml) → 89 paths Backward compatibility - Trigger applies forward only — existing rows unchanged. - nil/empty details from RecordEvent callers → nil out (preserves prior behaviour for the many existing call sites that pass nil). - Compliance superusers (provisioned out-of-band) bypass the trigger. Bundle 6 of the 2026-04-25 comprehensive audit.	2026-04-26 00:26:44 +00:00
shankar0123	ccd89c348f	fix(m2-pr-d): thread ctx through Job/Notification/Audit services ... Collapse CancelJobWithContext into CancelJob; eliminate 10 context.Background() hits across the Job+Notification+Audit service cluster by threading ctx through their handler-facing service interfaces. Services (ctx-first): - service/job.go: ListJobs, GetJob, CancelJob, ApproveJob, RejectJob now accept ctx; the CancelJobWithContext wrapper is removed (handler callers continue to invoke CancelJob, now ctx-aware). - service/notification.go: ListNotifications, GetNotification, MarkAsRead accept ctx. - service/audit.go: ListAuditEvents, GetAuditEvent accept ctx. Handlers (interface + callsites): - handler/jobs.go, handler/notifications.go, handler/audit.go: local service interfaces updated, r.Context() threaded at every callsite. Tests: - Mock services updated to match the new interfaces (ctx accepted and ignored via '_ context.Context' first parameter; Fn closure fields unchanged). - job_test.go / notification_test.go callsites thread context.Background() to match production shape. Verification: go build ./... ok go vet ./... ok go test -short ./... ok go test -race -short ./... ok golangci-lint run ./... 0 issues Locked decisions from the M-2 plan: D-1 ctx-only signatures (no dual forms) D-4 preserve handler method names facing the router D-5 domain types stay ctx-free Audit complete. Commit: 1f6cf0eafa. Sections: 12. Findings: 2/7/10/4/6.	2026-04-18 01:20:46 +00:00
shankar0123	9b4122b159	Fix runtime bugs, implement service layer, and overhaul documentation ... Runtime fixes: - Fix env var mismatch (CERTCTL_DB_URL → CERTCTL_DATABASE_URL) - Fix table name mismatches (certificates → managed_certificates, notifications → notification_events) - Add renewal_policy_id to certificate queries - Remove non-existent created_at from notification queries - Add env var fallback for agent CLI flags - Graceful degradation for missing notifiers/issuers in demo mode - Copy web/ directory in Dockerfile for dashboard serving Service layer: - Implement handler-service interface pattern across all services - Wire up certificate, agent, job, policy, team, owner, audit, notification services Documentation: - Add concepts.md: beginner-friendly guide to TLS, CAs, private keys - Rewrite quickstart.md with accurate API examples matching actual handlers - Add demo-advanced.md: interactive demo with cert issuance and automated script - Update architecture.md with correct table names and connector interfaces - Update connectors.md to match actual Go interface signatures - Update demo-guide.md with cross-references to new docs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:38:11 -04:00
shankar0123	3a9fe8ba37	Complete V1 scaffold	2026-03-14 20:01:53 -04:00
shankar0123	d395776a95	Initial scaffold: certificate control plane v0.1.0	2026-03-14 08:22:17 -04:00