mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 12:11:30 +00:00
master
21 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 shankar0123
|
151107c969 |
fix(test-compose): set CERTCTL_AGENT_BOOTSTRAP_TOKEN placeholder (deploy-vendor-e2e job)
deploy-vendor-e2e was hidden behind the go-build-and-test failure; once
that cleared (
|
||
|
shankar0123
|
7e2481b225 |
fix(deploy): SEC-014 — loopback-bind Postgres host port in compose files
Acquisition-audit SEC-014 closure (Sprint 2 ACQ, 2026-05-16). Both deploy/docker-compose.yml and deploy/docker-compose.test.yml published Postgres on `5432:5432` — the short Docker port-mapping form, which binds to 0.0.0.0 by default. On any host with a public-facing NIC, that quietly exposed the Postgres TCP listener to the internet. The certctl-server-to-postgres traffic itself goes over the `certctl-network` Docker bridge, not the host port; the host port mapping is a convenience for operator psql access and for the integration-test runner that lives on the host. Switch both mappings to `127.0.0.1:5432:5432` (loopback-only). Operator psql via `localhost` keeps working; the integration-test runner keeps working; cross-host exposure goes away. Audit trail: docs/operator/security.md (Postgres transport encryption subsection, SEC-014 paragraph). |
||
|
shankar0123
|
d364ace02a |
fix(ci): set CERTCTL_ACME_INSECURE_ACK=true in test compose
Phase 2 SEC-M4 (commit 5062624) added a fail-closed pairing requirement: when CERTCTL_ACME_INSECURE=true, the server refuses to start unless CERTCTL_ACME_INSECURE_ACK=true is also set. The integration test compose at deploy/docker-compose.test.yml has been setting CERTCTL_ACME_INSECURE=true (correct — Pebble's self-signed ACME directory needs TLS verification disabled) but never set the paired ACK, so the certctl-test-server container restart-loops with: Failed to load configuration: phase-2 SEC-M4 fail-closed guard: CERTCTL_ACME_INSECURE=true but CERTCTL_ACME_INSECURE_ACK is not true — refuse to start. This breaks the deploy-vendor-e2e CI job that exercises the EST/ACME integration stack. Fix: set CERTCTL_ACME_INSECURE_ACK=true alongside the existing CERTCTL_ACME_INSECURE=true. The ACK posture is correct here because the integration suite is built around Pebble's self-signed directory — that's the design. The guard's purpose (block accidental production deploys with TLS verify disabled) is preserved by the ACK still being explicit per-environment, not a fail-open default. |
||
|
shankar0123
|
a1c7741e1b |
fix(deploy/test) + ci(guard): drop dead SCEP profile from test compose
The deploy-vendor-e2e job has been failing with the certctl-test-server container restarting endlessly. Diagnostic dump (added in |
||
|
shankar0123
|
c4157fd196 |
fix(deploy/test) + ci(guard): unblock deploy-vendor-e2e — encryption-key length
Two-part complete-path fix for the deploy-vendor-e2e failure that has
been firing since the ci-pipeline-cleanup Phase 5 matrix collapse
started actually booting the certctl-test-server:
Failed to load configuration:
CERTCTL_CONFIG_ENCRYPTION_KEY too short (29 bytes; minimum 32).
Surfaced via the diagnostic-dump step landed in commit
|
||
|
shankar0123
|
c8624a7fae |
fix(deploy/test): libest IP collision with tls-init (10.30.50.9 → 10.30.50.10)
Two services on the certctl-test bridge network were pinned to the same static IP: certctl-tls-init (line 91) and libest-client (line 472). The pre-Phase-5 per-vendor matrix structurally hid this: - tls-init is profile-less ⇒ always runs - libest-client is profiles=[est-e2e] ⇒ only runs when est-e2e job brings it up - est-e2e and deploy-e2e historically lived in DIFFERENT CI jobs ⇒ separate docker networks ⇒ no collision The collision would surface the moment any single CI job invokes both `--profile deploy-e2e` and `--profile est-e2e`, or the moment a local operator runs `docker compose --profile=*` for full-stack debugging. Pre-emptive fix. Move libest to 10.30.50.10 (next free address; allocated range was 10.30.50.2-9 + 20-30, the entire 10-19 sub-range was unused). NOT the cause of the deploy-vendor-e2e "certctl-test-server is unhealthy" failure in CI run 25194251740 — libest isn't in profile=deploy-e2e and never started in that run. Real cause for that failure is being investigated in a separate commit (CI diagnostic dumping). |
||
|
shankar0123
|
a1fae33f40 |
fix(deploy/test): f5-mock-icontrol host-port collision (20443 → 20449)
CI run 25192994486 (deploy-vendor-e2e job) failed with:
Error response from daemon: failed to set up container networking:
driver failed programming external connectivity on endpoint
certctl-test-f5-mock: Bind for 0.0.0.0:20443 failed: port is already
allocated
apache-test (compose line 491) and f5-mock-icontrol (compose line 619)
both bound host port 20443. The pre-Phase-5 per-vendor matrix only ran
one sidecar at a time, so the collision was structurally hidden. The
ci-pipeline-cleanup Phase 5 collapse brings all 11 sidecars up
simultaneously — the bug surfaces.
This was a pre-existing latent bug in the deploy-hardening II Phase 1
(commit
|
||
|
shankar0123
|
b9a63a2521 |
ci-pipeline-cleanup Phase 6 follow-up: IIS operator playbook + matrix doc
Bundle: ci-pipeline-cleanup, Phase 6 follow-up. Phase 5+6 commit removed the deploy-vendor-e2e-windows matrix from ci.yml; this commit closes the Phase 6 deliverables that aren't ci.yml-side: 1. NEW docs/connector-iis.md::Operator validation playbook (Windows host) — the procedure operators run pre-release to flip the IIS / WinCertStore vendor-matrix cells from 'operator-playbook' → '✓'. Mirrors the Bundle II frozen decision 0.14 third-criterion (operator manual smoke required). 2. docs/deployment-vendor-matrix.md — IIS + WinCertStore rows status updated from 'pending' → 'operator-playbook' with link to the new playbook section. 3. deploy/docker-compose.test.yml — windows-iis-test sidecar comment updated to reflect that CI no longer activates this profile; sidecar definition preserved for operator local use via 'docker compose --profile deploy-e2e-windows up -d windows-iis-test'. Operator workflow going forward: - Pre-release: run the playbook on a Windows host - Record validation date + Windows Server version in cowork/<bundle>/iis-validation-receipts.md - Update docs/deployment-vendor-matrix.md cells if applicable |
||
|
shankar0123
|
c48a82c4c8 |
fix(ci): real digests + matrix→service mapping for deploy-vendor-e2e
Bundle II Phases 1+15 shipped fabricated @sha256 digests across 11
sidecars (deploy/docker-compose.test.yml) plus the f5-mock-icontrol
Dockerfile golang FROM line. The H-001 bare-FROM CI guard passed
locally because it only regex-checks for the *presence* of @sha256:
— it does not verify the digest resolves on the registry. Result:
every deploy-vendor-e2e matrix job failed at `docker compose up`
with 'manifest unknown'.
Two classes of fix:
1. Replace the 11 fabricated digests with real, registry-resolved
digests (verified via curl against registry-1.docker.io,
ghcr.io, mcr.microsoft.com manifest endpoints):
- httpd:2.4-alpine
- haproxy:3.0-alpine
- traefik:v3.1
- caddy:2.8-alpine
- envoyproxy/envoy:v1.32-latest
- boky/postfix:latest
- dovecot/dovecot:latest
- lscr.io/linuxserver/openssh-server:latest (via ghcr.io)
- kindest/node:v1.31.0
- mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2022
(manifest.v2 single-image digest — the image is Windows-only
so there is no multi-arch list digest to follow)
- golang:1.25.9-bookworm (in deploy/test/f5-mock-icontrol/Dockerfile)
debian:bookworm-slim was also fabricated under the comment
claiming it 'matches libest sidecar'; replaced with the real
amd64-linux digest.
2. Special-case the matrix.vendor → docker-compose service mapping
in .github/workflows/ci.yml::deploy-vendor-e2e step 'Bring up
vendor sidecar'. The original step assumed a uniform
'${{ matrix.vendor }}-test' suffix, but four matrix entries
don't conform:
- nginx → reuses apache-test (the legacy nginx sidecar in the
compose file is named 'nginx' with no profile; the nginx
vendor-edge tests in deploy/test/nginx_vendor_e2e_test.go
call requireSidecar(t,"apache") because the sidecar map
doesn't include an 'nginx' key — comment in source explains)
- ssh → openssh-test
- k8s → k8s-kind-test
- f5-mock → f5-mock-icontrol (must be built first; no published image)
- javakeystore → no sidecar (pure-Go placeholder stubs)
Wraps the bring-up in a case statement that maps every matrix
entry to its real sidecar name (or '' for the no-sidecar case),
and exits 0 cleanly for vendors that don't need a sidecar.
Per the CLAUDE.md 'never go from memory' + 'complete path' rules,
this fix:
- ground-truths every digest against the actual registry (curl
against the OCI v2 manifest endpoint with the right Accept
header), not memory or grep
- closes the 'lying field' footgun: H-001 guard now validates a
contract that's actually satisfied (digests exist + pull)
Verification: yaml parses on both files, H-001 guard simulation
returns no bare FROMs, all 12 manifest endpoints return HTTP 200
on the new digests.
|
||
|
shankar0123
|
889c1a5a9e |
feat(test): docker-compose deploy-e2e sidecar matrix — apache + haproxy + traefik + caddy + envoy + postfix + dovecot + openssh + f5-mock-icontrol + k8s-kind + windows-iis
Phase 1 of the deploy-hardening II master bundle. Adds the 11 missing
target sidecars to deploy/docker-compose.test.yml under
profiles: [deploy-e2e] (windows-iis-test under [deploy-e2e-windows]
because Windows containers run only on Windows hosts).
Per frozen decision 0.2: pull pre-built images from official
registries where they exist (NGINX, HAProxy, Traefik, Caddy, Envoy,
Postfix via boky, Dovecot, OpenSSH via lscr.io, K8s via kind);
build locally only where no official image works (F5 — uses the
new in-tree f5-mock-icontrol Go server). Every FROM digest-pinned
per H-001 guard.
NEW deploy/test/f5-mock-icontrol/ — in-tree Go server implementing
the iControl REST surface the F5 connector exercises:
- POST /mgmt/shared/authn/login (token-based auth)
- POST /mgmt/shared/file-transfer/uploads/<filename>
- POST /mgmt/tm/sys/crypto/cert + /key (install)
- POST /mgmt/tm/transaction (create) + /<txn-id> (commit)
- PATCH /mgmt/tm/ltm/profile/client-ssl/<name> (update SSL profile)
- GET / DELETE variants
- /healthz for sidecar readiness probes
- HTTPS via per-process self-signed ECDSA P-256 cert
- In-memory state map (lost on container restart; CI tests handle
via test-init re-auth)
Per frozen decision 0.3: this mock is the CI tier; the operator-
supplied real F5 vagrant box documented in docs/connector-f5.md
(Phase 14 deliverable) is the validation tier above. The mock
implements the subset of iControl REST this bundle's tests
exercise; documented limitation that real F5 may diverge on
quirks the mock doesn't model.
NEW per-vendor config bind-mounts (deploy/test/<vendor>/):
- apache/httpd-ssl.conf + init-cert.sh
- haproxy/haproxy.cfg
- traefik/traefik-dynamic.yml
- caddy/Caddyfile
- envoy/envoy.yaml
- dovecot/dovecot.conf
Each minimal config: bind /etc/<vendor>/certs to a named volume
so the e2e tests rotate certs via the per-connector atomic-deploy
primitive (Bundle I Phase 4-9).
Network IPs: 10.30.50.{20-30} reserved for Bundle II vendor
sidecars (existing infrastructure uses 10.30.50.{2-9}).
f5-mock-icontrol Go binary: gofmt clean, go vet clean, go build
clean. Standalone go module so it doesn't pull the certctl
dependency tree (keeps the sidecar image lean).
Phase 2 next: NGINX vendor-edge audit + 10 e2e tests.
|
||
|
shankar0123
|
5a682db8e2 |
EST RFC 7030 hardening master bundle Phases 10-11: libest sidecar e2e
+ Cisco IOS quirk fixtures + ManagedCertificate.Source provenance + EST bulk-revoke endpoint + 13 typed audit action codes. Phase 10.1 — libest reference-client sidecar: - deploy/test/libest/Dockerfile: multi-stage Debian-bookworm-slim build of Cisco's libest v3.2.0-2 from source (autoconf/automake/ libtool + libcurl4-openssl-dev + libssl-dev). Runtime stage carries only estclient + bash + openssl + ca-certificates so the exec surface stays small + predictable. - docker-compose.test.yml libest-client entry (profiles: [est-e2e]) with bind mounts for /config/est (test workspace) + /config/certs (certctl CA bundle for TLS pinning); IP 10.30.50.9 (10.30.50.8 was already taken by certctl-agent). - deploy/test/est/.gitkeep keeps the bind-mount target tracked. Phase 10.2 — 5 integration tests (//go:build integration) in deploy/test/est_e2e_test.go: - TestEST_LibESTClient_Enrollment_Integration (cacerts → simpleenroll → cert-shape assertion) - TestEST_LibESTClient_MTLSEnrollment_Integration (mTLS sibling-route cert auth; skip when bootstrap cert absent) - TestEST_LibESTClient_ServerKeygen_Integration (RFC 7030 §4.4 multipart; skip when profile gate disabled) - TestEST_LibESTClient_RateLimited_Integration (4th enroll trips per-principal cap, asserts 429-shaped error) - TestEST_LibESTClient_ChannelBinding_Integration (libest --tls-exporter; skip when libest build lacks the flag). - requireESTSidecar guard skips the suite when the operator forgot --profile est-e2e; helpful error message includes the exact command to bring the sidecar up. Phase 10.3 — Cisco IOS quirk fixtures + 3 unit tests in internal/api/handler/cisco_ios_quirks_test.go: - testdata/cisco_ios_15x_pem_csr.txt: PEM body sent with Content-Type application/x-pem-file. Handler dispatches on body-prefix not Content-Type — accepts cleanly. - testdata/cisco_ios_16x_trailing_newline_csr.txt: extra trailing newlines after base64 body. strings.TrimSpace tolerates. - testdata/cisco_ios_crlf_b64_csr.txt: CRLF-wrapped base64. base64.StdEncoding handles CRLF + LF identically. Phase 11.1 — ManagedCertificate.Source provenance: - New domain.CertificateSource enum (Unspecified/EST/SCEP/API/Agent). - Migration 000023_managed_certificates_source.up.sql adds source TEXT NOT NULL DEFAULT '' so existing rows scan as CertificateSourceUnspecified — back-compat: bulk-revoke filter treats empty as "any source". - Postgres repo Insert/Update/scan paths all wire the new column. Phase 11.2 — EST bulk-revoke endpoint: - BulkRevocationCriteria.Source field (Source-only requests rejected as too broad — must accompany at least one narrower criterion). - service.bulk_revocation.resolveCertificates post-filter by Source (empty=any, no SQL change so existing CertificateFilter callers unaffected). - New BulkRevocationHandler.BulkRevokeEST method pins Source=EST + dispatches; new route POST /api/v1/est/certificates/bulk-revoke (M-008 admin-gated). openapi.yaml documented + parity-guard green. Phase 11.3 — 13 typed audit action codes in internal/service/est_audit_actions.go: - est_simple_enroll_success / _failed - est_simple_reenroll_success / _failed - est_server_keygen_success / _failed - est_auth_failed_basic / _mtls / _channel_binding - est_rate_limited - est_csr_policy_violation - est_bulk_revoke - est_trust_anchor_reloaded - ESTService.processEnrollment + SimpleServerKeygen + ReloadTrust split-emit BOTH the legacy bare action codes (back-compat for the GUI activity-tab chip filters that match by exact string + existing audit-log analysers) AND the new typed _success / _failed variants (operator grep target + per-failure-mode counter). Tests: - internal/api/handler/bulk_revocation_est_test.go — 5 cases (admin-true happy path pins Source=EST + non-admin 403 + empty-criteria 400 + invalid-reason 400 + method-not-allowed). - internal/service/est_audit_actions_test.go — 5 cases (SimpleEnroll legacy+typed emission / SimpleReEnroll typed / IssuerError typed-failed / PolicyViolation triple-emit / unique-string invariant). Pre-commit verification (sandbox): gofmt clean, go vet clean (excluding repository/postgres testcontainers limit), staticcheck clean across api/handler/api/router/domain/service/deploy/test, go test -short -count=1 green for every non-postgres Go package + integration build (`go build -tags integration ./deploy/test/...`) clean. G-3 docs-drift guard reproduced locally clean (Phases 10-11 added zero new env vars). Spec preserved at cowork/est-rfc7030-hardening-prompt.md. Phases 12-13 (docs/est.md + WiFi/802.1X / IoT bootstrap / FreeRADIUS recipes; release prep + tag) remain — post-2.1.0 work. |
||
|
shankar0123
|
530593507b |
fix(scep-intune): close 11 audit gaps from 2026-04-29 pre-tag review
Closes the eleven gaps identified in the pre-v2.1.0 audit of the SCEP
RFC 8894 + Intune master bundle (cowork/scep-bundle-gap-closure-prompt.md).
Constitutional rule from cowork/CLAUDE.md::Operating Rules — 'Always
take the complete path, not the easy path' — drove this closure: each
gap was a load-bearing wire that crossed multiple layers (config →
validator → service wire-up → tests → docs) and shipping the bundle
without them would have produced lying-field footguns where operator-
visible config options stored values without affecting behavior.
WHAT LANDS:
Phase A — Clock-skew tolerance (master prompt §15 hazard closure)
internal/scep/intune/challenge.go: ValidateChallenge migrated from
positional args to ValidateOptions{} struct; new ClockSkewTolerance
field with default 0 (strict). 24 call sites updated mechanically.
Asymmetric application: now+tolerance >= iat AND now-tolerance < exp.
internal/config/config.go: SCEPIntuneProfileConfig.ClockSkewTolerance
default 60s + Validate() refusal when >= ChallengeValidity.
cmd/server/main.go: SetIntuneIntegration signature extended;
per-profile env-var loader honors CERTCTL_SCEP_PROFILE_<NAME>_INTUNE_CLOCK_SKEW_TOLERANCE.
internal/service/scep.go: intuneClockSkew field + IntuneStatsSnapshot
surfaces clock_skew_tolerance_ns. web/src/api/types.ts mirrors.
4 new tests in challenge_test.go covering accept-within-tolerance,
reject-beyond-tolerance, accept-expired-within-tolerance,
negative-treated-as-zero defensive normalization.
docs/scep-intune.md updated with the new env var + time-bounds rule.
Phase B — unknown-version-rejected golden test
internal/scep/intune/golden_helper_test.go: goldenUnknownVersionPayload
helper + signGoldenChallengeAny generic signer.
challenge_golden_test.go: TestGoldenChallenge_UnknownVersionRejected
uses an in-process ECDSA fixture (the on-disk PEM was generated with
a Go-stdlib version that produces different ecdsa.GenerateKey bytes
from the current call). TestRegenerateGoldenFixtures emits the new
unknown_version fixture file too.
Phase C — Two named Intune e2e tests
internal/api/handler/scep_intune_e2e_test.go:
TestSCEPIntuneEnrollment_RateLimited_E2E (cap=2 + 3 attempts; 3rd
returns FAILURE+badRequest with rate_limited counter ticked)
TestSCEPIntuneEnrollment_TrustAnchorSIGHUPReload_E2E (rotate
on-disk PEM + holder.Reload(); old-key challenge fails with
badMessageCheck; signature_invalid counter ticked)
intuneE2EFixture struct extended with trustHolder + trustPath fields
so tests can rotate.
Phase D — Four new ChromeOS hermetic tests (10 total now)
internal/api/handler/scep_chromeos_test.go:
_RAKeyMismatch — PKIMessage encrypted to wrong RA cert; handler
rejects without reaching service.
_3DESBackwardCompat — RFC 8894 §3.5.2 legacy fallback verified.
_RSACSR + _ECDSACSR — explicit matrix-pair pinning.
buildTestECDSACSR helper for ECDSA P-256 CSR construction;
tripleDESCBCEncrypt mirrors aesCBCEncrypt for 3DES-CBC;
assertChromeOSPositiveCertRep shared assertion.
Phase E — Per-profile counter isolation test
internal/api/handler/scep_profile_counter_isolation_test.go:
TestSCEPHandler_PerProfileIntuneCountersIsolated wires two
SCEPService instances + drives distinct PKIMessages + asserts
counter isolation. Guards against a future cmd/server/main.go
refactor that shares a *intuneCounterTab across profiles.
buildPerProfileIntuneFixture parameterized helper.
Phase F — Server-boot regression tests
cmd/server/preflight_scep_intune_test.go: 3 named tests covering
disabled-backward-compat, broken-config-with-PathID, expired-cert
refusal. preflightSCEPIntuneTrustAnchor signature extended with
pathID arg so error messages carry PathID= for operator log-grep.
Phase G — docs/connectors.md
Four new subsections under §EST/SCEP Integration: multi-profile
dispatch + mTLS sibling route + Intune Connector dispatcher + SCEP
probe in network scanner. Each has a one-paragraph operator
explanation + an env-var or endpoint table.
Phase H — Coverage uplift
internal/service/scep_probe_persist_test.go: 5 unit tests on
persistProbeResult (nil-safe + nil-repo-safe + repo-error swallow +
nil-logger guard) + ListRecentSCEPProbes (empty-slice-not-nil + repo
pass-through) + describeCertAlgorithm (RSA/ECDSA/QF1008-nil-curve
defensive branch/Ed25519/DSA/empty). CI gates (service ≥70, handler
≥75) PASS at 70.9% / 79.3%.
Phase I — deploy/test integration variant
deploy/test/scep_intune_e2e_test.go (//go:build integration):
TestSCEPIntuneEnrollment_Integration + _RateLimited_Integration
against the live docker-compose certctl container. Skip-when-
stack-missing semantics so sandbox + CI both work.
deploy/docker-compose.test.yml: new e2eintune SCEP profile env
vars + bind-mount of deploy/test/fixtures/.
deploy/test/fixtures/README.md: documents the deterministic trust
anchor regeneration recipe.
VERIFICATION (sandbox):
gofmt -d — clean for all changed files
staticcheck — clean for intune + handler + config + service +
cmd/server packages
go vet — clean for the same packages
go test -short — green for intune (95.3% cov), service (70.9%),
handler (79.3%), config (94.0%), cmd/server (boot
path; my preflight tests cover the directly-
testable function), pkcs7 (80.5% informational)
DEFERRED (per closure prompt §7 out-of-scope):
- V3-Pro Conditional Access gating + Microsoft Graph integration
- Standalone certctl-scan CLI binary
- OCSP rate-limiting, OCSP stapling, delta CRLs
Spec preserved at cowork/scep-bundle-gap-closure-prompt.md;
journal at cowork/scep-rfc8894-intune/progress.md (audit-closure
section appended).
|
||
|
shankar0123
|
a3d8b9c607 |
fix(deploy,db,handler): close fresh-clone postgres init failure + 4 ride-along audit findings (U-3 master)
GitHub #10 reopened: operator mikeakasully cloned v2.0.50 fresh and ran the canonical quickstart (docker compose -f deploy/docker-compose.yml up -d --build); postgres reported unhealthy indefinitely, dependent containers never started. Root cause: deploy/docker-compose.yml mounted a hand-curated subset of migrations/*.up.sql + seed.sql into postgres /docker-entrypoint-initdb.d/. Postgres applied them at initdb time. Once seed.sql referenced columns added by migrations *after* the mounted cutoff (e.g., policy_rules.severity from migration 000013), initdb crashed mid-seed and the container loop wedged. Two sources of truth (compose mount list vs in-tree migration ladder) diverged the moment a seed-touching migration shipped, and the only thing that fixed it was hand-editing the compose file every release. Fix: remove the dual source. Postgres boots empty; the server applies migrations + seed at startup via RunMigrations + RunSeed. Helm has used this pattern since day one (postgres-init emptyDir); compose now matches. Bundled with four ride-along audit findings whose fixes share the same schema/db code surface, so operators take the schema-change pain only once: cat-u-seed_initdb_schema_drift [P1, primary] — initdb-mount fix cat-o-retry_interval_unit_mismatch [P1] — column rename minutes→seconds cat-o-notification_created_at_dead_field [P2] — add column + populate cat-o-health_check_column_orphans [P1] — drop unwired columns cat-u-no_version_endpoint [P2] — add /api/v1/version Single migration (000017_db_coupling_cleanup) bundles the three schema changes under a DO \$\$ guard so re-application is safe; reduces operator-visible 'schema-change releases' from four to one. Backend - internal/repository/postgres/db.go: add RunSeed (baseline) + RunDemoSeed (gated by CERTCTL_DEMO_SEED). Both idempotent (ON CONFLICT DO NOTHING in every shipped INSERT) so repeated boots are safe; missing-file is no-op so custom packaging that strips seeds still boots cleanly. - cmd/server/main.go: invoke RunSeed (always) + RunDemoSeed (when flag set) immediately after RunMigrations. - internal/repository/postgres/notification.go: NotificationRepository.Create now sets created_at (with time.Now() fallback when caller leaves it zero); scanNotification reads it back; List + ListRetryEligible SELECT extended. - internal/repository/postgres/renewal_policy.go: column references updated to retry_interval_seconds across SELECT/INSERT/UPDATE sites. - internal/api/handler/version.go: new VersionHandler exposes {version, commit, modified, build_time, go_version} from runtime/debug.ReadBuildInfo() with ldflags-supplied Version override. - internal/api/router/router.go: register GET /api/v1/version through the no-auth chain (CORS + ContentType) alongside /health, /ready, /api/v1/auth/info. - cmd/server/main.go: add /api/v1/version to no-auth dispatch + audit ExcludePaths so rollout polling doesn't dominate the audit trail. - internal/config/config.go: add DatabaseConfig.DemoSeed + CERTCTL_DEMO_SEED env var. Migration - migrations/000017_db_coupling_cleanup.up.sql + .down.sql: (1) renewal_policies.retry_interval_minutes → retry_interval_seconds (DO \$\$ guard, idempotent re-application) (2) notification_events ADD COLUMN created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() (3) network_scan_targets DROP orphan health_check_enabled + health_check_interval_seconds - migrations/seed.sql: column reference updated to retry_interval_seconds. - migrations/seed_demo.sql: same column rename + applied at runtime now via RunDemoSeed (no longer initdb-mounted). Compose - deploy/docker-compose.yml: drop ALL initdb mounts (10 migration files + seed.sql); add start_period: 30s to postgres + certctl-server healthchecks to absorb the runtime migration + seed application window on first boot. - deploy/docker-compose.test.yml: same drop (+ ghost seed_test.sql mount removed; that file never existed); same healthcheck start_period. - deploy/docker-compose.demo.yml: replace seed_demo.sql initdb mount with CERTCTL_DEMO_SEED=true env var on certctl-server. Tests - internal/api/handler/version_handler_test.go: TestVersion_ReturnsBuildInfo, TestVersion_RejectsNonGet, TestVersion_LdflagsOverride. - internal/repository/postgres/seed_test.go: TestRunSeed_AppliesIdempotently, TestRunSeed_MissingFileIsNoOp, TestRunDemoSeed_AppliesIdempotently, TestMigration000017_RetryIntervalRename, TestMigration000017_NotificationCreatedAt, TestMigration000017_HealthCheckOrphansDropped (testcontainers, -short skips). - internal/repository/postgres/notification_test.go: TestNotificationRepository_CreatedAt_IsPersisted + TestNotificationRepository_CreatedAt_DefaultsToNow. CI guardrail - .github/workflows/ci.yml: new 'Forbidden migration mount in compose initdb (U-3)' step grep-fails the build if any migrations/*.sql or seed*.sql re-appears in /docker-entrypoint-initdb.d in any compose file. Catches future drift before a fresh-clone operator hits it. Spec / Docs - api/openapi.yaml: add /api/v1/version operation under Health tag. - docs/architecture.md: replace the 'initdb may run the same SQL' paragraph with a post-U-3 single-source-of-truth explanation. - CHANGELOG.md: full unreleased-section entry covering all 5 closures, breaking changes, and the new env var. Audit doc - coverage-gap-audit-2026-04-24-v5/unified-audit.md: add new P1 #14 cat-u-seed_initdb_schema_drift; flip the 4 ride-along findings to ✅ RESOLVED with closure prose pointing at this commit. Verification: build/vet/test -short -race all clean across all touched packages locally; govulncheck reports 0 vulnerabilities affecting our code; OpenAPI YAML parses; CI U-3 grep guardrail clears against the post-fix tree. |
||
|
shankar0123
|
55ce86b132 |
v2.0.48: swap self-signed TLS bootstrap algorithm ed25519 → ECDSA-P256
Follow-up to v2.0.47 (HTTPS-Everywhere). The Phase-3 self-signed bootstrap sidecar shipped an ed25519 server cert. Apple's TLS stack — Safari Network Framework and the macOS-bundled LibreSSL 3.3.6 /usr/bin/curl — does not advertise ed25519 in the ClientHello signature_algorithms extension for server certs, so the handshake fails with the server-side log line: tls: peer doesn't support any of the certificate's signature algorithms Homebrew OpenSSL 3.x, Chrome, Firefox, and Linux curl all accept ed25519 server certs fine. Apple is the outlier. Rather than gate the demo stack behind "install Homebrew OpenSSL first," swap the bootstrap algorithm to ECDSA-P256 with SHA-256 — universally supported, including on the Apple stack. Changes - deploy/docker-compose.yml: certctl-tls-init openssl invocation swapped to `-newkey ec -pkeyopt ec_paramgen_curve:P-256 -nodes`; header comment + echo line updated; multi-line rationale paragraph added. - deploy/docker-compose.test.yml: same openssl swap + echo update for the test harness sidecar that writes to the bind-mounted ./test/certs directory the Go integration_test.go pins via CERTCTL_TEST_CA_BUNDLE. - docs/tls.md: Pattern 1 description + code block updated; "Why ECDSA-P256 and not ed25519" rationale paragraph added covering pre-v2.0.48 history, the Apple diagnosis, accepting clients, and the operator migration command. Patterns 2 (existing Secret) and 3 (cert-manager) explicitly called out as unaffected. - docs/upgrade-to-tls.md: docker-compose procedure sentence updated with cross-reference to tls.md Pattern 1. - docs/test-env.md: "Get the CA bundle for curl" sentence updated. Migration Existing demo installs must tear the `certs` named volume down to pick up the new algorithm: docker compose -f deploy/docker-compose.yml down -v docker compose -f deploy/docker-compose.yml up -d --build Not touched - cmd/server/tls.go: algorithm-agnostic. TLS 1.3 min version with [X25519, P-256] curve preferences for key exchange is orthogonal to the server cert's signature algorithm. No Go code change needed. - Helm chart: Patterns 2 and 3 operators supply their own cert; this patch does not affect them. - Unrelated ed25519 uses (agent key algorithm detection, profile algorithm options, SSH key path examples, tlsprobe key metadata, cloud discovery key-algo display): all orthogonal to the server TLS bootstrap cert. Incidental cleanup - .gitignore: dropped dangling `strategy.md` entry (file doesn't exist in repo; entry was cruft). |
||
|
shankar0123
|
52248be717 |
v2.0.47: HTTPS Everywhere — TLS-only control plane, agents/CLI/MCP
Breaking change release. Plaintext HTTP listener removed. The certctl control plane now terminates TLS 1.3 on :8443 via http.Server.ListenAndServeTLS. No CERTCTL_TLS_ENABLED=false escape hatch. No dual-listener mode. One-step cutover per docs/upgrade-to-tls.md. Server - cmd/server/tls.go: certHolder with SIGHUP hot-reload + atomic cert swap, buildServerTLSConfig (TLS 1.3 min, GetCertificate callback), preflightServerTLS validation - cmd/server/main.go: ListenAndServeTLS in place of ListenAndServe, watchSIGHUP wiring, cert/key path config threading - tls_test.go: 418-line regression coverage of reload, preflight, callback behavior, SAN validation Config - CERTCTL_TLS_CERT_PATH / CERTCTL_TLS_KEY_PATH (required) - Plaintext rejection: agents/CLI/MCP pre-flight-fail on http:// URLs with a pointer to docs/upgrade-to-tls.md Agents, CLI, MCP - All three pre-flight-reject http:// URLs with fail-loud diagnostic - CERTCTL_SERVER_CA_BUNDLE_PATH for private-CA trust - CERTCTL_SERVER_TLS_INSECURE_SKIP_VERIFY for dev-only bypass (loud warning on startup) - install-agent.sh emits both vars as commented template lines docker-compose - certctl-tls-init sidecar generates SAN-valid self-signed cert into deploy/test/certs/ on first boot - All demo-stack curls pin against ca.crt with --cacert Helm chart - Three TLS provisioning modes, exactly one required: - server.tls.existingSecret (operator-supplied) - server.tls.certManager.enabled (cert-manager integration) - server.tls.selfSigned.enabled (eval only — not for production) - server-certificate.yaml template for cert-manager mode - helm install without a TLS source fails at template render with a pointer to docs/tls.md CI - .github/workflows/ci.yml Helm Chart Validation step renders the chart in both existingSecret and cert-manager modes, plus an inverse guard-regression test that asserts helm template MUST refuse to render when no TLS source is configured. Previously the single `helm template` invocation hit the certctl.tls.required fail-loud guard and exit-1'd CI. Four invocations now: lint (existingSecret), template (existingSecret), template (cert-manager), template (no args — must fail). Integration tests - deploy/test/integration_test.go stands up the Compose stack over HTTPS, extracts the CA bundle, and exercises every certctl API over https://localhost:8443 - All 34 integration subtests green (per Phase 8 local CI-parity) Documentation - New: docs/tls.md (provisioning patterns, rotation, SIGHUP reload) - New: docs/upgrade-to-tls.md (one-step cutover, no-downgrade warnings, fleet-roll sequencing) - CHANGELOG.md: v2.2.0 "HTTPS Everywhere — The Irony" entry (file heading unchanged; release tag is v2.0.47) - All curls in docs/, examples/, deploy/helm/ guides use https://localhost:8443 --cacert Verification - grep -rn "ListenAndServe[^T]" cmd/ internal/ → 0 hits - grep -rn "\"http://" cmd/ internal/ → 2 benign hits (Caddy admin API default, SSRF doc comment) — zero certctl endpoints - Tasks #197–#206 (Phases 0–8) all closed in the tracker Files: 65 changed, 3489 insertions, 372 deletions (pre-CI-fix). |
||
|
shankar0123
|
672e1d991d |
build: propagate HTTP_PROXY/HTTPS_PROXY/NO_PROXY through Docker build (M-4, Issue #9)
Addresses Medium finding M-4 in the audit report. The multi-stage
Dockerfiles previously had no ARG declarations for HTTP_PROXY,
HTTPS_PROXY, or NO_PROXY, so corporate-proxy environments silently
failed at 'npm ci' (frontend stage) and 'go mod download' (Go builder).
The npm retry idiom (`npm ci --include=dev || npm ci --include=dev`)
masked the failure because the upstream 'Exit handler never called!'
bug exits 0 despite the install crash.
Fix: thread HTTP_PROXY / HTTPS_PROXY / NO_PROXY ARGs through every
Docker build stage that performs network I/O, re-export them as ENV
with both upper- and lower-case aliases (apk/curl/npm read lowercase;
Go/Node read uppercase), and forward the host shell's environment via
`build.args:` in every compose file and `build-args:` in the release
workflow's docker/build-push-action steps. Defaults are empty strings
so un-proxied builds remain byte-identical to the pre-fix tree.
Scope: Dockerfile (frontend + Go builder stages), Dockerfile.agent
(Go builder stage), deploy/docker-compose.yml (server + agent),
deploy/docker-compose.dev.yml (server + agent), deploy/docker-compose.test.yml
(server + agent), .github/workflows/release.yml (both docker/build-push-action
v6 invocations). Zero Go, web, test, or runtime code changes. Zero
base-image changes. Existing npm `||` retry idiom and `ARG TARGETARCH`
preserved verbatim.
CWE-1173 (Improper Use of Validated Input) / CWE-16 (Configuration).
Verification:
- YAML parses clean across all four compose files and release.yml.
- yamllint -d relaxed: clean exit across all five YAML files.
- All six `build.args:` blocks expose HTTP_PROXY, HTTPS_PROXY, NO_PROXY
with default-empty ${VAR:-} substitution.
- Both release.yml docker/build-push-action steps expose the same
three keys sourced from ${{ secrets.HTTP_PROXY }}, etc.
- Dockerfiles contain 5 proxy ARG declarations total (Dockerfile has 2
stages × 3 ARGs = 6 lines, Dockerfile.agent has 1 stage × 3 ARGs = 3
lines); lowercase ENV aliases verified present in every stage.
- git diff --shortstat: 6 files changed, 117 insertions(+), 0 deletions.
Pure additive.
Docker-live verification (`docker build`, `docker compose config`)
deferred to CI / post-commit smoke because the sandbox has no Docker
runtime. hadolint, go, golangci-lint, govulncheck likewise unavailable
in the sandbox; per-layer CI coverage gates (service 55%, handler 60%,
domain 40%, middleware 30%) are trivially unaffected as M-4 touches
zero Go source files.
|
||
|
shankar0123
|
c6efa4ab39 |
docs: add Docker Compose environments guide and fix compose files
- New deploy/ENVIRONMENTS.md: comprehensive walkthrough of all 4 compose files with service-by-service explanations, beginner-friendly Docker concepts, and expert-level networking/config details - Fix docker-compose.dev.yml: agent LOG_LEVEL → CERTCTL_LOG_LEVEL (was silently ignored without the CERTCTL_ prefix) - Add CERTCTL_CONFIG_ENCRYPTION_KEY to base and test compose (enables M34/M35 dynamic issuer/target config encryption) - Add CERTCTL_DISCOVERY_DIRS to base compose agent (enables filesystem certificate discovery in default deployment) - Cross-link ENVIRONMENTS.md from README doc table and quickstart.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
shankar0123
|
8f146e08d6 |
feat(M36): onboarding wizard for first-run experience
4-step wizard (Connect CA → Deploy Agent → Add Certificate → Done) shown on fresh installs when no user-configured issuers or certificates exist. Auto-seeded env var issuers (source="env") are excluded from first-run detection. Wizard state latches to prevent query refetches from dismissing it mid-flow. Split docker-compose into clean default (wizard-compatible) and demo override (seed_demo.sql). Added missing migrations 000009/000010 to test compose. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
shankar0123
|
adfb682754 |
feat: Go integration test suite replacing bash end-to-end tests
Refactors deploy/test/run-test.sh into a typed Go test file with crypto/x509 certificate parsing, eliminating fragile openssl text scraping. 12 phases, 35 subtests covering Local CA, ACME, step-ca, revocation, discovery, renewal, EST, S/MIME, and API spot checks. - testClient HTTP helper with Bearer auth - testDB PostgreSQL helper (port 5432 now exposed) - waitFor/waitForJobsDone polling helpers - crypto/x509 for EKU, KeyUsage, SAN verification - crypto/tls for NGINX deployment verification - //go:build integration tag (not in CI yet) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
shankar0123
|
0822f748a5 |
feat: S/MIME certificate support in integration tests + test env docs
Add S/MIME (emailProtection EKU) end-to-end test coverage: - ValidateCommonName() now accepts email addresses for S/MIME certs - S/MIME test profile (prof-test-smime) in seed data - Phase 11 test: issuance, EKU, KeyUsage, email SAN verification - EST config enabled in test Docker Compose - Portable KeyUsage parsing (awk, works on BSD/GNU) - Full test environment documentation (docs/test-env.md) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
shankar0123
|
b059ec930f |
fix: end-to-end certificate lifecycle bugs + integration test environment
Fixes 12 production bugs preventing the full issuance→deployment flow from working with ACME (Pebble/Let's Encrypt) and step-ca issuers: ACME connector (acme.go): - Save orderURI before WaitOrder overwrites it (Go crypto/acme bug) - Add CreateOrderCert fallback via WaitOrder+FetchCert - Remove defer-reset in ValidateConfig that caused nil pointer panic - Add Insecure TLS option for self-signed ACME servers (Pebble) step-ca connector (stepca.go, jwe.go): - Real JWE provisioner key loading + decryption (was using ephemeral keys) - Fix JWT audience (/1.0/sign), sha claim (key fingerprint), kid header - Custom root CA trust via RootCertPath config - Remove hardcoded 90-day validity default (let step-ca decide) NGINX target connector (nginx.go): - Use sh -c for validate/reload commands (shell interpretation) - Use filepath.Dir instead of fragile string slicing - Add private key file writing (agent-mode keys were never deployed) - Make chain_path write conditional Server/service layer: - TriggerRenewalWithActor now creates actual Job records (was no-op) - createDeploymentJobs falls back to DB query when cert.TargetIDs empty - ProcessPendingJobs skips agent-routed deployment jobs - Agent cert pickup path parsing: len(parts)<4 → len(parts)<3 - Health/ready/auth-info endpoints bypass auth middleware - Write timeout 15s→120s for ACME issuance - Cert fingerprint computed on CSR submission Integration test environment (deploy/test/): - 10-phase test script covering Local CA, ACME, step-ca, revocation, discovery, renewal, and API spot checks - Docker Compose with 7 containers (server, agent, postgres, nginx, pebble, challtestsrv, step-ca) on isolated network - TLS verification checks SAN (not just Subject CN) for modern CA compat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |