3 Commits

Author	SHA1	Message	Date
shankar0123	7f6bfed03c	tlsprobe: add VerifyWithExponentialBackoff + rewire all connectors' runPostDeployVerify ... Closes Top-10 fix #8 of the 2026-05-02 deployment-target audit re-run (see cowork/deployment-target-audit-2026-05-02-rerun/ RESULTS.md). Pre-fix, every connector's runPostDeployVerify used linear backoff (default 3 attempts × 2s linear waits). Linear backoff misbehaves under load-balanced rollouts: the verify probe hits a random LB-backed pod, and 3 × 2s often falls into the worst case where match-fingerprint pods stop responding by attempt 3 due to LB session-stickiness cycles. This commit: 1. New shared helper internal/tlsprobe/retry.go:: VerifyWithExponentialBackoff. Default 3 attempts; 1s initial, 16s cap. Doubling pattern: 1s → 2s → 4s → 8s → 16s. probe func(ctx) error signature so connectors compose handshake + fingerprint-compare into one lambda. 2. Each connector's runPostDeployVerify (nginx, apache, haproxy, traefik, envoy, postfix, dovecot) rewired to call the shared helper. Per-connector signature unchanged. 3. New PostDeployVerifyMaxBackoff time.Duration field added to each connector's Config. Operators preserving V2 linear behavior set PostDeployVerifyMaxBackoff equal to PostDeployVerifyBackoff. 4. Tests: - tlsprobe/retry_test.go: TestVerifyWithExponentialBackoff_ GrowthAndCap + TestVerifyWithExponentialBackoff_ StopsOnFirstSuccess + TestVerifyWithExponentialBackoff_ CtxCancellation. - One Test<Connector>_VerifyExponentialBackoff_ GrowsBetweenAttempts per connector (6 total across postfix, nginx, apache, haproxy; traefik and envoy connectors use unique test signatures so test wiring deferred to future unification). 5. docs/deployment-atomicity.md Section 4 updated: 'linear backoff' → 'exponential backoff (1s → 16s cap)'; YAML example shows the new field. Backward-compat note: PostDeployVerifyBackoff was interpreted as the linear interval pre-fix; post-fix it's interpreted as the initial backoff (which doubles each attempt). Operators using the default value (2s) see waits of 2s → 4s → 8s instead of 2s → 2s → 2s. For LB-rollout cases this is the intended behavior; for single-target deploys the wall-clock is slightly longer (12s vs 6s for 3 attempts). Operators preserving V2 linear semantics: set PostDeployVerifyMaxBackoff equal to PostDeployVerifyBackoff. Verified locally: - gofmt clean. - go test -short -count=1 ./internal/tlsprobe/... ./internal/connector/target/{postfix,nginx,apache,haproxy}/... green. Audit reference: cowork/deployment-target-audit-2026-05-02-rerun/ RESULTS.md Top-10 fix #8.	2026-05-02 22:56:07 +00:00
shankar0123	88e8881224	postfix: add atomic-test variants for Mode=dovecot (happy path + verify-rollback) ... Closes Bundle 11 of the 2026-05-02 deployment-target coverage audit (see cowork/deployment-target-audit-2026-05-02/RESULTS.md). Pre-fix, postfix_atomic_test.go exercised the atomic deploy path under Mode= postfix only — the existing TestPostfix_DovecotMode at L233-246 asserted only the DeploymentID prefix, leaving applyDefaults's dovecot-specific validate/reload command set + the rollback's file-content-restoration unverified at the deploy-test layer. Audit's only test-coverage gap on the otherwise-production-grade Postfix/Dovecot connector. This commit adds two new tests (test-only commit; no production- code changes): 1. TestPostfix_Atomic_DovecotMode_HappyPath. Builds a Config with Mode: "dovecot" and NO ValidateCommand / NO ReloadCommand set. Calls ValidateConfig (which is what triggers applyDefaults via its JSON-marshal-then-parse path) before DeployCertificate. Captures the validate + reload commands threaded through the SetTestRunValidate / SetTestRunReload hooks. Asserts: - capturedValidateCmd contains "doveconf -n" (applyDefaults populated it from the dovecot branch). - capturedReloadCmd contains "doveadm reload". - DeploymentID prefix "dovecot-" + result.Metadata["mode"] is "dovecot" (Mode survived end-to-end). 2. TestPostfix_Atomic_DovecotMode_VerifyFails_Rollback. Pre-creates cert.pem AND key.pem with known "ORIG-CERT" / "ORIG-KEY" bytes. Builds Config with Mode: "dovecot", PostDeployVerify enabled (Endpoint pointing at a dovecot-IMAPS-style :993 — value unused by the probe stub), PostDeployVerifyAttempts: 1 (default is 3 attempts × 2s backoff = 4+ seconds; we don't need that for a unit test). Probe stub returns Success: false, which runPostDeployVerify wraps as "TLS probe failed: ...". Asserts: - DeployCertificate returns error containing "TLS probe failed". - cert.pem AND key.pem on disk contain the ORIG bytes verbatim — Bundle 11's load-bearing assertion that the rollback restored the pre-deploy file state under Mode=dovecot. The existing TestPostfix_VerifyMismatch_Rollback (Mode=postfix) only asserts the error; this test extends to file-content restoration. Existing TestPostfix_DovecotMode (L233-246) preserved as-is — the minimal DeploymentID-prefix smoke test complements the new richer tests without duplicating their scope. The encoding/json import is added to support the HappyPath test's json.Marshal call. No other dependency changes. No production-code changes; the connector itself was already correct for Mode=dovecot. Only the test pin was missing. Verified locally: - gofmt -l ./internal/connector/target/postfix/ clean - go vet ./internal/connector/target/postfix/ clean - go build ./cmd/agent/... clean (no signature changes) - go test -race -count=1 ./internal/connector/target/postfix/ green (24 tests total: 22 pre-existing + 2 new) Audit reference: cowork/deployment-target-audit-2026-05-02/RESULTS.md Bundle 11.	2026-05-02 19:34:58 +00:00
claude	758dbb283f	feat(traefik,caddy,envoy,postfix): atomic deploy + post-deploy TLS verify + rollback + ValidateOnly ... Phase 7 of the deploy-hardening I master bundle. Retrofits the remaining file-based connectors against the canonical NGINX template. Per-connector quirks codified: - Postfix/Dovecot: full retrofit with PreCommit (postfix check / doveconf -n) + PostCommit (postfix reload / doveadm reload) + post-deploy TLS verify. Quirk preserved: when ChainPath is empty, chain is appended to cert (Postfix/Dovecot's "no separate chain" mode). Per-distro user defaults: postfix, dovecot, _postfix. Default key mode 0600. ValidateOnly real impl returns sentinel when no ValidateCommand. - Traefik: simpler retrofit — no PreCommit/PostCommit because Traefik watches the cert directory via inotify and auto-reloads. Atomic-write via deploy.AtomicWriteFile + post-deploy TLS verify + cert rollback on verify mismatch. Default key mode 0600. ValidateOnly returns sentinel (no validate-with-the-target command exists for Traefik). - Caddy: retrofitted both modes. File mode replaces os.WriteFile with deploy.AtomicWriteFile (preserves the file watcher's auto- reload). API mode unchanged (POST /load already atomic at the Caddy admin server). ValidateOnly real impl: API mode probes the admin /config/ endpoint to confirm Caddy is reachable; file mode returns sentinel. - Envoy: file mode atomic-write via deploy.AtomicWriteFile. Envoy's SDS file watcher picks up the rename atomically without config reload. ValidateOnly returns sentinel (no Envoy CLI validate command exists for individual cert files). Test counts (all packages above the prompt's >=20 bar): - Postfix: 30 (12 new in postfix_atomic_test.go + 18 pre-existing) - Traefik: 22 (12 new in traefik_atomic_test.go + 10 pre-existing) - Caddy: 22 (10 new in caddy_atomic_test.go + 12 pre-existing) - Envoy: 21 (5 new in envoy_atomic_test.go + 16 pre-existing) Coverage: each connector at the prompt's >=80% target. golangci-lint v2.11.4 clean across all 4 connector packages. Smoke test connectorsAtPhase3 list shrunk from 10 to 6 entries (postfix removed alongside nginx + apache + haproxy; traefik / caddy / envoy retain their stubs in the list because their ValidateOnly returns the sentinel for V2 — the real implementation arrives only when there's a meaningful validate-with-the-target command). Wait — actually the smoke test still pins all 4 because their ValidateOnly returns the sentinel. Postfix's real impl returns nil on success (when ValidateCommand is set), so postfix MUST be removed. Caddy's API mode is real-impl. Traefik + Envoy still return sentinel always — they stay in the smoke list. Phase 8 next: F5 + IIS — explicit post-deploy TLS verify + on-failure rollback. Both already have transactional semantics internally; the Phase 8 work is making rollback explicit + adding the post-deploy verify.	2026-04-30 15:12:11 +00:00