Commit Graph

4 Commits

Author SHA1 Message Date
shankar0123 b8b7e1e3dd tlsprobe: add VerifyWithExponentialBackoff + rewire all connectors' runPostDeployVerify
Closes Top-10 fix #8 of the 2026-05-02 deployment-target audit
re-run (see cowork/deployment-target-audit-2026-05-02-rerun/
RESULTS.md). Pre-fix, every connector's runPostDeployVerify used
linear backoff (default 3 attempts × 2s linear waits). Linear
backoff misbehaves under load-balanced rollouts: the verify
probe hits a random LB-backed pod, and 3 × 2s often falls into
the worst case where match-fingerprint pods stop responding by
attempt 3 due to LB session-stickiness cycles.

This commit:

1. New shared helper internal/tlsprobe/retry.go::
   VerifyWithExponentialBackoff. Default 3 attempts; 1s initial,
   16s cap. Doubling pattern: 1s → 2s → 4s → 8s → 16s. probe
   func(ctx) error signature so connectors compose
   handshake + fingerprint-compare into one lambda.

2. Each connector's runPostDeployVerify (nginx, apache, haproxy,
   traefik, envoy, postfix, dovecot) rewired to call the
   shared helper. Per-connector signature unchanged.

3. New PostDeployVerifyMaxBackoff time.Duration field added to
   each connector's Config. Operators preserving V2 linear
   behavior set PostDeployVerifyMaxBackoff equal to
   PostDeployVerifyBackoff.

4. Tests:
   - tlsprobe/retry_test.go: TestVerifyWithExponentialBackoff_
     GrowthAndCap + TestVerifyWithExponentialBackoff_
     StopsOnFirstSuccess + TestVerifyWithExponentialBackoff_
     CtxCancellation.
   - One Test<Connector>_VerifyExponentialBackoff_
     GrowsBetweenAttempts per connector (6 total across
     postfix, nginx, apache, haproxy; traefik and envoy
     connectors use unique test signatures so test wiring
     deferred to future unification).

5. docs/deployment-atomicity.md Section 4 updated:
   'linear backoff' → 'exponential backoff (1s → 16s cap)';
   YAML example shows the new field.

Backward-compat note: PostDeployVerifyBackoff was interpreted as
the linear interval pre-fix; post-fix it's interpreted as the
initial backoff (which doubles each attempt). Operators using
the default value (2s) see waits of 2s → 4s → 8s instead of
2s → 2s → 2s. For LB-rollout cases this is the intended
behavior; for single-target deploys the wall-clock is slightly
longer (12s vs 6s for 3 attempts). Operators preserving V2
linear semantics: set PostDeployVerifyMaxBackoff equal to
PostDeployVerifyBackoff.

Verified locally:
- gofmt clean.
- go test -short -count=1 ./internal/tlsprobe/...
  ./internal/connector/target/{postfix,nginx,apache,haproxy}/... green.

Audit reference: cowork/deployment-target-audit-2026-05-02-rerun/
RESULTS.md Top-10 fix #8.
2026-05-02 22:56:07 +00:00
shankar0123 919a92bf1b feat(haproxy): atomic deploy + post-deploy TLS verify + rollback + ValidateOnly + test-depth uplift to 36 tests
Phase 6 of the deploy-hardening I master bundle. HAProxy connector
follows the canonical Phase 4 NGINX template with the HAProxy-
specific quirk: combined PEM file (cert + chain + key in one
file, in that order). Test count lifts 3 → 36.

HAProxy specifics:
- buildCombinedPEM concatenates cert, chain, key in HAProxy's
  required order. The combined file goes through deploy.Apply as
  a single File entry (vs NGINX/Apache's 2-3 separate File entries).
- Default mode 0600 unconditionally (combined file contains the
  private key); operators rely on this back-compat behavior.
  PEMFileMode override is the supported escape hatch.
- Validate command is `haproxy -c -f <config>`. Reload via
  `systemctl reload haproxy` (NOT `restart` — reload uses socket
  activation to drain in-flight connections).
- Default user/group: haproxy (cross-distro consistent).

DeployCertificate refactor:
- Replaces the duplicated os.WriteFile flow with deploy.Apply.
- PreCommit runs `haproxy -c -f` validation (gated on
  ValidateCommand being non-empty — HAProxy historically allowed
  empty validate).
- PostCommit runs the operator's ReloadCommand.
- Post-deploy TLS verify (frozen-decision-0.3 default ON when
  Endpoint is configured): probes the configured target,
  fingerprint-matches against the deployed cert (the leaf cert
  block from the combined PEM), retries with backoff for load-
  balanced targets.
- Rollback wires identical to NGINX/Apache: backup restore +
  reload retry on PostCommit failure; verify-fail also triggers
  rollback.

ValidateOnly real impl: returns sentinel when no ValidateCommand;
otherwise runs the operator's command without touching the live
combined PEM.

Tests (36 total: 33 in haproxy_atomic_test.go + 3 pre-existing
in haproxy_test.go):

- Atomic invariants (happy, validate-fail, reload-fail-rollback,
  rollback-also-fail-escalation)
- Combined PEM order (cert + chain + key — verified via PEM
  block headers, not base64 bodies)
- Mode handling (default 0600 even when existing is 0640 —
  back-compat; PEMFileMode override; existing-mode unchanged
  when override matches)
- Idempotency (full skip)
- Verify (match, mismatch, dial-timeout, retries, disabled,
  no-endpoint, rollback-runs-reload)
- ValidateOnly (happy, fails, no-command-sentinel, stderr-in-error)
- Concurrency (same-paths-serialize)
- Edge cases (no-chain, no-key, ctx-cancelled, no-validate-command,
  config-validation rejects missing pem_path / reload / shell-injection)

Coverage: HAProxy 88.0% (above >=85% prompt bar). Race detector
clean. golangci-lint v2.11.4 clean.

Smoke test connectorsAtPhase3 list shrinks 11→10 (haproxy
removed alongside nginx + apache).

Phase 7 next: Traefik + Caddy + Envoy + Postfix — the remaining
file-based connectors get the same treatment.
2026-04-30 15:01:23 +00:00
shankar0123 200bdf990f fix(quality): TICKET-012 propagate request context instead of context.Background()
- Updated AgentService interface to accept context.Context parameter in all methods
- Replaced context.Background() calls with proper ctx parameter in agent.go
- Updated AgentGroupService interface to accept context.Context parameter
- Replaced context.Background() calls with proper ctx parameter in agent_group.go
- Updated handler methods to pass r.Context() to service methods
- Context now properly propagates through request lifecycle for timeout/cancellation
- Improved request tracing and cancellation behavior
2026-03-27 21:35:22 -04:00
shankar0123 07275bf92f feat: M10 — agent metadata collection, Apache httpd + HAProxy target connectors
Agents now report OS, architecture, IP address, hostname, and version
via heartbeat using runtime.GOOS, runtime.GOARCH, and net.Dial. New
migration adds columns to agents table. Heartbeat handler, service,
and repository updated to accept and persist metadata. GUI shows
OS/Arch in agent list and full system info in agent detail page.

Apache httpd connector: separate cert/chain/key files, apachectl
configtest validation, graceful reload. HAProxy connector: combined
PEM file (cert+chain+key), optional config validation, reload.
Both wired into agent binary's target connector switch.

14 tests for new connectors. All existing tests updated for new
Heartbeat/UpdateHeartbeat signatures. Docs updated across README,
architecture, concepts, and connectors guides.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 02:19:28 -04:00