loadtest: per-connector deploy throughput scenarios + target sidecars + README baseline section

Closes Bundle 10 of the 2026-05-02 deployment-target coverage audit (see cowork/deployment-target-audit-2026-05-02/RESULTS.md). Pre-fix, deploy/test/loadtest/k6.js drove only the API-tier throughput path (POST /api/v1/certificates + GET /api/v1/certificates) — the operator- facing rate at which an automation client can submit cert requests. The deploy hot path (cert deployed to a target — connector-tier latency) had no benchmarks. Procurement asks "can certctl handle our 5,000-NGINX fleet at 47-day rotation?" and the answer should be a number with methodology, not a claim. This commit ships v1 of the connector-tier loadtest harness: 1. Target-side sidecars added to docker-compose.yml: nginx-target, apache-target, haproxy-target, f5-mock-target. Each daemon serves a starter cert (ECDSA P-256, multi-SAN) written into a shared ./fixtures/target-certs/ volume by a new target-tls-init container. f5-mock-target re-uses the in-tree deploy/test/f5-mock-icontrol/ image (already used by the deploy- vendor-e2e CI job) and generates its own self-signed cert via tls.go::selfSignedCert at startup. 2. Fixture configs committed under deploy/test/loadtest/fixtures/: - nginx.conf — minimal HTTPS server, single 200 OK location. - httpd.conf — self-contained Apache config with the minimum module set + SSL vhost. - haproxy.cfg — minimal SSL-terminating frontend backed by a static "ok" backend. 3. k6 scenarios added (4 new): nginx_handshake, apache_handshake, haproxy_handshake, f5_handshake. Each runs constant-arrival-rate at 100 conns/min for 5 minutes. Latency captured by k6's http_req_duration metric covers TCP connect + TLS handshake + tiny HTTP request/response — that's the end-to-end "connection readiness" latency a deploy connector cares about. 4. summary.json gains a connector_tier object with per-target p50/p95/p99/max/avg/error_rate/iterations breakdowns. Operators tracking a connector regression diff connector_tier.<type> between runs. Implementation: a new enrichWithConnectorTier helper that reads data.metrics keyed by target_type tag and shallow-merges the breakdown into the summary before serialisation. 5. Threshold contract per target type: - nginx/apache/haproxy: p99 < 3s, p95 < 1s. - f5-mock: p99 < 5s, p95 < 1.5s (iControl REST handler does slightly more work per request than pure TLS termination). - All scenarios: error rate < 1% (k6 default; any 4xx/5xx counts as failed). Any change pushing past these fails the workflow. 6. README documents the methodology + the baseline-number table for the connector tier. Numeric values are em-dash placeholders pending the first clean canonical-hardware run; the accompanying commit message in that follow-up captures the methodology line alongside the numbers. Out-of-scope is documented explicitly: - Full agent-driven deploy poll loop (POST cert with target binding → poll deployments endpoint → verify served cert). v2 of the harness — needs the agent registration + target- binding API surface plumbed end-to-end in the loadtest stack. - Kubernetes target via kind-in-docker. kind requires `privileged: true` and is operationally fragile in CI; deferred until Bundle 2 (real k8s.io/client-go) lands and a CI-friendly envtest harness is wired. - Real F5 BIG-IP. CI uses the in-tree f5-mock; real-appliance benchmarking is out of scope. 7. CI workflow .github/workflows/loadtest.yml timeout-minutes bumped from 15 to 25. The harness now boots four additional target sidecars before the k6 run; their healthchecks add ~30-60s. The k6 scenarios themselves are still 5 minutes (run in parallel, not serially). 25 minutes absorbs that plus slow CI runners and cold image caches without letting a stuck container consume the runner indefinitely. Trigger remains workflow_dispatch + cron — sustained 25-minute runs are too slow for per-PR signal. What this connector tier explicitly does NOT measure (documented in the k6.js header + README): - The agent-driven full deploy hot path (v2 follow-up). - K8s target (Bundle 2 dependency). - Real F5 appliance. - Issuer-side throughput (handled by issuer-coverage-audit fix #8). Verified locally: - python3 -c "import yaml; yaml.safe_load(...)" on docker-compose.yml and .github/workflows/loadtest.yml — clean. - node -c on k6.js — clean syntax. - gofmt / go vet on the rest of the tree (no Go diff in this commit). - Manual smoke against docker-compose pending — operator validates on the canonical-hardware first run; if any fixture config is off, fix-up commit lands separately so the methodology change and the numeric baseline have independent reviewability. No Go code changes; this is a loadtest-harness-only commit. Audit reference: cowork/deployment-target-audit-2026-05-02/RESULTS.md Bundle 10.
2026-06-07 17:12:04 +00:00 · 2026-05-02 19:28:45 +00:00
parent 08a86d355d
commit e292faafc6
8 changed files with 677 additions and 47 deletions
@@ -0,0 +1,29 @@
+# HAProxy target sidecar — Bundle 10 of the 2026-05-02 deployment-target audit.
+#
+# Minimal SSL-terminating config that boots green with the starter cert
+# written by target-tls-init. The k6 connector-tier scenarios connect at
+# sustained 100 conns/min and measure handshake-completion latency.
+
+global
+    log stdout local0 warning
+    maxconn 4096
+    # Bundle 10: starter cert+key live at /usr/local/etc/haproxy/certs/.
+    # HAProxy expects a SINGLE PEM file containing cert + key concatenated;
+    # the target-tls-init container writes target.pem in that combined form.
+    ssl-default-bind-options ssl-min-ver TLSv1.2
+
+defaults
+    log global
+    mode http
+    option dontlognull
+    timeout connect 5s
+    timeout client 30s
+    timeout server 30s
+
+frontend https-in
+    bind *:443 ssl crt /usr/local/etc/haproxy/certs/target.pem
+    default_backend ok
+
+backend ok
+    # Static 200 OK — handshake-only loadtest doesn't exercise the backend.
+    http-request return status 200 content-type text/plain string "ok\n"
@@ -0,0 +1,66 @@
+# Apache httpd target sidecar — Bundle 10 of the 2026-05-02 deployment-target audit.
+#
+# Self-contained httpd.conf that the httpd:2.4-alpine image will use as its
+# main configuration. Loads the minimum module set required for an HTTPS
+# server + serves a single SSL-enabled vhost backed by the starter cert
+# written by target-tls-init.
+
+ServerRoot "/usr/local/apache2"
+Listen 443
+
+# Module set is the minimum required for the SSL vhost below + the
+# directives Apache parses elsewhere in its bootstrap.
+LoadModule mpm_event_module modules/mod_mpm_event.so
+LoadModule authn_file_module modules/mod_authn_file.so
+LoadModule authn_core_module modules/mod_authn_core.so
+LoadModule authz_host_module modules/mod_authz_host.so
+LoadModule authz_user_module modules/mod_authz_user.so
+LoadModule authz_core_module modules/mod_authz_core.so
+LoadModule access_compat_module modules/mod_access_compat.so
+LoadModule auth_basic_module modules/mod_auth_basic.so
+LoadModule reqtimeout_module modules/mod_reqtimeout.so
+LoadModule filter_module modules/mod_filter.so
+LoadModule mime_module modules/mod_mime.so
+LoadModule log_config_module modules/mod_log_config.so
+LoadModule env_module modules/mod_env.so
+LoadModule headers_module modules/mod_headers.so
+LoadModule setenvif_module modules/mod_setenvif.so
+LoadModule version_module modules/mod_version.so
+LoadModule unixd_module modules/mod_unixd.so
+LoadModule dir_module modules/mod_dir.so
+LoadModule alias_module modules/mod_alias.so
+LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
+LoadModule ssl_module modules/mod_ssl.so
+
+User daemon
+Group daemon
+
+ServerName apache-target
+ServerAdmin loadtest@certctl.local
+
+# Quiet log so the run log stays diff-able. Errors still go to stderr
+# (/proc/self/fd/2) so docker compose logs surfaces them on startup
+# failure.
+ErrorLog /proc/self/fd/2
+LogLevel warn
+
+DocumentRoot "/usr/local/apache2/htdocs"
+
+# Bundle 10: starter cert+key from target-tls-init's shared volume.
+SSLEngine On
+SSLCertificateFile     /usr/local/apache2/conf/certs/target.crt
+SSLCertificateKeyFile  /usr/local/apache2/conf/certs/target.key
+SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
+SSLCipherSuite HIGH:!aNULL:!MD5
+SSLHonorCipherOrder on
+
+<Directory "/usr/local/apache2/htdocs">
+    AllowOverride None
+    Require all granted
+</Directory>
+
+# Quiet response — the loadtest scenarios only care that the handshake
+# completes. The body content is irrelevant.
+<Location />
+    Require all granted
+</Location>
@@ -0,0 +1,36 @@
+# nginx target sidecar — Bundle 10 of the 2026-05-02 deployment-target audit.
+#
+# Minimal HTTPS-only config that boots green with a starter cert from the
+# shared target-tls-init container. The k6 connector-tier scenarios connect
+# at sustained 100 conns/min and measure handshake-completion latency.
+# Production NGINX configs are far richer; this is a load-test fixture, not
+# a deployment template.
+
+worker_processes 1;
+events {
+    worker_connections 1024;
+}
+
+http {
+    # Quiet log so the loadtest run doesn't fill the docker-compose log.
+    access_log off;
+    error_log /var/log/nginx/error.log warn;
+
+    server {
+        listen 443 ssl;
+        server_name _;
+
+        # Bundle 10: starter cert+key written by target-tls-init into the
+        # shared volume. Not the deployed cert; this is what makes the
+        # daemon boot green so the loadtest scenarios have something to
+        # handshake against.
+        ssl_certificate     /etc/nginx/certs/target.crt;
+        ssl_certificate_key /etc/nginx/certs/target.key;
+        ssl_protocols TLSv1.2 TLSv1.3;
+
+        location / {
+            return 200 "ok\n";
+            add_header Content-Type text/plain;
+        }
+    }
+}