mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-21 18:13:25 +00:00
02b1cb2bca
Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit.
286 lines
11 KiB
Python
286 lines
11 KiB
Python
"""v1.5.0 R12 — Advanced wizard fields (HAProxy 2.4+ tuning).
|
|
|
|
Round 12 expanded the wizard surface with:
|
|
- BackendStep: extended balance algorithms (parameter-FREE only — `hdr` /
|
|
`url_param` rejected because they need a parameter), fullconn, cookie
|
|
persistence, default-server-* defaults, request/response headers,
|
|
validator forbidding cookie persistence with mode='tcp'.
|
|
- FrontendStep: maxconn, rate_limit, timeouts, compression, log_separate,
|
|
monitor_uri, header injection, tcp_request_rules.
|
|
- ServerStep: extended ssl_verify Literal, TLS version Literals.
|
|
- SSLChoice: ALPN, ssl_min_ver, ssl_max_ver, ciphers, ciphersuites,
|
|
strict-sni, HSTS opt-in.
|
|
|
|
These tests guarantee:
|
|
1. New fields default to safe / unset values (BACKWARD COMPAT — existing
|
|
wizard payloads from v1.5.0 first deploy must still parse, AND the
|
|
neutral defaults must NOT silently change the cipher / ALPN / HSTS
|
|
profile of frontends created via direct API calls).
|
|
2. Field constraints are enforced (numeric ranges, Literal lists).
|
|
3. New TLS Literals cover HAProxy 2.4+ supported versions.
|
|
"""
|
|
import pytest
|
|
from pydantic import ValidationError
|
|
|
|
from models.site_wizard import (
|
|
BackendStep,
|
|
FrontendStep,
|
|
ServerStep,
|
|
SSLChoice,
|
|
)
|
|
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# BackendStep advanced
|
|
# ----------------------------------------------------------------------------
|
|
|
|
|
|
def test_backend_step_new_balance_algorithms_accepted():
|
|
"""Round 12: only PARAMETER-FREE algorithms are accepted by the wizard.
|
|
Parametric ones (`hdr(name)`, `url_param(name)`) need an extra input
|
|
field that the wizard does not provide today — they are rejected to
|
|
avoid producing invalid HAProxy syntax (`balance hdr` without args)."""
|
|
for algo in ("roundrobin", "leastconn", "static-rr", "first", "source", "uri", "random"):
|
|
b = BackendStep(name="be_test", balance_method=algo)
|
|
assert b.balance_method == algo
|
|
|
|
|
|
@pytest.mark.parametrize("bad", ["hdr", "url_param", "rdp-cookie", "not-a-real-algo"])
|
|
def test_backend_step_invalid_balance_rejected(bad):
|
|
"""Parametric algorithms must be REJECTED until the wizard exposes a
|
|
parameter input. Otherwise HAProxy errors out at apply time with a
|
|
cryptic 'invalid keyword' message."""
|
|
with pytest.raises(ValidationError):
|
|
BackendStep(name="be_test", balance_method=bad)
|
|
|
|
|
|
def test_backend_step_cookie_on_tcp_mode_rejected():
|
|
"""Round 12 (Bulgu #f8): HAProxy `cookie` directive is HTTP-only.
|
|
The wizard must reject the combination at validation time so the
|
|
user gets a clear actionable error instead of an apply-time HAProxy
|
|
syntax failure."""
|
|
with pytest.raises(ValidationError, match="mode='http'"):
|
|
BackendStep(name="be_test", mode="tcp", cookie_name="SRVID")
|
|
|
|
|
|
def test_backend_step_cookie_on_http_mode_accepted():
|
|
"""Counter-test: cookie persistence on http mode is allowed."""
|
|
b = BackendStep(
|
|
name="be_test", mode="http", cookie_name="SRVID",
|
|
cookie_options="insert indirect nocache",
|
|
)
|
|
assert b.cookie_name == "SRVID"
|
|
|
|
|
|
def test_backend_step_tcp_mode_without_cookie_accepted():
|
|
"""Counter-test: TCP backend without cookie is fine."""
|
|
b = BackendStep(name="be_test", mode="tcp")
|
|
assert b.mode == "tcp"
|
|
assert b.cookie_name is None
|
|
|
|
|
|
def test_backend_step_advanced_defaults_unset():
|
|
b = BackendStep(name="be_test")
|
|
# New fields must default to None so existing payloads don't break.
|
|
assert b.fullconn is None
|
|
assert b.cookie_name is None
|
|
assert b.cookie_options is None
|
|
assert b.default_server_inter is None
|
|
assert b.default_server_fall is None
|
|
assert b.default_server_rise is None
|
|
assert b.request_headers is None
|
|
assert b.response_headers is None
|
|
|
|
|
|
def test_backend_step_cookie_persistence_roundtrip():
|
|
b = BackendStep(
|
|
name="be_test",
|
|
cookie_name="SRVID",
|
|
cookie_options="insert indirect nocache",
|
|
)
|
|
assert b.cookie_name == "SRVID"
|
|
assert b.cookie_options == "insert indirect nocache"
|
|
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# FrontendStep advanced
|
|
# ----------------------------------------------------------------------------
|
|
|
|
|
|
def test_frontend_step_advanced_defaults_safe():
|
|
f = FrontendStep(name="fe_test")
|
|
# No defaults that would change existing behaviour.
|
|
assert f.maxconn is None
|
|
assert f.rate_limit is None
|
|
assert f.compression is False
|
|
assert f.log_separate is False
|
|
assert f.monitor_uri is None
|
|
assert f.tcp_request_rules is None
|
|
assert f.request_headers is None
|
|
assert f.response_headers is None
|
|
|
|
|
|
def test_frontend_step_rate_limit_must_be_non_negative():
|
|
with pytest.raises(ValidationError):
|
|
FrontendStep(name="fe_test", rate_limit=-1)
|
|
|
|
|
|
def test_frontend_step_maxconn_must_be_positive():
|
|
with pytest.raises(ValidationError):
|
|
FrontendStep(name="fe_test", maxconn=0)
|
|
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# ServerStep advanced TLS literals
|
|
# ----------------------------------------------------------------------------
|
|
|
|
|
|
def test_server_step_ssl_verify_literal():
|
|
ServerStep(server_name="s", server_address="10.0.0.1", server_port=80, ssl_verify="none")
|
|
ServerStep(server_name="s", server_address="10.0.0.1", server_port=80, ssl_verify="required")
|
|
with pytest.raises(ValidationError):
|
|
ServerStep(server_name="s", server_address="10.0.0.1", server_port=80, ssl_verify="bogus")
|
|
|
|
|
|
# R18c audit fix (round 3 #15): TLSv1.0 / TLSv1.1 are formally
|
|
# deprecated by RFC 8996 and are now REJECTED by the wizard
|
|
# validators. The legacy assertion that "any of the four literal
|
|
# values is accepted" is therefore split: the modern values pass,
|
|
# the deprecated values raise ValidationError. This test is the
|
|
# back-compat hinge — if a future revision re-allows TLS 1.0 / 1.1
|
|
# for any reason, that change should be deliberate and visible
|
|
# here.
|
|
@pytest.mark.parametrize("ver", ["TLSv1.2", "TLSv1.3"])
|
|
def test_server_step_tls_version_literals(ver):
|
|
ServerStep(server_name="s", server_address="10.0.0.1", server_port=80, ssl_min_ver=ver)
|
|
|
|
|
|
@pytest.mark.parametrize("ver", ["TLSv1.0", "TLSv1.1"])
|
|
def test_server_step_tls_version_literals_rejects_deprecated(ver):
|
|
from pydantic import ValidationError
|
|
with pytest.raises(ValidationError):
|
|
ServerStep(
|
|
server_name="s",
|
|
server_address="10.0.0.1",
|
|
server_port=80,
|
|
ssl_enabled=True,
|
|
ssl_min_ver=ver,
|
|
)
|
|
|
|
|
|
def test_server_step_invalid_tls_version_rejected():
|
|
with pytest.raises(ValidationError):
|
|
ServerStep(server_name="s", server_address="10.0.0.1", server_port=80, ssl_min_ver="SSLv3")
|
|
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# SSLChoice advanced TLS / HSTS
|
|
# ----------------------------------------------------------------------------
|
|
|
|
|
|
def test_ssl_choice_alpn_default_none_for_backward_compat():
|
|
"""Round 12 backward-compat fix: ssl_alpn default MUST be None at the
|
|
Pydantic layer. Setting a non-empty default ('h2,http/1.1') would
|
|
silently enable HTTP/2 on HTTPS frontends created via direct API
|
|
callers (or saved drafts from the v1.5.0 first deploy). The wizard
|
|
UI populates a sensible form-level initial value; the model stays
|
|
neutral."""
|
|
s = SSLChoice(mode="none")
|
|
assert s.ssl_alpn is None
|
|
|
|
|
|
def test_ssl_choice_min_tls_default_none_for_backward_compat():
|
|
"""Round 12 backward-compat fix: ssl_min_ver default MUST be None.
|
|
A 'TLSv1.2' default would silently disable TLSv1.0/1.1 for any
|
|
HTTPS frontend that previously left ssl_min_ver unset — a behaviour
|
|
change for direct API callers. The wizard UI populates 'TLSv1.2' as
|
|
a form initial value; the model stays neutral."""
|
|
s = SSLChoice(mode="none")
|
|
assert s.ssl_min_ver is None
|
|
|
|
|
|
def test_ssl_choice_explicit_alpn_and_tls_versions_pass_through():
|
|
"""When the wizard form sends 'h2,http/1.1' and 'TLSv1.2', the
|
|
model accepts and round-trips them unchanged."""
|
|
s = SSLChoice(mode="acme", ssl_alpn="h2,http/1.1", ssl_min_ver="TLSv1.2")
|
|
assert s.ssl_alpn == "h2,http/1.1"
|
|
assert s.ssl_min_ver == "TLSv1.2"
|
|
|
|
|
|
def test_ssl_choice_hsts_off_by_default():
|
|
s = SSLChoice(mode="none")
|
|
assert s.hsts_enabled is False
|
|
assert s.hsts_max_age == 31536000 # 1 year
|
|
assert s.hsts_include_subdomains is True
|
|
assert s.hsts_preload is False
|
|
|
|
|
|
def test_ssl_choice_strict_sni_off_by_default():
|
|
s = SSLChoice(mode="none")
|
|
assert s.ssl_strict_sni is False
|
|
|
|
|
|
# R18c audit fix (round 3 #15): same TLS 1.0 / 1.1 rejection as
|
|
# the ServerStep validator, but on the HTTPS frontend bind.
|
|
@pytest.mark.parametrize("ver", ["TLSv1.2", "TLSv1.3"])
|
|
def test_ssl_choice_tls_version_literals(ver):
|
|
SSLChoice(mode="none", ssl_min_ver=ver)
|
|
|
|
|
|
@pytest.mark.parametrize("ver", ["TLSv1.0", "TLSv1.1"])
|
|
def test_ssl_choice_tls_version_literals_rejects_deprecated(ver):
|
|
from pydantic import ValidationError
|
|
with pytest.raises(ValidationError):
|
|
SSLChoice(mode="none", ssl_min_ver=ver)
|
|
|
|
|
|
def test_ssl_choice_invalid_tls_version_rejected():
|
|
with pytest.raises(ValidationError):
|
|
SSLChoice(mode="none", ssl_min_ver="TLSv0.99")
|
|
|
|
|
|
def test_ssl_choice_rejects_unknown_auto_renew_before_days_field():
|
|
"""Round 12 (Bulgu #f3): the per-cert renew-before-days override is
|
|
NOT plumbed into the renewal scheduler — so the field is removed
|
|
from the wizard payload to avoid offering a placebo control. The
|
|
Pydantic model must NOT accept it (stale draft would otherwise
|
|
appear to set a value that is silently ignored)."""
|
|
# SSLChoice does NOT have auto_renew_before_days. Direct callers
|
|
# passing it get an extra-field error (or it's ignored, depending on
|
|
# extra=). The strict assertion is just that the model has no such
|
|
# attribute.
|
|
s = SSLChoice(mode="acme")
|
|
assert not hasattr(s, "auto_renew_before_days"), (
|
|
"auto_renew_before_days must remain unimplemented until v1.6.0 "
|
|
"renewal scheduler picks up per-cert overrides."
|
|
)
|
|
|
|
|
|
# ----------------------------------------------------------------------------
|
|
# Backward compat: payload from v1.5.0 first deploy (sans advanced) parses
|
|
# ----------------------------------------------------------------------------
|
|
|
|
|
|
def test_v15_payload_without_advanced_still_parses():
|
|
"""If a saved draft from the v1.5.0 first deploy lacked all the new
|
|
advanced fields, the model must still validate."""
|
|
minimal_backend = BackendStep(name="be_legacy", balance_method="roundrobin", mode="http")
|
|
assert minimal_backend.fullconn is None # absence is fine
|
|
|
|
minimal_frontend = FrontendStep(
|
|
name="fe_legacy", mode="http", bind_address="*", bind_port=80
|
|
)
|
|
assert minimal_frontend.maxconn is None
|
|
|
|
minimal_server = ServerStep(
|
|
server_name="srv1", server_address="10.0.0.1", server_port=8080
|
|
)
|
|
assert minimal_server.ssl_enabled is False
|
|
|
|
legacy_ssl_acme = SSLChoice(mode="acme", auto_renew=True)
|
|
# Backward-compat: defaults stay None so v1.5.0-first-deploy callers
|
|
# get the same HAProxy ssl-* directive set they got before R12.
|
|
assert legacy_ssl_acme.ssl_alpn is None
|
|
assert legacy_ssl_acme.ssl_min_ver is None
|
|
assert legacy_ssl_acme.hsts_enabled is False
|