mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-11 21:38:55 +00:00
02b1cb2bca
Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit.
377 lines
13 KiB
Python
377 lines
13 KiB
Python
"""
|
|
SSL Certificate PEM parsing utilities
|
|
"""
|
|
import re
|
|
import logging
|
|
from datetime import datetime, timezone
|
|
from typing import Optional, Dict, Any
|
|
from cryptography import x509
|
|
from cryptography.hazmat.primitives import hashes, serialization
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
def parse_ssl_certificate(cert_content: str) -> Dict[str, Any]:
|
|
"""
|
|
Parse SSL certificate PEM content and extract domain, expiry date, and other details
|
|
|
|
Args:
|
|
cert_content: PEM format certificate content
|
|
|
|
Returns:
|
|
Dictionary containing parsed certificate information
|
|
"""
|
|
try:
|
|
# Clean up the certificate content
|
|
cert_content = cert_content.strip()
|
|
|
|
# Ensure proper PEM format
|
|
if not cert_content.startswith('-----BEGIN CERTIFICATE-----'):
|
|
cert_content = '-----BEGIN CERTIFICATE-----\n' + cert_content
|
|
if not cert_content.endswith('-----END CERTIFICATE-----'):
|
|
cert_content = cert_content + '\n-----END CERTIFICATE-----'
|
|
|
|
# Parse the certificate using cryptography library
|
|
cert_bytes = cert_content.encode('utf-8')
|
|
certificate = x509.load_pem_x509_certificate(cert_bytes)
|
|
|
|
# Extract domain names (Subject Alternative Names + Common Name)
|
|
domains = []
|
|
|
|
# Get SAN (Subject Alternative Names)
|
|
try:
|
|
san_extension = certificate.extensions.get_extension_for_oid(x509.ExtensionOID.SUBJECT_ALTERNATIVE_NAME)
|
|
san_domains = [name.value for name in san_extension.value]
|
|
domains.extend(san_domains)
|
|
except x509.ExtensionNotFound:
|
|
logger.debug("No SAN extension found in certificate")
|
|
|
|
# Get Common Name from subject
|
|
try:
|
|
subject = certificate.subject
|
|
cn = subject.get_attributes_for_oid(x509.NameOID.COMMON_NAME)
|
|
if cn:
|
|
common_name = cn[0].value
|
|
if common_name not in domains:
|
|
domains.append(common_name)
|
|
except Exception as e:
|
|
logger.debug(f"Could not extract common name: {e}")
|
|
|
|
# Primary domain (first one or common name)
|
|
primary_domain = domains[0] if domains else "unknown"
|
|
|
|
# Extract expiry date - use not_valid_after and force UTC timezone
|
|
try:
|
|
# Try not_valid_after first (more compatible)
|
|
expiry_date = certificate.not_valid_after
|
|
logger.info(f"🕐 Using not_valid_after: {expiry_date}, tzinfo: {expiry_date.tzinfo}")
|
|
except AttributeError:
|
|
# Fallback to not_valid_after_utc
|
|
expiry_date = certificate.not_valid_after_utc
|
|
logger.info(f"🕐 Using not_valid_after_utc: {expiry_date}, tzinfo: {expiry_date.tzinfo}")
|
|
|
|
# Force timezone to UTC regardless of what we got
|
|
if expiry_date.tzinfo is None:
|
|
expiry_date = expiry_date.replace(tzinfo=timezone.utc)
|
|
logger.info(f"🔧 Added UTC timezone: {expiry_date}")
|
|
else:
|
|
# Convert any timezone to UTC
|
|
expiry_date = expiry_date.astimezone(timezone.utc)
|
|
logger.info(f"🔧 Converted to UTC: {expiry_date}")
|
|
|
|
# Extract issuer
|
|
issuer_name = "Unknown"
|
|
try:
|
|
issuer = certificate.issuer
|
|
org = issuer.get_attributes_for_oid(x509.NameOID.ORGANIZATION_NAME)
|
|
if org:
|
|
issuer_name = org[0].value
|
|
else:
|
|
cn = issuer.get_attributes_for_oid(x509.NameOID.COMMON_NAME)
|
|
if cn:
|
|
issuer_name = cn[0].value
|
|
except Exception as e:
|
|
logger.debug(f"Could not extract issuer: {e}")
|
|
|
|
# Calculate certificate status
|
|
now = datetime.now(timezone.utc)
|
|
logger.debug(f"Now datetime: {now}, tzinfo: {now.tzinfo}")
|
|
logger.debug(f"Expiry datetime: {expiry_date}, tzinfo: {expiry_date.tzinfo}")
|
|
|
|
try:
|
|
days_until_expiry = (expiry_date - now).days
|
|
logger.debug(f"Days until expiry calculated: {days_until_expiry}")
|
|
except Exception as e:
|
|
logger.error(f"Error calculating days until expiry: {e}")
|
|
# Fallback to 0 if calculation fails
|
|
days_until_expiry = 0
|
|
|
|
if days_until_expiry < 0:
|
|
status = "expired"
|
|
elif days_until_expiry < 30:
|
|
status = "expiring_soon"
|
|
else:
|
|
status = "valid"
|
|
|
|
# Get certificate fingerprint
|
|
fingerprint = certificate.fingerprint(hashes.SHA256()).hex()
|
|
|
|
return {
|
|
"primary_domain": primary_domain,
|
|
"all_domains": domains,
|
|
"expiry_date": expiry_date,
|
|
"issuer": issuer_name,
|
|
"status": status,
|
|
"days_until_expiry": days_until_expiry,
|
|
"fingerprint": fingerprint,
|
|
"serial_number": str(certificate.serial_number),
|
|
"version": certificate.version.name
|
|
}
|
|
|
|
except Exception as e:
|
|
logger.error(f"Failed to parse SSL certificate: {e}")
|
|
return {
|
|
"primary_domain": "parse_error",
|
|
"all_domains": [],
|
|
"expiry_date": None,
|
|
"issuer": "Unknown",
|
|
"status": "invalid",
|
|
"days_until_expiry": 0,
|
|
"fingerprint": "",
|
|
"serial_number": "",
|
|
"version": "unknown",
|
|
"error": str(e)
|
|
}
|
|
|
|
def validate_private_key(key_content: str) -> bool:
|
|
"""
|
|
Validate private key PEM content
|
|
|
|
Args:
|
|
key_content: PEM format private key content
|
|
|
|
Returns:
|
|
Boolean indicating if key is valid
|
|
"""
|
|
try:
|
|
# Clean up the key content
|
|
key_content = key_content.strip()
|
|
|
|
# Try to parse as different key types
|
|
key_bytes = key_content.encode('utf-8')
|
|
|
|
# Try RSA private key
|
|
try:
|
|
serialization.load_pem_private_key(key_bytes, password=None)
|
|
return True
|
|
except Exception:
|
|
pass
|
|
|
|
# Try with password (empty password)
|
|
try:
|
|
serialization.load_pem_private_key(key_bytes, password=b'')
|
|
return True
|
|
except Exception:
|
|
pass
|
|
|
|
return False
|
|
|
|
except Exception as e:
|
|
logger.error(f"Failed to validate private key: {e}")
|
|
return False
|
|
|
|
def validate_certificate_chain(chain_content: str) -> bool:
|
|
"""
|
|
Validate certificate chain PEM content
|
|
|
|
Args:
|
|
chain_content: PEM format certificate chain content
|
|
|
|
Returns:
|
|
Boolean indicating if chain is valid
|
|
"""
|
|
if not chain_content or not chain_content.strip():
|
|
return True # Chain is optional
|
|
|
|
try:
|
|
# Clean up the chain content
|
|
chain_content = chain_content.strip()
|
|
|
|
# Split multiple certificates in the chain
|
|
cert_pattern = r'-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----'
|
|
certificates = re.findall(cert_pattern, chain_content, re.DOTALL)
|
|
|
|
if not certificates:
|
|
return False
|
|
|
|
# Validate each certificate in the chain
|
|
for cert_pem in certificates:
|
|
try:
|
|
cert_bytes = cert_pem.encode('utf-8')
|
|
x509.load_pem_x509_certificate(cert_bytes)
|
|
except Exception:
|
|
return False
|
|
|
|
return True
|
|
|
|
except Exception as e:
|
|
logger.error(f"Failed to validate certificate chain: {e}")
|
|
return False
|
|
|
|
def verify_certificate_key_match(
|
|
cert_content: str, private_key_content: str
|
|
) -> Dict[str, Any]:
|
|
"""Bulgu #23 (round-12 audit): verify cert public key == private key
|
|
public key.
|
|
|
|
Pre-fix the wizard / direct SSL upload route validated cert and
|
|
key INDEPENDENTLY. An operator who pasted a cert for site A and
|
|
the private key for site B (easy to mix up when juggling many
|
|
PEMs) saw success and only learned about the mismatch at the
|
|
agent's `haproxy -c`, which errors with:
|
|
|
|
unable to load SSL private key from PEM file '...':
|
|
crypto/x509/x509_cmp.c:...: X509_check_private_key:
|
|
key values mismatch
|
|
|
|
by which point the wizard had already created the cert row, the
|
|
HTTPS frontend row, and the PENDING config version. Recovery
|
|
required hunting through Apply Management to reject the version.
|
|
|
|
Returns:
|
|
{"match": bool, "reason": Optional[str]}
|
|
- match=True → cert and key share the same public key.
|
|
- match=False → mismatch (cert/key are for different sites
|
|
or the key was rotated without re-issuing the cert).
|
|
- match=None → could not compare (e.g. encrypted key,
|
|
unsupported key type). Caller falls back to validate-key
|
|
only (which already ran).
|
|
"""
|
|
from cryptography.hazmat.primitives import serialization
|
|
|
|
if not (cert_content or "").strip() or not (private_key_content or "").strip():
|
|
return {"match": None, "reason": "empty cert or key content"}
|
|
|
|
try:
|
|
cert_bytes = cert_content.strip().encode("utf-8")
|
|
certificate = x509.load_pem_x509_certificate(cert_bytes)
|
|
except Exception as cert_err:
|
|
return {"match": None, "reason": f"cert parse failed: {cert_err}"}
|
|
|
|
key_bytes = private_key_content.strip().encode("utf-8")
|
|
key_obj = None
|
|
for password in (None, b""):
|
|
try:
|
|
key_obj = serialization.load_pem_private_key(
|
|
key_bytes, password=password
|
|
)
|
|
break
|
|
except Exception:
|
|
continue
|
|
if key_obj is None:
|
|
return {"match": None, "reason": "key parse failed (encrypted?)"}
|
|
|
|
try:
|
|
cert_pub_der = certificate.public_key().public_bytes(
|
|
encoding=serialization.Encoding.DER,
|
|
format=serialization.PublicFormat.SubjectPublicKeyInfo,
|
|
)
|
|
key_pub_der = key_obj.public_key().public_bytes(
|
|
encoding=serialization.Encoding.DER,
|
|
format=serialization.PublicFormat.SubjectPublicKeyInfo,
|
|
)
|
|
except Exception as compare_err:
|
|
return {"match": None, "reason": f"public-key serialization failed: {compare_err}"}
|
|
|
|
return {
|
|
"match": cert_pub_der == key_pub_der,
|
|
"reason": (
|
|
None if cert_pub_der == key_pub_der
|
|
else "cert public key differs from private key's public key"
|
|
),
|
|
}
|
|
|
|
|
|
def domain_covered_by_cert(domain: str, cert_san_or_cn: list) -> bool:
|
|
"""Bulgu #25 (round-12 audit): check whether a `domain` is covered
|
|
by any entry in the cert's SAN / Common-Name list, accounting for
|
|
RFC 6125 single-label wildcards.
|
|
|
|
HAProxy's SNI / cert matching follows RFC 6125 / RFC 9525:
|
|
|
|
* Literal match: cert SAN `api.example.com` matches `api.example.com`.
|
|
* Wildcard: cert SAN `*.example.com` matches `api.example.com`
|
|
(single leftmost label) but does NOT match `api.sub.example.com`
|
|
(two leftmost labels) and does NOT match the bare apex
|
|
`example.com` (no leftmost label).
|
|
|
|
Pre-fix the wizard let an operator deploy a site with
|
|
`domains=['shop.example.com']` and a cert for `api.example.com`
|
|
— HAProxy loads happily but every TLS handshake serves the wrong
|
|
cert, browser shows NET::ERR_CERT_COMMON_NAME_INVALID, and the
|
|
site is effectively down.
|
|
"""
|
|
if not domain or not cert_san_or_cn:
|
|
return False
|
|
domain_lc = domain.lower().strip().rstrip(".")
|
|
if not domain_lc:
|
|
return False
|
|
for cd in cert_san_or_cn:
|
|
cd_lc = (cd or "").lower().strip().rstrip(".")
|
|
if not cd_lc:
|
|
continue
|
|
if cd_lc == domain_lc:
|
|
return True
|
|
if cd_lc.startswith("*."):
|
|
parent = cd_lc[2:]
|
|
if not parent or "." not in parent:
|
|
continue
|
|
suffix = "." + parent
|
|
if domain_lc.endswith(suffix):
|
|
prefix = domain_lc[: -len(suffix)]
|
|
if prefix and "." not in prefix:
|
|
return True
|
|
return False
|
|
|
|
|
|
def find_uncovered_domains(domains: list, cert_san_or_cn: list) -> list:
|
|
"""Return the subset of `domains` NOT covered by any SAN/CN entry,
|
|
preserving the input order so the error message lists them as
|
|
the operator typed them.
|
|
"""
|
|
if not domains:
|
|
return []
|
|
return [d for d in domains if not domain_covered_by_cert(d, cert_san_or_cn or [])]
|
|
|
|
|
|
def format_certificate_info(cert_info: Dict[str, Any]) -> str:
|
|
"""
|
|
Format certificate information for display
|
|
|
|
Args:
|
|
cert_info: Parsed certificate information
|
|
|
|
Returns:
|
|
Formatted string for display
|
|
"""
|
|
if cert_info.get("error"):
|
|
return f"❌ Invalid Certificate: {cert_info['error']}"
|
|
|
|
status_emoji = {
|
|
"valid": "✅",
|
|
"expiring_soon": "⚠️",
|
|
"expired": "❌",
|
|
"invalid": "❌"
|
|
}
|
|
|
|
emoji = status_emoji.get(cert_info["status"], "❓")
|
|
domain = cert_info["primary_domain"]
|
|
expiry = cert_info["expiry_date"]
|
|
issuer = cert_info["issuer"]
|
|
|
|
if expiry:
|
|
expiry_str = expiry.strftime("%Y-%m-%d")
|
|
days = cert_info["days_until_expiry"]
|
|
return f"{emoji} {domain} | Expires: {expiry_str} ({days} days) | Issuer: {issuer}"
|
|
else:
|
|
return f"{emoji} {domain} | Issuer: {issuer}"
|