feat(ssl): encrypt the pending CSR private key at rest (v1.10.1, closes #53)

Closes the follow-up filed during the v1.9.0 CSR review. The private key of a
PENDING CSR is now Fernet-encrypted in the database instead of being stored as
a raw PEM.

Why this key specifically: it is the one key in the system that sits idle. It
is generated at CSR creation, waits for an external CA to sign the request
(days to weeks), and is destroyed the moment the signed certificate is
imported. It is never transmitted to an agent and never leaves the server.
ssl_certificates.private_key_content and the ACME order keys are deliberately
NOT covered, because agents must receive those in plaintext on every poll, so
encrypting them at rest buys nothing without an end-to-end redesign.

Implementation follows the pattern already used for the VRRP secret, TOTP
secrets and DNS provider credentials: a new utils/csr_key_crypto.py with its
own CSR_ENCRYPTION_KEY env var and its own HKDF info string
("csr-private-key-v1"), so rotating one secret class never affects another.

No schema change and deliberately NO SCHEMA_VERSION bump: the Fernet token
replaces the PEM inside the existing ssl_csrs.private_key_pem TEXT column. A
bump would re-run the migration sequence and re-seed the four built-in roles to
their defaults, which is a needless side effect for a storage-format change.

Backward compatible with no data migration. Rows written before this release
hold a raw PEM and are still read unchanged; the discriminator is exact rather
than a heuristic, since a Fernet token is base64url and can never contain the
"-----BEGIN" marker. Legacy rows drain naturally because a CSR's key copy is
NULLed on import.

A key that cannot be decrypted (SECRET_KEY rotated while CSR_ENCRYPTION_KEY was
unset) now fails with an explicit "delete this CSR and create a new one" error.
Previously that situation would have surfaced as the far more confusing
"certificate does not match this CSR's private key".

Also documents all four per-purpose encryption keys in .env.template. Only
VIP_ENCRYPTION_KEY was listed; MFA_ENCRYPTION_KEY and
DNS_PROVIDER_ENCRYPTION_KEY had been missing since v1.6.0 and v1.8.0.

Verified before release, on a corporate pre-production environment and locally:
- Full backend suite 1234 -> 1243 passed (+9 new tests), 0 failed.
- Against a real Postgres: a CSR created through the API stores a Fernet token
  with no PEM header in the column, and imports successfully.
- Full 1.10.0 -> 1.10.1 -> 1.10.0 drill on one database volume. The upgrade
  logs "Schema already at version 10 (>= 10); skipping migration run", so no
  migration executes and the built-in roles are not re-seeded. A CSR created on
  1.10.0 with a plaintext key imports successfully after the upgrade, which is
  the backward-compatibility guarantee proven against a real row rather than a
  mock.
- rsa-2048, rsa-4096 and ecdsa-p384 all round-trip through create, encrypt,
  decrypt and import.
- Key derivation is stable across processes: two independent containers sharing
  SECRET_KEY decrypt each other's tokens (required for UVICORN_WORKERS > 1 and
  multi-replica deployments), while a different SECRET_KEY yields None rather
  than a wrong key or an exception.
- Downgrade behaviour was measured, not assumed: 1.10.0 cannot parse the token
  and fails with HTTP 500 "key parse failed (encrypted?)" rather than pairing a
  wrong key. The rollback note states the measured behaviour.
- No CSR endpoint returns the key in any form: list and detail responses
  contain neither a PEM nor a Fernet token.

Not changed here, from the issue's "worth folding in" list: the create rate
limit is not a concurrency guard, create_csr holds a pooled connection across
RSA key generation, detail=str(e) echoes internal error text (a repo-wide
convention), and is_global skips cluster validation in both routers/ssl.py and
routers/csr.py. None are storage concerns and each is a separate change.

.env.template

@@ -17,11 +17,34 @@ REDIS_URL=redis://redis:6379
 # Change this to a strong random string in production
 SECRET_KEY=your-secret-key-change-this-in-production
 
-# Optional: dedicated Fernet key for encrypting VRRP secrets of HA/VIP (Issue #27).
-# If unset, it is derived from SECRET_KEY (HKDF), exactly like MFA. Set an explicit
-# key (urlsafe-base64, 32 bytes) in production if you want independent key rotation.
+# ----------------------------------------------------------------------------
+# Optional per-purpose encryption keys.
+#
+# Every secret the application stores is encrypted at rest with Fernet. Each class
+# derives its own key, so rotating one never affects another. If a variable below is
+# unset, that class's key is derived from SECRET_KEY via HKDF — which works, but means
+# rotating SECRET_KEY makes the existing values of that class UNDECRYPTABLE. Set an
+# explicit key (urlsafe-base64, 32 bytes) in production if you want independent
+# rotation. Generate one with:
+#   python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
+# ----------------------------------------------------------------------------
+
+# VRRP secrets for HA/VIP (Issue #27).
 # VIP_ENCRYPTION_KEY=
 
+# TOTP secrets for multi-factor authentication (Issue #18).
+# MFA_ENCRYPTION_KEY=
+
+# Per-account DNS provider credentials for ACME DNS-01 (Issue #35).
+# Rotating this without re-entering credentials makes DNS-01 renewals fail until the
+# affected accounts' credentials are re-saved in ACME Automation.
+# DNS_PROVIDER_ENCRYPTION_KEY=
+
+# Private keys of PENDING CSRs, held only until the signed certificate is imported
+# (Issue #53). Rotating this while CSRs are out for signature makes those CSRs
+# unusable — they must be deleted and re-created.
+# CSR_ENCRYPTION_KEY=
+
 # ============================================================================
 # PUBLIC URL CONFIGURATION
 # ============================================================================

README.md

@@ -2474,6 +2474,7 @@ Developed with ❤️ for the HAProxy community
 
 ## Release Notes
 
+- **v1.10.1** (2026-08-08) — **CSR private key encrypted at rest** (Issue #53): the private key of a **pending** CSR is now Fernet-encrypted in the database instead of stored as PEM. It is the one key in the system worth protecting this way — it sits idle for the entire signing window (days to weeks), is never transmitted to an agent, and is destroyed the moment the signed certificate is imported; `ssl_certificates.private_key_content` and the ACME order keys are unchanged, because agents must receive those in plaintext on every poll. The token replaces the PEM in the **same column**, so there is **no schema change and no `SCHEMA_VERSION` bump** (and therefore no re-seed of the built-in roles). CSRs created before this release keep a raw PEM and are still read transparently, so anything already out for signature imports normally with no data migration. The key derives from `SECRET_KEY` via HKDF with its own info string, independent of the VIP/MFA/DNS keys, and an optional `CSR_ENCRYPTION_KEY` enables independent rotation — rotating `SECRET_KEY` without it makes pending CSR keys unrecoverable, which now fails with an explicit "delete and re-create this CSR" error rather than a misleading key-mismatch. `.env.template` now documents all four per-purpose encryption keys. No API, UI or agent change.
 - **v1.10.0** (2026-08-07) — **GoDaddy DNS provider for DNS-01** (Issue #35 follow-up): DNS-01 challenges can now be published and cleaned up automatically through **GoDaddy**, alongside the existing Manual and Cloudflare providers, so wildcard and internal-cluster certificates on GoDaddy-hosted zones **renew unattended**. Credentials are a **Production API Key + Secret** pair from `developer.godaddy.com/keys` (a **Personal Access Token** also works — paste it as the Key and leave the Secret blank, which is the forward path as GoDaddy retires `sso-key`); they are **verified against the GoDaddy API before being saved** and **encrypted at rest** (Fernet, the same path as Cloudflare), and are never returned by the API, logged, or written to an order event. GoDaddy's v1 API has **no per-value TXT write** — `PUT` replaces an entire RRset — so add/remove are read-modify-write with sibling values merged back, empty-`data` tombstones filtered out, and `DELETE` used for the last value (`PUT []` is rejected); this is what keeps the **apex + wildcard** case (two TXT values at one `_acme-challenge` name) working, and the record path is hard-gated so it can never collapse onto the zone-wide endpoint that would wipe SPF/DKIM/DMARC. Zone lookup probes the records API rather than the domain listing, so **delegated sub-zones** resolve and small accounts are not falsely rejected. Registry-only addition: one new provider module plus one registry line — no frontend change (the credential form is schema-driven). No schema, API-shape, agent, or rendered-config changes; Manual, Cloudflare and HTTP-01 are unaffected.
 - **v1.9.0** (2026-08-04) — **CSR creation** (in-app key + CSR generation and signed-certificate import): a new **CSR tab** on the SSL Certificates page generates a private key and Certificate Signing Request server-side (RSA 2048/4096 or ECDSA P-256/P-384; full subject — O/OU/L/ST/C/email — plus DNS SANs with wildcard support), for certificates signed by an **external or corporate CA**. The operator downloads/copies the CSR PEM, has it signed, then imports the signed certificate (+ optional chain): the backend verifies the certificate against the stored key (hard gate), rejects expired certs, warns on SAN drift, and creates a normal SSL certificate entry (source `CSR`) that flows through the standard **PENDING → Apply Management → agent pull** pipeline. The private key **never leaves the server** — no CSR endpoint returns it, and after import the CSR row's key copy is destroyed (the key then lives only on the certificate, like every other key). Additive schema change: one new table `ssl_csrs` (SCHEMA_VERSION 9 → 10, auto-migrated, no existing table altered); key generation runs off the event loop and is rate-limited per user; existing `ssl.*` permissions govern all new endpoints. No agent or rendered-config changes.
 - **v1.8.10** (2026-07-20) — **Security hardening** (GHSA-7rhv-c5pc-69r8, GHSA-3p5c-m5m4-mjpx, GHSA-3vh4): three advisory classes remediated, backend-only, no agent changes. (1) **RCE**: the agent script-template read/write endpoints now require the `agents.version` permission on top of authentication — a poisoned template is executed as root on every HAProxy node, so authentication alone was insufficient. (2) **Missing authentication**: operator/UI endpoints that were served without a JWT (dashboard stats, pool/cluster listings, agent inventory, WAF rules, config validate/optimize, SSL config-versions, health deep/agents/clusters) are now gated by a `require_authenticated_user` dependency, and agent data-plane endpoints that treated the `X-API-Key` header as *optional* (heartbeat, config, ssl-certificates, upgrade-status, pending-requests) now hard-reject a missing key. In every case the auth check was moved **ahead of** the handler's `try:` block so a 401 can no longer be rewritten into a 500 by the generic exception handler. (3) **SSRF**: a new `utils/ssrf_guard.py` (https-only, IPv4-pinned connector, all resolved addresses must be public, no redirects) protects the ACME directory fetch, the signed-request target and the ACME connection test, which accept operator- or DB-supplied URLs; the connection test also stopped reflecting arbitrary upstream JSON. Frontend dependency advisories patched in the same release. No schema, API-shape or rendered-config changes.

UPGRADE_GUIDE.md

@@ -1,3 +1,42 @@
+# Upgrade Notes — v1.10.1 (CSR private key encrypted at rest)
+
+**Backward compatible.** Nothing to do on upgrade, and nothing changes for existing clusters,
+agents or certificates:
+
+- **Schema:** **no `SCHEMA_VERSION` bump and no migration.** The Fernet token replaces the PEM
+  inside the *existing* `ssl_csrs.private_key_pem` TEXT column. As in v1.10.0, this means the
+  four built-in roles are **not** re-seeded, so any customization of `super_admin` / `operator` /
+  `security_admin` / `viewer` survives.
+- **Existing pending CSRs keep working.** Rows written before this release hold a raw PEM and are
+  still read transparently, so a CSR that is already out for signature can be imported normally
+  after the upgrade. There is no data migration and no downtime step. Those rows stay plaintext
+  until they are imported (which NULLs the key) — if you want everything encrypted immediately,
+  delete and re-create any long-pending CSRs.
+- **Scope:** this covers the PENDING CSR key only. It is the one key in the system that sits idle
+  for the whole signing window and is never transmitted. `ssl_certificates.private_key_content`
+  and the ACME order keys are unchanged, because agents must receive those in plaintext on every
+  poll.
+- **Optional env:** `CSR_ENCRYPTION_KEY` (see `.env.template`). If unset, the key is derived from
+  `SECRET_KEY` via HKDF with its own info string, so it is independent of the VIP, MFA and DNS
+  provider keys.
+  - **⚠️ Rotating `SECRET_KEY` while `CSR_ENCRYPTION_KEY` is unset makes pending CSR keys
+    unrecoverable.** Import then fails with an explicit "delete this CSR and create a new one"
+    error rather than a misleading key-mismatch. Set an explicit `CSR_ENCRYPTION_KEY` if you
+    rotate `SECRET_KEY`. Certificates already imported are unaffected — their key lives on the
+    certificate row.
+- **API / UI / agents:** unchanged. No CSR endpoint ever returned the private key before or now,
+  and nothing about the CSR tab changes.
+- **Rollback:** the application downgrades cleanly — 1.10.0 starts normally against the same
+  database and every other feature is unaffected. The one casualty is a CSR **created on 1.10.1
+  and still pending**: 1.10.0 has no decrypt step, so it hands the Fernet token straight to the
+  key-pairing check. Measured on a real downgrade, the import then fails with
+  `HTTP 500 — Could not verify the certificate/key pair: key parse failed (encrypted?)`; it does
+  **not** silently pair the wrong key, and it does not corrupt anything. Import or delete CSRs
+  created on 1.10.1 before downgrading. Certificates already imported are unaffected, since their
+  key lives on the certificate row, and CSRs created before 1.10.1 are plaintext and still work.
+
+---
+
 # Upgrade Notes — v1.10.0 (GoDaddy DNS-01 provider)
 
 **Backward compatible & additive.** Nothing changes unless you select **GoDaddy** as an ACME

backend/services/csr_service.py

@@ -19,8 +19,14 @@ subject instead of CN-only, ECDSA support, same PKCS8/NoEncryption key
 serialisation (the agent concatenates cert+key+chain into one PEM and HAProxy
 cannot read passphrase-protected keys).
 
-Private keys are stored PLAINTEXT, consistent with every other key in the
-system (ssl_certificates.private_key_content, letsencrypt_orders.cert_private_key).
+Private keys are ENCRYPTED AT REST from v1.10.1 (Issue #53): the Fernet token
+replaces the PEM in the same `ssl_csrs.private_key_pem` column, so there is no
+schema change and no SCHEMA_VERSION bump. Rows written earlier hold a raw PEM
+and are still read transparently — see utils/csr_key_crypto.py for the format
+discriminator and the key-rotation caveat. The pending CSR key is the one key
+in the system worth encrypting: it sits idle for the whole signing window and
+is never transmitted, unlike ssl_certificates.private_key_content and the ACME
+order keys, which agents must receive in plaintext on every poll.
 The key is NEVER returned by any CSR API response — `csr_row_to_dict` strips
 it unconditionally.
 """
@@ -39,6 +45,7 @@ from cryptography.hazmat.primitives.asymmetric import ec, rsa
 from cryptography.x509.oid import NameOID
 
 from services import ssl_service
+from utils.csr_key_crypto import decrypt_csr_private_key, encrypt_csr_private_key
 
 logger = logging.getLogger(__name__)
 
@@ -186,8 +193,14 @@ async def assert_csr_name_available(conn, name: str) -> None:
 
 
 async def insert_csr_row(conn, payload: Any, bundle: Dict[str, Any], user_id: Optional[int]) -> int:
-    """Persist a freshly generated CSR bundle. Returns the new csr id."""
+    """Persist a freshly generated CSR bundle. Returns the new csr id.
+
+    Issue #53 (v1.10.1): the private key is Fernet-encrypted before it is stored. The token goes
+    into the SAME private_key_pem TEXT column — no schema change — and is only ever decrypted
+    in-process by import_signed_certificate. No CSR endpoint returns the column either way.
+    """
     await assert_csr_name_available(conn, payload.name)
+    stored_key = encrypt_csr_private_key(bundle['private_key_pem'])
     try:
         csr_id = await conn.fetchval(
             """
@@ -203,7 +216,7 @@ async def insert_csr_row(conn, payload: Any, bundle: Dict[str, Any], user_id: Op
             json.dumps(bundle['sans']),
             payload.key_algorithm,
             bundle['csr_pem'],
-            bundle['private_key_pem'],
+            stored_key,
             user_id,
         )
     except asyncpg.exceptions.UniqueViolationError:
@@ -243,8 +256,7 @@ async def import_signed_certificate(conn, csr_id: int, imp: Any, user_id: Option
                     "Create a new CSR to reissue."
                 ),
             )
-        stored_key = row['private_key_pem']
-        if not stored_key:
+        if not row['private_key_pem']:
             raise HTTPException(
                 status_code=500,
                 detail=(
@@ -252,6 +264,23 @@ async def import_signed_certificate(conn, csr_id: int, imp: Any, user_id: Option
                     "corrupt. Delete it and create a new CSR."
                 ),
             )
+        # Issue #53: the column holds a Fernet token from v1.10.1 on, and a raw PEM for rows
+        # written before it. decrypt_csr_private_key accepts both, so no data migration is
+        # needed. A None here means the token cannot be decrypted — SECRET_KEY was rotated
+        # without CSR_ENCRYPTION_KEY set. Fail loudly: the key is gone, so the CA's certificate
+        # can never be paired with it, and silently falling through would surface as the far
+        # more confusing "certificate does not match this CSR's private key".
+        stored_key = decrypt_csr_private_key(row['private_key_pem'])
+        if not stored_key:
+            raise HTTPException(
+                status_code=500,
+                detail=(
+                    f"The stored private key for CSR '{row['name']}' cannot be decrypted. This "
+                    "happens when SECRET_KEY was rotated while CSR_ENCRYPTION_KEY was not set. "
+                    "The key is unrecoverable, so this CSR can no longer be completed — delete "
+                    "it and create a new one (then have the new CSR signed)."
+                ),
+            )
 
         effective_name = getattr(imp, 'name', None) or row['name']
 

backend/tests/test_csr_key_encryption.py

@@ -0,0 +1,129 @@
+"""Issue #53 (v1.10.1) — at-rest encryption for the pending CSR private key.
+
+Pure logic: no DB, no network. Covers the round-trip, the backward-compatible read of rows
+written before this release, the unrecoverable-key path after a key rotation, and a static
+assertion that the write path can no longer store a raw PEM.
+"""
+import os
+import re
+from pathlib import Path
+
+import pytest
+
+os.environ.setdefault("SECRET_KEY", "test-secret-key-for-csr-encryption-unit-tests")
+
+from cryptography.fernet import Fernet
+
+from utils.csr_key_crypto import (
+    decrypt_csr_private_key,
+    encrypt_csr_private_key,
+    is_encrypted,
+    reset_fernet_for_tests,
+)
+
+_SAMPLE_PEM = (
+    "-----BEGIN PRIVATE KEY-----\n"
+    "MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj\n"
+    "-----END PRIVATE KEY-----\n"
+)
+
+
+def test_roundtrip_and_ciphertext_does_not_contain_the_key():
+    reset_fernet_for_tests()
+    token = encrypt_csr_private_key(_SAMPLE_PEM)
+    # The stored form must not be the PEM, and must not leak any recognisable fragment of it.
+    assert token != _SAMPLE_PEM
+    assert "-----BEGIN" not in token
+    assert "MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj" not in token
+    assert decrypt_csr_private_key(token) == _SAMPLE_PEM
+
+
+def test_is_encrypted_discriminates_token_from_legacy_pem():
+    reset_fernet_for_tests()
+    assert is_encrypted(encrypt_csr_private_key(_SAMPLE_PEM)) is True
+    assert is_encrypted(_SAMPLE_PEM) is False
+    assert is_encrypted("") is False
+    assert is_encrypted(None) is False
+
+
+def test_legacy_plaintext_row_is_read_unchanged():
+    # Rows written before v1.10.1 hold a raw PEM. They must keep working with NO data migration,
+    # otherwise upgrading would strand every CSR that is out for signature.
+    reset_fernet_for_tests()
+    assert decrypt_csr_private_key(_SAMPLE_PEM) == _SAMPLE_PEM
+
+
+def test_empty_or_missing_value_returns_none():
+    reset_fernet_for_tests()
+    assert decrypt_csr_private_key(None) is None
+    assert decrypt_csr_private_key("") is None
+
+
+def test_key_rotation_makes_the_stored_key_unrecoverable_rather_than_wrong():
+    """After a rotation the caller must get None, never a silently wrong key."""
+    reset_fernet_for_tests()
+    token = encrypt_csr_private_key(_SAMPLE_PEM)
+
+    # Rotate: an explicit, different CSR_ENCRYPTION_KEY takes precedence over the derived one.
+    previous = os.environ.get("CSR_ENCRYPTION_KEY")
+    os.environ["CSR_ENCRYPTION_KEY"] = Fernet.generate_key().decode()
+    try:
+        reset_fernet_for_tests()
+        assert decrypt_csr_private_key(token) is None
+    finally:
+        if previous is None:
+            os.environ.pop("CSR_ENCRYPTION_KEY", None)
+        else:
+            os.environ["CSR_ENCRYPTION_KEY"] = previous
+        reset_fernet_for_tests()
+
+
+def test_explicit_env_key_is_used_and_survives_reset():
+    previous = os.environ.get("CSR_ENCRYPTION_KEY")
+    key = Fernet.generate_key().decode()
+    os.environ["CSR_ENCRYPTION_KEY"] = key
+    try:
+        reset_fernet_for_tests()
+        token = encrypt_csr_private_key(_SAMPLE_PEM)
+        # Decryptable with the same explicit key from a fresh instance...
+        reset_fernet_for_tests()
+        assert decrypt_csr_private_key(token) == _SAMPLE_PEM
+        # ...and independently verifiable with the raw Fernet key.
+        assert Fernet(key.encode()).decrypt(token.encode()).decode() == _SAMPLE_PEM
+    finally:
+        if previous is None:
+            os.environ.pop("CSR_ENCRYPTION_KEY", None)
+        else:
+            os.environ["CSR_ENCRYPTION_KEY"] = previous
+        reset_fernet_for_tests()
+
+
+def test_derivation_uses_its_own_hkdf_info_string():
+    """Each secret class derives an independent key, so rotating one never affects another."""
+    src = (Path(__file__).resolve().parent.parent / "utils" / "csr_key_crypto.py").read_text()
+    assert b"csr-private-key-v1".decode() in src
+    # Must NOT reuse another class's info string.
+    for foreign in ("dns-provider-creds-v1", "vip-vrrp-secret-v1", "mfa-totp-secret-v1"):
+        assert foreign not in src, f"CSR key derivation must not reuse the {foreign} info string"
+
+
+def test_write_path_stores_the_encrypted_form_not_the_pem():
+    """Static pin: insert_csr_row must encrypt before the INSERT.
+
+    A future refactor that passed bundle['private_key_pem'] straight through would silently
+    reintroduce plaintext storage, and no unit test with a mocked connection would notice.
+    """
+    src = (Path(__file__).resolve().parent.parent / "services" / "csr_service.py").read_text()
+    insert_fn = src[src.index("async def insert_csr_row("):]
+    insert_fn = insert_fn[: insert_fn.index("\nasync def ")]
+    assert "encrypt_csr_private_key(bundle['private_key_pem'])" in insert_fn
+    # The raw PEM must not be a bind parameter of the INSERT itself.
+    assert not re.search(r"^\s*bundle\['private_key_pem'\],\s*$", insert_fn, re.M)
+
+
+def test_import_path_decrypts_and_fails_closed_on_unrecoverable_key():
+    src = (Path(__file__).resolve().parent.parent / "services" / "csr_service.py").read_text()
+    fn = src[src.index("async def import_signed_certificate("):]
+    assert "decrypt_csr_private_key(row['private_key_pem'])" in fn
+    # A None decrypt must raise rather than fall through to the key-match comparison.
+    assert "cannot be decrypted" in fn

backend/utils/csr_key_crypto.py

@@ -0,0 +1,120 @@
+"""Issue #53 — at-rest encryption for the pending CSR private key (v1.10.1).
+
+Mirrors the established Fernet + HKDF(SECRET_KEY) pattern already used for the VRRP secret
+(services/keepalived_config.py), TOTP secrets (services/mfa_service.py) and DNS provider
+credentials (utils/dns_credentials.py): prefer an explicit CSR_ENCRYPTION_KEY env var (enables
+key rotation), else derive a stable key from SECRET_KEY via HKDF with its own versioned info
+string, so a rotation of one secret class never affects another.
+
+WHY this key and not every key in the system: the CSR private key is the one key that sits IDLE.
+It is generated at CSR creation, waits for an external CA to sign the request (days to weeks),
+and is destroyed the moment the signed certificate is imported — it is never transmitted to an
+agent and never leaves the server. `ssl_certificates.private_key_content` and the ACME order keys
+are different: agents must receive them in plaintext on every poll, so encrypting them at rest
+buys nothing without an end-to-end redesign.
+
+STORAGE: the Fernet token replaces the PEM in the SAME `ssl_csrs.private_key_pem` TEXT column.
+No new column, no new table, and deliberately NO `SCHEMA_VERSION` bump — a bump would re-run the
+migration sequence and re-seed the four built-in roles to their defaults (see UPGRADE_GUIDE.md),
+which is a needless side effect for a storage-format change.
+
+BACKWARD COMPATIBILITY: rows written before this release hold a raw PEM. `decrypt_csr_private_key`
+detects those by their `-----BEGIN` header and returns them unchanged. The discriminator is exact,
+not a heuristic: a Fernet token is base64url text and can never contain "-----". Legacy rows drain
+naturally, since a CSR's key copy is NULLed on import.
+
+KEY ROTATION: if SECRET_KEY rotates while CSR_ENCRYPTION_KEY is unset, previously stored keys
+become undecryptable and `decrypt_csr_private_key` returns None. Callers MUST surface a clear
+"delete this CSR and create a new one" error — the CSR is unusable at that point, because the
+signed certificate can no longer be paired with its key.
+"""
+from __future__ import annotations
+
+import base64
+import logging
+import os
+from typing import Optional
+
+from cryptography.fernet import Fernet, InvalidToken
+from cryptography.hazmat.primitives import hashes
+from cryptography.hazmat.primitives.kdf.hkdf import HKDF
+
+from config import SECRET_KEY
+
+logger = logging.getLogger(__name__)
+
+# A PEM private key always carries this header; a Fernet token is base64url and never can.
+_PEM_MARKER = "-----BEGIN"
+
+_fernet_instance: Optional[Fernet] = None
+
+
+def _resolve_fernet_key() -> bytes:
+    """Prefer an explicit CSR_ENCRYPTION_KEY; else derive from SECRET_KEY via HKDF with a
+    versioned info string (so stored keys survive restarts)."""
+    explicit = os.getenv("CSR_ENCRYPTION_KEY", "").strip()
+    if explicit:
+        try:
+            Fernet(explicit.encode())
+            return explicit.encode()
+        except Exception as exc:  # noqa: BLE001
+            logger.error("CSR_ENCRYPTION_KEY env var present but invalid: %s", exc)
+    logger.warning(
+        "CSR_ENCRYPTION_KEY not set; deriving the CSR private-key encryption key from SECRET_KEY. "
+        "Set CSR_ENCRYPTION_KEY to a Fernet key to enable key rotation."
+    )
+    hkdf = HKDF(algorithm=hashes.SHA256(), length=32, salt=None, info=b"csr-private-key-v1")
+    derived = hkdf.derive(SECRET_KEY.encode("utf-8"))
+    return base64.urlsafe_b64encode(derived)
+
+
+def _get_fernet() -> Fernet:
+    global _fernet_instance
+    if _fernet_instance is None:
+        _fernet_instance = Fernet(_resolve_fernet_key())
+    return _fernet_instance
+
+
+def reset_fernet_for_tests() -> None:
+    """Test-only hook to force re-resolution after env mutation."""
+    global _fernet_instance
+    _fernet_instance = None
+
+
+def is_encrypted(stored: Optional[str]) -> bool:
+    """True when the stored value is a Fernet token rather than a legacy raw PEM.
+
+    Single source of the format discriminator: `decrypt_csr_private_key` branches on this, so
+    the "what does a stored value look like" rule is stated exactly once.
+    """
+    return bool(stored) and _PEM_MARKER not in stored
+
+
+def encrypt_csr_private_key(pem: str) -> str:
+    """Fernet-encrypt a PEM private key to a storable token string."""
+    return _get_fernet().encrypt(pem.encode("utf-8")).decode("utf-8")
+
+
+def decrypt_csr_private_key(stored: Optional[str]) -> Optional[str]:
+    """Return the PEM private key for a stored value.
+
+    Accepts BOTH shapes so an upgrade needs no data migration:
+      - a raw PEM written before v1.10.1 -> returned unchanged
+      - a Fernet token -> decrypted
+
+    Returns None when the value is empty or cannot be decrypted (e.g. SECRET_KEY rotated without
+    CSR_ENCRYPTION_KEY). Callers MUST treat None as "this CSR's key is unrecoverable" and tell the
+    operator to delete it and create a new one; never fall through to a pairing attempt.
+    """
+    if not stored:
+        return None
+    if not is_encrypted(stored):
+        return stored  # legacy plaintext row, pre-v1.10.1
+    try:
+        return _get_fernet().decrypt(stored.encode("utf-8")).decode("utf-8")
+    except InvalidToken:
+        logger.warning("Failed to decrypt a stored CSR private key (invalid Fernet token)")
+        return None
+    except Exception as exc:  # noqa: BLE001
+        logger.error("Unexpected error decrypting a stored CSR private key: %s", exc)
+        return None

backend/version.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.10.0",
-  "releaseName": "GoDaddy DNS provider for ACME DNS-01",
-  "releaseDate": "2026-08-07"
+  "version": "1.10.1",
+  "releaseName": "CSR private key encrypted at rest",
+  "releaseDate": "2026-08-08"
 }

frontend/package.json

@@ -1,6 +1,6 @@
 {
   "name": "haproxy-openmanager-frontend",
-  "version": "1.10.0",
+  "version": "1.10.1",
   "description": "HAProxy Load Balancer Management UI",
   "license": "AGPL-3.0-or-later",
   "dependencies": {