fix: ACME certificate issuance improvements, null-safe hardening, guided setup UX, and order list enhancements (Issue #9)

- Fix critical cascading NULL status bug in ACME challenge flow that could cause 404s
- Add defense-in-depth NULL handling across all ACME service methods
- Add new GET /api/letsencrypt/prerequisites endpoint for configuration checks
- Add interactive ACME Setup Guide with step-by-step navigation links
- Add URL-based tab navigation in Settings and SSL Management pages
- Harden retry flow: return clear 409 errors for invalid/cancelled orders
- Allow cancellation of invalid orders (backend + frontend)
- Improve error message extraction with consistent getErrorMsg helper
- Add status filter tabs (Active/Completed/Failed/All) for order list
- Add visual dimming for cancelled/invalid orders
- Add enhanced pagination with size changer and total count
- Add comprehensive diagnostic logging with ACME: prefix
- Update README with ACME architecture docs, quick start guide, and troubleshooting

Resolves #9

Made-with: Cursor

README.md

@@ -865,6 +865,107 @@ When enabled, HAProxy configuration is automatically injected with:
 
 This injection only affects HTTP-mode frontends and is completely removed when ACME is disabled.
 
+#### ACME Distributed Architecture
+
+Understanding how HTTP-01 challenges work in a distributed HAProxy environment is critical for successful certificate issuance:
+
+```
+┌─────────────────┐    DNS A Record     ┌──────────────────┐
+│  Let's Encrypt   │ ──────────────────► │  HAProxy Node(s) │
+│  (CA Server)     │  HTTP GET :80       │  (VIP / Public)  │
+└─────────────────┘  /.well-known/      └────────┬─────────┘
+                      acme-challenge/             │
+                      {token}                     │ ACL match →
+                                                  │ use_backend
+                                                  │ _acme_challenge_backend
+                                                  ▼
+                                         ┌──────────────────┐
+                                         │  OpenManager      │
+                                         │  (Backend Server) │
+                                         │  Serves token     │
+                                         │  response from DB │
+                                         └──────────────────┘
+```
+
+**Key Points:**
+
+1. **Let's Encrypt connects to your HAProxy**, not directly to OpenManager
+2. **DNS must resolve** the requested domain to your HAProxy node's public IP (or VIP/NAT)
+3. **Port 80 must be open** from the internet to HAProxy for HTTP-01 validation
+4. **HAProxy routes** the `/.well-known/acme-challenge/` path to OpenManager via the injected backend
+5. **OpenManager serves** the challenge token response stored in the database
+6. The `MANAGEMENT_BASE_URL` or `acme.challenge_backend_url` setting tells HAProxy where to find OpenManager
+
+**Network Scenarios:**
+
+| Scenario | HAProxy | OpenManager | DNS Target |
+|----------|---------|-------------|------------|
+| Single server | localhost:80 | localhost:5000 | Server's public IP |
+| Separate servers | 10.0.0.10:80 | 10.0.0.20:5000 | HAProxy's public IP |
+| Behind NAT/VIP | VIP: 1.2.3.4:80 | Internal:5000 | VIP address |
+| Multi-cluster | Multiple HAProxy nodes | Central OpenManager | Each domain → respective HAProxy |
+
+#### ACME Quick Start Guide
+
+Follow these steps to obtain your first Let's Encrypt certificate:
+
+**Step 1: Configure ACME Settings**
+- Navigate to **Settings > ACME / SSL Automation** tab
+- Select your ACME provider (default: Let's Encrypt)
+- For testing, enable **Staging Mode** to avoid rate limits
+- Save changes
+
+**Step 2: Register an ACME Account**
+- On the **SSL Certificates > ACME Automation** page, find the ACME Account card
+- Click **Register Account** and provide a valid email address
+- Accept the Terms of Service
+
+**Step 3: Enable ACME on Clusters**
+- Go to **Cluster Management** > Edit your cluster
+- Enable **ACME Challenge Routing** toggle
+- Set the correct **ACME Backend URL** if OpenManager is on a different server
+- Save the cluster configuration
+
+**Step 4: Apply Configuration**
+- Go to **Apply Management** and apply the pending HAProxy configuration changes
+- This injects the ACME challenge routing rules into HAProxy
+
+**Step 5: Verify Challenge Routing**
+
+Test that the challenge path is reachable through HAProxy:
+```bash
+curl -v http://your-haproxy-ip/.well-known/acme-challenge/test
+```
+Expected: HTTP 404 from OpenManager (not HAProxy's default 503). This confirms routing works.
+
+**Step 6: Ensure DNS Resolution**
+- Your domain(s) must have DNS A/AAAA records pointing to your HAProxy node's public IP
+- Verify: `dig +short yourdomain.com` should return the HAProxy IP
+
+**Step 7: Request Certificate**
+- Go to **SSL Certificates > ACME Automation** tab
+- Click **New Certificate** and follow the wizard
+- The UI will guide you through prerequisite checks before submission
+
+#### Troubleshooting ACME
+
+If certificate issuance fails, check the following:
+
+| Issue | Diagnostic Step |
+|-------|----------------|
+| Challenge 404 | Check `ACME-CHALLENGE` log entries in OpenManager logs |
+| DNS mismatch | Verify `dig +short yourdomain.com` returns HAProxy IP |
+| Port 80 blocked | Test from external: `curl http://yourdomain.com/.well-known/acme-challenge/test` |
+| Config not applied | Check **Apply Management** for pending changes |
+| ACME not enabled | Verify cluster has **ACME Challenge Routing** enabled |
+| Account issues | Check ACME account status in **ACME Automation** page |
+
+**Diagnostic Logging:** OpenManager provides detailed ACME logging with prefixes:
+- `ACME:` — Order creation, challenge responses, finalization
+- `ACME-CHALLENGE:` — Incoming challenge token requests and responses
+
+Review application logs to trace the complete ACME flow when troubleshooting issues.
+
 ### 🛡️ **WAF Management** - *Web Application Firewall*
 - **Rate Limiting**: Request rate limiting by IP, URL, or custom patterns
 - **IP Filtering**: Whitelist/blacklist IP addresses and CIDR ranges
@@ -1451,7 +1552,7 @@ POST /api/letsencrypt/accounts
 DELETE /api/letsencrypt/accounts/{account_id}
 
 # Request certificate
-POST /api/letsencrypt/certificates/request
+POST /api/letsencrypt/certificates
 {
   "domains": ["example.com", "www.example.com"],
   "account_id": 1,
@@ -1473,18 +1574,18 @@ DELETE /api/letsencrypt/orders/{order_id}
 # Renew certificate (create new order)
 POST /api/letsencrypt/orders/{order_id}/renew
 
-# Complete certificate (download from CA)
-POST /api/letsencrypt/certificates/{order_id}/complete
-
 # Revoke certificate
 POST /api/letsencrypt/certificates/{cert_id}/revoke
 
 # Import CA chain certificates
-POST /api/letsencrypt/ca-chain/import
+POST /api/letsencrypt/import-ca-chain
 
 # Get renewal schedule
 GET /api/letsencrypt/renewal-schedule
 
+# Check ACME prerequisites (setup status)
+GET /api/letsencrypt/prerequisites
+
 # ACME Settings
 GET /api/settings/acme
 PUT /api/settings/acme

backend/main.py

@@ -653,20 +653,24 @@ async def get_version():
 @app.get("/.well-known/acme-challenge/{token}")
 async def serve_acme_challenge(token: str):
     """Serve ACME HTTP-01 challenge token. Public endpoint, no auth required."""
+    logger.info(f"ACME-CHALLENGE: Incoming request for token={token[:32]}...")
     conn = None
     try:
         conn = await get_database_connection()
         row = await conn.fetchrow(
-            "SELECT key_authorization FROM acme_challenges WHERE token = $1 AND status IN ('pending', 'processing') LIMIT 1",
+            "SELECT key_authorization FROM acme_challenges WHERE token = $1 AND (status IN ('pending', 'processing') OR status IS NULL) LIMIT 1",
             token
         )
         if row:
+            logger.info(f"ACME-CHALLENGE: Token found, serving key_authorization ({len(row['key_authorization'])} chars)")
             from fastapi.responses import PlainTextResponse
             return PlainTextResponse(row['key_authorization'])
+        logger.warning(f"ACME-CHALLENGE: Token NOT found in DB - no matching record with status pending/processing/null")
         raise HTTPException(status_code=404, detail="Challenge not found")
     except HTTPException:
         raise
-    except Exception:
+    except Exception as e:
+        logger.error(f"ACME-CHALLENGE: Error serving token {token[:32]}...: {e}")
         raise HTTPException(status_code=404, detail="Challenge not found")
     finally:
         if conn:

backend/routers/letsencrypt.py

@@ -156,6 +156,8 @@ async def request_certificate(body: CertificateRequest, authorization: str = Hea
     if not body.domains:
         raise HTTPException(status_code=400, detail="At least one domain is required")
 
+    logger.info(f"ACME: Certificate request initiated for domains={body.domains}, account_id={body.account_id}, cluster_ids={body.cluster_ids}")
+
     try:
         account_id = body.account_id
         if not account_id:
@@ -173,6 +175,26 @@ async def request_certificate(body: CertificateRequest, authorization: str = Hea
                 )
             account_id = account['id']
 
+        logger.info(f"ACME: Using account_id={account_id} for certificate request")
+
+        warnings = []
+        try:
+            conn_warn = await get_database_connection()
+            try:
+                acme_clusters = await conn_warn.fetchval(
+                    "SELECT COUNT(*) FROM haproxy_clusters WHERE acme_enabled = TRUE AND is_active = TRUE"
+                )
+                if acme_clusters == 0:
+                    logger.warning("ACME: No clusters with ACME Challenge Routing enabled - certificate validation will likely fail")
+                    warnings.append(
+                        "No clusters have ACME Challenge Routing enabled. "
+                        "Certificate validation will fail. Enable it in Cluster Management and Apply Changes first."
+                    )
+            finally:
+                await close_database_connection(conn_warn)
+        except Exception:
+            pass
+
         order = await acme_service.create_order(
             account_id=account_id,
             domains=body.domains,
@@ -181,12 +203,15 @@ async def request_certificate(body: CertificateRequest, authorization: str = Hea
 
         challenges = await acme_service.respond_to_challenges(order['order_id'])
 
+        logger.info(f"ACME: Order {order['order_id']} created, {len(challenges)} challenge(s) posted, status={order['status']}")
+
         return {
             "order_id": order['order_id'],
             "status": order['status'],
             "domains": body.domains,
             "challenges": challenges,
-            "message": "Order created. ACME challenges have been posted. Waiting for CA validation."
+            "message": "Order created. ACME challenges have been posted. Waiting for CA validation.",
+            "warnings": warnings,
         }
     except HTTPException:
         raise
@@ -265,15 +290,31 @@ async def retry_order(order_id: int, authorization: str = Header(None)):
         raise HTTPException(status_code=403, detail="Insufficient permissions: ssl.create required")
     try:
         status_info = await acme_service.check_order_status(order_id)
-        if status_info.get('status') == 'ready':
+        current_status = status_info.get('status')
+
+        if current_status == 'invalid':
+            raise HTTPException(
+                status_code=409,
+                detail="This order is invalid and cannot be retried. "
+                       "Please cancel it and create a new certificate request."
+            )
+        if current_status == 'cancelled':
+            raise HTTPException(
+                status_code=409,
+                detail="This order has been cancelled. Please create a new certificate request."
+            )
+
+        if current_status == 'ready':
             result = await acme_service.finalize_order(order_id)
             safe_result = {k: v for k, v in result.items() if k not in ('private_key_pem', 'cert_private_key')}
             return {"message": "Order finalized", **safe_result}
-        elif status_info.get('status') == 'valid' and status_info.get('certificate_url'):
+        elif current_status == 'valid' and status_info.get('certificate_url'):
             return await _complete_certificate(order_id)
         else:
             challenges = await acme_service.respond_to_challenges(order_id)
-            return {"message": "Challenges re-submitted", "status": status_info.get('status'), "challenges": challenges}
+            return {"message": "Challenges re-submitted", "status": current_status, "challenges": challenges}
+    except HTTPException:
+        raise
     except Exception as e:
         raise HTTPException(status_code=500, detail=str(e))
 
@@ -314,7 +355,7 @@ async def cancel_order(order_id: int, authorization: str = Header(None)):
     conn = await get_database_connection()
     try:
         result = await conn.execute(
-            "UPDATE letsencrypt_orders SET status = 'cancelled', updated_at = NOW() WHERE id = $1 AND status NOT IN ('valid', 'invalid', 'cancelled')",
+            "UPDATE letsencrypt_orders SET status = 'cancelled', updated_at = NOW() WHERE id = $1 AND status NOT IN ('valid', 'cancelled')",
             order_id
         )
         if result == "UPDATE 0":
@@ -670,3 +711,120 @@ async def _auto_apply_renewal(cert_id: int, cluster_ids: list):
             logger.info(f"ACME AUTO-APPLY: Notified {len(sync_results)} agents for cluster {cluster_id}")
         except Exception as notify_err:
             logger.error(f"ACME AUTO-APPLY: Agent notification failed for cluster {cluster_id}: {notify_err}")
+
+
+@router.get("/prerequisites")
+async def check_prerequisites(authorization: str = Header(None)):
+    """Check ACME prerequisites and return step-by-step setup status with navigation hints."""
+    from auth_middleware import get_current_user_from_token, check_user_permission
+    current_user = await get_current_user_from_token(authorization)
+    has_perm = await check_user_permission(current_user['id'], 'ssl', 'read')
+    if not has_perm:
+        raise HTTPException(status_code=403, detail="Insufficient permissions: ssl.read required")
+
+    conn = await get_database_connection()
+    try:
+        steps = []
+
+        # Step 1: ACME Settings configured
+        dir_url = await conn.fetchval(
+            "SELECT value FROM system_settings WHERE key = 'acme.directory_url'"
+        )
+        dir_url_str = ""
+        if dir_url:
+            try:
+                parsed = json.loads(dir_url) if isinstance(dir_url, str) else dir_url
+                dir_url_str = str(parsed) if not isinstance(parsed, str) else parsed
+            except (json.JSONDecodeError, TypeError, ValueError):
+                dir_url_str = str(dir_url)
+
+        provider_row = await conn.fetchval(
+            "SELECT value FROM system_settings WHERE key = 'acme.provider'"
+        )
+        provider = ""
+        if provider_row:
+            try:
+                parsed = json.loads(provider_row) if isinstance(provider_row, str) else provider_row
+                provider = str(parsed) if not isinstance(parsed, str) else parsed
+            except (json.JSONDecodeError, TypeError, ValueError):
+                provider = str(provider_row)
+
+        settings_ok = bool(dir_url_str and dir_url_str.startswith("http"))
+        settings_detail = f"Provider: {provider or 'Not set'}, Directory URL configured" if settings_ok else "ACME provider and directory URL not configured"
+        steps.append({
+            "key": "acme_settings",
+            "title": "Configure ACME Settings",
+            "ok": settings_ok,
+            "detail": settings_detail,
+            "navigate": "/settings?tab=acme",
+        })
+
+        # Step 2: ACME Account registered
+        account = await conn.fetchrow(
+            "SELECT id, email FROM letsencrypt_accounts WHERE status = 'valid' ORDER BY created_at DESC LIMIT 1"
+        )
+        account_ok = account is not None
+        account_detail = f"Active account: {account['email']}" if account else "No active ACME account registered"
+        steps.append({
+            "key": "acme_account",
+            "title": "Register ACME Account",
+            "ok": account_ok,
+            "detail": account_detail,
+            "action": "register_account",
+        })
+
+        # Step 3: Cluster ACME enabled
+        acme_cluster_count = await conn.fetchval(
+            "SELECT COUNT(*) FROM haproxy_clusters WHERE acme_enabled = TRUE AND is_active = TRUE"
+        )
+        acme_cluster_names = await conn.fetch(
+            "SELECT name FROM haproxy_clusters WHERE acme_enabled = TRUE AND is_active = TRUE"
+        )
+        cluster_ok = acme_cluster_count > 0
+        cluster_detail = (
+            f"Enabled on: {', '.join(r['name'] for r in acme_cluster_names)}"
+            if cluster_ok
+            else "No clusters have ACME Challenge Routing enabled"
+        )
+        steps.append({
+            "key": "cluster_acme_enabled",
+            "title": "Enable ACME on Cluster",
+            "ok": cluster_ok,
+            "detail": cluster_detail,
+            "navigate": "/clusters",
+        })
+
+        # Step 4: Configuration applied (depends on step 3)
+        if cluster_ok:
+            pending_count = await conn.fetchval("""
+                SELECT COUNT(*) FROM config_versions
+                WHERE status = 'PENDING'
+                AND cluster_id IN (SELECT id FROM haproxy_clusters WHERE acme_enabled = TRUE AND is_active = TRUE)
+            """)
+            config_ok = pending_count == 0
+            config_detail = "All ACME cluster configurations are applied" if config_ok else f"{pending_count} pending configuration change(s) need to be applied"
+        else:
+            config_ok = None
+            config_detail = "Enable ACME on a cluster first, then apply changes"
+        steps.append({
+            "key": "config_applied",
+            "title": "Apply Configuration Changes",
+            "ok": config_ok,
+            "detail": config_detail,
+            "navigate": "/apply-management",
+        })
+
+        # Step 5: DNS & Network (informational only)
+        steps.append({
+            "key": "network_dns",
+            "title": "Verify DNS and Network",
+            "ok": None,
+            "detail": "Domain DNS must point to HAProxy IP, Port 80 must be open from internet",
+            "navigate": None,
+        })
+
+        ready = all(step["ok"] is True for step in steps if step["ok"] is not None)
+
+        return {"ready": ready, "steps": steps}
+    finally:
+        await close_database_connection(conn)

backend/services/acme_service.py

@@ -256,7 +256,7 @@ class ACMEService:
                     account_url = $3, jwk_private_key = $4, status = $5, tos_agreed = $6, updated_at = NOW()
                 RETURNING id, email, directory_url, account_url, status, tos_agreed, created_at
             """, email, directory_url, account_url, pem,
-                data.get('status', 'valid'), tos_agreed, eab_kid)
+                data.get('status') or 'valid', tos_agreed, eab_kid)
             return dict(row)
         finally:
             await close_database_connection(conn)
@@ -287,7 +287,7 @@ class ACMEService:
 
             if status not in (200, 201):
                 logger.warning(f"ACME account deactivation returned {status}: {data}")
-                raise Exception(f"CA rejected deactivation (HTTP {status}): {data.get('detail', 'Unknown error')}")
+                raise Exception(f"CA rejected deactivation (HTTP {status}): {data.get('detail') or 'Unknown error'}")
 
             await conn.execute(
                 "UPDATE letsencrypt_accounts SET status = 'deactivated', updated_at = NOW() WHERE id = $1",
@@ -303,6 +303,7 @@ class ACMEService:
         domains: List[str],
         cluster_ids: Optional[List[int]] = None,
     ) -> dict:
+        logger.info(f"ACME: Creating order for domains={domains}, account_id={account_id}")
         conn = await get_database_connection()
         try:
             account = await conn.fetchrow(
@@ -326,9 +327,11 @@ class ACMEService:
             )
 
             if status not in (200, 201):
+                logger.error(f"ACME: Order creation failed: HTTP {status}, response={data}")
                 raise Exception(f"Order creation failed: {data}")
 
             order_url = headers.get('Location', '')
+            logger.info(f"ACME: Order created, order_url={order_url}, status={data.get('status')}, authorizations={len(data.get('authorizations', []))}")
             expires_at = None
             if data.get('expires'):
                 try:
@@ -341,8 +344,8 @@ class ACMEService:
                     (account_id, order_url, status, domains, finalize_url, expires_at, cluster_ids)
                 VALUES ($1, $2, $3, $4, $5, $6, $7)
                 RETURNING id
-            """, account_id, order_url, data.get('status', 'pending'),
-                json.dumps(domains), data.get('finalize', ''), expires_at,
+            """, account_id, order_url, data.get('status') or 'pending',
+                json.dumps(domains), data.get('finalize') or '', expires_at,
                 json.dumps(cluster_ids or []))
 
             order_id = order_row['id']
@@ -356,8 +359,8 @@ class ACMEService:
                     logger.warning(f"Failed to fetch authorization {auth_url}: HTTP {auth_status}")
                     continue
 
-                domain = auth_data.get('identifier', {}).get('value', '')
-                for challenge in auth_data.get('challenges', []):
+                domain = (auth_data.get('identifier') or {}).get('value', '')
+                for challenge in (auth_data.get('challenges') or []):
                     if challenge.get('type') == 'http-01':
                         token = challenge['token']
                         jwk = self._get_jwk(private_key)
@@ -368,15 +371,16 @@ class ACMEService:
                             INSERT INTO acme_challenges (order_id, domain, token, key_authorization, challenge_url, status)
                             VALUES ($1, $2, $3, $4, $5, $6)
                         """, order_id, domain, token, key_auth,
-                            challenge.get('url', ''), challenge.get('status', 'pending'))
+                            challenge.get('url') or '', challenge.get('status') or 'pending')
+                        logger.info(f"ACME: Challenge stored for domain={domain}, token={token[:20]}..., challenge_url={(challenge.get('url') or '')[:60]}")
 
             return {
                 "order_id": order_id,
                 "order_url": order_url,
-                "status": data.get('status', 'pending'),
+                "status": data.get('status') or 'pending',
                 "domains": domains,
-                "authorizations": data.get('authorizations', []),
-                "finalize": data.get('finalize', ''),
+                "authorizations": data.get('authorizations') or [],
+                "finalize": data.get('finalize') or '',
             }
         finally:
             await close_database_connection(conn)
@@ -385,7 +389,7 @@ class ACMEService:
         conn = await get_database_connection()
         try:
             challenges = await conn.fetch(
-                "SELECT * FROM acme_challenges WHERE order_id = $1 AND status = 'pending'",
+                "SELECT * FROM acme_challenges WHERE order_id = $1 AND (status = 'pending' OR status IS NULL)",
                 order_id
             )
             order = await conn.fetchrow(
@@ -397,9 +401,11 @@ class ACMEService:
 
             private_key = self._load_private_key(order['jwk_private_key'])
             results = []
+            logger.info(f"ACME: Responding to {len(challenges)} challenge(s) for order_id={order_id}")
 
             for ch in challenges:
                 if not ch['challenge_url']:
+                    logger.warning(f"ACME: Skipping challenge id={ch['id']} domain={ch['domain']} - no challenge_url")
                     continue
                 status, data, _ = await self._signed_request(
                     ch['challenge_url'],
@@ -408,7 +414,8 @@ class ACMEService:
                     {},
                     account_url=order['account_url'],
                 )
-                new_status = data.get('status', 'processing') if status == 200 else 'failed'
+                new_status = (data.get('status') or 'processing') if status == 200 else 'failed'
+                logger.info(f"ACME: Challenge response for domain={ch['domain']}, token={ch['token'][:20]}..., CA_HTTP={status}, CA_status_raw={data.get('status')!r}, stored_status={new_status}")
                 await conn.execute(
                     "UPDATE acme_challenges SET status = $1 WHERE id = $2",
                     new_status, ch['id']
@@ -432,6 +439,7 @@ class ACMEService:
                 raise Exception(f"Order {order_id} not found")
 
             domains = json.loads(order['domains']) if isinstance(order['domains'], str) else order['domains']
+            logger.info(f"ACME: Finalizing order_id={order_id}, domains={domains}, finalize_url={(order['finalize_url'] or 'N/A')[:60]}")
             private_key = self._load_private_key(order['jwk_private_key'])
 
             cert_key = rsa.generate_private_key(
@@ -464,15 +472,17 @@ class ACMEService:
             )
 
             if status not in (200, 201):
-                error_msg = data.get('detail', str(data))
+                error_msg = data.get('detail') or str(data)
+                logger.error(f"ACME: Finalize failed for order_id={order_id}: HTTP {status}, error={error_msg}")
                 await conn.execute(
                     "UPDATE letsencrypt_orders SET status = 'invalid', error_detail = $1, updated_at = NOW() WHERE id = $2",
                     error_msg, order_id
                 )
                 raise Exception(f"Finalize failed: {error_msg}")
 
-            order_status = data.get('status', 'processing')
-            certificate_url = data.get('certificate', '')
+            order_status = data.get('status') or 'processing'
+            certificate_url = data.get('certificate') or ''
+            logger.info(f"ACME: Finalize success for order_id={order_id}, status={order_status}, certificate_url={certificate_url[:60] if certificate_url else 'N/A'}")
 
             await conn.execute(
                 "UPDATE letsencrypt_orders SET status = $1, certificate_url = $2, cert_private_key = $3, updated_at = NOW() WHERE id = $4",
@@ -558,8 +568,8 @@ class ACMEService:
             )
 
             if status == 200:
-                new_status = data.get('status', order['status'])
-                certificate_url = data.get('certificate', order['certificate_url'])
+                new_status = data.get('status') or order['status']
+                certificate_url = data.get('certificate') or order['certificate_url']
                 await conn.execute(
                     "UPDATE letsencrypt_orders SET status = $1, certificate_url = $2, updated_at = NOW() WHERE id = $3",
                     new_status, certificate_url, order_id

frontend/src/components/ACMEAutomation.js

@@ -1,23 +1,30 @@
 import React, { useState, useEffect, useCallback } from 'react';
 import {
   Card, Table, Button, Tag, Space, Modal, Form, Input, Select, Steps,
-  message, Row, Col, Statistic, Alert, Tooltip, Switch, theme
+  message, Row, Col, Statistic, Alert, Tooltip, Switch, theme, Segmented
 } from 'antd';
 import {
   SafetyCertificateOutlined, PlusOutlined, ReloadOutlined,
   CheckCircleOutlined, ClockCircleOutlined, ExclamationCircleOutlined,
   SyncOutlined, CloseCircleOutlined,
   DeleteOutlined, EyeOutlined,
-  CloudDownloadOutlined, UserOutlined, InfoCircleOutlined
+  CloudDownloadOutlined, UserOutlined, InfoCircleOutlined,
+  RocketOutlined
 } from '@ant-design/icons';
+import { useNavigate } from 'react-router-dom';
 import axios from 'axios';
 
 const { Option } = Select;
 
+const getErrorMsg = (err, fallback) =>
+  err?.response?.data?.error?.message || err?.response?.data?.detail || fallback;
+
 const ACMEAutomation = () => {
+  const navigate = useNavigate();
   const [orders, setOrders] = useState([]);
   const [accounts, setAccounts] = useState([]);
   const [renewalSchedule, setRenewalSchedule] = useState([]);
+  const [prerequisites, setPrerequisites] = useState(null);
   const [loading, setLoading] = useState(false);
   const [wizardVisible, setWizardVisible] = useState(false);
   const [wizardStep, setWizardStep] = useState(0);
@@ -30,19 +37,22 @@ const ACMEAutomation = () => {
   const [registerForm] = Form.useForm();
   const [registering, setRegistering] = useState(false);
   const [accountDetailVisible, setAccountDetailVisible] = useState(false);
+  const [orderFilter, setOrderFilter] = useState('active');
   const { token } = theme.useToken();
 
   const fetchData = useCallback(async () => {
     setLoading(true);
     try {
-      const [ordersRes, accountsRes, renewalRes, clustersRes] = await Promise.allSettled([
+      const [ordersRes, accountsRes, renewalRes, clustersRes, prereqRes] = await Promise.allSettled([
         axios.get('/api/letsencrypt/orders'),
         axios.get('/api/letsencrypt/accounts'),
         axios.get('/api/letsencrypt/renewal-schedule'),
         axios.get('/api/clusters'),
+        axios.get('/api/letsencrypt/prerequisites'),
       ]);
       if (ordersRes.status === 'fulfilled') setOrders(ordersRes.value.data || []);
       if (accountsRes.status === 'fulfilled') setAccounts(accountsRes.value.data || []);
+      if (prereqRes.status === 'fulfilled') setPrerequisites(prereqRes.value.data);
       if (renewalRes.status === 'fulfilled') setRenewalSchedule(renewalRes.value.data || []);
       if (clustersRes.status === 'fulfilled') setClusters(clustersRes.value.data?.clusters || []);
     } catch (err) {
@@ -64,6 +74,14 @@ const ACMEAutomation = () => {
   const pendingOrders = orders.filter(o => o.status === 'pending' || o.status === 'processing');
   const activeAccount = accounts.find(a => a.status === 'valid') || null;
   const acmeAccount = activeAccount || (accounts.length > 0 ? accounts[accounts.length - 1] : null);
+  const acmeEnabledClusters = clusters.filter(c => c.acme_enabled && c.is_active);
+
+  const filteredOrders = orders.filter(o => {
+    if (orderFilter === 'active') return !['cancelled', 'invalid', 'valid'].includes(o.status);
+    if (orderFilter === 'completed') return o.status === 'valid';
+    if (orderFilter === 'failed') return o.status === 'cancelled' || o.status === 'invalid';
+    return true;
+  });
 
   const handleRequestCert = async () => {
     try {
@@ -80,12 +98,15 @@ const ACMEAutomation = () => {
         account_id: values.account_id || null,
       });
       message.success(res.data?.message || 'Certificate request submitted');
+      if (res.data?.warnings?.length > 0) {
+        res.data.warnings.forEach(w => message.warning(w, 8));
+      }
       setWizardVisible(false);
       setWizardStep(0);
       wizardForm.resetFields();
       fetchData();
     } catch (err) {
-      message.error(err?.response?.data?.detail || 'Failed to request certificate');
+      message.error(getErrorMsg(err, 'Failed to request certificate'));
     } finally {
       setSubmitting(false);
     }
@@ -97,7 +118,7 @@ const ACMEAutomation = () => {
       message.success(res.data?.message || 'Retry submitted');
       fetchData();
     } catch (err) {
-      message.error(err?.response?.data?.detail || 'Retry failed');
+      message.error(getErrorMsg(err, 'Retry failed'));
     }
   };
 
@@ -113,7 +134,7 @@ const ACMEAutomation = () => {
           message.success('Order cancelled');
           fetchData();
         } catch (err) {
-          message.error(err?.response?.data?.detail || 'Cancel failed');
+          message.error(getErrorMsg(err, 'Cancel failed'));
         }
       },
     });
@@ -138,7 +159,7 @@ const ACMEAutomation = () => {
           const res = await axios.post('/api/letsencrypt/import-ca-chain');
           message.success(res.data?.message || 'CA chain imported');
         } catch (err) {
-          message.error(err?.response?.data?.detail || 'Import failed');
+          message.error(getErrorMsg(err, 'Import failed'));
         }
       },
     });
@@ -157,7 +178,7 @@ const ACMEAutomation = () => {
       registerForm.resetFields();
       fetchData();
     } catch (err) {
-      message.error(err?.response?.data?.detail || 'Account registration failed');
+      message.error(getErrorMsg(err, 'Account registration failed'));
     } finally {
       setRegistering(false);
     }
@@ -183,7 +204,7 @@ const ACMEAutomation = () => {
           message.success('Account deactivated successfully');
           fetchData();
         } catch (err) {
-          message.error(err?.response?.data?.detail || 'Failed to deactivate account');
+          message.error(getErrorMsg(err, 'Failed to deactivate account'));
         }
       },
     });
@@ -209,7 +230,7 @@ const ACMEAutomation = () => {
           message.success('Account permanently removed');
           fetchData();
         } catch (err) {
-          message.error(err?.response?.data?.detail || 'Failed to remove account');
+          message.error(getErrorMsg(err, 'Failed to remove account'));
         }
       },
     });
@@ -225,7 +246,7 @@ const ACMEAutomation = () => {
       cancelled: { color: 'default', icon: <CloseCircleOutlined /> },
     };
     const cfg = map[status] || { color: 'default', icon: null };
-    return <Tag color={cfg.color} icon={cfg.icon}>{status?.toUpperCase()}</Tag>;
+    return <Tag color={cfg.color} icon={cfg.icon}>{(status || 'unknown').toUpperCase()}</Tag>;
   };
 
   const orderColumns = [
@@ -255,7 +276,7 @@ const ACMEAutomation = () => {
           <Tooltip title="View Details">
             <Button icon={<EyeOutlined />} size="small" onClick={() => handleViewOrder(record.id)} />
           </Tooltip>
-          {(record.status === 'pending' || record.status === 'ready') && (
+          {(record.status === 'pending' || record.status === 'processing' || record.status === 'ready') && (
             <Tooltip title="Retry / Finalize">
               <Button icon={<ReloadOutlined />} size="small" type="primary" ghost onClick={() => handleRetry(record.id)} />
             </Tooltip>
@@ -292,8 +313,6 @@ const ACMEAutomation = () => {
     },
   ];
 
-  const acmeEnabledClusters = clusters.filter(c => c.acme_enabled);
-
   const wizardSteps = [
     {
       title: 'Domains',
@@ -355,7 +374,13 @@ const ACMEAutomation = () => {
               type="error"
               showIcon
               message="No Active ACME Account"
-              description="Please register an active ACME account in Settings > ACME or from the ACME Account card."
+              description={
+                <span>
+                  Please register an active ACME account.{' '}
+                  <Button type="link" size="small" style={{ padding: 0 }} onClick={() => navigate('/settings?tab=acme')}>Go to Settings &gt; ACME</Button>
+                  {' '}or use the ACME Account card below.
+                </span>
+              }
               style={{ marginBottom: 16 }}
             />
           )}
@@ -364,7 +389,28 @@ const ACMEAutomation = () => {
               type="warning"
               showIcon
               message="No Clusters with ACME Enabled"
-              description="Enable ACME challenge routing on at least one cluster in Cluster Management."
+              description={
+                <span>
+                  Enable ACME challenge routing on at least one cluster.{' '}
+                  <Button type="link" size="small" style={{ padding: 0 }} onClick={() => navigate('/clusters')}>Go to Cluster Management</Button>
+                  {' '}then{' '}
+                  <Button type="link" size="small" style={{ padding: 0 }} onClick={() => navigate('/apply-management')}>Apply Changes</Button>.
+                </span>
+              }
+              style={{ marginBottom: 16 }}
+            />
+          )}
+          {prerequisites?.steps?.find(s => s.key === 'config_applied' && s.ok === false) && (
+            <Alert
+              type="warning"
+              showIcon
+              message="Configuration Not Applied"
+              description={
+                <span>
+                  ACME routing rules have not been pushed to HAProxy yet. Certificate validation will fail without this.{' '}
+                  <Button type="link" size="small" style={{ padding: 0 }} onClick={() => navigate('/apply-management')}>Go to Apply Management</Button>
+                </span>
+              }
               style={{ marginBottom: 16 }}
             />
           )}
@@ -387,6 +433,38 @@ const ACMEAutomation = () => {
 
   return (
     <div>
+      {prerequisites && !prerequisites.ready && (
+        <Card
+          size="small"
+          title={<span><RocketOutlined /> ACME Setup Guide</span>}
+          style={{ marginBottom: 16, borderColor: '#faad14' }}
+        >
+          <Steps
+            size="small"
+            direction="vertical"
+            items={(prerequisites.steps || []).map((step) => ({
+              title: (
+                <span>
+                  {step.title}
+                  {step.navigate && (
+                    <Button type="link" size="small" onClick={() => navigate(step.navigate)} style={{ marginLeft: 8, padding: 0 }}>
+                      Configure
+                    </Button>
+                  )}
+                  {step.action === 'register_account' && !step.ok && (
+                    <Button type="link" size="small" onClick={() => setRegisterVisible(true)} style={{ marginLeft: 8, padding: 0 }}>
+                      Register Account
+                    </Button>
+                  )}
+                </span>
+              ),
+              description: step.detail,
+              status: step.ok === true ? 'finish' : step.ok === false ? 'error' : 'wait',
+            }))}
+          />
+        </Card>
+      )}
+
       {pendingOrders.length > 0 && (
         <Alert
           type="warning"
@@ -470,13 +548,29 @@ const ACMEAutomation = () => {
         }
         style={{ marginBottom: 24 }}
       >
+        <div style={{ marginBottom: 12, display: 'flex', alignItems: 'center', gap: 12 }}>
+          <Segmented
+            value={orderFilter}
+            onChange={setOrderFilter}
+            options={[
+              { label: `Active (${orders.filter(o => !['cancelled','invalid','valid'].includes(o.status)).length})`, value: 'active' },
+              { label: `Completed (${orders.filter(o => o.status === 'valid').length})`, value: 'completed' },
+              { label: `Failed / Cancelled (${orders.filter(o => o.status === 'cancelled' || o.status === 'invalid').length})`, value: 'failed' },
+              { label: `All (${orders.length})`, value: 'all' },
+            ]}
+            size="small"
+          />
+        </div>
         <Table
           columns={orderColumns}
-          dataSource={orders}
+          dataSource={filteredOrders}
           rowKey="id"
           loading={loading}
-          pagination={{ pageSize: 10 }}
+          pagination={{ pageSize: 10, showSizeChanger: true, showTotal: (total) => `${total} orders` }}
           size="small"
+          rowClassName={(record) =>
+            record.status === 'cancelled' || record.status === 'invalid' ? 'acme-order-terminal' : ''
+          }
         />
       </Card>
 
@@ -521,7 +615,7 @@ const ACMEAutomation = () => {
             </Button>
           )}
           {wizardStep === wizardSteps.length - 1 && (
-            <Button type="primary" onClick={handleRequestCert} loading={submitting} disabled={!activeAccount}>
+            <Button type="primary" onClick={handleRequestCert} loading={submitting} disabled={!activeAccount || acmeEnabledClusters.length === 0}>
               Submit Request
             </Button>
           )}

frontend/src/components/SSLManagement.js

@@ -16,6 +16,7 @@ import {
   ThunderboltOutlined
 } from '@ant-design/icons';
 import axios from 'axios';
+import { useSearchParams } from 'react-router-dom';
 import { useCluster } from '../contexts/ClusterContext';
 import { useProgress } from '../contexts/ProgressContext';
 import { formatEntityForSync } from '../utils/agentSync';
@@ -26,6 +27,8 @@ const { TextArea } = Input;
 const { TabPane } = Tabs;
 
 const SSLManagement = () => {
+  const [searchParams] = useSearchParams();
+  const defaultTab = searchParams.get('tab') || 'certificates';
   const { token } = theme.useToken();
   const { selectedCluster, clusters } = useCluster();
   const [certificates, setCertificates] = useState([]);
@@ -893,6 +896,7 @@ const SSLManagement = () => {
         )}
       </Title>
       <Tabs
+        defaultActiveKey={defaultTab}
         items={[
           {
             key: 'certificates',

frontend/src/components/Settings.js

@@ -1,6 +1,7 @@
 import React, { useEffect, useState } from 'react';
 import { Card, Form, Switch, Button, InputNumber, message, Tabs, Input, Select, Collapse, Space, Alert, Tag, Spin, Tooltip } from 'antd';
 import { SafetyCertificateOutlined, ApiOutlined, CheckCircleOutlined, CloseCircleOutlined, InfoCircleOutlined } from '@ant-design/icons';
+import { useSearchParams } from 'react-router-dom';
 import axios from 'axios';
 
 const { Option } = Select;
@@ -33,6 +34,8 @@ const ACME_PROVIDERS = {
 };
 
 const Settings = () => {
+  const [searchParams] = useSearchParams();
+  const defaultTab = searchParams.get('tab') || 'general';
   const [form] = Form.useForm();
   const [acmeForm] = Form.useForm();
   const [acmeLoading, setAcmeLoading] = useState(false);
@@ -332,7 +335,7 @@ const Settings = () => {
 
   return (
     <div>
-      <Tabs items={tabItems} />
+      <Tabs items={tabItems} defaultActiveKey={defaultTab} />
     </div>
   );
 };

frontend/src/index.css

@@ -147,4 +147,8 @@ code {
 [data-theme='dark'] .search-input .ant-input-affix-wrapper:hover,
 [data-theme='dark'] .search-input.ant-input-affix-wrapper:hover {
   border-color: #177ddc !important;
+}
+
+.acme-order-terminal td {
+  opacity: 0.55;
 }