Until now the only record of what happened was `user_activity_logs`, which
stores non-GET 2xx operations with no bodies. When something failed you could
see that a counter went up, never what was sent or what came back.
This adds one queryable timeline covering both directions:
- inbound: every API call, including GETs and including 4xx/5xx, with the
user, client IP, status, duration and — redacted, size-capped — the request
and response bodies.
- outbound: every HTTP call the backend makes, tagged with who it went to
(ACME/Let's Encrypt, Cloudflare, GoDaddy, HAProxy stats, agents, the ACME
diagnostics probe).
Outbound rows inherit the inbound request's id, so one operator action and the
CA/DNS calls it triggered read as a single trace: opening a failed "Request
Certificate" shows the exact POST /acme/new-order and the CA's 429 underneath.
Implementation notes:
- Capture is a pure-ASGI middleware that TEES the request and response streams
rather than draining them. `await request.body()` inside a BaseHTTPMiddleware
would consume the receive channel and break the raw-body agent heartbeat
handler. Registered last so it is outermost: it then sees the final
client-visible response and seeds correlation_id_context before the error
handler reads it.
- Rows are written by a batching background writer with a bounded queue, so the
request path never awaits the database and a saturated logger drops rows
visibly (surfaced on the page) instead of blocking. Redaction runs on the
writer, off the request coroutine.
- Secrets never land: headers are an allowlist with Authorization/Cookie kept
only as a presence marker; body keys and value shapes are redacted
(passwords, tokens, api_token, API keys, private-key PEMs, JWTs); the ACME
JWS request body is never stored, because a stored protected+signature pair
is a replayable credential — a summary is logged instead; DNS-provider errors
record only the exception type; the ACME HTTP-01 challenge endpoint is
excluded so key_authorization is never captured.
- Retention is operator-configurable in Settings -> Request Log: separate day
counts for successful and failed rows (7 / 30) plus a hard row cap (500k),
whichever is reached first. Pruned in batches under a Postgres advisory lock,
with the day counts bound as parameters, never interpolated.
- New permissions requestlog.read / requestlog.manage. super_admin and
security_admin get both, operator gets read, viewer gets neither.
Schema: one new table (request_logs) plus its settings seed, SCHEMA_VERSION
10 -> 11, auto-migrated. No existing table altered, no agent or rendered-config
change. Kill switches: REQUEST_LOG_ENABLED=false (middleware never registered)
or the `enabled` toggle in Settings.
Tests: 245 new (7 backend files + 1 frontend), full suite 1655 backend +
17 frontend passing.
Follow-up to the RCE/missing-auth/SSRF remediation, from a thorough multi-lens
review (3 agents + a black-box audit of all 201 routes). Backend-only; no
agent-script changes.
Regression fix (introduced by the previous commit):
- GET /api/agents was made JWT-only, but deployed agents call it WITH X-API-Key
(not a JWT) to read their applied_config_version and avoid re-applying config on
restart. It now accepts EITHER a valid operator JWT OR a valid agent X-API-Key,
so agents no longer get 401 (which caused a spurious HAProxy reload every restart).
Completeness (GHSA-3p5c siblings the first pass missed — same data class, now JWT):
- dashboard.py: GET /api/haproxy-cluster-pools/{id}/agents (full agent inventory —
a direct anonymous bypass of the GET /api/agents lockdown), /api/pools,
/api/haproxy-cluster-pools, /api/dashboard/stats, /api/dashboard/overview
(auth was optional -> leaked stats/names/health/alerts anonymously),
/api/haproxy/stats.
- waf.py: GET /api/waf/rules. health.py: GET /api/health/errors.
- agent.py: GET /api/agents/generate-uninstall-script/{platform} (agent-management
endpoint; was anonymous) now requires JWT or agent key, like generate-install-script.
- config.py: POST /api/config/{validate,optimize,templates/{id}/generate} were
optional-auth (logging only) and run a HAProxy validator on caller input; now
require a JWT. (bulk-create, parse-bulk, diff and configuration/request were
already mandatory-auth — verified.)
All newly-gated endpoints are frontend-only (axios sends the JWT) or unused;
agents never call them.
SSRF (GHSA-3vh4) gap:
- acme_service._get_nonce fetched directory['newNonce'] (from the attacker-
influenceable directory JSON) with a bare session, http allowed, dual-stack, and
BEFORE the guarded _signed_request POST. Now guarded (assert_public_url +
safe_connector + no redirects + timeout), matching the other ACME sinks.
Correctness:
- Three agent webhooks (config-applied, config-validation-failed, config-sync)
swallowed their auth 401 into a 200 error body via a bare `except Exception`.
Added `except HTTPException: raise` so the 401/403 propagates.
Audit result (live black-box, all 201 routes probed unauthenticated): no data
leak and no unauthenticated mutation anywhere; every sensitive route returns
401/403 (a pre-existing group of read handlers wraps the 401 into a 500 via a
broad except — no data is exposed; left as-is, documented as cosmetic).
Verified: full pytest tests/ (1145 passed, 0 failed; +16 regression tests) + live
localtest stack smoke — agent-key GET /api/agents=200, anonymous=401, all newly
gated endpoints reject anonymous and admit JWT, the 3 webhooks return 401.
Addresses three reported advisories, all verified against the code. Fixes are
entirely server-side — deployed agents already send a valid X-API-Key on every
call, so enforcing it does not require any agent-script change or upgrade.
GHSA-7rhv-c5pc-69r8 (CRITICAL RCE — agent script-template poisoning):
- POST/GET /api/agents/script-templates/{platform} now require the agents.version
permission (was authentication-only), matching POST /versions. Blocks a viewer
JWT from overwriting the root install/upgrade script.
GHSA-3p5c-m5m4-mjpx (missing authentication):
- Agent data-plane endpoints now REQUIRE a valid X-API-Key (was optional/skipped
when the header was absent), checked before any DB access: config,
ssl-certificates (private keys!), upgrade-status, heartbeat (by-name and the
previously auth-less by-id), configuration pending-requests. Removes keyless
heartbeat spoofing and keyless rogue-agent auto-registration.
- Operator/UI endpoints now require a JWT: GET /api/agents, the entire
/api/dashboard-stats router, /api/health/{deep,agents,clusters}, and
/api/ssl/certificates/{id}/config-versions. The simple /api/health liveness
probe stays public. Adds shared auth_middleware.require_authenticated_user.
GHSA-3vh4-gvxx-wm2p (SSRF via ACME directory_url):
- New utils/ssrf_guard.py (https-only + public-IP-only, IPv4-pinned, no redirects),
applied to settings test-connection, acme_service.get_directory and
_signed_request, and validated at Let's Encrypt account creation. The
test-connection response no longer reflects arbitrary upstream JSON keys
(information-disclosure oracle) — only fixed ACME field names.
Verified: full pytest tests/ (1128 passed, 0 failed) + live localtest stack smoke
(valid JWT/key paths return 200/404 as expected; anonymous requests 401; SSRF to
metadata/private/loopback refused). No changes to backend/utils/agent_scripts/*.
ZeroSSL/Google account registration failed with
`malformed: The Replay Nonce could not be base64url-decoded`: the ACME client
(a process-wide singleton) kept a single anti-replay nonce shared across
certificate authorities, so a nonce issued by one CA could be sent to another,
and the auto-retry only covered `badNonce`.
- Scope the nonce per CA (self._nonce_by_dir keyed by directory_url): a nonce
from one CA is never sent to another; account registration always uses a fresh
nonce from the target CA.
- Broaden the 400 retry to also recover from the nonce-malformed rejection.
- Fix _b64url_decode padding (used for the EAB HMAC key).
Backend-only; HTTP-01 and Let's Encrypt are unaffected.
Addresses #35.
Add ACME DNS-01 (TXT-record) validation alongside the existing HTTP-01,
for internal/isolated clusters with no public port 80 and for wildcard
certificates. Opt-in via a global kill-switch (default off); HTTP-01 is
byte-for-byte unchanged, with zero agent or rendered-config changes.
- Pluggable DNS provider interface (Manual + Cloudflare). Per-account
credentials are Fernet-encrypted at rest, verified on save, and never
returned by the API or written to logs/events/error_detail.
- Non-blocking per-cycle orchestrator: publish (CAS) -> propagation grace
(across cycles, no in-loop sleep) -> respond -> finalize/download, with a
bounded fresh-order retry chain (1 original + 3 retries) on propagation lag.
- Manual flow: user publishes the TXT record and confirms; manual DNS-01
cannot auto-renew unattended (auto-renew forced off and surfaced in the UI).
- Migration v8: additive, idempotent columns on letsencrypt_accounts/orders
and acme_challenges, plus a new letsencrypt_account_dns_credentials table.
- Challenge-type-aware diagnostics (port80/routing/DNS checks skipped for
DNS-01) and a DNS-01 event timeline in the order detail.
- Frontend: DNS-01 account + credentials management, cert wizard adaptation,
order-detail TXT records + verify, orders/renewal Method columns, and a
Settings kill-switch. README, release notes, and API docs updated.
Implements #35.
- Fix critical cascading NULL status bug in ACME challenge flow that could cause 404s
- Add defense-in-depth NULL handling across all ACME service methods
- Add new GET /api/letsencrypt/prerequisites endpoint for configuration checks
- Add interactive ACME Setup Guide with step-by-step navigation links
- Add URL-based tab navigation in Settings and SSL Management pages
- Harden retry flow: return clear 409 errors for invalid/cancelled orders
- Allow cancellation of invalid orders (backend + frontend)
- Improve error message extraction with consistent getErrorMsg helper
- Add status filter tabs (Active/Completed/Failed/All) for order list
- Add visual dimming for cancelled/invalid orders
- Add enhanced pagination with size changer and total count
- Add comprehensive diagnostic logging with ACME: prefix
- Update README with ACME architecture docs, quick start guide, and troubleshooting
Resolves#9
Made-with: Cursor
mustafa.ulukaya