mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-12 05:48:58 +00:00
main
26 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 taylanbakircioglu
|
4e2d936c27 |
fix(requestlog): stop the table size from scaling with fleet size
The row rate of `request_logs` was a function of how many nodes are installed,
not of what anyone did. Counted from the agent loop in linux_install.sh, each
agent's 30s cycle issues three logged calls - config, pending-requests,
upgrade-status (the heartbeat is already on the default exclude list) - plus
keepalived-config and keepalived-status every fifth cycle. That is ~9 800
rows/day per agent, essentially all of them 200s meaning "nothing changed".
Measured on PostgreSQL 15 against the real DDL and all nine indexes, at 2 424
bytes/row:
20 agents ~196k rows/day 453 MB/day row cap reached in 2.5 days
200 agents ~2.0M rows/day 4.4 GB/day row cap reached in 6 hours
500 agents ~4.9M rows/day 11 GB/day row cap reached in 2 hours
The cap holds, so nothing runs away - but it holds by deleting, and what it
deletes is everything else. The shipped policy says 7 days of successes and 30
days of failures; on a 200-node fleet it delivers about six HOURS of both. The
forensic record the feature exists for is evicted by polling noise, and the
larger the installation the less history it keeps.
`requestlog.capture_agent_success`, default FALSE: a SUCCESSFUL inbound call
from an agent is not recorded. Failures always are, whatever the flag says -
they are what an operator needs and they are rare, so they cost nothing. With
this the table's size follows operator activity, and adding nodes does not
shorten anyone's retention.
Agent traffic is identified by header only, no database round-trip on the hot
path: the installed agent sends `X-API-Key` and never `Authorization`, the UI
sends a JWT and never an agent key. `generate-install-script`, the one endpoint
that accepts either, classifies correctly under the same rule - an operator
generating a script sends Authorization, a self-upgrading agent sends only the
key. The result is stored in the existing `target` column, which already means
"who was on the other end" for outbound rows and now means the same for inbound
ones, so no schema change and the existing target index applies.
Second half, and the reason this is one commit: `operator` holds
`requestlog.read` because, per the migration that grants it, "operators debug
failing applies and ACME orders". They could not. An apply fails on the NODE,
and the node reports that over its own API key, so the row carrying the
diagnosis has `user_id IS NULL` - and own-rows-only scoping hid it from exactly
the role the grant was written for. Scoping now admits agent rows alongside the
caller's own. Deliberately keyed on `target = 'agent'` rather than `user_id IS
NULL`: anonymous traffic is not agent traffic, so failed logins and their
usernames, and unauthenticated probes, stay admin-only.
Verified end to end through the real middleware: a successful agent poll is
dropped, a 422 from config-validation-failed is kept, operator and anonymous
calls are unaffected, and flipping the setting on restores the old behaviour.
|
||
|
taylanbakircioglu
|
4c84596215 |
feat(logging): unified request/response log with configurable retention
Applies PR #59 by Mustafa Ulukaya (github.com/taylanbakircioglu/haproxy-openmanager/pull/59,
head
|
||
|
mustafa.ulukaya
|
ef26860df9 |
feat(logging): unified request/response log with configurable retention (v1.11.0)
Until now the only record of what happened was `user_activity_logs`, which stores non-GET 2xx operations with no bodies. When something failed you could see that a counter went up, never what was sent or what came back. This adds one queryable timeline covering both directions: - inbound: every API call, including GETs and including 4xx/5xx, with the user, client IP, status, duration and — redacted, size-capped — the request and response bodies. - outbound: every HTTP call the backend makes, tagged with who it went to (ACME/Let's Encrypt, Cloudflare, GoDaddy, HAProxy stats, agents, the ACME diagnostics probe). Outbound rows inherit the inbound request's id, so one operator action and the CA/DNS calls it triggered read as a single trace: opening a failed "Request Certificate" shows the exact POST /acme/new-order and the CA's 429 underneath. Implementation notes: - Capture is a pure-ASGI middleware that TEES the request and response streams rather than draining them. `await request.body()` inside a BaseHTTPMiddleware would consume the receive channel and break the raw-body agent heartbeat handler. Registered last so it is outermost: it then sees the final client-visible response and seeds correlation_id_context before the error handler reads it. - Rows are written by a batching background writer with a bounded queue, so the request path never awaits the database and a saturated logger drops rows visibly (surfaced on the page) instead of blocking. Redaction runs on the writer, off the request coroutine. - Secrets never land: headers are an allowlist with Authorization/Cookie kept only as a presence marker; body keys and value shapes are redacted (passwords, tokens, api_token, API keys, private-key PEMs, JWTs); the ACME JWS request body is never stored, because a stored protected+signature pair is a replayable credential — a summary is logged instead; DNS-provider errors record only the exception type; the ACME HTTP-01 challenge endpoint is excluded so key_authorization is never captured. - Retention is operator-configurable in Settings -> Request Log: separate day counts for successful and failed rows (7 / 30) plus a hard row cap (500k), whichever is reached first. Pruned in batches under a Postgres advisory lock, with the day counts bound as parameters, never interpolated. - New permissions requestlog.read / requestlog.manage. super_admin and security_admin get both, operator gets read, viewer gets neither. Schema: one new table (request_logs) plus its settings seed, SCHEMA_VERSION 10 -> 11, auto-migrated. No existing table altered, no agent or rendered-config change. Kill switches: REQUEST_LOG_ENABLED=false (middleware never registered) or the `enabled` toggle in Settings. Tests: 245 new (7 backend files + 1 frontend), full suite 1655 backend + 17 frontend passing. |
||
|
mustafa.ulukaya
|
7dfd31832a |
feat(vip): store the keepalived.conf an agent finds on a node
Ingest side of adopting an existing VIP. A node reports the keepalived.conf it found and does NOT own to a new endpoint, and the finding is kept in a new vip_discoveries table (one row per agent, since the file is per-node). Reporting is read-only on the node. The heartbeat cannot carry this: it has the VIP address and a best-effort MASTER/BACKUP, while rendering a node's config needs eleven fields, so the file itself has to be read and parsed server-side - parsing keepalived's block syntax in bash is not something to attempt on a production load balancer. Secrets are split at ingest. The reported content may contain the VRRP auth_pass, so the password is Fernet-encrypted into its own column through the same key path as vip_instances, and the stored copy of the file has it masked. Nothing readable through the API, the UI preview or a database dump carries it in cleartext, and the parse result is never logged. A file that does not parse records its error rather than failing the agent's poll loop, and a report of "the file is gone" clears the row so the UI stops offering a stale candidate. The keepalived-config delivery gains allow_takeover and takeover_expected_hash. The agent refuses to overwrite a keepalived.conf without our ownership marker, which is the guard that protects a hand-maintained setup - and is exactly the guard adoption has to pass. Rather than weaken it, an adopted VIP authorises exactly ONE takeover of exactly the file that was analysed by pinning its md5, so a config edited between adoption and Apply is still refused instead of being silently overwritten. SCHEMA_VERSION 10 -> 11 for the new table and two additive columns. Nothing existing is altered, but the bump re-seeds the four built-in roles, which the upgrade notes call out. |
||
|
mustafa.ulukaya
|
a6166d11b9 |
feat(ssl): add CSR generation and signed-certificate import (backend)
New /api/ssl/csrs endpoint group: generate a private key + CSR server-side
(RSA 2048/4096, ECDSA P-256/P-384; full subject + DNS SANs with wildcard
support), list/detail/delete CSRs, and import the CA-signed certificate.
- New ssl_csrs table (SCHEMA_VERSION 9 -> 10, additive + idempotent); the
migration re-raises on failure so a failed run is retried instead of being
stamped as applied.
- Import verifies the certificate against the stored key as a hard gate
(match=None is treated as an integrity error, not a lenient pass), rejects
malformed and expired certificates with 400, warns on SAN drift, and
creates a normal ssl_certificates row (source=csr, cluster_id=NULL,
last_config_status=PENDING) so it flows through the standard
Apply Management -> agent pull pipeline.
- Concurrency: FOR UPDATE row lock serialises double-import and
delete-during-import; a partial unique index reserves pending CSR names;
soft-deleted same-name certs are reactivated preserving the row id.
- Security: no CSR endpoint ever returns the private key (explicit column
lists, enforced by a static test); the key copy on the CSR row is NULLed
after import; ssl.create/read/delete permissions enforced on every
endpoint incl. reads; per-user rate limit on key generation, which runs
in a worker thread; csr_id and cluster_ids are int32-guarded.
- ssl_service: extract _prepare_cert_fields from create_cert_row (behaviour
unchanged, extraction tests untouched) and add stage_ssl_config_versions
reusing the exact ssl-{id}-create-{ts} version-name scheme.
- Tests: crypto round-trip for all four algorithms, model validation,
import-flow unit tests, endpoint auth/permission pinning, migration and
key-non-exposure static assertions.
|
||
|
taylanbakircioglu
|
9e2ea04777 |
feat(haproxy): preserve SPOE filter + frontend log-format on import/edit (v1.8.8, Issue #38)
Bulk import / manual edit silently dropped `filter spoe engine ...` (Coraza WAF) and frontend `log-format` because the parser recognised only a fixed directive set. The regenerated config then missed the SPOE engine, so HAProxy failed with "unable to find SPOE engine 'coraza' used by the send-spoe-group". - parser: capture `filter` + `log-format`/`log-format-sd` into new ParsedFrontend fields - db: additive nullable `log_format` + `filters` TEXT columns on frontends (SCHEMA_VERSION 8->9) - generator: new `filter` bucket flushed before http-request rules so `filter` precedes `send-spoe-group`; `log-format` kept in prelude - bulk import: preview dict, change-detection, persist (create + merge-update); cluster-aware SPOE pre-flight advisories (missing-filter + host-prerequisite) surfaced in the UI - manual CRUD: full round-trip (get/create/update) incl. React form fields (no null-wipe) - reject/rollback: restore the new columns; restore path + wizard helper kept in parity - backend `option spop-check` recognised (suppresses spurious warning for coraza-spoa) - tests: test_spoe_filter_import.py; full suite green (1079 passed) |
||
|
taylanbakircioglu
|
c492b26bb1 |
feat(acme): DNS-01 challenge support with pluggable DNS providers (v1.8.0)
Add ACME DNS-01 (TXT-record) validation alongside the existing HTTP-01, for internal/isolated clusters with no public port 80 and for wildcard certificates. Opt-in via a global kill-switch (default off); HTTP-01 is byte-for-byte unchanged, with zero agent or rendered-config changes. - Pluggable DNS provider interface (Manual + Cloudflare). Per-account credentials are Fernet-encrypted at rest, verified on save, and never returned by the API or written to logs/events/error_detail. - Non-blocking per-cycle orchestrator: publish (CAS) -> propagation grace (across cycles, no in-loop sleep) -> respond -> finalize/download, with a bounded fresh-order retry chain (1 original + 3 retries) on propagation lag. - Manual flow: user publishes the TXT record and confirms; manual DNS-01 cannot auto-renew unattended (auto-renew forced off and surfaced in the UI). - Migration v8: additive, idempotent columns on letsencrypt_accounts/orders and acme_challenges, plus a new letsencrypt_account_dns_credentials table. - Challenge-type-aware diagnostics (port80/routing/DNS checks skipped for DNS-01) and a DNS-01 event timeline in the order detail. - Frontend: DNS-01 account + credentials management, cert wizard adaptation, order-detail TXT records + verify, orders/renewal Method columns, and a Settings kill-switch. README, release notes, and API docs updated. Implements #35. |
||
|
taylanbakircioglu
|
a1192e602d |
feat: HA / VIP (Keepalived) management from the UI (#27)
Manage highly-available virtual IPs backed by Keepalived (VRRP) directly from the OpenManager
UI — no more SSHing into nodes to install/configure Keepalived by hand. Builds on the agent
pull-architecture: define the VIP centrally, click Apply, and the agents converge.
Highlights:
- New "HA / VIP" tab: create a virtual IP, pick a per-node interface, select which pool nodes
participate (MASTER/BACKUP roles + priorities); live MASTER/BACKUP/FAULT per node.
- On Apply, agents install & configure Keepalived (unicast VRRP, cloud-safe default) across the
major distros (Debian/Ubuntu, RHEL/CentOS/Alma/Rocky, Fedora, SUSE/openSUSE, Alpine) with a
HAProxy health-check, so the VIP fails over automatically when HAProxy drops.
- Single-node (a managed floating IP without failover) and multi-node VRRP failover both work.
- VIP changes ride the standard Apply Management flow with the standard "View Change" diff.
- Approval-gated deletion (safety): deleting a running VIP is staged for approval and the VIP
keeps running, untouched, until you approve it — an agent never tears a VIP down without an
explicit human approval. Per-VIP Diagnostics view; opt-in package uninstall (only on nodes
where OpenManager installed it). A node already running a hand-managed Keepalived is detected
and never overwritten ("externally managed").
- Fully opt-in and backward compatible: nodes/clusters without a VIP are unaffected. Adds
vip_instances + vip_members tables (idempotent SCHEMA_VERSION bump; existing data and
passwords unaffected) and a `vip` RBAC permission group.
- Also includes a HAProxy config-generator robustness fix: auto-inject a stick-table when a
frontend uses a stick counter (track-sc / sc_*_rate) but declares none.
On-prem / L2 (VRRP) scope; the UI notes the cloud caveat.
|
||
|
taylanbakircioglu
|
b34d7cf811 |
fix: reactivate disabled backend servers from the UI + v1.6.3 (Issue #24)
A backend server toggled OFF (is_active=false) vanished from the UI with no way to reactivate it: GET /api/backends honored include_inactive for backends but the server sub-queries hardcoded 'AND is_active = TRUE'. - get_backends: server sub-queries now honor include_inactive (default callers unchanged); added last_config_status to the server payload so the UI can tell a DISABLED server (re-enableable) from a DELETION (pending delete). - toggle_server: persists an entity snapshot so an Apply-Management Reject rolls back is_active (previously left the server stuck disabled). - BackendServers.js: requests include_inactive, shows disabled servers with the ON/OFF switch + an 'Inactive' tag, hides only DELETION-pending servers, and keeps soft-deleted BACKENDS hidden (so include_inactive doesn't resurface them). - Config generation unchanged: disabled servers stay '# DISABLED:' comments and convert back to live lines when re-enabled. Startup migration hardening (multi-replica / rolling-deploy safety): create_essential_tables fails fast on lock contention and retries; run_all_migrations is serialized by a session advisory lock and gated by a schema_migrations version marker, so an already-current schema is skipped instead of issuing lock-heavy DDL that a serving replica's traffic could block at startup. Idempotent and fail-open. Version reported consistently across all layers (version.json, backend fallback, frontend package) -> 1.6.3. |
||
|
taylanbakircioglu
|
bd6a31cb0d |
feat: v1.6.0 — Multi-Factor Authentication (Issue #18)
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.
Highlights
==========
* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
`MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
`XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
+ `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
Users page; admins reset (single user or bulk) but never enable on
behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.
Security hardening
==================
* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
(prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
audit log so request bodies (TOTP / backup codes in flight) never
appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
XFF strategy so a single ingress address cannot exhaust the bucket
for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
`MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
with the same username can no longer occlude the active user
(also closes a small account-enumeration side channel).
Database
========
Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):
- users: mfa_enabled, mfa_method, mfa_secret_encrypted,
mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
- mfa_backup_codes (user_id ON DELETE CASCADE)
- mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
attempts, expires_at)
- mfa_pending_enrollments (user_id ON DELETE CASCADE)
Frontend
========
* Login page becomes a 3-phase state machine
(credentials → MFA → submitting); legacy single-step login is
preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
Admins viewing other users with MFA off see a non-actionable info
icon explaining that only the user themselves can enable MFA.
Deployment
==========
* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
a placeholder; `SECRET_KEY` is also placeholder-ized so both are
injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
uses `window.location.host` for `/api/*` and is routed by the
existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
into the production bundle.
Tests
=====
* New unit suites:
- `test_mfa_service.py` (TOTP, encryption, backup codes)
- `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
- `test_mfa_rate_limits.py` (env override + dataclass immutability)
- `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.
Documentation
=============
* README MFA section (overview, day-to-day operations, emergency
reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.
Issue: #18
|
||
|
taylanbakircioglu
|
02b1cb2bca |
feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13)
Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit. |
||
|
taylanbakircioglu
|
07942a82e8 |
feat: ACME stability & enterprise audit (v1.4.0) — fixes #10 #11 #12
Issue #10 — Silent timezone failure on ACME certificate save - Normalize tz-aware expiry_date to UTC tz-naive before INSERT/UPDATE in ssl_certificates (TIMESTAMP WITHOUT TIME ZONE) — restores ACME download path Issue #11 — Duplicate _acme_challenge_backend in generated config - Generator guard: skip auto-append when backend already rendered - Agent-sync filter: _should_sync_backend() drops system-managed backends and detaches their server rows to prevent orphans - Restore filter: IGNORED_BACKENDS skips reserved names during cluster restore - Parser warning: reserved_backend_names blocks accidental manual import - Cleanup migration: removes orphan rows + cascading server entries (idempotent) Issue #12 — Validated orders required manual completion - New 60s background task complete_pending_acme_orders, flag-independent, multi-replica safe via FOR UPDATE SKIP LOCKED + 30s updated_at watermark - Per-order pg_advisory_lock(0x41434D45, order_id) serializes UI-Complete and auto-task races; idempotency guard returns existing certificate cleanly - retry_order endpoint reports in_progress: true within 30s window so the UI surfaces an info toast instead of duplicating CA requests - ACMEAutomation surfaces stuck orders (status=valid && !ssl_certificate_id) with a one-click Complete action and Cancel fallback; conditional 30s polling Other hardening - ACME state machine error_detail persisted as structured JSON across challenge, finalize, download stages for actionable post-mortems - CertificateRequest Pydantic model: domain regex + min_length/max_length and cluster_ids defaulting to all ACME-enabled clusters when "global" is selected - Renewal cluster fallback now requires acme_enabled=TRUE in addition to active - _complete_certificate preserves manual cluster assignments on renewal, surfaces cluster_errors, raises explicit error on missing private key - Audit logging covers acme_certificate_requested/revoked, ca_chain_imported, account created/deactivated/purged, order retried/cancelled - Settings UI exposes acme.staging_url_override for private test CAs (Pebble) - Schema additions: acme_challenges.attempts (default 0) and last_attempt_at, index idx_letsencrypt_orders_status_updated; all migrations idempotent Tests (41/41 passing) - test_acme_expiry_normalize, test_acme_duplicate_backend, test_acme_state_machine, test_acme_pydantic_validation, test_acme_audit_logging, test_acme_concurrency CI / packaging - docker-build.yml reads version.json and pushes additional product-version tag (e.g. 1.4.0) alongside latest and timestamp build id Closes #10 Closes #11 Closes #12 |
||
|
taylanbakircioglu
|
cd4a94beb1 |
feat: agent IP/VIP live update + script update detection banner
- Agent scripts now detect and send ip_address in DAEMON heartbeat (Linux: ip route, macOS: ifconfig) - Backend validates agent-reported IPs via ipaddress stdlib, COALESCE preserves existing on NULL - IP/VIP change logging (non-critical, try/except wrapped) for operational visibility - New source_file_hash column on agent_script_templates for reliable update detection - Migration changed to ON CONFLICT DO NOTHING to prevent overwriting UI-customized scripts on restart - GET /versions returns script_update_available flag (disk hash vs DB hash comparison with fallback) - Frontend Alert banner warns users of new agent script versions and directs to Reset to Defaults - Reset to Defaults and Popconfirm modals explicitly warn about custom script edit loss - Full backward compatibility: old agents without ip_address field continue working unchanged Made-with: Cursor |
||
|
taylanbakircioglu
|
93d7ad8fdb |
feat: add ACME Auto SSL with Let's Encrypt integration (v1.1.0)
Add automated SSL certificate management via ACME protocol (RFC 8555): - Full ACME client implementation (account registration, HTTP-01 challenges, certificate issuance/renewal) - Configurable ACME providers (Let's Encrypt, ZeroSSL, custom CA) via Settings UI - Auto-renewal scheduler with PENDING -> Apply -> APPLIED flow alignment - ACME account management (register, deactivate) from UI - Certificate request wizard with domain validation and cluster targeting - Zero changes to HAProxy agent scripts - challenges routed through existing architecture - Comprehensive security hardening (no private key exposure in API responses) - Full backward compatibility with existing SSL, Apply, Rollback, and Restore workflows - Updated README, API documentation, and Kubernetes deployment notes - Version management embedded in code (v1.1.0) - UI messaging improvements for agent-pull architecture accuracy Made-with: Cursor |
||
|
taylanbakircioglu
|
77bd3ecead |
fix: add missing REJECTED and DELETION values to config_status enum
Root cause: config_status enum was created with only PENDING and APPLIED values. The REJECTED value was never added due to a silent duplicate_object exception in create_essential_tables(). This caused SSL certificate listing to crash with "invalid input value for enum config_status: REJECTED" on fresh installations. Also adds scrollable containers to Apply Management page to prevent Agent Sync Status card from being pushed off-screen. Closes #7 Made-with: Cursor |
||
|
taylanbakircioglu
|
922bc2ce6d |
improve: SSL certificate listing reliability, error visibility, and ARM64 support
Fixes #6 - Fix NameError in soft-deleted certificate reactivation path by reordering variable extraction before DB operations - Replace silent empty-array returns with HTTP 500 on SQL errors, making failures visible in both API responses and server logs - Add primary_domain migration for schema consistency across fresh and upgraded installations (backfill from legacy domain column) - Use primary_domain in non-cluster SSL query branch for schema compatibility - Surface SSL fetch errors in frontend via toast notifications - Harden connection cleanup in error handlers with try/except - Add QEMU + Buildx for linux/amd64,linux/arm64 multi-platform Docker image builds - Update GitHub Actions to latest versions (checkout v4, login v3, build-push v6) Made-with: Cursor |
||
|
taylanbakircioglu
|
b0feca3c2e |
feat: add keepalived VRRP state (MASTER/BACKUP) detection to agent heartbeat
- Add keepalive_state and keepalive_ip columns to agents table (migration + schema) - Add keepalive fields to AgentHeartbeat Pydantic model (backward compatible) - Update heartbeat endpoint to persist keepalive data to DB and cache in Redis - Add multi-method keepalived detection in agent scripts (journalctl, log files, VIP check) - Update dashboard-stats agents/status API with Redis-first keepalive lookup - Update GET /api/agents to include keepalive_state and keepalive_ip - Show MASTER/BACKUP tag in Dashboard AgentStatusCard - Show keepalive info in Agent Management registered agents table - Add Keepalive column to Cluster Management table with VIP search support Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
taylanbakircioglu
|
851377aedf |
feat: Add HAProxy proxy name collision prevention system
- Add preserved_listen_blocks column to agents table for storing agent's local listen block names - Implement reserved names check (stats, monitoring, admin, etc.) for frontend/backend creation - Add dynamic collision detection against agent's preserved listen blocks - Apply collision checks to CREATE, UPDATE endpoints and bulk import - Add debug mode for failed config validation (saves to /tmp/haproxy-failed-*.cfg) - Fix JSON character stripping for ACL and use_backend rules - Remove collision protection from agent scripts (now handled by backend) - All collision checks wrapped in try-except for backwards compatibility |
||
|
Taylan Bakırcıoğlu
|
b212fb92bc |
Add SSL Advanced Options support (Backend) - Part 1
FEATURE: Complete SSL Advanced Options implementation for frontend and backend server SSL ✅ DATABASE: - Added SSL parameter columns to frontends table: * ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites * ssl_min_ver, ssl_max_ver, ssl_strict_sni - Added SSL parameter columns to backend_servers table: * ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers - Migration functions: add_ssl_advanced_options_to_frontends() and add_ssl_advanced_options_to_servers() ✅ MODELS: - FrontendConfig: Added 7 new SSL fields for bind parameters - ServerConfig: Added 4 new SSL fields for server parameters - AgentHeartbeat: Added system_info field (fixes HTTP 422 validation error) ✅ BULK IMPORT PARSER: - Parse alpn, npn, ciphers, ciphersuites, ssl-min-ver, ssl-max-ver, strict-sni from bind lines - Parse sni, ssl-min-ver, ssl-max-ver, ciphers from server lines - Store parsed values in frontend/server objects - User-friendly warnings about imported SSL parameters ✅ CONFIG GENERATOR: - Generate bind lines with SSL advanced options: 'bind :443 ssl crt file.pem alpn h2,http/1.1 ciphers ...' - Generate server lines with SSL advanced options: 'server s1 addr:port ssl sni hostname ssl-min-ver TLSv1.2' - Support both NEW MODE (multiple certs) and OLD MODE (single cert) USER IMPACT: - Bulk import now correctly parses SSL configs with alpn/npn/ciphers - SSL parameters preserved during import (not lost anymore) - Agent heartbeat fixed (no more offline agents) - Ready for UI implementation (next commit) EXAMPLE USAGE: Frontend: bind 0.0.0.0:8443 ssl crt cert1.pem crt cert2.pem alpn h2,http/1.1 Server: server s1 10.1.1.1:443 ssl verify required sni backend.example.com ssl-min-ver TLSv1.2 NEXT: Frontend UI components for editing these SSL options |
||
|
taylanbakircioglu
|
0fc18fde38 |
feat: Add HAProxy options support for backends and frontends
Implemented comprehensive HAProxy options field support for both backend and frontend entities to enable standard HAProxy directives like 'option http-keep-alive', 'option httplog', 'option forwardfor', etc. Changes: - Database: Added 'options' TEXT column to backends and frontends tables - Models: Added options field to BackendConfig, BackendConfigUpdate, and FrontendConfig - API Endpoints: Updated CREATE, UPDATE, and GET endpoints to handle options field * Backend: CREATE/UPDATE/GET with options support * Frontend: CREATE/UPDATE/GET with options support (fixed 5 SELECT queries) - Config Generator: Added options block generation for both backends and frontends - Bulk Import Parser: * Added options field to ParsedBackend and ParsedFrontend dataclasses * Implemented option directive parsing with validation * Added unknown option warnings * Fixed bulk parse response to include options field - Bulk Import Merge: Added options field comparison in UPDATE logic - UI Components: * BackendServers.js: Added options TextArea form field * FrontendManagement.js: Added options TextArea form field Features: - Multi-line options support (newline-separated format) - Option validation with known HAProxy options list - Backward compatible (NULL options for existing entities) - Bulk import support with merge strategy - Full CRUD support for both manual and bulk operations Technical Details: - Format: Newline-separated TEXT field for multiple options - Validation: Warns about unknown options but allows them - Config Generation: Each option written as separate directive - Agent: Standard HAProxy config validation applies Total: 10 files modified, ~195 lines added, 26 integration points verified |
||
|
taylanbakircioglu
|
1ea1c6a29f |
feat: Major stability and feature improvements
This commit consolidates multiple improvements from internal development: ## Agent Stability Improvements - Add database connection pooling (min=10, max=50) for better performance - Prevent config reapply on agent restart by fetching last_applied_version from database - Optimize SSL fetch to only run when config changes (98% API call reduction) - Make SSL_SYNC_TIMESTAMP_FILE agent-specific to prevent race conditions - Fix agent offline display issue due to database connection bottleneck - 10x faster heartbeat response (200ms → 20ms) ## Bulk Import UPSERT Support - Parse endpoint detects existing entities (New/Existing status) - Bulk-create supports UPDATE for existing backends/frontends (merge strategy) - New servers can be added to existing backends - Existing servers preserved (no deletion in MVP) - Field-by-field value comparison (only changed fields updated) - Pending apply conflict prevention (409 error) - Fixed duplicate key error on server INSERT - Backend marked PENDING when servers added ## Apply Management Fixes - Fixed deleted entities not showing (include_inactive parameter) - Backend/Frontend GET endpoints support inactive entities for Apply Management - All pending changes now visible - Phantom backend bug protection maintained ## Backend Delete Improvements - Automatically clean ACL/use_backend rules from frontends - Prevents HAProxy validation errors after backend deletion - Frontend references automatically updated ## UI/UX Improvements - Cluster selector status dot auto-refreshes every 30 seconds - Real-time agent health monitoring (no page refresh needed) - Parse message shows only NEW entities (cleaner) - Status labels: 'Update' → 'Existing' (clearer meaning) - Multi-line parse success messages - Detailed summary breakdown with tooltips Technical Changes: - backend/database/connection.py: Connection pool implementation - backend/main.py: Pool initialization and cleanup - backend/routers/*: UPSERT logic, field comparison, include_inactive - backend/utils/agent_scripts/*: Applied version tracking, SSL optimization - frontend/src/components/*: UI improvements, status indicators - frontend/src/contexts/ClusterContext.js: Auto-refresh agent health Impact: - Supports 50+ concurrent agents (previously ~10) - Zero config reapply on restart/upgrade - Bulk import handles existing entities correctly - All pending changes visible in Apply Management - Real-time cluster health status - No HAProxy validation errors after backend delete |
||
|
taylanbakircioglu
|
281e23ea27 |
feat: Add SSL usage_type (Frontend/Server) with conditional private key requirement
This is a comprehensive update that adds SSL certificate differentiation for frontend (HAProxy bind) and server (backend verification) use cases. FEATURES: - SSL certificates can be marked as 'frontend' or 'server' usage type - Frontend SSL: Private key REQUIRED (for HAProxy bind ssl crt) - Server SSL: Private key OPTIONAL (CA cert only for backend verification) - UI dropdown for usage type selection - Dynamic form validation based on usage type - Filtering: Frontends see only Frontend SSL, Backends see only Server SSL DATABASE: - Added usage_type column to ssl_certificates (default: 'frontend') - Made private_key_content nullable for server SSL support - Migration automatically runs on pod restart BACKEND: - Pydantic v2 compatibility (@field_validator, @model_validator) - SSL router: usage_type filtering support - Agent endpoint: usage_type field included - Improved migration robustness with better error handling - Fixed duplicate ensure_agents_table() function - Fixed JSONB permissions insert with json.dumps() - Fixed ON CONFLICT constraints with explicit checks FRONTEND: - SSL Management: Usage Type dropdown with visual feedback - Frontend Management: Filters only Frontend SSL certificates - Backend Servers: Filters only Server SSL certificates - Dynamic private key validation (required for Frontend, optional for Server) - Improved form UX with color-coded hints AGENT SCRIPTS (Linux & macOS): - Support for Server SSL without private key - Conditional PEM file creation (cert+key vs cert-only) - usage_type awareness in SSL deployment - Backward compatible with existing Frontend SSL certificates DOCKER: - Increased npm timeout for slow networks (300s → 600s) - Increased fetch-retries (5 → 10) - Reduced maxsockets for stability (3 → 1) All changes are backward compatible. Existing SSL certificates default to 'frontend' type and continue working unchanged. Tested with: HAProxy 2.8+, PostgreSQL 15, React 18 |
||
|
taylanbakircioglu
|
6c24971c19 |
Migration: Convert use_backend_rules to JSONB for consistency
Database Type Consistency Fix:
- acl_rules: JSONB ✓
- redirect_rules: JSONB ✓
- use_backend_rules: TEXT ✗ (INCONSISTENT!)
Issue:
Different data types cause:
- JSON serialization inconsistencies
- Query performance differences
- Potential data corruption
Fix Applied:
1. CREATE TABLE (Line 2038):
Changed: use_backend_rules TEXT
To: use_backend_rules JSONB DEFAULT '[]'::jsonb
2. ALTER TABLE (Line 121):
Changed: ADD COLUMN use_backend_rules TEXT
To: ADD COLUMN use_backend_rules JSONB DEFAULT '[]'::jsonb
3. Type Conversion Migration (Line 2318-2334):
Added automatic conversion from TEXT to JSONB
Handles: NULL, empty string, existing JSON data
Safe conversion with CASE statement
Migration Logic:
IF column type is TEXT or VARCHAR:
- NULL → '[]'::jsonb
- Empty string → '[]'::jsonb
- Existing JSON → Parse to JSONB
- Invalid data → Fails gracefully
Benefits:
- Consistent JSONB type across all rule fields
- Better query performance (JSONB indexing)
- Type safety in application code
- Automatic array validation
Impact: SAFE - Migration runs automatically on startup
|
||
|
taylanbakircioglu
|
97fd0e7dad |
Migration: Add ssl_certificate_id to backend_servers + Remove emojis
Database Migration: - Added ssl_certificate_id column to backend_servers table - Added FK constraint to ssl_certificates table - ON DELETE SET NULL behavior - Idempotent migration (safe to run multiple times) Column Details: Name: ssl_certificate_id Type: INTEGER Nullable: YES Foreign Key: ssl_certificates(id) On Delete: SET NULL Migration Function: add_ssl_certificate_id_to_backend_servers() Called in run_migrations() at Line 1523 Code Cleanup: - Removed emojis from migration logs - Removed emojis from SSL dropdown status icons - Changed to text: Valid, Expiring, Expired - Changed to text: Global, Cluster Error Fixed: GET /api/backends - 500 column "ssl_certificate_id" does not exist After migration runs on startup, column will exist and API will work |
||
|
taylanbakircioglu
|
c72859d507 |
Fix: Add cluster_id to config requests to prevent wrong cluster selection
- Add cluster_id column to agent_config_requests table - Update config request endpoint to accept and validate cluster_id - Frontend now sends cluster_id with config requests - Fixes issue where agents in pools with multiple clusters get wrong config requests Technical Details: - Database migration adds cluster_id as nullable foreign key for backward compatibility - Backend validates that agent belongs to requested cluster via pool_id check - Improved logging includes cluster name for better traceability - No impact on existing features (apply management, sync status, entity CRUD) |
||
|
taylanbakircioglu
|
6aae0f4309 | Initial commit |