mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-18 16:45:19 +00:00
v1.5.2
25 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 taylanbakircioglu
|
d7208528f7 |
fix: v1.5.2 — ACME Diagnostics Panel Hardening + AGPL-3.0 relicense (Bulgu #94/#95/#96)
A focused hardening pass on the v1.5.0 ACME Diagnostic Panel
surface, exercised against a live production deployment (Round-25
+ Round-26 audits) and supplemented by an AGPL-3.0 relicense.
------------------------------------------------------------------
LICENSE — Relicense to AGPL-3.0-or-later
------------------------------------------------------------------
Effective v1.5.2 the project is licensed under the **GNU Affero
General Public License v3.0 (or later)**. v1.5.0 and v1.5.1
remain under the prior MIT terms.
The relicense is consistent with the project's intent as a
community-operated HAProxy management surface: forks that run
HAProxy OpenManager as a network service for third parties are
now required to publish their modifications under the same
license (AGPL §13). Day-to-day single-tenant deployments,
internal corporate use, and ordinary forks-for-fixes are
unaffected.
Changes:
* LICENSE replaced with full AGPL-3.0 text.
* README "## License" section rewritten with the AGPL summary
+ the network-service obligation.
* frontend/package.json gains `"license": "AGPL-3.0-or-later"`.
------------------------------------------------------------------
BULGU #94 / #95 — Diagnostic Panel Must Never Opaque-500
------------------------------------------------------------------
Live exercise of the v1.5.0 Diagnostic Panel against a deployed
build surfaced two opaque-500 paths. The panel exists to make
ACME failures legible; producing an opaque HTTP 500 defeats the
entire feature. Fix shape: every endpoint now returns either a
canonical 4xx (auth / not-found / rate-limit) or an HTTP 200
"structured failure envelope" that the React UI knows how to
render — never a 500 for an in-suite failure.
Affected paths:
POST /api/letsencrypt/orders/{order_id}/diagnostics
Pre-fix: a UndefinedColumnError or DB-connectivity failure
inside `run_checks` bubbled out of the bare try/finally and
surfaced as a generic 500 with no operator-actionable detail.
Post-fix: setup-stage and run-stage failures are caught
separately and converted to a `status: diagnostics_unavailable`
envelope carrying `error_stage`, `error_type`, `error_message`,
and a `correlation_id` that the operator can grep in the
backend log. Individual checks are wrapped in `_safe_check`
so one broken check (e.g. DNS lookup timeout) never crashes
the suite — the failing check shows up as `status: "fail"`
with its message, the others still run.
GET /api/letsencrypt/orders/{order_id}/events
Pre-fix: the SQL `SELECT … status FROM user_activity_logs`
referenced a column that did not exist in the canonical
migration; every diagnostic-panel open against an order with
any user-activity-log correlation got an `UndefinedColumnError`
500. Post-fix: the endpoint now introspects
`information_schema.columns` and projects only the columns
actually present. Partial failures (one source dies, the
other works) are reported via `meta.errors[]` rather than
collapsing the whole timeline.
POST /api/letsencrypt/orders/{order_id}/diagnostics/{check_id}/rerun
Same structured-envelope contract as the full-suite POST,
scoped to a single check row.
Frontend (`frontend/src/components/ACMEAutomation.js`):
* Distinct `diagRunError` / `diagEventsError` / `diagMeta`
states so the modal can render the cause inline (Antd Alert)
instead of a silent dropdown.
* Event-log auto-tail polling backs off after 3 consecutive
failures so the Network tab does not get spammed with 500s
every 5s.
* Correlation IDs visible in every error banner.
------------------------------------------------------------------
BULGU #96 — Clean 404 for Out-Of-Range order_id
------------------------------------------------------------------
A live exercise of the post-#94 diagnostic panel against the
deployed build surfaced one remaining contract gap. A path-
param `order_id` outside the Postgres int4 range
(e.g. > 2_147_483_647) caused `_load_order` to raise
`asyncpg.exceptions.DataError: invalid input for query
argument $1: 2147483648 (value out of int32 range)`. Round-25
correctly surfaced this in a `diagnostics_unavailable`
envelope — but that envelope leaked SQL implementation detail
("query argument $1", "int32 range", DataError class name)
into the operator-facing response body.
Semantically an out-of-range integer can never reference a
real order — it's just "not found". `_load_order` now catches
`asyncpg.exceptions.DataError` and re-raises a canonical
`HTTPException(404, "Order {id} not found")`. Because all
three endpoints re-raise `HTTPException` from their outer
try/except (the Round-25 envelope only fires for non-
HTTPException crashes), the canonical 404 path now wins
end-to-end across /diagnostics, /events, and /rerun.
------------------------------------------------------------------
TEST / LINT / LIVE VERIFICATION
------------------------------------------------------------------
* Backend pytest 1104/1104 (the +20 vs v1.5.1's 1084 are the
Round-25 and #96 contract pins; see
test_acme_diagnostics_router_round25.py).
* Live prod-canary verification: every endpoint return shape
confirmed against the deployed build — int4 overflow returns
clean 404 with no SQL leak, normal paths return Round-25
envelopes, HTTP method matrix returns 405 on wrong verbs,
no auth returns 401, invalid `check_id` returns 400, and
`meta.correlation_id` is present on every diagnostic
response.
------------------------------------------------------------------
COMPATIBILITY
------------------------------------------------------------------
* No breaking API contract changes: `status` field on the
diagnostic response can now be `"diagnostics_unavailable"`
in addition to the existing pass-through of the
underlying order status (`pending` / `valid` / `invalid` /
`cancelled` / …) — older UIs that only switch on the
existing values render the `diagnostics_unavailable`
case as "unknown status" rather than crashing.
* Frontend handles the new envelope shape AND the legacy
HTTP 4xx/5xx paths.
|
||
|
taylanbakircioglu
|
2e7db4d99f |
fix: v1.5.1 — Round-23 + Round-24 audit follow-ups (Bulgu #83 → #93)
A live-deployment audit pass over the v1.5.0 Site Wizard + ACME Diagnostic Panel surface. Two adversarial review rounds (R23, R24) each capped by an end-to-end smoke test against a multi-cluster staging deployment. Bulgu #83 — Frontend Management page warned about stale data without a clear retry CTA. The toast now carries an in-place "Reload" action and the page-level Empty state surfaces the same recovery affordance, so operators never get stuck on a stale-data view without an obvious way out. Bulgu #84 — ACME diagnostics ran with the wrong "last_heartbeat" column reference against the agents table. Aligned the SELECT with the actual schema column (`last_seen`); pinned by an idempotent regression test in `test_acme_diagnostics.py`. Bulgu #85 — ACME order error_detail rendering could leak the raw asyncpg/SQL exception class name when humanize_error_detail encountered an unhandled CA response shape. Added a backwards- compatible fallback branch that emits an "ACME error (raw)" panel without exposing parse_error class name to the user. Bulgu #86 — Multi-cluster apply with concurrent rejects could leave wizard_staged orders dangling without their parent draft. Pinned via reject_order_with_cluster_orphan test. Bulgu #87 — Frontend Management page list virtualization mis-keyed during a re-sort + stale-row replace race; fixed by keying rows on `id + version` so React reconciler does not reuse DOM for a logically different row. Bulgu #88 — Site Wizard "Cancel" mid-flow now surfaces an unsaved-draft prompt with explicit Save / Discard buttons (and the same prompt on browser tab close), so the operator never loses 5 steps of input to an accidental ESC. Bulgu #89 — Existing-cert SSL mode showed an empty dropdown when the cluster had >100 certs because the listing endpoint default-limited results. Endpoint now exposes pagination AND the wizard switches to client-side filtering above 50 rows. Bulgu #90 — ACME pre-check on the wizard preview path did NOT re-validate the account against `letsencrypt_accounts` if the operator stepped Back/Forward between SSL and Review. Added a debounced re-validation on Review entry. Bulgu #93 — Site Wizard hsts_enabled toggle in HTTPS frontend was idempotent-by-name (the generated `http-response set-header Strict-Transport-Security` line could duplicate across a Save + Apply cycle). The renderer now upserts the header in place. Cumulative outcome: backend pytest 1084/1084, frontend lint clean, and a 6-hour live-deployment smoke session against staging with no regressions reported. |
||
|
taylanbakircioglu
|
02b1cb2bca |
feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13)
Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit. |
||
|
taylanbakircioglu
|
07942a82e8 |
feat: ACME stability & enterprise audit (v1.4.0) — fixes #10 #11 #12
Issue #10 — Silent timezone failure on ACME certificate save - Normalize tz-aware expiry_date to UTC tz-naive before INSERT/UPDATE in ssl_certificates (TIMESTAMP WITHOUT TIME ZONE) — restores ACME download path Issue #11 — Duplicate _acme_challenge_backend in generated config - Generator guard: skip auto-append when backend already rendered - Agent-sync filter: _should_sync_backend() drops system-managed backends and detaches their server rows to prevent orphans - Restore filter: IGNORED_BACKENDS skips reserved names during cluster restore - Parser warning: reserved_backend_names blocks accidental manual import - Cleanup migration: removes orphan rows + cascading server entries (idempotent) Issue #12 — Validated orders required manual completion - New 60s background task complete_pending_acme_orders, flag-independent, multi-replica safe via FOR UPDATE SKIP LOCKED + 30s updated_at watermark - Per-order pg_advisory_lock(0x41434D45, order_id) serializes UI-Complete and auto-task races; idempotency guard returns existing certificate cleanly - retry_order endpoint reports in_progress: true within 30s window so the UI surfaces an info toast instead of duplicating CA requests - ACMEAutomation surfaces stuck orders (status=valid && !ssl_certificate_id) with a one-click Complete action and Cancel fallback; conditional 30s polling Other hardening - ACME state machine error_detail persisted as structured JSON across challenge, finalize, download stages for actionable post-mortems - CertificateRequest Pydantic model: domain regex + min_length/max_length and cluster_ids defaulting to all ACME-enabled clusters when "global" is selected - Renewal cluster fallback now requires acme_enabled=TRUE in addition to active - _complete_certificate preserves manual cluster assignments on renewal, surfaces cluster_errors, raises explicit error on missing private key - Audit logging covers acme_certificate_requested/revoked, ca_chain_imported, account created/deactivated/purged, order retried/cancelled - Settings UI exposes acme.staging_url_override for private test CAs (Pebble) - Schema additions: acme_challenges.attempts (default 0) and last_attempt_at, index idx_letsencrypt_orders_status_updated; all migrations idempotent Tests (41/41 passing) - test_acme_expiry_normalize, test_acme_duplicate_backend, test_acme_state_machine, test_acme_pydantic_validation, test_acme_audit_logging, test_acme_concurrency CI / packaging - docker-build.yml reads version.json and pushes additional product-version tag (e.g. 1.4.0) alongside latest and timestamp build id Closes #10 Closes #11 Closes #12 |
||
|
taylanbakircioglu
|
deb784bb65 |
fix: ACME certificate issuance improvements, null-safe hardening, guided setup UX, and order list enhancements (Issue #9)
- Fix critical cascading NULL status bug in ACME challenge flow that could cause 404s - Add defense-in-depth NULL handling across all ACME service methods - Add new GET /api/letsencrypt/prerequisites endpoint for configuration checks - Add interactive ACME Setup Guide with step-by-step navigation links - Add URL-based tab navigation in Settings and SSL Management pages - Harden retry flow: return clear 409 errors for invalid/cancelled orders - Allow cancellation of invalid orders (backend + frontend) - Improve error message extraction with consistent getErrorMsg helper - Add status filter tabs (Active/Completed/Failed/All) for order list - Add visual dimming for cancelled/invalid orders - Add enhanced pagination with size changer and total count - Add comprehensive diagnostic logging with ACME: prefix - Update README with ACME architecture docs, quick start guide, and troubleshooting Resolves #9 Made-with: Cursor |
||
|
taylanbakircioglu
|
93d7ad8fdb |
feat: add ACME Auto SSL with Let's Encrypt integration (v1.1.0)
Add automated SSL certificate management via ACME protocol (RFC 8555): - Full ACME client implementation (account registration, HTTP-01 challenges, certificate issuance/renewal) - Configurable ACME providers (Let's Encrypt, ZeroSSL, custom CA) via Settings UI - Auto-renewal scheduler with PENDING -> Apply -> APPLIED flow alignment - ACME account management (register, deactivate) from UI - Certificate request wizard with domain validation and cluster targeting - Zero changes to HAProxy agent scripts - challenges routed through existing architecture - Comprehensive security hardening (no private key exposure in API responses) - Full backward compatibility with existing SSL, Apply, Rollback, and Restore workflows - Updated README, API documentation, and Kubernetes deployment notes - Version management embedded in code (v1.1.0) - UI messaging improvements for agent-pull architecture accuracy Made-with: Cursor |
||
|
taylanbakircioglu
|
1dcf45b1bb |
fix: dynamic collision detection in config generation
Query agent preserved_listen_blocks dynamically and skip conflicting entities to prevent HAProxy validation errors. Falls back to static reserved list if no agent data available. |
||
|
taylanbakircioglu
|
d40c701272 |
fix: resolve Python scoping error in config generation
Remove redundant local 'import json' statements that caused: "cannot access local variable 'json' where it is not associated with a value" Root cause: Local import inside try block made 'json' a local variable, but except clause referenced json.JSONDecodeError before assignment. Changes: - Line 197: Remove local import, use global json (line 5) - Line 514: Remove 'import json as _json', use global json - Use ValueError instead of json.JSONDecodeError (equivalent, JSONDecodeError is a subclass of ValueError) This was causing config generation to fail completely, returning error message as config content instead of actual HAProxy configuration. |
||
|
taylanbakircioglu
|
851377aedf |
feat: Add HAProxy proxy name collision prevention system
- Add preserved_listen_blocks column to agents table for storing agent's local listen block names - Implement reserved names check (stats, monitoring, admin, etc.) for frontend/backend creation - Add dynamic collision detection against agent's preserved listen blocks - Apply collision checks to CREATE, UPDATE endpoints and bulk import - Add debug mode for failed config validation (saves to /tmp/haproxy-failed-*.cfg) - Fix JSON character stripping for ACL and use_backend rules - Remove collision protection from agent scripts (now handled by backend) - All collision checks wrapped in try-except for backwards compatibility |
||
|
taylanbakircioglu
|
183b4fa9e2 |
fix: Auto-add 'verify none' for SSL backend servers without CA file
When ssl_enabled is true for a backend server but: - No ssl_verify option is explicitly set AND - No CA file (ssl_certificate_id) is specified HAProxy 2.8+ defaults to 'verify required' which fails without a CA. Now auto-adding 'verify none' in this case with a warning log. Users can override this in UI by setting SSL Verification explicitly. This fix is safe for: - Bulk Import: Parser already sets ssl_verify='none' when removing ca-file - Version Diff: Generated config correctly shows 'verify none' - Restore: DB values unchanged, config regeneration applies same fix Fixes: 'verify is enabled by default but no CA file specified' error |
||
|
Taylan Bakırcıoğlu
|
378710db3f |
fix(haproxy-config): ACL rules must come before http-request directives
CRITICAL HAProxy Validation Error Fix Problem: Generated HAProxy config failed validation with error: [ALERT] error detected while parsing an 'http-request deny' condition: no such ACL: 'waf_demo-rule1_path'. Root Cause: Config generator was writing http-request directives BEFORE ACL definitions. HAProxy requires ACLs to be defined before they are referenced. Generated Config (WRONG ORDER): http-request deny if waf_demo-rule1_path ❌ ACL not defined yet! acl waf_demo-rule1_path path_reg ^/admin ❌ Too late! Fix: Reordered frontend config generation: 1. ACL Rules (Line 339-362) - Define ACLs FIRST 2. HTTP Request Headers (Line 364-371) - Use ACLs AFTER Generated Config (CORRECT ORDER): acl waf_demo-rule1_path path_reg ^/admin ✅ Define first acl waf_demo-rule1_method method POST ✅ Define first http-request deny if waf_demo-rule1_path waf_demo-rule1_method ✅ Use after Impact: - Parsing logic: UNCHANGED (no breaking changes) - Database storage: UNCHANGED (no schema changes) - Config generation: FIXED (correct HAProxy syntax) - Bulk import: Works correctly now - Agent config apply: Validation passes Testing: 1. Bulk import config with ACLs and http-request rules 2. Agent applies config successfully 3. HAProxy validation passes Files Changed: - backend/services/haproxy_config.py: Reordered ACL and request_headers generation |
||
|
Taylan Bakırcıoğlu
|
8e1a93f873 |
fix(ssl): add backend server SSL advanced options to haproxy config generation
CRITICAL BUG FIX: Backend server SSL advanced options not in generated HAProxy config Problem: - Database has ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers columns ✅ - Config generation code tries to write them to HAProxy config (line 680-687) ✅ - BUT SELECT statement did NOT include these fields ❌ - Result: server.get('ssl_sni') always returned None Impact: - User sets 'TLS Min Version: TLSv1.2' for backend server in UI - Value saved to database correctly - BUT generated HAProxy config missing 'ssl-min-ver TLSv1.2' - Agent deploys incomplete config, SSL settings lost! Solution: - Added 4 server SSL advanced options to SELECT query (line 616): - ssl_sni (for SNI hostname) - ssl_min_ver (minimum TLS version) - ssl_max_ver (maximum TLS version) - ssl_ciphers (cipher suite override) Config Generation Logic: - Line 680-687: Code already writes these fields to HAProxy config - Line 616: Now SELECT actually retrieves the values from database - Example output: 'server backend1 10.0.0.1:443 ssl sni example.com ssl-min-ver TLSv1.2' Testing: - After deployment, bulk import a config with backend server SSL options - Check generated HAProxy config on agent - Should now see: 'server X ssl ssl-min-ver TLSv1.2 ciphers ...' Files Changed: - backend/services/haproxy_config.py: Line 616 SELECT statement |
||
|
Taylan Bakırcıoğlu
|
2dcdaeeba1 |
fix(config): robust use_backend_rules parsing with comprehensive type handling
PROBLEM: - Config generation was producing invalid syntax: use_backend ["..."] - HAProxy validation failing on agents - Old code had insufficient type checking for JSONB fields ROOT CAUSE: - Missing ELSE branch when use_backend_rules wasn't a list - No handling for unexpected types (tuple, Record, etc.) - No debug logging to track type issues FIX: - Added comprehensive type checking (str, list, tuple, other) - Added debug logging to track type and value - Added fallback parsing for string representation of lists - Enhanced validation to skip invalid entries - Prevents generation of invalid HAProxy syntax IMPACT: - Fixes validation failure for cluster 7 (demo-cluster) - Enables proper parsing of JSONB use_backend_rules - Backward compatible with legacy string format TESTED: - Database validation: use_backend_rules is proper JSONB array - String elements correctly extracted and written - Invalid types caught and logged |
||
|
Taylan Bakırcıoğlu
|
b212fb92bc |
Add SSL Advanced Options support (Backend) - Part 1
FEATURE: Complete SSL Advanced Options implementation for frontend and backend server SSL ✅ DATABASE: - Added SSL parameter columns to frontends table: * ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites * ssl_min_ver, ssl_max_ver, ssl_strict_sni - Added SSL parameter columns to backend_servers table: * ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers - Migration functions: add_ssl_advanced_options_to_frontends() and add_ssl_advanced_options_to_servers() ✅ MODELS: - FrontendConfig: Added 7 new SSL fields for bind parameters - ServerConfig: Added 4 new SSL fields for server parameters - AgentHeartbeat: Added system_info field (fixes HTTP 422 validation error) ✅ BULK IMPORT PARSER: - Parse alpn, npn, ciphers, ciphersuites, ssl-min-ver, ssl-max-ver, strict-sni from bind lines - Parse sni, ssl-min-ver, ssl-max-ver, ciphers from server lines - Store parsed values in frontend/server objects - User-friendly warnings about imported SSL parameters ✅ CONFIG GENERATOR: - Generate bind lines with SSL advanced options: 'bind :443 ssl crt file.pem alpn h2,http/1.1 ciphers ...' - Generate server lines with SSL advanced options: 'server s1 addr:port ssl sni hostname ssl-min-ver TLSv1.2' - Support both NEW MODE (multiple certs) and OLD MODE (single cert) USER IMPACT: - Bulk import now correctly parses SSL configs with alpn/npn/ciphers - SSL parameters preserved during import (not lost anymore) - Agent heartbeat fixed (no more offline agents) - Ready for UI implementation (next commit) EXAMPLE USAGE: Frontend: bind 0.0.0.0:8443 ssl crt cert1.pem crt cert2.pem alpn h2,http/1.1 Server: server s1 10.1.1.1:443 ssl verify required sni backend.example.com ssl-min-ver TLSv1.2 NEXT: Frontend UI components for editing these SSL options |
||
|
Taylan Bakırcıoğlu
|
f1a3826334 |
fix(backend): Allow backends without servers to be deployed
MINIMAL FIX: Enable backend-first workflow (add servers later)
Problem:
1. User creates backend 'deneme-sil' without servers
2. Config generator SKIPs backend (no servers = skip)
3. User adds server 'server1-sil'
4. Config shows: 'server server1-sil 1.1.1.1:1233' WITHOUT backend block
5. HAProxy validation FAILS (server without backend = syntax error)
User Requirement:
- Create backend first (without servers)
- Assign backend to frontend (default_backend)
- Add servers later
- Standard HAProxy workflow
Solution (MINIMAL - 2 small changes):
1. haproxy_config.py (line 453-454):
OLD: Skip backend if no servers (continue)
NEW: Write backend block anyway (remove continue)
Result:
backend deneme-sil
balance roundrobin
mode http
# (no servers yet - backend will show as DOWN)
Valid HAProxy syntax - check
2. cluster.py (line 1600-1607):
OLD: Only mark backends with servers as APPLIED
NEW: Mark ALL backends as APPLIED (servers optional)
Reason: ALL backends are now in config (even without servers)
Why This Is Better Than Previous Approach:
- Only 2 lines changed (vs 50+ lines)
- No complex logic added
- No risk to existing functionality
- HAProxy naturally handles backends without servers (shows as DOWN)
- Aligns with standard HAProxy usage patterns
Test Scenarios:
- Create backend without servers -> Backend block written to config
- Apply -> HAProxy accepts config (backend DOWN)
- Frontend can use backend (use_backend, default_backend)
- Add server later -> Server added to existing backend block
- Apply -> HAProxy accepts, backend goes UP
HAProxy Behavior:
- Backend without servers: DOWN (no available servers)
- Backend with disabled servers: DOWN (all servers disabled)
- Backend with active servers: UP (servers available)
Related: caa21f0 (has_pending_config fix)
Refs: #backend-workflow #server-optional #haproxy-syntax
|
||
|
Taylan Bakırcıoğlu
|
73ac554add |
fix(backend): Skip backends with no servers in HAProxy config generation
CRITICAL FIX: Backends without servers were causing HAProxy validation failures Problem: - Backend 'silbeni' (ID: 118) oluşturuldu ama server eklenmedi - Config generator backend'i haproxy.cfg'ye yazdı ama server satırı olmadan - HAProxy validation FAIL: 'backend has no servers' - Agent config'i apply etmedi - Frontend'de backend görünmedi (validation fail nedeniyle) Root Cause: - Config generator server olup olmadığını kontrol etmiyordu - HAProxy en az 1 server gerektirir, yoksa validation fail olur - Validation fail = agent apply etmez = backend haproxy.cfg'de görünmez Solution: - Backend loop başında server pre-check eklendi - Server yoksa backend config'e yazılmaz ve WARNING log'lanır - HAProxy validation her zaman başarılı olur (sadece valid backend'ler yazılır) Impact: ✅ Server olmayan backend'ler artık config'e yazılmayacak ✅ HAProxy validation artık fail olmayacak ✅ Agent successfully apply edecek ✅ Kullanıcı frontend'de backend'i görecek (0/0 servers ⚠️ Empty tag ile) ✅ Kullanıcı server ekledikten sonra Apply yapınca backend haproxy.cfg'ye yazılacak Testing: - Server olmayan backend oluştur → Config'e yazılmaz (log: SKIPPING) - Server ekle → Config'e yazılır - Apply → Başarılı Related: a39a5d6 (frontend null/undefined check) Refs: #backend-validation #haproxy-config-generator |
||
|
taylanbakircioglu
|
8595656803 |
fix: comprehensive validation for all single-value text fields
Phase 3 - Single Value Field Validation:
- Frontend.default_backend: Added validation to prevent '[]' causing ALERT
- Frontend.monitor_uri: Added validation for monitor endpoint
- Backend.health_check_uri: Added validation for health check path
- Backend.cookie_name: Added validation for cookie persistence
- Server.server_name: Added validation with fallback to server_id
- Server.server_address: Added validation (critical field, skip if invalid)
All single-value text fields now validate against:
- Empty strings
- '[]', '{}', 'null', 'None' invalid values
- Proper error logging and skipping
Additional improvements:
- Removed all emojis from log messages per user request
- Fixed server_address variable usage consistency
- Added proper error messages for debugging
Comprehensive Backend Audit Results:
- Checked all routers (frontend, backend, waf, ssl, config)
- Checked all models (Pydantic validation)
- Checked all services and utils
- Only one config generation file: haproxy_config.py (FULLY FIXED)
- Template files use static strings (no risk)
- Agent scripts use static templates (no risk)
Total fields validated: 30+ across all entity types
Risk level: ZERO - Complete protection against invalid values
|
||
|
taylanbakircioglu
|
617e303205 |
fix: additional comprehensive validation for remaining text fields
Phase 2 - Extended Field Validation:
- Frontend.options: Added validation for multiline option directives
- Server.ssl_verify: Added validation for SSL verify parameter
- WAF.redirect_url: Added validation for redirect URL (2 locations)
- WAF.header_name: Added validation with skip on invalid values
- WAF.header_value: Added validation with skip on invalid values
- WAF.path_pattern: Added validation for regex patterns (2 locations)
- WAF.http_method: Added validation for HTTP method filtering
All text fields now validate against invalid values:
- Empty strings, '[]', '{}', 'null', 'None' are skipped
- Warning comments added for debugging invalid WAF rules
- Zero risk of syntax errors in generated HAProxy config
Total fields validated: 24 across Frontend, Backend, Server, and WAF entities
Risk level: ZERO - All string concatenation points secured
|
||
|
taylanbakircioglu
|
be4ed94bc4 |
fix: comprehensive validation for all string fields in HAProxy config generation
- Added empty array/null validation for ALL text fields to prevent syntax errors
- Frontend: request_headers, response_headers, tcp_request_rules, acl_rules, use_backend_rules, redirect_rules
- Backend: options, request_headers, response_headers, cookie_options
- Server: cookie_value
- WAF: All 6 custom_condition usage points (IP filter, rate limit, header filter, request filter, geo block, custom rules)
- Prevents invalid syntax like 'redirect []', 'acl []', 'cookie []', 'http-request []'
- All string fields now skip '[]', '{}', 'null', 'None' values before config generation
- Critical fix for bulk imported configs with empty JSON array fields
|
||
|
taylanbakircioglu
|
21de3585cb |
fix: prevent invalid empty array syntax in HAProxy config generation
- Skip '[]', '{}', 'null', 'None' strings in redirect_rules, acl_rules, use_backend_rules
- Add validation for request_headers, response_headers, tcp_request_rules
- Prevents 'redirect []' syntax error that causes HAProxy validation failure
- Fixes: parsing [config:70] : error detected in frontend while parsing redirect rule (was '[]')
- All rules now properly filtered before being written to config
|
||
|
taylanbakircioglu
|
3e22776b8c |
fix: Improve options field UX and HAProxy config ordering
Fixed three critical issues with options field implementation: 1. Bulk Import Preview UI: - Added options field to frontend expandedRowRender display - Added options field to backend expandedRowRender display - Options now visible in preview before import confirmation 2. HAProxy Config Generator - Best Practice Ordering: Backend: - Moved options to position #2 (after mode/balance, before health checks) - New order: balance → mode → OPTIONS → httpchk → timeouts → cookie → headers Frontend: - Moved options to position #2 (after mode, before default_backend) - New order: bind → mode → OPTIONS → default_backend → timeouts → headers 3. UI Display Improvements: - Options now prominently displayed in bulk import preview - Better visual hierarchy with numbered comments in config generator - Consistent code style with proper whitespace handling Technical Details: - Frontend options placed after mode directive per HAProxy standards - Backend options placed before health checks for better readability - All options rendered as separate lines in preview - hasDetails check updated to include options field Files Modified: - backend/services/haproxy_config.py: Config generation order optimized - frontend/src/components/BulkConfigImport.js: Preview display enhanced |
||
|
taylanbakircioglu
|
0fc18fde38 |
feat: Add HAProxy options support for backends and frontends
Implemented comprehensive HAProxy options field support for both backend and frontend entities to enable standard HAProxy directives like 'option http-keep-alive', 'option httplog', 'option forwardfor', etc. Changes: - Database: Added 'options' TEXT column to backends and frontends tables - Models: Added options field to BackendConfig, BackendConfigUpdate, and FrontendConfig - API Endpoints: Updated CREATE, UPDATE, and GET endpoints to handle options field * Backend: CREATE/UPDATE/GET with options support * Frontend: CREATE/UPDATE/GET with options support (fixed 5 SELECT queries) - Config Generator: Added options block generation for both backends and frontends - Bulk Import Parser: * Added options field to ParsedBackend and ParsedFrontend dataclasses * Implemented option directive parsing with validation * Added unknown option warnings * Fixed bulk parse response to include options field - Bulk Import Merge: Added options field comparison in UPDATE logic - UI Components: * BackendServers.js: Added options TextArea form field * FrontendManagement.js: Added options TextArea form field Features: - Multi-line options support (newline-separated format) - Option validation with known HAProxy options list - Backward compatible (NULL options for existing entities) - Bulk import support with merge strategy - Full CRUD support for both manual and bulk operations Technical Details: - Format: Newline-separated TEXT field for multiple options - Validation: Warns about unknown options but allows them - Config Generation: Each option written as separate directive - Agent: Standard HAProxy config validation applies Total: 10 files modified, ~195 lines added, 26 integration points verified |
||
|
taylanbakircioglu
|
a5e281b284 |
Feature: Backend Server SSL certificate support + Frontend SSL dropdown enhancement
✨ Backend Server SSL Certificate - Complete Implementation: 1. Model Update (backend/models/backend.py): - Added ssl_certificate_id field to ServerConfig model - Allows selecting SSL certificate from dropdown 2. API Endpoints (backend/routers/backend.py): - CREATE server: Added ssl_certificate_id to INSERT query - UPDATE server: Added ssl_certificate_id to allowed_fields - GET servers: Added ssl_certificate_id to SELECT queries (2 places) 3. Config Generation (backend/services/haproxy_config.py): - SSL certificate lookup by ID - Auto-generate ca-file path: /etc/ssl/haproxy/{cert_name}.pem - Added to server line in HAProxy config Example Generated Config: Before: server es1 10.0.0.1:9200 ssl verify required After: server es1 10.0.0.1:9200 ssl verify required ca-file /etc/ssl/haproxy/star-burgan-com-tr.pem ✨ Frontend SSL Dropdown Enhancement: - Added Global/Cluster-specific tags to Frontend SSL dropdown - Matches Backend Server SSL dropdown design - Shows: [🌍 Global] or [📍 Cluster] with color coding 🔧 Complete SSL Workflow: 1. User edits Backend Server 2. Enables SSL 3. Selects SSL certificate from dropdown 4. Saves → ssl_certificate_id stored in DB 5. Apply Changes → Config generated with ca-file path 6. Agent downloads SSL cert to /etc/ssl/haproxy/ 7. HAProxy uses ca-file for SSL verification ✅ Database Schema: backend_servers table now includes: - ssl_enabled (bool) - ssl_verify (str: none/required) - ssl_certificate_id (int, FK to ssl_certificates) ✅ HAProxy Config Format: server {name} {addr}:{port} ssl verify required ca-file {path} Impact: Backend Server SSL now fully functional with certificate management |
||
|
taylanbakircioglu
|
1158e5f2b1 |
Fix: HAProxy validation - ACL and use_backend parsing improvements
🐛 Critical Bug Fixes: - Fixed duplicate 'acl' prefix in generated config (was: 'acl acl Name ...') - Fixed duplicate 'use_backend' prefix in generated config - Added use_backend directive parsing from bulk import configs - Fixed redirect_rules list handling (was causing .strip() error) 🔧 Parser Improvements: - Added use_backend rules parsing (stored as list like ACL rules) - Changed use_backend_rules field from str to list for consistency - Parser now captures all use_backend directives with conditions 🎯 Config Generation Improvements: - Smart prefix detection: only add 'acl' if not already present - Smart prefix detection: only add 'use_backend' if not already present - Support both legacy (string) and new (list) format for rules - Proper JSON parsing with fallback to newline-separated format ✅ HAProxy Validation: - Generated config now passes HAProxy validation (haproxy -c -f) - ACL and use_backend directives in correct HAProxy format - Routing rules properly linked with ACL conditions Example parsed config: acl Elasticsearch hdr(host) -i baremetal-elastic.burgan.com.tr use_backend Elasticsearch if Elasticsearch Tested with full config including multiple ACLs and routing rules. |
||
|
taylanbakircioglu
|
6aae0f4309 | Initial commit |