gsadmin/haproxy-openmanager
1
0
Fork 0
mirror of https://github.com/taylanbakircioglu/haproxy-openmanager.git synced 2026-09-17 16:15:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
292 Commits 10 Branches 25 Tags
main
Commit Graph

10 Commits

Author SHA1 Message Date
taylanbakircioglu c79391cd13 feat(acl): accept HAProxy -f pattern-file references with advisory warnings (v1.8.9, Issue #38) 
The manual Frontend editor, wizard and visual ACL builder hard-rejected the ACL
`-f <file>` flag while bulk import accepted it. Worse, a frontend imported with
an `-f` ACL could not be edited at all (422) until the ACL was dropped.

The original guard predated the fail-safe apply flow: the agent runs `haproxy -c`
before every reload, so a missing pattern file is rejected safely and the previous
config keeps running. Pattern files are operator-managed host files — the same
policy adopted for SPOE filter configs in v1.8.8.

- models: remove the 5 `-f` hard rejects (frontend acl/redirect/use_backend
  validators + wizard string/dict-redirect guards); `$(`/backtick and X!X
  contradiction guards unchanged
- routers/frontend: `_pattern_file_warnings` helper; non-blocking warning on
  create + update responses listing referenced pattern files (empty when no
  rule uses `-f` — zero noise)
- routers/config: bulk-import preview advisory listing pattern files per
  frontend (cluster config-dir aware, next to the SPOE advisories)
- React: remove the FrontendManagement submit gate and SiteWizard step gate;
  ACLRuleBuilder renders informational notes instead of errors and re-adds
  `-f (pattern file on host)` to the flag dropdown; create path now renders
  server warnings like update
- tests: 4 reject-pins inverted to accept-pins; new test_acl_pattern_file_allow.py
  (accept/guards-kept/zero-noise/advisory); full suite green (1094 passed)
 2026-07-14 00:27:11 +03:00
taylanbakircioglu 9e2ea04777 feat(haproxy): preserve SPOE filter + frontend log-format on import/edit (v1.8.8, Issue #38) 
Bulk import / manual edit silently dropped `filter spoe engine ...` (Coraza WAF)
and frontend `log-format` because the parser recognised only a fixed directive
set. The regenerated config then missed the SPOE engine, so HAProxy failed with
"unable to find SPOE engine 'coraza' used by the send-spoe-group".

- parser: capture `filter` + `log-format`/`log-format-sd` into new ParsedFrontend fields
- db: additive nullable `log_format` + `filters` TEXT columns on frontends (SCHEMA_VERSION 8->9)
- generator: new `filter` bucket flushed before http-request rules so `filter` precedes
  `send-spoe-group`; `log-format` kept in prelude
- bulk import: preview dict, change-detection, persist (create + merge-update); cluster-aware
  SPOE pre-flight advisories (missing-filter + host-prerequisite) surfaced in the UI
- manual CRUD: full round-trip (get/create/update) incl. React form fields (no null-wipe)
- reject/rollback: restore the new columns; restore path + wizard helper kept in parity
- backend `option spop-check` recognised (suppresses spurious warning for coraza-spoa)
- tests: test_spoe_filter_import.py; full suite green (1079 passed)
 2026-07-10 18:34:36 +03:00
taylanbakircioglu 02b1cb2bca feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13) 
Closes #13, Closes #14.

This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0
shipped the ACME stability & enterprise audit (Issues #10/#11/#12).
v1.5.0 builds on that foundation with two co-equal headline features
plus a 22-round audit campaign hardening the prior configuration
surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0
lands in v1.5.2).

------------------------------------------------------------------
HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13)
------------------------------------------------------------------
A live pre-flight + post-failure diagnostic surface for every ACME
order, reachable from the ACME Automation page. The panel exists
to make ACME failures legible to operators who do NOT have shell
access to the API host.

Endpoints (`backend/routers/acme_diagnostics.py`):
  POST /api/letsencrypt/orders/{order_id}/diagnostics
       Run the full 5-check suite (DNS / port-80 / routing /
       account / agents) and humanize the order's `error_detail`
       (>=11 RFC-8555 problem types, backwards compatible with
       legacy plain-string failures).
  POST /api/letsencrypt/orders/{order_id}/diagnostics/
                                {check_id}/rerun
       Re-run a single check in place — used by the "Re-run"
       button on every row of the modal's pre-flight table.
  GET  /api/letsencrypt/orders/{order_id}/events
       Merged event timeline combining the typed
       `acme_order_events` rows with correlated
       `user_activity_logs` entries (resource_type =
       'letsencrypt_order' AND resource_id = order_id). The
       diagnostic modal auto-tails this timeline every 5 seconds
       while open.

Service-level checks (`backend/services/acme_diagnostics.py`):
  * DNS resolution via stdlib socket.gethostbyname_ex through
    run_in_executor (intentionally avoiding an aiodns runtime
    dep for v1.5.0).
  * Port-80 HEAD probe, target locked to the order's domains,
    success on HTTP 200 OR 404, warns on egress timeout
    (corp egress policies routinely blackhole outbound 80 —
    fail-hard would be too noisy).
  * SSRF guard: probe refuses non-public IPs and surfaces the
    skip in the diagnostic result; IPv4-mapped IPv6 normalisation
    closes the `::ffff:169.254.169.254` cloud-metadata vector.
  * HAProxy routing presence check: matches the order's
    cluster_ids to a port-80 HTTP frontend.
  * ACME account validity check against `letsencrypt_accounts`.
  * Agent presence check (>=1 active agent in target cluster).
  * Every sub-check wrapped in a wall-clock timeout to bound
    impact on the API event loop.

RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate
limit on both run and rerun, backed by the (user_id, action,
created_at DESC) composite index.

Frontend (`frontend/src/components/ACMEAutomation.js`):
  * "Diagnose" button on every order row + the existing
    "stuck order" warning row.
  * Modal with two tabs:
    - Pre-flight Checks (Antd Table with status pills + Re-run
      buttons + humanized error banner)
    - Event Log (Antd Timeline with auto-tail polling, scroll-
      to-bottom, pause-on-hover)
  * Correlation IDs surfaced in error banners and individual
    check fail details for backend-log lookup.

------------------------------------------------------------------
HEADLINE FEATURE B — Site Setup Wizard (Issue #14)
------------------------------------------------------------------
A single guided flow that creates a Backend + Servers + HTTP
Frontend (and optional HTTPS Frontend) in one atomic transaction.

Endpoints (`backend/routers/site_wizard.py`):
  POST /api/site-wizard/preview     — diff-preview the changeset
  POST /api/site-wizard/create      — atomic execute
  POST /api/site-wizard/reject      — clean rollback (including
                                       any wizard_staged ACME
                                       orders)
  GET  /api/site-wizard/drafts      — draft persistence
  PUT  /api/site-wizard/drafts/{id} — save/update
  DELETE /api/site-wizard/drafts/{id}

Feature surface:
  * One screen captures both backend (mode + servers) AND
    frontend (http + optional https + SSL mode) inputs.
  * SSL modes: ACME (new order, HTTP-01 only for v1.5.0),
    Upload (existing PEM), Existing (link to a stored cert),
    or None.
  * ACME-staged path: wizard_staged_until watermark on the
    `letsencrypt_orders` row defers finalisation until agent
    confirmation; per-mode reject cleanly cancels and rolls
    back the staged order.
  * Live diff preview against the cluster's current generated
    config (renderer-evolution noise stripped — track-sc<N>
    dedup, per-server cookie strip, defaults-cookie
    inheritance, listen-block flattening).
  * Draft persistence with PEM stripped at save time (private
    keys never round-trip through the drafts table).
  * Per-cluster multi-tenancy: drafts and wizard_staged orders
    are isolated to the creating user's cluster scope.

Frontend (`frontend/src/components/SiteWizard.js`):
  * 4-step Antd Steps flow: Backend → Frontend → SSL → Review.
  * Render the live diff preview inline before commit.
  * Antd Form-level validation mirrors backend Pydantic
    validators (numeric bounds, HAProxy reserved keywords, ALPN
    consistency, IPv6 scope-id, domain regex, server name
    dedup).

------------------------------------------------------------------
AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82)
------------------------------------------------------------------
v1.5.0 includes 22 adversarial review passes. Each round produced
its own commit set in the corporate development line; this squash
collapses those into the v1.5.0 release artefact. Highlights:

  Round 1-4   Site Wizard core: dry-run parity, single-line
              value injection guard, ACL -f pattern-file block,
              SSL parity, timeout regex, form-state pin.
  Round 5-7   defaults-cookie inheritance, server-named-cookie
              guard, fe/be mode mismatch, duplicate server
              names, health_check_uri + server_address
              validators.
  Round 8-10  cookie_name / cookie_options newline-injection
              guard, dry-run parity (round 9), TCP-mode HTTP-only
              feature blockers.
  Round 11    SSL name path traversal + health-check >= 1.
  Round 12-13 SSL & ACME deep dive (Bulgu #23-#32).
  Round 14    single-line value injection (Bulgu #33).
  Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy
              reserved keywords, ALPN/TLS consistency,
              all-backup, multi-domain & multi-user enterprise
              edges, drain/HSTS/post-completion (Bulgu
              #34-#53).
  Round 18-21 concurrency, agent state, TCP-mode HTTP-only,
              list size caps, IPv6 scope-id, preview account
              validation, TCP backend + balance uri reject
              (Bulgu #54-#61).
  Round 22    FE error visibility + 3x stale-data lockouts,
              referential integrity + cascade safety,
              authentication & authorization, multi-cluster
              isolation, apply_pending_changes concurrency,
              script injection + bulk import multi-tenancy,
              prefix-stripped signature comparison
              (Bulgu #62-#82).

------------------------------------------------------------------
NO CORPORATE-SPECIFIC ARTIFACTS
------------------------------------------------------------------
This squash deliberately sanitises corporate hostnames, container
registry references, and TLS secret names into generic
placeholders (`your-registry.example.com/your-org`,
`haproxy-openmanager*.example.com`, `wildcard-tls`,
`taylanbakircioglu/haproxy-openmanager-*`) so the public artefact
contains no internal infrastructure detail. Pilot / development
history that retained those values stays in the corporate fork
and is NOT part of this commit.
 2026-05-14 00:04:19 +03:00
taylanbakircioglu 71c717364c fix: allow dot character in entity names for UI and backend validation 
Bulk import accepted dots in frontend/backend/server names but UI and
backend validators rejected them with ^[a-zA-Z0-9_-]+$. After import,
entities with dots could not be edited. HAProxy itself allows dots in
section names, so the regex is expanded to ^[a-zA-Z0-9_.-]+$ across
all 12 validation points (5 React form rules, 1 ACL char-strip,
3 Pydantic validators, 1 WAF validator, 2 config-validator warnings).
 2026-04-14 01:26:03 +03:00
Taylan Bakırcıoğlu 589f865f59 fix(ssl): rollback support + improved ALPN validation error message 
PART 1: Rollback Support for SSL Advanced Options
===================================================
Problem: SSL advanced options lost on reject/rollback operations

Root Cause:
- Snapshot creation uses SELECT * (includes all fields) ✅
- old_values contains SSL advanced options ✅
- Rollback UPDATE did NOT restore SSL fields ❌

Impact - Frontend:
- User changes ssl_alpn, then clicks Reject
- Rollback skipped: ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites, ssl_min_ver, ssl_max_ver, ssl_strict_sni
- Result: All SSL advanced options lost (set to NULL)

Impact - Backend Server:
- User changes ssl_min_ver='TLSv1.2', then clicks Reject
- Rollback skipped: ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers
- Result: Security issue - TLS version constraints removed!

Fix:
- backend/utils/entity_snapshot.py line 285-286: Added 7 frontend SSL fields to rollback UPDATE
- backend/utils/entity_snapshot.py line 458: Added 4 server SSL fields to rollback UPDATE

PART 2: Improved ALPN Validation Error Message
===============================================
Problem: User enters 'http/2' in ALPN field, gets generic error

User feedback: Tried 'h2,http/1.1,http/2' → validation error not clear

Root Cause:
- ALPN standard uses 'h2' for HTTP/2 (not 'http/2')
- Validator rejected 'http/2' but didn't explain the correct format

Fix:
- backend/models/frontend.py line 255-260: Detect common mistakes (http/2, http2, http-2)
- Provide helpful error: 'For HTTP/2, use "h2" (not "http/2")'

Before:
  "Invalid ALPN protocol: http/2. Valid protocols: h2, http/1.1, ..."

After:
  "Invalid ALPN protocol: http/2. For HTTP/2, use 'h2' (not 'http/2'). Valid protocols: ..."

Testing:
1. Rollback test: Edit frontend SSL, reject, verify SSL fields restored
2. Validation test: Enter 'http/2', verify friendly error message

Files Changed:
- backend/utils/entity_snapshot.py: _rollback_update() for frontend and server
- backend/models/frontend.py: validate_alpn() with better error messages
 2025-11-18 21:58:05 +03:00
Taylan Bakırcıoğlu e8690245af fix(ssl): add change detection and validators for SSL advanced options 
Critical additions:
1. Change Detection (backend/routers/config.py)
   - Added SSL advanced options to bulk import change detection logic
   - Detects changes in ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites
   - Detects changes in ssl_min_ver, ssl_max_ver, ssl_strict_sni
   - Changes will now appear in version diff

2. Pydantic Validators (backend/models/frontend.py, backend/models/backend.py)
   - TLS version validator: Only allows valid versions (SSLv3, TLSv1.0-1.3)
   - ALPN protocol validator: Only allows h2, http/1.1, http/1.0, h2c, spdy/*
   - NPN protocol validator: Only allows http/1.1, http/1.0, spdy/*
   - Prevents invalid values from being saved to database
   - HAProxy validation will not fail due to invalid SSL options

Impact:
- Bulk import will correctly detect SSL option changes
- Apply Management diff will show SSL changes
- User cannot enter invalid TLS versions or protocols
- Improved UX with early validation errors

Previous fixes in this series:
- Frontend GET API: Added SSL fields to response
- Bulk Import UPDATE: Added SSL fields to UPDATE statement
- Backend Server GET API: Added SSL fields to response

Test: Bulk import with ALPN change → Should see change in version diff
 2025-11-18 21:58:05 +03:00
Taylan Bakırcıoğlu b212fb92bc Add SSL Advanced Options support (Backend) - Part 1 
FEATURE: Complete SSL Advanced Options implementation for frontend and backend server SSL

✅ DATABASE:
- Added SSL parameter columns to frontends table:
  * ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites
  * ssl_min_ver, ssl_max_ver, ssl_strict_sni
- Added SSL parameter columns to backend_servers table:
  * ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers
- Migration functions: add_ssl_advanced_options_to_frontends() and add_ssl_advanced_options_to_servers()

✅ MODELS:
- FrontendConfig: Added 7 new SSL fields for bind parameters
- ServerConfig: Added 4 new SSL fields for server parameters
- AgentHeartbeat: Added system_info field (fixes HTTP 422 validation error)

✅ BULK IMPORT PARSER:
- Parse alpn, npn, ciphers, ciphersuites, ssl-min-ver, ssl-max-ver, strict-sni from bind lines
- Parse sni, ssl-min-ver, ssl-max-ver, ciphers from server lines
- Store parsed values in frontend/server objects
- User-friendly warnings about imported SSL parameters

✅ CONFIG GENERATOR:
- Generate bind lines with SSL advanced options: 'bind :443 ssl crt file.pem alpn h2,http/1.1 ciphers ...'
- Generate server lines with SSL advanced options: 'server s1 addr:port ssl sni hostname ssl-min-ver TLSv1.2'
- Support both NEW MODE (multiple certs) and OLD MODE (single cert)

USER IMPACT:
- Bulk import now correctly parses SSL configs with alpn/npn/ciphers
- SSL parameters preserved during import (not lost anymore)
- Agent heartbeat fixed (no more offline agents)
- Ready for UI implementation (next commit)

EXAMPLE USAGE:
Frontend: bind 0.0.0.0:8443 ssl crt cert1.pem crt cert2.pem alpn h2,http/1.1
Server: server s1 10.1.1.1:443 ssl verify required sni backend.example.com ssl-min-ver TLSv1.2

NEXT: Frontend UI components for editing these SSL options
 2025-11-18 21:58:05 +03:00
taylanbakircioglu 0fc18fde38 feat: Add HAProxy options support for backends and frontends 
Implemented comprehensive HAProxy options field support for both backend and frontend entities to enable standard HAProxy directives like 'option http-keep-alive', 'option httplog', 'option forwardfor', etc.

Changes:
- Database: Added 'options' TEXT column to backends and frontends tables
- Models: Added options field to BackendConfig, BackendConfigUpdate, and FrontendConfig
- API Endpoints: Updated CREATE, UPDATE, and GET endpoints to handle options field
  * Backend: CREATE/UPDATE/GET with options support
  * Frontend: CREATE/UPDATE/GET with options support (fixed 5 SELECT queries)
- Config Generator: Added options block generation for both backends and frontends
- Bulk Import Parser:
  * Added options field to ParsedBackend and ParsedFrontend dataclasses
  * Implemented option directive parsing with validation
  * Added unknown option warnings
  * Fixed bulk parse response to include options field
- Bulk Import Merge: Added options field comparison in UPDATE logic
- UI Components:
  * BackendServers.js: Added options TextArea form field
  * FrontendManagement.js: Added options TextArea form field

Features:
- Multi-line options support (newline-separated format)
- Option validation with known HAProxy options list
- Backward compatible (NULL options for existing entities)
- Bulk import support with merge strategy
- Full CRUD support for both manual and bulk operations

Technical Details:
- Format: Newline-separated TEXT field for multiple options
- Validation: Warns about unknown options but allows them
- Config Generation: Each option written as separate directive
- Agent: Standard HAProxy config validation applies

Total: 10 files modified, ~195 lines added, 26 integration points verified
 2025-11-13 10:12:30 +03:00
taylanbakircioglu bfa0caa006 Feature: Full UI support for use_backend rules editing 
✨ New Features:
- Added use_backend_rules validator to Frontend model
- UI now supports editing use_backend rules from Frontend Management page
- Array to string conversion for use_backend rules in edit modal

🔧 Model Improvements:
- Changed use_backend_rules field type from Optional[str] to Any (list support)
- Added parse_use_backend_rules validator (same logic as ACL/redirect rules)
- Handles 3 formats: Array, Textarea string (newline-separated), JSON string

💡 UI Improvements:
- Frontend edit modal automatically converts use_backend array to multi-line text
- Users can edit routing rules line by line in textarea
- Format: 'use_backend BackendName if condition'

✅ Complete Workflow:
1. Bulk Import: Config parsed → ACL + use_backend stored as array
2. Frontend Edit: Arrays converted to multi-line string in textarea
3. User edits ACL/use_backend rules in UI
4. Save: Textarea string → validator → array → database
5. Config Generation: Array → HAProxy config format

Example workflow:
  Parse: ['use_backend API if is_api']
  → Edit UI: 'use_backend API if is_api' (textarea)
  → User edits: 'use_backend API_v2 if is_api_v2'
  → Save: ['use_backend API_v2 if is_api_v2']
  → Generate: 'use_backend API_v2 if is_api_v2' (HAProxy config)
 2025-11-07 11:51:14 +03:00
taylanbakircioglu 6aae0f4309 Initial commit 2025-10-27 12:14:03 +03:00