mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-19 17:15:12 +00:00
v1.10.2
36 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 taylanbakircioglu
|
eee0a4716a |
feat(ssl): encrypt the pending CSR private key at rest (v1.10.1, closes #53)
Closes the follow-up filed during the v1.9.0 CSR review. The private key of a
PENDING CSR is now Fernet-encrypted in the database instead of being stored as
a raw PEM.
Why this key specifically: it is the one key in the system that sits idle. It
is generated at CSR creation, waits for an external CA to sign the request
(days to weeks), and is destroyed the moment the signed certificate is
imported. It is never transmitted to an agent and never leaves the server.
ssl_certificates.private_key_content and the ACME order keys are deliberately
NOT covered, because agents must receive those in plaintext on every poll, so
encrypting them at rest buys nothing without an end-to-end redesign.
Implementation follows the pattern already used for the VRRP secret, TOTP
secrets and DNS provider credentials: a new utils/csr_key_crypto.py with its
own CSR_ENCRYPTION_KEY env var and its own HKDF info string
("csr-private-key-v1"), so rotating one secret class never affects another.
No schema change and deliberately NO SCHEMA_VERSION bump: the Fernet token
replaces the PEM inside the existing ssl_csrs.private_key_pem TEXT column. A
bump would re-run the migration sequence and re-seed the four built-in roles to
their defaults, which is a needless side effect for a storage-format change.
Backward compatible with no data migration. Rows written before this release
hold a raw PEM and are still read unchanged; the discriminator is exact rather
than a heuristic, since a Fernet token is base64url and can never contain the
"-----BEGIN" marker. Legacy rows drain naturally because a CSR's key copy is
NULLed on import.
A key that cannot be decrypted (SECRET_KEY rotated while CSR_ENCRYPTION_KEY was
unset) now fails with an explicit "delete this CSR and create a new one" error.
Previously that situation would have surfaced as the far more confusing
"certificate does not match this CSR's private key".
Also documents all four per-purpose encryption keys in .env.template. Only
VIP_ENCRYPTION_KEY was listed; MFA_ENCRYPTION_KEY and
DNS_PROVIDER_ENCRYPTION_KEY had been missing since v1.6.0 and v1.8.0.
Verified before release, on a corporate pre-production environment and locally:
- Full backend suite 1234 -> 1243 passed (+9 new tests), 0 failed.
- Against a real Postgres: a CSR created through the API stores a Fernet token
with no PEM header in the column, and imports successfully.
- Full 1.10.0 -> 1.10.1 -> 1.10.0 drill on one database volume. The upgrade
logs "Schema already at version 10 (>= 10); skipping migration run", so no
migration executes and the built-in roles are not re-seeded. A CSR created on
1.10.0 with a plaintext key imports successfully after the upgrade, which is
the backward-compatibility guarantee proven against a real row rather than a
mock.
- rsa-2048, rsa-4096 and ecdsa-p384 all round-trip through create, encrypt,
decrypt and import.
- Key derivation is stable across processes: two independent containers sharing
SECRET_KEY decrypt each other's tokens (required for UVICORN_WORKERS > 1 and
multi-replica deployments), while a different SECRET_KEY yields None rather
than a wrong key or an exception.
- Downgrade behaviour was measured, not assumed: 1.10.0 cannot parse the token
and fails with HTTP 500 "key parse failed (encrypted?)" rather than pairing a
wrong key. The rollback note states the measured behaviour.
- No CSR endpoint returns the key in any form: list and detail responses
contain neither a PEM nor a Fernet token.
Not changed here, from the issue's "worth folding in" list: the create rate
limit is not a concurrency guard, create_csr holds a pooled connection across
RSA key generation, detail=str(e) echoes internal error text (a repo-wide
convention), and is_global skips cluster validation in both routers/ssl.py and
routers/csr.py. None are storage concerns and each is a separate change.
|
||
|
mustafa.ulukaya
|
8e534ef170 |
feat(dns): add GoDaddy DNS-01 provider (API Key+Secret / PAT, additive RRset writes)
Registers a third pluggable DNS provider for ACME DNS-01 alongside Manual and Cloudflare. Credentials are an API Key + Secret pair; leaving the Secret blank sends the Key as a Personal Access Token (Bearer), which is the migration path as GoDaddy retires the sso-key scheme. GoDaddy's Domains API v1 has no per-value TXT write: PUT on a record set replaces every value at that name. A certificate covering example.com and *.example.com publishes two different TXT values at the same _acme-challenge.example.com, so add/remove are read-modify-write - read the current set, merge, put the whole list back - with empty-data tombstone rows filtered out (they are rejected on echo) and DELETE used for the last value, since PUT with an empty array is rejected. The zone-wide sibling endpoints (.../records/TXT and .../records) would wipe SPF/DKIM/DMARC and the whole zone respectively, so the record path is built in one place that refuses an empty or dot segment. An unreadable record-set read fails closed rather than being treated as an empty set, because the PUT that follows would otherwise destroy the coexisting values. Zone lookup walks name suffixes probing the records API rather than the domain listing, so zones delegated to GoDaddy nameservers resolve and accounts that are rejected from the domain-details endpoint still work. Credential and eligibility failures during the walk surface instead of being reported as "no managed domain". Provider errors are sanitized at the single point where GoDaddy-supplied text enters a message, since those strings are persisted to order events and shown in the UI. No new dependency, no schema change, no frontend change - the credential form is rendered from the provider schema. |
||
|
mustafa.ulukaya
|
a6166d11b9 |
feat(ssl): add CSR generation and signed-certificate import (backend)
New /api/ssl/csrs endpoint group: generate a private key + CSR server-side
(RSA 2048/4096, ECDSA P-256/P-384; full subject + DNS SANs with wildcard
support), list/detail/delete CSRs, and import the CA-signed certificate.
- New ssl_csrs table (SCHEMA_VERSION 9 -> 10, additive + idempotent); the
migration re-raises on failure so a failed run is retried instead of being
stamped as applied.
- Import verifies the certificate against the stored key as a hard gate
(match=None is treated as an integrity error, not a lenient pass), rejects
malformed and expired certificates with 400, warns on SAN drift, and
creates a normal ssl_certificates row (source=csr, cluster_id=NULL,
last_config_status=PENDING) so it flows through the standard
Apply Management -> agent pull pipeline.
- Concurrency: FOR UPDATE row lock serialises double-import and
delete-during-import; a partial unique index reserves pending CSR names;
soft-deleted same-name certs are reactivated preserving the row id.
- Security: no CSR endpoint ever returns the private key (explicit column
lists, enforced by a static test); the key copy on the CSR row is NULLed
after import; ssl.create/read/delete permissions enforced on every
endpoint incl. reads; per-user rate limit on key generation, which runs
in a worker thread; csr_id and cluster_ids are int32-guarded.
- ssl_service: extract _prepare_cert_fields from create_cert_row (behaviour
unchanged, extraction tests untouched) and add stage_ssl_config_versions
reusing the exact ssl-{id}-create-{ts} version-name scheme.
- Tests: crypto round-trip for all four algorithms, model validation,
import-flow unit tests, endpoint auth/permission pinning, migration and
key-non-exposure static assertions.
|
||
|
taylanbakircioglu
|
9e5185c458 |
fix(security): post-review hardening — agent-inventory regression, coverage gaps, SSRF newNonce
Follow-up to the RCE/missing-auth/SSRF remediation, from a thorough multi-lens
review (3 agents + a black-box audit of all 201 routes). Backend-only; no
agent-script changes.
Regression fix (introduced by the previous commit):
- GET /api/agents was made JWT-only, but deployed agents call it WITH X-API-Key
(not a JWT) to read their applied_config_version and avoid re-applying config on
restart. It now accepts EITHER a valid operator JWT OR a valid agent X-API-Key,
so agents no longer get 401 (which caused a spurious HAProxy reload every restart).
Completeness (GHSA-3p5c siblings the first pass missed — same data class, now JWT):
- dashboard.py: GET /api/haproxy-cluster-pools/{id}/agents (full agent inventory —
a direct anonymous bypass of the GET /api/agents lockdown), /api/pools,
/api/haproxy-cluster-pools, /api/dashboard/stats, /api/dashboard/overview
(auth was optional -> leaked stats/names/health/alerts anonymously),
/api/haproxy/stats.
- waf.py: GET /api/waf/rules. health.py: GET /api/health/errors.
- agent.py: GET /api/agents/generate-uninstall-script/{platform} (agent-management
endpoint; was anonymous) now requires JWT or agent key, like generate-install-script.
- config.py: POST /api/config/{validate,optimize,templates/{id}/generate} were
optional-auth (logging only) and run a HAProxy validator on caller input; now
require a JWT. (bulk-create, parse-bulk, diff and configuration/request were
already mandatory-auth — verified.)
All newly-gated endpoints are frontend-only (axios sends the JWT) or unused;
agents never call them.
SSRF (GHSA-3vh4) gap:
- acme_service._get_nonce fetched directory['newNonce'] (from the attacker-
influenceable directory JSON) with a bare session, http allowed, dual-stack, and
BEFORE the guarded _signed_request POST. Now guarded (assert_public_url +
safe_connector + no redirects + timeout), matching the other ACME sinks.
Correctness:
- Three agent webhooks (config-applied, config-validation-failed, config-sync)
swallowed their auth 401 into a 200 error body via a bare `except Exception`.
Added `except HTTPException: raise` so the 401/403 propagates.
Audit result (live black-box, all 201 routes probed unauthenticated): no data
leak and no unauthenticated mutation anywhere; every sensitive route returns
401/403 (a pre-existing group of read handlers wraps the 401 into a 500 via a
broad except — no data is exposed; left as-is, documented as cosmetic).
Verified: full pytest tests/ (1145 passed, 0 failed; +16 regression tests) + live
localtest stack smoke — agent-key GET /api/agents=200, anonymous=401, all newly
gated endpoints reject anonymous and admit JWT, the 3 webhooks return 401.
|
||
|
taylanbakircioglu
|
520b69a1c6 |
fix(security): remediate RCE, missing-auth and SSRF advisories (backend-only, no agent changes)
Addresses three reported advisories, all verified against the code. Fixes are
entirely server-side — deployed agents already send a valid X-API-Key on every
call, so enforcing it does not require any agent-script change or upgrade.
GHSA-7rhv-c5pc-69r8 (CRITICAL RCE — agent script-template poisoning):
- POST/GET /api/agents/script-templates/{platform} now require the agents.version
permission (was authentication-only), matching POST /versions. Blocks a viewer
JWT from overwriting the root install/upgrade script.
GHSA-3p5c-m5m4-mjpx (missing authentication):
- Agent data-plane endpoints now REQUIRE a valid X-API-Key (was optional/skipped
when the header was absent), checked before any DB access: config,
ssl-certificates (private keys!), upgrade-status, heartbeat (by-name and the
previously auth-less by-id), configuration pending-requests. Removes keyless
heartbeat spoofing and keyless rogue-agent auto-registration.
- Operator/UI endpoints now require a JWT: GET /api/agents, the entire
/api/dashboard-stats router, /api/health/{deep,agents,clusters}, and
/api/ssl/certificates/{id}/config-versions. The simple /api/health liveness
probe stays public. Adds shared auth_middleware.require_authenticated_user.
GHSA-3vh4-gvxx-wm2p (SSRF via ACME directory_url):
- New utils/ssrf_guard.py (https-only + public-IP-only, IPv4-pinned, no redirects),
applied to settings test-connection, acme_service.get_directory and
_signed_request, and validated at Let's Encrypt account creation. The
test-connection response no longer reflects arbitrary upstream JSON keys
(information-disclosure oracle) — only fixed ACME field names.
Verified: full pytest tests/ (1128 passed, 0 failed) + live localtest stack smoke
(valid JWT/key paths return 200/404 as expected; anonymous requests 401; SSRF to
metadata/private/loopback refused). No changes to backend/utils/agent_scripts/*.
|
||
|
taylanbakircioglu
|
9e2ea04777 |
feat(haproxy): preserve SPOE filter + frontend log-format on import/edit (v1.8.8, Issue #38)
Bulk import / manual edit silently dropped `filter spoe engine ...` (Coraza WAF) and frontend `log-format` because the parser recognised only a fixed directive set. The regenerated config then missed the SPOE engine, so HAProxy failed with "unable to find SPOE engine 'coraza' used by the send-spoe-group". - parser: capture `filter` + `log-format`/`log-format-sd` into new ParsedFrontend fields - db: additive nullable `log_format` + `filters` TEXT columns on frontends (SCHEMA_VERSION 8->9) - generator: new `filter` bucket flushed before http-request rules so `filter` precedes `send-spoe-group`; `log-format` kept in prelude - bulk import: preview dict, change-detection, persist (create + merge-update); cluster-aware SPOE pre-flight advisories (missing-filter + host-prerequisite) surfaced in the UI - manual CRUD: full round-trip (get/create/update) incl. React form fields (no null-wipe) - reject/rollback: restore the new columns; restore path + wizard helper kept in parity - backend `option spop-check` recognised (suppresses spurious warning for coraza-spoa) - tests: test_spoe_filter_import.py; full suite green (1079 passed) |
||
|
taylanbakircioglu
|
e86e86a53c |
fix(acme): scope ACME nonce per CA - fixes ZeroSSL registration (v1.8.2)
ZeroSSL/Google account registration failed with `malformed: The Replay Nonce could not be base64url-decoded`: the ACME client (a process-wide singleton) kept a single anti-replay nonce shared across certificate authorities, so a nonce issued by one CA could be sent to another, and the auto-retry only covered `badNonce`. - Scope the nonce per CA (self._nonce_by_dir keyed by directory_url): a nonce from one CA is never sent to another; account registration always uses a fresh nonce from the target CA. - Broaden the 400 retry to also recover from the nonce-malformed rejection. - Fix _b64url_decode padding (used for the EAB HMAC key). Backend-only; HTTP-01 and Let's Encrypt are unaffected. Addresses #35. |
||
|
taylanbakircioglu
|
70ebc02e09 |
fix(acme): Cloudflare token, ZeroSSL EAB, Apply Management (v1.8.1)
Follow-up fixes for the DNS-01 feature reported on #35: - Cloudflare: sanitize the API token (strip surrounding quotes + any non token68 chars) so a pasted token with quotes/spaces no longer fails with "Invalid request headers"; verify-on-save shows a precise hint when it cleaned the input. Covers the automated orchestrator path too. - ZeroSSL/Google EAB: enter the EAB Key ID and HMAC Key per-account in the Register Account dialog (falls back to the global Settings value when blank); base64-validate the HMAC key; humanize the externalAccountRequired failure; and preserve the deliberate 409/422 instead of downgrading them to 400. - Apply Management: cluster ACME enable/disable changes now show under a dedicated "ACME Challenge Routing" section, are counted in the Apply/Reject dialogs, and Apply/Reject All process them (previously "Rejected 0 HA/VIP change(s)") - consistent with every other entity. Reject rolls acme_enabled back to the original via ORDER BY created_at ASC over the snapshot chain. - getErrorMsg surfaces field-level validation messages. Backward compatible (additive / strict superset; HTTP-01 unchanged). Addresses #35. |
||
|
taylanbakircioglu
|
c492b26bb1 |
feat(acme): DNS-01 challenge support with pluggable DNS providers (v1.8.0)
Add ACME DNS-01 (TXT-record) validation alongside the existing HTTP-01, for internal/isolated clusters with no public port 80 and for wildcard certificates. Opt-in via a global kill-switch (default off); HTTP-01 is byte-for-byte unchanged, with zero agent or rendered-config changes. - Pluggable DNS provider interface (Manual + Cloudflare). Per-account credentials are Fernet-encrypted at rest, verified on save, and never returned by the API or written to logs/events/error_detail. - Non-blocking per-cycle orchestrator: publish (CAS) -> propagation grace (across cycles, no in-loop sleep) -> respond -> finalize/download, with a bounded fresh-order retry chain (1 original + 3 retries) on propagation lag. - Manual flow: user publishes the TXT record and confirms; manual DNS-01 cannot auto-renew unattended (auto-renew forced off and surfaced in the UI). - Migration v8: additive, idempotent columns on letsencrypt_accounts/orders and acme_challenges, plus a new letsencrypt_account_dns_credentials table. - Challenge-type-aware diagnostics (port80/routing/DNS checks skipped for DNS-01) and a DNS-01 event timeline in the order detail. - Frontend: DNS-01 account + credentials management, cert wizard adaptation, order-detail TXT records + verify, orders/renewal Method columns, and a Settings kill-switch. README, release notes, and API docs updated. Implements #35. |
||
|
taylanbakircioglu
|
a1192e602d |
feat: HA / VIP (Keepalived) management from the UI (#27)
Manage highly-available virtual IPs backed by Keepalived (VRRP) directly from the OpenManager
UI — no more SSHing into nodes to install/configure Keepalived by hand. Builds on the agent
pull-architecture: define the VIP centrally, click Apply, and the agents converge.
Highlights:
- New "HA / VIP" tab: create a virtual IP, pick a per-node interface, select which pool nodes
participate (MASTER/BACKUP roles + priorities); live MASTER/BACKUP/FAULT per node.
- On Apply, agents install & configure Keepalived (unicast VRRP, cloud-safe default) across the
major distros (Debian/Ubuntu, RHEL/CentOS/Alma/Rocky, Fedora, SUSE/openSUSE, Alpine) with a
HAProxy health-check, so the VIP fails over automatically when HAProxy drops.
- Single-node (a managed floating IP without failover) and multi-node VRRP failover both work.
- VIP changes ride the standard Apply Management flow with the standard "View Change" diff.
- Approval-gated deletion (safety): deleting a running VIP is staged for approval and the VIP
keeps running, untouched, until you approve it — an agent never tears a VIP down without an
explicit human approval. Per-VIP Diagnostics view; opt-in package uninstall (only on nodes
where OpenManager installed it). A node already running a hand-managed Keepalived is detected
and never overwritten ("externally managed").
- Fully opt-in and backward compatible: nodes/clusters without a VIP are unaffected. Adds
vip_instances + vip_members tables (idempotent SCHEMA_VERSION bump; existing data and
passwords unaffected) and a `vip` RBAC permission group.
- Also includes a HAProxy config-generator robustness fix: auto-inject a stick-table when a
frontend uses a stick counter (track-sc / sc_*_rate) but declares none.
On-prem / L2 (VRRP) scope; the UI notes the cloud caveat.
|
||
|
taylanbakircioglu
|
bd6a31cb0d |
feat: v1.6.0 — Multi-Factor Authentication (Issue #18)
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.
Highlights
==========
* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
`MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
`XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
+ `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
Users page; admins reset (single user or bulk) but never enable on
behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.
Security hardening
==================
* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
(prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
audit log so request bodies (TOTP / backup codes in flight) never
appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
XFF strategy so a single ingress address cannot exhaust the bucket
for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
`MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
with the same username can no longer occlude the active user
(also closes a small account-enumeration side channel).
Database
========
Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):
- users: mfa_enabled, mfa_method, mfa_secret_encrypted,
mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
- mfa_backup_codes (user_id ON DELETE CASCADE)
- mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
attempts, expires_at)
- mfa_pending_enrollments (user_id ON DELETE CASCADE)
Frontend
========
* Login page becomes a 3-phase state machine
(credentials → MFA → submitting); legacy single-step login is
preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
Admins viewing other users with MFA off see a non-actionable info
icon explaining that only the user themselves can enable MFA.
Deployment
==========
* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
a placeholder; `SECRET_KEY` is also placeholder-ized so both are
injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
uses `window.location.host` for `/api/*` and is routed by the
existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
into the production bundle.
Tests
=====
* New unit suites:
- `test_mfa_service.py` (TOTP, encryption, backup codes)
- `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
- `test_mfa_rate_limits.py` (env override + dataclass immutability)
- `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.
Documentation
=============
* README MFA section (overview, day-to-day operations, emergency
reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.
Issue: #18
|
||
|
taylanbakircioglu
|
d7208528f7 |
fix: v1.5.2 — ACME Diagnostics Panel Hardening + AGPL-3.0 relicense (Bulgu #94/#95/#96)
A focused hardening pass on the v1.5.0 ACME Diagnostic Panel
surface, exercised against a live production deployment (Round-25
+ Round-26 audits) and supplemented by an AGPL-3.0 relicense.
------------------------------------------------------------------
LICENSE — Relicense to AGPL-3.0-or-later
------------------------------------------------------------------
Effective v1.5.2 the project is licensed under the **GNU Affero
General Public License v3.0 (or later)**. v1.5.0 and v1.5.1
remain under the prior MIT terms.
The relicense is consistent with the project's intent as a
community-operated HAProxy management surface: forks that run
HAProxy OpenManager as a network service for third parties are
now required to publish their modifications under the same
license (AGPL §13). Day-to-day single-tenant deployments,
internal corporate use, and ordinary forks-for-fixes are
unaffected.
Changes:
* LICENSE replaced with full AGPL-3.0 text.
* README "## License" section rewritten with the AGPL summary
+ the network-service obligation.
* frontend/package.json gains `"license": "AGPL-3.0-or-later"`.
------------------------------------------------------------------
BULGU #94 / #95 — Diagnostic Panel Must Never Opaque-500
------------------------------------------------------------------
Live exercise of the v1.5.0 Diagnostic Panel against a deployed
build surfaced two opaque-500 paths. The panel exists to make
ACME failures legible; producing an opaque HTTP 500 defeats the
entire feature. Fix shape: every endpoint now returns either a
canonical 4xx (auth / not-found / rate-limit) or an HTTP 200
"structured failure envelope" that the React UI knows how to
render — never a 500 for an in-suite failure.
Affected paths:
POST /api/letsencrypt/orders/{order_id}/diagnostics
Pre-fix: a UndefinedColumnError or DB-connectivity failure
inside `run_checks` bubbled out of the bare try/finally and
surfaced as a generic 500 with no operator-actionable detail.
Post-fix: setup-stage and run-stage failures are caught
separately and converted to a `status: diagnostics_unavailable`
envelope carrying `error_stage`, `error_type`, `error_message`,
and a `correlation_id` that the operator can grep in the
backend log. Individual checks are wrapped in `_safe_check`
so one broken check (e.g. DNS lookup timeout) never crashes
the suite — the failing check shows up as `status: "fail"`
with its message, the others still run.
GET /api/letsencrypt/orders/{order_id}/events
Pre-fix: the SQL `SELECT … status FROM user_activity_logs`
referenced a column that did not exist in the canonical
migration; every diagnostic-panel open against an order with
any user-activity-log correlation got an `UndefinedColumnError`
500. Post-fix: the endpoint now introspects
`information_schema.columns` and projects only the columns
actually present. Partial failures (one source dies, the
other works) are reported via `meta.errors[]` rather than
collapsing the whole timeline.
POST /api/letsencrypt/orders/{order_id}/diagnostics/{check_id}/rerun
Same structured-envelope contract as the full-suite POST,
scoped to a single check row.
Frontend (`frontend/src/components/ACMEAutomation.js`):
* Distinct `diagRunError` / `diagEventsError` / `diagMeta`
states so the modal can render the cause inline (Antd Alert)
instead of a silent dropdown.
* Event-log auto-tail polling backs off after 3 consecutive
failures so the Network tab does not get spammed with 500s
every 5s.
* Correlation IDs visible in every error banner.
------------------------------------------------------------------
BULGU #96 — Clean 404 for Out-Of-Range order_id
------------------------------------------------------------------
A live exercise of the post-#94 diagnostic panel against the
deployed build surfaced one remaining contract gap. A path-
param `order_id` outside the Postgres int4 range
(e.g. > 2_147_483_647) caused `_load_order` to raise
`asyncpg.exceptions.DataError: invalid input for query
argument $1: 2147483648 (value out of int32 range)`. Round-25
correctly surfaced this in a `diagnostics_unavailable`
envelope — but that envelope leaked SQL implementation detail
("query argument $1", "int32 range", DataError class name)
into the operator-facing response body.
Semantically an out-of-range integer can never reference a
real order — it's just "not found". `_load_order` now catches
`asyncpg.exceptions.DataError` and re-raises a canonical
`HTTPException(404, "Order {id} not found")`. Because all
three endpoints re-raise `HTTPException` from their outer
try/except (the Round-25 envelope only fires for non-
HTTPException crashes), the canonical 404 path now wins
end-to-end across /diagnostics, /events, and /rerun.
------------------------------------------------------------------
TEST / LINT / LIVE VERIFICATION
------------------------------------------------------------------
* Backend pytest 1104/1104 (the +20 vs v1.5.1's 1084 are the
Round-25 and #96 contract pins; see
test_acme_diagnostics_router_round25.py).
* Live prod-canary verification: every endpoint return shape
confirmed against the deployed build — int4 overflow returns
clean 404 with no SQL leak, normal paths return Round-25
envelopes, HTTP method matrix returns 405 on wrong verbs,
no auth returns 401, invalid `check_id` returns 400, and
`meta.correlation_id` is present on every diagnostic
response.
------------------------------------------------------------------
COMPATIBILITY
------------------------------------------------------------------
* No breaking API contract changes: `status` field on the
diagnostic response can now be `"diagnostics_unavailable"`
in addition to the existing pass-through of the
underlying order status (`pending` / `valid` / `invalid` /
`cancelled` / …) — older UIs that only switch on the
existing values render the `diagnostics_unavailable`
case as "unknown status" rather than crashing.
* Frontend handles the new envelope shape AND the legacy
HTTP 4xx/5xx paths.
|
||
|
taylanbakircioglu
|
2e7db4d99f |
fix: v1.5.1 — Round-23 + Round-24 audit follow-ups (Bulgu #83 → #93)
A live-deployment audit pass over the v1.5.0 Site Wizard + ACME Diagnostic Panel surface. Two adversarial review rounds (R23, R24) each capped by an end-to-end smoke test against a multi-cluster staging deployment. Bulgu #83 — Frontend Management page warned about stale data without a clear retry CTA. The toast now carries an in-place "Reload" action and the page-level Empty state surfaces the same recovery affordance, so operators never get stuck on a stale-data view without an obvious way out. Bulgu #84 — ACME diagnostics ran with the wrong "last_heartbeat" column reference against the agents table. Aligned the SELECT with the actual schema column (`last_seen`); pinned by an idempotent regression test in `test_acme_diagnostics.py`. Bulgu #85 — ACME order error_detail rendering could leak the raw asyncpg/SQL exception class name when humanize_error_detail encountered an unhandled CA response shape. Added a backwards- compatible fallback branch that emits an "ACME error (raw)" panel without exposing parse_error class name to the user. Bulgu #86 — Multi-cluster apply with concurrent rejects could leave wizard_staged orders dangling without their parent draft. Pinned via reject_order_with_cluster_orphan test. Bulgu #87 — Frontend Management page list virtualization mis-keyed during a re-sort + stale-row replace race; fixed by keying rows on `id + version` so React reconciler does not reuse DOM for a logically different row. Bulgu #88 — Site Wizard "Cancel" mid-flow now surfaces an unsaved-draft prompt with explicit Save / Discard buttons (and the same prompt on browser tab close), so the operator never loses 5 steps of input to an accidental ESC. Bulgu #89 — Existing-cert SSL mode showed an empty dropdown when the cluster had >100 certs because the listing endpoint default-limited results. Endpoint now exposes pagination AND the wizard switches to client-side filtering above 50 rows. Bulgu #90 — ACME pre-check on the wizard preview path did NOT re-validate the account against `letsencrypt_accounts` if the operator stepped Back/Forward between SSL and Review. Added a debounced re-validation on Review entry. Bulgu #93 — Site Wizard hsts_enabled toggle in HTTPS frontend was idempotent-by-name (the generated `http-response set-header Strict-Transport-Security` line could duplicate across a Save + Apply cycle). The renderer now upserts the header in place. Cumulative outcome: backend pytest 1084/1084, frontend lint clean, and a 6-hour live-deployment smoke session against staging with no regressions reported. |
||
|
taylanbakircioglu
|
02b1cb2bca |
feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13)
Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit. |
||
|
taylanbakircioglu
|
07942a82e8 |
feat: ACME stability & enterprise audit (v1.4.0) — fixes #10 #11 #12
Issue #10 — Silent timezone failure on ACME certificate save - Normalize tz-aware expiry_date to UTC tz-naive before INSERT/UPDATE in ssl_certificates (TIMESTAMP WITHOUT TIME ZONE) — restores ACME download path Issue #11 — Duplicate _acme_challenge_backend in generated config - Generator guard: skip auto-append when backend already rendered - Agent-sync filter: _should_sync_backend() drops system-managed backends and detaches their server rows to prevent orphans - Restore filter: IGNORED_BACKENDS skips reserved names during cluster restore - Parser warning: reserved_backend_names blocks accidental manual import - Cleanup migration: removes orphan rows + cascading server entries (idempotent) Issue #12 — Validated orders required manual completion - New 60s background task complete_pending_acme_orders, flag-independent, multi-replica safe via FOR UPDATE SKIP LOCKED + 30s updated_at watermark - Per-order pg_advisory_lock(0x41434D45, order_id) serializes UI-Complete and auto-task races; idempotency guard returns existing certificate cleanly - retry_order endpoint reports in_progress: true within 30s window so the UI surfaces an info toast instead of duplicating CA requests - ACMEAutomation surfaces stuck orders (status=valid && !ssl_certificate_id) with a one-click Complete action and Cancel fallback; conditional 30s polling Other hardening - ACME state machine error_detail persisted as structured JSON across challenge, finalize, download stages for actionable post-mortems - CertificateRequest Pydantic model: domain regex + min_length/max_length and cluster_ids defaulting to all ACME-enabled clusters when "global" is selected - Renewal cluster fallback now requires acme_enabled=TRUE in addition to active - _complete_certificate preserves manual cluster assignments on renewal, surfaces cluster_errors, raises explicit error on missing private key - Audit logging covers acme_certificate_requested/revoked, ca_chain_imported, account created/deactivated/purged, order retried/cancelled - Settings UI exposes acme.staging_url_override for private test CAs (Pebble) - Schema additions: acme_challenges.attempts (default 0) and last_attempt_at, index idx_letsencrypt_orders_status_updated; all migrations idempotent Tests (41/41 passing) - test_acme_expiry_normalize, test_acme_duplicate_backend, test_acme_state_machine, test_acme_pydantic_validation, test_acme_audit_logging, test_acme_concurrency CI / packaging - docker-build.yml reads version.json and pushes additional product-version tag (e.g. 1.4.0) alongside latest and timestamp build id Closes #10 Closes #11 Closes #12 |
||
|
taylanbakircioglu
|
deb784bb65 |
fix: ACME certificate issuance improvements, null-safe hardening, guided setup UX, and order list enhancements (Issue #9)
- Fix critical cascading NULL status bug in ACME challenge flow that could cause 404s - Add defense-in-depth NULL handling across all ACME service methods - Add new GET /api/letsencrypt/prerequisites endpoint for configuration checks - Add interactive ACME Setup Guide with step-by-step navigation links - Add URL-based tab navigation in Settings and SSL Management pages - Harden retry flow: return clear 409 errors for invalid/cancelled orders - Allow cancellation of invalid orders (backend + frontend) - Improve error message extraction with consistent getErrorMsg helper - Add status filter tabs (Active/Completed/Failed/All) for order list - Add visual dimming for cancelled/invalid orders - Add enhanced pagination with size changer and total count - Add comprehensive diagnostic logging with ACME: prefix - Update README with ACME architecture docs, quick start guide, and troubleshooting Resolves #9 Made-with: Cursor |
||
|
taylanbakircioglu
|
93d7ad8fdb |
feat: add ACME Auto SSL with Let's Encrypt integration (v1.1.0)
Add automated SSL certificate management via ACME protocol (RFC 8555): - Full ACME client implementation (account registration, HTTP-01 challenges, certificate issuance/renewal) - Configurable ACME providers (Let's Encrypt, ZeroSSL, custom CA) via Settings UI - Auto-renewal scheduler with PENDING -> Apply -> APPLIED flow alignment - ACME account management (register, deactivate) from UI - Certificate request wizard with domain validation and cluster targeting - Zero changes to HAProxy agent scripts - challenges routed through existing architecture - Comprehensive security hardening (no private key exposure in API responses) - Full backward compatibility with existing SSL, Apply, Rollback, and Restore workflows - Updated README, API documentation, and Kubernetes deployment notes - Version management embedded in code (v1.1.0) - UI messaging improvements for agent-pull architecture accuracy Made-with: Cursor |
||
|
taylanbakircioglu
|
1dcf45b1bb |
fix: dynamic collision detection in config generation
Query agent preserved_listen_blocks dynamically and skip conflicting entities to prevent HAProxy validation errors. Falls back to static reserved list if no agent data available. |
||
|
taylanbakircioglu
|
d40c701272 |
fix: resolve Python scoping error in config generation
Remove redundant local 'import json' statements that caused: "cannot access local variable 'json' where it is not associated with a value" Root cause: Local import inside try block made 'json' a local variable, but except clause referenced json.JSONDecodeError before assignment. Changes: - Line 197: Remove local import, use global json (line 5) - Line 514: Remove 'import json as _json', use global json - Use ValueError instead of json.JSONDecodeError (equivalent, JSONDecodeError is a subclass of ValueError) This was causing config generation to fail completely, returning error message as config content instead of actual HAProxy configuration. |
||
|
taylanbakircioglu
|
851377aedf |
feat: Add HAProxy proxy name collision prevention system
- Add preserved_listen_blocks column to agents table for storing agent's local listen block names - Implement reserved names check (stats, monitoring, admin, etc.) for frontend/backend creation - Add dynamic collision detection against agent's preserved listen blocks - Apply collision checks to CREATE, UPDATE endpoints and bulk import - Add debug mode for failed config validation (saves to /tmp/haproxy-failed-*.cfg) - Fix JSON character stripping for ACL and use_backend rules - Remove collision protection from agent scripts (now handled by backend) - All collision checks wrapped in try-except for backwards compatibility |
||
|
taylanbakircioglu
|
183b4fa9e2 |
fix: Auto-add 'verify none' for SSL backend servers without CA file
When ssl_enabled is true for a backend server but: - No ssl_verify option is explicitly set AND - No CA file (ssl_certificate_id) is specified HAProxy 2.8+ defaults to 'verify required' which fails without a CA. Now auto-adding 'verify none' in this case with a warning log. Users can override this in UI by setting SSL Verification explicitly. This fix is safe for: - Bulk Import: Parser already sets ssl_verify='none' when removing ca-file - Version Diff: Generated config correctly shows 'verify none' - Restore: DB values unchanged, config regeneration applies same fix Fixes: 'verify is enabled by default but no CA file specified' error |
||
|
Taylan Bakırcıoğlu
|
378710db3f |
fix(haproxy-config): ACL rules must come before http-request directives
CRITICAL HAProxy Validation Error Fix Problem: Generated HAProxy config failed validation with error: [ALERT] error detected while parsing an 'http-request deny' condition: no such ACL: 'waf_demo-rule1_path'. Root Cause: Config generator was writing http-request directives BEFORE ACL definitions. HAProxy requires ACLs to be defined before they are referenced. Generated Config (WRONG ORDER): http-request deny if waf_demo-rule1_path ❌ ACL not defined yet! acl waf_demo-rule1_path path_reg ^/admin ❌ Too late! Fix: Reordered frontend config generation: 1. ACL Rules (Line 339-362) - Define ACLs FIRST 2. HTTP Request Headers (Line 364-371) - Use ACLs AFTER Generated Config (CORRECT ORDER): acl waf_demo-rule1_path path_reg ^/admin ✅ Define first acl waf_demo-rule1_method method POST ✅ Define first http-request deny if waf_demo-rule1_path waf_demo-rule1_method ✅ Use after Impact: - Parsing logic: UNCHANGED (no breaking changes) - Database storage: UNCHANGED (no schema changes) - Config generation: FIXED (correct HAProxy syntax) - Bulk import: Works correctly now - Agent config apply: Validation passes Testing: 1. Bulk import config with ACLs and http-request rules 2. Agent applies config successfully 3. HAProxy validation passes Files Changed: - backend/services/haproxy_config.py: Reordered ACL and request_headers generation |
||
|
Taylan Bakırcıoğlu
|
8e1a93f873 |
fix(ssl): add backend server SSL advanced options to haproxy config generation
CRITICAL BUG FIX: Backend server SSL advanced options not in generated HAProxy config Problem: - Database has ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers columns ✅ - Config generation code tries to write them to HAProxy config (line 680-687) ✅ - BUT SELECT statement did NOT include these fields ❌ - Result: server.get('ssl_sni') always returned None Impact: - User sets 'TLS Min Version: TLSv1.2' for backend server in UI - Value saved to database correctly - BUT generated HAProxy config missing 'ssl-min-ver TLSv1.2' - Agent deploys incomplete config, SSL settings lost! Solution: - Added 4 server SSL advanced options to SELECT query (line 616): - ssl_sni (for SNI hostname) - ssl_min_ver (minimum TLS version) - ssl_max_ver (maximum TLS version) - ssl_ciphers (cipher suite override) Config Generation Logic: - Line 680-687: Code already writes these fields to HAProxy config - Line 616: Now SELECT actually retrieves the values from database - Example output: 'server backend1 10.0.0.1:443 ssl sni example.com ssl-min-ver TLSv1.2' Testing: - After deployment, bulk import a config with backend server SSL options - Check generated HAProxy config on agent - Should now see: 'server X ssl ssl-min-ver TLSv1.2 ciphers ...' Files Changed: - backend/services/haproxy_config.py: Line 616 SELECT statement |
||
|
Taylan Bakırcıoğlu
|
2dcdaeeba1 |
fix(config): robust use_backend_rules parsing with comprehensive type handling
PROBLEM: - Config generation was producing invalid syntax: use_backend ["..."] - HAProxy validation failing on agents - Old code had insufficient type checking for JSONB fields ROOT CAUSE: - Missing ELSE branch when use_backend_rules wasn't a list - No handling for unexpected types (tuple, Record, etc.) - No debug logging to track type issues FIX: - Added comprehensive type checking (str, list, tuple, other) - Added debug logging to track type and value - Added fallback parsing for string representation of lists - Enhanced validation to skip invalid entries - Prevents generation of invalid HAProxy syntax IMPACT: - Fixes validation failure for cluster 7 (demo-cluster) - Enables proper parsing of JSONB use_backend_rules - Backward compatible with legacy string format TESTED: - Database validation: use_backend_rules is proper JSONB array - String elements correctly extracted and written - Invalid types caught and logged |
||
|
Taylan Bakırcıoğlu
|
b212fb92bc |
Add SSL Advanced Options support (Backend) - Part 1
FEATURE: Complete SSL Advanced Options implementation for frontend and backend server SSL ✅ DATABASE: - Added SSL parameter columns to frontends table: * ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites * ssl_min_ver, ssl_max_ver, ssl_strict_sni - Added SSL parameter columns to backend_servers table: * ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers - Migration functions: add_ssl_advanced_options_to_frontends() and add_ssl_advanced_options_to_servers() ✅ MODELS: - FrontendConfig: Added 7 new SSL fields for bind parameters - ServerConfig: Added 4 new SSL fields for server parameters - AgentHeartbeat: Added system_info field (fixes HTTP 422 validation error) ✅ BULK IMPORT PARSER: - Parse alpn, npn, ciphers, ciphersuites, ssl-min-ver, ssl-max-ver, strict-sni from bind lines - Parse sni, ssl-min-ver, ssl-max-ver, ciphers from server lines - Store parsed values in frontend/server objects - User-friendly warnings about imported SSL parameters ✅ CONFIG GENERATOR: - Generate bind lines with SSL advanced options: 'bind :443 ssl crt file.pem alpn h2,http/1.1 ciphers ...' - Generate server lines with SSL advanced options: 'server s1 addr:port ssl sni hostname ssl-min-ver TLSv1.2' - Support both NEW MODE (multiple certs) and OLD MODE (single cert) USER IMPACT: - Bulk import now correctly parses SSL configs with alpn/npn/ciphers - SSL parameters preserved during import (not lost anymore) - Agent heartbeat fixed (no more offline agents) - Ready for UI implementation (next commit) EXAMPLE USAGE: Frontend: bind 0.0.0.0:8443 ssl crt cert1.pem crt cert2.pem alpn h2,http/1.1 Server: server s1 10.1.1.1:443 ssl verify required sni backend.example.com ssl-min-ver TLSv1.2 NEXT: Frontend UI components for editing these SSL options |
||
|
Taylan Bakırcıoğlu
|
f1a3826334 |
fix(backend): Allow backends without servers to be deployed
MINIMAL FIX: Enable backend-first workflow (add servers later)
Problem:
1. User creates backend 'deneme-sil' without servers
2. Config generator SKIPs backend (no servers = skip)
3. User adds server 'server1-sil'
4. Config shows: 'server server1-sil 1.1.1.1:1233' WITHOUT backend block
5. HAProxy validation FAILS (server without backend = syntax error)
User Requirement:
- Create backend first (without servers)
- Assign backend to frontend (default_backend)
- Add servers later
- Standard HAProxy workflow
Solution (MINIMAL - 2 small changes):
1. haproxy_config.py (line 453-454):
OLD: Skip backend if no servers (continue)
NEW: Write backend block anyway (remove continue)
Result:
backend deneme-sil
balance roundrobin
mode http
# (no servers yet - backend will show as DOWN)
Valid HAProxy syntax - check
2. cluster.py (line 1600-1607):
OLD: Only mark backends with servers as APPLIED
NEW: Mark ALL backends as APPLIED (servers optional)
Reason: ALL backends are now in config (even without servers)
Why This Is Better Than Previous Approach:
- Only 2 lines changed (vs 50+ lines)
- No complex logic added
- No risk to existing functionality
- HAProxy naturally handles backends without servers (shows as DOWN)
- Aligns with standard HAProxy usage patterns
Test Scenarios:
- Create backend without servers -> Backend block written to config
- Apply -> HAProxy accepts config (backend DOWN)
- Frontend can use backend (use_backend, default_backend)
- Add server later -> Server added to existing backend block
- Apply -> HAProxy accepts, backend goes UP
HAProxy Behavior:
- Backend without servers: DOWN (no available servers)
- Backend with disabled servers: DOWN (all servers disabled)
- Backend with active servers: UP (servers available)
Related: caa21f0 (has_pending_config fix)
Refs: #backend-workflow #server-optional #haproxy-syntax
|
||
|
Taylan Bakırcıoğlu
|
73ac554add |
fix(backend): Skip backends with no servers in HAProxy config generation
CRITICAL FIX: Backends without servers were causing HAProxy validation failures Problem: - Backend 'silbeni' (ID: 118) oluşturuldu ama server eklenmedi - Config generator backend'i haproxy.cfg'ye yazdı ama server satırı olmadan - HAProxy validation FAIL: 'backend has no servers' - Agent config'i apply etmedi - Frontend'de backend görünmedi (validation fail nedeniyle) Root Cause: - Config generator server olup olmadığını kontrol etmiyordu - HAProxy en az 1 server gerektirir, yoksa validation fail olur - Validation fail = agent apply etmez = backend haproxy.cfg'de görünmez Solution: - Backend loop başında server pre-check eklendi - Server yoksa backend config'e yazılmaz ve WARNING log'lanır - HAProxy validation her zaman başarılı olur (sadece valid backend'ler yazılır) Impact: ✅ Server olmayan backend'ler artık config'e yazılmayacak ✅ HAProxy validation artık fail olmayacak ✅ Agent successfully apply edecek ✅ Kullanıcı frontend'de backend'i görecek (0/0 servers ⚠️ Empty tag ile) ✅ Kullanıcı server ekledikten sonra Apply yapınca backend haproxy.cfg'ye yazılacak Testing: - Server olmayan backend oluştur → Config'e yazılmaz (log: SKIPPING) - Server ekle → Config'e yazılır - Apply → Başarılı Related: a39a5d6 (frontend null/undefined check) Refs: #backend-validation #haproxy-config-generator |
||
|
taylanbakircioglu
|
8595656803 |
fix: comprehensive validation for all single-value text fields
Phase 3 - Single Value Field Validation:
- Frontend.default_backend: Added validation to prevent '[]' causing ALERT
- Frontend.monitor_uri: Added validation for monitor endpoint
- Backend.health_check_uri: Added validation for health check path
- Backend.cookie_name: Added validation for cookie persistence
- Server.server_name: Added validation with fallback to server_id
- Server.server_address: Added validation (critical field, skip if invalid)
All single-value text fields now validate against:
- Empty strings
- '[]', '{}', 'null', 'None' invalid values
- Proper error logging and skipping
Additional improvements:
- Removed all emojis from log messages per user request
- Fixed server_address variable usage consistency
- Added proper error messages for debugging
Comprehensive Backend Audit Results:
- Checked all routers (frontend, backend, waf, ssl, config)
- Checked all models (Pydantic validation)
- Checked all services and utils
- Only one config generation file: haproxy_config.py (FULLY FIXED)
- Template files use static strings (no risk)
- Agent scripts use static templates (no risk)
Total fields validated: 30+ across all entity types
Risk level: ZERO - Complete protection against invalid values
|
||
|
taylanbakircioglu
|
617e303205 |
fix: additional comprehensive validation for remaining text fields
Phase 2 - Extended Field Validation:
- Frontend.options: Added validation for multiline option directives
- Server.ssl_verify: Added validation for SSL verify parameter
- WAF.redirect_url: Added validation for redirect URL (2 locations)
- WAF.header_name: Added validation with skip on invalid values
- WAF.header_value: Added validation with skip on invalid values
- WAF.path_pattern: Added validation for regex patterns (2 locations)
- WAF.http_method: Added validation for HTTP method filtering
All text fields now validate against invalid values:
- Empty strings, '[]', '{}', 'null', 'None' are skipped
- Warning comments added for debugging invalid WAF rules
- Zero risk of syntax errors in generated HAProxy config
Total fields validated: 24 across Frontend, Backend, Server, and WAF entities
Risk level: ZERO - All string concatenation points secured
|
||
|
taylanbakircioglu
|
be4ed94bc4 |
fix: comprehensive validation for all string fields in HAProxy config generation
- Added empty array/null validation for ALL text fields to prevent syntax errors
- Frontend: request_headers, response_headers, tcp_request_rules, acl_rules, use_backend_rules, redirect_rules
- Backend: options, request_headers, response_headers, cookie_options
- Server: cookie_value
- WAF: All 6 custom_condition usage points (IP filter, rate limit, header filter, request filter, geo block, custom rules)
- Prevents invalid syntax like 'redirect []', 'acl []', 'cookie []', 'http-request []'
- All string fields now skip '[]', '{}', 'null', 'None' values before config generation
- Critical fix for bulk imported configs with empty JSON array fields
|
||
|
taylanbakircioglu
|
21de3585cb |
fix: prevent invalid empty array syntax in HAProxy config generation
- Skip '[]', '{}', 'null', 'None' strings in redirect_rules, acl_rules, use_backend_rules
- Add validation for request_headers, response_headers, tcp_request_rules
- Prevents 'redirect []' syntax error that causes HAProxy validation failure
- Fixes: parsing [config:70] : error detected in frontend while parsing redirect rule (was '[]')
- All rules now properly filtered before being written to config
|
||
|
taylanbakircioglu
|
3e22776b8c |
fix: Improve options field UX and HAProxy config ordering
Fixed three critical issues with options field implementation: 1. Bulk Import Preview UI: - Added options field to frontend expandedRowRender display - Added options field to backend expandedRowRender display - Options now visible in preview before import confirmation 2. HAProxy Config Generator - Best Practice Ordering: Backend: - Moved options to position #2 (after mode/balance, before health checks) - New order: balance → mode → OPTIONS → httpchk → timeouts → cookie → headers Frontend: - Moved options to position #2 (after mode, before default_backend) - New order: bind → mode → OPTIONS → default_backend → timeouts → headers 3. UI Display Improvements: - Options now prominently displayed in bulk import preview - Better visual hierarchy with numbered comments in config generator - Consistent code style with proper whitespace handling Technical Details: - Frontend options placed after mode directive per HAProxy standards - Backend options placed before health checks for better readability - All options rendered as separate lines in preview - hasDetails check updated to include options field Files Modified: - backend/services/haproxy_config.py: Config generation order optimized - frontend/src/components/BulkConfigImport.js: Preview display enhanced |
||
|
taylanbakircioglu
|
0fc18fde38 |
feat: Add HAProxy options support for backends and frontends
Implemented comprehensive HAProxy options field support for both backend and frontend entities to enable standard HAProxy directives like 'option http-keep-alive', 'option httplog', 'option forwardfor', etc. Changes: - Database: Added 'options' TEXT column to backends and frontends tables - Models: Added options field to BackendConfig, BackendConfigUpdate, and FrontendConfig - API Endpoints: Updated CREATE, UPDATE, and GET endpoints to handle options field * Backend: CREATE/UPDATE/GET with options support * Frontend: CREATE/UPDATE/GET with options support (fixed 5 SELECT queries) - Config Generator: Added options block generation for both backends and frontends - Bulk Import Parser: * Added options field to ParsedBackend and ParsedFrontend dataclasses * Implemented option directive parsing with validation * Added unknown option warnings * Fixed bulk parse response to include options field - Bulk Import Merge: Added options field comparison in UPDATE logic - UI Components: * BackendServers.js: Added options TextArea form field * FrontendManagement.js: Added options TextArea form field Features: - Multi-line options support (newline-separated format) - Option validation with known HAProxy options list - Backward compatible (NULL options for existing entities) - Bulk import support with merge strategy - Full CRUD support for both manual and bulk operations Technical Details: - Format: Newline-separated TEXT field for multiple options - Validation: Warns about unknown options but allows them - Config Generation: Each option written as separate directive - Agent: Standard HAProxy config validation applies Total: 10 files modified, ~195 lines added, 26 integration points verified |
||
|
taylanbakircioglu
|
a5e281b284 |
Feature: Backend Server SSL certificate support + Frontend SSL dropdown enhancement
✨ Backend Server SSL Certificate - Complete Implementation: 1. Model Update (backend/models/backend.py): - Added ssl_certificate_id field to ServerConfig model - Allows selecting SSL certificate from dropdown 2. API Endpoints (backend/routers/backend.py): - CREATE server: Added ssl_certificate_id to INSERT query - UPDATE server: Added ssl_certificate_id to allowed_fields - GET servers: Added ssl_certificate_id to SELECT queries (2 places) 3. Config Generation (backend/services/haproxy_config.py): - SSL certificate lookup by ID - Auto-generate ca-file path: /etc/ssl/haproxy/{cert_name}.pem - Added to server line in HAProxy config Example Generated Config: Before: server es1 10.0.0.1:9200 ssl verify required After: server es1 10.0.0.1:9200 ssl verify required ca-file /etc/ssl/haproxy/star-burgan-com-tr.pem ✨ Frontend SSL Dropdown Enhancement: - Added Global/Cluster-specific tags to Frontend SSL dropdown - Matches Backend Server SSL dropdown design - Shows: [🌍 Global] or [📍 Cluster] with color coding 🔧 Complete SSL Workflow: 1. User edits Backend Server 2. Enables SSL 3. Selects SSL certificate from dropdown 4. Saves → ssl_certificate_id stored in DB 5. Apply Changes → Config generated with ca-file path 6. Agent downloads SSL cert to /etc/ssl/haproxy/ 7. HAProxy uses ca-file for SSL verification ✅ Database Schema: backend_servers table now includes: - ssl_enabled (bool) - ssl_verify (str: none/required) - ssl_certificate_id (int, FK to ssl_certificates) ✅ HAProxy Config Format: server {name} {addr}:{port} ssl verify required ca-file {path} Impact: Backend Server SSL now fully functional with certificate management |
||
|
taylanbakircioglu
|
1158e5f2b1 |
Fix: HAProxy validation - ACL and use_backend parsing improvements
🐛 Critical Bug Fixes: - Fixed duplicate 'acl' prefix in generated config (was: 'acl acl Name ...') - Fixed duplicate 'use_backend' prefix in generated config - Added use_backend directive parsing from bulk import configs - Fixed redirect_rules list handling (was causing .strip() error) 🔧 Parser Improvements: - Added use_backend rules parsing (stored as list like ACL rules) - Changed use_backend_rules field from str to list for consistency - Parser now captures all use_backend directives with conditions 🎯 Config Generation Improvements: - Smart prefix detection: only add 'acl' if not already present - Smart prefix detection: only add 'use_backend' if not already present - Support both legacy (string) and new (list) format for rules - Proper JSON parsing with fallback to newline-separated format ✅ HAProxy Validation: - Generated config now passes HAProxy validation (haproxy -c -f) - ACL and use_backend directives in correct HAProxy format - Routing rules properly linked with ACL conditions Example parsed config: acl Elasticsearch hdr(host) -i baremetal-elastic.burgan.com.tr use_backend Elasticsearch if Elasticsearch Tested with full config including multiple ACLs and routing rules. |
||
|
taylanbakircioglu
|
6aae0f4309 | Initial commit |