Five confirmed findings from an adversarial review of the branch, four of them
regressions introduced by it.
Fall through to the next source when a stored URL cannot be resolved.
Stopping at the first non-empty candidate emitted a backend section with no
`server` line: the section exists so `haproxy -c` passes and Apply succeeds,
then every challenge request 503s from an empty backend with nothing to show
for it. Scheme-less values are common — the settings field was free text until
this branch — so this was reachable on real installs. Selection moved into
`select_acme_backend_source()` so it is testable and the skipped candidates are
logged rather than silently dropped.
Report a challenge backend with no server line. `extract_acme_backend_target`
returns None for that section, and the loopback filter skipped falsy targets,
so the case above would have been reported as "challenge route present in
applied config" — the new check confirming the very state it exists to catch.
Do not narrow the row set feeding the routing check's `fail` branch. Adding a
mode filter to the WHERE clause turned a tcp-only port-80 cluster from "ok"
into "fail", and the site wizard blocks submit on any failing check, so those
installs would have been locked on upgrade day. Mode is now examined in Python
and only downgrades to `warn`, using an expression that is character-for-
character the renderer's normalisation.
Match the agent's config selector. The applied-config lookup omitted
`is_active = TRUE`, so it could read a superseded row and report on a config
the nodes never received. Extraction now happens in SQL rather than pulling
whole configs — these run to hundreds of KB.
Select `acme_backend_url` when loading the existing cluster. It was absent, so
the entity snapshot recorded old_values as NULL unconditionally and rejecting
the pending version wiped the operator's per-cluster URL back to the global
loopback default — re-creating the exact failure this branch removes.
Also carry `acme_enabled` and `acme_backend_url` through cluster creation. The
create model declared neither and the INSERT wrote neither, so a cluster
created with ACME switched on came back switched off with no error shown.
Refuted and deliberately not changed: settings PUT re-validating a stored
loopback value (it validates only what is submitted), an apply-path connection
leak (the 422 propagates to a handler that closes it), and the modal discarding
backend rejection reasons (the envelope matches).
Correcting a wrong ACME challenge backend was impossible without a shell, and
even with one the correction did not reach the nodes.
The mint gate only fired when `acme_enabled` flipped. `acme_backend_url` was
written to the DB and minted nothing, so Apply answered "No pending changes to
apply" and the nodes kept the old address forever. It is now decided by
comparing the rendered `server _acme_mgmt` line against the active version —
the one line that answers "would the nodes talk to a different address?".
Comparing whole configs would flag every unrelated pending edit.
The field had no UI at all. Added to the cluster form with validation that
mirrors the backend rules, and keyed on `model_fields_set` so clearing it
reverts to the global setting — with a plain `is not None` test an empty box is
indistinguishable from "not submitted", so a value could never be removed.
Validation is asymmetric on purpose (utils/acme_backend_url):
- at the write boundary, reject what cannot express a reachable target —
including the two silent traps: a scheme-less value became `localhost`, and
an out-of-range port raised inside the generator and destroyed the config
- at render time, never reject. The shipped defaults are themselves loopback,
so refusing to render would make every acme_enabled cluster unappliable,
including for changes unrelated to ACME. Problems are logged and surfaced.
The port-less default stays 8080 rather than moving to HTTP's 80: the bundled
compose publishes nginx on 8080, so installs relying on it work today and the
first sign of breaking them would be the unattended renewal loop months later.
The omission is warned about instead.
RFC1918 is allowed and is usually the right answer here, and no DNS resolution
is performed — both deliberate departures from utils/ssrf_guard, whose policy
is the opposite of what this address needs. What the management host can
resolve says nothing about what the HAProxy node can reach.
Diagnostics stop reporting success on a dead path:
- check_port80 uses GET instead of HEAD and classifies the body. A proxy that
has lost its /.well-known/acme-challenge/ location serves its SPA with HTTP
200, which `status in (200, 404)` accepted as healthy. Warnings also surface
when other domains pass, which previously hid the most diagnostic outcome.
- check_routing filters `mode`, joins `acme_enabled` and reads the APPLIED
config instead of counting database rows, and reports a loopback target.
- every new condition is `warn`, never `fail`: the site wizard blocks submit on
any fail, so a new failing condition would lock every install on upgrade day.
Also: normalise `frontends.mode` once per frontend. It is nullable, and the
raw value was interpolated into `mode {}`, emitting a literal `mode None` that
HAProxy rejects — taking down the whole cluster config. The ACME gate and the
backend-mode check now read the same normalised value.
And stop hardcoding PUBLIC_URL / MANAGEMENT_BASE_URL in docker-compose, which
silently ignored the operator's .env and made the wrong default load-bearing.
Manage highly-available virtual IPs backed by Keepalived (VRRP) directly from the OpenManager
UI — no more SSHing into nodes to install/configure Keepalived by hand. Builds on the agent
pull-architecture: define the VIP centrally, click Apply, and the agents converge.
Highlights:
- New "HA / VIP" tab: create a virtual IP, pick a per-node interface, select which pool nodes
participate (MASTER/BACKUP roles + priorities); live MASTER/BACKUP/FAULT per node.
- On Apply, agents install & configure Keepalived (unicast VRRP, cloud-safe default) across the
major distros (Debian/Ubuntu, RHEL/CentOS/Alma/Rocky, Fedora, SUSE/openSUSE, Alpine) with a
HAProxy health-check, so the VIP fails over automatically when HAProxy drops.
- Single-node (a managed floating IP without failover) and multi-node VRRP failover both work.
- VIP changes ride the standard Apply Management flow with the standard "View Change" diff.
- Approval-gated deletion (safety): deleting a running VIP is staged for approval and the VIP
keeps running, untouched, until you approve it — an agent never tears a VIP down without an
explicit human approval. Per-VIP Diagnostics view; opt-in package uninstall (only on nodes
where OpenManager installed it). A node already running a hand-managed Keepalived is detected
and never overwritten ("externally managed").
- Fully opt-in and backward compatible: nodes/clusters without a VIP are unaffected. Adds
vip_instances + vip_members tables (idempotent SCHEMA_VERSION bump; existing data and
passwords unaffected) and a `vip` RBAC permission group.
- Also includes a HAProxy config-generator robustness fix: auto-inject a stick-table when a
frontend uses a stick counter (track-sc / sc_*_rate) but declares none.
On-prem / L2 (VRRP) scope; the UI notes the cloud caveat.
Mustafa ULUKAYA