mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-19 09:05:43 +00:00
v1.10.14
198 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 taylanbakircioglu
|
1d4e4286af |
fix(agent): keep acknowledging once converged, so a lost report self-heals (v1.10.14)
The deploy report is the server's only evidence that a member node applied its keepalived.conf, and it was sent on the write path alone. Once the rendered config was on disk the agent took the idempotency early return every cycle and never reported again, so a single lost report - a backend restart, a 5xx, a network blip - left the VIP reading SYNCING with an empty "Last ack" forever while the node was demonstrably running the right config. Nothing would ever reconcile the two; the only escape was to change the rendered config so the agent wrote it again, which means touching a live VIP to fix a display problem. The agent now re-asserts its state on that path too: one request per node per poll cycle (~2.5 min), nothing written, keepalived not reloaded. This gap dates from the original HA/VIP work rather than this release series; it only became visible when acknowledgements were dropped for an unrelated reason. A test pins that both daemon copies report BEFORE the early return, since placing it after would silently restore the old behaviour. Verified end to end on a real HA pair: discovery, instance-based adoption of both nodes, PENDING, Apply, agent pull, the validation gate, the hash-pinned takeover, the acknowledgement, and retirement of the one-shot authorisation. |
||
|
taylanbakircioglu
|
0eb587dfa8 |
fix: keepalived validation gate and deploy acknowledgements (v1.10.13)
Two agent-side fixes found while taking the VIP adoption flow through a real
HA pair, released together.
1. A VALID CONFIG WAS REJECTED BY ITS OWN WARNING (v1.10.12)
Before writing a rendered keepalived.conf the agent validates it with
keepalived -t and, on failure, keeps the running config and does not restart
keepalived. That fail-safe is right, but it treated ANY non-zero exit as
invalid - and keepalived's config-test exit code does not separate fatal from
benign. Measured on 2.2.8:
clean config ..................... 0
auth_pass longer than 8 chars .... 5 "Truncating auth_pass to 8 characters"
missing '}' ...................... 5 "There are 1 missing '}'s"
unknown keyword .................. 5 "Unknown keyword '...'"
script without script_security ... 6 "SECURITY VIOLATION ..."
Exit 5 covers both a harmless truncation and a broken file, so a VRRP password
over eight characters was enough to block every apply - including on a node
whose own running config emits the same warning and had been serving the VIP
for weeks. Accepting exit 5 would have accepted broken configs, so the gate now
judges the OUTPUT: known-benign messages are dropped and anything remaining
still fails. It fails CLOSED - an unrecognised message, or a non-zero exit with
no readable output at all, is fatal - and the filter is an allowlist, never a
denylist. The agent also reports what keepalived said, in its log and in the
status the HA/VIP page shows; discarding it left a correct refusal that nobody
could act on.
2. EVERY DEPLOY ACKNOWLEDGEMENT WAS DROPPED
The takeover-retirement clause on POST /agents/{name}/keepalived-status reused
one placeholder for both the assignment `last_deploy_hash=$n` and the
comparison inside its CASE. PostgreSQL types a placeholder per USE, so it came
out as text in one and character varying in the other and asyncpg rejected the
statement with AmbiguousParameterError. The whole UPDATE never ran, so no
member recorded an acknowledgement: VIPs sat at SYNCING with an empty Last ack
while the nodes were verifiably running the config, and teardown acks were lost
the same way. The hash now has its own placeholder, compared only against the
column.
Verified against real keepalived and a real PostgreSQL rather than by
inspection, including on busybox and bash 3.2, and a test asserts every $n in
those statements is bound exactly once.
|
||
|
taylanbakircioglu
|
9e64002f1c |
fix(vip): make the Adoptable tag name the real blocker (v1.10.11)
The tag and the disabled Adopt button were computed by two separate ladders and could disagree. Seen on a live pair: one node's keepalived.conf had an unbalanced brace, so it was excluded from the instance; its partner was then tagged "MASTER missing" - technically true, because the unreadable node's state MASTER had not been counted - while the actual reason (the peer cannot be taken over, so adopting would strand it) sat only in the button's tooltip. The label pointed the operator at the wrong node. groupState now makes ONE ordered decision and returns the label, its colour and the reason together, so the tag can never describe a different condition than the one disabling the button. A group held up by a node that references the same address but cannot be adopted with it reads "blocked by peer"; two MASTERs is distinguished from none. Display only: the endpoint's checks and refusals are untouched. Backend suite: 1366 passed, 152 skipped. Frontend build clean. |
||
|
taylanbakircioglu
|
4f24d5bdd9 |
fix(vip): list adoption blockers once per instance (v1.10.10)
Since the panel groups a VRRP instance into one row, the blocker lists of all its members are merged - and every node reports the SAME problems about the SAME shared config. A two-node pair therefore showed each issue twice, in the Adoptable tooltip and in the adopt dialog. Plain de-duplication does not collapse them because the two files report different line numbers for the same directive. mergeBlockers keys on the message with a leading "line N:" stripped and keeps the first occurrence, so each distinct problem appears once while the text the operator reads still carries a line reference. Display only: the endpoint already evaluated the combined set across every node, and what it accepts or refuses is unchanged. Backend suite: 1366 passed, 152 skipped. Frontend build clean. |
||
|
taylanbakircioglu
|
a87994e06a |
fix(vip): refuse adoption that strands a node or normalises a peer (v1.10.9)
Three findings from a second pass over the adoption flow, all of the same class: something real leaving the set silently. 1. STRANDING. _collect_instance_participants can only match a node it can READ, that is ENABLED, and that is in the SAME pool. Each of those is a door a genuine member of the VRRP group leaves through without a word, and the nodes that remain are rewritten while it keeps serving the same address from an unmanaged config. Found on a live pool: one node of a pair had an unclosed vrrp_instance block, so it parsed to nothing while its partner parsed cleanly. Rather than guard each door, ask the question directly: does any reported keepalived.conf mention THIS virtual address without being one of the nodes we are about to adopt? Refuses naming the node and the reason. Scoped on the address so an unrelated file elsewhere cannot block every adoption, and excluding nodes already under management (a standing VIP, or our ownership marker) because those are not stranded. 2. SILENT NORMALISATION. prefix_length, unicast/multicast mode, HAProxy tracking and the VRRP password are stored ONCE on the VIP and re-rendered onto EVERY member, so whichever node was clicked imposed its settings on the others. prefix_length is the sharpest: the design refuses to GUESS a netmask for a live VIP, and copying one node's netmask onto another is that same change wearing a different hat. All four must now agree, with both values named in the refusal. The VRRP secret is compared by decrypting each node's token - Fernet is non-deterministic, so ciphertexts cannot be compared - and a token that will not decrypt is an error rather than an assumed match. 3. THE TAKEOVER AUTHORISATION WAS NOT ONE-SHOT. takeover_expected_hash is the permission to overwrite a keepalived.conf that lacks our ownership marker. It was written at adoption and never cleared, so it stayed valid for that file content indefinitely: restoring the pre-adoption file would have been overwritten again with no fresh human approval. It is now retired when a member acknowledges our rendered config, gated on the acked hash matching applied_config_hash so a partial or failed deploy never drops it and leaves the VIP unable to converge. The panel applies the stranding rule too, so the Adopt button is disabled with the reason instead of letting the operator click into a 422. No schema change, no agent change, no API-shape break. Backend suite: 1366 passed, 152 skipped. Frontend build clean. |
||
|
taylanbakircioglu
|
7d95c737f0 |
fix(vip): adopt the whole VRRP instance, not one node (v1.10.8)
Four defects found while tracing the adoption flow end to end after v1.10.4
reached a live HA pair.
B3/B4 (one root, one fix). Adoption took only the node whose row was clicked:
- adopting the BACKUP alone produced a VIP that apply always rejects, because
apply requires exactly one MASTER member;
- adopting the MASTER alone left the peer unmanaged, and adopting it
afterwards hit the VRID-collision guard with 409, so a pair could never be
completed from the panel;
- on a UNICAST instance the single-member render dropped the unicast block
entirely (render_keepalived_conf emits it only when peer_ips is non-empty),
so keepalived fell back to multicast on the adopted node while its peer
stayed unicast. They stop seeing each other and BOTH claim the VIP.
Adoption now resolves the whole instance via _collect_instance_participants,
keyed on (virtual_router_id, virtual address) - the same key keepalived uses to
group nodes. Each participant becomes a member with the role, priority and
interface its own file declares, and its own one-shot takeover hash, so the
per-node overwrite guard is unchanged. It refuses, naming the reason, when the
group has other than one MASTER, when advert_int differs across nodes, when a
declared unicast peer is not among the nodes being adopted, or when a node is
already in a live VIP - the one-active-VIP-per-agent rule that create/update
enforce via _validate_members_against_pool and adoption never called.
B1. The Apply Management "View Change" regex matched vip-(create|update|delete)
only, so an adopt version fell through to the generic HAProxy diff and rendered
the cluster's whole haproxy.cfg as removed. Display-only, but alarming. A test
now asserts every action _stage_vip_version can stage is in that alternation.
B2. Rejecting an adoption hid the node from the panel permanently:
vip_discoveries.adopted_vip_id is write-once, a VIP is only ever soft-deleted so
the column's ON DELETE SET NULL never fires, and the agent does not re-report a
file whose hash has not changed. Rather than clearing the column on each path,
adoptability is derived from whether the linked VIP is still active, which
self-heals reject, undo-reject and approved teardown alike.
The panel now lists one row per instance instead of per node, and the adopt
dialog names every node that will be taken over. Blockers are aggregated across
all of them, matching what the endpoint checks.
No schema change, no agent change, no API-shape break: /api/vip/discoveries
gains a derived field and /api/vip/adopt keeps its request body.
Backend suite: 1359 passed, 152 skipped. Frontend build clean (no new lint
warnings in VIPManagement.js).
|
||
|
taylanbakircioglu
|
eda7f36c93 |
fix(vip): scope the HA/VIP page to the selected cluster (v1.10.7)
The page ignored the cluster picker in the header. Both the VIP table and the v1.10.4 "Unmanaged keepalived detected" panel queried the whole fleet, so on an install with more than one cluster the lists showed every cluster's nodes at once and did not change when the selection did - the panel looked stuck on one cluster's keepalived. - GET /api/vip/discoveries takes the same optional cluster_id the VIP list already took, mapped to the cluster's pool via haproxy_clusters exactly like list_vips does. Omitting it still returns the whole fleet, so no existing caller changes behaviour. - VIPManagement reads selectedCluster from ClusterContext (it only took the cluster list before) and sends cluster_id on both fetches. The fetch callbacks depend on the scope, so switching cluster refetches instead of showing stale rows. - tests/test_vip_discoveries_scope.py pins both defects this endpoint has had: that the route reaches its own handler rather than being parsed as a vip_id (asserting != 422 specifically, since the repo's generic endpoint-auth tests accept 422 alongside 401/403 and therefore could not catch it), and that the cluster filter resolves cluster -> pool and short-circuits when absent. Behaviour change worth calling out: the VIP table is now scoped to the selected cluster where it was fleet-wide before. The API still serves the fleet-wide view to any caller that omits cluster_id. Backend suite: 1345 passed, 152 skipped. Frontend production build clean. |
||
|
taylanbakircioglu
|
11e5bf57d9 |
fix(vip): make the adoption panel reachable again (v1.10.6)
Follow-up to #58. The "Unmanaged keepalived detected" panel it shipped never appeared on any deployment. GET /discoveries was declared after GET /{vip_id} in routers/vip.py, and FastAPI matches routes in declaration order, so every request for the discovery list was routed into get_vip, which takes vip_id: int and rejected "discoveries" with 422 before list_vip_discoveries ever ran. The failure was completely silent. The agents reported their configs correctly, the rows landed in vip_discoveries, and the HA/VIP page treats any non-OK response as "nothing to show" - so the feature was invisible with no error in any log. Confirmed against a real fleet: two discovery rows present in the database, one with a parsed candidate, and an empty panel. - move list_vip_discoveries above the /{vip_id} routes, with a comment stating the ordering requirement - add tests/test_router_path_shadowing.py: a static source scan that fails if any literal path in any router is declared after a parameterised route that would swallow it. The whole router tree is clean; the detector is itself tested against the pre-fix ordering so the guard cannot pass vacuously. POST /adopt is unaffected: no POST /{vip_id} route exists. Also carries the release metadata for #58 (v1.10.4) and #60 (v1.10.5), which are published together with this fix rather than as separate artifacts. No schema, API-shape, frontend or agent change. Data reported under the earlier code is not lost - existing rows show up as soon as this backend is deployed, with no agent action needed. |
||
|
taylanbakircioglu
|
a4c74f2a52 |
Merge pull request #60 from mustafaulukaya/fix/acme-http01-split-deployment
HTTP-01 challenge backend on split deployments |
||
|
taylanbakircioglu
|
a36dd87a74 |
fix(agent): port VIP adoption into the in-script daemon so self-upgraded agents get it
Found during an impact analysis of the agent-script change in PR #58. linux_install.sh contains TWO daemon implementations and which one runs depends on how the agent reached its current state: - Fresh install: the heredoc at lines 923-2646 is written to /usr/local/bin/haproxy-agent and systemd runs that file. - Self-upgrade: perform_agent_upgrade copies the downloaded INSTALLER script over that same path (`cp "$temp_script" "$current_script"` with current_script=/usr/local/bin/haproxy-agent). systemd then runs the installer with `daemon` + SKIP_TO_DAEMON=true, which takes the separate in-script daemon that lives after the heredoc. PR #58 added _kp_discover and the one-shot takeover only to the heredoc copy, so both were absent from the path that self-upgraded agents actually run — and self-upgrade is exactly the path the release notes tell operators to rely on ("nodes will pull the new script through the normal agent-upgrade path"). The feature would have worked on a freshly installed node and been silently inert on every upgraded one. The file's own banner warns about this ("check_agent_upgrade() - Multiple locations ... TIP: Search for function name to find all occurrences!"), and send_heartbeat / fetch_and_deploy_keepalived_config / get_haproxy_stats_csv are already maintained as parallel copies for the same reason. Verified empirically rather than by reading: the script was instrumented and run in a container exactly as systemd invokes it after an upgrade (SKIP_TO_DAEMON=true, `bash linux_install.sh daemon`), inspecting the live `declare -f fetch_and_deploy_keepalived_config`. before: DISCOVERY_YOK TAKEOVER_YOK after: DISCOVERY_VAR TAKEOVER_VAR ENDPOINT_VAR Both blocks are ported verbatim from the heredoc copy with indentation adjusted; no logic changed, so the guard semantics are identical in both paths — takeover still requires allow_takeover AND a non-empty expected hash AND a matching on-disk md5, and anything else falls through to the existing "externally managed — refusing to overwrite" branch. `bash -n` passes. Backend suite unchanged at 1263 passed. |
||
|
Mustafa ULUKAYA
|
9e6e4dd03b |
fix(acme): repair the diagnostics contract the GET probe broke
Running the suite properly — the previous rounds could not, pytest was not installed locally — surfaced four failures, all in the code this branch touches. main is clean at 1410 passed, so these were mine. Two were real defects, not stale expectations: check_port80 turned an unreadable body into a hard failure. The GET probe read the body outside any guard, so a connection reset mid-response, or a server that hangs after headers, made a healthy 404 fail. The body is EVIDENCE, not a precondition: when it cannot be read the check now falls back to the status-only semantics it has always had, and records body_class 'unread'. The stricter rule applies only when there is something to judge. check_routing raised KeyError on a row without `acme_enabled`. The column gates a warning, so a row shape lacking it should not take down the whole diagnostic. The rest were expectations that had to change, because the contract did: - The probe is a GET now, so the test doubles needed a body. `_FakeHEADResp` became `_FakeGETResp` with headers and a readable content stream. - "A port-80 frontend row exists" no longer means ok. That assertion is exactly the bug: it describes what the database wants, while the nodes run whatever was last applied — which is how this check reported success throughout an incident where the live config had no usable challenge route. New coverage for the branches that had none, including the case that started all of this: a proxy that has lost its /.well-known/acme-challenge/ location serves its SPA with HTTP 200, which the old `status in (200, 404)` rule accepted as healthy. Also the tcp-only cluster, which must warn rather than fail — SiteWizard blocks submit on any failing check, so failing there would lock those installs on upgrade day. Verified against real dependencies rather than a stub harness: the app boots with the challenge route registered; the field validator fires through pydantic on both cluster models, normalising whitespace and rejecting scheme-less and loopback values; `model_fields_set` really does distinguish an explicit null from an omitted key, which is what makes clearing the field work; the settings validator rejects both raw and jsonb-encoded bad values; and a legacy scheme-less value is skipped so the next source renders. Suite: 1477 passed on the branch, 1410 on main, 0 failed on either. |
||
|
Mustafa ULUKAYA
|
47cc79dcf7 |
fix(acme): address review findings on the challenge-backend hardening
Five confirmed findings from an adversarial review of the branch, four of them regressions introduced by it. Fall through to the next source when a stored URL cannot be resolved. Stopping at the first non-empty candidate emitted a backend section with no `server` line: the section exists so `haproxy -c` passes and Apply succeeds, then every challenge request 503s from an empty backend with nothing to show for it. Scheme-less values are common — the settings field was free text until this branch — so this was reachable on real installs. Selection moved into `select_acme_backend_source()` so it is testable and the skipped candidates are logged rather than silently dropped. Report a challenge backend with no server line. `extract_acme_backend_target` returns None for that section, and the loopback filter skipped falsy targets, so the case above would have been reported as "challenge route present in applied config" — the new check confirming the very state it exists to catch. Do not narrow the row set feeding the routing check's `fail` branch. Adding a mode filter to the WHERE clause turned a tcp-only port-80 cluster from "ok" into "fail", and the site wizard blocks submit on any failing check, so those installs would have been locked on upgrade day. Mode is now examined in Python and only downgrades to `warn`, using an expression that is character-for- character the renderer's normalisation. Match the agent's config selector. The applied-config lookup omitted `is_active = TRUE`, so it could read a superseded row and report on a config the nodes never received. Extraction now happens in SQL rather than pulling whole configs — these run to hundreds of KB. Select `acme_backend_url` when loading the existing cluster. It was absent, so the entity snapshot recorded old_values as NULL unconditionally and rejecting the pending version wiped the operator's per-cluster URL back to the global loopback default — re-creating the exact failure this branch removes. Also carry `acme_enabled` and `acme_backend_url` through cluster creation. The create model declared neither and the INSERT wrote neither, so a cluster created with ACME switched on came back switched off with no error shown. Refuted and deliberately not changed: settings PUT re-validating a stored loopback value (it validates only what is submitted), an apply-path connection leak (the 422 propagates to a handler that closes it), and the modal discarding backend rejection reasons (the envelope matches). |
||
|
Mustafa ULUKAYA
|
bb774141d4 |
fix(acme): make the challenge backend fixable from the panel
Correcting a wrong ACME challenge backend was impossible without a shell, and
even with one the correction did not reach the nodes.
The mint gate only fired when `acme_enabled` flipped. `acme_backend_url` was
written to the DB and minted nothing, so Apply answered "No pending changes to
apply" and the nodes kept the old address forever. It is now decided by
comparing the rendered `server _acme_mgmt` line against the active version —
the one line that answers "would the nodes talk to a different address?".
Comparing whole configs would flag every unrelated pending edit.
The field had no UI at all. Added to the cluster form with validation that
mirrors the backend rules, and keyed on `model_fields_set` so clearing it
reverts to the global setting — with a plain `is not None` test an empty box is
indistinguishable from "not submitted", so a value could never be removed.
Validation is asymmetric on purpose (utils/acme_backend_url):
- at the write boundary, reject what cannot express a reachable target —
including the two silent traps: a scheme-less value became `localhost`, and
an out-of-range port raised inside the generator and destroyed the config
- at render time, never reject. The shipped defaults are themselves loopback,
so refusing to render would make every acme_enabled cluster unappliable,
including for changes unrelated to ACME. Problems are logged and surfaced.
The port-less default stays 8080 rather than moving to HTTP's 80: the bundled
compose publishes nginx on 8080, so installs relying on it work today and the
first sign of breaking them would be the unattended renewal loop months later.
The omission is warned about instead.
RFC1918 is allowed and is usually the right answer here, and no DNS resolution
is performed — both deliberate departures from utils/ssrf_guard, whose policy
is the opposite of what this address needs. What the management host can
resolve says nothing about what the HAProxy node can reach.
Diagnostics stop reporting success on a dead path:
- check_port80 uses GET instead of HEAD and classifies the body. A proxy that
has lost its /.well-known/acme-challenge/ location serves its SPA with HTTP
200, which `status in (200, 404)` accepted as healthy. Warnings also surface
when other domains pass, which previously hid the most diagnostic outcome.
- check_routing filters `mode`, joins `acme_enabled` and reads the APPLIED
config instead of counting database rows, and reports a loopback target.
- every new condition is `warn`, never `fail`: the site wizard blocks submit on
any fail, so a new failing condition would lock every install on upgrade day.
Also: normalise `frontends.mode` once per frontend. It is nullable, and the
raw value was interpolated into `mode {}`, emitting a literal `mode None` that
HAProxy rejects — taking down the whole cluster config. The ACME gate and the
backend-mode check now read the same normalised value.
And stop hardcoding PUBLIC_URL / MANAGEMENT_BASE_URL in docker-compose, which
silently ignored the operator's .env and made the wrong default load-bearing.
|
||
|
Mustafa ULUKAYA
|
acfd32dd63 |
fix(acme): stop shipping config-generation failures as applied config
`generate_haproxy_config_for_cluster` reports failure by RETURNING a one-line
comment ("# Error generating configuration: ...") rather than raising. Nothing
in the backend checked for it, so the apply path hashed that comment, stored it
as an APPLIED config version and pushed it to every agent — silently replacing
a cluster's entire haproxy.cfg.
Any exception inside the generator triggers this. An out-of-range port in
`acme_backend_url` is enough: `urlparse('http://h:99999').port` raises
ValueError, the outer `except` swallows it, and the cluster loses its config.
- add `is_config_generation_error()` next to the sentinel definitions so call
sites stop matching the string by hand
- guard the apply path: refuse with 422 and leave the running config in force
- guard the ACME-toggle PENDING mint, and let HTTPException through the local
`except Exception`, which would otherwise report success while no pending
version exists
Also make the challenge backend diagnosable without shell access, since this
block previously emitted no log line at all:
- log the rendered `host:port` and WHICH source chose it (cluster override,
system setting, or the MANAGEMENT_BASE_URL fallback) under the greppable
`ACME-BACKEND` keyword
- warn when the rendered address is loopback: HAProxy resolves it on the node,
not on the management host, so it can only work on an all-in-one install —
and it is exactly what the shipped defaults produce
- log peer, X-Forwarded-For and Host on the challenge endpoint, which answers
"did the request arrive at all?" — the question that separates a wrong
backend address from a wrong response body
Refs the HTTP-01 investigation: a split deployment rendered
`server _acme_mgmt <mgmt>:8080` against a port with no listener, and every
existing check reported success.
|
||
|
mustafa.ulukaya
|
709817fec3 | chore(version): bump to 1.10.4 - adopt an existing keepalived VIP | ||
|
mustafa.ulukaya
|
822c441d34 |
test(vip): cover the adoption gate and the VRRP password masking
The gate decides whether an operator's working keepalived.conf gets replaced, so its rule is pinned directly: a supplied prefix resolves only the prefix blocker, accepting data loss resolves only the loss blocker, and setting both still cannot wave through an impossibility like an unknown VRID or an unsupported auth_type. The gate matches on substrings of the blocker prose the UI displays, which means a reworded message would silently stop being waivable. One test therefore feeds real parser output through it in both directions rather than hand-written strings, so the message and the rule are checked together. Also asserts that masking leaves no trace of a password containing spaces while keeping the rest of the config readable, using the router's own regex so the test breaks if it is ever loosened. |
||
|
mustafa.ulukaya
|
164841219a |
feat(agent): report an unmanaged keepalived.conf and honour a one-shot takeover
Two additions to the Linux agent, both inside the existing keepalived converge function so no new poll or timer is introduced. Discovery is strictly read-only: when the node has a keepalived.conf without OpenManager's ownership marker, the agent posts it so an existing VIP can be adopted from the UI. Nothing is written to the node. It is rate-limited by content - the md5 of the last report is cached next to the config, so a file that may carry the VRRP password is posted only when it actually changes rather than every cycle. Once we own the file there is nothing left to adopt, so the record is cleared exactly once. The content is JSON-encoded with `jq -Rs` so newlines survive verbatim and the hash the server pins the takeover to is the hash of what is really on disk. The ownership guard now has exactly one exception, and it does not weaken it. Previously any file without the marker was refused, which is what protects a hand-maintained setup - and also what would block adoption forever. The server authorises a single takeover of a specific file by sending the md5 the operator adopted from, and the agent overwrites only when the on-disk hash still matches. If the file changed in between, the agent refuses again and reports why, so an edit made after adoption wins over the stale adoption instead of being destroyed. The fallback latest Linux agent version moves 2.0.0 -> 2.1.0 so nodes pull the new script through the normal upgrade path. Discovery simply does not happen on a node that has not upgraded yet. |
||
|
mustafa.ulukaya
|
d92a7e9660 |
feat(vip): adopt a discovered keepalived instance into a managed VIP
GET /api/vip/discoveries lists what the agents found; POST /api/vip/adopt turns
one vrrp_instance into a managed VIP using the values from the node's own file
instead of retyping them. The VIP is created PENDING like any other, so nothing
reaches the node until it is applied from Apply Management.
Adoption replaces the operator's file with our render, so the gate is the
feature. Blockers fall into three kinds and only two are resolvable:
- a LOSS ("our renderer cannot reproduce this, so adopting would delete it")
can be accepted explicitly - that is an informed choice about a notify hook
or an LVS section;
- an UNKNOWN prefix length can be supplied, because picking a netmask for a
live VIP would change its routing;
- anything else is an IMPOSSIBILITY, not a loss: an absent virtual_router_id,
a fractional advert_int, an unsupported auth_type. No flag waves those
through.
That rule now lives in one place, remaining_blockers(), so the endpoint and the
UI cannot drift apart - and it is unit-testable, which matters because getting
it wrong destroys a working config.
Adoption keeps the VRRP identity it found: unlike create_vip, which allocates
the next free VRID, a VRID already used in the pool is a hard 409. Silently
renumbering would put the adopted node in a different VRRP domain from the
peers that still run the original config.
The member row records the reporting node's own role, priority and interface,
and carries the one-shot takeover hash. The response returns the instance's
unicast peers, because those nodes hold their own keepalived.conf and have to
be adopted or added as members before the render describes a complete group.
|
||
|
mustafa.ulukaya
|
7dfd31832a |
feat(vip): store the keepalived.conf an agent finds on a node
Ingest side of adopting an existing VIP. A node reports the keepalived.conf it found and does NOT own to a new endpoint, and the finding is kept in a new vip_discoveries table (one row per agent, since the file is per-node). Reporting is read-only on the node. The heartbeat cannot carry this: it has the VIP address and a best-effort MASTER/BACKUP, while rendering a node's config needs eleven fields, so the file itself has to be read and parsed server-side - parsing keepalived's block syntax in bash is not something to attempt on a production load balancer. Secrets are split at ingest. The reported content may contain the VRRP auth_pass, so the password is Fernet-encrypted into its own column through the same key path as vip_instances, and the stored copy of the file has it masked. Nothing readable through the API, the UI preview or a database dump carries it in cleartext, and the parse result is never logged. A file that does not parse records its error rather than failing the agent's poll loop, and a report of "the file is gone" clears the row so the UI stops offering a stale candidate. The keepalived-config delivery gains allow_takeover and takeover_expected_hash. The agent refuses to overwrite a keepalived.conf without our ownership marker, which is the guard that protects a hand-maintained setup - and is exactly the guard adoption has to pass. Rather than weaken it, an adopted VIP authorises exactly ONE takeover of exactly the file that was analysed by pinning its md5, so a config edited between adoption and Apply is still refused instead of being silently overwritten. SCHEMA_VERSION 10 -> 11 for the new table and two additive columns. Nothing existing is altered, but the bump re-seeds the four built-in roles, which the upgrade notes call out. |
||
|
mustafa.ulukaya
|
8ac567dfe0 |
feat(vip): parse an existing keepalived.conf so a VIP can be adopted
Groundwork for adopting a hand-maintained keepalived setup into HA/VIP management. The page is empty today because the flow is one-way: VIPs are declared in OpenManager and pushed to the node, and nothing reads what is already there. The heartbeat cannot drive adoption. It carries two keepalived facts - keepalive_state (MASTER/BACKUP, best-effort from logs) and keepalive_ip (the first address grepped out of virtual_ipaddress) - while render_keepalived_conf needs eleven: virtual_router_id, auth_pass, interface, priority, prefix_length, advert_int, unicast peers, track_haproxy, role, address and name. Guessing the rest is not a cosmetic risk: a wrong VRID puts the nodes in separate VRRP domains and a wrong auth_pass makes them reject each other, and either way both nodes claim the VIP. So the config itself has to be read. Extracting the fields is the easy half. Adoption REPLACES the operator's file with our render, so anything their file contains that the renderer cannot reproduce would be destroyed on takeover - a notify_master failover hook, an LVS virtual_server section, a sync group, a second address in one instance, a custom track_script. The parser therefore also returns everything it could not model, and build_adoption_candidate turns each entry into a blocker with the file's own line number. Values that are unknowable rather than unreproducible block too: a missing virtual_router_id, and a missing prefix length, because our renderer always writes an explicit prefix and picking one would silently change a live VIP's netmask. keepalived's own documented defaults (state BACKUP, priority 100, advert_int 1) are applied but reported in `defaulted`, so the UI can say which values were assumed rather than read. Handles the layout variation real files have: nested braces, `#` and `!` comments, blocks opened and closed on one line, quoted script paths containing spaces, and several vrrp_instance blocks in one file. A parse result carries auth_pass in cleartext, since that is the only way to re-render an identical config, so it must never be logged - noted on every function that returns one. Tests pin each blocker and the layout variants, and include the invariant that keeps the parser honest: a config the renderer itself produced must parse back with zero blockers, so adding a directive to render_keepalived_conf without teaching the parser fails the suite instead of making OpenManager's own output look unadoptable. Verified by mutation - eight deliberate weakenings of the safety checks are each caught by at least one test. No endpoint, no schema change and no agent change yet; nothing calls this. |
||
|
mustafa.ulukaya
|
dd7b7822bf | chore(version): bump to 1.10.3 - multi-account ACME wizard fix | ||
|
taylanbakircioglu
|
dbb9189f16 |
fix(ui): make Apply Management readable in dark mode (v1.10.2)
Three dark-mode defects reported on the Apply Management page, all the same
class of bug: light-mode colour literals hardcoded where theme tokens belong.
1. The "Pending Changes" box was painted background #fffbe6 with border
#ffe58f. In dark mode the text on top is light, so the version name,
timestamp and "View Change" link sat on a cream panel and were unreadable.
Measured contrast was 1.03:1; it is now 11.50:1 (secondary text 1.03:1 ->
7.02:1).
2. The added/removed rows in the View Change diff used #f6ffed/#52c41a and
#fff2f0/#ff4d4f, which stayed near-white inside the otherwise dark diff
panel. Now 5.49:1 (added) and 4.01:1 (removed), from 2.21:1 and 2.99:1.
3. The "Apply All Configuration Changes" confirm dialog came up white. This one
is not a colour literal: in Ant Design 5 the STATIC Modal.confirm / message /
notification APIs render into their own detached root and never see the app's
ConfigProvider, so they always fall back to the light algorithm. Registering
ConfigProvider.config({ holderRender }) once at the app root wraps that
detached root in the same ConfigProvider. Verified against the installed antd
5.29.3 source rather than assumed: config-provider/index.js sets
globalHolderRender, and modal/confirm.js wraps the dialog with it. This fixes
EVERY static dialog in the application — 12 components call Modal.confirm —
not just this page.
While in the file, six more instances of the same bug were fixed: the error
Alert border, the VIP pending-delete row, two ACME/pending version panels, the
applied-version panel and the agent-error recommendation box.
Light mode is byte-identical. Each token resolves under the default algorithm
to exactly the literal it replaced (colorWarningBg -> #fffbe6, colorSuccessBg ->
#f6ffed, colorErrorBg -> #fff2f0, colorInfoBg, colorErrorBorder, ...), so this
release can only change dark mode. Contrast was measured by resolving the real
design tokens under both algorithms and computing WCAG ratios, not by eye.
Note the diff rows were already low-contrast in LIGHT mode (2.21:1 and 2.99:1)
and remain so; that is the design system's own success/error pair and changing
it would alter the established light-mode appearance, so it is left alone.
holderRender is registered in an effect rather than during render, since
ConfigProvider.config() mutates antd module state; effects still run long
before a user can click anything that opens a static dialog.
Frontend only: no schema, no SCHEMA_VERSION bump, no API change, no environment
variable, zero agent impact. Backend suite unchanged at 1243 passed.
|
||
|
taylanbakircioglu
|
eee0a4716a |
feat(ssl): encrypt the pending CSR private key at rest (v1.10.1, closes #53)
Closes the follow-up filed during the v1.9.0 CSR review. The private key of a
PENDING CSR is now Fernet-encrypted in the database instead of being stored as
a raw PEM.
Why this key specifically: it is the one key in the system that sits idle. It
is generated at CSR creation, waits for an external CA to sign the request
(days to weeks), and is destroyed the moment the signed certificate is
imported. It is never transmitted to an agent and never leaves the server.
ssl_certificates.private_key_content and the ACME order keys are deliberately
NOT covered, because agents must receive those in plaintext on every poll, so
encrypting them at rest buys nothing without an end-to-end redesign.
Implementation follows the pattern already used for the VRRP secret, TOTP
secrets and DNS provider credentials: a new utils/csr_key_crypto.py with its
own CSR_ENCRYPTION_KEY env var and its own HKDF info string
("csr-private-key-v1"), so rotating one secret class never affects another.
No schema change and deliberately NO SCHEMA_VERSION bump: the Fernet token
replaces the PEM inside the existing ssl_csrs.private_key_pem TEXT column. A
bump would re-run the migration sequence and re-seed the four built-in roles to
their defaults, which is a needless side effect for a storage-format change.
Backward compatible with no data migration. Rows written before this release
hold a raw PEM and are still read unchanged; the discriminator is exact rather
than a heuristic, since a Fernet token is base64url and can never contain the
"-----BEGIN" marker. Legacy rows drain naturally because a CSR's key copy is
NULLed on import.
A key that cannot be decrypted (SECRET_KEY rotated while CSR_ENCRYPTION_KEY was
unset) now fails with an explicit "delete this CSR and create a new one" error.
Previously that situation would have surfaced as the far more confusing
"certificate does not match this CSR's private key".
Also documents all four per-purpose encryption keys in .env.template. Only
VIP_ENCRYPTION_KEY was listed; MFA_ENCRYPTION_KEY and
DNS_PROVIDER_ENCRYPTION_KEY had been missing since v1.6.0 and v1.8.0.
Verified before release, on a corporate pre-production environment and locally:
- Full backend suite 1234 -> 1243 passed (+9 new tests), 0 failed.
- Against a real Postgres: a CSR created through the API stores a Fernet token
with no PEM header in the column, and imports successfully.
- Full 1.10.0 -> 1.10.1 -> 1.10.0 drill on one database volume. The upgrade
logs "Schema already at version 10 (>= 10); skipping migration run", so no
migration executes and the built-in roles are not re-seeded. A CSR created on
1.10.0 with a plaintext key imports successfully after the upgrade, which is
the backward-compatibility guarantee proven against a real row rather than a
mock.
- rsa-2048, rsa-4096 and ecdsa-p384 all round-trip through create, encrypt,
decrypt and import.
- Key derivation is stable across processes: two independent containers sharing
SECRET_KEY decrypt each other's tokens (required for UVICORN_WORKERS > 1 and
multi-replica deployments), while a different SECRET_KEY yields None rather
than a wrong key or an exception.
- Downgrade behaviour was measured, not assumed: 1.10.0 cannot parse the token
and fails with HTTP 500 "key parse failed (encrypted?)" rather than pairing a
wrong key. The rollback note states the measured behaviour.
- No CSR endpoint returns the key in any form: list and detail responses
contain neither a PEM nor a Fernet token.
Not changed here, from the issue's "worth folding in" list: the create rate
limit is not a concurrency guard, create_csr holds a pooled connection across
RSA key generation, detail=str(e) echoes internal error text (a repo-wide
convention), and is_global skips cluster validation in both routers/ssl.py and
routers/csr.py. None are storage concerns and each is a separate change.
|
||
|
mustafa.ulukaya
|
6bf6d016f5 | chore(version): bump to 1.10.0 - GoDaddy DNS-01 provider | ||
|
mustafa.ulukaya
|
0a0226c758 |
test(dns): cover GoDaddy relative-name derivation, RRset merge and request handling
Twelve tests, no network and no database, in the existing pure-logic style. The merge helpers are covered directly (additive add, idempotent republish, tombstone filtering, remove-one-of-many, remove-the-last-value signalling DELETE), but helper math alone would stay green if the write path stopped using it, so add_txt_record and remove_txt_record are also driven against a recording stub: the assertions pin that a sibling value survives a publish, that an already-published value issues no write, that an unreadable read raises instead of replacing the set, that removing the last value emits DELETE and never an empty PUT, and that no call is ever aimed at a zone-wide path. _request is exercised through a fake response for the cases that only appear against the real API: an empty 204 body must not raise, a 3xx must not read as success (redirects are not followed), a transport failure mid-read must not be mistaken for an empty body, and each error status must produce a message naming what the operator has to fix. Also covers the auth header in both forms, that the sanitizer strips credentials from composed error text, that the module does not log at all, the credential-field schema against the upsert validator's own key and length rules, and the two-key encryption round trip. Verified by mutation: nine deliberate breakages of the provider - single-value PUT, empty PUT instead of DELETE, coercing an unreadable read to empty, treating 3xx as success, following redirects, swallowing transport errors, dropping the dot-segment guard, lowering the TTL below the API floor, and removing sanitization - are each caught by at least one test. |
||
|
mustafa.ulukaya
|
8e534ef170 |
feat(dns): add GoDaddy DNS-01 provider (API Key+Secret / PAT, additive RRset writes)
Registers a third pluggable DNS provider for ACME DNS-01 alongside Manual and Cloudflare. Credentials are an API Key + Secret pair; leaving the Secret blank sends the Key as a Personal Access Token (Bearer), which is the migration path as GoDaddy retires the sso-key scheme. GoDaddy's Domains API v1 has no per-value TXT write: PUT on a record set replaces every value at that name. A certificate covering example.com and *.example.com publishes two different TXT values at the same _acme-challenge.example.com, so add/remove are read-modify-write - read the current set, merge, put the whole list back - with empty-data tombstone rows filtered out (they are rejected on echo) and DELETE used for the last value, since PUT with an empty array is rejected. The zone-wide sibling endpoints (.../records/TXT and .../records) would wipe SPF/DKIM/DMARC and the whole zone respectively, so the record path is built in one place that refuses an empty or dot segment. An unreadable record-set read fails closed rather than being treated as an empty set, because the PUT that follows would otherwise destroy the coexisting values. Zone lookup walks name suffixes probing the records API rather than the domain listing, so zones delegated to GoDaddy nameservers resolve and accounts that are rejected from the domain-details endpoint still work. Credential and eligibility failures during the walk surface instead of being reported as "no managed domain". Provider errors are sanitized at the single point where GoDaddy-supplied text enters a message, since those strings are persisted to order events and shown in the UI. No new dependency, no schema change, no frontend change - the credential form is rendered from the provider schema. |
||
|
mustafa.ulukaya
|
69e12f7459 |
chore(version): bump to 1.9.0 - CSR creation
- backend/version.json + frontend package version to 1.9.0 - README: feature list entry, CSR workflow section, SSL CSR API reference, v1.9.0 release notes - UPGRADE_GUIDE: v1.9.0 section (additive ssl_csrs table, SCHEMA_VERSION 9 -> 10, no RBAC changes, zero agent impact, rollback note) |
||
|
mustafa.ulukaya
|
a6166d11b9 |
feat(ssl): add CSR generation and signed-certificate import (backend)
New /api/ssl/csrs endpoint group: generate a private key + CSR server-side
(RSA 2048/4096, ECDSA P-256/P-384; full subject + DNS SANs with wildcard
support), list/detail/delete CSRs, and import the CA-signed certificate.
- New ssl_csrs table (SCHEMA_VERSION 9 -> 10, additive + idempotent); the
migration re-raises on failure so a failed run is retried instead of being
stamped as applied.
- Import verifies the certificate against the stored key as a hard gate
(match=None is treated as an integrity error, not a lenient pass), rejects
malformed and expired certificates with 400, warns on SAN drift, and
creates a normal ssl_certificates row (source=csr, cluster_id=NULL,
last_config_status=PENDING) so it flows through the standard
Apply Management -> agent pull pipeline.
- Concurrency: FOR UPDATE row lock serialises double-import and
delete-during-import; a partial unique index reserves pending CSR names;
soft-deleted same-name certs are reactivated preserving the row id.
- Security: no CSR endpoint ever returns the private key (explicit column
lists, enforced by a static test); the key copy on the CSR row is NULLed
after import; ssl.create/read/delete permissions enforced on every
endpoint incl. reads; per-user rate limit on key generation, which runs
in a worker thread; csr_id and cluster_ids are int32-guarded.
- ssl_service: extract _prepare_cert_fields from create_cert_row (behaviour
unchanged, extraction tests untouched) and add stage_ssl_config_versions
reusing the exact ssl-{id}-create-{ts} version-name scheme.
- Tests: crypto round-trip for all four algorithms, model validation,
import-flow unit tests, endpoint auth/permission pinning, migration and
key-non-exposure static assertions.
|
||
|
taylanbakircioglu
|
0ebf6583ea |
chore(version): bump to 1.8.10 — security release (RCE/auth/SSRF advisories fixed)
Single-source version bump (backend/version.json) with frontend/package.json and package-lock kept in sync (test_version_consistency). Marks the release that ships the GHSA-7rhv / GHSA-3p5c / GHSA-3vh4 fixes. |
||
|
taylanbakircioglu
|
9e5185c458 |
fix(security): post-review hardening — agent-inventory regression, coverage gaps, SSRF newNonce
Follow-up to the RCE/missing-auth/SSRF remediation, from a thorough multi-lens
review (3 agents + a black-box audit of all 201 routes). Backend-only; no
agent-script changes.
Regression fix (introduced by the previous commit):
- GET /api/agents was made JWT-only, but deployed agents call it WITH X-API-Key
(not a JWT) to read their applied_config_version and avoid re-applying config on
restart. It now accepts EITHER a valid operator JWT OR a valid agent X-API-Key,
so agents no longer get 401 (which caused a spurious HAProxy reload every restart).
Completeness (GHSA-3p5c siblings the first pass missed — same data class, now JWT):
- dashboard.py: GET /api/haproxy-cluster-pools/{id}/agents (full agent inventory —
a direct anonymous bypass of the GET /api/agents lockdown), /api/pools,
/api/haproxy-cluster-pools, /api/dashboard/stats, /api/dashboard/overview
(auth was optional -> leaked stats/names/health/alerts anonymously),
/api/haproxy/stats.
- waf.py: GET /api/waf/rules. health.py: GET /api/health/errors.
- agent.py: GET /api/agents/generate-uninstall-script/{platform} (agent-management
endpoint; was anonymous) now requires JWT or agent key, like generate-install-script.
- config.py: POST /api/config/{validate,optimize,templates/{id}/generate} were
optional-auth (logging only) and run a HAProxy validator on caller input; now
require a JWT. (bulk-create, parse-bulk, diff and configuration/request were
already mandatory-auth — verified.)
All newly-gated endpoints are frontend-only (axios sends the JWT) or unused;
agents never call them.
SSRF (GHSA-3vh4) gap:
- acme_service._get_nonce fetched directory['newNonce'] (from the attacker-
influenceable directory JSON) with a bare session, http allowed, dual-stack, and
BEFORE the guarded _signed_request POST. Now guarded (assert_public_url +
safe_connector + no redirects + timeout), matching the other ACME sinks.
Correctness:
- Three agent webhooks (config-applied, config-validation-failed, config-sync)
swallowed their auth 401 into a 200 error body via a bare `except Exception`.
Added `except HTTPException: raise` so the 401/403 propagates.
Audit result (live black-box, all 201 routes probed unauthenticated): no data
leak and no unauthenticated mutation anywhere; every sensitive route returns
401/403 (a pre-existing group of read handlers wraps the 401 into a 500 via a
broad except — no data is exposed; left as-is, documented as cosmetic).
Verified: full pytest tests/ (1145 passed, 0 failed; +16 regression tests) + live
localtest stack smoke — agent-key GET /api/agents=200, anonymous=401, all newly
gated endpoints reject anonymous and admit JWT, the 3 webhooks return 401.
|
||
|
taylanbakircioglu
|
520b69a1c6 |
fix(security): remediate RCE, missing-auth and SSRF advisories (backend-only, no agent changes)
Addresses three reported advisories, all verified against the code. Fixes are
entirely server-side — deployed agents already send a valid X-API-Key on every
call, so enforcing it does not require any agent-script change or upgrade.
GHSA-7rhv-c5pc-69r8 (CRITICAL RCE — agent script-template poisoning):
- POST/GET /api/agents/script-templates/{platform} now require the agents.version
permission (was authentication-only), matching POST /versions. Blocks a viewer
JWT from overwriting the root install/upgrade script.
GHSA-3p5c-m5m4-mjpx (missing authentication):
- Agent data-plane endpoints now REQUIRE a valid X-API-Key (was optional/skipped
when the header was absent), checked before any DB access: config,
ssl-certificates (private keys!), upgrade-status, heartbeat (by-name and the
previously auth-less by-id), configuration pending-requests. Removes keyless
heartbeat spoofing and keyless rogue-agent auto-registration.
- Operator/UI endpoints now require a JWT: GET /api/agents, the entire
/api/dashboard-stats router, /api/health/{deep,agents,clusters}, and
/api/ssl/certificates/{id}/config-versions. The simple /api/health liveness
probe stays public. Adds shared auth_middleware.require_authenticated_user.
GHSA-3vh4-gvxx-wm2p (SSRF via ACME directory_url):
- New utils/ssrf_guard.py (https-only + public-IP-only, IPv4-pinned, no redirects),
applied to settings test-connection, acme_service.get_directory and
_signed_request, and validated at Let's Encrypt account creation. The
test-connection response no longer reflects arbitrary upstream JSON keys
(information-disclosure oracle) — only fixed ACME field names.
Verified: full pytest tests/ (1128 passed, 0 failed) + live localtest stack smoke
(valid JWT/key paths return 200/404 as expected; anonymous requests 401; SSRF to
metadata/private/loopback refused). No changes to backend/utils/agent_scripts/*.
|
||
|
taylanbakircioglu
|
c79391cd13 |
feat(acl): accept HAProxy -f pattern-file references with advisory warnings (v1.8.9, Issue #38)
The manual Frontend editor, wizard and visual ACL builder hard-rejected the ACL `-f <file>` flag while bulk import accepted it. Worse, a frontend imported with an `-f` ACL could not be edited at all (422) until the ACL was dropped. The original guard predated the fail-safe apply flow: the agent runs `haproxy -c` before every reload, so a missing pattern file is rejected safely and the previous config keeps running. Pattern files are operator-managed host files — the same policy adopted for SPOE filter configs in v1.8.8. - models: remove the 5 `-f` hard rejects (frontend acl/redirect/use_backend validators + wizard string/dict-redirect guards); `$(`/backtick and X!X contradiction guards unchanged - routers/frontend: `_pattern_file_warnings` helper; non-blocking warning on create + update responses listing referenced pattern files (empty when no rule uses `-f` — zero noise) - routers/config: bulk-import preview advisory listing pattern files per frontend (cluster config-dir aware, next to the SPOE advisories) - React: remove the FrontendManagement submit gate and SiteWizard step gate; ACLRuleBuilder renders informational notes instead of errors and re-adds `-f (pattern file on host)` to the flag dropdown; create path now renders server warnings like update - tests: 4 reject-pins inverted to accept-pins; new test_acl_pattern_file_allow.py (accept/guards-kept/zero-noise/advisory); full suite green (1094 passed) |
||
|
taylanbakircioglu
|
9e2ea04777 |
feat(haproxy): preserve SPOE filter + frontend log-format on import/edit (v1.8.8, Issue #38)
Bulk import / manual edit silently dropped `filter spoe engine ...` (Coraza WAF) and frontend `log-format` because the parser recognised only a fixed directive set. The regenerated config then missed the SPOE engine, so HAProxy failed with "unable to find SPOE engine 'coraza' used by the send-spoe-group". - parser: capture `filter` + `log-format`/`log-format-sd` into new ParsedFrontend fields - db: additive nullable `log_format` + `filters` TEXT columns on frontends (SCHEMA_VERSION 8->9) - generator: new `filter` bucket flushed before http-request rules so `filter` precedes `send-spoe-group`; `log-format` kept in prelude - bulk import: preview dict, change-detection, persist (create + merge-update); cluster-aware SPOE pre-flight advisories (missing-filter + host-prerequisite) surfaced in the UI - manual CRUD: full round-trip (get/create/update) incl. React form fields (no null-wipe) - reject/rollback: restore the new columns; restore path + wizard helper kept in parity - backend `option spop-check` recognised (suppresses spurious warning for coraza-spoa) - tests: test_spoe_filter_import.py; full suite green (1079 passed) |
||
|
taylanbakircioglu
|
97b2452bd2 |
fix(version): single-source version reporting to stop UI version drift (v1.8.7)
The version shown in the UI (backend-sourced via /api/version) could lag the
real release. The canonical version lived at repo-root version.json, but the
backend image is built with context ./backend, so that file did not reach the
container unless a pipeline staged it; the backend then fell back to a hardcoded
constant in main.py that had to be bumped by hand and had drifted (it reported
1.8.4 after 1.8.5 and 1.8.6 shipped).
- version.json moves to backend/version.json (single source of truth), now
committed and co-located with main.py, so COPY . . bakes it into every image
directly - correct version in every deployment, no pipeline staging required.
- backend/main.py reads the co-located file; its in-code fallback is no longer a
real version ("unknown") so it can never silently drift again.
- docker-build.yml reads backend/version.json and drops the staging step;
docker-compose.localtest drops the stale root mount.
- backend/tests/test_version_consistency.py fails the build if main.py hardcodes
a real version, if the canonical file is missing/invalid, or if
frontend/package.json drifts from it.
Bumped to 1.8.7. No functional, schema, API, or agent change. Full suite green.
|
||
|
taylanbakircioglu
|
23257b02cf |
perf(api): opt-in uvicorn workers + heartbeat query consolidation (v1.8.6, Issue #35)
A user running the API on a 2-core/4GB host reported slow-feeling API responses (Issue #35 follow-up). Review of the hot paths found no pathological defect; the dominant factors are the single uvicorn worker (one core serves all requests) and the constant agent-poll baseline (4 requests per agent every 30s). Two zero-risk improvements: - backend/Dockerfile: CMD now honors UVICORN_WORKERS, falling back to WEB_CONCURRENCY and then 1. Flagless uvicorn natively honors WEB_CONCURRENCY, so the fallback keeps any deployment that relied on it byte-for-byte compatible; with the final default of 1 worker uvicorn runs in-process exactly as before. >1 enables the multiprocess supervisor so multi-core hosts can use all cores. Background tasks are already multi-replica safe (FOR UPDATE SKIP LOCKED / advisory locks), as exercised by the k8s HPA deployment (2-10 replicas). `exec` keeps uvicorn as PID 1 (clean SIGTERM, verified ~1s docker stop with 2 workers). docker-compose.yml passes UVICORN_WORKERS through as empty-when-unset so a user-set WEB_CONCURRENCY is never overridden; .env.template documents it. - routers/agent.py heartbeat (by-name endpoint): the agent's status/version/upgrade_status were read with three separate single-column SELECTs against the same row; now one SELECT. Identical values and None semantics (single consistent snapshot instead of three reads); saves two round-trips per heartbeat per agent every 30s. The legacy by-id heartbeat endpoint is untouched; the heartbeat API contract is unchanged for agents of every version. - README: new "Performance Tuning" section (worker/replica scaling, and how to use the X-Response-Time header plus "Slow request detected" logs to pinpoint slow endpoints). Verification: full backend suite in docker green (1063 passed, 151 skipped; also re-run by the runtime image build); worker-count expansion matrix (unset->1, UVICORN_WORKERS=2->2, WEB_CONCURRENCY=3->3, both->UVICORN_WORKERS, empty->fallback) all correct; default run confirmed single-process with uvicorn as PID 1 and healthy API; UVICORN_WORKERS=2 confirmed parent + 2 workers, healthy API, clean shutdown; live heartbeats verified for register + existing-agent paths AND degraded agents (no stats socket / haproxy stopped / garbage stats CSV / unknown backend in server_statuses): all return 200, agent row updates correctly, zero backend errors. No schema, API, or agent changes. |
||
|
taylanbakircioglu
|
c8d144ca9d |
fix(acme): remove stray paren breaking ACME completion task (v1.8.5, Issue #35)
The background order-completion task (complete_pending_acme_orders, 60s
cycle) failed on EVERY cycle since v1.8.0 with:
[ACME-COMPLETE] Error in completion task: syntax error at or near ")"
Root cause: the bounded DNS-01 retry OR-arm added to the atomic order-claim
query in v1.8.0 (13b65d9) carried one extra closing parenthesis, making the
whole SELECT invalid PostgreSQL. The claim is the task's first statement, so
the generic except swallowed it each minute and NO background ACME work ever
ran on v1.8.0-v1.8.4:
- orders were never claimed for finalize -> download -> save (http-01 too);
- advance_dns01_order never ran, so the DNS-01 TXT record was never
published - DNS-01 with an automated provider (e.g. Cloudflare) could
never validate (exactly the report in Issue #35);
- wizard-staged orders never left wizard_staged (same try block);
- retry_invalid_dns01 / reconcile_dns01_cleanup never executed;
- hourly-created renewal orders could never complete in the background.
Fix: drop the stray ')' (one line). Query semantics are unchanged.
Why the suite missed it: the unit tests mock asyncpg, so raw SQL never
reaches a real parser. Added a regression test that AST-scans the ACME
modules' SQL string literals (comments/quoted literals stripped) and fails
on unbalanced parentheses - it is red on the pre-fix tree and would have
caught the v1.8.0 regression at commit time. Scanned all six ACME modules:
this query was the only unbalanced SQL.
Verification: full backend suite in docker green (1062 passed, 151
skipped); the fixed query EXPLAINs cleanly on postgres:15; live localtest
run shows zero completion-task errors and a seeded pending order is claimed
("[ACME-COMPLETE] Claimed 1 order(s)"). Backend-only, no schema/API/agent
changes; fully backward compatible.
Reported-by: @tkkost (GitHub Issue #35)
|
||
|
taylanbakircioglu
|
64d42663cd |
fix(agent): stop installer self-kill in pre-installation cleanup (v1.8.4)
The Linux/macOS agent installer could abort during "pre-installation cleanup"
(terminal showed `Killing processes matching: haproxy-agent` then `Killed`,
returning to the prompt) when the install script's own command line contained
"haproxy-agent". The cleanup killed processes via `pgrep -f "$pattern"` starting
with the bare string "haproxy-agent", which also matched the running installer's
own command line and a sudo/PAM ancestor that the $$/$PPID self-exclusion did not
cover, so the installer terminated itself before installing.
- linux_install.sh / macos_install.sh: the cleanup kill loop now targets ONLY
the installed agent - "$INSTALL_DIR/haproxy-agent" (the daemon binary path) and
the agent service/label ("haproxy-agent.service" / "com.haproxy.agent") - never
the bare "haproxy-agent" substring. Neither pattern can match the installer's
own command line. The redundant bare pattern is dropped (the service is stopped
separately, and the binary-path pattern still catches a running daemon).
- frontend (AgentManagement.js): name the downloaded scripts
install-agent-<platform>.sh / uninstall-agent-<platform>.sh (matching the
backend's suggested filename) - defense in depth so this cannot resurface.
Installer-only change. The running agent and its privilege model are unchanged
(it runs as root for HAProxy reload, config writes, keepalived, and self-upgrade).
The cleanup runs only on a full interactive install (gated by SKIP_TO_DAEMON), so
daemon mode, self-upgrade, and config/version apply are unaffected. Both agent
scripts kept in sync. Scripts parse on bash 4.2-5.2; full backend suite green.
Addresses #31.
|
||
|
taylanbakircioglu
|
27fbe48c4b |
fix(agent): tolerate empty system_info in heartbeat JSON (v1.8.3)
A self-hosted agent could fail every heartbeat with HTTP 400 `Invalid JSON: Expecting property name enclosed in double quotes` when the system-info block it collects came back empty on an unusual host. The agent builds the heartbeat JSON as text, so an empty `$system_info` collapsed the `$system_info,` line to a bare comma and broke the payload. - Agent script (linux + macos, kept in sync): guard the fragment-form register_agent and send_heartbeat builders so an empty system_info falls back to a valid key and can never emit a bare comma. Uses the most portable bash glob test (no POSIX class / pattern substitution; verified on bash 3.2-5.2 and on Ubuntu/Debian/Rocky/Alpine/Amazon Linux). True no-op for healthy agents. - Backend heartbeat endpoint: parse the body as-is first and only run the malformed-JSON repair when parsing fails, so a valid heartbeat from any agent version is byte-for-byte untouched. The repair (now a testable helper) recovers a leading or doubled comma (the empty-system_info artifact) in addition to the existing empty-value / trailing-comma fixes. No agent version bump; self-upgrade and daemon mode are unaffected. Healthy agents of every version behave identically. Full backend suite green. Addresses #31. |
||
|
taylanbakircioglu
|
e86e86a53c |
fix(acme): scope ACME nonce per CA - fixes ZeroSSL registration (v1.8.2)
ZeroSSL/Google account registration failed with `malformed: The Replay Nonce could not be base64url-decoded`: the ACME client (a process-wide singleton) kept a single anti-replay nonce shared across certificate authorities, so a nonce issued by one CA could be sent to another, and the auto-retry only covered `badNonce`. - Scope the nonce per CA (self._nonce_by_dir keyed by directory_url): a nonce from one CA is never sent to another; account registration always uses a fresh nonce from the target CA. - Broaden the 400 retry to also recover from the nonce-malformed rejection. - Fix _b64url_decode padding (used for the EAB HMAC key). Backend-only; HTTP-01 and Let's Encrypt are unaffected. Addresses #35. |
||
|
taylanbakircioglu
|
70ebc02e09 |
fix(acme): Cloudflare token, ZeroSSL EAB, Apply Management (v1.8.1)
Follow-up fixes for the DNS-01 feature reported on #35: - Cloudflare: sanitize the API token (strip surrounding quotes + any non token68 chars) so a pasted token with quotes/spaces no longer fails with "Invalid request headers"; verify-on-save shows a precise hint when it cleaned the input. Covers the automated orchestrator path too. - ZeroSSL/Google EAB: enter the EAB Key ID and HMAC Key per-account in the Register Account dialog (falls back to the global Settings value when blank); base64-validate the HMAC key; humanize the externalAccountRequired failure; and preserve the deliberate 409/422 instead of downgrading them to 400. - Apply Management: cluster ACME enable/disable changes now show under a dedicated "ACME Challenge Routing" section, are counted in the Apply/Reject dialogs, and Apply/Reject All process them (previously "Rejected 0 HA/VIP change(s)") - consistent with every other entity. Reject rolls acme_enabled back to the original via ORDER BY created_at ASC over the snapshot chain. - getErrorMsg surfaces field-level validation messages. Backward compatible (additive / strict superset; HTTP-01 unchanged). Addresses #35. |
||
|
taylanbakircioglu
|
c492b26bb1 |
feat(acme): DNS-01 challenge support with pluggable DNS providers (v1.8.0)
Add ACME DNS-01 (TXT-record) validation alongside the existing HTTP-01, for internal/isolated clusters with no public port 80 and for wildcard certificates. Opt-in via a global kill-switch (default off); HTTP-01 is byte-for-byte unchanged, with zero agent or rendered-config changes. - Pluggable DNS provider interface (Manual + Cloudflare). Per-account credentials are Fernet-encrypted at rest, verified on save, and never returned by the API or written to logs/events/error_detail. - Non-blocking per-cycle orchestrator: publish (CAS) -> propagation grace (across cycles, no in-loop sleep) -> respond -> finalize/download, with a bounded fresh-order retry chain (1 original + 3 retries) on propagation lag. - Manual flow: user publishes the TXT record and confirms; manual DNS-01 cannot auto-renew unattended (auto-renew forced off and surfaced in the UI). - Migration v8: additive, idempotent columns on letsencrypt_accounts/orders and acme_challenges, plus a new letsencrypt_account_dns_credentials table. - Challenge-type-aware diagnostics (port80/routing/DNS checks skipped for DNS-01) and a DNS-01 event timeline in the order detail. - Frontend: DNS-01 account + credentials management, cert wizard adaptation, order-detail TXT records + verify, orders/renewal Method columns, and a Settings kill-switch. README, release notes, and API docs updated. Implements #35. |
||
|
taylanbakircioglu
|
a1192e602d |
feat: HA / VIP (Keepalived) management from the UI (#27)
Manage highly-available virtual IPs backed by Keepalived (VRRP) directly from the OpenManager
UI — no more SSHing into nodes to install/configure Keepalived by hand. Builds on the agent
pull-architecture: define the VIP centrally, click Apply, and the agents converge.
Highlights:
- New "HA / VIP" tab: create a virtual IP, pick a per-node interface, select which pool nodes
participate (MASTER/BACKUP roles + priorities); live MASTER/BACKUP/FAULT per node.
- On Apply, agents install & configure Keepalived (unicast VRRP, cloud-safe default) across the
major distros (Debian/Ubuntu, RHEL/CentOS/Alma/Rocky, Fedora, SUSE/openSUSE, Alpine) with a
HAProxy health-check, so the VIP fails over automatically when HAProxy drops.
- Single-node (a managed floating IP without failover) and multi-node VRRP failover both work.
- VIP changes ride the standard Apply Management flow with the standard "View Change" diff.
- Approval-gated deletion (safety): deleting a running VIP is staged for approval and the VIP
keeps running, untouched, until you approve it — an agent never tears a VIP down without an
explicit human approval. Per-VIP Diagnostics view; opt-in package uninstall (only on nodes
where OpenManager installed it). A node already running a hand-managed Keepalived is detected
and never overwritten ("externally managed").
- Fully opt-in and backward compatible: nodes/clusters without a VIP are unaffected. Adds
vip_instances + vip_members tables (idempotent SCHEMA_VERSION bump; existing data and
passwords unaffected) and a `vip` RBAC permission group.
- Also includes a HAProxy config-generator robustness fix: auto-inject a stick-table when a
frontend uses a stick counter (track-sc / sc_*_rate) but declares none.
On-prem / L2 (VRRP) scope; the UI notes the cloud caveat.
|
||
|
taylanbakircioglu
|
9d7a718671 |
fix(security): re-pin nginx to 1.31.1-alpine for the poolslip advisory (v1.6.5)
Supersedes the v1.6.4 stable pin (1.30.2-alpine) with the mainline patched release. No config, schema, or behavior changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
taylanbakircioglu
|
62b1599354 |
fix(security): pin nginx to 1.30.2-alpine for the poolslip advisory (v1.6.4)
The bundled nginx reverse proxy was flagged for the nginx 'poolslip' advisory (affected: mainline <=1.31.0; fixed: stable 1.30.2+ / mainline 1.31.1+). The config-level mitigation (named capture groups instead of $1/$2 in rewrite) does not apply — the product's nginx config (nginx/nginx.conf and the k8s configmap) has no rewrite capture-group directives, only prefix locations + proxy_pass. So the fix is the version: pin nginx:alpine -> nginx:1.30.2-alpine in docker-compose.yml and k8s/manifests/10-nginx.yaml. No config/schema/behavior change (nginx only reverse-proxies). Version bumped to 1.6.4 across all layers. Verified in Docker: nginx -v=1.30.2; nginx -t OK on both the compose and production configmap configs; all proxied routes work through nginx; no nginx errors. |
||
|
taylanbakircioglu
|
b34d7cf811 |
fix: reactivate disabled backend servers from the UI + v1.6.3 (Issue #24)
A backend server toggled OFF (is_active=false) vanished from the UI with no way to reactivate it: GET /api/backends honored include_inactive for backends but the server sub-queries hardcoded 'AND is_active = TRUE'. - get_backends: server sub-queries now honor include_inactive (default callers unchanged); added last_config_status to the server payload so the UI can tell a DISABLED server (re-enableable) from a DELETION (pending delete). - toggle_server: persists an entity snapshot so an Apply-Management Reject rolls back is_active (previously left the server stuck disabled). - BackendServers.js: requests include_inactive, shows disabled servers with the ON/OFF switch + an 'Inactive' tag, hides only DELETION-pending servers, and keeps soft-deleted BACKENDS hidden (so include_inactive doesn't resurface them). - Config generation unchanged: disabled servers stay '# DISABLED:' comments and convert back to live lines when re-enabled. Startup migration hardening (multi-replica / rolling-deploy safety): create_essential_tables fails fast on lock contention and retries; run_all_migrations is serialized by a session advisory lock and gated by a schema_migrations version marker, so an already-current schema is skipped instead of issuing lock-heavy DDL that a serving replica's traffic could block at startup. Idempotent and fail-open. Version reported consistently across all layers (version.json, backend fallback, frontend package) -> 1.6.3. |
||
|
taylanbakircioglu
|
f3d4fb11bb |
fix(agent): accept agent token (X-API-Key) on cluster read endpoints (Issue #22)
Assigning a HAProxy agent failed with '401: Authorization header missing'
on GET /api/clusters. A cluster-read hardening had made GET /api/clusters and
GET /api/clusters/{id} accept only a user JWT in the Authorization header;
agents authenticate with their agent token in the X-API-Key header, so the
token was never read.
Both endpoints now accept either a user JWT (Authorization) or an agent token
(X-API-Key via validate_agent_api_key), mirroring the existing dual-auth on
POST /api/agents/generate-install-script. Anonymous access is still rejected,
so the original hardening is preserved. The auth guard is placed before the
try block so the failure surfaces as a clean 401 (not the 500-wrapped-401 in
the report). Agent install scripts now consistently send the token via
X-API-Key (pre-flight cluster check on linux/macos, and macOS get_cluster_paths
which previously used the wrong Authorization: Bearer header).
Also normalizes the platform in the uninstall-script generator so macOS agents
(which report platform 'darwin') no longer get a 400 from
GET /api/agents/generate-uninstall-script/darwin.
version 1.6.0 -> 1.6.2.
|
||
|
taylanbakircioglu
|
bd6a31cb0d |
feat: v1.6.0 — Multi-Factor Authentication (Issue #18)
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.
Highlights
==========
* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
`MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
`XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
+ `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
Users page; admins reset (single user or bulk) but never enable on
behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.
Security hardening
==================
* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
(prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
audit log so request bodies (TOTP / backup codes in flight) never
appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
XFF strategy so a single ingress address cannot exhaust the bucket
for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
`MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
with the same username can no longer occlude the active user
(also closes a small account-enumeration side channel).
Database
========
Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):
- users: mfa_enabled, mfa_method, mfa_secret_encrypted,
mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
- mfa_backup_codes (user_id ON DELETE CASCADE)
- mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
attempts, expires_at)
- mfa_pending_enrollments (user_id ON DELETE CASCADE)
Frontend
========
* Login page becomes a 3-phase state machine
(credentials → MFA → submitting); legacy single-step login is
preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
Admins viewing other users with MFA off see a non-actionable info
icon explaining that only the user themselves can enable MFA.
Deployment
==========
* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
a placeholder; `SECRET_KEY` is also placeholder-ized so both are
injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
uses `window.location.host` for `/api/*` and is routed by the
existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
into the production bundle.
Tests
=====
* New unit suites:
- `test_mfa_service.py` (TOTP, encryption, backup codes)
- `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
- `test_mfa_rate_limits.py` (env override + dataclass immutability)
- `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.
Documentation
=============
* README MFA section (overview, day-to-day operations, emergency
reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.
Issue: #18
|
||
|
taylanbakircioglu
|
d7208528f7 |
fix: v1.5.2 — ACME Diagnostics Panel Hardening + AGPL-3.0 relicense (Bulgu #94/#95/#96)
A focused hardening pass on the v1.5.0 ACME Diagnostic Panel
surface, exercised against a live production deployment (Round-25
+ Round-26 audits) and supplemented by an AGPL-3.0 relicense.
------------------------------------------------------------------
LICENSE — Relicense to AGPL-3.0-or-later
------------------------------------------------------------------
Effective v1.5.2 the project is licensed under the **GNU Affero
General Public License v3.0 (or later)**. v1.5.0 and v1.5.1
remain under the prior MIT terms.
The relicense is consistent with the project's intent as a
community-operated HAProxy management surface: forks that run
HAProxy OpenManager as a network service for third parties are
now required to publish their modifications under the same
license (AGPL §13). Day-to-day single-tenant deployments,
internal corporate use, and ordinary forks-for-fixes are
unaffected.
Changes:
* LICENSE replaced with full AGPL-3.0 text.
* README "## License" section rewritten with the AGPL summary
+ the network-service obligation.
* frontend/package.json gains `"license": "AGPL-3.0-or-later"`.
------------------------------------------------------------------
BULGU #94 / #95 — Diagnostic Panel Must Never Opaque-500
------------------------------------------------------------------
Live exercise of the v1.5.0 Diagnostic Panel against a deployed
build surfaced two opaque-500 paths. The panel exists to make
ACME failures legible; producing an opaque HTTP 500 defeats the
entire feature. Fix shape: every endpoint now returns either a
canonical 4xx (auth / not-found / rate-limit) or an HTTP 200
"structured failure envelope" that the React UI knows how to
render — never a 500 for an in-suite failure.
Affected paths:
POST /api/letsencrypt/orders/{order_id}/diagnostics
Pre-fix: a UndefinedColumnError or DB-connectivity failure
inside `run_checks` bubbled out of the bare try/finally and
surfaced as a generic 500 with no operator-actionable detail.
Post-fix: setup-stage and run-stage failures are caught
separately and converted to a `status: diagnostics_unavailable`
envelope carrying `error_stage`, `error_type`, `error_message`,
and a `correlation_id` that the operator can grep in the
backend log. Individual checks are wrapped in `_safe_check`
so one broken check (e.g. DNS lookup timeout) never crashes
the suite — the failing check shows up as `status: "fail"`
with its message, the others still run.
GET /api/letsencrypt/orders/{order_id}/events
Pre-fix: the SQL `SELECT … status FROM user_activity_logs`
referenced a column that did not exist in the canonical
migration; every diagnostic-panel open against an order with
any user-activity-log correlation got an `UndefinedColumnError`
500. Post-fix: the endpoint now introspects
`information_schema.columns` and projects only the columns
actually present. Partial failures (one source dies, the
other works) are reported via `meta.errors[]` rather than
collapsing the whole timeline.
POST /api/letsencrypt/orders/{order_id}/diagnostics/{check_id}/rerun
Same structured-envelope contract as the full-suite POST,
scoped to a single check row.
Frontend (`frontend/src/components/ACMEAutomation.js`):
* Distinct `diagRunError` / `diagEventsError` / `diagMeta`
states so the modal can render the cause inline (Antd Alert)
instead of a silent dropdown.
* Event-log auto-tail polling backs off after 3 consecutive
failures so the Network tab does not get spammed with 500s
every 5s.
* Correlation IDs visible in every error banner.
------------------------------------------------------------------
BULGU #96 — Clean 404 for Out-Of-Range order_id
------------------------------------------------------------------
A live exercise of the post-#94 diagnostic panel against the
deployed build surfaced one remaining contract gap. A path-
param `order_id` outside the Postgres int4 range
(e.g. > 2_147_483_647) caused `_load_order` to raise
`asyncpg.exceptions.DataError: invalid input for query
argument $1: 2147483648 (value out of int32 range)`. Round-25
correctly surfaced this in a `diagnostics_unavailable`
envelope — but that envelope leaked SQL implementation detail
("query argument $1", "int32 range", DataError class name)
into the operator-facing response body.
Semantically an out-of-range integer can never reference a
real order — it's just "not found". `_load_order` now catches
`asyncpg.exceptions.DataError` and re-raises a canonical
`HTTPException(404, "Order {id} not found")`. Because all
three endpoints re-raise `HTTPException` from their outer
try/except (the Round-25 envelope only fires for non-
HTTPException crashes), the canonical 404 path now wins
end-to-end across /diagnostics, /events, and /rerun.
------------------------------------------------------------------
TEST / LINT / LIVE VERIFICATION
------------------------------------------------------------------
* Backend pytest 1104/1104 (the +20 vs v1.5.1's 1084 are the
Round-25 and #96 contract pins; see
test_acme_diagnostics_router_round25.py).
* Live prod-canary verification: every endpoint return shape
confirmed against the deployed build — int4 overflow returns
clean 404 with no SQL leak, normal paths return Round-25
envelopes, HTTP method matrix returns 405 on wrong verbs,
no auth returns 401, invalid `check_id` returns 400, and
`meta.correlation_id` is present on every diagnostic
response.
------------------------------------------------------------------
COMPATIBILITY
------------------------------------------------------------------
* No breaking API contract changes: `status` field on the
diagnostic response can now be `"diagnostics_unavailable"`
in addition to the existing pass-through of the
underlying order status (`pending` / `valid` / `invalid` /
`cancelled` / …) — older UIs that only switch on the
existing values render the `diagnostics_unavailable`
case as "unknown status" rather than crashing.
* Frontend handles the new envelope shape AND the legacy
HTTP 4xx/5xx paths.
|
||
|
taylanbakircioglu
|
2e7db4d99f |
fix: v1.5.1 — Round-23 + Round-24 audit follow-ups (Bulgu #83 → #93)
A live-deployment audit pass over the v1.5.0 Site Wizard + ACME Diagnostic Panel surface. Two adversarial review rounds (R23, R24) each capped by an end-to-end smoke test against a multi-cluster staging deployment. Bulgu #83 — Frontend Management page warned about stale data without a clear retry CTA. The toast now carries an in-place "Reload" action and the page-level Empty state surfaces the same recovery affordance, so operators never get stuck on a stale-data view without an obvious way out. Bulgu #84 — ACME diagnostics ran with the wrong "last_heartbeat" column reference against the agents table. Aligned the SELECT with the actual schema column (`last_seen`); pinned by an idempotent regression test in `test_acme_diagnostics.py`. Bulgu #85 — ACME order error_detail rendering could leak the raw asyncpg/SQL exception class name when humanize_error_detail encountered an unhandled CA response shape. Added a backwards- compatible fallback branch that emits an "ACME error (raw)" panel without exposing parse_error class name to the user. Bulgu #86 — Multi-cluster apply with concurrent rejects could leave wizard_staged orders dangling without their parent draft. Pinned via reject_order_with_cluster_orphan test. Bulgu #87 — Frontend Management page list virtualization mis-keyed during a re-sort + stale-row replace race; fixed by keying rows on `id + version` so React reconciler does not reuse DOM for a logically different row. Bulgu #88 — Site Wizard "Cancel" mid-flow now surfaces an unsaved-draft prompt with explicit Save / Discard buttons (and the same prompt on browser tab close), so the operator never loses 5 steps of input to an accidental ESC. Bulgu #89 — Existing-cert SSL mode showed an empty dropdown when the cluster had >100 certs because the listing endpoint default-limited results. Endpoint now exposes pagination AND the wizard switches to client-side filtering above 50 rows. Bulgu #90 — ACME pre-check on the wizard preview path did NOT re-validate the account against `letsencrypt_accounts` if the operator stepped Back/Forward between SSL and Review. Added a debounced re-validation on Review entry. Bulgu #93 — Site Wizard hsts_enabled toggle in HTTPS frontend was idempotent-by-name (the generated `http-response set-header Strict-Transport-Security` line could duplicate across a Save + Apply cycle). The renderer now upserts the header in place. Cumulative outcome: backend pytest 1084/1084, frontend lint clean, and a 6-hour live-deployment smoke session against staging with no regressions reported. |
||
|
taylanbakircioglu
|
02b1cb2bca |
feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13)
Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit. |