Compare commits

..

4 Commits

Author SHA1 Message Date
taylanbakircioglu 78641d87ff feat(vip): let a VIP carry additional addresses (issue #73, v1.12.0)
A VIP held exactly one address. It can now carry up to 19 additional IPv4
addresses (with prefix) in the same virtual_ipaddress block, on the same interface,
failing over together with the virtual IP.

- vip_instances.additional_ips (JSONB, default []); an empty list renders exactly
  as before. Create/edit accept the list (omitted keeps, [] clears); it is part of
  the applied snapshot and restored by Reject and undo.
- Every address write takes one advisory lock and checks the addresses against
  node IPs and every other active VIP's primary and additional addresses.
- Adoption accepts an instance with several addresses: the ones after the first
  become additional addresses when they render back exactly (IPv4, explicit
  prefix, the instance's interface, no repeats); the peers must agree on them.
- UI: an "Additional addresses" list in the VIP form and a "+N" tag in the table.

v1.12.0: README, upgrade notes (delta migration, rolling-deploy and downgrade
rules, per-node live state), version files.

Fixes #73
2026-10-17 13:05:29 +03:00
taylanbakircioglu 349434fc4d feat(vip): let a node belong to several keepalived VIPs (issue #73)
A node could be a member of one VIP only: the form refused a second one, and the
delivery endpoint served each agent a single VIP's keepalived.conf.

- Delivery merges the node's applied VIPs into the one file it deploys: global_defs
  once, then per VIP a vrrp_script chk_haproxy_<id> (each keeps its own failover
  weight) and its vrrp_instance, ordered by VIP id. A node with one VIP gets the same
  bytes and hash as before, so existing nodes do not rewrite or reload.
- One ack converges every VIP in the merged file; a deleted VIP's membership is
  marked disabled once the node runs a file without it.
- A snapshot that cannot be merged leaves the node alone (status compose_error,
  members show the reason). A database error during delivery answers "unavailable"
  instead of "not_configured", which on a managed node meant teardown.
- Adoption refuses a keepalived.conf with several vrrp_instance blocks (taking it
  over would remove the others), also for analyses stored by older versions and at
  Apply for an adoption made before the upgrade. Discoveries record the parser
  version; files analysed by an older one are re-reported once by v1.11.1+ agents.
- Schema 13 is a delta from 12 (two additive columns); built-in roles are not
  re-seeded on a database already at 12.

No agent change.
2026-10-17 13:04:38 +03:00
taylanbakircioglu dd11fe1be5 fix(import): keep stick-table and stick rules on Config Import (v1.11.3)
A frontend re-defined with
  stick-table type ip size 200k expire 3h store http_req_rate(10s),gpc0_rate(10s),gpt0
kept rendering the old 100k/30s table after a Config Import of the full
frontend section, with no error or warning.

The import parser had no branch for `stick-table` (frontend or backend) or for
backend `stick on|match|store-request|store-response`, so those lines were
dropped silently. The frontend still used track-sc0 / sc_* counters, so the
renderer's v1.7.4 auto-inject added its default table in place of the
operator's one, and the gpc0 rules imported with it referenced counters the
table did not store.

  - Parser: these lines are kept verbatim in the frontend/backend `options`
    field (free-form, already rendered and snapshotted; no schema change)
    and listed as import warnings.
  - Import preview: options are merged per kind, so a stick-only import keeps
    the stored `option` lines and vice versa. The table the generator emits
    itself for Rate Limit / WAF rate_limit (type ip size 100k expire <N>s
    store http_req_rate(10s)) is not stored where the generator will render
    it, so round-trips show no change and later window edits still apply.
  - The preview stripped every `track-sc0 src` / `sc_http_req_rate(0)` line
    as auto-generated. It now strips only the exact generated forms, and
    only on frontends with Rate Limit or an active rate_limit WAF rule;
    operator trackers elsewhere were silently deleted on update.
  - Renderer: a backend emits only its first `stick-table`. A second one is
    fatal in HAProxy and made the agent reject the whole cluster config.
  - Non-blocking advisories: a table without http_req_rate while Rate Limit
    is active (HAProxy accepts it and the limit never fires), a table also
    left in Request Headers, `stick on` without a table, `peers` references.

Configs without stick lines parse and render byte-identical (S0 render diff
empty; 70/70 stick-free test configs parse identically). Generated configs
pass `haproxy -c` on 3.3.10 and 2.6. No schema change, no agent change.

Fixes #72
2026-10-16 14:13:36 +03:00
taylanbakircioglu b49db4aff7 fix(rbac): let read-only roles read SSL, and gate private keys on ssl.download (v1.11.2)
A user on the built-in viewer role got 403 from the SSL API while clusters,
frontends, backends, WAF rules, agents and VIPs were readable.

The UI never fills in user_pool_access, so every non-admin user created there
has no pool access. The SSL list with ?cluster_id= and the SSL detail required
pool access on read; the other read endpoints do not. The check also protected
little: the SSL list without cluster_id already returned every certificate.

  - New auth_middleware.validate_cluster_read_access: a role that grants
    <resource>.read reads that resource on every cluster, otherwise the
    router's pool check still applies. Used by the SSL list, SSL detail and
    GET /api/waf/stats. Write endpoints are unchanged.

While tracing this, GET /api/ssl/certificates/{id} and the SSL change diff in
Apply Management turned out to return the private key to any authenticated
user, viewer included.

  - The key is now included only with ssl.download (already in the role
    catalog, never enforced) plus the pool check that guarded the detail route
    before. Otherwise the detail returns private_key_content: null with
    private_key_redacted: true, and the diff shows a redaction line.
  - SSL Management shows the key as hidden, and on edit the key field becomes
    optional; an empty field keeps the stored key (the PUT already falls back).

Also: GET /api/waf/stats re-wrapped its 403/404 as 500 and leaked the
connection on that path.

No schema change, so the built-in roles are not re-seeded. Custom roles with
ssl.update but no ssl.download no longer see the key.
2026-10-16 14:13:08 +03:00
25 changed files with 2075 additions and 165 deletions
+4
View File
@@ -1096,6 +1096,7 @@ The HA / VIP page manages **highly-available virtual IPs** backed by Keepalived
- **VIP dashboard**: lists each virtual IP with its pool, VRID, per-node interface, and a **live MASTER / BACKUP / FAULT** column per node (refreshed every 30s from the existing keepalive-state heartbeat pipeline).
- **Creation form**: enter the virtual IP (+ prefix), pick the pool, and select which of the pool's HAProxy nodes participate — each with a network interface (from the node's reported interfaces), a VRRP role (MASTER/BACKUP) and a priority. VRID auto-allocates per pool; an optional VRRP secret can be set.
- **Hands-off automation**: on **Apply**, each member's `keepalived.conf` is rendered and delivered; the agent installs Keepalived if missing, writes the config + a HAProxy health-check (`vrrp_script` + `track_script`), validates with `keepalived -t`, and starts the service. When HAProxy drops on the active node, the health-check lowers VRRP priority and the **VIP fails over automatically** to a backup.
- **Several VIPs per node, several addresses per VIP** (v1.12.0): a node can belong to more than one VIP — each VIP is its own `vrrp_instance` (its own VRID, role and priority) and the node's `keepalived.conf` holds all of them. A VIP can also carry up to 19 **additional addresses** next to its virtual IP; they sit in the same `virtual_ipaddress` block, on the same interface, and fail over together.
**How it works (pull-based, isolated):**
- New tables `vip_instances` + `vip_members`; the agent polls `GET /api/agents/{name}/keepalived-config` (key-authenticated, agent-bound) and converges. It is fully **isolated** from the global HAProxy apply flow — VIP changes never regenerate `haproxy.cfg`.
@@ -2475,6 +2476,9 @@ Developed with ❤️ for the HAProxy community
## Release Notes
- **v1.12.0** (2026-10-16) — **Several keepalived VIPs on one node, and several addresses on one VIP**: a node could belong to only one VIP (the form refused a second one with "node already in VIP …") and a VIP held exactly one address. A node may now be a member of several VIPs: each VIP stays its own `vrrp_instance` with its own VRID, role and priority, and the delivery endpoint merges the node's applied VIPs into the one `keepalived.conf` the node deploys — one `global_defs`, then per VIP a `vrrp_script chk_haproxy_<id>` (each VIP keeps its own failover weight) and its `vrrp_instance`, ordered by VIP id. A node with a single VIP receives exactly the bytes and hash it received before, so existing installs see no change and no reload. A VIP can also carry up to 19 **additional addresses** (IPv4 + prefix) in the same `virtual_ipaddress` block and on the same interface; they are checked against node IPs and every other VIP's addresses under a lock, kept in the applied snapshot, and restored by Reject. The merged file passes `keepalived -t` with no output (which older agents require) on keepalived 2.0.19, 2.1.5, 2.2.4, 2.2.7 and 2.2.8; on 2.0.19, 2.1.5 and 2.2.8 a reload that adds a second VIP did not drop the first one's address, and a VIP with 20 addresses came up. If a node's VIPs cannot be merged, the node is left untouched and its members show the reason. A database error during delivery now answers a status the agent ignores instead of `not_configured`, which on a managed node meant teardown. **Adoption** refuses a `keepalived.conf` with several `vrrp_instance` blocks (taking it over would remove the others; earlier versions allowed it), now accepts an instance with several addresses (the extra ones become additional addresses), and asks agents to re-report files analysed by the older parser once. Schema `13` is a **delta** from `12`: two additive columns, built-in roles are not re-seeded. No agent change. See the upgrade notes for rolling-deploy and downgrade rules.
- **v1.11.3** (2026-10-16) — **Config Import keeps `stick-table` and `stick` rules instead of dropping them**: the import parser had no branch for `stick-table` (frontend or backend) or for backend `stick on|match|store-request|store-response`, so those lines were discarded **without a warning**. A frontend that kept its `track-sc0` / `sc_*` counter rules then received the renderer's fallback table (`type ip size 100k expire 30s store http_req_rate(10s)`), so a re-defined table such as `size 200k expire 3h store http_req_rate(10s),gpc0_rate(10s),gpt0` was silently replaced by the default while the `gpc0` rules imported alongside it referenced counters the table did not store. These lines are now imported as-is into the frontend/backend **Options** field (no schema change) with an import warning, and an import that carries only a stick-table keeps the existing option lines. The table the platform generates itself for Rate Limit / WAF `rate_limit` rules is recognised and not stored, so a round-trip of a generated config shows no change and later rule edits still take effect. The import preview also deleted every `http-request track-sc0 src` / `sc_http_req_rate(0)` line as "auto-generated"; it now does so only where the platform regenerates them (frontend Rate Limit or a WAF `rate_limit` rule), so operator trackers on other frontends survive. Backends now render a single `stick-table` (the first; a second one is a fatal HAProxy error) and the preview warns about a table that does not store `http_req_rate` while Rate Limit is active, a duplicate table left in Request Headers, `stick on` without a table, and `peers` references. Configs without stick lines import and render byte-for-byte as before. No schema change, no agent change.
- **v1.11.2** (2026-10-01) — **Read-only roles read SSL certificates, and private keys need `ssl.download`**: the UI never fills in `user_pool_access`, so every non-admin user created there has no pool access, and the SSL list (`?cluster_id=`) and SSL detail answered **403** on read while clusters, frontends, backends, WAF rules, agents and VIPs were readable. A role that grants `ssl.read` (or `waf.read` for WAF stats) now reads that resource on every cluster; without it the pool check still applies, and write endpoints are unchanged. Separately, `GET /api/ssl/certificates/{id}` and the SSL diff in Apply Management returned the **private key to any authenticated user**, including `viewer`. The key is now included only with `ssl.download` plus pool access; everyone else gets `private_key_content: null` with `private_key_redacted: true`, and the UI shows the key as hidden while editing keeps the stored key. Custom roles with `ssl.update` but no `ssl.download` lose sight of the key. No schema change, no agent change.
- **v1.11.1** (2026-08-15) — **A node can no longer be hidden from adoption for good, and Config Import reaches a freshly installed agent**: the agent posts an unmanaged `keepalived.conf` for adoption and caches the hash of what it sent, so the file (which carries the VRRP password) is re-posted only when it changes. Delivery was judged by `curl`'s exit code, which is **0 for 5xx as well**, so a report the server *rejected* was recorded as delivered — and because a hand-maintained config does not change on its own, that node dropped out of *Unmanaged keepalived detected* permanently, curable only by deleting a cache file on the node by hand. Now: the report is cached only on a 2xx; `GET /agents/{name}/keepalived-config` tells the agent whether the server actually holds a discovery for it, so nodes stuck from earlier releases recover by themselves on the next poll; a **400/413/422** records the refusal so identical bytes are not re-posted forever (4xx and 5xx are never sampled out of the request log, so an unattended loop would write a row carrying the whole config every cycle), while **401 and 404 keep retrying** because in this system they mean a token rotation or an agent row briefly absent, not a bad payload. The same exit-code mistake in the *clear* path is fixed too, where it left a managed node still being offered for adoption. Separately, **Config Import silently did nothing on any agent that had never self-upgraded**: `check_config_requests` was defined in the installer and in the self-upgrade daemon but not in the body a fresh install writes, and its call site is guarded by `type`, so the operator asked a node for its `haproxy.cfg` and nothing arrived, with no error anywhere. And the cluster whose `keepalived_config_path` is handed to an agent is now resolved deterministically — a pool may hold several clusters, and the unordered join could return a different one between polls, pointing the agent at a file that does not exist. Agent-script change: sync the script from Agent Management and let the agents upgrade. No schema change.
- **v1.11.0** (2026-08-14) — **Unified request/response log with configurable retention**: until now the only record of what happened was `user_activity_logs`, which stores non-GET **2xx** operations with no bodies — so when something failed you could see *that* the count went up, never *what was sent or what came back*. This release adds one queryable timeline covering **both directions**: every inbound API call (**including GETs and including 4xx/5xx**) with the user, client IP, status, duration and — redacted and size-capped — the request and response bodies; and every **outbound** HTTP call the backend makes, tagged with who it went to (ACME/Let's Encrypt, Cloudflare, GoDaddy, HAProxy stats, agents, the ACME diagnostics probe). Outbound rows **inherit the inbound request's id**, so one operator action and the CA/DNS calls it triggered read as a single trace — opening a failed *Request Certificate* shows the exact `POST /acme/new-order` and the CA's `429` body underneath it. Capture is a **pure-ASGI middleware that tees** the request and response streams rather than draining them, so no downstream handler is affected (notably the raw-body agent heartbeat), and rows are written by a **batching background writer** with a bounded queue, so the request path never waits on the database and a saturated logger drops rows visibly instead of blocking. Secrets never land: headers are an allowlist (`Authorization`/`Cookie` reduced to a presence marker), body keys and value shapes are redacted (passwords, tokens, API keys, private-key PEMs, JWTs), the **ACME JWS request body is never stored** (a stored `protected`+`signature` pair is a replayable credential — a summary is logged instead), DNS-provider errors record only the exception **type**, and the ACME HTTP-01 challenge endpoint is excluded so `key_authorization` is never captured. **Retention is operator-configurable** in *Settings → Request Log*: separate day counts for successful and failed rows (defaults 7 and 30) plus a hard row cap (500 000), whichever is reached first, pruned in **batches** under a Postgres advisory lock so a multi-million-row table cannot time out the delete or have every replica scan it at once. New **Request Log** page (`requestlog.read`) and retention/purge permission (`requestlog.manage`); `super_admin` and `security_admin` get both, `operator` gets read, `viewer` gets neither. **Successful agent polls are not logged** (`capture_agent_success`, default off; failures always are), which is what keeps the table's size a function of operator activity rather than of node count: measured at 2 424 bytes/row, a 200-node fleet would otherwise write 2.0M rows/day and reach the row cap in six hours, silently reducing the configured 7-day/30-day retention to a few hours for everything in the table. Cost is measured, not estimated: 27.7 µs per request on the hot path, 18.8 µs per row on the writer task, **0.096 % of one core at 500 nodes**. Adds one new table (`request_logs`) and its settings seed — SCHEMA_VERSION 11 → 12 (not 11: that number was taken by v1.10.4 while this was in review, and the version gate would have skipped the migration entirely on every existing install), auto-migrated, no existing table altered, no agent or rendered-config change. Kill switches: `REQUEST_LOG_ENABLED=false` (environment — the middleware is then never registered and costs nothing) or the `enabled` toggle in Settings (no restart).
- **v1.10.14** (2026-08-14) — **A converged node keeps acknowledging**: the deploy report is the server's only evidence that a member node applied its `keepalived.conf`, and it was sent on the write path alone. Once the rendered config was on disk the agent took the idempotency early return on every cycle and never reported again, so a **single lost report** — a backend restart, a 5xx, a network blip — left the VIP reading `SYNCING (0/n)` with an empty *Last ack* forever, while the node was demonstrably running the right config. Nothing would ever reconcile the two: the node was correct, the page was not, and the only way out was to change the rendered config so the agent wrote it again. The agent now re-asserts its state on the idempotent path too, which costs one request per node per ~2.5 minutes and touches nothing on the node — keepalived is not reloaded and the file is not rewritten. This is a long-standing gap from the original HA/VIP work, surfaced when acknowledgements were dropped for an unrelated reason in v1.10.12. Agent-script change: sync the script from Agent Management and let the agents upgrade. No schema or API change.
+121
View File
@@ -1,3 +1,124 @@
# Upgrade Notes — v1.12.0 (several keepalived VIPs on one node, several addresses on one VIP)
**Backend + UI change, schema `13` (delta), no agent change.** Agents of every version keep working;
no agent upgrade is needed.
- **Schema 12 → 13 is a delta.** A database at 12 adds two columns —
`vip_instances.additional_ips` (JSONB, default `[]`) and `vip_discoveries.analysis_version`
(SMALLINT, default `0`) — and nothing else runs. **Built-in roles are not re-seeded**, so role
customizations survive this upgrade (unlike v1.10.4 and v1.11.0). A database older than 12 runs the
full sequence as before, including the role re-seed.
- **Nodes with one VIP see no change.** Their `keepalived.conf` keeps the same bytes and hash, so the
agent does not rewrite or reload anything.
- **Check for nodes that already belong to several VIPs before upgrading.** Earlier versions refused
this when a VIP was saved, but without a lock, so two saves at the same moment could still create
it. Such a node switches from one VIP's file to the merged file on its first poll after the upgrade:
```sql
SELECT a.name, count(*) FROM vip_members m
JOIN vip_instances v ON v.id = m.vip_id JOIN agents a ON a.id = m.agent_id
WHERE v.is_active GROUP BY a.name HAVING count(*) > 1;
```
- **Rolling deploy:** do not add a second VIP to a node until **every backend replica** runs v1.12.0.
An older replica serves a node only one of its VIPs, so the node would alternate between two files.
An older replica applying a VIP also drops that VIP's additional addresses.
- **Rollback / downgrade:** first reduce every node to one VIP and remove the additional addresses
(edit, then Apply), then downgrade. The two new columns can stay; older versions ignore them.
- **How a node with several VIPs is served.** Each VIP is still rendered and applied on its own; the
delivery endpoint merges the node's applied VIPs into one file: `global_defs` once, then per VIP a
`vrrp_script chk_haproxy_<id>` and a `vrrp_instance VI_<id>`, ordered by VIP id. VRIDs are unique per
pool, so the instances never collide. One acknowledgement from the node converges all of its VIPs.
Applying two VIPs one after the other reloads keepalived twice; measured on keepalived 2.0.19,
2.1.5 and 2.2.8, a reload that adds or removes a VIP does not drop the other VIPs' addresses.
- **If the merge fails** (an applied snapshot that is not in the platform's own format), the node keeps
running what it has: the agent is told nothing new, and the node's members show
"cannot merge this node's VIPs …" with an ERROR status. Re-apply the VIPs to fix it.
- **Delivery errors no longer tear keepalived down.** A database error while serving
`keepalived-config` answered `not_configured`, which on a node whose file is ours meant teardown. It
now answers `unavailable`, which agents of every version ignore.
- **Additional addresses:** up to 19 per VIP (IPv4, prefix 1–32), on the VIP's interface on every node.
They must not be a node's IP or any address of another active VIP.
- **Live state is per node, not per VIP.** The MASTER/BACKUP column comes from the node's heartbeat,
which reports one state for the node. A node that is MASTER for one VIP and BACKUP for another shows
the same state in both rows; with several VIPs on a node, the heartbeat's config-based fallback can
also report MASTER on a BACKUP node. Check `ip addr` on the node when it matters. The VIPs themselves
fail over correctly; only the display is affected.
- **Adoption:** a `keepalived.conf` with several `vrrp_instance` blocks can no longer be adopted (it
replaces the whole file, so the other instances would be removed — earlier versions allowed it). An
adoption made before the upgrade from such a file is refused at Apply; reject it. An instance with
several addresses is now adoptable; the extra addresses become additional addresses. Files analysed
by an older version are re-reported once by v1.11.1+ agents; until then the list shows them with the
new blocker.
**Rollback:** see the downgrade rule above. With one VIP per node and no additional addresses,
rolling back is safe.
---
# Upgrade Notes — v1.11.3 (Config Import keeps `stick-table` and `stick` rules)
**Backend change, no schema change, no agent change.** No `SCHEMA_VERSION` bump and no agent
upgrade is needed.
- **`stick-table` is no longer dropped on Config Import.** Frontend `stick-table` and backend
`stick-table` / `stick on|match|store-request|store-response` lines are imported as-is into the
entity's **Options** field and listed as import warnings. Before this release they were discarded
silently, and a frontend that still used `track-sc` / `sc_*` counters rendered the default table
`stick-table type ip size 100k expire 30s store http_req_rate(10s)` instead of the imported one.
- **Re-import affected frontends and backends.** Entities imported with a custom `stick-table` on an
earlier version do not hold it; import the section again (the full frontend/backend block) to
store it. If you worked around the bug by putting a `stick-table` into Request Headers, remove it
from there after the re-import — the preview warns when both are present.
- **Options are merged by kind.** An import that carries only stick lines keeps the existing option
lines, and an import that carries only option lines keeps the existing stick lines. An import
carrying both replaces both, as before.
- **Generated lines are recognised, not stored.** The `stick-table type ip size 100k expire <N>s
store http_req_rate(10s)` the platform renders for Rate Limit / WAF `rate_limit` rules, and the
matching `track-sc0 src` / `sc_http_req_rate(0)` rules, are skipped (with an ℹ️ note) only where
the platform regenerates them. A `track-sc0` tracker on a frontend without Rate Limit or a WAF
`rate_limit` rule is now kept; earlier versions removed it from the preview.
- **One stick-table per backend.** HAProxy rejects a second `stick-table` in the same proxy as a
fatal error; backends now render the first one and log the skipped duplicate, as frontends already
did.
- **`peers`:** a stick-table that references `peers <name>` needs that section in each node's
`haproxy.cfg` above the first `defaults`/`frontend`/`backend`/`listen` section, which the agent
preserves. The platform does not manage `peers` sections.
- Configs without stick lines parse, import and render exactly as before.
**Rollback:** safe. No schema change. After a rollback the stick lines stay in Options and are still
rendered (Options is free-form); only the import behaviour returns to dropping them.
---
# Upgrade Notes — v1.11.2 (read-only roles read SSL; private keys need `ssl.download`)
**Backend and frontend change, no schema change, no agent change.** No `SCHEMA_VERSION` bump, so the
built-in roles are **not** re-seeded and no agent upgrade is needed.
- **A read-only user can read SSL certificates through the API and the UI.** `user_pool_access` is
never filled in by the UI, so every non-admin user created there has no pool access. The SSL list
with `?cluster_id=` and the SSL detail required pool access on read and answered **403**, while
clusters, frontends, backends, WAF rules, agents and VIPs were readable. Now a role that grants
`ssl.read` (or `waf.read` for `GET /api/waf/stats?cluster_id=`) reads that resource on every
cluster; a role without it still needs pool access. Write endpoints are unchanged.
- **The SSL private key is returned only with `ssl.download`.** Before this release
`GET /api/ssl/certificates/{id}` and the SSL change diff in Apply Management
(`GET /api/clusters/{id}/config-versions/{version_id}/diff`) returned the private key to **any**
authenticated user, including the `viewer` role. The key is now included only when the caller has
`ssl.download` **and** passes the pool check that guarded the detail route before. Otherwise the
detail returns `private_key_content: null` with `private_key_redacted: true`, and the diff shows a
redaction line in place of the key. Of the built-in roles, `super_admin`, `operator` and
`security_admin` hold `ssl.download`; `viewer` does not.
- **Check custom roles.** A custom role with `ssl.update` but without `ssl.download` no longer sees
the private key. Editing still works: the key field becomes optional and an empty field keeps the
stored key. Add `ssl.download` to the role if its users must see or copy the key.
- `GET /api/waf/stats` now returns 403/404 as such instead of wrapping them in a 500.
**Rollback:** safe. No schema or data change; reverting restores the previous behaviour, in which a
read-only user without pool access gets 403 on SSL reads and every authenticated user can read
private keys.
---
# Upgrade Notes — v1.11.1 (adoption cannot hide a node; Config Import on fresh installs)
**Agent-script change, no schema change.** No `SCHEMA_VERSION` bump, so the built-in roles are
+22
View File
@@ -299,6 +299,28 @@ async def check_user_permission(
logger.error(f"Error checking permission for user {user_id}: {e}")
return False
async def validate_cluster_read_access(
current_user: Dict[str, Any],
cluster_id: int,
conn,
resource: str,
validate_pool_access,
) -> bool:
"""Cluster gate for READ endpoints.
A role that grants ``<resource>.read`` (e.g. the built-in read-only
``viewer``) can read that resource on every cluster, the same way the
cluster, frontend, backend and WAF rule lists already behave. Without
the role permission the caller falls back to ``validate_pool_access``
(the router's ``user_pool_access`` check). Write endpoints keep using
the pool check directly; this helper must not be used for them.
"""
if await check_user_permission(current_user["id"], resource, "read", current_user=current_user):
if not await conn.fetchval("SELECT id FROM haproxy_clusters WHERE id = $1", cluster_id):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Cluster not found")
return True
return await validate_pool_access(current_user["id"], cluster_id, conn)
async def get_current_user_with_permissions(authorization: Optional[str] = None) -> Dict[str, Any]:
"""
Get current user with their permissions included
+56 -2
View File
@@ -1796,7 +1796,24 @@ async def ensure_agent_activity_logs_table():
# Additive + idempotent; no existing table is altered, agents never read this
# table. Same caveat as v1.10.4: this bump re-seeds the four built-in roles to
# their defaults, so export role customizations before upgrading.
SCHEMA_VERSION = 12
#
# v1.12.0 (issue #73 — several VIPs per node, several addresses per VIP): bumped
# 12 -> 13 as a DELTA. A database already at 12 runs only the steps listed in
# _delta_steps() — two additive columns — and keeps its role customizations; the full
# sequence (and with it the built-in role re-seed) runs only for databases older than
# _DELTA_BASE_VERSION or without a marker, exactly as before. Every delta step is ALSO
# called from _run_all_migrations_inner(), so both paths end at the same schema.
# A future change that alters seeded/role data needs a full bump: raise
# _DELTA_BASE_VERSION to the new SCHEMA_VERSION instead of adding a delta step.
SCHEMA_VERSION = 13
_DELTA_BASE_VERSION = 12
def _delta_steps():
"""(version, step) pairs a database at >= _DELTA_BASE_VERSION runs instead of the full
sequence. Each step takes the migration connection and raises on failure, so the version
marker is not written and the next start retries."""
return [(13, ensure_vip_multi_address_columns)]
async def run_all_migrations():
@@ -1845,7 +1862,13 @@ async def run_all_migrations():
logger.info(f"Schema already at version {applied_version} (>= {SCHEMA_VERSION}); skipping migration run.")
return
await _run_all_migrations_inner()
if applied_version is not None and applied_version >= _DELTA_BASE_VERSION:
for version, step in _delta_steps():
if version > applied_version:
logger.info(f"Applying schema delta {applied_version} -> {version}")
await step(lock_conn)
else:
await _run_all_migrations_inner()
try:
await lock_conn.execute("""
@@ -1935,6 +1958,7 @@ async def _run_all_migrations_inner():
# Issue #27 — HA/VIP Keepalived management (v1.7.0): two brand-new tables.
# MUST run after its FK targets (haproxy_cluster_pools/agents/users), all created above.
await ensure_vip_tables()
await ensure_vip_multi_address_columns()
# v1.9.0 — CSR creation: brand-new ssl_csrs table. FK-references
# ssl_certificates/users, both created above.
@@ -2408,6 +2432,36 @@ async def ensure_vip_tables():
if conn:
await close_database_connection(conn)
async def ensure_vip_multi_address_columns(conn=None):
"""v1.12.0 (issue #73). Additive + idempotent:
* vip_instances.additional_ips — the VIP's addresses beyond virtual_ip, as a JSON list
of {"address", "prefix_length"}. Empty for every existing VIP, which renders exactly
as before.
* vip_discoveries.analysis_version — which parser produced the stored analysis. Rows
written by an older parser read 0, and the agent is asked to re-report them once.
Raises on failure (unlike ensure_vip_tables): code reads these columns unconditionally,
so the schema version must not be recorded without them.
"""
own = conn is None
if own:
conn = await get_database_connection()
try:
# IF EXISTS: ensure_vip_tables() tolerates its own failure, and a missing VIP table
# must keep degrading only the VIP feature, not stop the backend from starting.
await conn.execute(
"ALTER TABLE IF EXISTS vip_instances ADD COLUMN IF NOT EXISTS additional_ips "
"JSONB NOT NULL DEFAULT '[]'::jsonb;")
await conn.execute(
"ALTER TABLE IF EXISTS vip_discoveries ADD COLUMN IF NOT EXISTS analysis_version "
"SMALLINT NOT NULL DEFAULT 0;")
logger.info("✅ VIP multi-address columns ensured (issue #73)")
finally:
if own:
await close_database_connection(conn)
async def add_ssl_certificate_id_to_backend_servers():
"""Add ssl_certificate_id column to backend_servers table for SSL certificate management"""
conn = None
+37
View File
@@ -50,6 +50,38 @@ def _validate_ipv4(v: str) -> str:
return s
# Issue #73: a VIP may carry further addresses in the same VRRP instance. 20 addresses in
# total keeps every advertisement within what VRRPv2 peers are known to accept.
MAX_ADDITIONAL_IPS = 19
class VIPAddressIn(BaseModel):
address: str
prefix_length: int = 24
@field_validator('address')
@classmethod
def _addr(cls, v):
return str(ipaddress.ip_address(_validate_ipv4(v)))
@field_validator('prefix_length')
@classmethod
def _prefix(cls, v):
if not isinstance(v, int) or not (1 <= v <= 32):
raise ValueError('prefix_length must be an integer between 1 and 32 (IPv4)')
return v
def _validate_additional_ips(extras: List['VIPAddressIn'], primary: Optional[str]) -> None:
if len(extras) > MAX_ADDITIONAL_IPS:
raise ValueError(f'a VIP may carry at most {MAX_ADDITIONAL_IPS} additional addresses')
addrs = [e.address for e in extras]
if len(set(addrs)) != len(addrs):
raise ValueError('each additional address may appear only once')
if primary is not None and str(ipaddress.ip_address(primary)) in addrs:
raise ValueError('an additional address must differ from virtual_ip')
class VIPMemberIn(BaseModel):
agent_id: int
network_interface: str
@@ -122,6 +154,7 @@ class VIPCreate(BaseModel):
auth_pass: Optional[str] = None # plaintext on the wire; stored Fernet-encrypted
use_unicast: bool = True
track_haproxy: bool = True
additional_ips: List[VIPAddressIn] = []
members: List[VIPMemberIn]
@field_validator('name')
@@ -172,6 +205,7 @@ class VIPCreate(BaseModel):
@model_validator(mode='after')
def _members_consistent(self):
_validate_members(self.members)
_validate_additional_ips(self.additional_ips, self.virtual_ip)
return self
@@ -187,6 +221,7 @@ class VIPUpdate(BaseModel):
auth_pass: Optional[str] = None # provide only to rotate; omit to keep existing
use_unicast: Optional[bool] = None
track_haproxy: Optional[bool] = None
additional_ips: Optional[List[VIPAddressIn]] = None # None keeps them; [] clears them
members: Optional[List[VIPMemberIn]] = None
@field_validator('name')
@@ -242,4 +277,6 @@ class VIPUpdate(BaseModel):
def _members_consistent(self):
if self.members is not None:
_validate_members(self.members)
if self.additional_ips is not None:
_validate_additional_ips(self.additional_ips, self.virtual_ip)
return self
+153 -24
View File
@@ -931,16 +931,17 @@ async def delete_agent(agent_id: int, authorization: str = Header(None)):
# with no way to be told to tear down (review MED-3). Make the operator remove it
# from the VIP first — that stages a clean PENDING change they can apply.
try:
vip_member = await conn.fetchrow(
# A node may belong to several VIPs (issue #73); name them all.
vip_names = [r["name"] for r in await conn.fetch(
"SELECT v.name FROM vip_members vm JOIN vip_instances v ON v.id = vm.vip_id "
"WHERE vm.agent_id = $1 AND v.is_active = TRUE LIMIT 1", agent_id)
"WHERE vm.agent_id = $1 AND v.is_active = TRUE ORDER BY v.name", agent_id)]
except Exception: # noqa: BLE001 — vip_* may not exist on older schemas; don't block delete
vip_member = None
if vip_member:
vip_names = []
if vip_names:
await close_database_connection(conn)
raise HTTPException(
status_code=409,
detail=(f"This node is a member of VIP '{vip_member['name']}'. Remove it from the "
detail=(f"This node is a member of VIP '{', '.join(vip_names)}'. Remove it from the "
f"VIP on the HA / VIP page (and apply) before deleting the agent."))
await conn.execute("DELETE FROM agents WHERE id = $1", agent_id)
@@ -2281,6 +2282,37 @@ async def get_agent_ssl_certificates(agent_name: str, since: Optional[str] = Non
raise HTTPException(status_code=500, detail=str(e))
async def _node_applied_vips(conn, agent_id: int):
"""Issue #73 — the active VIPs this node runs: every active membership that has an applied
snapshot, ordered by VIP id (the order compose_node_keepalived_conf writes them in)."""
return await conn.fetch("""
SELECT v.id AS vip_id, v.name AS vip_name, v.is_active, v.track_haproxy,
v.purge_on_teardown,
m.applied_config_content, m.applied_config_hash, m.takeover_expected_hash
FROM vip_members m JOIN vip_instances v ON v.id = m.vip_id
WHERE m.agent_id = $1 AND v.is_active = TRUE
AND m.applied_config_content IS NOT NULL AND m.applied_config_content <> ''
ORDER BY v.id
""", agent_id)
async def _flag_compose_error(conn, agent_id: int, vip_ids, exc: Exception) -> None:
"""Show a merge failure on the node's members. The agent never acks a status it does not
know, so without this the VIPs would sit at SYNCING with nothing saying why."""
if not vip_ids:
return
msg = f"cannot merge this node's VIPs into one keepalived.conf ({exc}); re-apply the VIPs"[:500]
try:
await conn.execute("""
UPDATE vip_members SET last_deploy_state='error', last_deploy_message=$3,
last_deploy_at=CURRENT_TIMESTAMP, updated_at=CURRENT_TIMESTAMP
WHERE agent_id=$1 AND vip_id = ANY($2::int[])
AND (last_deploy_state IS DISTINCT FROM 'error' OR last_deploy_message IS DISTINCT FROM $4)
""", agent_id, list(vip_ids), msg, msg)
except Exception as e: # noqa: BLE001
logger.error(f"could not record the keepalived compose failure: {e}")
@router.get("/{agent_name}/keepalived-config")
async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str] = Header(None)):
"""Issue #27 (v1.7.0) — deliver this agent's APPLIED keepalived snapshot, or a
@@ -2293,8 +2325,8 @@ async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str]
the agent API key is a SHARED/global install token, so the config is resolved by the
requested agent_name and a token/name mismatch is an advisory audit log, NOT a 403
(a hard 403 would break every VIP member whose name isn't the one row the shared token
resolves to — review HIGH-1). Any unexpected error degrades to not_configured (B-7) so
the agent stays inert; a node with no membership row always gets not_configured.
resolves to — review HIGH-1). Any unexpected error degrades to a status the agent ignores
(B-7) so the agent stays inert; a node with no membership row always gets not_configured.
"""
conn = None
try:
@@ -2303,6 +2335,7 @@ async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str]
# name mismatch — do not 403 (HIGH-1). Done before the agent lookup so an
# unauthenticated caller can't probe which agent names exist.
from auth_middleware import validate_agent_api_key
from services.keepalived_parser import ANALYSIS_VERSION
if not x_api_key:
raise HTTPException(status_code=401, detail="Agent API key required")
agent_auth = await validate_agent_api_key(x_api_key)
@@ -2333,7 +2366,12 @@ async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str]
-- good: the file never changes, so the agent never speaks again. Telling it
-- what we actually hold lets it recover on its own, with no extra request and
-- no one having to touch the node.
EXISTS (SELECT 1 FROM vip_discoveries vd WHERE vd.agent_id = a.id)
--
-- v1.12.0: only a discovery analysed by the CURRENT parser counts. An older
-- analysis reads as unknown, so the agent re-reports once and the stored
-- blockers/candidates catch up without anyone touching the node.
EXISTS (SELECT 1 FROM vip_discoveries vd
WHERE vd.agent_id = a.id AND vd.analysis_version >= $2)
AS discovery_known
FROM agents a
LEFT JOIN haproxy_clusters hc ON hc.pool_id = a.pool_id
@@ -2354,7 +2392,7 @@ async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str]
OR hc.keepalived_config_path = '/etc/keepalived/keepalived.conf'),
hc.id
LIMIT 1
""", agent_name)
""", agent_name, ANALYSIS_VERSION)
if not agent:
raise HTTPException(status_code=404, detail=f"Agent '{agent_name}' not found")
config_path = agent['keepalived_config_path'] or '/etc/keepalived/keepalived.conf'
@@ -2367,13 +2405,57 @@ async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str]
m.applied_config_content, m.applied_config_hash, m.takeover_expected_hash
FROM vip_members m JOIN vip_instances v ON v.id = m.vip_id
WHERE m.agent_id = $1
-- Active VIP first (an agent has at most one). With NO active VIP, pick the most
-- RECENTLY updated inactive membership so a teardown reflects the latest delete
-- (incl. its purge flag) — not a stale older VIP the node was once part of.
-- Active VIP first. With NO active VIP, pick the most RECENTLY updated inactive
-- membership so a teardown reflects the latest delete (incl. its purge flag) —
-- not a stale older VIP the node was once part of.
ORDER BY v.is_active DESC, v.updated_at DESC, v.id DESC
LIMIT 1
""", agent['id'])
# Issue #73: a node may belong to several active VIPs. Their applied snapshots are
# merged into the one keepalived.conf the node deploys; a single VIP is served exactly
# as before (same content, same hash).
# Any failure in this block must answer with a status the agent ignores: on a node
# whose file is ours the agent treats not_configured as "tear keepalived down". Here
# the members are also flagged, so the operator sees why the node is not converging.
node_vips = []
merged = None
if row and row['is_active']:
try:
node_vips = await _node_applied_vips(conn, agent['id'])
if len(node_vips) > 1:
from services.keepalived_config import (
build_haproxy_check_script, compose_node_keepalived_conf)
composed = compose_node_keepalived_conf(
[(r['vip_id'], r['applied_config_content']) for r in node_vips])
takeover = next((r['takeover_expected_hash'] for r in node_vips
if r['takeover_expected_hash']), None)
merged = {
"desired_state": "enabled",
"install_if_missing": True,
"vip_id": node_vips[0]['vip_id'],
"vip_ids": [r['vip_id'] for r in node_vips],
"vip_name": ", ".join(str(r['vip_name']) for r in node_vips),
"config_content": composed,
"config_hash": hashlib.md5(composed.encode("utf-8")).hexdigest(),
"check_script": build_haproxy_check_script()
if any(r['track_haproxy'] for r in node_vips) else "",
"allow_takeover": bool(takeover),
"takeover_expected_hash": takeover,
}
except Exception as ce: # noqa: BLE001
logger.error(f"keepalived compose failed for '{agent_name}': {ce}")
await _flag_compose_error(conn, agent['id'], [r['vip_id'] for r in node_vips], ce)
return {"agent_name": agent_name, "status": "compose_error", "config_path": config_path,
"discovery_known": bool(agent["discovery_known"]), "keepalived": None}
if merged is not None:
return {"agent_name": agent_name, "status": "available", "config_path": config_path,
"discovery_known": bool(agent["discovery_known"]), "keepalived": merged}
if len(node_vips) == 1:
# The most recently updated active VIP may be a pending one with nothing applied
# yet; serving it would read as not_configured and tear down the running VIP.
row = node_vips[0]
if not row:
return {"agent_name": agent_name, "status": "not_configured", "config_path": config_path, "discovery_known": bool(agent["discovery_known"]), "keepalived": None}
if not row['is_active']:
@@ -2413,8 +2495,10 @@ async def get_agent_keepalived_config(agent_name: str, x_api_key: Optional[str]
raise
except Exception as e:
logger.error(f"keepalived-config delivery failed for '{agent_name}': {e}")
# Degrade to no-op rather than 500 (B-7) — keeps the fleet inert on any error.
return {"agent_name": agent_name, "status": "not_configured",
# Degrade to a no-op rather than 500 (B-7). The status must be one the agent does not
# act on: not_configured is a teardown on a node whose file is ours, so a transient
# database error would have taken a running VIP down.
return {"agent_name": agent_name, "status": "unavailable",
"config_path": "/etc/keepalived/keepalived.conf", "keepalived": None}
finally:
if conn:
@@ -2476,12 +2560,54 @@ async def agent_keepalived_status(agent_name: str, status_data: dict, x_api_key:
WHERE agent_id=$1
""", agent['id'], state, message, config_hash, config_hash)
else:
await conn.execute(f"""
UPDATE vip_members SET last_deploy_state=$3, last_deploy_message=$4,
last_deploy_hash=$5, last_deploy_at=CURRENT_TIMESTAMP, updated_at=CURRENT_TIMESTAMP,
{_retire_takeover.format(p="$6")}
WHERE agent_id=$1 AND vip_id=$2
""", agent['id'], int(vip_id), state, message, config_hash, config_hash)
# Issue #73: a node running several VIPs acks the merged file once, with the first
# VIP's id and the merged file's hash. Spread the ack over every VIP in that file;
# each member is converged against its OWN applied hash.
node_vips = await _node_applied_vips(conn, agent['id'])
node_vip_ids = [r['vip_id'] for r in node_vips]
converged = False
if len(node_vips) > 1 and int(vip_id) in node_vip_ids:
from services.keepalived_config import compose_node_keepalived_conf
try:
composed = compose_node_keepalived_conf(
[(r['vip_id'], r['applied_config_content']) for r in node_vips])
composed_hash = hashlib.md5(composed.encode("utf-8")).hexdigest()
except Exception: # noqa: BLE001 — unmergeable means not converged, never a 500
composed_hash = None
converged = bool(composed_hash) and config_hash == composed_hash
if converged:
await conn.execute("""
UPDATE vip_members SET last_deploy_state=$3, last_deploy_message=$4,
last_deploy_hash=applied_config_hash, last_deploy_at=CURRENT_TIMESTAMP,
updated_at=CURRENT_TIMESTAMP, takeover_expected_hash=NULL
WHERE agent_id=$1 AND vip_id = ANY($2::int[])
""", agent['id'], node_vip_ids, state, message)
else:
await conn.execute("""
UPDATE vip_members SET last_deploy_state=$3, last_deploy_message=$4,
last_deploy_hash=$5, last_deploy_at=CURRENT_TIMESTAMP, updated_at=CURRENT_TIMESTAMP
WHERE agent_id=$1 AND vip_id = ANY($2::int[])
""", agent['id'], node_vip_ids, state, message, config_hash)
else:
await conn.execute(f"""
UPDATE vip_members SET last_deploy_state=$3, last_deploy_message=$4,
last_deploy_hash=$5, last_deploy_at=CURRENT_TIMESTAMP, updated_at=CURRENT_TIMESTAMP,
{_retire_takeover.format(p="$6")}
WHERE agent_id=$1 AND vip_id=$2
""", agent['id'], int(vip_id), state, message, config_hash, config_hash)
converged = any(r['vip_id'] == int(vip_id) and r['applied_config_hash'] == config_hash
for r in node_vips)
if state == "enabled" and converged:
# The file the node now runs holds only its active VIPs, so a deleted VIP's
# membership on this node is torn down even though no teardown was sent for it.
await conn.execute("""
UPDATE vip_members m SET last_deploy_state='disabled',
last_deploy_message='removed from this node''s keepalived.conf',
last_deploy_at=CURRENT_TIMESTAMP, updated_at=CURRENT_TIMESTAMP
FROM vip_instances v
WHERE v.id = m.vip_id AND m.agent_id=$1 AND v.is_active = FALSE
AND m.last_deploy_state IS DISTINCT FROM 'disabled'
""", agent['id'])
return {"status": "ok"}
except HTTPException:
raise
@@ -2539,7 +2665,8 @@ async def agent_keepalived_discovery(agent_name: str, payload: dict, x_api_key:
is_managed = bool(payload.get("is_managed"))
config_hash = hashlib.md5(content.encode("utf-8", "replace")).hexdigest()
from services.keepalived_parser import analyse_keepalived_conf, KeepalivedParseError
from services.keepalived_parser import (
ANALYSIS_VERSION, analyse_keepalived_conf, KeepalivedParseError)
from services.keepalived_config import encrypt_vrrp_secret
parse_error = None
@@ -2562,8 +2689,8 @@ async def agent_keepalived_discovery(agent_name: str, payload: dict, x_api_key:
await conn.execute("""
INSERT INTO vip_discoveries
(agent_id, config_path, config_hash, is_managed, raw_config_masked,
auth_pass_encrypted, analysis, parse_error, reported_at)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,CURRENT_TIMESTAMP)
auth_pass_encrypted, analysis, parse_error, analysis_version, reported_at)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,CURRENT_TIMESTAMP)
ON CONFLICT (agent_id) DO UPDATE SET
config_path = EXCLUDED.config_path,
config_hash = EXCLUDED.config_hash,
@@ -2572,9 +2699,11 @@ async def agent_keepalived_discovery(agent_name: str, payload: dict, x_api_key:
auth_pass_encrypted = EXCLUDED.auth_pass_encrypted,
analysis = EXCLUDED.analysis,
parse_error = EXCLUDED.parse_error,
analysis_version = EXCLUDED.analysis_version,
reported_at = CURRENT_TIMESTAMP
""", agent['id'], config_path, config_hash, is_managed, masked, auth_enc,
json.dumps(analysis) if analysis is not None else None, parse_error)
json.dumps(analysis) if analysis is not None else None, parse_error,
ANALYSIS_VERSION)
return {"status": "recorded", "config_hash": config_hash}
except HTTPException:
raise
+18 -2
View File
@@ -2646,9 +2646,25 @@ async def get_config_version_diff(cluster_id: int, version_id: int, authorizatio
cert_file_content_lines.append("")
cert_file_content_lines.append("# Private Key Content:")
# Add private key content
# Add private key content. Only ssl.download holders with
# pool access to this cluster see the key; for everyone else
# (e.g. the read-only viewer role) it is redacted.
if ssl_cert['private_key_content']:
cert_file_content_lines.extend(ssl_cert['private_key_content'].strip().split('\n'))
from auth_middleware import check_user_permission
can_view_private_key = await check_user_permission(
current_user['id'], "ssl", "download", current_user=current_user
)
if can_view_private_key:
try:
await validate_user_cluster_access(current_user['id'], cluster_id, conn)
except HTTPException as access_error:
if access_error.status_code != 403:
raise
can_view_private_key = False
if can_view_private_key:
cert_file_content_lines.extend(ssl_cert['private_key_content'].strip().split('\n'))
else:
cert_file_content_lines.append("# [redacted: ssl.download permission required]")
# Add chain content if exists
if ssl_cert['chain_content'] and ssl_cert['chain_content'].strip():
+128 -25
View File
@@ -17,6 +17,13 @@ from utils.config_templates import (
generate_config_from_template, TemplateCategory
)
from utils.haproxy_config_parser import parse_haproxy_config
from utils.stick_table_import import (
drop_platform_stick_tables,
frontend_uses_counter,
merge_options,
stick_advisories,
strip_auto_headers,
)
from utils.logging_config import log_with_correlation, PerformanceLogger
from auth_middleware import get_current_user_from_token, require_authenticated_user
from database.connection import get_database_connection, close_database_connection
@@ -978,26 +985,61 @@ async def parse_bulk_config(
# field, WAF rules table) that the parser cannot distinguish from user-defined
# configuration. Strip all auto-managed patterns so the preview and comparison
# only reflect user-defined configuration.
def _is_auto_header(line):
s = line.strip()
if "is_acme_challenge" in s:
return True
if "track-sc0 src" in s:
return True
if "sc_http_req_rate(0)" in s:
return True
if s.startswith("http-request") and " waf_" in s:
return True
return False
def _strip_auto_headers(headers_str):
if not headers_str:
return None
lines = [l for l in headers_str.split("\n") if not _is_auto_header(l)]
return "\n".join(lines) if lines else None
#
# Issue #72: the rate-limit lines (track-sc0 src / sc_http_req_rate(0)) are
# generator-owned only when the generator will emit them again for this
# frontend: its rate_limit is set or an active rate_limit WAF rule applies
# (same rule selection as services/haproxy_config.py). Otherwise they are
# the operator's own and were silently deleted on update. Errors fall back
# to "regenerates", i.e. the previous always-strip behaviour.
existing_frontend_rows = {}
platform_regenerates = {}
try:
global_rate_limit_waf = bool(await conn.fetchval("""
SELECT EXISTS (
SELECT 1 FROM waf_rules w
WHERE w.is_active = TRUE AND w.rule_type = 'rate_limit'
AND (w.cluster_id = $1 OR w.cluster_id IS NULL)
AND NOT EXISTS (
SELECT 1 FROM frontend_waf_rules fwr WHERE fwr.waf_rule_id = w.id
)
)
""", request.cluster_id))
except Exception as _waf_err:
logger.warning(f"BULK IMPORT: global rate_limit WAF lookup failed, assuming present: {_waf_err}")
global_rate_limit_waf = True
for frontend in frontends_data:
existing_frontend_rows[frontend["name"]] = await conn.fetchrow("""
SELECT * FROM frontends
WHERE name = $1 AND cluster_id = $2
ORDER BY is_active DESC
""", frontend["name"], request.cluster_id)
existing_frontend_ids = [r["id"] for r in existing_frontend_rows.values() if r]
frontends_with_rate_limit_waf = set()
if existing_frontend_ids and not global_rate_limit_waf:
try:
rows = await conn.fetch("""
SELECT DISTINCT fw.frontend_id
FROM waf_rules w
JOIN frontend_waf_rules fw ON w.id = fw.waf_rule_id
WHERE w.is_active = TRUE AND w.rule_type = 'rate_limit'
AND fw.frontend_id = ANY($1::int[])
""", existing_frontend_ids)
frontends_with_rate_limit_waf = {r["frontend_id"] for r in rows}
except Exception as _waf_err:
logger.warning(f"BULK IMPORT: assigned rate_limit WAF lookup failed, assuming present: {_waf_err}")
frontends_with_rate_limit_waf = set(existing_frontend_ids)
for frontend in frontends_data:
row = existing_frontend_rows[frontend["name"]]
platform_regenerates[frontend["name"]] = bool(
global_rate_limit_waf
or (row and row["rate_limit"])
or (row and row["id"] in frontends_with_rate_limit_waf)
)
backends_data = [b for b in backends_data if b["name"] != "_acme_challenge_backend"]
stick_policy_warnings = []
for frontend in frontends_data:
frontend["acl_rules"] = [
r for r in (frontend.get("acl_rules") or [])
@@ -1007,7 +1049,9 @@ async def parse_bulk_config(
r for r in (frontend.get("use_backend_rules") or [])
if "_acme_challenge_backend" not in r
]
frontend["request_headers"] = _strip_auto_headers(frontend.get("request_headers"))
fe_regenerates = platform_regenerates[frontend["name"]]
frontend["request_headers"] = strip_auto_headers(
frontend.get("request_headers"), fe_regenerates)
if frontend.get("tcp_request_rules"):
lines = [
l for l in frontend["tcp_request_rules"].split("\n")
@@ -1022,6 +1066,31 @@ async def parse_bulk_config(
else:
frontend["default_backend"] = None
# Issue #72: a generator-shaped stick-table that the generator will render
# itself must not be stored, or it would freeze later rate_limit / WAF
# window changes (Options are emitted first and win the dedup).
fe_uses_counter = frontend_uses_counter([
frontend.get(k) for k in (
"options", "request_headers", "response_headers", "tcp_request_rules",
"acl_rules", "use_backend_rules", "filters", "log_format",
)
])
frontend["options"], dropped_tables = drop_platform_stick_tables(
frontend.get("options"), fe_regenerates, fe_uses_counter)
for table_line in dropped_tables:
parse_result.warnings = [
w for w in parse_result.warnings
if not (w.startswith(f"Frontend '{frontend['name']}': stick-table imported as-is")
and table_line in w)
]
stick_policy_warnings.append(
f"ℹ️ Frontend '{frontend['name']}': stick-table '{table_line}' matches the "
f"table HAProxy OpenManager generates for Rate Limit / WAF rate_limit rules "
f"and is generated automatically for this frontend, so it is not stored."
)
parse_result.warnings.extend(stick_policy_warnings)
parse_result.warnings = [
w for w in parse_result.warnings if "_acme_challenge_backend" not in w
]
@@ -1166,12 +1235,27 @@ async def parse_bulk_config(
update_servers = 0
for frontend in frontends_data:
existing = await conn.fetchrow("""
SELECT * FROM frontends
WHERE name = $1 AND cluster_id = $2
ORDER BY is_active DESC
""", frontend["name"], request.cluster_id)
existing = existing_frontend_rows[frontend["name"]]
imported_options = frontend.get("options")
if existing:
frontend["options"] = merge_options(imported_options, existing.get("options"))
def _final(field):
# Bulk-create only overwrites a free-form field when the import carries it.
if frontend.get(field):
return frontend.get(field)
return existing.get(field) if existing else None
enhanced_warnings.extend(stick_advisories(
"Frontend", frontend["name"], frontend.get("options"), imported_options,
{
"Request Headers": _final("request_headers"),
"Response Headers": _final("response_headers"),
"TCP Request Rules": _final("tcp_request_rules"),
},
platform_regenerates[frontend["name"]],
))
if existing:
# Check if any field has actually changed AND track which fields changed
has_changes = False
@@ -1202,7 +1286,8 @@ async def parse_bulk_config(
has_changes = True
changes["maxconn"] = {"old": existing["maxconn"], "new": frontend["maxconn"]}
parsed_rh = frontend.get("request_headers")
db_rh = _strip_auto_headers(existing["request_headers"])
db_rh = strip_auto_headers(
existing["request_headers"], platform_regenerates[frontend["name"]])
if parsed_rh and parsed_rh != db_rh:
has_changes = True
changes["request_headers"] = {"old": db_rh, "new": parsed_rh}
@@ -1292,6 +1377,24 @@ async def parse_bulk_config(
WHERE name = $1 AND cluster_id = $2
ORDER BY is_active DESC
""", backend["name"], request.cluster_id)
imported_options = backend.get("options")
if existing:
backend["options"] = merge_options(imported_options, existing.get("options"))
def _final_be(field):
if backend.get(field):
return backend.get(field)
return existing.get(field) if existing else None
enhanced_warnings.extend(stick_advisories(
"Backend", backend["name"], backend.get("options"), imported_options,
{
"Request Headers": _final_be("request_headers"),
"Response Headers": _final_be("response_headers"),
},
False,
))
if existing:
# Check if any field has actually changed AND track which fields changed
+41 -9
View File
@@ -9,7 +9,11 @@ from datetime import datetime, timezone
# Import database and models
from database.connection import get_database_connection, close_database_connection
from auth_middleware import get_current_user_from_token, require_authenticated_user
from auth_middleware import (
get_current_user_from_token,
require_authenticated_user,
validate_cluster_read_access,
)
from models.ssl import SSLCertificate, SSLCertificateCreate, SSLCertificateUpdate, SSLCertificateResponse
from utils.ssl_parser import parse_ssl_certificate, validate_private_key, validate_certificate_chain, format_certificate_info
from utils.activity_log import log_user_activity
@@ -226,10 +230,14 @@ async def get_ssl_certificates(
try:
conn = await get_database_connection()
if current_user and cluster_id:
# Cluster-scoped enumeration must respect cluster access.
# Re-raises HTTPException(403/404) cleanly; finally below
# releases the connection.
await validate_user_cluster_access(current_user["id"], cluster_id, conn)
# Cluster-scoped enumeration must respect cluster access:
# an ssl.read role reads every cluster, anyone else needs
# validate_user_cluster_access (user_pool_access). Re-raises
# HTTPException(403/404) cleanly; finally below releases the
# connection.
await validate_cluster_read_access(
current_user, cluster_id, conn, "ssl", validate_user_cluster_access
)
# First check if ssl_certificates table exists
try:
@@ -761,10 +769,28 @@ async def get_ssl_certificate(cert_id: int, authorization: str = Header(None)):
await close_database_connection(conn)
raise HTTPException(status_code=404, detail="SSL certificate not found")
# Validate user access to cluster
# Validate user access to cluster (an ssl.read role reads every cluster)
if certificate['cluster_id']:
await validate_user_cluster_access(current_user['id'], certificate['cluster_id'], conn)
await validate_cluster_read_access(
current_user, certificate['cluster_id'], conn, "ssl", validate_user_cluster_access
)
# The private key is returned only to ssl.download holders who also
# pass the pool check that guarded this route before read access was
# widened. Everyone else (e.g. the read-only viewer role) gets the
# certificate without its key.
from auth_middleware import check_user_permission
can_view_private_key = await check_user_permission(
current_user['id'], "ssl", "download", current_user=current_user
)
if can_view_private_key and certificate['cluster_id']:
try:
await validate_user_cluster_access(current_user['id'], certificate['cluster_id'], conn)
except HTTPException as access_error:
if access_error.status_code != 403:
raise
can_view_private_key = False
# For now, assume no pending changes (SSL config versions need separate implementation)
pending_versions = 0
@@ -805,7 +831,13 @@ async def get_ssl_certificate(cert_id: int, authorization: str = Header(None)):
cert_dict['all_domains'] = []
cert_dict['has_pending_config'] = pending_versions > 0
cert_dict['private_key_redacted'] = (
not can_view_private_key and bool(cert_dict.get('private_key_content'))
)
if not can_view_private_key:
cert_dict['private_key_content'] = None
cert_dict['usage_count'] = len(used_by_frontends) + len(used_by_servers)
cert_dict['used_by_frontends'] = [
{'id': f['id'], 'name': f['name'], 'cluster_name': f['cluster_name']}
+157 -60
View File
@@ -16,6 +16,7 @@ Every DB access is wrapped so a (pathological) missing vip_* relation degrades t
empty/None result instead of a 500 (B-7) — preserving the fleet-wide no-op guarantee.
"""
import hashlib
import ipaddress
import json
import logging
import re
@@ -171,7 +172,7 @@ async def render_vip_config_masked(conn, vip_id: int):
vip_dict = {"id": v["id"], "name": v["name"], "virtual_ip": v["virtual_ip"],
"prefix_length": v["prefix_length"], "virtual_router_id": v["virtual_router_id"],
"advert_int": v["advert_int"], "use_unicast": v["use_unicast"],
"track_haproxy": v["track_haproxy"]}
"track_haproxy": v["track_haproxy"], "additional_ips": _extra_ips(v["additional_ips"])}
member_dicts = [{"role": m["role"], "priority": m["priority"],
"network_interface": m["network_interface"], "agent_id": m["agent_id"],
"ip_address": str(m["ip_address"]) if m["ip_address"] else ""} for m in members]
@@ -241,7 +242,8 @@ async def _capture_vip_pending_state(conn, vip_id: int):
"vip": {"name": v["name"], "description": v["description"], "virtual_ip": v["virtual_ip"],
"prefix_length": v["prefix_length"], "virtual_router_id": v["virtual_router_id"],
"advert_int": v["advert_int"], "use_unicast": v["use_unicast"],
"track_haproxy": v["track_haproxy"], "auth_pass_encrypted": v["auth_pass_encrypted"]},
"track_haproxy": v["track_haproxy"], "auth_pass_encrypted": v["auth_pass_encrypted"],
"additional_ips": _extra_ips(v["additional_ips"])},
"members": [{"agent_id": m["agent_id"], "network_interface": m["network_interface"],
"role": m["role"], "priority": m["priority"]} for m in members],
}
@@ -282,37 +284,31 @@ async def restore_vip_from_rejected_version(conn, version_id: int):
if not await conn.fetchrow("SELECT 1 FROM vip_instances WHERE id=$1", vip_id):
return "the VIP no longer exists — re-create it from the HA/VIP page"
sv, sm = ps["vip"], ps.get("members", [])
# One-VIP-per-agent must STILL hold after reactivation: if a member was meanwhile added
# to another active VIP (while this one was rejected/soft-deleted), undoing here would
# create a double-membership — and since the delivery endpoint serves one VIP per agent,
# reactivating this (never-applied → not_configured) VIP would tear down the other live
# VIP on that node. Block it with a clear message (review round-3 FINDING 2).
member_ids = [m["agent_id"] for m in sm]
if member_ids:
dup = await conn.fetchrow(
"SELECT a.name AS agent_name, v.name AS vip_name FROM vip_members vm "
"JOIN vip_instances v ON v.id = vm.vip_id JOIN agents a ON a.id = vm.agent_id "
"WHERE vm.agent_id = ANY($1) AND v.is_active = TRUE AND v.id <> $2 LIMIT 1",
member_ids, vip_id)
if dup:
return (f"node '{dup['agent_name']}' now belongs to active VIP '{dup['vip_name']}' — "
f"remove it there first, then undo")
# Issue #73: a member that meanwhile joined another active VIP keeps it. The delivery
# endpoint serves a node every active VIP that has an applied snapshot, so reactivating
# this one (never applied, or applied earlier) cannot displace the other.
# Preserve the running (applied) snapshot for members that remain, so undo of an edit
# never momentarily flips a live node to not_configured (T-1).
prev = {r["agent_id"]: r for r in await conn.fetch(
"SELECT agent_id, applied_config_content, applied_config_hash, last_deploy_state, "
"last_deploy_message, last_deploy_hash, last_deploy_at FROM vip_members WHERE vip_id=$1", vip_id)}
# Captured before v1.12.0 → no additional addresses.
extras = _extra_ips(sv.get("additional_ips") or [])
try:
async with conn.transaction():
await conn.execute("SELECT pg_advisory_xact_lock($1, $2)", *_VIP_ADDRESS_LOCK)
reason = await _address_clash(conn, sv["virtual_ip"], extras, vip_id)
if reason:
return f"cannot restore — {reason}"
await conn.execute("""
UPDATE vip_instances SET name=$2, description=$3, virtual_ip=$4, prefix_length=$5,
virtual_router_id=$6, advert_int=$7, use_unicast=$8, track_haproxy=$9,
auth_pass_encrypted=$10, is_active=TRUE, last_config_status='PENDING',
updated_at=CURRENT_TIMESTAMP
additional_ips=$11::jsonb, updated_at=CURRENT_TIMESTAMP
WHERE id=$1
""", vip_id, sv["name"], sv.get("description"), sv["virtual_ip"], sv["prefix_length"],
sv["virtual_router_id"], sv["advert_int"], sv["use_unicast"], sv["track_haproxy"],
sv.get("auth_pass_encrypted"))
sv.get("auth_pass_encrypted"), json.dumps(extras))
await conn.execute("DELETE FROM vip_members WHERE vip_id=$1", vip_id)
for mm in sm:
o = prev.get(mm["agent_id"])
@@ -395,7 +391,7 @@ async def list_vips(cluster_id: Optional[int] = None, authorization: str = Heade
if cluster_id is not None:
vips = await conn.fetch(f"""
SELECT v.id, v.name, v.description, v.pool_id, v.virtual_ip, v.prefix_length,
v.virtual_router_id, v.advert_int, v.use_unicast, v.track_haproxy,
v.virtual_router_id, v.advert_int, v.use_unicast, v.track_haproxy, v.additional_ips,
v.is_active, v.last_config_status, v.pending_delete,
(v.auth_pass_encrypted IS NOT NULL) AS auth_pass_set,
v.created_at, v.updated_at, p.name AS pool_name
@@ -408,7 +404,7 @@ async def list_vips(cluster_id: Optional[int] = None, authorization: str = Heade
else:
vips = await conn.fetch(f"""
SELECT v.id, v.name, v.description, v.pool_id, v.virtual_ip, v.prefix_length,
v.virtual_router_id, v.advert_int, v.use_unicast, v.track_haproxy,
v.virtual_router_id, v.advert_int, v.use_unicast, v.track_haproxy, v.additional_ips,
v.is_active, v.last_config_status, v.pending_delete,
(v.auth_pass_encrypted IS NOT NULL) AS auth_pass_set,
v.created_at, v.updated_at, p.name AS pool_name
@@ -434,6 +430,7 @@ async def list_vips(cluster_id: Optional[int] = None, authorization: str = Heade
v["last_config_status"], members, v["pending_delete"], v["is_active"])
result.append({
**dict(v),
"additional_ips": _extra_ips(v["additional_ips"]),
# Convergence-aware status for the table (issue #27 follow-up); the raw
# last_config_status is kept above for the PENDING gate / Apply button.
"deploy_status": deploy_status,
@@ -505,7 +502,7 @@ async def get_vip(vip_id: int, authorization: str = Header(None)):
try:
v = await conn.fetchrow("""
SELECT v.id, v.name, v.description, v.pool_id, v.virtual_ip, v.prefix_length,
v.virtual_router_id, v.advert_int, v.use_unicast, v.track_haproxy,
v.virtual_router_id, v.advert_int, v.use_unicast, v.track_haproxy, v.additional_ips,
v.is_active, v.last_config_status,
(v.auth_pass_encrypted IS NOT NULL) AS auth_pass_set,
v.created_at, v.updated_at
@@ -520,7 +517,8 @@ async def get_vip(vip_id: int, authorization: str = Header(None)):
FROM vip_members m LEFT JOIN agents a ON a.id = m.agent_id
WHERE m.vip_id = $1 ORDER BY m.priority DESC
""", vip_id)
return {**dict(v), "members": [dict(m) for m in members]}
return {**dict(v), "additional_ips": _extra_ips(v["additional_ips"]),
"members": [dict(m) for m in members]}
finally:
await close_database_connection(conn)
@@ -540,24 +538,9 @@ async def _validate_members_against_pool(conn, pool_id: int, members: List, vip_
if row["pool_id"] != pool_id:
raise HTTPException(status_code=400,
detail=f"agent {m.agent_id} is not in pool {pool_id}")
# One active VIP per agent (v1): a node deploys a single keepalived.conf, and the
# delivery endpoint serves one VIP per agent — a second active membership would never
# converge (stuck SYNCING) instead of erroring. Reject it up front (review MED-2).
agent_ids = [m.agent_id for m in members]
if agent_ids:
dq = ("SELECT a.name AS agent_name, v.name AS vip_name "
"FROM vip_members vm JOIN vip_instances v ON v.id = vm.vip_id "
"JOIN agents a ON a.id = vm.agent_id "
"WHERE vm.agent_id = ANY($1) AND v.is_active = TRUE")
dparams = [agent_ids]
if exclude_vip_id is not None:
dq += " AND v.id <> $2"
dparams.append(exclude_vip_id)
dup = await conn.fetchrow(dq + " LIMIT 1", *dparams)
if dup:
raise HTTPException(status_code=409,
detail=f"node '{dup['agent_name']}' is already a member of VIP "
f"'{dup['vip_name']}' — a node can belong to only one VIP")
# Issue #73: a node may belong to several VIPs. The delivery endpoint merges their applied
# snapshots into the node's one keepalived.conf; the pool-scoped VRID uniqueness
# (uq_vip_vrid_active) keeps their VRRP instances apart.
# VIP must not be an existing agent's primary IP (INET cast, SQL-1)
clash = await conn.fetchval("SELECT 1 FROM agents WHERE ip_address = $1::inet LIMIT 1", vip_virtual_ip)
if clash:
@@ -572,6 +555,57 @@ async def _validate_members_against_pool(conn, pool_id: int, members: List, vip_
raise HTTPException(status_code=409, detail=f"{vip_virtual_ip} is already used by another VIP")
# Serialises every write that sets a VIP's addresses. The primary address has a partial unique
# index; the additional ones live in a JSON list no index can cover, so two concurrent writes
# could otherwise both pass the clash check below with the same address.
_VIP_ADDRESS_LOCK = (18181873, 0)
def _norm_ip(value) -> str:
try:
return str(ipaddress.ip_address(str(value).strip()))
except ValueError:
return str(value)
def _extra_ips(value) -> List[dict]:
"""vip_instances.additional_ips (JSONB; asyncpg hands it over as a string) as a list."""
items = _jsonb_list(value)
return [{"address": str(i["address"]), "prefix_length": int(i["prefix_length"])}
for i in items if isinstance(i, dict) and i.get("address")]
async def _address_clash(conn, primary: str, extras: List[dict],
exclude_vip_id: Optional[int]) -> Optional[str]:
"""Issue #73 — why these addresses cannot be used, or None. Covers what the primary-address
checks do not: every additional address against node IPs and every active VIP's addresses,
and the primary against the other VIPs' additional addresses. Call it inside the write's
transaction after taking _VIP_ADDRESS_LOCK."""
other = ("SELECT name FROM vip_instances WHERE is_active = TRUE "
"AND ($2::int IS NULL OR id <> $2::int) AND {cond} LIMIT 1")
in_extras = "additional_ips @> jsonb_build_array(jsonb_build_object('address', $1::text))"
hit = await conn.fetchval(other.format(cond=in_extras), primary, exclude_vip_id)
if hit:
return f"{primary} is already an additional address of VIP '{hit}'"
for e in extras:
addr = e["address"]
if await conn.fetchval("SELECT 1 FROM agents WHERE ip_address = $1::inet LIMIT 1", addr):
return f"{addr} is already a node's IP"
hit = await conn.fetchval(
other.format(cond=f"(virtual_ip = $1::text OR {in_extras})"), addr, exclude_vip_id)
if hit:
return f"{addr} is already used by VIP '{hit}'"
return None
async def _lock_and_check_addresses(conn, primary: str, extras: List[dict],
exclude_vip_id: Optional[int]) -> None:
await conn.execute("SELECT pg_advisory_xact_lock($1, $2)", *_VIP_ADDRESS_LOCK)
reason = await _address_clash(conn, primary, extras, exclude_vip_id)
if reason:
raise HTTPException(status_code=409, detail=reason)
@router.post("")
async def create_vip(payload: VIPCreate, request: Request, authorization: str = Header(None)):
current_user = await _require(authorization, "create")
@@ -583,22 +617,25 @@ async def create_vip(payload: VIPCreate, request: Request, authorization: str =
await _validate_members_against_pool(conn, payload.pool_id, payload.members, payload.virtual_ip)
enc = encrypt_vrrp_secret(payload.auth_pass) if payload.auth_pass else None
extras = [e.model_dump() for e in payload.additional_ips]
# Allocate VRID + insert, retrying once on a unique-violation race (B-1).
last_err = None
for _attempt in range(2):
vrid = await _alloc_free_vrid(conn, payload.pool_id, payload.virtual_router_id)
try:
async with conn.transaction():
await _lock_and_check_addresses(conn, payload.virtual_ip, extras, None)
vip_id = await conn.fetchval("""
INSERT INTO vip_instances
(name, description, pool_id, virtual_ip, prefix_length, virtual_router_id,
advert_int, auth_pass_encrypted, use_unicast, track_haproxy,
is_active, last_config_status, created_by)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,TRUE,'PENDING',$11)
is_active, last_config_status, created_by, additional_ips)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,TRUE,'PENDING',$11,$12::jsonb)
RETURNING id
""", payload.name, payload.description, payload.pool_id, payload.virtual_ip,
payload.prefix_length, vrid, payload.advert_int, enc,
payload.use_unicast, payload.track_haproxy, current_user["id"])
payload.use_unicast, payload.track_haproxy, current_user["id"],
json.dumps(extras))
for m in payload.members:
await conn.execute("""
INSERT INTO vip_members (vip_id, agent_id, network_interface, role, priority)
@@ -606,6 +643,8 @@ async def create_vip(payload: VIPCreate, request: Request, authorization: str =
""", vip_id, m.agent_id, m.network_interface, m.role, m.priority)
last_err = None
break
except HTTPException:
raise
except Exception as ie: # noqa: BLE001
if "unique" in str(ie).lower() or "duplicate" in str(ie).lower():
last_err = ie
@@ -650,10 +689,17 @@ async def update_vip(vip_id: int, payload: VIPUpdate, request: Request, authoriz
"SELECT 1 FROM vip_instances WHERE virtual_ip=$1 AND is_active=TRUE AND id<>$2", new_ip, vip_id)
if dup:
raise HTTPException(status_code=409, detail=f"{new_ip} is already used by another VIP")
new_extras = ([e.model_dump() for e in payload.additional_ips]
if payload.additional_ips is not None else None)
extras = new_extras if new_extras is not None else _extra_ips(v["additional_ips"])
if _norm_ip(new_ip) in {e["address"] for e in extras}:
raise HTTPException(status_code=400, detail="an additional address must differ from virtual_ip")
enc_set = payload.auth_pass is not None and payload.auth_pass != ""
try:
async with conn.transaction():
if payload.virtual_ip or new_extras is not None:
await _lock_and_check_addresses(conn, new_ip, extras, vip_id)
await conn.execute("""
UPDATE vip_instances SET
name = COALESCE($2, name),
@@ -665,6 +711,7 @@ async def update_vip(vip_id: int, payload: VIPUpdate, request: Request, authoriz
use_unicast = COALESCE($8, use_unicast),
track_haproxy = COALESCE($9, track_haproxy),
auth_pass_encrypted = CASE WHEN $10 THEN $11 ELSE auth_pass_encrypted END,
additional_ips = COALESCE($12::jsonb, additional_ips),
last_config_status = 'PENDING',
-- Editing a VIP means you are KEEPING and changing it, so it cancels any
-- staged deletion (otherwise a later Apply would delete instead of applying
@@ -676,7 +723,8 @@ async def update_vip(vip_id: int, payload: VIPUpdate, request: Request, authoriz
""", vip_id, payload.name, payload.description, payload.virtual_ip,
payload.prefix_length, payload.virtual_router_id, payload.advert_int,
payload.use_unicast, payload.track_haproxy,
enc_set, encrypt_vrrp_secret(payload.auth_pass) if enc_set else None)
enc_set, encrypt_vrrp_secret(payload.auth_pass) if enc_set else None,
json.dumps(new_extras) if new_extras is not None else None)
if members is not None:
# T-1: preserve the APPLIED snapshot + deploy ack for members that REMAIN,
# so a pending member edit never momentarily flips a running node to
@@ -826,6 +874,22 @@ async def apply_vip(vip_id: int, request: Request, authorization: str = Header(N
master_prio = masters[0]["priority"]
if any(m["role"] == "BACKUP" and m["priority"] >= master_prio for m in members):
raise HTTPException(status_code=400, detail="MASTER priority must exceed every BACKUP")
# Issue #73: an adoption made before v1.12.0 may still be waiting for this Apply. Taking
# over a file with several vrrp_instance blocks would remove the others from the node.
multi = await conn.fetchrow("""
SELECT a.name AS agent_name, (d.analysis->>'instance_count')::int AS n
FROM vip_members m
JOIN vip_discoveries d ON d.agent_id = m.agent_id
JOIN agents a ON a.id = m.agent_id
WHERE m.vip_id = $1 AND m.takeover_expected_hash IS NOT NULL
AND d.config_hash = m.takeover_expected_hash
AND COALESCE((d.analysis->>'instance_count')::int, 0) > 1
LIMIT 1
""", vip_id)
if multi:
raise HTTPException(status_code=409, detail=(
f"{multi['agent_name']}'s keepalived.conf declares {multi['n']} vrrp_instance blocks; "
f"taking it over would remove the others from the node. Reject this adoption."))
auth_plain = decrypt_vrrp_secret(v["auth_pass_encrypted"]) if v["auth_pass_encrypted"] else None
# If a secret is set but can't be decrypted (SECRET_KEY/VIP_ENCRYPTION_KEY rotated or
@@ -844,7 +908,7 @@ async def apply_vip(vip_id: int, request: Request, authorization: str = Header(N
vip_dict = {"id": v["id"], "name": v["name"], "virtual_ip": v["virtual_ip"],
"prefix_length": v["prefix_length"], "virtual_router_id": v["virtual_router_id"],
"advert_int": v["advert_int"], "use_unicast": v["use_unicast"],
"track_haproxy": v["track_haproxy"]}
"track_haproxy": v["track_haproxy"], "additional_ips": _extra_ips(v["additional_ips"])}
snapshot_members = []
async with conn.transaction():
@@ -870,7 +934,8 @@ async def apply_vip(vip_id: int, request: Request, authorization: str = Header(N
"vip": {"name": v["name"], "description": v["description"], "virtual_ip": v["virtual_ip"],
"prefix_length": v["prefix_length"], "virtual_router_id": v["virtual_router_id"],
"advert_int": v["advert_int"], "use_unicast": v["use_unicast"],
"track_haproxy": v["track_haproxy"], "auth_pass_encrypted": v["auth_pass_encrypted"]},
"track_haproxy": v["track_haproxy"], "auth_pass_encrypted": v["auth_pass_encrypted"],
"additional_ips": vip_dict["additional_ips"]},
"members": snapshot_members}
await conn.execute(
"UPDATE vip_instances SET last_config_status='APPLIED', "
@@ -944,17 +1009,24 @@ async def reject_vip(vip_id: int, request: Request, authorization: str = Header(
sv = snap["vip"]
sm = snap.get("members", [])
# Applied before v1.12.0 → no additional addresses.
extras = _extra_ips(sv.get("additional_ips") or [])
try:
async with conn.transaction():
await conn.execute("SELECT pg_advisory_xact_lock($1, $2)", *_VIP_ADDRESS_LOCK)
reason = await _address_clash(conn, sv["virtual_ip"], extras, vip_id)
if reason:
raise HTTPException(status_code=409, detail=f"Cannot restore — {reason}")
await conn.execute("""
UPDATE vip_instances SET
name=$2, description=$3, virtual_ip=$4, prefix_length=$5, virtual_router_id=$6,
advert_int=$7, use_unicast=$8, track_haproxy=$9, auth_pass_encrypted=$10,
additional_ips=$11::jsonb,
last_config_status='APPLIED', updated_at=CURRENT_TIMESTAMP
WHERE id=$1
""", vip_id, sv["name"], sv.get("description"), sv["virtual_ip"], sv["prefix_length"],
sv["virtual_router_id"], sv["advert_int"], sv["use_unicast"], sv["track_haproxy"],
sv.get("auth_pass_encrypted"))
sv.get("auth_pass_encrypted"), json.dumps(extras))
await conn.execute("DELETE FROM vip_members WHERE vip_id=$1", vip_id)
for m in sm:
await conn.execute("""
@@ -1049,6 +1121,16 @@ def _discovery_row_to_api(row) -> dict:
analysis = json.loads(analysis)
except (json.JSONDecodeError, TypeError):
analysis = None
# Issue #73: an analysis stored by an older parser does not carry the multi-instance
# blocker yet; show it here exactly as adopt_vip enforces it.
count = int((analysis or {}).get("instance_count") or 0) if isinstance(analysis, dict) else 0
if count > 1:
from services.keepalived_parser import MULTI_INSTANCE_MARKER, multi_instance_blocker
for cand in analysis.get("candidates") or []:
blockers = cand.setdefault("blockers", [])
if not any(MULTI_INSTANCE_MARKER in b for b in blockers):
blockers.append(multi_instance_blocker(count))
cand["adoptable"] = False
return {
"agent_id": row["agent_id"],
"agent_name": row["agent_name"],
@@ -1133,6 +1215,7 @@ async def _collect_instance_participants(conn, *, pool_id: int, vrid: int, virtu
"agent_id": r["agent_id"], "agent_name": r["agent_name"],
"ip_address": r["ip_address"], "config_hash": r["config_hash"],
"auth_pass_encrypted": r["auth_pass_encrypted"],
"instance_count": int((analysis or {}).get("instance_count") or 0),
"candidate": cand,
})
break
@@ -1206,7 +1289,8 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
except (TypeError, ValueError):
raise HTTPException(status_code=400, detail="prefix_length must be an integer")
from services.keepalived_parser import remaining_blockers
from services.keepalived_parser import (
MULTI_INSTANCE_MARKER, multi_instance_blocker, remaining_blockers)
vrid = vip_fields.get("virtual_router_id")
virtual_ip = vip_fields.get("virtual_ip")
@@ -1236,6 +1320,9 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
prefix_supplied=vip_fields.get("prefix_length") is not None,
accept_data_loss=bool(payload.get("accept_data_loss")),
)
# Issue #73: also for discoveries analysed before the parser reported this itself.
if p["instance_count"] > 1 and not any(MULTI_INSTANCE_MARKER in b for b in rem):
rem.append(multi_instance_blocker(p["instance_count"]))
if rem:
raise HTTPException(status_code=422, detail={
"message": f"This keepalived config cannot be adopted as-is ({p['agent_name']})",
@@ -1263,18 +1350,21 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
# this VRID in the pool (the clash check above would have refused first);
# * a node whose file carries our ownership marker (is_managed) is ours already.
adopted_ids = [p["agent_id"] for p in participants]
# Issue #73: every address of the instance, not only the first.
extras = _extra_ips(vip_fields.get("additional_ips") or [])
all_addrs = [virtual_ip] + [e["address"] for e in extras]
stranded = await conn.fetch("""
SELECT a.name AS agent_name, a.pool_id, COALESCE(a.enabled, TRUE) AS enabled,
d.parse_error
FROM vip_discoveries d
JOIN agents a ON a.id = d.agent_id
LEFT JOIN vip_instances av ON av.id = d.adopted_vip_id
WHERE d.raw_config_masked LIKE '%' || $1 || '%'
WHERE d.raw_config_masked LIKE ANY($1::text[])
AND NOT (a.id = ANY($2::int[]))
AND COALESCE(av.is_active, FALSE) = FALSE
AND d.is_managed = FALSE
ORDER BY a.name
""", virtual_ip, adopted_ids)
""", [f"%{a}%" for a in all_addrs], adopted_ids)
if stranded:
def _why(r):
if r["parse_error"]:
@@ -1286,7 +1376,7 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
return "it does not report this vrrp_instance"
named = "; ".join(f"{r['agent_name']} — {_why(r)}" for r in stranded)
raise HTTPException(status_code=422, detail=(
f"These node(s) also reference {virtual_ip} but cannot be taken over with the rest "
f"These node(s) also reference {', '.join(all_addrs)} but cannot be taken over with the rest "
f"of the instance, so adopting now would rewrite the others and leave them running "
f"an unmanaged config on the same address: {named}. Resolve that first, then adopt."))
@@ -1296,11 +1386,17 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
# the sharpest: the design refuses to GUESS a netmask for a live VIP, and quietly copying
# one node's netmask onto another is the same change by another name.
def _vfield(p, key):
return (p["candidate"].get("vip") or {}).get(key)
value = (p["candidate"].get("vip") or {}).get(key)
if key == "additional_ips":
# Hashable and order-free; an analysis from before v1.12.0 has no key → none.
return tuple(sorted((e["address"], e["prefix_length"])
for e in _extra_ips(value or [])))
return value
for key, label in (("prefix_length", "prefix length"),
("use_unicast", "unicast/multicast mode"),
("track_haproxy", "HAProxy tracking")):
("track_haproxy", "HAProxy tracking"),
("additional_ips", "additional addresses")):
seen = {}
for p in participants:
seen.setdefault(_vfield(p, key), []).append(p["agent_name"])
@@ -1370,9 +1466,9 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
f"holding the VIP. Register those nodes as agents so they report their config, "
f"then adopt again."))
# One active VIP per agent — the delivery endpoint serves a single keepalived.conf per
# node, so a second active membership never converges. create/update enforce this via
# _validate_members_against_pool; adoption did not call it at all.
# Adoption hands a node's whole keepalived.conf over to OpenManager. A node that is
# already a member of a managed VIP is not taken over a second time (issue #73 allows
# several VIPs per node for VIPs created here, not for a second adoption).
busy = await conn.fetchrow("""
SELECT a.name AS agent_name, v.name AS vip_name
FROM vip_members vm JOIN vip_instances v ON v.id = vm.vip_id
@@ -1396,19 +1492,20 @@ async def adopt_vip(payload: dict, request: Request, authorization: str = Header
f"config must keep its VRID, so resolve the collision first"))
async with conn.transaction():
await _lock_and_check_addresses(conn, virtual_ip, extras, None)
vip_id = await conn.fetchval("""
INSERT INTO vip_instances
(name, description, pool_id, virtual_ip, prefix_length, virtual_router_id,
advert_int, auth_pass_encrypted, use_unicast, track_haproxy,
is_active, last_config_status, adopted_at, created_by)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,TRUE,'PENDING',CURRENT_TIMESTAMP,$11)
is_active, last_config_status, adopted_at, created_by, additional_ips)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,TRUE,'PENDING',CURRENT_TIMESTAMP,$11,$12::jsonb)
RETURNING id
""", name, payload.get("description") or f"Adopted from {disc['agent_name']}",
disc["pool_id"], virtual_ip, int(vip_fields["prefix_length"]), int(vrid),
int(vip_fields.get("advert_int") or 1),
disc["auth_pass_encrypted"], # already Fernet-encrypted at ingest
bool(vip_fields.get("use_unicast")), bool(vip_fields.get("track_haproxy")),
current_user["id"])
current_user["id"], json.dumps(extras))
# Every node of the instance becomes a member with the role/priority/interface ITS
# OWN file declares, and each carries its own one-shot takeover authorisation pinned
# to the hash of the file we analysed on THAT node. The guard stays per-node: a file
+12 -2
View File
@@ -91,6 +91,7 @@ async def validate_user_cluster_access(user_id: int, cluster_id: int, conn):
@router.get("/stats")
async def get_waf_stats(cluster_id: Optional[int] = None, authorization: str = Header(None)):
"""Get WAF statistics and overview, optionally filtered by cluster"""
conn = None
try:
# Get current user for cluster validation
from auth_middleware import get_current_user_from_token
@@ -98,9 +99,12 @@ async def get_waf_stats(cluster_id: Optional[int] = None, authorization: str = H
conn = await get_database_connection()
# Validate cluster access if cluster_id provided
# Validate cluster access if cluster_id provided (a waf.read role reads every cluster)
if cluster_id:
await validate_user_cluster_access(current_user['id'], cluster_id, conn)
from auth_middleware import validate_cluster_read_access
await validate_cluster_read_access(
current_user, cluster_id, conn, "waf", validate_user_cluster_access
)
# Build cluster filter condition
cluster_filter = ""
@@ -179,6 +183,12 @@ async def get_waf_stats(cluster_id: Optional[int] = None, authorization: str = H
]
}
except HTTPException:
# Keep 401/403/404 as-is instead of re-wrapping them as 500. These are
# only raised before the success-path close, so release the handle here.
if conn is not None:
await close_database_connection(conn)
raise
except Exception as e:
logger.error(f"Error fetching WAF stats: {e}")
raise HTTPException(status_code=500, detail=str(e))
+24 -4
View File
@@ -1379,7 +1379,27 @@ async def generate_haproxy_config_for_cluster(cluster_id: int, conn: Optional[An
logger.info(f"CONFIG GENERATION: Backend '{backend['name']}' has NO ACTIVE SERVERS. Writing backend block (will show as DOWN until servers added).")
config_lines.append(f"backend {backend['name']}")
# Issue #72: a second `stick-table` in one backend is FATAL in HAProxy
# ("stick-table name ... conflicts with table declared"), which makes the
# agent reject the WHOLE cluster config. Can happen when an imported table
# (options) meets one entered by hand in request/response headers. Keep the
# first one, like the frontend R3.3 dedup; configs with a single table are
# rendered byte-identical.
_be_stick_table_emitted = False
def _append_be_free_form(line_stripped: str) -> None:
nonlocal _be_stick_table_emitted
if line_stripped.startswith("stick-table"):
if _be_stick_table_emitted:
logger.warning(
f"BACKEND STICK-TABLE DEDUP: skipping duplicate declaration in "
f"backend '{backend['name']}': {line_stripped}"
)
return
_be_stick_table_emitted = True
config_lines.append(f" {line_stripped}")
# --- HAProxy Configuration Best Practice Order ---
# 1. Mode and balance method
if backend.get('balance_method'):
@@ -1396,7 +1416,7 @@ async def generate_haproxy_config_for_cluster(cluster_id: int, conn: Optional[An
if line_stripped and line_stripped not in ('[]', '{}', 'null', 'None'):
# Lines are complete HAProxy option directives
# Examples: "option http-keep-alive", "option forwardfor"
config_lines.append(f" {line_stripped}")
_append_be_free_form(line_stripped)
# 3. Health check configuration
if backend.get('health_check_uri'):
@@ -1452,7 +1472,7 @@ async def generate_haproxy_config_for_cluster(cluster_id: int, conn: Optional[An
# Skip empty strings, "[]", or invalid rules
if line_stripped and line_stripped not in ('[]', '{}', 'null', 'None'):
# Lines are already complete directives (e.g., "http-request set-header X-Backend test")
config_lines.append(f" {line_stripped}")
_append_be_free_form(line_stripped)
# 9. HTTP Response Headers
if backend.get('response_headers'):
@@ -1461,7 +1481,7 @@ async def generate_haproxy_config_for_cluster(cluster_id: int, conn: Optional[An
# Skip empty strings, "[]", or invalid rules
if line_stripped and line_stripped not in ('[]', '{}', 'null', 'None'):
# Lines are already complete directives (e.g., "http-response add-header X-Powered-By Backend")
config_lines.append(f" {line_stripped}")
_append_be_free_form(line_stripped)
# Get all servers for this backend (including inactive ones for commenting)
# CRITICAL FIX: Handle servers with NULL cluster_id (legacy data)
+100
View File
@@ -183,6 +183,8 @@ def render_keepalived_conf(*, vip: dict, members: List[dict], this_agent: dict,
lines.append(" }")
lines.append(" virtual_ipaddress {")
lines.append(f' {vip["virtual_ip"]}/{int(vip.get("prefix_length", 24))} dev {iface}')
for extra in vip.get("additional_ips") or []:
lines.append(f' {extra["address"]}/{int(extra["prefix_length"])} dev {iface}')
lines.append(" }")
if track:
lines.append(" track_script {")
@@ -190,3 +192,101 @@ def render_keepalived_conf(*, vip: dict, members: List[dict], this_agent: dict,
lines.append(" }")
lines.append("}")
return "\n".join(lines) + "\n"
# ---------------------------------------------------------------------------
# Issue #73 — several VIPs on one node
# ---------------------------------------------------------------------------
# A node deploys ONE keepalived.conf, and each member row stores the full single-VIP file
# render_keepalived_conf produced at apply time. A node that belongs to several VIPs gets
# those applied files merged into one: global_defs once, then every VIP's vrrp_script and
# vrrp_instance. The health-check script is renamed per VIP (chk_haproxy_<id>) because each
# VIP's failover weight is derived from its own priorities, and keepalived needs unique
# script names. A node with a single VIP is served its applied file unchanged.
class KeepalivedComposeError(ValueError):
"""An applied snapshot is not in the shape render_keepalived_conf produces."""
def _top_level_blocks(conf: str) -> List[tuple]:
"""Split our own render into (name, lines) top-level blocks; comment lines are skipped."""
blocks: List[tuple] = []
current: Optional[List[str]] = None
depth = 0
for line in conf.split("\n"):
stripped = line.strip()
if current is None:
if not stripped or stripped.startswith("#"):
continue
if not stripped.endswith("{"):
raise KeepalivedComposeError(f"unexpected top-level line: {stripped[:60]}")
current = [line]
depth = 1
continue
current.append(line)
if stripped.startswith("auth_pass "):
continue # the secret may contain braces; it is a single token, never a block
depth += stripped.count("{") - stripped.count("}")
if depth == 0:
blocks.append((" ".join(current[0].strip().rstrip("{").split()), current))
current = None
elif depth < 0:
raise KeepalivedComposeError("unbalanced braces")
if current is not None:
raise KeepalivedComposeError("unterminated block")
return blocks
def compose_node_keepalived_conf(applied: List[tuple]) -> str:
"""Merge the applied single-VIP files of one node into the file the node deploys.
`applied` is [(vip_id, applied_config_content), ...]. One entry is returned unchanged,
so nodes with a single VIP keep the exact bytes (and hash) they already run. Raises
KeepalivedComposeError when a snapshot does not have the expected shape; the caller
must then leave the node alone rather than deploy a partial file.
"""
if not applied:
raise KeepalivedComposeError("nothing to compose")
if len(applied) == 1:
return applied[0][1]
header: List[str] = [f"{OWNERSHIP_MARKER} — DO NOT EDIT"]
global_defs: Optional[List[str]] = None
scripts: List[str] = []
instances: List[str] = []
for vip_id, conf in sorted(applied, key=lambda item: int(item[0])):
vip_id = int(vip_id)
for line in (conf or "").split("\n"):
if line.startswith("# VIP "):
header.append(line)
blocks = _top_level_blocks(conf or "")
names = [name for name, _ in blocks]
if names.count("global_defs") != 1 or names.count(f"vrrp_instance {_vrrp_instance_name(vip_id)}") != 1:
raise KeepalivedComposeError(f"VIP {vip_id}: applied config has an unexpected layout")
if any(n not in ("global_defs", "vrrp_script chk_haproxy", f"vrrp_instance {_vrrp_instance_name(vip_id)}")
for n in names):
raise KeepalivedComposeError(f"VIP {vip_id}: applied config has an unexpected block")
script_name = f"chk_haproxy_{vip_id}"
for name, body in blocks:
if name == "global_defs":
if global_defs is None:
global_defs = body
elif name == "vrrp_script chk_haproxy":
scripts.append("\n".join([f"vrrp_script {script_name} {{"] + body[1:]))
else:
renamed, in_track = [], False
for line in body:
s = line.strip()
if s == "track_script {":
in_track = True
elif in_track and s == "}":
in_track = False
elif in_track and s == "chk_haproxy":
line = line.replace("chk_haproxy", script_name)
renamed.append(line)
instances.append("\n".join(renamed))
out: List[str] = header + global_defs + [""]
for block in scripts:
out.extend([block, ""])
out.append("\n\n".join(instances))
return "\n".join(out) + "\n"
+61 -5
View File
@@ -31,6 +31,12 @@ import ipaddress
import re
from typing import Any, Dict, List, Optional, Tuple
from models.vip import MAX_ADDITIONAL_IPS
# Stored with every discovery (vip_discoveries.analysis_version). Bump it whenever the
# analysis or its blockers change: rows from an older parser make the agent re-report once.
ANALYSIS_VERSION = 1
# Directives `render_keepalived_conf` emits, and therefore the only ones a takeover can
# reproduce. Anything else found in a vrrp_instance is reported in `unsupported`.
_SUPPORTED_INSTANCE_KEYS = {
@@ -379,6 +385,17 @@ _DEFAULT_STATE = "BACKUP"
OUR_CHECK_SCRIPT_NAME = "chk_haproxy"
MULTI_INSTANCE_MARKER = "vrrp_instance blocks"
def multi_instance_blocker(count: int) -> str:
"""Adoption replaces the node's whole keepalived.conf with ONE instance. Not resolvable:
the text must contain neither _LOSS_MARKER nor _PREFIX_MARKER."""
return (f"the file declares {count} {MULTI_INSTANCE_MARKER}; adoption takes over the whole "
f"file, so the other instances would be removed from the node — adopting from a "
f"keepalived.conf with several instances is not supported")
def build_adoption_candidate(parsed: Dict[str, Any], instance: Dict[str, Any]) -> Dict[str, Any]:
"""Map one parsed `vrrp_instance` onto vip_instances / vip_members fields.
@@ -399,17 +416,54 @@ def build_adoption_candidate(parsed: Dict[str, Any], instance: Dict[str, Any]) -
f"line {d['line']}: `{d['directive']}` — OpenManager's renderer cannot reproduce "
f"this, so adopting would delete it")
# Adoption replaces the node's whole keepalived.conf with the render of ONE instance, so
# every other instance in the file would disappear from the node. Not resolvable: none of
# the wording below may contain _LOSS_MARKER or _PREFIX_MARKER.
instance_total = len(parsed.get("instances") or [])
if instance_total > 1:
blockers.append(multi_instance_blocker(instance_total))
vips = instance.get("virtual_ips") or []
if len(vips) == 0:
blockers.append("the instance declares no virtual_ipaddress — nothing to adopt")
elif len(vips) > 1:
addrs = ", ".join(v["address"] for v in vips)
blockers.append(
f"the instance carries {len(vips)} addresses ({addrs}); a managed VIP holds exactly "
f"one, so adopting would drop all but the first")
vip_entry = vips[0] if vips else None
# Issue #73: the addresses after the first become the VIP's additional addresses. Each one
# must render back exactly, so nothing about it may be guessed.
additional_ips: List[Dict[str, Any]] = []
extra_entries = vips[1:]
if len(extra_entries) > MAX_ADDITIONAL_IPS:
blockers.append(
f"the instance carries {len(vips)} addresses; a managed VIP holds at most "
f"{MAX_ADDITIONAL_IPS + 1}")
seen_addrs = {vip_entry["address"]} if vip_entry else set()
for extra in extra_entries:
addr = extra["address"]
if ipaddress.ip_address(addr).version != 4:
blockers.append(f"additional address `{addr}` is IPv6; only IPv4 addresses are managed")
continue
if addr in seen_addrs:
blockers.append(f"additional address `{addr}` is listed more than once")
continue
seen_addrs.add(addr)
prefix = extra.get("prefix_length")
if prefix is None:
blockers.append(
f"additional address `{addr}` has no /prefix in the file; add it there so the "
f"netmask cannot change on takeover, then let the agent report again")
continue
if not (1 <= int(prefix) <= 32):
blockers.append(f"additional address `{addr}/{prefix}` has an invalid prefix length")
continue
if extra.get("dev") and instance.get("interface") and extra["dev"] != instance["interface"]:
blockers.append(
f"additional address `{addr}` is bound to `dev {extra['dev']}` but the instance "
f"uses `interface {instance['interface']}`; the render always uses the instance "
f"interface")
continue
additional_ips.append({"address": addr, "prefix_length": int(prefix)})
if instance.get("virtual_router_id") is None:
blockers.append("no virtual_router_id — it cannot be guessed: a wrong VRID puts the "
"nodes in separate VRRP domains and both would claim the VIP")
@@ -468,6 +522,7 @@ def build_adoption_candidate(parsed: Dict[str, Any], instance: Dict[str, Any]) -
"use_unicast": bool(peers),
"track_haproxy": OUR_CHECK_SCRIPT_NAME in tracked,
"auth_pass": instance.get("auth_pass"),
"additional_ips": additional_ips,
},
"member": {
"network_interface": instance.get("interface"),
@@ -526,6 +581,7 @@ def analyse_keepalived_conf(text: str) -> Dict[str, Any]:
"""Parse + map in one call: the shape the discovery endpoint stores and the UI renders."""
parsed = parse_keepalived_conf(text)
return {
"analysis_version": ANALYSIS_VERSION,
"instance_count": len(parsed["instances"]),
"sync_groups": parsed["sync_groups"],
"global_defs": parsed["global_defs"],
@@ -0,0 +1,300 @@
"""
Issue #72 regression tests: `stick-table` re-definition ignored by Config Import.
Bug: the bulk-config parser had no branch for `stick-table` (frontend and
backend) nor for backend `stick on|match|store-*`, so they were dropped without
a warning. A frontend that kept its `track-sc0` / counter rules then got the
renderer's stick-table auto-inject (`type ip size 100k expire 30s store
http_req_rate(10s)`), i.e. the operator's new table was silently replaced by
the default. Separately, the import preview stripped EVERY `track-sc0 src` /
`sc_http_req_rate(0)` line as "auto-generated", deleting operator trackers.
DB-free: parser behaviour, the import policy helpers and source guards for the
renderer / endpoint wiring.
"""
import dataclasses
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.haproxy_config_parser import parse_haproxy_config
from utils.stick_table_import import (
drop_platform_stick_tables,
frontend_uses_counter,
is_platform_rate_limit_header,
merge_options,
stick_advisories,
strip_auto_headers,
uses_stick_counter,
)
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
NEW_TABLE = "stick-table type ip size 200k expire 3h store http_req_rate(10s),gpc0_rate(10s),gpt0"
DEFAULT_TABLE = "stick-table type ip size 100k expire 30s store http_req_rate(10s)"
ISSUE_72_CONFIG = f"""
frontend fe_web
bind *:80
mode http
option httplog
{NEW_TABLE}
http-request track-sc0 src
http-request deny if {{ sc0_gpc0_rate gt 20 }}
http-response sc-inc-gpc0(0) if {{ status 404 }}
default_backend be_web
backend be_web
mode http
stick-table type ip size 1m expire 30m
stick on src
stick match src table fe_web
stick store-request src
server s1 10.0.0.1:80 check
"""
def _src(rel):
with open(os.path.join(ROOT, rel), encoding="utf-8") as f:
return f.read()
# ---------------------------------------------------------------- parser
def test_parser_keeps_frontend_stick_table_in_options():
r = parse_haproxy_config(ISSUE_72_CONFIG)
fe = r.frontends[0]
assert fe.options == f"option httplog\n{NEW_TABLE}"
assert any("stick-table imported as-is" in w and NEW_TABLE in w for w in r.warnings)
# The rest of the frontend is parsed exactly as before.
assert fe.request_headers == "http-request track-sc0 src\nhttp-request deny if { sc0_gpc0_rate gt 20 }"
assert fe.response_headers == "http-response sc-inc-gpc0(0) if { status 404 }"
def test_parser_keeps_backend_stick_directives_in_order():
be = parse_haproxy_config(ISSUE_72_CONFIG).backends[0]
assert be.options == (
"stick-table type ip size 1m expire 30m\n"
"stick on src\n"
"stick match src table fe_web\n"
"stick store-request src"
)
assert be.mode == "http"
assert [s.server_name for s in be.servers] == ["s1"]
def test_parser_ignores_lookalike_directives():
cfg = """
backend b
stickiness on
stick-tablex foo
server s1 10.0.0.1:80
"""
be = parse_haproxy_config(cfg).backends[0]
assert be.options is None
def test_parser_unchanged_without_stick_directives():
cfg = """
frontend f
bind *:80
option httplog
http-request set-header X-A 1
default_backend b
backend b
option forwardfor
server s1 10.0.0.1:80
"""
r = parse_haproxy_config(cfg)
fe = dataclasses.asdict(r.frontends[0])
be = dataclasses.asdict(r.backends[0])
assert fe["options"] == "option httplog"
assert be["options"] == "option forwardfor"
assert not any("stick" in w for w in r.warnings)
# ------------------------------------------------- generator-owned headers
def test_platform_rate_limit_header_exact_forms():
assert is_platform_rate_limit_header("http-request track-sc0 src")
assert is_platform_rate_limit_header("http-request deny if { sc_http_req_rate(0) gt 100 }")
assert is_platform_rate_limit_header(
"http-request capture req.hdr(User-Agent) len 64 if { sc_http_req_rate(0) gt 5 }")
assert not is_platform_rate_limit_header("http-request track-sc0 src table other")
assert not is_platform_rate_limit_header("http-request track-sc1 src")
assert not is_platform_rate_limit_header(
"http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }")
def test_strip_keeps_operator_rate_limit_lines_when_platform_does_not_regenerate():
rh = "http-request track-sc0 src\nhttp-request deny if { sc_http_req_rate(0) gt 100 }"
assert strip_auto_headers(rh, platform_regenerates=False) == rh
def test_strip_removes_generator_lines_when_platform_regenerates():
rh = ("http-request track-sc0 src\n"
"http-request deny if { sc_http_req_rate(0) gt 100 }\n"
"http-request set-header X-A 1")
assert strip_auto_headers(rh, platform_regenerates=True) == "http-request set-header X-A 1"
def test_strip_keeps_non_exact_forms_even_when_platform_regenerates():
rh = "http-request track-sc0 src table other\nhttp-request track-sc1 src"
assert strip_auto_headers(rh, platform_regenerates=True) == rh
def test_strip_always_removes_acme_and_waf_lines():
rh = ("http-request allow if is_acme_challenge\n"
"http-request deny if waf_block_1\n"
"http-request set-header X-A 1")
for regen in (True, False):
assert strip_auto_headers(rh, regen) == "http-request set-header X-A 1"
assert strip_auto_headers(None, True) is None
assert strip_auto_headers("http-request track-sc0 src", True) is None
# ------------------------------------------- generator-shaped stick-tables
def test_drop_platform_table_when_platform_regenerates():
opts = f"option httplog\nstick-table type ip size 100k expire 60s store http_req_rate(10s)"
kept, dropped = drop_platform_stick_tables(opts, True, False)
assert kept == "option httplog"
assert dropped == ["stick-table type ip size 100k expire 60s store http_req_rate(10s)"]
def test_drop_default_table_when_auto_inject_would_render_it():
kept, dropped = drop_platform_stick_tables(DEFAULT_TABLE, False, True)
assert kept is None and dropped == [DEFAULT_TABLE]
def test_keep_generator_shaped_table_the_platform_would_not_render():
# 60s with only auto-inject in play: auto-inject would render 30s, so keep it.
t60 = "stick-table type ip size 100k expire 60s store http_req_rate(10s)"
assert drop_platform_stick_tables(t60, False, True) == (t60, [])
# Default table with no counter use: nothing would render it (e.g. `table fe` refs).
assert drop_platform_stick_tables(DEFAULT_TABLE, False, False) == (DEFAULT_TABLE, [])
def test_never_drop_operator_table():
for regen in (True, False):
for counter in (True, False):
assert drop_platform_stick_tables(NEW_TABLE, regen, counter) == (NEW_TABLE, [])
def test_frontend_uses_counter_matches_renderer_rule():
assert uses_stick_counter("http-request track-sc0 src")
assert uses_stick_counter("http-request deny if { sc_http_req_rate(0) gt 1 }")
assert not uses_stick_counter("http-request set-header X-A 1")
assert frontend_uses_counter([None, "option httplog", ["acl a sc_http_req_rate(0) gt 1"]])
# stick-table lines themselves never count (they are the thing being decided on)
assert not frontend_uses_counter([DEFAULT_TABLE])
def test_uses_stick_counter_is_the_renderer_expression():
s = _src("services/haproxy_config.py")
assert 'if "track-sc" in stripped or ("sc_" in stripped and "_rate(" in stripped):' in s
m = _src("utils/stick_table_import.py")
assert 'return "track-sc" in s or ("sc_" in s and "_rate(" in s)' in m
def test_platform_table_regex_covers_every_generator_form():
s = _src("services/haproxy_config.py")
assert s.count("stick-table type ip size 100k expire 30s store http_req_rate(10s)") == 2
assert "stick-table type ip size 100k expire {waf_rule.get('rate_limit_window', 60)}s store http_req_rate(10s)" in s
for line in (DEFAULT_TABLE, "stick-table type ip size 100k expire 120s store http_req_rate(10s)"):
assert drop_platform_stick_tables(line, True, False)[1] == [line]
# ------------------------------------------------------------ options merge
def test_merge_stick_only_import_keeps_existing_option_lines():
assert merge_options(NEW_TABLE, "option forwardfor") == f"option forwardfor\n{NEW_TABLE}"
def test_merge_option_only_import_keeps_existing_stick_lines():
assert merge_options("option httplog", f"option forwardfor\n{DEFAULT_TABLE}") == \
f"option httplog\n{DEFAULT_TABLE}"
def test_merge_replaces_each_group_when_imported():
assert merge_options(f"option httplog\n{NEW_TABLE}", f"option forwardfor\n{DEFAULT_TABLE}") == \
f"option httplog\n{NEW_TABLE}"
def test_merge_without_imported_options_keeps_existing_verbatim():
assert merge_options(None, "option forwardfor") == "option forwardfor"
assert merge_options(None, None) is None
def test_merge_same_lines_returns_existing_unchanged():
existing = f"{NEW_TABLE}\noption httplog"
assert merge_options(f"option httplog\n{NEW_TABLE}", existing) is existing
def test_merge_matches_previous_behaviour_when_no_stick_lines():
# Pre-#72 bulk import replaced options wholesale when the import had option lines.
assert merge_options("option httplog", "option forwardfor") == "option httplog"
# ---------------------------------------------------------------- advisories
def test_advisory_table_without_http_req_rate_when_rate_limited():
t = "stick-table type ip size 200k expire 3h store gpc0_rate(10s),gpt0"
w = stick_advisories("Frontend", "f", t, t, {}, platform_regenerates=True)
assert any("does not store http_req_rate" in x for x in w)
assert not stick_advisories("Frontend", "f", t, t, {}, platform_regenerates=False)
assert not stick_advisories("Frontend", "f", NEW_TABLE, NEW_TABLE, {}, platform_regenerates=True)
def test_advisory_duplicate_table_in_other_field():
w = stick_advisories("Backend", "b", NEW_TABLE, NEW_TABLE,
{"Request Headers": "stick-table type ip size 1m"}, False)
assert any("also stored in Request Headers" in x for x in w)
# Not raised when this import did not bring a table.
assert not stick_advisories("Backend", "b", NEW_TABLE, None,
{"Request Headers": "stick-table type ip size 1m"}, False)
def test_advisory_stick_rule_without_table():
w = stick_advisories("Backend", "b", "stick on src", "stick on src", {}, False)
assert any("unable to find stick-table" in x for x in w)
assert not stick_advisories("Backend", "b", "stick on src table fe", "stick on src table fe", {}, False)
assert not stick_advisories("Backend", "b", "stick-table type ip size 1m\nstick on src",
"stick on src", {}, False)
def test_advisory_peers():
t = "stick-table type ip size 1m peers mypeers"
w = stick_advisories("Backend", "b", t, t, {}, False)
assert any("'mypeers'" in x and "BEFORE the 'defaults'" in x for x in w)
# ------------------------------------------------------------ wiring guards
def test_backend_renderer_dedups_stick_table():
s = _src("services/haproxy_config.py")
assert "_be_stick_table_emitted" in s
assert "BACKEND STICK-TABLE DEDUP" in s
block = s[s.index("_append_be_free_form(line_stripped: str)"):s.index("# 3. Health check configuration")]
assert "_append_be_free_form(line_stripped)" in block # options loop routed through it
tail = s[s.index("# 8. HTTP Request Headers"):s.index("# Get all servers for this backend")]
assert tail.count("_append_be_free_form(line_stripped)") == 2 # request + response headers
def test_parse_bulk_uses_conditional_policy():
s = _src("routers/config.py")
fn = s[s.index("async def parse_bulk_config("):s.index('@router.post("/bulk-create")')]
assert '"track-sc0 src" in s' not in fn # the unconditional strip is gone
assert "strip_auto_headers(" in fn and "platform_regenerates[" in fn
assert "drop_platform_stick_tables(" in fn
assert fn.count("merge_options(") == 2 # frontends + backends
assert fn.count("stick_advisories(") == 2
# Bulgu #82 guard window must still see the cluster access check.
assert "validate_user_cluster_access" in fn[:6000]
+48 -7
View File
@@ -18,7 +18,7 @@ os.environ.setdefault("SECRET_KEY", "test-secret-key-for-keepalived-parser-tests
from services import keepalived_config as kc # noqa: E402
from services.keepalived_parser import ( # noqa: E402
KeepalivedParseError, analyse_keepalived_conf, build_adoption_candidate,
parse_keepalived_conf,
parse_keepalived_conf, remaining_blockers,
)
@@ -78,6 +78,7 @@ def test_parses_a_handwritten_config_into_model_fields():
"use_unicast": True,
"track_haproxy": True,
"auth_pass": "s3cr3t",
"additional_ips": [],
}
assert cand["member"] == {"network_interface": "eth0", "role": "MASTER", "priority": 150}
assert cand["peers"] == ["10.0.0.12"] and cand["unicast_src_ip"] == "10.0.0.11"
@@ -141,11 +142,46 @@ def test_directives_we_cannot_render_block_adoption():
assert _only_candidate(text)["adoptable"] is False
def test_multiple_addresses_in_one_instance_block_adoption():
text = HANDWRITTEN.replace(" 10.0.0.100/24 dev eth0",
" 10.0.0.100/24 dev eth0\n 10.0.0.101/24 dev eth0")
blockers = _blockers_for(text)
assert "2 addresses" in blockers and "10.0.0.101" in blockers
def _with_extra(line):
return HANDWRITTEN.replace(" 10.0.0.100/24 dev eth0",
" 10.0.0.100/24 dev eth0\n" + line)
def test_multiple_addresses_in_one_instance_are_adopted_as_additional_ips():
"""Issue #73: the addresses after the first become the VIP's additional addresses."""
cand = _only_candidate(_with_extra(" 10.0.0.101/25 dev eth0\n 10.0.0.102/24"))
assert cand["adoptable"] is True, cand["blockers"]
assert cand["vip"]["virtual_ip"] == "10.0.0.100"
assert cand["vip"]["additional_ips"] == [
{"address": "10.0.0.101", "prefix_length": 25},
{"address": "10.0.0.102", "prefix_length": 24},
]
def test_an_additional_address_that_cannot_render_back_exactly_blocks_for_good():
"""No prefix, another dev, IPv6 or a repeat cannot be reproduced. None of them may be
waivable: their wording must carry neither the loss nor the prefix marker."""
cases = {
" 10.0.0.101": "no /prefix",
" 10.0.0.101/24 dev eth1": "dev eth1",
" fd00::1/64 dev eth0": "IPv6",
" 10.0.0.100/24 dev eth0": "more than once",
" 10.0.0.101/40 dev eth0": "invalid prefix",
}
for line, reason in cases.items():
cand = _only_candidate(_with_extra(line))
assert cand["adoptable"] is False, line
assert reason in " | ".join(cand["blockers"]), (line, cand["blockers"])
assert remaining_blockers(cand["blockers"], prefix_supplied=True,
accept_data_loss=True), line
def test_too_many_addresses_block_adoption():
from models.vip import MAX_ADDITIONAL_IPS
lines = "\n".join(f" 10.0.1.{i}/24 dev eth0" for i in range(1, MAX_ADDITIONAL_IPS + 2))
cand = _only_candidate(_with_extra(lines))
assert cand["adoptable"] is False
assert f"at most {MAX_ADDITIONAL_IPS + 1}" in " | ".join(cand["blockers"])
def test_missing_vrid_blocks_adoption_with_the_split_brain_reason():
@@ -237,7 +273,12 @@ vrrp_instance VI_2 {
names = [c["instance_name"] for c in analysed["candidates"]]
assert names == ["VI_1", "VI_2"]
assert [c["vip"]["virtual_ip"] for c in analysed["candidates"]] == ["10.0.0.100", "10.0.0.200"]
assert all(c["adoptable"] for c in analysed["candidates"])
# Issue #73: adoption takes over the whole file, so adopting either instance would remove
# the other from the node. Neither may be adoptable, and no flag may waive it.
for c in analysed["candidates"]:
assert c["adoptable"] is False
assert remaining_blockers(c["blockers"], prefix_supplied=True, accept_data_loss=True)
assert any("2 vrrp_instance blocks" in b for b in c["blockers"])
def test_unbalanced_braces_raise():
+227
View File
@@ -0,0 +1,227 @@
"""Read-only role access: read endpoints work without pool access, private keys stay hidden.
A user on the built-in ``viewer`` role (``ssl.read``, ``waf.read``, no
``ssl.download``) and no ``user_pool_access`` rows used to get 403 on the
cluster-scoped SSL list and SSL detail, while clusters, frontends, backends and
WAF rules were readable. These tests pin the fix:
* ``<resource>.read`` lets the caller read that resource on every cluster;
without it the pool check still applies.
* the SSL private key is returned only with ``ssl.download`` plus pool access,
both in the certificate detail and in the Apply Management SSL diff.
"""
from datetime import datetime, timezone
from unittest.mock import AsyncMock
import pytest
from fastapi import HTTPException
import auth_middleware
from routers import cluster as cluster_mod
from routers import ssl as ssl_mod
from routers import waf as waf_mod
KEY_PEM = "-----BEGIN PRIVATE KEY-----\nSECRETKEYMATERIAL\n-----END PRIVATE KEY-----"
CERT_PEM = "-----BEGIN CERTIFICATE-----\nPUBLICCERT\n-----END CERTIFICATE-----"
USER = {"id": 7, "username": "readonly", "is_admin": False}
def _no_pool_access():
async def _validate(_uid, _cid, _conn):
raise HTTPException(status_code=403, detail="You don't have access to this cluster.")
return _validate
def _pool_access_ok():
async def _validate(_uid, _cid, _conn):
return True
return _validate
def _patch_common(monkeypatch, module, conn, permissions, pool_validator):
async def _user(_auth):
return dict(USER)
async def _perm(_uid, resource, action, **_kw):
return f"{resource}.{action}" in permissions
async def _get_conn():
return conn
async def _close(_c):
return None
monkeypatch.setattr(auth_middleware, "check_user_permission", _perm)
monkeypatch.setattr(auth_middleware, "get_current_user_from_token", _user)
monkeypatch.setattr(module, "get_database_connection", _get_conn)
monkeypatch.setattr(module, "close_database_connection", _close)
monkeypatch.setattr(module, "validate_user_cluster_access", pool_validator)
if hasattr(module, "get_current_user_from_token"):
monkeypatch.setattr(module, "get_current_user_from_token", _user)
def _cert_row(cluster_id=1):
now = datetime(2026, 1, 1, tzinfo=timezone.utc)
return {
"id": 5, "name": "example", "domain": "example.com", "all_domains": '["example.com"]',
"certificate_content": CERT_PEM, "private_key_content": KEY_PEM, "chain_content": None,
"expiry_date": now, "issuer": "CA", "status": "valid", "days_until_expiry": 90,
"fingerprint": "ab", "cluster_id": cluster_id, "usage_type": "frontend", "source": "manual",
"is_active": True, "created_at": now, "updated_at": now, "is_global": False, "cluster_ids": [1],
}
def _detail_conn(cluster_id=1):
conn = AsyncMock()
conn.fetchrow.return_value = _cert_row(cluster_id)
conn.fetchval.return_value = cluster_id # cluster exists
conn.fetch.return_value = []
return conn
# ---------------------------------------------------------------- SSL detail
@pytest.mark.asyncio
async def test_viewer_reads_ssl_detail_without_pool_access_but_key_is_hidden(monkeypatch):
_patch_common(monkeypatch, ssl_mod, _detail_conn(), {"ssl.read"}, _no_pool_access())
out = await ssl_mod.get_ssl_certificate(5, authorization="Bearer x")
assert out["certificate_content"] == CERT_PEM
assert out["private_key_content"] is None
assert out["private_key_redacted"] is True
assert "SECRETKEYMATERIAL" not in repr(out)
@pytest.mark.asyncio
async def test_ssl_download_with_pool_access_returns_key(monkeypatch):
_patch_common(monkeypatch, ssl_mod, _detail_conn(), {"ssl.read", "ssl.download"}, _pool_access_ok())
out = await ssl_mod.get_ssl_certificate(5, authorization="Bearer x")
assert out["private_key_content"] == KEY_PEM
assert out["private_key_redacted"] is False
@pytest.mark.asyncio
async def test_ssl_download_without_pool_access_keeps_key_hidden(monkeypatch):
_patch_common(monkeypatch, ssl_mod, _detail_conn(), {"ssl.read", "ssl.download"}, _no_pool_access())
out = await ssl_mod.get_ssl_certificate(5, authorization="Bearer x")
assert out["private_key_content"] is None
assert out["private_key_redacted"] is True
@pytest.mark.asyncio
async def test_ssl_detail_without_ssl_read_still_needs_pool_access(monkeypatch):
_patch_common(monkeypatch, ssl_mod, _detail_conn(), set(), _no_pool_access())
with pytest.raises(HTTPException) as exc:
await ssl_mod.get_ssl_certificate(5, authorization="Bearer x")
assert exc.value.status_code == 403
# ------------------------------------------------------------------ SSL list
@pytest.mark.asyncio
async def test_viewer_lists_cluster_ssl_without_pool_access(monkeypatch):
conn = AsyncMock()
conn.fetchval.return_value = 1 # cluster exists / table exists
conn.fetch.return_value = []
_patch_common(monkeypatch, ssl_mod, conn, {"ssl.read"}, _no_pool_access())
out = await ssl_mod.get_ssl_certificates(cluster_id=1, usage_type=None, authorization="Bearer x")
assert out == []
@pytest.mark.asyncio
async def test_ssl_list_unknown_cluster_is_404_for_read_role(monkeypatch):
conn = AsyncMock()
conn.fetchval.return_value = None
_patch_common(monkeypatch, ssl_mod, conn, {"ssl.read"}, _no_pool_access())
with pytest.raises(HTTPException) as exc:
await ssl_mod.get_ssl_certificates(cluster_id=99, usage_type=None, authorization="Bearer x")
assert exc.value.status_code == 404
@pytest.mark.asyncio
async def test_ssl_list_without_ssl_read_still_needs_pool_access(monkeypatch):
conn = AsyncMock()
_patch_common(monkeypatch, ssl_mod, conn, set(), _no_pool_access())
with pytest.raises(HTTPException) as exc:
await ssl_mod.get_ssl_certificates(cluster_id=1, usage_type=None, authorization="Bearer x")
assert exc.value.status_code == 403
# ----------------------------------------------------------------- WAF stats
def _waf_conn():
conn = AsyncMock()
conn.fetchval.return_value = 1
conn.fetch.return_value = []
return conn
@pytest.mark.asyncio
async def test_viewer_reads_cluster_waf_stats_without_pool_access(monkeypatch):
_patch_common(monkeypatch, waf_mod, _waf_conn(), {"waf.read"}, _no_pool_access())
out = await waf_mod.get_waf_stats(cluster_id=1, authorization="Bearer x")
assert "overview" in out
@pytest.mark.asyncio
async def test_waf_stats_denial_is_403_not_500(monkeypatch):
_patch_common(monkeypatch, waf_mod, _waf_conn(), set(), _no_pool_access())
with pytest.raises(HTTPException) as exc:
await waf_mod.get_waf_stats(cluster_id=1, authorization="Bearer x")
assert exc.value.status_code == 403
# --------------------------------------------------- Apply Management SSL diff
def _diff_conn():
conn = AsyncMock()
version = {
"id": 11, "version_name": "ssl-5-update-1700000000", "config_content": "",
"created_at": datetime(2026, 1, 1, tzinfo=timezone.utc), "metadata": None,
}
ssl_cert = {
"id": 5, "name": "example", "primary_domain": "example.com",
"created_at": datetime(2026, 1, 1, tzinfo=timezone.utc),
"certificate_content": CERT_PEM, "private_key_content": KEY_PEM, "chain_content": None,
"usage_type": "frontend", "ssl_type": "Cluster-specific",
}
conn.fetchrow.side_effect = [version, ssl_cert]
return conn
@pytest.mark.asyncio
async def test_ssl_diff_redacts_private_key_without_ssl_download(monkeypatch):
_patch_common(monkeypatch, cluster_mod, _diff_conn(), {"apply.read"}, _pool_access_ok())
out = await cluster_mod.get_config_version_diff(1, 11, authorization="Bearer x")
lines = [c["line"] for c in out["changes"]]
assert not any("SECRETKEYMATERIAL" in line for line in lines)
assert any("redacted" in line for line in lines)
assert any("PUBLICCERT" in line for line in lines)
@pytest.mark.asyncio
async def test_ssl_diff_shows_private_key_with_ssl_download(monkeypatch):
_patch_common(monkeypatch, cluster_mod, _diff_conn(), {"ssl.download"}, _pool_access_ok())
out = await cluster_mod.get_config_version_diff(1, 11, authorization="Bearer x")
assert any("SECRETKEYMATERIAL" in c["line"] for c in out["changes"])
+2 -1
View File
@@ -460,7 +460,8 @@ def test_discovery_cache_defers_to_the_server():
assert agent_router.count('AS discovery_known') == 1, (
"the keepalived-config query must report whether a discovery row exists"
)
assert agent_router.count('"discovery_known": bool(agent["discovery_known"])') == 5, (
# 7 = the five original paths + the v1.12.0 merged-delivery and compose_error paths.
assert agent_router.count('"discovery_known": bool(agent["discovery_known"])') == 7, (
"every response path that knows the agent must carry the flag — the adoptable nodes are "
"precisely the not_configured ones"
)
+231
View File
@@ -0,0 +1,231 @@
"""Issue #73 (v1.12.0) — several VIPs on one node, several addresses on one VIP.
Pure tests for the merge of a node's applied single-VIP files, the extra-address rendering
and validation, plus source-level guards for the delivery / ack / adoption / migration
invariants that need a live database to exercise end to end.
"""
from __future__ import annotations
import hashlib
import os
import pathlib
import sys
import pytest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from models.vip import MAX_ADDITIONAL_IPS, VIPCreate, VIPUpdate # noqa: E402
from services import keepalived_config as kc # noqa: E402
from services.keepalived_parser import ( # noqa: E402
ANALYSIS_VERSION, analyse_keepalived_conf, parse_keepalived_conf,
)
BACKEND = pathlib.Path(__file__).resolve().parents[1]
AGENT_ROUTER = (BACKEND / "routers" / "agent.py").read_text()
VIP_ROUTER = (BACKEND / "routers" / "vip.py").read_text()
MIGRATIONS = (BACKEND / "database" / "migrations.py").read_text()
NODE = {"role": "MASTER", "priority": 150, "network_interface": "eth0",
"agent_id": 1, "ip_address": "10.0.0.11"}
PEER = {"role": "BACKUP", "priority": 100, "network_interface": "eth0",
"agent_id": 2, "ip_address": "10.0.0.12"}
def _vip(vid, ip, vrid, **over):
base = {"id": vid, "name": f"vip-{vid}", "virtual_ip": ip, "prefix_length": 24,
"virtual_router_id": vrid, "advert_int": 1, "use_unicast": True,
"track_haproxy": True}
base.update(over)
return base
def _applied(vip, auth="s3cr3t", this=NODE, members=(NODE, PEER)):
peers = [m["ip_address"] for m in members if m["agent_id"] != this["agent_id"]]
return kc.render_keepalived_conf(vip=vip, members=list(members), this_agent=this,
peer_ips=peers, auth_pass_plain=auth)
A = _vip(3, "10.0.0.100", 51)
B = _vip(7, "10.0.0.200", 52, track_haproxy=False)
class TestCompose:
def test_single_vip_is_served_byte_identical(self):
conf = _applied(A)
assert kc.compose_node_keepalived_conf([(3, conf)]) == conf
def test_two_vips_merge_into_one_valid_file(self):
out = kc.compose_node_keepalived_conf([(3, _applied(A)), (7, _applied(B))])
assert out.count("global_defs {") == 1
assert out.startswith(kc.OWNERSHIP_MARKER)
assert "vrrp_script chk_haproxy_3 {" in out
assert "vrrp_script chk_haproxy {" not in out
assert "vrrp_instance VI_3 {" in out and "vrrp_instance VI_7 {" in out
assert out.index("VI_3") < out.index("VI_7")
assert " chk_haproxy_3\n" in out
parsed = parse_keepalived_conf(out)
assert [i["instance_name"] for i in parsed["instances"]] == ["VI_3", "VI_7"]
assert [i["track_scripts"] for i in parsed["instances"]] == [["chk_haproxy_3"], []]
def test_order_follows_vip_id_not_input_order(self):
a, b = (3, _applied(A)), (7, _applied(B))
assert kc.compose_node_keepalived_conf([b, a]) == kc.compose_node_keepalived_conf([a, b])
def test_each_instance_keeps_its_own_failover_weight(self):
a2 = _vip(9, "10.0.0.150", 53)
low_peer = {**PEER, "priority": 50}
out = kc.compose_node_keepalived_conf([
(3, _applied(A)), (9, _applied(a2, members=(NODE, low_peer)))])
assert "weight -51" in out and "weight -101" in out
def test_braces_in_the_secret_do_not_break_the_merge(self):
out = kc.compose_node_keepalived_conf([(3, _applied(A, auth="a}b{")),
(7, _applied(B, auth="}x"))])
assert "auth_pass a}b{" in out and "auth_pass }x" in out
assert out.count("vrrp_instance") == 2
@pytest.mark.parametrize("bad", [
"",
"global_defs {\n}\n",
"vrrp_instance VI_3 {\n state MASTER\n",
"something else\n",
])
def test_unexpected_snapshot_fails_closed(self, bad):
with pytest.raises(kc.KeepalivedComposeError):
kc.compose_node_keepalived_conf([(3, _applied(A)), (7, bad)])
def test_foreign_block_fails_closed(self):
tampered = _applied(B) + "virtual_server 10.0.0.1 80 {\n}\n"
with pytest.raises(kc.KeepalivedComposeError):
kc.compose_node_keepalived_conf([(3, _applied(A)), (7, tampered)])
def test_snapshot_of_another_vip_id_fails_closed(self):
with pytest.raises(kc.KeepalivedComposeError):
kc.compose_node_keepalived_conf([(3, _applied(A)), (8, _applied(B))])
def test_nothing_to_compose_raises(self):
with pytest.raises(kc.KeepalivedComposeError):
kc.compose_node_keepalived_conf([])
class TestAdditionalAddresses:
def test_empty_list_renders_byte_identical(self):
assert _applied({**A, "additional_ips": []}) == _applied(A)
def test_extras_follow_the_primary_on_the_same_interface(self):
conf = _applied({**A, "additional_ips": [
{"address": "10.0.0.101", "prefix_length": 24},
{"address": "192.168.5.9", "prefix_length": 32}]})
lines = [l.strip() for l in conf.splitlines()]
i = lines.index("10.0.0.100/24 dev eth0")
assert lines[i + 1:i + 3] == ["10.0.0.101/24 dev eth0", "192.168.5.9/32 dev eth0"]
def test_rendered_extras_are_adopted_back_unchanged(self):
extras = [{"address": "10.0.0.101", "prefix_length": 25}]
conf = _applied({**A, "additional_ips": extras})
cand = analyse_keepalived_conf(conf)["candidates"][0]
assert cand["adoptable"] is True, cand["blockers"]
assert cand["vip"]["additional_ips"] == extras
def _members():
return [{"agent_id": 1, "network_interface": "eth0", "role": "MASTER", "priority": 150}]
class TestModels:
def test_default_is_empty(self):
v = VIPCreate(name="v", pool_id=1, virtual_ip="10.0.0.100", members=_members())
assert v.additional_ips == []
def test_address_is_normalised_and_prefix_defaults(self):
v = VIPCreate(name="v", pool_id=1, virtual_ip="10.0.0.100", members=_members(),
additional_ips=[{"address": " 10.0.0.101 "}])
assert v.additional_ips[0].model_dump() == {"address": "10.0.0.101", "prefix_length": 24}
@pytest.mark.parametrize("extras", [
[{"address": "10.0.0.100"}],
[{"address": "10.0.0.101"}, {"address": "10.0.0.101"}],
[{"address": "fd00::1"}],
[{"address": "nope"}],
[{"address": "10.0.0.101", "prefix_length": 33}],
[{"address": f"10.0.1.{i}"} for i in range(1, MAX_ADDITIONAL_IPS + 2)],
])
def test_invalid_extras_are_rejected(self, extras):
with pytest.raises(ValueError):
VIPCreate(name="v", pool_id=1, virtual_ip="10.0.0.100", members=_members(),
additional_ips=extras)
def test_update_omitted_keeps_and_empty_clears(self):
assert VIPUpdate().additional_ips is None
assert VIPUpdate(additional_ips=[]).additional_ips == []
class TestDeliveryAndAckGuards:
def test_failures_answer_with_statuses_the_agent_ignores(self):
assert '"status": "compose_error"' in AGENT_ROUTER
generic = AGENT_ROUTER.split("keepalived-config delivery failed", 1)[1][:600]
assert '"status": "unavailable"' in generic
assert "not_configured" not in generic.split("return", 1)[1].split("}", 1)[0]
def test_merged_delivery_hash_is_md5_of_the_composed_text(self):
assert 'hashlib.md5(composed.encode("utf-8")).hexdigest()' in AGENT_ROUTER
out = kc.compose_node_keepalived_conf([(3, _applied(A)), (7, _applied(B))])
assert len(hashlib.md5(out.encode("utf-8")).hexdigest()) == 32
def test_only_applied_snapshots_are_merged(self):
q = AGENT_ROUTER.split("async def _node_applied_vips", 1)[1].split("async def ", 1)[0]
assert "v.is_active = TRUE" in q
assert "m.applied_config_content IS NOT NULL" in q
def test_ack_failure_to_merge_is_not_a_500(self):
ack = AGENT_ROUTER.split("acks the merged file once", 1)[1][:1200]
assert "except Exception" in ack and "composed_hash = None" in ack
def test_discovery_known_asks_old_analyses_to_be_reported_again(self):
assert "vd.analysis_version >= $2" in AGENT_ROUTER
assert ANALYSIS_VERSION >= 1
assert analyse_keepalived_conf(_applied(A))["analysis_version"] == ANALYSIS_VERSION
class TestAdoptionGuards:
def test_apply_refuses_an_inflight_adoption_of_a_multi_instance_file(self):
apply = VIP_ROUTER.split("async def apply_vip", 1)[1].split("\nasync def ", 1)[0]
assert "d.config_hash = m.takeover_expected_hash" in apply
assert "(d.analysis->>'instance_count')::int, 0) > 1" in apply
assert "status_code=409" in apply
def test_old_analyses_get_the_blocker_in_the_list_and_on_adopt(self):
assert VIP_ROUTER.count("multi_instance_blocker(") >= 2
row_api = VIP_ROUTER.split("def _discovery_row_to_api", 1)[1].split("\ndef ", 1)[0]
assert 'cand["adoptable"] = False' in row_api
def test_address_writes_take_the_address_lock(self):
for fn in ("create_vip", "update_vip", "adopt_vip"):
body = VIP_ROUTER.split(f"async def {fn}", 1)[1].split("\nasync def ", 1)[0]
assert "_lock_and_check_addresses(" in body, fn
class TestDeltaMigration:
def test_versions(self):
assert "SCHEMA_VERSION = 13" in MIGRATIONS
assert "_DELTA_BASE_VERSION = 12" in MIGRATIONS
assert "return [(13, ensure_vip_multi_address_columns)]" in MIGRATIONS
def test_full_sequence_also_creates_the_columns(self):
inner = MIGRATIONS.split("async def _run_all_migrations_inner", 1)[1].split("\nasync def ", 1)[0]
assert inner.index("await ensure_vip_tables()") < inner.index(
"await ensure_vip_multi_address_columns()")
def test_delta_path_does_not_reseed_roles(self):
run = MIGRATIONS.split("async def run_all_migrations", 1)[1].split("\nasync def ", 1)[0]
delta = run.split("applied_version >= _DELTA_BASE_VERSION", 1)[1].split("else:", 1)[0]
assert "_delta_steps()" in delta
assert "_run_all_migrations_inner" not in delta
assert "update_system_roles" not in delta
def test_columns_are_additive_with_safe_defaults(self):
body = MIGRATIONS.split("async def ensure_vip_multi_address_columns", 1)[1].split(
"\nasync def ", 1)[0].replace('"\n "', "")
assert "ADD COLUMN IF NOT EXISTS additional_ips JSONB NOT NULL DEFAULT '[]'::jsonb" in body
assert "ADD COLUMN IF NOT EXISTS analysis_version SMALLINT NOT NULL DEFAULT 0" in body
+25
View File
@@ -11,6 +11,9 @@ from dataclasses import dataclass, asdict
logger = logging.getLogger(__name__)
_STICK_TABLE_LINE_RE = re.compile(r'^stick-table\s')
_STICK_RULE_LINE_RE = re.compile(r'^stick\s+(on|match|store-request|store-response)\s')
@dataclass
class ParsedServer:
@@ -277,6 +280,17 @@ class HAProxyConfigParser:
if re.match(r'^log-format(-sd)?\s', line, re.IGNORECASE):
log_format_list.append(line.strip())
continue
# Issue #72: `stick-table` matched no branch below and was dropped
# without a warning, so the renderer's auto-inject replaced the
# operator's table with the 100k/30s default. Kept verbatim in
# `options`; the renderer routes it to the `stick` bucket.
if _STICK_TABLE_LINE_RE.match(line):
options_list.append(line.strip())
self.warnings.append(
f"Frontend '{name}': stick-table imported as-is into Frontend Options: "
f"'{line.strip()}'."
)
continue
# Parse bind directive
# IMPORTANT: Handle multiple bind lines correctly
@@ -585,6 +599,17 @@ class HAProxyConfigParser:
options_list = []
for line in lines:
# Issue #72: `stick-table` and `stick on|match|store-*` were dropped
# without a warning. Kept verbatim in `options`, which the renderer
# writes as-is.
if _STICK_TABLE_LINE_RE.match(line) or _STICK_RULE_LINE_RE.match(line):
options_list.append(line.strip())
self.warnings.append(
f"Backend '{name}': stick directive imported as-is into Backend Options: "
f"'{line.strip()}'."
)
continue
# Parse mode
mode_match = re.match(r'^mode\s+(\S+)', line, re.IGNORECASE)
if mode_match:
+219
View File
@@ -0,0 +1,219 @@
"""
Issue #72 — Config Import policy for stick-table / stick rules and the rate-limit
lines the config generator injects.
Pure functions (no DB) so the import endpoint stays thin and the policy is unit
testable. "Platform regenerates" means the generator will itself emit the
rate-limit table/tracker/deny lines for a frontend: its `rate_limit` field is set
or an active `rate_limit` WAF rule applies to it (see services/haproxy_config.py).
"""
import re
from typing import Iterable, List, Optional, Tuple
# The only stick-table forms the generator emits (services/haproxy_config.py:
# frontend rate_limit, stick-table auto-inject, WAF rate_limit rule).
PLATFORM_STICK_TABLE_RE = re.compile(
r'^stick-table\s+type\s+ip\s+size\s+100k\s+expire\s+(\d+)s\s+store\s+http_req_rate\(10s\)$'
)
AUTO_INJECT_EXPIRE_SECONDS = 30
# Exact forms of the rate-limit request lines the generator emits.
_PLATFORM_RATE_LIMIT_HEADER_RES = (
re.compile(r'^http-request\s+track-sc0\s+src$'),
re.compile(r'^http-request\s+deny\s+if\s+\{\s*sc_http_req_rate\(0\)\s+gt\s+\d+\s*\}$'),
re.compile(
r'^http-request\s+capture\s+req\.hdr\(User-Agent\)\s+len\s+64\s+if\s+'
r'\{\s*sc_http_req_rate\(0\)\s+gt\s+\d+\s*\}$'
),
)
_STICK_RULE_RE = re.compile(r'^stick\s+(on|match|store-request|store-response)\s')
_TABLE_ARG_RE = re.compile(r'(?:^|\s)table\s+\S+')
_PEERS_ARG_RE = re.compile(r'(?:^|\s)peers\s+(\S+)')
def _lines(text: Optional[str]) -> List[str]:
if not text:
return []
return [l.strip() for l in text.split('\n') if l.strip()]
def is_stick_line(line: str) -> bool:
s = line.strip()
return s.startswith('stick-table') or s.startswith('stick ')
def is_stick_table_line(line: str) -> bool:
return line.strip().startswith('stick-table')
def uses_stick_counter(line: str) -> bool:
"""Same test the renderer uses to decide a frontend needs a stick-table
(services/haproxy_config.py `_emit_fe`)."""
s = line.strip()
return "track-sc" in s or ("sc_" in s and "_rate(" in s)
def is_platform_rate_limit_header(line: str) -> bool:
s = line.strip()
return any(r.match(s) for r in _PLATFORM_RATE_LIMIT_HEADER_RES)
def is_always_auto_header(line: str) -> bool:
"""ACME and WAF lines are generator-only: stripped regardless of context."""
s = line.strip()
if "is_acme_challenge" in s:
return True
if s.startswith("http-request") and " waf_" in s:
return True
return False
def strip_auto_headers(headers: Optional[str], platform_regenerates: bool) -> Optional[str]:
"""Drop generator-owned lines from a `request_headers` value.
Rate-limit lines are dropped only when the generator will emit them again for
this frontend AND they match its exact form; otherwise they are the
operator's own (e.g. a tracker feeding gpc0 rules) and are kept.
"""
if not headers:
return None
kept = []
for line in headers.split("\n"):
if is_always_auto_header(line):
continue
if platform_regenerates and is_platform_rate_limit_header(line):
continue
kept.append(line)
return "\n".join(kept) if kept else None
def drop_platform_stick_tables(
options: Optional[str],
platform_regenerates: bool,
frontend_uses_counter: bool,
) -> Tuple[Optional[str], List[str]]:
"""Remove imported stick-table lines the generator will reproduce itself.
Keeping them would freeze them in `options` (emitted first, so they win the
dedup) and hide later rate_limit / WAF window changes. A generator-shaped
line is dropped when rate_limit/WAF will emit a table, or when it is the
30s auto-inject default and the frontend uses a counter (auto-inject then
renders the identical line). Anything else is the operator's table.
"""
if not options:
return options, []
kept, dropped = [], []
for line in options.split("\n"):
m = PLATFORM_STICK_TABLE_RE.match(line.strip())
if m and (
platform_regenerates
or (int(m.group(1)) == AUTO_INJECT_EXPIRE_SECONDS and frontend_uses_counter)
):
dropped.append(line.strip())
continue
kept.append(line)
return ("\n".join(kept) if kept else None), dropped
def merge_options(parsed: Optional[str], existing: Optional[str]) -> Optional[str]:
"""Merge an imported `options` value into the stored one per directive type.
Bulk import only overwrote `options` when the import carried `option` lines;
with stick lines now landing in `options` too, a stick-only import must not
wipe hand-entered `option` lines (and vice versa). Each group is replaced
only when the import carries that group. Returns `existing` unchanged when
the merged lines are the same set, so re-imports don't flag a change.
"""
parsed_lines = _lines(parsed)
existing_lines = _lines(existing)
parsed_other = [l for l in parsed_lines if not is_stick_line(l)]
parsed_stick = [l for l in parsed_lines if is_stick_line(l)]
existing_other = [l for l in existing_lines if not is_stick_line(l)]
existing_stick = [l for l in existing_lines if is_stick_line(l)]
final = (parsed_other or existing_other) + (parsed_stick or existing_stick)
if not final:
return None
if sorted(final) == sorted(existing_lines):
return existing
return "\n".join(final)
def frontend_uses_counter(fields: Iterable) -> bool:
"""True if any non-stick line of the given frontend fields uses a counter."""
for value in fields:
if not value:
continue
items = value if isinstance(value, (list, tuple)) else str(value).split("\n")
for line in items:
if isinstance(line, str) and not is_stick_line(line) and uses_stick_counter(line):
return True
return False
def stick_advisories(
kind: str,
name: str,
final_options: Optional[str],
imported_options: Optional[str],
final_other_fields: dict,
platform_regenerates: bool,
) -> List[str]:
"""Non-blocking import-preview warnings for stick directives.
kind: 'Frontend' or 'Backend'. final_other_fields: {label: final value} of
the other free-form fields (what they hold after this import is applied).
"""
warnings: List[str] = []
final_lines = _lines(final_options)
tables = [l for l in final_lines if is_stick_table_line(l)]
imported_tables = [l for l in _lines(imported_options) if is_stick_table_line(l)]
if kind == "Frontend" and platform_regenerates:
for t in tables:
if not PLATFORM_STICK_TABLE_RE.match(t) and "http_req_rate" not in t:
warnings.append(
f"⚠️ Frontend '{name}': the imported stick-table does not store "
f"http_req_rate, but this frontend's Rate Limit / rate_limit WAF rule "
f"checks sc_http_req_rate(0) against it. HAProxy accepts this and the "
f"rate limit silently never triggers. Add http_req_rate(10s) to the "
f"table's 'store' list to keep it working."
)
if imported_tables:
for label, value in final_other_fields.items():
if any(is_stick_table_line(l) for l in _lines(value)):
effect = (
"only the first stick-table is rendered (Options), so the one in "
f"{label} is ignored"
)
warnings.append(
f"⚠️ {kind} '{name}': a stick-table is also stored in {label}; "
f"{effect}. Remove it from {label}."
)
if kind == "Backend":
has_own_table = bool(tables) or any(
is_stick_table_line(l) for v in final_other_fields.values() for l in _lines(v)
)
for l in final_lines:
if _STICK_RULE_RE.match(l) and not _TABLE_ARG_RE.search(l) and not has_own_table:
warnings.append(
f"⚠️ Backend '{name}': '{l}' needs a stick-table in this backend or a "
f"'table <name>' argument. HAProxy rejects it at apply "
f"(\"unable to find stick-table\"); the previous config keeps running."
)
for t in tables:
m = _PEERS_ARG_RE.search(t)
if m:
warnings.append(
f"ℹ️ {kind} '{name}': stick-table references peers section "
f"'{m.group(1)}'. HAProxy OpenManager does not manage peers sections: define "
f"it in each node's haproxy.cfg BEFORE the 'defaults' section (the agent "
f"keeps everything above the first defaults/listen/frontend/backend and "
f"replaces the rest)."
)
return warnings
+3 -3
View File
@@ -1,5 +1,5 @@
{
"version": "1.11.1",
"releaseName": "Adoption cannot hide a node; Config Import on fresh installs",
"releaseDate": "2026-08-15"
"version": "1.12.0",
"releaseName": "Several keepalived VIPs on one node, and several addresses on one VIP",
"releaseDate": "2026-10-16"
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "haproxy-openmanager-frontend",
"version": "1.11.1",
"version": "1.12.0",
"description": "HAProxy Load Balancer Management UI",
"license": "AGPL-3.0-or-later",
"dependencies": {
+32 -16
View File
@@ -1188,11 +1188,16 @@ const SSLManagement = () => {
{({ getFieldValue }) => {
const usageType = getFieldValue('usage_type');
const privateKeyValue = getFieldValue('private_key_content');
const isRequired = usageType === 'frontend';
// The stored key is withheld from users without ssl.download; leaving
// the field empty on edit keeps it (the backend falls back to it).
const storedKeyKept = !!selectedCertificate?.private_key_redacted;
const isRequired = usageType === 'frontend' && !storedKeyKept;
const hasValue = privateKeyValue && privateKeyValue.trim();
let extraMessage;
if (isRequired) {
if (storedKeyKept && !hasValue) {
extraMessage = <span style={{ color: '#999' }}>Stored private key is hidden and will be kept. Paste a new key only to replace it.</span>;
} else if (isRequired) {
extraMessage = hasValue
? <span style={{ color: '#52c41a' }}>Private key provided</span>
: <span style={{ color: '#ff4d4f' }}>Required for Frontend SSL</span>;
@@ -1233,7 +1238,7 @@ const SSLManagement = () => {
<TextArea
rows={8}
disabled={isLetsEncryptCert}
placeholder={isLetsEncryptCert ? "Managed by ACME automation" : "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEF...\n-----END PRIVATE KEY-----"}
placeholder={isLetsEncryptCert ? "Managed by ACME automation" : storedKeyKept ? "Stored private key hidden (ssl.download permission required)" : "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEF...\n-----END PRIVATE KEY-----"}
/>
</Form.Item>
);
@@ -1361,19 +1366,30 @@ const SSLManagement = () => {
)}
</TabPane>
<TabPane tab="Private Key" key="3">
<Alert
message="Security Warning"
description="Private key content is sensitive. Only view when necessary and ensure secure handling."
type="warning"
showIcon
style={{ marginBottom: 16 }}
/>
<TextArea
value={selectedCertificate.private_key_content}
rows={12}
readOnly
style={{ fontFamily: 'monospace' }}
/>
{selectedCertificate.private_key_redacted ? (
<Alert
message="Private key hidden"
description="A private key is stored for this certificate, but viewing it requires the ssl.download permission."
type="info"
showIcon
/>
) : (
<>
<Alert
message="Security Warning"
description="Private key content is sensitive. Only view when necessary and ensure secure handling."
type="warning"
showIcon
style={{ marginBottom: 16 }}
/>
<TextArea
value={selectedCertificate.private_key_content}
rows={12}
readOnly
style={{ fontFamily: 'monospace' }}
/>
</>
)}
</TabPane>
<TabPane tab={
<span>
+53 -4
View File
@@ -6,7 +6,7 @@ import {
import {
PlusOutlined, EditOutlined, DeleteOutlined, ReloadOutlined, WarningOutlined,
PlayCircleOutlined, SyncOutlined, CrownOutlined, ClockCircleOutlined, FileSearchOutlined,
InfoCircleOutlined
InfoCircleOutlined, MinusCircleOutlined
} from '@ant-design/icons';
import { useCluster } from '../contexts/ClusterContext';
import { extractApiError } from '../utils/apiError';
@@ -76,9 +76,13 @@ const mergeBlockers = (lists) => {
return Array.from(seen.values());
};
// Mirrors backend/models/vip.py MAX_ADDITIONAL_IPS.
const MAX_ADDITIONAL_IPS = 19;
const IPV4_RE = /^(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}$/;
// This component uses raw fetch(), but extractApiError expects an axios-shaped error
// (err.response.data). Read the fetch Response body and reuse the envelope-aware extractor
// so backend messages — e.g. the 409 "node already in VIP X" — actually reach the user.
// so backend messages — e.g. the 409 "address already used by VIP X" — actually reach the user.
const fetchApiError = async (res, fallback) => {
try { const data = await res.json(); return extractApiError({ response: { data } }, fallback); }
catch { return fallback; }
@@ -351,7 +355,8 @@ const VIPManagement = () => {
setSelectedPoolId(null);
setMemberRows([]);
form.resetFields();
form.setFieldsValue({ prefix_length: 24, advert_int: 1, use_unicast: true, track_haproxy: true });
form.setFieldsValue({ prefix_length: 24, advert_int: 1, use_unicast: true, track_haproxy: true,
additional_ips: [] });
setModalVisible(true);
};
@@ -365,6 +370,7 @@ const VIPManagement = () => {
virtual_ip: vip.virtual_ip, prefix_length: vip.prefix_length,
virtual_router_id: vip.virtual_router_id, advert_int: vip.advert_int,
use_unicast: vip.use_unicast, track_haproxy: vip.track_haproxy,
additional_ips: (vip.additional_ips || []).map((a) => ({ ...a })),
});
setModalVisible(true);
};
@@ -478,7 +484,19 @@ const VIPManagement = () => {
const columns = [
{ title: 'Name', dataIndex: 'name', key: 'name' },
{ title: 'Virtual IP', key: 'vip', render: (_, v) => <Text code>{v.virtual_ip}/{v.prefix_length}</Text> },
{ title: 'Virtual IP', key: 'vip', render: (_, v) => {
const extra = v.additional_ips || [];
return (
<Space size={4}>
<Text code>{v.virtual_ip}/{v.prefix_length}</Text>
{extra.length > 0 && (
<Tooltip title={extra.map((a) => `${a.address}/${a.prefix_length}`).join(', ')}>
<Tag>+{extra.length}</Tag>
</Tooltip>
)}
</Space>
);
} },
{ title: 'Pool', dataIndex: 'pool_name', key: 'pool' },
{ title: 'VRID', dataIndex: 'virtual_router_id', key: 'vrid' },
{ title: 'Members / Live state', key: 'members', render: renderMembers },
@@ -843,6 +861,37 @@ const VIPManagement = () => {
</Form.Item>
</Space>
<Form.List name="additional_ips">
{(fields, { add, remove }) => (
<div style={{ marginBottom: 16 }}>
<Text strong>Additional addresses</Text>
<div style={{ color: '#888', fontSize: 12, marginBottom: 8 }}>
Optional. Extra IPv4 addresses carried by the same VRRP instance — they move between
nodes together with the virtual IP, on the same interface (up to {MAX_ADDITIONAL_IPS}).
</div>
{fields.map(({ key, name, ...rest }) => (
<Space key={key} align="baseline">
<Form.Item {...rest} name={[name, 'address']}
rules={[{ required: true, message: 'Address required' },
{ pattern: IPV4_RE, message: 'Enter an IPv4 address' }]}>
<Input placeholder="10.0.0.101" style={{ width: 180 }} />
</Form.Item>
<Form.Item {...rest} name={[name, 'prefix_length']}
rules={[{ required: true, message: 'Prefix required' }]}>
<InputNumber min={1} max={32} addonBefore="/" />
</Form.Item>
<MinusCircleOutlined onClick={() => remove(name)} />
</Space>
))}
<Button type="dashed" size="small" icon={<PlusOutlined />}
disabled={fields.length >= MAX_ADDITIONAL_IPS}
onClick={() => add({ prefix_length: form.getFieldValue('prefix_length') || 24 })}>
Add address
</Button>
</div>
)}
</Form.List>
<Text strong>Participating nodes</Text>
<div style={{ color: '#888', fontSize: 12, marginBottom: 8 }}>
These are the HAProxy servers (agents) already enrolled in this pool — toggle which join the VIP.