Commit Graph

8735 Commits

Author SHA1 Message Date
courtmanr@gmail.com a38d21bb86 feat(alerts): add initial delivery routing 2026-07-30 15:37:53 +01:00
courtmanr@gmail.com 206d3a1c4d feat(kubernetes): scope overview workloads by cluster 2026-07-30 15:10:49 +01:00
courtmanr@gmail.com 4e03ecd9aa test(notifications): align email routing expectations 2026-07-30 15:05:44 +01:00
courtmanr@gmail.com 9245da363f docs(governance): record notification tag routing contract 2026-07-30 14:54:34 +01:00
courtmanr@gmail.com 51d4b12415 feat(notifications): route alerts by resource tags 2026-07-30 14:48:42 +01:00
courtmanr@gmail.com 6e710cd4aa feat(docker): show container update target details 2026-07-30 13:55:43 +01:00
courtmanr@gmail.com ff0a7a4827 Align Apprise topic copy tests 2026-07-30 13:22:47 +01:00
courtmanr@gmail.com 4ff4b61ccc Restore Apprise Telegram topic delivery 2026-07-30 13:10:59 +01:00
courtmanr@gmail.com f985e616d6 Allow in-place API token scope edits (#1629) 2026-07-30 12:06:19 +01:00
courtmanr@gmail.com d201b14f1b Document host-relative memory contracts (#1326) 2026-07-30 11:34:09 +01:00
courtmanr@gmail.com 046158bda5 Add host-relative guest memory view (#1326) 2026-07-30 11:31:00 +01:00
courtmanr@gmail.com 110ed3eb2b Support guest filesystem alert overrides (#1205) 2026-07-30 10:59:27 +01:00
courtmanr@gmail.com 69cfce92d3 Expose per-resource alert delay controls (#1497) 2026-07-30 10:27:39 +01:00
courtmanr@gmail.com cb1768d484 Update node modal form shape proofs 2026-07-30 09:48:24 +01:00
courtmanr@gmail.com 101735588c Add Docker host web interface access (#1276) 2026-07-30 09:31:28 +01:00
courtmanr@gmail.com 6922c8ad57 Fix PBS datastore exclusions before detail polling (#1105) 2026-07-30 09:21:57 +01:00
courtmanr@gmail.com 53c5122665 fix(updates): keep LXC installer guidance release-local (#1408)
Contract-Neutral: Issue #1408 keeps release-local signed-installer guidance; update APIs and deployment behavior are unchanged, with direct model coverage.
2026-07-30 01:02:50 +01:00
courtmanr@gmail.com c3c6b96cbd test(agent): keep QNAP watchdog proof cross-platform (#1617) 2026-07-30 00:54:04 +01:00
courtmanr@gmail.com 4d4567e4a4 fix(updates): stop recommending unsafe LXC helper (#1408)
Contract-Neutral: Issue #1408 changes update guidance only; update APIs and deployment behavior are unchanged, with direct UI regression coverage.
2026-07-30 00:49:27 +01:00
courtmanr@gmail.com 378ffcde16 docs: define QNAP watchdog singleton contract (#1617) 2026-07-30 00:43:31 +01:00
courtmanr@gmail.com 69475117be fix(agent): prevent duplicate QNAP watchdogs (#1617) 2026-07-30 00:40:27 +01:00
courtmanr@gmail.com 94c908419b Fix discovery blocklist for configured subnets (#1616) 2026-07-30 00:18:48 +01:00
courtmanr@gmail.com 41978482e7 Restore cross-platform verification coverage
Use a native absolute path in the SMART override test so Windows verifies the same contract as Unix. Move the metrics concurrency proof into the canonical verification artifact and document that live ingestion remains independent from lifecycle maintenance.
2026-07-29 23:17:58 +01:00
courtmanr@gmail.com 0c5944b8eb Preserve full poll budget for large PVE clusters
Do not let a short per-request connection timeout shrink the whole inventory cycle below its 90-second default. The seven-node support bundle for #1437 showed hundreds of successful guest reads followed by exact 30-second cancellation and dead-lettering, preventing a coherent generation from publishing. Larger request timeouts can still expand the cycle up to MAX_POLL_TIMEOUT.\n\nFixes #1437
2026-07-29 23:11:45 +01:00
courtmanr@gmail.com 7c91724191 Keep metrics ingestion live during maintenance
Run startup maintenance, rollups, and retention on a lifecycle-owned worker separate from buffered metric ingestion. SQLite still serializes write transactions, while maintenance CPU/read work can no longer stop writeCh from draining until it drops live batches. Join both workers before closing the shared database.\n\nFixes #1601
2026-07-29 23:07:20 +01:00
courtmanr@gmail.com f1ae0af646 Stop warning for permitted unsigned remote config
When remote-config signature enforcement is disabled, an unsigned response is an expected compatibility state rather than a warning. Keep its structured diagnostic at debug while preserving warning/error behavior for required, expired, future, or invalid signatures.\n\nFixes #1538
2026-07-29 23:03:00 +01:00
courtmanr@gmail.com 02d47ba514 Merge remote-tracking branch 'origin/main' into feature/external-probe-alerting 2026-07-29 22:26:35 +01:00
courtmanr@gmail.com 1684244f76 document integration contracts and canonical proofs 2026-07-29 22:22:46 +01:00
courtmanr@gmail.com c5d7941d08 Merge remote-tracking branch 'origin/main' into feature/external-probe-alerting 2026-07-29 22:21:04 +01:00
courtmanr@gmail.com ad054defaf fix outstanding integration and agent regressions 2026-07-29 22:10:04 +01:00
courtmanr@gmail.com 0ff0a37f8d Use server time for probe freshness 2026-07-29 21:28:26 +01:00
courtmanr@gmail.com d4e5326059 Ratchet private boundary audit debt 2026-07-29 20:09:23 +01:00
courtmanr@gmail.com 352c8cef7a Fix companion compatibility CI 2026-07-29 20:09:23 +01:00
courtmanr@gmail.com a53d45e2d3 Harden external probe outage alerting 2026-07-29 20:09:23 +01:00
courtmanr@gmail.com f4dd8096aa Prepare v6.2.0-rc.4 release
Fourth candidate on the v6.2.0 line, superseding v6.2.0-rc.3 and still following stable v6.1.2. This hardening cut includes action recovery, kiosk containment, alert and notification accounting, node-restricted Proxmox storage, container-runtime selection, and host CPU sampling fixes that landed after rc.3. Version pins move to 6.2.0-rc.4 across the repo root, Docker bootstrap defaults, and Helm metadata; stable install pointers remain on v6.1.2 until governed promotion.
v6.2.0-rc.4
2026-07-29 14:49:51 +01:00
courtmanr@gmail.com 8aeffc456d Parallelize Build and Test and add a local pre-push check
The Build and Test workflow ran everything in one serial job — frontend
lint/tests/build, the full go test -race sweep, and benchmarks — for a
37-55 minute wall clock on every push. The job now splits into parallel
jobs: frontend (lint, unit tests, type-check, bundle build and size gate),
backend tests sharded three ways with internal/api on its own shard since
it alone takes ~10m under -race, script smoke tests with the backend
build, and benchmarks with the existing baseline cache flow. Backend jobs
satisfy the frontend embed with a stub index.html, which unit tests do
not assert against; the real bundle is still built and size-gated by the
frontend job and by release builds. Expected wall clock is roughly the
slowest single job, 10-15 minutes.

A new changes job skips the build/test jobs when a push touches only
docs/ and Markdown, mirroring the path filters Core E2E already has. The
gitleaks secret scan deliberately stays unconditional since this workflow
is the only per-push scan and docs can leak secrets too.

scripts/dev-prepush.sh runs the checks that most often turn main red,
scoped to the outgoing commits: the canonical completion guard in CI
mode, the registry snapshot tests when registry.json changed, the
mutation registry audits, build plus tests for touched Go packages, and
the frontend type-check. It exists because CI verdicts arrive well after
the push; today's unclassified force-fail route would have failed it
locally in about a minute.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 14:32:01 +01:00
courtmanr@gmail.com 848e166f5d Fix alert and notification telemetry signals 2026-07-29 14:17:19 +01:00
rcourtman 000986f125 Keep kiosk sessions off settings surfaces and silence the truncated-status security banner (#1650)
Kiosk mode is frontend-only and a kiosk API token carries monitoring:read
alone, but the shell treated kiosk as a display preference rather than an
authority fact. Pressing Escape on a kiosk display revealed the full header,
the navigation, and a security-warning banner offering links into Settings.

Three separate bugs produced that:

Escape called toggleKioskMode() unconditionally. setKioskMode(false) writes
'false' to sessionStorage, so the exit was sticky for the rest of the browser
session and the auto-enable effect never re-armed. Escape now exits kiosk only
for a session that can reach settings; a scope-limited session gets the
temporary header reveal the hover and touch affordances already provide.

The redirect that keeps sessions off /settings, /patrol and the
alert-configuration tabs ran only while kiosk was on, so once Escape had turned
kiosk off a kiosk-token session could navigate into Settings. It now also keys
off whether the session's token can read settings, so it holds with kiosk off.
The panels themselves already 403 server-side; this is UI containment, not an
auth change.

/api/security/status truncates its payload by authority and names the tier in
detailLevel. A token without settings:read gets an 'authenticated' payload with
no posture fields at all, which the banner read as disabled controls and turned
into a false security score with an "Enable Security" link. The visibility
helper now refuses to assert anything the payload does not carry: an
authenticated payload raises nothing, a public payload is authoritative for
hasAuthentication only so a genuinely unauthenticated instance is still warned
about, and a privileged payload keeps the full assessment. The remaining global
banners deep-link into settings too, so that block is gated on the same
settings-access predicate.

Scope membership is now decided once by sessionHasSettingsAccess in
AppLayout.tsx and shared with App.tsx instead of being re-derived inline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 14:06:50 +01:00
courtmanr@gmail.com 8aac88588c Classify the action force-fail route in the mutation registry (#1649)
The Canonical Governance mutation registry audit scans the real router
sources and fails closed on any infrastructure route it cannot resolve to
a registry disposition, so the new POST /api/actions/{id}/force-fail
route broke the audit on main. Register it as the lifecycle entry
action.api.force-fail, executed by internal/actionlifecycle.Service.ForceFail
under the same execute_action capability, admin approval floor, and
committed-lifecycle delivery as execute, and classify the route in the
runtime surface catalogue.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 12:33:49 +01:00
Richard Courtman ccea5ba761 Fail actions stranded without agent completion evidence and add force-fail (#1649)
An action that reached executing had exactly one way out: a terminal
operation receipt from the agent. Expiry skips executing rows on purpose,
and reconciliation bailed for every non-terminal query answer, so a Docker
update whose agent restarted mid-pull sat in executing forever with no
operator route out of it.

The agent-side receipt store rewrites accepted and started receipts to
interrupted on every Open(), and nothing can move an interrupted or
tombstoned receipt back to terminal. An identity-correlated answer of
either kind is therefore proof the operation will never report, and the
three executors now settle the action immediately as inconclusive with a
message telling the operator Pulse cannot confirm the effect and the
resource needs checking by hand.

A not_found answer, or a receipt still merely accepted or started, may
still be completed, so those keep waiting and only settle once the
dispatch attempt is older than one hour, the same threshold the
pulse-intelligence telemetry already uses to call an executing action
stuck. Every typed operation timeout is far shorter than that, so an
in-flight mutation is never cut short. A transport error answering the
query is still not evidence and preserves receipt_pending unchanged.

RecoverExecutingActions drives all of this on the existing two-minute
recovery loop, so rows already wedged before the upgrade heal themselves
without anyone touching them.

For the residue that reconciliation cannot reach, an agent that was
reinstalled or a legacy executing row with no dispatch attempt, POST
/api/actions/{id}/force-fail writes the same inconclusive terminal truth
under an operator attribution. It never touches the transport, refuses
anything already terminal, and is gated on admin plus settings:write on
top of the execute capability check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 12:14:28 +01:00
Richard Courtman 97c5449bef Fix storage listed on nodes excluded by its node restriction (#1645)
Proxmox's per-node storage endpoint GET /nodes/{node}/storage does not
return what the node can actually use. It returns every storage in the
datacenter config and reports the ones the node is excluded from with
enabled:0/active:0 instead of leaving them out. pollStorageWithNodes
ingested every row it got back, so a storage restricted to one node
appeared on all the others as a disabled entry, which the UI renders as
Offline.

The poller already fetches the datacenter config once via GetAllStorage
and keeps it keyed by storage name, so the restriction is in hand at the
point each per-node row is turned into a models.Storage. Skip the row
when the config carries a non-empty nodes restriction that does not
include the node being polled. Node names are compared case-insensitively
to match how node identity is compared elsewhere in the poller.

Only the restriction drops a row. A storage with no restriction still
shows up on every node it is reported from, including when it is disabled
everywhere, so a datacenter-wide disabled storage stays visible as
disabled rather than vanishing. Because the shared-storage aggregation
builds Nodes/NodeIDs/NodeCount from the surviving per-node rows, this also
stops a restricted shared storage from claiming cluster members that
cannot mount it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 11:45:56 +01:00
courtmanr@gmail.com c7d7c3abea Add stranded pulse-intelligence gate section to the high-risk matrix
The pulse-intelligence-rg-01-rg-12 release gate was registered in
status.json but its matrix section only existed in a never-pushed
v6.1.3 packet staged in a local checkout, leaving
documentation_currentness_test failing on every branch since. Salvaged
verbatim from that staged edit; the gate's latest exercised record and
block conditions are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 23:45:24 +01:00
courtmanr@gmail.com e734bd0097 Fix transient rootless Podman socket outranking rootful Docker (#1647)
The installer's discover_rootless_container_runtime only globbed
/run/user/* sockets and never consulted the system Docker daemon, so a
socket-activated rootless Podman API socket (alive only for root's login
session on Debian/OMV) won discovery over a healthy rootful Docker and
pinned PULSE_DOCKER_RUNTIME=podman plus CONTAINER_HOST/PODMAN_HOST/
XDG_RUNTIME_DIR into the agent unit. The env-application block also ran
for explicit --enable-docker installs. Rootless discovery now defers to
system_docker_runtime_is_active (docker info with DOCKER_HOST stripped,
or a live /var/run/docker.sock probe) before touching rootless sockets.

On the agent, detectRuntime short-circuited to podman whenever the
preference said podman, mislabeling connections that actually fell
through to the Docker socket and disabling Swarm collection. The
preference is now an ordering hint: a podman-preferred connection landing
on a docker endpoint reports docker, while unlabeled endpoints with no
runtime signals still honor the pin. When the bound socket disappears
mid-run the agent re-runs runtime discovery after three consecutive
daemon-unavailable collects, swapping the connection behind a
swappableDockerClient so concurrent goroutines keep a stable handle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 22:49:48 +01:00
courtmanr@gmail.com 07a0d84464 Average host CPU usage across the report interval (#1648)
collectCPUUsage took a blocking 1-second spot sample once per report
interval, which overstates CPU on mostly-idle guests (reported on a
2-vCPU aarch64 KVM guest) and delayed every Collect() by a second.
Track the previous cumulative cpu.Times sample and report busy/total
across the elapsed interval instead, falling back to the spot sample
only for the first collection and counter resets (reboot, migration).
CPUUsagePercent keeps its meaning in the report schema.

Contract-Neutral: hostmetrics sampling behavior change with no subsystem contract surface delta
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 22:49:08 +01:00
courtmanr@gmail.com ad97ae890d Backfill agent-lifecycle governance for the marshal-hook race fix
68e557e9f moved the Docker agent's JSON-marshal hook from a package global
to a per-Agent field, which touches internal/dockeragent/agent.go and
internal/dockeragent/container_update.go. It meant to take the
Contract-Neutral bypass, but a blank line separated the trailer from
Co-Authored-By, so git's trailer parser dropped it, CI ran the completion
guard without a reason, and Canonical Governance went red on three counts:
missing contract docs/release-control/v6/internal/subsystems/agent-lifecycle.md,
missing verification artifact for "agent runtime transport trust proof", and
missing verification artifact for "Docker container recreate, rollback,
durable result, and live network-mode proof". The sibling timer-hook commit
8e5ef365d has the same defect, so this backfill covers both seams.

Document the seams in the agent-lifecycle contract's Current State: both
newTimerFn and jsonMarshalFn are unexported Agent fields reached through the
newTimer and jsonMarshal methods, nil falls back to time.NewTimer and
json.Marshal, no package-level hook global remains in internal/dockeragent,
and injection happens at construction only so the fields need no mutex. The
seams stay internal — no enrollment, transport trust, command admission,
acknowledgement, or recreate/rollback semantics move, and they must not be
promoted into NewAgent options or any server-facing surface.

Pin that shape in the two registered verification artifacts.
agent_internal_test.go asserts the field names, types, and unexported-ness,
parses every non-test source in the package to fail if either hook returns as
a package-level var, proves the nil defaults run the standard library, drives
two Agents with different hooks concurrently so the isolation is checked under
-race, and proves sendCommandAck marshals through the receiver's own seam.
container_update_test.go proves decodeUpdateContainerPayload routes through
a.jsonMarshal, that an injected failure never reaches a sibling Agent, and
pins the method form at compile time.

Guard dry-run over the staged set passes with no Contract-Neutral bypass, and
replaying 68e557e9f's file list plus these three files is green, so the
original commit would have passed had it carried them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v6.2.0-rc.3
2026-07-28 22:33:55 +01:00
courtmanr@gmail.com 68e557e9f0 Fix Docker agent test JSON-marshal race by making the hook per-Agent
Same class of race fixed for newTimerFn in 8e5ef365d: tests swapped the
package-level jsonMarshalFn hook while async goroutines leaked from
earlier tests (sendCommandAck ack retries via runAsync) could still be
reading it, tripping the race detector. Replace the global with a
per-Agent jsonMarshalFn seam (nil defaults to json.Marshal), make the
decode payload helpers Agent methods so they use it, and inject the
failing marshaller into the tests that previously swapped the global.
Verified with go test ./internal/dockeragent/ -race -count=20.

Contract-Neutral: test seam refactor to fix data race, no public contract delta

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 17:16:06 +01:00
courtmanr@gmail.com d1ee6e928b Fix multi-organization Proxmox connection identity 2026-07-28 17:15:54 +01:00
courtmanr@gmail.com 8e5ef365d4 Fix Docker agent test timer race by making the timer hook per-Agent
Tests swapped the package-level newTimerFn hook while async goroutines
leaked from earlier tests (backup-cleanup and stop-command paths) were
still reading it in waitForAsyncDelay, tripping the race detector on CI.
Replace the global with a per-Agent newTimerFn seam (nil defaults to
time.NewTimer), make waitForContextDelay an Agent method, and inject the
immediate timer into the tests that previously swapped the global.
Verified with go test ./internal/dockeragent/ -race -count=20.

Contract-Neutral: test seam refactor to fix data race, no public contract delta

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 16:27:30 +01:00
rcourtman 16ff5544c1 Prepare v6.2.0-rc.2 release
Second candidate on the v6.2.0 line, superseding v6.2.0-rc.1 and still
following stable v6.1.2. This is a hardening and bugfix candidate:
auto-update reliability including in-place migration of already-deployed
update units, PBS backup attribution across multiple Proxmox clusters,
Proxmox installer registration including per-canonical-type bootstrap
grants on combined PVE+PBS hosts, Patrol readiness streaming transport
plus a completed-verdict readiness gate, discovery-policy DNS and SSH
retry churn, explicit per-metric threshold off toggles, request-derived
SSO callback URLs, and the new Entra ID SSO guide. Version pins move to
6.2.0-rc.2 across the repo root, Docker bootstrap defaults, and Helm
metadata per the deployment-installability contract; stable install
pointers remain on v6.1.2 until governed promotion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v6.2.0-rc.2
2026-07-28 16:05:56 +01:00
courtmanr@gmail.com 72599bd1ec Allow one Proxmox bootstrap per canonical type on combined hosts (#1644)
A host running both PVE and PBS is a deployment the docs call officially
supported, and the agent's RunAll registers each product in turn from the
one install token. The bootstrap grant recorded consumption per TOKEN, so
the PVE leg spent it, the PBS leg came back canRegister=false, the agent
wrote a proxmox-pbs-registration-blocked marker, and install.sh printed an
ERROR banner over a PVE source that had registered perfectly well.

- server: consumption is now recorded per canonical type. One PVE create
  and one PBS create per token, each still one-shot — a second create of
  the same type takes the same 403. The bounds that are not per type stay
  singular: the 24h mint-age clock and the first-use bound_hostname are
  shared, so whichever type registers first pins the hostname for both and
  the second type cannot be aimed at another machine. The per-type ledger
  lives in proxmox_registration_consumed_types; a record carrying only
  proxmox_registration_completed=true predates it and still reads as every
  type consumed, so upgrading cannot revive a token already spent in the
  field. The unsuffixed completion block keeps tracking the most recent
  completion, which also leaves an older binary reading the same store
  failing closed.

- rollback: the consume-before-persist undo is scoped to the keys one
  consumption wrote, so a failed PBS source save restores the PBS grant
  without resurrecting the PVE grant that already produced a source.

- agent: RunAll no longer lets one product's failure speak for the host. It
  attempts and returns the remaining products, errors only when every
  detected product failed, and publishes the detected products in a
  proxmox-detected-types state marker.

- installer: report_proxmox_registration_outcome reads that marker, waits
  for an outcome from each detected product, and prints a success or denial
  line per product instead of one verdict. Agents predating the marker keep
  the old first-outcome-wins timing so a single-product host does not wait
  out the window. The blocked-marker path now only fires on genuine refusals.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:03:55 +01:00