Commit Graph

3059 Commits

Author SHA1 Message Date
rcourtman f9b10316b8 fix(licensing): surface unreadable persisted v5 licenses and retry failed startup exchanges
Two silent-downgrade paths in the v5→v6 migration are now visible and
self-healing:

License load/decrypt failure (was: one log line, no UI state): when
license.enc exists but cannot be read, getTenantComponents now persists
a terminal commercial_migration state with reason
persisted_license_unreadable, so the licence panel and global banner
tell the customer to re-enter their v5 key instead of leaving them to
discover missing Pro features.

Failed startup exchange (was: once per process, Community until manual
restart): exchange failures classified as pending — license-server
blips, DNS failures, rate limits — now schedule a background retry loop
with backoff (30s → 30m cap) that re-attempts the exchange until it
succeeds, hits a terminal classification, or the org's service activates
through another path. The loop stops cleanly on manual activation and
StopAllBackgroundLoops.

Registers the new files in the subsystem registry (cloud-paid
commercial-migration verification policy + shared ownership) and pins
both behaviors in the cloud-paid and api-contracts contracts.
2026-06-11 08:44:35 +01:00
rcourtman cce1b3c783 fix(licensing): decrypt v5.0.0-era license.enc sealed with raw machine-id material
v5.0.0 derived the license.enc AES key as sha256("pulse-license-" +
material) where material was the raw /etc/machine-id file content —
trailing newline included — with hostname and a fixed dev fallback as
alternates. v6 trims machine-id before hashing and dropped both
fallbacks, so licenses activated on v5.0.0 (before .license-key shipped
in v5.0.1) could never be decrypted and the December 2025 lifetime
cohort silently booted as Community.

Collect the raw v5 key materials at construction and try them verbatim
as legacy sha256 candidates during decrypt, alongside the existing
trimmed HKDF/legacy derivations. The write path is unchanged: persistent
random key + HKDF only. Pins the cloud-paid persistence contract to the
full v5.0.x compatibility-loader material set.
2026-06-11 08:36:11 +01:00
rcourtman e8455e84b7 fix(resources): pin canonical host IDs to durable identity so they survive restarts
Canonical IDs for merged-source hosts (PVE node + pulse-agent) were minted
from whichever identity keys the creating record happened to carry: the
agent record knows the machine ID, the Proxmox node record only knows
cluster+hostname. The registry rebuilds from scratch every tick, so a boot
window where the agent had not checked in yet minted a cluster-keyed ID
(agent-7a62... for delly) while steady state minted a machine-keyed one
(agent-bdd4...). Every restart re-ran the race, fragmenting the
resource_changes journal into per-boot eras (9.4k vs 6.1k rows for the
same host) and silently truncating report availability and UI timelines.

Fix, in the layer that owns identity:
- Persist identity pins (canonical_id <-> machine_id/dmi/cluster/hostname)
  in the previously schema-only resource_identities table, written by the
  store-backed registry after monitor-adapter rebuilds, diff-aware so
  steady-state ticks cost no writes.
- Complete weak incoming identities from the pins before matching and ID
  derivation, so a node-only boot window derives the same machine-keyed
  canonical ID as steady state. Derivation itself is unchanged; ephemeral
  nil-store registries behave exactly as before.
- Expand change-journal reads (Get/Count families, SQLite and memory) to
  the full era set recomputed from the pinned identity keys, healing
  historical journals at query time with no row migration. Reads keyed by
  a stale era ID resolve to the same merged timeline.

Regression tests cover both ingest orders, restart simulation via the
monitor adapter, era ID derivation, and era-merged journal reads on both
store implementations. Contracts updated: unified-resources obligation 25
(durable identity pins), monitoring obligation 10 (adapter rebuild
persistence).
2026-06-11 08:33:00 +01:00
rcourtman 3a51429a58 fix(licensing): honest installation-limit copy and v6 connectivity disclosure
Docs and frontend half of the licensing-policy client work from the v6
upgrade sweep (license findings 5 and 6); the Go classifier and grant
refresh fix land separately once the shared pre-commit guard unblocks.

A 409 MAX_INSTALLATIONS during v5 key migration rendered as 'another v6
activation handoff is still settling. Retry activation from this
instance.' Retrying can never succeed until a slot is freed server-side,
so that copy stranded users. The notice now says the key is already
active on its maximum number of v6 installations and points at
support@pulserelay.pro to free a slot, via the new
exchange_installation_limit reason and free_installation_slot action.

docs/UPGRADE_v6.md gains a 'Breaking Change: Paid Licensing Requires
Connectivity' section: v5 validated keys fully offline, v6 needs
periodic connectivity to license.pulserelay.pro, with roughly 10 days of
offline tolerance (72 hour grant plus 7 day grace) before paid features
drop to Community, plus air-gap guidance and the migrated-key
installation cap. docs/releases/V6_CHANGELOG.md gains the matching
breaking-change bullet.

Note: the two frontend files also carry two small additive hunks from
the parallel persisted-license-unreadable slice (a copy case and its
test, inert until its Go side lands); they share these files and are
included to avoid splitting them mid-flight.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-11 08:32:07 +01:00
rcourtman 5e78825f54 fix(install): make unattended auto-updates actually run after install and upgrade
Two GA-blocking bugs in the auto-update install path:

The pulse-update.service unit was rendered through an unquoted heredoc
containing $${PULSE_SERVICE_NAME}, which bash expands to the installer's
PID, so the unit shipped with ExecCondition checking a service like
"24757{PULSE_SERVICE_NAME}". The condition always failed and systemd
silently skipped every scheduled run, for fresh installs and upgrades
alike. The heredoc now renders the detected service name directly:
ExecCondition=/bin/sh -c 'systemctl is-active --quiet pulse'.

Upgrades also never refreshed the updater assets: a v5 box with
auto-updates enabled keeps pulse-update.timer and autoUpdateEnabled=true,
so the update/reinstall flows never re-ran setup_auto_updates and the
v5.1-pinned /usr/local/bin/pulse-auto-update.sh stayed in place, logging
"Already running latest version" forever instead of selecting v6
releases. The asset-install half of setup_auto_updates is now a shared
install_auto_update_assets(), and a new refresh_auto_updates() rewrites
the helper script and units unconditionally whenever the timer already
exists, without touching system.json or the timer's enabled state. All
five install flows (update, reinstall, --version, --source, fresh) are
wired.

Tests now render the real unit and execute the rendered ExecCondition
against a recording systemctl stub instead of asserting source-text
fragments, pin refresh_auto_updates behavior (stale helper replaced,
system.json and enablement untouched), and pin the call-site wiring.
The deployment-installability contract records both invariants.

Note: rewritten message only — the original local commit carried a
parallel agent's licensing commit message due to a shared-index race;
the tree is byte-identical to the hook-verified original.
2026-06-10 22:43:08 +01:00
rcourtman 43bc15c12f fix(config): restore legacy PORT fallback for FRONTEND_PORT with deprecation warning
v5 documented PORT as a legacy alias for FRONTEND_PORT. v6 dropped the
fallback silently, so an install configured with PORT=8080 upgrades to
v6 and comes up on 7655 with no explanation, unreachable behind its
existing port mapping or reverse proxy.

Honor PORT again at lowest precedence (FRONTEND_PORT wins when both are
set) and log a deprecation warning at startup, mirroring the existing
legacy DISABLE_AUTH handling. Document PORT as deprecated in
CONFIGURATION.md and note the compatibility behavior in the v6
changelog.
2026-06-10 22:32:28 +01:00
rcourtman 3229ed5bd1 Document connecting client Proxmox/PBS over the provider VPN in MSP.md
Polling direction (provider to client site, 8006/8007) versus agent
check-in direction, per-client steps with the guided pveum setup command,
privilege-separated token caveat, TOFU certificate pinning, and the
overlapping-RFC1918 note that container-per-client isolation makes moot.
2026-06-10 14:35:14 +01:00
rcourtman a17882458a Rename CP_TRIAL_ACTIVATION_PRIVATE_KEY to CP_ENTITLEMENT_SIGNING_PRIVATE_KEY in the provider MSP bundle
The variable signs hosted entitlement leases; the trial-activation name is
left over from the retired trial era and reads as trial machinery to an
operator generating their licensing root key. The provider MSP bundle has
no installed base yet, so the canonical rename is free today and frozen
the moment the first design partner installs.

- Control plane reads CP_ENTITLEMENT_SIGNING_PRIVATE_KEY first and falls
  back to CP_TRIAL_ACTIVATION_PRIVATE_KEY, so existing Pulse-hosted cloud
  deployments (deploy/cloud, hibernated snapshot) keep working unchanged.
- deploy/provider-msp (.env.example, compose, setup.sh), MSP.md, and the
  install-test pins use the canonical name; error messages name it too.
- deploy/cloud intentionally keeps the legacy name: that stack historically
  signed hosted trial activations, and its snapshot predates the rename.
2026-06-10 14:08:52 +01:00
rcourtman 2482d4acb6 Document provider-MSP deploy bundle as canonical install; surface lease signing public key in setup.sh
- MSP.md now leads with deploy/provider-msp/ (compose stack, setup.sh,
  upgrade.sh, run-install-proof.sh), documents the HTTPS requirement
  (__Host- portal session cookie) and the pulse.provider-msp.role labels
  workspace provisioning requires, and explains the licence/lease chain
  including licence-expiry behavior.
- setup.sh derives and prints the lease signing public key the provider
  MSP licence must bind (also via --print-lease-signing-public-key), and
  the missing-licence error now includes it with request instructions.
- .env.example documents the CP_TRIAL_ACTIVATION_PRIVATE_KEY binding.
2026-06-10 14:00:11 +01:00
rcourtman 96001d134e Harden MSP tenant isolation: scope org-bound tokens away from default org, propagate webhook allowlist to all tenants
Two gaps found by exercising the MSP pilot path live on a throwaway
multi-tenant instance:

1. CheckAccess granted any authenticated principal access to the default
   org, so a token bound to a client org could read the provider's own
   default-org estate if it leaked from a client site. Org-bound tokens
   now fall through to the explicit binding check for the default org;
   authenticated users and legacy unbound tokens keep default-org access,
   and binding "default" explicitly still grants it.

2. The webhook private-target allowlist (instance-wide system setting)
   only ever reached the default org's notification manager on
   startup/reload, and only the request-context org on settings update.
   Tenant orgs' webhooks to private targets (per-client Gotify over VPN,
   the canonical MSP alert route) failed SSRF validation with no org-side
   remedy, and any allowlist died with a restart. Settings updates and
   reloads now fan out to every live tenant manager via the new
   MultiTenantMonitor.ForEachMonitor, and tenant monitors inherit the
   persisted allowlist and public URL at creation.

Both fixes verified live: org-bound token vs default org returns 403;
client-org webhooks to a private target succeed after restart and for
orgs created after the allowlist was saved. MSP.md validation checklist
gains the default-org probe and the allowlist guidance; MULTI_TENANT.md
documents the binding semantics. Contracts updated for api-contracts,
security-privacy, and monitoring with adjacency notes for
agent-lifecycle, storage-recovery, and performance-and-scalability.
2026-06-10 11:37:39 +01:00
rcourtman cbd0311055 Inject workspace display name into hosted tenant runtime env
Hosted tenant containers received PULSE_TENANT_ID but not
PULSE_TENANT_NAME, so alert webhook payloads from provider-hosted client
runtimes fell back to the raw tenant ID instead of a human-readable
workspace label. Resolve the display name from the tenant registry at
container-create time via a ManagerConfig resolver and stamp it
alongside the tenant ID. Display-name changes after creation apply on
the next runtime rollout, which recreates the container with freshly
resolved env.
2026-06-10 11:13:37 +01:00
rcourtman 1cf92e6c10 Pin metadata GET zero-record payload contract
Adds TestContract_MetadataGetPayloadsUseZeroRecordsInsteadOf404: empty
guest/docker metadata maps must serialize as {} (never null) and a
missing resource must return a 200 zero record echoing the requested ID
(never a 404). This is the proof companion to the
metadata_handlers_shared.go consolidation in the previous commit — it
was authored with that change but lost to a shared-index race at commit
time.
2026-06-10 10:53:57 +01:00
rcourtman 6340cd36f2 Dedupe internal/api handler families behind shared flows and generics
Clears the sixteen dupl pair groups in internal/api plus the pkg/pulsecli
pair:

- router.go: privileged settings endpoints share the
  serveSetupTokenOrSettingsWrite gate; patrol findings convert via one
  unifiedFindingFromAI; the five infrastructure-summary chart loops share
  collectGuestChartData / fillChartSeriesFromBatch; the VM/LXC workloads
  summary loops share appendGuestWorkloadSummaries.
- deploy_handlers.go: preflight and job status/SSE handlers share
  handleDeployJobStatus / handleDeployJobEvents.
- recovery_handlers.go: points series/facets share
  parseRecoveryListPointsOptions.
- truenas_handlers.go / vmware_handlers.go / router_routes_registration.go:
  the connection update flow (locate, decode-with-fallback, normalize,
  preserve masked secrets, validate, save, redact) moves to the new
  platform_connection_shared.go (updatePlatformConnection +
  decodeOptionalInstanceRequest + the admin-gated item-route builder);
  per-platform wrappers carry nolint'd declarative wiring only.
- docker_metadata.go / guest_metadata.go: GET/PUT payload semantics move
  to metadata_handlers_shared.go; the zero-record-instead-of-404 contract
  is pinned by TestContract_MetadataGetPayloadsUseZeroRecordsInsteadOf404.
- kubernetes_agents.go / docker_agents.go: lifecycle PUTs share
  per-handler action helpers.
- config_node_handlers.go: PBS/PMG probes share
  testProxmoxPlatformConnection.
- cloud_handoff_handlers.go / purchase_return_redemptions.go: secrets
  sqlite stores open through openHardenedSecretsDB so permission
  hardening stays single-sourced.
- ai_handlers.go / chat_service_adapter.go: the GetMessages adapters are
  deliberate contract mirrors — suppressed with nolint and enforced by
  TestOrchestratorAndChatAdaptersMapTheSameMessageFields.
- pkg/pulsecli/actions.go: action subcommands seed env defaults via
  actionAPIDefaults (tested); the audit/events cobra registration pair is
  nolint'd parallel wiring.
- .golangci.yml: exclude gitignored tmp/ from ./... typechecking.
- subsystem_lookup_test.py: refresh the pinned api-contracts.md line
  numbers shifted by the contract additions.

golangci-lint run ./... is now fully green. Full internal/api and
pkg/pulsecli test suites pass.
2026-06-10 10:52:05 +01:00
rcourtman 26c7ca910c Document the MSP delivery surface: webhook contract, PSA samples, provider ops guide
- WEBHOOKS.md: stable Delivery Contract section (alert/resolved events,
  two-level severity model, alert types, tenant identity fields, retry
  layers, X-Pulse-Event-ID idempotency, signingSecret HMAC verification
  with a receiver example), plus full PSA sample payloads for critical,
  warning, and resolved events and ConnectWise mapping guidance.
- New MSP.md provider operations guide: deployment models, split-port
  ingress topology with firewall baseline, a no-host-jump validation
  checklist (agent port 404s, agent-token scope rejection, cross-tenant
  403 probe), per-client alert routing and reports, branding scope per
  deployment model, licensing and limits.
- docs index links the new guide.
2026-06-10 10:44:34 +01:00
rcourtman a223b02dd5 Stamp tenant identity into alert webhook payloads
Alert webhook payloads now carry the tenant that fired them, so MSP/PSA
receivers (ConnectWise and similar) can route tickets by client without
inferring the tenant from which webhook endpoint fired.

- WebhookPayloadData gains TenantID/TenantName, exposed to custom
  templates as {{.TenantID}}/{{.TenantName}}.
- Defaults come from PULSE_TENANT_ID/PULSE_TENANT_NAME (already injected
  into provider-hosted client runtimes; name falls back to ID).
- Shared-process multi-tenant orgs override via a lazy org-backed
  resolver wired in MultiTenantMonitor.GetMonitor, so display-name
  renames are picked up without restart.
- Generic service template emits a tenant block when identity is set,
  omits it otherwise; single-tenant payloads are unchanged.
- Notifications and monitoring subsystem contracts updated with the
  tenant-identity ownership boundary; guardrail test pins the org
  wiring.
2026-06-10 10:03:39 +01:00
rcourtman 8439ce6e6b Dedupe AI runtime guest-family, tool-pipeline, and provider clones
Clears the ten dupl pairs across internal/ai:

- patrol_intelligence.go, tools_query.go, tools_storage.go: VM and LXC
  system-container paths collapse into generics over read-state view
  method subsets (gatherGuestIntelligenceFromViews, canonicalGuestGetResult
  + guestViewGetResult, addCanonicalGuestSearchMatches +
  addGuestViewSearchMatches, appendGuestDiskSummaries).
- tools_file.go: append/write share executeFileMutation driven by
  fileMutationSpec (approval-command text, shell redirect, verification
  strategy stay per-action and verbatim).
- tools_kubernetes.go: deployment restart / pod delete share
  executeKubernetesResourceAction driven by kubernetesResourceAction.
- providers/anthropic.go + anthropic_oauth.go: message conversion shared
  via convertMessagesToAnthropic (the OAuth copy was annotated 'same as
  regular client').
- memory/changes.go + memory/remediation.go: history loading shared via
  the generic loadMemoryHistory in memory/paths.go (10 MiB cap, sort,
  missing-file semantics preserved via a found flag).
- findings.go and unified/alerts.go: Finding/findingJSON and
  UnifiedFinding/unifiedFindingJSON are deliberate marshal-mirror twins
  (AlertIdentifier json:"-" vs alert_identifier round-trip); merging
  would break every public literal. Suppressed with nolint:dupl and
  enforced instead by new reflect-based mirror-sync tests.

Contract Extension Points name the shared helpers and the mirror
invariant. Full ./internal/ai/... test tree passes.
2026-06-10 09:38:52 +01:00
rcourtman b855b21c7f Dedupe monitoring pollers behind shared provider, lifecycle, and seeding helpers
Clears the six dupl pairs in internal/monitoring:

- poll_providers.go: PVE/PBS/PMG listInstances / describeInstances /
  connectionStatus closures collapse into generic sortedClientNames,
  describeProviderInstances, and providerConnectionStatuses helpers; the
  PBS/PMG adapters are now built by newPrefixedPollProvider from a
  prefixedPollProviderSpec (prefix drives status keys, health keys, and
  the fallback instance name).
- truenas_poller.go / vmware_poller.go: the Start scheduling loop
  (double-start guard, stopped-channel handshake, sync+poll cadence)
  moves to startPollerLoop, and the active-connection config policy
  (enabled instances keyed by trimmed connection ID, defaults applied)
  moves to loadActiveInstanceConfigs, both in the new
  platform_poller_shared.go.
- monitor_polling_vm.go / monitor_polling_containers.go: traditional
  polling now records guest series via the existing canonical
  recordGuestMetric helper (io/network sentinels preserve the historical
  cpu/memory/disk-only behavior on this path); the source-shape
  guardrails in canonical_guardrails_test.go and
  memory_source_catalog_test.go pin the new delegation.
- mock_metrics_history.go: native and TrueNAS disk seeding share one
  seedDiskTelemetry closure.

Proof: new TestTrueNASPollerDoubleStartKeepsSingleLoop pins the shared
lifecycle loop; new TestSeedMockMetricsHistory_DiskTelemetryParityAcrossNativeAndTrueNAS
pins four-series parity for both disk sources. Contract Extension Points
name the shared wiring layer and poller scaffold.

Full internal/monitoring test suite passes.
2026-06-10 09:19:51 +01:00
rcourtman 74417de150 Share one Resource scaffold across namespaced Kubernetes adapters
The k8s workload/config adapters in unifiedresources each hand-rolled the
same Resource literal (Technology, LastSeen, UpdatedAt, Kubernetes facet,
label tags) plus namespacedKubernetesIdentity return; dupl flagged the
StatefulSet/Job and ReplicaSet/DaemonSet clones. Fold the scaffold into
namespacedKubernetesResource and delegate all 19 namespaced adapters
(Service included) to it; cluster-scoped kinds (PV, StorageClass,
Namespace) keep bespoke identity construction.

Contract: unified-resources Extension Points now name the shared scaffold
as the way new namespaced kinds assemble resources. Proof: new
TestNamespacedKubernetesResourceScaffold pins the scaffold fields and
namespaced hostname identity.

Full internal/unifiedresources test suite passes.
2026-06-10 09:16:14 +01:00
rcourtman f4d57c6836 Label key-collision lifecycle events in the finding timeline
Completes the deferred UI half of dad1152fc: the content_replaced
lifecycle event (emitted when a same-key re-detection's text is
substantially different from the existing finding — a key collision)
now renders as "Re-detected with different details" in the finding
timeline instead of the identifier-formatter fallback, so the operator
reads what actually happened in plain language. The event metadata
already carries the previous and new titles.

Ceremony this line deferred from the backend commit: patrol-intelligence
contract Current State entry (lifecycle label map rule + the new label),
ai-runtime cross-reference updated, and the label pinned in
FindingsPanel.test.ts lifecycleLabels (the subsystem's accepted proof).
111 FindingsPanel tests green.
2026-06-09 22:10:53 +01:00
rcourtman dad1152fc4 Surface finding-key collisions as lifecycle events instead of silent identity hijack
Patrol finding identity is the LLM-assigned key (resource+category+key
hash -> ID), and recordFinding adds by plain ID merge: when the LLM
reuses a key for a genuinely different issue on the same resource, the
new report silently overwrites the existing finding's title,
description, and evidence while inheriting its lifecycle — and on a
resolved finding, the reactivation counts as a regression of the OLD
issue, inflating the regression counter with a fiction.

Forking the key on dissimilar content would be worse: LLM titles vary
run to run, and splitting one real issue into duplicate findings is a
bigger trust hit than a conflated history. So the merge semantics stand,
and the identity shift is recorded honestly: when a same-ID
re-detection's title shares essentially no keywords with the existing
title (keywordOverlap <= findingIdentityShiftMaxTitleOverlap, 0.2 —
resource/category/key are equal by construction so text is the only
discriminating signal), FindingsStore.Add appends a content_replaced
lifecycle event preserving both titles in metadata and logs the
collision for frequency observability. Rephrasings and identical
re-detections stay event-free per the heartbeat rule. The UI renders
the event through the existing identifier-formatter fallback ("Content
replaced"); a dedicated label is patrol-intelligence UI work and lands
with that subsystem's own ceremony.

Teeth in findings_lifecycle_test.go: distinct-issue collision records
exactly one event with both titles; rephrased and identical
re-detections record none; the resolved-finding collision shows BOTH
content_replaced and regressed so the regression can be read for what
it is. Contract: Current State entry in ai-runtime.md. Full internal/ai
tree green.
2026-06-09 22:03:20 +01:00
rcourtman ab9552716c Auto-clear event/persistent Patrol findings on affirmative deterministic verification
The finding lifecycle was asymmetric around the deterministic-resolve
gate: the gate correctly blocks LLM resolves of event/persistent
findings (backup, reliability, security, general) when the verifier
still detects the signal, but nothing ever cleared such a finding when
the underlying issue WAS fixed — absence-based stale auto-resolve only
covers performance/capacity, so a fixed backup stayed an active finding
indefinitely unless the LLM happened to call patrol_resolve_finding.

reconcileStaleFindings now runs the deterministic verifier for seeded,
unreported event/persistent findings whose key has one, and resolves
ONLY on an affirmative "signal gone" verification. Still-present or
inconclusive results fail closed (same standard as the resolve gate);
verifications are capped per run (3) with deferred candidates logged
and retried next run. Test seam: PatrolService.verifyFixResolvedFn.

Two enabling repairs found during verification:
- hasDeterministicVerifierForKey listed 2 of the 7 keys the
  verifyFixDeterministically dispatch handles, so the LLM-resolve gate
  silently skipped existing verifiers for backup-stale and
  guest-unreachable findings. Now aligned; documented as the single
  source of truth for both consumers.
- The finding-key vocabulary was forked: the patrol_report_finding tool
  suggested keys (high-cpu, ...) the verifier switch never matched, so
  verification rarely engaged for new findings. normalizeFindingKey now
  aliases unambiguous directional synonyms onto the canonical verifier
  vocabulary (high-cpu -> cpu-high etc.; pbs-job-failed and node-offline
  deliberately NOT aliased — different resource models), and the tool
  description teaches the canonical keys.

NOT changed: the synthetic ai-patrol-error finding — verification showed
the reported "accumulation" is a non-problem (deterministic ID, store
merges repeats as heartbeats, successful runs auto-resolve it).

Teeth: 8 new tests in patrol_reconcile_test.go including the
idempotence invariant (repeat reconcile over unchanged state produces
zero resolves and zero lifecycle growth) and the cap-defers case.
Contract: verified stale-resolve clause added beside the
deterministic-resolve-gate in ai-runtime.md. Full internal/ai tree green.
2026-06-09 21:47:14 +01:00
rcourtman cb1ced8b8c Add the interaction-quality scenario corpus (chat-feel regression teeth)
Consolidates the OpenCode-restraint arc's user-visible promises into one
runnable corpus, mirroring the Discovery corpus pattern
(internal/servicediscovery/scenario_corpus_test.go). Each scenario
drives a full ExecuteStream turn against a scripted provider and pins
the browser-facing event stream: ordered/forbidden event types,
answer-text teeth, and payload teeth.

Six scenarios: clean plain-answer turn (done stamped with the model
route, no tool noise), greeting answered directly with tools offered
but unused, tool turn rendered as compact tool events with real tool
names and no provider call ids in the answer, clean no-narrative
fallback sentence (no raw JSON / call_ ids), invisible pre-event
provider retry (no error event), and exactly one clear error event on
terminal provider failure.

Teeth proven by mutation probes: a wrong ordered type and a wrong
answer assertion both fail with the promise named in the message.
Corpus documented in ai-runtime.md Current State as the canonical home
for interaction-quality regressions, so Patrol-phase churn on the
shared agentic loop cannot silently regress the chat feel.
2026-06-09 21:19:25 +01:00
rcourtman 945ed2198d Make the Assistant resolve a missing target before asking, not deflect
The system prompt framed ask-first as the only safe behavior for a
missing target ("Missing target information is not a safe default...
ask for the missing target"), so any "run X" / "check Y" request
without an explicit host deflected back to the operator — including on
single-host deployments with no real ambiguity. OpenCode-parity gap:
a competent operator looks first.

New policy: resolve-before-asking. Use read-only query/topology tools
to identify plausible targets; if exactly one plausible target matches,
run read-only diagnostics against it and name it in the answer; ask
only when several plausible targets remain or the action changes state.
Unchanged safety: placeholder targets (current_resource outside an
attached-resource turn) stay forbidden in all modes, never guess an
unresolved target, and write actions still need an explicit target.

Contract: resolve-before-asking entries in Extension Points and Current
State supersede the ask-first framing. Proof:
TestBuildSystemPrompt_CurrentResourceRequiresResourceHandoff pins the
new boundary strings. Live "just run X" behavior check is owed to the
interaction-quality corpus (next unit); dev-instance agent exec is
currently broken (known command-token issue), so it cannot be observed
end-to-end from here today.
2026-06-09 21:12:45 +01:00
rcourtman 36b9b2a53d Make Assistant tool selection model-owned; remove the prompt-keyword tool-scope router
assistantToolScopeForPrompt classified prompt wording into text-only /
query-only / full tool scopes — the prompt-keyword router the ai-runtime
contract forbids. It withheld every tool from greetings and exact-reply
turns, zeroed the manifest on false-positive phrases ("before using any
tools, tell me your plan"), filtered inventory prompts to pulse_query
only, and its query-only path injected a prefetched topology payload
into the user message and then withheld tools entirely.

Now every interactive turn that reaches the selected model carries the
full governed manifest from toolsForExecutionMode; the model decides
whether to answer, ask, or call tools. Removed with the router: the
query-only topology prefetch + text-only downgrade, the inventory
sanitizer allow-list span, and the summary-only pulse_query input
rewrite (preferSummaryOnlyQueries) that mutated model-chosen tool
inputs. The two contract-sanctioned behaviors survive unchanged: the
context-only resource handoff manifest boundary, and the deterministic
count-only local answer (pulse:local-inventory), now gated directly by
assistantPromptQualifiesForLocalInventoryCount whose false positives
fail safe — to the model, never away from it.

Contract: superseded the query-only/direct-text scoping paragraphs in
ai-runtime.md with the model-owned manifest rule. Proofs:
TestService_ExecuteStream_ToolManifestIsModelOwned,
TestService_ExecuteStream_InventoryBreakdownIsModelOwned,
TestAssistantPromptQualifiesForLocalInventoryCount. Full internal/ai +
internal/api trees green; chat package lints clean.
2026-06-09 21:07:28 +01:00
rcourtman 16f791a656 Harden install-time PVE token extraction with JSON-first parsing (#44/#1312)
The install-time auto-register path (auto_register_pve_node) parsed the API
token secret out of pveum's box-drawing table output with a fragile awk
column-split. The web-setup render path was already hardened to request
'pveum ... --output-format json' first and parse the value field, but this
secondary install.sh path was never ported.

auto_register_pve_node now requests --output-format json first (falling back
to the bare --privsep 1 form only when an older pveum rejects the JSON flag,
which keeps the secure-installer contract pin on that form satisfied) and
extracts the secret via a new extract_pve_token_value helper: JSON value-field
parse first, then a locale-independent box-drawing table fallback (normalizes
the column separator to a plain pipe byte-wise before splitting, so it works
regardless of host locale). This mirrors the hardened render path and removes
the silent-failure / mis-parse risk when pveum table formatting drifts.

Functional + contract tests in root_install_sh_test.go; deployment-installability
contract documents the deterministic extraction. The host-agent path
(internal/hostagent/proxmox_setup.go setupPVEToken) carries an agent-lifecycle
token-permission proof obligation and is left for a governed lane.
2026-06-09 15:47:12 +01:00
rcourtman 2ff2b18c74 Wire legacy sensor-proxy cleanup into install.sh --uninstall (#34)
A Proxmox host upgraded from v5 may still carry the legacy pulse-sensor-proxy
footprint (binary, systemd units, runtime/state dirs, dedicated service user,
and managed SSH keys in root's authorized_keys). install.sh --uninstall removed
everything for the Pulse server itself but left that legacy footprint behind,
so a 'complete uninstall' was not complete -- most notably it left SSH key
entries in /root/.ssh/authorized_keys.

uninstall_pulse now calls cleanup_local_sensor_proxy, which removes the LOCAL
footprint only: stop/disable the units, remove the binary/units/runtime/state
dirs, strip the '# pulse-managed-key' / '# pulse-proxy-key' entries, and remove
the service user/group. It is presence-gated (silent no-op when no proxy was
installed). The aggressive cluster-wide authorized_keys removal and
pulse-monitor@pam API-user deletion stay behind the explicit standalone
scripts/uninstall-sensor-proxy.sh, which we print a pointer to.

Functional + contract tests in scripts/installtests/root_install_sh_test.go;
deployment-installability contract documents the new uninstall removal scope.
2026-06-09 15:25:08 +01:00
rcourtman 55dfdcb1d3 Instruct the Assistant to attribute facts and caveat staleness
The model now receives provenance — per-fact source/confidence and discovery
freshness — but nothing told it to use it, so answers stated facts the user
couldn't trace or weigh. Add a GROUNDING & PROVENANCE section to the base system
prompt: briefly attribute facts to their source ("Debian 12, per
/etc/os-release"), note recency for time-sensitive claims, do not present stale
context as current, and keep attribution concise rather than citing every line.

This is the prose-attribution layer of the provenance work (the visible part
chosen as the first, lowest-risk step over a heavier citations UI). Whether the
model attributes well in practice is a live-behavior question to verify by
exercising the Assistant — the prompt only guarantees the instruction reaches it.

- Test: TestBuildSystemPrompt_IncludesProvenanceGuidance.
- Contract: ai-runtime base prompt must instruct provenance attribution.
2026-06-09 10:53:05 +01:00
rcourtman 9ec52406ad Carry discovery freshness into the cloud-safe context
The pushed cloud-safe operational context told the model a resource's access
pattern, paths, and ports but not how OLD the discovery was — so the Assistant
could present a 2-week-old cached scan as current. For a monitoring assistant,
recency is the most important provenance signal.

FormatCloudSafeContext now appends "Last discovered: <age>" (via the existing
FormatDiscoveryAge helper, previously unused) when the timestamp is known, and
the push-path conversion (cloudSafeOperationalContext) carries UpdatedAt through.
A timestamp is non-identifying, so it adds no PII. Omitted when unknown.

- Tests: FormatCloudSafeContext freshness present/absent; cloudSafeOperationalContext
  carries UpdatedAt end of the push conversion (and still emits no PII).
- Contract: ai-runtime cloud-safe context must carry discovery age when known.

Follow-up: same freshness line on the local/full path (formatSingleDiscovery).
2026-06-09 10:46:10 +01:00
rcourtman a24e944f1b Carry per-fact discovery provenance through to the model
buildDiscoveryToolResponse rebuilt each fact as {category,key,value} and dropped
the Source (the command/origin that produced it) and Confidence the DiscoveryFact
already carries — so the model could state facts it couldn't attribute or weight.
Serialize both per fact (omitted when empty) so the Assistant can report "Debian
12, per /etc/os-release" instead of a bare, untraceable claim.

This is the first piece of the provenance/trust groundwork: make the model
*receive* the source metadata it needs to cite. Surfacing it to the user (UI) and
push-path discovery freshness are follow-ups.

- Test: TestBuildDiscoveryToolResponse_IncludesFactProvenance (source+confidence
  present when set, omitted when empty).
- Contract: ai-runtime section 8 documents the pulse_discovery fact provenance.
2026-06-09 10:35:29 +01:00
rcourtman ec91dd43ea Deliver cloud-safe operational context to the model on the resource handoff path
The drawer "Ask Assistant" handoff anchored a resource but never delivered its
operational context (access command, config/data/log paths, ports) to the model
— that only reached the model via @-mentions. Route handoff resources through
the same prefetch path so the proactive path matches the @-mention path on cloud
turns; PII (hostname/IP/alias) stays redacted at the model boundary.

- Reconcile the handoff Data Boundary directive: the model may use the PII-free
  operational context to answer and guide the user, while still never revealing
  raw hostnames/IPs/aliases/secrets. Action Boundary unchanged (read-only; any
  mutation goes through the governed approval flow).
- Fix a pre-existing clobber: the plain-text resource resolver overwrote the
  prefetch summary on the @-mention path, dropping the operational context
  before it reached the model. Run it only as a fallback when no structured
  mention resolved.
- Add the first end-to-end test asserting the operational context reaches a
  cloud model on BOTH the handoff and @-mention paths while PII is redacted.
- Update the ai-runtime contract to reflect the reconciled handoff data boundary,
  handoff operational-context parity, and the plain-text fallback ordering.
2026-06-09 10:18:52 +01:00
rcourtman bdb212744c Remove the cloud_context_privacy dial; fix cloud context to a lean posture
Per maintainer decision: the cloud-context-privacy feature was bloat. The real
fix for the "useless Assistant on cloud" problem was the earlier sensitivity
recalibration (ordinary workloads = Internal, not redacted); the dial layered a
configurable knob on top of an already-solved problem, guarding mostly-non-secret
data on a destination the operator opted into, and demanded every model-bound path
stay dial-aware (a standing leak surface). The privacy control users actually
understand is the choice of model — cloud provider vs. local Ollama.

Removed entirely:
- AIConfig.CloudContextPrivacy dial + constants + GetCloudContextPrivacy /
  NormalizeCloudContextPrivacy, AND the now-dead legacy
  ShareOperationalContextWithCloud boolean + ShouldShareOperationalContextWithCloud
  (internal/config/ai.go); the config-load migration (persistence.go).
- Both fields from the /api/settings/ai request/response, validation, and sync
  (ai_handlers.go) + the JSON contract snapshots.
- The "Cloud model privacy" 3-option UI control, form field, presentation copy,
  and CloudContextPrivacy type (frontend), plus their tests.
- The dial branching in the seam: chat/service.go cloudPrivacyLevel,
  CloudContextPolicy.Level + local_only suppression + the localOnly directive
  (context_prefetch.go), the inventory resourceLabel dial logic (resource_context*),
  and the modelboundary RedactLocalOnlyResourcesOnly option.

Fixed lean posture (no setting): a cloud-routed model receives real infrastructure
context, with two always-on invariants enforced by the model-boundary sanitizer —
credentials are always stripped, and local-only/Restricted resources (the floor)
never leave the local trust boundary. Local (Ollama) always full. The sanitizer's
default is now the local-only floor; it remains the universal backstop installed on
EVERY model-bound path (chat, session compaction, discovery/report/analysis via the
shared helper). Kept the two standalone fixes from this effort: compaction now
routes through the sanitizer, and directives no longer inject the "redacted by
policy" placeholder.

Governance: ai-runtime contract rewritten to a fixed-posture rule; api-contracts /
frontend-primitives / agent-lifecycle / storage-recovery dial references removed.
Tests updated to the floor-only behavior (local-only redacted, Sensitive flows,
secrets stripped). Full internal/ai/..., config, api suites green; frontend
type-check + tests + lint green.
2026-06-09 09:43:21 +01:00
rcourtman 38740f1dc4 Honor the cloud_context_privacy dial on non-chat model paths (increment 2 completion)
While investigating a "bazarr → Unknown Service" discovery report, found that
increment 2 only wired the dial into the interactive chat seam. The shared helper
(*Service).requestSanitizerForModel — used by discovery analysis, the report and
fleet narrators, quick analysis, and the ExecuteAgentic paths — always installed a
FULL-redaction sanitizer regardless of the dial. So at the "full" dial those paths
silently over-redacted governed resources: e.g. discovery could not identify a
governed service even though the operator chose full.

(Note: bazarr itself is classified Internal/cloud-summary, so it is NOT redacted —
its "Unknown Service" is a discovery service-identification matter, not redaction.
This fix addresses the governed-resource case the same gap would break.)

Fix: requestSanitizerForModel now resolves the dial from the config snapshot
(fail-closed to redacted when absent) and passes RedactLocalOnlyResourcesOnly() at
"full", exactly like the chat seam. Local (Ollama) still gets no sanitizer. The
local-only hard floor still protects must-not-leave resources at full.

Proof: TestRequestSanitizerForModel_HonorsCloudPrivacyDial — at full a Sensitive
(local-first) identifier flows while a Restricted (local-only) one stays redacted
and the bearer token is always stripped; at redacted both identifiers are redacted;
local model gets a nil sanitizer. Contract: ai-runtime universal backstop rule now
states the shared helper must honor the dial too (functional parity), not just
install the sanitizer. Full internal/ai/... suite green (23 packages).
2026-06-09 08:46:40 +01:00
rcourtman 8d33986b5e Sanitize session-compaction transcript at the model boundary (privacy audit fix)
A systematic audit of all model-bound paths (not just the ones touched reactively)
found a leak: SummarizeSession (internal/ai/chat/session_compaction.go) sent the
PERSISTED chat transcript to the chat model via provider.Chat WITHOUT the
dial-aware model-boundary sanitizer. The transcript is built from persisted
messages — original user prompts and tool outputs — which carry raw resource
identifiers (hostnames/IPs/names) regardless of how the live turns were redacted.
On a cloud chat model at redacted/local_only, that shipped identifiers to the
provider, contradicting the cloud_context_privacy dial. (Secrets were already
redacted at transcript-build time via safety; resource identifiers were not.)

Fix: run the compaction ChatRequest through modelboundary.RequestSanitizerForModel
with the same dial resolution as a normal turn — fail closed to redacted when no
config snapshot, RedactLocalOnlyResourcesOnly() at full, nil (no-op) for local
Ollama. Mirrors the interactive seam exactly.

Audit also checked: report/fleet narrators, quick analysis, ExecuteAgentic(Stream),
discovery analysis, and Patrol all already install the sanitizer (verified). The
Patrol preflight self-test sends a fixed payload with no resource content, so it
needs no sanitizer (verified static, not trusted from the audit summary). agentcontext
output flows through the sanitized agentic loop. So compaction was the one gap.

Contract: ai-runtime gains a UNIVERSAL backstop rule — every model-bound path that
carries infrastructure-derived content must install the dial-aware sanitizer;
session compaction named explicitly; static no-identifier probes exempted. Proof:
TestServiceSummarizeSessionRedactsResourceIdentifiersForCloud (a transcript
hostname is stripped in the captured compaction request at the redacted dial);
existing compaction tests stay green (fix is additive — empty-model path unchanged).
Full internal/ai/... suite green (23 packages).
2026-06-08 22:10:49 +01:00
rcourtman bc5da3f4c5 Make the inventory resource context obey the cloud_context_privacy dial (increment 2 completion)
Increment 2 wired the dial into the prefetch and the model-boundary sanitizer but
missed a third model-bound path: the broad inventory context builder
(internal/ai/resource_context.go buildUnifiedResourceContextForModel) rendered
resource display names through unifiedresources.ResourcePolicyLabel, which redacts
genuinely-sensitive names UNCONDITIONALLY — ignoring the dial and even local-vs-cloud.

Symptom (reported live): on a cloud model the Assistant surfaced "redacted by
policy" and tried to run pulse_query with it as a search term, because sensitive
resources appeared redacted in the inventory regardless of the dial.

Fix: unifiedResourcePolicyContext now carries the dial + a known-local flag, and a
new resourceLabel() renders names per the dial — known-local (Ollama) always real;
cloud real only at "full" and only for resources NOT routed local-only (the same
hard floor as the sanitizer); unknown/empty destination fails closed to the
governed label (preserves the safe default for the no-destination context path).
All 15 ResourcePolicyLabel call sites in the inventory builder route through it.
This also fixes a latent inconsistency where local (Ollama) models over-redacted
inventory names despite "local is always full".

Proof: TestUnifiedResourcePolicyContext_ResourceLabelDialAware (local real;
cloud-full sensitive real, local-only floored; cloud-redacted both governed). The
existing AI-safe-summary inventory tests (no-destination path) stay green via the
fail-closed unknown-destination branch. Governance: ai-runtime contract delta —
the seam is now THREE dial-aware paths, inventory builder included. Full
internal/ai/... suite green (23 packages).
2026-06-08 18:16:50 +01:00
rcourtman fe46632e26 Stop injecting the "redacted by policy" placeholder into model directives (increment 4)
The resource-context handoff directives in internal/ai/chat/service.go and
internal/ai/chat/plain_text_resource_context.go named the literal redaction
placeholder ("Do not copy 'redacted by policy' into any tool argument", "labels
may be redacted by policy"). Naming the phrase in the prompt made the model echo
it back to the user as if it were the resource name — the confusing leakage
flagged after the sensitivity recalibration.

Reword the directives neutrally ("a withheld or placeholder label", "some labels
may be withheld") and add an explicit instruction not to repeat a withheld
placeholder back as the resource identity. The current_resource handle remains
the authoritative target, and the tool resolver still accepts the placeholder as
a defensive alias, so behavior is unchanged — only the prompt wording.

Proof: the plain-text resource-context test now pins that the directive does NOT
inject the redaction placeholder (previously it asserted the opposite, which was
pinning the leakage). Governance: substantive ai-runtime contract delta adding a
redaction-placeholder-hygiene rule for Pulse-authored model-bound directives.
Full internal/ai/... suite green.
2026-06-08 17:16:53 +01:00
rcourtman 3e3c601317 Make the redaction seam obey the cloud_context_privacy dial (increment 2)
The dial now drives model-bound redaction directly, replacing the legacy
ShareOperationalContextWithCloud read. internal/ai/chat/service.go resolves
cloudPrivacyLevel once per turn (failing closed to "redacted" when no config
snapshot is present) and threads it into the prefetch and the model boundary:

- full: the model-bound resource-policy sanitizer is invoked with
  modelboundary.RedactLocalOnlyResourcesOnly(), so real identifiers (hostname,
  IP, alias, name) for ordinary (Internal) and Sensitive (local-first) resources
  reach the cloud model — the core "answer with real detail" win. Resources the
  policy engine routes local-only (Restricted) stay redacted as a HARD FLOOR a
  blanket dial must never override, so default-full never ships a must-not-leave
  resource to a cloud vendor. Prompt-secret sanitation (credentials) always runs.
- redacted: every policied resource's identifiers are redacted as before, and the
  prefetch surfaces the PII-free operational context for governed resources.
- local_only: the prefetch injects NO proactive infrastructure context to the
  cloud turn (only a transparency directive pointing at the setting / a local
  model), and the sanitizer still redacts identifiers as a backstop.

CloudContextPolicy now carries the dial Level (failing closed to redacted for
empty/unknown) instead of a ShareOperationalContext bool; sharesCloudOperationalContext
covers full+redacted, suppressesCloudContext covers local_only. The obsolete
"Share operational context with cloud models" transparency string is replaced by
a local_only directive referencing "Cloud model privacy".

modelboundary gains RedactLocalOnlyResourcesOnly() + localOnlyRoutedResources();
the resource-redaction pass narrows to the local-only floor at full while
prompt-secret sanitation is unconditional. Local (Ollama) models never reach the
sanitizer.

Governance: substantive ai-runtime contract delta describing the dial-driven seam
and the local-only floor. Proofs: modelboundary sanitizer tests (full keeps the
floor + redacts secrets; default redacts all identifiers), CloudContextPolicy
level semantics + prefetch full/redacted/local_only behavior, and the handoff
relationship test pinned to redacted. Full internal/ai/..., unifiedresources,
and agentcontext suites green.
2026-06-08 17:08:14 +01:00
rcourtman 7cb45489e8 Add cloud_context_privacy dial (privacy redesign increment 1)
Introduce the single privacy dial that will govern what infrastructure context
cloud models may see, replacing the binary share_operational_context_with_cloud
toggle as the canonical operator control. This increment adds and surfaces the
setting; it does not change the redaction seam (that is increment 2).

Config (internal/config/ai.go): add AIConfig.CloudContextPrivacy with the
full|redacted|local_only levels, default "full", plus NormalizeCloudContextPrivacy
and the nil-safe GetCloudContextPrivacy getter. NewDefaultAIConfig defaults a fresh
self-hosted install to "full" so the Assistant answers with real resource detail
out of the box. The legacy ShareOperationalContextWithCloud boolean is retained as
the field the redaction seam still reads until it is wired into the dial directly.

Migration (internal/config/persistence.go): LoadAIConfig derives the dial from the
legacy toggle for pre-dial configs (legacy on -> full, off/absent -> redacted) and
persists it, leaving the legacy boolean untouched so existing installs keep their
current cloud behavior byte-for-byte. Fresh installs (no config file) default to full.

API (internal/api/ai_handlers.go): round-trip cloud_context_privacy through
/api/settings/ai field-by-field like discovery_enabled. The response always
serializes GetCloudContextPrivacy() (no omitempty) so the UI binds a 3-option
control to the concrete value; the update request carries an optional *string
validated against NormalizeCloudContextPrivacy (unknown values -> 400). When the
dial is provided it supersedes and re-syncs the legacy boolean (full -> true,
redacted/local_only -> false) so the existing seam honors the dial's full/redacted
axis without new redaction code paths.

Frontend: replace the binary "Share operational context with cloud models" toggle
with a "Cloud model privacy" 3-option FormSelect in AIRuntimeControlsSection.tsx,
bound to state.form.cloudContextPrivacy and the cloud_context_privacy payload via
useAISettingsState. CloudContextPrivacy type + payload fields in types/ai.ts;
label/help/option/summary copy in aiSettingsPresentation.ts.

Governance (ai-runtime + frontend-primitives substantive deltas; dependent
api-contracts, agent-lifecycle, storage-recovery notes): the contracts now name the
dial as canonical with the legacy boolean as the synced/migrated seam field.
Proofs: ai_config_test.go (getter/normalize/default), persistence_ai_test.go
(migration cases), ai_handlers_test.go (round-trip + legacy sync + 400),
contract_test.go JSON snapshots, settingsArchitecture + aiSettingsPresentation tests.
Live-verified in the preview drawer: dial renders with all three levels, the
migrated value (redacted) is selected, the summary updates reactively, and an
end-to-end UI save round-trips full (legacy sync true) then restores redacted.
2026-06-08 16:43:50 +01:00
rcourtman accebfb305 Infer resource_type from a canonical handle in pulse_query get
In the trace for 'hows esphome', the model called get with the canonical handle
'system-container-599a2e3...' as resource_id and no resource_type, and it failed
twice with 'resource_type is required' before recovering with 'get 102'. The user
sees those failed tool calls in the chat.

A canonical handle already encodes the type (unifiedresources/ids.go builds ids
as '<type>-<hash>'), so executeGetResource now infers resource_type from the
handle when it's omitted, via resourceTypeFromCanonicalID (the trailing hex hash
segment is unambiguous since no type word is all-hex). A bare numeric VMID still
requires an explicit type. Test TestResourceTypeFromCanonicalID covers it; full
internal/ai/tools green.
2026-06-08 15:49:09 +01:00
rcourtman 9573c6f625 Stop withholding tools from short resource lookups
User asked 'hows esphome' and the Assistant replied it had 'no infrastructure
context or diagnostic tools available' and asked them to run docker ps and say
where esphome lives — for a container (CT 102 esphome on delly) Pulse already
inventories.

Root cause: assistantPromptLooksConversational treated any prompt with <= 3 words
as chit-chat, routing it to the text-only scope that offers ZERO tools. So every
natural short lookup ('hows esphome', 'check frigate', 'grafana cpu') had its
tools withheld and the model genuinely couldn't query. This is the prompt-keyword
router anti-pattern the contract forbids.

Remove the word-count rule: only explicit greeting/meta prompts (hi, thanks, who
are you) are conversational; everything else is offered tools and the model
decides whether to use them. Verified live: 'hows esphome' now returns its real
status (Online, CPU 4.2%, mem 8.4%, no alerts) from Pulse data.

Regression test TestToolsForAssistantTurn_ShortResourceLookupGetsTools asserts
short lookups get tools while greetings stay text-only. Full internal/ai/chat green.
2026-06-08 15:36:58 +01:00
rcourtman ddc480ff3c Recalibrate resource sensitivity: ordinary workloads aren't secret
User on a cloud model saw 'redacted by policy' everywhere. Root cause: the
default classification (classifyResourceSensitivity) treated every VM, container,
pod, k8s workload, and docker service as 'Sensitive', which redacts their
hostname/IP/alias/path for cloud models. For Pulse's homelab/SMB audience that
crippled the cloud Assistant — a workload named 'grafana' isn't a secret, and its
private LAN IP isn't either.

Recalibrate: compute workloads classify as 'Internal' (cloud-summary, no
redaction) so cloud models can see their names/IPs. Escalation to
Sensitive/Restricted is by tag (database, backup, customer-data, secret, ...) or
by genuinely sensitive TYPE: storage/data-at-rest (storage, PBS, Ceph,
physical-disk, network-share, network, k8s PV/PVC/StorageClass), configuration
(docker-config, k8s-configmap), and security (k8s RBAC, secrets, PMG). Secrets
and PMG stay Restricted; the tag-based escalation is unchanged.

Tests: new TestRefreshPolicyMetadata_PlainComputeWorkloadsAreInternalNotRedacted
+ TestComputeWorkloadPolicyIsInternalUnlessEscalated lock it in. ~13 AI-subsystem
redaction tests that assumed plain compute = Sensitive updated to tag their
fixtures so they still exercise redaction on a genuinely-sensitive resource (no
assertions weakened). Contract: unified-resources Extension Points documents the
recalibrated classification. internal/ai/... + internal/unifiedresources/... green.
2026-06-08 15:21:51 +01:00
rcourtman 32523c8c94 Make the no-narrative fallback summary clean, not a JSON/call-id dump
When the model runs tools but returns no final narrative, Pulse synthesized a
fallback summary. It leaked the provider call ids as the 'tool names'
(normalizeToolUseID returns call_27f0f389… unchanged because the hex suffix
isn't all digits) and appended a raw JSON tool-output snippet — so the chat
showed 'I completed 4 successful check(s) using call_27f0f389…, … automatic
summary. Latest successful result snippet: {"systems":[]…}'. Far more
un-OpenCode than anything in the transcript.

buildAutomaticFallbackSummary now resolves each tool result's provider call id to
the real tool name from the assistant tool call (pulse_ prefix stripped), drops
opaque call_/toolu_/fc_ ids entirely, removes the raw-output snippet, and reads
as a clean operator message: 'I ran N checks (query, metrics) but the model
didn't return a written summary this time. Ask me again and I'll pull the
results together.'

New regression test reproduces the real OpenRouter shape (call_ ids on results,
real names on the tool calls) and asserts no call_ ids, real tool names, no raw
JSON. Full internal/ai/chat suite green.
2026-06-08 14:47:59 +01:00
rcourtman f68a75f012 Move Assistant live status to the footer, keep the transcript clean (OpenCode model)
User: the chat populates too much during a turn vs OpenCode. Read OpenCode's TUI
(packages/tui/src/routes/session/index.tsx): live 'working' state lives in ONE
pinned footer line (spinner + interrupt), and the scrolling transcript holds only
durable artifacts (user text, reasoning, tool calls, the answer). It never
narrates 'Preparing context / Reading inventory / Counting' into the timeline.

Pulse was doing both — workflow status rendered as transcript rows AND a header
chip AND (briefly) in the activity dock. Mirror OpenCode:
- MessageItem no longer renders workflow status in the transcript: the per-event
  role=status row (shouldRenderWorkflowStatusEvent) and the early-phase header
  chip (shouldShowHeaderWorkflowStatus) are both hard-false.
- The activity dock is now the single live indicator and persists for the whole
  turn: gate it on the streaming assistant message (assistantTurnActive) instead
  of chat.isLoading(), which flips false at visible-turn-complete and made the
  dock flash its status for a frame then vanish.

Result (verified live): transcript = user msg -> model route -> compact tool
rows -> answer; live status (spinner + 'Model is reasoning...' + route + Stop)
stays pinned in the footer while working, gone when done.

9 MessageItem tests that asserted the old transcript-row behavior rewritten to
assert footer-owned suppression; pacing/retry coverage stays in activeTurnStatus.
840 chat tests green (1 pre-existing ModelSelector failure, unrelated). Contract:
canonical footer-only rule added, supersedes the per-row transcript-status rules.
2026-06-08 14:39:36 +01:00
rcourtman 332fce866e Morph the streaming answer DOM instead of replacing innerHTML each tick
Final piece of the streaming-jank work. AssistantMarkdownBlock rendered
renderMarkdown(text) into innerHTML on every paced reveal, which rebuilds the
entire prose subtree — so a multi-paragraph or list/table answer flickers and
reflows every earlier line as it streams. Measured live: a 5-item numbered list
produced ~232 DOM mutations / 106 list-item removals over one turn.

Add markdownMorph.ts: reconcile old and new trees in place — identical nodes
untouched, same-tag nodes morphed (recursing into children so a growing
<ol>/<table> keeps its earlier <li>/<tr>), growing tail block updates its text
node rather than rebuilding. AssistantMarkdownBlock now feeds the sanitized HTML
to it via a ref+effect.

Same list answer after: ~22 mutations / 1 list-item removal — a ~10x drop. The
earlier lines stay put; only new items append and the tail updates in place.

Security unchanged: renderMarkdown (DOMPurify) remains the sole sanitization
gate; the morph only reconciles already-sanitized nodes. 9 morph unit tests +
286 MessageItem/AIChat tests green; tsc clean; render verified correct live.
2026-06-08 12:00:28 +01:00
rcourtman c3cd811b98 Render Assistant stream-event rows with <Index> so they stop re-mounting
Follow-up to the message-level reconcile fix. One level down, the per-message
stream-event/tool-row list in MessageItem rendered groupStreamEventsForDisplay()
through a reference-keyed <For>. That memo remaps blocks to fresh objects every
tick, so the streaming answer block (and completed tool rows) re-mounted — and
re-parsed markdown — on every content delta.

StreamDisplayEvent has no id, but the grouped list is strictly append-ordered
(the grouper only pushes new blocks or mutates the open content/thinking block in
place; never inserts mid-list or reorders), so positional keying is correct.
Switch the inner list to <Index>: each row keeps its DOM node across event-object
rebuilds at a stable position and updates in place.

Verified: <Index> reuses DOM nodes per position (probe); regression test proven
to fail on the original <For> (row re-created) and pass with <Index>; 286
MessageItem+AIChat tests green; tsc clean (it flagged every evt -> evt() spot).
2026-06-08 11:30:07 +01:00
rcourtman 7f3a0e3dd9 Stop the Assistant transcript re-mounting on every stream event
The chat felt janky during a turn — status rows popping in and out, the answer
flashing, the transcript jumping up and down — unlike OpenCode's stable timeline.

Root cause (measured with a live DOM mutation observer): useChat rebuilds its
message array immutably on every stream event, spreading a brand-new message
object each time. ChatMessages rendered that array through <For>, which keys by
object reference, so the whole MessageItem was torn down and recreated on every
content chunk / workflow-status change / tool update — dozens of re-mounts per
turn (observer showed the assistant message block DEL+ADD ~30x over 17s).

Fix: reconcile the incoming array into a keyed solid-js/store mirror in
ChatMessages so each message keeps a stable identity across updates. MessageItem
already reads every field through accessors, so once it stops re-mounting only
the genuinely changed text/rows update in place. After the fix the observer
showed the assistant block mount ONCE per turn (ADD:1, DEL:0).

Contained to ChatMessages.tsx — no changes to useChat's 18 update sites and no
new dependency. Regression test proven to fail on the old reference-keyed <For>.

Residual: the per-message stream-event/tool-row list still re-mounts on tool
turns (groupStreamEventsForDisplay remaps to new objects) — a separate, narrower
follow-up, noted in the ai-runtime contract.
2026-06-08 11:11:05 +01:00
rcourtman 3465a3bbf4 Give Assistant the current time so it stops deflecting time questions
Asked 'what's the time' in autonomous mode, Pulse Assistant deflected
('I don't have access to a real-time clock... tell me a target host and I can
run `date`') while OpenCode just ran date and answered. Root cause: the
Assistant's per-turn system prompt carried no clock, and the heavy target_host
framing pushed the model to demand a host for any command.

Inject the current wall-clock time (Pulse server clock) into the per-turn
prompt in AgenticLoop.getSystemPrompt. getSystemPrompt is re-evaluated each
turn, so the timestamp stays fresh; the base prompt is frozen at service start
and must not carry it. The time is PII-free and safe on cloud-routed turns.

The Assistant now answers time/date questions directly with no command and no
target host.
2026-06-08 10:39:18 +01:00
rcourtman 3f76da7932 Surface cloud operational-context sharing opt-in in AI settings
Wire AIConfig.ShareOperationalContextWithCloud through /api/settings/ai so the
existing chat-path opt-in (commit 32d597267) is operator-reachable, not
config-file-only.

Backend (internal/api/ai_handlers.go): add share_operational_context_with_cloud
to the AI settings response (always serialized so a toggle can bind to the
concrete value) and to the update request as an optional *bool, applied
field-by-field exactly like discovery_enabled (omitted = persisted opt-in
unchanged).

Frontend: add a 'Share operational context with cloud models' toggle to the
Assistant runtime controls, bound to the canonical useAISettingsState form and
the api/ai.ts AISettings/AISettingsUpdateRequest payload. Help/summary copy
(PII-free scope, hostnames/IPs/aliases stay redacted, default off, local Ollama
always gets full context) lives in aiSettingsPresentation.ts.

Governance: substantive ai-runtime + frontend-primitives deltas plus
dependent-contract notes (api-contracts, agent-lifecycle, storage-recovery);
path-policy proofs in ai_handlers_test.go (round-trip), settingsArchitecture
and aiSettingsPresentation tests. JSON snapshot contracts updated for the new
always-serialized field.
2026-06-08 10:05:37 +01:00
rcourtman 32d5972673 Share PII-free operational context with cloud Assistant when opted in
Governed resources (every sensitive guest) were redacted to a terse
summary on cloud-routed Assistant turns, so the Assistant went blind on
cloud models -- generic non-answers for the majority of users who run
cloud providers.

Add AIConfig.ShareOperationalContextWithCloud (default false). When the
operator opts in and the turn routes to an external provider, the chat
prefetch path injects servicediscovery.FormatCloudSafeContext (service
identity, access command, config/data/log paths, ports -- PII-free) in
place of the terse governed redaction, and the model-bound resource
sanitizer allow-lists those exact spans so they survive the provider
boundary. Hostname/IP/alias/platform-id stay redacted regardless of the
opt-in.

When sharing is off on a cloud turn, the prefetch path instructs the
Assistant to disclose the redaction and point at the setting instead of
silently degrading the answer. Local (Ollama) routing is unaffected and
always receives full context.

Proof: internal/ai/chat/context_prefetch_cloud_context_test.go (opt-in =>
access path present, no hostname/IP; opt-out => governed redaction +
transparency; model-bound sanitizer strips raw PII while the allow-listed
cloud-safe span survives) and internal/config/ai_config_test.go.
ai-runtime contract updated for the new opt-in behavior.
2026-06-08 09:38:22 +01:00
rcourtman 091ffc2f3d Keep Assistant chat model routes explicit
Fail unusable selected chat routes instead of substituting a same-provider default so retries and errors stay tied to the chosen model.
2026-06-08 07:21:38 +01:00
rcourtman d89257f9d8 Add Assistant tool chain fixture
Add a paced local Assistant fixture that exercises consecutive tool start, progress, completion, and replacement states without opening a provider request.
2026-06-08 07:13:37 +01:00