When the server disables command execution for an agent, we now properly
call Close() on the command client to tear down the WebSocket connection.
Previously we just set the pointer to nil which left the goroutine running
with an orphaned connection.
This implements full remote configuration for the AI command execution setting:
Backend:
- Add CommandsEnabled field to HostMetadata for persistent storage
- Add GetHostAgentConfig/UpdateHostAgentConfig methods to Monitor
- Add /api/agents/host/{id}/config endpoint (GET for agents, PATCH for UI)
- Server includes config in report response for immediate agent application
- Agent parses response and dynamically enables/disables command client
Frontend:
- Add 'AI Commands' toggle column in Managed Agents table
- Toggle immediately updates server config; agent applies on next heartbeat
- Add 'Enable AI command execution' checkbox in agent installer wizard
- Checkbox adds --enable-commands flag to generated install commands
This allows users to:
1. Enable at install time via checkbox in the wizard
2. Toggle remotely via the Managed Agents UI for existing agents
3. Agents apply changes automatically on their next report cycle
- Add CommandsEnabled field to AgentInfo in pkg/agents/host/report.go
- Agent now reports whether AI command execution is enabled
- Server stores and exposes this via Host model
- Frontend can now show which agents have commands enabled
- This provides visibility before implementing remote configuration
Users who accumulated AI patrol findings before the patrol-without-AI
bug was fixed (24c4bb0b) could not dismiss them because the dismiss and
resolve endpoints required a Pro license.
Changes:
- Remove Pro license requirement from /api/ai/patrol/dismiss endpoint
- Remove Pro license requirement from /api/ai/patrol/resolve endpoint
- Add ClearAll() method to FindingsStore for bulk clearing
- Add DELETE /api/ai/patrol/findings endpoint for clearing all findings
- Add "Clear All" button to AI Insights UI
Users can now dismiss or resolve any findings they can see, and admins
can clear all findings at once if needed.
Backup and snapshot polling runs asynchronously and could take 20-45 seconds to complete, but WebSocket broadcasts happened on a separate fixed-interval timer. This caused frontend to show stale data until a broadcast happened to coincide with completed polling - which could take hours.
Now broadcasts state immediately after backup/snapshot polling completes, ensuring users see changes within seconds.
Related to #895
BREAKING CHANGE: AI command execution on agents is now disabled by default.
Users who want AI auto-fix must explicitly enable it with --enable-commands
flag or PULSE_ENABLE_COMMANDS=true environment variable.
Changes:
- Add --enable-commands flag (opt-in for command execution)
- Commands disabled by default for security (defense-in-depth)
- --disable-commands is now deprecated (logs warning, no longer needed)
- PULSE_DISABLE_COMMANDS deprecated in favor of PULSE_ENABLE_COMMANDS
- Update installer script to use --enable-commands
- Backwards compatibility: PULSE_DISABLE_COMMANDS=false still enables commands
This addresses community feedback about secure defaults for arbitrary
command execution on production infrastructure.
Related to #889
Extended Issue #891 fix to cover manual node addition via the UI:
1. HandleAddNode now checks for duplicates by Host URL (not name)
2. Disambiguator applied to PVE, PBS, and PMG node creation
3. Error message updated: 'host URL already exists' instead of 'name already exists'
This ensures the fix works whether nodes are added via:
- Agent auto-registration ✓
- Manual UI setup ✓
All node creation paths now consistently:
- Match by Host URL only
- Disambiguate duplicate hostnames with IP: 'px1' → 'px1 (10.0.2.224)'
Follow-up to #891 fix - also match by name+tokenID to handle the case
where the same physical host gets a new IP (DHCP). This ensures:
1. Same hostname + DIFFERENT token = different physical hosts → create separate nodes
2. Same hostname + SAME token = same host with new IP → update existing node
Also updates the host URL when an existing node is matched, so IP changes
are properly reflected in the saved configuration.
PROBLEM:
When two Proxmox hosts have the same hostname (e.g., 'px1' on different networks),
the auto-registration was matching by name and overwriting the first with the second.
This has been a recurring issue (#104) with at least 3 prior fix attempts.
ROOT CAUSE:
The auto-register handler matched existing nodes by BOTH Host URL and Name.
Matching by name is incorrect - different physical hosts can share hostnames.
FIXES:
1. Remove name-based matching in auto-registration - match by Host URL only
2. Add disambiguateNodeName() to append IP when duplicate hostnames exist
3. Add regression tests to prevent this from breaking again
Now when registering two hosts named 'px1':
- First becomes: px1
- Second becomes: px1 (10.0.2.224)
Both are stored as separate nodes with their own credentials.
Users who haven't enabled AI were seeing AI patrol findings from
heuristic analysis that they couldn't dismiss (license-gated).
- IsPatrolEnabled() now checks if Enabled is true
- IsAlertTriggeredAnalysisEnabled() also checks Enabled
- Updated tests to reflect new behavior
AI patrol and alert-triggered analysis require AI to be enabled
as a master switch. This prevents confusing UX where users see
AI features without having configured them.
Added request_timeout_seconds field to:
- AISettingsResponse struct (for GET responses)
- AISettingsUpdateRequest struct (for PUT requests)
- HandleUpdateAISettings handler logic (validation + persistence)
- HandleGetAISettings response builder
The frontend was already sending request_timeout_seconds but the
backend was ignoring it. Now the setting persists correctly.
Adds RequestTimeoutSeconds to AI config (default 300s / 5 min).
Users with low-power hardware running Ollama can increase this
value in Settings to prevent timeouts on slower inference.
Docker users were losing their license activation on every update because
/etc/machine-id changes when the container is recreated.
Changes:
- Store persistent encryption key in /data/.license-key (survives container updates)
- Fall back to machine-id for backwards compatibility with existing installations
- Existing users only need to re-enter their key once, then it persists forever
Fixes issue reported by customer upgrading from RC to v5.0.0.
When a PVE cluster has unique self-signed certificates on each node, Pulse
would mark secondary nodes as unhealthy because only the primary node's
fingerprint was used for all connections.
Now, during cluster discovery, Pulse captures each node's TLS fingerprint
and uses it when connecting to that specific node. This enables
"Trust On First Use" (TOFU) for clusters with unique per-node certs.
Changes:
- Add Fingerprint field to ClusterEndpoint config
- Add FetchFingerprint() to tlsutil for capturing node certs
- validateNodeAPI() now captures and returns fingerprints during discovery
- NewClusterClient() accepts endpointFingerprints map for per-node certs
- All client creation paths use per-endpoint fingerprints when available
Related to #879
Alert-triggered AI analysis was passing nil for lastBackup when analyzing
guests, causing 'Never backed up' findings even when backup data existed.
- Pass actual LastBackup timestamp from VM/Container state in analyzeGuestFromAlert
- Add regression test to verify backup data is correctly passed through
Fixes false positive 'Never backed up' alerts appearing when CPU/memory alerts fire.
- Move deadline/pong handler setup BEFORE registering agent in map
- Use writeMu mutex consistently for all WebSocket writes
- Prevents race between registration response and ExecuteCommand calls
- Fixes flaky TestExecuteCommand_RoundTripViaWebSocket in CI
Whitelists /api/ai/execute in the DemoModeMiddleware so users can
interact with the mock AI assistant while keeping the rest of the
system read-only and hardened.
Adds structured XML finding responses to the demo mock AI service.
This prevents the background patrol service from failing with 'Analysis failed'
when running in demo mode without a real LLM provider.
Updates the demo data generator to create patrol runs spaced 6 hours apart
over the last 3 days, rather than clustering them all in the last hour.
This provides a more authentic viewing experience on the demo dashboard.
Ensures the AI settings endpoint reports enabled=true and configured=true
when running in demo mode (PULSE_MOCK_MODE=true), even if no provider is
configured. This unlocks the frontend UI to allow interaction with the
mock AI assistant.
Allows the AI Assistant to provide realistic canned responses on the
live demo server without needing a real API key. Handled automatically
when PULSE_MOCK_MODE=true and no provider is configured.
When PULSE_MOCK_MODE=true, automatically grant all Pro features
so the demo server can showcase AI Patrol findings without needing
a license. This is specifically for public demo instances.
When a user deletes an API token that was migrated from .env, track
the hash in a suppression list to prevent it from being re-migrated
on the next restart.
Changes:
- Add SuppressedEnvMigrations field to Config
- Add env_token_suppressions.json persistence
- Check suppression list during env token migration
- Record suppressed hash when deleting "Migrated from .env" tokens
- Update RemoveAPIToken to return the removed record
Related to #871
- Add validateAIFindings() that cross-checks AI findings against actual metrics
- Filter out low-confidence findings (CPU <50%, memory <60%, disk <70%)
- Always allow critical findings, backup issues, and reliability findings through
- Update AI system prompt with stricter thresholds and explicit noise examples
- Add 'before creating a finding' checklist for AI (the 3am test)
- Update AI.md docs with clear value proposition and expectations
- Add comprehensive tests for the validation layer
This ensures paying users get immediate value without noise.
PBS datastores are now displayed in the Storage overview alongside PVE
storage. Each PBS datastore is converted to a Storage entry with:
- type: 'pbs'
- content: 'backup'
- shared: true
- active: based on PBS instance status
This provides a complete picture of all storage resources in one view
while keeping detailed PBS info in the dedicated PBS section.
Closes#869
- Fix PVE nodes: buildNodeUrl in ProxmoxNodesSection.tsx now prioritizes
guestURL over host (was ignoring guestURL entirely)
- Add PBS support: GuestURL field added to PBSInstance config, model,
and API handlers
- Add PMG support: GuestURL field added to PMGInstance config, model,
and API handlers
- Update NodeSummaryTable to use guestURL for PBS nodes
- Frontend types updated for PBS/PMG guestURL support
The Guest URL setting in node configuration now works correctly across
all node types. When set, it takes priority over the Host URL when
clicking on node names to navigate to the Proxmox/PBS/PMG web UI.
Closes#870
- Add IsMockResource() helper to detect mock data resources by naming patterns
- Filter out heuristic findings from mock resources when PULSE_MOCK_MODE is off
- Mock findings still appear when demo mode is enabled
- Ensures users don't see mock data findings in production
The AI demo/mock findings were using a non-existent MOCK_ENABLED
env var. Changed to PULSE_MOCK_MODE which is the actual env var
used by the mock data system.
- Add AnalyzeKubernetes to PatrolConfig and enable by default
- Add analyzeKubernetesCluster() for K8s heuristic analysis
- Detect offline clusters, nodes not ready, CrashLoopBackOff pods
- Detect failed/pending pods and unavailable deployments
- Add K8s clusters to LLM context in buildInfrastructureSummary()
- Add kubernetes_cluster as valid resource type in AI prompt
- Enhance analyzeDockerHost() for Docker/Podman
- Podman-aware messaging based on runtime
- Add unhealthy container detection (health checks)
- Add exited container with error detection
- Add high CPU detection (>90%)
- Add stale host detection (10+ minutes)
- Tiered severity for restarts (>10 = critical)
- Add 20 new tests for K8s and Docker analysis
When MOCK_ENABLED=true, Pulse now injects realistic AI patrol
findings to showcase the AI features without requiring actual
LLM API calls. This enables the demo instance to demonstrate:
- Critical/warning/info findings with realistic content
- Patrol run history
- Actionable recommendations
Also includes refinements to dismissal logic from earlier work:
- Only 'not_an_issue' creates permanent suppression
- 'expected_behavior' and 'will_fix_later' just acknowledge
1. Fixed TestNewConfigPersistenceFailsWhenEncryptedDataPresentWithoutKey
- Test was picking up real encryption key from /etc/pulse during migration
- Now temporarily moves system key during test for proper isolation
- Uses t.Cleanup to ensure key is restored even on failure
2. Cleaned up console.log statements in production code
- Dashboard.tsx: replaced console.log with logger.debug for metadata events
- CompleteStep.tsx: removed verbose agent detection debug logs
These changes reduce log noise in production while maintaining debug
capability in development mode.
Addresses #866 - agents were logging 'WebSocket connection failed' warnings
even during normal reconnection scenarios (server restart, network blip, etc).
Changes:
- Normal close errors (1000, 1001, connection reset) now log at Debug level
- Only log Warning after 3+ consecutive failures
- Changed 'Connecting to Pulse' from Info to Debug to reduce noise
- Successful connections still log at Info level
The WebSocket is only used for AI command execution, not metrics, so
transient disconnections don't affect monitoring functionality.
User feedback fields (DismissedReason, UserNote, TimesRaised, Suppressed, Source)
were not being saved to disk, causing 'expected behavior' dismissals to be lost
after Pulse restarted.
- Add missing fields to AIFindingRecord in persistence.go
- Update FindingsPersistenceAdapter to save/load these fields
- Add comprehensive tests for dismissal persistence round-trip
Fixes issue where Frigate storage warning kept reappearing despite being
marked as expected behavior.
Since watch/info findings are filtered from the UI and never shown
to users, don't include them in the patrol run status summary.
This makes the summary consistent with what users actually see.
The LLM was confusing VMIDs because they weren't included in the
context. Now the formatted context shows:
### Container: ollama (VMID 200) on minipc
This prevents the AI from referencing the wrong VMID when generating
findings and recommendations.
When the service restarts, it now checks if a patrol ran within the
last hour. If so, it skips the initial patrol to avoid wasting API
tokens during development/maintenance when the service is restarted
frequently.
The scheduled patrol runs (every 6 hours) are not affected.
100 samples was causing 326k+ input tokens which is expensive.
24 samples (hourly resolution) still provides good pattern visibility
while significantly reducing token cost.
Estimated reduction: ~75% fewer metric tokens.
When AI patrol fails due to API issues like insufficient balance, invalid
API key, or rate limiting, we now create a finding that appears in the
AI Insights tab. This makes the issue visible to users rather than hidden
in logs.
The finding includes:
- Clear description of the issue (e.g., 'Insufficient API credits')
- Recommendation for how to fix it
- Evidence showing the actual error message
When a patrol run encounters errors (e.g., LLM call failed), don't
display 'All healthy' in the summary as that's misleading - the
analysis didn't complete properly.
Now shows 'Analysis incomplete (N errors)' instead, which correctly
explains why the status badge shows red/error.
Modern LLMs have 100k+ token contexts. 100 samples over 24h gives
~15 minute resolution while adding minimal token overhead.
This lets the LLM see fine-grained patterns, short spikes, and
accurately distinguish anomalies from normal behavior.
The in-memory MetricsHistory only retains 24 hours of data, not 7 days.
Changed computeGuestMetricSamples to use trendWindow24h instead of
trendWindow7d, and reduced sample count from 24 to 12 points.
This ensures the LLM actually receives metric samples in the context,
which wasn't happening before because the 7-day query returned empty data.
rcourtman