Addresses #450, #451, #406
- Initialize all variables at top of script to prevent "unbound variable" errors with set -u
- BUILD_FROM_SOURCE, SKIP_DOWNLOAD, IN_CONTAINER, IN_DOCKER now set at line 20-27
- ENABLE_AUTO_UPDATES, FORCE_VERSION, FORCE_CHANNEL, SOURCE_BRANCH also moved to top
- Removed duplicate assignments from argument parsing section
- Restore /bin/update command creation for ProxmoxVE LXC installations
- Creates update script that re-runs install.sh for easy updates
- Allows backend to properly detect ProxmoxVE deployment type
- Users can now run "update" in LXC console as documented
- Update deployment detection to recognize install.sh in update command
- Previously only looked for legacy "pulse.sh" reference
- Now checks for both pulse.sh and install.sh
Addresses multiple issues identified during comprehensive alert system audit:
1. Fix ZFS device loop lock issue
- Moved lock acquisition outside loop in checkZFSPoolHealth
- Changed clearAlert to clearAlertNoLock when lock already held
- Prevents multiple lock acquisitions in same iteration
2. Add alert deduplication on restore
- Prevents duplicate alerts after service restart
- Tracks seen alert IDs during LoadActiveAlerts
- Logs warnings for any duplicates found
3. Add API input validation
- validateAlertID function prevents DOS attacks
- Limit alert ID length to 500 characters
- Whitelist allowed characters (alphanumeric, -, _, :, /, .)
- Cap history limit parameter at 10,000 records
- Applied validation to acknowledge, unacknowledge, and clear endpoints
4. Add panic recovery to goroutines
- All SaveActiveAlerts goroutines now have defer/recover
- Cleanup goroutines protected from panics
- Contextual error logging for each goroutine type
5. Document lock ordering
- Added comprehensive documentation for Manager mutexes
- Explains m.mu and resolvedMutex relationship
- Clarifies acquisition rules to prevent deadlocks
- Inline comments for resolvedMutex field
These fixes improve stability, security, data integrity, and maintainability
of the alert system without breaking API compatibility.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Corrected widespread misinformation claiming API tokens cannot access guest agent data on Proxmox 9.
Changes:
- Rewrote VM_DISK_MONITORING.md with accurate technical explanation
- Deleted VM_DISK_STATS_TROUBLESHOOTING.md (contained false information)
- Updated FAQ.md with correct quick reference and troubleshooting link
- Added comprehensive VM disk troubleshooting section to TROUBLESHOOTING.md
- Fixed README.md troubleshooting reference
- Updated frontend tooltip to show accurate permission requirements
- Corrected backend log messages to remove "known limitation" language
- Updated test-vm-disk.sh diagnostic script with accurate guidance
Key corrections:
- API tokens work fine for guest agent queries on both PVE 8 and 9
- Proxmox API returning disk=0 is normal behavior, not a bug
- Both tokens and passwords work equally well
- Only requirements: guest agent installed + proper permissions
- Permission issues are config problems, not authentication method limitations
Documentation now provides clear user journey: FAQ → Troubleshooting → Full Guide
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds DEMO_MODE environment variable that blocks all write operations
while allowing full read/view functionality. Includes banner notification
in UI when demo mode is active.
Addresses need for safe public demo instances.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added streaming discovery that shows servers as they're found
- Backend sends WebSocket updates for each discovered server
- Frontend displays servers immediately without waiting for full scan
- Created sync-production-config.sh to preserve nodes when switching modes
- Updated toggle-mock.sh to sync config when disabling mock mode
- Dev environment now maintains separate config that syncs from production
- Enabled discovery service in dev environment by default
addresses real-time discovery UX and mock/production mode configuration persistence
The aggressive use of cluster/resources was breaking storage collection
for setups with multiple standalone nodes or improperly clustered nodes.
Now only uses cluster/resources when explicitly configured as a cluster,
falling back to traditional node-by-node polling otherwise.
This should fix the missing storage issue where one node's storage
wasn't showing after upgrading to rc5.
The issue was that when a node was successfully polled but returned empty storage
(e.g., due to API permissions), it was still marked as 'successfully polled'.
This prevented the preservation logic from keeping existing storage data.
Now if a node returns empty storage but we have existing storage for that node,
we don't mark it as polled, allowing the preservation logic to keep the data.
This should fix the issue where storage disappears from one node in #448.
- Send error result to channel when storage query times out so preservation logic works
- Ensures storage data is preserved for nodes that experience timeouts
- Fixes issue where storage/backups would disappear when a node times out
When a node's storage query times out, don't return empty storage which would wipe out existing data. Instead, skip the node entirely so the preservation logic can maintain the existing storage information.
Removed the flexible ID matching code that was added for backward compatibility. Since we've fixed the frontend to generate IDs consistently with the backend, we don't need the complexity of trying multiple ID formats.
This keeps the codebase simpler and more maintainable.
The frontend can save alert overrides with different ID formats depending on how the cluster is configured. This fix makes the backend more flexible in matching these overrides by:
1. Trying the exact guest ID first
2. Checking for partial matches that end with -node-vmid
3. Trying alternative ID formats like node-vmid and instance-node-vmid
This ensures custom alert thresholds work correctly regardless of the cluster name format used when saving overrides.
- Always try cluster/resources endpoint first (works on standalone nodes too)
- Only fall back to traditional polling for very old Proxmox versions
- Confirmed working on standalone nodes like pimox
- Significantly reduces API calls and improves performance
- Addresses efficiency concerns from #447
PVE polling is hardcoded to 10s since Proxmox cluster/resources endpoint only updates every 10s internally. Setting faster polling intervals was wasteful and provided no benefit.
Removed:
- POLLING_INTERVAL env variable and all references
- pollingInterval from config structs and API responses
- UI settings for polling interval (already removed)
- Dynamic polling interval updates via SIGHUP
- Legacy persistence code for saving polling settings
The monitoring loop now uses a hardcoded 10s interval matching Proxmox's update frequency.
Strip trailing slashes and paths from URLs before parsing host:port
to prevent "invalid port number" errors when users add nodes with
URLs like https://192.168.xxx.xxx:8006/
Mock VMs and containers were using 'node:qemu/vmid' format but the alert
system expects 'instance-node-vmid' format. This caused custom thresholds
to be ignored for mock guests.
- Add debug logging to guest agent filesystem API responses
- Better handle Windows drive mountpoints (C:\, D:\, etc.)
- Improve empty filesystem list detection and logging
- Add specific handling for Windows filesystems that may report differently
This should help diagnose why some VMs with guest agents installed still show 0% or missing disk usage, particularly on Windows systems.
- Add comprehensive disk detection diagnostics to /api/diagnostics
- Shows which nodes return disks and which don't
- Provides specific error messages and API responses
- Includes targeted recommendations based on failure type
- Helps users provide better debugging info when reporting issues
- Add specific warning for permission errors (401/403)
- Add info message for nodes without disk monitoring support
- Elevate error logging from debug to warn level for visibility
- Helps users understand why disks might not appear for certain nodes
- Increase WebSocket buffer sizes from 64KB to 4MB to handle large mock data
- Add robust reconnection logic with exponential backoff
- Implement heartbeat mechanism to detect stale connections faster
- Add manual reconnect button in UI when connection fails
- Fix unused variable warnings in monitor code
- Add debug logging to trace WebSocket state initialization
This resolves the issue where the frontend would hang after code changes
during hot-reload, especially when using mock mode with many nodes.
- Storage polling now preserves existing data when nodes are temporarily offline
- Backup polling preserves existing backups for unpolled nodes
- Prevents storage and backups from disappearing when nodes have connectivity issues
- Similar to the physical disk preservation fix
When a node is temporarily offline or has connectivity issues, preserve its last known physical disk data instead of removing it from the table. This prevents nodes from disappearing from the Physical Disks table when they have transient connection problems.
- VMs/Containers default to 10 seconds
- Nodes default to 15 seconds
- Storage defaults to 30 seconds
- PBS servers default to 30 seconds
This allows more appropriate delays for different resource types instead of a single global delay that doesn't fit all use cases. Storage and PBS can have longer delays since they're less critical and more prone to transient spikes during operations.
The 100% threshold disabling feature was incorrectly implemented and doesn't
make logical sense - metrics can legitimately reach 100% (CPU, memory, storage)
and those are critical conditions that should trigger alerts.
The correct way to disable specific alerts is already implemented:
- Set threshold to 0 or negative to disable a metric type globally
- Use per-resource overrides to disable specific metrics for specific resources
- Example: Overrides[guest-id].Memory = {Trigger: 0} disables memory alerts for that guest
This removes the confusing behavior where 100% thresholds would disable alerts
instead of alerting on actual 100% usage conditions.
Two critical issues fixed:
1. Storage threshold overrides were being loaded but never applied - the code
always used the default threshold instead of checking for overrides
2. Setting a threshold to 100% now properly disables alerts for that metric,
allowing users to suppress specific alerts they don't want
This fixes both the storage threshold persistence issue and the inability
to disable alerts by setting thresholds to 100%.
The issue was that PBS monitoring uses name-based IDs (pbs-<name>) while
the config system uses index-based IDs (pbs-0, pbs-1). When updating PBS
node configuration, the alert overrides were already being preserved but
the ID mismatch wasn't properly documented. Added explicit logging to
track PBS override preservation using the correct monitoring ID.
When detecting Proxmox cluster nodes, the Host field was being set to just the node name without a port. This caused validation to fail with "invalid Port number" error when qdevices were running.
Now cluster endpoints properly include the port (8006) in the Host field, allowing clusters with qdevices to be added successfully.
- Limit alert checking to 50 guests per cycle to prevent blocking
- Remove unnecessary state broadcast when alerts are resolved
- Fix deadlock in GetActiveAlerts by releasing lock quickly
- Enable handling of 800+ mock guests with sub-10ms response times
This allows Pulse to handle large-scale deployments efficiently for testing and production use.
The alert acknowledgment endpoints were hanging because GetState() was called
synchronously to broadcast updates via WebSocket, which could take significant
time with many nodes/guests. This caused the HTTP response to timeout, showing
an error to users even though the alert was successfully acknowledged.
Fixed by:
- Sending HTTP response immediately after acknowledging the alert
- Moving WebSocket broadcast to a goroutine to avoid blocking
- Applied fix to all alert endpoints (acknowledge, unacknowledge, clear, bulk ops)
This resolves the issue where users saw 'Failed to acknowledge alert' errors
but the alert was actually acknowledged (disappeared on refresh).
When updating PBS nodes through the node configuration UI, alert thresholds
were being reset to defaults. This was because alert overrides are stored
separately from node configuration and weren't being preserved during node updates.
The fix ensures that when a node is updated, the alert configuration (including
any custom threshold overrides) is reloaded and preserved. This applies to both
PBS and PVE nodes to ensure consistent behavior.
- handle PBS node status endpoint permission errors gracefully (returns nil instead of error for 403s)
- add required cf and timeframe parameters to RRD endpoint calls
- properly handle nil nodeStatus returns in monitor.go
these API calls now fail silently as PBS API tokens often lack the required permissions for these endpoints, which is expected behavior
- Add retry logic with delays to detectPVECluster function to handle API permission propagation
- Periodically re-check standalone nodes to detect if they're actually part of a cluster
- Increase timeout from 3 to 5 seconds for cluster detection attempts
- Skip retries for definitively standalone nodes (501 not implemented errors)
This addresses the issue where adding a PVE cluster doesn't detect it properly on first attempt,
requiring deletion and re-adding to work correctly. The retry mechanism gives time for
API permissions to fully propagate in Proxmox.
- Use centralized detectServiceName() function instead of duplicate logic
- Automatically detect whether system uses 'pulse' or 'pulse-backend' service
- Improves compatibility between official and community installer scripts
- Reduces confusion when users mix installation methods
rcourtman