PBS storage should respect Proxmox's shared configuration. When PBS
storage is configured without node restrictions in Proxmox, it appears
as shared storage accessible by all nodes. Updated mock data to match
real cluster behavior where PBS entries appear under 'cluster' group.
PBS storage was being marked as shared when it shouldn't be. Each PBS
namespace is node-specific even when connecting to the same PBS server.
Now PBS storage correctly shows under its respective node instead of
all being assigned to the last node.
When some cluster nodes are offline but the main node can still reach others,
show a yellow status dot instead of red to indicate partial connectivity.
This better represents the actual cluster health state.
- Added PULSE_PUBLIC_URL config option to specify the full URL to access Pulse
- Updated notification manager to use publicURL when constructing webhook payloads
- Modified prepareWebhookData to construct full URLs instead of just paths
- Fixed test webhooks to use configured publicURL instead of hardcoded values
- Gotify and ntfy notifications now include clickable links that work properly
- Windows System Reserved partitions now excluded from disk usage
- Added better handling for filesystems with zero total bytes
- Improved debug logging for skipped filesystems
- Prevents incorrect disk usage display for Windows VMs
- Added CheckStorage calls in parallel storage polling (was missing, causing storage alerts to not trigger)
- Fixed node cleanup logic to use alert.Node field directly instead of parsing IDs
- Removed auto-acknowledge on alert click - now only acknowledge button toggles state
- Added unacknowledge button for acknowledged alerts
- Fixed double-toggle issue with acknowledge button using race condition prevention
- Fixed tab menu width in Alerts and Settings pages (changed flex-shrink-0 to flex-1)
addresses #228 (storage alert threshold issue)
- Add dynamic metric fluctuations for VMs and containers in mock data
- Fix alert acknowledgment to dim instead of hide alerts
- Implement unacknowledge functionality with backend persistence
- Simplify alert UI to single-click toggle (remove selection system)
- Add proper hysteresis for alert resolution when metrics drop
- Fix SVG icon boundaries in alert displays
- Add webhook disable toggles for testing without notifications
- Fix frontend directory duplication issue (addresses frontend-modern recreation)
- Improve alert sorting to show most recent first
- Make mock system generate realistic metric changes for proper alert lifecycle
- Add DiskStatusReason field to track why disk stats are unavailable
- Show helpful tooltips in UI explaining specific issues:
- Proxmox 9 API token limitation (401 on guest agent endpoints)
- Guest agent not installed/running
- Special filesystems only (Live ISOs)
- Permission issues
- Add comprehensive troubleshooting guide (docs/VM_DISK_STATS_TROUBLESHOOTING.md)
- Document that API tokens cannot access guest agent data on PVE 9
- Tested and confirmed: only password/cookie auth works for guest agent on PVE 9
- Update README with quick reference to VM disk stats issue
This addresses issues #348, #367, and #71 by clearly explaining the root cause
(Proxmox API limitation) and providing actionable guidance to users.
- Ensure disk metrics from /nodes endpoint are preserved when GetNodeStatus fails
- Add better fallback logic to prevent showing 0% or '-' for disk usage
- Improve logging to distinguish between rootfs and /nodes endpoint metrics
- Handle cases where neither rootfs nor valid node disk data is available
This fixes the regression introduced in v4.12.1 where disk stats would show as
'-' when GetNodeStatus failed due to network issues or rate limiting
- Added goroutine-based parallel fetching for VMs, containers, and storage across nodes
- Each node is now polled concurrently instead of sequentially
- Added detailed performance logging showing duration and success/failure counts
- Should significantly improve performance for deployments with 10+ nodes
- Maintains backwards compatibility with existing error handling and retry logic
- nodes now properly show as disconnected when client creation fails
- all configured nodes are checked, not just ones with active clients
- connection health is tracked from the actual monitoring state
- added detailed logging when client creation fails to help diagnose issues
- storage and backup tabs should now properly reflect connection status
previously, if a node failed to create a client during initialization (due to auth issues, network problems, etc), it wouldn't be tracked at all, causing inconsistent UI state. now all configured nodes are properly tracked and their connection status is accurately reflected.
- Always attempt to get filesystem info from guest agent when cluster/resources or node APIs return 0 disk usage
- The Proxmox API often returns 0 for disk field even when guest agent has real data available
- More aggressively query guest agent for running VMs with 0 disk usage
- Improved logging to show when disk data is successfully retrieved from guest agent
Mock mode was bypassing alert checking entirely, causing alerts not to trigger even when thresholds were exceeded. Added checkMockAlerts() function that processes mock VMs, containers, nodes and storage through the alert manager.
Fixed an issue where powered-off VMs were triggering CPU/RAM alerts.
Changes made:
1. Alert manager now skips ANY non-running VM (not just "stopped")
- Handles all states: stopped, paused, suspended, etc.
- Only checks thresholds for VMs with status="running"
2. Zero out metrics for non-running VMs/containers
- Proxmox may report stale metrics for stopped VMs
- Now explicitly sets CPU, memory, disk, network to 0 for non-running guests
3. Added debug logging to track VM states and metric handling
This ensures alerts only fire for actively running VMs and containers.
Storage operations on slow nodes were timing out after 30 seconds.
Made several improvements to handle this better:
- Increased default CONNECTION_TIMEOUT from 30s to 45s
- Added retry logic for storage queries that timeout
- Changed error logs to warnings for recoverable storage issues
- Storage polling now continues with other nodes if one fails
Users experiencing timeouts can also set CONNECTION_TIMEOUT env variable
to a higher value (e.g., CONNECTION_TIMEOUT=60s) if needed.
Storage polling now continues even if cluster-level storage config fails,
falling back to node-level storage data. Added comprehensive logging to
help diagnose storage/backup visibility issues.
Changes:
- Continue storage polling when cluster storage API fails
- Add detailed debug logging for storage operations
- Downgrade error to warning when API calls fail but can recover
- Better visibility into storage polling process
- Added offline node status checks before attempting to access node resources
- Fixed pollStorageWithNodes to skip offline nodes
- Fixed GetNodeStatus calls to skip offline nodes
- Fixed storage metric collection to skip offline nodes
- Added test offline node (pve3) to mock data generator
This prevents the 595 "authentication failed" errors that occur when
trying to access resources on offline cluster nodes through other nodes.
addresses #379 - better handling of offline nodes in clusters
- Skip polling VMs/containers from offline nodes to avoid 595 errors
- Improved error message for 595 to distinguish between auth failures and offline node access
addresses #389 - improved error messaging
- Better detection of whether 595 is an auth issue or offline node issue
- Clearer error messages to help users diagnose the actual problem
The 595 error can occur when:
1. Authentication actually fails (wrong credentials)
2. Trying to access resources on an offline node through another node in the cluster
addresses #388 - LXC containers not showing due to VMID type mismatch
- Changed Container.VMID from int to FlexInt to handle string VMIDs from older Proxmox versions
- Updated all code that references Container.VMID to cast to int where needed
addresses #389 - connection timeout errors with Proxmox nodes
- Increased default CONNECTION_TIMEOUT from 10s to 30s to handle slower networks
- This should resolve "context deadline exceeded" errors when polling nodes
addresses #379 - authentication errors may have been related to timeouts
- restored Node, Owner, and Location columns to backups table
- fixed issue where PBS backups via Proxmox storage showed storage name in Node column
- now correctly shows actual Proxmox node in Node column for all backup types
- Location column shows storage/datastore name without duplication
- Fixed alert acknowledgment persistence bug in monitor.go
- Reordered columns in NodeSummaryTable for better logical grouping
- Consolidated development environment with improved hot-dev script
- Updated vite.config.dev.ts for consistent port usage (7656)
- Enhanced Go main.go with better development mode detection
- Removed obsolete development scripts for cleaner repository
- Added comprehensive development documentation in CLAUDE.md
Development improvements ensure consistent port 7655 usage and
eliminate conflicts between different development approaches.
- Shows PVE/PBS version in node summary tables
- Helps quickly identify nodes needing updates
- Extracts just version number from PVE (e.g. 9.0.5)
- Also improved mock mode system for local development
Mock mode now properly returns simulated data including PMG host backups.
The monitor's GetState() method now checks for mock mode and returns
mock data when enabled, allowing full testing of UI features without
real Proxmox nodes.
Addresses #359 - PMG host config backups with VMID=0 are now correctly
identified as "Host" type instead of being misidentified as LXC containers.
Added purple color scheme for Host type backups in the UI.
When multiple clusters were added, shared storage from different clusters
would use the same ID (e.g., 'shared-local'), causing storage from one
cluster to overwrite storage from another. Now using instance-specific IDs
for shared storage to ensure each cluster's storage is properly tracked.
- Changed deduplication display from ratio (14.4:1) to multiplier (14.4x)
- Added encryption indicators for PBS backups (lock icon)
- Added owner column showing who created each PBS backup
- Fixed owner display to use separate column instead of cramped next to node name
- Added owner field to PBSBackup model and populated from PBS API
These improvements make it easier to understand backup status at a glance
- Add PBS alert monitoring (CPU, memory, offline detection)
- Add storage offline detection with proper cluster awareness
- Remove bulk toggle feature from thresholds UI (unnecessary complexity)
- Add enable/disable buttons for PBS servers in thresholds tab
- Fix storage offline detection to avoid false positives in clusters
(only alert on truly offline storage, not inactive cluster storage)
Alert improvements:
- PBS instances now properly monitored like nodes
- Storage devices generate offline alerts with confirmation system
- All resource types support custom thresholds and disable toggles
- Consistent alert ID format across all resource types
- Proper hysteresis and confirmation counts to prevent flapping
addresses #123 (if there was an issue about missing PBS alerts)
- capture deduplication_factor from PBS API datastore status endpoint
- display average deduplication ratio in backup frequency chart header
- shows as green 'Deduplication: X.X:1' when PBS datastores provide this data
- Remove max-width constraint on search fields to utilize available space
- Node summary table now updates based on search/filter criteria
- Only show nodes with matching guests when filtering is active
- Calculate node metrics based on filtered guests only
- Show matched guest count in node summary when filtering
- Provides better visual feedback on what the filters are affecting
- Created comprehensive mock data generator for nodes, VMs, containers
- Added toggle scripts for easy switching between real and mock mode
- Integrated with backend-watch.sh for auto-rebuild with mock support
- Modified monitor to skip polling when mock mode is enabled
- Added CLAUDE.md documentation for future sessions
Note: Mock system initializes but data isn't fully integrated with GetState() yet.
Currently shows mixed real + mock data. Works for UI testing purposes.
TESTED AND CONFIRMED: API tokens CAN access guest agent data on PVE 9!
- Created test tokens and verified they work
- Guest agent API returns proper disk usage data
- The cluster/resources endpoint shows disk=0 but that's not what Pulse uses
- Pulse correctly fetches data via /nodes/{node}/qemu/{vmid}/agent/get-fsinfo
The misinformation about PVE 9 not working was completely wrong. It does work when properly configured with PVEAuditor role which includes VM.GuestAgent.Audit permission.
Stop making definitive claims about what works or doesn't work. The reality:
- Some users (like you) have it working fine in cluster configs
- Others report 0% disk usage
- The exact conditions that make it work are unclear
- Results vary between different setups
Updated all docs and messages to reflect this uncertainty rather than making false claims about non-existent workarounds or absolute limitations.
Previous advice was completely wrong. The facts:
- VM.Monitor permission doesn't exist in PVE 9 (was removed)
- It was replaced with VM.GuestAgent.Audit
- But even with correct permissions, API tokens CANNOT access guest agent data on PVE 9
- This is Proxmox bug #1373 with NO working workaround for API tokens
- Users must accept 0% VM disk usage on PVE 9 until Proxmox fixes it upstream
Updated all documentation and error messages to reflect this reality instead of giving false hope about non-existent workarounds.
The root@pam suggestion doesn't actually work since it requires the Linux system root password, not a Proxmox-specific password. Most users don't know or have disabled their Linux root password for security.
Updated all documentation and error messages to correctly advise users to grant VM.Monitor permission to their API token user instead.
When the instance name equals the node name (common in single-node setups),
avoid generating redundant IDs like "pve-pve-100" by using just "pve-100".
This fixes alert acknowledgment issues where the UI couldn't match alert
IDs due to the duplicate node name pattern.
Addresses #353
- Add verification steps for qemu-guest-agent service status
- Clarify that the service is socket-activated (not systemctl enable)
- Add diagnostic commands users can run to verify agent is working
- Update FAQ with correct troubleshooting steps for agent issues
This helps users like @RLSinRFV who were trying to enable the service
when it's actually socket-activated and should start automatically.
The real issue for PVE 8 users seeing 0% disk usage:
- Users who added nodes BEFORE v4.7 don't have VM.Monitor permission
- The setup script always created tokens with privsep=0, so that wasn't the issue
- Solution: Re-run the setup script or manually add VM.Monitor permission
Updated error messages and documentation to reflect the actual cause
and provide the correct fix for users experiencing this issue.
- Add detailed logging when VM disk monitoring fails due to permissions
- Explain Proxmox 9 limitation: API tokens cannot access guest agent data (PVE bug #1373)
- Explain Proxmox 8 requirements: VM.Monitor permission and privsep=0 for tokens
- Update setup script to show appropriate warnings for each PVE version
- Update FAQ with troubleshooting steps for 0% disk usage on VMs
- Log messages now clearly indicate workarounds for each scenario
The core issue: Proxmox 9 removed VM.Monitor permission and the replacement
permissions don't allow API tokens to access guest agent filesystem info.
This is a Proxmox upstream bug that affects their own web UI as well.
For users experiencing this issue:
- PVE 9: Use root@pam credentials or wait for Proxmox to fix upstream
- PVE 8: Ensure token has VM.Monitor and privsep=0
- All versions: QEMU guest agent must be installed in VMs
addresses #348
After extensive testing and research:
CONFIRMED: This is a Proxmox 9 API limitation, not a configuration issue
- Guest agent get-fsinfo works when called as root (qm agent <vmid> get-fsinfo)
- API tokens CANNOT access this data even with VM.GuestAgent.Audit permission
- Proxmox's own web UI also shows 0% for VM disk usage (bug #1373)
Updated:
- Setup script now clearly explains this is a known Proxmox limitation
- Changed log level from Warn to Debug for permission errors (expected on PVE 9)
- Added references to Proxmox bug #1373
Workarounds for users:
1. Use root@pam credentials instead of API tokens for full VM disk monitoring
2. Container (LXC) disk usage works correctly with tokens
3. Wait for Proxmox to fix this upstream
The guest agent returns the data (total-bytes, used-bytes) but Proxmox's
API doesn't allow token access to it. This is not something we can fix
in Pulse - it needs to be addressed in Proxmox itself.
addresses #348
- Updated setup script to properly detect and handle Proxmox 9 where VM.Monitor was removed
- For PVE 9+, now creates custom role with Sys.Audit permissions (replaces VM.Monitor)
- Attempts to add VM.Agent or Sys.Modify permissions for better guest agent access
- Added better error logging to identify permission issues with guest agent API
- Warns users about PVE 9 permission requirements if disk usage shows 0%
The setup script now:
1. Properly detects PVE version using pveversion command
2. Creates appropriate roles based on PVE version (VM.Monitor for PVE 8, Sys.Audit for PVE 9)
3. Provides clear instructions if guest agent access still doesn't work
Improved logging to help users diagnose why VM disk usage might not be showing:
- Clearly identify when agent is enabled in config but not running in guest OS
- Detect timeout issues with unresponsive agents
- Log when agent returns no filesystem info
- Show which filesystems are included/excluded from calculations
- Distinguish between no agent, agent not running, and agent working
This will help users understand exactly why their VM disk usage isn't showing
and what steps they need to take to fix it (install qemu-guest-agent, restart
the service, etc).
addresses discussion #344
The agent field in Proxmox can have values other than just 0 or 1 when features are enabled, causing the strict equality check (== 1) to fail. Changed to check for any value > 0 to properly detect when the agent is enabled.
addresses discussion #344
- Setup script no longer requires authentication (uses setup codes instead)
- Fixed discovery service not starting when toggled via settings
- Addresses #347 and discussion #344
Pulse Monitor