Commit Graph

588 Commits

Author SHA1 Message Date
rcourtman f783fef716 Fix frontend performance issues by caching system settings
Backend was loading system settings from disk on every HTTP request
causing massive log spam and UI responsiveness issues. System settings
are now cached at startup and reloaded only when updated.

Also fixed tab cursor showing text cursor instead of pointer cursor.

Changes:
- Cache system settings in Router struct with mutex protection
- Load settings once at startup instead of on every request
- Add reloadSystemSettings() method to refresh cache when settings change
- Hook up cache reload in config handlers after successful save
- Add cursor-pointer to tab CSS classes for proper hover cursor
2025-10-11 17:22:22 +00:00
rcourtman 9c21d86212 Add multi-target Docker agent support and update installer 2025-10-11 16:34:33 +00:00
rcourtman 6e6d2dec6c Add quiet hours category suppression 2025-10-11 13:23:27 +00:00
rcourtman 11e80bbcbb chore: bump version to v4.22.0-rc.6 2025-10-11 12:44:33 +00:00
rcourtman 1ffcfbeaf6 Guard storage backups from failed polls 2025-10-11 10:00:05 +00:00
rcourtman 96b6454463 Sync alert configuration updates 2025-10-11 07:36:54 +00:00
rcourtman d7cc972265 Improve LXC disk usage reporting 2025-10-11 07:35:44 +00:00
rcourtman 80b4f50532 fix: improve update experience and handle missing releases gracefully
Backend:
- Handle 'no releases found' error gracefully instead of returning 500
- Return proper response indicating no updates available for the channel
- Fixes the 500 error when checking for updates on development versions

Frontend:
- Detect when backend enters 'restarting' phase during updates
- Show 'Pulse is restarting...' message instead of getting stuck on 'Initializing...'
- Implement health check polling with exponential backoff (2s → 15s max)
- Auto-reload page when backend becomes healthy again
- Improve messaging: inform users page will reload automatically

This fixes the stuck update modal issue where users would see
'Initializing...' forever when the backend restarted during updates.
2025-10-10 21:31:36 +00:00
rcourtman b250febcdd refactor: remove update history tracking from Manager
The update history infrastructure (history.go, UI panel) already exists
and is used by InstallShAdapter. However, for a home lab monitoring tool,
tracking every update in the Manager is overkill.

Removed:
- History field and initialization from Manager
- Event tracking in ApplyUpdate function
- Update history population on success/failure

The history.go file and UI panel remain for InstallShAdapter usage,
but the main Manager update path is now simpler and more focused.
2025-10-10 20:43:25 +00:00
rcourtman f05f47f935 fix: comprehensive update system improvements
Fixed 12 critical issues in the update system:

Critical bugs:
- Cache invalidation when channel auto-detected vs explicitly provided
- Data race on cache access (added proper RWMutex locking)
- Incorrect error handling when already on latest version

Security improvements:
- Command injection prevention with version string validation
- Symlink vulnerabilities in backup/restore (replaced cp with safe Go implementations)
- SHA256 checksum verification for downloads
- Improved backup completeness (now includes binary and VERSION file)

Quality improvements:
- Channel-keyed cache map instead of single cache value
- Update history tracking with event IDs and rollback support
- Cleanup of old temp directories (24+ hours)
- Proper resource cleanup with Close() method
2025-10-10 20:35:11 +00:00
rcourtman fe1518457a fix: auto-detect update channel from current version and add background checker
- Auto-detect update channel from current version when config is empty
  (RC users automatically get RC updates, stable users get stable)
- Add background update checker that runs on startup and hourly
- Add GetCachedUpdateInfo() method to return cached update info
- Update /api/version endpoint to include updateAvailable from cache
- Fixes issue where RC users on rc.4 weren't seeing rc.5 updates

This ensures RC testers continue receiving RC notifications while
stable users stay on the stable channel, all without explicit config.
2025-10-10 19:46:52 +00:00
rcourtman 64df42c62c chore: bump version to v4.22.0-rc.5 2025-10-10 19:07:09 +00:00
rcourtman 3ad41829ab revert: undo version bump to 4.22.0-rc.5 2025-10-10 18:51:15 +00:00
rcourtman 07d9e389bd chore: bump version to v4.22.0-rc.5 2025-10-10 18:43:15 +00:00
rcourtman d6e61e73ec fix: resolve Docker container permission and path errors in v4.22.0-rc.4
- Fix update history using hardcoded /var/lib/pulse instead of configured data dir
- Skip mock.env watching in Docker environments to avoid 'no such file' errors

Fixes issue #529

Update History Path:
- Modified NewUpdateHandlers to accept dataDir parameter
- Pass r.config.DataPath from router (honors PULSE_DATA_DIR=/data in Docker)
- Maintains backward compatibility with default /var/lib/pulse when empty

Mock Env Watcher:
- Check PULSE_DOCKER env var and skip mock.env watching if true
- Only watch /opt/pulse/mock.env if directory exists (not in containers)
- Guard all mock.env operations to prevent errors when path is empty
- Clean up log messages to only show mock_env_path when actually watching
2025-10-10 18:17:38 +00:00
rcourtman 81f96537e2 fix: resolve docker-agent-install 401 error from trailing slash in PULSE_URL
- Strip trailing slash from PULSE_URL in install script to prevent double-slash URLs
- Add path normalization in router for defense-in-depth on public endpoint matching
- Fixes issue #528 where users copying URLs with trailing slashes got 401 errors

The install script now normalizes PULSE_URL with ${PULSE_URL%/} before concatenating
with /download/pulse-docker-agent, preventing https://example.com//download URLs.

The router normalization provides additional resilience for path matching, though the
existing path traversal check already blocks double slashes at ServeHTTP level.
2025-10-10 18:09:55 +00:00
rcourtman 91222face7 fix: comprehensive Docker agent token security improvements
This commit addresses 6 critical security and UX issues in Docker agent
token handling identified through code review:

**High Priority Fixes:**
1. Fix stale token in command preview - Changed DockerAgents to use
   getInstallCommandTemplate() that always returns placeholder, allowing
   CommandBuilder to handle all token substitution reactively. Command
   preview now updates live as user types.

2. Fix misleading "multiple tokens" messaging - Updated token generation
   modal to accurately reflect single-token backend model with red warning:
   "This will immediately invalidate your existing token". Prevents operators
   from unknowingly breaking active Docker agents.

3. Close proxy auth bypass vulnerability - Both HandleRegenerateAPIToken
   and HandleValidateAPIToken now explicitly reject requests when proxy auth
   is configured but validation fails (returns 401). Prevents unauthenticated
   access when proxy auth is enabled.

**Medium Priority Fixes:**
4. Disable buttons when no stored token - "Use This Token" and "Copy" buttons
   now properly disabled with contextual tooltips when browser hasn't saved
   a token, eliminating confusing silent no-ops.

5. Improve frontend error handling - Token validation now distinguishes
   between authentication errors (401/403), rate limiting (429), network
   failures, and actual invalid tokens. No longer mislabels auth failures
   as "invalid token".

**Low Priority Fixes:**
6. Add authentication to validate-token endpoint - Endpoint now requires
   admin authentication (same as regenerate-token), preventing unauthenticated
   token guessing oracle despite rate limiting.

**New Component:**
- CommandBuilder.tsx: Interactive command builder with live preview,
  state-based visual cues, inline token generation, and validation

**Security Impact:**
- Closes unauthenticated validation surface
- Enforces proper proxy auth gating
- Prevents accidental exposure of security model
- Rate limiting maintained (10 attempts/min)

**UX Impact:**
- Clear, accurate error messages
- Live-updating command preview
- Contextual token management
- Disabled states prevent confusion

Reviewed and verified by both Claude Code and Codex with no regressions found.
2025-10-10 17:54:53 +00:00
rcourtman a1649d2e03 fix: resolve test failures and TypeScript errors
- Add missing postfix queue endpoint to PMG test mock server
- Add 'pmg' to DiscoveredServer type in Settings.tsx
- Fix potentially undefined object in UpdateBanner.tsx
- Remove unused imports and variables in Dashboard.tsx, MailGateway.tsx, NodeSummaryTable.tsx
2025-10-10 16:08:09 +00:00
rcourtman 3aa8e387e4 chore: bump version to v4.22.0-rc.4 2025-10-10 16:04:27 +00:00
rcourtman 5294ed8e4e feat: integrate update system with API endpoints
Wire up the adapter-based update system to HTTP API:

**Enhanced UpdateHandlers:**
- Initialize UpdateHistory and UpdaterRegistry
- Register all deployment adapters (systemd, proxmoxve, docker, aur)
- Handlers now have access to history and updater registry

**New API Endpoints:**
- GET /api/updates/plan?version=X - Get update plan for deployment
  * Returns canAutoUpdate, instructions, prerequisites, estimated time
  * Deployment-specific based on detected type
- GET /api/updates/history?limit=N&status=X - List update history
  * Supports filtering by status (success/failed/in_progress)
  * Returns audit log entries with full metadata
- GET /api/updates/history/entry?id=X - Get specific history entry
  * Retrieve detailed information about past update

**Existing Endpoints Still Work:**
- GET /api/updates/check - Check for available updates
- POST /api/updates/apply - Apply update (legacy manager)
- GET /api/updates/status - Get current update status

The system now supports both:
- Legacy update flow (existing install.sh wrapper in manager.go)
- New adapter-based flow (prepared for frontend integration)

Next: Frontend components to consume new endpoints.
2025-10-10 15:42:21 +00:00
rcourtman 2408d8004c feat: add update system foundation with adapter pattern
Add infrastructure for deployment-agnostic update management:

**Update History (Audit Log):**
- JSONL-based audit log (/var/lib/pulse/update-history.jsonl)
- Tracks all update attempts with full metadata
- In-memory cache for fast queries
- Schema designed for future DB migration

**Updater Interface:**
- Defines contract for deployment-specific update logic
- SupportsApply(), PrepareUpdate(), Execute(), Rollback()
- Registry pattern for managing multiple adapters
- Progress callbacks for real-time UI updates

**Adapters Implemented:**
- InstallShAdapter: Wraps install.sh (systemd/LXC)
  * Full automation support
  * Captures stdout/stderr to log files
  * Parses backup paths from output
  * Maintains install.sh as single source of truth
- DockerUpdater: Instruction-only (manual)
- AURUpdater: Instruction-only (package manager)

Key design decisions:
- install.sh remains unchanged (backward compatible)
- All update mechanisms funnel through adapters
- Audit log records all updates (manual + automated)
- Thin wrapper approach - no logic duplication

Next: Wire up API endpoints and frontend integration.
2025-10-10 15:39:47 +00:00
rcourtman fb21f4c909 feat: add version-aware update notifications for RC releases
Implement smart update checking that shows appropriate updates based on user's current version:
- RC users now see both newer RC releases AND newer stable releases
- Stable users continue to see only stable releases (RCs filtered out)
- Both channels use version-aware filtering (only show updates > current version)

Key changes:
- Modified getLatestReleaseForChannel to accept currentVer parameter
- Removed /releases/latest shortcut; both channels now fetch all releases
- RC channel tracks both newest RC and newest stable, returns highest
- Stable channel filters prereleases and returns first stable > currentVer
- Added comprehensive test suite with 15 test cases

Respects semver ordering where 4.22.0 > 4.22.0-rc.3 per RFC.

Consulted with Codex for architectural direction.
2025-10-10 15:20:34 +00:00
rcourtman 30fa3fd810 feat: add complete Proxmox Mail Gateway (PMG) monitoring support
Add comprehensive PMG monitoring with mail statistics, queue depth tracking,
spam distribution analysis, and quarantine monitoring. Includes full discovery
support and UI consistency improvements across all Proxmox products.

Backend:
- Add pkg/pmg package with complete API client for PMG operations
- Implement mail statistics collection (inbound/outbound, spam, virus, bounces)
- Add queue depth monitoring (active, deferred, hold, incoming queues)
- Support spam score distribution and quarantine totals
- Add PMG-specific discovery logic to differentiate from PVE on port 8006
- Extend mock data generator with realistic PMG instances and metrics
- Add PMG node configuration support in config system

Frontend:
- Create MailGateway.tsx component with detailed PMG dashboard
- Display mail flow statistics with time-series charts
- Show queue depth with color-coded warnings (>50 messages or >30min age)
- Add spam distribution histogram and quarantine status
- Support cluster node status with individual queue monitoring
- Add PMG to network discovery with purple branding and mail icon
- Implement conditional navigation (hide PMG tab when no instances configured)
- Standardize discovery UI controls across PVE/PBS/PMG settings pages

API:
- Add /api/config/pmg endpoints for node configuration
- Support PMG-specific monitoring toggles (mail stats, queues, quarantine)
- Extend system settings with PMG configuration options

Discovery:
- Detect PMG vs PVE on shared port 8006 using /api2/json/statistics/mail endpoint
- Return 'pmg' type for mail gateway servers in discovery results
- Update DiscoveryModal to display PMG servers with appropriate styling

This completes ecosystem monitoring support for all three Proxmox products:
Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
2025-10-10 14:30:51 +00:00
rcourtman ba7a1df8aa chore: bump version to v4.22.0-rc.3 2025-10-09 23:24:03 +00:00
rcourtman 7427a75117 feat: support standard Authorization header for API authentication
Add support for Authorization: Bearer <token> header as a fallback
for environments that strip custom headers like X-API-Token. Docker
agent now sends both headers for maximum compatibility.
2025-10-09 23:03:46 +00:00
rcourtman 95f99767e5 Surface shared storage on all nodes refs #522 2025-10-09 22:55:27 +00:00
rcourtman 53012f73ca Improve snapshot polling resiliency 2025-10-09 22:13:06 +00:00
rcourtman 6e3c77b609 fix: properly fix test and remove unused variables
- Bypass UpdateConfig in test to allow TimeThreshold=0
- Remove unused helper functions and variables completely
2025-10-09 16:20:56 +00:00
rcourtman 75ac4d6699 fix: resolve test failures and unused variables
- Fix TestAcknowledgePersistsThroughCheckMetric by setting TimeThreshold to 0
- Prefix unused variables with underscore in ThresholdsTable.tsx
2025-10-09 16:18:45 +00:00
rcourtman 44f52b5718 chore: bump version to v4.22.0-rc.2 2025-10-09 16:16:10 +00:00
rcourtman 667590519a feat: add per-metric alert delay configuration
Add sophisticated alert delay system with per-metric overrides:
- Global delay defaults per resource type (guest/node/storage/pbs)
- Per-metric delay overrides with UI controls
- Dynamic tooltips showing effective delay values
- Clear buttons to reset metric-specific overrides
- Temperature threshold defaults for nodes (80°C/75°C)
- Improved delay configuration persistence and normalization
2025-10-09 16:14:39 +00:00
rcourtman 30364a56cb Improve alert type safety and add docker container global toggle
- Add explicit typing for threshold records to handle undefined values
- Add temperature and docker threshold support to alert types
- Implement global toggle to disable all docker container alerts
- Add test coverage for docker container toggle behavior
- Add concurrency test for multi-instance node updates
2025-10-09 11:23:25 +00:00
rcourtman 5758f1e016 Improve alert notification routing and frontend alert visuals 2025-10-09 10:58:53 +00:00
rcourtman db24633c9e chore: bump version to v4.22.0 2025-10-09 08:21:32 +00:00
rcourtman 0b796ba32b Fix Proxmox memory usage accounting 2025-10-09 08:07:11 +00:00
rcourtman b674e88664 Wire global offline alert toggles 2025-10-08 18:56:07 +00:00
rcourtman 3f82b6a473 Include OIDC in exports and refresh guest metadata 2025-10-08 18:14:24 +00:00
rcourtman 8584ff7454 chore: update docker client to v28 2025-10-08 17:24:01 +00:00
rcourtman bf1e1ecaff chore: prepare v4.22.0-rc.1 2025-10-08 16:52:06 +00:00
rcourtman ba4500a022 tweak node display naming 2025-10-08 15:17:00 +00:00
rcourtman 76741cf921 Normalize websocket forwarded proto handling 2025-10-08 13:04:12 +00:00
rcourtman 9b4a740626 Add per-resource powered-off alert severity control
Implement tri-state offline alert configuration (Off/Warning/Critical) for VMs, containers, and Docker containers with individual severity overrides and visual badge breakdown in alerts tab.

Changes:
- Add poweredOffSeverity field to resource models and override types
- Implement tri-state buttons (Off/Warn/Crit) in ResourceTable
- Add separate critical/warning badge counts in alerts tab
- Support per-resource severity overrides with proper defaults
- Include alert delay configuration column in thresholds table
- Update backend to honor per-resource severity levels
- Add proper state persistence in raw override config
2025-10-08 12:00:05 +00:00
rcourtman df7bc21caf Improve alert duration tracking and add time threshold UI
This commit addresses issues reported in #470 related to alert duration
tracking, time threshold configuration, and email notification debugging.

Backend Changes:
- Preserve alert StartTime in preserveAlertState() to maintain accurate
  duration calculations across monitoring cycles
- Add debug logging to track alert creation times and duration preservation
- Add comprehensive logging to notification pipeline for email delivery
  tracking including SMTP config, cooldown status, and delivery attempts

Frontend Changes:
- Add TimeThresholdSettings component to display and configure per-resource-type
  alert delays (VMs/Containers, Nodes, Storage, PBS)
- Integrate time threshold UI into Thresholds tab with clear labels explaining
  "seconds above threshold before triggering"
- Add informational help text about how alert delays work

Related to #470
2025-10-08 08:51:02 +00:00
rcourtman a4569875b8 Fix temperature collection for cluster nodes and FQDNs
Issues fixed:
- Temperature collection was using node name instead of actual hostname/IP
- SSH warnings were contaminating JSON output from sensors command
- ClusterEndpoint IPs were not being utilized for SSH connections

Changes:
1. Use ClusterEndpoint IP/Host for cluster nodes instead of node name
2. Use Host URL from config for standalone nodes
3. Fallback to node name for simple DNS/hosts setups
4. Use cmd.Output() instead of cmd.CombinedOutput() to avoid SSH stderr warnings

This resolves issue #101 where users with FQDNs (e.g., pve2.some.domain)
couldn't collect temperatures, and handles duplicate node names across
multiple Proxmox instances.
2025-10-08 08:36:58 +00:00
rcourtman 104b163ac7 Add UI toggle for physical disk monitoring to prevent HDD spin-up
Physical disk monitoring is now disabled by default with a clear UI toggle for users who want to enable it.

**The Problem:**
Pulse was polling `/nodes/{node}/disks/list` every 10 seconds to check physical disk SMART data, causing idle HDDs to constantly spin up.

**The Solution:**
- Physical disk monitoring OFF by default (no HDD spin-up)
- New UI toggle in node edit modal under "Advanced monitoring"
- Clear warning: "This will cause HDDs to spin up from standby"
- When enabled, polls every 5 minutes (configurable via PhysicalDiskPollingMinutes)
- Storage pool monitoring (ZFS/LVM) still active and catches most disk failures

**UI Implementation:**
- Added monitorPhysicalDisks checkbox to PVE node settings
- Help text explains HDD spin-up behavior
- Field properly saved/loaded when editing nodes
- Only shown for PVE nodes (not PBS)

**Backend:**
- MonitorPhysicalDisks defaults to false
- Configurable polling interval (default 5 min)
- Interval-based polling with per-instance tracking
- Skips polls when interval hasn't elapsed

Users can now make an informed choice about disk monitoring vs. HDD power management.

Fixes #514
2025-10-08 08:16:43 +00:00
rcourtman e6bbd384a7 Fix critical alert system issues: cooldown, cleanup, and pending alerts
**Issues Fixed:**

1. **Cooldown period not enforced** - Added LastNotified tracking and shouldNotifyAfterCooldown()
   - New alerts are notified immediately and LastNotified is set
   - Existing alerts re-notify only after cooldown period passes
   - Critical escalations bypass cooldown for immediate notification
   - Cooldown check respects quiet hours configuration

2. **No re-notification for existing alerts** - Alert updates now check cooldown
   - Alerts stuck above threshold now re-notify after cooldown expires
   - Level escalation to critical triggers immediate re-notification
   - Prevents alert fatigue while ensuring critical issues aren't missed

3. **alertRateLimit memory leak** - Added cleanup in Cleanup() method
   - Entries older than 1 hour are removed every 10 minutes
   - Empty entries are deleted entirely to prevent map growth
   - Prevents unbounded memory consumption

4. **Pending alerts not cleared on threshold disable** - Fixed in reevaluateActiveAlertsLocked()
   - When threshold is disabled/removed, pending alerts are now cleared
   - Prevents phantom pending state for disabled metrics
   - Logged for debugging

**Implementation Details:**
- Added LastNotified field to Alert struct with proper cloning
- Cooldown enforced separately from rate limiting (MaxAlertsHour)
- Quiet hours verified working correctly (non-critical only)
- All changes maintain thread safety with existing mutex patterns
2025-10-07 21:40:43 +00:00
rcourtman 52edbd4e78 Fix Docker alert cleanup and email provider UI
- Add HandleDockerHostRemoved to properly clean up all alerts and tracking
  when Docker hosts are removed from config
- Implement pruneStaleDockerAlerts to automatically clear orphaned Docker
  alerts during sync cycles
- Clean up restart tracking and exit code maps in addition to state confirmations
- Add comprehensive test coverage for Docker host removal cleanup
- Improve email provider select layout with consistent styling and instruction boxes
- Fix RemoveDockerHost to handle missing hosts gracefully and still clear alerts
2025-10-07 20:31:00 +00:00
rcourtman 8aebbe3b54 Revamp alerts and Docker host management 2025-10-07 14:51:55 +00:00
rcourtman 15a9fc646e Handle empty Proxmox polls and fix IO metric keys 2025-10-06 15:44:29 +00:00
rcourtman 863061b0f4 Fix Docker alert overrides and expose host toggle 2025-10-06 15:17:05 +00:00