Commit Graph

809 Commits

Author SHA1 Message Date
rcourtman 53926de33c cleanup: remove legacy AI context fallback (buildInfrastructureContext)
The AI service now uses only buildUnifiedResourceContext() for
infrastructure context, since the resourceProvider is always set
during router initialization.

Removed:
- buildInfrastructureContext() function (~288 lines of dead code)
- Legacy fallback path in buildSystemPrompt()

The unified resource context provides a cleaner, deduplicated view
of infrastructure that includes:
- All resources grouped by platform and type
- Top CPU/Memory/Disk consumers
- Active alerts on resources
- Infrastructure summary statistics

This completes the AI service migration to unified resources.
2025-12-08 09:21:03 +00:00
rcourtman 11ee325879 cleanup: consolidate unified resources adapters into useResourcesAsLegacy hook
This cleanup addresses transition debt from the unified resources migration:

Frontend cleanup:
- Move all Resource→Legacy type conversions to useResourcesAsLegacy() hook
- Add asNodes() and asDockerHosts() adapter functions to the hook
- Simplify DockerRoute, HostsRoute, DashboardView to use the centralized hook
- Remove ~300 lines of duplicated adapter code from App.tsx
- Remove debug console.log statements from Dashboard.tsx
- Fix CPU value conversion (divide by 100) for Dashboard compatibility

Backend fixes (from previous session):
- Fix parentID format in converters (VM, Container, Storage) to match Node.ID
- Format changed from 'instance/node/nodename' to 'instance-nodename'
- Update tests to match new parentID format

This consolidates all legacy type conversion logic in one place,
making future cleanup easier when components are migrated to use
unified resources directly.
2025-12-08 09:16:26 +00:00
rcourtman e1e83c8295 fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman 2d4ba2b765 test: add PopulateFromSnapshot test to verify resource conversion
Test confirms that StateSnapshot data is correctly converted to unified
resources. Validates conversion for nodes, VMs, containers, and hosts.
2025-12-07 23:38:15 +00:00
rcourtman bb053a9985 ensure resource store is populated before WebSocket broadcasts
- Added PopulateFromSnapshot method to resources.Store
- Extended ResourceStoreInterface to include PopulateFromSnapshot
- Monitor now calls updateResourceStore before broadcasts
- This ensures resources are fresh on every WebSocket broadcast

Without this, the store would only be populated when /api/resources or
/api/state endpoints are hit, leaving WebSocket broadcasts empty.
2025-12-07 23:15:02 +00:00
rcourtman c2a747d337 phase 1: add unified resources to WebSocket state broadcasts
- Extended StateFrontend with Resources field containing unified resource data
- Added ResourceFrontend and related types for frontend-compatible resource data
- Extended ResourceStoreInterface to include GetAll() method
- Monitor now injects resources into WebSocket broadcasts
- Added helper method getResourcesForBroadcast() to convert resources to frontend format
- All existing tests pass

This enables the frontend to access unified resources via WebSocket state.
2025-12-07 23:02:56 +00:00
rcourtman cb416d48b3 checkpoint: remove unified resources UI, keep backend model
- Removed /resources page and associated frontend components
- Removed ResourcesOverview.tsx, UnifiedResourceRow.tsx, columns.ts
- Removed frontend types/resource.ts
- Updated unified-resource-architecture.md to mark Phase 4 as ABANDONED
- Removed unified-view-migration-plan.md
- Backend unified resource model remains for AI context

This is a checkpoint before attempting full frontend migration to unified model.
2025-12-07 22:54:27 +00:00
rcourtman f7735cca62 fix: Populate resources on-demand when /api/resources is called
The Resources page was showing 0 resources because the store was only
populated when /api/state was called (from the dashboard). Now the
resources are populated on-demand when /api/resources is accessed.

Changes:
- Added StateProvider interface to ResourceHandlers
- SetStateProvider() method for injecting the monitor
- HandleGetResources now calls PopulateFromSnapshot before querying
- Router injects monitor as state provider during SetMonitor()

This ensures the /resources page works even when accessed directly
without visiting the main dashboard first.
2025-12-07 14:47:29 +00:00
rcourtman f34c28f970 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman 7b1ec9b5f5 Add host alert deduplication with tests
- Modified alerts.go for improved host alert handling
- Added host_dedup_test.go for deduplication test coverage
2025-12-07 12:38:38 +00:00
rcourtman e176861b35 AI features checkpoint: Host selection, memory sparklines, UI refinements
- Extended AI context selection to host rows in HostsOverview
- Added resourceId prop to StackedMemoryBar for sparkline support
- Relocated guest URL editing from GuestRow name click
- Added GuestNotes component with URL field in AI sidebar
- Refined host routing in AI service backend
- Minor animation and styling improvements
2025-12-07 12:25:26 +00:00
rcourtman 28ea85a0a0 Additional updates 2025-12-07 10:22:42 +00:00
rcourtman bcd7b550d4 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman 472a86dcdb feat: Add OS type display for LXC containers
- Extract ostype from LXC container config (debian, ubuntu, alpine, etc.)
- Map ostype values to human-readable names (e.g., "debian" -> "Debian")
- Add OSName field to Container model and ContainerFrontend
- Add icons for NixOS, openSUSE, and Gentoo in frontend
- LXC containers now show OS icons alongside VMs in the dashboard

Supported LXC OS types: alpine, archlinux, centos, debian, devuan,
fedora, gentoo, nixos, opensuse, ubuntu, unmanaged
2025-12-05 12:43:32 +00:00
rcourtman 6c03b42b2e wip: AI provider improvements and chat store enhancements
- Improve Anthropic provider error handling
- Add AI service enhancements
- Update AI chat store with additional state management
2025-12-05 12:14:00 +00:00
rcourtman 8948e84fe5 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman 53d7776d6b wip: AI chat integration with multi-provider support
- Add AI service with Anthropic, OpenAI, and Ollama providers
- Add AI chat UI component with streaming responses
- Add AI settings page for configuration
- Add agent exec framework for command execution
- Add API endpoints for AI chat and configuration
2025-12-04 20:16:53 +00:00
rcourtman da51449392 fix: Exclude TrueNAS Docker overlay mounts from disk stats
Host agent was including Docker overlay2 mounts from TrueNAS SCALE's
.ix-apps directory in disk totals. These mounts inherit the ZFS pool's
AVAIL space, causing massively inflated storage numbers (e.g., 173 TB
per container overlay instead of actual usage).

Changes:
- Add /mnt/.ix-apps/docker/ to container overlay path exclusions
- Use ShouldSkipFilesystem() in host agent disk collection (was only
  using ShouldIgnoreReadOnlyFilesystem() which missed container paths)
- Add test cases for TrueNAS overlay paths

Related to #718
2025-12-04 03:03:04 +00:00
rcourtman d0d989289a Refactor alert system: fix race conditions, memory leaks, and improve code quality
- Rename checkFlapping to checkFlappingLocked to clarify lock contract
- Replace goto statements with structured control flow
- Wire up unused recordAlertFired/recordAlertResolved metric hooks
- Add trackingMapCleanup goroutine to prevent memory leaks from stale entries
- Tighten alert ID validation to alphanumeric + safe punctuation
- Fix history save error handling to properly manage backup lifecycle
- Add auto-migration for deprecated GroupingWindow field
- Refactor 300+ line UpdateConfig into focused helper functions
- Unify duplicate evaluateVMCondition/evaluateContainerCondition
- Add constants for magic numbers (thresholds, timing, flapping)
- Update tests to match new backup behavior
2025-12-02 23:31:36 +00:00
rcourtman b4a33c4f2d Fix offline buffering: add tests, remove unused config, fix flaky test
- Add unit tests for internal/buffer package
- Fix misleading "ring buffer" comment (it's a bounded FIFO queue)
- Remove unused BufferCapacity config field from both agents
- Rewrite flaky integration test to use polling instead of fixed sleeps
2025-12-02 22:31:44 +00:00
courtmanr@gmail.com caf0c10206 feat: Implement offline buffering for host and docker agents
- Add internal/buffer package with generic ring buffer
- Add buffering logic to host agent for failed reports
- Add buffering logic to docker agent for failed reports
- Add BufferCapacity configuration option
- Add integration tests for buffering logic
2025-12-02 22:12:47 +00:00
rcourtman bda8056e48 Add refresh-cluster button to detect new Proxmox cluster members
When new nodes are added to a Proxmox cluster after Pulse was
initially configured, they weren't showing up in Settings. The
existing "Refresh" button only triggered network discovery, not
cluster membership re-detection.

Changes:
- Add POST /api/config/nodes/{id}/refresh-cluster endpoint
- Add "Refresh" button in cluster node panel in Settings
- Re-detect cluster membership and update stored endpoints

Related to #799
2025-12-02 22:01:00 +00:00
rcourtman 4f824ab148 style: Apply gofmt to 37 files
Standardize code formatting across test files and monitor.go.
No functional changes.
2025-12-02 17:21:48 +00:00
rcourtman 1a5acc2542 refactor: Remove duplicate IsPasswordHashed from auth package
The config package has a more robust IsPasswordHashed function that
handles truncated hashes. The auth package had a simpler duplicate
that was only used in tests. Removed the duplicate and its test
(already covered by config/config_utils_test.go).

Reduces deadcode findings from 7 to 6.
2025-12-02 17:19:07 +00:00
rcourtman e5f1289239 refactor: Remove duplicate isLoopback function from hostagent
The isLoopback function in internal/hostagent/agent.go was unused in
production code - it was a duplicate of the same function in
internal/hostmetrics/collector.go which is actively used.

Removed the dead code along with its associated tests to reduce
maintenance burden and improve code clarity.
2025-12-02 16:52:55 +00:00
rcourtman cf26ed7f12 security: Add request body size limits to remaining API handlers
Add http.MaxBytesReader to 8 additional handlers to complete API
hardening against memory exhaustion attacks:

- guest_metadata.go: HandleUpdateMetadata (16KB)
- notification_queue.go: RetryDLQItem, DeleteDLQItem (8KB each)
- temperature_proxy.go: HandleRegister (8KB)
- host_agents.go: HandleReport (256KB)
- updates.go: HandleApplyUpdate (8KB)
- docker_metadata.go: HandleUpdateMetadata (16KB)
- system_settings.go: UpdateSystemSettings (64KB)

All API handlers that decode JSON request bodies now have size limits.
2025-12-02 16:47:13 +00:00
rcourtman b4d497ce3b security: Add request body size limits to API handlers
Add http.MaxBytesReader to 16 additional handlers to prevent memory
exhaustion attacks via oversized request bodies:

- docker_agents.go: HandleReport (512KB), HandleCommandAck (8KB),
  HandleSetCustomDisplayName (8KB)
- alerts.go: UpdateAlertConfig (64KB), BulkAcknowledgeAlerts (32KB),
  BulkClearAlerts (32KB)
- config_handlers.go: HandleAddNode, HandleTestConnection,
  HandleUpdateNode, HandleTestNodeConfig (32KB each),
  HandleVerifyTemperatureSSH, HandleExportConfig, HandleDiscoverServers,
  HandleSetupScriptURL (8KB each), HandleImportConfig (1MB),
  HandleUpdateMockMode (16KB)
2025-12-02 16:43:13 +00:00
rcourtman 6eb7f06df1 security: Add request body size limits to notification handlers
Add http.MaxBytesReader limits to prevent memory exhaustion attacks:
- UpdateEmailConfig: 32KB limit
- UpdateAppriseConfig: 64KB limit
- CreateWebhook: 64KB limit
- UpdateWebhook: 64KB limit

This follows the pattern already used in system_settings.go for
SSH config validation.
2025-12-02 16:37:30 +00:00
rcourtman c4fef5e560 test: Fix unreachable code warning in panic recovery test
Refactor the test to avoid unreachable code after panic by
checking a flag set before the panic instead of after. This
resolves the go vet warning while maintaining test coverage.
2025-12-02 16:30:31 +00:00
rcourtman 322573157e refactor: Use zerolog instead of fmt.Printf in config export
Replace raw fmt.Printf calls with structured zerolog logging for
consistency with the rest of the codebase. This improves log
formatting and enables proper log level filtering.
2025-12-02 16:27:54 +00:00
rcourtman ed9907accb test: Add tests for SyncGuestBackupTimes and UpdateStorageBackupsForInstance
Add tests for the remaining uncovered State methods:
- SyncGuestBackupTimes: Tests backup time sync from storage and PBS backups
- UpdateStorageBackupsForInstance: Tests storage backup updates with instance isolation

Improves internal/models coverage from 89.6% to 95.6%.
2025-12-02 16:22:38 +00:00
rcourtman 1df5897369 test: Add tests for Ceph, backup, replication, and snapshot methods
Add tests for previously uncovered State methods:
- UpdateCephClustersForInstance
- UpdateBackupTasksForInstance
- UpdateReplicationJobsForInstance
- UpdateGuestSnapshotsForInstance

Improves internal/models coverage from 80.4% to 89.6%.
2025-12-02 16:20:11 +00:00
rcourtman f044b42e39 test: Add more State method tests for coverage
Add tests for previously uncovered State methods:
- UpdatePhysicalDisks
- UpdateStorageForInstance
- UpdatePBSInstances / UpdatePBSInstance
- UpdatePMGInstances / UpdatePMGInstance

Improves internal/models coverage from 71.9% to 80.4%.
2025-12-02 16:17:04 +00:00
rcourtman 9d57025a88 test: Add tests for state connection health and backup methods
Add tests for previously uncovered State methods:
- SetConnectionHealth / RemoveConnectionHealth
- UpdatePBSBackups
- UpdatePMGBackups

Improves internal/models coverage from 67.0% to 71.9%.
2025-12-02 16:14:40 +00:00
rcourtman b9db9c140b docs: Add godoc comments to more exported functions
Add missing godoc comments to:
- BuildGuestKey in alerts/alerts.go
- GenerateMockData in mock/generator.go
- NewDockerUpdater, NewAURUpdater in updates/adapter_installsh.go
- NewMockUpdater in updates/mock_updater.go
2025-12-02 16:03:57 +00:00
rcourtman c05817f9de docs: Add godoc comments to exported functions
Add missing godoc comments to:
- NewRateLimiter and Allow in ratelimit.go
- SnapshotSyncStatus in temperature_proxy.go
- NewClient and GetVersion in pkg/pmg/client.go
2025-12-02 15:58:59 +00:00
rcourtman 097976321b perf: Cache hostname lowercase in temperature proxy lookups
Pre-compute strings.ToLower(hostname) before loops that search for
matching PVE instances. Avoids repeated lowercasing in two functions.
2025-12-02 15:43:41 +00:00
rcourtman 98d170e087 perf: Cache err.Error() in cluster node validation
Cache err.Error() result in two locations in config_handlers.go:
- TLS mismatch detection (3x calls to 1)
- Standalone node detection (2x calls to 1)
2025-12-02 15:41:18 +00:00
rcourtman 0bc58f678e perf: Cache err.Error() in storage timeout error handling
Cache err.Error() result in two locations:
- monitor.go: storage query retry logic (2x calls to 1)
- monitor_polling.go: storage timeout handling (2x calls to 1)
2025-12-02 15:39:37 +00:00
rcourtman dc707b2225 perf: Cache err.Error() in disk monitoring error handling
Compute err.Error() once and reuse errStr instead of calling Error()
four times when checking disk monitoring error types.
2025-12-02 15:37:48 +00:00
rcourtman 85487b0058 perf: Cache lowercase RAID state in alerts processing
Compute strings.ToLower(array.State) once and reuse stateLower instead
of calling ToLower three times when checking RAID array status.
2025-12-02 15:35:32 +00:00
rcourtman 884c85c2ab chore: Remove debug logging that exposed config JSON
Removed two DEBUG log statements that were logging full nodes config
JSON at Info level. This was verbose and potentially exposed sensitive
configuration data (credentials, tokens) in logs.
2025-12-02 15:32:02 +00:00
rcourtman a4edd6229f perf: Cache lowercase error string in guest agent error handling
Compute strings.ToLower(errStr) once and reuse errStrLower instead of
calling ToLower three times on the same string in permission check.
2025-12-02 15:26:40 +00:00
rcourtman 9dbfc3216c perf: Remove redundant fmt.Sprintf in tempproxy contains function
The fmt.Sprintf("%v", s) call was pointless since s is already a string.
Removing this unnecessary conversion.
2025-12-02 15:24:40 +00:00
rcourtman c81bbba8a3 perf: Use strconv.Itoa instead of fmt.Sprintf for int conversion
strconv.Itoa is faster than fmt.Sprintf("%d", ...) because it doesn't
need to parse a format string. Changed 4 occurrences in monitoring
package where integers are converted to strings.
2025-12-02 15:21:41 +00:00
rcourtman 0989e9c671 perf: Pre-compile regexes in updates/version package
Move regex compilation from function bodies to package-level variables
to avoid recompilation when parsing version strings.

Affected regexes:
- semverRe: Matches semantic version format (X.Y.Z-prerelease+build)
- rcNumRe: Extracts RC number from prerelease strings

These are called multiple times during version comparison and update checks.
2025-12-02 15:14:15 +00:00
rcourtman 5d7bb8d461 perf: Pre-compile regexes in mdadm package
Move regex compilation from function bodies to package-level variables
to avoid recompilation on each RAID array check during polling cycles.

Affected regexes:
- mdDeviceRe: Matches md device names in /proc/mdstat
- slotRe: Matches device slot lines in mdadm --detail output
- speedRe: Extracts rebuild speed from /proc/mdstat
2025-12-02 15:11:04 +00:00
rcourtman f0da506524 fix: Properly show disabled storage status from Proxmox
Storage that is disabled in Proxmox (Datacenter > Storage > enabled=no)
was incorrectly showing as "available" in Pulse. The issue was that
Enabled/Active fields defaulted to true and were never set to false
from the per-node API response.

Now the model correctly initializes Enabled/Active from the Proxmox
per-node storage API response, and the status determination prioritizes
checking the disabled state first.

Related to #796
2025-12-02 15:03:40 +00:00
rcourtman 3cba9a32ee refactor: Remove unreachable dead code branches
- parseDetail: Remove len(parts) != 2 check after SplitN(line, ":", 2)
  when we already verified line contains ":" (SplitN always returns 2)

Function improved from 96.4% to 98.1% coverage.
2025-12-02 15:00:26 +00:00
rcourtman 80baf3ad95 refactor: Remove unreachable dead code branches
- checkOrigin: Remove redundant same-origin check at line 155 that was
  already handled at line 116 (origin == requestOrigin)

Function improved from 95.1% to 97.4% coverage.
2025-12-02 14:56:35 +00:00