mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-26 05:29:45 +00:00
Attach a machine's own alerts to agent performance reports
Every Proxmox node and standalone Pulse agent is a unified `agent` resource in v6, and the Reports picker requests it as one. enrichReportRequest only enriched node, VM and container reports, so an agent report attached no alerts: on a mock stack, West Production A read HEALTHY, "All systems operating normally" and Quick Stats Alerts 0 while pve1 had two open critical node alerts. Docker runtimes, containers and Swarm services, storage, physical disks, Kubernetes, PBS and PMG reports had the same gap. Reports now attach the alerts raised under their subject's identities, read from its unified resource rather than host or node names: the unified and metrics-target IDs; for a machine, the linked Proxmox node's source ID and, when the agent source is present, the agent's "agent:<host>" alerts with their component children; the Docker alert producer's host, container and service references; a Proxmox storage pool's source ID with its ZFS pool and device children, or the agent alert on an Unraid array; and the Proxmox health and wearout alerts on a disk's device path only when they record that disk's serial, since a replacement disk takes over the path. A node alert that moved to the agent and the agent's own alert for that metric land in the same report, where pkg/reporting counts them once. Guests, storage pools and disks a machine hosts keep the alerts raised under their own identities on their own reports. Agent reports also fill resource details from the Proxmox and agent payloads and list the storage pools and physical disks whose unified parent is the machine. Disk rows for node and agent reports now share one builder, so both show only a collected temperature, coloured by the disk temperature alert thresholds.
This commit is contained in:
rcourtman
@@ -2770,6 +2770,13 @@ agent inventory, registration state, or command-channel readiness.
|
||||
recovered. That is report presentation only: the report never decides
|
||||
agent linkage, ownership of a metric, or agent freshness, and it adds no
|
||||
lifecycle state.
|
||||
Reports on an `agent` resource (`internal/api/reporting_subject_alerts.go`)
|
||||
read the agent's host ID from the unified resource's agent payload, only
|
||||
when the agent source is present, to attach that agent's `agent:<host>`
|
||||
alerts and component children beside the linked node's alerts. The
|
||||
report reads linkage the registry already resolved; it never links,
|
||||
unlinks, identifies or authorizes an agent, and it adds no lifecycle
|
||||
state.
|
||||
The same isolation rule applies to Patrol investigation-record propagation
|
||||
through shared AI intelligence handlers and `internal/api/router.go`:
|
||||
lifecycle surfaces may observe the resulting resource context, but they must
|
||||
|
||||
@@ -9673,14 +9673,45 @@ aggregate, health card, table and resource blocks, and the fleet heuristic
|
||||
recommends checking the agent it moved to. "Triggered and resolved" counts
|
||||
recoveries only, and the report narrator's payload marks a covered handover
|
||||
with `successor_alert_listed` so the model describes the condition once.
|
||||
Reports requested as `agent`, which is how the Reports picker addresses every
|
||||
Proxmox node and standalone agent in v6, carry no alert rows yet:
|
||||
`enrichReportRequest` enriches node, VM and container reports only, so an
|
||||
agent report's deterministic verdict ignores its machine's alerts (HEALTHY,
|
||||
or NO DATA without metrics). The rule above reaches those reports once they
|
||||
attach the machine's alerts. The fleet heuristic offers its all-clear
|
||||
pattern and "No fleet-wide action required" only while no alert, moved
|
||||
ones included, is active.
|
||||
The fleet heuristic offers its all-clear pattern and "No fleet-wide action
|
||||
required" only while no alert, moved ones included, is active.
|
||||
Reports requested as `agent`, which is how the Reports picker addresses
|
||||
every Proxmox node and standalone Pulse agent in v6, attach the machine's
|
||||
alerts through `enrichAgentReport` in
|
||||
`internal/api/reporting_subject_alerts.go`. `reportAlertScopeFor` builds a
|
||||
subject's alert identities from its unified resource, never from host or
|
||||
node names, and an alert belongs to the report of each resource whose
|
||||
identity it was raised under. For a machine that is the requested unified
|
||||
ID, the metrics-target ID, the linked Proxmox node's source ID
|
||||
(`<instance>-<node>`, the node's own alerts) and, when the agent source is
|
||||
present, the Pulse agent's `agent:<host>` alerts with every component child
|
||||
the agent raises under that identity (`agent:<host>/disk:...`, disk
|
||||
temperature, SMART, RAID, the Unraid array, custom sensors). When the alert
|
||||
engine records a node alert's handover to the agent (`moved_to_agent`), the
|
||||
moved alert and the agent's own alert for that metric land in one report,
|
||||
where the rule above counts them once
|
||||
(`TestContract_AgentReportCountsAMovedNodeAlertOnce`). Guests, containers,
|
||||
Proxmox storage pools and physical disks raise alerts under their own
|
||||
identities, so those stay on their own reports, and a same-named node in
|
||||
another cluster never leaks in
|
||||
(`TestContract_AgentReportAttachesTheMachinesOwnAlerts`). The one alert
|
||||
raised under two identities is the agent's Unraid array alert
|
||||
(`agent:<host>/storage:unraid-array`), which appears on both the machine's
|
||||
and the array's report. The agent report also fills resource details from
|
||||
the resource's Proxmox and agent payloads and lists the storage pools and
|
||||
physical disks whose unified parent is the machine through the node
|
||||
report's row builders, so a disk shows only a collected temperature,
|
||||
coloured by the alert thresholds. Every other type
|
||||
without a dedicated enricher attaches the alerts in its scope: the unified
|
||||
and metrics-target IDs; `alerts.DockerHostResourceID`,
|
||||
`DockerContainerResourceID` and `DockerServiceResourceID` for Docker
|
||||
runtimes, containers (by name when they have no ID) and Swarm services; a Proxmox storage pool's source ID with its ZFS pool and device
|
||||
children; and the Proxmox health and wearout alerts on a disk's device-path
|
||||
reference (`ProxmoxPhysicalDiskAlertResourceID`) only when the alert's
|
||||
recorded `disk_serial` is the disk's, because a replacement disk in the same
|
||||
slot takes over the path (#2627)
|
||||
(`TestContract_UnenrichedReportTypesAttachTheSubjectsAlerts`). The legacy
|
||||
`node` enricher keeps its name-based match for direct API callers.
|
||||
Multi-resource fleet reports (`engine.GenerateMulti`) now also carry an
|
||||
optional fleet-level narrative through a distinct
|
||||
`pkg/reporting.FleetNarrator` interface, kept separate from the
|
||||
|
||||
@@ -1493,6 +1493,14 @@ recovery scope, or a storage/recovery-owned secret source.
|
||||
handover resolution (moved to a Pulse agent, not recovered). That changes
|
||||
how a report states alert health only; it opens no storage, backup or
|
||||
recovery path, and report backups still come from the recovery store.
|
||||
Machine (`agent`) reports now list the storage pools and physical disks
|
||||
whose unified parent is the machine, through the same row builders as
|
||||
node reports. Storage reports attach their pool's own alerts (the Proxmox
|
||||
source ID plus ZFS pool and device children, or the agent's alert on an
|
||||
Unraid array), and physical-disk reports attach the Proxmox health and
|
||||
wearout alerts that record the disk's serial. All of it reads the existing unified storage
|
||||
and disk projections only; it opens no storage, backup or recovery path
|
||||
and grants no storage action.
|
||||
Update-plan readiness payloads and apply-route readiness enforcement are
|
||||
adjacent shared API context only. Storage and recovery surfaces may observe
|
||||
the resulting update state if a future settings flow links to recovery
|
||||
|
||||
@@ -5049,6 +5049,361 @@ func TestContract_VMInventoryExportCSVHeaders(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// reportAlertResourceIDs lists a report request's alert rows by resource ID,
|
||||
// sorted, so a test can state exactly which alerts a report attributes to
|
||||
// its subject.
|
||||
func reportAlertResourceIDs(req reporting.MetricReportRequest) []string {
|
||||
ids := make([]string, 0, len(req.Alerts))
|
||||
for _, alert := range req.Alerts {
|
||||
ids = append(ids, alert.ResourceID)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
return ids
|
||||
}
|
||||
|
||||
// Every Proxmox node and standalone Pulse agent is an `agent` resource in v6,
|
||||
// and the Reports picker requests it as one. Its report must carry the
|
||||
// machine's own alerts, matched through the identities its unified resource
|
||||
// carries: the linked node's source ID, the agent's "agent:<host>" alerts and
|
||||
// their component children. Guests, storage pools and same-named nodes in
|
||||
// another cluster are not the machine.
|
||||
func TestContract_AgentReportAttachesTheMachinesOwnAlerts(t *testing.T) {
|
||||
now := time.Now()
|
||||
start := now.Add(-24 * time.Hour)
|
||||
machineID := "agent-west-a"
|
||||
memoryTotal, diskTotal := int64(64<<30), int64(1<<40)
|
||||
storageTotal, storageUsed := int64(2<<40), int64(1<<40)
|
||||
snapshot := reportingEnrichmentSnapshot{
|
||||
Resources: []unifiedresources.Resource{
|
||||
{
|
||||
ID: machineID,
|
||||
Type: unifiedresources.ResourceTypeAgent,
|
||||
Name: "West Production A",
|
||||
Status: unifiedresources.StatusOnline,
|
||||
Uptime: 86400,
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceProxmox, unifiedresources.SourceAgent},
|
||||
Identity: unifiedresources.ResourceIdentity{
|
||||
IPAddresses: []string{"192.0.2.10"},
|
||||
},
|
||||
Metrics: &unifiedresources.ResourceMetrics{
|
||||
Memory: &unifiedresources.MetricValue{Total: &memoryTotal},
|
||||
Disk: &unifiedresources.MetricValue{Total: &diskTotal},
|
||||
},
|
||||
Proxmox: &unifiedresources.ProxmoxData{
|
||||
SourceID: "cluster-a-pve1",
|
||||
NodeName: "pve1",
|
||||
Instance: "cluster-a",
|
||||
HostURL: "https://pve1.example:8006",
|
||||
PVEVersion: "8.2.4",
|
||||
KernelVersion: "6.8.12-1-pve",
|
||||
ClusterName: "cluster-a",
|
||||
CPUInfo: &unifiedresources.CPUInfo{Model: "Xeon E5-2680", Cores: 16, Sockets: 2},
|
||||
},
|
||||
Agent: &unifiedresources.AgentData{
|
||||
AgentID: "host-pve1",
|
||||
OSName: "Debian GNU/Linux",
|
||||
OSVersion: "12",
|
||||
},
|
||||
},
|
||||
{
|
||||
ID: "storage-west-a-zfs",
|
||||
Type: unifiedresources.ResourceTypeStorage,
|
||||
Name: "local-zfs",
|
||||
Status: unifiedresources.StatusOnline,
|
||||
ParentID: &machineID,
|
||||
Metrics: &unifiedresources.ResourceMetrics{
|
||||
Disk: &unifiedresources.MetricValue{Total: &storageTotal, Used: &storageUsed, Percent: 50},
|
||||
},
|
||||
Storage: &unifiedresources.StorageMeta{Type: "zfspool", Content: "images"},
|
||||
Proxmox: &unifiedresources.ProxmoxData{SourceID: "cluster-a-pve1-local-zfs"},
|
||||
},
|
||||
{
|
||||
ID: "physical-disk-west-a-nvme0",
|
||||
Type: unifiedresources.ResourceTypePhysicalDisk,
|
||||
ParentID: &machineID,
|
||||
PhysicalDisk: &unifiedresources.PhysicalDiskMeta{DevPath: "/dev/nvme0n1", Model: "Samsung 990", DiskType: "nvme", Health: "PASSED", Temperature: 41},
|
||||
},
|
||||
{
|
||||
// A disk in standby keeps its last-known temperature, which a
|
||||
// report must not tabulate as measured.
|
||||
ID: "physical-disk-west-a-sdb",
|
||||
Type: unifiedresources.ResourceTypePhysicalDisk,
|
||||
ParentID: &machineID,
|
||||
PhysicalDisk: &unifiedresources.PhysicalDiskMeta{
|
||||
DevPath: "/dev/sdb", DiskType: "hdd", Health: "PASSED", Temperature: 52,
|
||||
Collection: &diskinventory.CollectionStatus{
|
||||
Temperature: diskinventory.FieldStatus{State: diskinventory.FieldUnavailable, Reason: "standby"},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
ID: "vm-west-a-101",
|
||||
Type: unifiedresources.ResourceTypeVM,
|
||||
ParentID: &machineID,
|
||||
},
|
||||
},
|
||||
ActiveAlerts: []models.Alert{
|
||||
// The node's own alert, keyed by the Proxmox source ID.
|
||||
{ID: "node-cpu", Type: "cpu", Level: "critical", ResourceID: "cluster-a-pve1", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
// The agent's own alert and a filesystem it reports.
|
||||
{ID: "agent-memory", Type: "memory", Level: "warning", ResourceID: "agent:host-pve1", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
{ID: "agent-disk", Type: "disk", Level: "warning", ResourceID: "agent:host-pve1/disk:/", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
// The agent raises its Unraid array alert under its own identity,
|
||||
// so it belongs to the machine as well as to the array's report.
|
||||
{ID: "agent-array", Type: "unraid-array", Level: "warning", ResourceID: "agent:host-pve1/storage:unraid-array", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
// A provider incident keyed by the unified ID.
|
||||
{ID: "incident", Type: "resource-incident", Level: "warning", ResourceID: machineID, StartTime: now.Add(-time.Hour)},
|
||||
// Not the machine: a guest, a storage pool, a same-named node in
|
||||
// another cluster, and an agent whose host ID shares a prefix.
|
||||
{ID: "guest", Type: "powered-off", Level: "warning", ResourceID: "cluster-a-pve1-101", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
{ID: "pool", Type: "zfs-pool-errors", Level: "warning", ResourceID: "cluster-a-pve1-local-zfs/zfs-pool:local-zfs", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
{ID: "other-cluster", Type: "cpu", Level: "critical", ResourceID: "cluster-b-pve1", Node: "pve1", StartTime: now.Add(-time.Hour)},
|
||||
{ID: "other-agent", Type: "memory", Level: "critical", ResourceID: "agent:host-pve10", Node: "pve10", StartTime: now.Add(-time.Hour)},
|
||||
},
|
||||
RecentlyResolved: []models.ResolvedAlert{
|
||||
{
|
||||
Alert: models.Alert{ID: "node-temp", Type: "temperature", Level: "warning", ResourceID: "cluster-a-pve1", Node: "pve1", StartTime: now.Add(-3 * time.Hour)},
|
||||
ResolvedTime: now.Add(-2 * time.Hour),
|
||||
},
|
||||
{
|
||||
Alert: models.Alert{ID: "node-old", Type: "cpu", Level: "warning", ResourceID: "cluster-a-pve1", Node: "pve1", StartTime: now.Add(-50 * time.Hour)},
|
||||
ResolvedTime: now.Add(-48 * time.Hour),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
req := reporting.MetricReportRequest{ResourceType: "agent", ResourceID: machineID}
|
||||
NewReportingHandlers(nil, nil).enrichReportRequest(context.Background(), "default", &req, snapshot, start, now)
|
||||
|
||||
wantAlerts := []string{"agent-west-a", "agent:host-pve1", "agent:host-pve1/disk:/", "agent:host-pve1/storage:unraid-array", "cluster-a-pve1", "cluster-a-pve1"}
|
||||
if got := reportAlertResourceIDs(req); !reflect.DeepEqual(got, wantAlerts) {
|
||||
t.Fatalf("agent report alerts = %v, want %v", got, wantAlerts)
|
||||
}
|
||||
for _, alert := range req.Alerts {
|
||||
if alert.Type == "temperature" && (alert.ResolvedTime == nil || !alert.Recovered()) {
|
||||
t.Fatalf("node alert resolved inside the window must read as a recovery, got %+v", alert)
|
||||
}
|
||||
}
|
||||
|
||||
res := req.Resource
|
||||
if res == nil {
|
||||
t.Fatal("agent report carries no resource details")
|
||||
}
|
||||
if res.Name != "West Production A" || res.Host != "https://pve1.example:8006" || res.PVEVersion != "8.2.4" ||
|
||||
res.KernelVersion != "6.8.12-1-pve" || res.CPUModel != "Xeon E5-2680" || res.CPUCores != 16 || res.CPUSockets != 2 ||
|
||||
res.MemoryTotal != memoryTotal || res.DiskTotal != diskTotal || res.Uptime != 86400 ||
|
||||
res.OSName != "Debian GNU/Linux" || res.OSVersion != "12" || res.ClusterName != "cluster-a" ||
|
||||
!reflect.DeepEqual(res.IPAddresses, []string{"192.0.2.10"}) {
|
||||
t.Fatalf("agent report resource details = %+v", res)
|
||||
}
|
||||
if len(req.Storage) != 1 || req.Storage[0].Name != "local-zfs" || req.Storage[0].Type != "zfspool" || req.Storage[0].UsagePerc != 50 {
|
||||
t.Fatalf("agent report storage = %+v", req.Storage)
|
||||
}
|
||||
// Disk rows share the node report's builder: only a collected temperature
|
||||
// is tabulated, coloured by the disk temperature alert thresholds.
|
||||
if len(req.Disks) != 2 {
|
||||
t.Fatalf("agent report disks = %+v", req.Disks)
|
||||
}
|
||||
nvme, standby := req.Disks[0], req.Disks[1]
|
||||
if nvme.Device != "/dev/nvme0n1" || nvme.Health != "PASSED" || nvme.Temperature != 41 ||
|
||||
nvme.TemperatureCritical <= 0 || nvme.TemperatureWarning <= 0 || nvme.TemperatureWarning > nvme.TemperatureCritical {
|
||||
t.Fatalf("collected disk row = %+v", nvme)
|
||||
}
|
||||
if standby.Device != "/dev/sdb" || standby.Temperature != 0 {
|
||||
t.Fatalf("a retained disk temperature must not be reported as measured, got %+v", standby)
|
||||
}
|
||||
}
|
||||
|
||||
// A node alert that moved to the machine's Pulse agent and the agent's own
|
||||
// alert for that metric both belong to the machine's report, and the report
|
||||
// verdict counts the condition once.
|
||||
func TestContract_AgentReportCountsAMovedNodeAlertOnce(t *testing.T) {
|
||||
now := time.Now()
|
||||
machineID := "agent-west-a"
|
||||
snapshot := reportingEnrichmentSnapshot{
|
||||
Resources: []unifiedresources.Resource{{
|
||||
ID: machineID,
|
||||
Type: unifiedresources.ResourceTypeAgent,
|
||||
Name: "West Production A",
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceProxmox, unifiedresources.SourceAgent},
|
||||
Proxmox: &unifiedresources.ProxmoxData{SourceID: "cluster-a-pve1", NodeName: "pve1"},
|
||||
Agent: &unifiedresources.AgentData{AgentID: "host-pve1"},
|
||||
}},
|
||||
ActiveAlerts: []models.Alert{{
|
||||
ID: "agent-memory", Type: "memory", Level: "warning", ResourceID: "agent:host-pve1",
|
||||
Message: "Agent memory at 94%", StartTime: now.Add(-time.Minute),
|
||||
}},
|
||||
RecentlyResolved: []models.ResolvedAlert{{
|
||||
Alert: models.Alert{
|
||||
ID: "node-memory", Type: "memory", Level: "warning", ResourceID: "cluster-a-pve1",
|
||||
Message: "Node memory at 95%", StartTime: now.Add(-time.Hour),
|
||||
Resolution: &models.AlertResolution{
|
||||
Reason: "moved_to_agent",
|
||||
SuccessorResourceID: "agent:host-pve1",
|
||||
SuccessorName: "pve1 (Host Agent)",
|
||||
},
|
||||
},
|
||||
ResolvedTime: now.Add(-2 * time.Minute),
|
||||
}},
|
||||
}
|
||||
|
||||
req := reporting.MetricReportRequest{ResourceType: "agent", ResourceID: machineID}
|
||||
NewReportingHandlers(nil, nil).enrichReportRequest(context.Background(), "default", &req, snapshot, now.Add(-24*time.Hour), now)
|
||||
|
||||
if got := reportAlertResourceIDs(req); !reflect.DeepEqual(got, []string{"agent:host-pve1", "cluster-a-pve1"}) {
|
||||
t.Fatalf("agent report alerts = %v", got)
|
||||
}
|
||||
narrative, err := reporting.HeuristicNarrator{}.Narrate(context.Background(), reporting.NarrativeInput{Alerts: req.Alerts})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if narrative.HealthStatus != "WARNING" || narrative.HealthMessage != "1 warning detected - review recommended" {
|
||||
t.Fatalf("moved node alert and its successor must count once, got %q / %q", narrative.HealthStatus, narrative.HealthMessage)
|
||||
}
|
||||
}
|
||||
|
||||
// Report types without a dedicated enricher still list the alerts raised on
|
||||
// their subject: the unified ID, the metrics-target ID, and the IDs each
|
||||
// alert producer builds for a Docker runtime, a container or a storage pool's
|
||||
// ZFS components. Workloads under a runtime stay with their own reports.
|
||||
func TestContract_UnenrichedReportTypesAttachTheSubjectsAlerts(t *testing.T) {
|
||||
now := time.Now()
|
||||
resources := []unifiedresources.Resource{
|
||||
{
|
||||
// Docker runtimes are agent-typed internally and travel as
|
||||
// docker-host.
|
||||
ID: "agent-docker-1",
|
||||
Type: unifiedresources.ResourceTypeAgent,
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceDocker},
|
||||
Docker: &unifiedresources.DockerData{HostSourceID: "docker-host-1"},
|
||||
},
|
||||
{
|
||||
ID: "app-container-web",
|
||||
Type: unifiedresources.ResourceTypeAppContainer,
|
||||
Name: "web",
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceDocker},
|
||||
Docker: &unifiedresources.DockerData{HostSourceID: "docker-host-1", ContainerID: "web-1"},
|
||||
},
|
||||
{
|
||||
// A container reported without an ID is referenced by its name.
|
||||
ID: "app-container-sidecar",
|
||||
Type: unifiedresources.ResourceTypeAppContainer,
|
||||
Name: "sidecar",
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceDocker},
|
||||
Docker: &unifiedresources.DockerData{HostSourceID: "docker-host-1"},
|
||||
},
|
||||
{
|
||||
ID: "storage-zfs",
|
||||
Type: unifiedresources.ResourceTypeStorage,
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceProxmox},
|
||||
Proxmox: &unifiedresources.ProxmoxData{SourceID: "cluster-a-pve1-local-zfs"},
|
||||
},
|
||||
{
|
||||
ID: "k8s-cluster-prod",
|
||||
Type: unifiedresources.ResourceTypeK8sCluster,
|
||||
},
|
||||
{
|
||||
ID: "pmg-eu",
|
||||
Type: unifiedresources.ResourceTypePMG,
|
||||
MetricsTarget: &unifiedresources.MetricsTarget{ResourceType: "agent", ResourceID: "pmg-main"},
|
||||
},
|
||||
{
|
||||
ID: "docker-service-web",
|
||||
Type: unifiedresources.ResourceTypeDockerService,
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceDocker},
|
||||
Docker: &unifiedresources.DockerData{HostSourceID: "docker-host-1", ServiceID: "svc-1"},
|
||||
},
|
||||
{
|
||||
// A service without an ID is named by the adapter's resource
|
||||
// name, which the producer falls back to.
|
||||
ID: "docker-service-api",
|
||||
Type: unifiedresources.ResourceTypeDockerService,
|
||||
Name: "api",
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceDocker},
|
||||
Docker: &unifiedresources.DockerData{HostSourceID: "docker-host-1"},
|
||||
},
|
||||
{
|
||||
// A Pulse agent reports the Unraid array as storage keyed by
|
||||
// its own source ID, which is the metrics target.
|
||||
ID: "storage-unraid",
|
||||
Type: unifiedresources.ResourceTypeStorage,
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceAgent},
|
||||
MetricsTarget: &unifiedresources.MetricsTarget{ResourceType: "storage", ResourceID: "host-1/storage:unraid-array"},
|
||||
},
|
||||
{
|
||||
ID: "physical-disk-sda",
|
||||
Type: unifiedresources.ResourceTypePhysicalDisk,
|
||||
Sources: []unifiedresources.DataSource{unifiedresources.SourceProxmox},
|
||||
Proxmox: &unifiedresources.ProxmoxData{Instance: "cluster-a", NodeName: "pve1"},
|
||||
PhysicalDisk: &unifiedresources.PhysicalDiskMeta{DevPath: "/dev/sda", Serial: "SER-B"},
|
||||
},
|
||||
}
|
||||
active := func(resourceID string) models.Alert {
|
||||
return models.Alert{ID: resourceID + "-alert", Type: "status", Level: "warning", ResourceID: resourceID, StartTime: now.Add(-time.Hour)}
|
||||
}
|
||||
// Proxmox disk alerts reference the device path, which a replacement disk
|
||||
// in the same slot takes over, and record the serial of the disk they
|
||||
// were raised on.
|
||||
diskRef := unifiedresources.ProxmoxPhysicalDiskAlertResourceID("cluster-a", "pve1", "/dev/sda")
|
||||
diskAlert := func(id, serial string) models.Alert {
|
||||
alert := active(diskRef)
|
||||
alert.ID, alert.Type, alert.Message = id, "disk-health", id
|
||||
alert.Metadata = map[string]interface{}{"disk_serial": serial}
|
||||
return alert
|
||||
}
|
||||
snapshot := reportingEnrichmentSnapshot{
|
||||
Resources: resources,
|
||||
ActiveAlerts: []models.Alert{
|
||||
active(alerts.DockerHostResourceID("docker-host-1")),
|
||||
active(alerts.DockerContainerResourceID("docker-host-1", "web-1", "web")),
|
||||
active(alerts.DockerContainerResourceID("docker-host-1", "", "sidecar")),
|
||||
active("cluster-a-pve1-local-zfs"),
|
||||
active("cluster-a-pve1-local-zfs/zfs-pool:local-zfs"),
|
||||
active("cluster-a-pve1-local-zfs/zfs-pool:local-zfs/device:sda2"),
|
||||
active("cluster-a-pve1-local-zfs2"),
|
||||
active("k8s-cluster-prod"),
|
||||
active("pmg-main"),
|
||||
active(alerts.DockerServiceResourceID("docker-host-1", "svc-1", "web")),
|
||||
active(alerts.DockerServiceResourceID("docker-host-1", "", "api")),
|
||||
active("agent:host-1/storage:unraid-array"),
|
||||
diskAlert("disk-health-replacement", "SER-B"),
|
||||
diskAlert("disk-health-predecessor", "SER-A"),
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range []struct {
|
||||
resourceType string
|
||||
resourceID string
|
||||
want []string
|
||||
}{
|
||||
{"docker-host", "agent-docker-1", []string{alerts.DockerHostResourceID("docker-host-1")}},
|
||||
{"app-container", "app-container-web", []string{alerts.DockerContainerResourceID("docker-host-1", "web-1", "web")}},
|
||||
{"app-container", "app-container-sidecar", []string{alerts.DockerContainerResourceID("docker-host-1", "", "sidecar")}},
|
||||
{"storage", "storage-zfs", []string{
|
||||
"cluster-a-pve1-local-zfs",
|
||||
"cluster-a-pve1-local-zfs/zfs-pool:local-zfs",
|
||||
"cluster-a-pve1-local-zfs/zfs-pool:local-zfs/device:sda2",
|
||||
}},
|
||||
{"k8s", "k8s-cluster-prod", []string{"k8s-cluster-prod"}},
|
||||
{"pmg", "pmg-eu", []string{"pmg-main"}},
|
||||
{"app-container", "docker-service-web", []string{alerts.DockerServiceResourceID("docker-host-1", "svc-1", "web")}},
|
||||
{"app-container", "docker-service-api", []string{alerts.DockerServiceResourceID("docker-host-1", "", "api")}},
|
||||
{"storage", "storage-unraid", []string{"agent:host-1/storage:unraid-array"}},
|
||||
{"disk", "physical-disk-sda", []string{diskRef}},
|
||||
} {
|
||||
req := reporting.MetricReportRequest{ResourceType: tc.resourceType, ResourceID: tc.resourceID}
|
||||
NewReportingHandlers(nil, nil).enrichReportRequest(context.Background(), "default", &req, snapshot, now.Add(-24*time.Hour), now)
|
||||
if got := reportAlertResourceIDs(req); !reflect.DeepEqual(got, tc.want) {
|
||||
t.Fatalf("%s report alerts = %v, want %v", tc.resourceType, got, tc.want)
|
||||
}
|
||||
// Only the alert naming the replacement disk's serial is its own;
|
||||
// its predecessor's alert on the same path stays off its report.
|
||||
if tc.resourceType == "disk" && req.Alerts[0].Message != "disk-health-replacement" {
|
||||
t.Fatalf("disk report alert = %+v", req.Alerts[0])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestContract_ReportingCatalogJSONSnapshot(t *testing.T) {
|
||||
handler := NewReportingHandlers(nil, nil)
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/admin/reports/catalog", nil)
|
||||
|
||||
@@ -694,6 +694,10 @@ func (h *ReportingHandlers) enrichReportRequest(ctx context.Context, orgID strin
|
||||
h.enrichVMReport(ctx, orgID, req, snapshot, start, end)
|
||||
case "system-container", "oci-container":
|
||||
h.enrichContainerReport(ctx, orgID, req, snapshot, start, end)
|
||||
case "agent":
|
||||
h.enrichAgentReport(req, snapshot, start, end)
|
||||
default:
|
||||
attachReportAlerts(req, snapshot, reportAlertScopeFor(req, findReportSubjectResource(req, snapshot)), start, end)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -849,41 +853,7 @@ func (h *ReportingHandlers) enrichNodeReport(req *reporting.MetricReportRequest,
|
||||
if storageNode != node.Name {
|
||||
continue
|
||||
}
|
||||
|
||||
var total, used, available int64
|
||||
var usagePerc float64
|
||||
if r.Metrics != nil && r.Metrics.Disk != nil {
|
||||
if r.Metrics.Disk.Total != nil {
|
||||
total = *r.Metrics.Disk.Total
|
||||
}
|
||||
if r.Metrics.Disk.Used != nil {
|
||||
used = *r.Metrics.Disk.Used
|
||||
}
|
||||
if total > 0 {
|
||||
available = total - used
|
||||
}
|
||||
usagePerc = r.Metrics.Disk.Percent
|
||||
if usagePerc == 0 && total > 0 {
|
||||
usagePerc = (float64(used) / float64(total)) * 100
|
||||
}
|
||||
}
|
||||
|
||||
var storageType, content string
|
||||
if r.Storage != nil {
|
||||
storageType = r.Storage.Type
|
||||
content = r.Storage.Content
|
||||
}
|
||||
|
||||
req.Storage = append(req.Storage, reporting.StorageInfo{
|
||||
Name: r.Name,
|
||||
Type: storageType,
|
||||
Status: string(r.Status),
|
||||
Total: total,
|
||||
Used: used,
|
||||
Available: available,
|
||||
UsagePerc: usagePerc,
|
||||
Content: content,
|
||||
})
|
||||
req.Storage = append(req.Storage, reportStorageInfo(r))
|
||||
case unifiedresources.ResourceTypePhysicalDisk:
|
||||
if r.PhysicalDisk == nil {
|
||||
continue
|
||||
@@ -895,20 +865,7 @@ func (h *ReportingHandlers) enrichNodeReport(req *reporting.MetricReportRequest,
|
||||
if diskNode != node.Name {
|
||||
continue
|
||||
}
|
||||
pd := r.PhysicalDisk
|
||||
temperatureWarning, temperatureCritical := reportDiskTemperatureThresholds(snapshot.alertManager, pd.DiskType)
|
||||
req.Disks = append(req.Disks, reporting.DiskInfo{
|
||||
Device: pd.DevPath,
|
||||
Model: pd.Model,
|
||||
Serial: pd.Serial,
|
||||
Type: pd.DiskType,
|
||||
Size: pd.SizeBytes,
|
||||
Health: pd.Health,
|
||||
Temperature: reportDiskTemperature(*pd),
|
||||
WearLevel: pd.Wearout,
|
||||
TemperatureWarning: temperatureWarning,
|
||||
TemperatureCritical: temperatureCritical,
|
||||
})
|
||||
req.Disks = append(req.Disks, reportDiskInfo(*r.PhysicalDisk, snapshot.alertManager))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,363 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
|
||||
"github.com/rcourtman/pulse-go-rewrite/pkg/reporting"
|
||||
)
|
||||
|
||||
// reportAlertScope is the set of alert resource IDs a report attributes to
|
||||
// its subject. It is built from the identities the subject's unified
|
||||
// resource carries, never from host or node names: an alert belongs to the
|
||||
// report of each resource whose identity it was raised under. A machine's
|
||||
// report therefore leaves out the guests, containers, storage pools and
|
||||
// disks it hosts, whose alerts are raised under their own identities and
|
||||
// appear on their own reports.
|
||||
type reportAlertScope struct {
|
||||
ids map[string]struct{}
|
||||
// childPrefixes match component alerts raised under the subject's own
|
||||
// alert identity, such as "agent:<host>/disk:/" for a filesystem the
|
||||
// agent reports.
|
||||
childPrefixes []string
|
||||
// serialGatedIDs are path-shaped references the subject owns only for
|
||||
// alerts that record its serial: a replacement disk in the same slot
|
||||
// takes over a Proxmox disk's device path (#2627).
|
||||
serialGatedIDs map[string]string
|
||||
}
|
||||
|
||||
func (s *reportAlertScope) addID(id string) {
|
||||
if id = strings.TrimSpace(id); id != "" {
|
||||
s.ids[id] = struct{}{}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *reportAlertScope) addChildPrefix(parentID string) {
|
||||
if parentID = strings.TrimSpace(parentID); parentID != "" {
|
||||
s.childPrefixes = append(s.childPrefixes, parentID+"/")
|
||||
}
|
||||
}
|
||||
|
||||
// matchesAlert reports whether an alert belongs to the subject.
|
||||
func (s reportAlertScope) matchesAlert(alert models.Alert) bool {
|
||||
if s.matches(alert.ResourceID) {
|
||||
return true
|
||||
}
|
||||
serial, gated := s.serialGatedIDs[strings.TrimSpace(alert.ResourceID)]
|
||||
if !gated {
|
||||
return false
|
||||
}
|
||||
recorded, _ := alert.Metadata["disk_serial"].(string)
|
||||
return strings.TrimSpace(recorded) != "" && strings.EqualFold(strings.TrimSpace(recorded), serial)
|
||||
}
|
||||
|
||||
func (s reportAlertScope) matches(resourceID string) bool {
|
||||
resourceID = strings.TrimSpace(resourceID)
|
||||
if resourceID == "" {
|
||||
return false
|
||||
}
|
||||
if _, ok := s.ids[resourceID]; ok {
|
||||
return true
|
||||
}
|
||||
for _, prefix := range s.childPrefixes {
|
||||
if strings.HasPrefix(resourceID, prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// reportAlertScopeFor returns the alert identities of a report subject:
|
||||
// - the requested unified ID and the resource's metrics-target ID, which
|
||||
// key provider incidents and the alerts evaluated on unified resources
|
||||
// (vSphere hosts, TrueNAS systems, Kubernetes, PBS, PMG);
|
||||
// - for a machine, its linked Proxmox node's source ID
|
||||
// ("<instance>-<node>"), which keys the node's own alerts, and its
|
||||
// Pulse agent's "agent:<host>" alerts with their component children
|
||||
// ("agent:<host>/disk:<mount>", disk temperature, SMART, RAID, the Unraid
|
||||
// array, custom sensors). A node alert that moved to the agent and the
|
||||
// agent's own alert for that metric therefore land in the same report,
|
||||
// where pkg/reporting counts them once;
|
||||
// - for a Docker runtime, container or Swarm service, the IDs the Docker
|
||||
// alert producer builds (alerts.DockerHostResourceID,
|
||||
// DockerContainerResourceID, DockerServiceResourceID);
|
||||
// - for a storage pool, its source ID and the pool's ZFS components
|
||||
// ("<storage>/zfs-pool:<pool>/device:<dev>"), or for an agent-reported
|
||||
// Unraid array, the agent alert raised on it;
|
||||
// - for a Proxmox physical disk, the alerts on its device-path reference
|
||||
// that record its serial.
|
||||
//
|
||||
// resource may be nil when the request names no live unified resource; the
|
||||
// scope then holds only the request's own IDs.
|
||||
func reportAlertScopeFor(req *reporting.MetricReportRequest, resource *unifiedresources.Resource) reportAlertScope {
|
||||
scope := reportAlertScope{ids: make(map[string]struct{}), serialGatedIDs: make(map[string]string)}
|
||||
if req != nil {
|
||||
scope.addID(req.ResourceID)
|
||||
scope.addID(req.MetricsResourceID)
|
||||
}
|
||||
if resource == nil {
|
||||
return scope
|
||||
}
|
||||
scope.addID(resource.ID)
|
||||
switch unifiedresources.CanonicalResourceType(resource.Type) {
|
||||
case unifiedresources.ResourceTypeAgent:
|
||||
if resource.Proxmox != nil {
|
||||
scope.addID(resource.Proxmox.SourceID)
|
||||
}
|
||||
if agentID := reportHostAgentID(*resource); agentID != "" {
|
||||
// Alert evaluation names a Pulse agent "agent:<host ID>"
|
||||
// (alerts.hostResourceID), the same reference the canonical
|
||||
// identity indexes for per-resource policy (#1497).
|
||||
scope.addID("agent:" + agentID)
|
||||
scope.addChildPrefix("agent:" + agentID)
|
||||
}
|
||||
if resource.Docker != nil && strings.TrimSpace(resource.Docker.HostSourceID) != "" {
|
||||
scope.addID(alerts.DockerHostResourceID(resource.Docker.HostSourceID))
|
||||
}
|
||||
case unifiedresources.ResourceTypeAppContainer:
|
||||
if docker := resource.Docker; docker != nil && strings.TrimSpace(docker.HostSourceID) != "" {
|
||||
// A container without an ID is referenced by its name, which
|
||||
// the container adapter carries as the resource name.
|
||||
scope.addID(alerts.DockerContainerResourceID(docker.HostSourceID, docker.ContainerID, resource.Name))
|
||||
}
|
||||
case unifiedresources.ResourceTypeDockerService:
|
||||
if docker := resource.Docker; docker != nil && strings.TrimSpace(docker.HostSourceID) != "" {
|
||||
// The service adapter carries the service name as the resource
|
||||
// name; the producer falls back to it when the ID is empty.
|
||||
serviceName := docker.ServiceName
|
||||
if strings.TrimSpace(serviceName) == "" {
|
||||
serviceName = resource.Name
|
||||
}
|
||||
scope.addID(alerts.DockerServiceResourceID(docker.HostSourceID, docker.ServiceID, serviceName))
|
||||
}
|
||||
case unifiedresources.ResourceTypeStorage:
|
||||
if resource.Proxmox != nil {
|
||||
scope.addID(resource.Proxmox.SourceID)
|
||||
scope.addChildPrefix(resource.Proxmox.SourceID)
|
||||
}
|
||||
// A Pulse agent reports an Unraid array as storage keyed
|
||||
// "<host>/storage:unraid-array", which is then the metrics target, and
|
||||
// raises the array's alert under its own identity as
|
||||
// "agent:<host>/storage:unraid-array". That alert belongs to both the
|
||||
// array's report and its machine's.
|
||||
if req != nil && reportStorageReportedOnlyByAgent(*resource) && strings.TrimSpace(req.MetricsResourceID) != "" {
|
||||
scope.addID("agent:" + strings.TrimSpace(req.MetricsResourceID))
|
||||
}
|
||||
case unifiedresources.ResourceTypePhysicalDisk:
|
||||
if px, disk := resource.Proxmox, resource.PhysicalDisk; px != nil && disk != nil &&
|
||||
strings.TrimSpace(px.Instance) != "" && strings.TrimSpace(px.NodeName) != "" &&
|
||||
strings.TrimSpace(disk.DevPath) != "" && strings.TrimSpace(disk.Serial) != "" {
|
||||
// Proxmox disk health and wearout alerts are raised on the
|
||||
// disk's device-path reference and record the disk's serial
|
||||
// (disk_serial). The path is not hardware identity, so an alert
|
||||
// counts for this disk only when it names this disk's serial.
|
||||
ref := unifiedresources.ProxmoxPhysicalDiskAlertResourceID(px.Instance, px.NodeName, disk.DevPath)
|
||||
scope.serialGatedIDs[ref] = strings.TrimSpace(disk.Serial)
|
||||
}
|
||||
}
|
||||
return scope
|
||||
}
|
||||
|
||||
// reportStorageReportedOnlyByAgent reports whether a storage resource comes
|
||||
// from a Pulse agent alone, so its metrics target is the agent's source ID.
|
||||
func reportStorageReportedOnlyByAgent(resource unifiedresources.Resource) bool {
|
||||
if !slices.Contains(resource.Sources, unifiedresources.SourceAgent) {
|
||||
return false
|
||||
}
|
||||
for _, source := range resource.Sources {
|
||||
if source != unifiedresources.SourceAgent {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// reportHostAgentID is the host ID of the Pulse agent reporting a machine.
|
||||
// vSphere hosts and TrueNAS systems also fill AgentData, so the agent source
|
||||
// must be present before the ID names a Pulse agent.
|
||||
func reportHostAgentID(resource unifiedresources.Resource) string {
|
||||
if resource.Agent == nil || !slices.Contains(resource.Sources, unifiedresources.SourceAgent) {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(resource.Agent.AgentID)
|
||||
}
|
||||
|
||||
// findReportSubjectResource returns the unified resource the request names.
|
||||
func findReportSubjectResource(req *reporting.MetricReportRequest, snapshot reportingEnrichmentSnapshot) *unifiedresources.Resource {
|
||||
if req == nil {
|
||||
return nil
|
||||
}
|
||||
for i := range snapshot.Resources {
|
||||
if snapshot.Resources[i].ID == req.ResourceID {
|
||||
return &snapshot.Resources[i]
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// attachReportAlerts adds the subject's open alerts and the alerts resolved
|
||||
// inside the report window, keeping each alert's resolution.
|
||||
func attachReportAlerts(req *reporting.MetricReportRequest, snapshot reportingEnrichmentSnapshot, scope reportAlertScope, start, end time.Time) {
|
||||
for _, alert := range snapshot.ActiveAlerts {
|
||||
if scope.matchesAlert(alert) {
|
||||
req.Alerts = append(req.Alerts, reportActiveAlertInfo(alert))
|
||||
}
|
||||
}
|
||||
for _, resolved := range snapshot.RecentlyResolved {
|
||||
if scope.matchesAlert(resolved.Alert) &&
|
||||
resolved.ResolvedTime.After(start) && resolved.ResolvedTime.Before(end) {
|
||||
req.Alerts = append(req.Alerts, reportResolvedAlertInfo(resolved))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// enrichAgentReport fills a report on a machine: every Proxmox node and
|
||||
// standalone Pulse agent is a unified `agent` resource in v6. Details come
|
||||
// from the resource's Proxmox and agent payloads, storage pools and physical
|
||||
// disks from the resources whose unified parent is the machine, and alerts
|
||||
// from reportAlertScopeFor.
|
||||
func (h *ReportingHandlers) enrichAgentReport(req *reporting.MetricReportRequest, snapshot reportingEnrichmentSnapshot, start, end time.Time) {
|
||||
resource := findReportSubjectResource(req, snapshot)
|
||||
if resource != nil {
|
||||
req.Resource = reportMachineResourceInfo(*resource)
|
||||
for i := range snapshot.Resources {
|
||||
child := &snapshot.Resources[i]
|
||||
if child.ParentID == nil || *child.ParentID != resource.ID {
|
||||
continue
|
||||
}
|
||||
switch unifiedresources.CanonicalResourceType(child.Type) {
|
||||
case unifiedresources.ResourceTypeStorage:
|
||||
req.Storage = append(req.Storage, reportStorageInfo(*child))
|
||||
case unifiedresources.ResourceTypePhysicalDisk:
|
||||
if child.PhysicalDisk != nil {
|
||||
req.Disks = append(req.Disks, reportDiskInfo(*child.PhysicalDisk, snapshot.alertManager))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
attachReportAlerts(req, snapshot, reportAlertScopeFor(req, resource), start, end)
|
||||
}
|
||||
|
||||
// reportMachineResourceInfo projects a machine's unified payloads onto the
|
||||
// report's resource details. The Proxmox API is authoritative for PVE
|
||||
// facts; the Pulse agent adds the operating system.
|
||||
func reportMachineResourceInfo(resource unifiedresources.Resource) *reporting.ResourceInfo {
|
||||
info := &reporting.ResourceInfo{
|
||||
Name: resource.Name,
|
||||
Status: string(resource.Status),
|
||||
Uptime: resource.Uptime,
|
||||
IPAddresses: append([]string(nil), resource.Identity.IPAddresses...),
|
||||
Tags: append([]string(nil), resource.Tags...),
|
||||
}
|
||||
if resource.Temperature != nil {
|
||||
temp := *resource.Temperature
|
||||
info.Temperature = &temp
|
||||
}
|
||||
if metrics := resource.Metrics; metrics != nil {
|
||||
if metrics.Memory != nil && metrics.Memory.Total != nil {
|
||||
info.MemoryTotal = *metrics.Memory.Total
|
||||
}
|
||||
if metrics.Disk != nil && metrics.Disk.Total != nil {
|
||||
info.DiskTotal = *metrics.Disk.Total
|
||||
}
|
||||
}
|
||||
if px := resource.Proxmox; px != nil {
|
||||
info.Host = px.HostURL
|
||||
info.Instance = px.Instance
|
||||
info.KernelVersion = px.KernelVersion
|
||||
info.PVEVersion = px.PVEVersion
|
||||
info.ClusterName = px.ClusterName
|
||||
info.IsCluster = px.IsClusterMember
|
||||
info.LoadAverage = append([]float64(nil), px.LoadAverage...)
|
||||
if info.Uptime == 0 {
|
||||
info.Uptime = px.Uptime
|
||||
}
|
||||
if px.CPUInfo != nil {
|
||||
info.CPUModel = px.CPUInfo.Model
|
||||
info.CPUCores = px.CPUInfo.Cores
|
||||
info.CPUSockets = px.CPUInfo.Sockets
|
||||
}
|
||||
}
|
||||
if agent := resource.Agent; agent != nil && reportHostAgentID(resource) != "" {
|
||||
info.OSName = agent.OSName
|
||||
info.OSVersion = agent.OSVersion
|
||||
if info.KernelVersion == "" {
|
||||
info.KernelVersion = agent.KernelVersion
|
||||
}
|
||||
if info.CPUCores == 0 {
|
||||
info.CPUCores = agent.CPUCount
|
||||
}
|
||||
if len(info.LoadAverage) == 0 {
|
||||
info.LoadAverage = append([]float64(nil), agent.LoadAverage...)
|
||||
}
|
||||
if info.Uptime == 0 {
|
||||
info.Uptime = agent.UptimeSeconds
|
||||
}
|
||||
if info.MemoryTotal == 0 && agent.Memory != nil {
|
||||
info.MemoryTotal = agent.Memory.Total
|
||||
}
|
||||
}
|
||||
return info
|
||||
}
|
||||
|
||||
// reportStorageInfo is the report row for a storage pool resource.
|
||||
func reportStorageInfo(r unifiedresources.Resource) reporting.StorageInfo {
|
||||
var total, used, available int64
|
||||
var usagePerc float64
|
||||
if r.Metrics != nil && r.Metrics.Disk != nil {
|
||||
if r.Metrics.Disk.Total != nil {
|
||||
total = *r.Metrics.Disk.Total
|
||||
}
|
||||
if r.Metrics.Disk.Used != nil {
|
||||
used = *r.Metrics.Disk.Used
|
||||
}
|
||||
if total > 0 {
|
||||
available = total - used
|
||||
}
|
||||
usagePerc = r.Metrics.Disk.Percent
|
||||
if usagePerc == 0 && total > 0 {
|
||||
usagePerc = (float64(used) / float64(total)) * 100
|
||||
}
|
||||
}
|
||||
|
||||
var storageType, content string
|
||||
if r.Storage != nil {
|
||||
storageType = r.Storage.Type
|
||||
content = r.Storage.Content
|
||||
}
|
||||
|
||||
return reporting.StorageInfo{
|
||||
Name: r.Name,
|
||||
Type: storageType,
|
||||
Status: string(r.Status),
|
||||
Total: total,
|
||||
Used: used,
|
||||
Available: available,
|
||||
UsagePerc: usagePerc,
|
||||
Content: content,
|
||||
}
|
||||
}
|
||||
|
||||
// reportDiskInfo is the report row for a physical disk resource. Its
|
||||
// temperature is the collected reading only (reportDiskTemperature), coloured
|
||||
// by the tenant's disk temperature alert thresholds; a nil manager means the
|
||||
// factory alert configuration.
|
||||
func reportDiskInfo(pd unifiedresources.PhysicalDiskMeta, manager *alerts.Manager) reporting.DiskInfo {
|
||||
temperatureWarning, temperatureCritical := reportDiskTemperatureThresholds(manager, pd.DiskType)
|
||||
return reporting.DiskInfo{
|
||||
Device: pd.DevPath,
|
||||
Model: pd.Model,
|
||||
Serial: pd.Serial,
|
||||
Type: pd.DiskType,
|
||||
Size: pd.SizeBytes,
|
||||
Health: pd.Health,
|
||||
Temperature: reportDiskTemperature(pd),
|
||||
WearLevel: pd.Wearout,
|
||||
TemperatureWarning: temperatureWarning,
|
||||
TemperatureCritical: temperatureCritical,
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user