gsadmin/pulse
1
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-10-25 13:10:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Show agents awaiting a report after server restart

Browse Source 
Keep retained enrollments unknown while their accepted-report lease is
current, then show offline if no fresh report arrives. Preserve historical
timestamps, removals and report ordering without manufacturing telemetry.
Use the shared first-report state in Machines, resource details and Settings.

Refs #2980
This commit is contained in:
rcourtman
2026-10-24 16:45:39 +01:00
parent 0a98f0a8de
commit 5f0037fc25
45 changed files with 772 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/release-control/v6/internal/status.json
+41
View File
@@ -4785,6 +4785,29 @@
"run",
"type-check"
]
},
{
"id": "ra42-startup-agent-history",
"run": [
"go",
"test",
"./internal/api",
"./internal/api/chartapi",
"-run",
"TestContract_MetricsHistoryLivePointOnlyWhileSourceReports|TestChartsDoNotSynthesizeCurrentMetricsForPendingEnrollment",
"-count=1"
]
},
{
"id": "ra42-startup-agent-report",
"run": [
"go",
"test",
"./internal/monitoring",
"-run",
"TestHostAgentStartupWaitsWithinAcceptedReportLease",
"-count=1"
]
}
],
"evidence": [
@@ -4805,6 +4828,24 @@
"kind": "file",
"evidence_tier": "test-proof"
},
{
"repo": "pulse",
"path": "internal/api/chartapi/service_handlers_test.go",
"kind": "file",
"evidence_tier": "test-proof"
},
{
"repo": "pulse",
"path": "internal/api/contract_test.go",
"kind": "file",
"evidence_tier": "test-proof"
},
{
"repo": "pulse",
"path": "internal/monitoring/monitor_host_agents_test.go",
"kind": "file",
"evidence_tier": "test-proof"
},
{
"repo": "pulse",
"path": "tests/integration/tests/91-operational-trust-attention-workbench.spec.ts",
docs/release-control/v6/internal/subsystems/agent-lifecycle.md
+5 -1
View File
@@ -9890,7 +9890,11 @@ node that PVE polling keeps fresh. The agent connection on the Connections
list and the update-readiness agent-continuity check therefore age a silent
agent from its last report instead of from the PVE poll, and mark it stale at
their own heartbeat cutoff (`fleethealth.AgentStaleThreshold`, which is
separate from the monitoring reporting lease). Agent registration, enrolment,
separate from the monitoring reporting lease). A saved enrollment awaiting its
first report in the current monitor generation retains pending delivery and
its historical receipt under the monitoring-owned lease. Connections must not
label that enrollment active, or turn an explicitly expired lease active using
the longer heartbeat fallback. Agent registration, enrolment,
install, update, removal and report identity are unchanged.
### Mock-mode read views fold linked fixture agents
docs/release-control/v6/internal/subsystems/api-contracts.md
+12
View File
@@ -21,6 +21,13 @@
## Purpose
### Pending agent delivery supplies no live chart fallback
A saved enrollment awaiting its first accepted report remains queryable by its
existing metrics target. History and summary chart routes retain stored series
but cannot synthesize a now-stamped live sample from that enrollment's absent
metrics. An accepted report restores normal live fallback behaviour.
### Typed MCP fleet filters survive HTTP projection
GET/DELETE capability projection preserves public string and boolean query
@@ -8025,6 +8032,11 @@ API-backed lifecycle state. The retired
`InfrastructureOperationsController.tsx` shell and
`useInfrastructureReportingState.tsx` reporting path must not be reintroduced
as parallel transport owners.
The `/api/connections` agent projection preserves monitoring's pending
first-report state across a restart without renewing the saved last-seen
timestamp. An explicitly offline agent cannot become active through the
longer fleet heartbeat fallback. API provider errors and pending provider
polls retain their own connection states.
That same `/api/connections` projection also owns collection-method truth for
the infrastructure ledger: `useConnectionsLedger.ts` must derive one canonical
subtitle (`via platform API`, `via Pulse Agent`, or `via platform API and
docs/release-control/v6/internal/subsystems/frontend-primitives.md
+12
View File
@@ -20,6 +20,18 @@
## Purpose
### Agent first-report status
Machines, resource detail drawers and agent connection rows use the shared
"Waiting for first report" label for monitoring's bounded pending startup
state. Machines treats that informational state as unknown rather than an outage.
The collapsed machine row shows the wording beside its name and suppresses
cached metric bars until accepted telemetry arrives. Consumers must not infer
the wait from a browser timer or hide a provider failure behind it. The expanded
drawer presents the wait as information and does not synthesize a current
chart reading from absent or cached metrics. Stored history remains available.
Pending API polls retain their own wording.
### Backup posture failure and retry stay source-local (24 October 2026)
The Backups posture notice offers a keyboard-operable retry with the existing
docs/release-control/v6/internal/subsystems/monitoring.md
+15 -1
View File
@@ -2680,6 +2680,17 @@ the live last-known host snapshot. After a server restart, durable continuity
must still expire persisted operation alerts on the same accepted-telemetry
lease; missing telemetry then remains visible through the separate confirmed
connectivity lifecycle.
After a restart, a saved agent enrollment without a report in this monitor
generation waits only while its last accepted report remains inside that same
lease. The read projection publishes pending agent delivery and unknown
resource status, preserving identity and the historical receipt with no
metrics. An accepted report replaces that projection. Lease expiry publishes
offline even if no report arrives. Already-expired or removed enrollments do
not receive a new grace period, and provider failures remain authoritative.
Rejected duplicate or buffered reports cannot hydrate a live host or finish
the wait. They preserve transport liveness and the accepted-report watermark.
Realtime resource conversion carries the same source delivery map and original
receipt as the canonical API, including pending delivery and absent metrics.
PBS backup snapshot refresh is a bounded monitoring hot path: group-level
snapshot fetches must run through the fixed worker pool in
`internal/monitoring/monitor_backups.go`, reuse cached snapshots on per-group
@@ -4855,7 +4866,10 @@ path through `evaluateHostAgents` and the registry-backed store refresh.
The metrics-history API's live fallback relies on the same projection: it
treats a host row reading `offline` as past its lease, and an agent linked to
a Proxmox node then answers from the node row. For node rows it takes a
a Proxmox node then answers from the node row. A pending startup enrollment
also supplies no live fallback. History and summary charts keep any actual
stored observations without synthesizing zero-valued current samples.
For node rows it takes a
projected `offline` or `unknown` as the lapse and otherwise reads the
registry's sightings (`NodeView.SourceStatus`), because a projected `warning`
is ambiguous: the stale pass shows an unreachable node that way, and so does a
docs/release-control/v6/internal/subsystems/performance-and-scalability.md
+6
View File
@@ -3152,6 +3152,12 @@ the agent resumes, without a reload. A stale Proxmox source
can still lose the aggregate to a live agent while its own filesystem rows
remain; such a guest stays on the Proxmox rule. The agent's OS, addresses and
physical disk health carry no stale cue yet.
Realtime resource rows carry canonical source delivery status so an open
browser observes a server restart without a REST enrichment race. An explicit
pending agent delivery with unknown resource status withdraws that row's
previous current metrics and agent telemetry, while retaining identity and
the existing history target. The same full-snapshot and delta merge enforce
this boundary; unrelated rows retain their usual incremental update path.
Guest, node, and Docker-host drawer headers follow the same frontend-primitives dependency
boundary for collapse: the owning table (Workloads for guests, the Proxmox
nodes table for nodes, the Docker hosts table for Docker hosts) owns which
docs/release-control/v6/internal/subsystems/security-privacy.md
+7
View File
@@ -18,6 +18,13 @@
## Purpose
### Agent delivery is separate from authenticated transport
A saved agent enrollment or an authenticated rejected report does not establish
accepted current telemetry after a server restart. Pending delivery stays
informational, with its historical receipt, and cannot establish a live metric
reading or imply that credentials or command-channel policy were verified.
### Restricted outbound connections keep approved destinations pinned
`pkg/securityutil/outbound_http.go` owns restricted HTTP destination validation,
docs/release-control/v6/internal/subsystems/storage-recovery.md
+7
View File
@@ -21,6 +21,13 @@
## Purpose
### Agent startup is not storage evidence
An agent's pending first report after a server restart is a monitoring-owned
delivery state. Its retained enrollment and historical receipt carry no fresh
storage metrics, healthy storage verdict or recovery evidence. Storage and
recovery consumers continue to require their own source observations.
### Failed posture refreshes withdraw retained assessments (24 October 2026)
Protection posture remains visible while a refresh is pending, but a settled
docs/release-control/v6/internal/subsystems/unified-resources.md
+10 -2
View File
@@ -676,8 +676,8 @@ fleet CPU attribution.
### Host continuity overlays reuse one build per registry generation
`ReadStateWithHostContinuity` overlays standalone agents that are offline
across a restart onto the live read state by building a registry clone. The
`ReadStateWithHostContinuity` overlays standalone agents awaiting a report or
offline across a restart onto the live read state by building a registry clone. The
monitor requests that overlay on every canonical read-state lookup, which
alert evaluation makes once per resource per poll, so a single offline agent
made every poll rebuild the registry once per resource for up to 72 hours. The
@@ -911,6 +911,14 @@ and linked Proxmox state. Even a future-dated saved enrollment cannot overwrite
live truth. Skipped records cannot attach superseded IDs or migrate operator
state. Distinct machines sharing a hostname remain distinct, and absent hosts
retain their historical identity without manufacturing a live sighting.
During the monitoring-owned reporting lease, an absent saved agent has unknown
resource status and pending agent delivery. Registry ingestion, stale marking,
cloning and serialized read-state seeding preserve that pending delivery rather
than treating its historical timestamp as a current sighting. Continuity carries
no metrics. Its lease and transition to offline are owned by monitoring.
Realtime rows carry source delivery alongside resource status. A pending
enrollment explicitly withdraws an earlier browser observation rather than
retaining cached metrics or an older agent facet as current evidence.
`TestHostContinuityCannotOverwriteCurrentResource`,
`TestHostContinuityRetainsDistinctMachinesWithSameHostname`, and the monitoring
`TestHostContinuityPreservesLiveAgentAfterReenrollment` pin this boundary (#1913).
frontend-modern/browser-verification/7fa94a1e4ee7d130.json
+65
View File
@@ -0,0 +1,65 @@
{
"version": 1,
"base_sha": "0a98f0a8debb2c96870d5ef0133d3b31491e830b",
"verified_at": "2026-10-24T15:41:10.218Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/Infrastructure/ResourceDetailDrawerOverviewTab.tsx",
"frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx",
"frontend-modern/src/components/Infrastructure/resourceDetailDrawerMetricsHistoryModel.ts",
"frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts",
"frontend-modern/src/components/Settings/connectionsTableModel.ts",
"frontend-modern/src/components/Settings/useConnectionsLedger.ts",
"frontend-modern/src/features/standalone/AgentsMachinesTable.tsx",
"frontend-modern/src/features/standalone/standalonePageModel.ts",
"frontend-modern/src/utils/agentResources.ts",
"frontend-modern/src/utils/resourceStateAdapters.ts"
],
"content_sha256": {
"frontend-modern/src/components/Infrastructure/ResourceDetailDrawerOverviewTab.tsx": "406ff7d3ec98ea605ccc1accc2cfe8e26c594cad8557c75b9383f2bac6f48038",
"frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx": "05197e1b31e6f25812fe2c652d0676ebbc3b607c4dc67df11bd2303967793c1d",
"frontend-modern/src/components/Infrastructure/resourceDetailDrawerMetricsHistoryModel.ts": "9c800cdffe393b3c7671f05940c6f2b682e67dff9b0922f2a098ef726e4f6aba",
"frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts": "f636b3c362e516f6d9078aa1c4dc431db500fec333a2645f9d04c6502b140498",
"frontend-modern/src/components/Settings/connectionsTableModel.ts": "c62059c55e1f4ff5ab397a2306ac2ec72ae1f0a84c9cc65a084f49cfef3f2ef3",
"frontend-modern/src/components/Settings/useConnectionsLedger.ts": "30c286d18cf51dff8984ce49ce6c76d5db1ecdcee642c80dd2e3ac38055d2728",
"frontend-modern/src/features/standalone/AgentsMachinesTable.tsx": "0c11c74b2e92f84af5c83a5064d04e3ee13843a43273bcef0038c8070648fcbe",
"frontend-modern/src/features/standalone/standalonePageModel.ts": "f7de0eadcd75e9b6e575a6fb53a5430985578fbdca504757d7a2457d1ab8a9da",
"frontend-modern/src/utils/agentResources.ts": "c3c6a003da42e0075cad3b9a5530cc64bf718529c9ea648e5777b8779933b5c6",
"frontend-modern/src/utils/resourceStateAdapters.ts": "a13924bf6410c4f3439ce59e68b582c45b88553f455fc143d4bd35f73ac7624a"
},
"routes": [
"/standalone",
"/settings/infrastructure"
],
"viewports": [
{
"width": 1440,
"height": 1000
},
{
"width": 768,
"height": 1024
},
{
"width": 390,
"height": 844
}
],
"states": [
"Recent saved agents show Waiting for first report with unknown status and no current metric bars.",
"An overdue saved agent remains offline. An unreported enrollment becomes offline at its original lease deadline.",
"Removed saved enrollments remain absent after startup and restart.",
"Expanded pending drawer shows an informational wait and empty history without fabricated zero-valued current readings.",
"An accepted report ends the wait and shows live CPU and memory readings.",
"An open browser and inline drawer survive an actual server restart, withdraw current readings, and retain labelled stored history.",
"Settings agent rows show the first-report label on desktop and narrow layouts."
],
"interactions": [
"Focus the machine expansion button, open and close with Enter, press Escape, and collapse through the drawer header.",
"Open the machine drawer by selecting its name on desktop and narrow viewports; inspect wrapping, chart placement and dismissal.",
"Navigate between /standalone and /settings/infrastructure and scroll the narrow Settings page to inspect the last pending row in the viewport.",
"Send an authenticated synthetic agent report and observe the transition to healthy telemetry.",
"Keep the page and drawer open during a private server restart and inspect the restored pending state and stored history.",
"Wait through the original reporting deadline, reload, and verify the overdue and removed enrollment states."
]
}
frontend-modern/src/components/Infrastructure/ResourceDetailDrawerOverviewTab.tsx
+9
View File
@@ -13,6 +13,8 @@ import { getAlertAttentionCopy } from '@/features/alerts/metricAlertPresentation
import { useAlertsActivation } from '@/stores/alertsActivation';
import { getAlertsForUnifiedResource, getOwnAlertsForUnifiedResource } from '@/utils/alerts';
import { GuestDrawerHistory } from '@/components/Workloads/GuestDrawerHistory';
import { InlineNotice } from '@/components/shared/InlineNotice';
import { getAgentFirstReportStatusIndicator } from '@/utils/agentResources';
import {
getAlertStatedRowLabels,
getDrawerChartAlerts,
@@ -404,6 +406,13 @@ export const ResourceDetailDrawerOverviewTab: Component<ResourceDetailDrawerOver
return (
<div class="space-y-3">
<Show when={getAgentFirstReportStatusIndicator(resource())}>
{(waiting) => (
<InlineNotice tone="info" role="status">
{waiting().label}
</InlineNotice>
)}
</Show>
<DrawerAttentionSection items={attentionItems()} />
<Show when={attentionItems().length === 0}>
<DrawerNoAlertsLine availabilityPassing={availabilityHealthy()} />
frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx
+6 -3
View File
@@ -17,6 +17,7 @@ import {
import { TagBadges } from '@/components/shared/TagBadges';
import { formatBytes, formatRelativeTime, formatUptime } from '@/utils/format';
import { useRelativeTimeNow } from '@/utils/relativeTimeClock';
import { getAgentFirstReportStatusIndicator } from '@/utils/agentResources';
import { getDockerImageRegistryLink } from '@/features/docker/dockerImageReference';
import { getDockerContainerUpdateAction } from '@/features/docker/dockerContainerUpdateAction';
import { getDockerImageOperationalPresentation } from '@/features/docker/dockerImagePresentation';
@@ -419,8 +420,10 @@ const dockerVolumeSection = (resource: Resource, now: number): DetailSection | n
// A resource that is not reporting must say so in front, not only in Details:
// offline is danger-toned (with when it was last seen), and a state Pulse
// cannot vouch for is warning-toned, so the drawer lifts either.
const observedStateRow = (status: string | undefined, lastSeen: string | undefined) => {
const state = (status || 'unknown').trim().toLowerCase();
const observedStateRow = (resource: Resource, lastSeen: string | undefined) => {
const waiting = getAgentFirstReportStatusIndicator(resource);
if (waiting) return makeDetailRow('Observed state', waiting.label);
const state = (resource.status || 'unknown').trim().toLowerCase();
const label = state.charAt(0).toUpperCase() + state.slice(1);
const offline = state === 'offline';
return makeDetailRow(
@@ -544,7 +547,7 @@ export const useResourceSummarySections = (
'Observed state',
getDockerContainerRunStatePresentation(props.resource),
)
: observedStateRow(props.resource.status, props.drawer.lastSeen()),
: observedStateRow(props.resource, props.drawer.lastSeen()),
docker
? dockerRuntimeRow(
'Health check',
frontend-modern/src/components/Infrastructure/__tests__/ResourceDetailDrawer.machine-history.test.tsx
+16
View File
@@ -80,6 +80,22 @@ afterEach(() => {
describe('ResourceDetailDrawer machine inventory', () => {
const gib = 1024 ** 3;
it('keeps startup waiting informational and withholds the current history fallback', () => {
const [machine, setMachine] = createStore(
resource({
status: 'unknown',
sourceStatus: { agent: { status: 'pending' } },
cpu: { current: 12 },
}),
);
render(() => <ResourceDetailDrawer resource={machine} />);
expect(screen.getByRole('status')).toHaveTextContent('Waiting for first report');
expect(screen.queryByTestId('resource-drawer-exceptions')).toBeNull();
expect(screen.getByTestId('machine-history')).not.toHaveAttribute('data-cpu');
setMachine({ status: 'online', sourceStatus: { agent: { status: 'online' } } });
expect(screen.queryByRole('status')).toBeNull();
expect(screen.getByTestId('machine-history')).toHaveAttribute('data-cpu', '12');
});
it('states one facts line and keeps the inventory under a closed Details', () => {
render(() => (
<ResourceDetailDrawer
frontend-modern/src/components/Infrastructure/__tests__/resourceDetailDrawerMetricsHistoryModel.branchcov0712.test.ts
+11
View File
@@ -402,6 +402,17 @@ describe('getResourceMetricsHistoryTarget branch coverage', () => {
});
describe('getResourceMetricsHistoryCurrentMetrics branch coverage', () => {
it('does not supply a current fallback while a saved agent awaits its first report', () => {
const waiting = baseResource({
status: 'unknown',
sourceStatus: { agent: { status: 'pending' } },
cpu: { current: 12 },
memory: { current: 25 },
disk: { current: 30 },
});
expect(getResourceMetricsHistoryCurrentMetrics(waiting)).toEqual({});
expect(getResourceMetricsHistoryTarget(waiting)).not.toBeNull();
});
it('returns every metric as a finite number when the resource is fully populated', () => {
// All finiteMetric calls take the "finite number -> return value" arm;
// memory/disk ternaries take the TRUE arm and getMemoryPercent/getDiskPercent
frontend-modern/src/components/Infrastructure/resourceDetailDrawerMetricsHistoryModel.ts
+2
View File
@@ -21,6 +21,7 @@ import {
import { asTrimmedString } from '@/utils/stringUtils';
import { getMemoryObservationPresentation } from '@/utils/memoryObservation';
import { getWorkloadGuestDiskRead } from '@/utils/workloadGuestPresentation';
import { getAgentFirstReportStatusIndicator } from '@/utils/agentResources';
const finiteMetric = (value: number | undefined): number | undefined =>
typeof value === 'number' && Number.isFinite(value) ? value : undefined;
@@ -260,6 +261,7 @@ export const getResourceMetricsHistoryDeferredMetrics = (
export const getResourceMetricsHistoryCurrentMetrics = (
resource: Resource,
): Record<string, number | undefined> => {
if (getAgentFirstReportStatusIndicator(resource)) return {};
const disk = guestDiskReading(resource);
const diskPercent =
resource.disk && (!disk || disk.state === 'current')
frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts
+3 -1
View File
@@ -6,6 +6,7 @@ import type { ResourceVMwareMeta } from '@/types/resource';
import { formatAbsoluteTime, formatRelativeTime } from '@/utils/format';
import { useRelativeTimeNow } from '@/utils/relativeTimeClock';
import { getAgentStatusIndicator } from '@/utils/status';
import { getAgentFirstReportStatusIndicator } from '@/utils/agentResources';
import { mapDockerContainerStatus } from '@/features/docker/dockerPageModel';
import {
dedupeResourceBadges,
@@ -111,7 +112,8 @@ export const useResourceDetailDrawerDerivedState = (
const statusIndicator = createMemo(() =>
resource().type === 'app-container'
? mapDockerContainerStatus(resource())
: getAgentStatusIndicator({ status: resource().status }),
: (getAgentFirstReportStatusIndicator(resource()) ??
getAgentStatusIndicator({ status: resource().status })),
);
const healthIssue = createMemo(() => getResourceHealthIssuePresentation(resource()));
// An open drawer keeps its rows while a resource's timestamps stay the same,
frontend-modern/src/components/Settings/__tests__/InfrastructureSourceManager.test.tsx
+42 -1
View File
@@ -1,4 +1,12 @@
import { cleanup, fireEvent, render, screen, waitFor, within } from '@solidjs/testing-library';
import {
cleanup,
fireEvent,
render,
renderHook,
screen,
waitFor,
within,
} from '@solidjs/testing-library';
import { afterEach, describe, expect, it, vi } from 'vitest';
import infrastructureSourceManagerSource from '../InfrastructureSourceManager.tsx?raw';
import {
@@ -14,6 +22,9 @@ import {
type InfrastructureSystemRow,
} from '../connectionsTableModel';
import type { Connection } from '@/api/connections';
import { ConnectionsAPI } from '@/api/connections';
import { resetCreateNonSuspendingQueryCacheForTest } from '@/hooks/createNonSuspendingQuery';
import { useConnectionsLedger } from '../useConnectionsLedger';
const clusterDeployApi = vi.hoisted(() => ({
getCandidates: vi.fn(),
@@ -112,9 +123,39 @@ const member = (
describe('InfrastructureSourceManager setup summary', () => {
afterEach(() => {
cleanup();
resetCreateNonSuspendingQueryCacheForTest();
vi.restoreAllMocks();
Object.defineProperty(window, 'innerWidth', { configurable: true, value: 1024 });
});
it('renders the canonical first-report label through the real connection ledger', async () => {
resetCreateNonSuspendingQueryCacheForTest();
vi.spyOn(ConnectionsAPI, 'list').mockResolvedValue({
connections: [connectionFixture({ state: 'pending', name: 'Waiting host' })],
systems: [],
});
const ledger = renderHook(() => useConnectionsLedger());
await waitFor(() => expect(ledger.result.rows()).toHaveLength(1));
const onOpenConnection = vi.fn();
render(() => (
<InfrastructureSourceManager
rows={ledger.result.rows}
discoveredNodes={() => []}
discoveryEnabled={false}
discoveryScanStatus={() => ({ scanning: false })}
readOnly={false}
onAddSource={vi.fn()}
onAddInfrastructure={vi.fn()}
onOpenConnection={onOpenConnection}
onOpenAgentDoctor={vi.fn()}
/>
));
expect(screen.getByText('Waiting for first report')).toBeInTheDocument();
expect(screen.queryByText('Unreachable')).toBeNull();
fireEvent.click(screen.getByRole('button', { name: /^Manage$/i }));
expect(onOpenConnection).toHaveBeenCalled();
});
it('renders manual discovery as a visible operator action', () => {
const onRunDiscovery = vi.fn();
const onOpenDiscoverySettings = vi.fn();
frontend-modern/src/components/Settings/__tests__/InfrastructureWorkspace.test.tsx
+37
View File
@@ -3,6 +3,7 @@ import { createSignal } from 'solid-js';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { Connection } from '@/api/connections';
import {
agentAttachmentProblem,
infrastructureRowMonitoringPath,
type InfrastructureSystemMemberRow,
type InfrastructureSystemRow,
@@ -329,6 +330,42 @@ describe('InfrastructureWorkspace', () => {
const renderWorkspace = (propOverrides: Record<string, unknown> = {}) =>
render(() => (<InfrastructureWorkspace {...{ ...baseProps(), ...propOverrides }} />) as any);
it('keeps a working provider active while its attached agent awaits a first report', async () => {
const provider = connectionFixture();
const agent = connectionFixture({ id: 'agent:waiting', type: 'agent', state: 'pending' });
connectionState.rows = [
{
id: provider.id,
ownerType: provider.type,
name: provider.name,
subtitle: 'via platform API and Pulse Agent',
source: 'both',
host: provider.address,
coverageLabels: ['VMs'],
statusLabel: 'Active',
statusClassName: 'bg-green-100',
agentUpdateCount: 0,
lastActivityText: '1m ago',
...emptyFleetRow,
problem: agentAttachmentProblem(agent),
enabled: true,
canEdit: true,
canPause: false,
canRemove: true,
isAgent: false,
isCluster: false,
attachedConnections: [agent],
members: [],
connection: provider,
},
];
renderWorkspace();
await waitFor(() => expect(screen.getByText('Waiting for first report')).toBeInTheDocument());
expect(screen.getAllByText('Active').length).toBeGreaterThan(0);
expect(screen.queryByText('Unreachable')).toBeNull();
expect(screen.getByRole('button', { name: /^Manage$/i })).toBeInTheDocument();
});
it('renders the source manager landing without empty platform sections', async () => {
renderWorkspace();
frontend-modern/src/components/Settings/__tests__/connectionsTableModel.test.ts
+11
View File
@@ -1,6 +1,7 @@
import { describe, expect, it } from 'vitest';
import type { Connection, ConnectionAgentIdentity } from '@/api/connections';
import {
agentAttachmentProblem,
connectionAgentIdentitySummary,
fleetGovernanceSignalsForConnection,
primaryRowProblem,
@@ -26,6 +27,16 @@ const connectionFixture = (overrides: Partial<Connection> = {}): Connection => (
});
describe('visibleFleetGovernanceSignals', () => {
it('names a pending attached agent without calling it an outage', () => {
expect(agentAttachmentProblem(connectionFixture({ state: 'pending' }))).toMatchObject({
label: 'Waiting for first report',
tone: 'warning',
});
expect(agentAttachmentProblem(connectionFixture({ state: 'unreachable' }))).toMatchObject({
label: 'Agent unreachable',
tone: 'critical',
});
});
it('surfaces the failed module and its operator-facing reason', () => {
const signals = fleetGovernanceSignalsForConnection(
connectionFixture({
frontend-modern/src/components/Settings/__tests__/useConnectionsLedger.test.ts
+25
View File
@@ -21,6 +21,31 @@ describe('useConnectionsLedger', () => {
expect(useConnectionsLedgerSource).not.toContain('createResource');
});
it('names the first-report wait only for pending agents', async () => {
const connections = [
{ id: 'agent:waiting', type: 'agent', name: 'Waiting host', source: 'agent' },
{ id: 'pve:waiting', type: 'pve', name: 'Waiting API', source: 'manual' },
].map((value) => ({
...value,
address: value.name,
state: 'pending',
stateReason: '',
enabled: true,
surfaces: ['host'],
scope: { host: true },
lastSeen: null,
lastError: null,
capabilities: { supportsPause: false, supportsScope: false, supportsTest: false },
})) as Connection[];
vi.spyOn(ConnectionsAPI, 'list').mockResolvedValue({ connections, systems: [] });
const { result } = renderHook(() => useConnectionsLedger());
await waitFor(() => expect(result.rows()).toHaveLength(2));
expect(result.rows().find((row) => row.ownerType === 'agent')?.statusLabel).toBe(
'Waiting for first report',
);
expect(result.rows().find((row) => row.ownerType === 'pve')?.statusLabel).toBe('Pending');
});
it('renders standalone agent rows with compact host identity and endpoint context', async () => {
const connections: Connection[] = [
{
frontend-modern/src/components/Settings/connectionsTableModel.ts
+2 -1
View File
@@ -13,6 +13,7 @@ import type {
ConnectionFleetVersionDrift,
} from '@/api/connections';
import { getAgentHostProfileFamily } from '@/utils/platformSupportManifest';
import { AGENT_WAITING_FOR_FIRST_REPORT_LABEL } from '@/utils/agentResources';
import {
buildDockerPath,
buildKubernetesPath,
@@ -868,7 +869,7 @@ export const agentAttachmentProblem = (agent: Connection): InfrastructureRowProb
};
case 'pending':
return {
label: 'Agent pending first report',
label: AGENT_WAITING_FOR_FIRST_REPORT_LABEL,
detail: agent.stateReason || 'The Pulse Agent has not reported yet.',
tone: 'warning',
};
frontend-modern/src/components/Settings/useConnectionsLedger.ts
+6 -1
View File
@@ -1,6 +1,7 @@
import { createMemo } from 'solid-js';
import { createNonSuspendingQuery } from '@/hooks/createNonSuspendingQuery';
import { formatConnectionErrorMessage } from '@/utils/connectionErrorPresentation';
import { AGENT_WAITING_FOR_FIRST_REPORT_LABEL } from '@/utils/agentResources';
import {
ConnectionsAPI,
type Connection,
@@ -342,6 +343,10 @@ const buildRow = (
const clusterName = system?.clusterName?.trim();
const isCluster = ownerType === 'pve' && Boolean(clusterName) && members.length > 0;
const isStandaloneAgent = !isCluster && primaryConnection.type === 'agent';
const statusLabel =
rolledState === 'pending' && primaryConnection.type === 'agent'
? AGENT_WAITING_FOR_FIRST_REPORT_LABEL
: presentation.label;
const name =
ownerType === 'pve' && clusterName
? clusterName
@@ -414,7 +419,7 @@ const buildRow = (
source,
host,
coverageLabels,
statusLabel: presentation.label,
statusLabel,
statusClassName: presentation.badgeClass,
agentUpdateCount: agentUpdateCountFor(componentConnections),
lastActivityText: rollup
frontend-modern/src/features/standalone/AgentsMachinesTable.tsx
+22 -8
View File
@@ -63,7 +63,10 @@ import { useColumnVisibility } from '@/hooks/useColumnVisibility';
import type { Disk } from '@/types/api';
import type { Resource, ResourceAvailabilityMeta } from '@/types/resource';
import type { MetricDisplayThresholds } from '@/utils/metricThresholds';
import { getActionableAgentIdFromResource } from '@/utils/agentResources';
import {
getActionableAgentIdFromResource,
getAgentFirstReportStatusIndicator,
} from '@/utils/agentResources';
import { getUnifiedResourceAlertStyles } from '@/utils/alerts';
import { formatBytes, formatSpeed, formatObservedSpeed, normalizeDiskArray } from '@/utils/format';
import { STORAGE_KEYS } from '@/utils/localStorage';
@@ -1579,7 +1582,9 @@ export const AgentsMachinesTable: Component<{
return `Agent has stopped reporting.${when} Re-run the install command from the Pulse UI to refresh its token.`;
};
const indicator = () => getStandaloneResourceStatusIndicator(machine, now());
const canRenderMetrics = () => indicator().variant !== 'danger';
const awaitingReport = () => getAgentFirstReportStatusIndicator(machine);
const canRenderMetrics = () =>
!awaitingReport() && indicator().variant !== 'danger';
const telemetryFallback = () =>
canRenderMetrics()
? outdatedAgentTelemetryFallbackFor(machine, props.targetAgentVersion)
@@ -1721,12 +1726,21 @@ export const AgentsMachinesTable: Component<{
title={indicator().label}
ariaLabel={indicator().label}
/>
<ResourceNameWithWebInterfaceLink
name={name()}
url={savedWebInterfaceUrl()}
class="min-w-0"
nameClass="truncate font-semibold text-base-content"
/>
<div class="min-w-0">
<ResourceNameWithWebInterfaceLink
name={name()}
url={savedWebInterfaceUrl()}
class="min-w-0"
nameClass="truncate font-semibold text-base-content"
/>
<Show when={awaitingReport()}>
{(waiting) => (
<span class="block whitespace-normal text-[10px] leading-snug text-muted">
{waiting().label}
</span>
)}
</Show>
</div>
</div>
</TableCell>
<Show when={columnVisibility.isColumnVisible('system')}>
frontend-modern/src/features/standalone/__tests__/AgentsMachinesTable.test.tsx
+26
View File
@@ -162,6 +162,32 @@ afterEach(() => {
});
describe('AgentsMachinesTable', () => {
it('shows the first-report wait beside the name and hides cached metric bars until reporting', () => {
const [machine, setMachine] = createStore(
resource({
id: 'waiting',
name: 'Waiting host',
status: 'unknown',
lastSeen: Date.now(),
sourceStatus: { agent: { status: 'pending' } },
cpu: { current: 12 },
}),
);
render(() => (
<AgentsMachinesTable
resources={[machine]}
emptyIcon={emptyIcon}
emptyTitle="No machines"
emptyDescription="Install Pulse Agent."
/>
));
expect(screen.getByText('Waiting for first report')).toBeVisible();
expect(screen.queryByTestId('agent-machine-cpu-bar')).not.toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Offline, 0' })).toBeInTheDocument();
setMachine({ status: 'online', sourceStatus: { agent: { status: 'online' } } });
expect(screen.queryByText('Waiting for first report')).not.toBeInTheDocument();
expect(screen.getByTestId('agent-machine-cpu-bar')).toHaveAttribute('data-usage', '12');
});
it('keeps health counts and selected rows truthful for retained, arriving and withdrawn reports', () => {
const start = Date.parse('2026-10-11T17:00:00Z');
vi.useFakeTimers({ toFake: ['Date', 'setInterval', 'clearInterval'], now: start });
frontend-modern/src/features/standalone/__tests__/standalonePageModel.test.ts
+23
View File
@@ -3,6 +3,7 @@ import type { Resource } from '@/types/resource';
import {
buildStandalonePageModel,
buildStandalonePostureSummary,
getStandaloneResourceStatusIndicator,
sortStandaloneResourcesByAttention,
} from '../standalonePageModel';
@@ -21,6 +22,28 @@ const resource = (overrides: Partial<Resource>): Resource =>
}) as Resource;
describe('standalonePageModel', () => {
it('keeps startup waiting out of the outage and attention counts, then follows recovery or expiry', () => {
const saved = resource({ status: 'unknown', sourceStatus: { agent: { status: 'pending' } } });
expect(getStandaloneResourceStatusIndicator(saved)).toEqual({
variant: 'info',
label: 'Waiting for first report',
});
expect(buildStandalonePostureSummary([saved])).toMatchObject({
total: 1,
unknown: 1,
critical: 0,
attention: 0,
});
const now = Date.now();
const online = resource({
status: 'online',
lastSeen: now,
sourceStatus: { agent: { status: 'online' } },
});
expect(getStandaloneResourceStatusIndicator(online, now).label).toBe('Online');
const expired = resource({ status: 'offline', sourceStatus: { agent: { status: 'stale' } } });
expect(getStandaloneResourceStatusIndicator(expired).variant).toBe('danger');
});
it('classifies each resource once per attention sort and reclassifies on the next call', () => {
const now = 1_700_000_030_000;
let firstStatus: Resource['status'] = 'online';
frontend-modern/src/features/standalone/standalonePageModel.ts
+6 -1
View File
@@ -1,6 +1,9 @@
import type { Resource } from '@/types/resource';
import { getAvailabilityProbePresentation } from '@/utils/availabilityProbePresentation';
import { isPulseAgentPlatformResource } from '@/utils/agentResources';
import {
getAgentFirstReportStatusIndicator,
isPulseAgentPlatformResource,
} from '@/utils/agentResources';
import { getSimpleStatusIndicator } from '@/utils/status';
export interface StandalonePageModel {
@@ -28,6 +31,8 @@ export const isAgentlessAvailabilityResource = (resource: Resource): boolean =>
const AGENT_REPORT_STALE_AFTER_MS = 5 * 60 * 1000;
export const getStandaloneResourceStatusIndicator = (resource: Resource, nowMs = Date.now()) => {
const waiting = getAgentFirstReportStatusIndicator(resource);
if (waiting) return waiting;
const indicator = getSimpleStatusIndicator(resource.status);
if (indicator.variant === 'danger') return indicator;
const availability = getAvailabilityProbePresentation(resource, new Date(nowMs));
frontend-modern/src/utils/__tests__/agentResources.test.ts
+18
View File
@@ -1,6 +1,7 @@
import { describe, expect, it } from 'vitest';
import type { Resource } from '@/types/resource';
import {
getAgentFirstReportStatusIndicator,
getActionableAgentIdFromResource,
getActionableDockerRuntimeIdFromResource,
getActionableKubernetesClusterIdFromResource,
@@ -31,6 +32,23 @@ const makeResource = (overrides: Partial<Resource> = {}): Resource =>
}) as Resource;
describe('agentResources', () => {
it('labels an unobserved startup agent without overriding a provider verdict', () => {
const pending = { agent: { status: 'pending', lastSeen: '2026-10-24T12:00:00Z' } };
expect(
getAgentFirstReportStatusIndicator(
makeResource({ status: 'unknown', sourceStatus: pending }),
),
).toEqual({
variant: 'info',
label: 'Waiting for first report',
});
for (const status of ['online', 'offline', 'warning'] as const) {
expect(
getAgentFirstReportStatusIndicator(makeResource({ status, sourceStatus: pending })),
).toBeUndefined();
}
expect(getAgentFirstReportStatusIndicator(makeResource({ status: 'unknown' }))).toBeUndefined();
});
it('resolves explicit agent ids from canonical and platform facets', () => {
expect(
getExplicitAgentIdFromResource(
frontend-modern/src/utils/__tests__/resourceStateAdapters.test.ts
+47
View File
@@ -12,6 +12,53 @@ import {
} from '../resourceStateAdapters';
import type { Resource } from '@/types/resource';
it('withdraws a previous browser observation when realtime reports startup pending delivery', () => {
const existing: Resource = {
id: 'waiting-agent',
type: 'agent',
name: 'Host',
displayName: 'Host',
platformId: 'agent',
platformType: 'agent',
sourceType: 'agent',
status: 'online',
lastSeen: 1000,
sources: ['agent'],
cpu: { current: 12 },
memory: { current: 25 },
uptime: 60,
agent: { agentId: 'waiting-agent', cpuCount: 4 } as Resource['agent'],
};
const incoming: Resource = {
id: existing.id,
type: 'agent',
name: 'Host',
displayName: 'Host',
platformId: 'agent',
platformType: 'agent',
sourceType: 'agent',
status: 'unknown',
lastSeen: 1000,
sources: ['agent'],
sourceStatus: { agent: { status: 'pending' } },
agent: { agentId: 'waiting-agent' } as Resource['agent'],
};
for (const result of [
mergeCanonicalResourceSnapshot([incoming], [existing]),
mergeCanonicalResourceDeltaSnapshot([incoming], [existing], new Set([incoming.id])),
]) {
expect(result[0]).toMatchObject({
status: 'unknown',
lastSeen: 1000,
sourceStatus: incoming.sourceStatus,
});
expect(result[0]?.cpu).toBeUndefined();
expect(result[0]?.memory).toBeUndefined();
expect(result[0]?.uptime).toBeUndefined();
expect(result[0]?.agent?.cpuCount).toBeUndefined();
}
});
const createNodeResource = (platformData: Record<string, unknown>): Resource =>
({
id: 'node-1',
frontend-modern/src/utils/agentResources.ts
+10
View File
@@ -6,6 +6,16 @@ import {
} from '@/utils/discoveryTarget';
import { normalizeSourcePlatformKey } from '@/utils/sourcePlatforms';
import { asTrimmedString } from '@/utils/stringUtils';
import type { StatusIndicator } from '@/utils/status';
export const AGENT_WAITING_FOR_FIRST_REPORT_LABEL = 'Waiting for first report';
export const getAgentFirstReportStatusIndicator = (
resource: Pick<Resource, 'status' | 'sourceStatus'>,
): StatusIndicator | undefined =>
resource.sourceStatus?.agent?.status === 'pending' && resource.status === 'unknown'
? { variant: 'info', label: AGENT_WAITING_FOR_FIRST_REPORT_LABEL }
: undefined;
const AGENT_FACET_INFRASTRUCTURE_TYPES = new Set<ResourceType>(['agent', 'pbs', 'pmg']);
const AGENT_PROFILE_ASSIGNABLE_TYPES = new Set<ResourceType>([
frontend-modern/src/utils/resourceStateAdapters.ts
+16 -2
View File
@@ -27,6 +27,7 @@ import { $RAW } from 'solid-js/store';
import type { Resource } from '@/types/resource';
import {
getExplicitAgentIdFromResource,
getAgentFirstReportStatusIndicator,
getExplicitResourceClusterName,
hasDockerFacetEvidence,
} from '@/utils/agentResources';
@@ -161,8 +162,8 @@ const mergeAgentFacet = <T extends JsonRecord>(incoming?: T, existing?: T): T |
return next;
};
// Source status is per-source evidence the realtime payload never carries, so
// it survives merges as REST enrichment. Entries of sources the merged row no
// Thin producers can omit source status, so it survives as REST enrichment.
// Entries of sources the merged row no
// longer lists are stale: they would keep a split-off provider claiming the row.
const pruneSourceStatus = (
value: unknown,
@@ -941,6 +942,19 @@ export const mergeCanonicalResource = (incoming: Resource, existing?: Resource):
} else if (merged.sourceStatus !== undefined) {
delete merged.sourceStatus;
}
if (getAgentFirstReportStatusIndicator(incoming)) {
// A new monitor generation has no accepted telemetry for this enrollment.
// Its explicit delivery state withdraws the previous browser observation.
merged.cpu = undefined;
merged.memory = undefined;
merged.disk = undefined;
merged.network = undefined;
merged.diskIO = undefined;
merged.temperature = undefined;
merged.uptime = undefined;
merged.agent = incoming.agent;
merged.platformData = incoming.platformData;
}
return merged;
};
internal/api/chartapi/service.go
+9 -1
View File
@@ -362,7 +362,7 @@ func (r *Service) HandleCharts(w http.ResponseWriter, req *http.Request) {
if batchMetrics, ok := agentBatchMetrics[request.SQLResourceID]; ok {
oldestTimestamp = fillChartSeriesFromBatch(agentData[hID], batchMetrics, oldestTimestamp)
}
if len(agentData[hID]["cpu"]) == 0 {
if len(agentData[hID]["cpu"]) == 0 && !hostAgentAwaitingFirstReport(h) {
agentData[hID]["cpu"] = []MetricPoint{{Timestamp: currentTime, Value: h.CPUPercent()}}
agentData[hID]["memory"] = []MetricPoint{{Timestamp: currentTime, Value: h.MemoryPercent()}}
agentData[hID]["disk"] = []MetricPoint{{Timestamp: currentTime, Value: h.DiskPercent()}}
@@ -478,6 +478,11 @@ func parseWorkloadMaxPoints(raw string) int {
func ParseWorkloadMaxPoints(raw string) int { return parseWorkloadMaxPoints(raw) }
func hostAgentAwaitingFirstReport(host *unifiedresources.HostView) bool {
status, ok := host.SourceStatus(unifiedresources.SourceAgent)
return ok && status.Status == "pending"
}
func hostAgentChartRequest(host *unifiedresources.HostView) (string, monitoring.GuestChartRequest, bool) {
if host == nil {
return "", monitoring.GuestChartRequest{}, false
@@ -1959,6 +1964,9 @@ func (r *Service) HandleInfrastructureCharts(w http.ResponseWriter, req *http.Re
if len(agentData[hID][metricType]) > 0 {
continue
}
if hostAgentAwaitingFirstReport(h) {
continue
}
var value float64
hasFallbackValue := true
switch metricType {
internal/api/chartapi/service_handlers_test.go
+47
View File
@@ -7,10 +7,12 @@ import (
"net/http/httptest"
"reflect"
"sort"
"strings"
"testing"
"time"
"unsafe"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
"github.com/rcourtman/pulse-go-rewrite/internal/testutil"
@@ -236,6 +238,51 @@ func TestHandleCharts_UsesCanonicalMockUnifiedReadStateForVMwareHosts(t *testing
}
}
func TestChartsDoNotSynthesizeCurrentMetricsForPendingEnrollment(t *testing.T) {
monitor, state, history := newTestMonitor(t)
syncTestResourceStore(t, monitor, state)
continuity := config.NewHostContinuityStore(t.TempDir(), nil)
if err := continuity.Upsert(config.HostContinuityEntry{HostID: "waiting-agent", Hostname: "waiting", LastSeen: time.Now().Add(-time.Second), IntervalSeconds: 30}); err != nil {
t.Fatal(err)
}
setUnexportedField(t, monitor, "hostContinuityStore", continuity)
for _, stored := range []bool{false, true} {
if stored {
history.AddGuestMetric("agent:waiting-agent", "cpu", 37, time.Now().Add(-time.Minute))
}
for _, path := range []string{"/api/charts?range=5m", "/api/charts/infrastructure?range=5m"} {
service := NewService(fixedMonitorResolver{monitor: monitor})
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, path, nil)
if strings.HasPrefix(path, "/api/charts/infrastructure") {
service.HandleInfrastructureCharts(rec, req)
} else {
service.HandleCharts(rec, req)
}
var response struct {
AgentData map[string]VMChartData `json:"agentData"`
}
if rec.Code != http.StatusOK {
t.Fatalf("%s: status %d: %s", path, rec.Code, rec.Body.String())
}
if err := json.Unmarshal(rec.Body.Bytes(), &response); err != nil {
t.Fatal(err)
}
series := response.AgentData["waiting-agent"]
if _, ok := response.AgentData["waiting-agent"]; !ok {
t.Fatalf("%s lost the pending enrollment", path)
}
if stored {
if len(series["cpu"]) != 1 || series["cpu"][0].Value != 37 {
t.Fatalf("%s lost stored history: %+v", path, series)
}
} else if len(series) != 0 {
t.Fatalf("%s invented current readings: %+v", path, series)
}
}
}
}
func TestHandleInfrastructureCharts_Lightweight(t *testing.T) {
monitor, state, _ := newTestMonitor(t)
state.Nodes = []models.Node{{
internal/api/connections_aggregator.go
+10 -1
View File
@@ -613,7 +613,16 @@ func buildAgentConnection(host models.Host, expectedAgentVersion string, now tim
versionDrift := fleethealth.DeriveAgentVersionDrift(currentAgentVersion, expectedAgentVersion)
updateAvailable := versionDrift == fleethealth.AgentVersionBehind
agentIdentity := connectionAgentIdentityForHost(host)
switch fleethealth.DeriveAgentLiveness(host.LastSeen, now, host.IntervalSeconds) {
liveness := fleethealth.DeriveAgentLiveness(host.LastSeen, now, host.IntervalSeconds)
if host.Status == "pending" {
liveness = fleethealth.AgentLivenessPending
reason = "Waiting for first report after server restart"
} else if host.Status == "offline" && lastSeen != nil {
// An expired accepted-report lease is already an offline verdict. The
// fleet's longer fallback heartbeat cutoff cannot turn it active again.
liveness = fleethealth.AgentLivenessStale
}
switch liveness {
case fleethealth.AgentLivenessPending:
state = ConnectionStatePending
case fleethealth.AgentLivenessStale:
internal/api/connections_aggregator_test.go
+9
View File
@@ -180,6 +180,8 @@ func TestBuildConnections_AgentStateFromLastSeen(t *testing.T) {
now := time.Now()
in := aggregatorInputs{
hosts: []models.Host{
{ID: "startup", Hostname: "waiting", Status: "pending", LastSeen: now.Add(-10 * time.Second), IntervalSeconds: 30},
{ID: "expired-startup", Hostname: "missed-report", Status: "offline", LastSeen: now.Add(-4 * time.Minute), IntervalSeconds: 30},
{ID: "fresh", Hostname: "h1", LastSeen: now.Add(-10 * time.Second), IntervalSeconds: 30},
{ID: "stale", Hostname: "h2", LastSeen: now.Add(-5*time.Minute - time.Second), IntervalSeconds: 30},
{ID: "never", Hostname: "h3"},
@@ -200,6 +202,13 @@ func TestBuildConnections_AgentStateFromLastSeen(t *testing.T) {
if byID["agent:never"].State != ConnectionStatePending {
t.Fatalf("never-reported agent: got %q, want pending", byID["agent:never"].State)
}
startup := byID["agent:startup"]
if startup.State != ConnectionStatePending || startup.StateReason != "Waiting for first report after server restart" || startup.LastSeen == nil || !startup.LastSeen.Equal(now.Add(-10*time.Second)) {
t.Fatalf("saved startup agent reported live: %+v", startup)
}
if byID["agent:expired-startup"].State != ConnectionStateStale {
t.Fatal("expired startup lease became active without a report")
}
for _, c := range got {
if c.Capabilities.SupportsPause || c.Capabilities.SupportsScope {
t.Fatalf("agents must not advertise pause/scope capabilities: %+v", c)
internal/api/contract_test.go
+12
View File
@@ -13939,6 +13939,11 @@ func TestContract_MetricsHistoryLivePointOnlyWhileSourceReports(t *testing.T) {
setUnexportedField(t, monitor, "state", state)
setUnexportedField(t, monitor, "metricsHistory", monitoring.NewMetricsHistory(10, time.Hour))
setUnexportedField(t, monitor, "metricsStore", store)
continuity := config.NewHostContinuityStore(t.TempDir(), nil)
if err := continuity.Upsert(config.HostContinuityEntry{HostID: "agent-waiting", Hostname: "waiting", LastSeen: now.Add(-time.Second), IntervalSeconds: 30}); err != nil {
t.Fatal(err)
}
setUnexportedField(t, monitor, "hostContinuityStore", continuity)
router := &Router{monitor: monitor}
// Preserve the source's own verdict through the stale pass (#2634): a
@@ -13976,6 +13981,8 @@ func TestContract_MetricsHistoryLivePointOnlyWhileSourceReports(t *testing.T) {
}
for _, tc := range []struct{ resourceType, resourceID, metric string }{
{"agent", "agent-waiting", "cpu"},
{"agent", "agent-waiting", "temperature"},
{"agent", "agent-silent", "cpu"},
{"agent", "agent-silent", "temperature"},
{"node", "pve-unreachable", "memory"},
@@ -14032,6 +14039,11 @@ func TestContract_MetricsHistoryLivePointOnlyWhileSourceReports(t *testing.T) {
t.Fatalf("%s %s: current %s not served as a live point: source=%q points=%+v", tc.resourceType, tc.resourceID, tc.metric, resp.Source, resp.Points)
}
}
// A saved enrollment may still have real history from its previous run.
monitor.GetMetricsHistory().AddGuestMetric("agent:agent-waiting", "cpu", 37, now.Add(-time.Minute))
if resp, _ := history("resourceType=agent&resourceId=agent-waiting&metric=cpu"); resp.Source == "live" || len(resp.Points) != 1 || resp.Points[0].Value != 37 {
t.Fatalf("pending enrollment lost stored history or relabelled it as live: %+v", resp)
}
}
// A row lapses only when it records a reading source and every one it records
internal/api/router.go
+4 -2
View File
@@ -6537,9 +6537,11 @@ func (r *Router) handleMetricsHistory(w http.ResponseWriter, req *http.Request)
_, lapsed := lapsedRows[kind+"\x00"+id]
return lapsed
}
// HostsSnapshot reports an agent row past its reporting lease as offline.
// Saved enrollment is not live telemetry while awaiting its first report,
// and HostsSnapshot reports an expired reporting lease as offline.
hostReporting := func(host *models.Host) bool {
return !strings.EqualFold(strings.TrimSpace(host.Status), "offline") && !rowLapsed("host", host.ID)
status := strings.ToLower(strings.TrimSpace(host.Status))
return status != "offline" && status != "pending" && !rowLapsed("host", host.ID)
}
// The poller marks a node it cannot reach, or one the cluster lists but
// this poll did not observe, offline or unknown and keeps earlier
internal/models/converters.go
+2
View File
@@ -1154,6 +1154,7 @@ type ResourceConvertInput struct {
PlatformType string
SourceType string
Sources []string
SourceStatus json.RawMessage
ParentID string
ParentName string
ChildCount int
@@ -1241,6 +1242,7 @@ func ConvertResourceToFrontend(input ResourceConvertInput) ResourceFrontend {
PlatformType: input.PlatformType,
SourceType: input.SourceType,
Sources: append([]string(nil), input.Sources...),
SourceStatus: input.SourceStatus,
ParentID: input.ParentID,
ParentName: input.ParentName,
ChildCount: input.ChildCount,
internal/models/models_frontend.go
+5 -4
View File
@@ -1038,10 +1038,11 @@ type ResourceFrontend struct {
DisplayName string `json:"displayName"`
// Platform/Source
PlatformID string `json:"platformId"`
PlatformType string `json:"platformType"`
SourceType string `json:"sourceType"`
Sources []string `json:"sources,omitempty"`
PlatformID string `json:"platformId"`
PlatformType string `json:"platformType"`
SourceType string `json:"sourceType"`
Sources []string `json:"sources,omitempty"`
SourceStatus json.RawMessage `json:"sourceStatus,omitempty"`
// Hierarchy
ParentID string `json:"parentId,omitempty"`
internal/monitoring/monitor.go
+4
View File
@@ -3901,6 +3901,9 @@ func hostFromReadStateView(view *unifiedresources.HostView) models.Host {
func hostAgentLivenessFromReadStateView(view *unifiedresources.HostView, now time.Time) (string, time.Time) {
status, lastSeen := string(view.Status()), view.LastSeen()
agentSighting, ok := view.SourceStatus(unifiedresources.SourceAgent)
if ok && agentSighting.Status == "pending" {
return "pending", agentSighting.LastSeen
}
if !ok || agentSighting.LastSeen.IsZero() {
return status, lastSeen
}
@@ -7225,6 +7228,7 @@ func monitorResourceToConvertInput(resource unifiedresources.Resource) models.Re
PlatformType: monitorPlatformType(&resource, resourceType),
SourceType: monitorSourceType(resource.Sources),
Sources: monitorSourceKeys(resource.Sources),
SourceStatus: monitorRawJSON(resource.SourceStatus),
ParentID: monitorStringValue(resource.ParentID),
ParentName: resource.ParentName,
ChildCount: resource.ChildCount,
internal/monitoring/monitor_agents.go
+4 -1
View File
@@ -1812,8 +1812,11 @@ func (m *Monitor) applyRejectedHostReportLiveness(
if host.LastSeen.IsZero() ||
receivedAt.Sub(host.LastSeen) > hostAgentHealthWindow(host.IntervalSeconds) {
host.Status = "offline"
} else {
host.Status = "pending"
}
m.state.UpsertHost(host)
// A rejected report does not hydrate live telemetry after a restart.
// Leave the saved enrollment in its bounded first-report projection.
}
// An authenticated arrival proves the transport is reachable, but a report
internal/monitoring/monitor_host_agent_removal_lifecycle_test.go
+3 -3
View File
@@ -587,14 +587,14 @@ func TestHostAgentRemovalLifecycleKeepsOfflineRowRemovableAfterRestart(t *testin
}
// Reconstruct the monitor without another agent report. The durable row
// must remain visible as offline so its removal action stays reachable.
// must remain visible while awaiting a report so removal stays reachable.
restarted := newHostRemovalLifecycleMonitor(t, dataPath)
hosts := restarted.HostsSnapshot()
if len(hosts) != 1 || hosts[0].ID != host.ID {
t.Fatalf("restart host inventory = %+v, want continuity row %q", hosts, host.ID)
}
if hosts[0].Status != "offline" {
t.Fatalf("restart continuity status = %q, want offline", hosts[0].Status)
if hosts[0].Status != "pending" {
t.Fatalf("restart continuity status = %q, want pending", hosts[0].Status)
}
for i := 0; i < 3; i++ {
internal/monitoring/monitor_host_agents_test.go
+86
View File
@@ -35,6 +35,92 @@ import (
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
)
func TestHostAgentStartupWaitsWithinAcceptedReportLease(t *testing.T) {
dir := t.TempDir()
original := issue1654Monitor()
original.hostContinuityStore = config.NewHostContinuityStore(dir, nil)
report := issue1654Report(time.Now().UTC())
report.SequenceID = agentshost.FormatReportSequenceID("startup-report", 1)
host, err := original.ApplyHostReport(report, nil)
if err != nil {
t.Fatal(err)
}
restarted := issue1654Monitor()
restarted.hostContinuityStore = config.NewHostContinuityStore(dir, nil)
if err := restarted.hostContinuityStore.Upsert(config.HostContinuityEntry{
HostID: "removed-agent", Hostname: "removed-host", LastSeen: host.LastSeen,
IntervalSeconds: 30, RemovedAt: host.LastSeen,
}); err != nil {
t.Fatal(err)
}
base := unifiedresources.NewMonitorAdapter(unifiedresources.NewRegistry(nil))
restarted.SetResourceStore(base)
if rejected, err := restarted.ApplyHostReport(report, nil); err != nil || rejected.Status != "pending" {
t.Fatalf("duplicate startup report ended the first-report wait: %+v, %v", rejected, err)
}
lease := hostAgentHealthWindow(host.IntervalSeconds)
for _, tc := range []struct {
name string
age time.Duration
want string
}{
{"restart", time.Second, "pending"},
{"deadline", lease, "pending"},
{"no-report-before-deadline", lease + time.Nanosecond, "offline"},
} {
t.Run(tc.name, func(t *testing.T) {
now := host.LastSeen.Add(tc.age)
read := restarted.readStateWithStandaloneHostContinuityAt(base, now)
views := read.Hosts()
if len(views) != 1 {
t.Fatalf("saved hosts = %d, want 1", len(views))
}
status, seen := hostAgentLivenessFromReadStateView(views[0], now)
if status != tc.want || !seen.Equal(host.LastSeen) || views[0].AgentID() != host.ID {
t.Fatalf("restored status/receipt/identity = %s %s %s", status, seen, views[0].AgentID())
}
if tc.want == "pending" {
sighting, ok := views[0].SourceStatus(unifiedresources.SourceAgent)
if !ok || sighting.Status != "pending" || views[0].Status() != unifiedresources.StatusUnknown {
t.Fatalf("saved enrollment became a current observation: %+v", sighting)
}
}
if len(restarted.GetLiveHostsSnapshot()) != 0 || len(base.Hosts()) != 0 {
t.Fatal("read overlay mutated live state")
}
resources := read.(*unifiedresources.MonitorAdapter).GetAll()
if resources[0].Metrics != nil {
t.Fatal("saved enrollment manufactured telemetry before its first report")
}
frontend := models.ConvertResourceToFrontend(monitorResourceToConvertInput(resources[0]))
var delivery map[string]unifiedresources.SourceStatus
if err := json.Unmarshal(frontend.SourceStatus, &delivery); err != nil {
t.Fatal(err)
}
if tc.want == "pending" && (delivery["agent"].Status != "pending" || frontend.Status != "unknown") {
t.Fatalf("realtime projection lost startup delivery: %+v", delivery)
}
if frontend.CPU != nil || frontend.Memory != nil || frontend.Disk != nil || frontend.LastSeen != host.LastSeen.UnixMilli() {
t.Fatal("realtime projection changed the accepted receipt or supplied metrics")
}
})
}
// A real accepted report ends the wait and supplies the live observation.
report.Timestamp = report.Timestamp.Add(time.Second)
report.SequenceID = agentshost.FormatReportSequenceID("startup-report", 2)
if _, err := restarted.ApplyHostReport(report, nil); err != nil {
t.Fatal(err)
}
views := restarted.readStateWithStandaloneHostContinuity(restarted.GetUnifiedReadStateOrSnapshot()).Hosts()
if len(views) != 1 || views[0].Status() != unifiedresources.StatusOnline {
t.Fatal("first accepted report did not bring the saved agent online")
}
if sighting, ok := views[0].SourceStatus(unifiedresources.SourceAgent); !ok || sighting.Status != "online" {
t.Fatalf("first accepted report retained pending delivery: %+v", sighting)
}
}
func TestGuestPollOwnershipRuntimeDoesNotWithdrawIndependentAgentEvidence(t *testing.T) {
c := &guestPollOwnershipClient{cpu: .1}
m := guestPollOwnershipMonitor(t, c)
internal/monitoring/monitored_system_usage.go
+21 -1
View File
@@ -66,6 +66,13 @@ func (m *Monitor) MonitoredSystemUsage() MonitoredSystemUsageSnapshot {
func (m *Monitor) readStateWithStandaloneHostContinuity(
readState unifiedresources.ReadState,
) unifiedresources.ReadState {
return m.readStateWithStandaloneHostContinuityAt(readState, time.Now().UTC())
}
func (m *Monitor) readStateWithStandaloneHostContinuityAt(
readState unifiedresources.ReadState,
now time.Time,
) unifiedresources.ReadState {
if m == nil || readState == nil {
return readState
@@ -114,7 +121,20 @@ func (m *Monitor) readStateWithStandaloneHostContinuity(
// it, but never present persisted telemetry as a current online sighting.
host := hostFromContinuityEntry(entry)
host.Status = "offline"
records = append(records, unifiedresources.HostIngestRecord(host))
waiting := hostAgentReportCurrent(entry.LastSeen, entry.IntervalSeconds, now)
if waiting {
host.Status = "pending"
}
record := unifiedresources.HostIngestRecord(host)
record.Resource.Metrics = nil
if waiting {
// The saved receipt bounds the wait, but is not a delivery in this
// monitor generation. Keep its timestamp without claiming freshness.
record.Resource.SourceStatus = map[unifiedresources.DataSource]unifiedresources.SourceStatus{
unifiedresources.SourceAgent: {Status: "pending", LastSeen: entry.LastSeen},
}
}
records = append(records, record)
}
if len(records) == 0 {
return readState
internal/unifiedresources/registry.go
+6 -2
View File
@@ -1998,7 +1998,7 @@ func (rr *ResourceRegistry) markStaleLocked(now time.Time, thresholds map[DataSo
previousStatus := resource.Status
staleFound := false
for source, status := range resource.SourceStatus {
if status.LastSeen.IsZero() {
if status.LastSeen.IsZero() || status.Status == "pending" {
continue
}
if now.Sub(status.LastSeen) > sourceStaleThreshold(source, status, thresholds) {
@@ -3102,7 +3102,11 @@ func (rr *ResourceRegistry) ingestRecord(source DataSource, sourceID string, res
}
resource.Sources = []DataSource{source}
sighting := resource.SourceStatus[source]
sighting.Status = sourceSightingStatus(resource.LastSeen)
// Only continuity can describe an enrolled agent awaiting a report in the
// current monitor generation. Its old receipt is a deadline, not a sighting.
if !(onlyMissing && source == SourceAgent && sighting.Status == "pending") {
sighting.Status = sourceSightingStatus(resource.LastSeen)
}
sighting.LastSeen = resource.LastSeen
sighting.reported = resource.Status
resource.SourceStatus = map[DataSource]SourceStatus{source: sighting}
internal/unifiedresources/registry_test.go
+32
View File
@@ -14,6 +14,38 @@ import (
"github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"
)
func TestPendingHostContinuityPreservesUnknownDeliveryAcrossSerializedReads(t *testing.T) {
seen := time.Now().Add(-time.Minute).UTC()
record := HostIngestRecord(models.Host{ID: "waiting-agent", Hostname: "waiting-host", Status: "pending", LastSeen: seen})
record.Resource.SourceStatus = map[DataSource]SourceStatus{SourceAgent: {Status: "pending", LastSeen: seen}}
rr := NewRegistry(nil)
id := rr.ingestRecord(SourceAgent, record.SourceID, record.Resource, record.Identity, true)
rr.MarkStale(time.Now(), map[DataSource]time.Duration{SourceAgent: time.Second})
encoded, err := json.Marshal(rr.List())
if err != nil {
t.Fatal(err)
}
var resources []Resource
if err := json.Unmarshal(encoded, &resources); err != nil {
t.Fatal(err)
}
copy := NewRegistry(nil)
copy.IngestResourcesWithStaleThresholds(resources, map[DataSource]time.Duration{SourceAgent: time.Second})
got, ok := copy.Get(id)
if !ok || got.Status != StatusUnknown || got.SourceStatus[SourceAgent].Status != "pending" || !got.LastSeen.Equal(seen) || got.Agent.Stale {
t.Fatalf("waiting enrollment acquired a fresh or offline verdict: %+v", got)
}
// Normal ingest must replace the continuity marker with a real delivery.
rr.IngestSnapshot(models.StateSnapshot{Hosts: []models.Host{{ID: "waiting-agent", Hostname: "waiting-host", Status: "online", LastSeen: time.Now().UTC()}}})
got, ok = rr.Get(id)
if !ok || got.Status != StatusOnline || got.SourceStatus[SourceAgent].Status != "online" {
t.Fatalf("accepted live sighting did not replace pending continuity: %+v", got)
}
if replaced := rr.ingestRecord(SourceAgent, record.SourceID, record.Resource, record.Identity, true); replaced != "" {
t.Fatal("pending continuity replaced an existing live observation")
}
}
// The snapshot entry point shares the registry's journal ownership, without
// mutating either the snapshot or the alert's recorded metadata.
func TestProxmoxDiskAlertSnapshotOwnershipMatchesRegistry(t *testing.T) {