Add application response contracts to availability checks

This commit is contained in:
Pulse Test
2026-08-30 17:35:33 +01:00
parent 34849268e5
commit c0b190bfa1
27 changed files with 2020 additions and 248 deletions
@@ -1,7 +1,7 @@
# Infrastructure-Aware Service Assurance Plan
Last updated: 2026-08-30
Status: ACCEPTED — SLICE A IMPLEMENTED
Status: ACCEPTED — SLICES A AND B IMPLEMENTED
Governance surfaces:
- `status.json.coverage_gaps.infrastructure-aware-service-assurance`
@@ -228,7 +228,7 @@ Implementation record (2026-08-30): Slice A is delivered through the
monitoring-owned categorical history store and rollups, server-authored
configuration revisions and remote receipt timeline, the bounded
`/api/availability-history` batch contract, and the URL-owned Availability
fleet presentation. Its release proofs live in the owner contract. Slices B
fleet presentation. Its release proofs live in the owner contract. Slices C
through H remain ordered future work; acceptance of this product lane does not
imply that deferred breadth is already delivered.
@@ -258,6 +258,19 @@ Exit conditions:
content;
- current reachability-only configurations migrate without semantic change.
Implementation record (2026-08-30): Slice B is delivered through one bounded,
explicit HTTP/S contract shared by local and assigned-agent execution. The
saved target owns method, accepted status range, optional POST body, headers,
Basic or bearer authentication, text assertion, and structured JSON field
equality. Write-only values are encrypted at rest, omitted from API and
evidence payloads, and reused only while the endpoint origin is unchanged;
changing scheme, host, or effective port requires explicit credential
re-entry. Current status and unified-resource facets preserve transport
reachability separately from typed application correctness while the overall
result remains the alert and history outcome. Legacy targets without an
explicit contract retain their previous HEAD-with-bounded-GET-fallback
semantics. Slices C through H remain ordered future work.
### Slice C: Discovery-Led Assurance Onboarding
Deliver:
@@ -306,6 +306,17 @@ shape so local and remote checks derive identical trust vocabulary. Certificate
identity, validity, and trust are observation payloads only. They do not renew
the target lease, agent enrollment, host heartbeat, or command readiness beyond
the accepted report's existing server-receipt evidence.
An assigned HTTP/S application-response contract travels to the owning agent
only through its authenticated remote-config assignment. The agent may receive
the encrypted-at-rest request body, Basic password, bearer token, and operator
header values because it must execute the check, but it must never copy those
values, response bytes, or arbitrary remote content into logs, module status,
the host report, telemetry, or Patrol context. The shared probe returns only
the overall outcome, transport outcome, typed application outcome, HTTP status,
bounded failure code/message, latency, and optional certificate observation.
Local and assigned-agent execution must therefore use the same method, status,
body-size, assertion, redirect, and outbound-URL limits, while the report
ingestion boundary continues to reject stale revisions and non-owning agents.
Explicit disk inclusion is an agent lifecycle input. The CLI, persisted
service arguments, unified host agent, and Docker/Podman agent must carry the
@@ -1465,6 +1465,29 @@ payload shape change when the portal presents compact client rows.
the same secret-free `certificate` observation returned by the runtime
probe, while reachability success remains independent from trust posture so
an operator can diagnose an untrusted but reachable endpoint.
HTTP/S targets may also carry one explicit `http` application-response
contract. Its bounded vocabulary is `HEAD`, `GET`, or `POST`, one accepted
status range, at most sixteen operator headers, optional Basic or bearer
authentication, an optional POST body of at most 8 KiB, and bounded text
and JSON-path assertions over at most 64 KiB of response content. An absent
`http` block preserves the legacy HEAD-with-GET-fallback behavior for saved
targets without a migration rewrite. Every execution-field edit, including
a contract or secret change, advances the server-authored configuration
revision and therefore the existing availability-history revision boundary.
Request bodies, passwords, bearer tokens, and header values are write-only.
List/create/update responses return only `httpSecrets` presence markers;
Updates and unsaved tests that identify an existing target merge omitted
write-only values from encrypted persistence before validation/execution
only while scheme, host, and effective port remain unchanged. An endpoint
origin change clears omitted body/header values and requires Basic or
bearer credentials to be entered again before the target can be saved or
tested; stored credentials must never follow an address edit silently.
Test and current-status responses expose transport reachability separately
from the typed application outcome, status code, and bounded failure code,
but never return the request body, response body, credential values, or
arbitrary remote content. Contract tests must pin redaction, unchanged-
secret preservation, bounded evaluation, and the reachable-but-incorrect
application case.
Availability target protocol vocabulary is canonicalized at the API/config
boundary: clients may submit `ping` as a user-facing alias for the ICMP
check, but saved targets, API reads, probe status, connections rows, and
@@ -761,10 +761,14 @@ cleanup so readers cannot retain orphaned runtime or alert projections.
poll-provider path. `internal/monitoring/availability_poller.go` owns ICMP,
TCP, and HTTP probes, provider health, scheduler task construction, and
supplemental unified-resource records for saved availability targets.
HTTP and HTTPS probes start with `HEAD` and retry once with `GET` only when
the endpoint explicitly reports that `HEAD` is unsupported (`405 Method
Not Allowed` or `501 Not Implemented`). Other server-error responses remain
failed probes and must not be converted into reachability evidence.
Legacy HTTP and HTTPS targets without an explicit response contract start
with `HEAD` and retry once with `GET` only when the endpoint explicitly
reports that `HEAD` is unsupported (`405 Method Not Allowed` or `501 Not
Implemented`). Explicit application contracts instead execute their bounded
`HEAD`, `GET`, or `POST` request exactly once, distinguish a received HTTP
response from an application assertion failure, and inspect no more than
64 KiB of response content. Other server-error responses remain failed
probes and must not be converted into healthy application evidence.
Failed endpoint probes are observed runtime state for that target; they
must publish provider health and incidents without dead-lettering the
scheduler task itself.
@@ -1464,6 +1468,18 @@ retention lifecycle, are deleted with the target, and retain no target address,
agent identity, raw error, certificate detail, or customer identity. The
monitoring-read batch path is bounded to 200 targets and 120 presentation
buckets without a query per target.
HTTP/S application response contracts extend that same source-owned provider
without creating a second uptime product or result store. Current status and
unified-resource availability data carry the transport outcome separately from
the typed application outcome, HTTP status, and bounded failure code; the
overall availability outcome remains the alert/history state so a reachable
endpoint returning an incorrect application response can still fail the saved
service check. Request bodies, response bodies, Basic passwords, bearer tokens,
and operator header values never enter status, categorical history, incidents,
evidence envelopes, telemetry, logs, or Patrol context. Execution-defined
contract edits use the existing server-authored configuration revision so
history visibly breaks at the new proof definition instead of blending unlike
checks.
HTTPS checks also author one canonical certificate observation from the same
probe execution. `internal/availabilityprobe` captures the presented leaf and
`pkg/tlsutil/certificate.go` derives subject, issuer, SANs, SHA-256 fingerprint,
@@ -1747,6 +1747,16 @@ recovery scope, or a storage/recovery-owned secret source.
`certificate_*` incident on those availability facets remain monitoring and
alerts context. They do not prove storage encryption, backup integrity,
repository authenticity, protection coverage, or restore readiness.
The same neutrality applies to HTTP/S application-response contracts.
Transport and application outcomes may explain liveness, but accepted
status ranges, text/JSON assertions, and typed failure codes are not backup
verification or restore evidence. Request bodies, response bodies, Basic
passwords, bearer tokens, and operator header values remain encrypted
configuration inputs and must never enter storage/recovery inventory,
history, exports, evidence, diagnostics, or recovery authority.
Omitted write-only values may be reused only while the HTTP/S endpoint
origin is unchanged; a scheme, host, or effective-port change must not
replay stored values to the new endpoint.
31. Keep infrastructure summary chart bucketing and short response caching presentation-only on the adjacent shared API boundary. When `internal/api/chartapi/service.go` normalizes mixed-cadence infrastructure history into equal-time summary buckets or serves a cached summary payload for repeated operator-facing summary-card requests, storage and recovery may consume the resulting visual context only; they must not reinterpret those normalized chart samples, cached timestamps, or cache hits as recovery freshness windows, backup cadence, or restore evidence.
Expiry sweeps, oldest-payload eviction, and byte/cardinality limits in that
shared chart cache are transport memory controls only. Storage and recovery
@@ -4653,6 +4653,15 @@ the shared resource drawer by canonical resource ID, while
already carried by that resource. History absence or failure cannot overwrite
the unified resource's current status, correlation, source ownership, or
navigation identity.
For HTTP/S checks with an explicit application response contract, the
availability facet preserves two source-owned facts: `transportOutcome`
describes whether the endpoint answered, while `applicationOutcome`,
`applicationStatusCode`, and `applicationFailureCode` describe whether the
bounded saved contract passed. The overall resource status still follows the
saved check outcome, so an answering endpoint with an incorrect application
response is not presented as healthy. Resource payloads, evidence envelopes,
incidents, relationships, and history must never contain request or response
bodies, Basic passwords, bearer tokens, or operator header values.
Shared unified-resource consumers now also normalize org scope through
`frontend-modern/src/utils/orgScope.ts` before building cache keys or
multi-tenant resource fetch state, so the canonical resource hooks do not
+22 -15
View File
@@ -1,18 +1,22 @@
{
"version": 1,
"base_sha": "712f1a3bfeff8f78de242b97fc66526d8f7c5761",
"verified_at": "2026-08-30T15:06:15Z",
"base_sha": "34849268e58652489d2a359886fb7fdd5bddd959",
"verified_at": "2026-08-30T16:33:41Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/shared/Dialog.tsx",
"frontend-modern/src/components/shared/useDialogState.ts"
"frontend-modern/src/api/availabilityTargets.ts",
"frontend-modern/src/components/Infrastructure/AvailabilityProbeStatusCard.tsx",
"frontend-modern/src/components/Settings/ConnectionEditor/CredentialSlots/AvailabilityTargetSlot.tsx",
"frontend-modern/src/types/resource.ts"
],
"content_sha256": {
"frontend-modern/src/components/shared/Dialog.tsx": "58b41841c2ac21939a9e1493754c314e77a1650d1d024de152ebbb39ba3d67a0",
"frontend-modern/src/components/shared/useDialogState.ts": "290cab00130c6de5731499a008cc432eb588c97c62e24d0b21940946d37e6e9c"
"frontend-modern/src/api/availabilityTargets.ts": "e8eba84f95617889782888152ef55fd92668071d35873e622de3a4fb0f39fee5",
"frontend-modern/src/components/Infrastructure/AvailabilityProbeStatusCard.tsx": "0f498ccc635f02f4d64848995bc3bb9a1fc4cf060812d0f66fa2a918976c25a8",
"frontend-modern/src/components/Settings/ConnectionEditor/CredentialSlots/AvailabilityTargetSlot.tsx": "8f47a84f61d5ea083743a7945803adc85c7495b67bd1d97df807ec91750a2330",
"frontend-modern/src/types/resource.ts": "a524ae7b84ad2400e2a7cf1fab213824f3a9335cdadb9f42f1d73a5a0b21978f"
},
"routes": [
"/settings/infrastructure"
"/settings/monitoring/availability?add=target"
],
"viewports": [
{
@@ -25,15 +29,18 @@
}
],
"states": [
"The Add infrastructure dialog remained visible and horizontally contained with its close control focused at desktop and narrow viewports",
"The active dialog portal stayed interactive while the app root and every other body-level surface were inert",
"A body-level surface added after opening became inert before inspection",
"Closing restored ordinary surfaces while preserving a surface that was already inert before opening"
"new HTTP availability check with the explicit application-response contract",
"GET contract with text assertion and reachable endpoint whose application assertion failed",
"GET contract with reachable endpoint and passing application assertion",
"POST contract with request body, bearer authentication, and operator header controls",
"phone-width failure and success feedback positioned above the sticky action footer"
],
"interactions": [
"opened Add infrastructure from its real settings trigger at desktop and narrow viewports",
"verified a late-added background action could not be activated while the modal was open",
"closed with Escape and verified focus returned to the Add infrastructure trigger",
"activated the late-added action after close to verify restored operability"
"changed probe type from ICMP to HTTP",
"changed request method between GET and POST",
"selected bearer authentication and added then removed an operator header",
"entered an endpoint and text assertion and ran test-before-save",
"confirmed transport-reachable application failure and full contract success messages",
"verified the contract controls and feedback at desktop and phone widths"
]
}
@@ -6,6 +6,42 @@ const AVAILABILITY_TARGETS_PATH = '/api/availability-targets';
export type AvailabilityProbeProtocol = 'icmp' | 'tcp' | 'udp' | 'http' | 'https';
export type AvailabilityUDPMode = 'response_required' | 'open_or_filtered';
export type AvailabilityTargetKind = 'machine' | 'service' | 'device';
export type AvailabilityHTTPMethod = 'HEAD' | 'GET' | 'POST';
export type AvailabilityHTTPAuthType = 'none' | 'basic' | 'bearer';
export interface AvailabilityHTTPHeader {
id: string;
name: string;
/** Write-only. Omitted values preserve the stored secret on update. */
value?: string;
}
export interface AvailabilityHTTPConfig {
method: AvailabilityHTTPMethod;
headers?: AvailabilityHTTPHeader[];
authentication: {
type: AvailabilityHTTPAuthType;
username?: string;
/** Write-only. */
password?: string;
/** Write-only. */
bearerToken?: string;
};
/** Write-only POST body. */
body?: string;
expectedStatusMin: number;
expectedStatusMax: number;
textContains?: string;
jsonPath?: string;
jsonEquals?: string;
}
export interface AvailabilityHTTPSecretState {
bodyConfigured: boolean;
passwordConfigured: boolean;
bearerTokenConfigured: boolean;
headers?: Array<{ id: string; valueConfigured: boolean }>;
}
export interface AvailabilityProbeStatus {
targetId: string;
@@ -14,6 +50,10 @@ export interface AvailabilityProbeStatus {
address: string;
protocol: AvailabilityProbeProtocol | string;
outcome?: 'reachable' | 'unreachable' | 'indeterminate' | string;
transportOutcome?: 'reachable' | 'unreachable' | 'indeterminate' | string;
applicationOutcome?: 'not_configured' | 'passed' | 'failed' | string;
applicationStatusCode?: number;
applicationFailureCode?: string;
enabled: boolean;
available: boolean;
lastChecked?: string;
@@ -56,6 +96,8 @@ export interface AvailabilityTarget {
* string because the server decodes updates onto the existing record.
*/
probeAgentId?: string;
http?: AvailabilityHTTPConfig;
httpSecrets?: AvailabilityHTTPSecretState;
status?: AvailabilityProbeStatus;
}
@@ -63,6 +105,12 @@ export interface AvailabilityTestResponse {
success: boolean;
latencyMillis: number;
outcome?: 'reachable' | 'unreachable' | 'indeterminate' | string;
transportOutcome?: 'reachable' | 'unreachable' | 'indeterminate' | string;
application?: {
outcome: 'not_configured' | 'passed' | 'failed' | string;
statusCode?: number;
failureCode?: string;
};
error?: string;
certificate?: ResourceCertificateObservation;
}
@@ -74,6 +74,30 @@ export function AvailabilityProbeStatusCard(props: AvailabilityProbeStatusCardPr
return err.length > 40 ? `${err.slice(0, 40)}` : err;
};
const certificate = () => props.availability.certificate;
const transportLabel = () => {
switch (props.availability.transportOutcome) {
case 'reachable':
return 'Endpoint answered';
case 'unreachable':
return 'No response';
case 'indeterminate':
return 'Indeterminate';
default:
return null;
}
};
const applicationLabel = () => {
if (
!props.availability.applicationOutcome ||
props.availability.applicationOutcome === 'not_configured'
) {
return null;
}
const status = props.availability.applicationStatusCode
? ` · HTTP ${props.availability.applicationStatusCode}`
: '';
return `${props.availability.applicationOutcome === 'passed' ? 'Contract passed' : 'Contract failed'}${status}`;
};
const certificateTrust = () => {
switch (certificate()?.trustStatus) {
case 'trusted':
@@ -168,6 +192,22 @@ export function AvailabilityProbeStatusCard(props: AvailabilityProbeStatusCardPr
presentation()?.freshnessLabel === 'stale' ? 'text-amber-600 dark:text-amber-300' : ''
}
/>
<Show when={transportLabel()}>
{(label) => <InfoCardKeyValueRow label="Reachability" value={label()} />}
</Show>
<Show when={applicationLabel()}>
{(label) => (
<InfoCardKeyValueRow
label="Application"
value={label()}
valueClass={
props.availability.applicationOutcome === 'passed'
? 'text-emerald-600 dark:text-emerald-400'
: 'text-red-600 dark:text-red-400'
}
/>
)}
</Show>
<Show when={presentation()?.correlationLabel}>
{(label) => (
<InfoCardKeyValueRow
@@ -13,6 +13,8 @@ import { FormSelect } from '@/components/shared/FormSelect';
import {
AvailabilityTargetsAPI,
type AvailabilityProbeProtocol,
type AvailabilityHTTPAuthType,
type AvailabilityHTTPMethod,
type AvailabilityTarget,
type AvailabilityTargetKind,
type AvailabilityTestResponse,
@@ -63,6 +65,29 @@ interface AvailabilityForm {
failureThreshold: string;
monitorCertificate: boolean;
certificateExpiryWarningDays: string;
httpMethod: AvailabilityHTTPMethod;
httpStatusMin: string;
httpStatusMax: string;
httpTextContains: string;
httpJSONPath: string;
httpJSONEquals: string;
httpAuthType: AvailabilityHTTPAuthType;
httpUsername: string;
httpPassword: string;
httpPasswordConfigured: boolean;
httpBearerToken: string;
httpBearerTokenConfigured: boolean;
httpBody: string;
httpBodyConfigured: boolean;
httpBodyTouched: boolean;
httpHeaders: AvailabilityHTTPHeaderForm[];
}
interface AvailabilityHTTPHeaderForm {
id: string;
name: string;
value: string;
valueConfigured: boolean;
}
export interface AvailabilityTargetSlotProps {
@@ -100,28 +125,70 @@ const newAvailabilityForm = (
failureThreshold: '2',
monitorCertificate: true,
certificateExpiryWarningDays: '30',
httpMethod: 'GET',
httpStatusMin: '200',
httpStatusMax: '399',
httpTextContains: '',
httpJSONPath: '',
httpJSONEquals: '',
httpAuthType: 'none',
httpUsername: '',
httpPassword: '',
httpPasswordConfigured: false,
httpBearerToken: '',
httpBearerTokenConfigured: false,
httpBody: '',
httpBodyConfigured: false,
httpBodyTouched: false,
httpHeaders: [],
});
const formFromTarget = (target: AvailabilityTarget): AvailabilityForm => ({
id: target.id,
name: target.name ?? '',
targetKind: target.targetKind ?? 'service',
address: target.address ?? '',
protocol: target.protocol ?? 'icmp',
port: target.port ? String(target.port) : '',
path: target.path ?? '',
udpMode: target.udpMode ?? 'response_required',
udpRequest: target.udpRequest ?? '',
udpExpectedResponse: target.udpExpectedResponse ?? '',
linkedResourceId: target.linkedResourceId ?? '',
probeAgentId: target.probeAgentId ?? '',
enabled: target.enabled ?? true,
pollIntervalSeconds: String(target.pollIntervalSeconds ?? 60),
timeoutMillis: String(target.timeoutMillis ?? 2000),
failureThreshold: String(target.failureThreshold ?? 2),
monitorCertificate: target.certificateMonitoringDisabled !== true,
certificateExpiryWarningDays: String(target.certificateExpiryWarningDays ?? 30),
});
const formFromTarget = (target: AvailabilityTarget): AvailabilityForm => {
const headerSecrets = new Map(
(target.httpSecrets?.headers ?? []).map((header) => [header.id, header.valueConfigured]),
);
return {
id: target.id,
name: target.name ?? '',
targetKind: target.targetKind ?? 'service',
address: target.address ?? '',
protocol: target.protocol ?? 'icmp',
port: target.port ? String(target.port) : '',
path: target.path ?? '',
udpMode: target.udpMode ?? 'response_required',
udpRequest: target.udpRequest ?? '',
udpExpectedResponse: target.udpExpectedResponse ?? '',
linkedResourceId: target.linkedResourceId ?? '',
probeAgentId: target.probeAgentId ?? '',
enabled: target.enabled ?? true,
pollIntervalSeconds: String(target.pollIntervalSeconds ?? 60),
timeoutMillis: String(target.timeoutMillis ?? 2000),
failureThreshold: String(target.failureThreshold ?? 2),
monitorCertificate: target.certificateMonitoringDisabled !== true,
certificateExpiryWarningDays: String(target.certificateExpiryWarningDays ?? 30),
httpMethod: target.http?.method ?? 'GET',
httpStatusMin: String(target.http?.expectedStatusMin ?? 200),
httpStatusMax: String(target.http?.expectedStatusMax ?? 399),
httpTextContains: target.http?.textContains ?? '',
httpJSONPath: target.http?.jsonPath ?? '',
httpJSONEquals: target.http?.jsonEquals ?? '',
httpAuthType: target.http?.authentication.type ?? 'none',
httpUsername: target.http?.authentication.username ?? '',
httpPassword: '',
httpPasswordConfigured: target.httpSecrets?.passwordConfigured ?? false,
httpBearerToken: '',
httpBearerTokenConfigured: target.httpSecrets?.bearerTokenConfigured ?? false,
httpBody: '',
httpBodyConfigured: target.httpSecrets?.bodyConfigured ?? false,
httpBodyTouched: false,
httpHeaders: (target.http?.headers ?? []).map((header) => ({
id: header.id,
name: header.name,
value: '',
valueConfigured: headerSecrets.get(header.id) ?? false,
})),
};
};
const parsePositiveInt = (value: string): number | undefined => {
const parsed = Number.parseInt(value.trim(), 10);
@@ -129,7 +196,11 @@ const parsePositiveInt = (value: string): number | undefined => {
};
const availabilityTestSuccessDescription = (result: AvailabilityTestResponse): string => {
const base = `Probe reached the target in ${result.latencyMillis} ms.`;
const application = result.application;
const base =
application?.outcome === 'passed'
? `Endpoint answered in ${result.latencyMillis} ms and the application contract passed${application.statusCode ? ` (HTTP ${application.statusCode})` : ''}.`
: `Probe reached the target in ${result.latencyMillis} ms.`;
const certificate = result.certificate;
if (!certificate) return base;
const trust = certificate.trustStatus.replace('-', ' ');
@@ -144,8 +215,23 @@ const availabilityTestSuccessDescription = (result: AvailabilityTestResponse): s
})}.`;
};
const availabilityTestFailureDescription = (result: AvailabilityTestResponse): string => {
if (result.transportOutcome === 'reachable' && result.application?.outcome === 'failed') {
const status = result.application.statusCode ? ` HTTP ${result.application.statusCode}.` : '';
return `Endpoint answered in ${result.latencyMillis} ms, but the application contract failed.${status} ${result.error || 'The response did not satisfy the configured assertion.'}`;
}
return result.error || 'Probe failed.';
};
const payloadFromForm = (form: AvailabilityForm): AvailabilityTarget => {
const port = parsePositiveInt(form.port);
const isHTTP = form.protocol === 'http' || form.protocol === 'https';
const httpPassword = form.httpPassword ? form.httpPassword : undefined;
const httpBearerToken = form.httpBearerToken ? form.httpBearerToken : undefined;
const httpBody =
form.httpMethod === 'POST' && (form.httpBodyTouched || form.httpBody)
? form.httpBody
: undefined;
return {
id: form.id,
name: form.name.trim(),
@@ -169,6 +255,30 @@ const payloadFromForm = (form: AvailabilityForm): AvailabilityTarget => {
certificateMonitoringDisabled: form.protocol === 'https' ? !form.monitorCertificate : false,
certificateExpiryWarningDays:
form.protocol === 'https' ? parsePositiveInt(form.certificateExpiryWarningDays) : 0,
http: isHTTP
? {
method: form.httpMethod,
headers: form.httpHeaders.map((header) => ({
id: header.id,
name: header.name.trim(),
value: header.value || undefined,
})),
authentication: {
type: form.httpAuthType,
username: form.httpAuthType === 'basic' ? form.httpUsername.trim() : undefined,
password: form.httpAuthType === 'basic' ? httpPassword : undefined,
bearerToken: form.httpAuthType === 'bearer' ? httpBearerToken : undefined,
},
body: httpBody,
expectedStatusMin: parsePositiveInt(form.httpStatusMin) ?? 200,
expectedStatusMax: parsePositiveInt(form.httpStatusMax) ?? 399,
textContains:
form.httpMethod !== 'HEAD' ? form.httpTextContains.trim() || undefined : undefined,
jsonPath: form.httpMethod !== 'HEAD' ? form.httpJSONPath.trim() || undefined : undefined,
jsonEquals:
form.httpMethod !== 'HEAD' ? form.httpJSONEquals.trim() || undefined : undefined,
}
: undefined,
};
};
@@ -228,6 +338,12 @@ export const AvailabilityTargetSlot: Component<AvailabilityTargetSlotProps> = (p
const [testing, setTesting] = createSignal(false);
const [error, setError] = createSignal<string | null>(null);
const [testResult, setTestResult] = createSignal<AvailabilityTestResponse | null>(null);
let feedbackElement: HTMLDivElement | undefined;
const revealFeedback = () => {
if (typeof feedbackElement?.scrollIntoView === 'function') {
feedbackElement.scrollIntoView({ block: 'nearest' });
}
};
// Set when the server rejects a probe assignment with the canonical 402, so a
// stale cached capability set still lands on the upgrade gate.
const [probeLicenseRejected, setProbeLicenseRejected] = createSignal(false);
@@ -262,6 +378,29 @@ export const AvailabilityTargetSlot: Component<AvailabilityTargetSlotProps> = (p
setTestResult(null);
};
const updateHTTPHeader = (index: number, patch: Partial<AvailabilityHTTPHeaderForm>) => {
updateForm({
httpHeaders: form().httpHeaders.map((header, headerIndex) =>
headerIndex === index ? { ...header, ...patch } : header,
),
});
};
const addHTTPHeader = () => {
const id =
globalThis.crypto?.randomUUID?.() ??
`http-header-${Date.now()}-${form().httpHeaders.length + 1}`;
updateForm({
httpHeaders: [...form().httpHeaders, { id, name: '', value: '', valueConfigured: false }],
});
};
const removeHTTPHeader = (index: number) => {
updateForm({
httpHeaders: form().httpHeaders.filter((_, headerIndex) => headerIndex !== index),
});
};
const selectedPresetConfig = () => availabilityPresetById(selectedPreset());
const addressPlaceholder = () =>
@@ -328,8 +467,10 @@ export const AvailabilityTargetSlot: Component<AvailabilityTargetSlotProps> = (p
try {
const result = await AvailabilityTargetsAPI.test(payloadFromForm(form()));
setTestResult(result);
queueMicrotask(revealFeedback);
} catch (err) {
setError(err instanceof Error ? err.message : 'Availability test failed.');
queueMicrotask(revealFeedback);
} finally {
setTesting(false);
}
@@ -435,7 +576,7 @@ export const AvailabilityTargetSlot: Component<AvailabilityTargetSlotProps> = (p
? 'Use a hostname or IP address and the port to open.'
: form().protocol === 'udp'
? 'Use a hostname or unicast IP. UDP checks send one small datagram per poll.'
: 'Use a full URL or a hostname. HTTP statuses below 500 count as reachable.'}
: 'Use a full URL or hostname, then define the accepted application response below.'}
</span>
</label>
<FormSelect
@@ -525,6 +666,257 @@ export const AvailabilityTargetSlot: Component<AvailabilityTargetSlotProps> = (p
/>
</label>
</Show>
<Show when={form().protocol === 'http' || form().protocol === 'https'}>
<section class="space-y-4 rounded-lg border border-border bg-surface-alt/40 p-4 sm:col-span-2">
<div>
<h3 class="text-sm font-semibold text-base-content">
What proves this service is working?
</h3>
<p class="mt-1 text-xs text-muted">
Pulse records whether the endpoint answered separately from whether this response
contract passed. Response bodies and credentials are never stored as evidence.
</p>
</div>
<div class="grid gap-4 sm:grid-cols-3">
<FormSelect
label="Request method"
value={form().httpMethod}
onChange={(event) =>
updateForm({ httpMethod: event.currentTarget.value as AvailabilityHTTPMethod })
}
>
<option value="HEAD">HEAD</option>
<option value="GET">GET</option>
<option value="POST">POST</option>
</FormSelect>
<label class={formField}>
<span class={formLabel}>Accepted status from</span>
<input
class={formControl}
inputMode="numeric"
value={form().httpStatusMin}
onInput={(event) => updateForm({ httpStatusMin: event.currentTarget.value })}
placeholder="200"
/>
</label>
<label class={formField}>
<span class={formLabel}>Accepted status to</span>
<input
class={formControl}
inputMode="numeric"
value={form().httpStatusMax}
onInput={(event) => updateForm({ httpStatusMax: event.currentTarget.value })}
placeholder="399"
/>
</label>
</div>
<Show when={form().httpMethod === 'POST'}>
<label class={formField}>
<span class={formLabel}>Request body (optional)</span>
<textarea
class={`${formControl} min-h-24 font-mono text-xs`}
value={form().httpBody}
onInput={(event) =>
updateForm({
httpBody: event.currentTarget.value,
httpBodyConfigured: false,
httpBodyTouched: true,
})
}
placeholder={
form().httpBodyConfigured
? 'Stored securely — leave blank to keep it'
: '{"operation":"health"}'
}
/>
<div class="flex flex-wrap items-center justify-between gap-2">
<span class={formHelpText}>
Up to 8 KiB. Kept out of history and test output.
</span>
<Show when={form().httpBodyConfigured && !form().httpBodyTouched}>
<button
type="button"
class="text-xs font-medium text-error hover:underline"
onClick={() =>
updateForm({
httpBody: '',
httpBodyConfigured: false,
httpBodyTouched: true,
})
}
>
Remove stored body
</button>
</Show>
</div>
</label>
</Show>
<div class="grid gap-4 sm:grid-cols-2">
<FormSelect
label="Authentication"
value={form().httpAuthType}
onChange={(event) =>
updateForm({
httpAuthType: event.currentTarget.value as AvailabilityHTTPAuthType,
})
}
>
<option value="none">None</option>
<option value="basic">Basic authentication</option>
<option value="bearer">Bearer token</option>
</FormSelect>
<Show when={form().httpAuthType === 'basic'}>
<label class={formField}>
<span class={formLabel}>Username</span>
<input
class={formControl}
value={form().httpUsername}
autocomplete="username"
onInput={(event) => updateForm({ httpUsername: event.currentTarget.value })}
/>
</label>
<label class={formField}>
<span class={formLabel}>Password</span>
<input
class={formControl}
type="password"
autocomplete="new-password"
value={form().httpPassword}
onInput={(event) =>
updateForm({
httpPassword: event.currentTarget.value,
httpPasswordConfigured: false,
})
}
placeholder={
form().httpPasswordConfigured
? 'Stored securely — leave blank to keep it'
: 'Required'
}
/>
</label>
</Show>
<Show when={form().httpAuthType === 'bearer'}>
<label class={formField}>
<span class={formLabel}>Bearer token</span>
<input
class={formControl}
type="password"
autocomplete="new-password"
value={form().httpBearerToken}
onInput={(event) =>
updateForm({
httpBearerToken: event.currentTarget.value,
httpBearerTokenConfigured: false,
})
}
placeholder={
form().httpBearerTokenConfigured
? 'Stored securely — leave blank to keep it'
: 'Required'
}
/>
</label>
</Show>
</div>
<div class="space-y-3">
<div class="flex items-center justify-between gap-3">
<div>
<div class={formLabel}>Request headers</div>
<div class={formHelpText}>
Optional operator-defined headers. Values are write-only.
</div>
</div>
<Button variant="outline" size="settingsAction" onClick={addHTTPHeader}>
Add header
</Button>
</div>
<For each={form().httpHeaders}>
{(header, index) => (
<div class="grid gap-2 sm:grid-cols-[1fr_1fr_auto] sm:items-end">
<label class={formField}>
<span class={formLabel}>Header name</span>
<input
class={formControl}
value={header.name}
onInput={(event) =>
updateHTTPHeader(index(), { name: event.currentTarget.value })
}
placeholder="X-Health-Check"
/>
</label>
<label class={formField}>
<span class={formLabel}>Header value</span>
<input
class={formControl}
type="password"
value={header.value}
onInput={(event) =>
updateHTTPHeader(index(), {
value: event.currentTarget.value,
valueConfigured: false,
})
}
placeholder={
header.valueConfigured
? 'Stored securely — leave blank to keep it'
: 'Value'
}
/>
</label>
<Button
variant="dangerOutline"
size="settingsAction"
onClick={() => removeHTTPHeader(index())}
>
Remove
</Button>
</div>
)}
</For>
</div>
<Show when={form().httpMethod !== 'HEAD'}>
<div class="grid gap-4 sm:grid-cols-2">
<label class={formField}>
<span class={formLabel}>Response contains text (optional)</span>
<input
class={formControl}
value={form().httpTextContains}
onInput={(event) => updateForm({ httpTextContains: event.currentTarget.value })}
placeholder="healthy"
/>
</label>
<div />
<label class={formField}>
<span class={formLabel}>JSON path (optional)</span>
<input
class={formControl}
value={form().httpJSONPath}
onInput={(event) => updateForm({ httpJSONPath: event.currentTarget.value })}
placeholder="data.status"
/>
<span class={formHelpText}>
Dot fields and bounded array indexes are supported.
</span>
</label>
<label class={formField}>
<span class={formLabel}>JSON value equals (optional)</span>
<input
class={formControl}
value={form().httpJSONEquals}
onInput={(event) => updateForm({ httpJSONEquals: event.currentTarget.value })}
placeholder="ok"
/>
</label>
</div>
</Show>
</section>
</Show>
<Show when={form().protocol === 'https'}>
<div class="flex items-center rounded-md border border-border bg-surface-alt px-4 py-3">
<label class="flex items-center gap-3">
@@ -638,59 +1030,61 @@ export const AvailabilityTargetSlot: Component<AvailabilityTargetSlotProps> = (p
</div>
</div>
<Show when={testResult()}>
{(result) => (
<CalloutCard
role={result().success ? 'status' : 'alert'}
tone={
result().outcome === 'indeterminate'
? 'warning'
: result().success
? 'success'
: 'danger'
}
scale="compact"
padding="sm"
description={
result().outcome === 'indeterminate'
? `No UDP rejection was received in ${result().latencyMillis} ms. The port is open or filtered, not proven reachable.`
: result().success
? availabilityTestSuccessDescription(result())
: result().error || 'Probe failed.'
}
/>
)}
</Show>
<div ref={feedbackElement} class="scroll-mb-56 space-y-2 sm:scroll-mb-24">
<Show when={testResult()}>
{(result) => (
<CalloutCard
role={result().success ? 'status' : 'alert'}
tone={
result().outcome === 'indeterminate'
? 'warning'
: result().success
? 'success'
: 'danger'
}
scale="compact"
padding="sm"
description={
result().outcome === 'indeterminate'
? `No UDP rejection was received in ${result().latencyMillis} ms. The port is open or filtered, not proven reachable.`
: result().success
? availabilityTestSuccessDescription(result())
: availabilityTestFailureDescription(result())
}
/>
)}
</Show>
<Show when={error()}>
{(message) => (
<CalloutCard
role="alert"
tone="danger"
scale="compact"
padding="sm"
description={message()}
/>
)}
</Show>
<Show when={error()}>
{(message) => (
<CalloutCard
role="alert"
tone="danger"
scale="compact"
padding="sm"
description={message()}
/>
)}
</Show>
<Show when={props.deleteError}>
{(message) => (
<CalloutCard
role="alert"
tone="danger"
scale="compact"
padding="sm"
description={message()}
/>
)}
</Show>
<Show when={props.deleteError}>
{(message) => (
<CalloutCard
role="alert"
tone="danger"
scale="compact"
padding="sm"
description={message()}
/>
)}
</Show>
<Show when={props.deleteConfirming}>
<div class="rounded-md border border-border bg-surface-alt px-4 py-3 text-xs text-muted">
Click remove again to confirm. Historical resource data and alerts remain available.
</div>
</Show>
<Show when={props.deleteConfirming}>
<div class="rounded-md border border-border bg-surface-alt px-4 py-3 text-xs text-muted">
Click remove again to confirm. Historical resource data and alerts remain available.
</div>
</Show>
</div>
<div class="sticky bottom-0 -mx-4 mt-auto border-t border-border bg-surface px-4 py-3 shadow-[0_-8px_16px_rgba(15,23,42,0.04)]">
<div class="flex flex-col-reverse gap-2 sm:flex-row sm:items-center sm:justify-between">
@@ -63,6 +63,7 @@ const agentHostResource = (agentId: string, displayName: string) => ({
const mockedCreate = vi.mocked(AvailabilityTargetsAPI.create);
const mockedList = vi.mocked(AvailabilityTargetsAPI.list);
const mockedUpdate = vi.mocked(AvailabilityTargetsAPI.update);
const mockedTest = vi.mocked(AvailabilityTargetsAPI.test);
describe('AvailabilityTargetSlot', () => {
beforeEach(() => {
@@ -244,6 +245,180 @@ describe('AvailabilityTargetSlot', () => {
);
});
it('creates an HTTP application response contract from the proof question', async () => {
render(() => <AvailabilityTargetSlot onCancel={vi.fn()} onSaved={vi.fn()} />);
fireEvent.change(screen.getByLabelText('Probe'), { target: { value: 'https' } });
expect(
screen.getByRole('heading', { name: 'What proves this service is working?' }),
).toBeInTheDocument();
fireEvent.input(screen.getByLabelText('Name'), { target: { value: 'Orders API' } });
fireEvent.input(screen.getByLabelText(/^URL or host/), {
target: { value: 'https://orders.example.test/health' },
});
fireEvent.change(screen.getByLabelText('Request method'), { target: { value: 'POST' } });
fireEvent.input(screen.getByLabelText('Accepted status from'), { target: { value: '200' } });
fireEvent.input(screen.getByLabelText('Accepted status to'), { target: { value: '204' } });
fireEvent.input(screen.getByLabelText(/^Request body \(optional\)/), {
target: { value: '{"operation":"health"}' },
});
fireEvent.change(screen.getByLabelText('Authentication'), { target: { value: 'bearer' } });
fireEvent.input(screen.getByLabelText('Bearer token'), { target: { value: 'token-value' } });
fireEvent.click(screen.getByRole('button', { name: 'Add header' }));
fireEvent.input(screen.getByLabelText('Header name'), { target: { value: 'X-Tenant' } });
fireEvent.input(screen.getByLabelText('Header value'), { target: { value: 'tenant-a' } });
fireEvent.input(screen.getByPlaceholderText('data.status'), {
target: { value: 'data.status' },
});
fireEvent.input(screen.getByPlaceholderText('ok'), {
target: { value: 'healthy' },
});
fireEvent.click(screen.getByRole('button', { name: 'Add service/device check' }));
await waitFor(() => expect(mockedCreate).toHaveBeenCalled());
expect(mockedCreate.mock.calls.at(-1)?.[0]).toEqual(
expect.objectContaining({
protocol: 'https',
http: expect.objectContaining({
method: 'POST',
body: '{"operation":"health"}',
expectedStatusMin: 200,
expectedStatusMax: 204,
authentication: { type: 'bearer', bearerToken: 'token-value' },
headers: [expect.objectContaining({ name: 'X-Tenant', value: 'tenant-a' })],
jsonPath: 'data.status',
jsonEquals: 'healthy',
}),
}),
);
});
it('preserves write-only HTTP values when editing without re-entering them', async () => {
mockedList.mockResolvedValue([
{
id: 'target-1',
name: 'Orders API',
address: 'https://orders.example.test/health',
protocol: 'https',
enabled: true,
http: {
method: 'POST',
headers: [{ id: 'tenant-header', name: 'X-Tenant' }],
authentication: { type: 'basic', username: 'pulse' },
expectedStatusMin: 200,
expectedStatusMax: 299,
jsonPath: 'status',
jsonEquals: 'healthy',
},
httpSecrets: {
bodyConfigured: true,
passwordConfigured: true,
bearerTokenConfigured: false,
headers: [{ id: 'tenant-header', valueConfigured: true }],
},
},
]);
mockedUpdate.mockResolvedValue({
id: 'target-1',
name: 'Orders API',
address: 'https://orders.example.test/health',
protocol: 'https',
enabled: true,
});
render(() => (
<AvailabilityTargetSlot editingTargetId="target-1" onCancel={vi.fn()} onSaved={vi.fn()} />
));
await waitFor(() => expect(screen.getByLabelText('Request method')).toHaveValue('POST'));
expect(screen.getByLabelText('Password')).toHaveAttribute(
'placeholder',
'Stored securely — leave blank to keep it',
);
expect(screen.getByLabelText(/^Request body \(optional\)/)).toHaveAttribute(
'placeholder',
'Stored securely — leave blank to keep it',
);
expect(screen.getByLabelText('Header value')).toHaveAttribute(
'placeholder',
'Stored securely — leave blank to keep it',
);
fireEvent.click(screen.getByRole('button', { name: 'Save target' }));
await waitFor(() => expect(mockedUpdate).toHaveBeenCalled());
const [, payload] = mockedUpdate.mock.calls.at(-1)!;
expect(payload.http?.body).toBeUndefined();
expect(payload.http?.authentication.password).toBeUndefined();
expect(payload.http?.headers).toEqual([
{ id: 'tenant-header', name: 'X-Tenant', value: undefined },
]);
});
it('can explicitly remove a stored POST body without changing the request method', async () => {
mockedList.mockResolvedValue([
{
id: 'target-1',
name: 'Orders API',
address: 'https://orders.example.test/health',
protocol: 'https',
enabled: true,
http: {
method: 'POST',
headers: [],
authentication: { type: 'none' },
expectedStatusMin: 200,
expectedStatusMax: 299,
},
httpSecrets: {
bodyConfigured: true,
passwordConfigured: false,
bearerTokenConfigured: false,
headers: [],
},
},
]);
mockedUpdate.mockResolvedValue({
id: 'target-1',
name: 'Orders API',
address: 'https://orders.example.test/health',
protocol: 'https',
enabled: true,
});
render(() => (
<AvailabilityTargetSlot editingTargetId="target-1" onCancel={vi.fn()} onSaved={vi.fn()} />
));
await waitFor(() => expect(screen.getByLabelText('Request method')).toHaveValue('POST'));
fireEvent.click(screen.getByRole('button', { name: 'Remove stored body' }));
fireEvent.click(screen.getByRole('button', { name: 'Save target' }));
await waitFor(() => expect(mockedUpdate).toHaveBeenCalled());
const [, payload] = mockedUpdate.mock.calls.at(-1)!;
expect(payload.http?.method).toBe('POST');
expect(payload.http?.body).toBe('');
});
it('explains a reachable endpoint with a failing application contract', async () => {
mockedTest.mockResolvedValue({
success: false,
latencyMillis: 18,
outcome: 'unreachable',
transportOutcome: 'reachable',
application: { outcome: 'failed', statusCode: 503, failureCode: 'status_mismatch' },
error: 'http response status 503 was outside the expected 200-299 range',
});
render(() => <AvailabilityTargetSlot onCancel={vi.fn()} onSaved={vi.fn()} />);
fireEvent.change(screen.getByLabelText('Probe'), { target: { value: 'http' } });
fireEvent.input(screen.getByLabelText(/^URL or host/), {
target: { value: 'http://orders.example.test/health' },
});
fireEvent.click(screen.getByRole('button', { name: 'Test probe' }));
expect(
await screen.findByText(/Endpoint answered in 18 ms, but the application contract failed/),
).toBeInTheDocument();
expect(screen.getByText(/HTTP 503/)).toBeInTheDocument();
});
describe('external probe assignment', () => {
it('offers connected agent hosts and saves the assignment when licensed', async () => {
resourceMocks.resources = [agentHostResource('host-edge-01', 'Edge 01')];
@@ -99,4 +99,27 @@ describe('AvailabilityProbeStatusCard', () => {
expect(screen.getByTitle(/0123456789abcdef/)).toHaveTextContent('0123456789abcdef…');
expect(screen.getByTitle('Warning window: 30 days')).toHaveTextContent('2027');
});
it('separates endpoint reachability from application correctness', () => {
render(() => (
<AvailabilityProbeStatusCard
availability={{
targetId: 'probe-orders',
address: 'orders.example.test',
protocol: 'https',
enabled: true,
available: false,
lastChecked: new Date().toISOString(),
transportOutcome: 'reachable',
applicationOutcome: 'failed',
applicationStatusCode: 503,
applicationFailureCode: 'status_mismatch',
lastError: 'http response status 503 was outside the expected 200-299 range',
}}
/>
));
expect(screen.getByText('Endpoint answered')).toBeInTheDocument();
expect(screen.getByText('Contract failed · HTTP 503')).toBeInTheDocument();
});
});
+4
View File
@@ -1506,6 +1506,10 @@ export interface ResourceAvailabilityMeta {
address?: string;
protocol?: string;
probeOutcome?: string;
transportOutcome?: string;
applicationOutcome?: 'not_configured' | 'passed' | 'failed' | string;
applicationStatusCode?: number;
applicationFailureCode?: string;
udpMode?: string;
port?: number;
path?: string;
+167 -15
View File
@@ -7,6 +7,7 @@ import (
"strings"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/availabilityprobe"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/mock"
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
@@ -23,15 +24,30 @@ type AvailabilityHandlers struct {
type availabilityTargetResponse struct {
config.AvailabilityTarget
Status *monitoring.AvailabilityProbeStatus `json:"status,omitempty"`
Status *monitoring.AvailabilityProbeStatus `json:"status,omitempty"`
HTTPSecrets *availabilityHTTPSecretState `json:"httpSecrets,omitempty"`
}
type availabilityHTTPSecretState struct {
BodyConfigured bool `json:"bodyConfigured"`
PasswordConfigured bool `json:"passwordConfigured"`
BearerTokenConfigured bool `json:"bearerTokenConfigured"`
Headers []availabilityHTTPHeaderSecretState `json:"headers,omitempty"`
}
type availabilityHTTPHeaderSecretState struct {
ID string `json:"id"`
ValueConfigured bool `json:"valueConfigured"`
}
type availabilityTestResponse struct {
Success bool `json:"success"`
LatencyMillis int64 `json:"latencyMillis"`
Outcome string `json:"outcome,omitempty"`
Error string `json:"error,omitempty"`
Certificate *tlsutil.CertificateObservation `json:"certificate,omitempty"`
Success bool `json:"success"`
LatencyMillis int64 `json:"latencyMillis"`
Outcome string `json:"outcome,omitempty"`
TransportOutcome string `json:"transportOutcome,omitempty"`
Application *availabilityprobe.ApplicationResult `json:"application,omitempty"`
Error string `json:"error,omitempty"`
Certificate *tlsutil.CertificateObservation `json:"certificate,omitempty"`
}
func NewAvailabilityHandlers(
@@ -125,7 +141,7 @@ func (h *AvailabilityHandlers) HandleList(w http.ResponseWriter, r *http.Request
}
responses := make([]availabilityTargetResponse, 0, len(targets))
for _, target := range targets {
response := availabilityTargetResponse{AvailabilityTarget: config.NormalizeAvailabilityTarget(target)}
response := availabilityTargetAPIResponse(target)
if status, ok := statuses[target.ID]; ok {
statusCopy := status
response.Status = &statusCopy
@@ -175,7 +191,7 @@ func (h *AvailabilityHandlers) HandleAdd(w http.ResponseWriter, r *http.Request)
return
}
h.refreshMonitor(r.Context())
writeJSON(w, http.StatusCreated, target)
writeJSON(w, http.StatusCreated, availabilityTargetAPIResponse(target))
}
func (h *AvailabilityHandlers) HandleUpdate(w http.ResponseWriter, r *http.Request) {
@@ -211,11 +227,12 @@ func (h *AvailabilityHandlers) HandleUpdate(w http.ResponseWriter, r *http.Reque
}
previous := config.NormalizeAvailabilityTarget(targets[index])
target, ok := decodeAvailabilityTargetRequest(w, r, previous)
target, ok := decodeAvailabilityTargetRequest(w, r, availabilityTargetWithoutHTTPSecrets(previous))
if !ok {
return
}
target.ID = targetID
target = mergeAvailabilityHTTPSecrets(previous, target)
target = config.NormalizeAvailabilityTarget(target)
target.ConfigRevision = previous.ConfigRevision
if config.AvailabilityExecutionConfigChanged(previous, target) {
@@ -234,7 +251,7 @@ func (h *AvailabilityHandlers) HandleUpdate(w http.ResponseWriter, r *http.Reque
return
}
h.refreshMonitor(r.Context())
writeJSON(w, http.StatusOK, target)
writeJSON(w, http.StatusOK, availabilityTargetAPIResponse(target))
}
func (h *AvailabilityHandlers) HandleDelete(w http.ResponseWriter, r *http.Request) {
@@ -287,6 +304,20 @@ func (h *AvailabilityHandlers) HandleTestConnection(w http.ResponseWriter, r *ht
if !ok {
return
}
if strings.TrimSpace(target.ID) != "" {
if h != nil && h.getPersistence != nil {
if persistence := h.getPersistence(r.Context()); persistence != nil {
if targets, err := persistence.LoadAvailabilityTargets(); err == nil {
for _, saved := range targets {
if strings.TrimSpace(saved.ID) == strings.TrimSpace(target.ID) {
target = mergeAvailabilityHTTPSecrets(config.NormalizeAvailabilityTarget(saved), target)
break
}
}
}
}
}
}
h.testTarget(w, r, target)
}
@@ -336,10 +367,12 @@ func (h *AvailabilityHandlers) testTarget(w http.ResponseWriter, r *http.Request
latencyMs = 1
}
response := availabilityTestResponse{
Success: err == nil,
LatencyMillis: latencyMs,
Outcome: string(result.Outcome),
Certificate: result.Certificate.Clone(),
Success: err == nil,
LatencyMillis: latencyMs,
Outcome: string(result.Outcome),
TransportOutcome: string(result.TransportOutcome),
Application: result.Application,
Certificate: result.Certificate.Clone(),
}
if err != nil {
response.Error = err.Error()
@@ -350,7 +383,7 @@ func (h *AvailabilityHandlers) testTarget(w http.ResponseWriter, r *http.Request
func decodeAvailabilityTargetRequest(w http.ResponseWriter, r *http.Request, base config.AvailabilityTarget) (config.AvailabilityTarget, bool) {
r.Body = http.MaxBytesReader(w, r.Body, 16*1024)
defer r.Body.Close()
target := base
target := cloneAvailabilityTarget(base)
if err := json.NewDecoder(r.Body).Decode(&target); err != nil {
writeErrorResponse(w, http.StatusBadRequest, "invalid_request", "Invalid JSON body", nil)
return config.AvailabilityTarget{}, false
@@ -358,6 +391,125 @@ func decodeAvailabilityTargetRequest(w http.ResponseWriter, r *http.Request, bas
return target, true
}
func availabilityTargetAPIResponse(target config.AvailabilityTarget) availabilityTargetResponse {
target = config.NormalizeAvailabilityTarget(target)
response := availabilityTargetResponse{AvailabilityTarget: cloneAvailabilityTarget(target)}
if target.HTTP == nil {
return response
}
state := &availabilityHTTPSecretState{
BodyConfigured: target.HTTP.Body != nil && *target.HTTP.Body != "",
PasswordConfigured: target.HTTP.Authentication.Password != nil && *target.HTTP.Authentication.Password != "",
BearerTokenConfigured: target.HTTP.Authentication.BearerToken != nil && *target.HTTP.Authentication.BearerToken != "",
}
response.HTTP.Body = nil
response.HTTP.Authentication.Password = nil
response.HTTP.Authentication.BearerToken = nil
for i := range response.HTTP.Headers {
configured := target.HTTP.Headers[i].Value != nil && *target.HTTP.Headers[i].Value != ""
state.Headers = append(state.Headers, availabilityHTTPHeaderSecretState{
ID: target.HTTP.Headers[i].ID, ValueConfigured: configured,
})
response.HTTP.Headers[i].Value = nil
}
response.HTTPSecrets = state
return response
}
func availabilityTargetWithoutHTTPSecrets(target config.AvailabilityTarget) config.AvailabilityTarget {
target = cloneAvailabilityTarget(target)
if target.HTTP == nil {
return target
}
target.HTTP.Body = nil
target.HTTP.Authentication.Password = nil
target.HTTP.Authentication.BearerToken = nil
for i := range target.HTTP.Headers {
target.HTTP.Headers[i].Value = nil
}
return target
}
func mergeAvailabilityHTTPSecrets(previous, next config.AvailabilityTarget) config.AvailabilityTarget {
previous = config.NormalizeAvailabilityTarget(previous)
next = cloneAvailabilityTarget(next)
if previous.HTTP == nil || next.HTTP == nil {
return next
}
// Write-only values may be reused while editing the same endpoint, but must
// never follow a changed origin. Otherwise an address edit or unsaved test
// could silently replay a stored credential to another server.
if !sameAvailabilityHTTPOrigin(previous, next) {
return next
}
if next.HTTP.Body == nil {
next.HTTP.Body = cloneStringPointer(previous.HTTP.Body)
}
if next.HTTP.Authentication.Password == nil && next.HTTP.Authentication.Type == previous.HTTP.Authentication.Type {
next.HTTP.Authentication.Password = cloneStringPointer(previous.HTTP.Authentication.Password)
}
if next.HTTP.Authentication.BearerToken == nil && next.HTTP.Authentication.Type == previous.HTTP.Authentication.Type {
next.HTTP.Authentication.BearerToken = cloneStringPointer(previous.HTTP.Authentication.BearerToken)
}
previousHeaders := make(map[string]*string, len(previous.HTTP.Headers))
for _, header := range previous.HTTP.Headers {
previousHeaders[header.ID] = header.Value
}
for i := range next.HTTP.Headers {
if next.HTTP.Headers[i].Value == nil {
next.HTTP.Headers[i].Value = cloneStringPointer(previousHeaders[next.HTTP.Headers[i].ID])
}
}
return next
}
func sameAvailabilityHTTPOrigin(previous, next config.AvailabilityTarget) bool {
previousURL, err := previous.HTTPURL()
if err != nil {
return false
}
nextURL, err := next.HTTPURL()
if err != nil {
return false
}
previousPort := previousURL.Port()
if previousPort == "" {
previousPort = map[string]string{"http": "80", "https": "443"}[strings.ToLower(previousURL.Scheme)]
}
nextPort := nextURL.Port()
if nextPort == "" {
nextPort = map[string]string{"http": "80", "https": "443"}[strings.ToLower(nextURL.Scheme)]
}
return strings.EqualFold(previousURL.Scheme, nextURL.Scheme) &&
strings.EqualFold(previousURL.Hostname(), nextURL.Hostname()) &&
previousPort == nextPort
}
func cloneAvailabilityTarget(target config.AvailabilityTarget) config.AvailabilityTarget {
clone := target
if target.HTTP == nil {
return clone
}
httpClone := *target.HTTP
httpClone.Body = cloneStringPointer(target.HTTP.Body)
httpClone.Authentication.Password = cloneStringPointer(target.HTTP.Authentication.Password)
httpClone.Authentication.BearerToken = cloneStringPointer(target.HTTP.Authentication.BearerToken)
httpClone.Headers = append([]config.AvailabilityHTTPHeader(nil), target.HTTP.Headers...)
for i := range httpClone.Headers {
httpClone.Headers[i].Value = cloneStringPointer(httpClone.Headers[i].Value)
}
clone.HTTP = &httpClone
return clone
}
func cloneStringPointer(value *string) *string {
if value == nil {
return nil
}
clone := *value
return &clone
}
func availabilityTargetIDFromPath(path string) (string, bool) {
id := strings.TrimPrefix(path, availabilityTargetsPathPrefix)
id = strings.Trim(id, "/")
+132
View File
@@ -6,6 +6,7 @@ import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
@@ -207,6 +208,137 @@ func TestAvailabilityHandlersTestSavedTarget(t *testing.T) {
}
}
func TestAvailabilityHandlersRedactAndPreserveHTTPContractSecrets(t *testing.T) {
const password = "never-return-this-password"
const headerValue = "never-return-this-header"
const requestBody = `{"secret":"never-return-this-body"}`
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
username, gotPassword, ok := r.BasicAuth()
if !ok || username != "pulse" || gotPassword != password {
w.WriteHeader(http.StatusUnauthorized)
return
}
if r.Header.Get("X-Contract-Key") != headerValue {
w.WriteHeader(http.StatusForbidden)
return
}
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(`{"status":"healthy"}`))
}))
defer server.Close()
persistence := config.NewConfigPersistence(t.TempDir())
handler := NewAvailabilityHandlers(
func(_ context.Context) *config.ConfigPersistence { return persistence },
nil,
nil,
)
passwordValue, headerSecret, bodyValue := password, headerValue, requestBody
target := config.AvailabilityTarget{
ID: "contract-target", Name: "Contract target", Address: server.URL,
Protocol: config.AvailabilityProbeHTTP, Enabled: true, TimeoutMillis: 1000,
HTTP: &config.AvailabilityHTTPConfig{
Method: config.AvailabilityHTTPMethodPOST,
Headers: []config.AvailabilityHTTPHeader{{ID: "contract-key", Name: "X-Contract-Key", Value: &headerSecret}},
Authentication: config.AvailabilityHTTPAuthentication{Type: config.AvailabilityHTTPAuthBasic, Username: "pulse", Password: &passwordValue},
Body: &bodyValue, ExpectedStatusMin: 200, ExpectedStatusMax: 299,
JSONPath: "status", JSONEquals: "healthy",
},
}
createRec := httptest.NewRecorder()
handler.HandleAdd(createRec, httptest.NewRequest(http.MethodPost, "/api/availability-targets", availabilityRequestBody(t, target)))
if createRec.Code != http.StatusCreated {
t.Fatalf("HandleAdd status = %d, body=%s", createRec.Code, createRec.Body.String())
}
for _, secret := range []string{password, headerValue, requestBody} {
if strings.Contains(createRec.Body.String(), secret) {
t.Fatalf("create response leaked secret %q: %s", secret, createRec.Body.String())
}
}
var created availabilityTargetResponse
if err := json.NewDecoder(createRec.Body).Decode(&created); err != nil {
t.Fatalf("decode created response: %v", err)
}
if created.HTTPSecrets == nil || !created.HTTPSecrets.PasswordConfigured || !created.HTTPSecrets.BodyConfigured ||
len(created.HTTPSecrets.Headers) != 1 || !created.HTTPSecrets.Headers[0].ValueConfigured {
t.Fatalf("secret state = %+v, want configured flags", created.HTTPSecrets)
}
if created.HTTP == nil || created.HTTP.Authentication.Password != nil || created.HTTP.Body != nil || created.HTTP.Headers[0].Value != nil {
t.Fatalf("redacted target still contains secret values: %+v", created.HTTP)
}
created.Name = "Renamed contract target"
updateRec := httptest.NewRecorder()
handler.HandleUpdate(updateRec, httptest.NewRequest(http.MethodPut, "/api/availability-targets/contract-target", availabilityRequestBody(t, created.AvailabilityTarget)))
if updateRec.Code != http.StatusOK {
t.Fatalf("HandleUpdate status = %d, body=%s", updateRec.Code, updateRec.Body.String())
}
loaded, err := persistence.LoadAvailabilityTargets()
if err != nil || len(loaded) != 1 {
t.Fatalf("LoadAvailabilityTargets() = %+v, %v", loaded, err)
}
stored := loaded[0]
if stored.HTTP == nil || stored.HTTP.Authentication.Password == nil || *stored.HTTP.Authentication.Password != password ||
stored.HTTP.Body == nil || *stored.HTTP.Body != requestBody || stored.HTTP.Headers[0].Value == nil || *stored.HTTP.Headers[0].Value != headerValue {
t.Fatalf("stored contract did not preserve write-only secrets: %+v", stored.HTTP)
}
if stored.ConfigRevision != 1 {
t.Fatalf("display-only edit revision = %d, want 1", stored.ConfigRevision)
}
testRec := httptest.NewRecorder()
handler.HandleTestConnection(testRec, httptest.NewRequest(http.MethodPost, "/api/availability-targets/test", availabilityRequestBody(t, created.AvailabilityTarget)))
if testRec.Code != http.StatusOK {
t.Fatalf("HandleTestConnection status = %d, body=%s", testRec.Code, testRec.Body.String())
}
var tested availabilityTestResponse
if err := json.NewDecoder(testRec.Body).Decode(&tested); err != nil {
t.Fatalf("decode test response: %v", err)
}
if !tested.Success || tested.TransportOutcome != "reachable" || tested.Application == nil || tested.Application.Outcome != "passed" {
t.Fatalf("test response = %+v, want reachable transport and passing application", tested)
}
for _, secret := range []string{password, headerValue, requestBody, "healthy"} {
if strings.Contains(testRec.Body.String(), secret) {
t.Fatalf("test response leaked request or response content %q: %s", secret, testRec.Body.String())
}
}
changedOriginReached := false
changedOrigin := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
changedOriginReached = true
w.WriteHeader(http.StatusOK)
}))
defer changedOrigin.Close()
created.Address = changedOrigin.URL
changedOriginUpdateRec := httptest.NewRecorder()
handler.HandleUpdate(changedOriginUpdateRec, httptest.NewRequest(http.MethodPut, "/api/availability-targets/contract-target", availabilityRequestBody(t, created.AvailabilityTarget)))
if changedOriginUpdateRec.Code != http.StatusBadRequest {
t.Fatalf("changed-origin update status = %d, body=%s; want re-entry validation", changedOriginUpdateRec.Code, changedOriginUpdateRec.Body.String())
}
changedOriginTestRec := httptest.NewRecorder()
handler.HandleTestConnection(changedOriginTestRec, httptest.NewRequest(http.MethodPost, "/api/availability-targets/test", availabilityRequestBody(t, created.AvailabilityTarget)))
if changedOriginTestRec.Code != http.StatusBadRequest {
t.Fatalf("changed-origin test status = %d, body=%s; want re-entry validation", changedOriginTestRec.Code, changedOriginTestRec.Body.String())
}
if changedOriginReached {
t.Fatal("stored HTTP values were replayed to a changed origin")
}
created.Address = server.URL
created.HTTP.JSONEquals = "ready"
revisionRec := httptest.NewRecorder()
handler.HandleUpdate(revisionRec, httptest.NewRequest(http.MethodPut, "/api/availability-targets/contract-target", availabilityRequestBody(t, created.AvailabilityTarget)))
if revisionRec.Code != http.StatusOK {
t.Fatalf("contract-edit HandleUpdate status = %d, body=%s", revisionRec.Code, revisionRec.Body.String())
}
loaded, err = persistence.LoadAvailabilityTargets()
if err != nil || len(loaded) != 1 || loaded[0].ConfigRevision != 2 {
t.Fatalf("contract edit revision = %+v, %v; want revision 2", loaded, err)
}
}
func TestAvailabilityHandlersListReturnsMockTargetsInMockMode(t *testing.T) {
previous := mock.IsMockEnabled()
if err := mock.SetEnabled(true); err != nil {
+199 -23
View File
@@ -10,8 +10,11 @@
package availabilityprobe
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/url"
@@ -39,11 +42,30 @@ const (
OutcomeIndeterminate Outcome = "indeterminate"
)
// ApplicationOutcome is separate from transport reachability so Pulse can
// distinguish "the endpoint answered" from "the service returned what the
// operator defined as healthy".
type ApplicationOutcome string
const (
ApplicationNotConfigured ApplicationOutcome = "not_configured"
ApplicationPassed ApplicationOutcome = "passed"
ApplicationFailed ApplicationOutcome = "failed"
)
type ApplicationResult struct {
Outcome ApplicationOutcome `json:"outcome"`
StatusCode int `json:"statusCode,omitempty"`
FailureCode string `json:"failureCode,omitempty"`
}
// ProbeResult carries the reachability outcome plus HTTPS certificate posture
// when the target completed a TLS handshake.
type ProbeResult struct {
Outcome Outcome `json:"outcome"`
Certificate *tlsutil.CertificateObservation `json:"certificate,omitempty"`
Outcome Outcome `json:"outcome"`
TransportOutcome Outcome `json:"transportOutcome"`
Application *ApplicationResult `json:"application,omitempty"`
Certificate *tlsutil.CertificateObservation `json:"certificate,omitempty"`
}
// Run executes one agentless availability check.
@@ -77,19 +99,17 @@ func DetailedResult(ctx context.Context, target config.AvailabilityTarget) (Prob
switch target.Protocol {
case config.AvailabilityProbeICMP:
outcome, err := outcomeFromError(probeICMP(probeCtx, target))
return ProbeResult{Outcome: outcome}, err
return ProbeResult{Outcome: outcome, TransportOutcome: outcome}, err
case config.AvailabilityProbeTCP:
outcome, err := outcomeFromError(probeTCP(probeCtx, target))
return ProbeResult{Outcome: outcome}, err
return ProbeResult{Outcome: outcome, TransportOutcome: outcome}, err
case config.AvailabilityProbeUDP:
outcome, err := probeUDP(probeCtx, target)
return ProbeResult{Outcome: outcome}, err
return ProbeResult{Outcome: outcome, TransportOutcome: outcome}, err
case config.AvailabilityProbeHTTP, config.AvailabilityProbeHTTPS:
certificate, err := probeHTTP(probeCtx, target, timeout)
outcome, probeErr := outcomeFromError(err)
return ProbeResult{Outcome: outcome, Certificate: certificate}, probeErr
return probeHTTP(probeCtx, target, timeout)
default:
return ProbeResult{Outcome: OutcomeUnreachable}, fmt.Errorf("unsupported availability protocol %q", target.Protocol)
return ProbeResult{Outcome: OutcomeUnreachable, TransportOutcome: OutcomeUnreachable}, fmt.Errorf("unsupported availability protocol %q", target.Protocol)
}
}
@@ -262,20 +282,23 @@ func probeTCPViaSystem(ctx context.Context, host string, port, timeoutMillis int
return fmt.Errorf("tcp probe failed: %s", details)
}
func probeHTTP(ctx context.Context, target config.AvailabilityTarget, timeout time.Duration) (*tlsutil.CertificateObservation, error) {
func probeHTTP(ctx context.Context, target config.AvailabilityTarget, timeout time.Duration) (ProbeResult, error) {
u, err := target.HTTPURL()
if err != nil {
return nil, err
return httpTransportFailure(err)
}
opts := httpOutboundOptions()
u, err = securityutil.ValidateOutboundFetchURL(ctx, u.String(), opts)
if err != nil {
return nil, fmt.Errorf("http availability target URL validation failed: %w", err)
return httpTransportFailure(fmt.Errorf("http availability target URL validation failed: %w", err))
}
client := securityutil.NewRestrictedOutboundHTTPClient(timeout, opts)
if target.HTTP != nil {
return probeHTTPContract(ctx, client, u, target.HTTP)
}
req, err := http.NewRequestWithContext(ctx, http.MethodHead, u.String(), nil)
if err != nil {
return nil, fmt.Errorf("build http availability request: %w", err)
return httpTransportFailure(fmt.Errorf("build http availability request: %w", err))
}
req.Header.Set("User-Agent", "Pulse availability probe")
resp, err := client.Do(req)
@@ -286,15 +309,19 @@ func probeHTTP(ctx context.Context, target config.AvailabilityTarget, timeout ti
return probeHTTPGet(ctx, client, u)
}
if resp.StatusCode >= http.StatusInternalServerError {
return certificate, fmt.Errorf("http probe returned %s", resp.Status)
return ProbeResult{Outcome: OutcomeUnreachable, TransportOutcome: OutcomeReachable, Certificate: certificate}, fmt.Errorf("http probe returned status %d", resp.StatusCode)
}
return certificate, nil
return ProbeResult{Outcome: OutcomeReachable, TransportOutcome: OutcomeReachable, Certificate: certificate}, nil
}
if ctxErr := ctx.Err(); ctxErr != nil {
return nil, ctxErr
return httpTransportFailure(ctxErr)
}
return nil, fmt.Errorf("http probe failed: %w", err)
return httpTransportFailure(fmt.Errorf("http probe failed: %w", err))
}
func httpTransportFailure(err error) (ProbeResult, error) {
return ProbeResult{Outcome: OutcomeUnreachable, TransportOutcome: OutcomeUnreachable}, err
}
func httpOutboundOptions() securityutil.RestrictedOutboundHTTPOptions {
@@ -306,25 +333,174 @@ func httpOutboundOptions() securityutil.RestrictedOutboundHTTPOptions {
}
}
func probeHTTPGet(ctx context.Context, client *http.Client, u *url.URL) (*tlsutil.CertificateObservation, error) {
func probeHTTPGet(ctx context.Context, client *http.Client, u *url.URL) (ProbeResult, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, fmt.Errorf("build http availability fallback request: %w", err)
return httpTransportFailure(fmt.Errorf("build http availability fallback request: %w", err))
}
req.Header.Set("User-Agent", "Pulse availability probe")
resp, err := client.Do(req)
if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
return nil, ctxErr
return httpTransportFailure(ctxErr)
}
return nil, fmt.Errorf("http probe failed: %w", err)
return httpTransportFailure(fmt.Errorf("http probe failed: %w", err))
}
defer resp.Body.Close()
certificate := certificateObservationFromResponse(resp)
if resp.StatusCode >= http.StatusInternalServerError {
return certificate, fmt.Errorf("http probe returned %s", resp.Status)
return ProbeResult{Outcome: OutcomeUnreachable, TransportOutcome: OutcomeReachable, Certificate: certificate}, fmt.Errorf("http probe returned status %d", resp.StatusCode)
}
return certificate, nil
return ProbeResult{Outcome: OutcomeReachable, TransportOutcome: OutcomeReachable, Certificate: certificate}, nil
}
func probeHTTPContract(ctx context.Context, client *http.Client, u *url.URL, contract *config.AvailabilityHTTPConfig) (ProbeResult, error) {
var body io.Reader
if contract.Body != nil {
body = bytes.NewBufferString(*contract.Body)
}
req, err := http.NewRequestWithContext(ctx, string(contract.Method), u.String(), body)
if err != nil {
return httpTransportFailure(fmt.Errorf("build http availability request: %w", err))
}
req.Header.Set("User-Agent", "Pulse availability probe")
for _, header := range contract.Headers {
if header.Value != nil {
req.Header.Set(header.Name, *header.Value)
}
}
switch contract.Authentication.Type {
case config.AvailabilityHTTPAuthBasic:
req.SetBasicAuth(contract.Authentication.Username, valueOrEmpty(contract.Authentication.Password))
case config.AvailabilityHTTPAuthBearer:
req.Header.Set("Authorization", "Bearer "+valueOrEmpty(contract.Authentication.BearerToken))
}
resp, err := client.Do(req)
if err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
return httpTransportFailure(ctxErr)
}
return httpTransportFailure(fmt.Errorf("http probe failed: %w", err))
}
defer resp.Body.Close()
certificate := certificateObservationFromResponse(resp)
result := ProbeResult{
Outcome: OutcomeReachable,
TransportOutcome: OutcomeReachable,
Certificate: certificate,
Application: &ApplicationResult{
Outcome: ApplicationPassed,
StatusCode: resp.StatusCode,
},
}
if resp.StatusCode < contract.ExpectedStatusMin || resp.StatusCode > contract.ExpectedStatusMax {
return applicationFailure(result, "status_mismatch", fmt.Sprintf("http response status %d was outside the expected %d-%d range", resp.StatusCode, contract.ExpectedStatusMin, contract.ExpectedStatusMax))
}
if contract.TextContains == "" && contract.JSONPath == "" {
return result, nil
}
responseBody, readErr := io.ReadAll(io.LimitReader(resp.Body, config.MaxAvailabilityHTTPResponseBytes+1))
if readErr != nil {
return applicationFailure(result, "response_read_failed", "http response body could not be read")
}
if len(responseBody) > config.MaxAvailabilityHTTPResponseBytes {
return applicationFailure(result, "response_too_large", fmt.Sprintf("http response body exceeded the %d byte assertion limit", config.MaxAvailabilityHTTPResponseBytes))
}
if contract.TextContains != "" && !bytes.Contains(responseBody, []byte(contract.TextContains)) {
return applicationFailure(result, "text_mismatch", "http response did not contain the expected text")
}
if contract.JSONPath != "" {
var document any
decoder := json.NewDecoder(bytes.NewReader(responseBody))
decoder.UseNumber()
if err := decoder.Decode(&document); err != nil {
return applicationFailure(result, "json_invalid", "http response was not valid JSON")
}
value, ok := lookupJSONPath(document, contract.JSONPath)
if !ok {
return applicationFailure(result, "json_path_missing", "http response did not contain the expected JSON path")
}
if contract.JSONEquals != "" && normalizedJSONAssertionValue(value) != contract.JSONEquals {
return applicationFailure(result, "json_value_mismatch", "http response JSON value did not match the expected value")
}
}
return result, nil
}
func applicationFailure(result ProbeResult, code, message string) (ProbeResult, error) {
result.Outcome = OutcomeUnreachable
result.Application.Outcome = ApplicationFailed
result.Application.FailureCode = code
return result, fmt.Errorf("%s", message)
}
func valueOrEmpty(value *string) string {
if value == nil {
return ""
}
return *value
}
func normalizedJSONAssertionValue(value any) string {
if text, ok := value.(string); ok {
return text
}
encoded, err := json.Marshal(value)
if err != nil {
return ""
}
return string(encoded)
}
// lookupJSONPath supports a deliberately bounded field/index vocabulary such
// as "status", "data.healthy", or "items[0].state". It is not a scripting
// language and cannot execute operator-authored expressions.
func lookupJSONPath(document any, rawPath string) (any, bool) {
path := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(rawPath), "$"))
path = strings.TrimPrefix(path, ".")
if path == "" {
return document, true
}
current := document
for _, segment := range strings.Split(path, ".") {
if segment == "" {
return nil, false
}
name := segment
indexes := ""
if bracket := strings.Index(segment, "["); bracket >= 0 {
name = segment[:bracket]
indexes = segment[bracket:]
}
if name != "" {
object, ok := current.(map[string]any)
if !ok {
return nil, false
}
current, ok = object[name]
if !ok {
return nil, false
}
}
for indexes != "" {
if !strings.HasPrefix(indexes, "[") {
return nil, false
}
end := strings.IndexByte(indexes, ']')
if end <= 1 {
return nil, false
}
index, err := strconv.Atoi(indexes[1:end])
array, ok := current.([]any)
if err != nil || !ok || index < 0 || index >= len(array) {
return nil, false
}
current = array[index]
indexes = indexes[end+1:]
}
}
return current, true
}
func certificateObservationFromResponse(response *http.Response) *tlsutil.CertificateObservation {
+78
View File
@@ -2,8 +2,10 @@ package availabilityprobe
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
@@ -22,6 +24,82 @@ func TestAvailabilityHTTPOutboundOptionsUsesSharedPeerCertificateCapture(t *test
}
}
func TestDetailedResultEvaluatesBoundedHTTPApplicationContract(t *testing.T) {
requestBody := `{"operation":"health"}`
password := "secret-password"
headerValue := "tenant-a"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
t.Errorf("method = %s, want POST", r.Method)
}
username, gotPassword, ok := r.BasicAuth()
if !ok || username != "pulse" || gotPassword != password {
t.Errorf("basic auth = %q/%q/%v", username, gotPassword, ok)
}
if got := r.Header.Get("X-Tenant"); got != headerValue {
t.Errorf("X-Tenant = %q, want %q", got, headerValue)
}
body, _ := io.ReadAll(r.Body)
if string(body) != requestBody {
t.Errorf("body = %q, want %q", body, requestBody)
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
_, _ = w.Write([]byte(`{"data":{"status":"healthy"}}`))
}))
defer server.Close()
result, err := DetailedResult(context.Background(), config.AvailabilityTarget{
Address: server.URL, Protocol: config.AvailabilityProbeHTTP, Enabled: true, TimeoutMillis: 1000,
HTTP: &config.AvailabilityHTTPConfig{
Method: config.AvailabilityHTTPMethodPOST,
Headers: []config.AvailabilityHTTPHeader{{ID: "tenant", Name: "X-Tenant", Value: &headerValue}},
Authentication: config.AvailabilityHTTPAuthentication{Type: config.AvailabilityHTTPAuthBasic, Username: "pulse", Password: &password},
Body: &requestBody, ExpectedStatusMin: 200, ExpectedStatusMax: 299,
TextContains: "healthy", JSONPath: "data.status", JSONEquals: "healthy",
},
})
if err != nil {
t.Fatalf("DetailedResult() error = %v", err)
}
if result.Outcome != OutcomeReachable || result.TransportOutcome != OutcomeReachable {
t.Fatalf("result outcomes = %+v, want reachable transport and overall", result)
}
if result.Application == nil || result.Application.Outcome != ApplicationPassed || result.Application.StatusCode != http.StatusCreated {
t.Fatalf("application result = %+v, want passed HTTP 201", result.Application)
}
}
func TestDetailedResultKeepsReachabilityWhenHTTPApplicationContractFails(t *testing.T) {
const responseSecret = "top-secret-response-content"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusServiceUnavailable)
_, _ = w.Write([]byte(responseSecret))
}))
defer server.Close()
result, err := DetailedResult(context.Background(), config.AvailabilityTarget{
Address: server.URL, Protocol: config.AvailabilityProbeHTTP, Enabled: true, TimeoutMillis: 1000,
HTTP: &config.AvailabilityHTTPConfig{
Method: config.AvailabilityHTTPMethodGET,
Authentication: config.AvailabilityHTTPAuthentication{Type: config.AvailabilityHTTPAuthNone},
ExpectedStatusMin: 200, ExpectedStatusMax: 299,
},
})
if err == nil {
t.Fatal("DetailedResult() error = nil, want status assertion failure")
}
if strings.Contains(err.Error(), responseSecret) {
t.Fatalf("error leaked response content: %q", err)
}
if result.Outcome != OutcomeUnreachable || result.TransportOutcome != OutcomeReachable {
t.Fatalf("result outcomes = %+v, want unreachable overall but reachable transport", result)
}
if result.Application == nil || result.Application.Outcome != ApplicationFailed || result.Application.FailureCode != "status_mismatch" {
t.Fatalf("application result = %+v, want typed status mismatch", result.Application)
}
}
func TestDetailedResultCapturesHTTPSCertificatePosture(t *testing.T) {
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
+249 -1
View File
@@ -4,6 +4,8 @@ import (
"fmt"
"net"
"net/url"
"reflect"
"strconv"
"strings"
"github.com/google/uuid"
@@ -14,8 +16,59 @@ const (
DefaultAvailabilityTimeoutMillis = 2000
DefaultAvailabilityFailureThreshold = 2
DefaultCertificateExpiryWarningDays = 30
MaxAvailabilityHTTPBodyBytes = 8192
MaxAvailabilityHTTPResponseBytes = 65536
)
type AvailabilityHTTPMethod string
const (
AvailabilityHTTPMethodHEAD AvailabilityHTTPMethod = "HEAD"
AvailabilityHTTPMethodGET AvailabilityHTTPMethod = "GET"
AvailabilityHTTPMethodPOST AvailabilityHTTPMethod = "POST"
)
type AvailabilityHTTPAuthType string
const (
AvailabilityHTTPAuthNone AvailabilityHTTPAuthType = "none"
AvailabilityHTTPAuthBasic AvailabilityHTTPAuthType = "basic"
AvailabilityHTTPAuthBearer AvailabilityHTTPAuthType = "bearer"
)
// AvailabilityHTTPHeader uses a stable ID so API clients can edit a redacted
// header without receiving its stored value. A nil Value means "leave the
// stored value unchanged" on update; an explicit empty string clears it.
type AvailabilityHTTPHeader struct {
ID string `json:"id"`
Name string `json:"name"`
Value *string `json:"value,omitempty"`
}
// AvailabilityHTTPAuthentication contains execution credentials. Secret
// values are pointers so update requests can distinguish omitted from empty.
// The API must redact them before returning a target to a client.
type AvailabilityHTTPAuthentication struct {
Type AvailabilityHTTPAuthType `json:"type"`
Username string `json:"username,omitempty"`
Password *string `json:"password,omitempty"`
BearerToken *string `json:"bearerToken,omitempty"`
}
// AvailabilityHTTPConfig is an explicit application response contract. A nil
// config preserves the legacy HEAD-with-GET-fallback reachability semantics.
type AvailabilityHTTPConfig struct {
Method AvailabilityHTTPMethod `json:"method"`
Headers []AvailabilityHTTPHeader `json:"headers,omitempty"`
Authentication AvailabilityHTTPAuthentication `json:"authentication"`
Body *string `json:"body,omitempty"`
ExpectedStatusMin int `json:"expectedStatusMin"`
ExpectedStatusMax int `json:"expectedStatusMax"`
TextContains string `json:"textContains,omitempty"`
JSONPath string `json:"jsonPath,omitempty"`
JSONEquals string `json:"jsonEquals,omitempty"`
}
type AvailabilityProbeProtocol string
const (
@@ -75,6 +128,10 @@ type AvailabilityTarget struct {
// check runs from the local Pulse instance. Agent existence is validated at
// the API layer, not here, because config has no view of monitor state.
ProbeAgentID string `json:"probeAgentId,omitempty"`
// HTTP is absent for legacy targets. Its request body, credentials, and
// header values are encrypted with the rest of availability target storage
// and are never returned by the API.
HTTP *AvailabilityHTTPConfig `json:"http,omitempty"`
}
// NewAvailabilityTarget returns a new target with generated ID and defaults.
@@ -124,6 +181,25 @@ func (t *AvailabilityTarget) ApplyDefaults() {
t.UDPMode = AvailabilityUDPResponseRequired
}
}
if t.HTTP != nil {
if t.HTTP.Method == "" {
t.HTTP.Method = AvailabilityHTTPMethodGET
}
if t.HTTP.Authentication.Type == "" {
t.HTTP.Authentication.Type = AvailabilityHTTPAuthNone
}
if t.HTTP.ExpectedStatusMin == 0 {
t.HTTP.ExpectedStatusMin = 200
}
if t.HTTP.ExpectedStatusMax == 0 {
t.HTTP.ExpectedStatusMax = 399
}
for i := range t.HTTP.Headers {
if strings.TrimSpace(t.HTTP.Headers[i].ID) == "" {
t.HTTP.Headers[i].ID = uuid.NewString()
}
}
}
}
func (t AvailabilityTarget) EffectivePollIntervalSecs() int {
@@ -160,6 +236,7 @@ func AvailabilityExecutionConfigChanged(previous, next AvailabilityTarget) bool
previous.UDPMode != next.UDPMode ||
previous.UDPRequest != next.UDPRequest ||
previous.UDPExpected != next.UDPExpected ||
!reflect.DeepEqual(previous.HTTP, next.HTTP) ||
previous.EffectiveTimeoutMillis() != next.EffectiveTimeoutMillis() ||
previous.EffectivePollIntervalSecs() != next.EffectivePollIntervalSecs() ||
previous.ProbeAgentID != next.ProbeAgentID
@@ -232,6 +309,13 @@ func (t AvailabilityTarget) Validate() error {
} else if t.UDPMode != "" || t.UDPRequest != "" || t.UDPExpected != "" {
return fmt.Errorf("UDP settings may only be used with UDP availability targets")
}
if protocol == AvailabilityProbeHTTP || protocol == AvailabilityProbeHTTPS {
if err := validateAvailabilityHTTPConfig(t.HTTP); err != nil {
return err
}
} else if t.HTTP != nil {
return fmt.Errorf("HTTP response contracts may only be used with HTTP or HTTPS availability targets")
}
if protocol != AvailabilityProbeHTTPS && (t.CertificateMonitoringDisabled || t.CertificateExpiryWarningDays != 0) {
return fmt.Errorf("certificate monitoring settings may only be used with HTTPS availability targets")
}
@@ -277,7 +361,7 @@ func (t AvailabilityTarget) HTTPURL() (*url.URL, error) {
}
u, err := url.Parse(raw)
if err != nil {
return nil, fmt.Errorf("invalid http availability address: %w", err)
return nil, fmt.Errorf("invalid http availability address")
}
if u.Scheme != "http" && u.Scheme != "https" {
return nil, fmt.Errorf("http availability targets require http or https scheme")
@@ -285,6 +369,9 @@ func (t AvailabilityTarget) HTTPURL() (*url.URL, error) {
if strings.TrimSpace(u.Hostname()) == "" {
return nil, fmt.Errorf("http availability target host is required")
}
if u.User != nil {
return nil, fmt.Errorf("http availability target credentials must use the authentication fields")
}
if t.Port > 0 {
u.Host = net.JoinHostPort(u.Hostname(), fmt.Sprintf("%d", t.Port))
}
@@ -316,10 +403,171 @@ func NormalizeAvailabilityTarget(target AvailabilityTarget) AvailabilityTarget {
}
target.LinkedResourceID = strings.TrimSpace(target.LinkedResourceID)
target.ProbeAgentID = strings.TrimSpace(target.ProbeAgentID)
if target.HTTP != nil {
target.HTTP.Method = AvailabilityHTTPMethod(strings.ToUpper(strings.TrimSpace(string(target.HTTP.Method))))
target.HTTP.Authentication.Type = AvailabilityHTTPAuthType(strings.ToLower(strings.TrimSpace(string(target.HTTP.Authentication.Type))))
target.HTTP.Authentication.Username = strings.TrimSpace(target.HTTP.Authentication.Username)
target.HTTP.TextContains = strings.TrimSpace(target.HTTP.TextContains)
target.HTTP.JSONPath = strings.TrimSpace(target.HTTP.JSONPath)
target.HTTP.JSONEquals = strings.TrimSpace(target.HTTP.JSONEquals)
for i := range target.HTTP.Headers {
target.HTTP.Headers[i].ID = strings.TrimSpace(target.HTTP.Headers[i].ID)
target.HTTP.Headers[i].Name = strings.TrimSpace(target.HTTP.Headers[i].Name)
}
if target.HTTP.Authentication.Type != AvailabilityHTTPAuthBasic {
target.HTTP.Authentication.Username = ""
target.HTTP.Authentication.Password = nil
}
if target.HTTP.Authentication.Type != AvailabilityHTTPAuthBearer {
target.HTTP.Authentication.BearerToken = nil
}
if target.HTTP.Method != AvailabilityHTTPMethodPOST {
target.HTTP.Body = nil
}
}
if target.Protocol != AvailabilityProbeHTTP && target.Protocol != AvailabilityProbeHTTPS {
target.HTTP = nil
}
target.ApplyDefaults()
return target
}
func validateAvailabilityHTTPConfig(contract *AvailabilityHTTPConfig) error {
if contract == nil {
return nil
}
switch contract.Method {
case AvailabilityHTTPMethodHEAD, AvailabilityHTTPMethodGET, AvailabilityHTTPMethodPOST:
default:
return fmt.Errorf("HTTP response contract method must be HEAD, GET, or POST")
}
if contract.ExpectedStatusMin < 100 || contract.ExpectedStatusMin > 599 ||
contract.ExpectedStatusMax < 100 || contract.ExpectedStatusMax > 599 ||
contract.ExpectedStatusMin > contract.ExpectedStatusMax {
return fmt.Errorf("HTTP expected status range must be between 100 and 599")
}
if contract.Body != nil {
if contract.Method != AvailabilityHTTPMethodPOST {
return fmt.Errorf("HTTP request bodies may only be used with POST")
}
if len(*contract.Body) > MaxAvailabilityHTTPBodyBytes {
return fmt.Errorf("HTTP request body must be %d bytes or less", MaxAvailabilityHTTPBodyBytes)
}
}
if len(contract.TextContains) > 256 {
return fmt.Errorf("HTTP text assertion must be 256 bytes or less")
}
if len(contract.JSONPath) > 256 || len(contract.JSONEquals) > 512 {
return fmt.Errorf("HTTP JSON assertion is too long")
}
if contract.JSONPath != "" && !validAvailabilityJSONPath(contract.JSONPath) {
return fmt.Errorf("HTTP JSON path must use dot fields and numeric array indexes")
}
if contract.Method == AvailabilityHTTPMethodHEAD && (contract.TextContains != "" || contract.JSONPath != "") {
return fmt.Errorf("HTTP HEAD contracts cannot assert a response body")
}
if contract.JSONEquals != "" && contract.JSONPath == "" {
return fmt.Errorf("HTTP JSON expected value requires a JSON path")
}
if len(contract.Headers) > 16 {
return fmt.Errorf("HTTP response contracts support at most 16 request headers")
}
seenIDs := make(map[string]struct{}, len(contract.Headers))
seenNames := make(map[string]struct{}, len(contract.Headers))
totalHeaderValueBytes := 0
for _, header := range contract.Headers {
if header.ID == "" || header.Name == "" || !validAvailabilityHTTPHeaderName(header.Name) {
return fmt.Errorf("HTTP request headers require a valid name and stable id")
}
name := strings.ToLower(header.Name)
switch name {
case "authorization", "host", "content-length", "connection", "transfer-encoding", "proxy-authorization":
return fmt.Errorf("HTTP request header %q is reserved", header.Name)
}
if _, ok := seenIDs[header.ID]; ok {
return fmt.Errorf("HTTP request header ids must be unique")
}
if _, ok := seenNames[name]; ok {
return fmt.Errorf("HTTP request header names must be unique")
}
seenIDs[header.ID] = struct{}{}
seenNames[name] = struct{}{}
if header.Value != nil {
if strings.ContainsAny(*header.Value, "\r\n") {
return fmt.Errorf("HTTP request header values must not contain newlines")
}
totalHeaderValueBytes += len(*header.Value)
}
}
if totalHeaderValueBytes > 8192 {
return fmt.Errorf("HTTP request header values must total 8192 bytes or less")
}
switch contract.Authentication.Type {
case AvailabilityHTTPAuthNone:
case AvailabilityHTTPAuthBasic:
if contract.Authentication.Username == "" || contract.Authentication.Password == nil || *contract.Authentication.Password == "" {
return fmt.Errorf("HTTP basic authentication requires a username and password")
}
case AvailabilityHTTPAuthBearer:
if contract.Authentication.BearerToken == nil || *contract.Authentication.BearerToken == "" {
return fmt.Errorf("HTTP bearer authentication requires a token")
}
default:
return fmt.Errorf("HTTP authentication type must be none, basic, or bearer")
}
return nil
}
func validAvailabilityHTTPHeaderName(name string) bool {
if name == "" {
return false
}
for _, char := range name {
if !((char >= 'a' && char <= 'z') || (char >= 'A' && char <= 'Z') ||
(char >= '0' && char <= '9') || strings.ContainsRune("!#$%&'*+-.^_`|~", char)) {
return false
}
}
return true
}
func validAvailabilityJSONPath(rawPath string) bool {
path := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(rawPath), "$"))
path = strings.TrimPrefix(path, ".")
if path == "" {
return true
}
for _, segment := range strings.Split(path, ".") {
if segment == "" {
return false
}
name := segment
indexes := ""
if bracket := strings.IndexByte(segment, '['); bracket >= 0 {
name = segment[:bracket]
indexes = segment[bracket:]
}
if strings.ContainsAny(name, "[]") || (name == "" && indexes == "") {
return false
}
for indexes != "" {
if !strings.HasPrefix(indexes, "[") {
return false
}
end := strings.IndexByte(indexes, ']')
if end <= 1 {
return false
}
index, err := strconv.Atoi(indexes[1:end])
if err != nil || index < 0 {
return false
}
indexes = indexes[end+1:]
}
}
return true
}
func normalizeAvailabilityUDPMode(mode AvailabilityUDPMode) AvailabilityUDPMode {
return AvailabilityUDPMode(strings.ToLower(strings.TrimSpace(string(mode))))
}
+91 -1
View File
@@ -1,6 +1,10 @@
package config
import "testing"
import (
"os"
"strings"
"testing"
)
func TestNormalizeAvailabilityTargetPreservesHTTPAddress(t *testing.T) {
target := NormalizeAvailabilityTarget(AvailabilityTarget{
@@ -123,6 +127,43 @@ func TestAvailabilityTargetHTTPURLAppliesPortAndPath(t *testing.T) {
}
}
func TestAvailabilityTargetHTTPURLDoesNotEchoMalformedAddress(t *testing.T) {
const sentinel = "address-secret-sentinel"
target := AvailabilityTarget{
Address: "http://[" + sentinel,
Protocol: AvailabilityProbeHTTP,
}
_, err := target.HTTPURL()
if err == nil {
t.Fatal("HTTPURL() error = nil, want malformed address error")
}
if strings.Contains(err.Error(), sentinel) {
t.Fatalf("HTTPURL() error leaked malformed address content: %v", err)
}
}
func TestAvailabilityHTTPContractRejectsUnboundedOrExecutableShapes(t *testing.T) {
target := NormalizeAvailabilityTarget(AvailabilityTarget{
Address: "https://service.local/health", Protocol: AvailabilityProbeHTTPS, Enabled: true,
HTTP: &AvailabilityHTTPConfig{
Method: AvailabilityHTTPMethodGET,
Authentication: AvailabilityHTTPAuthentication{Type: AvailabilityHTTPAuthNone},
ExpectedStatusMin: 200, ExpectedStatusMax: 299,
JSONPath: "data[not-an-index].status",
},
})
if err := target.Validate(); err == nil {
t.Fatal("Validate() error = nil, want bounded JSON path validation")
}
target.HTTP.JSONPath = "data.status"
secret := "value"
target.HTTP.Headers = []AvailabilityHTTPHeader{{ID: "reserved", Name: "Authorization", Value: &secret}}
if err := target.Validate(); err == nil {
t.Fatal("Validate() error = nil, want reserved header validation")
}
}
func TestAvailabilityTargetValidateRejectsTCPWithoutPort(t *testing.T) {
target := NormalizeAvailabilityTarget(AvailabilityTarget{
Address: "device.local",
@@ -196,6 +237,55 @@ func TestAvailabilityTargetsRoundTripThroughPersistence(t *testing.T) {
}
}
func TestAvailabilityHTTPContractSecretsRemainEncryptedAtRest(t *testing.T) {
persistence := NewConfigPersistence(t.TempDir())
password := "plaintext-password-sentinel"
token := "plaintext-token-sentinel"
header := "plaintext-header-sentinel"
body := "plaintext-body-sentinel"
target := AvailabilityTarget{
ID: "secure-http", Address: "https://service.local/health", Protocol: AvailabilityProbeHTTPS, Enabled: true,
HTTP: &AvailabilityHTTPConfig{
Method: AvailabilityHTTPMethodPOST,
Headers: []AvailabilityHTTPHeader{{ID: "header-1", Name: "X-Health-Key", Value: &header}},
Authentication: AvailabilityHTTPAuthentication{Type: AvailabilityHTTPAuthBasic, Username: "pulse", Password: &password},
Body: &body, ExpectedStatusMin: 200, ExpectedStatusMax: 299,
},
}
bearerTarget := AvailabilityTarget{
ID: "secure-bearer", Address: "https://service.local/bearer-health", Protocol: AvailabilityProbeHTTPS, Enabled: true,
HTTP: &AvailabilityHTTPConfig{
Method: AvailabilityHTTPMethodGET,
Authentication: AvailabilityHTTPAuthentication{Type: AvailabilityHTTPAuthBearer, BearerToken: &token},
ExpectedStatusMin: 200, ExpectedStatusMax: 299,
},
}
if err := persistence.SaveAvailabilityTargets([]AvailabilityTarget{target, bearerTarget}); err != nil {
t.Fatalf("SaveAvailabilityTargets() error = %v", err)
}
raw, err := os.ReadFile(persistence.availabilityFile)
if err != nil {
t.Fatalf("ReadFile(%q) error = %v", persistence.availabilityFile, err)
}
for _, secret := range []string{password, token, header, body} {
if strings.Contains(string(raw), secret) {
t.Fatalf("encrypted availability file contains plaintext secret %q", secret)
}
}
loaded, err := persistence.LoadAvailabilityTargets()
if err != nil || len(loaded) != 2 || loaded[0].HTTP == nil || loaded[1].HTTP == nil {
t.Fatalf("LoadAvailabilityTargets() = %+v, %v", loaded, err)
}
if loaded[0].HTTP.Authentication.Password == nil || *loaded[0].HTTP.Authentication.Password != password ||
loaded[0].HTTP.Headers[0].Value == nil || *loaded[0].HTTP.Headers[0].Value != header ||
loaded[0].HTTP.Body == nil || *loaded[0].HTTP.Body != body {
t.Fatalf("decrypted contract did not round trip: %+v", loaded[0].HTTP)
}
if loaded[1].HTTP.Authentication.BearerToken == nil || *loaded[1].HTTP.Authentication.BearerToken != token {
t.Fatalf("decrypted bearer token did not round trip: %+v", loaded[1].HTTP)
}
}
func TestNormalizeAvailabilityTargetTrimsProbeAgentID(t *testing.T) {
target := NormalizeAvailabilityTarget(AvailabilityTarget{
Address: "device.local",
+15 -16
View File
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"fmt"
"reflect"
"sort"
"strings"
"sync"
@@ -129,15 +130,7 @@ func normalizeAvailabilityAssignments(targets []config.AvailabilityTarget) []con
}
func availabilityAssignmentsEqual(left, right []config.AvailabilityTarget) bool {
if len(left) != len(right) {
return false
}
for i := range left {
if left[i] != right[i] {
return false
}
}
return true
return reflect.DeepEqual(left, right)
}
func (m *availabilityProbeModule) assignments() []config.AvailabilityTarget {
@@ -243,13 +236,19 @@ func (m *availabilityProbeModule) check(ctx context.Context, target config.Avail
}
result := agentshost.AvailabilityProbeResult{
ObservationID: uuid.NewString(),
TargetID: target.ID,
ConfigRevision: target.ConfigRevision,
Outcome: string(probeResult.Outcome),
LatencyMillis: latency.Milliseconds(),
CheckedAt: m.now().UTC(),
Certificate: probeResult.Certificate.Clone(),
ObservationID: uuid.NewString(),
TargetID: target.ID,
ConfigRevision: target.ConfigRevision,
Outcome: string(probeResult.Outcome),
TransportOutcome: string(probeResult.TransportOutcome),
LatencyMillis: latency.Milliseconds(),
CheckedAt: m.now().UTC(),
Certificate: probeResult.Certificate.Clone(),
}
if probeResult.Application != nil {
result.ApplicationOutcome = string(probeResult.Application.Outcome)
result.ApplicationStatusCode = probeResult.Application.StatusCode
result.ApplicationFailureCode = probeResult.Application.FailureCode
}
if err != nil {
message := strings.TrimSpace(err.Error())
+29
View File
@@ -118,6 +118,35 @@ func TestAvailabilityTargetsFromSettingsHandlesMissingAndUnusableValues(t *testi
}
}
func TestAvailabilityTargetsFromSettingsPreservesHTTPExecutionSecrets(t *testing.T) {
settings := availabilitySetting(map[string]interface{}{
"id": "http-contract", "address": "https://service.local/health", "protocol": "https", "enabled": true,
"http": map[string]interface{}{
"method": "POST", "body": `{"operation":"health"}`,
"expectedStatusMin": float64(200), "expectedStatusMax": float64(299),
"authentication": map[string]interface{}{"type": "bearer", "bearerToken": "agent-secret-token"},
"headers": []interface{}{map[string]interface{}{"id": "tenant", "name": "X-Tenant", "value": "tenant-a"}},
"jsonPath": "status", "jsonEquals": "healthy",
},
})
targets, err := availabilityTargetsFromSettings(settings)
if err != nil || len(targets) != 1 || targets[0].HTTP == nil {
t.Fatalf("availabilityTargetsFromSettings() = %+v, %v", targets, err)
}
contract := targets[0].HTTP
if contract.Body == nil || *contract.Body != `{"operation":"health"}` ||
contract.Authentication.BearerToken == nil || *contract.Authentication.BearerToken != "agent-secret-token" ||
len(contract.Headers) != 1 || contract.Headers[0].Value == nil || *contract.Headers[0].Value != "tenant-a" {
t.Fatalf("decoded HTTP contract = %+v, want complete execution values", contract)
}
decodedAgain, err := availabilityTargetsFromSettings(settings)
if err != nil || !availabilityAssignmentsEqual(targets, decodedAgain) {
t.Fatalf("identical pointer-backed HTTP assignments were reported as changed: %+v, %v", decodedAgain, err)
}
}
func TestApplyRemoteAvailabilityTargetsReconcilesAssignments(t *testing.T) {
agent := &Agent{
logger: zerolog.New(io.Discard),
+88 -40
View File
@@ -25,23 +25,27 @@ type tlsCert = tlsutil.CertificateObservation
// AvailabilityProbeStatus captures the last observed state of an agentless
// endpoint probe.
type AvailabilityProbeStatus struct {
TargetID string `json:"targetId"`
Name string `json:"name"`
TargetKind string `json:"targetKind,omitempty"`
Address string `json:"address"`
Protocol string `json:"protocol"`
Outcome string `json:"outcome,omitempty"`
Enabled bool `json:"enabled"`
Available bool `json:"available"`
LastChecked time.Time `json:"lastChecked,omitempty"`
LastSuccess time.Time `json:"lastSuccess,omitempty"`
LatencyMillis int64 `json:"latencyMillis,omitempty"`
ConsecutiveFailures int `json:"consecutiveFailures,omitempty"`
LastError string `json:"lastError,omitempty"`
FailureThreshold int `json:"failureThreshold,omitempty"`
ProbeAgentID string `json:"probeAgentId,omitempty"`
Certificate *tlsCert `json:"certificate,omitempty"`
CertificateCurrent bool `json:"-"`
TargetID string `json:"targetId"`
Name string `json:"name"`
TargetKind string `json:"targetKind,omitempty"`
Address string `json:"address"`
Protocol string `json:"protocol"`
Outcome string `json:"outcome,omitempty"`
TransportOutcome string `json:"transportOutcome,omitempty"`
ApplicationOutcome string `json:"applicationOutcome,omitempty"`
ApplicationStatusCode int `json:"applicationStatusCode,omitempty"`
ApplicationFailureCode string `json:"applicationFailureCode,omitempty"`
Enabled bool `json:"enabled"`
Available bool `json:"available"`
LastChecked time.Time `json:"lastChecked,omitempty"`
LastSuccess time.Time `json:"lastSuccess,omitempty"`
LatencyMillis int64 `json:"latencyMillis,omitempty"`
ConsecutiveFailures int `json:"consecutiveFailures,omitempty"`
LastError string `json:"lastError,omitempty"`
FailureThreshold int `json:"failureThreshold,omitempty"`
ProbeAgentID string `json:"probeAgentId,omitempty"`
Certificate *tlsCert `json:"certificate,omitempty"`
CertificateCurrent bool `json:"-"`
// ProbeReportReceivedAt is server-authored freshness evidence for a remote
// observation. Keep it off the wire: LastChecked remains the agent's
// observation time, while disconnect detection must not trust agent clock
@@ -332,7 +336,7 @@ func (m *Monitor) pollAvailabilityTarget(ctx context.Context, target config.Avai
result, err := ProbeAvailabilityTargetDetailedResult(ctx, target)
latency := time.Since(start)
checkedAt := time.Now().UTC()
m.applyAvailabilityObservation(target, uuid.NewString(), checkedAt, latency, result.Outcome, err, result.Certificate, "", time.Time{})
m.applyAvailabilityObservationDetailed(target, uuid.NewString(), checkedAt, latency, result.Outcome, result.TransportOutcome, result.Application, err, result.Certificate, "", time.Time{})
m.updateResourceStore(m.GetState())
}
@@ -350,7 +354,26 @@ func (m *Monitor) applyAvailabilityObservation(
probeAgentID string,
probeReportReceivedAt time.Time,
) {
m.setAvailabilityStatusWithCertificate(target, checkedAt, latency, outcome, probeErr, certificate, probeAgentID, probeReportReceivedAt)
m.applyAvailabilityObservationDetailed(target, observationID, checkedAt, latency, outcome, outcome, nil, probeErr, certificate, probeAgentID, probeReportReceivedAt)
}
func (m *Monitor) applyAvailabilityObservationDetailed(
target config.AvailabilityTarget,
observationID string,
checkedAt time.Time,
latency time.Duration,
outcome AvailabilityProbeOutcome,
transportOutcome availabilityprobe.Outcome,
application *availabilityprobe.ApplicationResult,
probeErr error,
certificate *tlsutil.CertificateObservation,
probeAgentID string,
probeReportReceivedAt time.Time,
) {
if transportOutcome == "" {
transportOutcome = outcome
}
m.setAvailabilityStatusWithDetails(target, checkedAt, latency, outcome, transportOutcome, application, probeErr, certificate, probeAgentID, probeReportReceivedAt)
m.recordAvailabilityHistory(target, observationID, checkedAt, latency, outcome, probeErr, probeAgentID, probeReportReceivedAt)
if probeErr == nil {
@@ -435,12 +458,33 @@ func (m *Monitor) setAvailabilityStatusWithCertificate(
certificate *tlsutil.CertificateObservation,
probeAgentID string,
probeReportReceivedAt time.Time,
) {
m.setAvailabilityStatusWithDetails(target, checkedAt, latency, outcome, outcome, nil, probeErr, certificate, probeAgentID, probeReportReceivedAt)
}
func (m *Monitor) setAvailabilityStatusWithDetails(
target config.AvailabilityTarget,
checkedAt time.Time,
latency time.Duration,
outcome AvailabilityProbeOutcome,
transportOutcome availabilityprobe.Outcome,
application *availabilityprobe.ApplicationResult,
probeErr error,
certificate *tlsutil.CertificateObservation,
probeAgentID string,
probeReportReceivedAt time.Time,
) {
if m == nil {
return
}
status := availabilityStatusFromTarget(target)
status.Outcome = string(outcome)
status.TransportOutcome = string(transportOutcome)
if application != nil {
status.ApplicationOutcome = string(application.Outcome)
status.ApplicationStatusCode = application.StatusCode
status.ApplicationFailureCode = application.FailureCode
}
status.LastChecked = checkedAt
status.Certificate = certificate.Clone()
status.CertificateCurrent = status.Certificate != nil
@@ -530,27 +574,31 @@ func availabilityResourceFromTarget(target config.AvailabilityTarget, status Ava
}
resourceStatus := availabilityResourceStatus(target, status)
data := &unifiedresources.AvailabilityData{
TargetID: target.ID,
LinkedResourceID: strings.TrimSpace(target.LinkedResourceID),
Name: target.DisplayName(),
TargetKind: string(target.TargetKind),
Address: target.Address,
Protocol: string(target.Protocol),
ProbeOutcome: status.Outcome,
ProbeAgentID: status.ProbeAgentID,
UDPMode: string(target.UDPMode),
Port: target.Port,
Path: target.Path,
Enabled: target.Enabled,
Available: status.Available,
LastChecked: timePointerIfSet(status.LastChecked),
LastSuccess: timePointerIfSet(status.LastSuccess),
LatencyMillis: status.LatencyMillis,
ConsecutiveFailures: status.ConsecutiveFailures,
LastError: status.LastError,
FailureThreshold: target.EffectiveFailureThreshold(),
PollIntervalSeconds: target.EffectivePollIntervalSecs(),
TimeoutMillis: target.EffectiveTimeoutMillis(),
TargetID: target.ID,
LinkedResourceID: strings.TrimSpace(target.LinkedResourceID),
Name: target.DisplayName(),
TargetKind: string(target.TargetKind),
Address: target.Address,
Protocol: string(target.Protocol),
ProbeOutcome: status.Outcome,
TransportOutcome: status.TransportOutcome,
ApplicationOutcome: status.ApplicationOutcome,
ApplicationStatusCode: status.ApplicationStatusCode,
ApplicationFailureCode: status.ApplicationFailureCode,
ProbeAgentID: status.ProbeAgentID,
UDPMode: string(target.UDPMode),
Port: target.Port,
Path: target.Path,
Enabled: target.Enabled,
Available: status.Available,
LastChecked: timePointerIfSet(status.LastChecked),
LastSuccess: timePointerIfSet(status.LastSuccess),
LatencyMillis: status.LatencyMillis,
ConsecutiveFailures: status.ConsecutiveFailures,
LastError: status.LastError,
FailureThreshold: target.EffectiveFailureThreshold(),
PollIntervalSeconds: target.EffectivePollIntervalSecs(),
TimeoutMillis: target.EffectiveTimeoutMillis(),
}
data.CertificateMonitoring = target.CertificateMonitoringEnabled()
data.CertificateExpiryWarningDays = target.EffectiveCertificateExpiryWarningDays()
+53 -17
View File
@@ -26,14 +26,16 @@ const availabilityProbeStaleError = "no recent report from probe agent"
// ProbeAvailabilityResult is one availability observation reported by a remote
// host agent that owns the target's execution.
type ProbeAvailabilityResult struct {
ObservationID string
TargetID string
ConfigRevision int64
Outcome availabilityprobe.Outcome
LatencyMillis int64
CheckedAt time.Time
Error string
Certificate *tlsutil.CertificateObservation
ObservationID string
TargetID string
ConfigRevision int64
Outcome availabilityprobe.Outcome
TransportOutcome availabilityprobe.Outcome
Application *availabilityprobe.ApplicationResult
LatencyMillis int64
CheckedAt time.Time
Error string
Certificate *tlsutil.CertificateObservation
}
// availabilityProbeAssignmentTracker provides a grace reference for a newly
@@ -60,15 +62,23 @@ func probeAvailabilityResultsFromReport(reported []agentshost.AvailabilityProbeR
default:
outcome = availabilityprobe.OutcomeIndeterminate
}
transportOutcome := availabilityprobe.Outcome(strings.ToLower(strings.TrimSpace(entry.TransportOutcome)))
switch transportOutcome {
case availabilityprobe.OutcomeReachable, availabilityprobe.OutcomeUnreachable, availabilityprobe.OutcomeIndeterminate:
default:
transportOutcome = outcome
}
results = append(results, ProbeAvailabilityResult{
ObservationID: strings.TrimSpace(entry.ObservationID),
TargetID: strings.TrimSpace(entry.TargetID),
ConfigRevision: entry.ConfigRevision,
Outcome: outcome,
LatencyMillis: entry.LatencyMillis,
CheckedAt: entry.CheckedAt,
Error: strings.TrimSpace(entry.Error),
Certificate: entry.Certificate.Clone(),
ObservationID: strings.TrimSpace(entry.ObservationID),
TargetID: strings.TrimSpace(entry.TargetID),
ConfigRevision: entry.ConfigRevision,
Outcome: outcome,
TransportOutcome: transportOutcome,
Application: applicationResultFromReport(entry),
LatencyMillis: entry.LatencyMillis,
CheckedAt: entry.CheckedAt,
Error: strings.TrimSpace(entry.Error),
Certificate: entry.Certificate.Clone(),
})
}
return results
@@ -157,7 +167,7 @@ func (m *Monitor) applyProbeAvailabilityResultsAt(hostID string, results []Probe
if observationID == "" {
observationID = legacyProbeAvailabilityObservationID(hostID, result)
}
m.applyAvailabilityObservation(target, observationID, checkedAt.UTC(), latency, outcome, probeErr, result.Certificate, hostID, receivedAt)
m.applyAvailabilityObservationDetailed(target, observationID, checkedAt.UTC(), latency, outcome, result.TransportOutcome, result.Application, probeErr, result.Certificate, hostID, receivedAt)
applied++
}
@@ -167,6 +177,26 @@ func (m *Monitor) applyProbeAvailabilityResultsAt(hostID string, results []Probe
m.updateResourceStore(m.GetState())
}
func applicationResultFromReport(entry agentshost.AvailabilityProbeResult) *availabilityprobe.ApplicationResult {
outcome := availabilityprobe.ApplicationOutcome(strings.ToLower(strings.TrimSpace(entry.ApplicationOutcome)))
switch outcome {
case availabilityprobe.ApplicationNotConfigured, availabilityprobe.ApplicationPassed, availabilityprobe.ApplicationFailed:
default:
return nil
}
failureCode := strings.TrimSpace(entry.ApplicationFailureCode)
switch failureCode {
case "", "status_mismatch", "response_read_failed", "response_too_large", "text_mismatch", "json_invalid", "json_path_missing", "json_value_mismatch":
default:
failureCode = ""
}
statusCode := entry.ApplicationStatusCode
if statusCode < 100 || statusCode > 599 {
statusCode = 0
}
return &availabilityprobe.ApplicationResult{Outcome: outcome, StatusCode: statusCode, FailureCode: failureCode}
}
func legacyProbeAvailabilityObservationID(hostID string, result ProbeAvailabilityResult) string {
material := fmt.Sprintf("%s\x00%s\x00%d\x00%s\x00%d\x00%s",
strings.TrimSpace(hostID), strings.TrimSpace(result.TargetID), result.CheckedAt.UTC().UnixNano(),
@@ -324,5 +354,11 @@ func availabilityProbeAgentTargetPayload(target config.AvailabilityTarget) map[s
if target.UDPExpected != "" {
payload["udpExpectedResponse"] = target.UDPExpected
}
if target.HTTP != nil {
// Assigned agents need the complete execution contract. These values are
// delivered only through authenticated remote config and are never echoed
// in the result/report payload.
payload["http"] = target.HTTP
}
return payload
}
@@ -755,7 +755,7 @@ func TestApplyHostReportIngestsAssignedAvailabilityResults(t *testing.T) {
Platform: "linux",
},
AvailabilityResults: []agentshost.AvailabilityProbeResult{
{TargetID: "remote", Outcome: "reachable", LatencyMillis: 17, CheckedAt: checkedAt},
{TargetID: "remote", Outcome: "reachable", TransportOutcome: "reachable", ApplicationOutcome: "passed", ApplicationStatusCode: 204, LatencyMillis: 17, CheckedAt: checkedAt},
{TargetID: "foreign", Outcome: "reachable", LatencyMillis: 3, CheckedAt: checkedAt},
},
Timestamp: checkedAt,
@@ -780,6 +780,9 @@ func TestApplyHostReportIngestsAssignedAvailabilityResults(t *testing.T) {
if status.ProbeAgentID != "probe-host" {
t.Fatalf("probe agent attribution = %q, want probe-host", status.ProbeAgentID)
}
if status.TransportOutcome != "reachable" || status.ApplicationOutcome != "passed" || status.ApplicationStatusCode != 204 {
t.Fatalf("application evidence = %+v, want reachable transport and passed HTTP 204", status)
}
if !status.LastChecked.Equal(checkedAt) {
t.Fatalf("last checked = %v, want %v", status.LastChecked, checkedAt)
}
@@ -740,18 +740,18 @@ func TestAgentlessAvailabilityTargetKindStaysCanonical(t *testing.T) {
"LinkedResourceID string `json:\"linkedResourceId,omitempty\"`",
},
filepath.Join("..", "monitoring", "availability_poller.go"): {
"TargetKind string `json:\"targetKind,omitempty\"`",
"TargetKind string `json:\"targetKind,omitempty\"`",
"TargetKind: string(target.TargetKind),",
"TargetKind: string(target.TargetKind),",
"TargetKind: string(target.TargetKind),",
"tags = append(tags, string(target.TargetKind))",
"LinkedResourceID: strings.TrimSpace(target.LinkedResourceID),",
"LinkedResourceID: strings.TrimSpace(target.LinkedResourceID),",
},
"types.go": {
"Availability *AvailabilityData `json:\"availability,omitempty\"`",
"TargetKind string `json:\"targetKind,omitempty\"`",
"LinkedResourceID string `json:\"linkedResourceId,omitempty\"`",
"LastChecked *time.Time `json:\"lastChecked,omitempty\"`",
"LastSuccess *time.Time `json:\"lastSuccess,omitempty\"`",
"TargetKind string `json:\"targetKind,omitempty\"`",
"LinkedResourceID string `json:\"linkedResourceId,omitempty\"`",
"LastChecked *time.Time `json:\"lastChecked,omitempty\"`",
"LastSuccess *time.Time `json:\"lastSuccess,omitempty\"`",
},
filepath.Join("..", "..", "frontend-modern", "src", "api", "availabilityTargets.ts"): {
"export type AvailabilityTargetKind = 'machine' | 'service' | 'device';",
+30 -26
View File
@@ -1740,32 +1740,36 @@ const (
// AvailabilityData contains agentless endpoint probe metadata for a resource.
type AvailabilityData struct {
TargetID string `json:"targetId,omitempty"`
LinkedResourceID string `json:"linkedResourceId,omitempty"`
Name string `json:"name,omitempty"`
TargetKind string `json:"targetKind,omitempty"`
Address string `json:"address,omitempty"`
Protocol string `json:"protocol,omitempty"`
ProbeOutcome string `json:"probeOutcome,omitempty"`
ProbeAgentID string `json:"probeAgentId,omitempty"`
UDPMode string `json:"udpMode,omitempty"`
Port int `json:"port,omitempty"`
Path string `json:"path,omitempty"`
Enabled bool `json:"enabled"`
Available bool `json:"available"`
LastChecked *time.Time `json:"lastChecked,omitempty"`
LastSuccess *time.Time `json:"lastSuccess,omitempty"`
LatencyMillis int64 `json:"latencyMillis,omitempty"`
ConsecutiveFailures int `json:"consecutiveFailures,omitempty"`
LastError string `json:"lastError,omitempty"`
FailureThreshold int `json:"failureThreshold,omitempty"`
PollIntervalSeconds int `json:"pollIntervalSeconds,omitempty"`
TimeoutMillis int `json:"timeoutMillis,omitempty"`
CorrelationState AvailabilityCorrelationState `json:"correlationState,omitempty"`
CorrelationRule string `json:"correlationRule,omitempty"`
CorrelationReason string `json:"correlationReason,omitempty"`
CorrelationCandidates int `json:"correlationCandidates,omitempty"`
Evidence *operationaltrust.EvidenceEnvelope `json:"evidence,omitempty"`
TargetID string `json:"targetId,omitempty"`
LinkedResourceID string `json:"linkedResourceId,omitempty"`
Name string `json:"name,omitempty"`
TargetKind string `json:"targetKind,omitempty"`
Address string `json:"address,omitempty"`
Protocol string `json:"protocol,omitempty"`
ProbeOutcome string `json:"probeOutcome,omitempty"`
TransportOutcome string `json:"transportOutcome,omitempty"`
ApplicationOutcome string `json:"applicationOutcome,omitempty"`
ApplicationStatusCode int `json:"applicationStatusCode,omitempty"`
ApplicationFailureCode string `json:"applicationFailureCode,omitempty"`
ProbeAgentID string `json:"probeAgentId,omitempty"`
UDPMode string `json:"udpMode,omitempty"`
Port int `json:"port,omitempty"`
Path string `json:"path,omitempty"`
Enabled bool `json:"enabled"`
Available bool `json:"available"`
LastChecked *time.Time `json:"lastChecked,omitempty"`
LastSuccess *time.Time `json:"lastSuccess,omitempty"`
LatencyMillis int64 `json:"latencyMillis,omitempty"`
ConsecutiveFailures int `json:"consecutiveFailures,omitempty"`
LastError string `json:"lastError,omitempty"`
FailureThreshold int `json:"failureThreshold,omitempty"`
PollIntervalSeconds int `json:"pollIntervalSeconds,omitempty"`
TimeoutMillis int `json:"timeoutMillis,omitempty"`
CorrelationState AvailabilityCorrelationState `json:"correlationState,omitempty"`
CorrelationRule string `json:"correlationRule,omitempty"`
CorrelationReason string `json:"correlationReason,omitempty"`
CorrelationCandidates int `json:"correlationCandidates,omitempty"`
Evidence *operationaltrust.EvidenceEnvelope `json:"evidence,omitempty"`
CertificateMonitoring bool `json:"certificateMonitoring,omitempty"`
CertificateExpiryWarningDays int `json:"certificateExpiryWarningDays,omitempty"`
+12 -8
View File
@@ -136,14 +136,18 @@ type ProxmoxLXCContainer struct {
// ("reachable", "unreachable", "indeterminate"); anything else is treated as
// indeterminate by the server.
type AvailabilityProbeResult struct {
ObservationID string `json:"observationId,omitempty"`
TargetID string `json:"targetId"`
ConfigRevision int64 `json:"configRevision,omitempty"`
Outcome string `json:"outcome"`
LatencyMillis int64 `json:"latencyMillis"`
CheckedAt time.Time `json:"checkedAt"`
Error string `json:"error,omitempty"`
Certificate *tlsutil.CertificateObservation `json:"certificate,omitempty"`
ObservationID string `json:"observationId,omitempty"`
TargetID string `json:"targetId"`
ConfigRevision int64 `json:"configRevision,omitempty"`
Outcome string `json:"outcome"`
TransportOutcome string `json:"transportOutcome,omitempty"`
ApplicationOutcome string `json:"applicationOutcome,omitempty"`
ApplicationStatusCode int `json:"applicationStatusCode,omitempty"`
ApplicationFailureCode string `json:"applicationFailureCode,omitempty"`
LatencyMillis int64 `json:"latencyMillis"`
CheckedAt time.Time `json:"checkedAt"`
Error string `json:"error,omitempty"`
Certificate *tlsutil.CertificateObservation `json:"certificate,omitempty"`
}
// ClusterNodeSensors contains temperature sensor data collected from a Proxmox