Measure node connection test outcomes

Telemetry could see only saved connections, so an install that tried to
reach a node and could not was indistinguishable from one that never
opened the add-node dialog. Both report zero configured connections and
stall at the same activation stage. Fleet data shows that population is
real and concentrated three to one in container deployments, and nothing
recorded whether those installs attempted a connection at all.

Record node connection test attempts and failures in a bounded,
day-bucketed tally in the config directory, pruned to a 31-day retention
window, and report both over the install-ID rotation window as
node_test_attempts_30d and node_test_failures_30d.

Recording starts only once a request carries a target and credentials, so
an incomplete form is never counted as a node that could not be reached.
A host string that turns out to be unusable does count, because the
attempt was made and it failed. Only the add-node dialog endpoint is
instrumented: instrumenting the unused test-config endpoint as well would
double-count a single operator action.

The tally holds counts alone. Hosts, credentials, and error text never
enter it, which is why it is plain JSON rather than encrypted history.
This commit is contained in:
rcourtman
2026-08-24 10:17:17 +01:00
parent a3f25507bf
commit 9e37d629ac
16 changed files with 645 additions and 1 deletions
+2
View File
@@ -120,6 +120,8 @@ Every field is listed below with the reason it exists. Nothing else is included
| Update successes 30d | `1` | Count successful update attempts in the current 30-day telemetry window |
| Update failures 30d | `1` | Count failed or rolled-back update attempts in the current 30-day telemetry window without sending raw errors, logs, URLs, or command output |
| Update last failure category | `download` | Send only a coarse category for the latest update failure, such as `download`, `signature`, `checksum`, `disk_space`, `extract`, `backup`, `apply`, `restart`, `rolled_back`, or `unknown` |
| Node test attempts 30d | `3` | Count node connection tests that reached the connection stage in the current 30-day telemetry window, without sending hostnames, addresses, credentials, or error text |
| Node test failures 30d | `2` | Count those connection tests that could not reach or authenticate against the target, so an install that tried to add a node and failed is distinguishable from one that never attempted it, without sending hostnames, addresses, credentials, or error text |
| Pulse Intelligence loop configured | `true`/`false` | See whether Assistant, Patrol, governed actions, or external-agent access is configured so adoption can be measured without sending configuration details |
| Pulse Intelligence loop active 30d | `true`/`false` | See whether Assistant, Patrol, external-agent, or governed-action activity occurred in the current 30-day telemetry window |
| Pulse Intelligence complete operations loop 30d | `true`/`false` | See whether Patrol issue activity reached an approved or rejected governed-action decision without sending prompts, findings, resource identifiers, command text, or action output |
@@ -2286,6 +2286,13 @@ the `/api/recovery/points` or `/api/recovery/rollups` list meta; lifecycle
code MUST NOT treat that pagination meta as an agent enrollment, admission, or
report-ingestion contract.
The node connection test handlers under `internal/api/configapi/` record a
content-free outcome tally for telemetry. This adds no agent lifecycle surface:
it does not touch agent registration, capability negotiation, command dispatch,
token rotation, or removal state, and it records nothing about agents. Node
connection tests target Proxmox, PBS, and PMG endpoints directly and never
traverse an agent.
## Forbidden Paths
1. New install or update continuity behavior hidden only inside broad monitoring ownership.
@@ -3325,6 +3325,16 @@ and MUST NOT reconstruct it from `aggregations.bySource`. Proof:
`TestContract_ResourceListReportsPlatformAdmission` and the wire-shape pin in
`frontend-modern/src/types/__tests__/resource.test.ts`.
`POST /api/config/nodes/test-connection` records a node connection test outcome
for telemetry. Recording begins only after the request is validated as carrying
a target and credentials, so a request rejected for incomplete input is never
counted as a node that could not be reached. Recording wraps the response
writer and must not alter the response contract: status codes and sanitized
error bodies stay exactly as they were. `POST /api/config/nodes/test-config` is
deliberately not instrumented, because instrumenting both endpoints would
double-count a single operator action and corrupt the failure share the
counters exist to measure.
## Forbidden Paths
1. Handler-local payload shape drift without a contract test
@@ -568,6 +568,17 @@ the `white_label` branding entitlement.
configuration has no key for this boundary and must not gain one that
can re-enable credential reads a local operator disabled.
Node connection test telemetry reports two counters over the install-ID
rotation window: `node_test_attempts_30d` and `node_test_failures_30d`. They
count only tests that carried a target and credentials, and they carry counts
alone. Hostnames, addresses, ports, credentials, fingerprints, and error text
must never enter the tally or the outbound ping. The tally is stored as plain
JSON in the config directory precisely because it holds no secret material;
adding any field that identifies a target would invalidate that storage choice
and require the encrypted history path instead. Both fields must stay disclosed
in `docs/PRIVACY.md` and `frontend-modern/public/docs/PRIVACY.md`, which
`TestAllTelemetryFieldsAreDisclosed` enforces.
## Forbidden Paths
1. Changing telemetry payload semantics without updating the canonical privacy disclosure.
@@ -1857,6 +1857,13 @@ serving page size that does not move the shared handler bounds re-opens the
`totalPages` misreport that silently truncates rollup iteration for clients
walking the reported page count.
The node connection test tally persists as a bounded, day-bucketed JSON file in
the config directory, pruned to a 31-day retention window so it cannot grow
without limit however long an install runs. It is advisory telemetry state: it
is outside backup, restore, and recovery, and a missing or corrupt tally must be
discarded and rebuilt rather than surfaced as a recovery failure. Nothing may
take a correctness dependency on its contents.
## Forbidden Paths
1. Reintroducing storage or recovery product logic as ad hoc dashboard-only summaries without a canonical page-surface owner
+2
View File
@@ -120,6 +120,8 @@ Every field is listed below with the reason it exists. Nothing else is included
| Update successes 30d | `1` | Count successful update attempts in the current 30-day telemetry window |
| Update failures 30d | `1` | Count failed or rolled-back update attempts in the current 30-day telemetry window without sending raw errors, logs, URLs, or command output |
| Update last failure category | `download` | Send only a coarse category for the latest update failure, such as `download`, `signature`, `checksum`, `disk_space`, `extract`, `backup`, `apply`, `restart`, `rolled_back`, or `unknown` |
| Node test attempts 30d | `3` | Count node connection tests that reached the connection stage in the current 30-day telemetry window, without sending hostnames, addresses, credentials, or error text |
| Node test failures 30d | `2` | Count those connection tests that could not reach or authenticate against the target, so an install that tried to add a node and failed is distinguishable from one that never attempted it, without sending hostnames, addresses, credentials, or error text |
| Pulse Intelligence loop configured | `true`/`false` | See whether Assistant, Patrol, governed actions, or external-agent access is configured so adoption can be measured without sending configuration details |
| Pulse Intelligence loop active 30d | `true`/`false` | See whether Assistant, Patrol, external-agent, or governed-action activity occurred in the current 30-day telemetry window |
| Pulse Intelligence complete operations loop 30d | `true`/`false` | See whether Patrol issue activity reached an approved or rejected governed-action decision without sending prompts, findings, resource identifiers, command text, or action output |
@@ -770,6 +770,43 @@ func (h *ConfigHandlers) handleAddNode(w http.ResponseWriter, r *http.Request) {
// HandleTestConnection tests a node connection without saving
// nodeTestOutcomeRecorder captures the status a node connection test responded
// with, so the outcome can be tallied once without threading a result through
// every branch of the handler.
type nodeTestOutcomeRecorder struct {
http.ResponseWriter
status int
}
func newNodeTestOutcomeRecorder(w http.ResponseWriter) *nodeTestOutcomeRecorder {
return &nodeTestOutcomeRecorder{ResponseWriter: w, status: http.StatusOK}
}
func (rec *nodeTestOutcomeRecorder) WriteHeader(status int) {
rec.status = status
rec.ResponseWriter.WriteHeader(status)
}
// failed reports whether the test resolved to anything other than success. The
// success paths write a body without an explicit status, which is why the
// recorder starts at 200.
func (rec *nodeTestOutcomeRecorder) failed() bool {
return rec.status < 200 || rec.status >= 300
}
// recordNodeTestOutcome tallies one node connection test for telemetry. The
// tally is advisory, so a write failure is logged and never surfaced to the
// caller: a broken counter must not break a connection test.
func (h *ConfigHandlers) recordNodeTestOutcome(ctx context.Context, failed bool) {
persistence := h.getPersistence(ctx)
if persistence == nil {
return
}
if err := persistence.RecordNodeTestOutcome(failed, time.Now()); err != nil {
log.Warn().Err(err).Msg("Failed to record node connection test outcome")
}
}
func (h *ConfigHandlers) handleTestConnection(w http.ResponseWriter, r *http.Request) {
// Limit request body to 32KB to prevent memory exhaustion
r.Body = http.MaxBytesReader(w, r.Body, 32*1024)
@@ -851,6 +888,16 @@ func (h *ConfigHandlers) handleTestConnection(w http.ResponseWriter, r *http.Req
return
}
// Past this point the request carries a target and credentials, so it is a
// real attempt by someone to reach a node. Requests that never carried them
// are rejected above and stay out of the tally, which is what keeps an
// incomplete form from counting as a node that could not be reached. A host
// string that turns out to be unusable still counts: the attempt was made
// and it failed.
testRecorder := newNodeTestOutcomeRecorder(w)
w = testRecorder
defer func() { h.recordNodeTestOutcome(r.Context(), testRecorder.failed()) }()
// Test connection based on type
if req.Type == "pve" {
host := normalizedHost
@@ -0,0 +1,118 @@
package configapi
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
)
func nodeTestTallyCounts(t *testing.T, dataPath string) (attempts int, failures int) {
t.Helper()
tally, err := config.NewConfigPersistence(dataPath).LoadNodeTestTally()
if err != nil {
t.Fatalf("LoadNodeTestTally: %v", err)
}
since := time.Now().UTC().AddDate(0, 0, -30)
return tally.AttemptsSince(since), tally.FailuresSince(since)
}
func postNodeConnectionTest(t *testing.T, h *ConfigHandlers, body map[string]string) {
t.Helper()
encoded, err := json.Marshal(body)
if err != nil {
t.Fatalf("marshal request: %v", err)
}
req := httptest.NewRequest("POST", "/api/config/nodes/test-connection", bytes.NewBuffer(encoded))
h.HandleTestConnection(httptest.NewRecorder(), req)
}
// A request that never carried a target and credentials was not an attempt to
// reach a node, so counting it would inflate the failure share this counter
// exists to measure.
func TestNodeTestTallyIgnoresRequestsWithoutATarget(t *testing.T) {
dataPath := t.TempDir()
h := newTestConfigHandlers(t, &config.Config{DataPath: dataPath})
for _, body := range []map[string]string{
{"type": "pve"},
{"type": "pve", "host": "10.0.0.1"},
{"type": "unknown", "host": "10.0.0.1", "user": "root@pam", "password": "x"},
} {
postNodeConnectionTest(t, h, body)
}
attempts, failures := nodeTestTallyCounts(t, dataPath)
if attempts != 0 || failures != 0 {
t.Fatalf("targetless requests tallied: attempts=%d failures=%d, want 0 and 0", attempts, failures)
}
}
// The other side of that boundary: someone who typed an unusable host and got
// an error did attempt to reach a node, and that attempt failed. Excluding it
// would hide exactly the population this counter exists to find.
func TestNodeTestTallyCountsUnusableHostAsFailedAttempt(t *testing.T) {
dataPath := t.TempDir()
h := newTestConfigHandlers(t, &config.Config{DataPath: dataPath})
postNodeConnectionTest(t, h, map[string]string{
"type": "pve",
"host": "://invalid-url",
"user": "root@pam",
"password": "password",
})
attempts, failures := nodeTestTallyCounts(t, dataPath)
if attempts != 1 || failures != 1 {
t.Fatalf("attempts=%d failures=%d, want 1 and 1", attempts, failures)
}
}
// An unreachable target is the case the counter exists for. proxmox.NewClient
// authenticates eagerly, so this fails during client construction rather than
// on a later call, and it must still be counted.
func TestNodeTestTallyCountsUnreachableTargetAsFailure(t *testing.T) {
dataPath := t.TempDir()
h := newTestConfigHandlers(t, &config.Config{DataPath: dataPath})
postNodeConnectionTest(t, h, map[string]string{
"type": "pve",
"host": "127.0.0.1:1",
"user": "root@pam",
"password": "password",
})
attempts, failures := nodeTestTallyCounts(t, dataPath)
if attempts != 1 {
t.Fatalf("attempts = %d, want 1", attempts)
}
if failures != 1 {
t.Fatalf("failures = %d, want 1", failures)
}
}
// The success paths write a body without setting a status explicitly, so the
// recorder must treat an unset status as success rather than failure. Driving a
// real successful test would need a live Proxmox endpoint, so the mapping is
// asserted directly.
func TestNodeTestOutcomeRecorderTreatsImplicitStatusAsSuccess(t *testing.T) {
rec := newNodeTestOutcomeRecorder(httptest.NewRecorder())
if rec.failed() {
t.Fatal("implicit 200 reported as failure")
}
if _, err := rec.Write([]byte(`{"status":"success"}`)); err != nil {
t.Fatalf("write: %v", err)
}
if rec.failed() {
t.Fatal("body write without WriteHeader reported as failure")
}
rec.WriteHeader(http.StatusBadRequest)
if !rec.failed() {
t.Fatal("400 not reported as failure")
}
}
+40
View File
@@ -24432,3 +24432,43 @@ func TestContract_PatrolBlockedCauseTelemetryExportsEnumOnlyWhileBlocked(t *test
}
}
}
// The node connection test tally is only meaningful if it starts after the
// request is known to carry a target and credentials. Moving the recorder above
// those guards would silently begin counting incomplete forms as nodes that
// could not be reached, which is the exact confusion the counter exists to
// resolve, and nothing about the response would change to reveal it.
func TestContract_NodeTestTelemetryRecordsOnlyAfterTargetValidation(t *testing.T) {
source, err := os.ReadFile(filepath.Clean("configapi/config_node_handlers.go"))
if err != nil {
t.Fatalf("read node handlers: %v", err)
}
handlers := string(source)
start := strings.Index(handlers, "func (h *ConfigHandlers) handleTestConnection(")
if start < 0 {
t.Fatal("handleTestConnection not found")
}
body := handlers[start:]
if end := strings.Index(body[1:], "\nfunc "); end >= 0 {
body = body[:end+1]
}
recorderAt := strings.Index(body, "testRecorder := newNodeTestOutcomeRecorder(w)")
if recorderAt < 0 {
t.Fatal("handleTestConnection no longer records node connection test outcomes")
}
for _, guard := range []string{
`http.Error(w, "Host is required", http.StatusBadRequest)`,
`http.Error(w, "Invalid node type", http.StatusBadRequest)`,
`http.Error(w, "Authentication credentials required", http.StatusBadRequest)`,
} {
guardAt := strings.Index(body, guard)
if guardAt < 0 {
t.Fatalf("validation guard %q missing from handleTestConnection", guard)
}
if guardAt > recorderAt {
t.Fatalf("validation guard %q must run before the node connection test tally starts", guard)
}
}
}
+181
View File
@@ -0,0 +1,181 @@
package config
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"time"
"github.com/rs/zerolog/log"
)
const (
nodeTestTallyFileName = "node_test_tally.json"
nodeTestTallyVersion = 1
// nodeTestTallyRetentionDays bounds the tally. It exceeds the thirty-day
// telemetry window by a day so the day the window opens on is still
// present when the tally is read.
nodeTestTallyRetentionDays = 31
)
// NodeTestTallyData counts node connection tests per UTC day. Telemetry can
// otherwise see only the saved-connection count, so an install that tried to
// reach a node and failed is indistinguishable from one that never opened the
// add-node dialog. Both report zero connections and stall at the "secured"
// activation stage. Two day-bucketed integers separate them and cost one small
// map entry per retained day.
//
// The tally holds counts only. Hosts, credentials, and error text never enter
// it, which is why it is stored as plain JSON alongside the encrypted
// configuration rather than through the encrypted history helpers.
type NodeTestTallyData struct {
Version int `json:"version"`
LastSaved time.Time `json:"last_saved"`
// DailyAttempts counts tests that reached the connection stage, meaning
// the request carried a well-formed target and credentials.
DailyAttempts map[string]int `json:"daily_attempts,omitempty"`
// DailyFailures counts the subset of those attempts that could not reach
// or authenticate against the target. Successes are the remainder.
DailyFailures map[string]int `json:"daily_failures,omitempty"`
}
// NodeTestTallyDayKey renders the UTC day key used by NodeTestTallyData.
func NodeTestTallyDayKey(at time.Time) string {
return at.UTC().Format("2006-01-02")
}
func nodeTestTallyCountSince(daily map[string]int, since time.Time) int {
if len(daily) == 0 {
return 0
}
sinceDay := NodeTestTallyDayKey(since)
total := 0
for day, count := range daily {
if day >= sinceDay {
total += count
}
}
return total
}
// AttemptsSince counts node connection tests observed at or after since. The
// tally is day-granular, so the day containing since is counted whole.
func (data *NodeTestTallyData) AttemptsSince(since time.Time) int {
if data == nil {
return 0
}
return nodeTestTallyCountSince(data.DailyAttempts, since)
}
// FailuresSince counts failed node connection tests observed at or after since.
func (data *NodeTestTallyData) FailuresSince(since time.Time) int {
if data == nil {
return 0
}
return nodeTestTallyCountSince(data.DailyFailures, since)
}
// pruneNodeTestTally drops days outside the retention window so the tally stays
// bounded no matter how long an install runs.
func pruneNodeTestTally(data *NodeTestTallyData, now time.Time) {
if data == nil {
return
}
cutoff := NodeTestTallyDayKey(now.UTC().AddDate(0, 0, -nodeTestTallyRetentionDays))
for day := range data.DailyAttempts {
if day < cutoff {
delete(data.DailyAttempts, day)
}
}
for day := range data.DailyFailures {
if day < cutoff {
delete(data.DailyFailures, day)
}
}
}
func (c *ConfigPersistence) nodeTestTallyPath() string {
return filepath.Join(c.configDir, nodeTestTallyFileName)
}
// loadNodeTestTallyLocked reads the tally with c.mu already held. A missing,
// empty, or corrupt file yields an empty tally rather than an error: the
// counters are advisory telemetry and must never block a connection test.
func (c *ConfigPersistence) loadNodeTestTallyLocked() *NodeTestTallyData {
data := &NodeTestTallyData{Version: nodeTestTallyVersion}
if raw, err := c.fs.ReadFile(c.nodeTestTallyPath()); err == nil && len(raw) > 0 {
decoded := &NodeTestTallyData{}
if err := json.Unmarshal(raw, decoded); err == nil {
data = decoded
} else {
log.Warn().Err(err).Msg("Discarding unreadable node test tally")
}
}
if data.DailyAttempts == nil {
data.DailyAttempts = make(map[string]int, nodeTestTallyRetentionDays)
}
if data.DailyFailures == nil {
data.DailyFailures = make(map[string]int, nodeTestTallyRetentionDays)
}
return data
}
// LoadNodeTestTally returns the persisted node connection test tally.
func (c *ConfigPersistence) LoadNodeTestTally() (*NodeTestTallyData, error) {
if c == nil || c.configDir == "" {
return &NodeTestTallyData{}, nil
}
c.mu.RLock()
defer c.mu.RUnlock()
return c.loadNodeTestTallyLocked(), nil
}
// RecordNodeTestOutcome folds one node connection test into the daily tally.
// Callers report only tests that reached the connection stage, so a failure
// always means the target could not be reached or authenticated rather than
// that the form was incomplete.
func (c *ConfigPersistence) RecordNodeTestOutcome(failed bool, now time.Time) error {
if c == nil || c.configDir == "" {
return nil
}
if now.IsZero() {
now = time.Now()
}
now = now.UTC()
c.mu.Lock()
defer c.mu.Unlock()
data := c.loadNodeTestTallyLocked()
day := NodeTestTallyDayKey(now)
data.DailyAttempts[day]++
if failed {
data.DailyFailures[day]++
}
pruneNodeTestTally(data, now)
data.Version = nodeTestTallyVersion
data.LastSaved = now
encoded, err := json.Marshal(data)
if err != nil {
return fmt.Errorf("encode node test tally: %w", err)
}
path := c.nodeTestTallyPath()
if err := c.fs.MkdirAll(filepath.Dir(path), 0o700); err != nil {
return fmt.Errorf("create node test tally directory: %w", err)
}
tmpPath := path + ".tmp"
if err := c.fs.WriteFile(tmpPath, encoded, 0o600); err != nil {
return fmt.Errorf("write temp node test tally: %w", err)
}
if err := c.fs.Rename(tmpPath, path); err != nil {
if removeErr := c.fs.Remove(tmpPath); removeErr != nil && !errors.Is(removeErr, os.ErrNotExist) {
log.Warn().Err(removeErr).Str("tmp_path", tmpPath).Msg("Failed to remove temporary node test tally after failed rename")
}
return fmt.Errorf("commit node test tally: %w", err)
}
return nil
}
+109
View File
@@ -0,0 +1,109 @@
package config_test
import (
"os"
"path/filepath"
"testing"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
)
func TestRecordNodeTestOutcomeCountsAttemptsAndFailures(t *testing.T) {
cp := config.NewConfigPersistence(t.TempDir())
now := time.Now().UTC()
for i := 0; i < 3; i++ {
if err := cp.RecordNodeTestOutcome(true, now); err != nil {
t.Fatalf("RecordNodeTestOutcome(failed) = %v", err)
}
}
if err := cp.RecordNodeTestOutcome(false, now); err != nil {
t.Fatalf("RecordNodeTestOutcome(success) = %v", err)
}
tally, err := cp.LoadNodeTestTally()
if err != nil {
t.Fatalf("LoadNodeTestTally = %v", err)
}
since := now.AddDate(0, 0, -30)
if got := tally.AttemptsSince(since); got != 4 {
t.Fatalf("AttemptsSince = %d, want 4", got)
}
if got := tally.FailuresSince(since); got != 3 {
t.Fatalf("FailuresSince = %d, want 3", got)
}
}
// A success must never be counted as a failure, otherwise the failure share
// this counter exists to measure is meaningless.
func TestRecordNodeTestOutcomeSuccessLeavesFailuresZero(t *testing.T) {
cp := config.NewConfigPersistence(t.TempDir())
now := time.Now().UTC()
if err := cp.RecordNodeTestOutcome(false, now); err != nil {
t.Fatalf("RecordNodeTestOutcome = %v", err)
}
tally, err := cp.LoadNodeTestTally()
if err != nil {
t.Fatalf("LoadNodeTestTally = %v", err)
}
if got := tally.FailuresSince(now.AddDate(0, 0, -30)); got != 0 {
t.Fatalf("FailuresSince = %d, want 0", got)
}
if got := tally.AttemptsSince(now.AddDate(0, 0, -30)); got != 1 {
t.Fatalf("AttemptsSince = %d, want 1", got)
}
}
// Counts outside the reporting window must not leak into it, and days outside
// the retention window must be pruned so the file stays bounded.
func TestNodeTestTallyPrunesAndWindowsCorrectly(t *testing.T) {
cp := config.NewConfigPersistence(t.TempDir())
now := time.Now().UTC()
if err := cp.RecordNodeTestOutcome(true, now.AddDate(0, 0, -40)); err != nil {
t.Fatalf("RecordNodeTestOutcome(old) = %v", err)
}
if err := cp.RecordNodeTestOutcome(true, now.AddDate(0, 0, -10)); err != nil {
t.Fatalf("RecordNodeTestOutcome(recent) = %v", err)
}
if err := cp.RecordNodeTestOutcome(true, now); err != nil {
t.Fatalf("RecordNodeTestOutcome(now) = %v", err)
}
tally, err := cp.LoadNodeTestTally()
if err != nil {
t.Fatalf("LoadNodeTestTally = %v", err)
}
if got := tally.AttemptsSince(now.AddDate(0, 0, -30)); got != 2 {
t.Fatalf("AttemptsSince(30d) = %d, want 2", got)
}
if _, ok := tally.DailyAttempts[config.NodeTestTallyDayKey(now.AddDate(0, 0, -40))]; ok {
t.Fatalf("day outside retention window was not pruned")
}
}
// The counters are advisory. A corrupt tally must start over rather than make
// a connection test fail.
func TestRecordNodeTestOutcomeRecoversFromCorruptTally(t *testing.T) {
dir := t.TempDir()
cp := config.NewConfigPersistence(dir)
if err := os.WriteFile(filepath.Join(dir, "node_test_tally.json"), []byte("{not json"), 0o600); err != nil {
t.Fatalf("seed corrupt tally: %v", err)
}
now := time.Now().UTC()
if err := cp.RecordNodeTestOutcome(true, now); err != nil {
t.Fatalf("RecordNodeTestOutcome = %v", err)
}
tally, err := cp.LoadNodeTestTally()
if err != nil {
t.Fatalf("LoadNodeTestTally = %v", err)
}
if got := tally.AttemptsSince(now.AddDate(0, 0, -30)); got != 1 {
t.Fatalf("AttemptsSince = %d, want 1", got)
}
}
+20 -1
View File
@@ -160,7 +160,12 @@ const (
// is distinguishable in the fleet from one that runs and finds nothing.
// Field telemetry showed both present identically: high run counts with
// zero AI calls and zero findings.
TelemetrySchemaVersion = 10
// Schema v11 adds node connection test attempt and failure counts. The
// fleet shows a population that configures authentication and then never
// saves a connection, concentrated three to one in container deployments,
// and nothing recorded whether those installs tried to reach a node and
// failed or never attempted one at all.
TelemetrySchemaVersion = 11
)
type installIDRecord struct {
@@ -263,6 +268,16 @@ type Ping struct {
// Last coarse update failure category; never raw error text.
UpdateLastFailureCategory string `json:"update_last_failure_category,omitempty"`
// Node connection test outcomes over the install-ID rotation window.
// ConfiguredConnections counts only connections that were saved, so an
// install that tried to reach a node and could not is indistinguishable
// from one that never opened the add-node dialog: both report zero
// connections and stall at the "secured" activation stage. These separate
// the two. Counts only; hosts, credentials, and error text never leave the
// install. Successes are attempts minus failures.
NodeTestAttempts30d int `json:"node_test_attempts_30d"`
NodeTestFailures30d int `json:"node_test_failures_30d"`
// Core product outcomes. Alert history is retained locally for 30 days;
// notification delivery rows are locally retention-bounded to seven days.
AlertsFired30d int `json:"alerts_fired_30d"`
@@ -412,6 +427,8 @@ type Snapshot struct {
UpdateSuccesses30d int
UpdateFailures30d int
UpdateLastFailureCategory string
NodeTestAttempts30d int
NodeTestFailures30d int
AuthConfigured bool
ConfiguredConnections int
AlertsFired30d int
@@ -1019,6 +1036,8 @@ func applySnapshot(base Ping, fn SnapshotFunc) Ping {
ping.UpdateSuccesses30d = s.UpdateSuccesses30d
ping.UpdateFailures30d = s.UpdateFailures30d
ping.UpdateLastFailureCategory = s.UpdateLastFailureCategory
ping.NodeTestAttempts30d = s.NodeTestAttempts30d
ping.NodeTestFailures30d = s.NodeTestFailures30d
ping.AuthConfigured = s.AuthConfigured
ping.ConfiguredConnections = s.ConfiguredConnections
ping.AlertsFired30d = s.AlertsFired30d
+15
View File
@@ -1255,3 +1255,18 @@ func decodeInstallIDRecord(t *testing.T, data []byte) installIDRecord {
}
return record
}
// The node connection test counts are only useful if they leave the install, so
// assert the snapshot-to-ping mapping rather than trusting field order.
func TestBuildPingCarriesNodeTestCounts(t *testing.T) {
ping := BuildPingForSnapshot(Snapshot{
NodeTestAttempts30d: 9,
NodeTestFailures30d: 4,
})
if ping.NodeTestAttempts30d != 9 {
t.Fatalf("node_test_attempts_30d = %d, want 9", ping.NodeTestAttempts30d)
}
if ping.NodeTestFailures30d != 4 {
t.Fatalf("node_test_failures_30d = %d, want 4", ping.NodeTestFailures30d)
}
}
+1
View File
@@ -616,6 +616,7 @@ func Run(ctx context.Context, version string) error {
router.ApplyUpdateTelemetrySnapshot(&snap, now)
router.ApplyLicensedFeatureTelemetrySnapshot(&snap, now)
}
applyNodeTestTelemetrySnapshot(&snap, telemetryPersistence, now)
var actionSnapshot telemetry.PulseIntelligenceActionSnapshot
if router != nil {
actionSnapshot = router.GetPulseIntelligenceActionTelemetry(now.Add(-telemetry.PulseIntelligenceTelemetryWindow))
+40
View File
@@ -16,6 +16,7 @@ import (
"github.com/rcourtman/pulse-go-rewrite/internal/agentexec"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rcourtman/pulse-go-rewrite/internal/telemetry"
"github.com/rcourtman/pulse-go-rewrite/internal/updates"
"github.com/rcourtman/pulse-go-rewrite/pkg/extensions"
pkglicensing "github.com/rcourtman/pulse-go-rewrite/pkg/licensing"
@@ -394,3 +395,42 @@ func TestServerRun_RejectsWildcardTrustedProxyCIDR(t *testing.T) {
t.Fatalf("expected wildcard trusted proxy configuration to be rejected, got %v", err)
}
}
// Node connection test counts must reach the telemetry snapshot, and counts
// older than the reporting window must not, otherwise the add-node stall this
// counter exists to measure cannot be read from the fleet.
func TestApplyNodeTestTelemetrySnapshotReadsTallyWithinWindow(t *testing.T) {
dir := t.TempDir()
persistence := config.NewConfigPersistence(dir)
now := time.Now().UTC()
if err := persistence.RecordNodeTestOutcome(true, now.AddDate(0, 0, -40)); err != nil {
t.Fatalf("record stale outcome: %v", err)
}
if err := persistence.RecordNodeTestOutcome(true, now); err != nil {
t.Fatalf("record failure: %v", err)
}
if err := persistence.RecordNodeTestOutcome(false, now); err != nil {
t.Fatalf("record success: %v", err)
}
var snap telemetry.Snapshot
applyNodeTestTelemetrySnapshot(&snap, persistence, now)
if snap.NodeTestAttempts30d != 2 {
t.Fatalf("NodeTestAttempts30d = %d, want 2", snap.NodeTestAttempts30d)
}
if snap.NodeTestFailures30d != 1 {
t.Fatalf("NodeTestFailures30d = %d, want 1", snap.NodeTestFailures30d)
}
}
// A missing tally is the normal state on an install that has never opened the
// add-node dialog, and must report zero rather than fail the snapshot.
func TestApplyNodeTestTelemetrySnapshotToleratesMissingTally(t *testing.T) {
var snap telemetry.Snapshot
applyNodeTestTelemetrySnapshot(&snap, config.NewConfigPersistence(t.TempDir()), time.Now().UTC())
if snap.NodeTestAttempts30d != 0 || snap.NodeTestFailures30d != 0 {
t.Fatalf("counts = %d/%d, want 0/0", snap.NodeTestAttempts30d, snap.NodeTestFailures30d)
}
}
+35
View File
@@ -0,0 +1,35 @@
package server
import (
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/telemetry"
)
// applyNodeTestTelemetrySnapshot adds content-free node connection test counts
// from the local tally.
//
// ConfiguredConnections reports only connections that were saved, so without
// these an install that tried to reach a node and could not is indistinguishable
// from one that never opened the add-node dialog: both report zero connections
// and stall at the "secured" activation stage.
//
// The window is the shared install-ID rotation window, the same one the
// Pulse Intelligence counters use, so no counter outlives the pseudonymous
// identifier it is reported against.
func applyNodeTestTelemetrySnapshot(snap *telemetry.Snapshot, persistence *config.ConfigPersistence, now time.Time) {
if snap == nil || persistence == nil {
return
}
if now.IsZero() {
now = time.Now()
}
tally, err := persistence.LoadNodeTestTally()
if err != nil || tally == nil {
return
}
since := now.UTC().Add(-telemetry.PulseIntelligenceTelemetryWindow)
snap.NodeTestAttempts30d = tally.AttemptsSince(since)
snap.NodeTestFailures30d = tally.FailuresSince(since)
}