mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-25 21:20:00 +00:00
Count successful history reads beyond seven days (#2996)
Measure longer-history use before changing free and paid features. History and chart APIs count successfully written GET responses only when they return a metric point older than seven days. Empty, denied, failed and mock responses do not count. Cached responses and automatic refreshes do count. Only bounded daily totals stay locally. Schema v19 exports their rolling sum with no request, resource, metric, tenant or customer content. The payload preview and privacy disclosure match. The adoption report separates schema coverage from zero use and uses each reporting ID's latest count. Validation: affected Go packages, receiver telemetry tests, 44 report tests, schema parity, docs mirror and governance checks. Playwright exercised /settings/system-general and /docs/PRIVACY, including the visible counter at 1280x800 and 390x844, refresh and ID rotation. Receiver deployment and upgraded-client adoption are still needed before interpreting live usage.
This commit is contained in:
rcourtman
+20
-2
@@ -40,7 +40,7 @@ Every field is listed below with the reason it exists. Nothing else is included
|
||||
|
||||
| Field | Example | Purpose |
|
||||
|-------|---------|---------|
|
||||
| Schema version | `18` | Identify the exact payload contract so old and new signals are not mixed silently |
|
||||
| Schema version | `19` | Identify the exact payload contract so old and new signals are not mixed silently |
|
||||
| Sent at | `2026-07-23T08:30:00Z` | Date the individual heartbeat without sending a history of client activity |
|
||||
| Install ID | `a1b2c3d4-...` | Distinguish active installations within one rotation window without tying telemetry to an account or person |
|
||||
| Version | `6.0.0-rc.1` | Track the canonical release identity currently deployed |
|
||||
@@ -259,6 +259,7 @@ Every field is listed below with the reason it exists. Nothing else is included
|
||||
| Pulse Intelligence approved action refusals: other 30d | `1` | Count pre-dispatch refusals not covered by the fixed categories without sending raw error or action content |
|
||||
| Pulse Intelligence verified finding resolutions 30d | `1` | Count approved Patrol-origin actions whose execution succeeded and whose linked finding postcondition was independently confirmed, without sending finding IDs, action IDs, evidence, resources, or fix details |
|
||||
| Pulse Intelligence approved action last failure reason 30d | `plan_drift` | See one fixed machine reason code for the most recent approved-action failure in the current 30-day telemetry window without sending error text, action output, command text, resource IDs, or actors |
|
||||
| Successful history reads over 7d 30d (`successful_history_reads_over_7d_30d`) | `5` | Count successful GET responses for a history range longer than seven days that actually return a numeric metric point older than seven days. Includes successful cache hits, but excludes empty/recent-only results, denied requests, failed writes, cancelled requests, HEAD requests and synthetic data. Only a rolling count leaves the install, with no selected range, timestamp, metric value, resource, tenant, user or request identity |
|
||||
| Workload history preview sessions 30d | `18` | Count browser sessions that successfully displayed a populated workload history preview, deduplicated locally once per session without sending a guest, user, route, timing, or browser identifier |
|
||||
| Workload history scrub sessions 30d | `9` | Count browser sessions that inspected a point on a workload history graph, deduplicated locally once per session without sending coordinates, values, guests, or browser identity |
|
||||
| Workload history range change sessions 30d | `6` | Count browser sessions that changed the workload history range, deduplicated locally once per session without sending the selected range or browser identity |
|
||||
@@ -311,7 +312,7 @@ occurrence exists, leaving no trustworthy fired-alert denominator. Detected
|
||||
flapping episodes are not reported because their diagnostic event path may be
|
||||
dropped under pressure. Configuration adoption is reported instead.
|
||||
|
||||
The current telemetry contract is schema version 18. Schema v16 adds four
|
||||
The current telemetry contract is schema version 19. Schema v16 adds four
|
||||
workload-history adoption counts. The browser
|
||||
sends only one closed milestone name to the local Pulse server and deduplicates
|
||||
each milestone once per browser session. Pulse stores bounded UTC-day counts
|
||||
@@ -337,6 +338,22 @@ flag for whether it offered a newer release. They exist because a failed check
|
||||
never shows up in the update counters, which only see updates that were
|
||||
applied. No version, release URL, response, or error text is sent.
|
||||
|
||||
Schema v19 adds `successful_history_reads_over_7d_30d`. The server counts
|
||||
successfully written metric-history and chart responses containing a metric
|
||||
point older than seven days. Selecting a long range alone does not count.
|
||||
Empty results, denied requests, failed writes and mock data do not count.
|
||||
Pulse retains bounded UTC-day totals locally and sends only their rolling
|
||||
30-day sum, including the UTC day containing the cutoff. The day buckets
|
||||
are not exported. Requests and auto-refreshes count as reads, not people
|
||||
or proof of a human viewing a chart.
|
||||
|
||||
No customer identity, request URL, selected range, metric timestamp, metric
|
||||
value or infrastructure identifier is retained in this tally or sent. Older
|
||||
schemas cannot report this signal. Measurement starts after upgrading, so
|
||||
historical use cannot be reconstructed and an initial zero does not represent
|
||||
a full 30-day observation. Telemetry opt-out, installation-ID rotation,
|
||||
account-linking restrictions and 90-day receiver retention are unchanged.
|
||||
|
||||
#### Server-side handling and retention
|
||||
|
||||
- Telemetry pings are stored on the Pulse license server only for aggregate install/use analysis.
|
||||
@@ -389,6 +406,7 @@ Every change to the payload bumps the schema version, is listed here with its da
|
||||
|
||||
| Schema | Date | Change |
|
||||
|--------|------|--------|
|
||||
| 19 | 2026-10-16 | Content-free rolling count of successful metric-history responses that return data older than seven days |
|
||||
| 18 | 2026-09-27 | Effective update channel, closed outcome of the last update check on that channel, and whether it offered a newer release |
|
||||
| 17 | 2026-09-02 | Closed Patrol provider class, effective Patrol autonomy level, coarse 30-day Patrol token buckets, and per-outcome investigation counts |
|
||||
| 16 | 2026-08-30 | Four content-free workload-history adoption counters, each counted at most once per browser session |
|
||||
|
||||
@@ -5929,7 +5929,7 @@
|
||||
"status": "target-met",
|
||||
"completion": {
|
||||
"state": "complete",
|
||||
"summary": "Security, identity, and privacy is at the current governed floor: privacy disclosures, telemetry controls, RBAC boundaries, and shared token/auth settings surfaces now route through explicit subsystem ownership with exercised proof across the remaining shared token and auth boundaries.",
|
||||
"summary": "Security, identity, and privacy remains at the governed floor. Schema v19 adds only a content-free successful extended-history HTTP read count with private daily persistence, exact Preview disclosure, compatible receiver intake and explicit report coverage. Source and isolated browser proofs pass. Receiver deployment and client release adoption remain separate operational boundaries.",
|
||||
"tracking": []
|
||||
},
|
||||
"blockers": [],
|
||||
@@ -5988,6 +5988,11 @@
|
||||
"path": "internal/api/api_token_org_scope_integration_test.go",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "internal/api/chartapi/history_activity_test.go",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "internal/api/org_handlers_test.go",
|
||||
@@ -6013,15 +6018,40 @@
|
||||
"path": "internal/api/system_settings_telemetry_test.go",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "internal/api/telemetry_history_reads_test.go",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "internal/config/history_read_activity_test.go",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "internal/telemetry/telemetry.go",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "scripts/tests/test_telemetry_adoption_report.py",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "SECURITY.md",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse",
|
||||
"path": "tests/integration/tests/19-telemetry-disclosure.spec.ts",
|
||||
"kind": "file"
|
||||
},
|
||||
{
|
||||
"repo": "pulse-pro",
|
||||
"path": "license-server/telemetry_test.go",
|
||||
"kind": "file"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -9879,3 +9879,12 @@ controls establish no native guest restart, thaw or prior-workload restoration.
|
||||
existing update pipeline. They add no agent enrollment, command, install or
|
||||
reporting path, and agent continuity stays judged by the same update readiness
|
||||
verdict the apply route already computes for its target version.
|
||||
|
||||
### Extended history observation preserves existing read authority
|
||||
|
||||
The shared chart/history response writer now observes successful extended
|
||||
history delivery for the security-privacy schema-v19 aggregate. It adds no
|
||||
query, resource, tenant, credential or action authority. The observer receives
|
||||
only a successful-read marker after serialization/write, and existing API
|
||||
entitlement checks remain before that boundary. Read semantics remain owned
|
||||
by api-contracts and telemetry retention/disclosure by security-privacy.
|
||||
|
||||
@@ -12262,3 +12262,17 @@ The accepted upstream `diagnostics_token_visibility_test.go` controls and API
|
||||
reference route checks cover caller filtering and the documented authority.
|
||||
Final-source checks must retain this upstream behavior beside the reviewed
|
||||
reporting/help repairs; source proof is not installed credential recovery.
|
||||
|
||||
### Successful extended history HTTP reads
|
||||
|
||||
`chartapi.Service.WriteHistoryJSON` and `WriteHistoryPayload` share one
|
||||
post-write observation boundary for metric history and the main, workload,
|
||||
infrastructure and storage chart handlers, including cached bytes. Existing
|
||||
response bodies, range/entitlement checks and failure behavior remain the API
|
||||
contract. Only GET responses requested beyond seven days that contain a real
|
||||
returned numeric metric point strictly older than seven days qualify. Long
|
||||
empty windows, metadata timestamps, failed encoding/writes, cancellation,
|
||||
HEAD and mock responses cannot increment the tally. Mock and live cache keys
|
||||
are separated. The optional observer receives no request or content.
|
||||
`history_activity_test.go` and `telemetry_history_reads_test.go` exercise this
|
||||
boundary. Schema v19 count semantics and retention belong to security-privacy.
|
||||
|
||||
@@ -3692,3 +3692,15 @@ Mobile settings gate and Alerts push panel carry no upgrade prompt. The
|
||||
`relay` feature is labelled "Pulse Relay (Mobile Connection)" because it never
|
||||
provided remote access to the web UI. Pulse Mobile retires on 31 March 2027;
|
||||
until then licensed instances keep pairing and push unchanged.
|
||||
|
||||
### Extended history measurement does not establish customer identity
|
||||
|
||||
The license server accepts schema-v19 `successful_history_reads_over_7d_30d`
|
||||
as a nonnegative aggregate integer, migrates storage without rewriting older
|
||||
rows, and resets pre-v19 supplied values to zero. The public reporter treats
|
||||
older schemas or missing receiver columns as unmeasured. Current coarse
|
||||
`paid_license` posture is not a paying-subscription census and must not be
|
||||
joined to billing accounts. This adds no price, entitlement, refund or billing
|
||||
behavior. Receiver intake and migration tests pin the additive schema boundary.
|
||||
Deploying the compatible receiver and release adoption remain separate from
|
||||
source verification. Count and disclosure semantics belong to security-privacy.
|
||||
|
||||
@@ -15,6 +15,16 @@
|
||||
```
|
||||
|
||||
## Purpose
|
||||
### Schema-v19 release disclosure follows receiver compatibility
|
||||
|
||||
`docs/releases/TELEMETRY_SCHEMA_V19.md` carries the extended-history telemetry notice for
|
||||
inclusion in the first release containing schema v19. Source landing neither
|
||||
deploys the compatible license-server receiver nor releases upgraded clients.
|
||||
The reporter exposes missing receiver columns and pre-v19 clients as unknown
|
||||
coverage, never as evidence of zero use. Existing telemetry opt-out, ID rotation
|
||||
and retention are unchanged. Payload details belong to security-privacy.
|
||||
|
||||
|
||||
|
||||
### TrueNAS SCALE removal retires every owned middleware boot task before recovery erasure
|
||||
|
||||
|
||||
@@ -3259,6 +3259,7 @@
|
||||
"internal/api/alerting/notifications_test.go",
|
||||
"internal/api/audit_handlers_test.go",
|
||||
"internal/api/availability_handlers_test.go",
|
||||
"internal/api/chartapi/history_activity_test.go",
|
||||
"internal/api/contract_test.go",
|
||||
"internal/api/docker_agents_report_size_test.go",
|
||||
"internal/api/docker_container_action_executor_test.go",
|
||||
@@ -3270,6 +3271,7 @@
|
||||
"internal/api/metadata_handlers_test.go",
|
||||
"internal/api/patrol_autopilot_test.go",
|
||||
"internal/api/runtime_inventory_sources_test.go",
|
||||
"internal/api/telemetry_history_reads_test.go",
|
||||
"internal/api/truenas_disk_history_integration_test.go",
|
||||
"pulse-enterprise:test/extensions_contract_test.go"
|
||||
]
|
||||
@@ -4747,6 +4749,19 @@
|
||||
"scripts/release_control/release_note_visuals_test.py"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "telemetry-schema-v19-release-disclosure",
|
||||
"label": "telemetry schema v19 release disclosure proof",
|
||||
"match_prefixes": [],
|
||||
"match_files": [
|
||||
"docs/releases/TELEMETRY_SCHEMA_V19.md"
|
||||
],
|
||||
"allow_same_subsystem_tests": false,
|
||||
"test_prefixes": [],
|
||||
"exact_files": [
|
||||
"internal/telemetry/telemetry_test.go"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "release-promotion-metadata-runtime",
|
||||
"label": "release promotion and artifact dispatch proof",
|
||||
@@ -7618,6 +7633,7 @@
|
||||
"internal/cloudcp/auth/magiclink.go",
|
||||
"internal/cloudcp/auth/magiclink_store.go",
|
||||
"internal/config/config.go",
|
||||
"internal/config/history_read_activity.go",
|
||||
"internal/config/watcher.go",
|
||||
"internal/crypto/crypto.go",
|
||||
"internal/logging/logging.go",
|
||||
@@ -7636,6 +7652,7 @@
|
||||
"pkg/extensions/audit_admin.go",
|
||||
"pkg/server/server.go",
|
||||
"pkg/server/service_health.go",
|
||||
"pkg/server/telemetry_licensed_features.go",
|
||||
"pkg/server/telemetry_pulse_intelligence.go",
|
||||
"pkg/tlsutil/fingerprint.go",
|
||||
"scripts/telemetry_adoption_report.py",
|
||||
@@ -7667,10 +7684,12 @@
|
||||
"match_files": [
|
||||
"docs/PRIVACY.md",
|
||||
"frontend-modern/public/docs/PRIVACY.md",
|
||||
"internal/config/history_read_activity.go",
|
||||
"internal/telemetry/service_health.go",
|
||||
"internal/telemetry/telemetry.go",
|
||||
"pkg/server/server.go",
|
||||
"pkg/server/service_health.go",
|
||||
"pkg/server/telemetry_licensed_features.go",
|
||||
"pkg/server/telemetry_pulse_intelligence.go",
|
||||
"scripts/telemetry_adoption_report.py",
|
||||
"SECURITY.md"
|
||||
@@ -7680,11 +7699,15 @@
|
||||
"exact_files": [
|
||||
"frontend-modern/src/stores/__tests__/systemSettings.test.ts",
|
||||
"internal/api/system_settings_telemetry_test.go",
|
||||
"internal/api/telemetry_history_reads_test.go",
|
||||
"internal/config/history_read_activity_test.go",
|
||||
"internal/telemetry/service_health_test.go",
|
||||
"internal/telemetry/telemetry_test.go",
|
||||
"pkg/server/service_health_test.go",
|
||||
"pkg/server/telemetry_licensed_features_test.go",
|
||||
"pkg/server/telemetry_pulse_intelligence_test.go",
|
||||
"scripts/tests/test_telemetry_adoption_report.py"
|
||||
"scripts/tests/test_telemetry_adoption_report.py",
|
||||
"tests/integration/tests/19-telemetry-disclosure.spec.ts"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -2843,9 +2843,40 @@ actor, and every audit row read stay on the install.
|
||||
`TestWithAuditReadActivity_RecordIsContentFree` and
|
||||
`TestRecordAuditReadActivity_RejectsUnknownActivity` pin both properties.
|
||||
|
||||
### Extended history demand remains a content-free successful-read count
|
||||
|
||||
Schema v19 adds only `successful_history_reads_over_7d_30d`. Authenticated
|
||||
metric-history and chart HTTP handlers observe successfully written GET
|
||||
responses only when the requested duration exceeds seven days and an actual
|
||||
returned metric point is strictly older than seven days. Empty/denied/failed,
|
||||
cancelled and mock responses are excluded. Cache hits count as reads, as do
|
||||
automatic refreshes. Background queries and range selection alone do not count.
|
||||
|
||||
The router observer receives no request, tenant, actor, resource, range or
|
||||
metric value. Root persistence retains bounded UTC-day counts with private
|
||||
permissions and atomic replacement. Restart preserves them. The snapshot
|
||||
exports only their rolling 30-day sum including the cutoff UTC day, never the
|
||||
day buckets. Opt-out, rotating pseudonymous ID, account-join prohibition and
|
||||
receiver retention are unchanged. This is observed HTTP delivery, not proof
|
||||
that a person saw a chart or would pay for it.
|
||||
|
||||
The receiver migrates the integer column, clamps negatives, and discards this
|
||||
field from pre-v19 payloads. The report contributes only each reporting ID's
|
||||
latest rolling count, splits coarse nonfree/free/unknown posture, excludes
|
||||
nonproduction and inactive latest states, and exposes v19 coverage separately
|
||||
from unknown old/missing measurements. Neither an initial zero nor an older
|
||||
schema establishes 30 days of nonuse. No billing identity join is introduced.
|
||||
|
||||
Proof: `history_read_activity_test.go`, `chartapi/history_activity_test.go`,
|
||||
`telemetry_history_reads_test.go`, telemetry Preview/schema-disclosure tests,
|
||||
server snapshot tests, receiver migration/intake tests, report coverage tests,
|
||||
and `tests/integration/tests/19-telemetry-disclosure.spec.ts`. Canonical and
|
||||
shipped privacy documents must remain byte-identical. Pending release notes
|
||||
carry the schema notice for the first release that includes this field.
|
||||
|
||||
### Telemetry ingestion matches the released sender while storage stays compatible
|
||||
|
||||
The active outbound contract is schema v18. Schema v8 added content-free
|
||||
The active outbound contract is schema v19. Schema v8 added content-free
|
||||
approved-action refusal counters for target change, prerequisite failure, and
|
||||
invalid typed contract so agent-side pre-mutation failures no longer collapse
|
||||
into `other`. Schema v9 completes that split with a content-free `uncoded`
|
||||
|
||||
@@ -6932,3 +6932,12 @@ update pipeline's existing backup of the installed binary before replacing it
|
||||
and its existing history-entry rollback. The data directory, metrics and
|
||||
configuration stores, backups, snapshots and recovery points are untouched,
|
||||
and no new backup, retention or restore path is added.
|
||||
|
||||
### Extended history observation preserves existing read authority
|
||||
|
||||
The shared chart/history response writer now observes successful extended
|
||||
history delivery for the security-privacy schema-v19 aggregate. It adds no
|
||||
query, resource, tenant, credential or action authority. The observer receives
|
||||
only a successful-read marker after serialization/write, and existing API
|
||||
entitlement checks remain before that boundary. Read semantics remain owned
|
||||
by api-contracts and telemetry retention/disclosure by security-privacy.
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
# Pending release notes
|
||||
|
||||
## Telemetry
|
||||
|
||||
When telemetry is enabled, schema v19 reports `successful_history_reads_over_7d_30d`,
|
||||
an aggregate count of successful
|
||||
history reads that return metric points older than seven days. This helps
|
||||
measure actual longer-history use before changing free and paid features.
|
||||
It sends no resource names, metric values, timestamps, selected ranges or
|
||||
customer identifiers. Existing opt-out and retention settings are unchanged.
|
||||
See [Privacy](../PRIVACY.md) for the full field contract.
|
||||
|
||||
Include this notice in the first release carrying schema v19. Deployment of
|
||||
the compatible telemetry receiver must precede interpreting this signal.
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"version": 1,
|
||||
"base_sha": "6eafe916da44c4a4f232870d2924a78d417a4f0b",
|
||||
"verified_at": "2026-10-16T18:03:51Z",
|
||||
"result": "passed",
|
||||
"changed_paths": [
|
||||
"frontend-modern/src/api/settings.ts"
|
||||
],
|
||||
"content_sha256": {
|
||||
"frontend-modern/src/api/settings.ts": "77f8775e16371b52f0e55cb5107d3e68732b6616ce2dcdd2fb12f5b587a9f973"
|
||||
},
|
||||
"routes": [
|
||||
"/settings/system-general",
|
||||
"/docs/PRIVACY"
|
||||
],
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1280,
|
||||
"height": 800
|
||||
},
|
||||
{
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
{
|
||||
"width": 390,
|
||||
"height": 844
|
||||
}
|
||||
],
|
||||
"states": [
|
||||
"Private task-owned current embedded build on pulse-dev port17857 with mock fixtures and outbound telemetry disabled. Preview closed then open, schema19 exact field present at zero. Mock data does not create qualifying history reads.",
|
||||
"Desktop and narrow Preview pixels inspected with successful_history_reads_over_7d_30d visible. JSON remains horizontally scrollable and document has no horizontal overflow at390px.",
|
||||
"Shipped privacy document opens separately and discloses the field, successful-point criterion, excluded responses, daily aggregation, coverage limits and unchanged privacy controls."
|
||||
],
|
||||
"interactions": [
|
||||
"Focused Preview payload and activated it by keyboard. Refreshed real local Preview and verified count remains zero.",
|
||||
"Opened Full details in a new tab, scrolled to schema19 explanation, inspected pixels and closed the tab.",
|
||||
"Resized to390x844, accepted Reset ID confirmation, verified rotating identity changes while count remains zero, scrolled actual preview container to the new field and inspected pixels.",
|
||||
"Both Playwright disclosure tests passed after the final test helper change. Managed backend was stopped by the runner."
|
||||
]
|
||||
}
|
||||
@@ -40,7 +40,7 @@ Every field is listed below with the reason it exists. Nothing else is included
|
||||
|
||||
| Field | Example | Purpose |
|
||||
|-------|---------|---------|
|
||||
| Schema version | `18` | Identify the exact payload contract so old and new signals are not mixed silently |
|
||||
| Schema version | `19` | Identify the exact payload contract so old and new signals are not mixed silently |
|
||||
| Sent at | `2026-07-23T08:30:00Z` | Date the individual heartbeat without sending a history of client activity |
|
||||
| Install ID | `a1b2c3d4-...` | Distinguish active installations within one rotation window without tying telemetry to an account or person |
|
||||
| Version | `6.0.0-rc.1` | Track the canonical release identity currently deployed |
|
||||
@@ -259,6 +259,7 @@ Every field is listed below with the reason it exists. Nothing else is included
|
||||
| Pulse Intelligence approved action refusals: other 30d | `1` | Count pre-dispatch refusals not covered by the fixed categories without sending raw error or action content |
|
||||
| Pulse Intelligence verified finding resolutions 30d | `1` | Count approved Patrol-origin actions whose execution succeeded and whose linked finding postcondition was independently confirmed, without sending finding IDs, action IDs, evidence, resources, or fix details |
|
||||
| Pulse Intelligence approved action last failure reason 30d | `plan_drift` | See one fixed machine reason code for the most recent approved-action failure in the current 30-day telemetry window without sending error text, action output, command text, resource IDs, or actors |
|
||||
| Successful history reads over 7d 30d (`successful_history_reads_over_7d_30d`) | `5` | Count successful GET responses for a history range longer than seven days that actually return a numeric metric point older than seven days. Includes successful cache hits, but excludes empty/recent-only results, denied requests, failed writes, cancelled requests, HEAD requests and synthetic data. Only a rolling count leaves the install, with no selected range, timestamp, metric value, resource, tenant, user or request identity |
|
||||
| Workload history preview sessions 30d | `18` | Count browser sessions that successfully displayed a populated workload history preview, deduplicated locally once per session without sending a guest, user, route, timing, or browser identifier |
|
||||
| Workload history scrub sessions 30d | `9` | Count browser sessions that inspected a point on a workload history graph, deduplicated locally once per session without sending coordinates, values, guests, or browser identity |
|
||||
| Workload history range change sessions 30d | `6` | Count browser sessions that changed the workload history range, deduplicated locally once per session without sending the selected range or browser identity |
|
||||
@@ -311,7 +312,7 @@ occurrence exists, leaving no trustworthy fired-alert denominator. Detected
|
||||
flapping episodes are not reported because their diagnostic event path may be
|
||||
dropped under pressure. Configuration adoption is reported instead.
|
||||
|
||||
The current telemetry contract is schema version 18. Schema v16 adds four
|
||||
The current telemetry contract is schema version 19. Schema v16 adds four
|
||||
workload-history adoption counts. The browser
|
||||
sends only one closed milestone name to the local Pulse server and deduplicates
|
||||
each milestone once per browser session. Pulse stores bounded UTC-day counts
|
||||
@@ -337,6 +338,22 @@ flag for whether it offered a newer release. They exist because a failed check
|
||||
never shows up in the update counters, which only see updates that were
|
||||
applied. No version, release URL, response, or error text is sent.
|
||||
|
||||
Schema v19 adds `successful_history_reads_over_7d_30d`. The server counts
|
||||
successfully written metric-history and chart responses containing a metric
|
||||
point older than seven days. Selecting a long range alone does not count.
|
||||
Empty results, denied requests, failed writes and mock data do not count.
|
||||
Pulse retains bounded UTC-day totals locally and sends only their rolling
|
||||
30-day sum, including the UTC day containing the cutoff. The day buckets
|
||||
are not exported. Requests and auto-refreshes count as reads, not people
|
||||
or proof of a human viewing a chart.
|
||||
|
||||
No customer identity, request URL, selected range, metric timestamp, metric
|
||||
value or infrastructure identifier is retained in this tally or sent. Older
|
||||
schemas cannot report this signal. Measurement starts after upgrading, so
|
||||
historical use cannot be reconstructed and an initial zero does not represent
|
||||
a full 30-day observation. Telemetry opt-out, installation-ID rotation,
|
||||
account-linking restrictions and 90-day receiver retention are unchanged.
|
||||
|
||||
#### Server-side handling and retention
|
||||
|
||||
- Telemetry pings are stored on the Pulse license server only for aggregate install/use analysis.
|
||||
@@ -389,6 +406,7 @@ Every change to the payload bumps the schema version, is listed here with its da
|
||||
|
||||
| Schema | Date | Change |
|
||||
|--------|------|--------|
|
||||
| 19 | 2026-10-16 | Content-free rolling count of successful metric-history responses that return data older than seven days |
|
||||
| 18 | 2026-09-27 | Effective update channel, closed outcome of the last update check on that channel, and whether it offered a newer release |
|
||||
| 17 | 2026-09-02 | Closed Patrol provider class, effective Patrol autonomy level, coarse 30-day Patrol token buckets, and per-outcome investigation counts |
|
||||
| 16 | 2026-08-30 | Four content-free workload-history adoption counters, each counted at most once per browser session |
|
||||
|
||||
@@ -7,7 +7,8 @@ vi.mock('@/utils/apiClient', () => ({
|
||||
}));
|
||||
|
||||
const mockTelemetryPreviewPayload = {
|
||||
schema_version: 4,
|
||||
schema_version: 19,
|
||||
successful_history_reads_over_7d_30d: 7,
|
||||
sent_at: '2026-07-23T08:30:00Z',
|
||||
install_id: 'preview-install-id',
|
||||
version: '6.0.0',
|
||||
@@ -313,6 +314,7 @@ describe('SettingsAPI', () => {
|
||||
const result = await SettingsAPI.getTelemetryPreview();
|
||||
|
||||
expect(apiFetchJSON).toHaveBeenCalledWith('/api/system/settings/telemetry-preview');
|
||||
expect(result.payload.successful_history_reads_over_7d_30d).toBe(7);
|
||||
expect(result).toEqual(mockPreview);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -89,6 +89,7 @@ export interface TelemetryPingPreview {
|
||||
service_health_previous_healthy: boolean;
|
||||
node_test_attempts_30d: number;
|
||||
node_test_failures_30d: number;
|
||||
successful_history_reads_over_7d_30d: number;
|
||||
workload_history_preview_sessions_30d: number;
|
||||
workload_history_scrub_sessions_30d: number;
|
||||
workload_history_range_change_sessions_30d: number;
|
||||
|
||||
@@ -13,7 +13,8 @@ const flushAsync = async () => {
|
||||
const buildTelemetryPreviewPayload = (
|
||||
overrides: Partial<TelemetryPingPreview> = {},
|
||||
): TelemetryPingPreview => ({
|
||||
schema_version: 4,
|
||||
schema_version: 19,
|
||||
successful_history_reads_over_7d_30d: 0,
|
||||
sent_at: '2026-07-23T08:30:00Z',
|
||||
install_id: 'preview-install-id',
|
||||
version: '6.0.0',
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
package chartapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/mock"
|
||||
)
|
||||
|
||||
// ExtendedHistoryReadObserver receives only the fact that a qualifying response
|
||||
// was written. No request, tenant, resource, metric, range or actor is passed.
|
||||
type ExtendedHistoryReadObserver interface {
|
||||
RecordSuccessfulExtendedHistoryRead()
|
||||
}
|
||||
|
||||
func historyDataModeCacheKey(key string) string {
|
||||
if mock.IsMockEnabled() {
|
||||
return key + "|mock"
|
||||
}
|
||||
return key
|
||||
}
|
||||
|
||||
// WriteHistoryJSON preserves the JSON encoder's newline while sharing the same
|
||||
// successful-response observation boundary as cached chart payloads.
|
||||
func (r *Service) WriteHistoryJSON(w http.ResponseWriter, req *http.Request, value any, duration time.Duration) error {
|
||||
payload, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return r.WriteHistoryPayload(w, req, append(payload, '\n'), duration)
|
||||
}
|
||||
|
||||
// WriteHistoryPayload counts successful GET responses, including cache hits,
|
||||
// only when an actual returned metric is older than seven days. Requesting a
|
||||
// long range, empty results, HEAD, cancellation, mock data and failed writes
|
||||
// cannot create usage. HTTP completion does not prove a person viewed a chart.
|
||||
func (r *Service) WriteHistoryPayload(w http.ResponseWriter, req *http.Request, payload []byte, duration time.Duration) error {
|
||||
observedAt := time.Now().UTC()
|
||||
n, err := w.Write(payload)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n != len(payload) {
|
||||
return io.ErrShortWrite
|
||||
}
|
||||
if req.Method != http.MethodGet || req.Context().Err() != nil || duration <= 7*24*time.Hour || mock.IsMockEnabled() {
|
||||
return nil
|
||||
}
|
||||
observer, ok := r.resolver.(ExtendedHistoryReadObserver)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
if containsExtendedHistoryPayload(payload, observedAt.Add(-7*24*time.Hour).UnixMilli()) {
|
||||
observer.RecordSuccessfulExtendedHistoryRead()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func containsExtendedHistoryPayload(payload []byte, cutoff int64) bool {
|
||||
if !json.Valid(payload) {
|
||||
return false
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(payload))
|
||||
if token, _ := decoder.Token(); token != json.Delim('{') {
|
||||
return false
|
||||
}
|
||||
for decoder.More() {
|
||||
key, _ := decoder.Token()
|
||||
switch key {
|
||||
case "points", "metrics", "data", "nodeData", "storageData", "dockerData", "dockerHostData", "agentData", "pools", "disks":
|
||||
if found, _ := scanExtendedHistoryPoint(decoder, cutoff); found {
|
||||
return true
|
||||
}
|
||||
default:
|
||||
var discard json.RawMessage
|
||||
if decoder.Decode(&discard) != nil {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Scan only the metric containers in the known response contracts. Streaming
|
||||
// avoids retaining a second copy of a large decoded chart and stops at the
|
||||
// first qualifying point. Metadata and requested window bounds cannot count.
|
||||
func scanExtendedHistoryPoint(decoder *json.Decoder, cutoff int64) (bool, any) {
|
||||
token, err := decoder.Token()
|
||||
if err != nil {
|
||||
return false, nil
|
||||
}
|
||||
switch token {
|
||||
case json.Delim('{'):
|
||||
var timestamp float64
|
||||
var hasValue bool
|
||||
for decoder.More() {
|
||||
key, _ := decoder.Token()
|
||||
found, scalar := scanExtendedHistoryPoint(decoder, cutoff)
|
||||
if found {
|
||||
return true, nil
|
||||
}
|
||||
if number, ok := scalar.(float64); ok {
|
||||
if key == "timestamp" {
|
||||
timestamp = number
|
||||
} else if key == "value" {
|
||||
hasValue = true
|
||||
}
|
||||
}
|
||||
}
|
||||
_, _ = decoder.Token()
|
||||
return hasValue && timestamp > 0 && timestamp < float64(cutoff), nil
|
||||
case json.Delim('['):
|
||||
for decoder.More() {
|
||||
if found, _ := scanExtendedHistoryPoint(decoder, cutoff); found {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
_, _ = decoder.Token()
|
||||
return false, nil
|
||||
}
|
||||
return false, token
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
package chartapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/testutil"
|
||||
)
|
||||
|
||||
type historyReadResolver struct {
|
||||
monitor *monitoring.Monitor
|
||||
reads int
|
||||
}
|
||||
|
||||
func (r *historyReadResolver) MonitorForContext(context.Context) *monitoring.Monitor {
|
||||
return r.monitor
|
||||
}
|
||||
func (r *historyReadResolver) RecordSuccessfulExtendedHistoryRead() { r.reads++ }
|
||||
|
||||
type failedHistoryWriter struct{ short bool }
|
||||
|
||||
func (failedHistoryWriter) Header() http.Header { return http.Header{} }
|
||||
func (failedHistoryWriter) WriteHeader(int) {}
|
||||
func (w failedHistoryWriter) Write(p []byte) (int, error) {
|
||||
if w.short {
|
||||
return len(p) - 1, nil
|
||||
}
|
||||
return 0, errors.New("closed connection")
|
||||
}
|
||||
|
||||
func TestExtendedHistoryReadRequiresActualOldMetric(t *testing.T) {
|
||||
cutoff := time.Date(2026, 10, 9, 12, 0, 0, 0, time.UTC).UnixMilli()
|
||||
for _, tc := range []struct {
|
||||
name, body string
|
||||
want bool
|
||||
}{
|
||||
{"old metric", fmt.Sprintf(`{"points":[{"timestamp":%d,"value":0}]}`, cutoff-1), true},
|
||||
{"exact boundary", fmt.Sprintf(`{"points":[{"timestamp":%d,"value":1}]}`, cutoff), false},
|
||||
{"recent only", fmt.Sprintf(`{"points":[{"timestamp":%d,"value":1}]}`, cutoff+1), false},
|
||||
{"empty long window", `{"range":"90d","start":1,"stats":{"oldestDataTimestamp":1},"points":[]}`, false},
|
||||
{"metadata is not a metric", `{"stats":{"timestamp":1,"value":2},"points":[]}`, false},
|
||||
{"zero timestamp", `{"points":[{"timestamp":0,"value":2}]}`, false},
|
||||
{"invalid value", `{"points":[{"timestamp":1,"value":null}]}`, false},
|
||||
{"nested named resource", `{"data":{"value":{"timestamp":[{"timestamp":1,"value":2}]}}}`, true},
|
||||
{"invalid JSON", `{"points":[{"timestamp":1,"value":2}]}invalid`, false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := containsExtendedHistoryPayload([]byte(tc.body), cutoff); got != tc.want {
|
||||
t.Fatalf("got %v, want %v", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtendedHistoryReadIgnoresUnsuccessfulOrSyntheticResponses(t *testing.T) {
|
||||
testutil.SetMockMode(t, false)
|
||||
body := []byte(fmt.Sprintf(`{"points":[{"timestamp":%d,"value":1}]}`, time.Now().Add(-8*24*time.Hour).UnixMilli()))
|
||||
for _, tc := range []struct {
|
||||
name, method string
|
||||
duration time.Duration
|
||||
cancel, short, fail bool
|
||||
want int
|
||||
}{
|
||||
{"successful GET", http.MethodGet, 30 * 24 * time.Hour, false, false, false, 1},
|
||||
{"HEAD", http.MethodHead, 30 * 24 * time.Hour, false, false, false, 0},
|
||||
{"seven day request", http.MethodGet, 7 * 24 * time.Hour, false, false, false, 0},
|
||||
{"cancelled", http.MethodGet, 30 * 24 * time.Hour, true, false, false, 0},
|
||||
{"short write", http.MethodGet, 30 * 24 * time.Hour, false, true, false, 0},
|
||||
{"failed write", http.MethodGet, 30 * 24 * time.Hour, false, false, true, 0},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
resolver := &historyReadResolver{}
|
||||
service := NewService(resolver)
|
||||
req := httptest.NewRequest(tc.method, "/history", nil)
|
||||
if tc.cancel {
|
||||
ctx, cancel := context.WithCancel(req.Context())
|
||||
cancel()
|
||||
req = req.WithContext(ctx)
|
||||
}
|
||||
var writer http.ResponseWriter = httptest.NewRecorder()
|
||||
if tc.short || tc.fail {
|
||||
writer = failedHistoryWriter{short: tc.short}
|
||||
}
|
||||
_ = service.WriteHistoryPayload(writer, req, body, tc.duration)
|
||||
if resolver.reads != tc.want {
|
||||
t.Fatalf("reads = %d, want %d", resolver.reads, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
resolver := &historyReadResolver{}
|
||||
service := NewService(resolver)
|
||||
req := httptest.NewRequest(http.MethodGet, "/history", nil)
|
||||
if err := service.WriteHistoryJSON(httptest.NewRecorder(), req, map[string]any{"value": math.NaN()}, 30*24*time.Hour); err == nil {
|
||||
t.Fatal("invalid JSON must fail")
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtendedHistoryReadRejectsMockPayload(t *testing.T) {
|
||||
testutil.SetMockMode(t, true)
|
||||
resolver := &historyReadResolver{}
|
||||
service := NewService(resolver)
|
||||
req := httptest.NewRequest(http.MethodGet, "/history", nil)
|
||||
body := []byte(`{"points":[{"timestamp":1,"value":2}]}`)
|
||||
_ = service.WriteHistoryPayload(httptest.NewRecorder(), req, body, 30*24*time.Hour)
|
||||
if resolver.reads != 0 {
|
||||
t.Fatalf("failed encoding/mock reads counted: %d", resolver.reads)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtendedHistoryReadCountsCachedChartResponses(t *testing.T) {
|
||||
testutil.SetMockMode(t, false)
|
||||
monitor, state, _ := newTestMonitor(t)
|
||||
syncTestResourceStore(t, monitor, state)
|
||||
resolver := &historyReadResolver{monitor: monitor}
|
||||
service := NewService(resolver)
|
||||
body, _ := json.Marshal(map[string]any{"data": map[string]any{"resource": map[string]any{"cpu": []MetricPoint{{Timestamp: time.Now().Add(-8 * 24 * time.Hour).UnixMilli(), Value: 1}}}}})
|
||||
now := time.Now()
|
||||
service.chartPayloads.put(workloadChartsCachePrefix+"default|30d||", body, now.Add(time.Minute), now)
|
||||
for i := 0; i < 2; i++ {
|
||||
rec := httptest.NewRecorder()
|
||||
service.HandleWorkloadCharts(rec, httptest.NewRequest(http.MethodGet, "/api/charts/workloads?range=30d", nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d", rec.Code)
|
||||
}
|
||||
}
|
||||
if resolver.reads != 2 {
|
||||
t.Fatalf("successful cached responses = %d, want 2", resolver.reads)
|
||||
}
|
||||
}
|
||||
@@ -435,7 +435,7 @@ func (r *Service) HandleCharts(w http.ResponseWriter, req *http.Request) {
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if err := json.NewEncoder(w).Encode(response); err != nil {
|
||||
if err := r.WriteHistoryJSON(w, req, response, duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to encode chart data response")
|
||||
http.Error(w, "Internal server error", http.StatusInternalServerError)
|
||||
return
|
||||
@@ -1328,11 +1328,11 @@ func (r *Service) HandleWorkloadCharts(w http.ResponseWriter, req *http.Request)
|
||||
}
|
||||
cacheKey := orgID + "|" + timeRange + "|" + selectedNodeID + "|" + maxPointsRaw
|
||||
|
||||
cacheKey = workloadChartsCachePrefix + cacheKey
|
||||
cacheKey = historyDataModeCacheKey(workloadChartsCachePrefix + cacheKey)
|
||||
now := time.Now()
|
||||
if body, ok := r.chartPayloads.get(cacheKey, now); ok {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if _, err := w.Write(body); err != nil {
|
||||
if err := r.WriteHistoryPayload(w, req, body, duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to write cached workload chart data response")
|
||||
}
|
||||
return
|
||||
@@ -1360,7 +1360,7 @@ func (r *Service) HandleWorkloadCharts(w http.ResponseWriter, req *http.Request)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if _, err := w.Write(v.([]byte)); err != nil {
|
||||
if err := r.WriteHistoryPayload(w, req, v.([]byte), duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to write workload chart data response")
|
||||
}
|
||||
}
|
||||
@@ -1764,10 +1764,10 @@ func (r *Service) HandleInfrastructureCharts(w http.ResponseWriter, req *http.Re
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
cacheKey := infrastructureChartsCacheKey(req, timeRange, requestedMetricNames)
|
||||
cacheKey := historyDataModeCacheKey(infrastructureChartsCacheKey(req, timeRange, requestedMetricNames))
|
||||
if payload, ok := r.cachedInfrastructureChartsPayload(cacheKey, now); ok {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if _, err := w.Write(payload); err != nil {
|
||||
if err := r.WriteHistoryPayload(w, req, payload, duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to write cached infrastructure chart data response")
|
||||
}
|
||||
return
|
||||
@@ -2030,7 +2030,7 @@ func (r *Service) HandleInfrastructureCharts(w http.ResponseWriter, req *http.Re
|
||||
payload = append(payload, '\n')
|
||||
r.cacheInfrastructureChartsPayload(cacheKey, payload, now)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if _, err := w.Write(payload); err != nil {
|
||||
if err := r.WriteHistoryPayload(w, req, payload, duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to write infrastructure chart data response")
|
||||
}
|
||||
}
|
||||
@@ -2205,7 +2205,7 @@ func (r *Service) HandleStorageCharts(w http.ResponseWriter, req *http.Request)
|
||||
resp.Disks = disks
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if err := json.NewEncoder(w).Encode(resp.NormalizeCollections()); err != nil {
|
||||
if err := r.WriteHistoryJSON(w, req, resp.NormalizeCollections(), duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to encode storage chart data")
|
||||
http.Error(w, "Internal server error", http.StatusInternalServerError)
|
||||
}
|
||||
|
||||
@@ -7160,14 +7160,14 @@ func (r *Router) handleMetricsHistory(w http.ResponseWriter, req *http.Request)
|
||||
response["points"] = apiPoints
|
||||
response["source"] = source
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(response)
|
||||
r.ensureChartService().WriteHistoryJSON(w, req, response, duration)
|
||||
return
|
||||
}
|
||||
} else if apiData, source, ok := fallbackAll(); ok {
|
||||
response["metrics"] = apiData
|
||||
response["source"] = source
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(response)
|
||||
r.ensureChartService().WriteHistoryJSON(w, req, response, duration)
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -7223,7 +7223,7 @@ func (r *Router) handleMetricsHistory(w http.ResponseWriter, req *http.Request)
|
||||
"source": source,
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(response)
|
||||
r.ensureChartService().WriteHistoryJSON(w, req, response, duration)
|
||||
return
|
||||
}
|
||||
} else {
|
||||
@@ -7243,7 +7243,7 @@ func (r *Router) handleMetricsHistory(w http.ResponseWriter, req *http.Request)
|
||||
"source": source,
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(response)
|
||||
r.ensureChartService().WriteHistoryJSON(w, req, response, duration)
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -7500,7 +7500,7 @@ func (r *Router) handleMetricsHistory(w http.ResponseWriter, req *http.Request)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if err := json.NewEncoder(w).Encode(response); err != nil {
|
||||
if err := r.ensureChartService().WriteHistoryJSON(w, req, response, duration); err != nil {
|
||||
log.Error().Err(err).Msg("Failed to encode metrics history response")
|
||||
http.Error(w, "Internal server error", http.StatusInternalServerError)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/mock"
|
||||
"github.com/rs/zerolog/log"
|
||||
)
|
||||
|
||||
func (r routerChartMonitorResolver) RecordSuccessfulExtendedHistoryRead() {
|
||||
if r.router == nil || r.router.persistence == nil || mock.IsMockEnabled() {
|
||||
return
|
||||
}
|
||||
// This install-wide tally deliberately has no tenant or actor dimension.
|
||||
if err := r.router.persistence.RecordSuccessfulExtendedHistoryRead(time.Now().UTC()); err != nil {
|
||||
log.Debug().Msg("Unable to retain aggregate extended history read count")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/testutil"
|
||||
pkglicensing "github.com/rcourtman/pulse-go-rewrite/pkg/licensing"
|
||||
"github.com/rcourtman/pulse-go-rewrite/pkg/metrics"
|
||||
)
|
||||
|
||||
func TestMetricsHistoryTelemetryRequiresServedExtendedData(t *testing.T) {
|
||||
testutil.SetMockMode(t, false)
|
||||
for _, tc := range []struct {
|
||||
name, query string
|
||||
age time.Duration
|
||||
licensed bool
|
||||
want int
|
||||
}{
|
||||
{"old single metric", "&metric=cpu&range=30d", 8 * 24 * time.Hour, true, 1},
|
||||
{"old all metrics", "&range=90d", 8 * 24 * time.Hour, true, 1},
|
||||
{"recent data in long range", "&metric=cpu&range=30d", 24 * time.Hour, true, 0},
|
||||
{"denied range", "&metric=cpu&range=30d", 8 * 24 * time.Hour, false, 0},
|
||||
{"empty long range", "&metric=missing&range=30d", 8 * 24 * time.Hour, true, 0},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
monitor, _, _ := newTestMonitor(t)
|
||||
store, err := metrics.NewStore(metrics.DefaultConfig(t.TempDir()))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = store.Close() })
|
||||
store.WriteBatchSync([]metrics.WriteMetric{{ResourceType: "vm", ResourceID: "private-resource", MetricType: "cpu", Value: 42, Timestamp: time.Now().Add(-tc.age), Tier: metrics.TierDaily}})
|
||||
setUnexportedField(t, monitor, "metricsStore", store)
|
||||
persistence := config.NewConfigPersistence(t.TempDir())
|
||||
handlers := NewLicenseHandlers(config.NewMultiTenantPersistence(t.TempDir()), false)
|
||||
if tc.licensed {
|
||||
setMetricsHistoryLicenseTier(t, handlers, pkglicensing.TierPro)
|
||||
}
|
||||
router := &Router{monitor: monitor, persistence: persistence, licenseHandlers: handlers}
|
||||
rec := httptest.NewRecorder()
|
||||
router.handleMetricsHistory(rec, httptest.NewRequest(http.MethodGet, "/api/metrics-store/history?resourceType=vm&resourceId=private-resource"+tc.query, nil))
|
||||
if tc.licensed && rec.Code != http.StatusOK {
|
||||
t.Fatalf("status %d: %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
if got := persistence.CountSuccessfulExtendedHistoryReadsSince(now.AddDate(0, 0, -30), now); got != tc.want {
|
||||
t.Fatalf("reads = %d, want %d; response %s", got, tc.want, rec.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
)
|
||||
|
||||
const historyReadActivityFile = "history_read_activity.json"
|
||||
|
||||
// HistoryReadActivity retains daily counts only. Requests, ranges, metric
|
||||
// values, resources, actors and individual read timestamps are never stored.
|
||||
type HistoryReadActivity struct {
|
||||
DailySuccessfulReads map[string]int `json:"daily_successful_reads_over_7d"`
|
||||
}
|
||||
|
||||
func (c *ConfigPersistence) loadHistoryReadActivityLocked() (*HistoryReadActivity, error) {
|
||||
activity := &HistoryReadActivity{DailySuccessfulReads: map[string]int{}}
|
||||
raw, err := c.fs.ReadFile(filepath.Join(c.configDir, historyReadActivityFile))
|
||||
if os.IsNotExist(err) {
|
||||
return activity, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := json.Unmarshal(raw, activity); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if activity.DailySuccessfulReads == nil {
|
||||
activity.DailySuccessfulReads = map[string]int{}
|
||||
}
|
||||
return activity, nil
|
||||
}
|
||||
|
||||
// RecordSuccessfulExtendedHistoryRead folds one successfully written response
|
||||
// containing a genuine metric older than seven days into its UTC-day count.
|
||||
func (c *ConfigPersistence) RecordSuccessfulExtendedHistoryRead(now time.Time) error {
|
||||
if c == nil || c.configDir == "" {
|
||||
return nil
|
||||
}
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
activity, err := c.loadHistoryReadActivityLocked()
|
||||
if err != nil {
|
||||
return fmt.Errorf("load history read activity: %w", err)
|
||||
}
|
||||
day := now.UTC().Format("2006-01-02")
|
||||
cutoff := now.UTC().AddDate(0, 0, -30).Format("2006-01-02")
|
||||
for retainedDay := range activity.DailySuccessfulReads {
|
||||
if retainedDay < cutoff || retainedDay > day {
|
||||
delete(activity.DailySuccessfulReads, retainedDay)
|
||||
}
|
||||
}
|
||||
activity.DailySuccessfulReads[day]++
|
||||
raw, err := json.Marshal(activity)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
path := filepath.Join(c.configDir, historyReadActivityFile)
|
||||
if err := c.fs.MkdirAll(c.configDir, 0o700); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.fs.WriteFile(path+".tmp", raw, 0o600); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.fs.Rename(path+".tmp", path); err != nil {
|
||||
_ = c.fs.Remove(path + ".tmp")
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *ConfigPersistence) CountSuccessfulExtendedHistoryReadsSince(since, now time.Time) int {
|
||||
if c == nil || c.configDir == "" {
|
||||
return 0
|
||||
}
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
activity, err := c.loadHistoryReadActivityLocked()
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
first, last := since.UTC().Format("2006-01-02"), now.UTC().Format("2006-01-02")
|
||||
count := 0
|
||||
for day, reads := range activity.DailySuccessfulReads {
|
||||
if day >= first && day <= last && reads > 0 {
|
||||
count += reads
|
||||
}
|
||||
}
|
||||
return count
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package config_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
)
|
||||
|
||||
func TestExtendedHistoryReadActivityPersistsOnlyBoundedDailyCounts(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
persistence := config.NewConfigPersistence(dir)
|
||||
now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
|
||||
for _, at := range []time.Time{now.AddDate(0, 0, -40), now.AddDate(0, 0, -30), now, now} {
|
||||
if err := persistence.RecordSuccessfulExtendedHistoryRead(at); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
// Reopening models a restart. No request-level data is needed for continuity.
|
||||
reopened := config.NewConfigPersistence(dir)
|
||||
if got := reopened.CountSuccessfulExtendedHistoryReadsSince(now.AddDate(0, 0, -30), now); got != 3 {
|
||||
t.Fatalf("retained reads = %d, want 3", got)
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(dir, "history_read_activity.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var stored map[string]map[string]int
|
||||
if err := json.Unmarshal(raw, &stored); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(stored) != 1 || len(stored["daily_successful_reads_over_7d"]) != 2 {
|
||||
t.Fatalf("unexpected retained fields or days: %s", raw)
|
||||
}
|
||||
info, err := os.Stat(filepath.Join(dir, "history_read_activity.json"))
|
||||
if err != nil || info.Mode().Perm() != 0o600 {
|
||||
t.Fatalf("tally mode must be 0600: %v %v", info, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtendedHistoryReadActivitySerializesConcurrentRequests(t *testing.T) {
|
||||
persistence := config.NewConfigPersistence(t.TempDir())
|
||||
now := time.Now().UTC()
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 16; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := persistence.RecordSuccessfulExtendedHistoryRead(now); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if got := persistence.CountSuccessfulExtendedHistoryReadsSince(now.AddDate(0, 0, -30), now); got != 16 {
|
||||
t.Fatalf("reads = %d, want 16", got)
|
||||
}
|
||||
}
|
||||
@@ -225,7 +225,9 @@ const (
|
||||
// A 6.4.3-rc.1 update check failed on every call for weeks while its
|
||||
// telemetry looked healthy, because the update counters only see applied
|
||||
// updates. No version, URL, or error text is exported.
|
||||
TelemetrySchemaVersion = 18
|
||||
// Schema v19 adds only the rolling count of successfully served metric
|
||||
// history responses containing data older than seven days.
|
||||
TelemetrySchemaVersion = 19
|
||||
)
|
||||
|
||||
type installIDRecord struct {
|
||||
@@ -389,6 +391,7 @@ type Ping struct {
|
||||
// browser intake is session-deduplicated and contains only the closed
|
||||
// milestone; guest/user identity, routes, coordinates, and timings never
|
||||
// enter the tally or payload.
|
||||
SuccessfulHistoryReadsOver7d30d int `json:"successful_history_reads_over_7d_30d"`
|
||||
WorkloadHistoryPreviewSessions30d int `json:"workload_history_preview_sessions_30d"`
|
||||
WorkloadHistoryScrubSessions30d int `json:"workload_history_scrub_sessions_30d"`
|
||||
WorkloadHistoryRangeChangeSessions30d int `json:"workload_history_range_change_sessions_30d"`
|
||||
@@ -610,6 +613,7 @@ type Snapshot struct {
|
||||
UpdateAvailable bool
|
||||
NodeTestAttempts30d int
|
||||
NodeTestFailures30d int
|
||||
SuccessfulHistoryReadsOver7d30d int
|
||||
WorkloadHistoryPreviewSessions30d int
|
||||
WorkloadHistoryScrubSessions30d int
|
||||
WorkloadHistoryRangeChangeSessions30d int
|
||||
@@ -1354,6 +1358,7 @@ func applySnapshot(base Ping, fn SnapshotFunc) Ping {
|
||||
ping.UpdateAvailable = s.UpdateAvailable && ping.UpdateCheckOutcome == updates.UpdateCheckOutcomeAvailable
|
||||
ping.NodeTestAttempts30d = s.NodeTestAttempts30d
|
||||
ping.NodeTestFailures30d = s.NodeTestFailures30d
|
||||
ping.SuccessfulHistoryReadsOver7d30d = s.SuccessfulHistoryReadsOver7d30d
|
||||
ping.WorkloadHistoryPreviewSessions30d = s.WorkloadHistoryPreviewSessions30d
|
||||
ping.WorkloadHistoryScrubSessions30d = s.WorkloadHistoryScrubSessions30d
|
||||
ping.WorkloadHistoryRangeChangeSessions30d = s.WorkloadHistoryRangeChangeSessions30d
|
||||
|
||||
@@ -721,6 +721,43 @@ func TestAllTelemetryFieldsAreDisclosed(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTelemetryPrivacyReleaseNoticeCoversExtendedHistory(t *testing.T) {
|
||||
raw, err := os.ReadFile(filepath.Join("..", "..", "docs", "releases", "TELEMETRY_SCHEMA_V19.md"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
notice := string(raw)
|
||||
for _, required := range []string{"schema v19", "successful_history_reads_over_7d_30d", "customer identifiers", "opt-out"} {
|
||||
if !strings.Contains(notice, required) {
|
||||
t.Errorf("release notice must disclose %q", required)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuccessfulExtendedHistoryReadCounterAppearsInPreview(t *testing.T) {
|
||||
preview, err := BuildPreview(Config{
|
||||
Version: "6.6.0", DataDir: t.TempDir(),
|
||||
GetSnapshot: func() Snapshot { return Snapshot{SuccessfulHistoryReadsOver7d30d: 7} },
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if preview.SuccessfulHistoryReadsOver7d30d != 7 {
|
||||
t.Fatalf("preview reads = %d", preview.SuccessfulHistoryReadsOver7d30d)
|
||||
}
|
||||
raw, err := json.Marshal(preview)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(raw, &payload); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if payload["successful_history_reads_over_7d_30d"] != float64(7) {
|
||||
t.Fatalf("missing aggregate read count")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTelemetryPrivacyDocsDisclosePseudonymousIdentityAndIPHandling(t *testing.T) {
|
||||
anonymousTelemetryClaimPattern := regexp.MustCompile(`(?i)\banonymous\b[^\n]{0,120}\btelemetry\b|\btelemetry\b[^\n]{0,120}\banonymous\b`)
|
||||
for _, relativePath := range []string{
|
||||
|
||||
@@ -35,6 +35,8 @@ func applyLicensedFeatureConfigSnapshot(snap *telemetry.Snapshot, persistence *c
|
||||
snap.AgentProfiles = len(profiles)
|
||||
}
|
||||
|
||||
snap.SuccessfulHistoryReadsOver7d30d = persistence.CountSuccessfulExtendedHistoryReadsSince(now.Add(-telemetry.PulseIntelligenceTelemetryWindow), now)
|
||||
|
||||
// Audit adoption is a read count, not store presence: the SQLite audit
|
||||
// logger is installed on every install for defense in depth, so only an
|
||||
// operator reaching a license-gated read surface distinguishes an install
|
||||
|
||||
@@ -67,3 +67,16 @@ func TestApplyLicensedFeatureConfigSnapshot_ToleratesNilInputs(t *testing.T) {
|
||||
applyLicensedFeatureConfigSnapshot(&snap, nil, time.Now().UTC())
|
||||
applyLicensedFeatureConfigSnapshot(nil, config.NewConfigPersistence(t.TempDir()), time.Now().UTC())
|
||||
}
|
||||
|
||||
func TestApplyLicensedFeatureConfigSnapshotIncludesSuccessfulExtendedHistoryReads(t *testing.T) {
|
||||
persistence := config.NewConfigPersistence(t.TempDir())
|
||||
now := time.Now().UTC()
|
||||
if err := persistence.RecordSuccessfulExtendedHistoryRead(now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var snap telemetry.Snapshot
|
||||
applyLicensedFeatureConfigSnapshot(&snap, persistence, now)
|
||||
if snap.SuccessfulHistoryReadsOver7d30d != 1 {
|
||||
t.Fatalf("extended history reads = %d, want 1", snap.SuccessfulHistoryReadsOver7d30d)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1374,6 +1374,7 @@ class CanonicalCompletionGuardTest(unittest.TestCase):
|
||||
"internal/api/alerting/notifications_test.go",
|
||||
"internal/api/audit_handlers_test.go",
|
||||
"internal/api/availability_handlers_test.go",
|
||||
"internal/api/chartapi/history_activity_test.go",
|
||||
"internal/api/contract_test.go",
|
||||
"internal/api/docker_agents_report_size_test.go",
|
||||
"internal/api/docker_container_action_executor_test.go",
|
||||
@@ -1385,6 +1386,7 @@ class CanonicalCompletionGuardTest(unittest.TestCase):
|
||||
"internal/api/metadata_handlers_test.go",
|
||||
"internal/api/patrol_autopilot_test.go",
|
||||
"internal/api/runtime_inventory_sources_test.go",
|
||||
"internal/api/telemetry_history_reads_test.go",
|
||||
"internal/api/truenas_disk_history_integration_test.go",
|
||||
"pulse-enterprise:test/extensions_contract_test.go",
|
||||
],
|
||||
|
||||
@@ -1073,6 +1073,7 @@ REPORT_ROW_COLUMNS = tuple(
|
||||
"version_is_published_release",
|
||||
"platform",
|
||||
"notification_failures_7d",
|
||||
"successful_history_reads_over_7d_30d",
|
||||
*(key for key, _ in ALERT_QUALITY_COUNT_FIELDS),
|
||||
*(key for key, _ in ADOPTION_COUNT_FIELDS),
|
||||
*(key for key, _ in FEATURE_BOOL_FIELDS),
|
||||
@@ -1686,8 +1687,14 @@ def fetch_rows_local(db_path: str, since_days: int) -> dict[str, Any]:
|
||||
"""
|
||||
).fetchone()
|
||||
)
|
||||
available_columns = {row[1] for row in conn.execute("PRAGMA table_info(telemetry_pings)")}
|
||||
missing_columns = sorted(set(REPORT_ROW_COLUMNS) - available_columns)
|
||||
projection = ", ".join(
|
||||
name if name in available_columns else "0 AS " + name
|
||||
for name in REPORT_ROW_COLUMNS
|
||||
)
|
||||
rows_sql = (
|
||||
f"SELECT {REPORT_ROW_PROJECTION} "
|
||||
f"SELECT {projection} "
|
||||
"FROM telemetry_pings "
|
||||
"WHERE received_at >= datetime('now', ?) "
|
||||
"ORDER BY received_at DESC"
|
||||
@@ -1699,7 +1706,7 @@ def fetch_rows_local(db_path: str, since_days: int) -> dict[str, Any]:
|
||||
(f"-{since_days} days",),
|
||||
).fetchall()
|
||||
]
|
||||
return {"db_stats": db_stats, "rows": rows}
|
||||
return {"db_stats": db_stats, "rows": rows, "unavailable_columns": missing_columns}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
@@ -2793,6 +2800,53 @@ def summarize_user_base_signals(
|
||||
}
|
||||
|
||||
|
||||
def summarize_extended_history_usage(
|
||||
latest_by_install: dict[str, dict[str, Any]],
|
||||
published_versions: set[str],
|
||||
*,
|
||||
now: datetime,
|
||||
unavailable_columns: Iterable[str] = (),
|
||||
) -> dict[str, Any]:
|
||||
"""One latest rolling count per reporting ID, with explicit coverage.
|
||||
|
||||
No heartbeat summing or account joins. The license flag is coarse, and
|
||||
rotating IDs do not identify people or subscriptions.
|
||||
"""
|
||||
field = "successful_history_reads_over_7d_30d"
|
||||
source_available = field not in unavailable_columns
|
||||
cohorts = {
|
||||
posture: {"reporting_ids": 0, "covered_ids": 0, "unmeasured_ids": 0,
|
||||
"ids_with_reads": 0, "successful_reads_30d": 0}
|
||||
for posture in ("nonfree", "free", "unknown")
|
||||
}
|
||||
excluded = 0
|
||||
for row in latest_by_install.values():
|
||||
age = now - parse_received_at(str(row["received_at"]))
|
||||
if age < timedelta(0) or age > timedelta(days=7):
|
||||
continue
|
||||
identity = classify_row_version(row, published_versions)
|
||||
if (is_mock_fleet_row(row) or identity.is_development
|
||||
or not identity.is_published_release
|
||||
or not parse_optional_bool(row.get("monitoring_active"))):
|
||||
excluded += 1
|
||||
continue
|
||||
paid = parse_optional_bool(row.get("paid_license"))
|
||||
cohort = cohorts["nonfree" if paid is True else "free" if paid is False else "unknown"]
|
||||
cohort["reporting_ids"] += 1
|
||||
if (not source_available or field not in row
|
||||
or parse_optional_nonnegative_int(row.get("schema_version")) < 19):
|
||||
cohort["unmeasured_ids"] += 1
|
||||
continue
|
||||
cohort["covered_ids"] += 1
|
||||
reads = parse_optional_nonnegative_int(row.get(field))
|
||||
if reads > 0:
|
||||
cohort["ids_with_reads"] += 1
|
||||
cohort["successful_reads_30d"] += reads
|
||||
return {"window": "7d", "minimum_schema_version": 19,
|
||||
"source_available": source_available, "excluded_ids": excluded,
|
||||
"cohorts": cohorts}
|
||||
|
||||
|
||||
def analyze_target_release_rows(
|
||||
rows: Iterable[dict[str, Any]],
|
||||
published_versions: set[str],
|
||||
@@ -3321,6 +3375,7 @@ def summarize_rows(
|
||||
source_window_days: int | None = None,
|
||||
pulse_intelligence_analysis_facts: Iterable[dict[str, Any]] | None = None,
|
||||
target_release_analysis_facts: Iterable[dict[str, Any]] | None = None,
|
||||
unavailable_columns: Iterable[str] = (),
|
||||
) -> dict[str, Any]:
|
||||
row_list: list[dict[str, Any]] = []
|
||||
mock_fleet_rows = 0
|
||||
@@ -3403,6 +3458,10 @@ def summarize_rows(
|
||||
"installs": len(mock_fleet_installs),
|
||||
},
|
||||
"latest_install_windows": latest_install_windows,
|
||||
"extended_history_usage_7d": summarize_extended_history_usage(
|
||||
latest_by_install, published_versions, now=current_time,
|
||||
unavailable_columns=unavailable_columns,
|
||||
),
|
||||
"user_base_signals_7d": summarize_user_base_signals(
|
||||
latest_by_install,
|
||||
now=current_time,
|
||||
@@ -3620,6 +3679,23 @@ def format_text(summary: dict[str, Any], repo: str, since_days: int) -> str:
|
||||
"schema v3+ counts only terminal failed or dead-letter outcomes"
|
||||
)
|
||||
|
||||
extended_history = summary.get("extended_history_usage_7d")
|
||||
if extended_history:
|
||||
lines.extend(["", "Successful history reads beyond seven days (active production reporting IDs, 7d):"])
|
||||
for posture, cohort in extended_history["cohorts"].items():
|
||||
lines.append(
|
||||
f"- {posture}: {cohort['covered_ids']} measured, "
|
||||
f"{cohort['unmeasured_ids']} unmeasured, {cohort['ids_with_reads']} with reads, "
|
||||
f"{cohort['successful_reads_30d']} successful reads (rolling 30d)"
|
||||
)
|
||||
lines.append(
|
||||
"- interpretation: schema v19+ only. Older or unavailable measurements are unknown, "
|
||||
"not zero use. Each reporting ID contributes only its latest rolling count. "
|
||||
"Automatic refreshes count as reads. Rotating IDs are not people or subscriptions. "
|
||||
"The nonfree flag does not verify paying customers or willingness to pay. "
|
||||
"An upgrade starts measurement, so a zero is not a full 30-day observation."
|
||||
)
|
||||
|
||||
pulse_loop = summary.get("pulse_intelligence_value_loop_7d")
|
||||
if pulse_loop:
|
||||
lines.extend(
|
||||
@@ -4001,6 +4077,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||
baseline_version=args.baseline_version,
|
||||
include_mock_fleet=args.include_mock_fleet,
|
||||
source_window_days=args.since_days,
|
||||
unavailable_columns=source.get("unavailable_columns", []),
|
||||
pulse_intelligence_analysis_facts=source.get(
|
||||
"pulse_intelligence_analysis_facts"
|
||||
),
|
||||
|
||||
@@ -24,6 +24,45 @@ import telemetry_adoption_report as report
|
||||
|
||||
|
||||
class TelemetryAdoptionReportTest(unittest.TestCase):
|
||||
def test_extended_history_reports_coverage_and_latest_rolling_count(self) -> None:
|
||||
now = datetime(2026, 10, 16, 12, tzinfo=timezone.utc)
|
||||
base = {"received_at": "2026-10-16 11:00:00", "version": "6.6.0",
|
||||
"monitoring_active": 1, "schema_version": 19, "paid_license": 1,
|
||||
"successful_history_reads_over_7d_30d": 4}
|
||||
rows = [{**base, "install_id": "nonfree"},
|
||||
{**base, "install_id": "nonfree", "received_at": "2026-10-15 11:00:00", "successful_history_reads_over_7d_30d": 3},
|
||||
{**base, "install_id": "legacy", "schema_version": 18, "successful_history_reads_over_7d_30d": 99},
|
||||
{**base, "install_id": "zero", "successful_history_reads_over_7d_30d": 0},
|
||||
{**base, "install_id": "free", "paid_license": 0, "successful_history_reads_over_7d_30d": 2},
|
||||
{**base, "install_id": "inactive", "monitoring_active": 0},
|
||||
{**base, "install_id": "dev", "version_is_development": 1},
|
||||
{**base, "install_id": "mock", "kubernetes_pods": 120, "vmware_hosts": 7},
|
||||
{**base, "install_id": "stale", "received_at": "2026-10-01 11:00:00"}]
|
||||
summary = report.summarize_rows({}, rows, {"6.6.0"}, now=now)
|
||||
usage = summary["extended_history_usage_7d"]
|
||||
self.assertEqual(usage["cohorts"]["nonfree"], {
|
||||
"reporting_ids": 3, "covered_ids": 2, "unmeasured_ids": 1,
|
||||
"ids_with_reads": 1, "successful_reads_30d": 4})
|
||||
self.assertEqual(usage["cohorts"]["free"]["successful_reads_30d"], 2)
|
||||
self.assertEqual(usage["excluded_ids"], 2)
|
||||
unavailable = report.summarize_rows({}, rows, {"6.6.0"}, now=now,
|
||||
unavailable_columns=["successful_history_reads_over_7d_30d"])
|
||||
self.assertEqual(unavailable["extended_history_usage_7d"]["cohorts"]["nonfree"]["covered_ids"], 0)
|
||||
text = report.format_text(summary, "rcourtman/Pulse", 7)
|
||||
self.assertIn("2 measured, 1 unmeasured, 1 with reads, 4 successful reads", text)
|
||||
self.assertIn("not zero use", text)
|
||||
self.assertIn("does not verify paying customers", text)
|
||||
|
||||
def test_local_report_old_receiver_marks_new_history_field_unavailable(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
db_path = str(Path(directory) / "telemetry.sqlite")
|
||||
with sqlite3.connect(db_path) as conn:
|
||||
conn.execute("CREATE TABLE telemetry_pings (received_at TEXT, install_id TEXT, version TEXT)")
|
||||
conn.execute("INSERT INTO telemetry_pings VALUES (datetime('now'), 'fixture', '6.6.0')")
|
||||
source = report.fetch_rows_local(db_path, 7)
|
||||
self.assertIn("successful_history_reads_over_7d_30d", source["unavailable_columns"])
|
||||
self.assertEqual(source["rows"][0]["successful_history_reads_over_7d_30d"], 0)
|
||||
|
||||
def test_normalize_reported_version_strips_v_prefix(self) -> None:
|
||||
self.assertEqual(report.normalize_reported_version("v6.0.0-rc.1"), "6.0.0-rc.1")
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ import {
|
||||
import { createAuthenticatedStorageState } from "./helpers";
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const EXPECTED_TELEMETRY_SCHEMA_VERSION = 18;
|
||||
const EXPECTED_TELEMETRY_SCHEMA_VERSION = 19;
|
||||
|
||||
type WorkerFixtures = {
|
||||
authStorageStatePath: string;
|
||||
@@ -53,6 +53,10 @@ async function expectPopupDoc(
|
||||
await popup.waitForLoadState("domcontentloaded");
|
||||
expect(new URL(popup.url()).pathname).toBe(pathname);
|
||||
await expect(popup.locator("body")).toContainText(expectedText);
|
||||
await expect(popup.locator("body")).toContainText("successful_history_reads_over_7d_30d");
|
||||
await expect(popup.locator("body")).toContainText("Selecting a long range alone does not count");
|
||||
await popup.getByText("Schema v19 adds", { exact: false }).scrollIntoViewIfNeeded();
|
||||
await popup.screenshot({ path: test.info().outputPath("privacy-history-counter.png") });
|
||||
await popup.close();
|
||||
}
|
||||
|
||||
@@ -69,9 +73,29 @@ async function readTelemetryPreview(page: Page) {
|
||||
update_channel: string;
|
||||
update_check_outcome: string;
|
||||
update_available: boolean;
|
||||
successful_history_reads_over_7d_30d: number;
|
||||
};
|
||||
}
|
||||
|
||||
async function showExtendedHistoryCount(page: Page) {
|
||||
await page.locator('pre[aria-label="Telemetry payload preview"]').evaluate((pre) => {
|
||||
const node = pre.firstChild!;
|
||||
const offset = (node.textContent ?? "").indexOf("successful_history_reads_over_7d_30d");
|
||||
const range = document.createRange();
|
||||
range.setStart(node, offset);
|
||||
range.setEnd(node, offset + "successful_history_reads_over_7d_30d".length);
|
||||
for (let parent = pre.parentElement; parent; parent = parent.parentElement) {
|
||||
if (parent.scrollHeight > parent.clientHeight && /auto|scroll/.test(getComputedStyle(parent).overflowY)) {
|
||||
parent.scrollTop += range.getBoundingClientRect().top - parent.getBoundingClientRect().top - parent.clientHeight / 2;
|
||||
}
|
||||
}
|
||||
const rect = range.getBoundingClientRect();
|
||||
if (rect.top < 60 || rect.bottom > window.innerHeight - 60) {
|
||||
throw new Error("Extended history count must be in the visible preview before capture");
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
test.describe("Telemetry disclosure", () => {
|
||||
test.setTimeout(180_000);
|
||||
|
||||
@@ -125,7 +149,9 @@ test.describe("Telemetry disclosure", () => {
|
||||
});
|
||||
await page.waitForURL(/\/settings/, { timeout: 15_000 });
|
||||
|
||||
await page.getByRole("button", { name: "Preview payload" }).click();
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
await page.getByRole("button", { name: "Preview payload" }).focus();
|
||||
await page.getByRole("button", { name: "Preview payload" }).press("Enter");
|
||||
|
||||
const initialPreview = await readTelemetryPreview(page);
|
||||
expect(initialPreview.event).toBe("heartbeat");
|
||||
@@ -156,6 +182,12 @@ test.describe("Telemetry disclosure", () => {
|
||||
"error",
|
||||
]).toContain(initialPreview.update_check_outcome);
|
||||
expect(typeof initialPreview.update_available).toBe("boolean");
|
||||
expect(initialPreview.successful_history_reads_over_7d_30d).toBe(0);
|
||||
await page.getByRole("button", { name: "Refresh payload" }).click();
|
||||
expect((await readTelemetryPreview(page)).successful_history_reads_over_7d_30d).toBe(0);
|
||||
// Fixture data must not create real extended-history usage.
|
||||
await showExtendedHistoryCount(page);
|
||||
await page.screenshot({ path: testInfo.outputPath("telemetry-preview-desktop.png") });
|
||||
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
await expect(
|
||||
@@ -178,5 +210,8 @@ test.describe("Telemetry disclosure", () => {
|
||||
return refreshedPreview.install_id;
|
||||
})
|
||||
.not.toBe(initialPreview.install_id);
|
||||
expect((await readTelemetryPreview(page)).successful_history_reads_over_7d_30d).toBe(0);
|
||||
await showExtendedHistoryCount(page);
|
||||
await page.screenshot({ path: testInfo.outputPath("telemetry-preview-narrow.png") });
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user