Commit Graph

3446 Commits

Author SHA1 Message Date
rcourtman 990bccf191 Redact PBS probe failures in the diagnostics export
4815a5c80 split PBSDiagnostic into canonical fields plus a nested probe
object, moving probe failure text to pbs.probe.error and adding
pbs.stateReason. sanitizeDiagnosticsData spreads the PBS entry and redacts
only host, name, id and the top-level error, so transport strings such as
"dial tcp 192.168.1.20:8007: connect: connection refused" stopped being
IP-redacted. That text used to live in pbs.error and was scrubbed; it now
rode unredacted into the sanitized bundle operators attach to support
requests.

The new fields are spread conditionally so an entry without them keeps its
exact previous shape, which leaves the existing strict-equality coverage
untouched.

Both proofs assert on the serialized bundle rather than on the sanitizer's
source, because the failure mode is a payload field added later that the
sanitizer never learned about. Verified to fail with the redaction removed,
reporting the raw address inside the exported JSON.
2026-07-24 22:25:46 +01:00
rcourtman 889d065528 Redact webhook URLs in the rate limit drop logs
f744e0700 moved webhook URL redaction onto the canonical
RedactWebhookURLSecrets helper and applied it at five log sites, but missed
two. checkWebhookRateLimit logged the raw webhookURL, and the enhanced
sender logged the raw webhook.URL on the same event. The enhanced path is
the one Gotify and other token-in-URL destinations actually take, and a
rate-limit drop is the event most likely to repeat for a misconfigured
destination, so the token reached the logs on every retry.

Both sites now redact. The reason the sweep missed them is that no test
asserted log content, so the new proof captures zerolog output and fails on
the raw token rather than reading the call sites, which a source scan cannot
do reliably.

Regression coverage: TestWebhookRateLimitLogsRedactURLSecrets, verified to
fail against the unredacted call site with the token visible in the captured
log line.
2026-07-24 22:22:52 +01:00
rcourtman afc0c865d4 Stop the availability projection writing into the live registry
removeAvailabilityProjectionLocked compacted each resource's relationships
in place with a Relationships[:0] filter. That is only safe if registry
clones own their backing array, and they did not: cloneResource deep-copies
roughly twenty-five slice and map fields but Relationships was left as the
shallow copy from the struct assignment, so a clone shared the original's
array.

The reachable path is ordinary. GET /api/resources rebuilds a registry from
Monitor.UnifiedResourceSnapshot, whose records come from GetAll clones, so
the filter wrote through into the live monitor registry from an HTTP
goroutine without holding that registry's lock. Every request performed
unconditional same-value writes, and whenever a legacy checks edge was
genuinely dropped the compaction left a duplicated trailing relationship on
the live host resource. The corrupting case is the post-upgrade seed the
projection exists to repair.

cloneResource now clones Relationships like every other slice, and the
filter builds a fresh slice so correctness no longer depends on each caller
having cloned first.

Regression coverage:
- TestCloneResourceGivesRelationshipsTheirOwnBackingArray pins the clone
  isolation, and fails with the clone line removed
- TestAvailabilityRepeatedProjectionKeepsHostRelationshipsStable drives the
  real ingest path across repeated re-projection
2026-07-24 22:19:47 +01:00
rcourtman 6c6deb8e2b Restore the TrueNAS pool topology discriminator
599c8e634 changed the TrueNAS pool projection to publish the ZFS data vdev
layout ("mirror", "raidz2", "stripe") as Storage.Topology. Topology is the
closed discriminator vocabulary every storage consumer keys on, and TrueNAS
pools travel as ResourceTypeStorage, so the frontend identifies them solely
by storage.topology === 'pool'. On any appliance whose pools report data
vdevs, every pool therefore fell out of the TrueNAS page, its datasets
orphaned, and per-system pool counts read zero.

Nothing caught it because internal/truenas/fixtures.go carries no VDevs, so
the projection still returned "pool" for every fixture while the producer
side pinned the new "mirror" value. Both suites stayed green and disagreed
with each other.

Pools now publish Topology "pool" again, with the layout in the new
Storage.VDevLayout field. poolVDevLayout returns an empty string when the
native report carries no data vdevs so callers fall back rather than
inventing a layout, and the presentation surfaces that previously read the
layout out of topology now prefer vdevLayout and fall back to topology.

Alert typing was unaffected (unified_eval.go already routes non-dataset
storage to truenas-pool through its default arm); incident metadata carried
the layout string and now carries the discriminator again.

Regression coverage pins the boundary from both sides, driving real vdev
layouts rather than the vdev-less fixtures:
- TestPoolTopologyStaysDiscriminatorAcrossVDevLayouts (producer, five layouts)
- TestStorageTopologyAndVDevLayoutAreDistinctIdentityFields (clone and wire)
- pool identity boundary cases in truenasPageModel.test.ts (consumer)
- topology label cases in storageAdapters.test.ts and the TrueNAS drawer model
2026-07-24 22:11:51 +01:00
rcourtman 24494a0afd Sort the registered alerts proof file into lexicographic order
The registry audit requires exact_files to be sorted, and 5715ec754 inserted
useAlertOverridesState.test.tsx before useAlertDestinationsTabState.test.tsx
rather than after it, which failed the audit.

Verified: registry_audit.py --check and contract_audit.py --check both clean.
2026-07-24 21:30:50 +01:00
rcourtman 5715ec754d Record the threshold section identity rule and register its proof
2118e3345 moved the Virtualization Hosts threshold section off a bare
`type === 'agent'` selection and onto provider-owned nodes, but landed without
the contract delta or a registered verification artifact, so the completion
guard blocked it.

The alerts contract now states the rule the fix depends on. Threshold sections
are keyed by override identity rather than resource type, and a resource that
appears in two sections has two different override keys, so a threshold saved
from the wrong row is written where alerting never reads it. Frontend
primitives records that surfaces classify through the shared
`isPulseAgentPlatformResource` classifier instead of re-deriving the standalone
versus provider-owned distinction from a raw resource type.

`useAlertOverridesState.test.tsx` already carried the right proof and simply
was not on the alerts-frontend-surface accepted list, so it is registered
rather than duplicated elsewhere.

Verified: canonical completion guard passes on the staged set, registry.json
parses.
2026-07-24 21:26:49 +01:00
rcourtman d82d3bd627 Record the wearout evidence boundary in contracts and proofs
Adds the subsystem contract deltas and verification artifacts for the
preceding wearout and physical-disk-node fixes, which landed without them.

The contracts now state the rule rather than leaving it implicit in each
caller. Wearout has two absent states and one real zero, -1 is the unreported
sentinel, and 0 is evidence only from a device that reports endurance at all.
storagehealth.WearoutReported is the single authority, and alerting, the
unified risk projection, and the frontend all gate on it. Storage-recovery
records that disk-to-node association is evidence-directional, so a disk with
no Proxmox scope still matches on node name while a scoped disk stays inside
its instance.

Proofs cover the predicate itself, a spent SSD alerting where it previously
stayed silent, absent evidence staying quiet on both the unreported sentinel
and a rotational zero, the view returning the sentinel rather than the struct
zero value, and the presentation helpers agreeing with the server.

Verified: full frontend suite 1061 files / 19405 tests, canonical completion
guard, gofmt, go build ./..., and internal/alerts, internal/monitoring,
internal/storagehealth, internal/unifiedresources.
2026-07-24 21:08:32 +01:00
rcourtman afc419ddab Add durable Proxmox node display names 2026-07-24 15:46:56 +01:00
rcourtman c41edb65a0 Fix agent command channel admission 2026-07-24 13:00:03 +01:00
rcourtman 478a9e933d Complete canonical web interface links
Preserve tenant-scoped metadata through partial URL updates and project stable URLs across runtime identities. Use a safe adjacent launch control across overview tables with desktop and mobile regression coverage.
2026-07-24 12:42:16 +01:00
rcourtman 599c8e6349 feat: add evidence-based pool health alerts 2026-07-24 12:40:05 +01:00
rcourtman 4a2335ce7f fix(ai): harden local compatible providers 2026-07-24 12:35:31 +01:00
rcourtman 9da4d4e966 Add powered-off alert tolerance 2026-07-24 12:23:29 +01:00
rcourtman 3a913752de Fix Proxmox cluster membership reconciliation 2026-07-24 11:21:27 +01:00
rcourtman 2ac70dab70 Fix audit storage migration and viewer races
Refs #1464
2026-07-24 10:33:57 +01:00
rcourtman 49217d284d fix: correct Proxmox guest rate sampling 2026-07-24 10:26:50 +01:00
rcourtman ac0fb263c2 Fix Proxmox workload refresh coherence 2026-07-24 10:11:56 +01:00
rcourtman cdb5797468 Expand metrics checkpoint profiling 2026-07-24 10:07:33 +01:00
rcourtman b811a7b331 Reduce metrics SQLite write amplification 2026-07-24 09:58:33 +01:00
rcourtman 4815a5c80d Fix PBS connection health authority 2026-07-24 09:45:05 +01:00
rcourtman 6e93cb3b5c Fix Unraid SMART probing and disk authority 2026-07-24 09:38:39 +01:00
rcourtman 96c35f2c35 Complete disk exclusion contracts 2026-07-24 09:25:39 +01:00
rcourtman bf67ba9201 Fix cache-aware Linux memory fallbacks 2026-07-24 00:19:40 +01:00
rcourtman b81ba7dd06 Migrate TrueNAS to supported JSON-RPC transport 2026-07-24 00:10:17 +01:00
rcourtman 0c17922055 Fix Docker report size contract drift 2026-07-23 23:57:42 +01:00
rcourtman c77571685d Bound long-lived frontend history caches 2026-07-23 23:54:40 +01:00
rcourtman 952e593d74 Fix agent update reconciliation across cached proxies 2026-07-23 23:50:28 +01:00
rcourtman 8d23529c02 Fix availability check identity projection 2026-07-23 23:46:34 +01:00
rcourtman b4877b4b44 Fix Proxmox agent registration authorization flow 2026-07-23 23:44:05 +01:00
rcourtman 3ad35750cb Optimize telemetry adoption reporting 2026-07-23 23:40:49 +01:00
rcourtman 0b6f7ad49a Fix notification delivery health reporting 2026-07-23 23:38:35 +01:00
rcourtman 3c6d933e9d Fix Proxmox cluster connection authority
Keep the configured cluster URL as primary and recover discovered member
endpoints asynchronously while it is healthy. Reset member reachability
evidence when the effective address changes.

Refs #1437 and #1493
2026-07-23 22:43:34 +01:00
rcourtman 15dbabfe9a Preserve resource URLs across runtime identity changes
Contract-Neutral: Docker URL migration touches shared monitor ingest files without changing the host deletion or re-enrollment contract.
2026-07-23 22:33:07 +01:00
rcourtman 0a95b25f41 Harden host agent re-enrollment lifecycle 2026-07-23 22:29:39 +01:00
rcourtman 87fe506c0a Bound state refresh and recovery ingest ownership 2026-07-23 22:24:52 +01:00
rcourtman d618fb8b74 Harden hypervisor guest CPU authority 2026-07-23 22:19:35 +01:00
rcourtman b8ea840f11 fix(monitoring): make Unraid task state freshness-safe 2026-07-23 22:17:25 +01:00
rcourtman d235aab3cf Fix RBAC assignment migration and loading 2026-07-23 22:11:48 +01:00
rcourtman 8c341753ae Normalize physical disk endurance evidence 2026-07-23 22:11:22 +01:00
rcourtman 13c4b57885 Fix VMware connection test degradation handling 2026-07-23 22:06:57 +01:00
rcourtman 615fa4418b Restore Docker agents to Hosts inventory 2026-07-23 22:06:56 +01:00
rcourtman 6a527ba9f2 Fix Proxmox physical disk inventory continuity 2026-07-23 22:05:09 +01:00
rcourtman b14cb376bc Prove resolved alert lock discipline
Route canonical cooldown refires through one resolved-state critical section and document the m.mu to resolvedMutex invariant. Exercise production refire, recovery, broadcast, alias repair, cleanup, and shutdown paths under the race detector while preserving cooldown history and start-time semantics.
2026-07-23 21:59:46 +01:00
rcourtman 5772589363 Prove TrueNAS alert threshold identity persistence 2026-07-23 21:54:57 +01:00
rcourtman 8580b48abc Fix Docker container update recreation lifecycle 2026-07-23 21:39:35 +01:00
rcourtman 2b7d202d8d Fix bounded Docker update checks
Keep manual check commands active until registry collection completes. Deduplicate replayed and concurrent commands, bound collection and acknowledgement retries, surface registry result counts, and prove timeout, rate-limit, and replay behavior.
2026-07-23 21:38:20 +01:00
rcourtman d7793a255d Fix TrueNAS boot and replication health classification 2026-07-23 21:37:26 +01:00
rcourtman e1a05104d9 Fix remote telemetry adoption reports 2026-07-23 20:38:09 +01:00
rcourtman 2ec5ef8d1a Record v6.1.1 release completion 2026-07-23 17:22:48 +01:00
rcourtman 5fde028276 Govern mobile Relay observability 2026-07-23 16:55:54 +01:00