The discovery-policy check resolved endpoint hostnames with a bare
net.LookupIP while the actual dials went through pkg/tlsutil's process-global
cached resolver, so the policy and the connection reasoned about two different
DNS views. That split is why 108aa4e20 had to skip resolution entirely for the
default policy, which left the injected 169.254.0.0/16 blocklist enforced only
against literal IPs: a hostname endpoint pointed at the metadata range walked
straight through.
Resolve through tlsutil.LookupHostCached instead. The shared resolver caches
answers and lookup failures alike until its next refresh, so repeat poll cycles
cost a cache hit rather than a query and the per-poll DNS volume that opened
#1638 stays gone. With that in place the default-policy skip is removed and the
blocklist applies to resolved addresses again, and the five-minute decision
cache is dropped rather than kept: it bought nothing on top of the resolver
cache, made the verdict trail the configuration, and memoized the fail-open
"resolution failed, allow" outcome for minutes even with an explicit allowlist
configured. Its claim to match a DNS refresh interval that operators configure
through DNS_CACHE_TIMEOUT goes with it.
The SSH backoffs now only escalate for work that ran. A knownhosts manager
suppressing a call inside its own window reports ErrKeyscanSuppressed, and the
temperature layer neither records a failure nor pays for the RPi fallback in
that case. An expired collection deadline is our own budget rather than
evidence about the host, so it holds the window at the floor. Both backoffs
decay once a retry deadline is more than one window past, and replacing the
temperature SSH key on disk clears both maps so a repaired key is tried on the
next cycle instead of after fifteen minutes.
Refs discussion #1638.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Recorded PVE 8 and PVE 9 guest rrddata responses (fixtures under
pkg/proxmox/testdata/rrd/) prove guest RRD never carries the cache-aware
memused/memavailable columns — they exist only in node RRD — so every
consumer branch reading them was dead code that #1634's listing fallback
(7d7d2b6a3) had already routed around.
Drop the two fields from GuestRRDPoint (now time/maxmem only, matching
the recordings), delete the dead VM RRD memory fallback and its
getVMRRDMetrics/getVMRRDMemory helpers plus the vmRRDMemCache they fed,
remove the pointless per-poll guest RRD fetch from the LXC memory path,
and retire the guest RRD lookups from PVEClientInterface. VMMemoryRaw
loses its never-populated RRD diagnostic fields, and guest reliability
scoring no longer treats the node-only rrd-* sources as trusted guest
evidence. The knownDeadGuestRRDFields allowlist in the fixture
alignment test is gone; a new reflection guard in
code_standards_test.go keeps GuestRRDPoint pinned to recorded columns,
and cleanupRRDCache pruning of the guest-agent meminfo cache gains
direct coverage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Issue #1634 happened because GuestRRDPoint declared memused/memavailable
columns that real PVE guest rrddata responses never contain (they exist
only in node RRD), and every test mocked the fictional columns, so CI
validated the assumption instead of the API.
Add recorded fixtures for the guest and node rrddata endpoints in both
generations: PVE 9 responses captured live from a pve-manager 9.2.3
cluster via pvesh, and PVE 8 responses recovered from the host's
pre-migration pve2-vm/pve2-node RRD databases, serialized the way
PVE::RRD::create_rrd_data emits them. Decode tests in pkg/proxmox now
replay these recordings through the real client and fail whenever a
NodeRRDPoint or GuestRRDPoint field references a column absent from
every recorded response; the two known-dead guest fields are held in an
explicit allowlist that also fails if they are removed or ever start
appearing in recordings.
Companion to 7d7d2b6a3, which restored the LXC listing fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add availability_probe_targets and availability_probe_agents to the
telemetry ping - counts only, no agent names or addresses - with the
disclosure table updated in both privacy doc copies. Document the
feature in the availability-checks configuration guide and the
unified agent guide, including the ICMP capability caveat for
containerized probes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The unified agent gains an availability module: probe assignments
arrive through the signed remote-config channel (missing key clears
the schedule), each enabled target runs on its own clamped interval
through the shared probe core, and results queue in a bounded
drop-oldest buffer. A result is offered to the primary server until
one delivery succeeds and never again after - buffered offline
reports are stripped of availability results so the disk buffer
cannot replay observations the queue still holds. ApplyHostReport
feeds accepted reports into the probe ingestion path, where the
ownership check and failure accounting live, and the probe agent id
is projected onto unified availability resources for source
attribution in the UI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add external_probe to the licensing feature enumeration, Pro tier
membership, self-hosted feature catalog, feature map, upgrade matrix
and pricing handoff, with the entitlement contract goldens updated.
The capability is served by the community binary - the entitlement
alone gates it, like relay. No existing free functionality changes:
local availability checks stay ungated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scheduled multi-guest vzdump jobs run under a single UPID whose VMID slot
is empty, so pollBackupTasks stored them with VMID 0 and the guest-centric
backups coverage view dropped them entirely: only individually backed-up
guests ever showed task status. (Regressed with the v6.0.0 guest-centric
redesign, which removed the flat task table that used to render job runs.)
pollBackupTasks now fetches the job task's log and parses the per-guest
markers ("Starting Backup of VM", "Finished Backup of VM (duration)",
"Backup of VM failed - reason") into synthetic per-guest BackupTask
entries. Their IDs embed the parent UPID, keeping them stable across polls
and distinct from individually-run backups; per-guest times are
reconstructed from the job start plus the printed durations. Finished
jobs' logs are immutable, so results are cached per instance|UPID and each
finished run is fetched at most once, with a per-cycle fetch cap so a
historical backlog trickles in without stalling the backup poll budget.
The task listing now uses source=all + typefilter=vzdump, so running jobs
are visible too: guests covered by an in-progress job get a "running"
synthetic task, which also feeds resolveBackupIntentContext and
suppresses offline/backup alerts for guests the job is actively backing
up. The frontend needs no changes - synthetic tasks carry real VMIDs and
flow through the existing coverage model, recovery mapper, and alert
intent paths.
Contract: monitoring.md completion obligation 13 records the per-guest
synthesis boundary; proofs land in monitor_backup_job_tasks_test.go,
monitor_alert_intent_test.go, and cluster_client_api_test.go.
Reported by Johannes Strasser (support thread "PBS Bug").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ceph Quincy and later (including Squid on PVE 9) dropped the monmap
mons array and the mgrmap active_name/standbys arrays from ceph status
output, replacing them with monmap.num_mons and mgrmap.num_standbys,
with quorum membership reported at the top level of the payload. Both
the host agent parser and the Proxmox API path only understood the
legacy arrays, so modern clusters showed 0 monitors and undercounted
managers.
- hostagent: read num_mons/num_standbys and top-level quorum data,
taking the largest available signal, and base the mon/mgr service
rows on the same counts
- pkg/proxmox: decode mgrmap num_standbys and top-level
quorum_names/quorum on CephStatus
- monitoring: fall back to the new fields when counting MON/MGR
daemons, and log Ceph 401/403 failures at warn level with a hint to
grant Sys.Audit on / instead of hiding them at debug
- models: prefer the larger non-zero MON/MGR counts when merging Ceph
cluster records from multiple sources
Fixes#1626, Refs discussion #1290
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A third pass on partially covered functions, led by the alert evaluation
predicates where a wrong arm means a missed or spurious alert.
- internal/alerts/specs: matches 47.9 to 100, and all six matches helpers
(severity threshold, change threshold, baseline anomaly, health assessment,
posture threshold, and the severity latch) from 50 to 75 percent up to 100.
Each threshold is pinned at, just below and just above, and the latch arm is
exercised both latched and unlatched with concrete verdicts.
- pkg/audit: exportCSV 76 to 88 with commas, quotes and newlines in the detail
field asserted through a parsed round-trip; NewSigner error arms, both
IsPersistent predicates and VerifySignature against a tampered payload and a
wrong key.
- internal/agentupdate: retryBackoffDelay, sleepWithContext, Snapshot and
writeSelfTestTokenFile to 100, the token file exercised under t.TempDir
including the unwritable-directory arm.
- internal/notifications: writeMultipartBodyPart and alertNodeDisplay to 100,
attachment handling to 69, all asserted on the produced MIME text. No test
opens a network or SMTP connection.
- internal/unifiedresources: the three pure action-dispatch helpers to 100.
- internal/alerts/config: CanonicalResourceTypeKeys 34.3 to 78.4.
Five targets deliberately did not move and are recorded rather than faked:
the error arms of writeEmailThreadingHeaders, buildMultipartEmailMessage and
copyWebhookConfig are unreachable because those functions write only into a
local bytes.Buffer, which never errors; exportJSON's only gap is a
json.MarshalIndent failure that its event struct cannot produce; and
verifyBinaryMagic's remaining gap is a deferred close-error handler.
No source file is modified. Adversarial review returned no rejects and flagged
seven near-duplicate subtests; all were removed and every target function
re-measured at an identical percentage.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
A second pass targeting PARTIALLY covered functions rather than untouched
ones, so every case here is an arm the existing suites never reached.
Percentages are per-function coverage, measured before and after.
- pkg/securityutil: the SSRF guards, which is where the uncovered arms
actually matter. isCarrierGradeNATIPv4 and isLocalNetworkIP are pinned at
the first and last address of 100.64.0.0/10 and just outside both ends,
across loopback, link-local, every RFC1918 range and IPv6 unique-local.
joinURLPath, IsLocalNetworkHost, resolveOutboundIPAddrs and
cloneRestrictedTransport to 100 percent, resolvePermittedOutboundIPs to
96.8, with the transport clone asserted independent of its source.
- internal/models: eleven deep-copy helpers from as low as 25 percent to 100.
Every one asserts real independence, mutating each nested slice, map and
pointer field of the clone and checking the original is untouched, which is
the failure mode a deep-copy helper actually has.
- internal/alerts: metricClearThreshold 28.6 to 100, resourceTypeLabel and
alertspecsMetricTriggered 50 to 100, the four canonical spec-id and
tracking-key builders 66.7 to 100, inferCanonicalKindFromLegacyAlert to 100,
and the backup-snapshot and ack-identity predicates.
- internal/servicediscovery: the four fingerprint generators to 100, each
asserted for both stability and sensitivity; the three command builders and
ValidateResourceID on their exact output and each rejection reason.
- internal/storagehealth: zfsScanActive and firstNonEmpty.
cephClusterSourceRank is deliberately left at 75 percent: its default arm is
unreachable because normalizeCephClusterSource can only return the two cases
above it. That is recorded rather than faked.
No source file is modified. Adversarial review returned no rejects and flagged
nine re-hit subtests; all nine were removed and every target function
re-measured at an identical percentage, proving they carried nothing.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
d235aab3c put MigrateFromFiles on a production path for the default org and
failed NewSQLiteManager whenever the import was rejected. That propagated to
GetManager, so every RBAC route returned 503 including ResetAdminRole, which
is the operator's only way back. The realistic trigger is a v5 install that
recreated a role the v6 store already holds under the same ID, so an
ordinary upgrade could leave an operator with no route to repair it short of
hand-editing the legacy JSON.
Rejecting the import is right and stays: importLegacyRBAC is transactional
and the legacy files are left in place, so a failure leaves the database
un-migrated rather than half-migrated. Missing roles deny access rather than
granting it, which is why the store is safe to keep serving.
The store now stays live and records the failure on MigrationError. The
deliberate fail-closed behaviour of the management surface is preserved
rather than removed: the handler manager accessors surface the migration
failure as the same 503 rbac_store_unavailable as before. Recovery reaches
the provider directly, so it is exempt by construction.
The existing 503 contract test is what caught the first attempt at this,
which simply let the surface serve un-migrated data. It now additionally
asserts recovery is reachable, so the two halves are pinned together.
Keep the configured cluster URL as primary and recover discovered member
endpoints asynchronously while it is healthy. Reset member reachability
evidence when the effective address changes.
Refs #1437 and #1493
Three new branch-coverage tests taking eighteen previously unreached functions
from 0.0% to covered.
internal/cloudcp/registry: the workspace limit error message on both the nil
and populated receiver, the active workspace count per account, the tenant
lookup across owning account, foreign account and missing rows, the invitation
listing by email including case and whitespace normalization, and the
invitation delete for both an existing pair and a pair that never existed.
internal/unifiedresources: the Docker container and Kubernetes node typed
views, the presentation listing and the metrics target, each asserted on the
nil receiver, the nil nested payload and the populated case, with returned
slices proven independent of the store.
pkg/db: the tracing wrappers around BeginTx, Query, QueryRow and their context
variants plus the pool setters, with exact histogram deltas proving the
observe wiring rather than that SQLite works.
All three files are new; no source or existing test was touched.
Contract-Neutral: test-only: new Go branch-coverage tests, no source or contract change
Retention deletes freed pages inside audit.db but the file never
shrank on disk, the same bloat class #1496 fixed for
unified_resources.db. Migrate existing databases to incremental
auto-vacuum at startup and return freed pages to the OS after each
retention pass, capped per cycle so a backlog drains gradually.
The metrics store capped its SQLite pool at one connection, so every UI
history read queued behind every buffered-write commit, and behind the
WAL checkpoints those commits pick up at the 4000-page threshold. On
write-heavy installs (many Docker agents with churning containers) that
serialization presented as sustained 120-260ms COMMIT warnings and an
unresponsive UI even with idle CPU and fast disks. Writes were never the
risk: flush, rollup, retention, and maintenance already funnel through
the single background worker goroutine, and the WriteBatchSync poller
path serializes on the WAL write lock via busy_timeout.
Raising the pool exposed a second bug: auto_vacuum(INCREMENTAL) in the
per-connection DSN pragmas replays as a database-header write whenever
the pool opens a new connection, which blocks connection creation behind
the active writer for up to the full 30s busy_timeout. auto_vacuum is a
persistent database property that migrateAutoVacuum already establishes
once at startup, so the per-connection copy is dropped.
Refs #1601
Contract-Neutral: behavioral fix: metrics store read concurrency and per-connection auto_vacuum pragma removal, no public contract delta (#1601)
Test-only wave, contract-neutral. New *_branchcov0719pm_test.go files cover
previously-uncovered pure value-in/value-out helpers, each verified to move
its target functions from 0% to full coverage:
- internal/agentcontext: formatKubernetesServicePorts (empty/single/cap/overflow
arms) and addMetricFact (nil-metric, percent/value/ratio arms) now 100%.
- pkg/reporting: reportLogoTypeFromPath 0->100, reportLogoTypeFromData 28.6->100,
scaledLogoSize 70->90 (extension and aspect branches).
- internal/cloudcp/email: RenderMagicLinkEmail 0->80 (render success path; the
compile-time template-error arm is unreachable and left uncovered).
- internal/recovery: recoveryDetailString (nil map, missing key, non-string,
string arms) and recoveryPointObservedAt 40->100.
- internal/ai/tools: ErrStrictResolution/ErrRoutingMismatch ToToolResponse 100.
- internal/ai/providers: every NormalizeCollections receiver 0->100.
No source or existing test modified.
New *_branchcov0718_test.go files raise coverage of previously-uncovered pure
functions across ten packages. Covered areas include securityutil SSRF and URL
validation, truenas path and telemetry parse helpers, storagehealth SMART and
physical-disk risk assessment, vmware inventory sort keys and error classifiers,
servicediscovery token filtering and readiness, telemetry evidence-from-history,
models ToFrontend converters and frontend NormalizeCollections normalizers,
actionplanner type predicates and canonical resource-id sort, config API-token
accessors, and licensing state accessors.
Test-only, with no runtime or subsystem-contract change. Verified in a clean
worktree at HEAD with go vet and package tests green, gofmt clean, and every
named target function moved from 0 percent to covered.
The 30d window said 18 of 26 approved action attempts failed but
carried zero cause information. Every approved attempt that is not a
verified success now lands in exactly one content-free bucket:
- pre_dispatch: terminally refused before dispatch (plan drift, expiry,
emergency stop, policy authorization)
- execution: dispatched execution failed or ended inconclusive
- unverified: execution succeeded but outcome verification was not
confirmed (reads as success in the UI, counted as failure here)
- stuck_executing: still executing over an hour after dispatch
plus one sanitized machine reason code for the most recent failure.
Successes, failure buckets, and recent in-flight attempts partition the
attempt count, so the next window explains its own gap.
RefuseActionExecution now persists the specific refusal code
(plan_drift, action_plan_expired, ...) as the canonical execution
reason code instead of the generic pre_dispatch_refused, so audit truth
and telemetry distinguish refusal causes without message parsing.
Privacy disclosures updated in both PRIVACY.md copies; the adoption
report script surfaces the new counters.
Contract-Neutral: additive content-free telemetry counters + specific pre-dispatch refusal reason codes; privacy disclosures updated in-commit per security-privacy extension point
The restricted outbound dialer resolved a hostname, validated the results,
then dialed only the first permitted IP. When localhost resolves to ::1
ahead of 127.0.0.1 and the target service binds only 127.0.0.1 (Ollama's
default), the dial failed with 'connect: connection refused' even though
curl and browsers connect fine via address-family fallback. This broke the
AI provider setup quickstart: the blessed http://localhost:11434 Ollama URL
failed Pulse's connection test on IPv6-first hosts (setup-friction class of
#847/#1003).
Try each permitted IP from the validating resolution in order until one
connects. Every candidate is still validated against the outbound policy
from the same resolution, so the DNS-rebinding pinning guarantee is
unchanged. Fixes all consumers of the shared client (Ollama provider, OIDC,
SSO, connection probes, availability poller); api-contracts wording updated
to match. Live-verified against Ollama 0.32.1 bound to 127.0.0.1 only:
TestConnection via http://localhost:11434 failed before, passes after.
The docker container lifecycle codec in agentexec had a contract test that never
exercised the codec functions at runtime, leaving decode, bind, validate and
identity helpers at zero coverage. This adds a table-driven test taking each to
near or full coverage including every validation error arm. In aicontracts,
CloneActionReference, IsNilAlertPayload and DefaultEngineConfig were uncovered,
so this covers the nil and populated arms and asserts CloneActionReference
produces an independent deep copy.
Both are new test-only files. No source changed. Verified with go test, gofmt
and go vet, with each target function confirmed to move off zero coverage.
Cover previously-unexecuted branches in modules that gained behavior
since the last wave. All six are newly added *_w0716_coverage_test.go
files that modify no source and no existing test.
- internal/ai/chat investigation-budget injection plus the agentic
tool-event and string helpers
- internal/ai/qualification manifest Validate error arms and the Patrol
docker-predicate and finding-prerequisite validators
- internal/unifiedresources adapters transforms and ContractResourceType
- internal/config AIConfig nil-receiver default branches
- pkg/aicontracts orchestrator error methods and investigation config
Adds table-driven branch-coverage unit tests for previously uncovered pure
functions across internal/alerts/config, internal/ai/modelresolution,
internal/monitoring, internal/ai, internal/ai/qualification and pkg/licensing.
New test files only, with no source changes.
Covers per-subsystem alert-default normalization, configured model and provider
resolution, fleet-doctor identity and cluster-endpoint helpers, docker-state and
infrastructure-key mapping, investigation and SMART issue helpers, JSON and
autonomy normalization, and licensing feature-tier resolution. 65 TestBranchCov
functions in 7 files, all vet and gofmt clean.
Adds table-driven branch-coverage unit tests for previously uncovered pure
functions across internal/models, internal/config, pkg/securityutil and
pkg/aicontracts. New test files only, with no source changes.
Covers model collection normalizers, reflect-based metric deep-copy,
removed-host agent bookkeeping, report-schedule and entitlement-billing
normalization, effective AI control and Patrol autonomy resolution,
websocket-origin host normalization, and command-safety classifiers. 61
TestBranchCov functions in 8 files, all vet and gofmt clean.
Adds table-driven branch-coverage unit tests for previously untested pure
helper functions across internal/alerts/config, internal/config,
internal/ai/safety, internal/ai/modelresolution, internal/operationreceipt,
internal/models, internal/securityutil and pkg/securityutil. New test files
only, with no source changes.
Covers alert-config normalization and validation, sensitive-path and
redaction classifiers, URL normalizers, provider model resolution,
operation-receipt decoding, credential masking, and account-to-org role
mapping. 57 TestBranchCov functions in 12 files, all vet and gofmt clean.
Fleets on networks numbered from nominally public IP space (issue
#1522: an AD estate on 192.20.0.0/16) cannot pass the agent's
local-network plaintext heuristic, and the only workaround was pointing
a .internal DNS alias at the server, which bypasses the same control
less visibly than a flag would. --allow-plaintext-http
(PULSE_AGENT_ALLOW_PLAINTEXT_HTTP) records process-wide consent once at
agent startup before any module validates a URL, covers every agent
transport including the websocket command channel, warns at startup
that the API token travels in cleartext, defaults closed, and is never
emitted by generated install commands or settable by the server.
courtmanr@gmail.com