99ad8c2c4 routed collectDiskIO through fsfilters.IsVirtualBlockDevice to
stop ZFS zvols being collected. That helper answers "can this device report
SMART", which is the wrong question for I/O accounting: it also matches
vd* and xvd*, so every agent running inside a KVM, Proxmox or Xen VM lost
host disk I/O entirely, since vda is the real disk on those machines. It
also dropped md, nbd, rbd, drbd and pmem, none of which were part of the
report.
Add IsNonPhysicalDiskIODevice for the I/O question and use it here. It
covers only loop, ram, zram, dm- and zd, so the zvol fix for #1671 stands
while everything that was collected before 99ad8c2c4 is collected again.
Tests assert vda and xvda survive the I/O filter while still being excluded
from SMART, so the two predicates cannot be conflated again.
Contract-Neutral: behavioural regression fix, no contract delta
Refs #1671
Two mandatory round-trips stood between an interested MSP and their first
screen, and neither was technical.
setup.sh required four image digests shipped as literal <pin>
placeholders, so the only way to get them was to ask. All four images are
publicly readable, so there was never anything to hand out. setup.sh now
resolves each blank pin to an immutable digest from its published tag via
buildx imagetools and writes it back to .env; hand-set values are left
alone.
setup.sh then died outright without a licence file, so nobody could start
the stack, create a workspace, or see the portal until a human minted a
licence for them. The control plane already ran unlicensed via
ProviderMSPPlanSourceEnvFallback; only the installer refused. A licence
path that is set but missing is still a hard failure, since that is a
misconfiguration rather than a choice.
Unlicensed now means evaluation rather than the cheapest paid tier. The
env fallback defaulted to msp_starter, handing every unlicensed
deployment the full 5-client Starter allowance and leaving no boundary
between evaluating and buying. Adds msp_eval at 2 workspaces: same
capabilities, smaller cap, not purchasable, not on the public ladder.
An isolation guarantee is the one claim a provider cannot evaluate from a
screenshot, and both MSP leads this year went quiet at exactly this step.
Contracts: cloud-paid records the unlicensed plan rule and the
strictly-below-paid invariant; deployment-installability records
credential-free, correspondence-free installability.
Verification: TestMSPEvalCapStaysBelowCheapestPaidTier,
TestCanonicalizePlanVersion_MSPEval,
TestProviderMSPSetupScriptSupportsUnlicensedEvaluation. The last was
negative-tested by reintroducing a <pin> placeholder and confirming it
fails. ensure_image_pins exercised against the live registries.
licensing, cloudcp, control-plane and installtests all green.
golangci-lint had accumulated 12 findings since 5abb2d8f4. All fixed with
real dedup (no nolint suppressions) and the repo's existing errcheck idioms:
- dupl internal/monitoring: docker/host identity-conflict trackers were
structural clones; extracted a shared identityFlapTracker core with a
domain-neutral identityConflict result. Per-domain files now hold only
the window const and the model translation. Tracker-behavior tests
consolidated into identity_flap_tracker_test.go; Monitor-level
translation and Apply*Report integration tests remain per domain.
- dupl internal/api/router.go: VM/container workload chart loops shared a
16-line live-fallback block; extracted guestChartSeriesWithLiveFallback
over a guestLiveMetricsView interface both views satisfy.
- dupl internal/storagehealth/risk.go: SMART attribute copying extracted
into applySMARTAttributes shared by both assessors (same
*models.SMARTAttributes type on both inputs).
- errcheck pkg/audit/sqlite_logger.go: three defer tx.Rollback() sites
now use the repo-wide defer func() { _ = tx.Rollback() }() idiom.
- errcheck telemetry/notifications tests: send() errors now fail the
test; queue.Stop() uses the package's _ = idiom.
Full test suites pass for all six touched packages.
Contract-Neutral: lint-hygiene restoration: dupl dedup (identical logic extracted to shared helpers) and errcheck idiom fixes; no public-contract or behavioral delta
A node joining a PVE cluster after setup could never be trusted when the
primary was fingerprint-pinned (#1664). Two independent gaps: cluster
discovery validation only relaxed TLS when client construction failed,
but a pinned-fingerprint mismatch surfaces from the first API call, so
the member was judged not-a-Proxmox-node and its captured fingerprint
discarded; and the cluster client's TOFU refresh only ran for endpoints
that already had a per-endpoint fingerprint, handling rotation but never
first trust. Validation now retries with the member's own captured
fingerprint before rejecting it, TOFU capture runs on first use, and
discovery failures now distinguish DNS, refused, timeout, and TLS
causes in the endpoint error instead of one generic message.
Contract-Neutral: cluster discovery TLS validation bug fix; no wire contract or payload change
Transport-level failures from httpClient.Do (connection refused/reset,
DNS, TLS, timeouts) were wrapped as plain errors, so they fell through
userFriendlyActivationError's LicenseServerError branch to the generic
contact-support fallback instead of the retryable temporarily-unavailable
message. Users behind flaky networks got a dead-end message, and the
2026-08-01 rc.6 paid-runtime gate failure was misdiagnosed the same way.
Classify Do failures at the client layer as a retryable LicenseServerError
(code license_server_unreachable, StatusCode 0 so revocation/suspension/
migration status gates never match) with the cause reachable via Unwrap so
the status poller's context.Canceled check keeps working. The cloud-paid
transport boundary contract now names this classification. Tests pin the
classification, the cancellation chain, and the end-to-end user-facing
retryable message.
The net.IP predicates every SSRF guard is built from read only the literal
address bytes, so an IPv6 transition address smuggles an internal IPv4
destination past all of them. 64:ff9b::a9fe:a9fe reaches 169.254.169.254
while IsLoopback, IsPrivate, IsLinkLocalUnicast and To4 all report an
ordinary public address, defeating both the webhook URL validator and the
restricted outbound transport with the same input.
Add securityutil.EmbeddedIPv4Candidates, which unwraps NAT64 (RFC 6052
well-known and RFC 8215 local-use prefixes), 6to4, Teredo, ISATAP,
IPv4-compatible and IPv4-translated encodings, and run every candidate
through the caller's own policy in both layers. The embedded destination
inherits the outer policy rather than a stricter one, so a NAT64 address
wrapping a permitted public target stays permitted and AllowPrivateIPs /
AllowLoopback relax the embedded check the same way they relax the outer.
Reported by tonghuaroot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Use a native absolute path in the SMART override test so Windows verifies the same contract as Unix. Move the metrics concurrency proof into the canonical verification artifact and document that live ingestion remains independent from lifecycle maintenance.
Run startup maintenance, rollups, and retention on a lifecycle-owned worker separate from buffered metric ingestion. SQLite still serializes write transactions, while maintenance CPU/read work can no longer stop writeCh from draining until it drops live batches. Join both workers before closing the shared database.\n\nFixes #1601
The discovery-policy check resolved endpoint hostnames with a bare
net.LookupIP while the actual dials went through pkg/tlsutil's process-global
cached resolver, so the policy and the connection reasoned about two different
DNS views. That split is why 108aa4e20 had to skip resolution entirely for the
default policy, which left the injected 169.254.0.0/16 blocklist enforced only
against literal IPs: a hostname endpoint pointed at the metadata range walked
straight through.
Resolve through tlsutil.LookupHostCached instead. The shared resolver caches
answers and lookup failures alike until its next refresh, so repeat poll cycles
cost a cache hit rather than a query and the per-poll DNS volume that opened
#1638 stays gone. With that in place the default-policy skip is removed and the
blocklist applies to resolved addresses again, and the five-minute decision
cache is dropped rather than kept: it bought nothing on top of the resolver
cache, made the verdict trail the configuration, and memoized the fail-open
"resolution failed, allow" outcome for minutes even with an explicit allowlist
configured. Its claim to match a DNS refresh interval that operators configure
through DNS_CACHE_TIMEOUT goes with it.
The SSH backoffs now only escalate for work that ran. A knownhosts manager
suppressing a call inside its own window reports ErrKeyscanSuppressed, and the
temperature layer neither records a failure nor pays for the RPi fallback in
that case. An expired collection deadline is our own budget rather than
evidence about the host, so it holds the window at the floor. Both backoffs
decay once a retry deadline is more than one window past, and replacing the
temperature SSH key on disk clears both maps so a repaired key is tried on the
next cycle instead of after fifteen minutes.
Refs discussion #1638.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Recorded PVE 8 and PVE 9 guest rrddata responses (fixtures under
pkg/proxmox/testdata/rrd/) prove guest RRD never carries the cache-aware
memused/memavailable columns — they exist only in node RRD — so every
consumer branch reading them was dead code that #1634's listing fallback
(7d7d2b6a3) had already routed around.
Drop the two fields from GuestRRDPoint (now time/maxmem only, matching
the recordings), delete the dead VM RRD memory fallback and its
getVMRRDMetrics/getVMRRDMemory helpers plus the vmRRDMemCache they fed,
remove the pointless per-poll guest RRD fetch from the LXC memory path,
and retire the guest RRD lookups from PVEClientInterface. VMMemoryRaw
loses its never-populated RRD diagnostic fields, and guest reliability
scoring no longer treats the node-only rrd-* sources as trusted guest
evidence. The knownDeadGuestRRDFields allowlist in the fixture
alignment test is gone; a new reflection guard in
code_standards_test.go keeps GuestRRDPoint pinned to recorded columns,
and cleanupRRDCache pruning of the guest-agent meminfo cache gains
direct coverage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Issue #1634 happened because GuestRRDPoint declared memused/memavailable
columns that real PVE guest rrddata responses never contain (they exist
only in node RRD), and every test mocked the fictional columns, so CI
validated the assumption instead of the API.
Add recorded fixtures for the guest and node rrddata endpoints in both
generations: PVE 9 responses captured live from a pve-manager 9.2.3
cluster via pvesh, and PVE 8 responses recovered from the host's
pre-migration pve2-vm/pve2-node RRD databases, serialized the way
PVE::RRD::create_rrd_data emits them. Decode tests in pkg/proxmox now
replay these recordings through the real client and fail whenever a
NodeRRDPoint or GuestRRDPoint field references a column absent from
every recorded response; the two known-dead guest fields are held in an
explicit allowlist that also fails if they are removed or ever start
appearing in recordings.
Companion to 7d7d2b6a3, which restored the LXC listing fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add availability_probe_targets and availability_probe_agents to the
telemetry ping - counts only, no agent names or addresses - with the
disclosure table updated in both privacy doc copies. Document the
feature in the availability-checks configuration guide and the
unified agent guide, including the ICMP capability caveat for
containerized probes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The unified agent gains an availability module: probe assignments
arrive through the signed remote-config channel (missing key clears
the schedule), each enabled target runs on its own clamped interval
through the shared probe core, and results queue in a bounded
drop-oldest buffer. A result is offered to the primary server until
one delivery succeeds and never again after - buffered offline
reports are stripped of availability results so the disk buffer
cannot replay observations the queue still holds. ApplyHostReport
feeds accepted reports into the probe ingestion path, where the
ownership check and failure accounting live, and the probe agent id
is projected onto unified availability resources for source
attribution in the UI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add external_probe to the licensing feature enumeration, Pro tier
membership, self-hosted feature catalog, feature map, upgrade matrix
and pricing handoff, with the entitlement contract goldens updated.
The capability is served by the community binary - the entitlement
alone gates it, like relay. No existing free functionality changes:
local availability checks stay ungated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scheduled multi-guest vzdump jobs run under a single UPID whose VMID slot
is empty, so pollBackupTasks stored them with VMID 0 and the guest-centric
backups coverage view dropped them entirely: only individually backed-up
guests ever showed task status. (Regressed with the v6.0.0 guest-centric
redesign, which removed the flat task table that used to render job runs.)
pollBackupTasks now fetches the job task's log and parses the per-guest
markers ("Starting Backup of VM", "Finished Backup of VM (duration)",
"Backup of VM failed - reason") into synthetic per-guest BackupTask
entries. Their IDs embed the parent UPID, keeping them stable across polls
and distinct from individually-run backups; per-guest times are
reconstructed from the job start plus the printed durations. Finished
jobs' logs are immutable, so results are cached per instance|UPID and each
finished run is fetched at most once, with a per-cycle fetch cap so a
historical backlog trickles in without stalling the backup poll budget.
The task listing now uses source=all + typefilter=vzdump, so running jobs
are visible too: guests covered by an in-progress job get a "running"
synthetic task, which also feeds resolveBackupIntentContext and
suppresses offline/backup alerts for guests the job is actively backing
up. The frontend needs no changes - synthetic tasks carry real VMIDs and
flow through the existing coverage model, recovery mapper, and alert
intent paths.
Contract: monitoring.md completion obligation 13 records the per-guest
synthesis boundary; proofs land in monitor_backup_job_tasks_test.go,
monitor_alert_intent_test.go, and cluster_client_api_test.go.
Reported by Johannes Strasser (support thread "PBS Bug").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ceph Quincy and later (including Squid on PVE 9) dropped the monmap
mons array and the mgrmap active_name/standbys arrays from ceph status
output, replacing them with monmap.num_mons and mgrmap.num_standbys,
with quorum membership reported at the top level of the payload. Both
the host agent parser and the Proxmox API path only understood the
legacy arrays, so modern clusters showed 0 monitors and undercounted
managers.
- hostagent: read num_mons/num_standbys and top-level quorum data,
taking the largest available signal, and base the mon/mgr service
rows on the same counts
- pkg/proxmox: decode mgrmap num_standbys and top-level
quorum_names/quorum on CephStatus
- monitoring: fall back to the new fields when counting MON/MGR
daemons, and log Ceph 401/403 failures at warn level with a hint to
grant Sys.Audit on / instead of hiding them at debug
- models: prefer the larger non-zero MON/MGR counts when merging Ceph
cluster records from multiple sources
Fixes#1626, Refs discussion #1290
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A third pass on partially covered functions, led by the alert evaluation
predicates where a wrong arm means a missed or spurious alert.
- internal/alerts/specs: matches 47.9 to 100, and all six matches helpers
(severity threshold, change threshold, baseline anomaly, health assessment,
posture threshold, and the severity latch) from 50 to 75 percent up to 100.
Each threshold is pinned at, just below and just above, and the latch arm is
exercised both latched and unlatched with concrete verdicts.
- pkg/audit: exportCSV 76 to 88 with commas, quotes and newlines in the detail
field asserted through a parsed round-trip; NewSigner error arms, both
IsPersistent predicates and VerifySignature against a tampered payload and a
wrong key.
- internal/agentupdate: retryBackoffDelay, sleepWithContext, Snapshot and
writeSelfTestTokenFile to 100, the token file exercised under t.TempDir
including the unwritable-directory arm.
- internal/notifications: writeMultipartBodyPart and alertNodeDisplay to 100,
attachment handling to 69, all asserted on the produced MIME text. No test
opens a network or SMTP connection.
- internal/unifiedresources: the three pure action-dispatch helpers to 100.
- internal/alerts/config: CanonicalResourceTypeKeys 34.3 to 78.4.
Five targets deliberately did not move and are recorded rather than faked:
the error arms of writeEmailThreadingHeaders, buildMultipartEmailMessage and
copyWebhookConfig are unreachable because those functions write only into a
local bytes.Buffer, which never errors; exportJSON's only gap is a
json.MarshalIndent failure that its event struct cannot produce; and
verifyBinaryMagic's remaining gap is a deferred close-error handler.
No source file is modified. Adversarial review returned no rejects and flagged
seven near-duplicate subtests; all were removed and every target function
re-measured at an identical percentage.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
A second pass targeting PARTIALLY covered functions rather than untouched
ones, so every case here is an arm the existing suites never reached.
Percentages are per-function coverage, measured before and after.
- pkg/securityutil: the SSRF guards, which is where the uncovered arms
actually matter. isCarrierGradeNATIPv4 and isLocalNetworkIP are pinned at
the first and last address of 100.64.0.0/10 and just outside both ends,
across loopback, link-local, every RFC1918 range and IPv6 unique-local.
joinURLPath, IsLocalNetworkHost, resolveOutboundIPAddrs and
cloneRestrictedTransport to 100 percent, resolvePermittedOutboundIPs to
96.8, with the transport clone asserted independent of its source.
- internal/models: eleven deep-copy helpers from as low as 25 percent to 100.
Every one asserts real independence, mutating each nested slice, map and
pointer field of the clone and checking the original is untouched, which is
the failure mode a deep-copy helper actually has.
- internal/alerts: metricClearThreshold 28.6 to 100, resourceTypeLabel and
alertspecsMetricTriggered 50 to 100, the four canonical spec-id and
tracking-key builders 66.7 to 100, inferCanonicalKindFromLegacyAlert to 100,
and the backup-snapshot and ack-identity predicates.
- internal/servicediscovery: the four fingerprint generators to 100, each
asserted for both stability and sensitivity; the three command builders and
ValidateResourceID on their exact output and each rejection reason.
- internal/storagehealth: zfsScanActive and firstNonEmpty.
cephClusterSourceRank is deliberately left at 75 percent: its default arm is
unreachable because normalizeCephClusterSource can only return the two cases
above it. That is recorded rather than faked.
No source file is modified. Adversarial review returned no rejects and flagged
nine re-hit subtests; all nine were removed and every target function
re-measured at an identical percentage, proving they carried nothing.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
d235aab3c put MigrateFromFiles on a production path for the default org and
failed NewSQLiteManager whenever the import was rejected. That propagated to
GetManager, so every RBAC route returned 503 including ResetAdminRole, which
is the operator's only way back. The realistic trigger is a v5 install that
recreated a role the v6 store already holds under the same ID, so an
ordinary upgrade could leave an operator with no route to repair it short of
hand-editing the legacy JSON.
Rejecting the import is right and stays: importLegacyRBAC is transactional
and the legacy files are left in place, so a failure leaves the database
un-migrated rather than half-migrated. Missing roles deny access rather than
granting it, which is why the store is safe to keep serving.
The store now stays live and records the failure on MigrationError. The
deliberate fail-closed behaviour of the management surface is preserved
rather than removed: the handler manager accessors surface the migration
failure as the same 503 rbac_store_unavailable as before. Recovery reaches
the provider directly, so it is exempt by construction.
The existing 503 contract test is what caught the first attempt at this,
which simply let the surface serve un-migrated data. It now additionally
asserts recovery is reachable, so the two halves are pinned together.
Keep the configured cluster URL as primary and recover discovered member
endpoints asynchronously while it is healthy. Reset member reachability
evidence when the effective address changes.
Refs #1437 and #1493
Three new branch-coverage tests taking eighteen previously unreached functions
from 0.0% to covered.
internal/cloudcp/registry: the workspace limit error message on both the nil
and populated receiver, the active workspace count per account, the tenant
lookup across owning account, foreign account and missing rows, the invitation
listing by email including case and whitespace normalization, and the
invitation delete for both an existing pair and a pair that never existed.
internal/unifiedresources: the Docker container and Kubernetes node typed
views, the presentation listing and the metrics target, each asserted on the
nil receiver, the nil nested payload and the populated case, with returned
slices proven independent of the store.
pkg/db: the tracing wrappers around BeginTx, Query, QueryRow and their context
variants plus the pool setters, with exact histogram deltas proving the
observe wiring rather than that SQLite works.
All three files are new; no source or existing test was touched.
Contract-Neutral: test-only: new Go branch-coverage tests, no source or contract change
Retention deletes freed pages inside audit.db but the file never
shrank on disk, the same bloat class #1496 fixed for
unified_resources.db. Migrate existing databases to incremental
auto-vacuum at startup and return freed pages to the OS after each
retention pass, capped per cycle so a backlog drains gradually.
The metrics store capped its SQLite pool at one connection, so every UI
history read queued behind every buffered-write commit, and behind the
WAL checkpoints those commits pick up at the 4000-page threshold. On
write-heavy installs (many Docker agents with churning containers) that
serialization presented as sustained 120-260ms COMMIT warnings and an
unresponsive UI even with idle CPU and fast disks. Writes were never the
risk: flush, rollup, retention, and maintenance already funnel through
the single background worker goroutine, and the WriteBatchSync poller
path serializes on the WAL write lock via busy_timeout.
Raising the pool exposed a second bug: auto_vacuum(INCREMENTAL) in the
per-connection DSN pragmas replays as a database-header write whenever
the pool opens a new connection, which blocks connection creation behind
the active writer for up to the full 30s busy_timeout. auto_vacuum is a
persistent database property that migrateAutoVacuum already establishes
once at startup, so the per-connection copy is dropped.
Refs #1601
Contract-Neutral: behavioral fix: metrics store read concurrency and per-connection auto_vacuum pragma removal, no public contract delta (#1601)
rcourtman