A password login can end while its response is being read: the caller
cancels, its deadline passes, or the client's own login timeout expires.
Go's HTTP transport can still hand back the server's answer to that
abandoned request. The construction fixture answers a departed client
with an empty 200, which surfaced as "invalid authentication response"
and failed TestClientConstructionHonoursContext and its client-timeout
twin intermittently in the required backend check.
The shared password exchange now reports the ending instead of
diagnosing that answer, keeps transport errors that already carry the
ending, and starts no form attempt after an ended JSON refusal. This
mirrors the PBS client fix from 18 October. Renewal shares the exchange,
so the same rule applies when its own bounded timeout ends a login.
Complete sessions are still accepted, and construction still discards a
client whose caller has ended.
A new control ends the login during response reading, by caller
cancellation and by a simulated login deadline, for every unusable
answer; without this change all of its cases fail. It also checks that
both encoding attempts carry the one login budget the client timeout
sets. The TLS fixture now aborts a request whose client has left rather
than answering it with an empty 200.
Patrol investigations inferred their outcome from prose markers, so a
bold or bulleted RESOLVED line became the terminal needs_attention and
the finding was never investigated again. Timeouts clustered on
subscription agents. Patrol's 60s stream idle timeout bounds a whole
buffered CLI turn there, three investigations queued on one serial CLI
slot, and the final summary call was capped at 20s.
- Add patrol_conclude_investigation, an investigation-only,
side-effect-free capture of resolved, cannot_fix or needs_attention.
It is carried on InvestigationRunResult and
OrchestratorInvestigationResult.
- After a conclusion, the next turn is the summary. The final turn
offers only the tools that end a run. A deadline reserve makes the
current turn final before the run times out, and the summary call
may use the time left.
- Give investigation turns on subscription agents a 4-minute turn
budget, a 25-minute default timeout and one investigation at a time.
These limits are republished on config reload without holding the
service lock across the Patrol lock.
- Derive Patrol's outer deadline from the effective timeout. Sessions
now carry their own deadline, and the stuck-run sweep measures a live
run against it, so neither a lowered timeout nor the previous run's
time resets a running investigation.
- Record a run orphaned by a restart as interrupted, not timed out. A
first investigation cut short that way used to stay running for
good, which also blocked a manual re-run. Recovery applies only while
the finding still shows that run, takes a result the run's session
already recorded, and keeps an action outcome such as fix_queued.
- Add unclassified and interrupted outcomes, with lifecycle, telemetry
bucket, digest and record handling.
Five stores set private page caches of 16 to 64 MB per connection.
The modernc driver allocates that cache with anonymous mmap outside
the Go heap, so GC and GOMEMLIMIT never see it, and each page costs
about twice its size. Measured: reading a 107 MB database through a
64 MB cache_size added 117 MB of resident memory, against 6 MB at
SQLite's 2 MB default. On a long-running dogfood install about 165 MB
of 360 MB resident sat outside the Go heap, with a 126 MB audit
database behind a 64 MB cache (#1803).
Audit, RBAC, notification queue, unified resources and the alert
event log (writer and both readers) now take pdb.PageCachePragma.
Pages that miss it come from the kernel file cache, which the kernel
can reclaim. The audit suite, including its large legacy migration,
takes the same time as before (309 s on both trees).
pkg/db tests reject any other cache_size in production code and bound
the resident growth of a 30 MB scan. Each store's test pins its
effective value. Negative controls: restoring the audit cache fails
the source scan, and a 64 MB reader cache grows a scan by 55 MB
against a 16 MB budget. modernc sorts _pragma values before applying
them, so a second cache_size in one DSN cannot reliably override.
Related to #1803
Every Buy, Upgrade and Renew action opened the Pulse Account portal on cloud.pulserelay.pro, which has returned a Cloudflare 522 since hosted Cloud was hibernated on 2026-06-01, so no one could buy from inside Pulse. The licence server now serves the same handoff-to-checkout page at https://license.pulserelay.pro/buy, using the existing portal handoff and checkout session APIs, so payment still returns to the instance and activates it.
Raw fleet reads copied resource IDs across SQLite for every observation and
merged tier branches by those repeated strings. Bind the requested scope once,
return statement-local resource numbers and restore each detached series to
stable timestamp order in the shared reader. Preserve indexed tier coverage,
metric filters, chunk limits and ordered streamed display buckets.
Bind the source tier and time window once for both clustered rollup stages.
Reusing the ordinary aggregate's six SQLite slots removes the duplicated
argument array and boxing that exceeded the small-rollup allocation budget.
Preserve indexed ranges, aggregation results and the legacy query path.
Use the covering time index to discover resources in each rollup window,
then aggregate their contiguous primary-key ranges. Group on a query-local
resource number to avoid repeating composite identity comparisons for every
point. Preserve the legacy aggregate until its physical migration completes.
Keep nullable extrema, bucket boundaries, resource identities and idempotence
equivalent to the legacy aggregate, and guard both indexed query stages.
Retained chart reads found each sample in the identity index and then fetched its payload from the rowid table. Store observations in the time-major primary table to remove that second lookup while preserving sample identity and retention.
Migrate installed stores through deferred atomic WAL maintenance, preserving old read snapshots, queued writes, nullable bounds, tier reconciliation and rollup metadata. Share the rewrite deadline with auto-vacuum and retain the installed schema when capacity or migration checks fail.
Keep the existing write ceiling, query-cache bounds, chart sampling and 90 ms SLO unchanged. Qualify migration cancellation and crash recovery, cross-Store cached reads, restart and production checkpoint behavior.
PR3063 landed endpoint namespaces while PR3068 was being prepared from an
older release base. Keep that existing fixture identity boundary and its
uncertainty-fence regression, and remove the redundant global-registry
resets and helper move added by PR3068.
The Proxmox files return to the first parent of PR3068. The installer
reader-readiness correction remains in place. This keeps one HTTP fixture
isolation mechanism and preserves earlier uncertainty fences throughout
the tests. Production guest admission and release controls are unchanged.
Refs #3063, #3068
Release preparation #3065 failed on two fixture boundaries: the PTY
could signal Bash before its read setup finished, and a guest-agent HTTP
case could inherit a previous fixture’s cooldown after ephemeral port
reuse. Wait for the actual blocked reader and isolate the registry per
serial HTTP scenario, retaining shared admission across clients and
aliases within each scenario.
Controlled regressions fail before the fixes and pass afterward,
covering premature signal delivery and inherited cooldown state. This
changes test fixtures only and preserves the production installer,
guest-agent safeguards and release qualification controls.
Validation:
- Full config API and Proxmox packages passed on Linux.
- Real BusyBox input and blocked-reader regression passed five
repetitions.
- Guest-agent tests passed with the race detector.
- Status and subsystem registry audits passed.
Refs #3065.
The two CI attempts for release preparation #3065 failed in fixture setup:
the PTY signalled a running Bash reader before input setup completed, and
an HTTP fixture inherited an earlier origin's uncertainty cooldown when
an ephemeral port was reused.
Wait for noncanonical, silent terminal input and a blocked Linux reader
before signalling. Isolate the shared guest-agent registry per serial
HTTP scenario while retaining admission sharing across its clients and
aliases. Move the existing isolation helper to the shared fixture file
and update its callers. Production guard and installer code are unchanged.
Both new regressions reproduce the old failures before the fixture fixes.
Retain those failures and the earlier unsuccessful compile proof. Source
test stabilization is separate from release qualification and publication.
Refs #3065
Reuse the existing per-case endpoint namespace helper for guest HTTP-status
tests so a recycled loopback address cannot inherit another fixture's
intentional uncertainty fence. Keep diagnostic and cluster clients within
each fixture on the same identity, preserving all runtime cooldown rules.
A shared-listener control reproduces the old bare-endpoint failure and
verifies independent first commands plus unchanged earlier fences. Targeted
and full Proxmox tests pass. Record this separate test-stabilization slice.
Use the canonical timestamp/value model directly from the shared retained
read engine. Preserve the full extrema reader, tier reconciliation, Docker
observation provenance, filters and chart sampling while eliminating unused
column transport and a second series copy.
Verify both projections, detached reads and chart sampling equivalence.
Record the measured memory reduction and retain inconclusive timing and
failed capacity evidence. The idle-worker 90 ms proof remains open.
Keep absent bounds nullable across the SQLite row boundary and apply their
existing value fallback during decoding. Preserve explicit bounds and the
same fallback before SQL and streamed display aggregation.
The full metrics suite passes. Ten alternating diagnostic samples reduce
allocations by 24.59% and bytes by 4.24%; their latency difference remains
inconclusive. Preserve the strict candidate failure and competing-capacity
receipt without claiming a qualified 90 ms result.
Contract-Neutral: Retained chart values, extrema, tier reconciliation, sampling and cache boundaries remain unchanged; only decoding allocation cost and capacity-error reporting change.
Stream ungrouped retained reads in the existing time-major index order,
avoiding temporary metric sorts without changing tier reconciliation or
chart samples. Extend query-plan and filtered-result coverage.
Add an explicit strict mode for the existing 90 ms API budget and an
optional fresh-worker CI proof with fixed executable identities and raw
capacity evidence. Keep ordinary hosted budgets and required gates intact.
Contract-Neutral: Chart payloads, retained-tier reconciliation, sampling, index layout and source/release gates are unchanged; the implementation removes an unnecessary sort and adds controlled verification of existing latency budgets.
A cancelled login can consume an empty response before the transport reports
cancellation. Preserve that context outcome for the owner and joined readers,
reject late session publication and prevent a cancelled refusal from retrying
another encoding. Keep live response validation and later recovery intact.
Initial password authentication has one owner and must end with its
constructor. Keep its complete JSON/form exchange within the caller and
client timeout. Retained-session renewal still serves concurrent readers
independently of any one caller's cancellation.
Retain generation-based coalescing and the shared parser, exercise both
initial encodings under the client timeout, and document the ownership
boundary alongside existing startup and cluster cancellation controls.
Retain the failed CI and reproduced baseline in the existing reliability
record while final-source qualification remains open.
Preserve selected memory freshness and the VM/LXC interpretation.
Retain both native tool contracts and current API concurrency controls.
Qualify this source separately from release and installed recovery.
Contract-Neutral: Integrate already-governed parents without a new contract change. Incoming verification files already match main. Fresh integration tests and source-bound browser proof cover the combined memory and installer surfaces.
Serial pinned-address dialing could spend the request deadline on a stalled
first address while a validated fallback was healthy. Share one bounded
two-worker dialer with restricted HTTP clients and retain cancellation in
webhook DNS admission and dialing.
Add real stalled-TCP and DNS cancellation controls, persistent queue verdict
readback, shared transport ownership and explicit proof routing.
Record successful response writes, scheduled generation, file saves and SMTP
outcomes in a durable content-free tally. Distinct active days include manual
exports, while saved-schedule periods deduplicate retries and monthly reuse.
Publish bounded scalars through the existing opt-out-controlled heartbeat and
report missing coverage explicitly. Preserve pricing and customer entitlements.
Retain backup execution outcomes and collection coverage in each
organization's recovery store so provider inventory changes and restarts
cannot erase report evidence. Keep missing and incomplete history distinct
from an observed period with no attempts.
Reuse existing report scopes, calendar schedules, branding and delivery
recovery for client PDF and CSV reports. Register the paid renderer through
the existing private runtime boundary. Pricing and grants are unchanged.
The report engine read every report's metrics from the default
organization's store, whatever organization the request came from. A
tenant request for a resource ID outside its own inventory fell through
to that store with the caller's ID, so one organization could read the
default organization's history, and even a tenant's own resources were
reported from the wrong store. Reporting was closed to org owners until
this was fixed.
The engine now resolves the metrics store of each request's OrgID
through the multi-tenant monitor and refuses a request that names none,
so no report path can fall back to the default store. Report handlers
reject resources outside the requesting organization's inventory,
scheduled runs resolve their scope against their owning organization's
inventory and skip resources it no longer holds, and Pulse Assistant's
pulse_summarize reads its executor's organization.
With every report read scoped, the reporting routes move to
ensureOrgAdminSession and reportingRead is offered to org managers
again, so providers can build reports inside client workspaces.
The report engine wiring lives in pkg/server/server.go, which the
completion guard only let telemetry tests prove. server.go now has its
own security-privacy path policy that keeps those proofs and also
accepts pkg/server/reporting_engine_test.go; the other telemetry files
keep their existing policy unchanged.
Session renewal became single-flight in 97e4c8eac7, but the caller that
started the login ran it on its own context. When that caller was a
short-lived read (a 10s guest-agent call, a guest-metadata fetch cancelled
on invalidation, an API request whose browser disconnected), cancelling it
aborted the shared login and every concurrent waiter failed with
"re-authentication failed: context canceled" despite live contexts. The
cluster client treats that text as an authentication error and skips
failover, and a failed GetNodes marks the instance's nodes unreachable for
the cycle.
Run the shared login detached from any single caller's cancellation,
bounded by the client's per-request timeout, and have every caller,
starter included, stop waiting only when its own context ends. A genuine
login failure is still shared with all waiters, and a login whose starter
has left still completes and is reused instead of being repeated.
Contract-Neutral: Proxmox password-session renewal keeps its API fields, TLS policy and guest-command rules; only which context bounds the shared login changes.
Measure delivered extended history before changing free and paid features.
Count only successful responses containing an older metric point and retain
only bounded daily totals. Expose the rolling sum through schema v19 and
Preview with matching privacy disclosure and explicit report coverage.
Proxmox Mail Gateway 9 returns the cluster status cid as a string, so
decoding it into an int failed the whole cluster read and the poller
lost the node list (issue 2960). Pulse never reads cid, so the field is
no longer decoded and any format works.
With no nodes, the poller then requested backups from a node named
after the connection's display name ("PMG-01"), which PMG rejected with
a hostname-lookup error that pointed users at DNS. That fallback now
runs only when PMG answered with no cluster entries; after a failed
cluster read the request could never return backups, and the poll
already records the cluster failure.
Stream every nested result field without imposing a new uniqueness policy on completed filesystem measurements. Existing counter validation retains unusable/partial results without a cooldown; a later null or case alias cannot erase an earlier foreign interface field. Correct the legacy HTTP-200 error-payload assertion to the documented unverified-completion behavior.
Contract-Neutral: Restore the documented completed-measurement semantics and strengthen tests of existing foreign-reply refusal; no additional contract delta.
Change-source: pulse-maintainer
A structurally valid reply for another guest command must not become an empty current inventory, acquire filesystem identity or release the serial-command fence. Check known command-exclusive fields and information-result shape before postflight lock checks; preserve null/empty/shared/unknown fields, partial row decoding, unusable file measurements, trust and independent live polling. Add six-reader cross-command controls, alias/expiry/recovery controls and actual builder/state/broadcast/History proof. This detects contradictory payload shapes, not native command correlation, thaw or report attribution.
Change-source: pulse-maintainer
Distinguish observed 403 permission denials from 401 authentication and other HTTP failures, rather than prescribing token rotation or user-only grants. Carry fixed, identity-free recovery guidance into the existing Diagnostics error summary while preserving live-monitor/PBS probe attribution, TLS trust and ordinary request behavior. Add wire-status, ACL-scope, same-credential recovery and serializer controls with their subsystem contracts.
Change-source: pulse-maintainer
Current qualification exposes Go's #NN repeated-name suffix as an invalid URL fragment in the new fixture path. Hash the complete evidence label so clients retain the same per-case key without using test labels as URL syntax. The actual hostname validation and guest fences remain unchanged.
Contract-Neutral: Fixture-only identity encoding repair; no runtime or trust-contract change.
Change-source: pulse-maintainer
Retained full-suite proof reports an inherited agent-cooldown before the invalid-UTF8 lock case dispatches any command. Give each synthetic case a distinct base path, keeping all clients in that case on its shared key. A same-listener control retains the prior uncertainty fence unchanged. No runtime guest guard, native target or assertion is weakened.
Change-source: pulse-maintainer
The retained 277ec211 source proof times out on a correctly retained target, with injected requests zero and its pin unchanged. This test owns routing and trust, not a latency SLO. Use explicit five-second client and ten-second caller fixture bounds, record elapsed work, and retain every routing/pin/guest assertion. Runtime timeouts and timeout controls are unchanged; the original failed receipt remains adverse evidence, not a diagnosed host fault.
Change-source: pulse-maintainer
Carry exact duplicate-row conflicts into owned client trust before map deduplication. Keep unresolved saved trust through initialization, missing-client retry and topology refresh without replacing pins or stopping trusted siblings. Cover actual monitor lifecycle routes and member projections alongside client TLS controls.
Change-source: pulse-maintainer
Resolve all saved pins for an endpoint before credential-bearing construction. Contradictory trust is an explicit failure, not a missing-pin fallback or first-use opportunity. Preserve independent authorities, agreeing formats and saved configuration; prove zero provider work, sibling monitoring and normal reload through the member projection.
Change-source: pulse-maintainer
Require whole bounded, unambiguous enumerations on the efficient and node fallback routes. Preserve scoped empty inventories, LXC integer compatibility and ordinary counter/receipt semantics. Validate retained readings, History, breaches and template policy through actual fallback and recovery.
Change-source: pulse-maintainer
Consume the bounded response through EOF before admitting nodes or a cluster probe. Reject missing, ambiguous and incomplete inventories without leaking response details, retain scoped empty arrays and provider metadata, and cover trusted startup, recovery, cached reads and consuming endpoint projection.
Change-source: pulse-maintainer
Contract-Neutral: Tightens existing node-inventory and recovery-health admission without changing the public response schema, API surface, permissions, TLS or guest-command policy.
Replace the old optimistic-health expectation with cached selection, failed-request and later-success controls. Advance no production clock or safety fence and avoid an irrelevant constructor attempt in the in-memory fixture.
Contract-Neutral: Test-only correction to the recovery contract established in 4147f48db3; runtime and browser inputs are unchanged.
Change-source: pulse-maintainer
Require the actual nodes/login probe to succeed at startup, recovery and lazy selection instead of accepting guest-like response text. Retain failed single-endpoint health until an ordinary request succeeds, preserving cached last-resort reads, real guest error handling, temporary rate-limit handling and trust policy. Add producing-path and connected projection regressions and revise the old nodes-as-guest fixture contract.
Change-source: pulse-maintainer
The projection fixture now uses the actual saved ClusterEndpoint schema. Give each malformed-filesystem fixture an independent base path so reused listener ports cannot borrow a previous live uncertainty fence; all real guard state and response assertions remain.
Contract-Neutral: Test-only schema correction and independent synthetic endpoint identity; no production policy, deadline, wire or configuration changes.
Change-source: pulse-maintainer
A completed first-use retry does not return its nodes payload. Omit that unknown diagnostic count rather than reporting zero. Concurrent ordinary reads may legitimately open additional TLS sockets, so the construction control bounds logins and probe reads rather than pooled read connections.
Contract-Neutral: Diagnostic-count fidelity and fixture-only connection counting; no wire, trust, polling, authority or consumer contract delta.
Change-source: pulse-maintainer
The shared probe must release the health mutex for first-use trust. Retain the prior single construction boundary so joined readers do not duplicate capture and password login. The actual TLS concurrency control checks eight ordinary reads and nonblocking health inspection.
Contract-Neutral: Preserve existing serialized lazy client construction while releasing only the health and trust mutex for first-use capture; no wire, authority, configuration or consumer contract changes.
Change-source: pulse-maintainer
Use one bounded first-use probe for constructor and nodes-read pin failures at startup, recovery and lazy selection. Keep capture credential-free, primary and saved pins immutable, and full polling-session acceptance separate. Qualify all existing auth encodings and refusal, cancellation and consuming projection controls.
Change-source: pulse-maintainer
The containing race suite observed a prior cooldown before the invalid-success matrix sent its first HTTP request. Use per-case synthetic base paths instead of recycled listener authority; prove two first admissions and preserved cross-client fences on one shared listener. Production admission keys, aliases and safety state are unchanged.
Contract-Neutral: Isolate synthetic test identities; no production guest admission, TLS, session, polling or History contract delta.
Change-source: pulse-maintainer
Reject incomplete or ambiguous 200 login bodies before replacing ticket, CSRF, expiry and PVE generation. Preserve retained sessions, joined renewal failures, normal encoding fallback and later ordinary recovery; cover real pinned startup/poll/broadcast consumption.
Contract-Neutral: Restore existing complete-session and failed-renewal preservation guarantees; no API/wire schema, TLS trust, credential authority, guest lifecycle, polling cadence or History contract delta.
Change-source: pulse-maintainer
Carry cancellation through PVE/PBS construction, cluster probes and retained sessions, startup retries and port fallback. Preserve prior health/errors on caller termination; accepted clients retain independent ordinary reads and full timeouts. Cover real pinned HTTPS cancellation, deadlines, form fallback and first-poll publication.
Contract-Neutral: Restore existing caller cancellation and usable-session guarantees; no wire/API schema, agent lifecycle, TLS trust, guest safety, polling cadence or History contract delta.
Change-source: pulse-maintainer
Retain the actual authentication refusal and existing client when the full-timeout session cannot be constructed after a successful short probe. Cover pinned HTTPS rejection, malformed and form-refused sessions, later ordinary recovery and the monitor endpoint projection.
Change-source: pulse-maintainer
Validate result/content/truncated types before shared guest-command admission is released. Scalar results must not become empty inventories, and wrongly typed file fields must not release admission before decoder failure. Preserve compatible empty/null rows and completed but unusable measurements. Add six-reader, alias/expiry and actual-state polling/broadcast/History controls with untouched healthy-read origins. Native thaw and restart recovery remain separately unverified.
Change-source: pulse-maintainer
Retain the shared resolver and request authority while retrying only failed socket establishment. Usable addresses share one bounded dial budget; cancellation prevents further attempts and TLS still authenticates the original hostname and pin. Connected controls reproduce the first-address defect and cover cache reuse, families, deadlines and cancellation.
Change-source: pulse-maintainer
Bind the credential-free capture handshake to the polling context under the existing five-second ceiling. Recheck cancellation before publishing a runtime pin; retain established trust, endpoint selection and ordinary declared-member first use. Add local stalled-handshake, peer-closure, no-dial and later-recovery controls.
Change-source: pulse-maintainer
Reject impossible physical counters and overflowing cache estimates. Keep the original trusted source through repeated failed/negative-cache polls within the unchanged observation-age and capacity fences, instead of clearing the incident on an independent provider fallback after one retained poll.
Exercise all eight refusals through both collectors and both HTTP clients, original percentage/byte History, backup admission, real alert decisions, valid recovery and measured zero. Preserve TLS, operation locks, shared uncertainty, deadlines and native recovery exclusions. This completes the original guest-memory outcome coherently, not a replay of source-xubz2c16.
Change-source: pulse-maintainer
rcourtman