Across the last 25 bot PRs the median open to merge time was about 33
minutes for one PR and about 45 when three opened together. The critical
path was the required Backend tests (api) check, where one go test -race
./internal/api step took about 27 of its 31 minutes. Every other required
check finishes within about 12 minutes.
internal/api now runs as four Backend tests (api-N) shards. Each shard
lists the package's tests with go test -race -list from the commit under
test and runs one contiguous quarter of that list in go test's own order,
so every test, including ones added later, runs in exactly one shard, and
a shard that resolves no tests fails. The split is contiguous rather than
interleaved by name because some internal/api tests depend on package
state left by the tests just before them. A round-robin split by sorted
name failed dozens of tests locally (admin bypass and session store
globals), while the four contiguous slices and the full run all pass. A
Backend tests (api) verdict job keeps the required check name and fails
unless every shard succeeded. Backend tests (rest-0) and (rest-1) keep
their names and package split. Local non-race timings put the heaviest
slice at about 42 percent of the package, so the api check should drop
from about 31 minutes to roughly 14.
Core E2E no longer runs the non-gating probation tier on pull requests.
It could not gate a PR and the promotion rule counts only main runs, so
on a PR it only held each of the eight shard runners about eight minutes
longer, which fed the runner queueing seen with concurrent PRs. Push and
manual runs still execute it.
Go test steps still run for frontend-only changes. Go tests read
frontend sources directly (internal/api contract tests,
internal/unifiedresources walking frontend-modern/src, and the
internal/telemetry repository-wide wording scan), so skipping them by
path would drop real coverage.
Dispatch the post-publication check from protected main while binding its packet to the immutable RC tag. Accept only canonical Pulse checkout origins before normalising the spelling required by the attester, and document the deployment contract and source guard together.
Change-source: pulse-maintainer
TestInstallSHRecoversV5ProcessArgsWithoutProcfs stubs the running agent
as pid 4242 and stubs ps, but running_agent_arg_stream reads
/proc/<pid>/cmdline whenever it exists and only falls back to ps when it
does not. On a Linux runner where pid 4242 happened to belong to another
process, recovery parsed that process's arguments and the test failed
with NOT_READY (seen in Backend tests rest-1). It never failed on macOS,
which has no /proc.
Wrap the real function so this test always resolves against an empty
proc root, which is the no-procfs host it describes. Also extract the
env-stream helper it was calling without, and fail on NOT_READY directly
since "READY" is a substring of it and never caught a failed recovery.
setup.sh resolves the image pins in .env to digests once, so upgrade.sh
only ever re-pulled the release a provider first installed. Moving to a
new release meant finding the new digests by hand, so none of a release's
fixes, and no new feature such as the portal Plan tab, reached an
existing install.
Run from a newly extracted release bundle, upgrade.sh now works on the
existing install: it resolves the bundle's control-plane and runtime tags
to digests, runs the status gate, preflight and verified backup with the
new release's control plane through a shell override while nothing on
disk changes, then keeps a copy of .env, installs the bundle files and
writes the new pins before starting the new images. --keep-image-pins
keeps hand-set pins.
It also pulls the tenant runtime image before the first status check,
which otherwise failed every upgrade to a new CP_PULSE_IMAGE with "not
present locally", including the documented pin-editing route.
Exercise two provider installations with the same tenant ID, unowned-network collision and cleanup isolation. Gate the release-line push before packet freeze and repeat against digest-verified candidate images during container qualification. Update the deployment-installability contract and pin both workflow gates in the release-asset verification suite.
Change-source: pulse-maintainer
Keep first-run evaluation and summary guidance conditional on actual Plan availability and confirmed active limits. Remove the ordinary manual-key purchase prompt while retaining the separate custom-license binding path, and update the installability contract and focused assertions in the same commit.
Change-source: pulse-maintainer
setup.sh still told an evaluating provider to "request an upgrade" at
/msp.html#request, a form that waited on a human reply with a checkout
link. The portal now sells plans itself (Plan tab, Stripe checkout,
applied in seconds), so both the evaluation log line and the closing
summary send the provider there. The summary also stops printing the
lease signing public key as something the licence "must bind": the
portal purchase binds it without a copy step, and the key is still
available through --print-lease-signing-public-key for a custom licence.
Start the provider services after licensed compose validation so the printed bootstrap and portal link have a live control plane. Correct the first-run prompt to name only operator-provided values and retain an install contract regression test.
Change-source: pulse-maintainer
Keep the canonical release and provider-MSP frontend builders on the proposed immutable amd64 Node 24 digest. Bind the same-commit change to the deployment contract, release-build metadata proof and provider-MSP rollout-control proof; hosted build and installed rollout remain separate evidence.
Change-source: pulse-maintainer
The install-metadata contract encoded only the emergency no-RC stable-patch shape. A stable patch with a same-version RC promotes the exercised candidate (promotion_mode stable-rc-promotion), so the contract must assert that shape and derive the rollback target from the last published stable.
previousStablePatchVersion hardcoded only v6.4.2 as unpublished, so v6.4.5 rolled back to the never-published v6.4.4. Add v6.4.3 and v6.4.4, which only ever shipped as prereleases, so v6.4.5 resolves to the published v6.4.1 that the promotion resolver also derives.
Add stablePatchPromotedPrerelease to detect the same-version candidate from the retained release notes, and branch the stable-patch install-metadata tests between the emergency and RC-derived shapes. Unit tests pin the rollback derivation and the candidate detection.
Change-source: pulse-maintainer
The FreeBSD rc.d branch printed a log hint pointing at the system log,
but pfSense has no /var/log/messages (its system log is
/var/log/system.log). More importantly, daemon(8) does not capture the
child's stdout and rc.d has no journal, so the agent's zerolog output was
never written to any file: correcting the path alone would still point
users at a log with no agent entries.
Give the agent its own rotating log via --log-file on FreeBSD (both the
TrueNAS CORE and pfSense/OPNsense/vanilla FreeBSD branches) and point the
completion hint at it, matching the Unraid/QNAP/TrueNAS-Linux convention.
The shell installer is a canonical path shared by agent-lifecycle and
deployment-installability, so record the FreeBSD log-capture obligation in
both subsystem contracts.
Refs #2123.
Change-source: pulse-maintainer
Integrate the reviewed failed-staging config-backup discard repair (07d54cd345)
with its deployment-installability contract, installtest proof and registered
completion pair (ccfd22da7f, b36e9609dd). The candidate auto-update RETURN-trap
repair (9b004cc787) duplicates the already-integrated 564c235e32, so the merge
keeps the canonical trap implementation and both regression suites and retains
both registry entries. Change-source: pulse-maintainer
Change-source: pulse-maintainer
Record the deployment-installability contract section for the auto-update
RETURN-trap and failed-staging config-backup repairs, and add the Go proof
that exercises the extracted perform_update and asserts the trap does not leak
into a later function return. This completes the two runtime commits for
canonical governance as one reviewed unit each.
Change-source: pulse-maintainer
Follow-up to 564c235e32 (fix(update): disarm perform_update RETURN trap after
it runs). That runtime change touches the deployment-installability subsystem,
so this commit adds the matching substantive contract section in
docs/release-control/v6/internal/subsystems/deployment-installability.md and
the registry-accepted deployment-script-runtime proof in
scripts/installtests/pulse_auto_update_test.go
(TestPerformUpdateDoesNotLeakReturnTrap). The pair is registered in
scripts/release_control/canonical_completion_history.json by the next commit.
Change-source: pulse-maintainer
backup_existing snapshots the configuration before the update stages, but a
staging disk-headroom failure then exited and left that snapshot behind. On a
small root filesystem each automatic retry added another full backup and made
the low-space condition progressively worse until updates could no longer run
(#2127). Record the snapshot and remove it when download_pulse aborts at the
headroom check, before anything was replaced. Add regression coverage.
Change-source: pulse-maintainer
Dependabot #2094, #2095 and #2096 bump actions/setup-go 6.4.0 -> 7.0.0, signpath/github-action-submit-signing-request 2 -> 3.0 and actions/github-script 8.0.0 -> 9.0.0. Each proposal is blocked only by the reviewed pin constants, the workflow-trust allowlist, the release-consumer action manifest and the installer governance assertions that hard-code the previous revisions. Carry all three bumps with those artifacts in one commit so the proposals can be closed as superseded.
All three actions keep the node24 runtime and their existing inputs and outputs; no consumer interface, installer behaviour or public contract changes. The deployment-installability and agent-lifecycle workflow references are pin-only.
Contract-Neutral: Reviewed action pin refresh with identical node24 consumer interfaces and unchanged inputs/outputs; no public-contract or installer-behaviour delta.
Change-source: pulse-maintainer
FreeBSD base ships sha256(1) but not GNU sha256sum or Perl shasum. Record the fallback contract for the unified and MCP installers and add the MCP installer proof the deployment-installability verification policy requires.
Change-source: pulse-maintainer
FreeBSD base ships sha256(1) but not GNU sha256sum or Perl shasum, so a
fresh pfSense/FreeBSD host without coreutils failed the download checksum
and refused to install. Route every installer checksum through
installer_file_sha256 and fall back through sha256sum, sha256 -q, shasum and
openssl dgst. install-mcp.sh gets the same sha256 branch. Add a focused test
that hides sha256sum/shasum and asserts the FreeBSD fallback returns the
digest.
Change-source: pulse-maintainer
The native installer suite still required the retired persistent runner
label. Require the fresh hosted runner while preserving the canonical
partition command, capacity checks, watchdogs and cache constraints.
Persistent prerelease runners can retain state from earlier source execution
and influence later admission, build output or qualification. Use the
existing hosted stable-release path for every channel while preserving
exact-source checks, resource planning, watchdogs and release gates.
A draft GitHub release does not hide its public Git tag or registry
images. Publishing those before qualification consumed a beta identity
when the candidate later failed.
Stage drafts without Git refs, join exact-source checks before public
tag, Docker and Helm publication, and preserve exposed tag identities.
Keep final digest verification before release activation.
The grouped action upgrade leaves signing, network and publication consumer assertions on superseded pins. Align those contracts and check every consumer against reviewed immutable upstream manifests, retaining exact dispatch and release trust boundaries without claiming hosted execution.
Change-source: pulse-maintainer
Replace obsolete v6.2.1 guidance using retained signed v6.4.1 delivery and payload evidence. Limit privacy claims to the evaluated licence request, synchronize the shipped guide, and adapt the registered deployment regression and contract without claiming installed onboarding acceptance.
Change-source: pulse-maintainer
PR #1933 failed paired normalization benchmarks despite unchanged hot-path source. Preserve actual checkout identities, selected toolchains and sample order so investigation does not confuse PR head metadata with measured source. Keep thresholds and sample collection unchanged.
Change-source: pulse-maintainer
PR #1921 targets release/v6.4 but receives only docs and boundary checks because build and E2E triggers still name the historical release branch. Include versioned release trains for push and pull request events without changing job gates or path filters. A focused regression fails all four event/workflow combinations before repair and passes after it.
Change-source: pulse-maintainer
The exact-byte public package check superseded metadata-only chart inspection, but the broader deployment contract still required the removed command and failed deterministically. Require the consumer-path pull and byte comparison so the contract matches the stronger workflow guarantee.
Change-source: pulse-maintainer
Stable continuity run 33592377446 was rejected before any job ran: its read-only caller invoked a reusable job requesting contents:write. Extract the unchanged smoke execution into a body that inherits the caller budget, keeping the existing draft-capable entry point and its write-level draft GET access. Continuity now calls the shared body directly without broadening its token. Pin the permission boundary in regression coverage; do not relax immutable-release admission.
Change-source: pulse-maintainer
Issue #1890 reports macOS agent updates stopping because the root group does not exist. Use numeric superuser ownership in the two shared lifecycle writes without relaxing failure handling or the least-privilege group boundary. Add a regression fixture that rejects named root ownership and checks that chown failures still prevent replacement.
Change-source: pulse-maintainer
Reapply the reviewed Go 1.26.8, OCI Helm recovery, and non-login-shell test expectations that later upstream merges accidentally replaced while retaining the corresponding implementations.
Contract-Neutral: test-only reconciliation; no product or runtime behavior changed
Change-source: pulse-maintainer
A failed v6.4.2 release left newer stable-shaped source and public container tags behind after its GitHub Release packet was removed. Extend the scheduled continuity audit across the stable tag and public registry frontier so this partial publication cannot remain hidden behind an older latest release.
Change-source: pulse-maintainer
The retry hardening added in #1885 bounded attempts but not time. npm's own
fetch-timeout defaults to five minutes and it retries internally, so three
"attempts" against a hanging advisory endpoint ran for 10m56s on job
100986651307, and a second audit step added 3m36s. The Frontend job was
cancelled 31s into type-check with all 1183 test files already passing, and a
cancelled job reports as a failed required check, so a green run blocked every
pull request. #1888 raised the job timeout to 40 minutes to unblock delivery;
this decides the policy instead.
Each attempt now runs under a hard wall-clock bound and the sequence stops at
a total deadline (60s and 240s by default). npm's internal retry loop is
disabled in favour of this one, since it was the hidden multiplier. The bound
is enforced by a watchdog subshell rather than timeout(1), which is not
present on every developer machine.
What happens when the endpoint stays unreachable is unchanged, because that
split was already right: the run fails when the change touches the dependency
graph and the answer is genuinely unknown, and warns without failing when it
does not, because the graph is then identical to a base commit that already
produced a passing answer. Any advisory at any severity still fails.
Also drops the production-only audit from the per-pull-request path. It audits
a subset of the same packages, so it reports a subset of the same advisories,
and because the complete audit fails the job on any finding, the production
step could only ever execute in the cases where it was already guaranteed
clean. The dev-versus-production split still runs for every npm workspace in
the scheduled security-scan job, where it informs rather than blocks delivery,
and Dependabot security updates remain the route for advisories published
against unchanged dependencies.
With the audit bounded to 4 minutes against an ~11 minute baseline, the job
timeout returns to 30: a stalled endpoint should surface as a warning, not be
absorbed by a budget large enough to hide it.
Replace the remaining Node 20 action pins before GitHub removes that runtime, and make the reviewed Node 24 pins a workflow trust invariant.
Change-source: pulse-maintainer
On 2026-09-03 registry.npmjs.org returned 503s and then timeouts from its
bulk advisory endpoint for over an hour. `npm audit` exits non-zero both
for a real advisory and for an endpoint it cannot reach, so the Frontend
job failed four times running and, because it is a required check, no
pull request could land at all - including Go-only ones that touch no
JavaScript. Every one of those failures was the outage. In two of the
runs the other audit call in the same job succeeded and reported zero
vulnerabilities.
The audits now run through scripts/npm-audit-retry.sh, which separates
the two cases and does nothing else. A conclusive result is acted on
immediately, and the gate stays exactly as strict as before: any
vulnerability at any severity still fails, and no severity threshold is
introduced. Only an unreachable endpoint is retried, with backoff.
When retries are exhausted the outcome depends on whether the answer is
actually unknown. A change that touches frontend-modern/package.json,
frontend-modern/package-lock.json, or the runner itself fails, because
the dependency graph moved and no result means no answer. A change that
touches none of them warns and continues, because the graph is then
identical to the base commit that already produced a passing answer.
Advisories published later against unchanged dependencies are what
Dependabot security updates are for, not a per-pull-request audit.
Deliberately not done: relaxing the severity threshold. That was my first
instinct, but the evidence does not support it. None of the four failures
was an advisory, the lockfile reports zero vulnerabilities at every
severity, and a threshold would have weakened the gate without fixing
anything. The contract's rule that audit suppression is not a valid
closure stands.
The pinning test now requires the runner's invocation and the dependency
detection wiring, and additionally asserts the runner carries no
--audit-level flag, so the strictness cannot be quietly traded away
later.
Go 1.26.8 supersedes the prior patch release, so every release builder and local toolchain guard must move together to prevent candidate artifacts from retaining an older compiler and runtime.
Contract-Neutral: toolchain-only patch update; no product or runtime contract changed
Change-source: pulse-maintainer
Retry only explicit registry failures with bounded one-minute attempts while preserving immediate advisory failures. Defer the aggregate audit verdict so frontend tests and builds still report during npm service incidents.
Keep convergence recoverable after short-lived Actions artifacts expire by preserving the exact digest-bound package already verified from GHCR.
Change-source: pulse-maintainer
Three defects that the v6.4.3-rc.1 cut exposed, each with its contract note
and proof:
1. scripts/release-preflight-worker.sh published the smoke stack on host port
7655 and probed it directly. pulse-dev also hosts the maintainer's dogfood
Pulse instance on 127.0.0.1:7655 and a second instance on 17655, so the
release smoke failed to start after every other stage had passed on the
qualified head. The worker now honours PULSE_RELEASE_PREFLIGHT_E2E_PORT,
otherwise takes the first candidate host port pair it verifies free and
fails fast when none is, probes health and update status on that port, and
hands Playwright the same base URL.
2. .github/workflows/publish-helm-chart.yml resolved the chart version before
checking out the repository, but de41ea1883 made that step call
scripts/write_github_output.py, so every chart publish since failed with
"can't open file". Run 33580123246 hit it after all other staging passed;
the checkout now precedes the resolver.
3. docs/release-control/control_plane.json maps the 6.4.3 version prefix to
release/v6.4. Run 33579042375, dispatched from main, failed inside the
compiler dispatch because main advanced one minute after the pipeline
pinned its source SHA; with pull requests landing every few minutes a
candidate cannot hold its SHA between prepare and compile. release/v6.4
was created from main at the exact-SHA-qualified commit 56e51e622e and
carries these same fixes; v6.4.3-rc.1 published from it.
The deployment-installability contract records the worker port and toolchain
rules and the Helm checkout order; release_preflight_test.py and
build_release_assets_test.go pin them.
Rootful evidence must reject non-running systemd states and prove no unrelated container runtime survives. Require exact manager state, mask distro containerd, recheck readiness after scenarios, and remove the complete Podman socket boundary.
The disposable host reached multi-user.target in a degraded state because distro Podman housekeeping used overlay storage inside the outer container. Mask unused units, force VFS for explicit daemons, and fail closed on systemd unit failures so retained rootful evidence remains trustworthy.
A workflow dispatch by branch can resolve after that branch moves, allowing an unreviewed tip to enter the release pipeline. Require every publishing dispatch to name its expected source SHA and make the workflow reject a different source or workflow commit before checkout.
Change-source: pulse-maintainer
(cherry picked from commit a461fc9c0a)
courtmanr@gmail.com