Commit Graph

284 Commits

Author SHA1 Message Date
rcourtman 11bf0c9e74 Fix native agent release lifecycle verification 2026-07-10 01:49:38 +01:00
rcourtman a3ef1226b7 Fail fast on missing native signing configuration 2026-07-09 23:44:22 +01:00
rcourtman 51c60df3a4 Exercise native signing in release rehearsals 2026-07-09 23:36:06 +01:00
rcourtman 255c7c23d4 Modernize Unified Agent lifecycle and platform support 2026-07-09 23:20:35 +01:00
rcourtman a5b8d9a3ee Allow complete release candidate validation 2026-07-09 23:11:38 +01:00
rcourtman cc0952e491 Run signed candidate builds during dispatched rehearsals 2026-07-09 22:32:40 +01:00
rcourtman 8dda0b6efa Build releases once and promote verified candidates 2026-07-09 22:21:34 +01:00
rcourtman 910418c3b2 Make stable patch releases unattended 2026-07-09 20:16:13 +01:00
rcourtman 2be167331d Harden demo SSH setup for IP targets 2026-07-09 17:06:52 +01:00
rcourtman 76ced45c3a Harden demo SSH setup for private deploy hosts 2026-07-09 16:51:38 +01:00
rcourtman 5fe6bfde57 Harden release integration diagnostics and login retries 2026-07-09 12:50:55 +01:00
rcourtman e240f162e9 Reserve the latest markers for the highest stable release
A maintenance cut of an older line (v5.1.36 after v6 GA, or a future
6.0.x patch after 6.1 ships) was allowed to move Docker/GHCR :latest and
the GitHub latest release marker onto itself, silently downgrading every
install that follows latest. Promote :MAJOR and :MAJOR.MINOR
unconditionally, but :latest and make_latest only when the tag is the
highest stable semver. workflow_dispatch gains force_latest as the
explicit rollback escape hatch.
2026-07-09 09:37:40 +01:00
rcourtman 54a6118d17 Fix CI false positives in the canonical completion guard
The guard judged substantive contract updates by diffing HEAD against
the index. In CI nothing is staged, the index equals HEAD, so every
contract file piped in via --files-from-stdin looked unchanged and the
guard blocked compliant pushes. Concretely, run 28944317805 blocked
7645965af even though its deployment-installability.md addition sits
inside the Current State section.

The guard now accepts --diff-base <ref> (requires --files-from-stdin),
resolves it to its merge base with HEAD so the comparison anchor
matches the three-dot changed-file list, and compares base vs HEAD
contract texts in that mode. Pre-commit keeps the index comparison.
The canonical-governance workflow passes the push or PR range base.
2026-07-08 14:35:07 +01:00
rcourtman 0a9a29d63d ci: join tailnet via OAuth client instead of static TS_AUTHKEY 2026-07-08 14:28:51 +01:00
rcourtman 7645965afe Derive the rollback target for scheduled release rehearsals
The weekly release-dry-run schedule failed at 'Resolve rehearsal
metadata' because GitHub does not apply workflow_dispatch input
defaults to schedule events, so rollback_version arrived empty and
resolve_release_promotion.py rejected the run.

Scheduled runs now pass --derive-rollback-latest-stable, which fills
an empty rollback_version with the latest stable tag preceding the
rehearsal version (currently v6.0.4 for 6.0.5-rc.3). Manual dispatches
keep the explicit rollback_version requirement; the stale prefilled
5.1.29 default is removed so operators state the target themselves.
The deployment-installability contract records the scoped scheduled
exception.
2026-07-08 13:52:48 +01:00
rcourtman a9ac8251ba Queue superseded e2e runs instead of cancelling in-progress ones
Agents push to main every few minutes and a sharded run takes about 30,
so cancel-in-progress meant a busy main could never complete a verdict.
With cancel-in-progress off, the in-flight run finishes and GitHub
collapses queued runs to the newest pending one, so intermediate pushes
still skip without killing the run that is about to report.
2026-07-08 08:29:08 +01:00
rcourtman 99a9560c1f Install WebKit for the mobile-safari e2e project, cancel superseded runs
The mobile-safari Playwright project (iPhone 12) launches WebKit, but CI
only installed chromium. The sequential run never reached a mobile-safari
test before the 45-minute cancel, so the gap stayed invisible until shard
4 of run 28923995416 hit it: 20 straight browserType.launch failures.

Also add a per-ref concurrency group so rapid successive pushes cancel
superseded runs instead of stacking four shard jobs each.
2026-07-08 08:18:13 +01:00
rcourtman c728539f07 Restore completed Core E2E verdicts: shard CI, drop release tag from test image
Every main push since the v6 branch flip was cancelled at the 45-minute
job timeout with no verdict. The flip brought the full 94-spec suite onto
main (the last green run, 2026-06-29, ran only 2 specs on the v5 main),
and it runs sequentially against a release-tagged image whose mock-fixture
gate returns 403 without a demo entitlement. Dozens of specs fail, retry
twice each, and burn the budget: of the 31 minutes of suite time in run
28907574469, 18.8 minutes were failing attempts.

- Add GO_BUILD_TAGS build arg (default release) and build the pulse:test
  e2e image with it empty, matching the dev harness the suite is green
  under. Shipped images keep the release tag; release-gate behavior keeps
  its dedicated -tags release Go tests.
- Shard Playwright 4 ways across a CI matrix (214/202/205/203 tests per
  shard) with per-shard report artifacts and an aggregate verdict job.
- Cap CI at 20 failures so an env-broken run reports red in minutes
  instead of grinding into a no-verdict cancellation.
2026-07-08 08:00:55 +01:00
rcourtman 4d6935f4fa Restore update-flow coverage as a v6 Playwright spec, retire test-updates workflow
The Update Integration Tests workflow lost its Go test
(tests/integration/api) in the v6 release commit and was reduced to a
diagnostic smoke test that duplicated the test-e2e stack boot. Replace
it with tests/79-update-flow.spec.ts in the main suite, which runs via
test-e2e.yml on the same trigger paths:

- stable-channel check returns the mock v99.0.0 release and filters
  the v99.1.0-rc.1 prerelease (regression guard for the auto-update
  prerelease bug); rc-channel check surfaces the prerelease
- update plan reports honest manual instructions for the docker
  deployment with readiness attached
- apply refuses prerelease download URLs on the stable channel (409)
- apply of an unsigned artifact fails closed at SSHSIG verification;
  a completed update against the unsigned mock artifact would mean
  the pinned-key trust root was bypassed

The old happy-path apply test is intentionally not revived: v6 made
SSHSIG verification against the pinned pulse-installer key mandatory,
so completing an apply would require shipping the real signing key to
the harness or weakening the trust root.

mock-github-server now serves v-prefixed asset names and download
paths like real Pulse releases (pulse-v99.0.0-linux-amd64.tar.gz);
the in-app updater only recognizes v-prefixed versions in download
URLs, so the old unprefixed shape made every apply fail validation
before reaching the paths under test. Unknown non-tarball sidecar
files (e.g. .sshsig) now 404 instead of falling back to tarball bytes.

The spec self-skips when the update check is not served by the mock
server, so managed-local-backend runs are unaffected.
2026-07-08 01:06:00 +01:00
rcourtman f8e5642ae7 Point update integration smoke test at surviving v6 coverage
The workflow still invoked TestUpdateFlowIntegration from
tests/integration/api, but that package was removed in the v6 release
commit, and the remaining Playwright diagnostic spec skips itself
unless PULSE_E2E_DIAGNOSTIC is set, so the step ran zero tests and
then failed on the missing Go package. Enable the diagnostic spec so
the step actually exercises the pulse:test stack and drop the dead Go
test invocation.
2026-07-07 23:00:09 +01:00
rcourtman 0ad22fe2d5 Mirror the canonical workspace layout in governance CI
The release-control audits resolve repo identity from the checkout
directory name and expect evidence repos as siblings under one repos
root. The hosted runner checked the repo out at Pulse/Pulse, so
canonical_repo_id returned Pulse instead of pulse and the registry
audit treated every local file reference as untracked (2655 errors).
Check out the main repo at repos/pulse and the evidence repos as
repos/pulse-pro, repos/pulse-enterprise, and repos/pulse-mobile, run
all steps from repos/pulse, and point the PULSE_REPO_ROOT_* env vars
at the new paths.
2026-07-07 22:41:38 +01:00
rcourtman 6c181f5f82 Restore pulse:test image build in update integration tests
The dual-key revert (1490a6e6e) removed the docker build line for the
pulse:test image instead of restoring the single-key version, leaving
the step with a bare cd and nothing building the image. Compose then
tried to pull pulse:test from Docker Hub and every run failed before
test execution. Build the runtime target the same way test-e2e.yml
does. The PULSE_LICENSE_PUBLIC_KEY env on the step was dead config:
env vars do not reach docker build and the Dockerfile no longer
declares that ARG.
2026-07-07 22:32:36 +01:00
rcourtman 6f5771d973 Authenticate governance evidence repo checkouts with WORKFLOW_PAT
The Canonical Governance workflow checked out the private pulse-pro,
pulse-enterprise, and pulse-mobile evidence repos with the default
workflow token, which cannot see other private repos, so every run
failed at the pulse-pro checkout. Use the existing WORKFLOW_PAT
secret (already used by create-release.yml to dispatch private Pro
workflows) and avoid persisting the credential in the checkout.
2026-07-07 22:32:25 +01:00
rcourtman 8113f3e8c3 Ensure Helm Pages publishes release chart 2026-07-07 19:21:48 +01:00
rcourtman 155023d86a Add mobile impact gate to release dispatch 2026-07-07 18:21:20 +01:00
rcourtman 1ba7bb06a0 Verify public Helm chart publication 2026-07-05 09:40:20 +01:00
rcourtman d029e4dc42 Fix demo fixture entitlement recovery
Seed the hidden demo fixture entitlement during stable demo updates so release builds can enable governed mock resources after runtime configuration is restored.

Keep the deployment contract and release policy checks aligned with the release-build entitlement gate.
2026-07-04 22:44:43 +01:00
rcourtman cd6b250ae6 Fix demo verification and agent update recovery
Refs #1515

- restore demo runtime env and verify mock fixtures even when the target version is already installed
- require recovered agent update state to include both URL and token before reporting success
2026-07-04 22:30:58 +01:00
rcourtman 4fcb90673b Allow stable patch artifact publishing
Reuse shared release-line validation for Docker, floating-tag, and Helm artifact workflows so stable patch tags can publish from the previous stable tag without a fabricated same-version RC.
2026-07-04 20:49:47 +01:00
rcourtman de0c5ff5c1 Retire v6 preview demo target after GA 2026-07-04 16:16:57 +01:00
rcourtman 72c8d9677a Prune nested demo runtime stores 2026-07-04 15:47:00 +01:00
rcourtman d2ab2b793d Prune demo volatile stores for release deploy 2026-07-04 15:39:35 +01:00
rcourtman d8487b568a Tighten demo backup cleanup headroom 2026-07-04 15:34:08 +01:00
rcourtman dc5f0a35c0 Harden installer and demo backup handling 2026-07-04 15:28:42 +01:00
rcourtman c04ac2f7cf Make private Pro publication block v6 releases 2026-06-15 15:37:13 +01:00
rcourtman 377fd5131d Harden release integration bootstrap gate 2026-06-14 22:26:13 +01:00
rcourtman 6169f3cea5 Bound Go benchmark release gate 2026-06-14 20:47:23 +01:00
rcourtman 50cbc27b71 Refresh workflow action pins 2026-06-14 19:26:21 +01:00
rcourtman ba8cfc8229 Harden release gate workflow 2026-06-14 19:19:39 +01:00
rcourtman cbb3b42ce0 Cancel superseded Build and Test runs
Refs CI notification noise on pulse/v6-release.
2026-06-11 16:32:09 +01:00
rcourtman b0f22f1130 Stop posting retest boilerplate on maintainer issue reopens
The retest-comment workflow fired on 'reopened' as well as 'opened', and
canPostRetestComment gated on the issue author's association, not the
reopen actor's. Since non-collaborator reporters cannot reopen
maintainer-closed issues, a reopened event is in practice always a
deliberate maintainer decision, and the bot would post version-retest
boilerplate that contradicts the maintainer's own comment while planting
the needs-retest auto-close marker on an issue they chose to keep open
(seen on #1471).

Restrict the trigger and the guard to 'opened' and pin the reopened skip
with a test.
2026-06-11 15:10:33 +01:00
rcourtman a405bfc7e8 test(go): stamp parity storage fixture LastSeen; raise CI go test timeout
Two failures surfaced once CI reached the Go steps for the first time
in ~100 runs:

- TestParityStorageFields predates models.Storage.LastSeen (e2a036ce2,
  zero = never seen, no fabricated stamps), so the fixture ingested as
  never-seen and LastSeen() was honestly zero. Stamp the fixture like
  the real pollers do and assert exact round-trip, matching the docker
  host parity test.
- internal/api alone takes ~10m under -race on a fast machine, so the
  10m per-binary budget set when the package was smaller times out on
  CI runners. Raise to 25m. The package has no sleep-debt (all test
  sleeps are milliseconds); it is legitimately heavy.
2026-06-11 12:02:29 +01:00
rcourtman edd7f001ab Restore pulse:test image build in the e2e workflow
The Core E2E workflow on this branch lost the 'docker build -t
pulse:test --target runtime .' line that release/5.1 still has, so
every dispatched run failed at compose-up with 'pull access denied for
pulse'. Masked until now because the workflow only auto-triggers on
main.
2026-06-10 12:10:32 +01:00
rcourtman bd6f77e093 Prepare v6.0.0 release candidate
Tighten v5-to-v6 upgrade safety, release installability, provider MSP mode handling, AI cost accounting, metrics flushing, and frontend guardrails for the v6.0.0 GA candidate.
2026-06-04 14:07:14 +01:00
rcourtman 0d0eb4bf11 Stabilize v6 release dry-run backend gate 2026-06-03 18:12:42 +01:00
rcourtman 42fb8eed3f Publish provider MSP control-plane image 2026-06-02 15:51:21 +01:00
rcourtman 52fd847ac6 Let install.sh smoke harness opt past Docker-environment refusal
install.sh refuses to run inside Docker (correct behavior for end
users), but the install-sh-smoke gate runs the documented systemd
install path inside a privileged systemd-in-Docker container — the
one legitimate bypass case. Added PULSE_INSTALL_ALLOW_DOCKER=1
escape hatch in check_docker_environment() and set it on the docker
exec in the smoke workflow. Takes effect on the next RC; v6.0.0-rc.6's
published install.sh predates this and is signed-frozen, so its
post-publish smoke gate will remain red until the next prerelease.
2026-05-27 22:12:17 +01:00
rcourtman 7c35977fcb Fix post-publish pipeline failures for v6.0.0-rc.6
promote-floating-tags.yml waited on rcourtman/pulse-agent:${TAG} and
promoted floating tags for it, but publish-docker.yml never pushes the
agent image — Pulse Agent ships as GitHub Release binaries
(publish-docker.yml line 199 confirms). The wait timed out after 5
minutes on every release since the pulse-agent push step was removed,
leaving floating tags unpromoted. Stripped the vestigial wait block,
the agent promote step, and the agent line in the summary. Updated
header comments in both files to drop the past-tense reference.

install-sh-smoke.yml booted jrei/systemd-debian:12 without
--cgroupns=host, so on GHA ubuntu-24.04 (cgroup v2 unified hierarchy)
the container's systemd PID 1 exited before mounting the cgroup tree
and the container disappeared during readiness polling. Added
--cgroupns=host, an explicit /run/lock tmpfs, and dropped --rm so the
container persists for diagnostic capture on failure (trap handles
cleanup). Added an "is container still running" probe inside the
readiness loop and richer diagnostic output on timeout.
2026-05-27 22:08:15 +01:00
rcourtman 1dfc4ee6d3 Require .sshsig sidecar for install.sh --archive path
The --archive PATH flag (and the Proxmox LXC bootstrap which
propagates --archive into the container) reached install_pulse_archive
without going through download_release_archive, which is the only
place that previously verified the cryptographic signature on a
Pulse release tarball. That made --archive a clean signature-
verification bypass: anyone who could swap a local tarball
between download and install (shared /tmp on a Proxmox host, a
maintainer testing a swapped local file, etc.) would get
arbitrary root code execution at extract time.

install.sh:
- install_pulse_archive now requires ${archive_path}.sshsig and
  verifies it via the existing verify_release_signature helper
  before tar runs. Missing or invalid signature is a hard
  fail-closed.
- download_release_archive now keeps the .sshsig alongside the
  archive at ${archive_path}.sshsig instead of stashing it in a
  mktemp file that got deleted post-verify. install_pulse_archive
  re-verifies it; no behavior change for the curl|bash path beyond
  the extra integrity check at extract time.
- Proxmox LXC bootstrap now pct push'es ${container_archive_source}.sshsig
  alongside the archive into the container so the in-container
  install_pulse_archive run can verify. Missing sidecar fails the
  bootstrap (cleanup_on_error tears down the half-provisioned
  container).
- Cleanup paths that remove the temp archive on success/failure
  now also remove the sidecar.
- --help advertises the .sshsig requirement.

install-sh-smoke.yml:
- Smoke test now pulls ${tarball}.sshsig alongside the tarball
  before running install.sh --archive inside the test container.

Trade-off: maintainer flows that did `install.sh --archive
/path/to/local.tar.gz` against an unsigned local tarball will now
fail with a clear "Required signature sidecar not found" error.
Maintainers can either pull the corresponding .sshsig from the
release alongside the tarball, or skip --archive in favour of the
canonical download path.
2026-05-21 16:13:57 +01:00
rcourtman 2305131d8a Move eval-model-matrix off self-hosted runner
Self-hosted runners attached to a public repo are a known
severe-risk pattern: any workflow that gains the ability to run
PR-controlled code escalates to RCE on the runner host. Even
though this workflow is workflow_dispatch only today, the runner
itself sits one workflow edit away from arbitrary code execution
on whatever machine it's pinned to.

Changes:

- runs-on: self-hosted -> ubuntu-24.04. Reachability of base_url
  from the GitHub-hosted runner is now the dispatcher's
  responsibility. Add tailscale/* steps (like deploy-v6-preview-
  demo.yml or deploy-demo-server.yml already do) if pointing at
  a private Pulse instance.

- Drop the admin/admin fallback on PULSE_EVAL_USER and
  PULSE_EVAL_PASS. The previous default was
  ${{ secrets.PULSE_EVAL_USER || 'admin' }} which silently ran
  the eval as admin/admin if the secrets were missing. Added a
  fail-closed precheck that refuses to proceed unless both
  secrets are set.

- Route inputs.scenario, inputs.models, inputs.providers, and
  inputs.base_url through env: indirection instead of inline
  ${{ }} interpolation into shell. Defense-in-depth against
  script injection from the dispatch payload.

Net result: kills the audit's only CRITICAL finding and the
companion MEDIUM (admin/admin fallback). Eval can still run
exactly as before once secrets are set and base_url is reachable.
2026-05-21 16:09:09 +01:00