From 91f9d656ae108e5fee7c44beffaa84860ef8ca16 Mon Sep 17 00:00:00 2001 From: Oliver Kieselbach Date: Wed, 19 Aug 2026 15:09:33 +0200 Subject: [PATCH] ci(scripts): publish customer scripts on push, decoupled from the agent release The scripts behind download.autopilotmonitor.com/agent/ rotated only on a publish_as_stable run of build-agent.yml, so a bootstrap fix sat unpublished until the next agent cutover -- while the WDP bootstrap MSI downloads the published copy on every enrolling device. Publish-BootstrapScripts.ps1 becomes the single owner of the chain: dev render with anchor check, $ScriptVersion bump guard, uploads (eu + fail-soft legacy mirror, no-cache), the two version oracles, and verification through the alias. publish-scripts.yml calls it on every push to main touching the sources; the publish_as_stable branch of build-agent.yml calls the same code, so a release still lands a consistent set. Blobs are published in a deterministic CRLF/UTF-8-without-BOM form. Git normalises line endings on checkout, so raw working-copy bytes would differ per environment and make every byte comparison -- guard and verify alike -- depend on where the workflow ran. The form reproduces the currently published bytes exactly. The Pester (PS 5.1) and ASCII gates move into the reusable bootstrap-script-gates.yml and now block the upload; pester.yml keeps the pull-request view and drops its push trigger. AdminConfiguration .LatestBootstrapV2ScriptVersion gets exactly one writer. Co-Authored-By: Claude Opus 5 (1M context) --- .github/scripts/Publish-BootstrapScripts.ps1 | 317 +++++++++++++++++++ .github/workflows/bootstrap-script-gates.yml | 63 ++++ .github/workflows/build-agent.yml | 59 ++-- .github/workflows/pester.yml | 58 +--- .github/workflows/publish-scripts.yml | 56 ++++ docs/index.md | 1 + docs/log.md | 4 + docs/script-publishing.md | 128 ++++++++ 8 files changed, 604 insertions(+), 82 deletions(-) create mode 100644 .github/scripts/Publish-BootstrapScripts.ps1 create mode 100644 .github/workflows/bootstrap-script-gates.yml create mode 100644 .github/workflows/publish-scripts.yml create mode 100644 docs/script-publishing.md diff --git a/.github/scripts/Publish-BootstrapScripts.ps1 b/.github/scripts/Publish-BootstrapScripts.ps1 new file mode 100644 index 00000000..eb421f94 --- /dev/null +++ b/.github/scripts/Publish-BootstrapScripts.ps1 @@ -0,0 +1,317 @@ +<# +.SYNOPSIS + Publishes the customer-facing PowerShell scripts behind download.autopilotmonitor.com/agent/. + +.DESCRIPTION + Single owner of the script-publishing chain. Called from two places: + + * publish-scripts.yml -- on every push to main that touches the sources, + so a script fix reaches customers without waiting + for an agent release. + * build-agent.yml -- on a publish_as_stable run, so a release always + lands a consistent set (ZIP + manifest + scripts). + + Both callers run the same code. The scripts on the download alias are the live + copy the WDP bootstrap MSI (Invoke-BootstrapDownload.ps1) downloads on every + enrolling device, so "current" is a correctness property, not cosmetics. + + Steps, in order: + 1. Parse $ScriptVersion from the bootstrap source. + 2. Render the -Dev variant by literal substitution of the two URL/manifest + defaults. A missing anchor is a hard failure -- a silently un-substituted + dev script would point the dev fleet at the stable agent. + 3. Version-bump guard: if the published bootstrap differs from the source + but carries the same $ScriptVersion, abort. Otherwise the docs badge, + version.json.bootstrapVersion and the portal's "your script is outdated" + hint would all keep asserting a version that is no longer what ships. + 4. Upload every script blob with Cache-Control: no-cache (they rotate in + place), mirrored fail-soft to the legacy account. + 5. Reconcile the two version oracles: version.json.bootstrapVersion + (read-modify-write under If-Match, so a concurrent agent release cannot + lose its agent fields) and AdminConfiguration.LatestBootstrapV2ScriptVersion. + 6. Verify through the alias -- re-download each blob and compare SHA-256. + +.PARAMETER SasToken + Container SAS for the eu storage account with write permission. Leading '?' tolerated. + +.PARAMETER LegacySasToken + Container SAS for the legacy account. Optional -- a mirror failure warns, never fails. + +.PARAMETER TableSasToken + Table SAS for AdminConfiguration. Optional -- omitted or 'XXX' skips the oracle update. + +.PARAMETER RepoRoot + Repository root. Defaults to the parent of .github/scripts. + +.PARAMETER DryRun + Run every check and render, upload nothing. + +.EXAMPLE + ./Publish-BootstrapScripts.ps1 -DryRun + +.EXAMPLE + ./Publish-BootstrapScripts.ps1 -SasToken $env:SAS -TableSasToken $env:TABLE_SAS +#> +[CmdletBinding()] +param( + [string]$SasToken, + [string]$LegacySasToken, + [string]$TableSasToken, + [string]$RepoRoot, + [switch]$DryRun +) + +$ErrorActionPreference = 'Stop' + +if (-not $RepoRoot) { + $RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '../..')).Path +} +if (-not $DryRun -and [string]::IsNullOrWhiteSpace($SasToken)) { + throw 'SasToken is required unless -DryRun is used.' +} + +$ContainerUrl = 'https://autopilotmonitoreu.blob.core.windows.net/agent' +$LegacyContainerUrl = 'https://autopilotmonitor.blob.core.windows.net/agent' +$AliasUrl = 'https://download.autopilotmonitor.com/agent' +$TableUrl = "https://autopilotmonitoreu.table.core.windows.net/AdminConfiguration(PartitionKey='GlobalConfig',RowKey='config')" + +$BootstrapSource = Join-Path $RepoRoot 'scripts/Bootstrap/Install-AutopilotMonitor.ps1' +$BootstrapBlob = 'Install-AutopilotMonitor.ps1' +$DevBlob = 'Install-AutopilotMonitor-Dev.ps1' +$ScriptContentType = 'text/plain; charset=utf-8' + +function Get-HttpStatus { + param($ErrorRecord) + if ($ErrorRecord.Exception.PSObject.Properties['Response'] -and $ErrorRecord.Exception.Response) { + return [int]$ErrorRecord.Exception.Response.StatusCode + } + return 0 +} + +function Get-Sha256 { + param([byte[]]$Bytes) + $sha = [System.Security.Cryptography.SHA256]::Create() + try { return [BitConverter]::ToString($sha.ComputeHash($Bytes)).Replace('-', '').ToLowerInvariant() } + finally { $sha.Dispose() } +} + +# Raw bytes of a published blob, or $null when it does not exist yet. +function Get-PublishedBytes { + param([string]$Url) + try { + $resp = Invoke-WebRequest -Uri $Url -Method Get -UseBasicParsing + } catch { + if ((Get-HttpStatus $_) -eq 404) { return $null } + throw + } + if ($resp.Content -is [byte[]]) { return $resp.Content } + return [System.Text.Encoding]::UTF8.GetBytes([string]$resp.Content) +} + +# Deterministic publish form: CRLF, UTF-8 without BOM. Git normalises line endings on +# checkout (a Windows runner yields CRLF, a core.autocrlf=false clone yields LF), so +# publishing the raw working-copy bytes would emit different blobs for identical source +# and make every byte comparison -- guard and verify alike -- environment-dependent. +# CRLF is also what PS 5.1 on the device expects and what is published today. +function Get-PublishBytes { + param([string]$Text) + $normalised = $Text.Replace("`r`n", "`n").Replace("`n", "`r`n") + return (New-Object System.Text.UTF8Encoding $false).GetBytes($normalised) +} + +function Get-BootstrapScriptVersion { + param([string]$Content, [string]$Origin) + if ($Content -match '\$ScriptVersion\s*=\s*"([\d\.\-a-zA-Z]+)"') { return $Matches[1] } + throw "Could not parse ScriptVersion from $Origin" +} + +# ------------------------------------------------------------------ 1. source +$sourceContent = Get-Content $BootstrapSource -Raw +$sourceBytes = Get-PublishBytes $sourceContent +$scriptVersion = Get-BootstrapScriptVersion -Content $sourceContent -Origin $BootstrapSource +Write-Host "Bootstrap script version: $scriptVersion" + +# ------------------------------------------------------------------ 2. dev render +$stableUrlLiteral = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip"' +$devUrlLiteral = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent-dev.zip"' +$stableManLiteral = '$VersionJsonName = "version.json"' +$devManLiteral = '$VersionJsonName = "version-dev.json"' + +if ($sourceContent.IndexOf($stableUrlLiteral) -lt 0 -or $sourceContent.IndexOf($stableManLiteral) -lt 0) { + throw "Bootstrap dev-render: anchor literal missing in $BootstrapSource" +} +$devContent = $sourceContent.Replace($stableUrlLiteral, $devUrlLiteral).Replace($stableManLiteral, $devManLiteral) +Write-Host "Rendered $DevBlob (dev agent URL + version-dev.json)" + +# ------------------------------------------------------------------ 3. bump guard +$publishedBytes = Get-PublishedBytes "$AliasUrl/$BootstrapBlob" +if ($null -eq $publishedBytes) { + Write-Host 'No published bootstrap yet -- version-bump guard skipped (first publish)' +} elseif ((Get-Sha256 $publishedBytes) -eq (Get-Sha256 $sourceBytes)) { + Write-Host 'Published bootstrap is already byte-identical to the source' +} else { + $publishedVersion = Get-BootstrapScriptVersion -Content ([System.Text.Encoding]::UTF8.GetString($publishedBytes)) -Origin "$AliasUrl/$BootstrapBlob" + if ($publishedVersion -eq $scriptVersion) { + throw ("Version-bump guard: $BootstrapBlob changed but ScriptVersion is still $scriptVersion. " + + "Bump it in $BootstrapSource -- the docs badge, version.json.bootstrapVersion and the " + + 'portal outdated-script hint all read that value and would keep reporting a version ' + + 'that no longer matches what customers download.') + } + Write-Host "Version bump: $publishedVersion -> $scriptVersion" +} + +# ------------------------------------------------------------------ 4. upload +# Sources beyond the bootstrap pair, read straight from the repo. +$extraSources = @( + @{ Path = 'scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1'; BlobName = 'Test-ShouldBootstrapAgent.ps1' } + # Customer-side Graph add-on grant script -- docs and admin UI link it directly. + @{ Path = 'scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1'; BlobName = 'Grant-AutopilotMonitorAddOn.ps1' } +) +# Uninstall-AutopilotMonitor.ps1 is deliberately NOT published: nothing links it, and +# an unauthenticated uninstall script on the public download host is not a feature. + +$publishSet = [System.Collections.Generic.List[object]]::new() +$publishSet.Add([pscustomobject]@{ BlobName = $BootstrapBlob; Bytes = $sourceBytes }) +$publishSet.Add([pscustomobject]@{ BlobName = $DevBlob; Bytes = (Get-PublishBytes $devContent) }) +foreach ($extra in $extraSources) { + $extraPath = Join-Path $RepoRoot $extra.Path + if (-not (Test-Path $extraPath)) { + throw "Publish source missing: $extraPath" + } + $publishSet.Add([pscustomobject]@{ BlobName = $extra.BlobName; Bytes = (Get-PublishBytes (Get-Content $extraPath -Raw)) }) +} + +$legacySas = if ($LegacySasToken) { $LegacySasToken.TrimStart('?') } else { '' } +$writeSas = if ($SasToken) { $SasToken.TrimStart('?') } else { '' } + +foreach ($item in $publishSet) { + $bytes = $item.Bytes + $item | Add-Member -NotePropertyName Sha256 -NotePropertyValue (Get-Sha256 $bytes) -Force + + if ($DryRun) { + # Doubles as a drift check: says per blob whether the alias already serves this source. + $served = Get-PublishedBytes "$AliasUrl/$($item.BlobName)" + $state = if ($null -eq $served) { 'MISSING on the alias' } + elseif ((Get-Sha256 $served) -eq $item.Sha256) { 'already current' } + else { 'STALE on the alias' } + Write-Host " [dry-run] $($item.BlobName) ($($bytes.Length) bytes, sha256 $($item.Sha256)) -- $state" + continue + } + + # no-cache on every blob: they all rotate in place and the download alias fronts them + # with Front Door. Route caching is disabled there, but a stale script paired with a + # fresh manifest would fail the bootstrap SHA check -- this keeps that class of bug + # impossible even if caching is ever re-enabled. + $headers = @{ 'x-ms-blob-type' = 'BlockBlob'; 'Content-Type' = $ScriptContentType; 'x-ms-blob-cache-control' = 'no-cache' } + Invoke-RestMethod -Uri "$ContainerUrl/$($item.BlobName)?$writeSas" -Method Put -Headers $headers -Body $bytes | Out-Null + Write-Host " uploaded $($item.BlobName) ($($bytes.Length) bytes)" + + if ($legacySas) { + try { + Invoke-RestMethod -Uri "$LegacyContainerUrl/$($item.BlobName)?$legacySas" -Method Put -Headers $headers -Body $bytes | Out-Null + Write-Host " mirrored $($item.BlobName) to legacy storage" + } catch { + Write-Host "::warning::legacy mirror failed for $($item.BlobName) -- $($_.Exception.Message)" + } + } +} + +# ------------------------------------------------------------------ 5. version oracles +# version.json is the agent manifest; only bootstrapVersion belongs to us. Read from the +# blob (not the alias) for an authoritative ETag, write back under If-Match so a concurrent +# agent release cannot lose its version/sha256 fields. +if (-not $DryRun) { + try { + $manifestResp = Invoke-WebRequest -Uri "$ContainerUrl/version.json" -Method Get -UseBasicParsing + $manifestRaw = if ($manifestResp.Content -is [byte[]]) { + [System.Text.Encoding]::UTF8.GetString($manifestResp.Content) + } else { [string]$manifestResp.Content } + $manifest = $manifestRaw | ConvertFrom-Json + + if ($manifest.bootstrapVersion -eq $scriptVersion) { + Write-Host "version.json already reports bootstrapVersion $scriptVersion" + } else { + $etag = '"' + ([string]($manifestResp.Headers['ETag'] | Select-Object -First 1)).Trim('"') + '"' + $manifest | Add-Member -NotePropertyName 'bootstrapVersion' -NotePropertyValue $scriptVersion -Force + $body = [System.Text.Encoding]::UTF8.GetBytes(($manifest | ConvertTo-Json -Compress)) + $manifestHeaders = @{ + 'x-ms-blob-type' = 'BlockBlob' + 'Content-Type' = 'application/json' + 'x-ms-blob-cache-control' = 'no-cache' + 'If-Match' = $etag + } + Invoke-RestMethod -Uri "$ContainerUrl/version.json?$writeSas" -Method Put -Headers $manifestHeaders -Body $body | Out-Null + Write-Host "version.json bootstrapVersion -> $scriptVersion (agent fields untouched)" + + if ($legacySas) { + try { + $legacyHeaders = @{ 'x-ms-blob-type' = 'BlockBlob'; 'Content-Type' = 'application/json'; 'x-ms-blob-cache-control' = 'no-cache' } + Invoke-RestMethod -Uri "$LegacyContainerUrl/version.json?$legacySas" -Method Put -Headers $legacyHeaders -Body $body | Out-Null + } catch { + Write-Host "::warning::legacy mirror failed for version.json -- $($_.Exception.Message)" + } + } + } + } catch { + if ((Get-HttpStatus $_) -eq 412) { + throw 'version.json changed while publishing (concurrent agent release). Re-run this workflow.' + } + throw + } +} + +$tableSas = if ($TableSasToken) { $TableSasToken.TrimStart('?') } else { '' } +if ($DryRun) { + Write-Host " [dry-run] would set AdminConfiguration.LatestBootstrapV2ScriptVersion = $scriptVersion" +} elseif ([string]::IsNullOrEmpty($tableSas) -or $tableSas -eq 'XXX') { + Write-Host 'SKIPPED: AdminConfiguration update -- set AZURE_TABLE_ADMIN_SAS_TOKEN to enable the portal version oracle' +} else { + $tableHeaders = @{ + 'Content-Type' = 'application/json' + 'Accept' = 'application/json;odata=nometadata' + 'If-Match' = '*' + } + $tableBody = @{ LatestBootstrapV2ScriptVersion = $scriptVersion } | ConvertTo-Json + Invoke-RestMethod -Uri "$TableUrl`?$tableSas" -Method Merge -Headers $tableHeaders -Body $tableBody | Out-Null + Write-Host "AdminConfiguration.LatestBootstrapV2ScriptVersion = $scriptVersion" +} + +# ------------------------------------------------------------------ 6. verify via alias +if (-not $DryRun) { + $pending = [System.Collections.ArrayList]::new() + $publishSet | ForEach-Object { [void]$pending.Add($_) } + + for ($attempt = 1; $attempt -le 6 -and $pending.Count -gt 0; $attempt++) { + if ($attempt -gt 1) { Start-Sleep -Seconds 20 } + foreach ($item in @($pending)) { + $servedBytes = Get-PublishedBytes "$AliasUrl/$($item.BlobName)" + if ($null -ne $servedBytes -and (Get-Sha256 $servedBytes) -eq $item.Sha256) { + Write-Host " verified $($item.BlobName) via $AliasUrl" + $pending.Remove($item) + } + } + if ($pending.Count -gt 0) { + Write-Host " attempt $attempt : $($pending.Count) blob(s) not current on the alias yet" + } + } + + if ($pending.Count -gt 0) { + $names = ($pending | ForEach-Object { $_.BlobName }) -join ', ' + throw ("Alias still serves stale content for: $names. The blobs were written, so this is a Front Door " + + 'cache issue -- purge with: az afd endpoint purge --resource-group rg-autopilotmonitor-prd-gwc ' + + "--profile-name autopilotmonitor-fd --endpoint-name apm-download --content-paths '/agent/*'") + } +} + +if ($env:GITHUB_STEP_SUMMARY) { + $mode = if ($DryRun) { 'DRY RUN -- nothing uploaded' } else { 'published + verified via the download alias' } + $summary = @( + "### Bootstrap scripts: $mode", + '', + "Bootstrap script version: **$scriptVersion**", + '' + ) + ($publishSet | ForEach-Object { "- $($_.BlobName) -- sha256 $($_.Sha256)" }) + ($summary -join [Environment]::NewLine) | Out-File -FilePath $env:GITHUB_STEP_SUMMARY -Append -Encoding utf8 +} + +Write-Host 'Done.' diff --git a/.github/workflows/bootstrap-script-gates.yml b/.github/workflows/bootstrap-script-gates.yml new file mode 100644 index 00000000..46cd1045 --- /dev/null +++ b/.github/workflows/bootstrap-script-gates.yml @@ -0,0 +1,63 @@ +name: Bootstrap script gates + +# Quality gate for the customer-facing PowerShell scripts, factored out so it exists +# once and both callers run the identical checks: +# * pester.yml -- pull requests (reports, does not block the merge button) +# * publish-scripts.yml -- pushes to main, where it BLOCKS the upload to the +# download alias. Devices execute these scripts as SYSTEM +# during enrollment, so a red suite must never ship. + +on: + workflow_call: + +permissions: + contents: read + +jobs: + gates: + name: Bootstrap guard tests (PS 5.1) + runs-on: windows-latest + steps: + - uses: actions/checkout@v7 + + - name: Install Pester 5+ + shell: powershell + run: | + Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force -Scope CurrentUser | Out-Null + Install-Module Pester -Scope CurrentUser -Force -SkipPublisherCheck -MinimumVersion 5.5.0 + + # Windows PowerShell 5.1 on purpose: that is the runtime IME uses to execute + # platform scripts, so guard/relax logic is verified without resetting a device. + - name: Run Pester + shell: powershell + run: | + Import-Module Pester -MinimumVersion 5.5.0 + $c = New-PesterConfiguration + $c.Run.Path = 'scripts/Bootstrap' + $c.Run.Exit = $true + $c.Output.Verbosity = 'Detailed' + Invoke-Pester -Configuration $c + + - name: Guard scripts stay pure ASCII + shell: powershell + run: | + $failed = $false + Get-ChildItem 'scripts/Bootstrap' -Filter '*.ps1' | ForEach-Object { + $bytes = [System.IO.File]::ReadAllBytes($_.FullName) + for ($i = 0; $i -lt $bytes.Length; $i++) { + if ($bytes[$i] -gt 127) { + Write-Host "::error file=scripts/Bootstrap/$($_.Name)::Non-ASCII byte 0x$('{0:X2}' -f $bytes[$i]) at offset $i. PS 5.1 (IME) reads BOM-less files as ANSI and corrupts multi-byte chars." + $failed = $true + break + } + } + } + if ($failed) { exit 1 } + Write-Host 'All bootstrap scripts are pure ASCII.' + + # The publish chain renders the -Dev variant by literal substitution and refuses to + # ship when an anchor moved. Running it dry here surfaces that (and any drift against + # the alias) on the PR instead of on the push that would have published. + - name: Publish dry run (dev-render anchors + version-bump guard + drift) + shell: pwsh + run: ./.github/scripts/Publish-BootstrapScripts.ps1 -DryRun diff --git a/.github/workflows/build-agent.yml b/.github/workflows/build-agent.yml index 01128339..3d63c8a3 100644 --- a/.github/workflows/build-agent.yml +++ b/.github/workflows/build-agent.yml @@ -33,6 +33,13 @@ permissions: id-token: write # keyless Sigstore signing (artifact attestation) attestations: write # store the provenance attestation +# Shared with publish-scripts.yml: both write version.json (this workflow the agent fields, +# the script publisher bootstrapVersion under If-Match). Serialising them keeps a script push +# from colliding with a release. Never cancel -- a half-published set is worse than a queue. +concurrency: + group: agent-blob-publish + cancel-in-progress: false + env: LINE_MAJOR: 2 AGENT_PROJECT: src/Agent/AutopilotMonitor.Agent.V2/AutopilotMonitor.Agent.V2.csproj @@ -46,8 +53,6 @@ env: VERSIONED_ZIP: AutopilotMonitor-Agent-v2.zip STABLE_ZIP: AutopilotMonitor-Agent.zip STABLE_MANIFEST: version.json - STABLE_BOOTSTRAP: Install-AutopilotMonitor.ps1 - STABLE_BOOTSTRAP_DEV: Install-AutopilotMonitor-Dev.ps1 jobs: build: @@ -181,6 +186,8 @@ jobs: # commit/buildUtc are the same two fields the backend and MCP manifests carry — # additive, and the bootstrap reads this file by field name, so older installers # ignore them. sha256/bootstrapVersion stay: those are the customer contract. + # bootstrapVersion is seeded here so the published manifest is never briefly + # missing it; Publish-BootstrapScripts.ps1 owns and re-asserts the field afterwards. $manifest | Add-Member -NotePropertyName "sha256" -NotePropertyValue $sha256Hash -Force $manifest | Add-Member -NotePropertyName "commit" -NotePropertyValue '${{ github.sha }}'.Substring(0, 7) -Force $manifest | Add-Member -NotePropertyName "buildUtc" -NotePropertyValue ([DateTime]::UtcNow.ToString('o')) -Force @@ -190,29 +197,12 @@ jobs: $manifest | ConvertTo-Json -Compress | Set-Content $env:MANIFEST_NAME -Encoding UTF8 Write-Host "$env:MANIFEST_NAME content: $(Get-Content $env:MANIFEST_NAME)" - # Bootstrap = single generic source. Stable variant = source 1:1. - # Dev variant = literal-string substitution of the two URL/manifest defaults. - # Both are uploaded only on -PublishAsStable runs (versioned namespace doesn't carry bootstrap). - if ('${{ inputs.publish_as_stable }}' -eq 'true') { - $stableUrl = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip"' - $devUrl = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent-dev.zip"' - $stableMan = '$VersionJsonName = "version.json"' - $devMan = '$VersionJsonName = "version-dev.json"' - - Copy-Item $env:BOOTSTRAP_SOURCE $env:STABLE_BOOTSTRAP -Force - $content = Get-Content $env:BOOTSTRAP_SOURCE -Raw - if ($content.IndexOf($stableUrl) -lt 0 -or $content.IndexOf($stableMan) -lt 0) { - Write-Error "Bootstrap dev-render: anchor literal missing in $env:BOOTSTRAP_SOURCE" - exit 1 - } - $devContent = $content.Replace($stableUrl, $devUrl).Replace($stableMan, $devMan) - [System.IO.File]::WriteAllText("$env:STABLE_BOOTSTRAP_DEV", $devContent, (New-Object System.Text.UTF8Encoding $false)) - } + # The customer-facing scripts (stable + dev render, test script, grant script) are + # published by .github/scripts/Publish-BootstrapScripts.ps1 -- see the upload step. "versioned_zip_release=$versionedZip" >> $env:GITHUB_OUTPUT "sha256=$sha256Hash" >> $env:GITHUB_OUTPUT "exe_sha256=$exeSha256Hash" >> $env:GITHUB_OUTPUT - "bootstrap_version=$bootstrapVersion" >> $env:GITHUB_OUTPUT - name: Upload to Azure Blob Storage if: inputs.upload_blob @@ -264,22 +254,21 @@ jobs: Write-Host "PUBLISH AS STABLE -- writing V2 into stable namespace (customer-facing)" -ForegroundColor Magenta Send-Blob -Local $env:VERSIONED_ZIP -BlobName $env:STABLE_ZIP -ContentType "application/zip" Send-Blob -Local $env:MANIFEST_NAME -BlobName $env:STABLE_MANIFEST -ContentType "application/json" - Send-Blob -Local $env:STABLE_BOOTSTRAP -BlobName $env:STABLE_BOOTSTRAP -ContentType "text/plain; charset=utf-8" - Send-Blob -Local $env:STABLE_BOOTSTRAP_DEV -BlobName $env:STABLE_BOOTSTRAP_DEV -ContentType "text/plain; charset=utf-8" - if (Test-Path "scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1") { - Send-Blob -Local "scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1" -BlobName "Test-ShouldBootstrapAgent.ps1" -ContentType "text/plain; charset=utf-8" - } - - # Customer-side Graph add-on grant script -- docs and admin UI link - # https://download.autopilotmonitor.com/agent/Grant-AutopilotMonitorAddOn.ps1 - if (Test-Path "scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1") { - Send-Blob -Local "scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1" -BlobName "Grant-AutopilotMonitorAddOn.ps1" -ContentType "text/plain; charset=utf-8" - } + # Customer-facing scripts: same publisher publish-scripts.yml runs on every push + # to main, so a release lands a set that is consistent with what is already live + # (dev render, version-bump guard, bootstrapVersion oracles, alias verification). + # Runs AFTER version.json so its bootstrapVersion read sees this build's manifest. + ./.github/scripts/Publish-BootstrapScripts.ps1 ` + -SasToken '${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}' ` + -LegacySasToken '${{ secrets.AZURE_BLOB_AGENT_LEGACY_SAS_TOKEN }}' ` + -TableSasToken '${{ secrets.AZURE_TABLE_ADMIN_SAS_TOKEN }}' } # Update AdminConfiguration LatestAgentV2* -- ONLY on stable cutover. The hash-oracle # must never point at a build that is not actually in the stable namespace. + # LatestBootstrapV2ScriptVersion is NOT set here: it belongs to the script publisher + # above, which also owns it on script-only pushes. One writer per field. $tableSasToken = "${{ secrets.AZURE_TABLE_ADMIN_SAS_TOKEN }}".TrimStart('?') if ('${{ inputs.publish_as_stable }}' -ne 'true') { Write-Host "SKIPPED: AdminConfiguration update (versioned-only run -- stable namespace unchanged)" @@ -289,22 +278,18 @@ jobs: $sha256 = "${{ steps.artifact.outputs.sha256 }}" $exeSha256 = "${{ steps.artifact.outputs.exe_sha256 }}" $agentVersion = "${{ steps.version.outputs.version }}" - $bootstrapVersion = "${{ steps.artifact.outputs.bootstrap_version }}" $tableUrl = "https://autopilotmonitoreu.table.core.windows.net/AdminConfiguration(PartitionKey='GlobalConfig',RowKey='config')?$tableSasToken" $tableBody = @{ LatestAgentV2Version = $agentVersion LatestAgentV2Sha256 = $sha256 LatestAgentV2ExeSha256 = $exeSha256 } - if ($bootstrapVersion) { - $tableBody["LatestBootstrapV2ScriptVersion"] = $bootstrapVersion - } Invoke-RestMethod -Uri $tableUrl -Method Merge -Headers @{ "Content-Type" = "application/json" "Accept" = "application/json;odata=nometadata" "If-Match" = "*" } -Body ($tableBody | ConvertTo-Json) - Write-Host "AdminConfiguration updated: V2 version=$agentVersion, sha256=$sha256, exeSha256=$exeSha256, bootstrapVersion=$bootstrapVersion" + Write-Host "AdminConfiguration updated: V2 version=$agentVersion, sha256=$sha256, exeSha256=$exeSha256" } # Sigstore keyless provenance: binds the ZIP digest to this workflow run (repo, ref, diff --git a/.github/workflows/pester.yml b/.github/workflows/pester.yml index 1204a221..c5c09fcc 100644 --- a/.github/workflows/pester.yml +++ b/.github/workflows/pester.yml @@ -1,16 +1,20 @@ name: Pester (Bootstrap scripts) -# Runs the Pester suite for the Intune bootstrap scripts under Windows PowerShell 5.1 -# (the runtime IME uses to execute platform scripts) so guard/relax logic changes are -# verified without resetting a device. Like ci.yml it reports rather than blocks. +# Pull-request view of the bootstrap script gates. Like ci.yml it reports rather than +# blocks the merge button. +# +# There is deliberately no push trigger: publish-scripts.yml runs the very same +# reusable gate workflow on every push to main and blocks the upload to the download +# alias on it. A push trigger here would only duplicate that run, with no consequence. on: - push: - branches: [main] - paths: ['scripts/Bootstrap/**'] pull_request: branches: [main] - paths: ['scripts/Bootstrap/**'] + paths: + - 'scripts/Bootstrap/**' + - 'scripts/CustomerSetup/**' + - '.github/scripts/Publish-BootstrapScripts.ps1' + - '.github/workflows/bootstrap-script-gates.yml' workflow_dispatch: permissions: @@ -21,41 +25,5 @@ concurrency: cancel-in-progress: true jobs: - pester: - name: Bootstrap guard tests (PS 5.1) - runs-on: windows-latest - steps: - - uses: actions/checkout@v7 - - - name: Install Pester 5+ - shell: powershell - run: | - Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force -Scope CurrentUser | Out-Null - Install-Module Pester -Scope CurrentUser -Force -SkipPublisherCheck -MinimumVersion 5.5.0 - - - name: Run Pester - shell: powershell - run: | - Import-Module Pester -MinimumVersion 5.5.0 - $c = New-PesterConfiguration - $c.Run.Path = 'scripts/Bootstrap' - $c.Run.Exit = $true - $c.Output.Verbosity = 'Detailed' - Invoke-Pester -Configuration $c - - - name: Guard scripts stay pure ASCII - shell: powershell - run: | - $failed = $false - Get-ChildItem 'scripts/Bootstrap' -Filter '*.ps1' | ForEach-Object { - $bytes = [System.IO.File]::ReadAllBytes($_.FullName) - for ($i = 0; $i -lt $bytes.Length; $i++) { - if ($bytes[$i] -gt 127) { - Write-Host "::error file=scripts/Bootstrap/$($_.Name)::Non-ASCII byte 0x$('{0:X2}' -f $bytes[$i]) at offset $i. PS 5.1 (IME) reads BOM-less files as ANSI and corrupts multi-byte chars." - $failed = $true - break - } - } - } - if ($failed) { exit 1 } - Write-Host 'All bootstrap scripts are pure ASCII.' + gates: + uses: ./.github/workflows/bootstrap-script-gates.yml diff --git a/.github/workflows/publish-scripts.yml b/.github/workflows/publish-scripts.yml new file mode 100644 index 00000000..47016900 --- /dev/null +++ b/.github/workflows/publish-scripts.yml @@ -0,0 +1,56 @@ +name: Publish customer scripts + +# Keeps the scripts behind download.autopilotmonitor.com/agent/ current WITHOUT waiting +# for an agent release. Before this workflow existed, the script blobs rotated only on a +# publish_as_stable run of build-agent.yml, so a bootstrap fix sat unpublished until the +# next agent cutover -- while the WDP bootstrap MSI downloads the published copy on every +# enrolling device. +# +# Same publish code as the release path (.github/scripts/Publish-BootstrapScripts.ps1); +# this workflow only supplies a second trigger for it. + +on: + push: + branches: [main] + paths: + - 'scripts/Bootstrap/**' + - 'scripts/CustomerSetup/**' + - '.github/scripts/Publish-BootstrapScripts.ps1' + workflow_dispatch: + inputs: + dry_run: + description: 'Run every check and report drift, upload nothing' + required: true + type: boolean + default: false + +permissions: + contents: read + +# Shared with build-agent.yml: a script publish and an agent release both touch +# version.json, and the bootstrapVersion read-modify-write must not interleave with the +# release writing its agent fields. Never cancel -- a half-published set is worse than a queue. +concurrency: + group: agent-blob-publish + cancel-in-progress: false + +jobs: + gates: + uses: ./.github/workflows/bootstrap-script-gates.yml + + publish: + name: Publish to the download alias + needs: gates + runs-on: windows-latest + steps: + - uses: actions/checkout@v7 + + - name: Publish bootstrap + customer scripts + shell: pwsh + run: | + $dryRun = '${{ inputs.dry_run }}' -eq 'true' + ./.github/scripts/Publish-BootstrapScripts.ps1 ` + -SasToken '${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}' ` + -LegacySasToken '${{ secrets.AZURE_BLOB_AGENT_LEGACY_SAS_TOKEN }}' ` + -TableSasToken '${{ secrets.AZURE_TABLE_ADMIN_SAS_TOKEN }}' ` + -DryRun:$dryRun diff --git a/docs/index.md b/docs/index.md index 8cd0ec19..39f7cdb4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,6 +13,7 @@ https://docs.autopilotmonitor.com (separate repository). * [Architecture Guide](architecture.md) - High-level architecture overview and solution layout for contributors. * [Version Contract](versioning.md) - One version shape for agent, backend and MCP: the ETag-CAS counter blob that mints build numbers, and the manifests published only after a deploy is verified live. +* [Customer Script Publishing](script-publishing.md) - How the customer-facing PowerShell scripts reach download.autopilotmonitor.com/agent/, why publishing is decoupled from agent releases, and the guards keeping the published copy, the version badge and the portal's version oracle in agreement. * [URL Registry](url-registry.md) - Every well-known host lives in one registry file per component (Constants.cs / utils/config.ts / src/config.ts), enforced by guard tests; includes the agent-download migration to the download alias and its deploy-order constraint. * [Drift Guards](drift-guards.md) - The 2026-08 fragility-audit follow-up contracts: SignalR message-name catalog (typed on both ends), the inverted-tick RowKey codec, tombstone table tagging, the shrink-only OkAsync baseline, per-section config PATCH with the section field map, and the web's scoped-endpoint builders + sidebar visibility matrix. diff --git a/docs/log.md b/docs/log.md index fb81983b..b832c904 100644 --- a/docs/log.md +++ b/docs/log.md @@ -1,5 +1,9 @@ # Log +## 2026-08-19 + +* **New**: `script-publishing.md` — the customer script publishing chain. `.github/scripts/Publish-BootstrapScripts.ps1` becomes the single owner (dev render, version-bump guard, uploads, version oracles, alias verification), called both by the new `publish-scripts.yml` (push to `main` on `scripts/Bootstrap/**` and `scripts/CustomerSetup/**`) and by the `publish_as_stable` branch of `build-agent.yml`. Script publishing no longer waits for an agent release. Blobs are published in a deterministic CRLF/UTF-8-no-BOM form so byte comparisons do not depend on the checkout; the Pester + ASCII gates move into the reusable `bootstrap-script-gates.yml` and block the upload; `AdminConfiguration.LatestBootstrapV2ScriptVersion` now has exactly one writer. + ## 2026-08-18 * **New**: `agent/wdp-scenario-gates.md` — WDP scenario gates (afee7ae0 audit): deterministic ExecutionContext marker as the only gate key (fail-toward-Classic), suppressed Autopilot/ESP-only emissions and hosts (autopilot_profile*/ZTD probe, esp_config unknown null-gate, EspPolicyProviderStallHost, ProvisioningStatusTracker, ModernDeployment noise IDs 100/417/418/1005 + error-backfill skip, starved sweep), and the WDP completion model (High-confidence v2 seed via `enrollmentTypeDeterministic`, desktop-based arm D, 30-min `device_prep_completion` backstop, parked-tripwire WDP branch). diff --git a/docs/script-publishing.md b/docs/script-publishing.md new file mode 100644 index 00000000..007c8c24 --- /dev/null +++ b/docs/script-publishing.md @@ -0,0 +1,128 @@ +--- +type: Concept +title: Customer Script Publishing — download.autopilotmonitor.com/agent/ +description: How the customer-facing PowerShell scripts reach the download alias, why publishing is decoupled from agent releases, and the guards that keep the published copy, the version badge and the portal's version oracle in agreement. +resource: /.github/scripts/Publish-BootstrapScripts.ps1 +tags: + - build + - ci + - release + - bootstrap + - download-alias +timestamp: 2026-08-19T00:00:00+02:00 +--- + +# Overview + +Four PowerShell scripts are served to customers from `https://download.autopilotmonitor.com/agent/`: + +| Blob | Source | Consumed by | +| --- | --- | --- | +| `Install-AutopilotMonitor.ps1` | `scripts/Bootstrap/Install-AutopilotMonitor.ps1` | Intune platform script (customer uploads a copy); downloaded live on every device by the WDP bootstrap MSI | +| `Install-AutopilotMonitor-Dev.ps1` | rendered from the same source | dev fleet | +| `Test-ShouldBootstrapAgent.ps1` | `scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1` | customer dry run via `irm ... \| iex` | +| `Grant-AutopilotMonitorAddOn.ps1` | `scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1` | optional Graph permission grant | + +`Uninstall-AutopilotMonitor.ps1` is deliberately not published — nothing links it, and an +unauthenticated uninstall script on the public download host is not a feature. + +The alias is a Front Door route in front of the `agent` container; see +[url-registry.md](url-registry.md) for the host registry and the migration to the alias. + +# Publishing chain + +`.github/scripts/Publish-BootstrapScripts.ps1` is the single owner. Two workflows call it +with identical arguments: + +* **`publish-scripts.yml`** — on every push to `main` touching `scripts/Bootstrap/**`, + `scripts/CustomerSetup/**` or the publisher itself, plus `workflow_dispatch` with a + `dry_run` input. +* **`build-agent.yml`** — inside the `publish_as_stable` branch, after `version.json` is + uploaded, so a release lands a set consistent with what is already live. + +Before this split, the script blobs rotated **only** on a stable agent release. A bootstrap +fix therefore sat unpublished until the next agent cutover — while the WDP MSI downloads the +published copy on every enrolling device. Script publishing is now decoupled from the agent +release cadence; the release path still publishes, so it can never fall behind either. + +Both workflows share the concurrency group `agent-blob-publish` (`cancel-in-progress: false`). +Both write `version.json`, and the publisher's `bootstrapVersion` read-modify-write must not +interleave with a release writing the agent fields. + +# Publisher steps + +1. **Parse** `$ScriptVersion` from the bootstrap source. +2. **Dev render** — literal substitution of the two `$AgentDownloadUrl` / `$VersionJsonName` + defaults. A missing anchor literal is a hard failure: a silently un-substituted dev script + would point the dev fleet at the stable agent. +3. **Version-bump guard** — if the published bootstrap differs from the source but carries the + same `$ScriptVersion`, the run aborts. See [Version oracles](#version-oracles). +4. **Upload** every blob with `Cache-Control: no-cache` (they all rotate in place), mirrored + fail-soft to the legacy storage account. +5. **Reconcile the version oracles.** +6. **Verify through the alias** — re-download each blob and compare SHA-256, six attempts + 20 s apart. A persistent mismatch means blobs were written but Front Door serves stale + content, and the error names the `az afd endpoint purge` command. + +## Deterministic publish form + +Every blob is published as **CRLF, UTF-8 without BOM**, normalized by the publisher rather +than taken from the working copy. Git normalizes line endings on checkout — a Windows runner +yields CRLF, a `core.autocrlf=false` clone yields LF — so publishing raw working-copy bytes +would emit different blobs for identical source and make every byte comparison, guard and +verification alike, depend on where the workflow ran. CRLF is also what PS 5.1 expects on the +device and what has always been published. + +The ASCII-only rule for `scripts/Bootstrap/*.ps1` (enforced by the gates below) is unrelated +but adjacent: PS 5.1 reads BOM-less files as ANSI and corrupts multi-byte characters. + +## Version oracles + +Three places state the bootstrap script version, and all three are derived from +`$ScriptVersion` in the source: + +* `version.json.bootstrapVersion` — read by the docs badge. Written by the publisher as a + read-modify-write under `If-Match`, so a concurrent agent release keeps its own fields. + `build-agent.yml` seeds the field when it builds the manifest so the published manifest is + never briefly missing it; the publisher owns and re-asserts it afterwards. +* `AdminConfiguration.LatestBootstrapV2ScriptVersion` — read by the portal to tell a customer + their uploaded copy is outdated. **Only** the publisher writes it; `build-agent.yml` writes + the `LatestAgentV2*` fields and nothing else. One writer per field. +* The `Bootstrap script version: v` marker the script logs, parsed out of captured platform + script stdout by `utils/bootstrapVersion.ts`. + +The version-bump guard exists because those three would otherwise keep asserting a version +that no longer matches what customers download: changed content under an unchanged version is +invisible to every consumer. + +# Gates + +`.github/workflows/bootstrap-script-gates.yml` is a `workflow_call` workflow holding the +checks once: + +* Pester suite under **Windows PowerShell 5.1** — the runtime IME uses for platform scripts. +* ASCII-only scan over `scripts/Bootstrap/*.ps1`. +* `Publish-BootstrapScripts.ps1 -DryRun` — exercises the dev-render anchors and the + version-bump guard, and reports per blob whether the alias is current, stale or missing. + +Two callers: `pester.yml` on pull requests (reports, does not block the merge button), and +`publish-scripts.yml` on pushes to `main`, where the gates job is a `needs:` dependency of the +publish job and therefore **blocks** the upload. `pester.yml` deliberately has no push trigger +— it would duplicate the gate run with no consequence. + +# Operating it + +* Dry run and drift check, no credentials needed for the read paths: + `pwsh .github/scripts/Publish-BootstrapScripts.ps1 -DryRun` +* Republish without a code change: `workflow_dispatch` on **Publish customer scripts** with + `dry_run=false`. +* Front Door serving stale content after a successful upload: + `az afd endpoint purge --resource-group rg-autopilotmonitor-prd-gwc --profile-name autopilotmonitor-fd --endpoint-name apm-download --content-paths '/agent/*'` + +# Citations + +* `.github/scripts/Publish-BootstrapScripts.ps1` +* `.github/workflows/publish-scripts.yml`, `.github/workflows/bootstrap-script-gates.yml`, `.github/workflows/pester.yml` +* `.github/workflows/build-agent.yml` (the `publish_as_stable` branch of the upload step) +* `src/Agent/AutopilotMonitor.BootstrapMsi/Invoke-BootstrapDownload.ps1` +* `src/Web/autopilot-monitor-web/utils/bootstrapVersion.ts`