mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 20:31:30 +00:00
shankar0123
2643a427ac
CI run #376 (commit a1c7741, Frontend Build job) failed with:
digest does not resolve: mcr.microsoft.com/windows/servercore/iis:
windowsservercore-ltsc2022@sha256:8d0b0e651ad514e3fb05978db66f38036
118812e1b9314a48f10419cad8a3462
A re-run with no code changes went green. The digest itself is fine —
verified against MCR directly (HTTP 200 from
mcr.microsoft.com/v2/windows/servercore/iis/manifests/sha256:8d0b...),
and the tag `:windowsservercore-ltsc2022` currently resolves to that
exact digest. Microsoft hasn't rotated.
Root cause is registry-side rate-limiting. MCR throttles unauthenticated
GET-by-digest requests by source IP. GitHub-hosted runners share a small
pool of egress IPs across many users; bursts trip the throttle and
return non-200. Re-run = different runner = different IP = throttle
window has reset = pass. This will recur on roughly N% of pushes
indefinitely, until either (a) Microsoft loosens MCR rate limits, (b)
GitHub buys more runner IPs, or (c) we stop verifying digests CI doesn't
actually use.
The deeper issue is structural, not transient. The Windows IIS image is
gated behind compose `profiles: [deploy-e2e-windows]`
(deploy/docker-compose.test.yml:700). The comment block above the
service definition (lines 675-691) explicitly says "Linux CI never
activates this profile." All 10 TestVendorEdge_IIS_*_E2E tests are on
scripts/vendor-e2e-skip-allowlist.txt because the sidecar is never
started. The whole Windows matrix was DELETED in ci-pipeline-cleanup
Phase 6 / frozen decision 0.5 (revising Bundle II decision 0.4); IIS
validation moved to docs/connector-iis.md::Operator validation playbook.
So `digest-validity.sh` is verifying a digest that no CI job ever pulls
— paying CI brittleness against MCR rate-limiting we can't control, for
an image whose only purpose in compose is documentation for an
operator's manual workflow on a real Windows host.
The fix matches the guard's stated purpose ("every digest CI actually
depends on is valid"): exclude images CI never pulls.
Implementation. Add an EXCLUDED_PATTERNS array near the top of the
script with one entry — the IIS image path
`mcr.microsoft.com/windows/servercore/iis` — and a comment block above
it documenting:
- WHY it's excluded (gated profile, never started, all tests on
skip-allowlist)
- WHEN it would need re-inclusion (if a Windows CI runner is added
that actually starts the sidecar)
- WHAT this list is NOT for (transient flake silencing — that gets
fixed via retry logic in the script, not via exclusion)
The match is by image-path substring, not by digest, so future tag/
digest updates of the same image still hit the exclusion without
needing this list to be re-edited.
Loop logic gains a 6-line check that runs the exclusion match before
any registry work. Excluded refs log as "SKIP (excluded) <ref>" so
operator-facing CI logs stay informative — at a glance you can see
which digests were verified vs which were intentionally not.
The success message updates to differentiate verified vs excluded
counts: "digest-validity: clean — N verified, M excluded (CI never
pulls)" when M > 0; original message preserved when M == 0.
Verified manually:
- Clean repo: 15 verified, 1 excluded, exit 0.
- Fabricated bogus httpd digest: ::error:: emitted for the bad
digest, IIS still SKIP-excluded, exit 1. (Real regressions still
caught.)
- Restore: 15 verified, 1 excluded, exit 0 again.
Other recurring MCR-hosted images would warrant the same treatment if
they get added later. The exclusion list pattern scales: each new entry
needs its own "WHY this is doc-only" justification block.
What this is NOT:
- Not a generic flake-silencer. The exclusion is justified by the
image being doc-only, not by the test being noisy.
- Not a global retry/resilience layer. If MCR rate-limits an image CI
DOES pull, that's a real CI dependency on an unreliable external
service — fix by retry-with-backoff, not by excluding.