Files
Jamie Tanna d91e6088ff fixup! test: set up mutation testing with Stryker
The `mutation-pr`/`mutation-full` CI jobs were failing outright, and a
full `stryker run` never got past the initial dry run, so none of this
was actually usable yet.

- `mutation.yml`: pin all actions to a commit SHA, add a `permissions`
  block, set `persist-credentials: false`, and pass the base ref/changed
  files through `env` instead of interpolating them into the shell
  script directly (zizmor was flagging all of these). Also switch to the
  project's own `./.github/actions/setup-node` composite action instead
  of `pnpm/action-setup`/`actions/setup-node` (which additionally
  referenced a non-existent `.node-version` file), and fix the `pnpm
  mutation`/`pnpm stryker run --mutate` calls to use the actual
  `test:mutation`/`test:mutation:changed` script names.
- Drop the now-unused `@stryker-mutator/typescript-checker` dependency,
  left over from when `stryker.config.json` still had `checkers:
  ["typescript"]`.
- `lib/util/timestamp.ts`: the local-timezone-offset fallback used in
  `asTimestamp()` was cached in a module-level constant computed once at
  import time. Combined with vitest's `threads` pool (used by Stryker's
  vitest-runner) not reliably honouring a `process.env.TZ` set from our
  own `test/setup.ts`, this made `asTimestamp()` parse plain dates (e.g.
  Jenkins' `buildDate: 'Jan 02, 2020'`) using the host's real timezone
  instead of UTC. Compute the offset per call instead, and set `TZ=UTC`
  in the actual environment via the `test:mutation*` scripts, since that's
  the one thing that reliably works across pools.
- `test/setup.ts`: restore real timers in a top-level `afterAll`, so a
  spec that leaves fake timers running (there's at least one) can't leak
  a frozen clock into whichever spec runs next in the same worker -
  something that's only reachable under low-concurrency/single-threaded
  execution.
- Raise `dryRunTimeoutMinutes` and the global vitest `testTimeout`: our
  full suite's per-test coverage instrumentation run took ~25 minutes
  end-to-end, well past Stryker's 5 minute default, and a couple of
  CPU-heavy specs (WASM PGP crypto) timed out under that contention too.
- Ignore `.stryker-tmp/` (Stryker's sandbox dir, which can survive an
  interrupted run) in ls-lint/oxlint/biome/prettier/vitest, since a
  leftover sandbox copy of the repo was otherwise picked up as extra
  source/test files and broke those tools outright.

Co-Authored-By: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
2026-09-10 15:38:01 +01:00

111 lines
4.1 KiB
YAML

name: Mutation Testing
on:
schedule:
- cron: '0 3 * * 0' # Weekly on Sundays at 3am
workflow_dispatch:
pull_request:
types: [opened, synchronize]
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
NODE_VERSION: 24
jobs:
mutation-full:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
continue-on-error: true
steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
show-progress: false
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
node-version: ${{ env.NODE_VERSION }}
os: ${{ runner.os }}
- run: pnpm test:mutation
- name: Upload mutation report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: always()
with:
name: mutation-report
path: reports/mutation/
mutation-pr:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
continue-on-error: true
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
show-progress: false
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
node-version: ${{ env.NODE_VERSION }}
os: ${{ runner.os }}
- name: Get changed source files
id: changed
env:
BASE_REF: ${{ github.base_ref }}
run: |
FILES=$(git diff --name-only "origin/$BASE_REF...HEAD" -- 'lib/**/*.ts' ':!lib/**/*.spec.ts' ':!lib/**/*.generated.ts' | tr '\n' ',')
echo "files=$FILES" >> "$GITHUB_OUTPUT"
echo "has_files=$( [ -n "$FILES" ] && echo true || echo false )" >> "$GITHUB_OUTPUT"
- name: Run mutation tests on changed files
if: steps.changed.outputs.has_files == 'true'
env:
CHANGED_FILES: ${{ steps.changed.outputs.files }}
run: pnpm test:mutation:changed "$CHANGED_FILES"
- name: Post PR comment
if: always() && steps.changed.outputs.has_files == 'true'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const fs = require('fs');
let body = '## 🧬 Mutation Testing Results\n\n';
try {
const report = JSON.parse(fs.readFileSync('reports/mutation/mutation.json', 'utf8'));
const score = report.schemaVersion ?
(report.files ? Object.values(report.files).reduce((acc, f) => acc + f.mutationScore, 0) / Object.keys(report.files).length : 0) : 0;
body += `**Mutation Score:** ${score.toFixed(1)}%\n\n`;
body += '| File | Score | Survived | Killed | Total |\n|------|-------|----------|--------|-------|\n';
for (const [file, data] of Object.entries(report.files || {})) {
body += `| ${file} | ${data.mutationScore.toFixed(1)}% | ${data.survived || 0} | ${data.killed || 0} | ${data.totalMutants || 0} |\n`;
}
} catch (e) {
body += '_No mutation report generated (possibly no mutants in changed files)._\n';
}
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number
});
const existing = comments.find(c => c.body.includes('🧬 Mutation Testing Results'));
if (existing) {
await github.rest.issues.updateComment({ owner: context.repo.owner, repo: context.repo.repo, comment_id: existing.id, body });
} else {
await github.rest.issues.createComment({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, body });
}