mirror of
https://github.com/renovatebot/renovate.git
synced 2026-11-02 10:33:39 +00:00
d91e6088ff
The `mutation-pr`/`mutation-full` CI jobs were failing outright, and a full `stryker run` never got past the initial dry run, so none of this was actually usable yet. - `mutation.yml`: pin all actions to a commit SHA, add a `permissions` block, set `persist-credentials: false`, and pass the base ref/changed files through `env` instead of interpolating them into the shell script directly (zizmor was flagging all of these). Also switch to the project's own `./.github/actions/setup-node` composite action instead of `pnpm/action-setup`/`actions/setup-node` (which additionally referenced a non-existent `.node-version` file), and fix the `pnpm mutation`/`pnpm stryker run --mutate` calls to use the actual `test:mutation`/`test:mutation:changed` script names. - Drop the now-unused `@stryker-mutator/typescript-checker` dependency, left over from when `stryker.config.json` still had `checkers: ["typescript"]`. - `lib/util/timestamp.ts`: the local-timezone-offset fallback used in `asTimestamp()` was cached in a module-level constant computed once at import time. Combined with vitest's `threads` pool (used by Stryker's vitest-runner) not reliably honouring a `process.env.TZ` set from our own `test/setup.ts`, this made `asTimestamp()` parse plain dates (e.g. Jenkins' `buildDate: 'Jan 02, 2020'`) using the host's real timezone instead of UTC. Compute the offset per call instead, and set `TZ=UTC` in the actual environment via the `test:mutation*` scripts, since that's the one thing that reliably works across pools. - `test/setup.ts`: restore real timers in a top-level `afterAll`, so a spec that leaves fake timers running (there's at least one) can't leak a frozen clock into whichever spec runs next in the same worker - something that's only reachable under low-concurrency/single-threaded execution. - Raise `dryRunTimeoutMinutes` and the global vitest `testTimeout`: our full suite's per-test coverage instrumentation run took ~25 minutes end-to-end, well past Stryker's 5 minute default, and a couple of CPU-heavy specs (WASM PGP crypto) timed out under that contention too. - Ignore `.stryker-tmp/` (Stryker's sandbox dir, which can survive an interrupted run) in ls-lint/oxlint/biome/prettier/vitest, since a leftover sandbox copy of the repo was otherwise picked up as extra source/test files and broke those tools outright. Co-Authored-By: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
111 lines
4.1 KiB
YAML
111 lines
4.1 KiB
YAML
name: Mutation Testing
|
|
|
|
on:
|
|
schedule:
|
|
- cron: '0 3 * * 0' # Weekly on Sundays at 3am
|
|
workflow_dispatch:
|
|
pull_request:
|
|
types: [opened, synchronize]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
NODE_VERSION: 24
|
|
|
|
jobs:
|
|
mutation-full:
|
|
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
|
runs-on: ubuntu-latest
|
|
continue-on-error: true
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
persist-credentials: false
|
|
show-progress: false
|
|
|
|
- name: Setup Node.js
|
|
uses: ./.github/actions/setup-node
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
os: ${{ runner.os }}
|
|
|
|
- run: pnpm test:mutation
|
|
|
|
- name: Upload mutation report
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
if: always()
|
|
with:
|
|
name: mutation-report
|
|
path: reports/mutation/
|
|
|
|
mutation-pr:
|
|
if: github.event_name == 'pull_request'
|
|
runs-on: ubuntu-latest
|
|
continue-on-error: true
|
|
permissions:
|
|
contents: read
|
|
pull-requests: write
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
show-progress: false
|
|
|
|
- name: Setup Node.js
|
|
uses: ./.github/actions/setup-node
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
os: ${{ runner.os }}
|
|
|
|
- name: Get changed source files
|
|
id: changed
|
|
env:
|
|
BASE_REF: ${{ github.base_ref }}
|
|
run: |
|
|
FILES=$(git diff --name-only "origin/$BASE_REF...HEAD" -- 'lib/**/*.ts' ':!lib/**/*.spec.ts' ':!lib/**/*.generated.ts' | tr '\n' ',')
|
|
echo "files=$FILES" >> "$GITHUB_OUTPUT"
|
|
echo "has_files=$( [ -n "$FILES" ] && echo true || echo false )" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Run mutation tests on changed files
|
|
if: steps.changed.outputs.has_files == 'true'
|
|
env:
|
|
CHANGED_FILES: ${{ steps.changed.outputs.files }}
|
|
run: pnpm test:mutation:changed "$CHANGED_FILES"
|
|
|
|
- name: Post PR comment
|
|
if: always() && steps.changed.outputs.has_files == 'true'
|
|
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
|
with:
|
|
script: |
|
|
const fs = require('fs');
|
|
let body = '## 🧬 Mutation Testing Results\n\n';
|
|
try {
|
|
const report = JSON.parse(fs.readFileSync('reports/mutation/mutation.json', 'utf8'));
|
|
const score = report.schemaVersion ?
|
|
(report.files ? Object.values(report.files).reduce((acc, f) => acc + f.mutationScore, 0) / Object.keys(report.files).length : 0) : 0;
|
|
body += `**Mutation Score:** ${score.toFixed(1)}%\n\n`;
|
|
body += '| File | Score | Survived | Killed | Total |\n|------|-------|----------|--------|-------|\n';
|
|
for (const [file, data] of Object.entries(report.files || {})) {
|
|
body += `| ${file} | ${data.mutationScore.toFixed(1)}% | ${data.survived || 0} | ${data.killed || 0} | ${data.totalMutants || 0} |\n`;
|
|
}
|
|
} catch (e) {
|
|
body += '_No mutation report generated (possibly no mutants in changed files)._\n';
|
|
}
|
|
const { data: comments } = await github.rest.issues.listComments({
|
|
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number
|
|
});
|
|
const existing = comments.find(c => c.body.includes('🧬 Mutation Testing Results'));
|
|
if (existing) {
|
|
await github.rest.issues.updateComment({ owner: context.repo.owner, repo: context.repo.repo, comment_id: existing.id, body });
|
|
} else {
|
|
await github.rest.issues.createComment({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, body });
|
|
}
|