mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
423927fdd8
* chore(mutation): shrink the lane to report-only (#1457, #1781 wave 2) The mutation harness's two real catches (#1474, #1475) both came from humans reading the weekly score report. The ratchet half never operated: the baseline was committed exactly twice (8cce0ef6b,60400d04b), both times with `stableRuns: 0, gating: false`, and was never updated after the very fixes it triggered — the weekly job computed a new baseline and then `git checkout --`d it, uploading a proposal nobody applied in 3+ weeks. A gate nobody arms is harness weight; the report is the part that paid. Deletes ratchet.ts + ratchet.test.ts, mutation-baselines/, and every baseline/graduation/gating path in run.ts (`--update`, `mutation:baseline`). run.ts now exits non-zero only on a harness failure, never on a score. The report renders the per-kernel table (kernel, score, killed, survived, total, timeouts) plus the surviving mutants a strengthening PR works from. Kernel scoping stays: stryker.config.json and KERNEL_MODULES are untouched. * fix(mutation): restore denominator coverage and publish the table before judging the shard set Review of #1828: - `report.test.ts` re-asserts that Ignored/CompileError/RuntimeError leave the denominator — the one behaviour `ratchet.test.ts` covered and nothing replaced. A `tally()` edit that counted tool noise would have deflated every published score with a green `mutation:test`. - `assertShardsCoverModules` now runs after `emit()`, so an incomplete shard set still publishes the kernels that completed instead of only an error string. This makes the workflow comments' claim about the job summary true rather than re-wording them down. * chore(mutation): trigger the affected lane on exactly the paths that can select mutants The PR lane returns an empty matrix unless the diff touches the harness, so the kernel-source and `**/*.test.ts` triggers only bought a 1-4 min no-op job on ~96% of PRs. `on.pull_request.paths` is now exactly `LANE_TOOLING` plus the workflow file, asserted in both directions by workflow.test.ts against the exported constant — a missing path would let a harness change merge unproven, an extra one starts a job that can only answer `[]`. Also drops the workflow header's contradictory scope paragraph: it claimed the lane selects on kernel sources and any test reaching one, which has not been true since the ratchet went. * fix(mutation): score and publish a short shard set before failing on the count The expected-count check ran inside readShardedReports, before anything was summarized, so on the weekly's real `--expect-shards 10` one dead shard threw away the nine that had reported — the earlier reorder only moved the zero-mutants check. The merge now returns the shard count, and both verdicts run after emit() with the same exit code and `score` stage. Regression uses the weekly argument shape (`--expect-shards 10`, one shard present) and asserts the reporting kernel's row reaches stdout while the run still fails.
182 lines
6.4 KiB
YAML
182 lines
6.4 KiB
YAML
name: Mutation Affected
|
|
|
|
# PR-side half of the decision-kernel mutation lane (issue #1415).
|
|
#
|
|
# Advisory only — the lane reports and never gates (#1457). `scripts/mutation/run.ts`
|
|
# exits non-zero on a harness failure, never on a score.
|
|
#
|
|
# Cost control: the kernel report that pays is the weekly sweep, so this lane
|
|
# spends mutants on one thing only — proving the harness still runs end to end
|
|
# when the harness itself changes. `select` therefore returns an empty matrix
|
|
# unless the diff touches the lane's own sources. Selecting on derived kernel
|
|
# ownership instead would run the full ten-shard sweep on 24 of the last 40
|
|
# merged PRs.
|
|
#
|
|
# On a harness diff the matrix is the canary (`LANE_CANARY`, the registry's
|
|
# cheapest real sweep — lane sources own no kernel, so derivation alone would
|
|
# select nothing) plus any kernel that same diff reaches through the import
|
|
# graph (scripts/mutation/ownership.ts): a kernel source, or a test that reaches
|
|
# one. Selected modules are sharded like the weekly sweep, so the PR's wall
|
|
# clock is one module rather than their sum.
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
# Exactly `LANE_TOOLING` in scripts/mutation/run.ts plus this file — the
|
|
# only diffs that can produce a non-empty matrix, asserted both ways by
|
|
# scripts/mutation/workflow.test.ts. A kernel or test path here would only
|
|
# start a job that selects nothing.
|
|
- 'scripts/mutation/**'
|
|
- 'scripts/lib/**'
|
|
- 'stryker.config.json'
|
|
- '.github/workflows/mutation-affected.yml'
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: ci-${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
select:
|
|
name: Select affected kernels
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
outputs:
|
|
modules: ${{ steps.select.outputs.result }}
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Setup toolchain
|
|
uses: ./.github/actions/setup-node-pnpm
|
|
|
|
- name: Harness self-test
|
|
uses: ./.github/actions/run-gate
|
|
with: { gate: mutation-model }
|
|
|
|
- id: select
|
|
name: Derive the affected shard matrix
|
|
uses: ./.github/actions/run-gate
|
|
with:
|
|
gate: mutation-affected
|
|
args: |
|
|
--list-affected
|
|
--base
|
|
origin/${{ github.event.pull_request.base.ref }}
|
|
|
|
# The self-test and the derivation run before any mutant, so a failure here
|
|
# would leave the lane with no envelope at all (#1430).
|
|
- name: Record a failed lane envelope
|
|
if: failure()
|
|
run: |
|
|
pnpm gate mutation --affected --fail-envelope \
|
|
"affected selection failed before any mutant ran (run ${{ github.run_id }})" || true
|
|
|
|
- name: Upload selection envelope
|
|
if: failure()
|
|
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
with:
|
|
name: mutation-affected-select
|
|
path: .tmp/mutation/lane-envelope.json
|
|
if-no-files-found: warn
|
|
|
|
mutants:
|
|
name: Mutants (${{ matrix.name }})
|
|
needs: select
|
|
if: needs.select.outputs.modules != '[]'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include: ${{ fromJSON(needs.select.outputs.modules) }}
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Setup toolchain
|
|
uses: ./.github/actions/setup-node-pnpm
|
|
|
|
- name: Run mutants for ${{ matrix.name }}
|
|
uses: ./.github/actions/run-gate
|
|
with:
|
|
gate: mutation
|
|
args: |
|
|
--modules
|
|
${{ matrix.module }}
|
|
${{ matrix.shard && '--shard' || '' }}
|
|
${{ matrix.shard || '' }}
|
|
|
|
- name: Record a failed shard envelope
|
|
if: failure()
|
|
run: |
|
|
pnpm gate mutation --affected --modules ${{ matrix.module }} --fail-envelope \
|
|
"shard ${{ matrix.name }} failed before producing a report (run ${{ github.run_id }})" || true
|
|
|
|
- name: Upload shard report
|
|
if: always()
|
|
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
with:
|
|
name: mutation-affected-shard-${{ matrix.name }}
|
|
path: |
|
|
.tmp/mutation/mutation.json
|
|
.tmp/mutation/mutation.html
|
|
.tmp/mutation/lane-envelope.json
|
|
if-no-files-found: warn
|
|
|
|
report:
|
|
name: Affected decision-kernel mutants
|
|
needs: [select, mutants]
|
|
if: always() && needs.select.result == 'success'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Setup toolchain
|
|
uses: ./.github/actions/setup-node-pnpm
|
|
|
|
- name: Download shard reports
|
|
if: needs.select.outputs.modules != '[]'
|
|
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
|
with:
|
|
pattern: mutation-affected-shard-*
|
|
path: .tmp/mutation/shards
|
|
|
|
# No shards means no affected kernel: run.ts reports "nothing to mutate"
|
|
# and still writes the envelope, so the lane is never silently absent.
|
|
# Either invocation writes the score table to $GITHUB_STEP_SUMMARY.
|
|
- name: Score the affected modules
|
|
run: |
|
|
if [ -d .tmp/mutation/shards ]; then
|
|
expected=$(echo '${{ needs.select.outputs.modules }}' | jq length)
|
|
pnpm gate mutation-check --report-dir .tmp/mutation/shards --affected \
|
|
--expect-shards "$expected" \
|
|
--base "origin/${{ github.event.pull_request.base.ref }}"
|
|
else
|
|
pnpm gate mutation-affected --base "origin/${{ github.event.pull_request.base.ref }}"
|
|
fi
|
|
|
|
- name: Record a failed lane envelope
|
|
if: failure()
|
|
run: |
|
|
pnpm gate mutation --affected --fail-envelope \
|
|
"affected report failed before producing a score table (run ${{ github.run_id }})" || true
|
|
|
|
- name: Upload mutation report
|
|
if: always()
|
|
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
with:
|
|
name: mutation-affected
|
|
path: |
|
|
.tmp/mutation/shards
|
|
.tmp/mutation/lane-envelope.json
|
|
if-no-files-found: ignore
|