Files
Michał Pierzchała 423927fdd8 chore(mutation): shrink to report-only — drop the ratchet, baseline and graduation (#1457, #1781) (#1828)
* chore(mutation): shrink the lane to report-only (#1457, #1781 wave 2)

The mutation harness's two real catches (#1474, #1475) both came from humans
reading the weekly score report. The ratchet half never operated: the baseline
was committed exactly twice (8cce0ef6b, 60400d04b), both times with
`stableRuns: 0, gating: false`, and was never updated after the very fixes it
triggered — the weekly job computed a new baseline and then `git checkout --`d
it, uploading a proposal nobody applied in 3+ weeks. A gate nobody arms is
harness weight; the report is the part that paid.

Deletes ratchet.ts + ratchet.test.ts, mutation-baselines/, and every
baseline/graduation/gating path in run.ts (`--update`, `mutation:baseline`).
run.ts now exits non-zero only on a harness failure, never on a score. The
report renders the per-kernel table (kernel, score, killed, survived, total,
timeouts) plus the surviving mutants a strengthening PR works from.

Kernel scoping stays: stryker.config.json and KERNEL_MODULES are untouched.

* fix(mutation): restore denominator coverage and publish the table before judging the shard set

Review of #1828:
- `report.test.ts` re-asserts that Ignored/CompileError/RuntimeError leave the
  denominator — the one behaviour `ratchet.test.ts` covered and nothing replaced.
  A `tally()` edit that counted tool noise would have deflated every published
  score with a green `mutation:test`.
- `assertShardsCoverModules` now runs after `emit()`, so an incomplete shard set
  still publishes the kernels that completed instead of only an error string.
  This makes the workflow comments' claim about the job summary true rather than
  re-wording them down.

* chore(mutation): trigger the affected lane on exactly the paths that can select mutants

The PR lane returns an empty matrix unless the diff touches the harness, so the
kernel-source and `**/*.test.ts` triggers only bought a 1-4 min no-op job on
~96% of PRs. `on.pull_request.paths` is now exactly `LANE_TOOLING` plus the
workflow file, asserted in both directions by workflow.test.ts against the
exported constant — a missing path would let a harness change merge unproven,
an extra one starts a job that can only answer `[]`.

Also drops the workflow header's contradictory scope paragraph: it claimed the
lane selects on kernel sources and any test reaching one, which has not been
true since the ratchet went.

* fix(mutation): score and publish a short shard set before failing on the count

The expected-count check ran inside readShardedReports, before anything was
summarized, so on the weekly's real `--expect-shards 10` one dead shard threw
away the nine that had reported — the earlier reorder only moved the
zero-mutants check. The merge now returns the shard count, and both verdicts
run after emit() with the same exit code and `score` stage.

Regression uses the weekly argument shape (`--expect-shards 10`, one shard
present) and asserts the reporting kernel's row reaches stdout while the run
still fails.
2026-08-18 17:47:29 +02:00

174 lines
6.6 KiB
TypeScript

// Which kernel module a changed file belongs to, DERIVED — never hand-listed.
//
// A mutation score is a statement about the tests that kill the mutants, so a
// selection is only honest if it follows *those* tests. An enumerated list of
// test files cannot state that: it silently omits tests that
// exercise a kernel indirectly (`src/__tests__/daemon-error.test.ts` reaches
// `normalizeError` through `src/daemon.ts`), and nothing fails when a new test
// is added. So ownership is computed from the static import graph instead: a test
// file owns every kernel module whose mutated sources it can reach.
//
// The derivation is deliberately a superset — reaching a kernel is cheaper to
// prove than killing its mutants, so an unrelated diff can select a module and
// pay for a report. False positives cost runner minutes; a false negative would
// leave a kernel whose tests changed unmeasured.
//
// Non-test source changes outside the registry are NOT owned: they can only move
// a score through the tests that reach the kernel, and the weekly full sweep is
// what re-measures the whole surface. The derived claim is narrower on purpose —
// kernel sources plus the tests that exercise them.
import fs from 'node:fs';
import path from 'node:path';
import {
readWorkspacePackages,
workspaceSpecifierTargets,
} from '../layering/package-boundaries.ts';
import { walkFiles } from '../lib/walk-files.ts';
import {
affectedModules,
isKernelTestFile,
KERNEL_MODULES,
normalizePath,
type ModuleId,
type KernelModule,
} from './modules.ts';
import { expandMutateFiles } from './test-scope.ts';
/** Test files the mutation lane can attribute to a kernel at all. */
export function isTestFile(filePath: string): boolean {
return isKernelTestFile(filePath);
}
/**
* Workspace package specifiers resolved through the shared exports-map reader
* (scripts/layering/package-boundaries.ts) — never hand-listed. Without this
* the graph walk stops at every `@agent-device/*` edge and a kernel living in
* `packages/` silently loses all of its owned tests.
*/
let cachedExportTargets: { repoRoot: string; targets: Map<string, string> } | undefined;
function exportTargetsFor(repoRoot: string): Map<string, string> {
if (cachedExportTargets?.repoRoot !== repoRoot) {
cachedExportTargets = { repoRoot, targets: workspaceSpecifierTargets(repoRoot) };
}
return cachedExportTargets.targets;
}
/**
* Repository-relative modules a file imports: relative specifiers plus
* workspace package specifiers resolved through their `exports` maps.
*/
function importsOf(file: string, repoRoot: string, cache: Map<string, string[]>): string[] {
const cached = cache.get(file);
if (cached) return cached;
const absolute = path.join(repoRoot, file);
const text = fs.existsSync(absolute) ? fs.readFileSync(absolute, 'utf8') : '';
const specifiers = [...text.matchAll(/(?:from|import)\s*\(?\s*'(?<spec>[.@][^']+)'/g)].map(
(match) => match.groups!.spec,
);
const exportTargets = exportTargetsFor(repoRoot);
const resolved = [
...new Set(
specifiers.flatMap((specifier) => {
if (specifier.startsWith('@')) {
const target = exportTargets.get(specifier);
return target && fs.existsSync(path.join(repoRoot, target)) ? [target] : [];
}
const base = path.posix.normalize(path.posix.join(path.posix.dirname(file), specifier));
return [base, `${base}.ts`, `${base}/index.ts`].filter((candidate) =>
fs.existsSync(path.join(repoRoot, candidate)),
);
}),
),
].filter((candidate) => candidate.endsWith('.ts'));
cache.set(file, resolved);
return resolved;
}
/** Every repository-relative module `file` reaches through the import graph. */
export function reachableFrom(
file: string,
repoRoot: string,
cache: Map<string, string[]> = new Map(),
): Set<string> {
const seen = new Set<string>();
const queue = [normalizePath(file)];
while (queue.length > 0) {
const current = queue.shift()!;
if (seen.has(current)) continue;
seen.add(current);
queue.push(...importsOf(current, repoRoot, cache));
}
return seen;
}
/** The concrete sources Stryker mutates for a module. */
export function mutatedSources(module: KernelModule, repoRoot: string): string[] {
return expandMutateFiles(module.mutate, repoRoot);
}
type Deriver = {
/** Kernel modules a single test file exercises, in registry order. */
ownersOf: (testFile: string) => ModuleId[];
};
/** A deriver with caches shared across files — one graph walk per module, not per query. */
export function ownershipDeriver(repoRoot: string): Deriver {
const importCache = new Map<string, string[]>();
const sources = KERNEL_MODULES.map((module) => ({
id: module.id,
sources: new Set(mutatedSources(module, repoRoot)),
}));
return {
ownersOf(testFile) {
const reachable = reachableFrom(testFile, repoRoot, importCache);
return sources
.filter((entry) => [...entry.sources].some((source) => reachable.has(source)))
.map((entry) => entry.id);
},
};
}
/**
* Kernel modules a diff affects: registry-owned paths plus every module the
* changed tests reach. Registry order, deduplicated.
*/
export function derivedAffectedModules(
changedFiles: readonly string[],
repoRoot: string,
): ModuleId[] {
const ids = new Set<ModuleId>(affectedModules(changedFiles));
const tests = changedFiles.filter(isTestFile).map(normalizePath);
if (tests.length > 0) {
const deriver = ownershipDeriver(repoRoot);
for (const testFile of tests) {
for (const id of deriver.ownersOf(testFile)) ids.add(id);
}
}
return KERNEL_MODULES.filter((module) => ids.has(module.id)).map((module) => module.id);
}
/**
* Every test file in the repository, per module that owns it — one graph walk
* over root `src/` plus every workspace package's `src/`, so a kernel tested
* only from inside its own package (`target-annotation-serde`) is not
* silently unownable.
*/
export function ownedTestFiles(repoRoot: string): Map<ModuleId, string[]> {
const deriver = ownershipDeriver(repoRoot);
const owned = new Map<ModuleId, string[]>(KERNEL_MODULES.map((module) => [module.id, []]));
const testRoots = [
path.join(repoRoot, 'src'),
...readWorkspacePackages(repoRoot).map((pkg) => path.join(repoRoot, pkg.dir, 'src')),
];
for (const root of testRoots) {
for (const file of walkFiles(root, (file) => file.endsWith('.test.ts'))) {
const relative = normalizePath(path.relative(repoRoot, file));
for (const id of deriver.ownersOf(relative)) owned.get(id)!.push(relative);
}
}
for (const files of owned.values()) files.sort();
return owned;
}