mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
9c22467832
* test(ci): prove every registered gate is owned and reachable (#1429) A check that silently stops running looks exactly like a green build. Two suites had already stopped: `check:tmpdir-leaks` (with its model tests) and `test:fixture-cache` are real package scripts that no workflow ran, reachable only through the `check:unit` aggregate CI never invokes. `CHECK_CATALOG` becomes the registry of every check and `pnpm gate <id>` the only way CI runs one, so finding what a lane runs is a scan for `pnpm gate` rather than an attempt to interpret shell. `pnpm check:gate-manifest` then asserts against the real workflows that every registered check is run by some qualifying lane (per unit, not per script name), that every check the real selector activates for a path is run by a lane that path would start (#1420's class), and that every Vitest project and suite script belongs to a check. The wiring that keeps those honest is asserted too: a gate id must name a registered check, an `if:` must be ruled on in GATE_CONDITIONS so `if: false` unowns what it guards, an action declared to run a gate is proven to, and a job whose steps the loader cannot open fails closed. It deliberately does not try to prove CI runs project code only through `pnpm gate`. Whether a shell block executes project code is not decidable from its text, so shell this model does not recognise earns no ownership credit — the failure direction is a check reported unowned, never one waved through. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * test(ci): update the two suites that assert on rewired workflow text `scripts/mutation/workflow.test.ts` and `test/ci/trusted-fixture-artifact.test.mjs` read the workflow and action files and assert on their command text, so routing those steps through `pnpm gate <id>` moved what they were matching. They are the two suites the manifest cannot help with: it proves a gate is still run, not that a test asserting on how CI spells a command was updated with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * fix(ci): credit gates by execution shape, and keep every guard Three ways the manifest could report a gate as owned when it does not run. 1. Crediting was a substring scan over `run:`, which #1429 explicitly rules out — "do not infer reachability from a command name merely appearing in workflow text". `false && pnpm gate x`, a gate inside `if false; then … fi`, one named in a heredoc, and `echo pnpm gate x` all credited it. There is a live instance: conformance-regenerate.yml's "Fail if regeneration changed anything" step names `pnpm gate maestro-regenerate` inside an error message telling a human to run it, and that credited the gate. A gate now counts only as the first command segment of a line, and a body carrying shell structure earns nothing. Reachability inside a script is not decidable, so this does not try: unrecognised shape means no credit and the check reports unowned. `VAR=$(pnpm gate x …)` is read, since the assignment form is unambiguous and the gate runs. 2. Job-level `if:` was not modelled at all, though six live jobs carry one, so a job that cannot run still credited every gate inside it. Two conditions on the mutation lanes are now declared. 3. A caller's `if:` REPLACED the guard on a nested composite-action step (`guard[0] ?? step.condition`), so an outer `always()` erased an inner `if: false`. Steps carry every guard between the lane and the step. Also corrects two source comments that still claimed project code run outside the runner fails the manifest. It does not: such a step earns no credit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * ci: add the run-gate action that names a gate structurally The seam the ownership proof will read instead of shell. A lane says which gate it runs in `with.gate`, a typed input the manifest reads straight out of the YAML and validates against CHECK_CATALOG. Nothing here is wired yet — the ~60 call sites and the model change follow. Added first so the target of that conversion is reviewable on its own. `args` cannot select which gate runs; it is appended after the id, so the worst a wrong value does is fail the gate it already named. There is no `|| true` and no output capture: the gate's exit code is the step's exit code, so a gate cannot run without being able to fail its lane. Part of #1429. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * merge: main (#1770) and route its three new steps through the runner #1770 landed the orphan-check fix on main, wiring `check:tmpdir-leaks`, `check:tmpdir-leaks:test` and `test:fixture-cache` into Coverage, Layering Guard and Integration Tests. This branch had wired the same three through `pnpm gate`, so the merge produced two steps per check rather than a conflict — each check ran twice. Kept main's steps, with the placement and reasoning reviewed on #1770, and changed only their `run:` line to the canonical runner. Dropped this branch's duplicates. Net effect on CI is unchanged: the same three checks, in the same three lanes, once each. Gate manifest green after the merge: 47 checks wired across 33 lanes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * fix(ci): address review — suite detection, freerange, glob, vacuous skip-list Six review findings plus the mutation blocker. [bug] `registered` was shape-only, so a `test:*` script running `node src/bin.ts test <dir>` resolved to a `script:` leaf and was invisible. Four `test:replay:*` scripts were owned only because someone hand-registered them; `test:replay:android` was neither registered nor reported while the nightly ran the same six .ad files by inlining them. A `test:*` script is now a suite by name. `replay-android` is registered, and the nightly runs the script instead of re-listing its files so the two cannot drift. The nightly invokes it inside `reactivecircus/android-emulator-runner`'s `script:` input — shell handed to a third-party action this loader does not read — so the suite executes but cannot be credited. Recorded in UNPROVABLE_OWNERS with that exact reason rather than assumed. The fixed detector also found a second orphan the review did not name: `test:integration:progress`. That one is a reporter whose `--check` sibling is the registered gate, so it is declared in REPORTING_SCRIPTS — a declaration that itself fails when inert. [bug] `freerange` defaulted to localRunnable, so fail-open ran `fr` (a Bun binary) on the pre-push path. Now false. [suggestion] The `--run` skip-list asserted `build:android-snapshot-helper`, a name `android-helpers` no longer uses, so it could not fail. Derived from the catalog instead. [suggestion] `matchesGlob` joined `**` splits with `.*`, making the adjacent slash mandatory — GitHub's `**` matches zero directories, so `src/**/*.test.ts` did not match `src/a.test.ts`. Pinned against `packages/*/src/**/*.test.ts`. [suggestion] Deleted the unwired `run-gate` action. It had no callers, was absent from GATE_ACTIONS, and its comment described a system that had not shipped. It returns with the rewiring, not before. [suggestion] Collapsed the module headers that narrated discarded designs. Mutation: `daemon entrypoint publishes HTTP metadata and cleans up on shutdown` is the only test here that spawns a real daemon process. It takes ~1.1s alone but exceeds Vitest's 5s default inside Stryker's dry run, which aborts the sweep before a single mutant runs. Given 30s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * fix(mutation): order sandbox aliases longest-first so subpaths resolve Every shard of the mutation sweep aborted in Stryker's dry run with: Cannot find package '@agent-device/selectors/engine' imported from .tmp/stryker/sandbox-*/src/core/selector-pipeline.ts The alias was generated correctly; it just never won. Vite matches a STRING alias by prefix and takes the first hit, and `workspaceSpecifierTargets` emitted the bare `@agent-device/selectors` ahead of the subpath entries. The bare entry therefore captured `@agent-device/selectors/engine` and rewrote it to `…/src/index.ts/engine`, which does not exist; Node fell back to real package resolution, could not find the subpath inside the sandbox, and the dry run failed before a single mutant ran — so the shard uploaded an empty envelope instead of a report and the ratchet failed for want of one. Sorting longest specifier first makes the most specific alias win: @agent-device/selectors/engine -> packages/selectors/src/engine.ts @agent-device/selectors/ast -> packages/selectors/src/ast.ts @agent-device/selectors -> packages/selectors/src/index.ts `/ast` never tripped this because nothing in a related test set imported it; `selector-pipeline.ts` introduced the first subpath import that mattered (#1744), so the mutation lane has been unable to run since that landed. Any PR touching `scripts/mutation/**` — which fails open into the full sweep — would have hit it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ * refactor: derive gate ownership from workflow structure * fix: run gates without optional arguments * fix: resolve mutation workspace subpaths exactly --------- Co-authored-by: Claude <noreply@anthropic.com>
291 lines
10 KiB
TypeScript
291 lines
10 KiB
TypeScript
// Entry point for `pnpm check:affected --base <ref>`.
|
|
//
|
|
// Derives the affected local check set from the diff against <ref>, prints a
|
|
// stable machine-readable plan (with per-check reasoning), and optionally runs
|
|
// the locally-runnable checks. Fails open to the full set on anything it cannot
|
|
// classify. Existing GitHub CI stays authoritative.
|
|
|
|
import fs from 'node:fs';
|
|
import path from 'node:path';
|
|
import { pathToFileURL } from 'node:url';
|
|
import { runCmdStreaming, runCmdSync } from '../../src/utils/exec.ts';
|
|
import { parseScriptArgs } from '../lib/cli-args.ts';
|
|
import { runEntrypoint } from '../lib/cli-entrypoint.ts';
|
|
import { DEFAULT_VITEST_MAX_WORKERS } from '../lib/vitest-concurrency.ts';
|
|
import {
|
|
assertCatalogComplete,
|
|
CHECK_CATALOG,
|
|
getCheckSpec,
|
|
resolveCommand,
|
|
type CheckSpec,
|
|
} from './checks.ts';
|
|
import { loadModel, owningLanes } from '../gate/model.ts';
|
|
import { ALL_CHECKS, selectChecks, type CheckId, type CheckPlan } from './model.ts';
|
|
|
|
// Which GitHub jobs run each check, read off the workflows rather than declared
|
|
// next to the check. A skipped check tells the reader where it is authoritative,
|
|
// and that pointer is only useful if it cannot drift from the workflows.
|
|
function ciJobsByCheck(): Map<CheckId, string[]> {
|
|
return owningLanes(loadModel(repoRoot, []));
|
|
}
|
|
|
|
type Args = { base: string; head: string; json: boolean; run: boolean };
|
|
|
|
const repoRoot = runCmdSync('git', ['rev-parse', '--show-toplevel']).stdout.trim();
|
|
|
|
const USAGE = 'Usage: pnpm check:affected [--base <ref>] [--head <ref>] [--json] [--run]\n';
|
|
|
|
function parseArgs(argv: readonly string[]): Args {
|
|
const values = parseScriptArgs(argv, USAGE, {
|
|
base: { type: 'string', default: 'origin/main' },
|
|
head: { type: 'string', default: 'HEAD' },
|
|
json: { type: 'boolean', default: false },
|
|
run: { type: 'boolean', default: false },
|
|
});
|
|
return {
|
|
base: values.base ?? 'origin/main',
|
|
head: values.head ?? 'HEAD',
|
|
json: Boolean(values.json),
|
|
run: Boolean(values.run),
|
|
};
|
|
}
|
|
|
|
function gitLines(args: string[], cwd: string): string[] {
|
|
return runCmdSync('git', args, { cwd }).stdout.split('\n').filter(Boolean);
|
|
}
|
|
|
|
// Collect every changed file a local plan must account for. The committed diff
|
|
// (base..head via merge-base) is the baseline; `--no-renames` keeps BOTH sides
|
|
// of a rename so a moved file cannot look docs-only by its destination alone.
|
|
// In local mode (head === HEAD) we also fold in working-tree changes and
|
|
// untracked files, which the committed diff never sees — ignoring uncommitted
|
|
// edits would be an unsafe narrowing of the local feedback loop. The staged
|
|
// (`--cached`) and unstaged diffs are collected separately and unioned: a
|
|
// single `git diff HEAD` nets index against working tree, so a staged add and
|
|
// an unstaged delete of the same file would cancel and hide it.
|
|
export function readChangedFiles(base: string, head: string, cwd: string = repoRoot): string[] {
|
|
const files = new Set<string>(
|
|
gitLines(['diff', '--name-only', '--no-renames', '--merge-base', base, head], cwd),
|
|
);
|
|
if (head === 'HEAD') {
|
|
for (const args of [
|
|
['diff', '--name-only', '--no-renames', '--cached'], // staged vs HEAD
|
|
['diff', '--name-only', '--no-renames'], // unstaged (working tree vs index)
|
|
['ls-files', '--others', '--exclude-standard'], // untracked
|
|
]) {
|
|
for (const file of gitLines(args, cwd)) files.add(file);
|
|
}
|
|
}
|
|
return [...files].sort();
|
|
}
|
|
|
|
type PackageJson = {
|
|
scripts: Record<string, string>;
|
|
exports?: Record<string, { import?: string }>;
|
|
};
|
|
|
|
function loadPackageJson(): PackageJson {
|
|
return JSON.parse(fs.readFileSync(path.join(repoRoot, 'package.json'), 'utf8')) as PackageJson;
|
|
}
|
|
|
|
// Public package surface = the source files behind package.json `exports`.
|
|
function packageEntryFiles(pkg: PackageJson): string[] {
|
|
return Object.values(pkg.exports ?? {})
|
|
.map((entry) => entry.import)
|
|
.filter((target): target is string => typeof target === 'string')
|
|
.map((target) => target.replace(/^\.\/dist\//, '').replace(/\.js$/, '.ts'));
|
|
}
|
|
|
|
function printPlanJson(plan: CheckPlan, args: Args): void {
|
|
const ciJobs = ciJobsByCheck();
|
|
const checks = plan.checks.map((id) => {
|
|
const spec = getCheckSpec(id);
|
|
return {
|
|
id,
|
|
label: spec.label,
|
|
ciJobs: ciJobs.get(id) ?? [],
|
|
localRunnable: spec.localRunnable,
|
|
reasons: plan.reasons.filter((reason) => reason.check === id),
|
|
};
|
|
});
|
|
const notSelected = ALL_CHECKS.filter((id) => !plan.checks.includes(id));
|
|
process.stdout.write(
|
|
`${JSON.stringify(
|
|
{
|
|
base: args.base,
|
|
head: args.head,
|
|
failOpen: plan.failOpen,
|
|
failOpenReasons: plan.failOpenReasons,
|
|
docsOnlyPaths: plan.docsOnlyPaths,
|
|
checks,
|
|
notSelected,
|
|
},
|
|
null,
|
|
2,
|
|
)}\n`,
|
|
);
|
|
}
|
|
|
|
function writeLine(line: string): void {
|
|
process.stdout.write(`${line}\n`);
|
|
}
|
|
|
|
function printCheckLine(plan: CheckPlan, id: (typeof plan.checks)[number]): void {
|
|
const spec = getCheckSpec(id);
|
|
const local = spec.localRunnable ? '' : ' (GitHub-authoritative; not run locally)';
|
|
writeLine(` - ${id}: ${spec.label}${local}`);
|
|
if (plan.failOpen) return;
|
|
for (const reason of plan.reasons.filter((entry) => entry.check === id)) {
|
|
writeLine(` · ${reason.path} [${reason.rule}] — ${reason.detail}`);
|
|
}
|
|
}
|
|
|
|
function printFailOpen(plan: CheckPlan): void {
|
|
writeLine('Fail-open: selecting the full check set.');
|
|
for (const reason of plan.failOpenReasons) {
|
|
writeLine(` ! ${reason.path} [${reason.rule}] — ${reason.detail}`);
|
|
}
|
|
}
|
|
|
|
function printSelected(plan: CheckPlan): void {
|
|
if (plan.checks.length === 0) {
|
|
writeLine('No local checks selected.');
|
|
return;
|
|
}
|
|
writeLine(`Selected ${plan.checks.length} check(s):`);
|
|
for (const id of plan.checks) printCheckLine(plan, id);
|
|
}
|
|
|
|
function printPlanHuman(plan: CheckPlan, args: Args): void {
|
|
writeLine(`check:affected — diff ${args.base}...${args.head}`);
|
|
if (plan.failOpen) printFailOpen(plan);
|
|
printSelected(plan);
|
|
if (plan.docsOnlyPaths.length > 0) {
|
|
writeLine(`Docs-only changes ignored: ${plan.docsOnlyPaths.length} file(s).`);
|
|
}
|
|
}
|
|
|
|
// How a resolved command is executed. Injectable so the entrypoint's `--run`
|
|
// propagation (order, skip of GitHub-authoritative checks, stop-on-failure) is
|
|
// testable without spawning real processes.
|
|
export type CommandExecutor = (command: string[], cwd: string) => Promise<number>;
|
|
|
|
const streamingExecutor: CommandExecutor = async (command, cwd) => {
|
|
const result = await runCmdStreaming(command[0]!, command.slice(1), {
|
|
cwd,
|
|
allowFailure: true,
|
|
onStdoutChunk: (chunk) => void process.stdout.write(chunk),
|
|
onStderrChunk: (chunk) => void process.stderr.write(chunk),
|
|
});
|
|
return result.exitCode;
|
|
};
|
|
|
|
export async function runChecks(
|
|
plan: CheckPlan,
|
|
pkg: PackageJson,
|
|
args: Args,
|
|
options: { cwd?: string; execute?: CommandExecutor; changedFiles?: readonly string[] } = {},
|
|
): Promise<number> {
|
|
const cwd = options.cwd ?? repoRoot;
|
|
const execute = options.execute ?? streamingExecutor;
|
|
const runnable = plan.checks.map(getCheckSpec).filter((spec: CheckSpec) => spec.localRunnable);
|
|
const skipped = plan.checks.map(getCheckSpec).filter((spec: CheckSpec) => !spec.localRunnable);
|
|
const coverageSelected = plan.checks.includes('coverage');
|
|
const ciJobs = skipped.length > 0 ? ciJobsByCheck() : new Map<CheckId, string[]>();
|
|
for (const spec of skipped) {
|
|
const jobs = ciJobs.get(spec.id) ?? [];
|
|
process.stdout.write(`\n[skip] ${spec.id} — GitHub-authoritative (jobs: ${jobs.join(', ')})\n`);
|
|
}
|
|
for (const spec of runnable) {
|
|
if (isCoveredByAffectedCoverage(spec, coverageSelected)) {
|
|
process.stdout.write(`\n[dedupe] ${spec.id} — covered by affected LCOV or GitHub CI\n`);
|
|
continue;
|
|
}
|
|
const commands = resolveCheckCommands(spec, pkg, args, options.changedFiles ?? []);
|
|
for (const command of commands) {
|
|
process.stdout.write(`\n[run] ${spec.id}: ${command.join(' ')}\n`);
|
|
const exitCode = await execute(command, cwd);
|
|
if (exitCode !== 0) {
|
|
process.stderr.write(`\ncheck:affected: ${spec.id} failed.\n`);
|
|
return 1;
|
|
}
|
|
}
|
|
}
|
|
process.stdout.write('\ncheck:affected: all runnable checks passed.\n');
|
|
return 0;
|
|
}
|
|
|
|
function isCoveredByAffectedCoverage(spec: CheckSpec, coverageSelected: boolean): boolean {
|
|
return (
|
|
coverageSelected &&
|
|
(spec.id === 'vitest-related' || spec.id === 'unit' || spec.id === 'provider-integration')
|
|
);
|
|
}
|
|
|
|
function resolveCheckCommands(
|
|
spec: CheckSpec,
|
|
pkg: PackageJson,
|
|
args: Args,
|
|
changedFiles: readonly string[],
|
|
): string[][] {
|
|
return spec.id === 'coverage'
|
|
? resolveAffectedCoverageCommands(pkg.scripts, args.base, changedFiles)
|
|
: [resolveCommand(spec, pkg.scripts, args.base, changedFiles)];
|
|
}
|
|
|
|
function resolveAffectedCoverageCommands(
|
|
scripts: Readonly<Record<string, string>>,
|
|
base: string,
|
|
changedFiles: readonly string[],
|
|
): string[][] {
|
|
if (!('check:coverage-changed' in scripts)) {
|
|
throw new Error('Required package.json script "check:coverage-changed" does not exist.');
|
|
}
|
|
return [
|
|
[
|
|
'pnpm',
|
|
'exec',
|
|
'vitest',
|
|
'related',
|
|
'--run',
|
|
'--passWithNoTests',
|
|
// `related` spans every configured Vitest project for broad diffs. The
|
|
// machine-derived default can start enough projects concurrently to
|
|
// starve otherwise-green subprocess/provider tests past their exact
|
|
// timeout budgets. Bound this aggregate feedback lane without changing
|
|
// the suites' own timeout or serialization contracts.
|
|
`--maxWorkers=${DEFAULT_VITEST_MAX_WORKERS}`,
|
|
'--coverage',
|
|
'--coverage.reporter=lcov',
|
|
'--coverage.thresholds.statements=0',
|
|
'--coverage.thresholds.lines=0',
|
|
...changedFiles,
|
|
],
|
|
['pnpm', 'run', 'check:coverage-changed', '--base', base],
|
|
];
|
|
}
|
|
|
|
async function main(argv = process.argv.slice(2)): Promise<number> {
|
|
assertCatalogComplete();
|
|
const args = parseArgs(argv);
|
|
const pkg = loadPackageJson();
|
|
// Validate every catalog command resolves before selecting, so a broken
|
|
// catalog fails loudly rather than silently dropping a gate.
|
|
for (const spec of CHECK_CATALOG) resolveCommand(spec, pkg.scripts, args.base);
|
|
const changedFiles = readChangedFiles(args.base, args.head);
|
|
const plan = selectChecks({
|
|
changedFiles,
|
|
packageEntryFiles: packageEntryFiles(pkg),
|
|
});
|
|
|
|
if (args.json) printPlanJson(plan, args);
|
|
else printPlanHuman(plan, args);
|
|
|
|
if (args.run) return await runChecks(plan, pkg, args, { changedFiles });
|
|
return 0;
|
|
}
|
|
|
|
if (import.meta.url === pathToFileURL(process.argv[1] ?? '').href) {
|
|
runEntrypoint('check:affected', () => main());
|
|
}
|