306 Commits

Author	SHA1	Message	Date
Michał Pierzchała	f4fc089099	0.20.6	2026-08-07 08:34:50 +02:00
Michał Pierzchała	d85072d935	perf(cli): route command aliases through the help fast path (#1641) ... * perf(cli): route command aliases through the help fast path bin.ts's `--help` fast path resolved aliases through a hand-written two-entry table that had drifted out of sync with the real CLI_COMMAND_ALIASES registry (five entries). `tap`, `launch`, and `relaunch` missed the table and silently fell through to a full runCli() bootstrap just to print static help text (~150-165ms vs ~45-50ms for aliases already in the table). Delegate to the shared normalizeCliCommandAlias registry instead of the stale local table, so every alias the registry knows about gets the fast path automatically. * test(cli): add R12 layering guard for bin.ts's alias delegation The unit test added for the alias fast-path fix (cli-help-alias-fast-path.test.ts) calls normalizeCliCommandAlias directly, so it stays green even if bin.ts itself reverts to a hand-rolled table — it pins the registry composition, not bin.ts's own wiring, and bin.ts cannot be safely unit-imported (it runs unguarded top-level dispatch on import and is deliberately excluded from coverage). Add an AST-based structural guard instead, in the style already established by scripts/layering/session-state.ts, facade-exports.ts, and zero-dep-jobs.ts (oxc-parser's module/program records, not a line scan, so a fixture's string literal can't produce a false hit). R12 asserts two facts about src/bin.ts: it holds a value import of normalizeCliCommandAlias from commands/cli-command-aliases.ts, and it contains none of the registry's own alias tokens as string literals. The token list is read out of the registry's own source (CLI_COMMAND_ALIASES's `alias:` property values), not hard-coded, so a future sixth alias is covered automatically. Both facts were false on the pre-fix bin.ts, verified by reverting locally and capturing the failure before restoring the fix. Wired into the existing check:layering chain (already part of check:tooling), next to R7's session-state ownership rule, which pins the same "delegate to your single owner" shape. * test(cli): pin the alias-resolver call into buildCommandUsageText (R12 P2) Maintainer review of R12 (PR #1641): import-presence and literal-absence alone let bin.ts regress to buildCommandUsageText(helpTarget) while the normalizeCliCommandAlias import stays in place, used harmlessly elsewhere (or not at all) — the real-tree gate stayed green through that exact regression. Add a third fact: bin.ts's call to buildCommandUsageText must receive, as its argument, a call to the LOCAL binding the resolver was imported as (aliasResolverLocalName + usageTextCallsResolver, both AST-based). Binding by local name rather than the literal export name means a renamed import (`as resolveAlias`) still verifies, and an unrelated same-named local cannot be mistaken for it. Verified by reverting locally to exactly the missed regression — import left in place, call reverted to buildCommandUsageText(helpTarget) — and confirming R12 now fails where the two-fact version passed; restored after. Two negative fixtures pin the scenario going forward: import present but unused, and import present but used only unrelated to the call. * test(cli): make R12's delegation fact universal and value-bound The previous fact 3 asked whether *any* `buildCommandUsageText(resolver(...))` existed in bin.ts. That quantifier is satisfied by a decoy call while the line that actually ships resolves nothing: void buildCommandUsageText(normalizeCliCommandAlias('open')); const commandHelp = buildCommandUsageText(helpTarget); Fact 3 now requires EVERY `buildCommandUsageText` call to receive the imported resolver applied to the fast path's own help-target binding, which rejects both lines above independently. The help-target name is read from bin.ts (the variable initialized by `resolveSimpleHelpTarget`), so renaming it re-points the guard instead of disarming it. Because fact 3 claims binding identity by name, it also now rejects a local shadow of the resolver and an ambiguous second help-target declaration — a same-named local would otherwise let the composition read as delegation while calling something that resolves nothing. The predicate returns the reason rather than a boolean, so the gate names which of the several distinct failures happened. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Rva4YGtSCAKJqH5PbpcCU --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-06 21:00:22 +02:00
Michał Pierzchała	3d2a9a05e8	ci: remove package smoke workflow (#1624) ... * ci: remove package smoke workflow * ci: align affected package check * ci: verify packaged tarball before publish	2026-08-06 13:31:36 +02:00
Michał Pierzchała	4269ca6d88	fix: update MCP registry namespace (#1618) ... * fix: update MCP registry namespace * feat: inherit MCP descriptions from CLI help * refactor: project command guidance per surface * refactor: make command guidance a single canonical description The guidance type carried seven fields, but only three were ever set, and all twenty call sites used it the same way: to hold a second, hand-written MCP string next to a near-identical CLI one. That is the drift the abstraction was meant to remove, so the type no longer offers a per-surface description at all. A command now has one canonical description plus an optional tail per surface: guidance: { description: 'Shared body.', cliDetail: 'Flags, positional syntax, terminal examples.', mcpDetail: 'When-to-use and sequencing hints.', } Because a surface can only append, CLI help and MCP tool text cannot diverge — the guard against CLI syntax in MCP descriptions becomes structural rather than a review tripwire, since flag vocabulary only lives in cliDetail. All twenty commands that previously carried two descriptions now share one body. Also: - Drop `summary` from the description fallback chain. It is the short list-view line, so falling back to it replaced the full description with a fragment on both surfaces: artifacts, boot, and shutdown each lost their real description. - Stop writing the MCP variant back over `metadata.description`. That field feeds CLI help, `explain`, and docs; `explain` was printing MCP-only text. MCP now reads a separate `mcpDescription`. - Drop `mcp.parameters`. It restated inputSchema property descriptions inside the tool description — 1232 characters duplicated verbatim across six tools, and three of sixteen declared hints silently rendered nothing because the property had no description. Those properties are documented in the schema instead, which serves MCP, --help, and docs at once. - Drop `cli.flags`. Its one use appended "Relevant flags: --surface, --launch-console." to help text that already named both flags inline. Tests assert the structural property (both surfaces share a canonical prefix) and the summary-fallback regression, alongside the existing CLI-syntax guard. CLI help wording assertions follow the new copy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf * test: gate undocumented MCP tool inputs Guidance no longer restates input fields in prose, so a tool's inputSchema is the only place its inputs are documented — for the model, for --help, and for the docs site. An undescribed property is a silent gap rather than a cosmetic one, which is exactly the failure mode the removed `mcp.parameters` selection had: it dropped hints for properties that carried no description and reported nothing. Describe the two trigger-app-event inputs that mechanism used to name, and add a ratcheting gate over every MCP tool input. A property key that is not already in the budget fails immediately; the total may never grow, and lowering it is required once properties gain descriptions, so the 132 remaining stay visible instead of settling in as permanent debt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf * fix: project the canonical description to every surface Storing only the MCP variant left the shared body unpropagated: `metadata.description` and the executable definition kept their pre-guidance text, so `explain click` reported "Click or tap a semantic UI target..." while CLI help and the MCP tool both used the canonical "Activate a UI target...". 53 commands were affected — the CLI schema base, `explain`, and docs all read `metadata.description`. `projectCommandGuidance` now returns the canonical body plus the MCP-only tail, and `defineCommandFacet` writes the body to both metadata and the definition. Only the tail is stored apart, as `mcpDetail`, so the body has exactly one home instead of a second full copy that could drift; `composeMcpDescription` joins them for the tool surface. The surface gate pins the invariant: definition, metadata, and `explain` must report the identical body, and neither CLI help nor the MCP description may do anything but extend it. Both arms verified by breaking them. Also replace the undocumented-input ratchet's bare-key allowlist plus aggregate budget with exact `tool.property` identities. The old shape stayed green while a gap migrated: describing `foo.text` and adding an undescribed `bar.text` left both the allowed-name set and the total of 132 unchanged, and stale names kept authorizing later gaps. Verified with that exact scenario — `app` was already an allowed name via push/reinstall/settings and the total held at 132, yet a newly undescribed `open.app` now fails. Recording a fix requires deleting its baseline entry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf * refactor: drop guidance.description in favour of the command's own `guidance.description` restated what `metadata.description` already is. Setting it shadowed the metadata literal rather than replacing it, so every command that used it shipped two bodies: the canonical one and a terse original that no surface could observe — 768 bytes of unreachable strings across 20 commands. Move each canonical body to the metadata literal where it belongs and delete the field. Guidance is now tails only, `cliDetail` and `mcpDetail`, which also removes the question of where a body is written: there is one place, and no chain to consult. Three guidance blocks held nothing else and are gone entirely. registry.js drops 1117 bytes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf * refactor: give every command one text block with a mandatory summary The CLI carried four prose fields with a fallback chain between them, and one of them — `helpDescription` — was authored on 45 commands and generated on the rest. That ambiguity is why surfaces drifted: whichever field a reader looked at, some other field might be the one actually rendered. Prose now lives in a single `CommandText`, and `CommandSchema` keeps only grammar: summary what is this command, in a list of ninety? (mandatory) description what does it do, and when do I reach for it? (mandatory) cliDetail flags, argument shapes, terminal examples mcpDetail sequencing and cross-tool hints `--help`, the command list, the MCP tool description and `explain` are projections computed where they render, so nothing derived is stored and no field can be both input and output. The four-field model was validated against the whole surface before the migration: all 67 commands reproduce their MCP text exactly and derive their help body from `description`, so none needed a fifth field. Making `summary` mandatory fixes a regression this branch introduced. 23 commands had none, so the command list fell back to the full detail paragraph; lengthening those descriptions earlier turned `click`'s list entry from 61 characters into 267, `fill`'s into 214, `devices`' into 159. Every command now states its own line, and a gate holds them under 72 characters, non-empty, period-free, and distinct from the description. Two duplications go with it: the per-command help printed its synopsis twice, once as a header and again under `Usage:`, and `press` said "use longpress" in both its body and its tail. * refactor: tighten the command text plumbing Self-review follow-ups on the text model, all quality-only: `command-text.ts` moves from `cli-schema/` to `commands/`. It is a command concept that MCP reads as much as the CLI does; living under `cli-schema` made the MCP surface import a CLI module to render its own tool descriptions. `defineCommandFacet` no longer casts. It took a facet and returned it with the schema completed, but claimed to return the input type, which needed `as unknown as` — a double cast is the type system reporting that the signature was wrong. Splitting `CommandFacetInput` from `CommandFacet` states the completion in the return type, so both that cast and the registry's `as CommandSchema` go. `push`'s summary duplicated its description apart from a trailing period, which the gate missed by comparing exact strings; it now compares normalised text, and the summary says something the description does not. `install-from-source`'s summary loses a clause it did not need. --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-05 18:51:21 +02:00
Michał Pierzchała	23a3016e9b	fix: stop the unit suite from leaking temp directories (#1593) ... * fix: stop the unit suite from leaking temp directories ~650 test call sites across the unit suite create scratch directories via fs.mkdtemp(path.join(os.tmpdir(), ...)) or shared factories (makeSessionStore) with no cleanup, ever. Over time this accumulated 1.16M+ orphaned directories in the real system tmpdir, slow enough to make tools that enumerate $TMPDIR at startup (e.g. opencode) take 1-2 minutes to launch. Rather than migrate every call site, redirect os.tmpdir() itself for the lifetime of the whole `vitest run` invocation: scripts/vitest-tmpdir-global-setup.ts wires in as vitest's globalSetup/globalTeardown, points TMPDIR at one /tmp-rooted directory (verified: env mutations here propagate to every forked worker, confirmed empirically), and removes it in one recursive rm after every worker across every project finishes. Since os.tmpdir() reads TMPDIR on every call, this covers all ~650 call sites without touching any of them. Rooted at /tmp rather than nested inside the current (already deep, on macOS) os.tmpdir(): that broke real AF_UNIX socket tests (runner-usbmux.test.ts) by pushing socket paths past the 104-byte sun_path limit. A per-file afterAll hook was tried first but proved unreliable — 5 of 7 workers in one run never ran it before their process was torn down; the global setup/teardown pair (one process, confirmed single execution) is the mechanism that's actually guaranteed to run once. Also adds: - scripts/check-tmpdir-leaks.ts: CI/local guard asserting no agent-device-test-run-* directory survives a run (a leftover one means a worker was killed before cleanup could run). - src/__tests__/test-utils/tmp-dir.ts: documented mkdtempForTest / mkdtempForTestSync helpers, the discoverable way to get a scratch dir going forward (mirrors the src/utils/exec.ts pattern for node:child_process). - scripts/check-test-tmpdir-helper.ts: ratchet guard capping raw fs.mkdtemp/mkdtempSync call sites in test files at today's count (632); it can only shrink as call sites migrate to the helper. * fix: make check-tmpdir-leaks scan the same root the fix actually uses check-tmpdir-leaks.ts was scanning os.tmpdir() for leftover run directories, but vitest-tmpdir-global-setup.ts creates them under a hard-coded /tmp. On macOS those are different paths (TMPDIR is a deep per-user /var/folders/.../T/ directory) — the guard could never find a leak on the exact platform the original leak happened on, only on Linux CI where os.tmpdir() already is /tmp. Export TEST_RUN_TMP_ROOT and TEST_RUN_TMP_PREFIX from the global-setup module and import them in the leak check instead of recomputing a path that can drift. Switched from a fixed pid-based directory name to fs.mkdtempSync so a same-named leftover from a prior killed run (or, on a shared machine, another user) can't collide with a live run. Also fixes two stale comments (in this file and ci.yml) that still described the per-file afterAll hook design that was abandoned in favor of the global setup/teardown pair, and notes the check only covers vitest runs, not the node --test lanes (test:smoke, test:integration:node). Verified live: with the old code the guard reported no leaks even with a real orphaned /tmp/agent-device-test-run-* directory present (left by a command that got killed mid-run); with this fix it correctly found and reported it. * simplify: drop the tmpdir ratchet guard, keep the leak check local-only Two guards were more than this needed: - check:test-tmpdir-helper (ratchet on raw fs.mkdtemp call counts) protects nothing a bug could actually trigger — the leak is already fixed architecturally regardless of call-site count, so this was pure style/discoverability nudging. Dropped the script and its check:tooling/CI wiring; kept mkdtempForTest/mkdtempForTestSync in tmp-dir.ts as the documented option without enforcing it. - check:tmpdir-leaks in CI added little: GitHub-hosted runners are destroyed after each job, so a leftover directory there is harmless by construction, and a worker getting killed mid-run would already surface as a job failure some other way. Its real value is local, on the long-lived dev machines where the original leak actually accumulated — kept it wired into check:unit, dropped the CI step. * fix: don't flag a concurrent vitest run's tmpdir as a leak check-tmpdir-leaks.ts reported every agent-device-test-run-* directory as a leak, but a concurrent vitest run in another worktree legitimately keeps its own directory present until its own teardown finishes. On a machine that regularly runs several worktrees at once, that made check:unit fail on unrelated in-progress work. Embed the owning process's pid in the directory name (still random- suffixed via mkdtempSync, so same-pid reuse across separate runs can't collide) and have the leak check skip any directory whose pid is still alive (process.kill(pid, 0)) — only directories whose owning process already exited without running its globalTeardown are real leaks. Split the pure logic into check-tmpdir-leaks-model.ts (findLeakedRunDirectories, with an injectable liveness check for testing) so it has a real regression suite, including the concurrent-run case, instead of only being exercised by hand. * refactor: migrate raw fs.mkdtemp call sites to mkdtempForTest(Sync) Migrates 629 raw fs.mkdtemp(Sync)(path.join(os.tmpdir(), PREFIX)) call sites across 168 test files to the mkdtempForTest / mkdtempForTestSync helpers (src/__tests__/test-utils/tmp-dir.ts), so there's one documented, discoverable way to get a scratch dir in a test — cleanup already didn't depend on the call-site shape (the global TMPDIR redirect covers any of them), this is purely for consistency and discoverability, same reasoning as src/utils/exec.ts for node:child_process. Existing manual per-test cleanup (fs.rm in finally/afterEach/onTestFinished blocks) is untouched — the global teardown is a fallback for killed workers, not a replacement for tests cleaning up after themselves. Migrated with a one-off AST-based codemod (oxc-parser, since regex mismatched multi-line calls and complex prefix expressions like `options?.tempPrefix ?? 'default-'`) rather than by hand across 168 files. The codemod isn't included — it doesn't need to survive this commit. Caught and fixed one real bug in it during review: a small number of files declare a second import statement later in the file, after some of the matched call sites, which broke a naive "insert after the textually-last ImportDeclaration" placement; fixed to insert after the top contiguous import block instead, plus a self-check that re-parses every generated file before writing it. Also fixes 5 fallow dead-code findings the branch introduced: the vitest globalSetup functions (setup/teardown) are only referenced by the config-string path vitest.config.ts hands to globalSetup, invisible to static analysis — suppressed with the documented convention. isProcessAlive didn't need to be exported (nothing outside the module uses it). And dropped a barrel re-export of the new helpers from test-utils/index.ts: nothing actually imports through the barrel (matching the existing makeSessionStore convention, which is imported directly from store-factory.ts everywhere despite also being barrel-exported), so the re-export was genuinely dead. Documents the convention in docs/agents/testing.md. Verified: full unit suite (5308 tests) passes except the one pre-existing, unrelated package-exports.test.ts failure; typecheck, lint, and format all clean; fallow audit clean against the PR base. * fix: correct fallow suppression token and drop unused barrel re-export These were meant to be part of 397cdd6d7 (verified locally before that commit) but didn't actually get staged — caught by CI's Fallow Code Quality check re-running against the pushed commit, which still had the plural 'unused-exports' token (fallow expects singular 'unused-export') and the dead barrel re-export. * fix: migrate the two mkdtemp call sites the rebase silently reintroduced Rebasing onto main pulled in #1594's two new test cases in this file, added independently of this branch's migration, still using raw fs.mkdtempSync(path.join(os.tmpdir(), ...)). Git's line-based merge found no textual conflict with this branch's removal of the os import (the changes touch non-overlapping regions), so it silently produced a file that doesn't typecheck. Migrated both to mkdtempForTestSync for consistency with the rest of the file, caught by CI's Typecheck, Fallow Code Quality, and FreeRange checks re-running against the pushed commit. * test: pin Vitest tmpdir lifecycle * fix: preserve the Swift cache across test runs	2026-08-05 08:56:39 +02:00
Michał Pierzchała	4f8dc3f31e	refactor: move selector engine into workspace package (#1589) ... * refactor: move selector engine into workspace package * refactor(selectors): trim the package façade to its real consumers Follow-up to the selector-package cutover, from a structural review of it. - Drop 15 façade symbols with no consumer anywhere in the repo: selectorUsesKey (added by the cutover, never called), isNodeVisible / isNodeEditable (the real helpers are contracts/snapshot's), normalizeText, splitIsSelectorArgs, IS_PREDICATE_REQUIRED_MESSAGE, four nested Replay types, SelectorDisambiguationDisclosure, and the four kernel type re-exports every consumer already imports from kernel directly. - Delete SelectorCapturePolicyInput.selectorExpression, which deriveSelectorCapturePolicy never read; the policy varies only by predicate, so it takes one now. Two of the four tests asserted that the unread parameter had no effect and could not fail; they go with it. - Return the Maestro export vocabulary to the maestro package. The cutover inlined MAESTRO_TEXT/STATE_SELECTOR_KEYS' values into the CLI call site, leaving both constants dead in the package that owns the concept and no gate over the two copies. MAESTRO_SELECTOR_PROJECTION is now the one statement of it. - Dedupe SelectorDiagnostics and SelectorDisambiguationDisclosure, declared character-for-character twice across the AST/string seam, and name the two shared option shapes once instead of five inline copies. The parser-side resolution types take an Ast prefix so the twins read as twins. - Delete three identity wrappers: parsePrivateSelector, selectorExpressionToMaestro, and the formatSelectorFailure forwarder — nothing passes it a chain any more, so the SelectorChain | string union and its branch go too. - Delete internal/index.ts, an AST barrel whose only consumer was one test in the same directory (renamed to engine.test.ts), and the match.ts pass-through that existed to feed it. - ReplaySelectorGrammar had three variants for two behaviors; 'wait' and 'ordinary' were the same path. It is 'is' | 'positional' now. - Drop the deleted src/sdk/selectors.ts from .fallowrc.json's entry list. Behavior unchanged. pnpm check green: 598 unit files / 5278 tests, smoke 35 passed / 3 live skipped, layering 71/71, depgraph 22/22, mutation config 45/45, fallow clean, package smoke sound. Counterfactual: pointing MAESTRO_SELECTOR_PROJECTION.textKeys at the state keys turns three replay-maestro-export cells red; restored before commit. * test(selectors): split the engine aggregation test by source concept `internal/index.test.ts` (renamed `engine.test.ts` when its barrel went away) was a 708-line aggregation over the whole engine — past the 500-line tripwire and mirroring no source module, so it also ran as one serial unit. It becomes five files that each mirror what they test, plus the parser cells folded into the existing parse test: resolve.test.ts alternative fallback, strict uniqueness, first-match existence resolve-disambiguation.test.ts ADR 0012 ranking: deepest, smallest-area, winner-vs-challenger disclosure, tie fallback resolve-viewport.test.ts the visibility half: on-screen beats off-screen, including inside an off-screen scroll container match.test.ts per-key matching semantics (text, role, focused, appname/windowtitle, decoded newline labels) arguments.test.ts where the selector ends and the command's positionals begin, both grammars parse.test.ts +6 grammar/escape cells beside the existing property tests The login-form tree shared by resolve.test.ts and match.test.ts moves to `__tests__/login-form-nodes.ts` rather than being copied into both. All 27 cells are carried over unchanged and still pass; no file now exceeds 224 lines. pnpm check green: 602 unit files / 5278 tests, layering 71/71, depgraph 22/22, mutation config 45/45, fallow clean over 127 changed files. * revert(selectors): keep agent-device/selectors public, behind one AST subpath The cutover removed the `agent-device/selectors` public subpath as part of tightening the API. It is in use, so the removal is reverted: the subpath ships the same ten symbols v0.20.5 shipped, with the same signatures. That has to coexist with the reason the package façade is string-only, so the AST leaves through one named door instead of the main one: @agent-device/selectors string-in/string-out; every in-repo consumer @agent-device/selectors/ast the published parser surface; one consumer, src/sdk/selectors.ts `packages/selectors/src/ast.ts` re-exports parseSelectorChain, tryParseSelectorChain, isSelectorToken, the AST-taking findSelectorChainMatch and resolveSelectorChain, isNodeVisible, isNodeEditable, and types SelectorChain / SelectorDiagnostics. `formatSelectorFailure` keeps its published `SelectorChain | string` first parameter as a shim here rather than widening internal/resolve.ts back to a union — the compatibility obligation sits at the boundary that owes it. This is strictly narrower than main, where the AST was reachable from anywhere in src/ via src/selectors/*. Two gates hold it there: facade-symbols.ts pins ./ast to exactly the v0.20.5 list, and package-boundaries.test.ts asserts src/sdk/selectors.ts is the only file outside the package that imports it. Restored alongside: the ./selectors export and tsdown entry/chunk group, the .fallowrc.json entry, the package-exports supported-subpath list, and both client-api.md sections. No CHANGELOG entry — nothing is removed any more. pnpm check green: 602 unit files / 5278 tests, smoke 35 passed / 3 live skipped, layering 71/71 (10 packages, 32 subpaths), depgraph 22/22, mutation config 45/45, fallow clean over 129 changed files, package smoke imported all 12 published entry points with publint and attw passing. Verified functionally against the built dist: the doc's parse -> findSelectorChainMatch example returns the same shapes as before, resolveSelectorChain still returns an AST `selector`, and formatSelectorFailure still accepts a chain. * fix(selectors): correct the two expectations that still assume the removal Review P1s on a792415a: restoring the public subpath left two gates asserting it was gone. - installed-package-metro.test.ts moved `agent-device/selectors` into the blocked-specifier list. It goes back to the subpath smoke set, running the same `isSelectorToken('||')` + `parseSelectorChain` check it ran before the removal, so the file's only remaining delta from main is a formatter reflow. - owner-files-no-leak.test.ts asserted `dist/src/sdk-selectors.js` was absent. It requires the stable named chunk again, and still rejects an auto-numbered `selectors2.js` fallback — the pair is what proves the restored tsdown chunk group is doing its job, verified against a clean build. PR body corrected: the removal is no longer described as intentional API tightening. * refactor(selectors): satisfy the widened fallow scope after rebase main's #1591 (the follow-up filed from this review) removed `packages/**` from .fallowrc.json's ignorePatterns, so the new package is audited for the first time. Everything below is a finding fallow could not previously see. Dead surface, all confirmed consumer-free: - 12 type re-exports from the `.` façade whose shapes consumers only ever reach structurally. - MAESTRO_TEXT_SELECTOR_KEYS / MAESTRO_STATE_SELECTOR_KEYS, orphaned by this branch's own MAESTRO_SELECTOR_PROJECTION change, and the test-util SELECTOR_VALUE_HAZARDS. All three are module-local now. - IS_PREDICATE_USAGE_HINT fails --production because its only consumer is the is-argument-surface parity test. It gets a commented `ignoreExports` entry rather than deletion: the constant is what makes the daemon and CLI raise ONE hint instead of two copied strings (ADR 0010), so the test asserting that is the point, not an accident. `fast-check` is now declared by the package that imports it. Duplication, split by what could be proven: - `isUsefulVisibilityAnchor` existed character-for-character in both packages/selectors and packages/maestro. Moved to @agent-device/contracts/snapshot, which both already depend on and which already owns this vocabulary. Safe because the `normalizeType` each copy called is itself character-identical to the contracts one — checked before moving, since a different normalizer would have silently changed which nodes anchor. - maestro additionally reimplemented `normalizeType`, `buildSnapshotNodeMap` (as `buildSnapshotNodeByIndex`) and `findSnapshotAncestor`, all character-identical to contracts'. Deleted in favour of the shared ones. - The three scroll-ancestor walks are NOT deduped. They are structurally the same walk but each uses a different scrollable predicate, and I have no evidence the three agree; collapsing them would be a Maestro-conformance change, not a cleanup. Both maestro sites now say so, and the work is filed separately. `projectSelectorExpression` (15 cyclomatic / 22 cognitive, written by the cutover) splits into a dispatcher plus `readAgreedTextValue` and `projectSelectorTerms`; all three are under threshold. Rebase note: the one conflict, in package-boundaries.test.ts, resolved to NEITHER side — #1591 had already deleted `AdReplayVerifiedTargetGuard` as an unused export, and this branch deletes the seven ReplaySelectorPort names, so the conflicting block is empty. * build: record fast-check for packages/selectors in the lockfile Declaring the dependency in packages/selectors/package.json without regenerating pnpm-lock.yaml made every CI job fail in its install step with ERR_PNPM_OUTDATED_LOCKFILE. My local `pnpm install --frozen-lockfile` printed "+ 1 dependencies were added: fast-check@^4.9.0" and exited 0, which read as success but was the same mismatch CI refuses. Regenerated with the pinned pnpm 11.17.0, not the 11.5.3 on this machine: 11.5.3 rewrites peer-dependency resolution keys repo-wide (dropping `(supports-color@7.2.0)` suffixes) and produced a 222-line diff. With the pinned version the diff is the 4 lines this change actually needs, plus pnpm's alphabetical re-sort of the root selectors entry.	2026-08-04 19:06:39 +02:00
Michał Pierzchała	80feff42d6	build: verify the published tarball instead of grepping the bundle (#1578) ... * build: verify the published tarball instead of grepping the bundle Replaces the bundle-dependency grep with one gate that packs the tarball npm would publish and proves it sound from a clean consumer install: publint and attw on the tarball, a two-way dependency-closure audit, an import of every `exports` subpath, and the CLI smoke run — all from outside the workspace, where no pnpm link can mask an unresolvable specifier. Also stops the build from emitting a publishable bundle in the first place: a missing workspace link now fails `pnpm build` instead of warning and exiting 0, which is how 0.20.4 shipped an unresolvable `@agent-device/ad-script` import. publint found 12 real defects in the current package — every `exports` entry listed `types` after `import`, so TypeScript resolved declarations by accident rather than by condition. The dependency audit found `pngjs` declared as a runtime dependency while tsdown inlines it, an install every user paid for and no shipped code reached; it moves to devDependencies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NamFJUgn9DGHrT2za11JbD * fix(ci): run the package gate without pnpm on the Node floor pnpm 11.17 requires Node >= 22.13, so `pnpm check:package` could not start on the 22.12 floor the Packaged CLI job exists to cover. The gate needs only `node` and `npm`, so the job invokes the script directly. Splits the dependency-closure audit into a collector and a message builder to clear Fallow's complexity threshold, and classifies both packaging linters in ignoreDependencies: they are subprocess CLIs with no importable API here, which dependency analysis cannot follow to an import. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NamFJUgn9DGHrT2za11JbD * fix(publishing): read every literal resolution form in the closure audit The dependency-closure audit derived shipped imports from the ESM module record alone, so it could not see a package resolved through `require` or a `createRequire` result: neither produces a module-record entry. A lazy `createRequire('@agent-device/…')` would therefore clear the audit, the all-export probe and the exercised CLI paths, reintroducing the 0.20.4 published-install failure class for another command. Measuring the built bundle turned up a second, larger hole in the same reader. The shipped files are minified, and the minifier rewrites every string literal to a no-substitution template literal, so the dynamic-import extraction — which accepted quoted strings only — matched 0 of the 99 dynamic imports the bundle contains. The lazy `import()` path that broke 0.20.4 was reported as covered while checking nothing. Specifiers now come from the module record plus an AST walk over every literal runtime-resolution form: `import()`, `require()`, `require.resolve()`, an immediately-invoked `createRequire(...)`, and calls through a `createRequire` result under any import or minified alias. Both spellings of a string literal count everywhere, and `.cjs` joins the scanned extensions. Computed specifiers stay explicitly out of scope, and are pinned as such. Rejecting them is not available: minifiers reuse short identifiers across scopes, and the packed bundle really does contain an unrelated `a(h[t],f,g,l,e,m)` that no name-based match can distinguish from a require call. Those are covered by the gate's runtime half instead, which resolves them for real. Bare-identifier calls need the one-string-argument shape for the same reason. The audit moves to scripts/lib/shipped-imports.ts so fixture packages can exercise it. The gate needs a real `npm pack` behind minutes of Swift and Android builds, so every check that runs it can only watch a healthy package pass — which is how a reader that matched nothing looked covered. The new fixtures assert the failure direction per resolution form: 16 of the 22 fail against the previous reader, and the 6 that pass are the quoted-spelling and pinned-limitation cases. A wiring assertion keeps the audit and both runtime probes attached to the gate, since fixtures alone would stay green if the call were deleted. Verified against the real built bundle: the closure resolves to exactly the two declared dependencies, so the stricter reader adds no false positives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NamFJUgn9DGHrT2za11JbD --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-04 12:28:05 +02:00
Michał Pierzchała	016577e6bc	docs: remove superseded architecture proposals and design prototypes (#1478 P7) (#1580) ... P7 cleanup per #1478's ratified defer decision: delete the daemon-modularity proposal, module-interface-principles (durable kernel folded into CONTEXT.md), the pre-package Maestro debt map, and the daemon-boundary prototypes with their package scripts; refresh CONTEXT.md's R10 bullet to post-arc state. Also reconciles the interaction façade symbol pin with #1570's wait symbols, which had left check:layering red on main via #1570 × #1574 branch-race skew. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-04 11:14:51 +02:00
Michał Pierzchała	83322a3f2f	test(layering): pin exact façade symbols for all workspace packages (#1574) ... * test(layering): pin exact façade symbols for all workspace packages #1555 added the repo's first exact exported-symbol gate, pinning @agent-device/ad-replay's named export list. Every other workspace package was still covered only by the exports-subpath locks, which prove which files a package exposes but say nothing about what those files name — so any façade could grow a symbol silently. Pin all 29 exported subpaths across the remaining 8 packages: ad-script, contracts (14), kernel (8), maestro, provider-limrun, provider-webdriver, replay-test, and xml. The lists are the honest current surface, untrimmed — contracts/interaction alone names 140 symbols, and pinning the real number is what makes the next widening visible. The table is checked in both directions, so a new package or subpath that nobody pinned fails rather than being silently skipped. Pinning contracts needed the export-discovery helper widened: 13 of its 14 façades are bare `export * from '../x.ts'` barrels, and readNamedExports throws on those by design, because given only a source string the contributed set is genuinely unknowable. Given the FILE it is not, so readFacadeExports resolves the relative re-export chain and enumerates it. Resolution stays narrow — a package-specifier star still throws (that would mean re-entering another package's exports map, the unbounded widening the gate exists to refuse), cycles are visit-guarded, and a default export still throws through a barrel. Helper unit tests cover the shapes the merged AST scan handles but left unpinned: `export { default as x }` (the form between the two rejection rules — named, so reported, never `default`), a local `export { … }` list with no `from`, and multi-declarator `export const a = 1, b = 2`. Plant-verified per package rather than asserted: a stray export on ad-script, one two files deep behind contracts' `export *` chain, and an unpinned new subpath on xml each failed with a named diff; each reverted to green. Gates: check:layering (63 tests, up from 53) / typecheck / lint / format:check — green. * fix(layering): model real `export *` semantics; split the pinned table out Addresses both P1 findings on #1574. P1 — `readFacadeExports` did not model `export *` façade semantics. It unioned every child name and threw on every child default. Both are wrong: - Per GetExportedNames, a star export excludes the child's `default`, so a private `export default` in a leaf is not reachable through the barrel and does not widen the façade. It is now passed over rather than rejected; the previous test codified that false positive and is replaced. A default on the ENTRY file is still a real default export of the façade and still throws. - Per ResolveExport, a name two star sources resolve differently is `ambiguous` — importing it is a SyntaxError, so it is not part of the surface at all. Unioning would pin a symbol no consumer can import; ambiguity now throws and names both origins. Origins are tracked by declaring module rather than by path taken, so a diamond (two barrels reaching one declaration) resolves normally, and an explicit export shadows a star-provided name of the same name as the spec's own precedence does. Both counterfactuals are tested alongside the two rejection cases. P1 — module size. The 885-line generated FACADE_SYMBOLS table moves to a focused sibling, scripts/layering/facade-symbols.ts, leaving the behavioral tests at 642 lines (from 1,455) so the test file stays one bounded read per AGENTS.md. Gates: check:layering (66 tests, up from 63) / typecheck / lint / format:check — green. Contracts plant re-verified under the corrected semantics: a stray two files deep behind the `export *` chain still fails with a named diff, and reverts to green. * fix(layering): resolve re-export identity transitively; extract facade-exports Addresses both P1 findings on the second review round. P1 — named re-export identity stopped at the immediate source. Given `a` re-exporting `x` from `b`, `c` re-exporting `x` from `a`, and a façade starring both, ESM resolves ONE binding (b's `x`), but the walker identified the two paths as `b#x` and `a#x` and falsely rejected the façade as ambiguous. Reproduced before fixing. Origins now resolve through the chain to the binding a name ultimately names, by asking the child's own already-resolved map instead of synthesizing an identity from the specifier. A package specifier keeps a stable synthetic identity (it is not a file this gate reads), and a cycle in progress falls back to the immediate source. Two tests, counterfactual-verified against each other: the chain diamond now resolves to one name (confirmed failing with the old immediate-source identity, passing with the fix), and a same-depth chain whose branches bottom out in two genuinely distinct declarations still throws — so the fix cannot be satisfied by simply collapsing every duplicate. P1 — context-safety extraction was incomplete. Façade export enumeration moves to scripts/layering/facade-exports.ts (219 lines) with its own facade-exports.test.ts (245), registered in check:layering. package-boundaries.ts drops to 338 from 528 and its test file to 450 from 642: the boundary rules answer "may this file import that one?", this module answers "what does this façade name?". Every layering file is now under the 500-line tripwire except the generated symbol table, which the rule exempts. Gates: check:layering (68 tests, up from 66) / typecheck / lint / format:check — green. Contracts plant re-verified after the split. * fix(layering): filter `default` at the star, not at its source The reported P1 does not reproduce: intermediate `export { default } from './x.ts'` links are reported by oxc as kind `Name` with the name `default`, not kind `Default`, so they already resolve transitively; and for a terminal `export default <decl>`, the fallback identity `${child}#default` is exactly the canonical binding, so both paths agree. The exact five-module scenario from the review returns ['x']. That behavior is now pinned by a test so it cannot silently regress. Investigating it did surface a real spec violation in the opposite direction. Because a re-exported `default` is a named entry, it landed in the module's map and was then copied wholesale by star enumeration, so `export * from './mid.ts'` reported `default` as part of the surface — a name `GetExportedNames` explicitly skips, and which oxc itself labels `AllButDefault` on the star's own import. `default` is now filtered at the star rather than at the source. That placement is the point: the name has to stay in the module's map so a later `export { default as x }` can resolve its binding, while never being reachable through a star. Filtering at the source would have broken identity resolution — the very thing the review round before this one fixed. A façade entry re-exporting a default under the name `default` is now rejected too. It carries a default export exactly as `export default …` does; only the parse shape differs, and only the declared form was being caught. Three tests: the star filter (counterfactual-verified — removing the filter fails it — with a sibling name proving the module is still read), entry-level rejection, and the two-paths-to-one-default-binding case from the review. Gates: check:layering (71 tests, up from 68) / typecheck / lint / format:check — green. Contracts plant re-verified. --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-04 10:34:57 +02:00
Michał Pierzchała	a9f8fc8da1	0.20.5	2026-08-03 22:34:42 +02:00
Michał Pierzchała	178a36c419	fix: prevent publishing unresolved workspace imports (#1577)	2026-08-03 22:34:15 +02:00
Michał Pierzchała	761317deb7	refactor(daemon): extract native .ad replay to packages/ad-replay (#1478 P5) (#1555) ... * refactor(replay): move the dependency-free engine leaves into packages/ad-replay Stage A of the #1478 P5 extraction: vars, plan-digest (+canonical-json, sole consumer), the target-identity classification core, report-action, and suggestion-ranking move verbatim; imports updated. The package facade temporarily re-exports the moved symbols so root consumers keep compiling; a later stage narrows it to inspectAdReplay/runAdReplay only. * chore(layering): register packages/ad-replay in the workspace and DAG * refactor(replay): define the three-operation replay selector port with dual adapters (#1478 P5) * refactor(daemon): route replay handlers through the selector port (#1478 P5) * refactor(replay): split target verification into engine policy and daemon authority (#1478 P5) * refactor(replay): move the .ad step loop behind inspectAdReplay/runAdReplay (#1478 P5) * refactor(replay): lock the ad-replay façade to its real consumers (#1478 P5) * test(replay): prove shared-id demotion on both selector-port adapters (#1555 review) * fix(replay): restore invalid replayBackend rejection on the native path (#1555 review) * refactor(replay): move shared .ad vocabulary to its owner, packages/ad-script (#1555 review) * refactor(replay): neutral step/run outcomes and digest/resume behind inspectAdReplay (#1555 review) P1 "do not smuggle daemon wire failures through a generic": drop the TResponse generic from AdReplayStepRuntime/runAdReplay. executeStep and handleActionFailure now return neutral tagged AdReplayStepOutcome/ AdReplayStepFailure values (kind/message/artifactPaths only); runAdReplay returns a neutral completed/failed AdReplayRunOutcome. The engine never holds or returns a DaemonResponse. The daemon adapter (createAdReplayStepRuntime, session-replay-runtime.ts) keeps its real wire response in a local side-map as it builds each neutral outcome, and runReplayScriptFile reads it back once runAdReplay reports which step failed, so the final response is byte-identical to before this split. P1 "parsing/planning/digest/resume must also occur behind runAdReplay": relocate computeReplayPlanDigest's call site and the --from/--plan-digest resume-point math (resolveReplayEntryIndex) behind inspectAdReplay's manifest as planDigest and a resolveEntryIndex closure. Neither is a new top-level export -- inspectAdReplay/runAdReplay stay the only two. Timing is preserved exactly (still called eagerly in prepareReplayPlan, before prepareReplaySession's coordinator-mutating side effects) since moving resume validation to run inside runAdReplay itself would let a rejected --from request mutate coordinator/session state first -- a real ordering hazard, not just a cosmetic one. computeReplayPlanDigest/ReplayPlanDigestMetadata/resolveReplayEntryIndex leave the ad-replay façade; request-router-repair-expired.test.ts and prepareReplayPlan read the digest/resume result off the manifest instead. * refactor(replay): relocate classifyTargetBindingMatch and pin the ad-replay façade (#1555 review) P1 "complete the binding façade instead of documenting deviations": classifyTargetBindingMatch never had a real consumer reachable through inspectAdReplay/runAdReplay -- both its callers (the daemon's record-time self-check in session-target-evidence.ts and its replay-time classification wrapper in session-replay-target-classification.ts) are daemon files that imported it directly. It interprets TargetAnnotationV1 evidence semantics shared beyond the engine, so it moves to packages/ad-script alongside target-annotation-identity.ts (new target-annotation-classification.ts + its test), and both daemon call sites now import it from there instead of @agent-device/ad-replay. One deviation remains and is reported rather than papered over per the review's own instruction: the four target-verification policy functions (planPreDispatchTargetVerification, planPostResolutionTargetVerification, deriveReplayTargetGuardMismatchEvidence, deriveWaitLandmarkMismatchEvidence) and the ReplaySelectorPort type family stay exported. Their sole caller, session-replay-target-verification.ts, interleaves these pure decisions with daemon-only async work (capture, SessionStore, coordinator/resume stamping, wire shaping) that must stay outside the engine by design; moving their call sites to live only behind runAdReplay would require restructuring that whole orchestration into new fine-grained AdReplayStepRuntime capabilities, which is out of scope for this pass. See packages/ad-replay/src/index.ts's header comment for the full reasoning. P1 "add the reviewer-required exact exported-symbol gate": adds readNamedExports (scripts/layering/package-boundaries.ts), a small parser over a façade's `export { .. } from`, `export type { .. } from`, and direct-declaration forms, and pins @agent-device/ad-replay's exact 21-symbol export list in package-boundaries.test.ts. Plant-verified: a stray `export const` addition failed the assertion; removed it and the gate went green again. * refactor(replay): drive target verification from the engine step loop (#1555 review) Moves the verify-then-dispatch decision flow into packages/ad-replay's step loop so the four target-verification policy functions (plan{PostResolution,PreDispatch}TargetVerification, derive{ReplayTargetGuardMismatch,WaitLandmark}MismatchEvidence) become engine-private and leave the ad-replay façade. The daemon (session-replay-target-verification.ts) shrinks to the narrow AdReplayStepRuntime capabilities the engine drives: routing (beginTargetVerification), capture (captureObservation), classification (classifyTarget), dispatch (dispatchStep), and wire-building (buildRecordedUnverifiableFailure, buildTargetBindingFailure, buildPostDispatchTargetBindingFailure). Wire output and replay-compat stay byte-identical; the exact-symbol façade gate is updated to the shrunken export list. * refactor(daemon): decompose the replay adapter's two over-threshold functions (#1555) * refactor(replay): fold #1554's keep-session terminal-lifecycle policy into the ad-replay engine Rebasing p5/extract-ad-replay onto main pulled in #1554's --keep-session feature, which had grown its own daemon-side terminal-close-suppression predicate (session-replay-terminal-lifecycle.ts's resolveSuppressedTerminalCloseIndex/countExecutedReplayActions) independently of this branch's own engine-side one (step-loop.ts's isRepairArmedTerminalCloseAction). Both are the same decision family — replay --keep-session and an active --save-script repair now share ONE structural resolution (resolveSuppressedTerminalCloseIndex, generalized to "terminal among EXECUTABLE actions" rather than the old physical-last-index check) and one suppression check inside runAdReplay, gated on keepSession OR runtime.isRepairArmed(). AdReplayRunRequest grew a keepSession field; the neutral 'replayed' count in AdReplayRunOutcome is now computed inline in the loop instead of the daemon's old actions.length - entryIndex approximation. requireLiveSessionForKeepSession (the --keep-session live-session postcondition) stays daemon-side, inlined into session-replay-runtime.ts, since it inspects SessionStore state the engine never sees. The daemon-only session-replay-terminal-lifecycle.ts this arrived with is deleted entirely — its isExecutableReplayAction was a duplicate of the engine's own. runReplayScriptFile's Maestro-format routing (including the new --keep-session Maestro rejection) was extracted into routeMaestroReplay to keep the function under fallow's complexity threshold after re-threading keepSession through it. Added packages/ad-replay/src/internal/__tests__/step-loop.test.ts covering the unified suppression decision (both keepSession and repair-armed) directly against runAdReplay, including the terminal-among-executable-actions case with a trailing nested replay marker. The daemon-level integration tests (6 tests in session-replay-terminal-lifecycle.test.ts, exercising the same behavior through runReplayScriptFile) and the SDK provider-scenario test (active-session-script-publication.test.ts) needed no changes and pass unmodified. * refactor(daemon): decompose session-replay-runtime.ts into three modules (#1555) Splits the ~1096-line replay runtime into cohesive pieces, keeping session-replay-runtime.ts as thin orchestration (~240 LOC): - session-replay-runtime-engine-adapter.ts: the AdReplayStepRuntime adapter (createAdReplayStepRuntime, the build*Failure capability implementations, and the lastResponse/lastObservation side-map mechanics), extracted verbatim. - session-replay-runtime-plan.ts: extended with the plan-side helpers (validateReplayBackendFlag, inspectReplayPlanManifest, resolveReplayPlanEntryIndex, prepareReplayPlan, routeMaestroReplay) alongside the buildReplayMetadataFlags helper already there — buildReplayMetadataFlags is now module-private since its one caller moved into the same file. Also introduces ReplayScriptFileParams, named here (instead of derived via Parameters<typeof runReplayScriptFile>) so routeMaestroReplay can reference the shape without importing back from session-replay-runtime.ts. - session-replay-runtime-session.ts (new): session preparation (prepareReplaySession and its coordinator arming/repair-preflight helpers), extracted verbatim. Coordinator ownership is unchanged: createReplayCoordinator is still constructed only in session-replay-runtime.ts, matching replay-coordinator-ownership.test.ts's allowlist as-is — every extracted module receives the already-constructed ReplayCoordinator as a parameter. Pure move; no behavior change. * test(replay): cover pre-step artifact ordering and resume-before-mutation (#1555) Two invariants found during the P5 decomposition pass now have direct counterfactual-verified coverage: - packages/ad-replay/src/internal/__tests__/step-loop.test.ts: a post-dispatch target-binding mismatch (dispatchWithGuard) must report the accumulated PRE-step artifact snapshot it was called with, never the artifacts the failed dispatch itself produced. Verified red by swapping the buildPostDispatchTargetBindingFailure call to outcome.artifactPaths. - src/daemon/handlers/__tests__/session-replay-runtime-plan.test.ts: a rejected --from/--plan-digest resume must never reach prepareReplaySession's coordinator-mutating writes (the R2 ordering invariant) — a pre-armed repair transaction and corrective-resume watermark are asserted byte-for-byte unchanged after rejection. Verified red by calling prepareReplaySession before honoring the plan-validation rejection. * fix(ad-replay): enforce the exact two-entrypoint facade (#1555 review P1) packages/ad-replay/src/index.ts now exports exactly two value symbols, inspectAdReplay and runAdReplay, and zero types — formatReplaySuccessMessage (presentation) moves beside its one caller in session-replay-runtime.ts, and every type a root daemon file needs is derived structurally off the two entrypoints in the one new src/daemon/ad-replay-facade-types.ts module instead of being named off the façade. scripts/layering/package-boundaries.ts's readNamedExports is rewritten on oxc-parser's own static-export table instead of a regex, so it can no longer silently miss a widening export form: a bare `export *` re-export or an `export default` now throws (an un-enumerable, and therefore un-pinnable, export), while `export * as ns` and every other enumerable form is still counted. The pinned exact-symbol assertion in package-boundaries.test.ts is narrowed to ['inspectAdReplay', 'runAdReplay']. * fix(ad-replay): translate wire failures before the engine boundary (#1555 review P1) AdReplayDispatchOutcome's guard-mismatch/landmark-mismatch variants carried a generic `details: Record<string, unknown> | undefined` bag straight off the wire response — a daemon wire projection crossing into the engine even though the outcome itself was already a neutral type. The daemon adapter (session-replay-runtime-engine-adapter.ts) now narrows that bag into the typed AdReplayGuardMismatchEvidence/AdReplayLandmarkMismatchEvidence shapes (observed identity, expected/observed structural denotation, ancestry entries, match count) before returning the outcome; the unknown-parsing readers move there with the wire-reading responsibility they always were. target-verification.ts's deriveReplayTargetGuardMismatchEvidence/ deriveWaitLandmarkMismatchEvidence now consume only the typed values — no `unknown`-valued record type remains on any engine-crossing signature. * fix(ad-replay): move variable semantics/planning behind runAdReplay (#1555 review P1) The daemon assembled the `${VAR}` scope (buildPreparedReplayScope) and interpolated actions at two independent call sites: dispatch's own (invokeReplayAction) and target verification's separate one (resolveTargetVerificationEntry) — duplicated orchestration the P5 design assigns to the engine. runAdReplay's request now carries the raw scope INPUTS (varSources: plain builtins/file/shell/cli-env data, plus actionLines/actionSourcePaths/ resolvedPath for interpolation-error location) instead of a built scope; the engine builds the scope and resolves each action exactly once per step, handing the RESOLVED action to dispatchStep/beginTargetVerification while every other capability still receives the ORIGINAL recorded action (a target-binding divergence reports the recorded selector, never an expanded ${VAR}). This is the one resolution site now — session-replay-action-runtime.ts's invokeReplayAction and session-replay-target-verification.ts's resolveTargetVerificationEntry no longer hold a scope or call resolveReplayAction themselves. Scrub-value collection (collectReplayScrubbableVarValues, for divergence-report redaction) is kept single-sourced in the engine too: it's computed from the engine's own live scope and threaded to each build-failure/handleActionFailure capability as an explicit scrubVars argument, rather than the daemon recomputing it from a second scope object (which would have gone stale, since expandedBuiltinNames tracking now only happens engine-side). The Maestro replay path's own daemon-side vars usage is unrelated (a different engine) and is out of scope here. * fix(ad-script): make ${VAR} interpolation a linear scanner CodeQL flagged the interpolation regex's fallback group as js/polynomial-redos once vars.ts moved into packages/ (library-input classification): every ${NAME:- prefix of an unclosed input rescanned to end-of-string, quadratic overall — 1,857 ms measured on 20k repetitions of '${A:-['. Replaced with a single-pass scanner; failed fallback scans emit their span verbatim and resume after it (escape-pair alignment is identical from every candidate start inside the span, so no later candidate can terminate where the failed scan could not). Equivalence: 200k-trial differential fuzz against the retired regex over the adversarial alphabet, zero mismatches; both adversarial shapes now resolve in 1-2 ms. * refactor(ad-replay): typed façade replaces the zero-type rule (#1555 structural-quality review) Reverses the exact-two-value zero-type export shape #1555's second review pass established: it forced every root type derivation through one shim (src/daemon/ad-replay-facade-types.ts) and left four daemon-side twin types (TargetVerificationEntry, TargetClassificationOutcome, TargetBindingFailureEvidence, ReplayVerifiedTargetGuard) plus a toDaemonEvidence copy translator shadowing the engine's own shapes. packages/ad-replay/src/index.ts now exports inspectAdReplay/runAdReplay (unchanged, still the only two values) plus the neutral vocabulary their signatures are built from, by name — following packages/maestro's façade precedent. The exact-symbol gate in scripts/layering/package-boundaries.test.ts is widened to pin the full sorted list (values + types). The four daemon twins are deleted; session-replay-target-verification.ts and session-replay-runtime-engine-adapter.ts now use the engine's own AdReplayVerificationEntry/AdReplayTargetClassification/ AdReplayTargetBindingEvidence/AdReplayVerifiedTargetGuard directly. TargetBindingDivergenceBuilt's array fields are now readonly-compatible, so toDaemonEvidence's copy is gone — evidence flows through unchanged. * fix(ad-replay): honor the selector port's own contract in the parse gate target-verification.ts's planPreDispatchTargetVerification used resolveRecordedTarget (operation 2, resolve) over an empty node tree purely to read its parse-invalid reason — a resolve call standing in for a parse call, even though readSelectorExpression (operation 1, parse) exists to answer exactly that question and was already unused inside the engine. Replaced with port.readSelectorExpression('ordinary', [token]). The mapping is not 'invalid' -> skip: production's 'ordinary'/'wait' grammars only ever record a boundary once it has already parsed, so a single malformed token can only come back 'not-applicable' there ('invalid' is unreachable from this call site on the production adapter). Both non-'expression' outcomes map to skip, matching the historical behavior (a single parse-invalid reason covered both cases). platform dropped from the function's params — it was only ever threaded to the resolve call this replaces. Added a contract-suite cell pinning the exact (diverging) discriminant each adapter reports for a selector-shaped-but-malformed bare token, and why the divergence is harmless for the one real consumer. * refactor(ad-replay): split step-loop.ts and shrink the daemon adapter (#1555 structural-quality review) step-loop.ts (810 LOC) splits three ways, following packages/maestro's own precedent: - internal/runtime-port-types.ts: the AdReplayStepRuntime boundary vocabulary (all the neutral types the engine/daemon exchange). - internal/verify-dispatch.ts: verifyAndDispatchStep + its dispatchNoGuard/ dispatchWithGuard helpers. - internal/step-loop.ts: runAdReplay itself plus the terminal-close/ executable-action structural logic (isExecutableReplayAction, resolveSuppressedTerminalCloseIndex). packages/ad-replay/src/index.ts's type exports now source from runtime-port-types.ts. step-loop.test.ts's AdReplayStepRuntime import moves to the new path (no assertion changes). src/daemon/handlers/session-replay-runtime-engine-adapter.ts (553 LOC after item 1's twin removal) shrinks to 294 via two further extractions: - session-replay-dispatch-narrowing.ts: the wire `details` bag -> typed evidence narrowing and dispatch-failure classification. - session-replay-runtime-step-support.ts: ReplayStepContext (moved here to avoid a cycle with the adapter, which re-exports it by name) plus the failure-wrapping/diagnostics-support helpers. Final LOC: adapter 294, dispatch-narrowing 148, step-support 153, step-loop 225, verify-dispatch 246, runtime-port-types 374. * test(ad-replay): package-local tests for resume.ts/target-verification.ts + terminal-lifecycle test rename resume.test.ts covers resolveReplayEntryIndex directly (previously only exercised transitively through the daemon's session-replay-runtime-plan tests): no --from/--plan-digest, the paired-flags requirement, in-range --from, out-of-range rejection, stale-digest rejection, the authorized empty-tail boundary (actionCount + 1) gated on a matching watermark, and the unperformed-record-and-heal growth check. Counterfactual run and restored: widening describeOutOfRangeResumeFrom's bound turns the out-of-range/ empty-tail-without-watermark assertions red (2 failures observed). target-verification.test.ts covers all four engine policy functions directly: the two plan* pre-capture gates and the two derive* post-dispatch evidence builders, including item 2's own new decision surface (a fake ReplaySelectorPort proving both non-'expression' readSelectorExpression outcomes map to skip). Counterfactual run and restored: narrowing the check to the literal `'invalid' -> skip` reading turns the 'not-applicable' case red (reports recorded-unverifiable instead of skip). session-replay-terminal-lifecycle.test.ts renamed to session-replay-runtime-keep-session.test.ts: its production module (session-replay-terminal-lifecycle.ts) was already deleted by the #1554 fold-in, and its six cases drive the full runReplayScriptFile round trip against a real SessionStore (including daemon-only postconditions the engine's step loop never reaches) rather than testing engine policy through the façade in isolation — the engine's own terminal-close-suppression decision already has direct, cheaper coverage in step-loop.test.ts. No assertion changes; both files' header comments cross-reference the split. * refactor(ad-replay): compute scrub values once per step, one name end to end collectReplayScrubbableVarValues(scope) was called fresh at 5 separate return points inside one verifyAndDispatchStep invocation plus once more in handleActionFailure — always the same result, since nothing between them mutates scope. step-loop.ts's runAdReplay now computes scrubVars ONCE per step, right after resolveReplayAction (the one call that can grow the scope's expanded-builtins set), and threads it as a plain readonly AdReplayScrubValue[] value; verify-dispatch.ts no longer imports ReplayVarScope or collectReplayScrubbableVarValues at all. "One name" end to end: the daemon's TargetBindingDivergenceContext.scrubVars and withReplayFailureDiagnostics's scrubVars param used a separately-derived ReturnType<typeof collectReplayScrubbableVarValues> (mutable array) instead of the engine's own AdReplayScrubValue, requiring a [...scrubVars] copy at every daemon call site to satisfy the mutable-array type. Both now use readonly AdReplayScrubValue[]/readonly ReplayVarScrubEntry[] (structurally identical, already readonly-safe downstream — scrubReplayVarValues and createReplayDivergenceSanitizer already accepted readonly arrays), so the four [...scrubVars] copies in session-replay-runtime-engine-adapter.ts are gone. * fix(daemon): make lastObservation genuinely per-step, not per-run createAdReplayStepRuntime's lastObservation closure lives for the whole replay run (one factory call covers every step), but was never reset between steps. Every current buildTargetBindingFailure call site happens to be preceded by this same step's own captureObservation, so the `lastObservation ?? { reason: 'observation-missing' }` fallback could never actually fire — but if it ever did (a future call path reaching buildTargetBindingFailure without capturing first), it would silently attach the PREVIOUS step's screen instead of reporting the missing-capture condition the fallback message claims. armStep runs exactly once per step, before any of that step's other capabilities (verified against step-loop.ts's runAdReplay loop order) — the natural per-step boundary. It now clears lastObservation first. No behavior change on any reachable path today (full daemon + ad-replay suite: 1766/1766 green); an unrelated device-claim-prune contention flake was observed once and did not reproduce on isolated or full-suite reruns. * docs(ad-replay): fix decayed review-changelog comments naming defunct symbols Four comments named symbols/paths that no longer exist, left behind by earlier review passes describing PR history rather than the current constraint: - session-replay-runtime-step-support.ts / session-replay-runtime.ts (2 sites): referenced a function called executeStep, which was never reintroduced under that name after the P5 split — the actual mechanism is the runtime's dispatch/build-failure capabilities recording into the lastResponse side-map. - session-replay-runtime.ts: referenced an engine collectArtifactPaths capability that does not exist — artifactPaths is a daemon-side Set the adapter mutates via collectReplayActionArtifactPaths. - packages/ad-replay/src/internal/selector-port.ts: pointed at ./testing/in-memory-selector-port.ts, the in-memory adapter's pre-stage-D location — it has lived at src/__tests__/test-utils/in-memory-replay-selector-port.ts since. - session-replay-repair-hint.ts / session-replay-runtime-step-support.ts (2 sites): named target-identity.ts, which does not exist (the real file is target-identity-node.ts); the second site additionally mislabeled classifyReplayTarget as engine-side when it is daemon-side (session-replay-target-classification.ts). Comment-only; no behavior change. * refactor(ad-script): move declaredScriptPlatform to its natural shared owner packages/ad-replay/src/internal/inspect.ts's declaredScriptPlatform and src/daemon/replay-device-selection.ts's readScriptReplaySelection each kept their own copy of the same "platform declared before the first open" scan over runtime/open actions — .ad script semantics, not engine or daemon policy, needed independently by ad-replay's plan-digest precedence and the daemon's device-selection platform resolution. Verified this was a genuine duplicate (not the single-sourced state I initially reported): readScriptReplaySelection's platform-tracking loop computes the identical result via a differently-shaped traversal fused with its own app-target scan. resolveDeclaredScriptPlatform now lives in packages/ad-script (its natural owner: the one package both ad-replay and the daemon already depend on, avoiding the R11 issue that justified the original duplication). The daemon's app-target scan stays its own separate pass; fusing it back into the shared function would smuggle a daemon-only concern into ad-script for no measurable cost (the actions array is small, and the shared function already stops at the same point the app-target scan needs to look). * docs(ad-replay): fix package.json description to match the current façade Described "target-identity, variable substitution, plan-digest, and report primitives" — the wide pre-#1555-review façade shape. Vars/identity/report vocabulary moved to ad-script/daemon across the P5 and #1555 review passes; the package now exports exactly inspectAdReplay/runAdReplay plus the neutral AdReplayStepRuntime vocabulary. Description updated to match. * refactor(daemon): fold the step-support fragment back into the engine adapter A simplicity audit judged session-replay-runtime-step-support.ts a size-target fragment, not a concern boundary: four unrelated concerns, one consumer, and a header comment admitting it existed to satisfy the <300 LOC metric. Folded back; the previously-exported helpers are module-private again; the adapter's honest size is renegotiated from the plan metric (dispatch-narrowing stays extracted — it has one nameable job).	2026-08-03 17:25:13 +02:00
Michał Pierzchała	95b4623461	0.20.4	2026-08-03 16:49:15 +02:00
Michał Pierzchała	b9509fe006	refactor: extract the .ad script codec into packages/ad-script (#1478) (#1536) ... * refactor: extract the .ad script codec into packages/ad-script Moves the mutually-coupled .ad read/write codec (script.ts, script-utils.ts, script-formatting.ts, open-script.ts) plus the target-v1 annotation SERDE slice of target-identity.ts into a new private leaf package, @agent-device/ad-script, exporting only `.`. This is option 1 from the P5 scoping dossier on #1478: the codec is shared by the daemon's session-script publication writer, the future replay engine, the CLI's `replay export`, and Maestro's failure-label formatting, so it can no longer live in root src/ once packages/ad-replay lands (R11 forbids a package reaching into root src), and a second export subpath or writer-half duplication are both ruled out by existing gates/tests. target-identity.ts keeps only the record/replay-shared classification core (classifyTargetBindingMatch, local-identity/ancestry-prefix matching), importing its shared types from the new package. Every real consumer (re-derived by grep, not the dossier's list alone) is rewired to @agent-device/ad-script. Refs #1478 Co-Authored-By: Claude <noreply@anthropic.com> * refactor: trim the ad-script façade to real consumers, lock the one-export boundary - packages/ad-script/src/index.ts: drop parseReplaySeriesFlags, formatTargetAnnotationCommentLine, parseTargetAnnotationCommentLine, TargetAnnotationLineParseResult, and TargetRect from the public façade — none has a consumer outside the package (re-swept every remaining export by grep; everything else kept has at least one real external importer). The functions/types stay exported from their declaring internal modules for the package's own internal use (script.ts, script-formatting.ts). - scripts/layering/package-boundaries.test.ts: add the parallel R11 assertions "the real tree parses, declares, and passes R11" already makes for maestro/provider-webdriver/provider-limrun/xml — ad-script exports exactly `.`, depends on exactly contracts+kernel, and is declared in root package.json — plus ad-script entries in the deep-resolution rejection coverage. Verified the lock catches a regression: temporarily added a fake `./codec` export to packages/ad-script/package.json and confirmed both the export-key-list assertion and the deep-resolution-rejection assertion fail; removed the plant and reconfirmed green. Refs #1478 Co-Authored-By: Claude <noreply@anthropic.com> * fix: remove polynomial-redos ambiguity from the target-v1 annotation line regex CodeQL js/polynomial-redos flagged TARGET_ANNOTATION_LINE_RE (packages/ad-script/src/internal/target-annotation-serde.ts): the payload group's `\s+(.*)` let `\s+` and the unconstrained `.*` both match whitespace, so a run of separator whitespace that ultimately fails to complete the match has many `\s+`/`.*` splits to backtrack through before concluding failure. Anchor the payload group on `\S` (the exact complement of `\s`), so the mandatory `\s+` separator and the payload's first character can never overlap — the split point becomes unique and no backtracking is possible. Behavior-preserving: the only caller (parseTargetAnnotationCommentLine) always matches against an already-.trim()-ed line, whose last character (whenever the tag matches at all) is never whitespace — so a payload section `\S.*` would reject (content that is entirely whitespace) can never reach this regex through the real call path. Verified against the frozen replay-compat corpus and the full serde/parser test suites, unmodified. Added a regression test with the exact adversarial shape CodeQL/the reviewer cited (many tab pairs after the version digits), asserting sub-second parse. Refs #1478 Co-Authored-By: Claude <noreply@anthropic.com> * test(ad-script): pin the annotation-line pattern's linear rejection directly The entry-point adversarial case matched greedily even with the retired regex (trim strips edge whitespace and per-line input carries no newline), so it proved nothing about the pattern. The regression surface is the pattern itself: an interior tab run with an x-newline tail fails the match, which the retired form re-split quadratically (3.7s at 100k tabs) and the \S anchor rejects in one attempt. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-01 20:21:38 +02:00
Michał Pierzchała	cbe1a57094	refactor(replay-test): extract packages/replay-test (#1478 P3b) (#1525) ... * refactor(replay-test): source the manifest device vocabulary from the kernel `session-test-types.ts` reached `ReplayScriptMetadata['platform']` and `['target']` through `replay/script.ts` — the native `.ad` engine. A format-neutral scheduler must not name an engine module, and P5 relocates that engine into `packages/ad-replay` regardless, so the import had to go before the scheduler can move. Both members already resolve to neutral kernel types (`Exclude<PlatformSelector, 'web'>` and `DeviceTarget` from `@agent-device/kernel/device`), so this re-sources them directly and the manifest shape is unchanged. Only the import direction differs. First increment of P3b; the scheduler still has request-global, engine and daemon imports to port before the physical move. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): inject the progress sink instead of reading a request global The scheduler called `emitRequestProgress` in eight places, which reads a sink out of a request-global `AsyncLocalStorage`. That is ambient authority a format-neutral scheduler cannot hold once it lives in `packages/replay-test`, and #1505 recorded it as a shrink-only R10 entry. The host now injects the capability through the existing `ReplayTestRuntimeDependencies` seam established in P3a, so no new seam is invented. `session-replay.ts` supplies `emitProgress: emitRequestProgress`; `src/request/progress.ts` keeps the sink and its AsyncLocalStorage binding for every other caller. The port is deliberately narrower than `RequestProgressSink`: it accepts only `ReplayTestSuiteProgressEvent | ReplayTestProgressEvent`, so the scheduler is not handed the ability to emit `CommandProgressEvent`. Authority narrows again one hop down: `runReplayTestAttempt` spread the whole dependency bag but uses three of its members and never publishes progress, so it now takes `Pick<..., 'runReplay' | 'cleanupSession' | 'finalizeAttempt'>`. That is why no runtime test fixture needed changing — the attempt runtime never gained the capability in the first place. Also drops the last two `replay/script.ts` type references from `session-test-runtime.ts`, so the engine import is gone from that file too. Reporter contract preserved: `session-test-reporter-values.test.ts` and `session-test-reporter-values-maestro.test.ts` both pass unmodified (27 tests green across the five scheduler suites). Typecheck clean. Remaining scheduler boundary for P3b: `request/cancel.ts`, `replay/format.ts`, `replay/script.ts` in discovery, `session-store.ts`, `daemon/types.ts`, `replay-source-discovery.ts`, `core/dispatch*`, `utils/diagnostics.ts`. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): ask the host whether the suite is canceled The scheduler called `isRequestCanceled(requestId)` in five places. That both reaches a request-global registry and forces the scheduler to name a daemon request id as the cancellation key — neither survives the move into `packages/replay-test`. The host now binds the predicate to its own request and passes `isCanceled: () => boolean`. The scheduler asks a question it is entitled to ask and learns nothing about how cancellation is tracked. `shouldStopReplayTestExecution` takes the capability rather than a request id, so no scheduler function threads a daemon identifier for this purpose any more. `session-test-attempt.ts` and `session-test.ts` no longer import `request/cancel.ts` at all. It remains in `session-test-runtime.ts`, which does something different — `registerRequestAbort`, `markRequestCanceled` and the parent-abort relay are cancellation *binding*, which the brief assigns to the daemon adapter, so that split is its own step. Behavior preserved: both pinned reporter characterizations pass unmodified, 32/33 across the five scheduler suites. The one failure is the pre-existing P2/#1506 discovery-ordering regression, unrelated and untouched here. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): drop the dead request-tracking call from attempt ids `buildReplayTestAttemptRequestId` wrapped its template in `resolveRequestTrackingId`, pulling `request/cancel.ts` into the scheduler. That wrapper substitutes a generated id only when its first argument is an empty string. The template here always contains `:test:`, so it is never empty and the wrapper always returned it unchanged — the call is unreachable in this path. Probed all three shapes (explicit request id, suite-id fallback with a shard, and degenerate empty inputs); every one returns the template verbatim. Removing it takes `request/cancel.ts` out of discovery without altering a single produced id. The scheduler mints attempt identity itself, which is what the brief asks for. Evidence the ids are byte-identical: the pinned reporter characterizations assert exact session strings such as `default:test:suite-reporter:1-02-retry:attempt-1` and pass unmodified — 30 tests green across the reporter, suite and discovery suites. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): move cancellation binding and diagnostics to the host `session-test-runtime.ts` held the last two request-globals in the scheduler: `request/cancel.ts` (registerRequestAbort, markRequestCanceled, clearRequestCanceled, plus the parent-abort relay) and `utils/diagnostics.ts`. These are different in kind from the earlier ports. The brief gives the daemon adapter the job of mapping an attempt id to daemon request identifiers and *binding cancellation*, while timeout policy stays scheduler-owned. So the scheduler now receives a per-attempt capability with exactly two verbs — `cancel()` on timeout and `release()` when the attempt settles — and every registry interaction, including `relayReplayTestAbortFromParent`, moved to `session-replay.ts` next to the rest of the adapter. Diagnostics became a narrow publish capability for the same reason: `emitDiagnostic` reads a request-global scope. The level vocabulary is spelled out at the seam rather than imported, so nothing engine- or daemon-shaped crosses it. The runtime fixtures drive the real exported host binding rather than a stub. They assert cancellation through `isRequestCanceled`, and a stubbed binding would have kept those assertions passing while proving nothing. 24 tests green across the runtime, suite and both reporter characterizations, which pass unmodified. Typecheck, lint and oxfmt clean. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): split discovery into host inspection and scheduler policy discoverReplayTestEntries expanded paths, read every file, and called both engines — readReplayScriptMetadata for .ad, inspectMaestroFlow for Maestro — plus resolveReplayFormat to choose between them. Four imports a format-neutral scheduler cannot hold. Inspection is now the host's discoverSources capability. What stays in the scheduler is the genuinely neutral half: which sources a --platform filter runs, which it skips and with what message, and the empty-suite error. The manifest carries exactly the four fields the scheduler consumes (platform, target, retries, timeoutMs) plus the reporter's title, per the brief's instruction not to add more without a demonstrated call site. The platform tag is what removes the last format leak. The filter used to ask resolveReplayFormat(...) === 'maestro' to decide whether a missing platform was disqualifying. It now reads a tag: caller-bound means the invocation supplies the platform, unspecified means the source declared none. Maestro is what caller-bound looks like from the scheduler's side, and the format cannot be recovered from it. Discovery tests drive the real inspection capability, writing actual .ad and Maestro sources — a stubbed host half would have kept them green while proving nothing about the composition they exist to pin. 35 tests green across discovery, suite, runtime and both reporter characterizations, which pass unmodified. The Maestro one is the direct check that titles still flow, since they now arrive via the manifest. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): build attempt ids from named segments; trim comments Review feedback on the attempt-id builder: the comment explained a deletion that git already records, and it sat above an opaque template literal. The id is now a segment list joined on ':', so its shape is readable without prose. Output is byte-identical — the reporter characterizations assert exact session and attempt strings and pass unmodified. Applied the same standard to four other docblocks in this PR that narrated what the code used to do rather than what it does. The durable 'why' stays: which side of the seam owns what, and why the vocabulary is neutral. The migration history goes, since git carries it and these docblocks will outlive the migration. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): move shard device binding to the host buildReplayTestShardPlan called listDeviceInventory to discover what to shard across, and buildReplayTestShardFlags constructed daemon CommandFlags for the nested request. Inventory enumeration, allowlists, simulator set paths, explicit --device selectors and the too-few-devices error are host concerns; what is scheduler-owned is deciding how many shards exist and which entries each one runs. The scheduler now receives resolved shard targets through a capability. The target is neutral: id and name for session labels and progress metadata, plus platform and target, which are already kernel vocabulary. DeviceInfo no longer crosses into scheduling. One behavior note: an explicit --device selector could in principle name a web target, which is not a shardable device. That is now rejected with INVALID_ARGS rather than widening the neutral platform vocabulary to carry something the scheduler can never run. Implicit selection already filtered to mobile. 919 of 920 handler tests pass. The one failure, session-test-runner.test.ts 'binds each replay script to its declared platform metadata', fails identically on clean origin/main in this container and is unrelated: directory discovery walks with opendirSync/readSync and directory results are deduped but not sorted, while glob results are sorted, so suite order is filesystem-dependent. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(replay-test): extract packages/replay-test behind a façade Completes the P3b extraction. The scheduler, attempt runtime, discovery policy, sharding distribution, artifacts and neutral types now live in packages/replay-test/src/internal/, with one package-root export. The façade takes a neutral ReplayTestSuiteRequest and returns a tagged ReplayTestSuiteOutcome. DaemonRequest, DaemonResponse and CommandFlags no longer reach the scheduler; the adapter translates flags and meta in, and the outcome back to a daemon response. Eight flags were read by the scheduler and each became a field it owns. Host work moved to daemon adapters: source inspection (both engines and format routing), shard device binding and shard-flag parsing, and artifacts-dir home expansion, which is why the package can resolve paths without SessionStore. The one remaining shared concern was the timing trace: the host writes video lifecycle events into the same trace the scheduler owns. Rather than export a writer from the façade, each attempt hands the host an appendTimingEvent closure, so the trace format stays private and the authority is scoped to that attempt. Tests mirror the topology. Discovery tests split along the seam they now cross: ordering, traversal and routing are pinned host-side against real files, filtering policy is pinned in the package against fake sources. The runtime tests assert the scheduler's cancellation obligation (cancel once on timeout, always release) against a recording binding, and a new daemon test pins the adapter's half — registry entries, the parent-abort relay, and detach on release — so that coverage moved rather than disappeared. R10 retargeted to packages/replay-test/src/ and the zone ranked alongside maestro. R11 confirms zero root-src imports from the package. 914 of 915 handler and package tests pass. The one failure, session-test-runner 'binds each replay script to its declared platform metadata', fails identically on clean main here: directory discovery walks with opendirSync and dedupes without sorting, while globs sort, so suite order is filesystem-dependent. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * refactor(daemon): simplify replay-test request translation Fallow flagged toReplayTestSuiteRequest at 14 cyclomatic in 18 lines. The branches were self-inflicted: every req.flags?.x is one, and each optional field was written as a conditional spread to avoid setting an undefined key. exactOptionalPropertyTypes is not enabled, so assigning undefined to an optional field is equivalent and the spreads bought nothing. Destructuring flags once and extracting two flag readers removes most of the rest. One correctness note on the simplification itself: the first version used `artifactsDir && expandHome(...)`, which returns '' for an empty-string flag where the previous code called expandHome(''). Replaced with an explicit undefined check so the empty-string path is unchanged. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * test(live): share the replay test-suite harness across iOS and Android Both live journeys invoked the public test command and then re-derived the same value-contract assertions by hand — suite totals, per-script status, replay counts, non-empty JUnit. Those are claims about the published suite result and are identical on every platform, and they had already drifted: iOS iterated with readReplayCommands inline, Android cast data.tests at the call site. The shared helper owns exactly that boundary. It takes the caller's runStep rather than binding a context type, so it is not a platform-configured runner and cannot template a platform's journey. Everything a platform genuinely differs on stays with the caller: which scripts run, the retry policy (iOS 2, Android none — itself a claim worth keeping), which commands each script exercises, and the behavioral evidence. Both callers keep every verify* call they had. 67 lines removed, 15 added. Residual risk: this container has no iOS or Android devices, so the live suites could not be executed here. Typecheck and lint pass; the harness needs a run on real targets before the claim that behavior is unchanged is evidence rather than inference. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * test: pin directory enumeration in the platform-binding suite test The test wrote two scripts into a temp directory and assumed discovery would return them in creation order. Directory expansion deliberately preserves filesystem order to match Maestro — only glob expansion sorts, and 'preserves Maestro directory filesystem order' pins that with a mocked opendirSync. So the ordering contract is correct; this test's assumption about enumeration was not. It passes on CI, where small directories usually enumerate in creation order, and fails on filesystems that do not — identically on clean main, where the platform-to-script binding appears reversed. Pinning enumeration the way the discovery tests already do keeps the subject intact (each script binds to ITS declared platform, and session numbering follows discovery order) without depending on the host filesystem. The fs import became a default import because vi.spyOn cannot redefine an ESM namespace export. 915 of 915 handler and package tests now pass here. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * test: scope the enumeration spy and restore it in a finally The spy I added restored only on the happy path and asserted on its argument inside the mock implementation. Either would misfire for anything else sharing the worker: an assertion thrown from inside fs, or a leaked global opendirSync, surfaces as a worker crash with no failed test rather than a readable failure. It now delegates to the real implementation for any directory but this suite's own, and restores in a finally. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * fix(replay-test): put package tests where they are actually run Review found the moved package tests were neither executed nor typechecked. They sat under packages/replay-test/test/, but vitest's unit-core lane includes packages/*/src/**/*.test.ts, and neither the root nor the package tsconfig covers a top-level test directory. A plain unit-core run discovered zero files under the package. That is why they looked green: my earlier runs passed those paths explicitly on the command line, which masked that the default run skipped them. The count is the proof — 550 files/4741 tests before, 553/4753 now, and the delta is exactly the three files and twelve tests that were being skipped. The runtime test also imported runReplayTestAttempt from the package specifier, which the facade does not export. It would have failed the moment it was discovered. It now imports internally, like the rest of the internal tests. Also removed replayTestAttemptFailure from the facade: zero consumers outside the package, so exporting it widened the boundary for nothing. P3 asks for a one-function facade. 553 test files and 4753 tests pass; lint and the layering guard are clean. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * style: format the facade after removing the unused export A scripted edit removed the export line but left a stray blank line; oxfmt was not re-run on that file afterward, so Lint & Format caught what pnpm lint alone does not. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * fix(replay-test): typecheck the package and fix a type-only import Review found the moved package tests were transpiled by vitest but never typechecked: the root typecheck script builds six packages via tsc -b and packages/replay-test was not among them, so its tsconfig was never used. That hid a real TS2459. session-test-runtime.test.ts imported ReplayTestAttemptOutcome from ../session-test-runtime.ts, which imports that type but does not re-export it. It now imports from ../session-test-types.ts, where the type is defined. Adding the package to the tsc -b list closes the gap. Verified empirically rather than assumed: planting a string-to-number error in a package test makes typecheck fail, and removing it makes it pass. This is the second finding of the same shape on this PR — first the tests were not discovered by vitest, now they were not covered by typecheck — so the gate was confirmed to reach the files rather than trusted to. 12 package tests pass, lint, format and the layering guard are clean, and typecheck is clean with the package included. Refs #1478 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-31 20:16:18 +02:00
Michał Pierzchała	da93191201	refactor: move Limrun provider behind package facade (#1518) ... * refactor: move Limrun provider behind package facade * fix: preserve Limrun public provider types * fix: tighten Limrun provider facade boundaries * test: harden Limrun compatibility coverage * fix: narrow Limrun public type exports * fix: narrow Limrun provider exports	2026-07-31 15:35:49 +02:00
Michał Pierzchała	b125435989	refactor: extract WebDriver provider package (#1504) ... * refactor: extract webdriver provider package * refactor: consolidate shared XML codec	2026-07-31 09:10:04 +02:00
Michał Pierzchała	0e51007b04	refactor: isolate maestro engine package (#1506) ... * refactor: isolate maestro engine package * perf: deepen maestro facade boundaries	2026-07-30 20:58:20 +02:00
Michał Pierzchała	0ee2a86129	refactor: extract contracts workspace package (#1499) ... * refactor: extract contracts workspace package * fix: preserve screenshot diff result contract * test: stabilize Android keyboard smoke	2026-07-30 17:07:46 +02:00
Michał Pierzchała	7ef83bc8bc	refactor: restore pngjs codec (#1498)	2026-07-30 14:12:50 +02:00
Michał Pierzchała	76453add71	refactor: pnpm workspace + @agent-device/kernel pilot (#1490 W0) (#1494) ... * refactor: pnpm workspace + @agent-device/kernel pilot (#1490 W0) Extend the workspace with packages/* and move the kernel behind an enforced public API: packages/kernel with nine consumer-earned subpath exports (errors, device, snapshot, contracts, collections, rect, redaction, daemon-error, bounds — the last absorbed from utils as Rect vocabulary). Every kernel import repo-wide becomes the @agent-device/kernel/<sub> specifier; kernel tests move to src/__tests__/kernel/ and exercise the package surface. The root declares the package in devDependencies (workspace:*), tsdown bundles it (noExternal) so the published artifact and its runtime dependency manifest are unchanged. Gate rewiring in the same change, per the W0 brief: - R1 kernel-sink retires (physically subsumed); new R11 package-boundaries guards no-root-back-imports, relative tunnelling past exports maps, undeclared workspace deps, and non-exported subpaths, with runtime resolution pins via import.meta.resolve. - resolveImportEdges and mutation ownership follow workspace specifiers through exports maps, keeping R4 cycle checks, depgraph, and derived test ownership connected across the seam (kernel-errors still owns 495 tests). listSourceFiles includes packages/*/src. - kernel becomes an unranked zone; mutation registry, stryker mutate globs, and the mutation-affected workflow path filter move to packages/kernel/src/errors.ts. - check:affected gains packages/ ownership (manifests fail open); vitest and coverage include packages/*/src; fallow ignores packages/** (its resolver cannot follow workspace specifiers). - The affected-selector CI job installs dependencies: its closure now crosses workspace specifiers, and the R8 relative exception is unsafe for production src files (Node ESM does not realpath, so dual specifier/relative loads would instantiate modules twice). The R8 zero-dep set is pinned empty with that rationale. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix: address W0 review — mutation sandbox, exports-map resolution, tsc -b Review findings on #1494, all five: 1. contracts-schema-public.test.ts reads the kernel source at its packages/ path (fs access invisible to the codemod and typecheck). 2. Mutation lane: Stryker sandboxes the tree but pnpm's node_modules symlink resolves @agent-device/* back to the real repo, so mutants in the sandbox never load and vitest.related finds no tests. vitest.mutation.config.ts now aliases each EXPORTED specifier to its source (derived from exports maps, never a wildcard), keeping resolution inside the mutated tree. Validated: kernel-errors module runs end to end (dry run 3,984 tests, mutants killed, exit 0). 3. Layering/depgraph resolve workspace specifiers through the exports-derived map (workspaceSpecifierTargets) instead of reconstructing paths, so '.'-facade packages resolve; the positional fallback remains only for map-less fixtures (P0 pin). 4. Per-package project references implemented: packages/kernel is composite (emitDeclarationOnly -> dist-types, gitignored), the root references it, and typecheck becomes tsc -b — probed to catch type errors on both sides under TypeScript 7 native. 5. R11's relative-route exception now requires membership in an actual R8 zero-dep job closure (zeroDepClosureFiles walks entries), not mere scripts/ placement — closing the dual-instantiation bypass. Also from review discussion: daemon-error moves out of the kernel package to src/client/ — its consumers (cli, client facade) rehydrate wire DaemonErrors client-side; the daemon only produces them. Kernel drops to 8 exported subpaths before any of them ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * refactor: one exports-map reader for mutation alias and ownership Fallow flagged workspaceExportAliases (cognitive 15, CRAP 90). The manifest-reading logic already exists as workspaceSpecifierTargets in scripts/layering/package-boundaries.ts, so both the Stryker sandbox alias table and the mutation ownership walker now consume it instead of carrying near-clones. Behavior unchanged; mutation suite 45/45 and changed-code fallow green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix: composite kernel without a root references edge FreeRange runs plain `tsc -p tsconfig.json`, and a root `references` entry makes non-build-mode TypeScript demand the referenced project's built declarations (TS6305) — a standing "build first" tax on every plain -p consumer (fr, editors). Keep the per-package composite project and build it in typecheck (`tsc -b packages/kernel` before the root and examples/sdk passes), but drop the root references edge: root consumption resolves through exports to source, identical to runtime and to the bundler. Probed: plain -p green with no prebuilt output; kernel-side type errors still caught by its own build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix: R11 uses the layering parser; mutation config is a fallow entry Review blockers on #1494: - R11's private single-quote regex could miss a double-quoted or re-export route into packages/*/src. specifierSites now delegates to the layering model's parseImports (both quote styles, side-effect imports, re-exports, dynamic imports), with direct regressions for each formerly-invisible form. - vitest.mutation.config.ts becomes a declared fallow entry instead of a tolerated unused-file finding: the full-repo audit now reports it reachable (unused files 2 -> 1; the remainder predates this PR). FreeRange clean-checkout evidence: with packages/kernel/dist-types and every *.tsbuildinfo deleted, `pnpm check:freerange` reports 0 findings on this head — the TS6305 topology died with the root references edge in the previous commit; check:freerange has no build precondition. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-30 12:12:46 +02:00
Michał Pierzchała	b4487f57a7	0.20.3	2026-07-30 10:23:47 +02:00
Michał Pierzchała	adcbdda8f0	perf: speed up unit tests and streamline checks (#1488) ... * perf: speed up unit tests and streamline checks * fix: validate canonical packaging workflows	2026-07-29 18:53:10 +02:00
Michał Pierzchała	2316fd32c5	test: pin daemon modularity migration contracts (#1487) ... * test: pin daemon modularity migration contracts * refactor: tighten daemon modularity ratchets * test: pin reporter live hook isolation	2026-07-29 18:05:00 +02:00
Michał Pierzchała	90e900a54b	0.20.2	2026-07-29 12:55:05 +02:00
devin-ai-integration[bot]	885c1486bb	test(ci): single-retry policy for enumerated contention-flaky files (timeouts only) (#1448) ... * test(ci): single-retry policy for enumerated contention-flaky files * fix: satisfy fallow * test(ci): read failures through a lane reporter so timeouts stay distinguishable * test(ci): cover the lane reporter and drop its duplicated boilerplate * chore(fallow): own the retry lane's tool-loaded export seams * test(ci): block retries on non-test failures and classify timeouts structurally * test(ci): decide retry eligibility from runner metadata and route gate verdicts through blockers * refactor(ci): name the retry policy's rules in code instead of comments * test(ci): mark runner-aborted timeouts inside the runner instead of inferring them * test(ci): make timeout provenance a per-run secret, not a writable flag Cover direct task.meta mutation in the real child-Vitest fixture gate. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ci): retry the failed files in the first run's project and coverage modes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: drop deleted repo-health file from the retry list after #1480 Rebase onto main post-#1480: the SkillGym/repo-health descope deleted scripts/repo-health/run.test.ts, whose CONTENTION_RETRY_FILES entry would now fail this PR's own missing-file check, and inlined the slow-test budgets into the reporter, resolving the budgets-module import. Envelope comments now point at scripts/lib/lane-envelope.ts instead of the closed #1430. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>	2026-07-29 12:53:33 +02:00
Michał Pierzchała	d5eb785e6f	docs: propose daemon module boundaries (#1451) ... * docs: propose daemon module boundaries * docs: preserve repair close retry state in proposal * docs: align session boundary probe with handoff	2026-07-29 12:51:57 +02:00
Michał Pierzchała	53e4be5f86	Remove SkillGym suite and repo-health snapshot infrastructure (#1480) ... * chore: drop SkillGym and the repo-health aggregator (#1412 descope) Remove the SkillGym harness (test/skillgym/), its check-affected lane, package scripts, and devDependency — the help-conformance bench is now the single non-gating small-model oracle. skills/ markdown classifies as docs in the affected-check selector instead of failing open. Remove scripts/repo-health: its only gating assertion duplicated the Layering Guard job, its case-count metric imported the deleted SkillGym suite, and its sole planned consumer (#1424 / PR #1477) was closed with the Track C descope on #1412. Verified: check-affected node --test suites, oxfmt, oxlint, tsc, check:layering, fallow audit vs origin/main, and the full unit suite (unit-core + subprocess-stub) all pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix(scripts): fold slow-test budgets into the reporter for production-exports The Fallow production-exports gate flagged all three budget exports: their in-file consumer (SLOW_TEST_RATCHET) and the repo-health entry point that kept the module reachable were both removed in the descope, leaving the config-loaded reporter as the only consumer — invisible to --production analysis. The data-only module's second consumer is gone, so per the boundaries-are-earned norm the constants move into the reporter instead of gaining a suppression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * docs: align skills/ format policy and purge last SkillGym mention Address both P2 review findings on #1480: the testing-matrix row and the selector's formatGate both still claimed oxfmt covers skills/, while selectChecks classifies skills/*.md docs-only (oxfmt ignores **/*.md, so the claim was a no-op even before). The matrix now states the docs-only policy and formatGate drops the dead underSkills fact. The merged examples/README.md index (from #1469) loses its skillgym mention. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-29 11:45:22 +02:00
Michał Pierzchała	630dc7c99b	feat(examples): add runnable Node.js SDK examples under examples/sdk/ (#1469) ... * feat(examples): add runnable Node.js SDK examples under examples/sdk/ examples/test-app is a fixture and the repo's only prior examples/ content; the real SDK usage patterns lived only in website/docs/docs/client-api.md with no runnable script anywhere. Adds four standalone, typechecked examples covering the minimum surface from #1463: root client session (create -> open -> snapshot/tap -> close with typed error handling), agent-device/metro (normalizeBaseUrl/resolveRuntimeTransport), agent-device/contracts (centerOfRect on a snapshot node), and agent-device/batch (runBatch for a custom transport). Each imports the published `agent-device/...` subpaths rather than relative src/ paths. examples/sdk/tsconfig.json path-maps those subpaths to src/sdk/ so `pnpm typecheck` (now also run against this tsconfig) checks the examples in CI without a prior build, workspace link, or publish step. Running an example for real still resolves `agent-device` as a self-referencing package after `pnpm build`. src/__tests__/client-api-examples-drift.test.ts guards the examples against drifting from client-api.md's subpath API manifest in both directions, picked up automatically by the existing unit-core vitest project (no new script or workflow needed). examples/README.md indexes the new examples and notes that test-app/ remains a fixture, not an example; it is not renamed or moved. Refs #1463 * fix: address Fallow findings on the new SDK examples Fallow flagged the four examples/sdk/*.ts files as unused files (not reachable from any entry point) and three functions as high complexity. - Register the examples as manual entry points in .fallowrc.json, matching how other standalone scripts (scripts/patch-xcuitest-runner-icon.ts, scripts/runner-request-count/run.ts) are already declared. - Reduce complexity in client-session.ts and contracts-result.ts by extracting device-resolution/error-reporting and rect-assertion helpers out of main(). - Reduce complexity in the drift guard's parseSubpathManifest by splitting bullet-matching and backtick-name extraction into their own functions. Verified: pnpm check:fallow --base <PR base sha> now reports no issues, and pnpm check:tooling / pnpm test:unit stay green. Refs #1463 * fix: compile client-api.md's actual code snippets, not just its symbol manifest Addresses review feedback on #1463's drift guard: the existing guard only parsed the doc's "Public subpath API" bullet manifest and compared imported symbol names, so a fenced ```ts snippet could drift or stop compiling without the guard noticing. Added test/integration/client-api-doc-snippets.test.ts, which extracts every fenced ```ts block from client-api.md and typechecks it against the real agent-device/* sources (reusing examples/sdk/tsconfig.json's existing paths mapping, read via `tsc --showConfig` so there's one source of truth). Free identifiers that continue a `client`/`snapshot` from an earlier snippet are stubbed — typed against the real SDK return type, not `any`, so continuation snippets still get real checking. Lives in the Node integration lane (test/integration/*.test.ts), not vitest's unit-core: it spawns a real tsc Program, well past the unit suite's 2.5s budget. Running this check against the existing doc surfaced real, pre-existing snippet bugs (unrelated to the new examples), fixed here: - "sessions.artifacts": `result.cloudArtifacts` accessed without narrowing the `CloudArtifactsResult | DaemonArtifactsResult` union first. - "Device cloud sessions": `platform`/`device` were passed into the client constructor config, which doesn't accept them; moved to the `apps.open()` call where those fields actually belong. - "Android ADB providers": the inline `exec` handler had no parameter types, so it failed under strict/noImplicitAny; annotated with the real `AndroidAdbExecutorOptions` type. Two further gaps the check surfaced are pre-existing product/API-surface questions out of scope for this PR (not the new examples), so they're allowlisted in KNOWN_DOC_GAPS with comments rather than silently patched: - "Remote Metro helpers" documents prepareRemoteMetro/reloadRemoteMetro/ stopMetroTunnel/resolveRemoteConfigProfile as public, but none of them are exported from agent-device/metro or agent-device/remote-config today. - "Web sessions"/audio probe pass `platform` to `observability.network()`/ `.audio()`, but NetworkOptions/AudioOptions have no `platform` field even though the CLI's network/audio commands accept `--platform`. Refs #1463 * fix: close the doc-snippet compiler's stubbing hole and the two suppressed gaps Addresses the second round of review feedback on #1463's drift guard: 1. stubFreeNamesAndRecompile auto-stubbed every "Cannot find name" as `any`, so a typo like `cliet.apps.open()` would silently pass on the second compile. It now only stubs identifiers in an explicit allowlist (KNOWN_FREE_NAME_STUB_TYPES) — the real SDK-derived continuations (`client`, `androidClient`, `snapshot`) plus the doc's own invented host-glue names, each typed precisely rather than loosely. Anything else is left as a real compile failure. Added a regression test that feeds a `cliet` typo through the guard and asserts it fails. 2. KNOWN_DOC_GAPS filtered six real compiler errors out of the final assertion while the test claimed every snippet compiles. Investigated both and fixed the actual contracts instead of suppressing them: - `prepareMetroRuntime`/`reloadMetro` (src/metro/client-metro.ts) and `stopMetroTunnel` (src/metro/metro.ts) already existed and matched the doc's described workflow almost exactly (same result shape) but were never re-exported from `agent-device/metro`; same for `resolveRemoteConfigProfile` and `agent-device/remote-config`. Added the four exports and fixed the doc's stale function names (`prepareRemoteMetro`/`reloadRemoteMetro`) and one stale field name (`profileKey` -> `companionProfileKey` on the prepare call) to match. - `NetworkOptions`/`AudioOptions` (src/contracts/client-observability.ts) had no `platform` field even though the CLI's `network`/`audio` commands accept `--platform` for the same use case, and the client methods already forward the options object to the daemon generically (`executeCommand('network'|'audio', options)`) — so this was a type gap, not a runtime one. Switched both from AgentDeviceRequestOverrides to DeviceCommandBaseOptions (matching PerfOptions' existing pattern), closing the gap for real instead of stripping `platform` from the doc. - The "Android installFromSource()" snippet was missing its `createAgentDeviceClient` import outright; added it. KNOWN_DOC_GAPS is gone — every fenced snippet now compiles for real, and the test's assertion matches what it claims. 3. Switched the raw `execFileSync` calls to `runCmdSync` from src/utils/exec.ts, per AGENTS.md's process-execution invariant (this is a .ts integration test, not a packaging fixture that needs to stay dependency-free). Refs #1463 * docs: fix stale reloadRemoteMetro() prose reference to reloadMetro() The prose right after the Remote Metro helpers snippet still named the old function; the compile guard only checks the fenced snippet, not surrounding prose, so it didn't catch this leftover from the prior rename. Refs #1463 --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-28 21:36:07 +02:00
Michał Pierzchała	4e4ecdea0d	test(ios): expand simulator e2e coverage (#1408) ... * test(ios): expand simulator e2e coverage * test(ios): make coverage checks host portable * test(ios): handle deep link confirmation * test(ios): fix deep link prompt selector * ci: stabilize full simulator nightly * test(ios): stabilize permission prompt lifecycle * test(e2e): wait for route-specific landmarks * test(e2e): reset permissions from inactive app * test(e2e): redeliver trusted cold deep links * test(ios): verify orientation native readback * test(ios): stabilize simulator permission coverage * test(ios): wait for tab target after deep link * test(ios): paginate full event timeline * test(ios): simplify event pagination coverage * test(ios): stabilize simulator e2e coverage * test(ios): model simulator recorder lifetime * ci(test-app): cache fixture dependencies * fix(ci): isolate test app cache by node * fix(ios): settle fixture route navigation * test(ci): waive unbenchmarked ios system UI help * fix(ios): tolerate delayed simulator scale lookup * 0.20.1 * test(ci): remove superseded system UI waiver * fix(ios): harden simulator e2e reliability * chore: clarify Apple runner CI steps * fix(ios): wait before fixture home snapshot * fix(ios): require exact catalog navigation * chore(ci): format rebased workflows * fix(ios): retry unobserved fixture navigation * refactor(test): remove iOS e2e workarounds * fix(ci): verify fixture artifact provenance * fix(ci): align fixture artifact fingerprints * fix(ci): use unified Android helper packager * perf(ci): scope fixture build concurrency * chore: format fixture artifact tests * test(ios): update split Apple coverage owner * fix(ios): accept deep-link confirmation alerts	2026-07-28 20:08:45 +02:00
devin-ai-integration[bot]	6544e9a0c5	obs: repo-health snapshot command aggregating existing analyzers into one JSON (#1471) ... * obs: add repo-health snapshot command aggregating existing analyzers into one JSON Adds `pnpm repo-health [--json]`, an offline command that aggregates the signals this repo already computes into one deterministic JSON snapshot by reusing each analyzer (depgraph, layering ratchets, coverage json-summary, size-report, fallow baselines, slow-test ratchet, bench/skillgym registries) rather than reimplementing any metric. Carries mandatory v1 provenance (schemaVersion, commit, per-analyzer content hashes, input provenance) so #1424 can persist history. Component metrics (instability/abstractness/main-sequence distance) are observatory-only. The only gating behaviour is the depgraph-vs-layering R6 consistency assertion, already wired into the Layering Guard job via scripts/depgraph/model.test.ts. Closes #1423 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * repo-health: hash read artifacts and flag staleness in provenance Coverage and size are artifacts repo-health reads but does not produce, so they carry no producing-commit stamp. Previously provenance recorded only their paths and the current HEAD, so a snapshot taken after a source edit without rerunning those producers would pair prior metrics with the new SHA — #1424 would persist a false commit-indexed history entry. Now each read artifact is bound via artifactProvenance() to its content hash (so history keys on the bytes the metrics came from, not a commit they may predate) and an explicit `stale` flag (true when a production source file is newer than the artifact). The coverage/size analyzer-config hashes are added to provenance.tool, and the human summary marks stale artifacts. Adds a pure stale-artifact regression test. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * repo-health: bind artifact freshness to the whole producer input set The stale check derived freshness only from listSourceFiles() (src/*.ts) and applied it to both coverage and size. Coverage also depends on tests and vitest config; size-report.mjs reads package.json and runs npm pack — so a change to a non-src producer input could leave an old artifact marked stale:false while provenance recorded the current HEAD, a commit the metrics predate. Each read artifact now declares its full producer input set (PRODUCER_INPUTS) and is flagged stale when ANY tracked input in that set is newer than the artifact (via git ls-files mtimes). Freshness that cannot be proven — an empty observable input set — is reported stale, never falsely fresh. The input set is recorded as provenance.inputs.*.producerInputs. Adds isArtifactStale() with a regression covering a non-src input newer than the artifact and the unprovable case. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * repo-health: prove artifact freshness from a stamped producer commit; route git through runCmdSync Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 20:07:45 +02:00
Michał Pierzchała	ce7b7a93d9	feat: expose Limrun runtime (#1476) ... * feat: expose Limrun runtime * fix: address Limrun export feedback	2026-07-28 20:05:33 +02:00
devin-ai-integration[bot]	e0b8463ef8	feat(scripts): blast-radius query — dependents, owning gates, live-coverage owners (#1425) (#1470) ... * feat(scripts): blast-radius query over the depgraph model (#1425) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(scripts): keep --limit bound to its value in depgraph affected Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 18:20:39 +02:00
Michał Pierzchała	ba1a5efbc6	refactor(layering): declare R1-R3 as a policy table, and test them (#1449) ... R1-R3 were three hand-written predicate functions. Each was short, but each buried its boundary in control flow: you had to read the early-returns to learn that R3 tolerates dynamic imports, or that R1 opens exactly one door. They are now data in scripts/layering/zone-policy.ts -- which zones a boundary governs, which import kinds it tolerates, which path prefixes are its declared seam -- walked by one small evaluator. A fourth zone boundary becomes a table entry rather than a fourth predicate to keep consistent with the other three. R1 is deliberately two entries rather than one with a special case, because "kernel may import contracts type-only" and "kernel may import nothing else at all" are two statements, and writing them separately is what makes the single open door visible. The refactor exposed a real gap: checkLayeringRules had NO unit test. The only thing exercising R1-R3 was the real tree, which is clean, so a rule that had silently stopped matching would have looked exactly like a rule being obeyed. zone-policy.test.ts now asserts each boundary fires and each documented exemption holds, including that src/daemon/client/ is excluded from the daemon seam. Verified end-to-end by injecting one violation per rule plus an exempt file: the gate reports 4 zone-policy violations (R3 twice, catching the daemon/client case), ignores the type-only and dynamic edges, and exits 1. Also records in docs/dependency-graph-findings.md the result of spiking eslint-plugin-boundaries under oxlint's jsPlugins, so nobody repeats it. It does work -- jsPlugins loads npm ESLint plugins with no ESLint install, and R1-R3 are all expressible once you know importKind: "value" and settings["boundaries/dependency-nodes"]. Not adopted: no ratchet mechanism (the thing that took R6 from 61 to 7 incrementally), it cannot express R4-R9 so the architecture would be defined in two places, it misreads inline `{ type Foo }` specifiers as value imports (one false positive on providers/limrun/android.ts), message interpolation renders empty under the current selector syntax, and it costs 230 transitive packages on an API documented as alpha. No behaviour change: 932 source files, R6 = 7, R7 = 41 fields, R8 clean, R9 = 102, all identical to before. Claude-Session: https://claude.ai/code/session_01Bfu8HofkhybiAm5LECfqur Co-authored-by: Claude <noreply@anthropic.com>	2026-07-28 17:40:55 +02:00
Michał Pierzchała	a0aa02579b	build(android): unify the 4 helper build/package scripts behind one parameterized pair (#1466) ... * build(android): unify snapshot/ime helper build+package scripts Replace the four ~75%-duplicated shell scripts with one parameterized build script and one parameterized package script, mirroring scripts/build-xcuitest-apple.sh's env-var-driven pattern. The helper is selected via AGENT_DEVICE_ANDROID_HELPER or a first positional arg; per-helper differences (HELPER_DIR/PACKAGE_NAME, snapshot's test-compile+run step, ime's aapt2 resource-compile step, and the manifest JSON fields) live in small case blocks. All package.json entry points keep their names and output paths. Verified byte-level equivalence between main and this branch: identical unzip -l listings, identical classes.dex SHA-256 for both helpers, and identical manifest fields (only the per-signing-run sha256 differs). Fixes #1461 * chore: drop stale android/multitouch-helper .gitignore entries The multitouch helper was consolidated away in #1281; these two lines were never cleaned up.	2026-07-28 17:39:35 +02:00
devin-ai-integration[bot]	61f8696d28	test(ci): gate PRs on changed-line coverage (#1418) (#1447) ... * test(ci): gate PRs on changed-line coverage (#1418) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor: split coverage-changed model into small helpers for fallow Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ci): simplify coverage-changed reporting and CLI surface Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 14:25:10 +02:00
devin-ai-integration[bot]	edca35d122	chore(deps): Renovate config, packageManager-derived pnpm in CI, repo-wide format (#1444) ... * chore(deps): add Renovate config and enforce packageManager pnpm version in CI Refs #1422 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: bump pnpm to 11.17.0 and format the whole repo with oxfmt format/format:check drop their hand-maintained path list: oxfmt already skips node_modules and honors .gitignore, so the only exclusion list is .oxfmtrc.json ignorePatterns. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mutation): accept either quote style in the affected-lane path filter Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(deps): keep fixture-app runtime deps as individual Renovate PRs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 13:50:39 +02:00
devin-ai-integration[bot]	e89ef9e69b	test: gate public command surface against commands.md reference (#1446) ... * test: gate public command surface against commands.md reference Enumerate PUBLIC_COMMANDS against website/docs/docs/commands.md in both directions with a waivered unit-lane gate, and document the drift found on main (doctor, react-native). Refs #1420 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: recognize tilde-fenced code blocks in command-doc gate Refs #1420 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 13:43:15 +02:00
devin-ai-integration[bot]	006c4cadc9	test: nightly parser fuzz lane — parser input fails as typed AppErrors, never hangs (#1414) (#1438) ... * test: nightly parser fuzz lane with typed-AppError invariant (#1414) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fuzz): run envelope, artifact promotion, and harness self-check tests (#1414) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fuzz): shared scheduled-lane envelope on every terminal path, watchdog after ready (#1414) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fuzz): envelope for malformed options; add scheduled-lane health consumer (#1414, #1430) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(lanes): actions:read scope, terminal error envelope, first-due grace (#1414, #1430) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(lanes): anchor first-run grace to schedule registration, use exec helper in tests (#1414, #1430) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(lanes): portable POSIX pickaxe pattern for schedule registration (#1414, #1430) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(fuzz): fast-check generators over the shared hazard list, drop the bundled lane-health work (#1414) - Strip scripts/scheduled-lane/* and scheduled-lane-health.yml: that watcher is #1430's own deliverable and collides with PR #1439's implementation of the same lane. What this lane owes (a per-run envelope) moves into scripts/fuzz/envelope.ts. - Rebase onto #1437 and rebuild the generator layer on fast-check: cases come from arbitraries sharing SELECTOR_VALUE_HAZARDS with the property suite, and counterexamples are shrunk, so a failure names a minimal input plus fast-check's seed/path instead of a 20k-char random string. - Route harness.test.ts into the serialized subprocess-stub project. - Drop the AGENT_DEVICE_FUZZ_STARTUP_DELAY_MS test seam: the ready handshake is now proven by a case budget far below real worker startup. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fuzz): replay the regression corpus through the worker watchdog (#1414) A promoted hang case used to wedge the unit job until the CI timeout, because corpus replay called checkCase in-process. It now goes through the same worker-backed watchdog the nightly lane uses, so such a case fails against a 5s per-case budget; the file moves to the serialized subprocess-stub project with the rest of the worker-driven fuzz tests. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fuzz): let the watchdog outlive vitest's default case timeout (#1414) A wedged parser was surfacing as a bare 'Test timed out in 5000ms' instead of the named hang: failure that says which input wedged, because the file's vitest timeout was shorter than the watchdog budget times the number of replayed cases. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(fuzz): complete drift provenance in the lane envelope (#1414) configHash now covers every input that decides what a seed generates (generate.ts and the shared property arbitraries, not just the arbitraries/targets/invariant), and tool records fast-check's installed version. A generation-loop edit or a fast-check upgrade previously changed the case set while the envelope looked unchanged. A test recomputes the hash with each input omitted so a future omission fails. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 11:29:25 +02:00
devin-ai-integration[bot]	e545544dfa	test(daemon): seeded concurrency torture lane for session/lease/lock invariants (#1439) ... * test(daemon): seeded concurrency torture lane for session/lease/lock invariants Refs #1416 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): drive torture lane through real lock plan + review fixes - derive each op's lock plan from production resolveRequestExecutionLockKeys via a fake device-inventory provider, so reverting the router's same-device serialization trips the overlap invariant (verified) - assert exact replay: full scheduler trace, terminal outcome, contention - assert real same-device lock contention in the sweep + a forced 2-client case - split harness into bindings/invariants/envelope modules (all <500 LOC) - emit #1430 scheduled-lane envelope (schema/SHA/hash/seed range/duration/result) and upload it from the nightly workflow Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): pass claim data as plain view accessors (fallow) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): gate lock plan on shouldLockSessionExecution; sweep replay + forced-device contention; whole-lane envelope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): add real-scope runLocked serialization guard; whole-lane envelope duration Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): allow seed 0 replay; add seed-0 regression (TORTURE_SEED must accept 0) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * obs(#1430): add scheduled-lane freshness/cadence health watcher Discovers schedule: workflows from .github/workflows/, reads recent scheduled runs via the GitHub API, and opens/pings a tracking issue when a lane misses or fails two consecutive cadences. Pure model unit-tested and gated on PRs; API I/O + issue open/ping run nightly. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): give scheduled-lane watcher a two-cadence newborn grace Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): anchor lane grace on schedule-introduction, not workflow age Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): derive schedule-activation semantically via git, through runCmdSync Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): add merge-commit regression pinning first-parent + committer time Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): pin scheduled-lane-health issue-write route via stubbed fetch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): unbundle #1430 watcher; make torture lane nightly-only Strip the scheduled-lane-health watcher (scripts + workflow + PR gate) — it is #1430's deliverable and collides with PR #1438's workflow of the same filename; keep only this lane's #1430 envelope writer. Move the torture lane under test/integration/nightly/ so it is out of the test:integration:node glob, and run it via an explicit, disclosed PR step plus the nightly sweep. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: fix stale torture-lane paths after nightly/ move Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): adopt shared lane-envelope for torture lane Rebase onto main (post-#1441) and replace the lane-local LaneEnvelope dialect with the shared scripts/lib/lane-envelope.ts builder, so the #1430 health watcher parses one schema: commitSha->commit, sourceHash-> configHash, seedRange/runs moved into the typed data payload, and the sweep encoded as seed "<start>-<end>". Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 10:29:41 +02:00
devin-ai-integration[bot]	8cce0ef6b8	test: ratchet mutation score over enumerated decision kernels (#1441) ... * test: ratchet mutation score over enumerated decision kernels Adds a Stryker (vitest runner) mutation lane scoped to the decision kernels, a per-module baseline with tool/config provenance, and a ratchet that only lets scores rise. Non-gating until two consecutive stable weekly sweeps. Refs #1415 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: declare the mutation test-scope seam for production-export analysis Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: own kernel tests in the mutation registry and ship the #1430 lane envelope - restore bench:help-conformance, broken by a formatting-path edit - kernel test files select their module on PRs (registry `tests` + workflow paths), asserted to reach the kernel through the import graph - every mutation run writes the standard scheduled-lane artifact envelope - move src/utils/__tests__/errors.test.ts beside its source per the mirror rule Refs #1415, #1430 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: derive kernel test ownership and land the scheduled-lane health monitor Ownership of a kernel's tests is now computed from the static import graph (scripts/mutation/ownership.ts) instead of a hand-listed set, so a test that reaches a kernel indirectly -- src/__tests__/daemon-error.test.ts through src/daemon.ts -- selects that kernel on a PR. The PR lane triggers on every src test and shards the derived modules, keeping wall clock at one module. The lane envelope (#1430) is now written on every exit path with the stage it reached, so a crash before any mutant runs is distinguishable from a lane that never ran. Adds the derived cadence monitor (scripts/lane-health, daily workflow): scheduled lanes are enumerated from .github/workflows/ and reported dark, failing, or never-run against their own cron cadence. * fix: merge only Stryker reports from a shard directory The shard artifacts now carry the lane envelope beside mutation.json, and the merge globbed every .json under the download path, so the ratchet job fed the envelope to the report parser and died after the mutants had already run. * fix(mutation): fail on incomplete shard sets and envelope pre-run failures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mutation): downgrade a passing envelope when a later lane step fails Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(mutation): shard by registry, defer the PR lane, drop the bundled watcher Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: describe registry sharding and the deferred PR mutation lane Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mutation): make the pre-graduation tooling exception select real mutants Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mutation): give the worktree fixture commits their own identity Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 10:07:14 +02:00
devin-ai-integration[bot]	d747ef6230	test: frozen replay-compat corpus with expected verdicts (#1417) (#1436) ... * test: frozen replay-compat corpus with expected verdicts (#1417) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: pin replay-compat corpus bytes to released blobs and assert via parseReplayInput Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: lock replay-compat provenance kind by corpus area and verify it in CI Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: describe corpus provenance-kind lock and CI job Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: prune replay-compat corpus to minimal witnesses per shipped form Reviewer feedback on #1436: the mechanism earns its place, the dataset did not. Drop the 30 corpus entries whose bytes repeat a syntactic form or a migration refusal another entry already witnesses (platform twins and adjacent-release re-recordings), leaving 22 deliberate entries; make note required and state per entry which form or refusal it is the sole witness of. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: address corpus review nits (typed coverage list, cap rationale, derived-citation note) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: split corpus rule — form from the release, verdict from today's parser Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: format corpus README emphasis markers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-28 10:06:00 +02:00
devin-ai-integration[bot]	8670a88afa	test: property-based testing foundation (fast-check) over parse/print and geometry kernels (#1437) ... * test: property-based testing foundation over parse/print and geometry kernels (#1413) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: derive selector and replay command domains from their owning registries Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-07-27 17:15:39 +02:00
Michał Pierzchała	2d1d70613f	feat(bench): renderer-pinned samples, topic-coverage gate, error-recovery quizzes; trim skillgym to agentic checks (#1411) ... * feat(bench): renderer-pinned samples, topic-coverage gate, error quizzes; trim skillgym to agentic checks The help conformance bench's quoted CLI output is now sourced from scripts/help-conformance-sample-outputs.mjs, and every sample is rebuilt through the real production renderers (settle output formatters, printHumanError, formatSnapshotText, refMutationAdmissionResponse) by scripts/__tests__/help-conformance-sample-outputs.test.ts — a rendering or message change fails deterministically instead of leaving the bench grading against output the CLI no longer prints. This retires the fabricated recoverable-failure envelope (production never throws a textual settle timeout; that case is replaced by a real DEVICE_IN_USE recovery quiz). Bench cases move to scripts/help-conformance-cases.mjs and are enumerated against the help-topic registry: helpTopicIds() is exported from cli-help, and scripts/__tests__/help-conformance-topic-coverage.test.ts fails when a help topic has neither a bench case nor an explicit waiver. New case families: error-envelope recovery quizzes (device-in-use, stale pinned ref, ambiguous find match, app-not-installed) pinned to real error text, topic coverage for tv/web/react-native/debugging/workflow, and a metamorphic twin of the settled-diff quiz. The skillgym smoke suite shrinks from 119 cases to the 5 that measure what only an agentic runner can show: skill routing plus output interpretation with a proven local CLI help probe (local-cli-help-policy). Its embedded samples now import the same pinned constants, replacing hand-transcribed output that had already drifted from the renderer. Knowledge checks belong to the bench; live fixture behavior belongs to the iOS simulator e2e suite. * review: drive error samples through the real producers; enforce local-help on the routing smoke The DEVICE_IN_USE, AMBIGUOUS_MATCH, and APP_NOT_INSTALLED parity tests no longer hand-author the producer message before rendering: each drives the actual producer — buildDeviceInUseBySessionError (extracted in session-open.ts and called by the handler), buildAmbiguousMatchError (now exported from find.ts), and buildAppNotInstalledError (extracted in app-resolution.ts and thrown by the resolver). Because each factory is exported from its producer file and called by the production path, dropping the production call would make it test-only and fail check:production-exports — the wiring is gate-enforced, not conventional. open-and-snapshot now sets requireLocalCliHelp and allowOnlyLocalCliHelpCommands, so the 'skill plus local help' claim is observed rather than assumed; without them the case can pass on model prior alone.	2026-07-27 14:24:36 +02:00
Michał Pierzchała	f19864e486	feat(scripts): dependency-graph report over the layering gate's model (#1410) ... * feat(scripts): dependency-graph report over the layering gate's model Reports what the layering gate deliberately does not enforce, as JSON plus a short summary. No renderer: the productive artifact is the JSON. pnpm depgraph # -> .tmp/depgraph/graph.json + summary pnpm depgraph:test Dependency graph: 898 files, 4627 edges, 25 zones value-import cycles (R4): 0 type-only/dynamic cycles (not gate-rejected): 8 spine back-edges (R5): 0 type-only spine inversions (R6): 42 transitively redundant value edges: 1338 The two numbers worth having are the ones CI cannot give you. Transitively redundant value edges — where the target is still reachable at distance >= 2, so the direct import changes nothing about what the module can see — need a real reachability pass, not a grep. And cycle detection over type-only and dynamic edges covers the loops R4 excludes by design. Both are candidate lists, never work lists; at ~1300 the redundancy set is a place to look. It reuses scripts/layering/model.ts, the same module check.ts uses in CI, so the file set, zone partition, edge kinds and cycle definition are the enforced ones. A second extractor would describe a graph nobody gates. Consequence worth having: its R6 count reproduces TYPE_INVERSION_BASELINE, so a mismatch means one of the two is stale. This is the analysis half of a viewer that was built and dropped. The render cost ~2200 lines and needed a Fallow exemption for a 920-line canvas file, and nobody read it. Everything here clears the repo's bar with NO exemption — scripts/depgraph is deliberately absent from ignorePatterns, unlike scripts/layering, scripts/perf and scripts/maestro-conformance. Getting there meant fixing rather than suppressing: extracted `valueSuccessors` (the value-edge adjacency was built identically in two places — a real clone), split `buildGraph` into four named aggregation steps, split `reachableBeyondDirectEdge` out of `markRedundantEdges`, extracted `compareZoneEdges`/`crossedZonePair`, extracted `edgeKindCode`/`edgeFlags` from a nested ternary scoring CRAP 42, and deleted `fileGroup` plus the `group` node field once the cluster layout went. Two additive exports on scripts/layering/model.ts: `zoneRank` and `targetDagZone` (previously module-private). The gate's behaviour is unchanged. `pnpm check` green, 4488 unit tests, 5 model tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bfu8HofkhybiAm5LECfqur * ci(layering): assert the depgraph report reproduces the gate's baseline The report reads the same model as the gate, so its inversion count must equal TYPE_INVERSION_BASELINE. That agreement was previously a nice property nobody checked; the Layering Guard job now runs scripts/depgraph/model.test.ts, so the two cannot be green independently. Verified by bumping a baseline entry by one and confirming the job fails with a message naming the fix. The count feeding the check is computed by `typeInversionsByPair`, which applies the gate's rule — once per FILE pair, over the raw resolved edges — rather than reading the collapsed edge list. That matters: `collapseEdges` keeps one edge per pair with the strongest kind winning, and `dynamic` outranks `type`, so a module imported both lazily and for its types would collapse to `dynamic` and drop out of the count. No such pair exists today (measured: 0 of 42 inverting pairs), but a number wired into a CI equality check must not be able to drift for a reason unrelated to layering. Stated honestly in the README and the test: this is a cross-check of the report's extraction and the baseline against the real tree, not two independent algorithms. The gate remains the authority — if they disagree, the baseline or the tree is wrong, never the test. TYPE_INVERSION_BASELINE is now exported for this purpose. Not done here, deliberately: the ~1338 transitively redundant value edges are a candidate for a loose growth-only ratchet later. They are a candidate list, not a work list, and a hard count would be noise. `pnpm check` green, 4488 unit tests, 6 depgraph model tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bfu8HofkhybiAm5LECfqur * fix(depgraph): make the source reviewable, and stop overclaiming removability Four review findings, all of them real. P1 - the implementation was binary. scripts/depgraph/model.ts contained two raw NUL bytes used as map-key delimiters, so Git classified a ~346-line file as binary and hid its entire diff behind `- -`. Replaced with a unicode escape: identical at runtime, textual on disk. I had seen the symptom repeatedly - every grep on that file printed "binary file matches" - and worked around it with python instead of asking why, which is how it survived to review. Guarded repo-wide rather than for this one file: a new test asserts no tracked .ts under src/ or scripts/ contains a raw NUL, verified by reintroducing one and watching it fail. Nothing else would catch a recurrence, and the failure mode is silent - the code works, the review does not. P1 - "transitively redundant" claimed removability it cannot support. Module reachability does not carry bindings: if `a` imports `{ c }` while `b` only re-exports it as `{ c as b }`, the path a -> b -> c exists and deleting a -> c still breaks `a`. The fixture in model.test.ts is exactly that shape and its comment said "removable". Reachability also says nothing about when a module's side effects run. Renamed throughout to what it measures - `transitivelyReachable`, `markTransitivelyReachableEdges`, and a summary line reading "value edges whose target is also reachable at distance >= 2 (reachability only - not a removability claim)". The caveats and the counterexample are now stated in the marker function, the fixture comment and the README, and symbol-level analysis is named as what deciding any individual edge would actually require. P2 - build.ts had no coverage. Every test exercised model.ts, so the CLI could break its output path, wire shape or summary silently. Added three subprocess tests: default path plus summary-agrees-with-payload, `--out` honoured and valid JSON written, and a trailing `--out` falling back rather than crashing (pinned so it is a decision, not an accident). `pnpm depgraph:test` now runs inside `check:tooling`, so `pnpm check` covers it. P2 - README was wrong three ways: it queried `.tmp/depgraph/index.json` after the output moved to `graph.json` (the documented command failed as written), it derived inversions from collapsed `zoneEdges`, which can undercount, and it claimed both that the report runs in CI and that nothing here runs in CI. The query now reads `typeInversions` and was run verbatim; the CI sentence names exactly which single test runs and states that nothing else gates a merge. pnpm check green, 4488 unit tests, 10 depgraph tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bfu8HofkhybiAm5LECfqur --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-27 13:59:53 +02:00
Michał Pierzchała	ab913c9720	feat: strengthen agent help benchmarks (#1404) ... * feat: strengthen agent help benchmarks * fix: harden help benchmark review findings * fix: close help benchmark validation bypasses * fix: make selector scoring quote-insensitive	2026-07-27 10:17:46 +02:00
Michał Pierzchała	074ca140a2	test: serialize PATH-stub daemon/apple tests to end contention flakes (#1362) ... `src/daemon/__tests__/runtime-hints.test.ts` and `src/platforms/apple/core/__tests__/index.test.ts` inject stub `adb`/`xcrun` binaries by mutating `process.env.PATH` and then spawn them, so each case waits real subprocess time. Run in the `unit-core` project at ~7x file parallelism they contend for CPU with every other stub-spawning file; a starved stub spawn pushes production down a generic failure path that returns a different error than the assertion expects. The failures present as assertion errors (not timeouts) and the failing subset shifts between runs — a pre-existing contention flake, reproducible on a clean tree. The repo already serializes the android scripted-adb tests into their own `fileParallelism: false, maxWorkers: 1` project for exactly this reason. This generalizes that project (renamed `android-adb` -> `subprocess-stub`) and adds the two files, so at most one real-stub-spawning file runs at a time across the whole suite. Assertions no longer let host wall-clock decide which error path runs. Option of injecting the spawn budget instead was not viable: runtime-hints passes no timeout to inject, and adding an exec-options seam purely for the test would be a test-only DI seam the CI gate forbids (AGENTS.md). Claude-Session: https://claude.ai/code/session_01NFepuH3Rh6ciRkvYJSCCtU Co-authored-by: Claude <noreply@anthropic.com>	2026-07-22 17:13:50 +02:00
Michał Pierzchała	a55a1a2600	0.20.0	2026-07-21 20:50:53 +02:00
Michał Pierzchała	32ba4b67f4	chore: add FreeRange range-analysis check (#1354) ... * chore: add freerange check * ci: install bun for freerange check * fix: preserve numeric range contracts * fix: guard diff overlay geometry * refactor: isolate diff overlay bounds	2026-07-21 11:49:33 +02:00
Michał Pierzchała	856d5d4900	test: replace the hand-typed Maestro fixture with a generated conformance oracle (#1289) ... * test: replace the hand-typed Maestro fixture with a generated conformance oracle Closes #1274. The old harness (scripts/maestro-conformance*) compared 5 hand-authored flows against a hand-typed transcription of Maestro 2.5.1's command model. It proved parser self-consistency, not conformance: all four bug classes that cost #1217 days of live debugging slipped past it by construction, and it verified no upstream SHAs despite parsing them. Every expected value here is generated from the pinned upstream artifacts. dev.mobile:maestro-orchestra:2.5.1 is published on Maven Central, so the harness runs the real parser and reads the real bytecode — no full Maestro source build. Layer 1 (parser): a Gradle/Kotlin harness drives the pinned YamlCommandReader over a corpus of 42 vendored maestro-test flows (sha256-recorded) plus authored bug-class, coverage, and invalid flows, capturing each parse. The verifier parses each flow with the live engine and classifies it identical / both-reject / we-reject / mismatch / we-are-lenient. Every non-identical outcome must be a declared divergence, so the 17 we-reject entries in expected-divergence.ts are the mechanical parity backlog (assertTrue, clipboard, travel, killApp, and option-level gaps) rather than silent drift. Layer 2 (semantics): ASM reads static-final constants straight from the pinned bytecode without initializing driver classes (MAX_RETRIES_ALLOWED=3, SCREENSHOT_DIFF_THRESHOLD=0.005, ANIMATION_TIMEOUT_MS=15000, erase cap, and the iOS pre-tap gate we intentionally omit), plus the parser-observed 400ms swipe default. Each is cross-checked against MAESTRO_COMPATIBILITY_PRESETS. Layer 3 (differential): scheduled device scenarios. Cross-engine comparison is outcome parity only and says so; finer behavior is asserted engine-side via invariants over replay-timing.ndjson. Bug class 4's detector — a tap must not consume the whole settle budget, since a full-budget tap means the stability loop never latched while the flow still passes — is pure and unit-tested against synthetic traces; only the device run is scheduled-only. regenerate.mjs verifies the pinned jar SHA-256s before trusting output and is byte-deterministic across runs. Layers 1-2 verify in normal CI via node --test with no Java (the job installs deps: unlike the layering guard it copies, the verifier parses with the live engine, which imports the `yaml` package). Acceptance: the four bug classes each have a fixture; every command in SUPPORTED_MAESTRO_COMMAND_NAMES (the parser's own dispatch table, now exported as the single source of truth) is corpus-covered or listed unverified; the five documented deviations are expected-divergence entries. * fix: address review findings on the conformance oracle P1 — layer-3 scenarios could never run. They pointed at layer-1 corpus flows, which exist only to be PARSED: they name a fictional com.example.app and elements that exist on no device. A device run would have failed before exercising any runtime behavior, making bug class 4's detector silently vacuous. Layer 3 now has its own flows under differential/flows/ driving the real fixture app (examples/test-app, com.callstack.agentdevicelab); the workflow builds and installs it and hard-fails if it is missing. A test enforces the separation so a scenario can never point back at the parse corpus. Nothing else in this repo builds or installs the Expo fixture app, so those steps are new and unproven. The workflow is therefore dispatch-only: the cron is removed until a supervised first run proves the path. A nightly job that fails at 05:00 every day teaches nothing. P2 — layer 3 installed whatever version the online installer served. It now pins MAESTRO_VERSION from pinned-upstream.json, so layer 3 cannot drift from the version layers 1-2 claim, and asserts `maestro --version` matches. P2 — fixture content was not bound to regeneration. CI compared only the embedded upstream metadata, so a hand edit to a captured command or constant passed: the transcription failure mode this oracle exists to remove. Two-layer fix, because per-PR CI must stay Java-free and cannot re-derive: - Each fixture now carries a contentHash seal that the verifier recomputes, so editing a capture breaks the build. Tamper-evident, and tested by actually tampering rather than assuming a hash comparison works. - New scheduled conformance-regenerate job re-runs the harness against the pinned jars and fails on any byte difference. Forgery cannot survive a real re-derivation. This is what makes "generated from upstream" enforced. P3 — boot-ios-test-simulator requires runtime-version; now passed alongside preferred-device-name, as the other iOS workflows do. * tmp: trigger layer-3 differential on this branch to prove the device path workflow_dispatch cannot run pre-merge (it registers from the default branch), so this temporary push trigger exists only to execute the never-run device path on the PR head and capture evidence. Removed before merge. * fix(ci): install the fixture app unfrozen for the layer-3 device run First live run of the device path failed at the very first step: ERR_PNPM_LOCKFILE_CONFIG_MISMATCH. CI implies --frozen-lockfile and the fixture app's lockfile is out of sync with its package.json overrides. No CI job has ever built examples/test-app, so that drift was never surfaced. * fix: drop --ignore-workspace from test-app:install (defeats #649 security overrides) The first live run of the layer-3 device path failed at ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, and the cause is a real latent bug rather than a stale lockfile. #649 moved the fixture app's `overrides` into examples/test-app/pnpm-workspace.yaml precisely because pnpm only honors overrides from a workspace root — they pin transitive deps (ws, brace-expansion, xmldom, postcss, uuid, shell-quote) to versions that clear Dependabot alerts. But `test-app:install` passes --ignore-workspace, which ignores that very file, so the overrides are dropped and no longer match the lockfile that has them baked in. It goes unnoticed locally because interactive installs are not frozen, and no CI job has ever installed this app. Dropping --ignore-workspace makes examples/test-app resolve as its own workspace root (it has its own pnpm-workspace.yaml and is not a member of the repo-root workspace), so the overrides apply and a frozen install succeeds. Verified both directions locally: with the flag + --frozen-lockfile reproduces the CI failure; without it, a frozen install completes and the lockfile's overrides stay intact. Note the workaround this replaces would have been actively harmful: installing with --no-frozen-lockfile resolves the mismatch by regenerating the lockfile WITHOUT the overrides, silently reverting the app to the vulnerable transitive versions #649 pinned away. * fix: make layer-3 scenarios prove what they claim, and parse the Maestro version Run 3 (29497919702) got the whole device path working: Expo build (30m), app installed, simctl check, pinned Maestro CLI install. Only the version ASSERTION failed — `maestro --version` prints an analytics banner before the version, and `tr -d '[:space:]'` mashed banner+version into one string. The CLI was correctly 2.5.1. Match the semver line instead, and set MAESTRO_CLI_NO_ANALYTICS (CI should not phone home). Verified the parse against the exact CI output: banner and clean forms both yield 2.5.1, wrong/empty still fail. tap-retry-if-no-change was vacuous: it tapped a navigating control, so the first tap always succeeded and retryIfNoChange never ran — it passed while proving nothing. It now taps the app's non-interactive title so the screen cannot change and the retry path is forced, and asserts tapRetries >= 1 from the trace (MaestroRuntimeMetrics already records it per step). A new metricAtLeast invariant kind carries the assertion; a test reproduces the old vacuity. percent-swipe no longer claims bug class 1. Truncation vs rounding is a <=1px delta that no app-observable device outcome can distinguish, so pass/pass could never back that claim up. The runtime half is instead pinned exactly by a pure unit test of resolveMaestroCoordinate (it short-circuits on a known viewport, so no device is needed) — verified to catch the regression by flipping trunc->round, which turns 3 of 6 tests red. Truncation had no test coverage at all before this. A test now forbids any device scenario from re-claiming bug class 1. * fix(ci): pass --maestro and match the fixture app's real UI in layer-3 flows Run 4 (29500262301) reached the differential itself — build, install, simctl check and the pinned Maestro 2.5.1 verification all passed — and surfaced two real bugs, both mine: 1. The runner invoked `agent-device test <flow>` without --maestro, so every scenario failed with "test does not support this file type". The repo's own scripts/run-test-app-maestro-suite.mjs passes it; the flag is what routes a .yaml through the Maestro compat engine. 2. settle-after-tap and percent-swipe assumed home-open-form is on screen at launch. It is not: real Maestro reported "Element not found: home-open-form", and the app's own helper flow scrolls it into view first. settle-after-tap now scrolls before tapping, mirroring that helper; percent-swipe no longer navigates at all and swipes the scrollable home screen, so it tests the conversion and nothing else. The remaining two flows already reported maestro=pass, so only the agent-device invocation was wrong for those. Note the settle invariant correctly reported "no-data: no completed tapOn steps" and FAILED rather than passing — a detector that cannot run is a failure, as intended. * feat: declare layer-3 divergences and schedule the differential Layer 3 ran both engines for the first time (29504440599) and immediately found a real engine bug. Blocking the measurement instrument on repairing what it just measured inverts the dependency, so layer 3 now gets the contract layer 1 already had: every divergence is a decision on the record. Adds `knownDivergence: { reason, tracking }` to the scenario type — the layer-3 twin of FLOW_DIVERGENCES. A declared divergence keeps the run green; only UNDECLARED ones fail. Two rules stop that from rotting, both enforced mechanically rather than by prose discipline: - `tracking` is required and must be a real issue URL (run.test.ts), because a declaration with nothing behind it is how "temporarily expected" becomes permanent without anyone deciding to. - a stale declaration FAILS: if a declared-divergent scenario starts passing, the run goes red until the declaration is removed. The fix PR must delete it, and the differential then enforces the gap stays closed — the oracle is the acceptance test for its own findings. Declared: - settle-after-tap -> #1299. Our scrollUntilVisible times out finding home-open-form where Maestro 2.5.1 scrolls to it and passes. Real engine correctness bug in an advertised command, found by this differential. Blocks bug class 4's device detector until fixed. - tap-retry-if-no-change -> #1300. The invariant caught the scenario being vacuous: both engines pass but tapRetries was 0, so retryIfNoChange never ran. Needs an inert fixture control; a scenario defect, not an engine one. Proven green on both engines and enforced now: percent-swipe, optional-warned-not-failed — the latter is real device-verified warned-vs-failed parity. With declarations in place the differential is green, so the schedule goes in (cron 05:00) per #1274. A green run still prints what it is not proving. * fix: park the flaky retry scenario instead of declaring it a divergence Run 29510020718 fired the stale-declaration guard on its first outing and caught my own mistake. tap-retry-if-no-change measured tapRetries=0 in run 29504440599 and tapRetries=1 in 29510020718 — same flow, same commit. So it is not vacuous as #1300 originally claimed: it is NON-DETERMINISTIC. The tap sometimes holds the hierarchy signature still and sometimes does not, because the fixture home screen carries live content. That exposes a real limit of the mechanism added in the previous commit: knownDivergence assumes the divergence REPRODUCES. A declared-but-flaky scenario flips between known-divergence (green) and stale-declaration (red) at random — a coin-flip scheduled job, which is worse than no scenario because it teaches people to ignore the differential. So the scenario is parked, not declared. The flow and the tapRetries invariant stay implemented and unit-tested, so the fix PR only re-adds the scenario once the fixture has an inert control. retryIfNoChange therefore has NO device coverage right now — tracked in #1300 and stated plainly rather than disguised by a green run. A test keeps it out of the active set until then. #1300 updated with the corrected diagnosis and both runs' evidence. Active differential: settle-after-tap (declared divergence, #1299), percent-swipe and optional-warned-not-failed (both enforced, pass/pass on real devices). * fix: make a knownDivergence waiver cover exactly one failure, not any failure P1 from re-review, and a real flaw: the code did not do what its own comment claimed. runScenario() collapsed every unexpected outcome and every invariant failure into `misbehaved`, then turned ANY of them green if the scenario carried a declaration. So while the #1299 scrollUntilVisible waiver is open, upstream Maestro could start failing too — or a different invariant could break — and the scheduled job would still report known-divergence and pass. A waiver for one bug was silently amnesty for the next. That is the exact failure this oracle exists to prevent, committed one commit after building the guard against it. knownDivergence now requires an `expected` signature: both engines' outcomes plus each declared invariant's status. The runner matches it exactly — - matches -> known-divergence (green, tracked) - misbehaves differently -> failed (red): not the failure the waiver covers - stops misbehaving -> stale-declaration (red): remove the declaration #1299's signature pins what runs 29504440599/29510020718 actually observed: maestro=pass, agent-device=fail, settle invariant no-data. Tests prove unrelated failures stay red under an open waiver: upstream also failing, our engine unexpectedly passing, a different invariant status, and a new invariant appearing are each NOT covered. A signature where both engines pass is rejected outright as describing no divergence. Also retains replay-timing.ndjson as a run artifact (review evidence note): the invariants are computed from that trace, so a report saying "tapRetries was 0" cannot be audited once the runner is gone without it. * perf(ci): cache the fixture app build for the layer-3 differential The differential job took ~30 minutes, of which 1331s (22 min, 79%) was building the Expo fixture app and only 347s was the differential itself — rebuilt from scratch on every run for an app that changes almost never. Cache the built .app, keyed on everything that can change the binary: the app's sources, native config, dependency graph, the build step itself, the iOS runtime, and the Xcode version. Mirrors the existing setup-apple-replay prebuilt-runner cache (same action pin, same Xcode-key + source-hash shape). On a hit the build is skipped entirely and the bundle is installed straight onto the booted simulator (~seconds), taking the job to roughly 8 minutes. On a miss it falls back to exactly the previous behaviour and repopulates, so the worst case is unchanged. The existing simctl verification still gates both paths, so a bad cache cannot produce a vacuous green: if the app is not installed, the job fails loudly rather than running scenarios against nothing. Note the first run after this lands is necessarily a miss. * refactor(ci): extract setup-fixture-app so any job can use the cached app The fixture-app build + cache was inline in the differential workflow, so nothing else could reach it. Extracted to a composite action mirroring setup-apple-replay, because the capability is what #320 has been missing: it wants replay coverage moved off Apple system apps onto a controlled fixture with stable ids, and that fixture (examples/test-app) already exists — CI just had no way to build and install it. The cache is genuinely shared. GitHub caches are per-repository and readable across workflows, and a run restores from its own branch or the default branch, so once a run on main populates it every workflow gets the hit and only the first one pays the ~22 minutes. The key is computed inside the action from a fixed input list and deliberately contains nothing caller-specific — folding a caller's workflow path into it would silently unshare the cache. Also removes a duplication risk: the action reads the bundle id from the built app's Info.plist rather than hardcoding it, so it cannot drift from what was actually built, and it fails loudly if the app is not installed. The conformance workflow keeps its own narrower assertion — that the installed id is the one its scenarios target — since that is its concern, not the action's. Usage: - uses: ./.github/actions/setup-fixture-app with: runtime-version: ${{ env.IOS_RUNTIME_VERSION }} # outputs: app-path, app-id, cache-hit * chore(ci): remove the temporary branch push trigger Run 29519848340 on this head executed both engines against the real fixture app and came back green, so the trigger that existed only to prove the never-run device path has done its job. Merged config is now cron (05:00) + workflow_dispatch, as required by #1274. known-divergence settle-after-tap maestro=pass agent-device=fail (#1299) ok percent-swipe maestro=pass agent-device=pass ok optional-warned-not-failed maestro=pass agent-device=pass This commit will not itself trigger a run: GitHub evaluates triggers at the pushed commit, and the push trigger is gone in it.	2026-07-16 20:22:28 +02:00