8 Commits

Author	SHA1	Message	Date
Michał Pierzchała	338aa2a0d5	refactor: route every native selector resolution through the policy interface (#1715) ... * refactor: route every native selector resolution through the policy interface #1649 declared the per-caller ambiguity matrix; four native call sites still bypassed it, spreading `selectorResolutionKnobs(row)` into a raw `resolveSelectorChain` instead of naming the row. That left the "one interface" claim aspirational: a caller could restate its contract as engine knobs and nothing would notice. - `is` non-exists, `get text`/`get attrs`, find's read actions, and the covered-selector diagnosis probe now call `resolveSelectorChainWithPolicy` with their existing row. Semantics are byte-identical: the knob-backed branch of that interface forwards to the same engine call the call sites built by hand. - The façade drops `resolveSelectorChain` and `selectorResolutionKnobs`, so no knob-taking resolver is reachable from outside the package and a call site cannot re-acquire the knobs even by accident. `requireUnique`/`disambiguateAmbiguous` are now named in exactly one function, which `resolve-with-policy.ts` and the replay resolver both derive through. - `get` names the two rows it may consume as a type, so pointing it at any other ambiguity contract is a compile error. Tests: selector-read-policy.test.ts pins which row each read command consumes, end to end, on one ambiguous fixture — the only tree the rows disagree on. Each assertion was proven red by re-pointing its caller at a neighbouring row. The knob-consistency check moves into the package beside the now-private helper. Test call sites that used the raw resolver move to `resolveRecordedTarget`, the same knobs and the path that actually replays a recorded chain. Extracting the failure branch drops `resolveSelectorInteractionTarget` below the complexity threshold; its `fallow-ignore` waiver is removed (verified load-bearing before the extraction, unnecessary after). Closes #1630. Structural stages (occlusion, off-screen, promotion, poll budget) stay per-caller pipeline code, tracked in #1656. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * test: observe which node find's row selected, not just that one existed #1715 review, P2: the find row assertion was only half a pin. `find exists` returns `found: true` for any resolved node, and the `list` call it leaned on goes through listFindMatches — a path that consumes no policy row at all. So repointing findFirstLocatorMatch at `readText` left both assertions green while selection silently moved from the document-order head to the tiebreak winner. Assert through `find get_attrs`, which returns the ref of the node the row actually selected. Both neighbouring rows are now red: `readText` fails '@e3' !== '@e2' (the move the old test missed), `readUnique` fails by refusing the ambiguous screen. `exists` stays as a second, weaker assertion on the same resolution. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * refactor: route is exists through the matrix, collapse the double match pass Follow-up tightening on the same seam. `is exists` reached findSelectorChainMatch directly while the `readAny` row's own doc claimed to serve "`exists` and find's read-only actions" — true of the docs, not of the code, which is the unverifiable-claim shape #1656's review called out. It now names `readAny`, the row it always described. Equivalent by construction: both take the first alternative with any match under requireRect: false, and disclose that alternative's count. That leaves the root façade with no consumer for findSelectorChainMatch, so it goes the way of resolveSelectorChain — dropped from the string-only façade, kept on the published ./ast surface. Its façade-twin type SelectorChainMatch dies with it (fallow caught it). resolveSelectorChainWithPolicy matched twice on the uniqueness path: once via resolveSelectorChain, then again to fill matchedNodes. Hoisting the single list call above the row switch removes that second pass, collapses two duplicated ambiguous literals into one helper, and drops a `?? [resolution.node]` fallback that was unreachable — a resolution implies its alternative matched, so the list is never null there. While hoisting: the resolved arm's matchedNodes can describe a different alternative than resolution.selector, because uniqueness skips an ambiguous alternative to try the next one. Unreachable today (only first-match callers read it, where both come from one list), and left as-is rather than silently changed — but the doc claimed "the alternative it came from", so it now says what is actually true. Tests: is exists gets a caller-level pin on the shared ambiguous fixture — passes with matches: 2 where its fail-closed siblings refuse — proven red by pointing it at readUnique. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * test: discriminate is exists's row by alternative, guard the façade structurally #1715 review, second regression-validity gap. The `is exists` pin observed only `pass: true` and `matches: 2` on a fixture whose first alternative was merely TIEBREAKABLE — so disambiguation succeeded there and reported the same count first-match would. `readAny`, `readText`, and the pre-migration raw lookup all produced that, and only the readUnique swap I had checked went red. One mutation proven is not the same as the row being pinned. `exists` exposes no node ref, so the row has to be read off WHICH alternative answered. New fixture: alternative one matches two nodes that are genuinely indistinguishable (same depth, same area, both on screen) so the tiebreak declines; alternative two matches exactly one. First-match answers from alternative one; every uniqueness row skips the undecidable alternative and answers from alternative two. Asserting the selector now separates them — readText and readUnique both fail with `id="save-unique"` where `label="Save"` is expected. Restoring the raw lookup stays behaviourally invisible, though: findSelectorChainMatch is equivalent to the readAny row it migrated to, which is precisely why that migration preserved semantics. No fixture assertion can catch that revert, so the guard is structural — the façade's export list must not carry resolveSelectorChain, findSelectorChainMatch, or selectorResolutionKnobs. Follows the packages/maestro index.test.ts absence-assertion precedent. Verified red by re-exporting the lookup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * fix: cover selector routes in device replays * test: simplify selector replay regression --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-11 07:34:11 +02:00
Michał Pierzchała	8a6ddbc11d	fix(test): repair Android replay fixtures against live device reality (#1538) ... * fix(test): repair Android replay fixtures against live device reality Three Android fixture defects from the #1482/#1484 full-tier suite, none of which ever executed in CI (both nightlies since failed on adb infra before the suite ran). All three verified live on a fresh API 36 emulator with a pixel_7-geometry AVD and a Release fixture APK: - 01-navigation-scroll.ad clicked label=Catalog, but the expo-router NativeTabs cart badge leaks '0 new notifications' into the tab's content description even while hidden, and unselected native tabs expose no child text node - exact match can never hit. Target the composed label the device actually exposes (deterministic at fixture start: cart is 0 after --relaunch). The badge does not leak on iOS, so the iOS twin keeps label="Catalog". - checkout-form-android.ad opened by iOS display name 'Agent Device Tester'; Android open resolves packages (the APK label is 'Agentdevicelab'), so APP_NOT_INSTALLED was guaranteed. Use the package id, matching gesture-lab-android.ad. - gesture-lab-android.ad aimed every gesture at y=700, above the gesture card (its targets span y754-1329 on pixel_7 geometry; the home screen gained content above the card since authoring). Re-aim pans inside the exact-two-pointer zone, flings on the image clear of that zone, and pinch/rotate/transform at the card center. Verified: full suite passes 2/2 via the public test command (20 + 32 steps replayed). Refs #1478 * docs(test): pin the Android gesture fixture's validated emulator geometry The re-aimed coordinates are validated on CI's profile (pixel_7 1080x2400 @420); any booted emulator can receive them via test-app:replay:android, so the fixture and README now say which geometry the numbers mean and what a mismatch failure looks like. The checkout twin is selector-driven and unconstrained. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-01 13:50:48 +02:00
Michał Pierzchała	32b9db2d7a	test: add Android full emulator coverage (#1484) ... * test: add Android full emulator coverage * ci: package Android helpers before nightly coverage * fix: harden Android nightly runtime evidence * fix: expose trace artifacts in MCP schema * style: format Android coverage manifest * fix: address Android coverage review findings * refactor: share live device coverage helpers * fix: restore fixture landmarks in device smokes * refactor: centralize live artifact assertions * fix: normalize fixture canary visibility	2026-07-30 21:07:06 +02:00
Michał Pierzchała	4e4ecdea0d	test(ios): expand simulator e2e coverage (#1408) ... * test(ios): expand simulator e2e coverage * test(ios): make coverage checks host portable * test(ios): handle deep link confirmation * test(ios): fix deep link prompt selector * ci: stabilize full simulator nightly * test(ios): stabilize permission prompt lifecycle * test(e2e): wait for route-specific landmarks * test(e2e): reset permissions from inactive app * test(e2e): redeliver trusted cold deep links * test(ios): verify orientation native readback * test(ios): stabilize simulator permission coverage * test(ios): wait for tab target after deep link * test(ios): paginate full event timeline * test(ios): simplify event pagination coverage * test(ios): stabilize simulator e2e coverage * test(ios): model simulator recorder lifetime * ci(test-app): cache fixture dependencies * fix(ci): isolate test app cache by node * fix(ios): settle fixture route navigation * test(ci): waive unbenchmarked ios system UI help * fix(ios): tolerate delayed simulator scale lookup * 0.20.1 * test(ci): remove superseded system UI waiver * fix(ios): harden simulator e2e reliability * chore: clarify Apple runner CI steps * fix(ios): wait before fixture home snapshot * fix(ios): require exact catalog navigation * chore(ci): format rebased workflows * fix(ios): retry unobserved fixture navigation * refactor(test): remove iOS e2e workarounds * fix(ci): verify fixture artifact provenance * fix(ci): align fixture artifact fingerprints * fix(ci): use unified Android helper packager * perf(ci): scope fixture build concurrency * chore: format fixture artifact tests * test(ios): update split Apple coverage owner * fix(ios): accept deep-link confirmation alerts	2026-07-28 20:08:45 +02:00
Michał Pierzchała	287cc18c29	fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216) (#1393) ... * fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216) #1315 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe` and `gesture rotate`'s `velocity`, but shipped without the migration guide, the repository sweep, or the parse-time error that issue #1216's own checklist gates a removal on. The sweep finds what that left behind: both `06-swipe-gestures.ad` integration fixtures still carried the 5-argument swipe and would fail at replay, two tests still asserted the removed shapes, and two branches still read the retired positional. Argument arity for every public gesture syntax now lives in one table keyed off the canonical `GESTURE_KINDS`, so a new kind cannot skip it and a form removed from the CLI is removed from `.ad` in the same edit. Both callers read it: the CLI argv parse, and a new `.ad` preflight. A stale script now fails when it is parsed — before the replay executes any device action — naming the line and computing its rewrite, instead of running up to that step and failing as a repairable divergence. The preflight checks arity only: `${VAR}` tokens resolve after planning, and interpolation never splits a token, so the count is decidable while the values are not. Deleting the dead duration read in `readSwipeGeometry` would have left `replay export` emitting no duration, handing Maestro's 400ms default to a gesture the script runs at 100ms, so the export now states `duration: 100`. `.ad` positional gesture parsing is NOT removed. Its only remaining callers are the CLI argv parse and the `.ad` line parse, both the current public syntax rather than a bridge to an older one, so there is nothing to migrate off. ADR 0013 records that and drops the "compatibility" framing that made it read as debt. Both migrated fixtures verified on real devices with the repo's own CLI: iOS simulator 34.9s, Android emulator 45.9s. * fix(gestures): reject removed swipe input at the Node/MCP boundary Review findings on d88c6ed8. P1: `interactionDaemonWriters.swipe` hand-projects five fields, so a JavaScript caller's `durationMs` was dropped before the daemon's `readSwipeInput` could reject it and a default-duration fling ran instead — the exact silent reinterpretation the guide promises does not happen. `gesture` was already safe because its writer runs `readGestureInput` -> `readGesturePayload`, which rejects the removed keys; `swipe` was the one surface with no reader of its own. The rejection now lives in contracts and is shared by the client writer and the daemon handler, so there is one rule and one message. The SDK regression covers all four removed keys and asserts the transport is never reached; reverting the writer call fails it on `swipe durationMs`. P2: the preflight's retired-slot test required a numeric token, so `swipe 197 650 197 300 ${DURATION}` fell back to bare usage text. An unresolved `${VAR}` now counts as the retired slot and is carried into the pan rewrite, while a stray flag or word stays a plain usage error. P2: the removal shipped in 0.20.0, not 0.21 — removal commit 6d99914f4 is contained in tag v0.20.0. The guide said 0.21 because the CHANGELOG still files it under `Unreleased`; the tag is the truth (headings lag several releases repo-wide, so that is pre-existing and left alone). The `.ad` grep recipe now matches variable-backed durations too. * docs(gestures): make the migration sweep and MCP claim accurate Re-review findings on 328bad8e (both migration-guide accuracy). The saved-script sweep matched only the five-argument `swipe` form; it missed timed `gesture fling`, timed `gesture swipe`, and `gesture rotate ... velocity`, so the repository-cleanliness step was incomplete. Provide one grep per retired form (number-or-`${VAR}` token), each verified to flag the removed shape and skip the valid one. The MCP section claimed the structured rejection carries the CLI's concrete replacement command. It does not: `readGesturePayload` and `assertNoRemovedSwipeInput` return a message that names the removed key and the replacement command (e.g. `gesture fling does not accept durationMs; use gesture pan for timed movement`) but not the fully-substituted coordinate rewrite, because the structured request carries no positional string to rewrite. Describe what the structured path actually returns. * docs(gestures): make the .ad sweep parser-aligned Re-review finding on 4fd5d39: the documented sweep required literal single spaces and unquoted numeric tokens, but the `.ad` tokenizer separates on any whitespace (`/\s/`, so tabs too) and accepts double-quoted tokens, so tab-separated or quoted-duration stale lines the parser rejects were missed. Rewrite the patterns to follow the tokenizer: `[[:space:]]+` between tokens and a numeric slot that accepts a bare or double-quoted number (optionally signed) or `${VAR}`. Requiring a digit in the numeric slot keeps a trailing flag like `--count` from being read as the retired positional. Verified against a fixture of tab/space/quoted/quoted-var/negative encodings that the four greps flag exactly the lines `parseReplayScriptDetailed` rejects and none it accepts, and that the live repo sweeps clean. Also state plainly what a regex cannot promise: the parser is the authoritative gate — every retired form is rejected at parse time before any device action, so running the suite finds every stale line by construction and a missed grep can never reach execution. The grep stays a bulk pre-flight, and the "every affected line" claim is scoped to that.	2026-07-26 11:41:02 +02:00
Michał Pierzchała	8bcefb754a	feat: add runtime command boundary (#412) ... * feat: add runtime command boundary * refactor: harden runtime command boundary * fix: address runtime boundary review * fix: preserve selector snapshot flags * fix: preserve selector get and screenshot cleanup * fix: harden runtime boundary follow-ups * fix: close runtime parity gaps * test: harden android replay navigation * fix: close screenshot surface edge cases * test: harden packaged runtime API smoke * fix: close runtime review edge cases * test: isolate CLI state dir in unit helpers	2026-04-16 13:03:31 +02:00
Michał Pierzchała	497a54deb7	test: expand replay coverage and CI reporting with JUnit (#317) ... * test: add benchmark .ad scripts for iOS and Android (02-06) Add 10 new replay scripts covering deep navigation, scroll discovery, text input & keyboard, app lifecycle, and swipe gestures for both iOS simulator and Android emulator. Each script combines multiple command types (snapshot, click, wait, is, find, get, scroll, scrollintoview, swipe, back, home, keyboard, screenshot, appstate) to provide comprehensive baseline coverage for version-over-version comparison. https://claude.ai/code/session_01VkHLzVc7NULQFaWHnhuBCm * test: move replay benchmarks out of CI suite * test: trim replay benchmark script comments * test: relaunch replay app opens * fix: stop focusing iOS simulator for screenshots * fix: open Simulator app on cold boot * test: trim iOS cold boot coverage * test: move replay benchmarks into integration suites * ci: align Android emulator with local setup * ci: raise iOS replay daemon timeout * test: add replay junit reports * ci: use supported Android emulator profile * fix: stabilize replay cleanup and Android CI flows * test: trim flaky replay coverage * fix: exclude replay tests from daemon request timeout * test: relax android scroll discovery replay * refactor: rename ios simulator app opener * ci: remove obsolete android replay timeout override * test: remove duplicate android replay opens * test: relax ios swipe replay assertion * ci: move full replay suites to nightly * ci: extract shared replay workflow setup * ci: fail nightly replay jobs on test errors --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-03-30 11:45:49 +02:00
Michał Pierzchała	c393ffe612	feat: add light test runner for .ad scripts (#266) ... * feat: add metadata-aware ad test runner * test: migrate platform integrations to replay suites * fix: stop macos runner during close * fix: harden ad test runner * fix: tighten ad test runner retries and reporting * refactor: tighten replay test suite runtime * test: cover replay suite helpers * docs: trim replay test guidance in skill * refactor: extract replay test cli helpers	2026-03-26 19:21:16 +01:00