4 Commits

Author	SHA1	Message	Date
Michał Pierzchała	e3f3a2488e	test(android): restore full-tier lifecycle and observability scenarios (#1781 A1) (#1793) ... * test(android): restore full-tier lifecycle and observability scenarios (#1781 A1) The nightly Android job has failed on `click id="automation-request-microphone"` since the full tier landed: `settings permission reset microphone` runs `pm revoke`, and revoking a *granted* runtime permission kills the app process, so the round after an accept clicked into an empty launcher surface. Reproduced on the pinned CI image (android-36 google_apis_playstore, Pixel 7): pm revoke leaves pid 4259 alive when the permission is denied and kills it when it is granted, with NexusLauncher resumed afterwards. Fixing that exposed the rest of a scenario that had never executed end to end: the post-revoke readback cold-started on the tabs home instead of Automation lab, the relaunched Automation lab needed its controls revealed, the Form tab does not exist on the Automation root route, that section needs the system IME back, and the IME diagnostic sits above the bottom of the form. The observability scenario then failed the same way (reveal distances tuned for a taller device) plus an event-timeline walk whose page was smaller than the events each page read appends. Validated live against a local Pixel_7_CI emulator (API 36, same profile as the lane): the full tier now runs bootstrap -> inventory -> automation-system -> form-input -> keyboard-ime -> capture-close -> lifecycle-system -> observability-artifacts and stops only in full:fixture-replays. * test(android): repair the drifted fixture replays and pin the catalog canary (#1781 A1) Review follow-up. The nested batch regression now checks a sibling card instead of the notice that owns `dismiss-notice`: resolving a child already proves its parent is present, so the old target could not fail on its own. Confirmed on a Pixel 7 / API 36 emulator that `gesture-lab-card` and `dismiss-notice` are on screen together at the scenario's existing 0.3 reveal (both present at 0.2-0.4; the card is gone by 0.5). Getting a full-tier run to complete then required repairing what the lane had never executed: - `01-navigation-scroll.ad` clicked `label="Catalog, 0 new notifications"`. #1543 made the cart badge conditional, so the live label is `Catalog` — what the iOS twin already used. - The catalog scroll canary lives inside the scrolling content, and Android accessibility snapshots carry on-screen nodes only, so every state except the initial `top` was unobservable: `wait "Catalog scroll: down|bottom|up"` could never pass, whatever the swipe coordinates were. `stickyHeaderIndices` pins that one line, which makes all four states readable at any offset on both platforms rather than tuning the .ad around a canary that scrolls away. - `gesture-lab-android.ad` started its multi-pointer gestures at y=1040, inside the target when the file was last repaired but 90px from its top edge after #1567 moved the card (targets now span y=949-1525). The second pointer landed outside the view, which reads as "the gesture did nothing". Multi-pointer gestures now start at the target centre, and the header comment records the geometry they depend on. Evidence: the lane's own command (`AGENT_DEVICE_ANDROID_E2E_TIER=full` over smoke-android-emulator.test.ts) passes end to end on a Pixel 7 / API 36 AVD with a CI-equivalent fixture APK (cached native + head JS through the same repack the workflow runs): 9/9 scenarios, 153s.	2026-08-18 09:57:24 +02:00
Michał Pierzchała	338aa2a0d5	refactor: route every native selector resolution through the policy interface (#1715) ... * refactor: route every native selector resolution through the policy interface #1649 declared the per-caller ambiguity matrix; four native call sites still bypassed it, spreading `selectorResolutionKnobs(row)` into a raw `resolveSelectorChain` instead of naming the row. That left the "one interface" claim aspirational: a caller could restate its contract as engine knobs and nothing would notice. - `is` non-exists, `get text`/`get attrs`, find's read actions, and the covered-selector diagnosis probe now call `resolveSelectorChainWithPolicy` with their existing row. Semantics are byte-identical: the knob-backed branch of that interface forwards to the same engine call the call sites built by hand. - The façade drops `resolveSelectorChain` and `selectorResolutionKnobs`, so no knob-taking resolver is reachable from outside the package and a call site cannot re-acquire the knobs even by accident. `requireUnique`/`disambiguateAmbiguous` are now named in exactly one function, which `resolve-with-policy.ts` and the replay resolver both derive through. - `get` names the two rows it may consume as a type, so pointing it at any other ambiguity contract is a compile error. Tests: selector-read-policy.test.ts pins which row each read command consumes, end to end, on one ambiguous fixture — the only tree the rows disagree on. Each assertion was proven red by re-pointing its caller at a neighbouring row. The knob-consistency check moves into the package beside the now-private helper. Test call sites that used the raw resolver move to `resolveRecordedTarget`, the same knobs and the path that actually replays a recorded chain. Extracting the failure branch drops `resolveSelectorInteractionTarget` below the complexity threshold; its `fallow-ignore` waiver is removed (verified load-bearing before the extraction, unnecessary after). Closes #1630. Structural stages (occlusion, off-screen, promotion, poll budget) stay per-caller pipeline code, tracked in #1656. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * test: observe which node find's row selected, not just that one existed #1715 review, P2: the find row assertion was only half a pin. `find exists` returns `found: true` for any resolved node, and the `list` call it leaned on goes through listFindMatches — a path that consumes no policy row at all. So repointing findFirstLocatorMatch at `readText` left both assertions green while selection silently moved from the document-order head to the tiebreak winner. Assert through `find get_attrs`, which returns the ref of the node the row actually selected. Both neighbouring rows are now red: `readText` fails '@e3' !== '@e2' (the move the old test missed), `readUnique` fails by refusing the ambiguous screen. `exists` stays as a second, weaker assertion on the same resolution. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * refactor: route is exists through the matrix, collapse the double match pass Follow-up tightening on the same seam. `is exists` reached findSelectorChainMatch directly while the `readAny` row's own doc claimed to serve "`exists` and find's read-only actions" — true of the docs, not of the code, which is the unverifiable-claim shape #1656's review called out. It now names `readAny`, the row it always described. Equivalent by construction: both take the first alternative with any match under requireRect: false, and disclose that alternative's count. That leaves the root façade with no consumer for findSelectorChainMatch, so it goes the way of resolveSelectorChain — dropped from the string-only façade, kept on the published ./ast surface. Its façade-twin type SelectorChainMatch dies with it (fallow caught it). resolveSelectorChainWithPolicy matched twice on the uniqueness path: once via resolveSelectorChain, then again to fill matchedNodes. Hoisting the single list call above the row switch removes that second pass, collapses two duplicated ambiguous literals into one helper, and drops a `?? [resolution.node]` fallback that was unreachable — a resolution implies its alternative matched, so the list is never null there. While hoisting: the resolved arm's matchedNodes can describe a different alternative than resolution.selector, because uniqueness skips an ambiguous alternative to try the next one. Unreachable today (only first-match callers read it, where both come from one list), and left as-is rather than silently changed — but the doc claimed "the alternative it came from", so it now says what is actually true. Tests: is exists gets a caller-level pin on the shared ambiguous fixture — passes with matches: 2 where its fail-closed siblings refuse — proven red by pointing it at readUnique. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * test: discriminate is exists's row by alternative, guard the façade structurally #1715 review, second regression-validity gap. The `is exists` pin observed only `pass: true` and `matches: 2` on a fixture whose first alternative was merely TIEBREAKABLE — so disambiguation succeeded there and reported the same count first-match would. `readAny`, `readText`, and the pre-migration raw lookup all produced that, and only the readUnique swap I had checked went red. One mutation proven is not the same as the row being pinned. `exists` exposes no node ref, so the row has to be read off WHICH alternative answered. New fixture: alternative one matches two nodes that are genuinely indistinguishable (same depth, same area, both on screen) so the tiebreak declines; alternative two matches exactly one. First-match answers from alternative one; every uniqueness row skips the undecidable alternative and answers from alternative two. Asserting the selector now separates them — readText and readUnique both fail with `id="save-unique"` where `label="Save"` is expected. Restoring the raw lookup stays behaviourally invisible, though: findSelectorChainMatch is equivalent to the readAny row it migrated to, which is precisely why that migration preserved semantics. No fixture assertion can catch that revert, so the guard is structural — the façade's export list must not carry resolveSelectorChain, findSelectorChainMatch, or selectorResolutionKnobs. Follows the packages/maestro index.test.ts absence-assertion precedent. Verified red by re-exporting the lookup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HuKzQWn6WQcMYaAZVvJzdD * fix: cover selector routes in device replays * test: simplify selector replay regression --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-11 07:34:11 +02:00
Michał Pierzchała	8a6ddbc11d	fix(test): repair Android replay fixtures against live device reality (#1538) ... * fix(test): repair Android replay fixtures against live device reality Three Android fixture defects from the #1482/#1484 full-tier suite, none of which ever executed in CI (both nightlies since failed on adb infra before the suite ran). All three verified live on a fresh API 36 emulator with a pixel_7-geometry AVD and a Release fixture APK: - 01-navigation-scroll.ad clicked label=Catalog, but the expo-router NativeTabs cart badge leaks '0 new notifications' into the tab's content description even while hidden, and unselected native tabs expose no child text node - exact match can never hit. Target the composed label the device actually exposes (deterministic at fixture start: cart is 0 after --relaunch). The badge does not leak on iOS, so the iOS twin keeps label="Catalog". - checkout-form-android.ad opened by iOS display name 'Agent Device Tester'; Android open resolves packages (the APK label is 'Agentdevicelab'), so APP_NOT_INSTALLED was guaranteed. Use the package id, matching gesture-lab-android.ad. - gesture-lab-android.ad aimed every gesture at y=700, above the gesture card (its targets span y754-1329 on pixel_7 geometry; the home screen gained content above the card since authoring). Re-aim pans inside the exact-two-pointer zone, flings on the image clear of that zone, and pinch/rotate/transform at the card center. Verified: full suite passes 2/2 via the public test command (20 + 32 steps replayed). Refs #1478 * docs(test): pin the Android gesture fixture's validated emulator geometry The re-aimed coordinates are validated on CI's profile (pixel_7 1080x2400 @420); any booted emulator can receive them via test-app:replay:android, so the fixture and README now say which geometry the numbers mean and what a mismatch failure looks like. The checkout twin is selector-driven and unconstrained. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-01 13:50:48 +02:00
Michał Pierzchała	32b9db2d7a	test: add Android full emulator coverage (#1484) ... * test: add Android full emulator coverage * ci: package Android helpers before nightly coverage * fix: harden Android nightly runtime evidence * fix: expose trace artifacts in MCP schema * style: format Android coverage manifest * fix: address Android coverage review findings * refactor: share live device coverage helpers * fix: restore fixture landmarks in device smokes * refactor: centralize live artifact assertions * fix: normalize fixture canary visibility	2026-07-30 21:07:06 +02:00