7 Commits

Author	SHA1	Message	Date
Michał Pierzchała	957a6727f8	fix(android): publish covered state from exact order evidence (#1981) ... * fix(android): unify snapshot occlusion across API levels * fix(android): preserve exact occlusion evidence * fix(android): restore collective occlusion coverage * fix(android): preserve snapshot evidence across consumers	2026-08-24 18:09:05 +02:00
Michał Pierzchała	84e6f9bf2d	refactor(android): raw is the acquired tree; one presentation for dialog recovery; residues declared (#1832 C3) (#1865) ... * refactor(android): raw is the acquired tree; one presentation for dialog recovery; residues declared (#1832 C3) - C3: the three regular-projection pruners (invisible subtrees, stale application windows, covered same-window surfaces) move out of parseUiHierarchyTree into the projection as a non-mutating classification (collectAndroidHiddenNodes in ui-hierarchy-visibility.ts). --raw presents the acquired tree; interactive ⊆ regular ⊆ raw by construction. Hidden-content hints and the scope root are derived per projection over retained children, which is what the mutating pruners implied. Property-checked identical to main for regular/-i/depth/scope over 12,000 random tree × projection pairs; raw grew on 2,514/3,000 and never shrank. - Android blocking-dialog recovery routes through buildSnapshotState (the one presentation), which moves to src/daemon/snapshot-state.ts below the daemon-server type cycle; importers repointed, its tests mirror the module. - Freshness route signature drops role/selected (Android never carries them). - Residues declared at their sites and in CONTEXT.md; docs + CHANGELOG. - ui-hierarchy.ts split by question: node predicates (ui-hierarchy-node.ts), regular-projection visibility (ui-hierarchy-visibility.ts), scope (ui-hierarchy-scope.ts); 974 → 634 LOC. * test: lower the snapshot.test.ts size pin to its new length * fix(android): dialog recovery acts on the presentation's occlusion result Review P1 on #1865: routing blocking-dialog recovery through buildSnapshotState made the occlusion result available but nothing consumed it. containsBlockingDialog scanned every node and findCloseAppButton returned the first text match with a rect, so a stale ANR surface left under the foreground one could still trigger recovery, and a covered "Close app" could be tapped ahead of the visible top button — the disagreement the routing was supposed to remove. Both decisions now filter through isSnapshotNodeInteractionBlocked, the shared predicate over the annotator's structured result. Two regressions cover it, both proven red against an unfiltered selection: a covered Close app preceding a visible one (asserts the visible center is tapped) and a fully covered dialog signal (asserts recovery does not trigger, no tap dispatched). Rebase reconciliation: screenshot-runtime.ts arrived on main (#1878) importing buildSnapshotState from its old home; repointed to src/daemon/snapshot-state.ts with the other importers.	2026-08-19 18:17:45 +02:00
Michał Pierzchała	cd9a7ce41b	test(android): add comprehensive emulator E2E coverage (#1482) ... * test(android): add catalog emulator smoke coverage * test(android): use stable snapshot diff mutation * test(android): assert actual back destination * test(android): separate keyboard and fill IMEs * fix(ci): keep Android timing report in one shell * refactor(test): simplify simulator e2e coverage * test(android): assert stable diff landmarks * fix(android): release snapshot helper gracefully * fix(android): fully release snapshot helper runtime * test(android): report coverage classifications * fix(android): stabilize accessibility root capture * fix(android): bound UiAutomation connection * ci: upload worktree daemon diagnostics * fix(android): cancel stalled wait captures * fix(android): bound helper fallback lifecycle * fix(android): harden emulator e2e lifecycle * fix: align e2e changes with kernel package * test(android): prove alert helper reuse directly * fix(android): cancel stalled settle captures * fix(android): separate helper retirement budgets	2026-07-30 15:10:21 +02:00
Michał Pierzchała	a0aa02579b	build(android): unify the 4 helper build/package scripts behind one parameterized pair (#1466) ... * build(android): unify snapshot/ime helper build+package scripts Replace the four ~75%-duplicated shell scripts with one parameterized build script and one parameterized package script, mirroring scripts/build-xcuitest-apple.sh's env-var-driven pattern. The helper is selected via AGENT_DEVICE_ANDROID_HELPER or a first positional arg; per-helper differences (HELPER_DIR/PACKAGE_NAME, snapshot's test-compile+run step, ime's aapt2 resource-compile step, and the manifest JSON fields) live in small case blocks. All package.json entry points keep their names and output paths. Verified byte-level equivalence between main and this branch: identical unzip -l listings, identical classes.dex SHA-256 for both helpers, and identical manifest fields (only the per-signing-run sha256 differs). Fixes #1461 * chore: drop stale android/multitouch-helper .gitignore entries The multitouch helper was consolidated away in #1281; these two lines were never cleaned up.	2026-07-28 17:39:35 +02:00
Michał Pierzchała	f087a5938e	fix: align Android Maestro gesture dispatch (#1356) ... * fix: align Android Maestro gesture dispatch * fix: preserve Android gesture guarantees	2026-07-21 18:44:08 +02:00
Michał Pierzchała	1a1ef7c419	feat(android): one persistent automation helper owning snapshot + viewport + canonical injection (#1281) ... * feat(android): consolidate touch injection and gesture viewport into the persistent snapshot helper (#1275) One Android automation helper now owns snapshot capture, gesture viewport resolution, and canonical one-/two-pointer plan injection. A live persistent helper session executes gesture/viewport commands over its socket protocol; without a session the same APK runs one-shot via am instrument. The separate one-shot multitouch helper APK is deleted (atomic replacement, no fallback). Touch scheduling/injection is extracted into focused Java classes (TouchPlan, TouchPlanInjector, PointerEventSchedule, GestureViewportReader) instead of growing SnapshotInstrumentation. ADR 0013 amended. * fix(android): stop a structurally-failed helper session before the one-shot viewport retry A structured ok=false viewport response leaves the session process alive, and Android permits only one instrumentation owner of UiAutomation - running the one-shot fallback against a still-live helper contends with it and masks the original structured failure. Stop the session first; regression pins that the one-shot retry only executes once the session is gone. * refactor(android): extract helper touch dispatch into focused classes; split session tests; document helper API v2 (PR #1281 review) Addresses findings 2 and 3 from PR #1281 review (finding 1, viewport session-stop ordering, was already fixed in 5961b9247). - Extract SnapshotInstrumentation.java's one-shot/session touch dispatch into TouchCommandHandler.java (viewport/gesture population, UiAutomation- parameterized) and SessionResponseWriter.java (session response encoding), with shared PROTOCOL/HELPER_API_VERSION/OUTPUT_FORMAT constants moved to a tiny HelperProtocol.java. SnapshotInstrumentation.java shrinks from 908 to 803 lines; wire format (header keys/values, error shapes) is unchanged. - Split touch-helper.test.ts (~720 lines) into touch-helper.test.ts (normalize/parse/one-shot gesture+viewport+result envelope) and touch-helper-session.test.ts (persistent-session transport + fake-session harness), moving shared device/plan/install-probe fixtures used by both files into touch-helper.fixtures.ts. - Update android/snapshot-helper/README.md to document helper API v2: the one-shot viewport/gesture modes, the android-touch-plan-v1 payload shape, and the persistent session's socket command/response contract. * fix(android): invalidate helper session after APK replacement; recycle viewport windows; align ADR 0002 (PR #1281 re-review) - prepareAndroidTouchHelper now mirrors the snapshot path: when ensureAndroidSnapshotHelper replaces the APK (install.installed), the persistent session started against the previous binary is stopped before any touch command, so gestures run one-shot against the fresh install instead of a dead/stale session socket. Regression: 'an APK replacement stops the stale session and the gesture runs one-shot' drives a live fake session through an outdated-install probe (new outdatedVersionAdb fixture) and asserts the session socket receives no gesture, the one-shot instrumentation path executes, and the session is gone. - GestureViewportReader.read no longer leaks AccessibilityWindowInfo: a single pass copies the active/focused and first-application bounds into locals, every window is recycled in a finally, and the existing precedence (active/focused app bounds, root-in-active-window, fallback app bounds, IllegalStateException) is applied afterwards, unchanged. - ADR 0002's touch-synthesis paragraph is amended (2026-07, issue #1275) to the shared-helper model, consistent with ADR 0013: a live persistent helper session executes touch commands directly, one-shot otherwise; the old stop-before-gesture requirement is kept as historical context. * fix(android): resolve touch helper artifact from the ADB provider like snapshots do (PR #1281 re-review) - prepareAndroidTouchHelper now uses the same artifact precedence as snapshot capture: the scoped adbProvider's snapshotHelperArtifact when present, otherwise the bundled resolver (whose strict unavailable error is preserved). The provider artifact drives both the install decision and the instrumentationRunner used for one-shot commands, so an ADB-backed provider that supplies a helper artifact but no native touch override runs snapshots and gestures against the same single helper (issue #1275). Regression: 'a provider-supplied snapshotHelperArtifact overrides the bundled artifact for touch' pins the provider packageName on the install probe, the provider apkPath on the install call, the provider instrumentationRunner on the am instrument args, and that the bundled resolver is never invoked. - ADR 0002 now states explicitly that one-shot retry applies only to idempotent reads (viewport) after the failed session is stopped; non-idempotent gesture failures surface directly. - Helper README session transport corrected: a persistent process serving one short-lived socket connection per request (the server closes each accepted connection), not a single long-lived connection. * fix(android): guard touch session reuse on helper identity, stop mismatched sessions (PR #1281 re-review) Persistent helper sessions are keyed by device, so touch reuse must also prove the live session runs the helper binary the command selected. The session record now stores its helper identity (packageName, runner, helperVersion, helperVersionCode — the same values that feed the snapshot session identity), and runAndroidSnapshotHelperSessionTouchCommand takes the requesting helper identity: on mismatch (packageName/runner always; version/versionCode when both sides define them) it stops the session and returns undefined, so the touch command runs one-shot against the selected artifact — gestures never start sessions; the next snapshot restarts one with the right artifact. Matching identity reuses the session as before. Snapshot capture identity and behavior are unchanged. Regression: 'a provider artifact that mismatches the live session helper stops it and runs one-shot' — a live fake session from the bundled fixture artifact, then a gesture through an ADB provider supplying an already-current artifact with a distinct packageName/runner (no install): the old session socket receives zero gesture commands, the session is stopped, the one-shot am instrument args end with the provider runner, and helperTransport is 'instrumentation'. * fix(android): include artifact sha in helper session identity; evict stale install memo entries (PR #1281 re-review) Same-version binary replacement changes only the APK sha, so identity guards keyed on package/runner/version/versionCode could not detect a crossover between two artifacts that differ only in bytes: - The artifact sha256 now joins the helper identity end-to-end: AndroidSnapshotHelperCaptureOptions gains helperSha256 (snapshot.ts passes artifact.manifest.sha256 alongside version/versionCode), the session record stores it, createSessionIdentity includes it (making snapshot session reuse sha-aware, consistent with the install path's existing sha check), and the touch identity guard compares it via the same both-defined rule. - ensureAndroidSnapshotHelper's install memo now evicts every other cached decision for the same device+package when it records an install/current decision, so installing B invalidates A's stale 'current' memo and a later command selecting A re-inspects the device instead of skipping the sha check. Regressions: 'a same-version artifact with a different sha stops the live session and runs one-shot' (touch-helper-session.test.ts — B owns the live session, a gesture selecting same-version different-sha A sends zero commands to B's socket, stops it, and completes one-shot) and 'installing a same-version different-sha helper evicts the stale install memo' (snapshot-helper.test.ts — A:current cached, B installed, selecting A re-inspects and reinstalls instead of serving the stale memo). Both verified to fail without their fix.	2026-07-16 17:04:42 +02:00
Michał Pierzchała	e58cbcdb5f	refactor: colocate native platform sources under android/, apple/, linux/ (#1273) ... Move the scattered root-level native projects into per-platform folders and drop the now-redundant platform prefix: - android-ime-helper/ -> android/ime-helper/ - android-multitouch-helper/ -> android/multitouch-helper/ - android-snapshot-helper/ -> android/snapshot-helper/ - apple-runner/ -> apple/runner/ - macos-helper/ -> apple/macos-helper/ - src/platforms/linux/atspi-dump.py -> linux/atspi-dump.py Only repo source paths move. Identity surfaces stay frozen so no user's runner cache is invalidated on upgrade: the derived-cache key hashes source paths relative to AgentDeviceRunner and excludes packageVersion, and the ~/.agent-device/{apple-runner,macos-helper} namespaces, the agent-device-android-*-helper artifact/manifest/protocol names, the AgentDeviceRunner Xcode project, and the `prepare ios-runner` CLI command are unchanged. Updates build/package scripts, CI, package.json files+scripts, ignore/attr/fallow configs, runtime path resolvers, and test fixtures. Also: re-base repo-root-relative refs inside the moved apple/runner for the added nesting level (gated XCUITest fixture walk + two doc links), and clean the legacy dist/apple-runner packaged output so the relocated runner can't double-ship into the wholesale-included dist (with a regression test).	2026-07-15 21:47:38 +02:00