* feat(android): consolidate touch injection and gesture viewport into the persistent snapshot helper (#1275)
One Android automation helper now owns snapshot capture, gesture viewport
resolution, and canonical one-/two-pointer plan injection. A live persistent
helper session executes gesture/viewport commands over its socket protocol;
without a session the same APK runs one-shot via am instrument. The separate
one-shot multitouch helper APK is deleted (atomic replacement, no fallback).
Touch scheduling/injection is extracted into focused Java classes
(TouchPlan, TouchPlanInjector, PointerEventSchedule, GestureViewportReader)
instead of growing SnapshotInstrumentation. ADR 0013 amended.
* fix(android): stop a structurally-failed helper session before the one-shot viewport retry
A structured ok=false viewport response leaves the session process alive, and
Android permits only one instrumentation owner of UiAutomation - running the
one-shot fallback against a still-live helper contends with it and masks the
original structured failure. Stop the session first; regression pins that the
one-shot retry only executes once the session is gone.
* refactor(android): extract helper touch dispatch into focused classes; split session tests; document helper API v2 (PR #1281 review)
Addresses findings 2 and 3 from PR #1281 review (finding 1, viewport
session-stop ordering, was already fixed in 5961b9247).
- Extract SnapshotInstrumentation.java's one-shot/session touch dispatch
into TouchCommandHandler.java (viewport/gesture population, UiAutomation-
parameterized) and SessionResponseWriter.java (session response encoding),
with shared PROTOCOL/HELPER_API_VERSION/OUTPUT_FORMAT constants moved to
a tiny HelperProtocol.java. SnapshotInstrumentation.java shrinks from 908
to 803 lines; wire format (header keys/values, error shapes) is unchanged.
- Split touch-helper.test.ts (~720 lines) into touch-helper.test.ts
(normalize/parse/one-shot gesture+viewport+result envelope) and
touch-helper-session.test.ts (persistent-session transport + fake-session
harness), moving shared device/plan/install-probe fixtures used by both
files into touch-helper.fixtures.ts.
- Update android/snapshot-helper/README.md to document helper API v2: the
one-shot viewport/gesture modes, the android-touch-plan-v1 payload shape,
and the persistent session's socket command/response contract.
* fix(android): invalidate helper session after APK replacement; recycle viewport windows; align ADR 0002 (PR #1281 re-review)
- prepareAndroidTouchHelper now mirrors the snapshot path: when
ensureAndroidSnapshotHelper replaces the APK (install.installed), the
persistent session started against the previous binary is stopped before
any touch command, so gestures run one-shot against the fresh install
instead of a dead/stale session socket. Regression: 'an APK replacement
stops the stale session and the gesture runs one-shot' drives a live fake
session through an outdated-install probe (new outdatedVersionAdb fixture)
and asserts the session socket receives no gesture, the one-shot
instrumentation path executes, and the session is gone.
- GestureViewportReader.read no longer leaks AccessibilityWindowInfo: a
single pass copies the active/focused and first-application bounds into
locals, every window is recycled in a finally, and the existing precedence
(active/focused app bounds, root-in-active-window, fallback app bounds,
IllegalStateException) is applied afterwards, unchanged.
- ADR 0002's touch-synthesis paragraph is amended (2026-07, issue #1275) to
the shared-helper model, consistent with ADR 0013: a live persistent
helper session executes touch commands directly, one-shot otherwise; the
old stop-before-gesture requirement is kept as historical context.
* fix(android): resolve touch helper artifact from the ADB provider like snapshots do (PR #1281 re-review)
- prepareAndroidTouchHelper now uses the same artifact precedence as
snapshot capture: the scoped adbProvider's snapshotHelperArtifact when
present, otherwise the bundled resolver (whose strict unavailable error
is preserved). The provider artifact drives both the install decision
and the instrumentationRunner used for one-shot commands, so an
ADB-backed provider that supplies a helper artifact but no native touch
override runs snapshots and gestures against the same single helper
(issue #1275). Regression: 'a provider-supplied snapshotHelperArtifact
overrides the bundled artifact for touch' pins the provider packageName
on the install probe, the provider apkPath on the install call, the
provider instrumentationRunner on the am instrument args, and that the
bundled resolver is never invoked.
- ADR 0002 now states explicitly that one-shot retry applies only to
idempotent reads (viewport) after the failed session is stopped;
non-idempotent gesture failures surface directly.
- Helper README session transport corrected: a persistent process serving
one short-lived socket connection per request (the server closes each
accepted connection), not a single long-lived connection.
* fix(android): guard touch session reuse on helper identity, stop mismatched sessions (PR #1281 re-review)
Persistent helper sessions are keyed by device, so touch reuse must also
prove the live session runs the helper binary the command selected. The
session record now stores its helper identity (packageName, runner,
helperVersion, helperVersionCode — the same values that feed the snapshot
session identity), and runAndroidSnapshotHelperSessionTouchCommand takes
the requesting helper identity: on mismatch (packageName/runner always;
version/versionCode when both sides define them) it stops the session and
returns undefined, so the touch command runs one-shot against the selected
artifact — gestures never start sessions; the next snapshot restarts one
with the right artifact. Matching identity reuses the session as before.
Snapshot capture identity and behavior are unchanged.
Regression: 'a provider artifact that mismatches the live session helper
stops it and runs one-shot' — a live fake session from the bundled fixture
artifact, then a gesture through an ADB provider supplying an
already-current artifact with a distinct packageName/runner (no install):
the old session socket receives zero gesture commands, the session is
stopped, the one-shot am instrument args end with the provider runner, and
helperTransport is 'instrumentation'.
* fix(android): include artifact sha in helper session identity; evict stale install memo entries (PR #1281 re-review)
Same-version binary replacement changes only the APK sha, so identity
guards keyed on package/runner/version/versionCode could not detect a
crossover between two artifacts that differ only in bytes:
- The artifact sha256 now joins the helper identity end-to-end:
AndroidSnapshotHelperCaptureOptions gains helperSha256 (snapshot.ts
passes artifact.manifest.sha256 alongside version/versionCode), the
session record stores it, createSessionIdentity includes it (making
snapshot session reuse sha-aware, consistent with the install path's
existing sha check), and the touch identity guard compares it via the
same both-defined rule.
- ensureAndroidSnapshotHelper's install memo now evicts every other
cached decision for the same device+package when it records an
install/current decision, so installing B invalidates A's stale
'current' memo and a later command selecting A re-inspects the device
instead of skipping the sha check.
Regressions: 'a same-version artifact with a different sha stops the live
session and runs one-shot' (touch-helper-session.test.ts — B owns the live
session, a gesture selecting same-version different-sha A sends zero
commands to B's socket, stops it, and completes one-shot) and 'installing
a same-version different-sha helper evicts the stale install memo'
(snapshot-helper.test.ts — A:current cached, B installed, selecting A
re-inspects and reinstalls instead of serving the stale memo). Both
verified to fail without their fix.
Move the scattered root-level native projects into per-platform folders and drop
the now-redundant platform prefix:
- android-ime-helper/ -> android/ime-helper/
- android-multitouch-helper/ -> android/multitouch-helper/
- android-snapshot-helper/ -> android/snapshot-helper/
- apple-runner/ -> apple/runner/
- macos-helper/ -> apple/macos-helper/
- src/platforms/linux/atspi-dump.py -> linux/atspi-dump.py
Only repo source paths move. Identity surfaces stay frozen so no user's runner
cache is invalidated on upgrade: the derived-cache key hashes source paths
relative to AgentDeviceRunner and excludes packageVersion, and the
~/.agent-device/{apple-runner,macos-helper} namespaces, the
agent-device-android-*-helper artifact/manifest/protocol names, the
AgentDeviceRunner Xcode project, and the `prepare ios-runner` CLI command are
unchanged. Updates build/package scripts, CI, package.json files+scripts,
ignore/attr/fallow configs, runtime path resolvers, and test fixtures.
Also: re-base repo-root-relative refs inside the moved apple/runner for the
added nesting level (gated XCUITest fixture walk + two doc links), and clean the
legacy dist/apple-runner packaged output so the relocated runner can't
double-ship into the wholesale-included dist (with a regression test).
Michał Pierzchała