Consolidate the repeated `var x = default; catchException({ x = expr }); if let m {
NSLog("..._IGNORED_EXCEPTION=%@", m); return default }; return x` shape used around
exception-prone XCUITest queries into one generic helper (RunnerTests+Exceptions.swift):
func safely<T>(_ tag:, _ fallback:, _ block:) -> T
func safely<T>(_ tag:, _ block: () -> T?) -> T? // nil-default convenience
11 uniform sites adopt it: SystemModal (4), Snapshot (2), TextEntry (2), Interaction (3).
Each drops from ~7-12 lines to 1-3. The NSLog format is preserved byte-for-byte via the tag arg
(tag "MODAL_QUERY" -> "AGENT_DEVICE_RUNNER_MODAL_QUERY_IGNORED_EXCEPTION=%@"), so the
silently-logged-and-continued path keeps its searchable format and now has a single place to add
per-tag exception telemetry.
Left inline by design (not the uniform pattern): the silent `_ = catchException` KVC reads
(elementHasFocus, snapshotHasFocus), the throw-on-AX snapshot path, executeOnMainSafely's
retry-driving catch, and the bespoke pressKeyboardReturn fallback / performElementTap branches
whose result depends on the exception message.
Behavior-preserving: same defaults, same log output, same returned values. catchException is
non-escaping, so the inout capture in safeIsActionableCandidate is preserved.
Verified: xcodebuild build-for-testing -> TEST BUILD SUCCEEDED.
Collapse the gesture-command boilerplate in the executeOnMain switch behind two seams:
- performGesture(app, idleTimeout:) -> (timing, outcome): folds measureGesture + the scroll
idle-timeout/quiescence-skip wrap. Parameterized: touch gestures wrap (idleTimeout: true,
default); synthesis gestures (pinch/rotate/transform) pass idleTimeout: false because
RunnerSynthesizedGesture governs its own timing — a distinction that was previously implicit.
- gestureResponse(message:timing:frame:) over a GestureFrame (none/touch/drag): one factory for
the success DataPayload (message + gesture timing + optional touch/drag visualization frame).
The 13 gesture cases (tap x3, tapSeries x2, longPress, drag, dragSeries, swipe, mouseClick, pinch,
rotateGesture, transformGesture) now use the seams; ~120 lines of repeated quartet (measureGesture +
idle-wrap + unsupported check + 8-field DataPayload) collapse. CommandExecution.swift 949 -> 867.
Behavior-preserving: identical outcomes, timing capture, DataPayload fields/messages, and the same
touch-wrapped vs synthesis-unwrapped distinction. No wire change (the factory emits the already
wire-locked DataPayload internally). mouseClick (throws) and swipe (returns an optional frame) keep
their bespoke measure/wrap but route the success payload through gestureResponse.
Verified: xcodebuild build-for-testing -> TEST BUILD SUCCEEDED (no warnings).
Behavior-preserving relocation: move the text-entry concern out of the (formerly 1805-line)
RunnerTests+Interaction.swift — the repo's most-changed file — into a new
RunnerTests+TextEntry.swift. Interaction.swift drops to 1068 lines (-737).
Moved verbatim (no logic changes):
- value types: TextTypingRepairMode, TextEntryTiming, TextEntryResult, TextEntryTarget
- the focus -> type -> verify -> repair pipeline (typeTextReliably + focus orchestration +
readiness polling + dropped-char/repair heuristics)
- clearTextInput and the text-entry leaf helpers (editableTextValue, isPlaceholderValue,
isGenericTextInputLabel, normalizedElementText, moveCaretToEnd, estimatedDeleteCount,
keyboardBecameVisible, keyboardElementExists)
The whole cluster is text-entry-exclusive (its only callers are within the moved code or the
already-internal command entry points), so every symbol keeps its original visibility — no
access widened. Shared helpers used by gestures/snapshot/get-text (isKeyboardVisible,
visibleKeyboardFrame, textInputAt, textInputCandidatesAt, readableText, ...) stay in
Interaction. The new file is auto-included via the project's file-system-synchronized group.
This is the safe, cosmetic half of the "TextEntry engine" architecture candidate. A deeper
extraction behind a real seam/type is intentionally deferred: it carries the #245 revert risk
and wants stronger text-entry e2e coverage first.
Verified: xcodebuild build-for-testing -> TEST BUILD SUCCEEDED. Pure relocation, no behavior change.
* fix: resolve test-app dependabot alerts
The postcss/uuid overrides added in #464 stopped applying once test-app
ended up nested under the repo-root pnpm-workspace.yaml: pnpm only honors
overrides from a workspace root, so test-app's package.json `pnpm.overrides`
were silently ignored and the lockfile drifted back to vulnerable versions.
Move the overrides into a dedicated examples/test-app/pnpm-workspace.yaml so
test-app is its own pnpm root and the overrides are honored, and add scoped
overrides for the two remaining alerts:
- postcss 8.4.49 -> 8.5.12 (XSS in CSS stringify)
- uuid 7.0.3 -> 14.0.0 (missing buffer bounds check)
- ws@8 8.20.0 -> 8.21.0 (uninitialized memory disclosure)
- brace-expansion@5 5.0.5 -> 5.0.6 (ReDoS / max bypass)
ws and brace-expansion overrides are scoped to the vulnerable majors so the
non-vulnerable ws@7 / brace-expansion@1 copies in the tree are left untouched.
* chore: drop dead lodash-es override, document test-app workspace
- Remove the no-op `lodash-es` override from the root package.json (leftover
from #368). lodash-es is no longer in the dependency tree, so the override
resolved to nothing; regenerating the root lockfile is a no-op.
- Add a comment to examples/test-app/pnpm-workspace.yaml explaining why the
file exists, so it isn't "tidied away" and the override drift reintroduced.
* fix(ios): unify multi-touch gestures on two-finger synthesis + hinted unsupported errors
Make RunnerSynthesizedGesture the single iOS multi-touch engine and drop the older
incompatible models:
- rotateGesture now drives the two-finger XCTest synthesis path (dx=dy=0, scale=1,
degrees), mirroring the pinch migration in #634. The native XCUIElement.rotate(withVelocity:)
injected a single synthetic rotation that React Native's rotation recognizer did not read
reliably; synthesis fixes it. velocity is ignored on iOS (kept in the wire contract for
compatibility; rotation direction comes from the sign of degrees).
- pinch is now synthesis on iOS and a clear UNSUPPORTED_OPERATION on tvOS/macOS. The macOS
coordinate double-tap+drag heuristic (performCoordinatePinch) is removed: synthesis is
iOS-only, so macOS multi-touch is reported honestly as unsupported rather than approximated.
- RunnerInteractionOutcome.unsupported now carries an actionable hint, mapped to ErrorPayload.hint
(#639). Every unsupported gesture/tvOS path returns a concise message plus a next-step hint
(existing messages kept verbatim).
Net: on iOS, pinch/rotate/transform all flow through one synthesis primitive. swipe/scroll/pan/
fling remain single-finger drags (correct, unchanged).
Coverage: examples/test-app/replays/gesture-lab.ad exercises pinch + rotate against the gesture
lab and asserts "pinch changed yes" / "rotate changed yes".
* fix(ios): fail-fast macOS pinch + align capability/docs with synthesis-only
Follow-through for removing the macOS coordinate pinch path (the runner now returns
UNSUPPORTED_OPERATION for macOS pinch): reject it at admission instead of round-tripping.
- capabilities.ts: pinch now matches rotate-gesture/transform-gesture (Android + iOS
simulator only); macOS dropped. Removes the now-unused isMacOsOrMobileAppleSimulator helper.
- capabilities.test.ts: pinch expected unsupported on macOS and tvOS.
- website/docs/docs/commands.md: pinch listed for Android + iOS simulators only (removed from
the macOS app-session list); documents that iOS rotate ignores the optional velocity arg
(synthesis uses a fixed duration; direction comes from the sign of degrees).
Addresses PR #645 review HIGH #2 and MEDIUM #3.
* fix(ios): surface a hinted unsupported error for synthesis gestures at admission
Removing macOS pinch from the capability matrix makes macOS pinch (and the already-excluded
rotate-gesture/transform-gesture on macOS/tvOS/physical iOS) fail fast in ensureGenericCommandReady
before reaching the runner. That left the runner's macOS-specific hint unreachable on the daemon
path, so callers only saw the generic "<cmd> is not supported on this device".
Add an optional unsupportedHint to the capability matrix and surface it at admission, so the
synthesis-only gestures fail fast (no runner round-trip) AND return an actionable hint pointing to
where they work (Android + iOS simulator). Applied to pinch / rotate-gesture / transform-gesture.
Addresses PR #645 review (P2: route macOS pinch to the hinted failure).
* refactor(ios): single CommandTraits table for runner command classification
Replace the three hand-maintained switches in RunnerTests+Lifecycle.swift
(isInteractionCommand / isReadOnlyCommand / isRunnerLifecycleCommand) with one
source of truth: CommandType.traits, an exhaustive switch returning a
CommandTraits struct (interaction / readOnly / lifecycle axes), collocated with
CommandType in RunnerTests+Models.swift.
Pure refactor: every command's classification is reproduced verbatim, and the
three predicates become one-line lookups with unchanged signatures, so call
sites are untouched. The exhaustive switch makes it a compile error to add a
CommandType without classifying it, closing the drift that historically let
tapSeries/dragSeries/keyboardReturn fall out of isInteractionCommand.
readOnly is a 3-state enum (.always/.never/.conditional); .conditional preserves
alert's action-dependent read-only behavior, resolved in isReadOnlyCommand.
Classification feeds ADR-0002 session invalidation (the read-only retry that
nulls currentApp/currentBundleId), so behavior is intentionally unchanged.
Adds the "Runner command traits" term to CONTEXT.md.
* docs(ios): note CommandTraits.readOnly .conditional is alert-only (review follow-up)
* fix(ios): classify tapSeries/dragSeries/keyboardReturn as interaction commands (#643)
* fix(ios): classify tapSeries/dragSeries/keyboardReturn as interaction commands
tapSeries and dragSeries are the series forms of tap/drag (already interaction
commands); keyboardReturn is the sibling of keyboardDismiss (already an
interaction command). All three were missing from the historical
isInteractionCommand switch — a drift the new CommandTraits table (#642) makes
visible. Classifying them as interaction commands gives them the foreground-guard
+ stabilization preflight that their single-shot/sibling forms already get.
Behavior change: these three commands now re-activate a backgrounded target to
foreground and pay the stabilization delays before running. Ships separately from
the CommandTraits refactor (#642) and should land after that bakes.
mouseClick left unchanged: macOS-only and the foreground guard interacts with
bespoke macOS activation, so it needs a macOS smoke check first.
* test: cover iOS runner series commands in perf harness
On iOS the runner lowered `pinch` to performCoordinatePinch — a tap() then a
single-finger press(forDuration:thenDragTo:). React Native reads that as a pan,
so the pinch scale never changes (reported in #629: scale stays 1.00).
Route iOS pinch through the existing two-finger XCTest synthesis path
(transformGesture / RunnerSynthesizedGesture) with zero translation and rotation,
so RN's pinch recognizer fires. macOS keeps the coordinate path.
Validated on iPhone 17 Pro against examples/test-app: the gesture-lab.ad oracle
now passes 1/1 (fling/pan/pinch/rotate); it previously failed at the pinch step.
Refs #629
* perf(ios): early-exit text-entry readiness when the keyboard is visible
The XCUITest text-entry focus/readiness loops keyed their fast-exit on
focusedTextInput(), which is intentionally hardcoded to return nil on iOS (focus
predicates are stale there). As a result stabilizeTextInputBeforeTyping always
burned its full focusTimeout (0.4s) and waitForTextEntryReadiness burned its full
readinessTimeout (2.0s) in the normal case where the software keyboard appears —
~2.4s of dead wait before a single keystroke on every type/fill.
The software keyboard becoming visible is the reliable iOS readiness signal, so
both loops now return as soon as isKeyboardVisible() is true. The warmup-first-char
echo check and post-type verify/repair remain as drop safety nets.
Measured on iPhone 17 sim (Settings search field), median type time:
25 chars: 3342ms -> 1379ms (2.4x)
52 chars: 3969ms -> 2190ms
313 chars: 10.3s -> 8.6s (remainder is genuine per-char XCUITest typing)
Reliability unchanged: 64/65 trials exact (incl. a 50-word lorem ipsum, verified
by read-back + screenshot); the lone miss triggered the existing verify/repair.
* fix(ios): don't clear an already-empty text field (fixes fill mis-navigation)
clearTextInput unconditionally ran moveCaretToEnd (an edge-tap computed from the
element frame) + a 24-key delete burst, even when the field was empty. On a field
that repositions on focus — e.g. the Settings search bar jumping bottom->top and
revealing a 'Suggestions' list — that edge-tap used a stale frame and landed on an
adjacent row (Developer), navigating away instead of clearing. fill (replace) into
the search field went to the Developer pane (0/3 correct).
Skip the clear entirely when the field's value is already empty (placeholder
treated as empty): replacing into an empty field is a no-op, and skipping avoids
the stray edge-tap. fill into the Settings search now types correctly and stays
put: 5/5 exact (read-back + screenshot).
* perf(ios): resolve text fields via typed queries, not full-tree enumeration
textInputAt used app.descendants(.any).allElementsBoundByIndex (snapshots EVERY
element) to find the text input at a point. fill drove this repeatedly: once it has
coordinates, resolveTextEntryElement re-runs textInputAt on every verify/repair poll
iteration whenever the focused-field reference goes stale (e.g. the Settings search
bar repositioning bottom->top), so the full-tree enum dominated fill latency.
Query the text-input element types directly (app.textFields/secureTextFields/
searchFields/textViews) instead. Same matches, but XCUITest resolves typed queries
without snapshotting the whole tree. Measured (iPhone 17 sim, warm runner): fill 25
chars ~14.5s -> ~4.5s (3.2x), 6/6 exact. Same primitive #632 killed for get text.
* fix(ios): address review — focus-change gate + secure-field clear
Review P1 (focus race): the isKeyboardVisible early-exit in stabilizeTextInputBeforeTyping
and waitForTextEntryReadiness fired the instant the keyboard was visible — but when it was
ALREADY up from a previous field (back-to-back fills), that is before first-responder moves
to the newly-tapped field, so app.typeText could target the old field. Gate the fast-path on
a keyboard hidden->visible TRANSITION via a shared keyboardBecameVisible(wasVisibleAtEntry:)
helper; when the keyboard was already up, fall back to the settle/timeout (the prior, correct
behavior) instead of the ~2.4s dead wait the fresh case avoids.
Review P1 (F2): clearTextInput used editableTextValue(...) ?? "" and skipped clearing on
empty — but editableTextValue returns nil for secure (and unknown) fields, so secure fields
were NEVER cleared and replace concatenated stale+new. Distinguish nil (clear) from "" (skip).
Device-validated: fresh fill fast-path preserved + exact; a second fill with the keyboard
already up still types into the correct field and replaces (not concatenates).
* perf(ios): skip backend re-read in get text for non-editable elements
readTextForNode dispatched a coordinate 'read' to the iOS XCUITest runner for
every get text, where readTextAt() enumerates the full element tree
(allElementsBoundByIndex) — ~20x slower than the snapshot already captured to
resolve the node. That re-read only recovers fuller text for editable/expandable
inputs (textField/searchField/textView/…); for all other element types the
freshly-captured snapshot node text is authoritative.
Return the snapshot node text directly for non-editable nodes with non-empty
readable text, skipping the round-trip. Measured on iPhone 17 sim: get text on a
labeled control drops from ~25s to ~0.3s steady-state. Editable inputs keep the
backend re-read (live value can exceed the snapshot).
* fix(ios): gate get text snapshot-text fast-path to iOS only
Review (P1): the fast-path skipped the backend read on Android/Linux/macOS too,
but those backends read value-first (macOS helper: AXValue→title→description;
Linux similar) whereas snapshot readable text is label-first for non-editables —
so skipping their read changed get text output. Restrict the optimization to the
iOS XCUITest path (the slow allElementsBoundByIndex re-read it targets). Adds a
test asserting non-iOS platforms still dispatch the backend read.
* feat: add e2e command perf benchmark harness + nightly CI
Adds scripts/perf, a cheap end-to-end perf benchmark that drives the built
CLI through an ordered Settings tour of ~24 commands for N rounds, on a fully
isolated daemon/state-dir and self-cleaning device, and emits JSON + Markdown
reports. Per-command timing comes from wrapping each batchable command in its
own single-step batch (daemon durationMs) plus wall-clock around the process.
Wires a scheduled + workflow_dispatch CI job (perf-nightly.yml) that reuses the
cached iOS XCUITest runner (setup-apple-replay) and the Android replay host, and
runs the CLI from source via --experimental-strip-types (no dist build).
* refactor(perf): drive the harness CLI via runCmdSync, not spawnSync
Review (P2): repo rule is to spawn processes through src/utils/exec.ts, not
node:child_process directly. Switch the perf harness's invokeCli to runCmdSync
(allowFailure so non-zero exits are recorded as samples) and add a maxBuffer
option to ExecOptions/runCmdSync (snapshot payloads exceed Node's ~1MB default).
* perf(harness): warm the runner after open so the first measured command is clean
The first interaction after open/relaunch pays the one-time iOS XCUITest runner
startup (~10s+ cold) and a per-relaunch first-AX-query settle cost (~4s). That was
landing on the first measured command each round (snapshot -i), inflating it ~10x
vs the next snapshot. Run an untimed warmup snapshot -i after establishSession, after
each round's reset-open, and after every freshRoot relaunch, so no measured command
absorbs runner startup. Noted in the report header.
* refactor(perf): address review + fix Fallow CI
- exec.ts: extract spawnRejectionError + commandCloseFailure helpers, deduping the
error/close handler clones (Fallow duplication ✗ that surfaced once the maxBuffer
change pulled exec.ts into the audit scope).
- .fallowrc: exclude scripts/perf/** (non-shipped benchmark tooling, like examples/
test-app) so its naturally-moderate functions don't trip the complexity gate.
- config.ts: drop unused exports CLI_BIN/DEFAULT_OUT_DIR; add readIntValue so
--n/--rounds/--warmup report the actual flag + reject non-integers clearly.
- harness.ts: extract toSample(); type sampleError param as CliResult.
- scenario.ts: ScenarioStep is now a discriminated union on execMode (removes step.step!/
step.args ?? []).
- comment/legend rewords (platform defaults are local-convenience/CI-overridden;
elements = node count). check:fallow now green; typecheck/lint/unit pass.
* perf(harness): downgrade sample ok when a batch step reports ok:false
Defensive belt-and-suspenders for the Codex review note: stop-only batch already
surfaces a failed step as a top-level failure (caught by invokeCli), but if an
on-error=continue mode ever keeps the batch ok while a step fails, don't silently
count that step as a successful sample — derive ok from the step's own result.ok.
* feat: forward --launch-args to adb shell am start on Android
Stacked on top of the iOS-only --launch-args PR (#598). Removes the
Android UNSUPPORTED_OPERATION guard added with the Maestro work and
threads launchArgs through all five Android open paths.
Per-path threading:
- openAndroidPackage (-p package launch + activity-fallback)
- openAndroidPackageActivity (-n component override)
- openAndroidIntent (named intent action)
- openAndroidDeepLink (-a VIEW -d <url>, with optional -p)
- openAndroidAppBoundDeepLink (-a VIEW -d <url> -p <resolved>)
`adb shell` joins its argv with spaces and feeds the result to a
device shell, which re-tokenises. The other am-start arguments are
well-known and never contain shell-significant characters, so they
round-trip untouched. Launch arguments are user-supplied and may
contain JSON, spaces, `#`, etc.; each is single-quoted unless it
consists entirely of safe shell characters (the same approach long
used in adb-driven tooling for the same reason).
Help text on --launch-args is updated to describe the Android shape
(`adb shell am start args, e.g. --es key value` for typed Intent
extras) and macOS remains the only rejected platform.
Tests:
- src/platforms/android/__tests__/index.test.ts: five new tests
cover package, activity-override, deep-link URL, app-bound URL,
and JSON-with-shell-metacharacters quoting paths.
- src/core/__tests__/dispatch-open.test.ts: the previous
"rejects Android launch arguments" test is inverted into a
forwarding test that asserts openAndroidApp receives the args.
Validated end-to-end on a Pixel emulator running a debug build whose
launcher activity reads an Intent extra to bootstrap test
configuration: a JSON value containing `#`, `/`, `:` survived
single-quoted transit through `adb shell` and arrived at the
activity unchanged.
* fix: reject unsupported launch args openings
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Michał Pierzchała