Files
callstack__agent-device/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift
Michał Pierzchała c77bc40d48 refactor(daemon): Wave 6 — migrate clipboard, app-switcher, trigger-app-event, settings, alert, react-native and capabilities onto request-bound runtimes (R55–R63) (#2021)
* refactor(daemon): migrate clipboard onto request-bound runtimes (R55)

Wave 6 unit 1 of the ADR 0019 platform-free daemon migration (#1739).
`clipboard` leaves the legacy dispatch projection: admission is now the
action-selected `readClipboard`/`writeClipboard` fact the parsed subcommand
names, and the only execution is that one bound operation.

- new `@agent-device/contracts/clipboard-runtime` facet, riding the existing
  `Interactor` seam through `interactor-operation-binding.ts`; read and write
  are separate cells because a provider can genuinely expose one half only.
- every owner states its own cells: Apple gains a `system/` facts module
  (simulator or the macOS host, matching the retired
  `supportsHostOrSimulatorSurface` closure), Android admits every real kind,
  Linux the desktop device, and HarmonyOS/Vega/web refuse -- none ever carried
  a bucket. Limrun reuses the local Android interactor and refuses on iOS;
  WebDriver rides interactor reachability like `back`/`home`.
- retires the `core/dispatch.ts` clipboard arm and handler, the descriptor's
  capability bucket and `dispatch` leaf, and the Apple plugin's clipboard
  admission closure. `handlers/session.ts` loses its inline handler (and its
  last `dispatchCommand`/`requireCommandSupported` imports) to the new
  `handlers/session-clipboard.ts`.
- `bindLocalInteractorOperationSet` collapses the byte-identical local
  interaction bind list Android and Linux each held a copy of.

Cutover row R55 with its retirement, admission-member and single-bind claims.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* refactor(daemon): migrate app-switcher onto request-bound runtimes (R56)

Wave 6 unit 2 of the ADR 0019 platform-free daemon migration (#1739).
`app-switcher` leaves the legacy dispatch projection: admission is the owner's
`appSwitcher` fact and the only execution is that one bound operation, resolved
by the generic route alongside back/home/orientation/tv-remote.

- new `@agent-device/contracts/app-switcher-runtime` facet on the shared
  `Interactor` seam, bound through the interactor catalog.
- Apple states one springboard reading for `home` and `app-switcher` (parity:
  the retired `supportsAppAndDeviceLifecycle` closure gated both off the same
  per-AppleOS row, so macOS and watchOS refuse); Android admits every real kind;
  HarmonyOS admits both kinds, restating the retired overlay membership;
  Linux/Vega/web refuse. Limrun reuses the local Android interactor and refuses
  on iOS; WebDriver rides interactor reachability.
- retires the `core/dispatch.ts` arm, the capability bucket, the `dispatch`
  leaf, `HARMONYOS_SUPPORTED_COMMANDS` membership, the Apple plugin closure, and
  the now-readerless `appAndDeviceLifecycle` row in the per-AppleOS table.
- router tests that used `app-switcher` as their legacy-dispatch stand-in move
  onto bound operations; the typed-error `supportedOn` test moves to `perf`, the
  one command that keeps a capability-matrix row after this wave.

Cutover row R56 with its retirement, admission-member and single-bind claims.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* refactor(daemon): migrate trigger-app-event onto request-bound runtimes (R57)

Wave 6 unit 3 of the ADR 0019 platform-free daemon migration (#1739).
`trigger-app-event` leaves the legacy dispatch projection: admission is the
owner's `triggerAppEvent` fact and the only execution is that one bound
operation.

The split follows ADR 0019 §2 — a facet input names no command, request, or CLI
flag. The event name pattern, the payload size limit, and the per-platform
`AGENT_DEVICE_*_APP_EVENT_URL_TEMPLATE` are daemon policy and stay in
`core/app-events.ts`; what reaches the owner is a resolved URL to open. They
also stay downstream of admission, where the retired `dispatchCommand` ran them,
so an unsupported device still reports its unsupported cell rather than an
argument error.

- new `@agent-device/contracts/app-event-runtime` facet on the shared
  `Interactor` seam, bound through the interactor catalog.
- Apple admits every leaf with a constructible interactor (no closure ever gated
  this command beyond its bucket), Android every real kind, and
  Linux/HarmonyOS/Vega/web refuse. It is the one system leaf both Limrun legs
  serve, since each implements `open`; WebDriver rides interactor reachability.
- retires the `core/dispatch.ts` arm and handler, the capability bucket, the
  `dispatch` leaf, and the session route's last
  capability-gate-then-`dispatchCommand` thunk: every leaf on that route now
  supplies a bind-and-execute thunk.
- the end-to-end delivery tests keep their shell-level assertions and move onto
  the migrated composition.

Cutover row R57 with its retirement and single-bind claims.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* refactor(daemon): migrate settings onto request-bound runtimes and retire the legacy dispatcher (R58)

Wave 6 unit 4 of the ADR 0019 platform-free daemon migration (#1739). `settings`
was the last `DISPATCH_HANDLERS` arm, so this change closes the command and
retires the legacy command dispatcher whole.

- new `@agent-device/contracts/settings-runtime` facet on the shared
  `Interactor` seam, bound through the interactor catalog. What reaches the
  owner is its own settings vocabulary (setting, state, resolved app id, typed
  coordinates); the CLI parse, the macOS setting-name gate, the clear-app-state
  app-id check and the coordinate typing are daemon policy and stay daemon-side,
  downstream of admission where the retired leaf ran them.
- Apple shares clipboard's exact host-or-simulator reading (the retired
  admission intersected the `settings` bucket with the same
  `supportsHostOrSimulatorSurface` closure); Android admits every real kind;
  HarmonyOS matches its retired overlay membership; Linux/Vega/web refuse.
  Limrun splits Android-reuse / iOS-refusal like `app-switcher`; WebDriver
  refuses unconditionally, since its interactor declares settings unsupported.
- retires `dispatchCommand`, `dispatchWithInteractor`, `dispatchKnownCommand`,
  `DISPATCH_HANDLERS`, `listRegisteredDispatchCommandNames`, and the request
  router's `executeGenericPlatformCommand` fallback. `core/dispatch.ts` keeps
  only `dispatchGestureViewport`, whose last consumers are replay/test.

Retiring the dispatcher surfaced two callers broken since Wave 5 moved `press`
onto a bound runtime: react-native overlay dismissal and the opt-in interaction
no-change retry both called `dispatchCommand(device, 'press', …)`, which has
thrown `INVALID_ARGS: Unknown command: press` on main since R48. Both now run
the same bound `tapPoint` every other touch leaf uses. The retry declares its
own callback seam rather than importing runtime admission, so the policy stays
readable without the binding stack — and that inversion, plus the dispatcher's
retirement, drops the largest type-level import cycle from 25 files to 21.

Cutover row R58 with its retirement and single-bind claims.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* refactor(daemon): migrate alert, react-native and capabilities onto facts (R59/R61/R63)

Wave 6 units 5, 7 and 9 of the ADR 0019 platform-free daemon migration (#1739),
plus the residue reclassification the tracker asks for as an analysis task.

R59 `alert` — new `@agent-device/contracts/alert-runtime` facet with four
action-selected legs (`readAlert`, `awaitAlert`, `acceptAlert`, `dismissAlert`)
on the shared `Interactor` seam. The daemon route admits and binds exactly the
leg the parsed subcommand names, and the poll and retry windows move to the
owners with it: how long a transient sheet takes to appear, and how many times
to re-ask a runner that says it is not there yet, are family mechanics, not
request policy. `src/platforms/apple/alert.ts` now holds the Apple windows
verbatim (with the macOS-helper / XCTest-runner split), and Android's legs read
the same presented tree `snapshot` publishes, which is why their occlusion
reading still holds.

Apple's cell is the retired `supportsAlertSurface` closure restated as facts —
the host-or-simulator reading widened by physical iOS — and that closure was the
per-AppleOS capability table's last reader, so `src/platforms/apple/capabilities.ts`
goes with it.

R61 `react-native` — the command's device work moved onto a bound `tapPoint`
with R48; this retires the capability gate that still stood in front of it and
moves admission ahead of the observing capture, so an owner that cannot dismiss
an overlay refuses without first spending a snapshot on it. That exposed a real
defect: the request handler chain never forwarded the request's runtime bindings
to this route, so the dismissal leg had been reaching a missing gateway ever
since R48 — only the no-overlay-detected path returned early enough to hide it.
Fixed, with a chain-level regression test.

R63 `capabilities` — the projection now reads each command's own declared
`platformExecution` uses instead of a hand-written map plus a "no capability
bucket means supported everywhere" fallback. That fallback is what let a stopped
Android AVD advertise `snapshot press fill` it cannot run, and a Vega VVD
advertise every migrated command; both collapse to the fact-derived set here.
The command itself executes nothing on a device, so it declares `none`.

Residue: `batch`, `debug` and `events` reclassify to `none` — each reaches no
device and delegates nothing that does. `replay`/`test` keep their gesture
viewport and boot-diagnostics edges, `daemon`/`web` hold platform imports in
their own CLI modules, and `react-devtools` still injects device-runtime
`runtime`, so all five stay `legacy`.

Cutover rows R59 and R61 with their retirement and single-bind claims.
Descriptors: 32 legacy at the wave checkpoint, 9 now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(daemon): restore two settings/alert sequences the migration had shifted

Self-review of the Wave 6 diff against `origin/main` found two places where the
migrated routes were faithful in what they did but not in when:

- `settings` typed its location coordinates before expiring the ref frame, so a
  request that failed on a bad coordinate no longer expired it. The retired route
  expired the frame first, then emitted its diagnostic, then typed the
  coordinates inside the leaf. Same order again.
- `alert` narrowed a frontmost-app session to "no bundle" in the daemon, which
  also stripped the bundle from the XCTest runner leg. That narrowing was only
  ever the macOS helper's, and it already lives in `platforms/apple/alert.ts`;
  the runner leg gets `session.appBundleId` unconditionally again, pinned by a
  test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(daemon): address adversarial review of the Wave 6 cutovers

Three independent reviews (behavior parity, correctness, ADR 0019 conformance)
ran against the branch. What they found, and what changed:

Correctness

- The R48 retry seam was unreachable. `captureSnapshot` builds it from the
  request's runtime bindings, but no caller forwarded them, so every retry
  resolved to a skip. The `snapshot` route now threads `inspectFacts`/
  `bindDevice` through `createSnapshotRuntime` and the daemon snapshot backend
  down to the capture.
- A retry tap that rejected escaped the capture it was decorating and turned a
  plain `snapshot` into an error. It is caught and reported as a skip, matching
  what the seam's own contract already claimed.
- The attempt is spent before the device work again, as the retired route did,
  so an owner that fails mid-flight cannot be re-attempted from a full budget.
- `react-native dismiss-overlay` reached its required `tapPoint` through `?.`
  and answered `dismissed: true` when the operation was absent. It refuses.
- `factOwnedCapabilityAvailable` indexed the facts map unguarded, and treated an
  empty `required` as proof (`[].every` is vacuously true). Both fail closed.

ADR 0019 conformance

- §6 forbids a `none` descriptor from binding a device, and `capabilities` bound
  three times to answer `logs`/`network`/`record`. Every owner composes a
  binding's facts with the same function `inspectFacts` calls, so those probes
  read back values the single inspection already carries — at the cost of a
  device claim on a read-only query. They are gone, and with them the last three
  empty-`required` admission uses.
- §9 is one admission per handler; the retry tap re-admitted on every retry
  round. It memoizes per device.
- `installFamilyCapabilityAvailable` was scaffolding this wave was scheduled to
  retire: the general projection returns the same verdict for all four
  install-family commands. Deleted.

Leftovers the cutovers created

- `requireCommandSupported` lost its last production caller when R56 migrated
  `app-switcher`: every generic-route command is admitted from owner facts
  before the dispatcher runs. The dead arm, the function, and
  `commandUsesDeviceRuntimeExecution` are removed.
- `CommandDispatchFacet`, `descriptor.dispatch`, and `explain`'s `dispatch=`
  field described a dispatcher R58 deleted.
- `request-router-android-modal.test.ts` asserted on a `dispatchCommand` mock
  whose module export no longer exists, so three assertions were vacuous.
- `generic-route-runtime-completeness.test.ts` now exists — a comment claimed it
  did. It pins the routing table as total over the generic route.
- Comments and test names describing the retired dispatcher, the deleted AppleOS
  capability table, and a react-native regression that never shipped.

Also records two deliberate provider cell changes the migration made (physical
Apple `clipboard` admitted, provider `alert` refused) and the react-native
widening to Linux, web and HarmonyOS, and drops a scratch probe file that was
committed by accident.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(apple): type the alert-absence retry instead of matching error prose

Review blocker 1 on #2021. The Apple alert legs decided retry and hint
eligibility by substring-matching error messages for "alert not found" / "no
alert", and `alert wait` swallowed *every* read failure. A dead runner, an
unreachable macOS helper or a canceled request was therefore spent as poll
budget and finally reported as `alert wait timed out`, hiding the real cause.

Both backends now state absence as typed evidence:

- The XCTest runner answers `ErrorPayload(code: "ALERT_NOT_FOUND", ...)`. It is
  diagnostic-only, so it stays `COMMAND_FAILED` on the wire and surfaces as
  `details.runnerErrorCode` — the same shape `RUNNER_BUSY` already used.
- The macOS helper adds `reason: "alert-not-found"` to its JSON error details,
  which the helper client already forwards verbatim.

`isAlertNotFoundError` reads only those two fields. `awaitAppleAlert` re-throws
anything that is not a typed absence instead of polling through it, and the
scoped-snapshot fallback hint attaches to typed absence alone.

The three tests the review asked for, plus coverage the daemon-altitude copies
could not express: a non-absence failure propagates immediately from `wait`; an
action does not retry a failure whose message merely reads like an absence; the
macOS helper's typed reason is retried like the runner's. The daemon-level
non-absence test moved to the family suite that owns this policy since R59,
lowering that file's size pin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(runtime): admit clipboard and provider operations from what execution checks

Review blocker on #2021, reproduced on a Pixel 9 Pro XL / Android 36 emulator:
`capabilities` advertised `clipboard`, then `clipboard read` failed with
`UNSUPPORTED_OPERATION: Android shell clipboard read is not supported on this
device.` Admission and execution were consulting different authorities, which
ADR 0019 §2 forbids — a bound operation must already be admitted.

Android. `cmd clipboard` has no shell implementation on every build, and the
retired bucket admitted both halves on every real Android kind, leaving the leaf
to discover the refusal after the fact. Support is now a fact: the owner probes
once per device (cached for its lifetime — a build's shell command set cannot
change while the device is up) and states `owner-capability-missing` when adb
names the condition. The probe is definitive in one direction only: adb saying
so means unsupported, a probe that cannot run means unknown, and reporting
unknown as unsupported would hide a working clipboard behind a transport
hiccup. The predicate moves to `@agent-device/contracts/android-clipboard-support`
so admission and the leaf's own defense-in-depth check cannot drift apart.

Cost, stated plainly: the first facts inspection per device now spends one adb
round trip, including for requests that never touch the clipboard.

WebDriver. `webdriver-interactor.ts` refuses through `capabilitySupported`,
while fact generation admitted from interactor reachability alone — so a
provider configured with `capabilityOverrides: { 'clipboard.read': 'unsupported' }`
was admitted and then thrown out of. The declared capability map is now an input
to fact generation, and the refusal carries the map author's own note. Applied to
every operation with an unambiguous capability key, not just the two named in
review: the mechanism is identical and a half-applied fix would leave the same
defect for `back`/`home`/`orientation`/`tap`/`fill`/`type`/`scroll`. Behavior is
unchanged by default — every one of those is `supported` or `partial` in the base
map — so only an explicit override bites. `focus`, the gesture tiers and
`trigger-app-event` keep reachability: no capability key maps to them 1:1.

Also collapses the eight identical `*RetiredDispatchProjectionProof` wrappers in
the cutover table into one parameterized factory (second review point).

Parity tests: an Android build reporting either unsupported-shell phrasing, the
probe cache, an adb failure staying admitted, and a WebDriver override refused at
admission for each keyed operation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* refactor(layering): split the Wave 6 cutover rows into a sibling module

Second review P2 on #2021. `runtime-command-cutover-table.ts` had reached 1,325
lines, past the point where one read covers it.

Wave 6's eight rows move to `runtime-command-cutover-table-wave6.ts` and are
spread back in, leaving the table at 1,095 lines. The split is by wave because
that is how these rows are retired: a wave's rows are deleted together once the
ADR declares its commands' migrations closed, and deleting a whole file is a
cleaner end than excising a run of literals from the middle of a larger one.

`retiredDispatchProjectionProof` moves to the shared extensions module, since
both tables now use it — the main table for `snapshot`/`diff`, the sibling for
its own eight.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(android): never fabricate clipboard availability from a failed probe

Review blocker on #2021. The probe I added had a `catch { return true }`, then
cached that result by device id for the runtime owner's lifetime. A transient
adb offline or timeout therefore made `capabilities` advertise the clipboard on
a build with no clipboard shell — recreating the exact lie the fix was for, and
pinning it for the rest of the session. A test locked the behavior in.

Support is now a typed verdict with three states, because "we could not ask" is
not "it works": `supported | unsupported | probe-failed`. Only a definitive
answer is cached; `probe-failed` refuses conservatively with a hint saying
support could not be determined, and is deliberately not remembered, so the next
inspection asks again.

The same change repairs the ownership boundary. Turning raw adb stdout/stderr
into a verdict is Android tool knowledge, so it belongs to the Android owner, not
to shared vocabulary — `@agent-device/contracts/android-clipboard-support` now
carries the typed union alone. The parser returns to `src/platforms/android/adb.ts`
and runs in exactly one place, behind a new `AndroidToolHost.probeClipboardShellSupport`
that hands owners the verdict. That also settles which Android home owns it:
R13 lets only `src/platform-runtime.ts` import `@agent-device/platform-android`,
so a parser shared between the package and the root leaf cannot live in the
package either.

Tests now cover the failure path the previous ones locked the wrong way: a failed
probe refuses instead of admitting, its refusal says it could not determine
support rather than claiming the build lacks it, and it is not cached — a second
inspection re-probes and admits once the device answers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* refactor(contracts): declare each interactor operation once

Second review P1 on #2021. `interactor-operation-catalog.ts` declared the same
operation set three times — a name tuple, a complete local binder map, and a
complete provider binder map — and each facet carried a mirrored
`bindLocal…Interactor`/`bindProvider…Interactor` pair whose only difference was
which interactor source to use and which label a refusal names.

There is now one row per operation, carrying its facts key, its provider refusal
label, and the facet's own executor. The local/provider split lives in the two
adapters, which differ by exactly the thing that differs: the interactor source.
Adding an operation is adding one row.

Deleted: the parallel tuple, both binder maps, 32 mirrored wrappers across ten
facet modules, and the per-facet `Local…`/`Provider…InteractorResolver` aliases
that existed only to be re-exported. Kept: every facet's typed executor, now
exported as its binding surface.

Net −563 production lines in `packages/contracts`.

Two consumers moved onto the catalog's public entry point rather than keeping a
private path to a single operation: the app-event delivery test and the provider
scenario fixture, whose two hand-bound keyboard legs are now whichever legs its
facts admit. Each facet's tests spell out the composition the retired wrappers
performed, so every assertion still exercises one executor reached through one
source.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(android): let only a clean adb exit prove clipboard support

Third review P1 on #2021. The typed verdict landed one layer too high. The
adapter probe runs `adb shell cmd clipboard get text` with `allowFailure`, so a
non-zero exit comes back as an ordinary result rather than a throw — and the
only thing standing between that result and `supported` was the missing-shell
prose check. A device that had gone offline, was unauthorized, timed out, or
failed for any other reason produced none of that prose, so it fell through to
`supported` and was then cached by device id for the runtime owner's lifetime.
The `catch` I added guarded the one path adb almost never takes.

Each adb outcome now proves only what it can:

- `exitCode === 0` is the sole evidence of support, because it is the only
  result that shows the command ran.
- The recognized missing-shell prose is the sole evidence of absence, and is
  read before the exit code — adb reports that condition non-zero, so checking
  the code first would turn every honest `unsupported` into a refusal.
- Everything else — non-zero without that prose, and the transport throw — is
  `probe-failed`, which admission refuses and the cache does not remember.

The package tests mocked the typed verdict, so they sat downstream of the bug
and could not see it. The regression is therefore at the adapter, over the raw
adb result: four planted reds (offline, unauthorized, device-not-found, generic
failure) that all returned `supported` before this change, plus the two
definitive verdicts and the ordering case that keeps `unsupported` reachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(android): never read adb's refusal prose out of the clipboard's contents

Fourth review P1 on #2021, and a second instance of the same bug it names.

The previous fix read `isClipboardShellUnsupported(stdout, stderr)` before the
exit code. On a *successful* `cmd clipboard get text`, stdout is the clipboard's
contents — arbitrary user text. Anyone who had copied "unknown command" or "no
shell command implementation" (from a terminal, a bug report, this repo) had
their own working clipboard classified `unsupported`, and the runtime owner
cached that for its lifetime. Ordering prose ahead of the exit code to keep
`unsupported` reachable traded one wrong admission for another.

The exit code is decisive on its own when it is zero, so it goes first. Only a
call that failed can carry prose about the call itself, which makes the missing-
shell phrases meaningful on non-zero exits alone:

    if (result.exitCode === 0) return 'supported';
    return isClipboardShellUnsupported(...) ? 'unsupported' : 'probe-failed';

`isClipboardShellUnsupported` now states that precondition, because reading it
on a successful call is exactly the mistake to prevent.

The same defect was already shipped in the helper's other caller.
`runAndroidClipboardShellCommand` in `src/platforms/android/device-input-state.ts`
has checked the prose before the exit code since #1950, so `clipboard read` on a
clipboard holding either phrase threw `UNSUPPORTED_OPERATION` — telling the user
their device does not support a clipboard it had just read correctly. It is not
this wave's code and not reachable from the migration, but it is the same helper
misused the same way, and documenting a precondition while leaving a caller that
violates it invites the next regression. Repaired here, with the failure ordering
otherwise unchanged: a non-zero exit still reports missing-shell as
`UNSUPPORTED_OPERATION` and anything else as the adb result error.

Both repairs are pinned by regressions that fail against the code they replace:
four exit-0 cases at the adapter (verified red against the ordering this commit
removes), and three at `readAndroidClipboardWithAdb` (verified red against
`origin/main`) covering contents that look like a refusal, a genuine missing
command, and an unrelated non-zero failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* fix(cli): bring the workflow help card back under its size budget

`Coverage (2)` has been red on `main` and on every PR branched from it since
#2020, which replaced three short Bootstrap lines with one longer line carrying
the new selection semantics. It updated the content matcher for that line but
not the size assertion beside it, so the card went to 9003 bytes against the
`< 9000` both `cli-help.test.ts` and `cli-help-topics.test.ts` enforce.

Nothing #2020 added is removed here — all of it is pinned by the matcher it
shipped, and it is the sentence agents most need. The bytes come back from a
clumsy repetition elsewhere in the card, where "settle" named itself twice in
one clause:

  ... only when you did not settle, settle reported not settled, or ...
  ... only when you did not settle, it reported not settled, or ...

which reads better short and puts the card at 8999.

That is one byte inside the budget, which is the real finding: the card has no
slack left, and the next sentence anyone adds re-opens this. The durable fix is
a base-owner call between raising the budget and moving a block down into its
sub-topic — the mechanism the card already uses, and which its own test
documents. Flagged on #2021 rather than decided here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

* test(cli): raise the workflow help-card budget to 9100

The card is a curated agent-facing reference, and #2020 grew it for a good
reason: the selection semantics it added are what an agent needs to predict
which device a bare `open` picks. Holding that content to a limit set before it
existed just moves the cost onto whoever writes the next sentence.

9100 is headroom, not a target. The previous commit left the card at 8999 of
9000 -- one byte -- which is not a state anyone should have to work in, and I
had already established there is no slack left to reclaim: no trailing
whitespace, and the only repeated runs are the deliberate column alignment in
the Escalate footer. Trimming further would have meant deleting content the
tests pin as load-bearing.

This is explicitly interim. The card is ~9KB of dense prose in one string, and
the real answer is to move a block down into its owning sub-topic -- the
mechanism the card already uses and its own test documents ("Deep content moved
out of the compact card, not deleted"). Raising the ceiling buys room to do that
deliberately instead of under a red CI.

Both enforcement sites move together, since they measure the same card through
different surfaces: `cli-help.test.ts` reads it through the CLI, and
`cli-help-topics.test.ts` through `usageForCommand`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-25 17:47:35 +02:00

2825 lines
104 KiB
Swift

import XCTest
#if AGENT_DEVICE_RUNNER_UNIT_TESTS && os(iOS)
import ObjectiveC.runtime
private final class RunnerSynthesizedSwipeFailureStub: NSObject {
@objc(synthesizeSwipeWithApplication:x:y:x2:y2:durationMs:)
class func synthesizeSwipe(
application: XCUIApplication,
x: Double,
y: Double,
x2: Double,
y2: Double,
durationMs: Double
) -> String? {
"forced private synthesis failure"
}
}
private final class RunnerSynthesizedTapFailureStub: NSObject {
@objc(synthesizeTapWithApplication:x:y:)
class func synthesizeTap(application: XCUIApplication, x: Double, y: Double) -> String? {
"forced private synthesis failure"
}
}
#endif
extension RunnerTests {
// MARK: - Main Thread Dispatch
private func currentUptimeMs() -> Double {
ProcessInfo.processInfo.systemUptime * 1000
}
private func measureGesture(_ action: () -> Void) -> (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double) {
let gestureStartUptimeMs = currentUptimeMs()
action()
return (gestureStartUptimeMs, currentUptimeMs())
}
func synthesizedSwipeFallbackHoldDuration(durationMs: Double) -> TimeInterval {
min(max((durationMs / 5.0) / 1000.0, 0.016), 0.120)
}
func coordinateDragHoldDuration() -> TimeInterval {
0.050
}
func unsupportedResponse(for outcome: RunnerInteractionOutcome) -> Response? {
switch outcome {
case .performed:
return nil
case .unsupported(let message, let hint):
return Response(
ok: false,
error: ErrorPayload(code: "UNSUPPORTED_OPERATION", message: message, hint: hint)
)
}
}
/// Optional visualization frame returned with a gesture response.
enum GestureFrame {
case none
case touch(TouchVisualizationFrame?)
case drag(DragVisualizationFrame)
}
struct GestureFallback {
let strategy: String
let message: String
let hint: String?
}
private func gestureFallback(strategy: String, from outcome: RunnerInteractionOutcome) -> GestureFallback? {
switch outcome {
case .performed:
return nil
case .unsupported(let message, let hint):
return GestureFallback(strategy: strategy, message: message, hint: hint)
}
}
/// Runs a gesture action with uniform timing capture. Touch gestures pass `idleTimeout: true`
/// (the default) to run inside the scroll idle-timeout + quiescence-skip wrapper; synthesis
/// pointer-plan gestures pass `false` because RunnerSynthesizedGesture governs their
/// own timing. Returns the captured timing and the action's outcome.
///
/// NOTE: a new SYNTHESIS gesture must pass `idleTimeout: false` — the default `true` would wrap
/// it in the scroll idle-timeout/quiescence-skip path and change its runtime behavior.
func performGesture(
_ app: XCUIApplication,
idleTimeout: Bool = true,
_ action: () -> RunnerInteractionOutcome
) -> (timing: (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double), outcome: RunnerInteractionOutcome) {
var outcome = RunnerInteractionOutcome.performed
let timing = measureGesture {
if idleTimeout {
withTemporaryScrollIdleTimeoutIfSupported(app) { outcome = action() }
} else {
outcome = action()
}
}
return (timing, outcome)
}
/// Single factory for the success payload every gesture returns (message + gesture timing +
/// an optional touch/drag visualization frame), so the field shape lives in one place.
private func gestureResponse(
message: String,
timing: (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double),
frame: GestureFrame = .none,
fallback: GestureFallback? = nil,
maestroNonHittableCoordinateFallbackUsed: Bool? = nil
) -> Response {
let data: DataPayload
switch frame {
case .none:
data = DataPayload(
message: message,
gestureStartUptimeMs: timing.gestureStartUptimeMs,
gestureEndUptimeMs: timing.gestureEndUptimeMs,
gestureFallback: fallback?.strategy,
gestureFallbackMessage: fallback?.message,
gestureFallbackHint: fallback?.hint,
maestroNonHittableCoordinateFallbackUsed: maestroNonHittableCoordinateFallbackUsed
)
case .touch(let f):
data = DataPayload(
message: message,
gestureStartUptimeMs: timing.gestureStartUptimeMs,
gestureEndUptimeMs: timing.gestureEndUptimeMs,
x: f?.x,
y: f?.y,
referenceWidth: f?.referenceWidth,
referenceHeight: f?.referenceHeight,
gestureFallback: fallback?.strategy,
gestureFallbackMessage: fallback?.message,
gestureFallbackHint: fallback?.hint,
maestroNonHittableCoordinateFallbackUsed: maestroNonHittableCoordinateFallbackUsed
)
case .drag(let f):
data = DataPayload(
message: message,
gestureStartUptimeMs: timing.gestureStartUptimeMs,
gestureEndUptimeMs: timing.gestureEndUptimeMs,
x: f.x,
y: f.y,
x2: f.x2,
y2: f.y2,
referenceWidth: f.referenceWidth,
referenceHeight: f.referenceHeight,
gestureFallback: fallback?.strategy,
gestureFallbackMessage: fallback?.message,
gestureFallbackHint: fallback?.hint
)
}
return Response(ok: true, data: data)
}
/// Gesture plans already return canonical centroid endpoints from the portable runtime.
/// Keep runner timing/fallback diagnostics, but do not leak the coordinate-drag adapter's
/// visualization frame into only the fast-fling response shape.
private func canonicalPlannedGestureResponse(_ response: Response) -> Response {
guard response.ok, let data = response.data else { return response }
return Response(
ok: true,
data: DataPayload(
message: data.message,
gestureStartUptimeMs: data.gestureStartUptimeMs,
gestureEndUptimeMs: data.gestureEndUptimeMs,
gestureFallback: data.gestureFallback,
gestureFallbackMessage: data.gestureFallbackMessage,
gestureFallbackHint: data.gestureFallbackHint
)
)
}
private func plannedGestureResponse(
plan: RunnerGesturePlan,
timing: (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double),
outcome: RunnerInteractionOutcome
) -> Response {
if let response = unsupportedResponse(for: outcome) {
return response
}
return gestureResponse(message: plan.intent, timing: timing)
}
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
func testGestureResponseIncludesSynthesizedTapFallbackDiagnostics() {
let response = gestureResponse(
message: "tapped",
timing: (gestureStartUptimeMs: 1, gestureEndUptimeMs: 2),
fallback: GestureFallback(
strategy: "xctest-coordinate-tap",
message: "Runner synthesized coordinate tap is unavailable",
hint: "Using XCTest coordinate tap fallback."
)
)
XCTAssertEqual(response.ok, true)
XCTAssertEqual(response.data?.gestureFallback, "xctest-coordinate-tap")
XCTAssertEqual(
response.data?.gestureFallbackMessage,
"Runner synthesized coordinate tap is unavailable"
)
XCTAssertEqual(response.data?.gestureFallbackHint, "Using XCTest coordinate tap fallback.")
}
func testGestureResponseIncludesMaestroNonHittableFallbackUsage() {
let response = gestureResponse(
message: "tapped via non-hittable coordinate fallback",
timing: (gestureStartUptimeMs: 1, gestureEndUptimeMs: 2),
frame: .touch(nil),
maestroNonHittableCoordinateFallbackUsed: true
)
XCTAssertEqual(response.data?.maestroNonHittableCoordinateFallbackUsed, true)
}
func testCanonicalPlannedGestureResponseOmitsDragFrameAndPreservesDiagnostics() {
let response = gestureResponse(
message: "fling",
timing: (gestureStartUptimeMs: 1, gestureEndUptimeMs: 2),
frame: .drag(
DragVisualizationFrame(
x: 160,
y: 150,
x2: 40,
y2: 150,
referenceWidth: 200,
referenceHeight: 300
)
),
fallback: GestureFallback(
strategy: "xctest-coordinate-drag",
message: "Private synthesis unavailable",
hint: "Using XCTest coordinate fallback."
)
)
let canonical = canonicalPlannedGestureResponse(response)
XCTAssertEqual(canonical.data?.gestureStartUptimeMs, 1)
XCTAssertEqual(canonical.data?.gestureEndUptimeMs, 2)
XCTAssertEqual(canonical.data?.gestureFallback, "xctest-coordinate-drag")
XCTAssertEqual(canonical.data?.gestureFallbackMessage, "Private synthesis unavailable")
XCTAssertEqual(canonical.data?.gestureFallbackHint, "Using XCTest coordinate fallback.")
XCTAssertNil(canonical.data?.x)
XCTAssertNil(canonical.data?.y)
XCTAssertNil(canonical.data?.x2)
XCTAssertNil(canonical.data?.y2)
XCTAssertNil(canonical.data?.referenceWidth)
XCTAssertNil(canonical.data?.referenceHeight)
}
#if os(iOS)
func testSinglePointerFlingFallsBackToXCTestCoordinateDragWhenPrivateSynthesisFails() throws {
let selector = NSSelectorFromString(
"synthesizeSwipeWithApplication:x:y:x2:y2:durationMs:"
)
guard
let synthesizedSwipeMethod = class_getClassMethod(RunnerSynthesizedGesture.self, selector),
let failureStubMethod = class_getClassMethod(RunnerSynthesizedSwipeFailureStub.self, selector)
else {
XCTFail("unable to install synthesized swipe failure stub")
return
}
let originalImplementation = method_getImplementation(synthesizedSwipeMethod)
method_setImplementation(
synthesizedSwipeMethod,
method_getImplementation(failureStubMethod)
)
app.launch()
runnerAccessibilityHealth = .healthy
defer {
method_setImplementation(synthesizedSwipeMethod, originalImplementation)
invalidateCachedTarget(reason: "unit_test_cleanup")
app.terminate()
}
let command = try runnerCommandFixture(
"""
{"command":"gesture","commandId":"gesture-fling-fallback","gesturePlan":{"topology":"single","intent":"fling","executionProfile":"endpoint-hold","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}}
"""
)
let response = try executeOnMainPrepared(command: command, activeApp: app)
XCTAssertTrue(response.ok)
XCTAssertEqual(response.data?.message, "fling")
XCTAssertEqual(response.data?.gestureFallback, "xctest-coordinate-drag")
XCTAssertEqual(response.data?.gestureFallbackMessage, "forced private synthesis failure")
XCTAssertEqual(
response.data?.gestureFallbackHint,
"Private XCTest event synthesis is required for AX-free coordinate drag on iOS; update Xcode if this persists."
)
XCTAssertNil(response.data?.x)
XCTAssertNil(response.data?.y)
XCTAssertNil(response.data?.x2)
XCTAssertNil(response.data?.y2)
}
func testSelectorTapFallsBackToXCTestCoordinateWhenPrivateSynthesisFails() throws {
let selector = NSSelectorFromString("synthesizeTapWithApplication:x:y:")
guard
let synthesizedTapMethod = class_getClassMethod(RunnerSynthesizedGesture.self, selector),
let failureStubMethod = class_getClassMethod(RunnerSynthesizedTapFailureStub.self, selector)
else {
XCTFail("unable to install synthesized tap failure stub")
return
}
let originalImplementation = method_getImplementation(synthesizedTapMethod)
method_setImplementation(
synthesizedTapMethod,
method_getImplementation(failureStubMethod)
)
app.launch()
currentApp = app
runnerAccessibilityHealth = .healthy
defer {
method_setImplementation(synthesizedTapMethod, originalImplementation)
invalidateCachedTarget(reason: "unit_test_cleanup")
app.terminate()
}
let command = try runnerCommandFixture(
#"{"command":"tap","commandId":"selector-tap-fallback","selectorKey":"label","selectorValue":"Agent Device Runner","synthesized":true}"#
)
let response = try executeOnMainPrepared(command: command, activeApp: app)
XCTAssertTrue(response.ok)
XCTAssertEqual(response.data?.message, "tapped")
XCTAssertEqual(response.data?.gestureFallback, "xctest-coordinate-tap")
XCTAssertEqual(response.data?.gestureFallbackMessage, "forced private synthesis failure")
XCTAssertEqual(
response.data?.gestureFallbackHint,
"Falling back to XCTest coordinate tap may be slower and can still need a healthy accessibility tree."
)
}
#endif
#if os(iOS)
func testTypeWithoutResolvedInputReturnsTypedFailureBeforeDispatchingText() throws {
let command = try runnerCommandFixture(
#"{"command":"type","commandId":"type-without-focus","text":"hello"}"#
)
let response = executeTypeCommand(
activeApp: XCUIApplication(bundleIdentifier: "com.example.agentdevice.missing-input"),
command: command
)
XCTAssertFalse(response.ok)
XCTAssertEqual(response.error?.code, "TEXT_INPUT_NOT_FOCUSED")
XCTAssertEqual(
response.error?.hint,
"Focus a visible text input, then retry type or fill. If the input is not exposed by accessibility, use a coordinate focus command before typing."
)
}
func testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden() throws {
// The fixture uses a real text responder with an empty input view to model hardware-keyboard input.
app.launchArguments = ["--agent-device-text-entry-regression"]
app.launch()
defer {
invalidateCachedTarget(reason: "unit_test_cleanup")
app.terminate()
}
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
let textField = app.textFields["agent-device-hardware-keyboard-input"]
XCTAssertTrue(textField.waitForExistence(timeout: appExistenceTimeout))
let frame = textField.frame
XCTAssertFalse(frame.isEmpty)
let tapCommand = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-hardware-keyboard-input","selectorKey":"id","selectorValue":"agent-device-hardware-keyboard-input"}"#
)
let tapResponse = try executeOnMainPrepared(command: tapCommand, activeApp: app)
XCTAssertTrue(tapResponse.ok, String(describing: tapResponse.error))
XCTAssertFalse(
isKeyboardVisible(app: app),
"the test must exercise a focused responder with the software keyboard hidden"
)
let failureCountBefore = currentXCTestFailureCount()
let typeCommand = try runnerCommandFixture(
#"{"command":"type","commandId":"type-hardware-keyboard","text":"hardware-keyboard"}"#
)
let typeResponse = executeTypeCommand(activeApp: app, command: typeCommand)
XCTAssertTrue(typeResponse.ok, String(describing: typeResponse.error))
XCTAssertFalse(didRecordXCTestFailure(since: failureCountBefore))
XCTAssertEqual(typeResponse.data?.textEntryRoute, "synthesized-first-responder")
XCTAssertEqual(String(describing: textField.value ?? ""), "hardware-keyboard")
let secondFailureCountBefore = currentXCTestFailureCount()
let secondTypeCommand = try runnerCommandFixture(
#"{"command":"type","commandId":"type-hardware-keyboard-again","text":"-again"}"#
)
let secondTypeResponse = executeTypeCommand(activeApp: app, command: secondTypeCommand)
XCTAssertFalse(secondTypeResponse.ok)
XCTAssertEqual(secondTypeResponse.error?.code, "TEXT_INPUT_NOT_FOCUSED")
XCTAssertFalse(didRecordXCTestFailure(since: secondFailureCountBefore))
XCTAssertEqual(String(describing: textField.value ?? ""), "hardware-keyboard")
}
func testBareDelayedTypeFailsWhenTappedInputDisappearsMidCommand() throws {
app.launchArguments = [
"--agent-device-text-entry-regression",
"--agent-device-text-entry-disappear-after-input",
]
app.launch()
defer {
invalidateCachedTarget(reason: "unit_test_cleanup")
app.terminate()
}
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
let textField = app.textFields["agent-device-hardware-keyboard-input"]
XCTAssertTrue(textField.waitForExistence(timeout: appExistenceTimeout))
let tapCommand = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-disappearing-input","selectorKey":"id","selectorValue":"agent-device-hardware-keyboard-input"}"#
)
let tapResponse = try executeOnMainPrepared(command: tapCommand, activeApp: app)
XCTAssertTrue(tapResponse.ok, String(describing: tapResponse.error))
let failureCountBefore = currentXCTestFailureCount()
let typeCommand = try runnerCommandFixture(
#"{"command":"type","commandId":"type-disappearing-input","text":"ab","delayMs":50}"#
)
let typeResponse = executeTypeCommand(activeApp: app, command: typeCommand)
XCTAssertFalse(didRecordXCTestFailure(since: failureCountBefore))
XCTAssertFalse(typeResponse.ok)
XCTAssertEqual(typeResponse.error?.code, "TEXT_INPUT_NOT_FOCUSED")
XCTAssertFalse(textField.exists)
}
#endif
func testInjectedTapRecordedFailureGateIsTapOnlyAndCountGated() {
// The seam's recording side cannot run in-bundle (a real XCTIssue would
// fail this very test run — same constraint the record(_:) suppression
// tests document); the live daemon proof covers it. This pins the gate.
XCTAssertFalse(RunnerTests.shouldInjectTapRecordedFailure(command: .tap, remaining: 0))
XCTAssertTrue(RunnerTests.shouldInjectTapRecordedFailure(command: .tap, remaining: 1))
XCTAssertFalse(RunnerTests.shouldInjectTapRecordedFailure(command: .type, remaining: 1))
XCTAssertFalse(RunnerTests.shouldInjectTapRecordedFailure(command: .snapshot, remaining: 1))
}
func testXCTestRecordedFailureResponseFailsMutatingSuccesses() throws {
let command = try runnerCommandFixture(#"{"command":"tap","commandId":"tap-1"}"#)
let response = Response(ok: true, data: DataPayload(message: "tapped"))
let failureResponse = xctestRecordedFailureResponse(command: command, response: response)
XCTAssertEqual(failureResponse?.ok, false)
XCTAssertEqual(failureResponse?.error?.code, "XCTEST_RECORDED_FAILURE")
XCTAssertEqual(
failureResponse?.error?.message,
"XCTest recorded a failure while executing tap; the action may not have been performed."
)
}
func testXCTestRecordedFailureResponseDoesNotWrapReadOnlyOrRunnerFatalResponses() throws {
let snapshotCommand = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-1"}"#)
let tapCommand = try runnerCommandFixture(#"{"command":"tap","commandId":"tap-1"}"#)
let runnerFatalResponse = Response(
ok: true,
data: DataPayload(runnerFatal: true, runnerFatalReason: "ax_snapshot_unavailable")
)
XCTAssertNil(
xctestRecordedFailureResponse(
command: snapshotCommand,
response: Response(ok: true, data: DataPayload(nodes: [], truncated: false))
)
)
XCTAssertNil(xctestRecordedFailureResponse(command: tapCommand, response: runnerFatalResponse))
}
// Simulator-only from here to the matching #endif: these launch the host app, route through
// SpringBoard, or assert the iOS-only alert/system-modal branches. Tests outside the
// `os(iOS)` regions in this file are pure runner decisions and also run on the macOS host
// lane (ci.yml) — see the classification convention in RunnerTests.swift.
#if os(iOS)
func testMissingBundleCommandInvalidatesCompleteCachedTargetState() throws {
app.launch()
currentApp = app
currentBundleId = "com.example.stale-target"
currentAppProcessIdentifier = 42
snapshotXCTestPenaltyWarmupExemptionPending = true
defer {
invalidateCachedTarget(reason: "unit_test_cleanup")
app.terminate()
}
let command = try runnerCommandFixture(
#"{"command":"snapshot","commandId":"snapshot-without-bundle"}"#
)
_ = prepareActiveCommandContext(command: command)
XCTAssertNil(currentApp)
XCTAssertNil(currentBundleId)
XCTAssertNil(currentAppProcessIdentifier)
XCTAssertFalse(snapshotXCTestPenaltyWarmupExemptionPending)
}
func testSkipAppActivationPreflightIncludesForegroundCachedCoordinateOnlyTaps() throws {
app.launch()
currentApp = app
currentBundleId = nil
defer {
currentApp = nil
currentBundleId = nil
app.terminate()
}
let tap = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-1","x":10,"y":20}"#
)
XCTAssertTrue(shouldSkipAppActivationPreflight(tap))
}
func testSkipAppActivationPreflightRejectsMissingChangedAndBackgroundTargets() throws {
let coordinateTap = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-1","x":10,"y":20}"#
)
currentApp = nil
currentBundleId = nil
XCTAssertFalse(shouldSkipAppActivationPreflight(coordinateTap))
app.launch()
currentApp = app
currentBundleId = "com.example.current"
defer {
currentApp = nil
currentBundleId = nil
app.terminate()
}
let changedBundleTap = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-2","appBundleId":"com.example.other","x":10,"y":20}"#
)
XCTAssertFalse(shouldSkipAppActivationPreflight(changedBundleTap))
app.terminate()
currentApp = app
currentBundleId = nil
XCTAssertFalse(shouldSkipAppActivationPreflight(coordinateTap))
}
func testPrepareActiveCommandContextRoutesBlockingSystemModalToSpringboard() throws {
blockingSystemModalPresenceOverrideForTesting = true
currentApp = nil
currentBundleId = nil
defer {
blockingSystemModalPresenceOverrideForTesting = nil
currentApp = nil
currentBundleId = nil
}
let tap = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-1","x":10,"y":20}"#
)
let preparation = prepareActiveCommandContext(
command: tap,
routeToSpringboard: shouldRouteToSpringboardBlockingSystemModal(tap)
)
guard case .context(let context) = preparation else {
XCTFail("expected command context")
return
}
XCTAssertTrue(context.app === springboard)
}
func testExecuteDispatchedReturnsBusyBeforeBlockingSystemModalProbeDrains() throws {
app.launch()
currentApp = app
currentBundleId = nil
defer {
currentApp = nil
currentBundleId = nil
systemModalProbeOverrideForTesting = nil
clearSnapshotXCTestChannelPenalty(reason: "test-cleanup")
app.terminate()
}
final class ResultBox {
var response: Response?
var error: Error?
var commandRecoveredBeforeRelease = false
var wasBusyBeforeRelease = false
var hadAbandonedProbeBeforeRelease = false
var drained = false
}
let box = ResultBox()
let probeStarted = expectation(description: "system-modal routing probe started")
let verificationFinished = expectation(description: "command recovery and modal probe drain verified")
let probeReleaseGate = DispatchSemaphore(value: 0)
let commandFinishedGate = DispatchSemaphore(value: 0)
systemModalProbeOverrideForTesting = { _ in
probeStarted.fulfill()
_ = probeReleaseGate.wait(timeout: .now() + 15)
return DataPayload(message: "late system modal")
}
let command = try runnerCommandFixture(
#"{"command":"tap","commandId":"bounded-modal-routing","x":10,"y":20}"#
)
DispatchQueue(label: "agent-device.runner.tests.modal-routing-probe").async {
do {
box.response = try self.executeDispatched(command: command)
} catch {
box.error = error
}
commandFinishedGate.signal()
}
DispatchQueue(label: "agent-device.runner.tests.modal-routing-probe-verifier").async {
let commandWait = commandFinishedGate.wait(
timeout: .now() + self.systemModalProbeBudget + 3
)
box.commandRecoveredBeforeRelease = commandWait == .success
&& box.error == nil
&& box.response?.error?.code == "RUNNER_BUSY"
if case .busy = self.currentMainThreadBusyState() {
box.wasBusyBeforeRelease = true
}
box.hadAbandonedProbeBeforeRelease = self.hasAbandonedTreeCapture()
// The XCTest main thread is blocked inside the injected probe, so this verifier owns the
// ordered release after recording the command result and abandoned-work state above.
probeReleaseGate.signal()
let deadline = Date().addingTimeInterval(5)
while self.hasAbandonedTreeCapture(), Date() < deadline {
self.sleepFor(0.002)
}
box.drained = !self.hasAbandonedTreeCapture()
verificationFinished.fulfill()
}
wait(for: [probeStarted, verificationFinished], timeout: 15)
XCTAssertTrue(
box.commandRecoveredBeforeRelease,
"the public coordinate tap must return RUNNER_BUSY before the blocked modal probe drains"
)
XCTAssertTrue(box.wasBusyBeforeRelease)
XCTAssertTrue(box.hadAbandonedProbeBeforeRelease)
XCTAssertTrue(box.drained)
guard case .idle = currentMainThreadBusyState() else {
return XCTFail("expected the runner to become idle after the routing probe drained")
}
XCTAssertFalse(hasAbandonedTreeCapture())
}
func testSkipAppActivationPreflightRejectsSelectorAndMixedSequenceGestures() throws {
app.launch()
currentApp = app
currentBundleId = nil
defer {
currentApp = nil
currentBundleId = nil
app.terminate()
}
let selectorTap = try runnerCommandFixture(
#"{"command":"tap","commandId":"tap-1","selectorKey":"label","selectorValue":"Search","synthesized":true}"#
)
let standardDrag = try runnerCommandFixture(
#"{"command":"drag","commandId":"drag-1","x":10,"y":20,"x2":30,"y2":40}"#
)
let mixedSequence = try runnerCommandFixture(
"""
{"command":"sequence","commandId":"seq-1","steps":[
{"kind":"tap","x":10,"y":20,"synthesized":true},
{"kind":"doubleTap","x":30,"y":40}
]}
"""
)
XCTAssertFalse(shouldSkipAppActivationPreflight(selectorTap))
XCTAssertFalse(shouldSkipAppActivationPreflight(standardDrag))
XCTAssertFalse(shouldSkipAppActivationPreflight(mixedSequence))
}
// Launches nothing, but still simulator-only: `shouldSkipAppActivationPreflight` is
// `#if os(iOS) …guards… #else return false #endif`, so on macOS this asserts a compile-time
// literal and no edit to the iOS body could make it red. Its five siblings above and below
// are gated for the same reason.
func testSkipAppActivationPreflightRequiresCachedForegroundTarget() throws {
currentApp = nil
currentBundleId = nil
let scroll = try runnerCommandFixture(
#"{"command":"scroll","commandId":"scroll-1","direction":"down","pixels":400}"#
)
XCTAssertFalse(shouldSkipAppActivationPreflight(scroll))
}
func testSkipAppActivationPreflightKeepsDragScrollAndSequenceOnForegroundGuard() throws {
app.launch()
currentApp = app
currentBundleId = nil
defer {
currentApp = nil
currentBundleId = nil
app.terminate()
}
let drag = try runnerCommandFixture(
#"{"command":"drag","commandId":"drag-1","x":10,"y":20,"x2":30,"y2":40,"synthesized":true}"#
)
let scroll = try runnerCommandFixture(
#"{"command":"scroll","commandId":"scroll-1","direction":"down","pixels":400}"#
)
let sequence = try runnerCommandFixture(
"""
{"command":"sequence","commandId":"seq-1","steps":[
{"kind":"tap","x":10,"y":20,"synthesized":true},
{"kind":"longPress","x":10,"y":200,"durationMs":300}
]}
"""
)
XCTAssertFalse(shouldSkipAppActivationPreflight(drag))
XCTAssertFalse(shouldSkipAppActivationPreflight(scroll))
XCTAssertFalse(shouldSkipAppActivationPreflight(sequence))
}
func testSkipAppActivationPreflightIncludesAlertCommands() throws {
let alert = try runnerCommandFixture(
#"{"command":"alert","commandId":"alert-1","action":"get"}"#
)
XCTAssertTrue(shouldSkipAppActivationPreflight(alert))
}
#endif
func testExecuteDispatchedReturnsBusyBeforeMainThreadFastPath() throws {
let command = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-busy"}"#)
abandonedMainThreadWorkCount = 1
abandonedMainThreadWorkSince = Date(timeIntervalSinceNow: -2)
defer {
abandonedMainThreadWorkCount = 0
abandonedMainThreadWorkSince = nil
}
let response = try executeDispatched(command: command)
XCTAssertFalse(response.ok)
XCTAssertEqual(response.error?.code, "RUNNER_BUSY")
XCTAssertTrue(response.error?.message.contains("previous command") == true)
}
func testExecuteDispatchedReturnsWedgedBeforeMainThreadFastPath() throws {
let command = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-wedged"}"#)
abandonedMainThreadWorkCount = 1
abandonedMainThreadWorkSince = Date(timeIntervalSinceNow: -(mainThreadWedgeThreshold + 1))
defer {
abandonedMainThreadWorkCount = 0
abandonedMainThreadWorkSince = nil
}
let response = try executeDispatched(command: command)
XCTAssertFalse(response.ok)
XCTAssertEqual(response.error?.code, "RUNNER_WEDGED")
XCTAssertTrue(response.error?.hint?.contains("runner session will be restarted") == true)
}
#if os(iOS)
func testAlertResolutionCannotBypassRequestedDeadline() throws {
final class ResultBox {
var error: Error?
var probeDeadline: Date?
var observedDeadline: Date?
var commandStartedAt: Date?
}
let box = ResultBox()
let releaseResolution = DispatchSemaphore(value: 0)
let resolutionExited = expectation(description: "bounded alert resolution exited")
let commandFinished = expectation(description: "alert command respected its deadline")
let command = try runnerCommandFixture(
#"{"command":"alert","commandId":"alert-deadline","appBundleId":"com.apple.springboard","action":"get","timeoutMs":500}"#
)
currentApp = springboard
currentBundleId = Self.springboardBundleId
systemModalProbeOverrideForTesting = { deadline in
box.probeDeadline = deadline
return nil
}
alertResolutionOverrideForTesting = { deadline in
box.observedDeadline = deadline
_ = releaseResolution.wait(timeout: .now() + 1)
resolutionExited.fulfill()
return nil
}
defer {
releaseResolution.signal()
systemModalProbeOverrideForTesting = nil
alertResolutionOverrideForTesting = nil
currentApp = nil
currentBundleId = nil
}
DispatchQueue(label: "agent-device.runner.tests.alert-deadline").async {
box.commandStartedAt = Date()
do {
_ = try self.executeDispatched(command: command)
} catch {
box.error = error
}
commandFinished.fulfill()
}
wait(for: [commandFinished], timeout: 1)
let error = box.error as NSError?
XCTAssertEqual(error?.domain, RunnerErrorDomain.general)
XCTAssertEqual(error?.code, RunnerErrorCode.mainThreadExecutionTimedOut)
XCTAssertNotNil(box.probeDeadline)
XCTAssertNotNil(box.observedDeadline)
if let probeDeadline = box.probeDeadline,
let observedDeadline = box.observedDeadline,
let commandStartedAt = box.commandStartedAt
{
XCTAssertEqual(probeDeadline.timeIntervalSince(observedDeadline), 0, accuracy: 0.01)
XCTAssertEqual(observedDeadline.timeIntervalSince(commandStartedAt), 0.5, accuracy: 0.05)
}
releaseResolution.signal()
wait(for: [resolutionExited], timeout: 1)
}
#endif
func testRunMainThreadWorkExecutesOffMainCallerOnMainThread() {
final class ResultBox {
var observedMainThread: Bool?
var error: Error?
}
let box = ResultBox()
let finished = expectation(description: "off-main caller finished")
DispatchQueue(label: "agent-device.runner.tests.off-main").async {
do {
box.observedMainThread = try self.runMainThreadWork(
timeout: 1,
timeoutError: self.mainThreadExecutionTimeoutError
) {
Thread.isMainThread
}
} catch {
box.error = error
}
finished.fulfill()
}
wait(for: [finished], timeout: 2)
XCTAssertNil(box.error)
XCTAssertEqual(box.observedMainThread, true)
}
func testRunMainThreadWorkTimeoutMarksAbandonedUntilDrained() {
final class ResultBox {
var error: Error?
var abandonedCount: Int?
var abandonedSinceSet: Bool?
var drainedCount: Int?
var drainedSinceCleared: Bool?
}
let box = ResultBox()
let releaseWork = DispatchSemaphore(value: 0)
let observedAbandoned = DispatchSemaphore(value: 0)
let finished = expectation(description: "off-main caller timed out")
let drained = expectation(description: "abandoned main work drained")
DispatchQueue(label: "agent-device.runner.tests.timeout").async {
do {
_ = try self.runMainThreadWork(
timeout: 0,
timeoutError: self.mainThreadExecutionTimeoutError,
onAbandoned: {
box.abandonedCount = self.abandonedMainThreadWorkCount
box.abandonedSinceSet = self.abandonedMainThreadWorkSince != nil
observedAbandoned.signal()
},
onDrained: {
self.mainThreadWorkLock.lock()
box.drainedCount = self.abandonedMainThreadWorkCount
box.drainedSinceCleared = self.abandonedMainThreadWorkSince == nil
self.mainThreadWorkLock.unlock()
drained.fulfill()
}
) {
_ = releaseWork.wait(timeout: .now() + 1)
return true
}
} catch {
box.error = error
}
finished.fulfill()
}
DispatchQueue(label: "agent-device.runner.tests.release-timeout").async {
_ = observedAbandoned.wait(timeout: .now() + 1)
releaseWork.signal()
}
wait(for: [finished, drained], timeout: 2)
XCTAssertEqual((box.error as NSError?)?.code, RunnerErrorCode.mainThreadExecutionTimedOut)
XCTAssertEqual(box.abandonedCount, 1)
XCTAssertEqual(box.abandonedSinceSet, true)
XCTAssertEqual(box.drainedCount, 0)
XCTAssertEqual(box.drainedSinceCleared, true)
}
func testPostSnapshotDelayMarkDoesNotQueueBehindAbandonedTreeCapture() {
abandonedTreeCaptureCount = 1
defer {
abandonedTreeCaptureCount = 0
needsPostSnapshotInteractionDelay = false
}
let finished = expectation(description: "off-main caller finished")
DispatchQueue(label: "agent-device.runner.tests.post-snapshot-delay").async {
self.setNeedsPostSnapshotInteractionDelay()
finished.fulfill()
}
wait(for: [finished], timeout: 1)
mainThreadWorkLock.lock()
let abandonedWorkCount = abandonedMainThreadWorkCount
mainThreadWorkLock.unlock()
XCTAssertEqual(abandonedWorkCount, 0)
}
#endif
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
func execute(command: Command) throws -> Response {
if command.command == .status {
return executeStatus(command: command)
}
if command.command == .uptime {
return executeUptime()
}
commandJournal.accept(command: command)
return try executeAccepted(command: command)
}
#endif
func executeAccepted(command: Command) throws -> Response {
commandJournal.start(command: command)
do {
let response = try executeDispatched(command: command)
commandJournal.finish(command: command, response: response)
return response
} catch {
commandJournal.fail(command: command, error: error)
throw error
}
}
func executeStatus(command: Command) -> Response {
guard
let statusCommandId = command.statusCommandId?.trimmedNonEmpty
else {
return Response(
ok: false,
error: ErrorPayload(
code: "INVALID_ARGS",
message: "status requires statusCommandId",
hint: "Set statusCommandId to the commandId of the runner command to inspect."
)
)
}
return Response(ok: true, data: commandJournal.status(normalizedCommandId: statusCommandId))
}
func executeUptime() -> Response {
// Placeholder value: the transport layer (jsonResponse) overwrites currentUptimeMs with a
// fresher send-time stamp on every ok response; kept so direct callers still get a value.
Response(
ok: true,
data: DataPayload(currentUptimeMs: currentUptimeMs())
)
}
/// Tracks one main-queue dispatch so the watchdog and the dispatched block can agree —
/// under `mainThreadWorkLock` — on exactly one of: finished in time, or abandoned.
private final class MainThreadWorkState {
var finished = false
var abandoned = false
}
struct ActiveCommandContext {
let app: XCUIApplication
}
enum ActiveCommandPreparation {
case response(Response)
case context(ActiveCommandContext)
}
enum MainThreadBusyState {
case idle
case busy(abandonedForSeconds: TimeInterval)
case wedged(abandonedForSeconds: TimeInterval)
}
func currentMainThreadBusyState() -> MainThreadBusyState {
mainThreadWorkLock.lock()
defer { mainThreadWorkLock.unlock() }
guard abandonedMainThreadWorkCount > 0 else { return .idle }
let abandonedFor = abandonedMainThreadWorkSince.map { Date().timeIntervalSince($0) } ?? 0
if abandonedFor > mainThreadWedgeThreshold {
return .wedged(abandonedForSeconds: abandonedFor)
}
return .busy(abandonedForSeconds: abandonedFor)
}
private func runnerBusyResponse(command: Command, abandonedForSeconds: TimeInterval) -> Response {
NSLog(
"AGENT_DEVICE_RUNNER_BUSY command=%@ commandId=%@ abandonedForSeconds=%.1f",
command.command.rawValue,
command.commandId ?? "",
abandonedForSeconds
)
return Response(
ok: false,
error: ErrorPayload(
code: "RUNNER_BUSY",
message:
"The iOS runner is still finishing a previous command that exceeded its execution watchdog (usually an accessibility capture on a heavy or animating screen).",
hint:
"Wait a few seconds and retry. If snapshots keep failing on this screen, use screenshot as visual truth and interact by coordinates, or navigate to another screen."
)
)
}
private func runnerWedgedResponse(command: Command, abandonedForSeconds: TimeInterval) -> Response {
NSLog(
"AGENT_DEVICE_RUNNER_WEDGED command=%@ commandId=%@ abandonedForSeconds=%.1f",
command.command.rawValue,
command.commandId ?? "",
abandonedForSeconds
)
return Response(
ok: false,
error: ErrorPayload(
code: "RUNNER_WEDGED",
message:
"The iOS runner main thread has been stuck in abandoned work for \(Int(abandonedForSeconds)) seconds and cannot recover on its own.",
hint:
"The runner session will be restarted. Retry the command after the restart; if this screen keeps wedging captures, use screenshot as visual truth and interact by coordinates."
)
)
}
private func runnerUnavailableResponse(command: Command) -> Response? {
switch currentMainThreadBusyState() {
case .idle:
return nil
case .busy(let abandonedForSeconds):
return runnerBusyResponse(command: command, abandonedForSeconds: abandonedForSeconds)
case .wedged(let abandonedForSeconds):
return runnerWedgedResponse(command: command, abandonedForSeconds: abandonedForSeconds)
}
}
private func executeDispatched(command: Command) throws -> Response {
// XCTest work cannot be cancelled mid-flight: once the watchdog abandons a main-queue
// block, queueing more main-thread commands behind it only buries the runner deeper.
// Refuse fast instead so the daemon backs off while the abandoned work drains; past the
// wedge threshold, escalate so the daemon recycles this runner (#1105).
if let unavailable = runnerUnavailableResponse(command: command) {
return unavailable
}
let alertDeadline = command.command == .alert
? Date().addingTimeInterval(Self.alertCommandTimeout(timeoutMs: command.timeoutMs))
: nil
if Thread.isMainThread {
let routeToSpringboard = shouldRouteToSpringboardBlockingSystemModal(
command,
deadline: alertDeadline
)
return try executeOnMainSafely(
command: command,
alertDeadline: alertDeadline,
routeToSpringboard: routeToSpringboard
)
}
// Resolve this before the command's outer main-thread block. If the bounded probe abandons
// slow XCTest enumeration, return the established recoverable response instead of queueing
// command preparation behind work that may outlive the 30-second command watchdog.
let routeToSpringboard = shouldRouteToSpringboardBlockingSystemModal(
command,
deadline: alertDeadline
)
if let unavailable = runnerUnavailableResponse(command: command) {
return unavailable
}
if command.command == .snapshot {
return try executeSnapshotDispatched(command: command)
}
if command.command == .alert, let deadline = alertDeadline {
return try runMainThreadWork(
timeout: max(0.001, deadline.timeIntervalSinceNow),
timeoutError: mainThreadExecutionTimeoutError
) {
try self.executeOnMainSafely(
command: command,
alertDeadline: deadline,
routeToSpringboard: routeToSpringboard
)
}
}
return try runMainThreadWork(
timeout: mainThreadExecutionTimeout,
timeoutError: mainThreadExecutionTimeoutError
) {
try self.executeOnMainSafely(command: command, routeToSpringboard: routeToSpringboard)
}
}
func runMainThreadWork<T>(
timeout: TimeInterval,
timeoutError: @escaping () -> Error,
onAbandoned: (() -> Void)? = nil,
onDrained: (() -> Void)? = nil,
_ work: @escaping () throws -> T
) throws -> T {
if Thread.isMainThread {
return try work()
}
var result: Result<T, Error>?
let semaphore = DispatchSemaphore(value: 0)
let workState = MainThreadWorkState()
DispatchQueue.main.async {
do {
result = .success(try work())
} catch {
result = .failure(error)
}
self.mainThreadWorkLock.lock()
if workState.abandoned {
self.abandonedMainThreadWorkCount -= 1
if self.abandonedMainThreadWorkCount == 0 {
self.abandonedMainThreadWorkSince = nil
NSLog("AGENT_DEVICE_RUNNER_ABANDONED_WORK_DRAINED")
}
self.mainThreadWorkLock.unlock()
onDrained?()
} else {
workState.finished = true
self.mainThreadWorkLock.unlock()
}
semaphore.signal()
}
let waitResult = semaphore.wait(timeout: .now() + timeout)
if waitResult == .timedOut {
mainThreadWorkLock.lock()
let stillRunning = !workState.finished
if stillRunning {
workState.abandoned = true
abandonedMainThreadWorkCount += 1
if abandonedMainThreadWorkSince == nil {
abandonedMainThreadWorkSince = Date()
}
onAbandoned?()
}
mainThreadWorkLock.unlock()
throw timeoutError()
}
switch result {
case .success(let value):
return value
case .failure(let error):
throw error
case .none:
throw NSError(
domain: RunnerErrorDomain.general,
code: RunnerErrorCode.noResponseFromMainThread,
userInfo: [NSLocalizedDescriptionKey: "no response from main thread"]
)
}
}
private func mainThreadExecutionTimeoutError() -> Error {
NSError(
domain: RunnerErrorDomain.general,
code: RunnerErrorCode.mainThreadExecutionTimedOut,
userInfo: [NSLocalizedDescriptionKey: "main thread execution timed out"]
)
}
// MARK: - Command Handling
private func executeOnMainSafely(
command: Command,
alertDeadline: Date? = nil,
routeToSpringboard: Bool
) throws -> Response {
var hasRetried = false
while true {
var response: Response?
var swiftError: Error?
let failureCountBefore = currentXCTestFailureCount()
let exceptionMessage = RunnerObjCExceptionCatcher.catchException({
do {
response = try self.executeOnMain(
command: command,
alertDeadline: alertDeadline,
routeToSpringboard: routeToSpringboard
)
} catch {
swiftError = error
}
})
if let exceptionMessage {
invalidateCachedTarget(reason: "objc_exception")
if !hasRetried, shouldRetryException(command, message: exceptionMessage) {
NSLog(
"AGENT_DEVICE_RUNNER_RETRY command=%@ reason=objc_exception",
command.command.rawValue
)
hasRetried = true
sleepFor(retryCooldown)
continue
}
throw NSError(
domain: RunnerErrorDomain.exception,
code: RunnerErrorCode.objcException,
userInfo: [NSLocalizedDescriptionKey: exceptionMessage]
)
}
if let swiftError {
throw swiftError
}
guard let response else {
throw NSError(
domain: RunnerErrorDomain.general,
code: RunnerErrorCode.commandReturnedNoResponse,
userInfo: [NSLocalizedDescriptionKey: "command returned no response"]
)
}
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
// #1605 merge gate: the REAL gesture already executed above; recording a
// production-shaped issue here makes the per-command failure-count
// conversion below fire exactly as in the field (bsky-24: activation
// lands, bookkeeping records a failure). Compiled out of production.
if consumeInjectedTapRecordedFailureForTesting(command: command.command) {
record(
XCTIssue(
type: .assertionFailure,
compactDescription: "Injected tap recorded-failure (#1605 corroboration merge gate)"
)
)
}
#endif
if didRecordXCTestFailure(since: failureCountBefore),
let failureResponse = xctestRecordedFailureResponse(command: command, response: response)
{
invalidateCachedTarget(reason: "xctest_recorded_failure")
return failureResponse
}
if !hasRetried, shouldRetryCommand(command), shouldRetryResponse(response) {
NSLog(
"AGENT_DEVICE_RUNNER_RETRY command=%@ reason=response_unavailable",
command.command.rawValue
)
hasRetried = true
invalidateCachedTarget(reason: "response_unavailable")
sleepFor(retryCooldown)
continue
}
return response
}
}
private func executeSnapshotDispatched(command: Command) throws -> Response {
try executeDispatchedWithRecovery(command: command) {
try self.executeSnapshotDispatchedOnce(command: command)
}
}
/// The dispatched snapshot recovery loop: read-only retry + XCTest-recorded-failure invalidation,
/// matching what `executeOnMainSafely` gives the generic path. `perform` runs the capture and its
/// own bounded main-thread work.
func executeDispatchedWithRecovery(
command: Command,
perform: () throws -> Response
) throws -> Response {
var hasRetried = false
while true {
let failureCountBefore = try runMainThreadWork(
timeout: mainThreadExecutionTimeout,
timeoutError: mainThreadExecutionTimeoutError
) {
self.currentXCTestFailureCount()
}
let response = try perform()
// Recovered independently — re-entering main for bookkeeping would queue behind the still-
// abandoned XCTest query and re-stall the command (#1244), so skip it until that work drains.
if hasAbandonedTreeCapture() {
NSLog(
"AGENT_DEVICE_RUNNER_DISPATCH_RECOVERY_SKIPPED_XCTEST_OCCUPIED command=%@",
command.command.rawValue
)
return response
}
let recordedFailureResponse = try runMainThreadWork(
timeout: mainThreadExecutionTimeout,
timeoutError: mainThreadExecutionTimeoutError
) {
self.didRecordXCTestFailure(since: failureCountBefore)
? self.xctestRecordedFailureResponse(command: command, response: response)
: nil
}
if let recordedFailureResponse {
try runMainThreadWork(
timeout: mainThreadExecutionTimeout,
timeoutError: mainThreadExecutionTimeoutError
) {
self.invalidateCachedTarget(reason: "xctest_recorded_failure")
}
return recordedFailureResponse
}
if !hasRetried, shouldRetryCommand(command), shouldRetryResponse(response) {
NSLog(
"AGENT_DEVICE_RUNNER_RETRY command=%@ reason=response_unavailable",
command.command.rawValue
)
hasRetried = true
try runMainThreadWork(
timeout: mainThreadExecutionTimeout,
timeoutError: mainThreadExecutionTimeoutError
) {
self.invalidateCachedTarget(reason: "response_unavailable")
self.sleepFor(self.retryCooldown)
}
continue
}
return response
}
}
private func executeSnapshotDispatchedOnce(command: Command) throws -> Response {
let preparation = try runMainThreadWork(
timeout: mainThreadExecutionTimeout,
timeoutError: mainThreadExecutionTimeoutError
) {
try self.prepareActiveCommandContextSafely(command: command, routeToSpringboard: false)
}
switch preparation {
case .response(let response):
return response
case .context(let context):
return try executeSnapshotPrepared(command: command, activeApp: context.app)
}
}
/// Pure command→options projection, extracted so the runner unit bundle can
/// prove the decoded wire field actually reaches presentation options (#1634 P2).
static func presentationOptions(from command: Command) -> PresentationOptions {
let customActions = command.customActions ?? false
return PresentationOptions(
interactiveOnly: command.interactiveOnly ?? false,
depth: command.depth,
scope: command.scope,
raw: command.raw ?? false,
// Custom actions are only readable through the private AX client, so
// asking for them pins that backend rather than silently returning a
// capture that structurally cannot carry them. An explicit pin wins.
preferredBackend: command.preferredBackend
?? (customActions ? SnapshotBackendKind.privateAX.rawValue : nil),
customActions: customActions
)
}
private func executeSnapshotPrepared(command: Command, activeApp: XCUIApplication) throws -> Response {
let options = Self.presentationOptions(from: command)
do {
let payload: DataPayload
if options.raw {
payload = try snapshotRaw(app: activeApp, options: options)
} else {
payload = try snapshotFast(app: activeApp, options: options)
}
setNeedsPostSnapshotInteractionDelay()
return Response(ok: true, data: payload)
} catch let failure as SnapshotCaptureFailure {
invalidateCachedTargetAfterSnapshotFailure()
return Response(
ok: false,
error: ErrorPayload(
code: failure.code,
message: failure.message,
hint: failure.hint
)
)
}
}
private func setNeedsPostSnapshotInteractionDelay() {
if Thread.isMainThread {
needsPostSnapshotInteractionDelay = true
return
}
guard !hasAbandonedTreeCapture() else {
NSLog("AGENT_DEVICE_RUNNER_POST_SNAPSHOT_DELAY_MARK_SKIPPED_XCTEST_OCCUPIED")
return
}
do {
try runMainThreadWork(
timeout: 1,
timeoutError: mainThreadExecutionTimeoutError
) {
self.needsPostSnapshotInteractionDelay = true
}
} catch {
NSLog("AGENT_DEVICE_RUNNER_POST_SNAPSHOT_DELAY_MARK_FAILED=%@", String(describing: error))
}
}
private func invalidateCachedTargetAfterSnapshotFailure() {
if Thread.isMainThread {
invalidateCachedTarget(reason: "ax_snapshot_failure")
return
}
do {
try runMainThreadWork(
timeout: 1,
timeoutError: mainThreadExecutionTimeoutError
) {
self.invalidateCachedTarget(reason: "ax_snapshot_failure")
}
} catch {
NSLog("AGENT_DEVICE_RUNNER_SNAPSHOT_INVALIDATION_FAILED=%@", String(describing: error))
}
}
private func prepareActiveCommandContextSafely(
command: Command,
routeToSpringboard: Bool
) throws -> ActiveCommandPreparation {
var preparation: ActiveCommandPreparation?
let exceptionMessage = RunnerObjCExceptionCatcher.catchException({
preparation = self.prepareActiveCommandContext(
command: command,
routeToSpringboard: routeToSpringboard
)
})
if let exceptionMessage {
throw NSError(
domain: RunnerErrorDomain.exception,
code: RunnerErrorCode.objcException,
userInfo: [NSLocalizedDescriptionKey: exceptionMessage]
)
}
guard let preparation else {
throw NSError(
domain: RunnerErrorDomain.general,
code: RunnerErrorCode.commandReturnedNoResponse,
userInfo: [NSLocalizedDescriptionKey: "snapshot preflight returned no response"]
)
}
return preparation
}
private func executeOnMain(
command: Command,
alertDeadline: Date?,
routeToSpringboard: Bool
) throws -> Response {
let preparation = prepareActiveCommandContext(
command: command,
routeToSpringboard: routeToSpringboard
)
let activeApp: XCUIApplication
switch preparation {
case .response(let response):
return response
case .context(let context):
activeApp = context.app
}
switch command.command {
case .status:
return executeStatus(command: command)
case .targetReset:
return resetTargetAfterExternalRelaunch()
case .shutdown:
stopRecordingIfNeeded()
return Response(ok: true, data: DataPayload(message: "shutdown"))
case .recordStart:
guard
let requestedOutPath = command.outPath?.trimmingCharacters(in: .whitespacesAndNewlines),
!requestedOutPath.isEmpty
else {
return Response(ok: false, error: ErrorPayload(message: "recordStart requires outPath"))
}
let hasAppBundleId = !(command.appBundleId?
.trimmingCharacters(in: .whitespacesAndNewlines)
.isEmpty ?? true)
guard hasAppBundleId else {
return Response(ok: false, error: ErrorPayload(message: "recordStart requires appBundleId"))
}
if activeRecording != nil {
return Response(ok: false, error: ErrorPayload(message: "recording already in progress"))
}
if let requestedFps = command.fps, (requestedFps < minRecordingFps || requestedFps > maxRecordingFps) {
return Response(ok: false, error: ErrorPayload(message: "recordStart fps must be between \(minRecordingFps) and \(maxRecordingFps)"))
}
do {
let resolvedOutPath = resolveRecordingOutPath(requestedOutPath)
let fpsLabel = command.fps.map(String.init) ?? String(RunnerTests.defaultRecordingFps)
NSLog(
"AGENT_DEVICE_RUNNER_RECORD_START requestedOutPath=%@ resolvedOutPath=%@ fps=%@",
requestedOutPath,
resolvedOutPath,
fpsLabel
)
let recorder = ScreenRecorder(
outputPath: resolvedOutPath,
fps: command.fps.map { Int32($0) }
)
try recorder.start { [weak self] in
return self?.captureRunnerFrame()
}
activeRecording = recorder
return Response(ok: true, data: DataPayload(message: "recording started"))
} catch {
activeRecording = nil
return Response(ok: false, error: ErrorPayload(message: "failed to start recording: \(error.localizedDescription)"))
}
case .recordStop:
guard let recorder = activeRecording else {
// The runner protocol is the durable cleanup primitive. A daemon may crash after the
// native stop succeeds but before it commits the resource transition, so exact-owner
// recovery must be able to repeat this command safely. Public `record stop` still owns
// its user-facing no-active validation through the daemon session manifest.
return Response(ok: true, data: DataPayload(message: "recording already stopped"))
}
do {
try recorder.stop()
activeRecording = nil
return Response(ok: true, data: DataPayload(message: "recording stopped"))
} catch {
activeRecording = nil
return Response(ok: false, error: ErrorPayload(message: "failed to stop recording: \(error.localizedDescription)"))
}
case .uptime:
return executeUptime()
case .activate:
guard
let bundleId = command.appBundleId?.trimmingCharacters(in: .whitespacesAndNewlines),
!bundleId.isEmpty
else {
return Response(ok: false, error: ErrorPayload(message: "activate requires appBundleId"))
}
// prepareActiveCommandContext already activated this bundle. Keep this case as the
// explicit acknowledgement after that preflight, not as a second activation.
return Response(ok: true, data: DataPayload(message: "app activated"))
case .terminate:
guard
let bundleId = command.appBundleId?.trimmingCharacters(in: .whitespacesAndNewlines),
!bundleId.isEmpty
else {
return Response(ok: false, error: ErrorPayload(message: "terminate requires appBundleId"))
}
XCUIApplication(bundleIdentifier: bundleId).terminate()
if currentBundleId == bundleId {
invalidateCachedTarget(reason: "target_terminated")
}
return Response(ok: true, data: DataPayload(message: "app terminated"))
default:
break
}
return try executeOnMainPrepared(
command: command,
activeApp: activeApp,
alertDeadline: alertDeadline
)
}
private func prepareActiveCommandContext(
command: Command,
routeToSpringboard: Bool = false
) -> ActiveCommandPreparation {
var activeApp = currentApp ?? app
if routeToSpringboard {
activeApp = springboard
} else if shouldSkipAppActivationPreflight(command) {
activeApp = resolveAppWithoutActivation(command: command)
} else if !isRunnerLifecycleCommand(command.command) {
let normalizedBundleId = command.appBundleId?
.trimmingCharacters(in: .whitespacesAndNewlines)
let requestedBundleId = (normalizedBundleId?.isEmpty == true) ? nil : normalizedBundleId
if let bundleId = requestedBundleId {
if currentBundleId != bundleId || currentApp == nil {
_ = activateTarget(bundleId: bundleId, reason: "bundle_changed")
} else {
refreshCachedTargetIfProcessChanged(bundleId: bundleId)
}
} else {
// Do not reuse stale bundle targets when the caller does not explicitly request one.
invalidateCachedTarget(reason: "missing_app_bundle")
}
activeApp = currentApp ?? app
if let bundleId = requestedBundleId, targetNeedsActivation(activeApp) {
activeApp = activateTarget(bundleId: bundleId, reason: "stale_target")
} else if requestedBundleId == nil, targetNeedsActivation(activeApp) {
ensureRunnerHostAppActive(reason: "missing_app_bundle")
activeApp = app
}
let skipExistenceWait = canUseFastForegroundAppGuard(
activeApp: activeApp,
requestedBundleId: requestedBundleId,
command: command.command
)
if !skipExistenceWait && !activeApp.waitForExistence(timeout: appExistenceTimeout) {
if let bundleId = requestedBundleId {
activeApp = activateTarget(bundleId: bundleId, reason: "missing_after_wait")
guard activeApp.waitForExistence(timeout: appExistenceTimeout) else {
return .response(Response(ok: false, error: ErrorPayload(message: "app '\(bundleId)' is not available")))
}
} else {
return .response(Response(ok: false, error: ErrorPayload(message: "runner app is not available")))
}
}
if isInteractionCommand(command.command) {
if let bundleId = requestedBundleId, activeApp.state != .runningForeground {
activeApp = activateTarget(bundleId: bundleId, reason: "interaction_foreground_guard")
} else if requestedBundleId == nil, activeApp.state != .runningForeground {
ensureRunnerHostAppActive(reason: "interaction_missing_app_bundle")
activeApp = app
}
let skipInteractionExistenceWait = canUseFastForegroundAppGuard(
activeApp: activeApp,
requestedBundleId: requestedBundleId,
command: command.command
)
if !skipInteractionExistenceWait && !activeApp.waitForExistence(timeout: 2) {
if let bundleId = requestedBundleId {
return .response(Response(ok: false, error: ErrorPayload(message: "app '\(bundleId)' is not available")))
}
return .response(Response(ok: false, error: ErrorPayload(message: "runner app is not available")))
}
applyInteractionStabilizationIfNeeded()
}
}
return .context(ActiveCommandContext(app: activeApp))
}
func executeOnMainPrepared(
command: Command,
activeApp: XCUIApplication,
alertDeadline: Date? = nil
) throws -> Response {
var activeApp = activeApp
if command.command != .tap && command.command != .type && !isReadOnlyCommand(command) {
clearRememberedTextEntryTap()
}
switch command.command {
case .status, .activate, .terminate, .targetReset, .shutdown, .recordStart, .recordStop, .uptime:
return Response(
ok: false,
error: ErrorPayload(
code: "UNSUPPORTED_OPERATION",
message: "\(command.command.rawValue) cannot be executed through the prepared command path"
)
)
case .tap:
if let selectorKey = command.selectorKey, let selectorValue = command.selectorValue {
let expectedPoint: CGPoint?
if command.allowNonHittableCoordinateFallback == true,
let x = command.x,
let y = command.y
{
expectedPoint = CGPoint(x: x, y: y)
} else {
expectedPoint = nil
}
let match = findElement(
app: activeApp,
selectorKey: selectorKey,
selectorValue: selectorValue,
allowNonHittableFallback: command.allowNonHittableCoordinateFallback == true,
expectedPoint: expectedPoint
)
if match.isAmbiguous {
clearRememberedTextEntryTap()
return Response(ok: false, error: ErrorPayload(code: "AMBIGUOUS_MATCH", message: "selector matched multiple elements"))
}
if let element = match.element {
let frame = element.frame
// XCTest reports closed-drawer/off-viewport items as hittable, then
// "taps" coordinates outside the visible window as a silent no-op.
// Refuse instead; the daemon falls back to tree-based resolution,
// which can prefer an on-screen candidate or explain the off-screen
// state. The check uses the main window frame, not app.frame: on RN
// apps app.frame unions transformed subtrees (a closed drawer at
// negative x), so it happily "contains" unreachable coordinates.
// The DECISION lives in TapPointPolicy (golden parity table with the
// TS twin); onScreenWindowFrame only supplies the frame.
if !match.usedNonHittableFallback
&& !TapPointPolicy.isAllowed(
elementFrame: frame,
windowFrame: onScreenWindowFrame(app: activeApp)
) {
clearRememberedTextEntryTap()
return Response(ok: false, error: ErrorPayload(
code: "ELEMENT_OFFSCREEN",
message: "element resolved off-screen at (\(Int(frame.midX)), \(Int(frame.midY)))"))
}
let isTextEntry = isTextEntryElement(element)
let touchPoint = expectedPoint ?? CGPoint(x: frame.midX, y: frame.midY)
let touchFrame = resolvedTouchVisualizationFrame(
app: activeApp,
x: touchPoint.x,
y: touchPoint.y
)
var fallback: GestureFallback?
if command.synthesized == true {
let policyKind = SynthesizedGesturePolicyKind.coordinateTap
let context = synthesizedCoordinateContext(
app: activeApp,
policy: synthesizedGesturePolicy(policyKind)
)
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
synthesizedTapAt(
app: activeApp,
x: touchPoint.x,
y: touchPoint.y,
context: context
)
}
if case .performed = outcome {
logSynthesizedGesturePolicyDecision(
kind: policyKind,
context: context,
fallbackAttempted: false
)
if isTextEntry {
waitForTextEntryReadinessAfterTap(app: activeApp, element: element)
}
rememberTextEntryTap(isTextEntry ? element : nil)
return gestureResponse(
message: match.usedNonHittableFallback
? "tapped via non-hittable coordinate fallback"
: "tapped",
timing: timing,
frame: .touch(touchFrame),
maestroNonHittableCoordinateFallbackUsed:
command.allowNonHittableCoordinateFallback == true
? match.usedNonHittableFallback
: nil
)
}
logSynthesizedGesturePolicyDecision(
kind: policyKind,
context: context,
fallbackAttempted: true
)
fallback = gestureFallback(strategy: "xctest-coordinate-tap", from: outcome)
}
let (timing, outcome) = performGesture(activeApp) {
if expectedPoint != nil || match.usedNonHittableFallback {
// Maestro compatibility: RN E2E backdoor controls can be 1x1 and
// reported non-hittable by XCTest. Snapshot-resolved targets also
// need coordinate delivery because XCUIElement.activate() does not
// invoke every React Native accessibility wrapper.
return tapAt(app: activeApp, x: touchPoint.x, y: touchPoint.y)
}
return activateElement(app: activeApp, element: element, action: "tap by selector")
}
if let response = unsupportedResponse(for: outcome) {
clearRememberedTextEntryTap()
return response
}
if isTextEntry {
waitForTextEntryReadinessAfterTap(app: activeApp, element: element)
}
rememberTextEntryTap(isTextEntry ? element : nil)
return gestureResponse(
message: match.usedNonHittableFallback ? "tapped via non-hittable coordinate fallback" : "tapped",
timing: timing,
frame: .touch(touchFrame),
fallback: fallback,
maestroNonHittableCoordinateFallbackUsed:
command.allowNonHittableCoordinateFallback == true
? match.usedNonHittableFallback
: nil
)
}
clearRememberedTextEntryTap()
return Response(ok: false, error: ErrorPayload(code: "ELEMENT_NOT_FOUND", message: "element not found"))
}
if let x = command.x, let y = command.y {
let xCTestChannelPenalized = isSnapshotXCTestChannelPenalized(
bundleId: currentBundleId
)
let xCTestTextInputProbeSkipped = !shouldProbeCoordinateTapTextInput(
xCTestChannelPenalized: xCTestChannelPenalized
)
let textInput: XCUIElement?
if !xCTestTextInputProbeSkipped {
textInput = textInputAt(app: activeApp, x: x, y: y)
} else {
// A process-scoped tap cannot authorize later typing without concrete element identity.
textInput = nil
NSLog(
"AGENT_DEVICE_RUNNER_COORDINATE_TAP_TEXT_INPUT_PROBE_SKIPPED bundle=%@",
currentBundleId ?? ""
)
}
var fallback: GestureFallback?
if command.synthesized == true {
let policyKind = SynthesizedGesturePolicyKind.coordinateTap
let context = synthesizedCoordinateContext(
app: activeApp,
policy: synthesizedGesturePolicy(policyKind)
)
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
synthesizedTapAt(app: activeApp, x: x, y: y, context: context)
}
if case .performed = outcome {
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: false)
rememberTextEntryTap(textInput)
return gestureResponse(message: "tapped", timing: timing)
}
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: true)
fallback = gestureFallback(strategy: "xctest-coordinate-tap", from: outcome)
}
let touchFrame = resolvedTouchVisualizationFrame(app: activeApp, x: x, y: y)
let (timing, outcome) = performGesture(activeApp) { tapAt(app: activeApp, x: x, y: y) }
if let response = unsupportedResponse(for: outcome) {
clearRememberedTextEntryTap()
return response
}
rememberTextEntryTap(textInput)
return gestureResponse(
message: "tapped",
timing: timing,
frame: .touch(touchFrame),
fallback: fallback
)
}
clearRememberedTextEntryTap()
return Response(ok: false, error: ErrorPayload(message: "tap requires a selector or x/y"))
case .mouseClick:
guard let x = command.x, let y = command.y else {
return Response(ok: false, error: ErrorPayload(message: "mouseClick requires x and y"))
}
let touchFrame = resolvedTouchVisualizationFrame(app: activeApp, x: x, y: y)
do {
// mouseClick throws (it has no RunnerInteractionOutcome), so it keeps raw measureGesture
// and only routes the success payload through gestureResponse.
var clickError: Error?
let timing = measureGesture {
do {
try mouseClickAt(app: activeApp, x: x, y: y, button: command.button ?? "primary")
} catch {
clickError = error
}
}
if let clickError {
throw clickError
}
return gestureResponse(message: "clicked", timing: timing, frame: .touch(touchFrame))
} catch {
return Response(ok: false, error: ErrorPayload(message: error.localizedDescription))
}
case .longPress:
guard let x = command.x, let y = command.y else {
return Response(ok: false, error: ErrorPayload(message: "longPress requires x and y"))
}
let duration = (command.durationMs ?? 800) / 1000.0
let touchFrame = resolvedTouchVisualizationFrame(app: activeApp, x: x, y: y)
let (timing, outcome) = performGesture(activeApp) {
longPressAt(app: activeApp, x: x, y: y, duration: duration)
}
if let response = unsupportedResponse(for: outcome) {
return response
}
return gestureResponse(message: "long pressed", timing: timing, frame: .touch(touchFrame))
case .drag:
guard let x = command.x, let y = command.y, let x2 = command.x2, let y2 = command.y2 else {
return Response(ok: false, error: ErrorPayload(message: "drag requires x, y, x2, and y2"))
}
let defaults = runnerDragCommandDefaults(command)
return executeDragGesture(
activeApp: activeApp,
x: x,
y: y,
x2: x2,
y2: y2,
durationMs: defaults.durationMs,
synthesized: command.synthesized == true,
message: "dragged",
synthesizedPolicyKind: .synthesizedDrag
)
case .scroll:
// Fused frame-resolve + drag scroll for non-tvOS. On iOS this intentionally stays on the
// AX-free synthesized coordinate lane so scroll keeps working when XCTest cannot serialize
// the accessibility tree.
guard let rawDirection = command.direction,
let direction = RunnerScrollDirection(rawValue: rawDirection)
else {
return invalidScrollDirectionResponse(commandName: "scroll")
}
let scrollPolicyKind = SynthesizedGesturePolicyKind.scroll
guard let scrollContext = synthesizedCoordinateContext(
app: activeApp,
policy: synthesizedGesturePolicy(scrollPolicyKind)
) else {
return Response(
ok: false,
error: ErrorPayload(message: "scroll could not resolve a usable interaction frame")
)
}
let frame = scrollReferenceFrame(app: activeApp, context: scrollContext)
guard frame.width > 0, frame.height > 0 else {
return Response(
ok: false,
error: ErrorPayload(message: "scroll could not resolve a usable interaction frame")
)
}
let defaults = runnerDragCommandDefaults(command)
guard let plan = runnerScrollGesturePlan(
direction: direction,
amount: defaults.scrollAmount,
pixels: command.pixels,
referenceWidth: frame.width,
referenceHeight: frame.height
) else {
return Response(
ok: false,
error: ErrorPayload(
code: "INVALID_ARGS",
message: "scroll could not compute a gesture plan"
)
)
}
guard scrollDurationIsValid(command.durationMs) else {
return invalidScrollDurationResponse(commandName: "scroll")
}
return executeScrollDragGesture(
activeApp: activeApp,
x: frame.minX + plan.x1,
y: frame.minY + plan.y1,
x2: frame.minX + plan.x2,
y2: frame.minY + plan.y2,
durationMs: defaults.durationMs,
message: "scrolled",
context: scrollContext.withReferenceFrame(frame),
releaseBehavior: command.scrollReleaseBehavior
)
case .desktopScroll:
guard let rawDirection = command.direction,
let direction = RunnerScrollDirection(rawValue: rawDirection)
else {
return invalidScrollDirectionResponse(commandName: "desktopScroll")
}
let appFrame = activeApp.frame
let frame = resolvedTouchReferenceFrame(app: activeApp, appFrame: appFrame)
guard frame.width > 0, frame.height > 0 else {
return Response(
ok: false,
error: ErrorPayload(message: "desktopScroll could not resolve a usable interaction frame")
)
}
guard let plan = runnerScrollGesturePlan(
direction: direction,
amount: command.amount,
pixels: command.pixels,
referenceWidth: frame.width,
referenceHeight: frame.height
) else {
return Response(
ok: false,
error: ErrorPayload(
code: "INVALID_ARGS",
message: "desktopScroll could not compute a wheel plan"
)
)
}
let x = frame.midX
let y = frame.midY
let localX = x - (appFrame.isEmpty ? frame.minX : appFrame.minX)
let localY = y - (appFrame.isEmpty ? frame.minY : appFrame.minY)
guard scrollDurationIsValid(command.durationMs) else {
return invalidScrollDurationResponse(commandName: "desktopScroll")
}
let touchFrame = resolvedTouchVisualizationFrame(
app: activeApp,
x: localX,
y: localY
)
do {
var scrollError: Error?
let timing = measureGesture {
do {
try desktopScrollAt(
app: activeApp,
x: x,
y: y,
direction: direction,
pixels: plan.travelPixels,
durationMs: command.durationMs
)
} catch {
scrollError = error
}
}
if let scrollError {
throw scrollError
}
return gestureResponse(message: "scrolled", timing: timing, frame: .touch(touchFrame))
} catch {
return Response(ok: false, error: ErrorPayload(message: error.localizedDescription))
}
case .remotePress:
guard let button = tvRemoteButton(from: command.remoteButton) else {
return Response(ok: false, error: ErrorPayload(message: "remotePress requires remoteButton"))
}
let duration = (command.durationMs ?? 0) / 1000.0
guard pressTvRemote(button, duration: duration) else {
return Response(
ok: false,
error: ErrorPayload(code: "UNSUPPORTED_OPERATION", message: "remotePress is only supported on tvOS")
)
}
return Response(ok: true, data: DataPayload(message: "remote pressed"))
case .type:
var response: Response?
withTemporaryScrollIdleTimeoutIfSupported(activeApp) {
response = executeTypeCommand(activeApp: activeApp, command: command)
}
return response ?? Response(ok: false, error: ErrorPayload(message: "type produced no response"))
case .swipe:
guard let direction = command.direction else {
return Response(ok: false, error: ErrorPayload(message: "swipe requires direction"))
}
// swipe returns an optional frame (tvOS-only) rather than a RunnerInteractionOutcome, so it
// keeps raw measureGesture and only routes the success payload through gestureResponse.
var executedFrame: DragVisualizationFrame?
let timing = measureGesture {
withTemporaryScrollIdleTimeoutIfSupported(activeApp) {
executedFrame = swipe(app: activeApp, direction: direction)
}
}
guard let dragFrame = executedFrame else {
return Response(ok: false, error: ErrorPayload(message: "swipe is only supported on tvOS"))
}
return gestureResponse(message: "swiped", timing: timing, frame: .drag(dragFrame))
case .findText:
guard let text = command.text else {
return Response(ok: false, error: ErrorPayload(message: "findText requires text"))
}
let found = findElement(app: activeApp, text: text) != nil
return Response(ok: true, data: DataPayload(found: found))
case .querySelector:
guard let selectorKey = command.selectorKey, let selectorValue = command.selectorValue else {
return Response(ok: false, error: ErrorPayload(message: "querySelector requires selectorKey and selectorValue"))
}
return queryElement(app: activeApp, selectorKey: selectorKey, selectorValue: selectorValue)
case .readText:
guard let x = command.x, let y = command.y else {
return Response(ok: false, error: ErrorPayload(message: "readText requires x and y"))
}
guard let text = readTextAt(app: activeApp, x: x, y: y) else {
return Response(ok: false, error: ErrorPayload(message: "readText did not resolve text"))
}
return Response(ok: true, data: DataPayload(text: text))
case .snapshot:
return try executeSnapshotPrepared(command: command, activeApp: activeApp)
case .screenshot:
let screenshot: XCUIScreenshot
#if os(macOS)
// macOS keeps the app-targeted capture behavior for window-level screenshots.
if let bundleId = command.appBundleId, !bundleId.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
let targetApp = XCUIApplication(bundleIdentifier: bundleId)
targetApp.activate()
activeApp = targetApp
// Brief wait for the app transition animation to complete
sleepFor(0.5)
}
if command.fullscreen == true {
screenshot = XCUIScreen.main.screenshot()
} else if let bundleId = command.appBundleId, !bundleId.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
screenshot = screenshotRoot(app: activeApp).screenshot()
} else {
screenshot = XCUIScreen.main.screenshot()
}
#else
screenshot = XCUIScreen.main.screenshot()
#endif
guard let pngData = runnerPngData(for: screenshot.image) else {
return Response(ok: false, error: ErrorPayload(message: "Failed to encode screenshot as PNG"))
}
if command.inlineScreenshot == true {
return Response(
ok: true,
data: DataPayload(imageBase64: pngData.base64EncodedString())
)
}
let fileName = "screenshot-\(Int(Date().timeIntervalSince1970 * 1000)).png"
let filePath = (NSTemporaryDirectory() as NSString).appendingPathComponent(fileName)
do {
try pngData.write(to: URL(fileURLWithPath: filePath))
} catch {
return Response(ok: false, error: ErrorPayload(message: "Failed to write screenshot: \(error.localizedDescription)"))
}
#if os(macOS)
return Response(ok: true, data: DataPayload(message: filePath))
#else
// Return path relative to app container root (tmp/ maps to NSTemporaryDirectory)
return Response(ok: true, data: DataPayload(message: "tmp/\(fileName)"))
#endif
case .back, .backInApp:
if tapInAppBackControl(app: activeApp) {
let message = command.command == .back ? "back" : "backInApp"
return Response(ok: true, data: DataPayload(message: message))
}
return Response(ok: false, error: ErrorPayload(message: "in-app back control is not available"))
case .backSystem:
if performSystemBackAction(app: activeApp) {
return Response(ok: true, data: DataPayload(message: "backSystem"))
}
return Response(ok: false, error: ErrorPayload(message: "system back is not available"))
case .home:
pressHomeButton()
return Response(ok: true, data: DataPayload(message: "home"))
case .rotate:
return executeRotateCommand(command)
case .appSwitcher:
performAppSwitcherGesture(app: activeApp)
return Response(ok: true, data: DataPayload(message: "appSwitcher"))
case .keyboardDismiss:
let result = dismissKeyboard(app: activeApp)
if result.wasVisible && !result.dismissed {
return Response(
ok: false,
error: ErrorPayload(
code: "UNSUPPORTED_OPERATION",
message: "Unable to dismiss the iOS keyboard: the keyboard exposes no dismiss key, and background taps are never attempted (no tap outside the keyboard can be proven side-effect-free)",
hint:
"The on-screen keyboard usually does not block agent-device interactions: press the next target directly instead of retrying dismiss. If that press fails or reports no visible effect, scroll the target into view, or use keyboard enter to press the return key when submission is wanted."
)
)
}
return Response(
ok: true,
data: DataPayload(
message: "keyboardDismiss",
visible: result.visible,
wasVisible: result.wasVisible,
dismissed: result.dismissed,
keyboardDismissMechanism: result.mechanism?.rawValue
)
)
case .keyboardReturn:
let result = pressKeyboardReturn(app: activeApp)
if !result.pressed {
return Response(
ok: false,
error: ErrorPayload(
code: "UNSUPPORTED_OPERATION",
message: "Unable to press the iOS keyboard return key"
)
)
}
return Response(
ok: true,
data: DataPayload(
message: "keyboardReturn",
visible: result.visible,
wasVisible: result.wasVisible
)
)
case .alert:
let action = (command.action ?? "get").lowercased()
let deadline = alertDeadline ?? Date().addingTimeInterval(
Self.alertCommandTimeout(timeoutMs: command.timeoutMs)
)
guard let alert = resolveAlert(app: activeApp, deadline: deadline) else {
// Typed so the host retries on absence alone: a transport or runner failure carries no
// code and must not be mistaken for "no alert yet" (ALERT_NOT_FOUND_RUNNER_CODE).
return Response(
ok: false,
error: ErrorPayload(code: "ALERT_NOT_FOUND", message: "alert not found")
)
}
return handleAlert(alert, action: action, deadline: deadline)
case .gesture:
guard let plan = command.gesturePlan else {
return Response(
ok: false,
error: ErrorPayload(code: "INVALID_ARGS", message: "gesture requires gesturePlan")
)
}
if let validationError = plannedGestureValidationError(plan) {
return Response(
ok: false,
error: ErrorPayload(code: "INVALID_ARGS", message: validationError)
)
}
switch plannedGestureExecution(for: plan) {
case .fastSwipe:
// Validation above guarantees a non-empty, single-pointer path for this execution kind.
let first = plan.pointers[0].samples.first!.point
let last = plan.pointers[0].samples.last!.point
return canonicalPlannedGestureResponse(
executeDragGesture(
activeApp: activeApp,
x: first.x,
y: first.y,
x2: last.x,
y2: last.y,
durationMs: plan.durationMs,
synthesized: true,
message: plan.intent,
synthesizedPolicyKind: .synthesizedDrag,
synthesizedProfile: .fastSwipe
)
)
case .sampled:
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
sampledPlannedGesture(app: activeApp, plan: plan)
}
return plannedGestureResponse(plan: plan, timing: timing, outcome: outcome)
}
case .gestureViewport:
let frame = resolvedTouchReferenceFrame(app: activeApp, appFrame: activeApp.frame)
guard !frame.isNull, !frame.isInfinite, !frame.isEmpty else {
return Response(ok: false, error: ErrorPayload(code: "COMMAND_FAILED", message: "Active app interaction viewport is unavailable"))
}
return Response(ok: true, data: DataPayload(message: "gestureViewport", x: frame.minX, y: frame.minY, x2: frame.width, y2: frame.height))
case .sequence:
return executeSequence(command: command, activeApp: activeApp)
}
}
private func invalidScrollDirectionResponse(commandName: String) -> Response {
Response(
ok: false,
error: ErrorPayload(
code: "INVALID_ARGS",
message: "\(commandName) requires direction up|down|left|right"
)
)
}
private func scrollDurationIsValid(_ durationMs: Double?) -> Bool {
guard let durationMs else { return true }
return durationMs.isFinite && durationMs >= 0 && durationMs <= 10000
}
private func invalidScrollDurationResponse(commandName: String) -> Response {
return Response(
ok: false,
error: ErrorPayload(
code: "INVALID_ARGS",
message: "\(commandName) durationMs must be between 0 and 10000"
)
)
}
private func executeScrollDragGesture(
activeApp: XCUIApplication,
x: Double,
y: Double,
x2: Double,
y2: Double,
durationMs: Double,
message: String,
context: SynthesizedCoordinateContext,
releaseBehavior: ScrollReleaseBehavior?
) -> Response {
#if os(iOS)
return executeDragGesture(
activeApp: activeApp,
x: x,
y: y,
x2: x2,
y2: y2,
durationMs: durationMs,
synthesized: true,
message: message,
synthesizedContext: context,
synthesizedPolicyKind: .scroll,
synthesizedProfile: scrollDragProfile(releaseBehavior: releaseBehavior)
)
#else
return executeDragGesture(
activeApp: activeApp,
x: x,
y: y,
x2: x2,
y2: y2,
durationMs: durationMs,
synthesized: false,
message: message,
synthesizedPolicyKind: .scroll
)
#endif
}
/// Shared drag execution for explicit drag commands. The iOS synthesized lane keeps its
/// fallback policy explicit; viewport scrolling owns a separate single drag specification.
private func executeDragGesture(
activeApp: XCUIApplication,
x: Double,
y: Double,
x2: Double,
y2: Double,
durationMs: Double?,
synthesized: Bool,
message: String,
synthesizedContext: SynthesizedCoordinateContext? = nil,
synthesizedPolicyKind: SynthesizedGesturePolicyKind,
synthesizedProfile: SynthesizedDragProfile = .continuous
) -> Response {
let durationMs = durationMs ?? runnerDefaultDragDurationMs
let commandName = dragCommandName(message: message)
guard x.isFinite, y.isFinite, x2.isFinite, y2.isFinite else {
return Response(
ok: false,
error: ErrorPayload(code: "INVALID_ARGS", message: "\(commandName) requires finite coordinates")
)
}
if synthesized, let synthesizedResponse = executeSynthesizedDragGesture(
activeApp: activeApp,
x: x,
y: y,
x2: x2,
y2: y2,
durationMs: durationMs,
message: message,
context: synthesizedContext,
policyKind: synthesizedPolicyKind,
profile: synthesizedProfile
) {
return synthesizedResponse
}
let dragPoints = keyboardAvoidingDragPoints(app: activeApp, x: x, y: y, x2: x2, y2: y2)
let dragFrame = resolvedDragVisualizationFrame(
app: activeApp,
x: dragPoints.x,
y: dragPoints.y,
x2: dragPoints.x2,
y2: dragPoints.y2
)
var fallback: GestureFallback?
if synthesized {
let durationMs = min(max(durationMs, 16), 10000)
let context = synthesizedCoordinateContext(
app: activeApp,
policy: synthesizedGesturePolicy(synthesizedPolicyKind)
)
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
synthesizedDragAt(
app: activeApp,
x: dragPoints.x,
y: dragPoints.y,
x2: dragPoints.x2,
y2: dragPoints.y2,
durationMs: durationMs,
profile: synthesizedProfile,
context: context
)
}
if case .performed = outcome {
return gestureResponse(message: message, timing: timing, frame: .drag(dragFrame))
}
fallback = gestureFallback(strategy: "xctest-coordinate-drag", from: outcome)
}
let holdDuration = synthesized
? synthesizedSwipeFallbackHoldDuration(durationMs: durationMs)
: coordinateDragHoldDuration()
let (timing, outcome) = performGesture(activeApp) {
dragAt(
app: activeApp,
x: dragPoints.x,
y: dragPoints.y,
x2: dragPoints.x2,
y2: dragPoints.y2,
holdDuration: holdDuration
)
}
if let response = unsupportedResponse(for: outcome) {
return response
}
return gestureResponse(
message: message,
timing: timing,
frame: .drag(dragFrame),
fallback: fallback
)
}
private func executeSynthesizedDragGesture(
activeApp: XCUIApplication,
x: Double,
y: Double,
x2: Double,
y2: Double,
durationMs: Double,
message: String,
context: SynthesizedCoordinateContext?,
policyKind: SynthesizedGesturePolicyKind,
profile: SynthesizedDragProfile
) -> Response? {
#if os(iOS)
let policy = synthesizedGesturePolicy(policyKind)
let context = context ?? synthesizedCoordinateContext(app: activeApp, policy: policy)
guard let plan = axFreeSynthesizedDragPlan(
app: activeApp,
x: x,
y: y,
x2: x2,
y2: y2,
context: context
)
else {
if context?.allowsXCTestCoordinateFallback == true {
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: true)
return executeCoordinateDragFallback(
activeApp: activeApp,
x: x,
y: y,
x2: x2,
y2: y2,
durationMs: durationMs,
message: message,
fallback: nil
)
}
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: false)
return Response(
ok: false,
error: ErrorPayload(
code: "INVALID_ARGS",
message: "\(dragCommandName(message: message)) could not resolve a finite synthesized coordinate frame"
)
)
}
let durationMs = min(max(durationMs, 16), 10000)
let dragFrame = axFreeDragVisualizationFrame(
x: plan.points.x,
y: plan.points.y,
x2: plan.points.x2,
y2: plan.points.y2,
referenceFrame: plan.referenceFrame
)
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
synthesizedDragAt(
app: activeApp,
x: plan.points.x,
y: plan.points.y,
x2: plan.points.x2,
y2: plan.points.y2,
durationMs: durationMs,
profile: profile,
context: plan.context
)
}
if case .performed = outcome {
logSynthesizedGesturePolicyDecision(kind: policyKind, context: plan.context, fallbackAttempted: false)
return gestureResponse(message: message, timing: timing, frame: .drag(dragFrame))
}
if plan.context.allowsXCTestCoordinateFallback {
logSynthesizedGesturePolicyDecision(kind: policyKind, context: plan.context, fallbackAttempted: true)
return executeCoordinateDragFallback(
activeApp: activeApp,
x: plan.points.x,
y: plan.points.y,
x2: plan.points.x2,
y2: plan.points.y2,
durationMs: durationMs,
message: message,
fallback: gestureFallback(strategy: "xctest-coordinate-drag", from: outcome)
)
}
logSynthesizedGesturePolicyDecision(kind: policyKind, context: plan.context, fallbackAttempted: false)
return unsupportedResponse(for: outcome)
#else
return nil
#endif
}
private func executeCoordinateDragFallback(
activeApp: XCUIApplication,
x: Double,
y: Double,
x2: Double,
y2: Double,
durationMs: Double,
message: String,
fallback: GestureFallback?
) -> Response {
let dragPoints = keyboardAvoidingDragPoints(app: activeApp, x: x, y: y, x2: x2, y2: y2)
let dragFrame = resolvedDragVisualizationFrame(
app: activeApp,
x: dragPoints.x,
y: dragPoints.y,
x2: dragPoints.x2,
y2: dragPoints.y2
)
let holdDuration = synthesizedSwipeFallbackHoldDuration(durationMs: durationMs)
let (timing, outcome) = performGesture(activeApp) {
dragAt(
app: activeApp,
x: dragPoints.x,
y: dragPoints.y,
x2: dragPoints.x2,
y2: dragPoints.y2,
holdDuration: holdDuration
)
}
if let response = unsupportedResponse(for: outcome) {
return response
}
return gestureResponse(
message: message,
timing: timing,
frame: .drag(dragFrame),
fallback: fallback
)
}
private func scrollReferenceFrame(app: XCUIApplication, context: SynthesizedCoordinateContext) -> CGRect {
#if os(iOS)
return synthesizedFrameAvoidingKeyboardWhenAllowed(app: app, context: context)
#else
return resolvedTouchReferenceFrame(app: app, appFrame: app.frame)
#endif
}
private func dragCommandName(message: String) -> String {
return message == "scrolled" ? "scroll" : "drag"
}
func currentXCTestFailureCount() -> Int {
return testRun?.failureCount ?? 0
}
func didRecordXCTestFailure(since failureCountBefore: Int) -> Bool {
return currentXCTestFailureCount() > failureCountBefore
}
private func xctestRecordedFailureResponse(command: Command, response: Response) -> Response? {
guard response.ok else { return nil }
if response.data?.runnerFatal == true {
return nil
}
guard !isReadOnlyCommand(command), !isRunnerLifecycleCommand(command.command) else {
return nil
}
return Response(
ok: false,
error: ErrorPayload(
code: "XCTEST_RECORDED_FAILURE",
message: "XCTest recorded a failure while executing \(command.command.rawValue); the action may not have been performed.",
hint: "The iOS runner session was invalidated. Re-observe with a fresh snapshot before retrying; if the accessibility tree is unavailable, use screenshot plus coordinate commands instead of retrying the tap blindly."
)
)
}
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
func runnerCommandFixture(_ json: String) throws -> Command {
try JSONDecoder().decode(Command.self, from: Data(json.utf8))
}
#endif
private func shouldSkipAppActivationPreflight(_ command: Command) -> Bool {
#if os(iOS)
if command.command == .alert {
return true
}
// Coordinate-only synthesized taps can run after an AX-fatal foreground screen because they do not
// need app activation, window lookup, keyboard lookup, or element resolution. Selector/text
// interactions intentionally stay on the normal AX path because they need an element query.
// Scroll/drag/sequence keep the normal foreground guard and stabilization path.
guard command.text == nil, command.selectorKey == nil else { return false }
guard hasCachedTargetForActivationSkip(command: command) else { return false }
return isCoordinateOnlyTap(command)
#else
return false
#endif
}
private func shouldRouteToSpringboardBlockingSystemModal(
_ command: Command,
deadline: Date? = nil
) -> Bool {
#if os(iOS)
guard command.command == .alert || isCoordinateOnlyTap(command) else {
return false
}
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
if let override = blockingSystemModalPresenceOverrideForTesting {
return override
}
#endif
let budgetDeadline = Date().addingTimeInterval(systemModalProbeBudget)
let probeDeadline = deadline.map { min($0, budgetDeadline) } ?? budgetDeadline
// `runMainThreadWork` executes inline for a main-thread caller, so that path cannot use its
// timeout machinery. Direct main-thread dispatch keeps the prior synchronous modal check;
// normal off-main command dispatch uses the bounded probe and post-probe busy recovery.
if Thread.isMainThread {
return firstBlockingSystemModal(
in: springboard,
deadline: probeDeadline
) != nil
}
return boundedBlockingSystemAlertSnapshot(
deadline: probeDeadline
) != nil
#else
return false
#endif
}
private func isCoordinateOnlyTap(_ command: Command) -> Bool {
return command.command == .tap
&& command.text == nil
&& command.selectorKey == nil
&& command.x != nil
&& command.y != nil
}
private func hasCachedTargetForActivationSkip(command: Command) -> Bool {
guard let currentApp, currentApp.state == .runningForeground else { return false }
guard let bundleId = command.appBundleId?.trimmingCharacters(in: .whitespacesAndNewlines),
!bundleId.isEmpty
else {
return true
}
return currentBundleId == bundleId
}
private func resolveAppWithoutActivation(command: Command) -> XCUIApplication {
guard let bundleId = command.appBundleId?
.trimmingCharacters(in: .whitespacesAndNewlines),
!bundleId.isEmpty
else {
return currentApp ?? app
}
if currentBundleId == bundleId, let currentApp {
return currentApp
}
return XCUIApplication(bundleIdentifier: bundleId)
}
func executeTypeCommand(activeApp: XCUIApplication, command: Command) -> Response {
guard let text = command.text else {
return Response(ok: false, error: ErrorPayload(message: "type requires text"))
}
let delaySeconds = Double(max(command.delayMs ?? 0, 0)) / 1000.0
let textEntryMode = resolveTextEntryMode(command)
let target: TextEntryTarget
var resolvedCoordinateContext: SynthesizedCoordinateContext?
var maestroNonHittableCoordinateFallbackUsed: Bool?
if command.allowNonHittableCoordinateFallback == true,
command.x != nil,
command.y != nil
{
// The shared runtime has already resolved this node as non-hittable and
// deliberately selected Maestro's coordinate compatibility route.
maestroNonHittableCoordinateFallbackUsed = true
}
let focusStartedAt = Date()
#if os(iOS)
let xCTestChannelPenalized = isSnapshotXCTestChannelPenalized(bundleId: currentBundleId)
var resolvedCoordinateTarget: TextEntryTarget?
if Self.shouldUseResolvedCoordinateTextEntryRoute(
repairMode: textEntryMode,
hasX: command.x != nil,
hasY: command.y != nil,
xCTestChannelPenalized: xCTestChannelPenalized
), let x = command.x, let y = command.y {
let policyKind = SynthesizedGesturePolicyKind.coordinateTap
let context = synthesizedCoordinateContext(
app: activeApp,
policy: synthesizedGesturePolicy(policyKind)
)
let (_, outcome) = performGesture(activeApp, idleTimeout: false) {
synthesizedTapAt(app: activeApp, x: x, y: y, context: context)
}
if Self.shouldFallbackFromSynthesizedTextEntryFocus(outcome) {
logSynthesizedGesturePolicyDecision(
kind: policyKind,
context: context,
fallbackAttempted: true
)
} else {
logSynthesizedGesturePolicyDecision(
kind: policyKind,
context: context,
fallbackAttempted: false
)
resolvedCoordinateContext = context
resolvedCoordinateTarget = TextEntryTarget(
element: nil,
refreshPoint: CGPoint(x: x, y: y),
prefersFocusedElement: false
)
}
}
#else
let xCTestChannelPenalized = false
let resolvedCoordinateTarget: TextEntryTarget? = nil
#endif
if let resolvedCoordinateTarget {
target = resolvedCoordinateTarget
} else if let selectorKey = command.selectorKey, let selectorValue = command.selectorValue {
// Released daemons may still send selector-keyed type commands even though current
// daemons resolve fill selectors through the runtime tree before reaching the runner.
let match = findElement(
app: activeApp,
selectorKey: selectorKey,
selectorValue: selectorValue,
allowNonHittableFallback: command.allowNonHittableCoordinateFallback == true
)
if match.isAmbiguous {
return Response(ok: false, error: ErrorPayload(code: "AMBIGUOUS_MATCH", message: "selector matched multiple elements"))
}
guard let element = match.element else {
return Response(ok: false, error: ErrorPayload(code: "NO_MATCH", message: "selector did not match an element"))
}
guard isTextEntryElement(element) else {
return Response(ok: false, error: ErrorPayload(code: "INVALID_TARGET", message: "selector did not match a text input"))
}
if command.allowNonHittableCoordinateFallback == true {
maestroNonHittableCoordinateFallbackUsed = match.usedNonHittableFallback
}
target = focusTextInputForTextEntry(app: activeApp, element: element)
} else {
target = focusTextInputForTextEntry(app: activeApp, x: command.x, y: command.y)
}
NSLog(
"AGENT_DEVICE_RUNNER_TEXT_ENTRY_PHASE commandId=%@ phase=focus durationMs=%.1f chars=%d mode=%@",
command.commandId ?? "",
Date().timeIntervalSince(focusStartedAt) * 1000.0,
text.count,
textEntryModeName(textEntryMode)
)
if textEntryMode == .replacement {
#if os(iOS)
let canReplaceResolvedFirstResponder = Self.shouldUseSynthesizedFirstResponderReplacement(
hasResolvedElement: target.element != nil,
hasRefreshPoint: target.refreshPoint != nil,
xCTestChannelPenalized: xCTestChannelPenalized
)
#else
let canReplaceResolvedFirstResponder = false
#endif
guard target.element != nil || canReplaceResolvedFirstResponder else {
let message =
(command.x != nil && command.y != nil)
? "no text input found at the provided coordinates to clear"
: "no focused text input to clear"
return Response(ok: false, error: ErrorPayload(message: message))
}
}
let textResult = typeTextReliably(
app: activeApp,
target: target,
text: text,
delaySeconds: delaySeconds,
repairMode: textEntryMode,
xCTestChannelPenalized: xCTestChannelPenalized,
synthesizer: PrivateXCTestTextEntrySynthesizer(),
commandId: command.commandId
)
if let failure = textResult.failure {
return Response(
ok: false,
error: ErrorPayload(code: failure.rawValue, message: failure.message, hint: failure.hint)
)
}
if textResult.verified == false {
let expected = textResult.expectedText ?? ""
let observed = textResult.observedText ?? ""
return Response(
ok: false,
error: ErrorPayload(
code: "TEXT_ENTRY_MISMATCH",
message: "text entry verification failed: expected \"\(expected)\", observed \"\(observed)\""
)
)
}
let point = target.refreshPoint
let frame: CGRect
if let resolvedCoordinateContext {
frame = resolvedCoordinateContext.referenceFrame
} else if point != nil {
frame = activeApp.frame
} else {
// Bare `type` has no coordinate response to normalize. Avoid serializing the
// application AX tree only to emit unused reference dimensions.
frame = .zero
}
return Response(
ok: true,
data: DataPayload(
message: textResult.repaired ? "typed after repair" : "typed",
x: point.map { Double($0.x) },
y: point.map { Double($0.y) },
referenceWidth: frame.isEmpty ? nil : Double(frame.width),
referenceHeight: frame.isEmpty ? nil : Double(frame.height),
maestroNonHittableCoordinateFallbackUsed: maestroNonHittableCoordinateFallbackUsed,
textEntryRoute: textResult.textEntryRoute
)
)
}
}