mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
c77bc40d48
* refactor(daemon): migrate clipboard onto request-bound runtimes (R55) Wave 6 unit 1 of the ADR 0019 platform-free daemon migration (#1739). `clipboard` leaves the legacy dispatch projection: admission is now the action-selected `readClipboard`/`writeClipboard` fact the parsed subcommand names, and the only execution is that one bound operation. - new `@agent-device/contracts/clipboard-runtime` facet, riding the existing `Interactor` seam through `interactor-operation-binding.ts`; read and write are separate cells because a provider can genuinely expose one half only. - every owner states its own cells: Apple gains a `system/` facts module (simulator or the macOS host, matching the retired `supportsHostOrSimulatorSurface` closure), Android admits every real kind, Linux the desktop device, and HarmonyOS/Vega/web refuse -- none ever carried a bucket. Limrun reuses the local Android interactor and refuses on iOS; WebDriver rides interactor reachability like `back`/`home`. - retires the `core/dispatch.ts` clipboard arm and handler, the descriptor's capability bucket and `dispatch` leaf, and the Apple plugin's clipboard admission closure. `handlers/session.ts` loses its inline handler (and its last `dispatchCommand`/`requireCommandSupported` imports) to the new `handlers/session-clipboard.ts`. - `bindLocalInteractorOperationSet` collapses the byte-identical local interaction bind list Android and Linux each held a copy of. Cutover row R55 with its retirement, admission-member and single-bind claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * refactor(daemon): migrate app-switcher onto request-bound runtimes (R56) Wave 6 unit 2 of the ADR 0019 platform-free daemon migration (#1739). `app-switcher` leaves the legacy dispatch projection: admission is the owner's `appSwitcher` fact and the only execution is that one bound operation, resolved by the generic route alongside back/home/orientation/tv-remote. - new `@agent-device/contracts/app-switcher-runtime` facet on the shared `Interactor` seam, bound through the interactor catalog. - Apple states one springboard reading for `home` and `app-switcher` (parity: the retired `supportsAppAndDeviceLifecycle` closure gated both off the same per-AppleOS row, so macOS and watchOS refuse); Android admits every real kind; HarmonyOS admits both kinds, restating the retired overlay membership; Linux/Vega/web refuse. Limrun reuses the local Android interactor and refuses on iOS; WebDriver rides interactor reachability. - retires the `core/dispatch.ts` arm, the capability bucket, the `dispatch` leaf, `HARMONYOS_SUPPORTED_COMMANDS` membership, the Apple plugin closure, and the now-readerless `appAndDeviceLifecycle` row in the per-AppleOS table. - router tests that used `app-switcher` as their legacy-dispatch stand-in move onto bound operations; the typed-error `supportedOn` test moves to `perf`, the one command that keeps a capability-matrix row after this wave. Cutover row R56 with its retirement, admission-member and single-bind claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * refactor(daemon): migrate trigger-app-event onto request-bound runtimes (R57) Wave 6 unit 3 of the ADR 0019 platform-free daemon migration (#1739). `trigger-app-event` leaves the legacy dispatch projection: admission is the owner's `triggerAppEvent` fact and the only execution is that one bound operation. The split follows ADR 0019 §2 — a facet input names no command, request, or CLI flag. The event name pattern, the payload size limit, and the per-platform `AGENT_DEVICE_*_APP_EVENT_URL_TEMPLATE` are daemon policy and stay in `core/app-events.ts`; what reaches the owner is a resolved URL to open. They also stay downstream of admission, where the retired `dispatchCommand` ran them, so an unsupported device still reports its unsupported cell rather than an argument error. - new `@agent-device/contracts/app-event-runtime` facet on the shared `Interactor` seam, bound through the interactor catalog. - Apple admits every leaf with a constructible interactor (no closure ever gated this command beyond its bucket), Android every real kind, and Linux/HarmonyOS/Vega/web refuse. It is the one system leaf both Limrun legs serve, since each implements `open`; WebDriver rides interactor reachability. - retires the `core/dispatch.ts` arm and handler, the capability bucket, the `dispatch` leaf, and the session route's last capability-gate-then-`dispatchCommand` thunk: every leaf on that route now supplies a bind-and-execute thunk. - the end-to-end delivery tests keep their shell-level assertions and move onto the migrated composition. Cutover row R57 with its retirement and single-bind claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * refactor(daemon): migrate settings onto request-bound runtimes and retire the legacy dispatcher (R58) Wave 6 unit 4 of the ADR 0019 platform-free daemon migration (#1739). `settings` was the last `DISPATCH_HANDLERS` arm, so this change closes the command and retires the legacy command dispatcher whole. - new `@agent-device/contracts/settings-runtime` facet on the shared `Interactor` seam, bound through the interactor catalog. What reaches the owner is its own settings vocabulary (setting, state, resolved app id, typed coordinates); the CLI parse, the macOS setting-name gate, the clear-app-state app-id check and the coordinate typing are daemon policy and stay daemon-side, downstream of admission where the retired leaf ran them. - Apple shares clipboard's exact host-or-simulator reading (the retired admission intersected the `settings` bucket with the same `supportsHostOrSimulatorSurface` closure); Android admits every real kind; HarmonyOS matches its retired overlay membership; Linux/Vega/web refuse. Limrun splits Android-reuse / iOS-refusal like `app-switcher`; WebDriver refuses unconditionally, since its interactor declares settings unsupported. - retires `dispatchCommand`, `dispatchWithInteractor`, `dispatchKnownCommand`, `DISPATCH_HANDLERS`, `listRegisteredDispatchCommandNames`, and the request router's `executeGenericPlatformCommand` fallback. `core/dispatch.ts` keeps only `dispatchGestureViewport`, whose last consumers are replay/test. Retiring the dispatcher surfaced two callers broken since Wave 5 moved `press` onto a bound runtime: react-native overlay dismissal and the opt-in interaction no-change retry both called `dispatchCommand(device, 'press', …)`, which has thrown `INVALID_ARGS: Unknown command: press` on main since R48. Both now run the same bound `tapPoint` every other touch leaf uses. The retry declares its own callback seam rather than importing runtime admission, so the policy stays readable without the binding stack — and that inversion, plus the dispatcher's retirement, drops the largest type-level import cycle from 25 files to 21. Cutover row R58 with its retirement and single-bind claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * refactor(daemon): migrate alert, react-native and capabilities onto facts (R59/R61/R63) Wave 6 units 5, 7 and 9 of the ADR 0019 platform-free daemon migration (#1739), plus the residue reclassification the tracker asks for as an analysis task. R59 `alert` — new `@agent-device/contracts/alert-runtime` facet with four action-selected legs (`readAlert`, `awaitAlert`, `acceptAlert`, `dismissAlert`) on the shared `Interactor` seam. The daemon route admits and binds exactly the leg the parsed subcommand names, and the poll and retry windows move to the owners with it: how long a transient sheet takes to appear, and how many times to re-ask a runner that says it is not there yet, are family mechanics, not request policy. `src/platforms/apple/alert.ts` now holds the Apple windows verbatim (with the macOS-helper / XCTest-runner split), and Android's legs read the same presented tree `snapshot` publishes, which is why their occlusion reading still holds. Apple's cell is the retired `supportsAlertSurface` closure restated as facts — the host-or-simulator reading widened by physical iOS — and that closure was the per-AppleOS capability table's last reader, so `src/platforms/apple/capabilities.ts` goes with it. R61 `react-native` — the command's device work moved onto a bound `tapPoint` with R48; this retires the capability gate that still stood in front of it and moves admission ahead of the observing capture, so an owner that cannot dismiss an overlay refuses without first spending a snapshot on it. That exposed a real defect: the request handler chain never forwarded the request's runtime bindings to this route, so the dismissal leg had been reaching a missing gateway ever since R48 — only the no-overlay-detected path returned early enough to hide it. Fixed, with a chain-level regression test. R63 `capabilities` — the projection now reads each command's own declared `platformExecution` uses instead of a hand-written map plus a "no capability bucket means supported everywhere" fallback. That fallback is what let a stopped Android AVD advertise `snapshot press fill` it cannot run, and a Vega VVD advertise every migrated command; both collapse to the fact-derived set here. The command itself executes nothing on a device, so it declares `none`. Residue: `batch`, `debug` and `events` reclassify to `none` — each reaches no device and delegates nothing that does. `replay`/`test` keep their gesture viewport and boot-diagnostics edges, `daemon`/`web` hold platform imports in their own CLI modules, and `react-devtools` still injects device-runtime `runtime`, so all five stay `legacy`. Cutover rows R59 and R61 with their retirement and single-bind claims. Descriptors: 32 legacy at the wave checkpoint, 9 now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(daemon): restore two settings/alert sequences the migration had shifted Self-review of the Wave 6 diff against `origin/main` found two places where the migrated routes were faithful in what they did but not in when: - `settings` typed its location coordinates before expiring the ref frame, so a request that failed on a bad coordinate no longer expired it. The retired route expired the frame first, then emitted its diagnostic, then typed the coordinates inside the leaf. Same order again. - `alert` narrowed a frontmost-app session to "no bundle" in the daemon, which also stripped the bundle from the XCTest runner leg. That narrowing was only ever the macOS helper's, and it already lives in `platforms/apple/alert.ts`; the runner leg gets `session.appBundleId` unconditionally again, pinned by a test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(daemon): address adversarial review of the Wave 6 cutovers Three independent reviews (behavior parity, correctness, ADR 0019 conformance) ran against the branch. What they found, and what changed: Correctness - The R48 retry seam was unreachable. `captureSnapshot` builds it from the request's runtime bindings, but no caller forwarded them, so every retry resolved to a skip. The `snapshot` route now threads `inspectFacts`/ `bindDevice` through `createSnapshotRuntime` and the daemon snapshot backend down to the capture. - A retry tap that rejected escaped the capture it was decorating and turned a plain `snapshot` into an error. It is caught and reported as a skip, matching what the seam's own contract already claimed. - The attempt is spent before the device work again, as the retired route did, so an owner that fails mid-flight cannot be re-attempted from a full budget. - `react-native dismiss-overlay` reached its required `tapPoint` through `?.` and answered `dismissed: true` when the operation was absent. It refuses. - `factOwnedCapabilityAvailable` indexed the facts map unguarded, and treated an empty `required` as proof (`[].every` is vacuously true). Both fail closed. ADR 0019 conformance - §6 forbids a `none` descriptor from binding a device, and `capabilities` bound three times to answer `logs`/`network`/`record`. Every owner composes a binding's facts with the same function `inspectFacts` calls, so those probes read back values the single inspection already carries — at the cost of a device claim on a read-only query. They are gone, and with them the last three empty-`required` admission uses. - §9 is one admission per handler; the retry tap re-admitted on every retry round. It memoizes per device. - `installFamilyCapabilityAvailable` was scaffolding this wave was scheduled to retire: the general projection returns the same verdict for all four install-family commands. Deleted. Leftovers the cutovers created - `requireCommandSupported` lost its last production caller when R56 migrated `app-switcher`: every generic-route command is admitted from owner facts before the dispatcher runs. The dead arm, the function, and `commandUsesDeviceRuntimeExecution` are removed. - `CommandDispatchFacet`, `descriptor.dispatch`, and `explain`'s `dispatch=` field described a dispatcher R58 deleted. - `request-router-android-modal.test.ts` asserted on a `dispatchCommand` mock whose module export no longer exists, so three assertions were vacuous. - `generic-route-runtime-completeness.test.ts` now exists — a comment claimed it did. It pins the routing table as total over the generic route. - Comments and test names describing the retired dispatcher, the deleted AppleOS capability table, and a react-native regression that never shipped. Also records two deliberate provider cell changes the migration made (physical Apple `clipboard` admitted, provider `alert` refused) and the react-native widening to Linux, web and HarmonyOS, and drops a scratch probe file that was committed by accident. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(apple): type the alert-absence retry instead of matching error prose Review blocker 1 on #2021. The Apple alert legs decided retry and hint eligibility by substring-matching error messages for "alert not found" / "no alert", and `alert wait` swallowed *every* read failure. A dead runner, an unreachable macOS helper or a canceled request was therefore spent as poll budget and finally reported as `alert wait timed out`, hiding the real cause. Both backends now state absence as typed evidence: - The XCTest runner answers `ErrorPayload(code: "ALERT_NOT_FOUND", ...)`. It is diagnostic-only, so it stays `COMMAND_FAILED` on the wire and surfaces as `details.runnerErrorCode` — the same shape `RUNNER_BUSY` already used. - The macOS helper adds `reason: "alert-not-found"` to its JSON error details, which the helper client already forwards verbatim. `isAlertNotFoundError` reads only those two fields. `awaitAppleAlert` re-throws anything that is not a typed absence instead of polling through it, and the scoped-snapshot fallback hint attaches to typed absence alone. The three tests the review asked for, plus coverage the daemon-altitude copies could not express: a non-absence failure propagates immediately from `wait`; an action does not retry a failure whose message merely reads like an absence; the macOS helper's typed reason is retried like the runner's. The daemon-level non-absence test moved to the family suite that owns this policy since R59, lowering that file's size pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(runtime): admit clipboard and provider operations from what execution checks Review blocker on #2021, reproduced on a Pixel 9 Pro XL / Android 36 emulator: `capabilities` advertised `clipboard`, then `clipboard read` failed with `UNSUPPORTED_OPERATION: Android shell clipboard read is not supported on this device.` Admission and execution were consulting different authorities, which ADR 0019 §2 forbids — a bound operation must already be admitted. Android. `cmd clipboard` has no shell implementation on every build, and the retired bucket admitted both halves on every real Android kind, leaving the leaf to discover the refusal after the fact. Support is now a fact: the owner probes once per device (cached for its lifetime — a build's shell command set cannot change while the device is up) and states `owner-capability-missing` when adb names the condition. The probe is definitive in one direction only: adb saying so means unsupported, a probe that cannot run means unknown, and reporting unknown as unsupported would hide a working clipboard behind a transport hiccup. The predicate moves to `@agent-device/contracts/android-clipboard-support` so admission and the leaf's own defense-in-depth check cannot drift apart. Cost, stated plainly: the first facts inspection per device now spends one adb round trip, including for requests that never touch the clipboard. WebDriver. `webdriver-interactor.ts` refuses through `capabilitySupported`, while fact generation admitted from interactor reachability alone — so a provider configured with `capabilityOverrides: { 'clipboard.read': 'unsupported' }` was admitted and then thrown out of. The declared capability map is now an input to fact generation, and the refusal carries the map author's own note. Applied to every operation with an unambiguous capability key, not just the two named in review: the mechanism is identical and a half-applied fix would leave the same defect for `back`/`home`/`orientation`/`tap`/`fill`/`type`/`scroll`. Behavior is unchanged by default — every one of those is `supported` or `partial` in the base map — so only an explicit override bites. `focus`, the gesture tiers and `trigger-app-event` keep reachability: no capability key maps to them 1:1. Also collapses the eight identical `*RetiredDispatchProjectionProof` wrappers in the cutover table into one parameterized factory (second review point). Parity tests: an Android build reporting either unsupported-shell phrasing, the probe cache, an adb failure staying admitted, and a WebDriver override refused at admission for each keyed operation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * refactor(layering): split the Wave 6 cutover rows into a sibling module Second review P2 on #2021. `runtime-command-cutover-table.ts` had reached 1,325 lines, past the point where one read covers it. Wave 6's eight rows move to `runtime-command-cutover-table-wave6.ts` and are spread back in, leaving the table at 1,095 lines. The split is by wave because that is how these rows are retired: a wave's rows are deleted together once the ADR declares its commands' migrations closed, and deleting a whole file is a cleaner end than excising a run of literals from the middle of a larger one. `retiredDispatchProjectionProof` moves to the shared extensions module, since both tables now use it — the main table for `snapshot`/`diff`, the sibling for its own eight. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(android): never fabricate clipboard availability from a failed probe Review blocker on #2021. The probe I added had a `catch { return true }`, then cached that result by device id for the runtime owner's lifetime. A transient adb offline or timeout therefore made `capabilities` advertise the clipboard on a build with no clipboard shell — recreating the exact lie the fix was for, and pinning it for the rest of the session. A test locked the behavior in. Support is now a typed verdict with three states, because "we could not ask" is not "it works": `supported | unsupported | probe-failed`. Only a definitive answer is cached; `probe-failed` refuses conservatively with a hint saying support could not be determined, and is deliberately not remembered, so the next inspection asks again. The same change repairs the ownership boundary. Turning raw adb stdout/stderr into a verdict is Android tool knowledge, so it belongs to the Android owner, not to shared vocabulary — `@agent-device/contracts/android-clipboard-support` now carries the typed union alone. The parser returns to `src/platforms/android/adb.ts` and runs in exactly one place, behind a new `AndroidToolHost.probeClipboardShellSupport` that hands owners the verdict. That also settles which Android home owns it: R13 lets only `src/platform-runtime.ts` import `@agent-device/platform-android`, so a parser shared between the package and the root leaf cannot live in the package either. Tests now cover the failure path the previous ones locked the wrong way: a failed probe refuses instead of admitting, its refusal says it could not determine support rather than claiming the build lacks it, and it is not cached — a second inspection re-probes and admits once the device answers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * refactor(contracts): declare each interactor operation once Second review P1 on #2021. `interactor-operation-catalog.ts` declared the same operation set three times — a name tuple, a complete local binder map, and a complete provider binder map — and each facet carried a mirrored `bindLocal…Interactor`/`bindProvider…Interactor` pair whose only difference was which interactor source to use and which label a refusal names. There is now one row per operation, carrying its facts key, its provider refusal label, and the facet's own executor. The local/provider split lives in the two adapters, which differ by exactly the thing that differs: the interactor source. Adding an operation is adding one row. Deleted: the parallel tuple, both binder maps, 32 mirrored wrappers across ten facet modules, and the per-facet `Local…`/`Provider…InteractorResolver` aliases that existed only to be re-exported. Kept: every facet's typed executor, now exported as its binding surface. Net −563 production lines in `packages/contracts`. Two consumers moved onto the catalog's public entry point rather than keeping a private path to a single operation: the app-event delivery test and the provider scenario fixture, whose two hand-bound keyboard legs are now whichever legs its facts admit. Each facet's tests spell out the composition the retired wrappers performed, so every assertion still exercises one executor reached through one source. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(android): let only a clean adb exit prove clipboard support Third review P1 on #2021. The typed verdict landed one layer too high. The adapter probe runs `adb shell cmd clipboard get text` with `allowFailure`, so a non-zero exit comes back as an ordinary result rather than a throw — and the only thing standing between that result and `supported` was the missing-shell prose check. A device that had gone offline, was unauthorized, timed out, or failed for any other reason produced none of that prose, so it fell through to `supported` and was then cached by device id for the runtime owner's lifetime. The `catch` I added guarded the one path adb almost never takes. Each adb outcome now proves only what it can: - `exitCode === 0` is the sole evidence of support, because it is the only result that shows the command ran. - The recognized missing-shell prose is the sole evidence of absence, and is read before the exit code — adb reports that condition non-zero, so checking the code first would turn every honest `unsupported` into a refusal. - Everything else — non-zero without that prose, and the transport throw — is `probe-failed`, which admission refuses and the cache does not remember. The package tests mocked the typed verdict, so they sat downstream of the bug and could not see it. The regression is therefore at the adapter, over the raw adb result: four planted reds (offline, unauthorized, device-not-found, generic failure) that all returned `supported` before this change, plus the two definitive verdicts and the ordering case that keeps `unsupported` reachable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(android): never read adb's refusal prose out of the clipboard's contents Fourth review P1 on #2021, and a second instance of the same bug it names. The previous fix read `isClipboardShellUnsupported(stdout, stderr)` before the exit code. On a *successful* `cmd clipboard get text`, stdout is the clipboard's contents — arbitrary user text. Anyone who had copied "unknown command" or "no shell command implementation" (from a terminal, a bug report, this repo) had their own working clipboard classified `unsupported`, and the runtime owner cached that for its lifetime. Ordering prose ahead of the exit code to keep `unsupported` reachable traded one wrong admission for another. The exit code is decisive on its own when it is zero, so it goes first. Only a call that failed can carry prose about the call itself, which makes the missing- shell phrases meaningful on non-zero exits alone: if (result.exitCode === 0) return 'supported'; return isClipboardShellUnsupported(...) ? 'unsupported' : 'probe-failed'; `isClipboardShellUnsupported` now states that precondition, because reading it on a successful call is exactly the mistake to prevent. The same defect was already shipped in the helper's other caller. `runAndroidClipboardShellCommand` in `src/platforms/android/device-input-state.ts` has checked the prose before the exit code since #1950, so `clipboard read` on a clipboard holding either phrase threw `UNSUPPORTED_OPERATION` — telling the user their device does not support a clipboard it had just read correctly. It is not this wave's code and not reachable from the migration, but it is the same helper misused the same way, and documenting a precondition while leaving a caller that violates it invites the next regression. Repaired here, with the failure ordering otherwise unchanged: a non-zero exit still reports missing-shell as `UNSUPPORTED_OPERATION` and anything else as the adb result error. Both repairs are pinned by regressions that fail against the code they replace: four exit-0 cases at the adapter (verified red against the ordering this commit removes), and three at `readAndroidClipboardWithAdb` (verified red against `origin/main`) covering contents that look like a refusal, a genuine missing command, and an unrelated non-zero failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * fix(cli): bring the workflow help card back under its size budget `Coverage (2)` has been red on `main` and on every PR branched from it since #2020, which replaced three short Bootstrap lines with one longer line carrying the new selection semantics. It updated the content matcher for that line but not the size assertion beside it, so the card went to 9003 bytes against the `< 9000` both `cli-help.test.ts` and `cli-help-topics.test.ts` enforce. Nothing #2020 added is removed here — all of it is pinned by the matcher it shipped, and it is the sentence agents most need. The bytes come back from a clumsy repetition elsewhere in the card, where "settle" named itself twice in one clause: ... only when you did not settle, settle reported not settled, or ... ... only when you did not settle, it reported not settled, or ... which reads better short and puts the card at 8999. That is one byte inside the budget, which is the real finding: the card has no slack left, and the next sentence anyone adds re-opens this. The durable fix is a base-owner call between raising the budget and moving a block down into its sub-topic — the mechanism the card already uses, and which its own test documents. Flagged on #2021 rather than decided here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX * test(cli): raise the workflow help-card budget to 9100 The card is a curated agent-facing reference, and #2020 grew it for a good reason: the selection semantics it added are what an agent needs to predict which device a bare `open` picks. Holding that content to a limit set before it existed just moves the cost onto whoever writes the next sentence. 9100 is headroom, not a target. The previous commit left the card at 8999 of 9000 -- one byte -- which is not a state anyone should have to work in, and I had already established there is no slack left to reclaim: no trailing whitespace, and the only repeated runs are the deliberate column alignment in the Escalate footer. Trimming further would have meant deleting content the tests pin as load-bearing. This is explicitly interim. The card is ~9KB of dense prose in one string, and the real answer is to move a block down into its owning sub-topic -- the mechanism the card already uses and its own test documents ("Deep content moved out of the compact card, not deleted"). Raising the ceiling buys room to do that deliberately instead of under a red CI. Both enforcement sites move together, since they measure the same card through different surfaces: `cli-help.test.ts` reads it through the CLI, and `cli-help-topics.test.ts` through `usageForCommand`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019RpfS12XApXqasuAWZJaEX --------- Co-authored-by: Claude <noreply@anthropic.com>
2825 lines
104 KiB
Swift
2825 lines
104 KiB
Swift
import XCTest
|
|
|
|
#if AGENT_DEVICE_RUNNER_UNIT_TESTS && os(iOS)
|
|
import ObjectiveC.runtime
|
|
|
|
private final class RunnerSynthesizedSwipeFailureStub: NSObject {
|
|
@objc(synthesizeSwipeWithApplication:x:y:x2:y2:durationMs:)
|
|
class func synthesizeSwipe(
|
|
application: XCUIApplication,
|
|
x: Double,
|
|
y: Double,
|
|
x2: Double,
|
|
y2: Double,
|
|
durationMs: Double
|
|
) -> String? {
|
|
"forced private synthesis failure"
|
|
}
|
|
}
|
|
|
|
private final class RunnerSynthesizedTapFailureStub: NSObject {
|
|
@objc(synthesizeTapWithApplication:x:y:)
|
|
class func synthesizeTap(application: XCUIApplication, x: Double, y: Double) -> String? {
|
|
"forced private synthesis failure"
|
|
}
|
|
}
|
|
#endif
|
|
|
|
extension RunnerTests {
|
|
// MARK: - Main Thread Dispatch
|
|
|
|
private func currentUptimeMs() -> Double {
|
|
ProcessInfo.processInfo.systemUptime * 1000
|
|
}
|
|
|
|
private func measureGesture(_ action: () -> Void) -> (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double) {
|
|
let gestureStartUptimeMs = currentUptimeMs()
|
|
action()
|
|
return (gestureStartUptimeMs, currentUptimeMs())
|
|
}
|
|
|
|
func synthesizedSwipeFallbackHoldDuration(durationMs: Double) -> TimeInterval {
|
|
min(max((durationMs / 5.0) / 1000.0, 0.016), 0.120)
|
|
}
|
|
|
|
func coordinateDragHoldDuration() -> TimeInterval {
|
|
0.050
|
|
}
|
|
|
|
func unsupportedResponse(for outcome: RunnerInteractionOutcome) -> Response? {
|
|
switch outcome {
|
|
case .performed:
|
|
return nil
|
|
case .unsupported(let message, let hint):
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: "UNSUPPORTED_OPERATION", message: message, hint: hint)
|
|
)
|
|
}
|
|
}
|
|
|
|
/// Optional visualization frame returned with a gesture response.
|
|
enum GestureFrame {
|
|
case none
|
|
case touch(TouchVisualizationFrame?)
|
|
case drag(DragVisualizationFrame)
|
|
}
|
|
|
|
struct GestureFallback {
|
|
let strategy: String
|
|
let message: String
|
|
let hint: String?
|
|
}
|
|
|
|
private func gestureFallback(strategy: String, from outcome: RunnerInteractionOutcome) -> GestureFallback? {
|
|
switch outcome {
|
|
case .performed:
|
|
return nil
|
|
case .unsupported(let message, let hint):
|
|
return GestureFallback(strategy: strategy, message: message, hint: hint)
|
|
}
|
|
}
|
|
|
|
|
|
/// Runs a gesture action with uniform timing capture. Touch gestures pass `idleTimeout: true`
|
|
/// (the default) to run inside the scroll idle-timeout + quiescence-skip wrapper; synthesis
|
|
/// pointer-plan gestures pass `false` because RunnerSynthesizedGesture governs their
|
|
/// own timing. Returns the captured timing and the action's outcome.
|
|
///
|
|
/// NOTE: a new SYNTHESIS gesture must pass `idleTimeout: false` — the default `true` would wrap
|
|
/// it in the scroll idle-timeout/quiescence-skip path and change its runtime behavior.
|
|
func performGesture(
|
|
_ app: XCUIApplication,
|
|
idleTimeout: Bool = true,
|
|
_ action: () -> RunnerInteractionOutcome
|
|
) -> (timing: (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double), outcome: RunnerInteractionOutcome) {
|
|
var outcome = RunnerInteractionOutcome.performed
|
|
let timing = measureGesture {
|
|
if idleTimeout {
|
|
withTemporaryScrollIdleTimeoutIfSupported(app) { outcome = action() }
|
|
} else {
|
|
outcome = action()
|
|
}
|
|
}
|
|
return (timing, outcome)
|
|
}
|
|
|
|
/// Single factory for the success payload every gesture returns (message + gesture timing +
|
|
/// an optional touch/drag visualization frame), so the field shape lives in one place.
|
|
private func gestureResponse(
|
|
message: String,
|
|
timing: (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double),
|
|
frame: GestureFrame = .none,
|
|
fallback: GestureFallback? = nil,
|
|
maestroNonHittableCoordinateFallbackUsed: Bool? = nil
|
|
) -> Response {
|
|
let data: DataPayload
|
|
switch frame {
|
|
case .none:
|
|
data = DataPayload(
|
|
message: message,
|
|
gestureStartUptimeMs: timing.gestureStartUptimeMs,
|
|
gestureEndUptimeMs: timing.gestureEndUptimeMs,
|
|
gestureFallback: fallback?.strategy,
|
|
gestureFallbackMessage: fallback?.message,
|
|
gestureFallbackHint: fallback?.hint,
|
|
maestroNonHittableCoordinateFallbackUsed: maestroNonHittableCoordinateFallbackUsed
|
|
)
|
|
case .touch(let f):
|
|
data = DataPayload(
|
|
message: message,
|
|
gestureStartUptimeMs: timing.gestureStartUptimeMs,
|
|
gestureEndUptimeMs: timing.gestureEndUptimeMs,
|
|
x: f?.x,
|
|
y: f?.y,
|
|
referenceWidth: f?.referenceWidth,
|
|
referenceHeight: f?.referenceHeight,
|
|
gestureFallback: fallback?.strategy,
|
|
gestureFallbackMessage: fallback?.message,
|
|
gestureFallbackHint: fallback?.hint,
|
|
maestroNonHittableCoordinateFallbackUsed: maestroNonHittableCoordinateFallbackUsed
|
|
)
|
|
case .drag(let f):
|
|
data = DataPayload(
|
|
message: message,
|
|
gestureStartUptimeMs: timing.gestureStartUptimeMs,
|
|
gestureEndUptimeMs: timing.gestureEndUptimeMs,
|
|
x: f.x,
|
|
y: f.y,
|
|
x2: f.x2,
|
|
y2: f.y2,
|
|
referenceWidth: f.referenceWidth,
|
|
referenceHeight: f.referenceHeight,
|
|
gestureFallback: fallback?.strategy,
|
|
gestureFallbackMessage: fallback?.message,
|
|
gestureFallbackHint: fallback?.hint
|
|
)
|
|
}
|
|
return Response(ok: true, data: data)
|
|
}
|
|
|
|
/// Gesture plans already return canonical centroid endpoints from the portable runtime.
|
|
/// Keep runner timing/fallback diagnostics, but do not leak the coordinate-drag adapter's
|
|
/// visualization frame into only the fast-fling response shape.
|
|
private func canonicalPlannedGestureResponse(_ response: Response) -> Response {
|
|
guard response.ok, let data = response.data else { return response }
|
|
return Response(
|
|
ok: true,
|
|
data: DataPayload(
|
|
message: data.message,
|
|
gestureStartUptimeMs: data.gestureStartUptimeMs,
|
|
gestureEndUptimeMs: data.gestureEndUptimeMs,
|
|
gestureFallback: data.gestureFallback,
|
|
gestureFallbackMessage: data.gestureFallbackMessage,
|
|
gestureFallbackHint: data.gestureFallbackHint
|
|
)
|
|
)
|
|
}
|
|
|
|
private func plannedGestureResponse(
|
|
plan: RunnerGesturePlan,
|
|
timing: (gestureStartUptimeMs: Double, gestureEndUptimeMs: Double),
|
|
outcome: RunnerInteractionOutcome
|
|
) -> Response {
|
|
if let response = unsupportedResponse(for: outcome) {
|
|
return response
|
|
}
|
|
return gestureResponse(message: plan.intent, timing: timing)
|
|
}
|
|
|
|
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
|
|
func testGestureResponseIncludesSynthesizedTapFallbackDiagnostics() {
|
|
let response = gestureResponse(
|
|
message: "tapped",
|
|
timing: (gestureStartUptimeMs: 1, gestureEndUptimeMs: 2),
|
|
fallback: GestureFallback(
|
|
strategy: "xctest-coordinate-tap",
|
|
message: "Runner synthesized coordinate tap is unavailable",
|
|
hint: "Using XCTest coordinate tap fallback."
|
|
)
|
|
)
|
|
|
|
XCTAssertEqual(response.ok, true)
|
|
XCTAssertEqual(response.data?.gestureFallback, "xctest-coordinate-tap")
|
|
XCTAssertEqual(
|
|
response.data?.gestureFallbackMessage,
|
|
"Runner synthesized coordinate tap is unavailable"
|
|
)
|
|
XCTAssertEqual(response.data?.gestureFallbackHint, "Using XCTest coordinate tap fallback.")
|
|
}
|
|
|
|
func testGestureResponseIncludesMaestroNonHittableFallbackUsage() {
|
|
let response = gestureResponse(
|
|
message: "tapped via non-hittable coordinate fallback",
|
|
timing: (gestureStartUptimeMs: 1, gestureEndUptimeMs: 2),
|
|
frame: .touch(nil),
|
|
maestroNonHittableCoordinateFallbackUsed: true
|
|
)
|
|
|
|
XCTAssertEqual(response.data?.maestroNonHittableCoordinateFallbackUsed, true)
|
|
}
|
|
|
|
func testCanonicalPlannedGestureResponseOmitsDragFrameAndPreservesDiagnostics() {
|
|
let response = gestureResponse(
|
|
message: "fling",
|
|
timing: (gestureStartUptimeMs: 1, gestureEndUptimeMs: 2),
|
|
frame: .drag(
|
|
DragVisualizationFrame(
|
|
x: 160,
|
|
y: 150,
|
|
x2: 40,
|
|
y2: 150,
|
|
referenceWidth: 200,
|
|
referenceHeight: 300
|
|
)
|
|
),
|
|
fallback: GestureFallback(
|
|
strategy: "xctest-coordinate-drag",
|
|
message: "Private synthesis unavailable",
|
|
hint: "Using XCTest coordinate fallback."
|
|
)
|
|
)
|
|
|
|
let canonical = canonicalPlannedGestureResponse(response)
|
|
|
|
XCTAssertEqual(canonical.data?.gestureStartUptimeMs, 1)
|
|
XCTAssertEqual(canonical.data?.gestureEndUptimeMs, 2)
|
|
XCTAssertEqual(canonical.data?.gestureFallback, "xctest-coordinate-drag")
|
|
XCTAssertEqual(canonical.data?.gestureFallbackMessage, "Private synthesis unavailable")
|
|
XCTAssertEqual(canonical.data?.gestureFallbackHint, "Using XCTest coordinate fallback.")
|
|
XCTAssertNil(canonical.data?.x)
|
|
XCTAssertNil(canonical.data?.y)
|
|
XCTAssertNil(canonical.data?.x2)
|
|
XCTAssertNil(canonical.data?.y2)
|
|
XCTAssertNil(canonical.data?.referenceWidth)
|
|
XCTAssertNil(canonical.data?.referenceHeight)
|
|
}
|
|
|
|
#if os(iOS)
|
|
func testSinglePointerFlingFallsBackToXCTestCoordinateDragWhenPrivateSynthesisFails() throws {
|
|
let selector = NSSelectorFromString(
|
|
"synthesizeSwipeWithApplication:x:y:x2:y2:durationMs:"
|
|
)
|
|
guard
|
|
let synthesizedSwipeMethod = class_getClassMethod(RunnerSynthesizedGesture.self, selector),
|
|
let failureStubMethod = class_getClassMethod(RunnerSynthesizedSwipeFailureStub.self, selector)
|
|
else {
|
|
XCTFail("unable to install synthesized swipe failure stub")
|
|
return
|
|
}
|
|
let originalImplementation = method_getImplementation(synthesizedSwipeMethod)
|
|
method_setImplementation(
|
|
synthesizedSwipeMethod,
|
|
method_getImplementation(failureStubMethod)
|
|
)
|
|
app.launch()
|
|
runnerAccessibilityHealth = .healthy
|
|
defer {
|
|
method_setImplementation(synthesizedSwipeMethod, originalImplementation)
|
|
invalidateCachedTarget(reason: "unit_test_cleanup")
|
|
app.terminate()
|
|
}
|
|
let command = try runnerCommandFixture(
|
|
"""
|
|
{"command":"gesture","commandId":"gesture-fling-fallback","gesturePlan":{"topology":"single","intent":"fling","executionProfile":"endpoint-hold","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}}
|
|
"""
|
|
)
|
|
|
|
let response = try executeOnMainPrepared(command: command, activeApp: app)
|
|
|
|
XCTAssertTrue(response.ok)
|
|
XCTAssertEqual(response.data?.message, "fling")
|
|
XCTAssertEqual(response.data?.gestureFallback, "xctest-coordinate-drag")
|
|
XCTAssertEqual(response.data?.gestureFallbackMessage, "forced private synthesis failure")
|
|
XCTAssertEqual(
|
|
response.data?.gestureFallbackHint,
|
|
"Private XCTest event synthesis is required for AX-free coordinate drag on iOS; update Xcode if this persists."
|
|
)
|
|
XCTAssertNil(response.data?.x)
|
|
XCTAssertNil(response.data?.y)
|
|
XCTAssertNil(response.data?.x2)
|
|
XCTAssertNil(response.data?.y2)
|
|
}
|
|
|
|
func testSelectorTapFallsBackToXCTestCoordinateWhenPrivateSynthesisFails() throws {
|
|
let selector = NSSelectorFromString("synthesizeTapWithApplication:x:y:")
|
|
guard
|
|
let synthesizedTapMethod = class_getClassMethod(RunnerSynthesizedGesture.self, selector),
|
|
let failureStubMethod = class_getClassMethod(RunnerSynthesizedTapFailureStub.self, selector)
|
|
else {
|
|
XCTFail("unable to install synthesized tap failure stub")
|
|
return
|
|
}
|
|
let originalImplementation = method_getImplementation(synthesizedTapMethod)
|
|
method_setImplementation(
|
|
synthesizedTapMethod,
|
|
method_getImplementation(failureStubMethod)
|
|
)
|
|
app.launch()
|
|
currentApp = app
|
|
runnerAccessibilityHealth = .healthy
|
|
defer {
|
|
method_setImplementation(synthesizedTapMethod, originalImplementation)
|
|
invalidateCachedTarget(reason: "unit_test_cleanup")
|
|
app.terminate()
|
|
}
|
|
let command = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"selector-tap-fallback","selectorKey":"label","selectorValue":"Agent Device Runner","synthesized":true}"#
|
|
)
|
|
|
|
let response = try executeOnMainPrepared(command: command, activeApp: app)
|
|
|
|
XCTAssertTrue(response.ok)
|
|
XCTAssertEqual(response.data?.message, "tapped")
|
|
XCTAssertEqual(response.data?.gestureFallback, "xctest-coordinate-tap")
|
|
XCTAssertEqual(response.data?.gestureFallbackMessage, "forced private synthesis failure")
|
|
XCTAssertEqual(
|
|
response.data?.gestureFallbackHint,
|
|
"Falling back to XCTest coordinate tap may be slower and can still need a healthy accessibility tree."
|
|
)
|
|
}
|
|
#endif
|
|
|
|
#if os(iOS)
|
|
func testTypeWithoutResolvedInputReturnsTypedFailureBeforeDispatchingText() throws {
|
|
let command = try runnerCommandFixture(
|
|
#"{"command":"type","commandId":"type-without-focus","text":"hello"}"#
|
|
)
|
|
|
|
let response = executeTypeCommand(
|
|
activeApp: XCUIApplication(bundleIdentifier: "com.example.agentdevice.missing-input"),
|
|
command: command
|
|
)
|
|
|
|
XCTAssertFalse(response.ok)
|
|
XCTAssertEqual(response.error?.code, "TEXT_INPUT_NOT_FOCUSED")
|
|
XCTAssertEqual(
|
|
response.error?.hint,
|
|
"Focus a visible text input, then retry type or fill. If the input is not exposed by accessibility, use a coordinate focus command before typing."
|
|
)
|
|
}
|
|
|
|
func testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden() throws {
|
|
// The fixture uses a real text responder with an empty input view to model hardware-keyboard input.
|
|
app.launchArguments = ["--agent-device-text-entry-regression"]
|
|
app.launch()
|
|
defer {
|
|
invalidateCachedTarget(reason: "unit_test_cleanup")
|
|
app.terminate()
|
|
}
|
|
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
|
|
|
|
let textField = app.textFields["agent-device-hardware-keyboard-input"]
|
|
XCTAssertTrue(textField.waitForExistence(timeout: appExistenceTimeout))
|
|
let frame = textField.frame
|
|
XCTAssertFalse(frame.isEmpty)
|
|
|
|
let tapCommand = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-hardware-keyboard-input","selectorKey":"id","selectorValue":"agent-device-hardware-keyboard-input"}"#
|
|
)
|
|
let tapResponse = try executeOnMainPrepared(command: tapCommand, activeApp: app)
|
|
XCTAssertTrue(tapResponse.ok, String(describing: tapResponse.error))
|
|
XCTAssertFalse(
|
|
isKeyboardVisible(app: app),
|
|
"the test must exercise a focused responder with the software keyboard hidden"
|
|
)
|
|
|
|
let failureCountBefore = currentXCTestFailureCount()
|
|
let typeCommand = try runnerCommandFixture(
|
|
#"{"command":"type","commandId":"type-hardware-keyboard","text":"hardware-keyboard"}"#
|
|
)
|
|
let typeResponse = executeTypeCommand(activeApp: app, command: typeCommand)
|
|
|
|
XCTAssertTrue(typeResponse.ok, String(describing: typeResponse.error))
|
|
XCTAssertFalse(didRecordXCTestFailure(since: failureCountBefore))
|
|
XCTAssertEqual(typeResponse.data?.textEntryRoute, "synthesized-first-responder")
|
|
XCTAssertEqual(String(describing: textField.value ?? ""), "hardware-keyboard")
|
|
|
|
let secondFailureCountBefore = currentXCTestFailureCount()
|
|
let secondTypeCommand = try runnerCommandFixture(
|
|
#"{"command":"type","commandId":"type-hardware-keyboard-again","text":"-again"}"#
|
|
)
|
|
let secondTypeResponse = executeTypeCommand(activeApp: app, command: secondTypeCommand)
|
|
|
|
XCTAssertFalse(secondTypeResponse.ok)
|
|
XCTAssertEqual(secondTypeResponse.error?.code, "TEXT_INPUT_NOT_FOCUSED")
|
|
XCTAssertFalse(didRecordXCTestFailure(since: secondFailureCountBefore))
|
|
XCTAssertEqual(String(describing: textField.value ?? ""), "hardware-keyboard")
|
|
}
|
|
|
|
func testBareDelayedTypeFailsWhenTappedInputDisappearsMidCommand() throws {
|
|
app.launchArguments = [
|
|
"--agent-device-text-entry-regression",
|
|
"--agent-device-text-entry-disappear-after-input",
|
|
]
|
|
app.launch()
|
|
defer {
|
|
invalidateCachedTarget(reason: "unit_test_cleanup")
|
|
app.terminate()
|
|
}
|
|
XCTAssertTrue(app.waitForExistence(timeout: appExistenceTimeout))
|
|
|
|
let textField = app.textFields["agent-device-hardware-keyboard-input"]
|
|
XCTAssertTrue(textField.waitForExistence(timeout: appExistenceTimeout))
|
|
let tapCommand = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-disappearing-input","selectorKey":"id","selectorValue":"agent-device-hardware-keyboard-input"}"#
|
|
)
|
|
let tapResponse = try executeOnMainPrepared(command: tapCommand, activeApp: app)
|
|
XCTAssertTrue(tapResponse.ok, String(describing: tapResponse.error))
|
|
|
|
let failureCountBefore = currentXCTestFailureCount()
|
|
let typeCommand = try runnerCommandFixture(
|
|
#"{"command":"type","commandId":"type-disappearing-input","text":"ab","delayMs":50}"#
|
|
)
|
|
let typeResponse = executeTypeCommand(activeApp: app, command: typeCommand)
|
|
|
|
XCTAssertFalse(didRecordXCTestFailure(since: failureCountBefore))
|
|
XCTAssertFalse(typeResponse.ok)
|
|
XCTAssertEqual(typeResponse.error?.code, "TEXT_INPUT_NOT_FOCUSED")
|
|
XCTAssertFalse(textField.exists)
|
|
}
|
|
#endif
|
|
|
|
func testInjectedTapRecordedFailureGateIsTapOnlyAndCountGated() {
|
|
// The seam's recording side cannot run in-bundle (a real XCTIssue would
|
|
// fail this very test run — same constraint the record(_:) suppression
|
|
// tests document); the live daemon proof covers it. This pins the gate.
|
|
XCTAssertFalse(RunnerTests.shouldInjectTapRecordedFailure(command: .tap, remaining: 0))
|
|
XCTAssertTrue(RunnerTests.shouldInjectTapRecordedFailure(command: .tap, remaining: 1))
|
|
XCTAssertFalse(RunnerTests.shouldInjectTapRecordedFailure(command: .type, remaining: 1))
|
|
XCTAssertFalse(RunnerTests.shouldInjectTapRecordedFailure(command: .snapshot, remaining: 1))
|
|
}
|
|
|
|
func testXCTestRecordedFailureResponseFailsMutatingSuccesses() throws {
|
|
let command = try runnerCommandFixture(#"{"command":"tap","commandId":"tap-1"}"#)
|
|
let response = Response(ok: true, data: DataPayload(message: "tapped"))
|
|
|
|
let failureResponse = xctestRecordedFailureResponse(command: command, response: response)
|
|
|
|
XCTAssertEqual(failureResponse?.ok, false)
|
|
XCTAssertEqual(failureResponse?.error?.code, "XCTEST_RECORDED_FAILURE")
|
|
XCTAssertEqual(
|
|
failureResponse?.error?.message,
|
|
"XCTest recorded a failure while executing tap; the action may not have been performed."
|
|
)
|
|
}
|
|
|
|
func testXCTestRecordedFailureResponseDoesNotWrapReadOnlyOrRunnerFatalResponses() throws {
|
|
let snapshotCommand = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-1"}"#)
|
|
let tapCommand = try runnerCommandFixture(#"{"command":"tap","commandId":"tap-1"}"#)
|
|
let runnerFatalResponse = Response(
|
|
ok: true,
|
|
data: DataPayload(runnerFatal: true, runnerFatalReason: "ax_snapshot_unavailable")
|
|
)
|
|
|
|
XCTAssertNil(
|
|
xctestRecordedFailureResponse(
|
|
command: snapshotCommand,
|
|
response: Response(ok: true, data: DataPayload(nodes: [], truncated: false))
|
|
)
|
|
)
|
|
XCTAssertNil(xctestRecordedFailureResponse(command: tapCommand, response: runnerFatalResponse))
|
|
}
|
|
|
|
// Simulator-only from here to the matching #endif: these launch the host app, route through
|
|
// SpringBoard, or assert the iOS-only alert/system-modal branches. Tests outside the
|
|
// `os(iOS)` regions in this file are pure runner decisions and also run on the macOS host
|
|
// lane (ci.yml) — see the classification convention in RunnerTests.swift.
|
|
#if os(iOS)
|
|
func testMissingBundleCommandInvalidatesCompleteCachedTargetState() throws {
|
|
app.launch()
|
|
currentApp = app
|
|
currentBundleId = "com.example.stale-target"
|
|
currentAppProcessIdentifier = 42
|
|
snapshotXCTestPenaltyWarmupExemptionPending = true
|
|
defer {
|
|
invalidateCachedTarget(reason: "unit_test_cleanup")
|
|
app.terminate()
|
|
}
|
|
let command = try runnerCommandFixture(
|
|
#"{"command":"snapshot","commandId":"snapshot-without-bundle"}"#
|
|
)
|
|
|
|
_ = prepareActiveCommandContext(command: command)
|
|
|
|
XCTAssertNil(currentApp)
|
|
XCTAssertNil(currentBundleId)
|
|
XCTAssertNil(currentAppProcessIdentifier)
|
|
XCTAssertFalse(snapshotXCTestPenaltyWarmupExemptionPending)
|
|
}
|
|
|
|
func testSkipAppActivationPreflightIncludesForegroundCachedCoordinateOnlyTaps() throws {
|
|
app.launch()
|
|
currentApp = app
|
|
currentBundleId = nil
|
|
defer {
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
app.terminate()
|
|
}
|
|
let tap = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-1","x":10,"y":20}"#
|
|
)
|
|
|
|
XCTAssertTrue(shouldSkipAppActivationPreflight(tap))
|
|
}
|
|
|
|
func testSkipAppActivationPreflightRejectsMissingChangedAndBackgroundTargets() throws {
|
|
let coordinateTap = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-1","x":10,"y":20}"#
|
|
)
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(coordinateTap))
|
|
|
|
app.launch()
|
|
currentApp = app
|
|
currentBundleId = "com.example.current"
|
|
defer {
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
app.terminate()
|
|
}
|
|
let changedBundleTap = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-2","appBundleId":"com.example.other","x":10,"y":20}"#
|
|
)
|
|
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(changedBundleTap))
|
|
|
|
app.terminate()
|
|
currentApp = app
|
|
currentBundleId = nil
|
|
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(coordinateTap))
|
|
}
|
|
|
|
func testPrepareActiveCommandContextRoutesBlockingSystemModalToSpringboard() throws {
|
|
blockingSystemModalPresenceOverrideForTesting = true
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
defer {
|
|
blockingSystemModalPresenceOverrideForTesting = nil
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
}
|
|
let tap = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-1","x":10,"y":20}"#
|
|
)
|
|
|
|
let preparation = prepareActiveCommandContext(
|
|
command: tap,
|
|
routeToSpringboard: shouldRouteToSpringboardBlockingSystemModal(tap)
|
|
)
|
|
|
|
guard case .context(let context) = preparation else {
|
|
XCTFail("expected command context")
|
|
return
|
|
}
|
|
XCTAssertTrue(context.app === springboard)
|
|
}
|
|
|
|
func testExecuteDispatchedReturnsBusyBeforeBlockingSystemModalProbeDrains() throws {
|
|
app.launch()
|
|
currentApp = app
|
|
currentBundleId = nil
|
|
defer {
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
systemModalProbeOverrideForTesting = nil
|
|
clearSnapshotXCTestChannelPenalty(reason: "test-cleanup")
|
|
app.terminate()
|
|
}
|
|
|
|
final class ResultBox {
|
|
var response: Response?
|
|
var error: Error?
|
|
var commandRecoveredBeforeRelease = false
|
|
var wasBusyBeforeRelease = false
|
|
var hadAbandonedProbeBeforeRelease = false
|
|
var drained = false
|
|
}
|
|
let box = ResultBox()
|
|
let probeStarted = expectation(description: "system-modal routing probe started")
|
|
let verificationFinished = expectation(description: "command recovery and modal probe drain verified")
|
|
let probeReleaseGate = DispatchSemaphore(value: 0)
|
|
let commandFinishedGate = DispatchSemaphore(value: 0)
|
|
systemModalProbeOverrideForTesting = { _ in
|
|
probeStarted.fulfill()
|
|
_ = probeReleaseGate.wait(timeout: .now() + 15)
|
|
return DataPayload(message: "late system modal")
|
|
}
|
|
|
|
let command = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"bounded-modal-routing","x":10,"y":20}"#
|
|
)
|
|
DispatchQueue(label: "agent-device.runner.tests.modal-routing-probe").async {
|
|
do {
|
|
box.response = try self.executeDispatched(command: command)
|
|
} catch {
|
|
box.error = error
|
|
}
|
|
commandFinishedGate.signal()
|
|
}
|
|
DispatchQueue(label: "agent-device.runner.tests.modal-routing-probe-verifier").async {
|
|
let commandWait = commandFinishedGate.wait(
|
|
timeout: .now() + self.systemModalProbeBudget + 3
|
|
)
|
|
box.commandRecoveredBeforeRelease = commandWait == .success
|
|
&& box.error == nil
|
|
&& box.response?.error?.code == "RUNNER_BUSY"
|
|
if case .busy = self.currentMainThreadBusyState() {
|
|
box.wasBusyBeforeRelease = true
|
|
}
|
|
box.hadAbandonedProbeBeforeRelease = self.hasAbandonedTreeCapture()
|
|
|
|
// The XCTest main thread is blocked inside the injected probe, so this verifier owns the
|
|
// ordered release after recording the command result and abandoned-work state above.
|
|
probeReleaseGate.signal()
|
|
let deadline = Date().addingTimeInterval(5)
|
|
while self.hasAbandonedTreeCapture(), Date() < deadline {
|
|
self.sleepFor(0.002)
|
|
}
|
|
box.drained = !self.hasAbandonedTreeCapture()
|
|
verificationFinished.fulfill()
|
|
}
|
|
|
|
wait(for: [probeStarted, verificationFinished], timeout: 15)
|
|
XCTAssertTrue(
|
|
box.commandRecoveredBeforeRelease,
|
|
"the public coordinate tap must return RUNNER_BUSY before the blocked modal probe drains"
|
|
)
|
|
XCTAssertTrue(box.wasBusyBeforeRelease)
|
|
XCTAssertTrue(box.hadAbandonedProbeBeforeRelease)
|
|
XCTAssertTrue(box.drained)
|
|
guard case .idle = currentMainThreadBusyState() else {
|
|
return XCTFail("expected the runner to become idle after the routing probe drained")
|
|
}
|
|
XCTAssertFalse(hasAbandonedTreeCapture())
|
|
}
|
|
|
|
func testSkipAppActivationPreflightRejectsSelectorAndMixedSequenceGestures() throws {
|
|
app.launch()
|
|
currentApp = app
|
|
currentBundleId = nil
|
|
defer {
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
app.terminate()
|
|
}
|
|
let selectorTap = try runnerCommandFixture(
|
|
#"{"command":"tap","commandId":"tap-1","selectorKey":"label","selectorValue":"Search","synthesized":true}"#
|
|
)
|
|
let standardDrag = try runnerCommandFixture(
|
|
#"{"command":"drag","commandId":"drag-1","x":10,"y":20,"x2":30,"y2":40}"#
|
|
)
|
|
let mixedSequence = try runnerCommandFixture(
|
|
"""
|
|
{"command":"sequence","commandId":"seq-1","steps":[
|
|
{"kind":"tap","x":10,"y":20,"synthesized":true},
|
|
{"kind":"doubleTap","x":30,"y":40}
|
|
]}
|
|
"""
|
|
)
|
|
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(selectorTap))
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(standardDrag))
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(mixedSequence))
|
|
}
|
|
|
|
// Launches nothing, but still simulator-only: `shouldSkipAppActivationPreflight` is
|
|
// `#if os(iOS) …guards… #else return false #endif`, so on macOS this asserts a compile-time
|
|
// literal and no edit to the iOS body could make it red. Its five siblings above and below
|
|
// are gated for the same reason.
|
|
func testSkipAppActivationPreflightRequiresCachedForegroundTarget() throws {
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
let scroll = try runnerCommandFixture(
|
|
#"{"command":"scroll","commandId":"scroll-1","direction":"down","pixels":400}"#
|
|
)
|
|
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(scroll))
|
|
}
|
|
|
|
func testSkipAppActivationPreflightKeepsDragScrollAndSequenceOnForegroundGuard() throws {
|
|
app.launch()
|
|
currentApp = app
|
|
currentBundleId = nil
|
|
defer {
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
app.terminate()
|
|
}
|
|
let drag = try runnerCommandFixture(
|
|
#"{"command":"drag","commandId":"drag-1","x":10,"y":20,"x2":30,"y2":40,"synthesized":true}"#
|
|
)
|
|
let scroll = try runnerCommandFixture(
|
|
#"{"command":"scroll","commandId":"scroll-1","direction":"down","pixels":400}"#
|
|
)
|
|
let sequence = try runnerCommandFixture(
|
|
"""
|
|
{"command":"sequence","commandId":"seq-1","steps":[
|
|
{"kind":"tap","x":10,"y":20,"synthesized":true},
|
|
{"kind":"longPress","x":10,"y":200,"durationMs":300}
|
|
]}
|
|
"""
|
|
)
|
|
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(drag))
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(scroll))
|
|
XCTAssertFalse(shouldSkipAppActivationPreflight(sequence))
|
|
}
|
|
|
|
func testSkipAppActivationPreflightIncludesAlertCommands() throws {
|
|
let alert = try runnerCommandFixture(
|
|
#"{"command":"alert","commandId":"alert-1","action":"get"}"#
|
|
)
|
|
|
|
XCTAssertTrue(shouldSkipAppActivationPreflight(alert))
|
|
}
|
|
#endif
|
|
|
|
func testExecuteDispatchedReturnsBusyBeforeMainThreadFastPath() throws {
|
|
let command = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-busy"}"#)
|
|
abandonedMainThreadWorkCount = 1
|
|
abandonedMainThreadWorkSince = Date(timeIntervalSinceNow: -2)
|
|
defer {
|
|
abandonedMainThreadWorkCount = 0
|
|
abandonedMainThreadWorkSince = nil
|
|
}
|
|
|
|
let response = try executeDispatched(command: command)
|
|
|
|
XCTAssertFalse(response.ok)
|
|
XCTAssertEqual(response.error?.code, "RUNNER_BUSY")
|
|
XCTAssertTrue(response.error?.message.contains("previous command") == true)
|
|
}
|
|
|
|
func testExecuteDispatchedReturnsWedgedBeforeMainThreadFastPath() throws {
|
|
let command = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-wedged"}"#)
|
|
abandonedMainThreadWorkCount = 1
|
|
abandonedMainThreadWorkSince = Date(timeIntervalSinceNow: -(mainThreadWedgeThreshold + 1))
|
|
defer {
|
|
abandonedMainThreadWorkCount = 0
|
|
abandonedMainThreadWorkSince = nil
|
|
}
|
|
|
|
let response = try executeDispatched(command: command)
|
|
|
|
XCTAssertFalse(response.ok)
|
|
XCTAssertEqual(response.error?.code, "RUNNER_WEDGED")
|
|
XCTAssertTrue(response.error?.hint?.contains("runner session will be restarted") == true)
|
|
}
|
|
|
|
#if os(iOS)
|
|
func testAlertResolutionCannotBypassRequestedDeadline() throws {
|
|
final class ResultBox {
|
|
var error: Error?
|
|
var probeDeadline: Date?
|
|
var observedDeadline: Date?
|
|
var commandStartedAt: Date?
|
|
}
|
|
let box = ResultBox()
|
|
let releaseResolution = DispatchSemaphore(value: 0)
|
|
let resolutionExited = expectation(description: "bounded alert resolution exited")
|
|
let commandFinished = expectation(description: "alert command respected its deadline")
|
|
let command = try runnerCommandFixture(
|
|
#"{"command":"alert","commandId":"alert-deadline","appBundleId":"com.apple.springboard","action":"get","timeoutMs":500}"#
|
|
)
|
|
currentApp = springboard
|
|
currentBundleId = Self.springboardBundleId
|
|
systemModalProbeOverrideForTesting = { deadline in
|
|
box.probeDeadline = deadline
|
|
return nil
|
|
}
|
|
alertResolutionOverrideForTesting = { deadline in
|
|
box.observedDeadline = deadline
|
|
_ = releaseResolution.wait(timeout: .now() + 1)
|
|
resolutionExited.fulfill()
|
|
return nil
|
|
}
|
|
defer {
|
|
releaseResolution.signal()
|
|
systemModalProbeOverrideForTesting = nil
|
|
alertResolutionOverrideForTesting = nil
|
|
currentApp = nil
|
|
currentBundleId = nil
|
|
}
|
|
|
|
DispatchQueue(label: "agent-device.runner.tests.alert-deadline").async {
|
|
box.commandStartedAt = Date()
|
|
do {
|
|
_ = try self.executeDispatched(command: command)
|
|
} catch {
|
|
box.error = error
|
|
}
|
|
commandFinished.fulfill()
|
|
}
|
|
|
|
wait(for: [commandFinished], timeout: 1)
|
|
let error = box.error as NSError?
|
|
XCTAssertEqual(error?.domain, RunnerErrorDomain.general)
|
|
XCTAssertEqual(error?.code, RunnerErrorCode.mainThreadExecutionTimedOut)
|
|
XCTAssertNotNil(box.probeDeadline)
|
|
XCTAssertNotNil(box.observedDeadline)
|
|
if let probeDeadline = box.probeDeadline,
|
|
let observedDeadline = box.observedDeadline,
|
|
let commandStartedAt = box.commandStartedAt
|
|
{
|
|
XCTAssertEqual(probeDeadline.timeIntervalSince(observedDeadline), 0, accuracy: 0.01)
|
|
XCTAssertEqual(observedDeadline.timeIntervalSince(commandStartedAt), 0.5, accuracy: 0.05)
|
|
}
|
|
|
|
releaseResolution.signal()
|
|
wait(for: [resolutionExited], timeout: 1)
|
|
}
|
|
#endif
|
|
|
|
func testRunMainThreadWorkExecutesOffMainCallerOnMainThread() {
|
|
final class ResultBox {
|
|
var observedMainThread: Bool?
|
|
var error: Error?
|
|
}
|
|
let box = ResultBox()
|
|
let finished = expectation(description: "off-main caller finished")
|
|
|
|
DispatchQueue(label: "agent-device.runner.tests.off-main").async {
|
|
do {
|
|
box.observedMainThread = try self.runMainThreadWork(
|
|
timeout: 1,
|
|
timeoutError: self.mainThreadExecutionTimeoutError
|
|
) {
|
|
Thread.isMainThread
|
|
}
|
|
} catch {
|
|
box.error = error
|
|
}
|
|
finished.fulfill()
|
|
}
|
|
|
|
wait(for: [finished], timeout: 2)
|
|
XCTAssertNil(box.error)
|
|
XCTAssertEqual(box.observedMainThread, true)
|
|
}
|
|
|
|
func testRunMainThreadWorkTimeoutMarksAbandonedUntilDrained() {
|
|
final class ResultBox {
|
|
var error: Error?
|
|
var abandonedCount: Int?
|
|
var abandonedSinceSet: Bool?
|
|
var drainedCount: Int?
|
|
var drainedSinceCleared: Bool?
|
|
}
|
|
let box = ResultBox()
|
|
let releaseWork = DispatchSemaphore(value: 0)
|
|
let observedAbandoned = DispatchSemaphore(value: 0)
|
|
let finished = expectation(description: "off-main caller timed out")
|
|
let drained = expectation(description: "abandoned main work drained")
|
|
|
|
DispatchQueue(label: "agent-device.runner.tests.timeout").async {
|
|
do {
|
|
_ = try self.runMainThreadWork(
|
|
timeout: 0,
|
|
timeoutError: self.mainThreadExecutionTimeoutError,
|
|
onAbandoned: {
|
|
box.abandonedCount = self.abandonedMainThreadWorkCount
|
|
box.abandonedSinceSet = self.abandonedMainThreadWorkSince != nil
|
|
observedAbandoned.signal()
|
|
},
|
|
onDrained: {
|
|
self.mainThreadWorkLock.lock()
|
|
box.drainedCount = self.abandonedMainThreadWorkCount
|
|
box.drainedSinceCleared = self.abandonedMainThreadWorkSince == nil
|
|
self.mainThreadWorkLock.unlock()
|
|
drained.fulfill()
|
|
}
|
|
) {
|
|
_ = releaseWork.wait(timeout: .now() + 1)
|
|
return true
|
|
}
|
|
} catch {
|
|
box.error = error
|
|
}
|
|
finished.fulfill()
|
|
}
|
|
|
|
DispatchQueue(label: "agent-device.runner.tests.release-timeout").async {
|
|
_ = observedAbandoned.wait(timeout: .now() + 1)
|
|
releaseWork.signal()
|
|
}
|
|
|
|
wait(for: [finished, drained], timeout: 2)
|
|
XCTAssertEqual((box.error as NSError?)?.code, RunnerErrorCode.mainThreadExecutionTimedOut)
|
|
XCTAssertEqual(box.abandonedCount, 1)
|
|
XCTAssertEqual(box.abandonedSinceSet, true)
|
|
XCTAssertEqual(box.drainedCount, 0)
|
|
XCTAssertEqual(box.drainedSinceCleared, true)
|
|
}
|
|
|
|
func testPostSnapshotDelayMarkDoesNotQueueBehindAbandonedTreeCapture() {
|
|
abandonedTreeCaptureCount = 1
|
|
defer {
|
|
abandonedTreeCaptureCount = 0
|
|
needsPostSnapshotInteractionDelay = false
|
|
}
|
|
|
|
let finished = expectation(description: "off-main caller finished")
|
|
DispatchQueue(label: "agent-device.runner.tests.post-snapshot-delay").async {
|
|
self.setNeedsPostSnapshotInteractionDelay()
|
|
finished.fulfill()
|
|
}
|
|
|
|
wait(for: [finished], timeout: 1)
|
|
mainThreadWorkLock.lock()
|
|
let abandonedWorkCount = abandonedMainThreadWorkCount
|
|
mainThreadWorkLock.unlock()
|
|
XCTAssertEqual(abandonedWorkCount, 0)
|
|
}
|
|
#endif
|
|
|
|
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
|
|
func execute(command: Command) throws -> Response {
|
|
if command.command == .status {
|
|
return executeStatus(command: command)
|
|
}
|
|
if command.command == .uptime {
|
|
return executeUptime()
|
|
}
|
|
commandJournal.accept(command: command)
|
|
return try executeAccepted(command: command)
|
|
}
|
|
#endif
|
|
|
|
func executeAccepted(command: Command) throws -> Response {
|
|
commandJournal.start(command: command)
|
|
do {
|
|
let response = try executeDispatched(command: command)
|
|
commandJournal.finish(command: command, response: response)
|
|
return response
|
|
} catch {
|
|
commandJournal.fail(command: command, error: error)
|
|
throw error
|
|
}
|
|
}
|
|
|
|
func executeStatus(command: Command) -> Response {
|
|
guard
|
|
let statusCommandId = command.statusCommandId?.trimmedNonEmpty
|
|
else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "INVALID_ARGS",
|
|
message: "status requires statusCommandId",
|
|
hint: "Set statusCommandId to the commandId of the runner command to inspect."
|
|
)
|
|
)
|
|
}
|
|
return Response(ok: true, data: commandJournal.status(normalizedCommandId: statusCommandId))
|
|
}
|
|
|
|
func executeUptime() -> Response {
|
|
// Placeholder value: the transport layer (jsonResponse) overwrites currentUptimeMs with a
|
|
// fresher send-time stamp on every ok response; kept so direct callers still get a value.
|
|
Response(
|
|
ok: true,
|
|
data: DataPayload(currentUptimeMs: currentUptimeMs())
|
|
)
|
|
}
|
|
|
|
/// Tracks one main-queue dispatch so the watchdog and the dispatched block can agree —
|
|
/// under `mainThreadWorkLock` — on exactly one of: finished in time, or abandoned.
|
|
private final class MainThreadWorkState {
|
|
var finished = false
|
|
var abandoned = false
|
|
}
|
|
|
|
struct ActiveCommandContext {
|
|
let app: XCUIApplication
|
|
}
|
|
|
|
enum ActiveCommandPreparation {
|
|
case response(Response)
|
|
case context(ActiveCommandContext)
|
|
}
|
|
|
|
enum MainThreadBusyState {
|
|
case idle
|
|
case busy(abandonedForSeconds: TimeInterval)
|
|
case wedged(abandonedForSeconds: TimeInterval)
|
|
}
|
|
|
|
func currentMainThreadBusyState() -> MainThreadBusyState {
|
|
mainThreadWorkLock.lock()
|
|
defer { mainThreadWorkLock.unlock() }
|
|
guard abandonedMainThreadWorkCount > 0 else { return .idle }
|
|
let abandonedFor = abandonedMainThreadWorkSince.map { Date().timeIntervalSince($0) } ?? 0
|
|
if abandonedFor > mainThreadWedgeThreshold {
|
|
return .wedged(abandonedForSeconds: abandonedFor)
|
|
}
|
|
return .busy(abandonedForSeconds: abandonedFor)
|
|
}
|
|
|
|
private func runnerBusyResponse(command: Command, abandonedForSeconds: TimeInterval) -> Response {
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_BUSY command=%@ commandId=%@ abandonedForSeconds=%.1f",
|
|
command.command.rawValue,
|
|
command.commandId ?? "",
|
|
abandonedForSeconds
|
|
)
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "RUNNER_BUSY",
|
|
message:
|
|
"The iOS runner is still finishing a previous command that exceeded its execution watchdog (usually an accessibility capture on a heavy or animating screen).",
|
|
hint:
|
|
"Wait a few seconds and retry. If snapshots keep failing on this screen, use screenshot as visual truth and interact by coordinates, or navigate to another screen."
|
|
)
|
|
)
|
|
}
|
|
|
|
private func runnerWedgedResponse(command: Command, abandonedForSeconds: TimeInterval) -> Response {
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_WEDGED command=%@ commandId=%@ abandonedForSeconds=%.1f",
|
|
command.command.rawValue,
|
|
command.commandId ?? "",
|
|
abandonedForSeconds
|
|
)
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "RUNNER_WEDGED",
|
|
message:
|
|
"The iOS runner main thread has been stuck in abandoned work for \(Int(abandonedForSeconds)) seconds and cannot recover on its own.",
|
|
hint:
|
|
"The runner session will be restarted. Retry the command after the restart; if this screen keeps wedging captures, use screenshot as visual truth and interact by coordinates."
|
|
)
|
|
)
|
|
}
|
|
|
|
private func runnerUnavailableResponse(command: Command) -> Response? {
|
|
switch currentMainThreadBusyState() {
|
|
case .idle:
|
|
return nil
|
|
case .busy(let abandonedForSeconds):
|
|
return runnerBusyResponse(command: command, abandonedForSeconds: abandonedForSeconds)
|
|
case .wedged(let abandonedForSeconds):
|
|
return runnerWedgedResponse(command: command, abandonedForSeconds: abandonedForSeconds)
|
|
}
|
|
}
|
|
|
|
private func executeDispatched(command: Command) throws -> Response {
|
|
// XCTest work cannot be cancelled mid-flight: once the watchdog abandons a main-queue
|
|
// block, queueing more main-thread commands behind it only buries the runner deeper.
|
|
// Refuse fast instead so the daemon backs off while the abandoned work drains; past the
|
|
// wedge threshold, escalate so the daemon recycles this runner (#1105).
|
|
if let unavailable = runnerUnavailableResponse(command: command) {
|
|
return unavailable
|
|
}
|
|
let alertDeadline = command.command == .alert
|
|
? Date().addingTimeInterval(Self.alertCommandTimeout(timeoutMs: command.timeoutMs))
|
|
: nil
|
|
if Thread.isMainThread {
|
|
let routeToSpringboard = shouldRouteToSpringboardBlockingSystemModal(
|
|
command,
|
|
deadline: alertDeadline
|
|
)
|
|
return try executeOnMainSafely(
|
|
command: command,
|
|
alertDeadline: alertDeadline,
|
|
routeToSpringboard: routeToSpringboard
|
|
)
|
|
}
|
|
// Resolve this before the command's outer main-thread block. If the bounded probe abandons
|
|
// slow XCTest enumeration, return the established recoverable response instead of queueing
|
|
// command preparation behind work that may outlive the 30-second command watchdog.
|
|
let routeToSpringboard = shouldRouteToSpringboardBlockingSystemModal(
|
|
command,
|
|
deadline: alertDeadline
|
|
)
|
|
if let unavailable = runnerUnavailableResponse(command: command) {
|
|
return unavailable
|
|
}
|
|
if command.command == .snapshot {
|
|
return try executeSnapshotDispatched(command: command)
|
|
}
|
|
if command.command == .alert, let deadline = alertDeadline {
|
|
return try runMainThreadWork(
|
|
timeout: max(0.001, deadline.timeIntervalSinceNow),
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
try self.executeOnMainSafely(
|
|
command: command,
|
|
alertDeadline: deadline,
|
|
routeToSpringboard: routeToSpringboard
|
|
)
|
|
}
|
|
}
|
|
return try runMainThreadWork(
|
|
timeout: mainThreadExecutionTimeout,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
try self.executeOnMainSafely(command: command, routeToSpringboard: routeToSpringboard)
|
|
}
|
|
}
|
|
|
|
func runMainThreadWork<T>(
|
|
timeout: TimeInterval,
|
|
timeoutError: @escaping () -> Error,
|
|
onAbandoned: (() -> Void)? = nil,
|
|
onDrained: (() -> Void)? = nil,
|
|
_ work: @escaping () throws -> T
|
|
) throws -> T {
|
|
if Thread.isMainThread {
|
|
return try work()
|
|
}
|
|
var result: Result<T, Error>?
|
|
let semaphore = DispatchSemaphore(value: 0)
|
|
let workState = MainThreadWorkState()
|
|
DispatchQueue.main.async {
|
|
do {
|
|
result = .success(try work())
|
|
} catch {
|
|
result = .failure(error)
|
|
}
|
|
self.mainThreadWorkLock.lock()
|
|
if workState.abandoned {
|
|
self.abandonedMainThreadWorkCount -= 1
|
|
if self.abandonedMainThreadWorkCount == 0 {
|
|
self.abandonedMainThreadWorkSince = nil
|
|
NSLog("AGENT_DEVICE_RUNNER_ABANDONED_WORK_DRAINED")
|
|
}
|
|
self.mainThreadWorkLock.unlock()
|
|
onDrained?()
|
|
} else {
|
|
workState.finished = true
|
|
self.mainThreadWorkLock.unlock()
|
|
}
|
|
semaphore.signal()
|
|
}
|
|
let waitResult = semaphore.wait(timeout: .now() + timeout)
|
|
if waitResult == .timedOut {
|
|
mainThreadWorkLock.lock()
|
|
let stillRunning = !workState.finished
|
|
if stillRunning {
|
|
workState.abandoned = true
|
|
abandonedMainThreadWorkCount += 1
|
|
if abandonedMainThreadWorkSince == nil {
|
|
abandonedMainThreadWorkSince = Date()
|
|
}
|
|
onAbandoned?()
|
|
}
|
|
mainThreadWorkLock.unlock()
|
|
throw timeoutError()
|
|
}
|
|
switch result {
|
|
case .success(let value):
|
|
return value
|
|
case .failure(let error):
|
|
throw error
|
|
case .none:
|
|
throw NSError(
|
|
domain: RunnerErrorDomain.general,
|
|
code: RunnerErrorCode.noResponseFromMainThread,
|
|
userInfo: [NSLocalizedDescriptionKey: "no response from main thread"]
|
|
)
|
|
}
|
|
}
|
|
|
|
private func mainThreadExecutionTimeoutError() -> Error {
|
|
NSError(
|
|
domain: RunnerErrorDomain.general,
|
|
code: RunnerErrorCode.mainThreadExecutionTimedOut,
|
|
userInfo: [NSLocalizedDescriptionKey: "main thread execution timed out"]
|
|
)
|
|
}
|
|
|
|
// MARK: - Command Handling
|
|
|
|
private func executeOnMainSafely(
|
|
command: Command,
|
|
alertDeadline: Date? = nil,
|
|
routeToSpringboard: Bool
|
|
) throws -> Response {
|
|
var hasRetried = false
|
|
while true {
|
|
var response: Response?
|
|
var swiftError: Error?
|
|
let failureCountBefore = currentXCTestFailureCount()
|
|
let exceptionMessage = RunnerObjCExceptionCatcher.catchException({
|
|
do {
|
|
response = try self.executeOnMain(
|
|
command: command,
|
|
alertDeadline: alertDeadline,
|
|
routeToSpringboard: routeToSpringboard
|
|
)
|
|
} catch {
|
|
swiftError = error
|
|
}
|
|
})
|
|
|
|
if let exceptionMessage {
|
|
invalidateCachedTarget(reason: "objc_exception")
|
|
if !hasRetried, shouldRetryException(command, message: exceptionMessage) {
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_RETRY command=%@ reason=objc_exception",
|
|
command.command.rawValue
|
|
)
|
|
hasRetried = true
|
|
sleepFor(retryCooldown)
|
|
continue
|
|
}
|
|
throw NSError(
|
|
domain: RunnerErrorDomain.exception,
|
|
code: RunnerErrorCode.objcException,
|
|
userInfo: [NSLocalizedDescriptionKey: exceptionMessage]
|
|
)
|
|
}
|
|
if let swiftError {
|
|
throw swiftError
|
|
}
|
|
guard let response else {
|
|
throw NSError(
|
|
domain: RunnerErrorDomain.general,
|
|
code: RunnerErrorCode.commandReturnedNoResponse,
|
|
userInfo: [NSLocalizedDescriptionKey: "command returned no response"]
|
|
)
|
|
}
|
|
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
|
|
// #1605 merge gate: the REAL gesture already executed above; recording a
|
|
// production-shaped issue here makes the per-command failure-count
|
|
// conversion below fire exactly as in the field (bsky-24: activation
|
|
// lands, bookkeeping records a failure). Compiled out of production.
|
|
if consumeInjectedTapRecordedFailureForTesting(command: command.command) {
|
|
record(
|
|
XCTIssue(
|
|
type: .assertionFailure,
|
|
compactDescription: "Injected tap recorded-failure (#1605 corroboration merge gate)"
|
|
)
|
|
)
|
|
}
|
|
#endif
|
|
if didRecordXCTestFailure(since: failureCountBefore),
|
|
let failureResponse = xctestRecordedFailureResponse(command: command, response: response)
|
|
{
|
|
invalidateCachedTarget(reason: "xctest_recorded_failure")
|
|
return failureResponse
|
|
}
|
|
if !hasRetried, shouldRetryCommand(command), shouldRetryResponse(response) {
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_RETRY command=%@ reason=response_unavailable",
|
|
command.command.rawValue
|
|
)
|
|
hasRetried = true
|
|
invalidateCachedTarget(reason: "response_unavailable")
|
|
sleepFor(retryCooldown)
|
|
continue
|
|
}
|
|
return response
|
|
}
|
|
}
|
|
|
|
private func executeSnapshotDispatched(command: Command) throws -> Response {
|
|
try executeDispatchedWithRecovery(command: command) {
|
|
try self.executeSnapshotDispatchedOnce(command: command)
|
|
}
|
|
}
|
|
|
|
/// The dispatched snapshot recovery loop: read-only retry + XCTest-recorded-failure invalidation,
|
|
/// matching what `executeOnMainSafely` gives the generic path. `perform` runs the capture and its
|
|
/// own bounded main-thread work.
|
|
func executeDispatchedWithRecovery(
|
|
command: Command,
|
|
perform: () throws -> Response
|
|
) throws -> Response {
|
|
var hasRetried = false
|
|
while true {
|
|
let failureCountBefore = try runMainThreadWork(
|
|
timeout: mainThreadExecutionTimeout,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
self.currentXCTestFailureCount()
|
|
}
|
|
let response = try perform()
|
|
// Recovered independently — re-entering main for bookkeeping would queue behind the still-
|
|
// abandoned XCTest query and re-stall the command (#1244), so skip it until that work drains.
|
|
if hasAbandonedTreeCapture() {
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_DISPATCH_RECOVERY_SKIPPED_XCTEST_OCCUPIED command=%@",
|
|
command.command.rawValue
|
|
)
|
|
return response
|
|
}
|
|
let recordedFailureResponse = try runMainThreadWork(
|
|
timeout: mainThreadExecutionTimeout,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
self.didRecordXCTestFailure(since: failureCountBefore)
|
|
? self.xctestRecordedFailureResponse(command: command, response: response)
|
|
: nil
|
|
}
|
|
if let recordedFailureResponse {
|
|
try runMainThreadWork(
|
|
timeout: mainThreadExecutionTimeout,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
self.invalidateCachedTarget(reason: "xctest_recorded_failure")
|
|
}
|
|
return recordedFailureResponse
|
|
}
|
|
if !hasRetried, shouldRetryCommand(command), shouldRetryResponse(response) {
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_RETRY command=%@ reason=response_unavailable",
|
|
command.command.rawValue
|
|
)
|
|
hasRetried = true
|
|
try runMainThreadWork(
|
|
timeout: mainThreadExecutionTimeout,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
self.invalidateCachedTarget(reason: "response_unavailable")
|
|
self.sleepFor(self.retryCooldown)
|
|
}
|
|
continue
|
|
}
|
|
return response
|
|
}
|
|
}
|
|
|
|
private func executeSnapshotDispatchedOnce(command: Command) throws -> Response {
|
|
let preparation = try runMainThreadWork(
|
|
timeout: mainThreadExecutionTimeout,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
try self.prepareActiveCommandContextSafely(command: command, routeToSpringboard: false)
|
|
}
|
|
switch preparation {
|
|
case .response(let response):
|
|
return response
|
|
case .context(let context):
|
|
return try executeSnapshotPrepared(command: command, activeApp: context.app)
|
|
}
|
|
}
|
|
|
|
/// Pure command→options projection, extracted so the runner unit bundle can
|
|
/// prove the decoded wire field actually reaches presentation options (#1634 P2).
|
|
static func presentationOptions(from command: Command) -> PresentationOptions {
|
|
let customActions = command.customActions ?? false
|
|
return PresentationOptions(
|
|
interactiveOnly: command.interactiveOnly ?? false,
|
|
depth: command.depth,
|
|
scope: command.scope,
|
|
raw: command.raw ?? false,
|
|
// Custom actions are only readable through the private AX client, so
|
|
// asking for them pins that backend rather than silently returning a
|
|
// capture that structurally cannot carry them. An explicit pin wins.
|
|
preferredBackend: command.preferredBackend
|
|
?? (customActions ? SnapshotBackendKind.privateAX.rawValue : nil),
|
|
customActions: customActions
|
|
)
|
|
}
|
|
|
|
private func executeSnapshotPrepared(command: Command, activeApp: XCUIApplication) throws -> Response {
|
|
let options = Self.presentationOptions(from: command)
|
|
do {
|
|
let payload: DataPayload
|
|
if options.raw {
|
|
payload = try snapshotRaw(app: activeApp, options: options)
|
|
} else {
|
|
payload = try snapshotFast(app: activeApp, options: options)
|
|
}
|
|
setNeedsPostSnapshotInteractionDelay()
|
|
return Response(ok: true, data: payload)
|
|
} catch let failure as SnapshotCaptureFailure {
|
|
invalidateCachedTargetAfterSnapshotFailure()
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: failure.code,
|
|
message: failure.message,
|
|
hint: failure.hint
|
|
)
|
|
)
|
|
}
|
|
}
|
|
|
|
private func setNeedsPostSnapshotInteractionDelay() {
|
|
if Thread.isMainThread {
|
|
needsPostSnapshotInteractionDelay = true
|
|
return
|
|
}
|
|
guard !hasAbandonedTreeCapture() else {
|
|
NSLog("AGENT_DEVICE_RUNNER_POST_SNAPSHOT_DELAY_MARK_SKIPPED_XCTEST_OCCUPIED")
|
|
return
|
|
}
|
|
do {
|
|
try runMainThreadWork(
|
|
timeout: 1,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
self.needsPostSnapshotInteractionDelay = true
|
|
}
|
|
} catch {
|
|
NSLog("AGENT_DEVICE_RUNNER_POST_SNAPSHOT_DELAY_MARK_FAILED=%@", String(describing: error))
|
|
}
|
|
}
|
|
|
|
private func invalidateCachedTargetAfterSnapshotFailure() {
|
|
if Thread.isMainThread {
|
|
invalidateCachedTarget(reason: "ax_snapshot_failure")
|
|
return
|
|
}
|
|
do {
|
|
try runMainThreadWork(
|
|
timeout: 1,
|
|
timeoutError: mainThreadExecutionTimeoutError
|
|
) {
|
|
self.invalidateCachedTarget(reason: "ax_snapshot_failure")
|
|
}
|
|
} catch {
|
|
NSLog("AGENT_DEVICE_RUNNER_SNAPSHOT_INVALIDATION_FAILED=%@", String(describing: error))
|
|
}
|
|
}
|
|
|
|
private func prepareActiveCommandContextSafely(
|
|
command: Command,
|
|
routeToSpringboard: Bool
|
|
) throws -> ActiveCommandPreparation {
|
|
var preparation: ActiveCommandPreparation?
|
|
let exceptionMessage = RunnerObjCExceptionCatcher.catchException({
|
|
preparation = self.prepareActiveCommandContext(
|
|
command: command,
|
|
routeToSpringboard: routeToSpringboard
|
|
)
|
|
})
|
|
if let exceptionMessage {
|
|
throw NSError(
|
|
domain: RunnerErrorDomain.exception,
|
|
code: RunnerErrorCode.objcException,
|
|
userInfo: [NSLocalizedDescriptionKey: exceptionMessage]
|
|
)
|
|
}
|
|
guard let preparation else {
|
|
throw NSError(
|
|
domain: RunnerErrorDomain.general,
|
|
code: RunnerErrorCode.commandReturnedNoResponse,
|
|
userInfo: [NSLocalizedDescriptionKey: "snapshot preflight returned no response"]
|
|
)
|
|
}
|
|
return preparation
|
|
}
|
|
|
|
private func executeOnMain(
|
|
command: Command,
|
|
alertDeadline: Date?,
|
|
routeToSpringboard: Bool
|
|
) throws -> Response {
|
|
let preparation = prepareActiveCommandContext(
|
|
command: command,
|
|
routeToSpringboard: routeToSpringboard
|
|
)
|
|
let activeApp: XCUIApplication
|
|
switch preparation {
|
|
case .response(let response):
|
|
return response
|
|
case .context(let context):
|
|
activeApp = context.app
|
|
}
|
|
|
|
switch command.command {
|
|
case .status:
|
|
return executeStatus(command: command)
|
|
case .targetReset:
|
|
return resetTargetAfterExternalRelaunch()
|
|
case .shutdown:
|
|
stopRecordingIfNeeded()
|
|
return Response(ok: true, data: DataPayload(message: "shutdown"))
|
|
case .recordStart:
|
|
guard
|
|
let requestedOutPath = command.outPath?.trimmingCharacters(in: .whitespacesAndNewlines),
|
|
!requestedOutPath.isEmpty
|
|
else {
|
|
return Response(ok: false, error: ErrorPayload(message: "recordStart requires outPath"))
|
|
}
|
|
let hasAppBundleId = !(command.appBundleId?
|
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
.isEmpty ?? true)
|
|
guard hasAppBundleId else {
|
|
return Response(ok: false, error: ErrorPayload(message: "recordStart requires appBundleId"))
|
|
}
|
|
if activeRecording != nil {
|
|
return Response(ok: false, error: ErrorPayload(message: "recording already in progress"))
|
|
}
|
|
if let requestedFps = command.fps, (requestedFps < minRecordingFps || requestedFps > maxRecordingFps) {
|
|
return Response(ok: false, error: ErrorPayload(message: "recordStart fps must be between \(minRecordingFps) and \(maxRecordingFps)"))
|
|
}
|
|
do {
|
|
let resolvedOutPath = resolveRecordingOutPath(requestedOutPath)
|
|
let fpsLabel = command.fps.map(String.init) ?? String(RunnerTests.defaultRecordingFps)
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_RECORD_START requestedOutPath=%@ resolvedOutPath=%@ fps=%@",
|
|
requestedOutPath,
|
|
resolvedOutPath,
|
|
fpsLabel
|
|
)
|
|
let recorder = ScreenRecorder(
|
|
outputPath: resolvedOutPath,
|
|
fps: command.fps.map { Int32($0) }
|
|
)
|
|
try recorder.start { [weak self] in
|
|
return self?.captureRunnerFrame()
|
|
}
|
|
activeRecording = recorder
|
|
return Response(ok: true, data: DataPayload(message: "recording started"))
|
|
} catch {
|
|
activeRecording = nil
|
|
return Response(ok: false, error: ErrorPayload(message: "failed to start recording: \(error.localizedDescription)"))
|
|
}
|
|
case .recordStop:
|
|
guard let recorder = activeRecording else {
|
|
// The runner protocol is the durable cleanup primitive. A daemon may crash after the
|
|
// native stop succeeds but before it commits the resource transition, so exact-owner
|
|
// recovery must be able to repeat this command safely. Public `record stop` still owns
|
|
// its user-facing no-active validation through the daemon session manifest.
|
|
return Response(ok: true, data: DataPayload(message: "recording already stopped"))
|
|
}
|
|
do {
|
|
try recorder.stop()
|
|
activeRecording = nil
|
|
return Response(ok: true, data: DataPayload(message: "recording stopped"))
|
|
} catch {
|
|
activeRecording = nil
|
|
return Response(ok: false, error: ErrorPayload(message: "failed to stop recording: \(error.localizedDescription)"))
|
|
}
|
|
case .uptime:
|
|
return executeUptime()
|
|
case .activate:
|
|
guard
|
|
let bundleId = command.appBundleId?.trimmingCharacters(in: .whitespacesAndNewlines),
|
|
!bundleId.isEmpty
|
|
else {
|
|
return Response(ok: false, error: ErrorPayload(message: "activate requires appBundleId"))
|
|
}
|
|
// prepareActiveCommandContext already activated this bundle. Keep this case as the
|
|
// explicit acknowledgement after that preflight, not as a second activation.
|
|
return Response(ok: true, data: DataPayload(message: "app activated"))
|
|
case .terminate:
|
|
guard
|
|
let bundleId = command.appBundleId?.trimmingCharacters(in: .whitespacesAndNewlines),
|
|
!bundleId.isEmpty
|
|
else {
|
|
return Response(ok: false, error: ErrorPayload(message: "terminate requires appBundleId"))
|
|
}
|
|
XCUIApplication(bundleIdentifier: bundleId).terminate()
|
|
if currentBundleId == bundleId {
|
|
invalidateCachedTarget(reason: "target_terminated")
|
|
}
|
|
return Response(ok: true, data: DataPayload(message: "app terminated"))
|
|
default:
|
|
break
|
|
}
|
|
return try executeOnMainPrepared(
|
|
command: command,
|
|
activeApp: activeApp,
|
|
alertDeadline: alertDeadline
|
|
)
|
|
}
|
|
|
|
private func prepareActiveCommandContext(
|
|
command: Command,
|
|
routeToSpringboard: Bool = false
|
|
) -> ActiveCommandPreparation {
|
|
var activeApp = currentApp ?? app
|
|
if routeToSpringboard {
|
|
activeApp = springboard
|
|
} else if shouldSkipAppActivationPreflight(command) {
|
|
activeApp = resolveAppWithoutActivation(command: command)
|
|
} else if !isRunnerLifecycleCommand(command.command) {
|
|
let normalizedBundleId = command.appBundleId?
|
|
.trimmingCharacters(in: .whitespacesAndNewlines)
|
|
let requestedBundleId = (normalizedBundleId?.isEmpty == true) ? nil : normalizedBundleId
|
|
if let bundleId = requestedBundleId {
|
|
if currentBundleId != bundleId || currentApp == nil {
|
|
_ = activateTarget(bundleId: bundleId, reason: "bundle_changed")
|
|
} else {
|
|
refreshCachedTargetIfProcessChanged(bundleId: bundleId)
|
|
}
|
|
} else {
|
|
// Do not reuse stale bundle targets when the caller does not explicitly request one.
|
|
invalidateCachedTarget(reason: "missing_app_bundle")
|
|
}
|
|
|
|
activeApp = currentApp ?? app
|
|
if let bundleId = requestedBundleId, targetNeedsActivation(activeApp) {
|
|
activeApp = activateTarget(bundleId: bundleId, reason: "stale_target")
|
|
} else if requestedBundleId == nil, targetNeedsActivation(activeApp) {
|
|
ensureRunnerHostAppActive(reason: "missing_app_bundle")
|
|
activeApp = app
|
|
}
|
|
|
|
let skipExistenceWait = canUseFastForegroundAppGuard(
|
|
activeApp: activeApp,
|
|
requestedBundleId: requestedBundleId,
|
|
command: command.command
|
|
)
|
|
if !skipExistenceWait && !activeApp.waitForExistence(timeout: appExistenceTimeout) {
|
|
if let bundleId = requestedBundleId {
|
|
activeApp = activateTarget(bundleId: bundleId, reason: "missing_after_wait")
|
|
guard activeApp.waitForExistence(timeout: appExistenceTimeout) else {
|
|
return .response(Response(ok: false, error: ErrorPayload(message: "app '\(bundleId)' is not available")))
|
|
}
|
|
} else {
|
|
return .response(Response(ok: false, error: ErrorPayload(message: "runner app is not available")))
|
|
}
|
|
}
|
|
|
|
if isInteractionCommand(command.command) {
|
|
if let bundleId = requestedBundleId, activeApp.state != .runningForeground {
|
|
activeApp = activateTarget(bundleId: bundleId, reason: "interaction_foreground_guard")
|
|
} else if requestedBundleId == nil, activeApp.state != .runningForeground {
|
|
ensureRunnerHostAppActive(reason: "interaction_missing_app_bundle")
|
|
activeApp = app
|
|
}
|
|
let skipInteractionExistenceWait = canUseFastForegroundAppGuard(
|
|
activeApp: activeApp,
|
|
requestedBundleId: requestedBundleId,
|
|
command: command.command
|
|
)
|
|
if !skipInteractionExistenceWait && !activeApp.waitForExistence(timeout: 2) {
|
|
if let bundleId = requestedBundleId {
|
|
return .response(Response(ok: false, error: ErrorPayload(message: "app '\(bundleId)' is not available")))
|
|
}
|
|
return .response(Response(ok: false, error: ErrorPayload(message: "runner app is not available")))
|
|
}
|
|
applyInteractionStabilizationIfNeeded()
|
|
}
|
|
}
|
|
return .context(ActiveCommandContext(app: activeApp))
|
|
}
|
|
|
|
func executeOnMainPrepared(
|
|
command: Command,
|
|
activeApp: XCUIApplication,
|
|
alertDeadline: Date? = nil
|
|
) throws -> Response {
|
|
var activeApp = activeApp
|
|
if command.command != .tap && command.command != .type && !isReadOnlyCommand(command) {
|
|
clearRememberedTextEntryTap()
|
|
}
|
|
switch command.command {
|
|
case .status, .activate, .terminate, .targetReset, .shutdown, .recordStart, .recordStop, .uptime:
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "UNSUPPORTED_OPERATION",
|
|
message: "\(command.command.rawValue) cannot be executed through the prepared command path"
|
|
)
|
|
)
|
|
case .tap:
|
|
if let selectorKey = command.selectorKey, let selectorValue = command.selectorValue {
|
|
let expectedPoint: CGPoint?
|
|
if command.allowNonHittableCoordinateFallback == true,
|
|
let x = command.x,
|
|
let y = command.y
|
|
{
|
|
expectedPoint = CGPoint(x: x, y: y)
|
|
} else {
|
|
expectedPoint = nil
|
|
}
|
|
let match = findElement(
|
|
app: activeApp,
|
|
selectorKey: selectorKey,
|
|
selectorValue: selectorValue,
|
|
allowNonHittableFallback: command.allowNonHittableCoordinateFallback == true,
|
|
expectedPoint: expectedPoint
|
|
)
|
|
if match.isAmbiguous {
|
|
clearRememberedTextEntryTap()
|
|
return Response(ok: false, error: ErrorPayload(code: "AMBIGUOUS_MATCH", message: "selector matched multiple elements"))
|
|
}
|
|
if let element = match.element {
|
|
let frame = element.frame
|
|
// XCTest reports closed-drawer/off-viewport items as hittable, then
|
|
// "taps" coordinates outside the visible window as a silent no-op.
|
|
// Refuse instead; the daemon falls back to tree-based resolution,
|
|
// which can prefer an on-screen candidate or explain the off-screen
|
|
// state. The check uses the main window frame, not app.frame: on RN
|
|
// apps app.frame unions transformed subtrees (a closed drawer at
|
|
// negative x), so it happily "contains" unreachable coordinates.
|
|
// The DECISION lives in TapPointPolicy (golden parity table with the
|
|
// TS twin); onScreenWindowFrame only supplies the frame.
|
|
if !match.usedNonHittableFallback
|
|
&& !TapPointPolicy.isAllowed(
|
|
elementFrame: frame,
|
|
windowFrame: onScreenWindowFrame(app: activeApp)
|
|
) {
|
|
clearRememberedTextEntryTap()
|
|
return Response(ok: false, error: ErrorPayload(
|
|
code: "ELEMENT_OFFSCREEN",
|
|
message: "element resolved off-screen at (\(Int(frame.midX)), \(Int(frame.midY)))"))
|
|
}
|
|
let isTextEntry = isTextEntryElement(element)
|
|
let touchPoint = expectedPoint ?? CGPoint(x: frame.midX, y: frame.midY)
|
|
let touchFrame = resolvedTouchVisualizationFrame(
|
|
app: activeApp,
|
|
x: touchPoint.x,
|
|
y: touchPoint.y
|
|
)
|
|
var fallback: GestureFallback?
|
|
if command.synthesized == true {
|
|
let policyKind = SynthesizedGesturePolicyKind.coordinateTap
|
|
let context = synthesizedCoordinateContext(
|
|
app: activeApp,
|
|
policy: synthesizedGesturePolicy(policyKind)
|
|
)
|
|
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
|
|
synthesizedTapAt(
|
|
app: activeApp,
|
|
x: touchPoint.x,
|
|
y: touchPoint.y,
|
|
context: context
|
|
)
|
|
}
|
|
if case .performed = outcome {
|
|
logSynthesizedGesturePolicyDecision(
|
|
kind: policyKind,
|
|
context: context,
|
|
fallbackAttempted: false
|
|
)
|
|
if isTextEntry {
|
|
waitForTextEntryReadinessAfterTap(app: activeApp, element: element)
|
|
}
|
|
rememberTextEntryTap(isTextEntry ? element : nil)
|
|
return gestureResponse(
|
|
message: match.usedNonHittableFallback
|
|
? "tapped via non-hittable coordinate fallback"
|
|
: "tapped",
|
|
timing: timing,
|
|
frame: .touch(touchFrame),
|
|
maestroNonHittableCoordinateFallbackUsed:
|
|
command.allowNonHittableCoordinateFallback == true
|
|
? match.usedNonHittableFallback
|
|
: nil
|
|
)
|
|
}
|
|
logSynthesizedGesturePolicyDecision(
|
|
kind: policyKind,
|
|
context: context,
|
|
fallbackAttempted: true
|
|
)
|
|
fallback = gestureFallback(strategy: "xctest-coordinate-tap", from: outcome)
|
|
}
|
|
let (timing, outcome) = performGesture(activeApp) {
|
|
if expectedPoint != nil || match.usedNonHittableFallback {
|
|
// Maestro compatibility: RN E2E backdoor controls can be 1x1 and
|
|
// reported non-hittable by XCTest. Snapshot-resolved targets also
|
|
// need coordinate delivery because XCUIElement.activate() does not
|
|
// invoke every React Native accessibility wrapper.
|
|
return tapAt(app: activeApp, x: touchPoint.x, y: touchPoint.y)
|
|
}
|
|
return activateElement(app: activeApp, element: element, action: "tap by selector")
|
|
}
|
|
if let response = unsupportedResponse(for: outcome) {
|
|
clearRememberedTextEntryTap()
|
|
return response
|
|
}
|
|
if isTextEntry {
|
|
waitForTextEntryReadinessAfterTap(app: activeApp, element: element)
|
|
}
|
|
rememberTextEntryTap(isTextEntry ? element : nil)
|
|
return gestureResponse(
|
|
message: match.usedNonHittableFallback ? "tapped via non-hittable coordinate fallback" : "tapped",
|
|
timing: timing,
|
|
frame: .touch(touchFrame),
|
|
fallback: fallback,
|
|
maestroNonHittableCoordinateFallbackUsed:
|
|
command.allowNonHittableCoordinateFallback == true
|
|
? match.usedNonHittableFallback
|
|
: nil
|
|
)
|
|
}
|
|
clearRememberedTextEntryTap()
|
|
return Response(ok: false, error: ErrorPayload(code: "ELEMENT_NOT_FOUND", message: "element not found"))
|
|
}
|
|
if let x = command.x, let y = command.y {
|
|
let xCTestChannelPenalized = isSnapshotXCTestChannelPenalized(
|
|
bundleId: currentBundleId
|
|
)
|
|
let xCTestTextInputProbeSkipped = !shouldProbeCoordinateTapTextInput(
|
|
xCTestChannelPenalized: xCTestChannelPenalized
|
|
)
|
|
let textInput: XCUIElement?
|
|
if !xCTestTextInputProbeSkipped {
|
|
textInput = textInputAt(app: activeApp, x: x, y: y)
|
|
} else {
|
|
// A process-scoped tap cannot authorize later typing without concrete element identity.
|
|
textInput = nil
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_COORDINATE_TAP_TEXT_INPUT_PROBE_SKIPPED bundle=%@",
|
|
currentBundleId ?? ""
|
|
)
|
|
}
|
|
var fallback: GestureFallback?
|
|
if command.synthesized == true {
|
|
let policyKind = SynthesizedGesturePolicyKind.coordinateTap
|
|
let context = synthesizedCoordinateContext(
|
|
app: activeApp,
|
|
policy: synthesizedGesturePolicy(policyKind)
|
|
)
|
|
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
|
|
synthesizedTapAt(app: activeApp, x: x, y: y, context: context)
|
|
}
|
|
if case .performed = outcome {
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: false)
|
|
rememberTextEntryTap(textInput)
|
|
return gestureResponse(message: "tapped", timing: timing)
|
|
}
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: true)
|
|
fallback = gestureFallback(strategy: "xctest-coordinate-tap", from: outcome)
|
|
}
|
|
let touchFrame = resolvedTouchVisualizationFrame(app: activeApp, x: x, y: y)
|
|
let (timing, outcome) = performGesture(activeApp) { tapAt(app: activeApp, x: x, y: y) }
|
|
if let response = unsupportedResponse(for: outcome) {
|
|
clearRememberedTextEntryTap()
|
|
return response
|
|
}
|
|
rememberTextEntryTap(textInput)
|
|
return gestureResponse(
|
|
message: "tapped",
|
|
timing: timing,
|
|
frame: .touch(touchFrame),
|
|
fallback: fallback
|
|
)
|
|
}
|
|
clearRememberedTextEntryTap()
|
|
return Response(ok: false, error: ErrorPayload(message: "tap requires a selector or x/y"))
|
|
case .mouseClick:
|
|
guard let x = command.x, let y = command.y else {
|
|
return Response(ok: false, error: ErrorPayload(message: "mouseClick requires x and y"))
|
|
}
|
|
let touchFrame = resolvedTouchVisualizationFrame(app: activeApp, x: x, y: y)
|
|
do {
|
|
// mouseClick throws (it has no RunnerInteractionOutcome), so it keeps raw measureGesture
|
|
// and only routes the success payload through gestureResponse.
|
|
var clickError: Error?
|
|
let timing = measureGesture {
|
|
do {
|
|
try mouseClickAt(app: activeApp, x: x, y: y, button: command.button ?? "primary")
|
|
} catch {
|
|
clickError = error
|
|
}
|
|
}
|
|
if let clickError {
|
|
throw clickError
|
|
}
|
|
return gestureResponse(message: "clicked", timing: timing, frame: .touch(touchFrame))
|
|
} catch {
|
|
return Response(ok: false, error: ErrorPayload(message: error.localizedDescription))
|
|
}
|
|
case .longPress:
|
|
guard let x = command.x, let y = command.y else {
|
|
return Response(ok: false, error: ErrorPayload(message: "longPress requires x and y"))
|
|
}
|
|
let duration = (command.durationMs ?? 800) / 1000.0
|
|
let touchFrame = resolvedTouchVisualizationFrame(app: activeApp, x: x, y: y)
|
|
let (timing, outcome) = performGesture(activeApp) {
|
|
longPressAt(app: activeApp, x: x, y: y, duration: duration)
|
|
}
|
|
if let response = unsupportedResponse(for: outcome) {
|
|
return response
|
|
}
|
|
return gestureResponse(message: "long pressed", timing: timing, frame: .touch(touchFrame))
|
|
case .drag:
|
|
guard let x = command.x, let y = command.y, let x2 = command.x2, let y2 = command.y2 else {
|
|
return Response(ok: false, error: ErrorPayload(message: "drag requires x, y, x2, and y2"))
|
|
}
|
|
let defaults = runnerDragCommandDefaults(command)
|
|
return executeDragGesture(
|
|
activeApp: activeApp,
|
|
x: x,
|
|
y: y,
|
|
x2: x2,
|
|
y2: y2,
|
|
durationMs: defaults.durationMs,
|
|
synthesized: command.synthesized == true,
|
|
message: "dragged",
|
|
synthesizedPolicyKind: .synthesizedDrag
|
|
)
|
|
case .scroll:
|
|
// Fused frame-resolve + drag scroll for non-tvOS. On iOS this intentionally stays on the
|
|
// AX-free synthesized coordinate lane so scroll keeps working when XCTest cannot serialize
|
|
// the accessibility tree.
|
|
guard let rawDirection = command.direction,
|
|
let direction = RunnerScrollDirection(rawValue: rawDirection)
|
|
else {
|
|
return invalidScrollDirectionResponse(commandName: "scroll")
|
|
}
|
|
let scrollPolicyKind = SynthesizedGesturePolicyKind.scroll
|
|
guard let scrollContext = synthesizedCoordinateContext(
|
|
app: activeApp,
|
|
policy: synthesizedGesturePolicy(scrollPolicyKind)
|
|
) else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(message: "scroll could not resolve a usable interaction frame")
|
|
)
|
|
}
|
|
let frame = scrollReferenceFrame(app: activeApp, context: scrollContext)
|
|
guard frame.width > 0, frame.height > 0 else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(message: "scroll could not resolve a usable interaction frame")
|
|
)
|
|
}
|
|
let defaults = runnerDragCommandDefaults(command)
|
|
guard let plan = runnerScrollGesturePlan(
|
|
direction: direction,
|
|
amount: defaults.scrollAmount,
|
|
pixels: command.pixels,
|
|
referenceWidth: frame.width,
|
|
referenceHeight: frame.height
|
|
) else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "INVALID_ARGS",
|
|
message: "scroll could not compute a gesture plan"
|
|
)
|
|
)
|
|
}
|
|
guard scrollDurationIsValid(command.durationMs) else {
|
|
return invalidScrollDurationResponse(commandName: "scroll")
|
|
}
|
|
return executeScrollDragGesture(
|
|
activeApp: activeApp,
|
|
x: frame.minX + plan.x1,
|
|
y: frame.minY + plan.y1,
|
|
x2: frame.minX + plan.x2,
|
|
y2: frame.minY + plan.y2,
|
|
durationMs: defaults.durationMs,
|
|
message: "scrolled",
|
|
context: scrollContext.withReferenceFrame(frame),
|
|
releaseBehavior: command.scrollReleaseBehavior
|
|
)
|
|
case .desktopScroll:
|
|
guard let rawDirection = command.direction,
|
|
let direction = RunnerScrollDirection(rawValue: rawDirection)
|
|
else {
|
|
return invalidScrollDirectionResponse(commandName: "desktopScroll")
|
|
}
|
|
let appFrame = activeApp.frame
|
|
let frame = resolvedTouchReferenceFrame(app: activeApp, appFrame: appFrame)
|
|
guard frame.width > 0, frame.height > 0 else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(message: "desktopScroll could not resolve a usable interaction frame")
|
|
)
|
|
}
|
|
guard let plan = runnerScrollGesturePlan(
|
|
direction: direction,
|
|
amount: command.amount,
|
|
pixels: command.pixels,
|
|
referenceWidth: frame.width,
|
|
referenceHeight: frame.height
|
|
) else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "INVALID_ARGS",
|
|
message: "desktopScroll could not compute a wheel plan"
|
|
)
|
|
)
|
|
}
|
|
let x = frame.midX
|
|
let y = frame.midY
|
|
let localX = x - (appFrame.isEmpty ? frame.minX : appFrame.minX)
|
|
let localY = y - (appFrame.isEmpty ? frame.minY : appFrame.minY)
|
|
guard scrollDurationIsValid(command.durationMs) else {
|
|
return invalidScrollDurationResponse(commandName: "desktopScroll")
|
|
}
|
|
let touchFrame = resolvedTouchVisualizationFrame(
|
|
app: activeApp,
|
|
x: localX,
|
|
y: localY
|
|
)
|
|
do {
|
|
var scrollError: Error?
|
|
let timing = measureGesture {
|
|
do {
|
|
try desktopScrollAt(
|
|
app: activeApp,
|
|
x: x,
|
|
y: y,
|
|
direction: direction,
|
|
pixels: plan.travelPixels,
|
|
durationMs: command.durationMs
|
|
)
|
|
} catch {
|
|
scrollError = error
|
|
}
|
|
}
|
|
if let scrollError {
|
|
throw scrollError
|
|
}
|
|
return gestureResponse(message: "scrolled", timing: timing, frame: .touch(touchFrame))
|
|
} catch {
|
|
return Response(ok: false, error: ErrorPayload(message: error.localizedDescription))
|
|
}
|
|
case .remotePress:
|
|
guard let button = tvRemoteButton(from: command.remoteButton) else {
|
|
return Response(ok: false, error: ErrorPayload(message: "remotePress requires remoteButton"))
|
|
}
|
|
let duration = (command.durationMs ?? 0) / 1000.0
|
|
guard pressTvRemote(button, duration: duration) else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: "UNSUPPORTED_OPERATION", message: "remotePress is only supported on tvOS")
|
|
)
|
|
}
|
|
return Response(ok: true, data: DataPayload(message: "remote pressed"))
|
|
case .type:
|
|
var response: Response?
|
|
withTemporaryScrollIdleTimeoutIfSupported(activeApp) {
|
|
response = executeTypeCommand(activeApp: activeApp, command: command)
|
|
}
|
|
return response ?? Response(ok: false, error: ErrorPayload(message: "type produced no response"))
|
|
case .swipe:
|
|
guard let direction = command.direction else {
|
|
return Response(ok: false, error: ErrorPayload(message: "swipe requires direction"))
|
|
}
|
|
// swipe returns an optional frame (tvOS-only) rather than a RunnerInteractionOutcome, so it
|
|
// keeps raw measureGesture and only routes the success payload through gestureResponse.
|
|
var executedFrame: DragVisualizationFrame?
|
|
let timing = measureGesture {
|
|
withTemporaryScrollIdleTimeoutIfSupported(activeApp) {
|
|
executedFrame = swipe(app: activeApp, direction: direction)
|
|
}
|
|
}
|
|
guard let dragFrame = executedFrame else {
|
|
return Response(ok: false, error: ErrorPayload(message: "swipe is only supported on tvOS"))
|
|
}
|
|
return gestureResponse(message: "swiped", timing: timing, frame: .drag(dragFrame))
|
|
case .findText:
|
|
guard let text = command.text else {
|
|
return Response(ok: false, error: ErrorPayload(message: "findText requires text"))
|
|
}
|
|
let found = findElement(app: activeApp, text: text) != nil
|
|
return Response(ok: true, data: DataPayload(found: found))
|
|
case .querySelector:
|
|
guard let selectorKey = command.selectorKey, let selectorValue = command.selectorValue else {
|
|
return Response(ok: false, error: ErrorPayload(message: "querySelector requires selectorKey and selectorValue"))
|
|
}
|
|
return queryElement(app: activeApp, selectorKey: selectorKey, selectorValue: selectorValue)
|
|
case .readText:
|
|
guard let x = command.x, let y = command.y else {
|
|
return Response(ok: false, error: ErrorPayload(message: "readText requires x and y"))
|
|
}
|
|
guard let text = readTextAt(app: activeApp, x: x, y: y) else {
|
|
return Response(ok: false, error: ErrorPayload(message: "readText did not resolve text"))
|
|
}
|
|
return Response(ok: true, data: DataPayload(text: text))
|
|
case .snapshot:
|
|
return try executeSnapshotPrepared(command: command, activeApp: activeApp)
|
|
case .screenshot:
|
|
let screenshot: XCUIScreenshot
|
|
#if os(macOS)
|
|
// macOS keeps the app-targeted capture behavior for window-level screenshots.
|
|
if let bundleId = command.appBundleId, !bundleId.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
|
|
let targetApp = XCUIApplication(bundleIdentifier: bundleId)
|
|
targetApp.activate()
|
|
activeApp = targetApp
|
|
// Brief wait for the app transition animation to complete
|
|
sleepFor(0.5)
|
|
}
|
|
if command.fullscreen == true {
|
|
screenshot = XCUIScreen.main.screenshot()
|
|
} else if let bundleId = command.appBundleId, !bundleId.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
|
|
screenshot = screenshotRoot(app: activeApp).screenshot()
|
|
} else {
|
|
screenshot = XCUIScreen.main.screenshot()
|
|
}
|
|
#else
|
|
screenshot = XCUIScreen.main.screenshot()
|
|
#endif
|
|
guard let pngData = runnerPngData(for: screenshot.image) else {
|
|
return Response(ok: false, error: ErrorPayload(message: "Failed to encode screenshot as PNG"))
|
|
}
|
|
if command.inlineScreenshot == true {
|
|
return Response(
|
|
ok: true,
|
|
data: DataPayload(imageBase64: pngData.base64EncodedString())
|
|
)
|
|
}
|
|
let fileName = "screenshot-\(Int(Date().timeIntervalSince1970 * 1000)).png"
|
|
let filePath = (NSTemporaryDirectory() as NSString).appendingPathComponent(fileName)
|
|
do {
|
|
try pngData.write(to: URL(fileURLWithPath: filePath))
|
|
} catch {
|
|
return Response(ok: false, error: ErrorPayload(message: "Failed to write screenshot: \(error.localizedDescription)"))
|
|
}
|
|
#if os(macOS)
|
|
return Response(ok: true, data: DataPayload(message: filePath))
|
|
#else
|
|
// Return path relative to app container root (tmp/ maps to NSTemporaryDirectory)
|
|
return Response(ok: true, data: DataPayload(message: "tmp/\(fileName)"))
|
|
#endif
|
|
case .back, .backInApp:
|
|
if tapInAppBackControl(app: activeApp) {
|
|
let message = command.command == .back ? "back" : "backInApp"
|
|
return Response(ok: true, data: DataPayload(message: message))
|
|
}
|
|
return Response(ok: false, error: ErrorPayload(message: "in-app back control is not available"))
|
|
case .backSystem:
|
|
if performSystemBackAction(app: activeApp) {
|
|
return Response(ok: true, data: DataPayload(message: "backSystem"))
|
|
}
|
|
return Response(ok: false, error: ErrorPayload(message: "system back is not available"))
|
|
case .home:
|
|
pressHomeButton()
|
|
return Response(ok: true, data: DataPayload(message: "home"))
|
|
case .rotate:
|
|
return executeRotateCommand(command)
|
|
case .appSwitcher:
|
|
performAppSwitcherGesture(app: activeApp)
|
|
return Response(ok: true, data: DataPayload(message: "appSwitcher"))
|
|
case .keyboardDismiss:
|
|
let result = dismissKeyboard(app: activeApp)
|
|
if result.wasVisible && !result.dismissed {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "UNSUPPORTED_OPERATION",
|
|
message: "Unable to dismiss the iOS keyboard: the keyboard exposes no dismiss key, and background taps are never attempted (no tap outside the keyboard can be proven side-effect-free)",
|
|
hint:
|
|
"The on-screen keyboard usually does not block agent-device interactions: press the next target directly instead of retrying dismiss. If that press fails or reports no visible effect, scroll the target into view, or use keyboard enter to press the return key when submission is wanted."
|
|
)
|
|
)
|
|
}
|
|
return Response(
|
|
ok: true,
|
|
data: DataPayload(
|
|
message: "keyboardDismiss",
|
|
visible: result.visible,
|
|
wasVisible: result.wasVisible,
|
|
dismissed: result.dismissed,
|
|
keyboardDismissMechanism: result.mechanism?.rawValue
|
|
)
|
|
)
|
|
case .keyboardReturn:
|
|
let result = pressKeyboardReturn(app: activeApp)
|
|
if !result.pressed {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "UNSUPPORTED_OPERATION",
|
|
message: "Unable to press the iOS keyboard return key"
|
|
)
|
|
)
|
|
}
|
|
return Response(
|
|
ok: true,
|
|
data: DataPayload(
|
|
message: "keyboardReturn",
|
|
visible: result.visible,
|
|
wasVisible: result.wasVisible
|
|
)
|
|
)
|
|
case .alert:
|
|
let action = (command.action ?? "get").lowercased()
|
|
let deadline = alertDeadline ?? Date().addingTimeInterval(
|
|
Self.alertCommandTimeout(timeoutMs: command.timeoutMs)
|
|
)
|
|
guard let alert = resolveAlert(app: activeApp, deadline: deadline) else {
|
|
// Typed so the host retries on absence alone: a transport or runner failure carries no
|
|
// code and must not be mistaken for "no alert yet" (ALERT_NOT_FOUND_RUNNER_CODE).
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: "ALERT_NOT_FOUND", message: "alert not found")
|
|
)
|
|
}
|
|
return handleAlert(alert, action: action, deadline: deadline)
|
|
case .gesture:
|
|
guard let plan = command.gesturePlan else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: "INVALID_ARGS", message: "gesture requires gesturePlan")
|
|
)
|
|
}
|
|
if let validationError = plannedGestureValidationError(plan) {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: "INVALID_ARGS", message: validationError)
|
|
)
|
|
}
|
|
switch plannedGestureExecution(for: plan) {
|
|
case .fastSwipe:
|
|
// Validation above guarantees a non-empty, single-pointer path for this execution kind.
|
|
let first = plan.pointers[0].samples.first!.point
|
|
let last = plan.pointers[0].samples.last!.point
|
|
return canonicalPlannedGestureResponse(
|
|
executeDragGesture(
|
|
activeApp: activeApp,
|
|
x: first.x,
|
|
y: first.y,
|
|
x2: last.x,
|
|
y2: last.y,
|
|
durationMs: plan.durationMs,
|
|
synthesized: true,
|
|
message: plan.intent,
|
|
synthesizedPolicyKind: .synthesizedDrag,
|
|
synthesizedProfile: .fastSwipe
|
|
)
|
|
)
|
|
case .sampled:
|
|
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
|
|
sampledPlannedGesture(app: activeApp, plan: plan)
|
|
}
|
|
return plannedGestureResponse(plan: plan, timing: timing, outcome: outcome)
|
|
}
|
|
case .gestureViewport:
|
|
let frame = resolvedTouchReferenceFrame(app: activeApp, appFrame: activeApp.frame)
|
|
guard !frame.isNull, !frame.isInfinite, !frame.isEmpty else {
|
|
return Response(ok: false, error: ErrorPayload(code: "COMMAND_FAILED", message: "Active app interaction viewport is unavailable"))
|
|
}
|
|
return Response(ok: true, data: DataPayload(message: "gestureViewport", x: frame.minX, y: frame.minY, x2: frame.width, y2: frame.height))
|
|
case .sequence:
|
|
return executeSequence(command: command, activeApp: activeApp)
|
|
}
|
|
}
|
|
|
|
private func invalidScrollDirectionResponse(commandName: String) -> Response {
|
|
Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "INVALID_ARGS",
|
|
message: "\(commandName) requires direction up|down|left|right"
|
|
)
|
|
)
|
|
}
|
|
|
|
private func scrollDurationIsValid(_ durationMs: Double?) -> Bool {
|
|
guard let durationMs else { return true }
|
|
return durationMs.isFinite && durationMs >= 0 && durationMs <= 10000
|
|
}
|
|
|
|
private func invalidScrollDurationResponse(commandName: String) -> Response {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "INVALID_ARGS",
|
|
message: "\(commandName) durationMs must be between 0 and 10000"
|
|
)
|
|
)
|
|
}
|
|
|
|
private func executeScrollDragGesture(
|
|
activeApp: XCUIApplication,
|
|
x: Double,
|
|
y: Double,
|
|
x2: Double,
|
|
y2: Double,
|
|
durationMs: Double,
|
|
message: String,
|
|
context: SynthesizedCoordinateContext,
|
|
releaseBehavior: ScrollReleaseBehavior?
|
|
) -> Response {
|
|
#if os(iOS)
|
|
return executeDragGesture(
|
|
activeApp: activeApp,
|
|
x: x,
|
|
y: y,
|
|
x2: x2,
|
|
y2: y2,
|
|
durationMs: durationMs,
|
|
synthesized: true,
|
|
message: message,
|
|
synthesizedContext: context,
|
|
synthesizedPolicyKind: .scroll,
|
|
synthesizedProfile: scrollDragProfile(releaseBehavior: releaseBehavior)
|
|
)
|
|
#else
|
|
return executeDragGesture(
|
|
activeApp: activeApp,
|
|
x: x,
|
|
y: y,
|
|
x2: x2,
|
|
y2: y2,
|
|
durationMs: durationMs,
|
|
synthesized: false,
|
|
message: message,
|
|
synthesizedPolicyKind: .scroll
|
|
)
|
|
#endif
|
|
}
|
|
|
|
/// Shared drag execution for explicit drag commands. The iOS synthesized lane keeps its
|
|
/// fallback policy explicit; viewport scrolling owns a separate single drag specification.
|
|
private func executeDragGesture(
|
|
activeApp: XCUIApplication,
|
|
x: Double,
|
|
y: Double,
|
|
x2: Double,
|
|
y2: Double,
|
|
durationMs: Double?,
|
|
synthesized: Bool,
|
|
message: String,
|
|
synthesizedContext: SynthesizedCoordinateContext? = nil,
|
|
synthesizedPolicyKind: SynthesizedGesturePolicyKind,
|
|
synthesizedProfile: SynthesizedDragProfile = .continuous
|
|
) -> Response {
|
|
let durationMs = durationMs ?? runnerDefaultDragDurationMs
|
|
let commandName = dragCommandName(message: message)
|
|
guard x.isFinite, y.isFinite, x2.isFinite, y2.isFinite else {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: "INVALID_ARGS", message: "\(commandName) requires finite coordinates")
|
|
)
|
|
}
|
|
if synthesized, let synthesizedResponse = executeSynthesizedDragGesture(
|
|
activeApp: activeApp,
|
|
x: x,
|
|
y: y,
|
|
x2: x2,
|
|
y2: y2,
|
|
durationMs: durationMs,
|
|
message: message,
|
|
context: synthesizedContext,
|
|
policyKind: synthesizedPolicyKind,
|
|
profile: synthesizedProfile
|
|
) {
|
|
return synthesizedResponse
|
|
}
|
|
let dragPoints = keyboardAvoidingDragPoints(app: activeApp, x: x, y: y, x2: x2, y2: y2)
|
|
let dragFrame = resolvedDragVisualizationFrame(
|
|
app: activeApp,
|
|
x: dragPoints.x,
|
|
y: dragPoints.y,
|
|
x2: dragPoints.x2,
|
|
y2: dragPoints.y2
|
|
)
|
|
var fallback: GestureFallback?
|
|
if synthesized {
|
|
let durationMs = min(max(durationMs, 16), 10000)
|
|
let context = synthesizedCoordinateContext(
|
|
app: activeApp,
|
|
policy: synthesizedGesturePolicy(synthesizedPolicyKind)
|
|
)
|
|
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
|
|
synthesizedDragAt(
|
|
app: activeApp,
|
|
x: dragPoints.x,
|
|
y: dragPoints.y,
|
|
x2: dragPoints.x2,
|
|
y2: dragPoints.y2,
|
|
durationMs: durationMs,
|
|
profile: synthesizedProfile,
|
|
context: context
|
|
)
|
|
}
|
|
if case .performed = outcome {
|
|
return gestureResponse(message: message, timing: timing, frame: .drag(dragFrame))
|
|
}
|
|
fallback = gestureFallback(strategy: "xctest-coordinate-drag", from: outcome)
|
|
}
|
|
let holdDuration = synthesized
|
|
? synthesizedSwipeFallbackHoldDuration(durationMs: durationMs)
|
|
: coordinateDragHoldDuration()
|
|
let (timing, outcome) = performGesture(activeApp) {
|
|
dragAt(
|
|
app: activeApp,
|
|
x: dragPoints.x,
|
|
y: dragPoints.y,
|
|
x2: dragPoints.x2,
|
|
y2: dragPoints.y2,
|
|
holdDuration: holdDuration
|
|
)
|
|
}
|
|
if let response = unsupportedResponse(for: outcome) {
|
|
return response
|
|
}
|
|
return gestureResponse(
|
|
message: message,
|
|
timing: timing,
|
|
frame: .drag(dragFrame),
|
|
fallback: fallback
|
|
)
|
|
}
|
|
|
|
private func executeSynthesizedDragGesture(
|
|
activeApp: XCUIApplication,
|
|
x: Double,
|
|
y: Double,
|
|
x2: Double,
|
|
y2: Double,
|
|
durationMs: Double,
|
|
message: String,
|
|
context: SynthesizedCoordinateContext?,
|
|
policyKind: SynthesizedGesturePolicyKind,
|
|
profile: SynthesizedDragProfile
|
|
) -> Response? {
|
|
#if os(iOS)
|
|
let policy = synthesizedGesturePolicy(policyKind)
|
|
let context = context ?? synthesizedCoordinateContext(app: activeApp, policy: policy)
|
|
guard let plan = axFreeSynthesizedDragPlan(
|
|
app: activeApp,
|
|
x: x,
|
|
y: y,
|
|
x2: x2,
|
|
y2: y2,
|
|
context: context
|
|
)
|
|
else {
|
|
if context?.allowsXCTestCoordinateFallback == true {
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: true)
|
|
return executeCoordinateDragFallback(
|
|
activeApp: activeApp,
|
|
x: x,
|
|
y: y,
|
|
x2: x2,
|
|
y2: y2,
|
|
durationMs: durationMs,
|
|
message: message,
|
|
fallback: nil
|
|
)
|
|
}
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: context, fallbackAttempted: false)
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "INVALID_ARGS",
|
|
message: "\(dragCommandName(message: message)) could not resolve a finite synthesized coordinate frame"
|
|
)
|
|
)
|
|
}
|
|
let durationMs = min(max(durationMs, 16), 10000)
|
|
let dragFrame = axFreeDragVisualizationFrame(
|
|
x: plan.points.x,
|
|
y: plan.points.y,
|
|
x2: plan.points.x2,
|
|
y2: plan.points.y2,
|
|
referenceFrame: plan.referenceFrame
|
|
)
|
|
let (timing, outcome) = performGesture(activeApp, idleTimeout: false) {
|
|
synthesizedDragAt(
|
|
app: activeApp,
|
|
x: plan.points.x,
|
|
y: plan.points.y,
|
|
x2: plan.points.x2,
|
|
y2: plan.points.y2,
|
|
durationMs: durationMs,
|
|
profile: profile,
|
|
context: plan.context
|
|
)
|
|
}
|
|
if case .performed = outcome {
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: plan.context, fallbackAttempted: false)
|
|
return gestureResponse(message: message, timing: timing, frame: .drag(dragFrame))
|
|
}
|
|
if plan.context.allowsXCTestCoordinateFallback {
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: plan.context, fallbackAttempted: true)
|
|
return executeCoordinateDragFallback(
|
|
activeApp: activeApp,
|
|
x: plan.points.x,
|
|
y: plan.points.y,
|
|
x2: plan.points.x2,
|
|
y2: plan.points.y2,
|
|
durationMs: durationMs,
|
|
message: message,
|
|
fallback: gestureFallback(strategy: "xctest-coordinate-drag", from: outcome)
|
|
)
|
|
}
|
|
logSynthesizedGesturePolicyDecision(kind: policyKind, context: plan.context, fallbackAttempted: false)
|
|
return unsupportedResponse(for: outcome)
|
|
#else
|
|
return nil
|
|
#endif
|
|
}
|
|
|
|
private func executeCoordinateDragFallback(
|
|
activeApp: XCUIApplication,
|
|
x: Double,
|
|
y: Double,
|
|
x2: Double,
|
|
y2: Double,
|
|
durationMs: Double,
|
|
message: String,
|
|
fallback: GestureFallback?
|
|
) -> Response {
|
|
let dragPoints = keyboardAvoidingDragPoints(app: activeApp, x: x, y: y, x2: x2, y2: y2)
|
|
let dragFrame = resolvedDragVisualizationFrame(
|
|
app: activeApp,
|
|
x: dragPoints.x,
|
|
y: dragPoints.y,
|
|
x2: dragPoints.x2,
|
|
y2: dragPoints.y2
|
|
)
|
|
let holdDuration = synthesizedSwipeFallbackHoldDuration(durationMs: durationMs)
|
|
let (timing, outcome) = performGesture(activeApp) {
|
|
dragAt(
|
|
app: activeApp,
|
|
x: dragPoints.x,
|
|
y: dragPoints.y,
|
|
x2: dragPoints.x2,
|
|
y2: dragPoints.y2,
|
|
holdDuration: holdDuration
|
|
)
|
|
}
|
|
if let response = unsupportedResponse(for: outcome) {
|
|
return response
|
|
}
|
|
return gestureResponse(
|
|
message: message,
|
|
timing: timing,
|
|
frame: .drag(dragFrame),
|
|
fallback: fallback
|
|
)
|
|
}
|
|
|
|
private func scrollReferenceFrame(app: XCUIApplication, context: SynthesizedCoordinateContext) -> CGRect {
|
|
#if os(iOS)
|
|
return synthesizedFrameAvoidingKeyboardWhenAllowed(app: app, context: context)
|
|
#else
|
|
return resolvedTouchReferenceFrame(app: app, appFrame: app.frame)
|
|
#endif
|
|
}
|
|
|
|
private func dragCommandName(message: String) -> String {
|
|
return message == "scrolled" ? "scroll" : "drag"
|
|
}
|
|
|
|
func currentXCTestFailureCount() -> Int {
|
|
return testRun?.failureCount ?? 0
|
|
}
|
|
|
|
func didRecordXCTestFailure(since failureCountBefore: Int) -> Bool {
|
|
return currentXCTestFailureCount() > failureCountBefore
|
|
}
|
|
|
|
private func xctestRecordedFailureResponse(command: Command, response: Response) -> Response? {
|
|
guard response.ok else { return nil }
|
|
if response.data?.runnerFatal == true {
|
|
return nil
|
|
}
|
|
guard !isReadOnlyCommand(command), !isRunnerLifecycleCommand(command.command) else {
|
|
return nil
|
|
}
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "XCTEST_RECORDED_FAILURE",
|
|
message: "XCTest recorded a failure while executing \(command.command.rawValue); the action may not have been performed.",
|
|
hint: "The iOS runner session was invalidated. Re-observe with a fresh snapshot before retrying; if the accessibility tree is unavailable, use screenshot plus coordinate commands instead of retrying the tap blindly."
|
|
)
|
|
)
|
|
}
|
|
|
|
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
|
|
func runnerCommandFixture(_ json: String) throws -> Command {
|
|
try JSONDecoder().decode(Command.self, from: Data(json.utf8))
|
|
}
|
|
#endif
|
|
|
|
private func shouldSkipAppActivationPreflight(_ command: Command) -> Bool {
|
|
#if os(iOS)
|
|
if command.command == .alert {
|
|
return true
|
|
}
|
|
// Coordinate-only synthesized taps can run after an AX-fatal foreground screen because they do not
|
|
// need app activation, window lookup, keyboard lookup, or element resolution. Selector/text
|
|
// interactions intentionally stay on the normal AX path because they need an element query.
|
|
// Scroll/drag/sequence keep the normal foreground guard and stabilization path.
|
|
guard command.text == nil, command.selectorKey == nil else { return false }
|
|
guard hasCachedTargetForActivationSkip(command: command) else { return false }
|
|
return isCoordinateOnlyTap(command)
|
|
#else
|
|
return false
|
|
#endif
|
|
}
|
|
|
|
private func shouldRouteToSpringboardBlockingSystemModal(
|
|
_ command: Command,
|
|
deadline: Date? = nil
|
|
) -> Bool {
|
|
#if os(iOS)
|
|
guard command.command == .alert || isCoordinateOnlyTap(command) else {
|
|
return false
|
|
}
|
|
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
|
|
if let override = blockingSystemModalPresenceOverrideForTesting {
|
|
return override
|
|
}
|
|
#endif
|
|
let budgetDeadline = Date().addingTimeInterval(systemModalProbeBudget)
|
|
let probeDeadline = deadline.map { min($0, budgetDeadline) } ?? budgetDeadline
|
|
// `runMainThreadWork` executes inline for a main-thread caller, so that path cannot use its
|
|
// timeout machinery. Direct main-thread dispatch keeps the prior synchronous modal check;
|
|
// normal off-main command dispatch uses the bounded probe and post-probe busy recovery.
|
|
if Thread.isMainThread {
|
|
return firstBlockingSystemModal(
|
|
in: springboard,
|
|
deadline: probeDeadline
|
|
) != nil
|
|
}
|
|
return boundedBlockingSystemAlertSnapshot(
|
|
deadline: probeDeadline
|
|
) != nil
|
|
#else
|
|
return false
|
|
#endif
|
|
}
|
|
|
|
private func isCoordinateOnlyTap(_ command: Command) -> Bool {
|
|
return command.command == .tap
|
|
&& command.text == nil
|
|
&& command.selectorKey == nil
|
|
&& command.x != nil
|
|
&& command.y != nil
|
|
}
|
|
|
|
private func hasCachedTargetForActivationSkip(command: Command) -> Bool {
|
|
guard let currentApp, currentApp.state == .runningForeground else { return false }
|
|
guard let bundleId = command.appBundleId?.trimmingCharacters(in: .whitespacesAndNewlines),
|
|
!bundleId.isEmpty
|
|
else {
|
|
return true
|
|
}
|
|
return currentBundleId == bundleId
|
|
}
|
|
|
|
private func resolveAppWithoutActivation(command: Command) -> XCUIApplication {
|
|
guard let bundleId = command.appBundleId?
|
|
.trimmingCharacters(in: .whitespacesAndNewlines),
|
|
!bundleId.isEmpty
|
|
else {
|
|
return currentApp ?? app
|
|
}
|
|
if currentBundleId == bundleId, let currentApp {
|
|
return currentApp
|
|
}
|
|
return XCUIApplication(bundleIdentifier: bundleId)
|
|
}
|
|
|
|
func executeTypeCommand(activeApp: XCUIApplication, command: Command) -> Response {
|
|
guard let text = command.text else {
|
|
return Response(ok: false, error: ErrorPayload(message: "type requires text"))
|
|
}
|
|
let delaySeconds = Double(max(command.delayMs ?? 0, 0)) / 1000.0
|
|
let textEntryMode = resolveTextEntryMode(command)
|
|
let target: TextEntryTarget
|
|
var resolvedCoordinateContext: SynthesizedCoordinateContext?
|
|
var maestroNonHittableCoordinateFallbackUsed: Bool?
|
|
if command.allowNonHittableCoordinateFallback == true,
|
|
command.x != nil,
|
|
command.y != nil
|
|
{
|
|
// The shared runtime has already resolved this node as non-hittable and
|
|
// deliberately selected Maestro's coordinate compatibility route.
|
|
maestroNonHittableCoordinateFallbackUsed = true
|
|
}
|
|
let focusStartedAt = Date()
|
|
#if os(iOS)
|
|
let xCTestChannelPenalized = isSnapshotXCTestChannelPenalized(bundleId: currentBundleId)
|
|
var resolvedCoordinateTarget: TextEntryTarget?
|
|
if Self.shouldUseResolvedCoordinateTextEntryRoute(
|
|
repairMode: textEntryMode,
|
|
hasX: command.x != nil,
|
|
hasY: command.y != nil,
|
|
xCTestChannelPenalized: xCTestChannelPenalized
|
|
), let x = command.x, let y = command.y {
|
|
let policyKind = SynthesizedGesturePolicyKind.coordinateTap
|
|
let context = synthesizedCoordinateContext(
|
|
app: activeApp,
|
|
policy: synthesizedGesturePolicy(policyKind)
|
|
)
|
|
let (_, outcome) = performGesture(activeApp, idleTimeout: false) {
|
|
synthesizedTapAt(app: activeApp, x: x, y: y, context: context)
|
|
}
|
|
if Self.shouldFallbackFromSynthesizedTextEntryFocus(outcome) {
|
|
logSynthesizedGesturePolicyDecision(
|
|
kind: policyKind,
|
|
context: context,
|
|
fallbackAttempted: true
|
|
)
|
|
} else {
|
|
logSynthesizedGesturePolicyDecision(
|
|
kind: policyKind,
|
|
context: context,
|
|
fallbackAttempted: false
|
|
)
|
|
resolvedCoordinateContext = context
|
|
resolvedCoordinateTarget = TextEntryTarget(
|
|
element: nil,
|
|
refreshPoint: CGPoint(x: x, y: y),
|
|
prefersFocusedElement: false
|
|
)
|
|
}
|
|
}
|
|
#else
|
|
let xCTestChannelPenalized = false
|
|
let resolvedCoordinateTarget: TextEntryTarget? = nil
|
|
#endif
|
|
if let resolvedCoordinateTarget {
|
|
target = resolvedCoordinateTarget
|
|
} else if let selectorKey = command.selectorKey, let selectorValue = command.selectorValue {
|
|
// Released daemons may still send selector-keyed type commands even though current
|
|
// daemons resolve fill selectors through the runtime tree before reaching the runner.
|
|
let match = findElement(
|
|
app: activeApp,
|
|
selectorKey: selectorKey,
|
|
selectorValue: selectorValue,
|
|
allowNonHittableFallback: command.allowNonHittableCoordinateFallback == true
|
|
)
|
|
if match.isAmbiguous {
|
|
return Response(ok: false, error: ErrorPayload(code: "AMBIGUOUS_MATCH", message: "selector matched multiple elements"))
|
|
}
|
|
guard let element = match.element else {
|
|
return Response(ok: false, error: ErrorPayload(code: "NO_MATCH", message: "selector did not match an element"))
|
|
}
|
|
guard isTextEntryElement(element) else {
|
|
return Response(ok: false, error: ErrorPayload(code: "INVALID_TARGET", message: "selector did not match a text input"))
|
|
}
|
|
if command.allowNonHittableCoordinateFallback == true {
|
|
maestroNonHittableCoordinateFallbackUsed = match.usedNonHittableFallback
|
|
}
|
|
target = focusTextInputForTextEntry(app: activeApp, element: element)
|
|
} else {
|
|
target = focusTextInputForTextEntry(app: activeApp, x: command.x, y: command.y)
|
|
}
|
|
NSLog(
|
|
"AGENT_DEVICE_RUNNER_TEXT_ENTRY_PHASE commandId=%@ phase=focus durationMs=%.1f chars=%d mode=%@",
|
|
command.commandId ?? "",
|
|
Date().timeIntervalSince(focusStartedAt) * 1000.0,
|
|
text.count,
|
|
textEntryModeName(textEntryMode)
|
|
)
|
|
if textEntryMode == .replacement {
|
|
#if os(iOS)
|
|
let canReplaceResolvedFirstResponder = Self.shouldUseSynthesizedFirstResponderReplacement(
|
|
hasResolvedElement: target.element != nil,
|
|
hasRefreshPoint: target.refreshPoint != nil,
|
|
xCTestChannelPenalized: xCTestChannelPenalized
|
|
)
|
|
#else
|
|
let canReplaceResolvedFirstResponder = false
|
|
#endif
|
|
guard target.element != nil || canReplaceResolvedFirstResponder else {
|
|
let message =
|
|
(command.x != nil && command.y != nil)
|
|
? "no text input found at the provided coordinates to clear"
|
|
: "no focused text input to clear"
|
|
return Response(ok: false, error: ErrorPayload(message: message))
|
|
}
|
|
}
|
|
let textResult = typeTextReliably(
|
|
app: activeApp,
|
|
target: target,
|
|
text: text,
|
|
delaySeconds: delaySeconds,
|
|
repairMode: textEntryMode,
|
|
xCTestChannelPenalized: xCTestChannelPenalized,
|
|
synthesizer: PrivateXCTestTextEntrySynthesizer(),
|
|
commandId: command.commandId
|
|
)
|
|
if let failure = textResult.failure {
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(code: failure.rawValue, message: failure.message, hint: failure.hint)
|
|
)
|
|
}
|
|
if textResult.verified == false {
|
|
let expected = textResult.expectedText ?? ""
|
|
let observed = textResult.observedText ?? ""
|
|
return Response(
|
|
ok: false,
|
|
error: ErrorPayload(
|
|
code: "TEXT_ENTRY_MISMATCH",
|
|
message: "text entry verification failed: expected \"\(expected)\", observed \"\(observed)\""
|
|
)
|
|
)
|
|
}
|
|
let point = target.refreshPoint
|
|
let frame: CGRect
|
|
if let resolvedCoordinateContext {
|
|
frame = resolvedCoordinateContext.referenceFrame
|
|
} else if point != nil {
|
|
frame = activeApp.frame
|
|
} else {
|
|
// Bare `type` has no coordinate response to normalize. Avoid serializing the
|
|
// application AX tree only to emit unused reference dimensions.
|
|
frame = .zero
|
|
}
|
|
return Response(
|
|
ok: true,
|
|
data: DataPayload(
|
|
message: textResult.repaired ? "typed after repair" : "typed",
|
|
x: point.map { Double($0.x) },
|
|
y: point.map { Double($0.y) },
|
|
referenceWidth: frame.isEmpty ? nil : Double(frame.width),
|
|
referenceHeight: frame.isEmpty ? nil : Double(frame.height),
|
|
maestroNonHittableCoordinateFallbackUsed: maestroNonHittableCoordinateFallbackUsed,
|
|
textEntryRoute: textResult.textEntryRoute
|
|
)
|
|
)
|
|
}
|
|
}
|