mirror of https://github.com/callstack/agent-device.git synced 2026-09-14 20:06:34 +08:00

Files

T

Michał Pierzchała 5d10ced2b5 perf(ios): add lifecycle-safe runner sequence command for hot press series

Adds a narrow 'sequence' runner command that batches an explicit
allowlist of coordinate steps (tap, longPress, drag) into one
lifecycle-tracked request with stop-on-first-failure and small bounded
per-step results. iOS press series with hold/jitter now issue one
sequence request per ~20-step chunk (also budgeted to stay under the
runner's 30s main-thread watchdog) instead of one request per press.
Sequence responses are journaled and retained, so lost-response recovery
returns observed results without replaying the gesture sequence.

Closes #669

2026-06-11 10:06:04 +02:00

AgentDeviceRunner

perf(ios): add lifecycle-safe runner sequence command for hot press series

2026-06-11 10:06:04 +02:00

README.md

fix: keep iOS runner hot across app closes (#700 )

2026-06-06 21:26:12 +02:00

RUNNER_PROTOCOL.md

feat: add recording quality flag (#409 )

2026-04-15 13:18:57 +02:00

README.md

agent-device iOS Runner

This folder contains the lightweight XCUITest runner used to provide element-level automation for Apple-family targets.

Intent

Provide a minimal XCTest target that exposes UI automation over a small HTTP server.
Allow local builds via xcodebuild and caching for faster subsequent runs.
Support simulator prebuilds where compatible.

Status

Current internal runner for iOS, tvOS, and macOS desktop automation.

Protocol and maintenance references:

Protocol overview: RUNNER_PROTOCOL.md
TypeScript client: ../src/platforms/ios/runner-client.ts
Swift wire models: AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift

UITest Runner File Map

AgentDeviceRunnerUITests/RunnerTests is split into focused files to reduce context size for contributors and LLM agents.

RunnerTests.swift: shared state/constants, setUp(), and testCommand() entry flow.
RunnerTests+Models.swift: wire protocol models (Command, Response, snapshot payload models).
RunnerTests+Environment.swift: environment and CLI argument helpers (RunnerEnv).
RunnerTests+Transport.swift: TCP request handling and HTTP parsing/encoding.
RunnerTests+CommandExecution.swift: command dispatch (execute*) and command switch.
RunnerTests+Lifecycle.swift: activation/retry/stabilization and recording lifecycle helpers.
RunnerTests+Interaction.swift: tap/drag/swipe/type/back/home/rotate/app-switcher helpers.
RunnerTests+Snapshot.swift: fast/raw snapshot builders and include/filter helpers.
RunnerTests+SystemModal.swift: SpringBoard/system modal detection and modal snapshot shaping.
RunnerTests+ScreenRecorder.swift: nested ScreenRecorder implementation.

Snapshot Strategy

iOS snapshots have two explicit capture modes:

full/raw snapshots use recursive XCTest snapshots for rich hierarchy and diagnostics;
compact interactive snapshots use a bounded flat XCTest query path for fast agent-facing refs.

Some simulator apps expose accessibility trees that lower-level AX services can inspect but XCTest cannot serialize reliably. In those cases compact interactive snapshots may return a sparse root quickly, while full snapshots preserve the XCTest error. See ../docs/adr/0004-ios-snapshot-backend-strategy.md for the backend boundary and the rationale for a future simulator AX-service backend.

Protocol Notes

The daemon posts JSON commands to POST /command on the runner's local HTTP listener.
The runner responds with a JSON envelope shaped as { ok, data?, error? }.
The protocol is internal to agent-device; when adding or renaming commands, update both wire models and the protocol tests/docs in the same change.