# Physical iOS device support Try it now (cabled iPhone, unlocked, Developer Mode on): ```bash argent run list-devices # copy the 25-char UDID argent run launch-app --udid <UDID> --bundleId com.apple.Preferences ``` First launch builds and signs the on-device runner: 3 to 5 minutes, keep the phone unlocked. Every later start: seconds, from cache. No signing setup needed if an Apple ID is signed into Xcode; the team is detected from the keychain. ## What this adds - Physical iPhone support, on by default: a cabled iPhone simply appears in `list-devices`. iPhone-only: physical iPads are skipped at discovery. - 15 tools ported to hardware. Flows replay on device. - Zero-config signing from the keychain; `ARGENT_IOS_TEAM_ID` is an override, nothing is persisted. - Self-contained npm install: the runner's Swift sources ship in the package. New package `@argent/ios-device-runner` (an XCUITest bundle hosting an HTTP command server on the device), new tool-server backend under `src/utils/ios-device/`. - The one always-on cost: `list-devices` on macOS now runs physical discovery in parallel with the other platform probes, under the existing 8s devicectl timeout; off-macOS it early-returns. ## Suggested review order Rough estimate: half a day for the full pass. 1. `packages/ios-device-runner/PROTOCOL.md`, then `README.md` (wire contract and reliability model, ~15 min) 2. `packages/tool-server/src/utils/ios-device/` (devicectl, usbmux, team-detect, runner-build, runner-commands: the core) 3. `packages/tool-server/src/blueprints/ios-device-runner.ts` (service lifecycle, crash recovery) 4. Per-tool device arms and the capability gates (`ios-physical-device-classification.test.ts` is the ratchet) 5. Docs page, the two new skills, the Swift runner sources ## Requirements 1. macOS host with full Xcode (iOS platform installed); uses `xcrun devicectl` and `xcodebuild`. 2. An Apple ID signed into Xcode; automatic signing mints the runner's provisioning profile. 3. USB cable only. Wi-Fi is deliberately unsupported: the transport is usbmux, and a Wi-Fi-only device lists as `paired`, not `connected`. 4. iPhone paired with and trusting the Mac, Developer Mode on, A12/2018 or newer (modern 8+16-hex UDID, the CoreDevice floor). Legacy 40-hex UDIDs are not classified as iOS. 5. Signing resolves as: `ARGENT_IOS_TEAM_ID` in the tool-server's environment (explicit override, CI) -> keychain detection (newest identity first, memoized per server run, never persisted). One certificate: nothing to do. Several teams: newest wins; the first tool result names the pick, the runner-ups, and the override. No certificate: the error walks through Xcode > Settings > Accounts and Manage Certificates > + > Apple Development, then retry; it never names the env var, since a team id alone cannot sign. Bundle ids derive per team (`com.argent.runner.t<teamid>`). `ARGENT_IOS_RUNNER_PROJECT` overrides the project path for unusual layouts. Build artifacts cache under `~/.argent/ios-device-runner`, keyed by runner sources, Xcode/SDK version, and signing. A team with no registered devices self-heals: the failed generic build retries against the connected phone, which registers it. ## How it works 1. `xcrun devicectl` (CoreDevice) is the control plane: discovery, readiness, app lifecycle. Readiness gates on the wired transport, so an unplugged phone is refused before any build starts. 2. A signed XCUITest runner is built lazily with the resolved team, started via `xcodebuild test-without-building`, and parks as an HTTP command server for 24h. On an npm install the Xcode project is the copy shipped inside the package. 3. The listener binds to the device's loopback interface; the host reaches it over usbmuxd with a dependency-free `node:net` client. 4. Mutating commands are journaled by `commandId`; a lost reply is resolved by asking `status` for the journaled fate, never by replay (at-most-once delivery). 5. Routing hangs off `resolveDevice` UDID-shape classification plus a per-tool capability allowlist. CLI and MCP layers are untouched; outside tool-server, docs, and the runner package the diff is failure-code constants (registry), two agent skills plus rule updates (skills), the pack step (packages/argent), and the e2e smoke script. ## Supported tools | Tool | Status | Notes | | ----------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | launch-app | Supported | Plain `devicectl` launch. System UI (SpringBoard/Spotlight) registers a session only; no launch attempt. Carries a note when signing is not resolvable yet. | | restart-app | Supported | System UI bundle ids rejected before any device contact. | | reinstall-app | Supported | Same system UI pre-flight. | | screenshot | Supported | Served through the runner; device-wide, needs no registered app. (No host-side devicectl screenshot exists on any toolchain; that dead route was removed.) | | describe | Supported | AX tree snapshot via the runner. Snapshots the session app's process, so another process's dialogs are not in the tree. A backgrounded session app fails with an actionable error instead of being silently re-fronted. Focus polling is off for this source; typing uses a fixed settle. | | keyboard | Supported | Types into the focused element; typing with nothing focused is refused with "tap the text field first". `enter` and `backspace` are the named keys (both exist as software-keyboard characters; there is no per-keycode HID surface on hardware). Whitespace-only keys rejected, not silently "typed". | | gesture-tap | Supported | `clickCount > 1` dispatches as one native multi-tap gesture, not separate taps. | | gesture-swipe | Supported | Single-finger drag between two points. | | gesture-custom | Supported | Two shapes only: press-hold and straight single-finger drag. Second finger or `Move` waypoints rejected with authoring guidance, before any runner starts. | | button | Supported | `home`, `volumeUp`, `volumeDown`, `actionButton` (volume buttons are simulator-unavailable in the SDK, not device-unavailable). `power`/`appSwitch` have no public XCUIDevice API and reject before device contact; a button the model lacks gets a typed refusal via `hasHardwareButton`. | | await-ui-element | Supported | Polls the same runner snapshot describe returns; waits and taps see identical frames. Degrades to `cause: "unreadable"` with a note on a backgrounded app, never a raw throw. | | await-screen-idle | Supported | Same describe path; resolves not-settled instead of throwing, so a follow-up screenshot still runs. | | run-sequence | Supported | Outer gate only; each step pre-flights its own capability. | | open-url | Supported | devicectl launches the receiving app with the URL as its launch payload. http(s) defaults to Safari; any other scheme needs the new optional `bundleId` param (hardware cannot resolve a scheme's handler system-wide). | | screenshot-diff | Supported | Live captures through the runner (full-resolution, device-wide, no registered app needed); the diff itself runs on the host. Baselines are per device model/resolution; `rotation` is ignored on hardware. | Flows run through these tools; the device flow tree gets the same `subtreeText` hoisting as every other platform, so container-scoped `text` asserts work. ## Not supported (400 at the capability gate) 1. gesture-pinch, gesture-rotate: no two-finger coordinate API on hardware. Pinch/rotate flow directives also fail upfront with a clear error. 2. screen-recording-start/stop (video): simulator-only capture pipeline. 3. rotate, paste, shake, settings-permissions: simulator-only backends, not ported (paste also hits the iOS 16+ cross-app paste permission prompt). 4. native-devtools, debugger, profiler families: simulator-side instrumentation (DYLD injection does not exist on hardware). 5. Physical iPads: filtered at discovery for now. A test ratchet (`ios-physical-device-classification.test.ts`) keeps `apple.device` declarations in sync with the ported list. ## Core contracts 1. Observation never mutates the screen: a backgrounded session app makes `describe`/`await-*` fail with an error naming the three ways out (screenshot, launch-app, or launch-app com.apple.springboard). Mutating commands do re-front, and their result carries `reactivated: true`. 2. The runner listener is unauthenticated; mitigation is the loopback-only bind. 3. `list-devices` states: `connected` (cabled now, usable) vs `paired` (unreachable, never auto-bound). Wired transport alone decides; a Wi-Fi-migrated CoreDevice tunnel does not count. 4. The multi-team signing pick rides the first affected tool result as a one-shot note (the pre-existing notes channel). The `list-devices` result shape is unchanged from main: devices and avds, nothing else. 5. Rejections are typed 400 validation errors end to end, so a batch flow run skips one failing flow instead of aborting. Also: the runner is a first-class device-owned service (`stop-simulator-server` and `stop-all-simulator-servers` reap it, detached xcodebuild child included; a crashed runner respawns on the next call). A passcode-locked phone refuses launches; the devicectl 10002 hint also covers a pending system prompt on an unlocked phone. Build failures surface xcodebuild's own `error:` lines with signing hints (no registered devices, free-team app-id cap, missing provisioning, the `errSecInternalComponent` partition-list fix). An app XCTest reports as `.unknown` (a swipe-killed foreign app) is refused with launch-app guidance. Status-bar pinning is skipped on hardware; pixel-settle masks the top band. ## Testing - Unit coverage, hardware-independent: UDID classification and capability routing; usbmux framing and transport; devicectl list/hints parsing incl. the silent iPhone-only filter; keychain detection against real certificate fixtures; runner build/cache self-heals (poisoned cache, concurrent checkout, fresh-team retry); crash/journal recovery; the backgrounded-observation and reactivated contracts; keyboard guards incl. the not-focused refusal; system-UI pre-flight; per-tool device arms. - Manual smoke on a real device: `scripts/e2e-ios-physical-device.mjs`. ## Docs and skills 1. `features/physical-ios-devices.mdx`: the user-facing walkthrough (zero-config signing with the sign-in fallback, prerequisites, app-scoped model, troubleshooting). 2. `fundamentals/supported-platforms.mdx`: physical iPhones in the platform matrix (iPads not supported yet). 3. `reference/tools.mdx` and `reference/configuration.mdx`: list-devices states and the `ARGENT_IOS_TEAM_ID` override. 4. Two skills ship with the package via `argent init`: `argent-ios-device-setup` (setup ladder plus a troubleshooting table keyed to real error texts) and `argent-ios-device-interact` (the app-scoped hardware contract, supported subset with alternatives, gesture recipes, pitfalls). The agent rule and sibling skills route to them. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added experimental USB-connected physical iPhone support. * Physical iPhones now support discovery, app lifecycle actions, URL opening, accessibility snapshots, screenshots, taps, swipes, custom gestures, keyboard input, waits, flows, and screenshot comparisons. * Added automatic signing-team detection with optional `ARGENT_IOS_TEAM_ID` configuration. * Added clear handling for paired or unavailable devices and unsupported hardware actions. * **Documentation** * Added setup, troubleshooting, capability, and physical-device usage guidance. * Clarified that physical iPads and several simulator-only tools and gestures remain unsupported. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
5.4 KiB
@argent/ios-device-runner
Argent's on-device automation runner for physical iOS devices: an XCUITest
bundle that, instead of running a scripted test, starts an HTTP command server
and parks in an XCTWaiter wait for 24 hours. XCUITest is the only
Apple-supported process allowed to drive arbitrary apps on real hardware, so
Argent hosts its automation server inside one.
Layout
ArgentRunner/
ArgentRunner.xcodeproj Two targets: ArgentRunner (placeholder host app)
and ArgentRunnerUITests (the runner itself).
ArgentRunner/ Host app: a single black "Argent Runner" screen.
ArgentRunnerUITests/
ArgentRunnerSession.swift Entry test + dispatch, coalescing, busy gate
ArgentRunnerSession+Commands.swift Target resolution, exception guard,
recorded-failure detection,
button / shutdown
ArgentRunnerSession+Gestures.swift tap / longPress / drag / viewport
ArgentRunnerSession+TextEntry.swift type / keyboardReturn / keyboardDelete
ArgentRunnerSession+Snapshot.swift one-shot AX tree capture + flattening
ArgentRunnerSession+Screenshot.swift inline base64 PNG
RunnerProtocol.swift wire models (see PROTOCOL.md)
RunnerHTTPServer.swift NWListener HTTP/1.1 framing
CommandJournal.swift send-once bookkeeping for lost replies
MainThreadGate.swift watchdog + busy/wedged reporting
ArgentExceptionGuard.{h,m} @try/@catch shim for XCTest NSExceptions
The TypeScript side (build orchestration, transport, command client) lives in
packages/tool-server/src/utils/ios-device/.
How it runs
- The tool-server builds this project lazily with
xcodebuild build-for-testing(signed with the user's team; see the environment variables below) into~/.argent/ios-device-runner/derived, stamped with a fingerprint of sources + Xcode + signing; a stamp mismatch (an Argent or Xcode update, a team change) rebuilds in place. - Per session it launches
xcodebuild test-without-buildingdetached; testmanagerd installs and starts the runner on the device. The runner binds a system-assigned loopback port and logs it asARGENT_RUNNER_LISTENING port=N; the tool-server reads that line from the launch log before it sends the first command. - Commands travel as one HTTP POST per command over usbmux (USB cable
only).
PROTOCOL.mddocuments the contract.
Build-time configuration
The signing team resolves in two steps, and the environment variable wins:
ARGENT_IOS_TEAM_ID(optional): your Apple Developer Team ID, a 10-character code. Find it in Xcode > Settings > Accounts (select your Apple ID and team) or at developer.apple.com/account under Membership.- When it is not set, the tool-server detects the team from the Mac's keychain: the Apple Development certificates name their team (and the legacy iPhone Developer name is checked too), and with several teams the one with the newest certificate wins. The detection result is memoized for the tool-server process. With no certificate at all, the build fails with a prompt to sign into Xcode and mint one (Manage Certificates > + > Apple Development).
Everything else is derived or fixed. The build always uses automatic signing
(an Apple ID signed into Xcode is needed so it can mint the provisioning
profile), and the bundle ids are derived from the team
(com.argent.runner.t<teamid> plus its .uitests sibling), which makes them
unique per team by construction. The project intentionally hardcodes no team
and only placeholder bundle ids; the tool-server injects the real values on
the xcodebuild command line. The device needs Developer Mode enabled
(Settings > Privacy & Security > Developer Mode), and on first install iOS
asks to trust the developer on the phone: Settings > General > VPN & Device
Management.
The Xcode project itself needs no locating either: it always sits next to the
bundled tool-server. An npm install of @swmansion/argent ships these Swift
sources there (the build step copies ArgentRunner/ next to the bundle), and
in the argent repo npm run build in packages/argent produces the same
layout.
ARGENT_IOS_RUNNER_PROJECT overrides the location for tool-server runs
outside the bundle (ts-node, tests) or unusual layouts.
Reliability model
- Issue suppression:
record(_:)mutes two benign "Failed to get matching snapshot" accessibility issue shapes that would otherwise tear the long-lived test down; hung-query timeouts still record. - Watchdog: command work runs on the main thread under a hard budget;
overruns are abandoned (XCTest work cannot be cancelled) and the runner
answers
RUNNER_BUSY, escalating toRUNNER_WEDGEDwhen stuck. - Send-once: commands are journaled by
commandId(responses are retained for mutating commands only); a client that lost a reply asksstatusfor the command's fate instead of replaying it. Duplicate in-flight sends coalesce onto the single execution. - Explicit targeting: app commands require
appBundleId; the runner never redirects them to its own host app. - Observation without side effects: the read-only
snapshotnever re-fronts a backgrounded target; it answersAPP_BACKGROUNDEDinstead. Mutating commands, and theviewportread that precedes a gesture, re-front a backgrounded target and stamp the replyreactivated: true.