callstack__agent-device/docs/agent-setup.md

# AI Agent Setup

`agent-device` is built for AI agents, but humans usually install it, grant device permissions, and decide which agent client should use it.

Use this page to wire Cursor, Codex, Claude Code, Windsurf, Cline, Goose, or another coding agent into mobile, TV, desktop, and web app verification. It covers skills, project rules, and MCP setup for React Native QA, Expo app verification, iOS Simulator automation, Android Emulator automation, tvOS checks, Android TV checks, Vega OS VVD control, web browser sessions, debugging, profiling, and exploratory QA.

The short version: install the CLI, let the agent start normal work with the requested app, and use version-matched help only for specialized work or an unclear command shape. MCP tools use command contracts backed by the same `AgentDeviceClient` execution path as the CLI adapters.

## Prerequisite: install the CLI

```bash
npm install -g agent-device@latest
agent-device --version
agent-device help workflow
```

For one-off human use without a global install:

```bash
npx agent-device --version
npx agent-device help workflow
```

Global install is better for normal agent workflows because repeated commands and terminal sessions resolve to one stable version. Project-local installs are also good when you want a lockfile-pinned agent-device version.

Avoid telling agents to choose an npm version or run `npx -y agent-device@latest` autonomously: it fetches and executes a mutable npm package without a human prompt. For unattended agent use, prefer a trusted installed binary, a project-local install, or a version supplied by the user or project config.

For Node, Xcode, Android SDK, macOS, and iOS device prerequisites, see [Installation](/agent-device/docs/installation.md).

## Install the skills

Install the CLI first, then install the repository skills when your agent runtime supports them:

```bash
npx skills add callstack/agent-device
```

Skills are distributed from the GitHub repository rather than the npm package. The [agent-device skill](https://github.com/callstack/agent-device/blob/main/skills/agent-device/SKILL.md) is the canonical router for skill-aware clients. For focused simulator work, use the [iOS Simulator skill](https://github.com/callstack/agent-device/blob/main/skills/ios-simulator/SKILL.md) or [Android Emulator skill](https://github.com/callstack/agent-device/blob/main/skills/android-emulator/SKILL.md). They start normal work directly and route agents to the separately installed, version-matched CLI help only when the task is specialized or a command shape is unclear.

## Recommended agent rule

Add this as a project rule, custom instruction, or skill equivalent when your agent client supports it:

```text
Use agent-device only for app/device automation tasks. For a normal app-driving task, start immediately. Do not probe first with `--help`, `--version`, `devices`, `appstate`, `snapshot`, or `screenshot`; open the requested app in the foreground and continue from its initial interactive snapshot. For TV, Fire TV, or Vega OS tasks, read `agent-device help tv`. For exploratory QA, read `agent-device help dogfood`. For logs, network, audio, traces, or runtime failures, read `agent-device help debugging`. For React Native component trees, props/state/hooks, slow renders, or rerenders, read `agent-device help react-devtools`. For React Native JavaScript heap growth, heap snapshots, allocation hotspots, or retained-object leaks, read `agent-device help cdp`. For React Native apps, overlays, Metro/Fast Refresh blockers, and routing to React DevTools or debugging evidence, read `agent-device help react-native`.

Use MCP tools or the CLI in the integrated terminal. If `agent-device` is not on PATH but the user installed it globally in another shell, resolve the command the same way the user would from a normal terminal session and run that absolute path instead. This may require inspecting shell startup behavior or package-manager/global bin locations; do not assume the agent process `PATH` is the user's `PATH`. Do not silently fall back to `npx -y agent-device@latest`; ask or use an exact version. MCP exposes structured tools backed by the agent-device client; it does not expose generic shell execution. Prefer `open -> snapshot -i -> act -> re-snapshot -> verify -> close` where the target supports capture and selectors; otherwise follow target-specific help. Use current refs such as `@e3` for exploration and selectors for durable replay. Keep mutating commands against one session serial. Capture screenshots, logs, network, audio, perf, traces, recordings, and `.ad` replay scripts only when they add evidence.
```

## MCP server

`agent-device mcp` starts the official stdio MCP server. It exposes direct structured tools for installed CLI commands. Tools run through command contracts and `AgentDeviceClient`; local-only workflows stay CLI-only rather than subprocess fallbacks.

For web automation, MCP tools can target `platform: "web"` after the managed backend is available, but `agent-device web setup` and `agent-device web doctor` are CLI-only. Run setup from a terminal in the same effective state directory before asking an MCP client to drive a browser session.

The server also ships its own guidance, so MCP-only clients do not depend on a separately installed skill: the handshake (`initialize` and `server/discover`) returns compact `instructions` with the workflow card (start with `open`, act with `settle: true`, verify, `close`, ref fidelity, sparse-AX recovery), and an MCP-only `help` tool returns the same guides as `agent-device help <topic|command>` on demand. `help` is not a startup step; agents call it for specialized work (gestures, scripting, TV, macOS, web, remote, debugging) or an unclear command shape.

Tool execution failures are returned as MCP tool results with `isError: true`; clients and agents should inspect the tool result, not only the successful JSON-RPC envelope.

MCP clients must not use this server as a generic shell runner. If the CLI is missing, agents should ask a human before installing or updating packages, reconnect the server after setup, and retry the intended app-driving command without adding version/help probes.

Global install configuration:

```json
{
  "mcpServers": {
    "agent-device": {
      "command": "agent-device",
      "args": ["mcp"]
    }
  }
}
```

No global install variant. Pin a user- or project-selected package version for unattended agent use:

```json
{
  "mcpServers": {
    "agent-device": {
      "command": "npx",
      "args": ["-y", "agent-device@<reviewed-version>", "mcp"]
    }
  }
}
```

Registry metadata uses MCP name `io.github.callstack/agent-device`, npm package `agent-device`, stdio transport, `mcpName` package verification, `server.json`, `glama.json`, and `smithery.yaml`. Glama lists the server at [callstack/agent-device](https://glama.ai/mcp/servers/callstack/agent-device).

## Cursor

Cursor works well with either the plain CLI or MCP tools. Use the CLI path when you want the most auditable setup and terminal-visible commands. Add MCP when you want Cursor Agent to discover structured `agent-device` tools directly from chat.

### Cursor path A: CLI only

Create a project rule:

```bash
mkdir -p .cursor/rules
cat > .cursor/rules/agent-device.mdc <<'EOF'
---
description: Use agent-device for app and device automation
alwaysApply: true
---

Use agent-device only for app/device automation tasks.
For a normal app-driving task, start immediately. Do not probe first with `--help`, `--version`, `devices`, `appstate`, `snapshot`, or `screenshot`; open the requested app in the foreground and continue from its initial interactive snapshot.
For TV, Fire TV, or Vega OS tasks, read `agent-device help tv`.
For exploratory QA, read `agent-device help dogfood`.
For logs, network, audio, traces, or runtime failures, read `agent-device help debugging`.
For React Native component trees, props/state/hooks, slow renders, or rerenders, read `agent-device help react-devtools`.
For React Native JavaScript heap growth, heap snapshots, or retained-object leaks, read `agent-device help cdp`.
For React Native apps, overlays, Metro/Fast Refresh blockers, and routing to React DevTools or debugging evidence, read `agent-device help react-native`.

Use the CLI in Cursor's integrated terminal.
If `agent-device` is not on PATH but the user installed it globally in another shell, resolve the absolute binary path instead of using `npx -y agent-device@latest`.
Prefer `open -> snapshot -i -> act -> re-snapshot -> verify -> close` where supported; otherwise follow target-specific help.
Keep mutating commands against one session serial.
EOF
```

Then ask Cursor Agent to run:

```bash
agent-device open <app-or-url> --platform ios --foreground
```

### Cursor path B: MCP tools

Create project MCP config:

```bash
mkdir -p .cursor
cat > .cursor/mcp.json <<'JSON'
{
  "mcpServers": {
    "agent-device": {
      "command": "agent-device",
      "args": ["mcp"]
    }
  }
}
JSON
```

Restart Cursor or reconnect MCP from Cursor settings, then ask Cursor Agent:

```text
Use the agent-device MCP tools to inspect the iOS app. Open the app, take an interactive snapshot, act on visible refs/selectors, verify with another snapshot, and close the session.
```

If the MCP server fails because Cursor cannot find the global binary, use the absolute binary path in `.cursor/mcp.json`:

```json
{
  "mcpServers": {
    "agent-device": {
      "command": "/absolute/path/to/agent-device",
      "args": ["mcp"]
    }
  }
}
```

## Codex

Put the recommended rule in `AGENTS.md` or the project instructions. Let Codex run `agent-device` in the terminal:

```bash
agent-device open <app-or-url> --platform ios --foreground
```

Some agent clients run commands in an environment that differs from the user's normal install shell. If the user installed `agent-device` globally but the agent cannot find it, resolve the command the same way the user would from a normal terminal session, then use the absolute binary path for the intended `open` and subsequent commands. This may require inspecting shell startup behavior or package-manager/global bin locations; do not assume the agent process `PATH` is the user's `PATH`.

For reviews or planning-only tasks, tell the agent not to run devices unless explicitly requested.

## Claude Code

Claude Code works through the terminal CLI and through the VS Code extension panel. The VS Code extension can use MCP servers configured by the Claude CLI and managed with `/mcp`.

### Claude path A: CLI only

Put this in `CLAUDE.md`:

```bash
cat > CLAUDE.md <<'EOF'
# agent-device

Use agent-device only for app/device automation tasks.
For a normal app-driving task, start immediately. Do not probe first with `--help`, `--version`, `devices`, `appstate`, `snapshot`, or `screenshot`; open the requested app in the foreground and continue from its initial interactive snapshot.
For TV, Fire TV, or Vega OS tasks, read `agent-device help tv`.
For exploratory QA, read `agent-device help dogfood`.
For logs, network, audio, traces, or runtime failures, read `agent-device help debugging`.
For React Native component trees, props/state/hooks, slow renders, or rerenders, read `agent-device help react-devtools`.
For React Native JavaScript heap growth, heap snapshots, or retained-object leaks, read `agent-device help cdp`.
For React Native apps, overlays, Metro/Fast Refresh blockers, and routing to React DevTools or debugging evidence, read `agent-device help react-native`.

Use the CLI in the integrated terminal.
If `agent-device` is not on PATH but the user installed it globally in another shell, resolve the absolute binary path instead of using `npx -y agent-device@latest`.
Prefer `open -> snapshot -i -> act -> re-snapshot -> verify -> close` where supported; otherwise follow target-specific help.
Keep mutating commands against one session serial.
EOF
```

Then ask Claude Code to run:

```bash
agent-device open <app-or-url> --platform android --foreground
```

### Claude path B: MCP tools

Add a user-scoped server:

```bash
claude mcp add --transport stdio --scope user agent-device -- agent-device mcp
claude mcp list
```

Or add it to the current project so teammates can review the generated `.mcp.json`:

```bash
claude mcp add --transport stdio --scope project agent-device -- agent-device mcp
```

In Claude Code or the VS Code extension, run:

```text
/mcp
```

Confirm `agent-device` is connected, then ask:

```text
Use the agent-device MCP tools to verify the app. Open the app, take an interactive snapshot, use refs/selectors for actions, verify with another snapshot, and close the session.
```

If Claude cannot start the MCP server because the extension process cannot find the global binary, remove and re-add it with an absolute path:

```bash
claude mcp remove agent-device
claude mcp add --transport stdio --scope user agent-device -- /absolute/path/to/agent-device mcp
```

The same CLI commands remain available in the integrated terminal for long-running or manual workflows.

## Windsurf, Cline, Goose, and other MCP clients

Use the [MCP server](#mcp-server) configuration when the client supports `mcpServers`, then tell the agent to use MCP tools or terminal CLI commands for device workflows.

If the client has project rules or custom instructions, add the recommended agent rule above. If it does not, ask the agent to open the requested app and continue from the initial interactive snapshot; introduce a help topic only when the task is specialized or a command shape is unclear.

## Why this setup works

The CLI stays the auditable automation surface, installed help stays version-matched with the commands, skills and rules route agents toward the right help topics, and MCP gives compatible clients direct structured tools backed by the same daemon/client implementation.

For the broader positioning, supported targets, observability features, and how `agent-device` differs from scripted test frameworks, see [Introduction](/agent-device/docs/introduction.md). For exact command groups and platform behavior, see [Commands](/agent-device/docs/commands.md).

For the local execution model, permissions, artifacts, and sensitive data guidance, see [Security & Trust](/agent-device/docs/security-trust.md).

## Agent-readable docs

Use [llms-full.txt](https://oss.callstack.com/agent-device/llms-full.txt) when an agent needs a single text bundle of the current docs. The installed CLI remains authoritative for exact command syntax:

```bash
agent-device help
agent-device help workflow
agent-device help dogfood
```