mirror of
https://github.com/actionbook/actionbook.git
synced 2026-09-19 03:28:06 +08:00
0949311c2e
Address Codex review feedback on PR #235 — extension-mode users would hit runtime errors copying examples for wait-idle, console, fetch, emulate, and switch-frame. Add [CDP] inline markers and section-level <sup>CDP</sup> tags so the limitation is visible before users try. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
446 lines
16 KiB
Plaintext
446 lines
16 KiB
Plaintext
---
|
|
title: "Browser Automation"
|
|
description: "Control browsers with Actionbook CLI — modes, commands, multi-session, and security"
|
|
---
|
|
|
|
The `actionbook browser` command lets AI agents automate any Chromium-based browser. Two modes are available depending on the use case:
|
|
|
|
<Note>
|
|
Normal usage: run `actionbook setup` once, then use `actionbook browser ...` commands only.
|
|
</Note>
|
|
|
|
## Modes
|
|
|
|
| Mode | How It Works | Best For |
|
|
|------|-------------|----------|
|
|
| **Isolated** *(default)* | Launches a new Chrome with a fresh, dedicated profile | Automated tasks, CI/CD, headless, no user data needed |
|
|
| **Extension** | Connects to the user's running Chrome via extension bridge | Tasks requiring existing login sessions, cookies, personal context |
|
|
|
|
Both modes are designed for AI agents. **Isolated** gives agents a clean, reproducible environment. **Extension** lets agents operate inside the user's real browser — ideal when the task requires existing authentication (e.g., "book my flight", "reply to that email").
|
|
|
|
## Configuration
|
|
|
|
The recommended way to set your preferred browser mode is via `actionbook setup`.
|
|
It writes your choice to `~/.actionbook/config.toml`, so you can run browser commands without extra flags.
|
|
|
|
### Set Default Mode
|
|
|
|
Create or edit `~/.actionbook/config.toml`:
|
|
|
|
```toml
|
|
[browser]
|
|
# Use "isolated" (default) or "extension"
|
|
mode = "extension"
|
|
|
|
[browser.extension]
|
|
# Auto-install local debug fallback when Web Store install is unavailable
|
|
auto_install = true
|
|
```
|
|
|
|
Once configured, all browser commands will automatically use your chosen mode:
|
|
|
|
```bash
|
|
# No flags needed - uses configured mode
|
|
actionbook browser open "https://example.com"
|
|
actionbook browser click "button[type='submit']"
|
|
```
|
|
|
|
<Note>
|
|
In current extension mode, bridge communication uses fixed `127.0.0.1:19222`.
|
|
</Note>
|
|
|
|
## Isolated Mode
|
|
|
|
Works out of the box — no setup required. The CLI launches a temporary Chrome process with a dedicated profile and connects via CDP.
|
|
|
|
```bash
|
|
# If isolated is your default mode (or not configured)
|
|
actionbook browser open "https://example.com"
|
|
actionbook browser click "button[type='submit']"
|
|
actionbook browser fill "actionbook" "#search"
|
|
actionbook browser screenshot output.png
|
|
```
|
|
|
|
When done, close the browser:
|
|
|
|
```bash
|
|
actionbook browser close
|
|
```
|
|
|
|
## Extension Mode Setup
|
|
|
|
If you've configured `mode = "extension"` in your config file, you only need to perform this one-time setup.
|
|
|
|
<Steps>
|
|
<Step title="Install from Chrome Web Store (recommended)">
|
|
1. Open [Actionbook on Chrome Web Store](https://chromewebstore.google.com/detail/actionbook/bebchpafpemheedhcdabookaifcijmfo)
|
|
2. Click **Add to Chrome**
|
|
3. Confirm **Add extension**
|
|
</Step>
|
|
|
|
<Step title="Verify with any browser command">
|
|
```bash
|
|
actionbook browser open "https://example.com"
|
|
actionbook browser snapshot
|
|
```
|
|
|
|
The bridge auto-starts in the background and the extension auto-connects within a few seconds.
|
|
</Step>
|
|
|
|
<Step title="Fallback: local debug install (only if Web Store install fails)">
|
|
1. Install the fallback package:
|
|
|
|
```bash
|
|
actionbook extension install
|
|
```
|
|
|
|
2. Open `chrome://extensions`
|
|
3. Enable **Developer mode**
|
|
4. Click **Load unpacked**
|
|
5. Select the path from:
|
|
|
|
```bash
|
|
actionbook extension path
|
|
```
|
|
</Step>
|
|
</Steps>
|
|
|
|
<Tip>
|
|
**Upgrading from v0.2.0 or earlier?**
|
|
- The extension path moved from `~/.config/actionbook/extension/` to `~/.actionbook/extension/` — files migrate automatically on first use
|
|
- Token-based pairing is no longer needed — the extension auto-connects when the bridge is running
|
|
- The bridge auto-starts with browser commands — you no longer need to run `actionbook extension serve` manually
|
|
</Tip>
|
|
|
|
## Browser Commands
|
|
|
|
Most commands work in both modes. Commands marked with **<sup>CDP</sup>** require isolated mode (CDP) and are not available in extension mode.
|
|
|
|
### Navigation
|
|
|
|
```bash
|
|
actionbook browser open "https://example.com"
|
|
actionbook browser goto "https://example.com/login"
|
|
actionbook browser back
|
|
actionbook browser forward
|
|
actionbook browser reload
|
|
actionbook browser wait-nav # wait for navigation to complete
|
|
```
|
|
|
|
### Page Interaction
|
|
|
|
```bash
|
|
actionbook browser click "button[type='submit']"
|
|
actionbook browser fill "demo" "#username"
|
|
actionbook browser type "hello" "#search" # append text (does not clear)
|
|
actionbook browser select "#country" "US"
|
|
actionbook browser hover ".menu-item"
|
|
actionbook browser focus "#search-input"
|
|
actionbook browser press "Enter"
|
|
actionbook browser hotkey "Control+A" # keyboard shortcuts
|
|
actionbook browser scroll down 500 # scroll by pixels
|
|
actionbook browser wait "#username" --timeout 10000
|
|
actionbook browser wait-idle # wait for network idle [CDP]
|
|
actionbook browser wait-fn "document.readyState === 'complete'"
|
|
```
|
|
|
|
<Tip>
|
|
Use `--ref` with `click`, `type`, or `fill` to target elements by snapshot ref (e.g., `--ref e5`) instead of CSS selectors.
|
|
Add `--human` to `click` or `type` for human-like mouse movement and typing delays.
|
|
</Tip>
|
|
|
|
### Debug & Export
|
|
|
|
```bash
|
|
actionbook browser screenshot output.png --full-page
|
|
actionbook browser pdf report.pdf
|
|
actionbook browser eval "document.title"
|
|
actionbook browser snapshot # accessibility tree
|
|
actionbook browser snapshot --interactive # only interactive elements
|
|
actionbook browser snapshot --diff # show changes since last snapshot
|
|
actionbook browser html
|
|
actionbook browser text
|
|
actionbook browser text --mode readability # smart content extraction
|
|
actionbook browser console --level error # capture console logs [CDP]
|
|
actionbook browser info "#submit" # element bounding box & attributes
|
|
actionbook browser viewport # get viewport dimensions
|
|
actionbook browser inspect 100 200 # inspect element at coordinates
|
|
```
|
|
|
|
### Cookie & Storage Operations
|
|
|
|
```bash
|
|
# Cookies
|
|
actionbook browser cookies list
|
|
actionbook browser cookies get session_id
|
|
actionbook browser cookies set demo_cookie hello
|
|
actionbook browser cookies delete demo_cookie
|
|
actionbook browser cookies clear --domain example.com --yes
|
|
|
|
# Local/Session Storage
|
|
actionbook browser storage list
|
|
actionbook browser storage get myKey
|
|
actionbook browser storage set myKey "myValue"
|
|
actionbook browser storage remove myKey
|
|
actionbook browser storage clear
|
|
```
|
|
|
|
### File Upload
|
|
|
|
```bash
|
|
actionbook browser upload photo.jpg # auto-detect file input
|
|
actionbook browser upload doc.pdf -s "input[type=file]" # specific selector
|
|
actionbook browser upload a.png b.png # multiple files
|
|
```
|
|
|
|
### Fetch (One-Shot) <sup>CDP</sup>
|
|
|
|
Fetch page content in a single command — opens a temporary browser, extracts content, then closes:
|
|
|
|
```bash
|
|
actionbook browser fetch "https://example.com" # text output
|
|
actionbook browser fetch "https://example.com" --format html # full HTML
|
|
actionbook browser fetch "https://example.com" --lite # HTTP-first, browser fallback
|
|
actionbook browser fetch "https://example.com" --max-tokens 4000 # truncate for LLM context
|
|
```
|
|
|
|
### Batch Actions
|
|
|
|
Execute multiple actions from a JSON file for complex workflows:
|
|
|
|
```bash
|
|
actionbook browser batch --file actions.json --delay 100
|
|
```
|
|
|
|
### Advanced Features
|
|
|
|
```bash
|
|
# IFrame context switching [CDP]
|
|
actionbook browser switch-frame "iframe#editor" # switch into iframe
|
|
actionbook browser switch-frame parent # back to parent
|
|
actionbook browser switch-frame default # back to main frame
|
|
|
|
# Device emulation [CDP]
|
|
actionbook browser emulate iphone-14
|
|
actionbook browser emulate pixel-7
|
|
actionbook browser emulate "1920x1080" # custom resolution
|
|
|
|
# Connection & Remote Browsers
|
|
actionbook browser connect 9222 # connect by port
|
|
actionbook browser connect "ws://host:9222/..." # connect by WebSocket URL
|
|
actionbook browser connect "wss://..." -H "authorization:Bearer token"
|
|
|
|
# Fingerprint rotation (stealth)
|
|
actionbook browser fingerprint rotate
|
|
actionbook browser fingerprint rotate --os windows --screen 1920x1080
|
|
|
|
# Browser status
|
|
actionbook browser status # show detection & connection info
|
|
```
|
|
|
|
### Tab Management
|
|
|
|
```bash
|
|
actionbook browser tab list # list all tabs
|
|
actionbook browser tab active # show current tab
|
|
actionbook browser tab new # open a new blank tab
|
|
actionbook browser tab switch <id> # switch to tab by ID
|
|
actionbook browser tab close <id> # close a tab by ID
|
|
```
|
|
|
|
<Tip>
|
|
For running **parallel tasks** across multiple tabs, use [Multi-Session](#multi-session-parallel-tabs) with the `-S` flag instead of manual tab switching.
|
|
</Tip>
|
|
|
|
## Multi-Session (Parallel Tabs)
|
|
|
|
<Note>
|
|
Available since v0.11.0. Requires isolated mode (default).
|
|
</Note>
|
|
|
|
Multi-session lets you run **multiple independent tabs in a single browser instance**. Each named session is bound to its own tab — commands sent to one session never affect another.
|
|
|
|
### Basic Usage
|
|
|
|
Use the `-S` (or `--session`) flag to name a session:
|
|
|
|
```bash
|
|
# Open three sites in separate sessions
|
|
actionbook browser open "https://arxiv.org" -S research
|
|
actionbook browser open "https://x.com" -S social
|
|
actionbook browser open "https://amazon.com" -S shopping
|
|
|
|
# Each session operates independently
|
|
actionbook browser snapshot -S research # only sees arxiv.org
|
|
actionbook browser snapshot -S social # only sees x.com
|
|
actionbook browser click "#search" -S shopping # only affects amazon.com
|
|
```
|
|
|
|
Without `-S`, all commands share a single `default` session.
|
|
|
|
### Session Management
|
|
|
|
```bash
|
|
# List all active sessions
|
|
actionbook browser session list
|
|
|
|
# Show which session is currently active
|
|
actionbook browser session active
|
|
|
|
# Destroy a session (closes its tab)
|
|
actionbook browser session destroy research
|
|
```
|
|
|
|
### Common Mistakes
|
|
|
|
<Warning>
|
|
**Don't create multiple profiles or ports for parallel tabs.** Use `-S` instead.
|
|
</Warning>
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="❌ Creating separate profiles with different ports">
|
|
```bash
|
|
# WRONG — wasteful, launches 3 Chrome instances
|
|
actionbook profile create work --cdp-port 9301
|
|
actionbook profile create work --cdp-port 9302
|
|
actionbook profile create work --cdp-port 9303
|
|
```
|
|
|
|
```bash
|
|
# ✅ CORRECT — one browser, three sessions
|
|
actionbook browser open "https://arxiv.org" -S research
|
|
actionbook browser open "https://x.com" -S social
|
|
actionbook browser open "https://amazon.com" -S shopping
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="❌ Using --browser-mode isolated per site">
|
|
```bash
|
|
# WRONG — each command launches a new isolated Chrome
|
|
actionbook browser open "https://arxiv.org" -P work --browser-mode isolated
|
|
actionbook browser open "https://x.com" -P work --browser-mode isolated
|
|
actionbook browser open "https://amazon.com" -P work --browser-mode isolated
|
|
```
|
|
|
|
```bash
|
|
# ✅ CORRECT — one browser, named sessions for isolation
|
|
actionbook browser open "https://arxiv.org" -P work -S research
|
|
actionbook browser open "https://x.com" -P work -S social
|
|
actionbook browser open "https://amazon.com" -P work -S shopping
|
|
```
|
|
</Accordion>
|
|
|
|
<Accordion title="❌ Running snapshot without session distinction">
|
|
```bash
|
|
# WRONG — all three snapshots see the same (last opened) tab
|
|
actionbook browser snapshot -P work
|
|
actionbook browser snapshot -P work
|
|
actionbook browser snapshot -P work
|
|
```
|
|
|
|
```bash
|
|
# ✅ CORRECT — each snapshot targets a specific session
|
|
actionbook browser snapshot -P work -S research
|
|
actionbook browser snapshot -P work -S social
|
|
actionbook browser snapshot -P work -S shopping
|
|
```
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Recommended Workflow
|
|
|
|
Combine action manuals with browser automation for the best results:
|
|
|
|
```bash
|
|
# 1. Find the action manual
|
|
actionbook search "github search repos"
|
|
|
|
# 2. Get verified selectors
|
|
actionbook get "<area_id>"
|
|
|
|
# 3. Execute (uses your configured mode)
|
|
actionbook browser open "https://github.com"
|
|
actionbook browser fill "actionbook" "#query-builder-test"
|
|
actionbook browser click "button[type='submit']"
|
|
```
|
|
|
|
## Shutdown
|
|
|
|
```bash
|
|
actionbook browser close # close the browser
|
|
actionbook browser restart # restart with same profile/session
|
|
```
|
|
|
|
In extension mode, if you manually started `actionbook extension serve` for debugging, stop it separately with `actionbook extension stop`.
|
|
|
|
## Extension Security
|
|
|
|
Extension mode includes multiple security layers to protect your browser.
|
|
|
|
### Risk Levels
|
|
|
|
| Level | Description | Example Operations |
|
|
|-------|-------------|-------------------|
|
|
| **L1** | Read-only, safe | `eval`, `snapshot`, `screenshot` |
|
|
| **L2** | Page interaction | `click`, `fill`, `select`, navigation |
|
|
| **L3** | Sensitive operations | Cookie writes on banking/payment sites |
|
|
|
|
L3 operations on sensitive domains trigger a confirmation popup in the extension — you must approve them manually.
|
|
|
|
### Localhost Trust Model
|
|
|
|
- The bridge only accepts connections from `127.0.0.1` (loopback interface)
|
|
- No token or pairing is required — local origin is sufficient for authentication
|
|
- The extension identifies itself via a WebSocket handshake with role and version fields
|
|
- The bridge validates the connection source (origin + extension ID) before accepting commands
|
|
|
|
### Network Isolation
|
|
|
|
- The bridge server only listens on `127.0.0.1` (loopback)
|
|
- No external network access is possible
|
|
- Origin validation enforces exact host matching
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Browser command waits for extension connection">
|
|
- Run any browser command once: `actionbook browser open <url>` (this auto-starts the bridge)
|
|
- Keep Chrome open and confirm the Actionbook extension is enabled in `chrome://extensions`
|
|
- First startup can take a few seconds while the bridge starts and the extension reconnects
|
|
</Accordion>
|
|
|
|
<Accordion title="Extension popup shows 'Waiting for bridge' or keeps reconnecting">
|
|
- This usually means the WebSocket handshake did not complete
|
|
- Update CLI: `npm update -g @actionbookdev/cli`
|
|
- Reinstall from [Chrome Web Store](https://chromewebstore.google.com/detail/actionbook/bebchpafpemheedhcdabookaifcijmfo)
|
|
- If needed, use fallback install: `actionbook extension install --force`, then reload it in `chrome://extensions`
|
|
- Click **Connect** in the extension popup to retry
|
|
</Accordion>
|
|
|
|
<Accordion title="Commands not using my configured mode">
|
|
- Re-run `actionbook setup` and choose your preferred browser mode
|
|
- Check effective config with `actionbook config show`
|
|
- Environment variables (for example `ACTIONBOOK_BROWSER_MODE`) can override config
|
|
</Accordion>
|
|
|
|
<Accordion title="No tab attached">
|
|
- Make sure Chrome has at least one visible, active tab
|
|
- Run `actionbook browser open <url>` first to attach a tab
|
|
- Avoid `chrome://` pages — the debugger cannot attach to them
|
|
</Accordion>
|
|
|
|
<Accordion title="Port 19222 in use">
|
|
- Browser mode currently uses a fixed bridge address: `ws://127.0.0.1:19222`
|
|
- Stop the conflicting process on that port (macOS/Linux: `lsof -i :19222`)
|
|
- Then retry your browser command; the bridge will auto-start again
|
|
</Accordion>
|
|
|
|
<Accordion title="Chrome Web Store install failed">
|
|
- Retry Web Store install: [Actionbook on Chrome Web Store](https://chromewebstore.google.com/detail/actionbook/bebchpafpemheedhcdabookaifcijmfo)
|
|
- Use local debug fallback: `actionbook extension install`
|
|
- Load it manually from `chrome://extensions` with **Load unpacked**
|
|
- If GitHub API is rate-limited during fallback, wait and retry or download from [GitHub Releases](https://github.com/actionbook/actionbook/releases)
|
|
</Accordion>
|
|
|
|
</AccordionGroup>
|