30 Commits

Author SHA1 Message Date
yan5xu 057ab7872c refactor: unified command definitions (commands.ts)
Single source of truth for all command metadata:
- ParamDef with type, required, position, description
- CommandDef with method, group, description, requiresTab, params
- commandToJsonSchema() for Hub registration
- getCommand() / getCommandsByGroup() helpers

Hub registration now uses COMMANDS directly instead of
hardcoded arrays. Removed zod dependency from shared package.

CLI and daemon dispatch unchanged — they can consume COMMANDS
in a future refactor.
2026-05-22 20:24:45 +08:00
yan5xu 8559b772c9 refactor: Phase 2 — new protocol format, command renames, --tab required
Protocol:
- action → method, success/data/error → result/error (mutually exclusive)
- Remove id field from Request/Response
- Error is now {message, hint} object, not plain string
- HTTP method filter renamed: method → httpMethod (avoids conflict)

Command renames:
- snapshot → snap
- refresh → reload
- Remove tab_select (use open --tab)
- Remove tab_close (use close --tab)

--tab required:
- Most commands require --tab, error with hint if missing
- Exceptions: open (optional), site (auto-match domain), tab list/new, daemon

Removed commands:
- fetch (use eval or site run)
- wait (agent polls snap)
- history (not browser control)

Other:
- get supports all 5 attributes (text/url/title/value/html)
- -1400 lines across 36 files
2026-05-22 18:45:35 +08:00
yan5xu c7bd48a900 feat: move site adapter execution from CLI to daemon (Phase 3)
- New: packages/daemon/src/site-adapter.ts — site adapter scanning,
  @meta parsing, domain-based tab matching, and JS evaluation via CDP
- Daemon: add site_run, site_list, site_info, site_search actions
- CLI: site commands now send HTTP to daemon instead of executing locally
- Provider: executeSiteCommand uses daemon site_run instead of shelling
  out to CLI subprocess (removes execFile dependency)
- Protocol: add siteName, siteArgs, query fields to Request type

Site adapter execution is now a single path: CLI/Provider → daemon
→ CDP evaluate. No more three-layer process nesting.
2026-05-22 17:44:58 +08:00
yan5xu 7bb8cd1624 refactor: Phase 1 cleanup — remove MCP, dead code, unify CDP port
- Remove packages/mcp/ entirely (Agent uses CLI or Hub invoke)
- Remove dead code: reload.ts (never routed)
- Unify default CDP port to 9222 (daemon was 19825, viewer was 9222)
- Remove --mcp CLI flag
- Update comments to reflect MCP removal
2026-05-22 17:16:42 +08:00
Yanwu 4815d776c1 feat: remote browser viewing via Clip Web UI (#220)
* feat(daemon): expose CDP host/port in daemon.json and /status

- Add cdpHost, cdpPort to DaemonInfo (optional in shared, required in daemon)
- Write cdpHost/cdpPort from discovered cdpEndpoint to daemon.json
- Return cdpHost/cdpPort in GET /status response
- Enables external tools (bb-viewer) to discover the Chrome CDP connection

* feat(provider): add view.start/answer/close commands + bb-viewer sidecar management

- Register view.start, view.answer, view.close commands in browser clip
- Viewer sidecar: auto-spawn bb-viewer on first view.start, kill on exit
- Read CDP info from daemon.json (cdpHost/cdpPort) for viewer config
- TURN config from env vars: TURN_URL, TURN_USER, TURN_CRED
- bb-viewer binary lookup: ~/.bb-browser/bin/bb-viewer or PATH fallback
- Sidecar health check with 10s startup timeout

* feat(provider): add Clip Web UI for remote browser viewing

- web/view.html: remote viewer page using @pinixai/core/web invoke()
  - WebRTC video with playoutDelayHint=0 for low latency
  - DataChannel binary input: mouse, keyboard, scroll, IME
  - X11 keysym mapping, composition handling
  - Connect/Disconnect with auto-cleanup
- provider: set hasWeb=true for browser clip
- provider: handle GetClipWebCommand to serve web/ files
  - MIME type detection, ETag, range request support
  - Path security validation

* feat(provider): TURN ephemeral credentials via TURN REST API

- Generate time-limited TURN credentials using HMAC-SHA1 shared secret
- Credentials valid for 24h, auto-generated at viewer sidecar spawn
- Config: TURN_URL + TURN_SECRET env vars (no user-facing config needed)
- Replaces static TURN_USER/TURN_CRED with secure ephemeral credentials

* fix(web): replace esm.sh import with direct /api/ fetch

@pinixai/core is a private package, not available on esm.sh.
Use direct fetch to /api/{command} which the Hub's subdomain
handler proxies to provider invoke.
2026-05-21 18:01:49 +08:00
yan5xu 1938ce46b1 fix(snapshot): clean up stale highlight overlays before each snapshot and screenshot
Highlights accumulated across multiple snapshot calls because the
cleanup function was commented out. Now buildDomTree clears previous
overlays at the start, and screenshot removes them before capture.

Closes #173

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 15:21:48 +08:00
Yanwu 03c6bf2010 refactor: provider rewrite + cleanup (-1891 lines) (#176)
* refactor: rewrite Edge Clip provider using @pinixai/hub-client (#175)

Replace hand-written protobuf descriptor in bb-browserd.ts with typed
imports from @pinixai/hub-client. The new bb-browser-provider.ts uses
create() from @bufbuild/protobuf with generated schemas, registers site
adapters as separate per-platform clips, and routes invocations to
either the daemon HTTP layer (browser commands) or CLI (site commands).

- Delete bin/bb-browserd.ts (1140 lines)
- Add bin/bb-browser-provider.ts (~710 lines)
- Add @pinixai/hub-client dependency
- Add bb-browser-provider bin entry and tsup entry point
- Add esbuild plugin to resolve hub-client gen subpath import

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: import from @pinixai/hub-client top-level exports

Use hub-client 0.1.2 which exports ProviderStream types directly.
Remove esbuild plugin hack for deep gen/hub_pb import.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: delete dead CDP code, fix MCP auth, extract shared daemon client

- Delete cdp-client.ts (1212 lines), cdp-monitor.ts (521 lines),
  cdp-monitor-state.ts (182 lines) — all dead code from pre-daemon era
- Fix MCP server to read daemon.json for token/port instead of
  hardcoded DAEMON_BASE_URL without auth
- Extract httpJson, readDaemonJson, isProcessAlive, DaemonInfo to
  packages/shared/src/daemon-client.ts — eliminates duplication across
  CLI daemon-manager, MCP server, and Edge Clip provider
- Fix scroll left/right: deltaX was hardcoded to 0, now properly
  maps direction to deltaX/deltaY
- Align daemon tsup target from node20 to node18 (matches engines)
- Remove cdp-monitor from tsup entry points

Net: -1891 lines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: lint — use DAEMON_DIR from shared, remove unused os import

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review feedback — refresh platformClipAliases on reconnect, remove dead DAEMON_BASE_URL

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove duplicate bun shebang from provider source

tsup adds #!/usr/bin/env node via banner config. The source file's
#!/usr/bin/env bun caused a double shebang in dist/provider.js,
breaking node execution.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 06:04:43 +08:00
Yanwu fd0d7956e2 feat: unified command registry for CLI/MCP/Edge Clip (#171)
* feat: add unified command registry (shared/commands.ts)

Single source of truth for all 30+ commands. CLI/MCP/Edge Clip
will auto-generate interfaces from this registry.

Part of #170

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: complete command registry (history args, fetch, site commands)

- Add missing `query` and `days` args to `history` command
- Add 6 site commands: site_list, site_search, site_info, site_recommend,
  site_run, site_update (category: "site")
- Verified `tab_new` is a real dispatch case (handled before ensurePageTarget)
- Verified `get` attribute enum is complete (text, url, title, value, html)
- Confirmed `fetch` is CLI-only (composes eval), not a daemon action — skip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: use Zod schemas in command registry

Replace custom CommandArg interface with Zod schemas for runtime
validation, JSON Schema generation, and zero-conversion MCP integration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: MCP + Edge Clip auto-generate from command registry

MCP: 25 → 37 tools, auto-generated from COMMANDS registry.
13 new tools: back, forward, refresh, check, uncheck, select,
dialog, frame, frame_main, tab_select, tab_close, trace, history.
Special handlers preserved for snapshot/screenshot/eval/network etc.

Edge Clip: 7 → 37 commands. Removed all direct CDP code (~400 lines).
Now routes through daemon HTTP API like CLI and MCP.
Uses inline zod-to-json-schema converter for Hub registration.

Adding a new command = 1 registry entry + 1 dispatch case.
All three interfaces (CLI/MCP/Edge Clip) get it automatically.

Part of #170

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review issues (since type, MCP compat, description loss, scroll default)

- Fix since field type: use z.union([z.literal("last_action"), z.number()])
  instead of z.string() for network/console/errors commands
- Add MCP backward compat: accept old arg names (time, command) alongside
  new ones (ms, networkCommand, consoleCommand, errorsCommand)
- Fix bb-browserd ZodOptional/ZodDefault unwrap losing outer description
- Fix typo: executeViaDeamon → executeViaDaemon
- Remove unnecessary tab arg from history command (daemon mode unsupported)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prefix unused cmd params with underscore for lint

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: route Edge Clip site commands through CLI instead of daemon

Site commands (site_list, site_run, etc.) are handled by CLI, not
daemon. Edge Clip now routes them via CLI execFile, matching MCP's
approach. Other commands still go through daemon HTTP API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: Edge Clip site commands use installed bb-browser CLI, not source

bun running CLI source fails because __BB_BROWSER_VERSION__ is a tsup
define not available at runtime. Use installed bb-browser binary with
fallback to dist/cli.js.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 22:16:28 +08:00
yan5xu 7936566ec5 feat!: remove Chrome Extension package
Move buildDomTree.js to packages/shared/. Delete packages/extension/
entirely. Update path references in daemon and CLI. Remove
build:extension script and release-please manifest sync.

The daemon connects directly to Chrome via CDP WebSocket since v0.11.0.
The Extension is no longer needed.

BREAKING CHANGE: Chrome Extension is removed. Users who relied on the
extension-based flow should use the daemon-direct mode instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 05:33:17 +08:00
Xeon 37f8d64959 fix: align daemon host defaults and restore daemon command (#137) 2026-04-02 05:29:36 +08:00
Yanwu fb1a9b0644 chore: add harness — tests, CI, protocol types, design invariants (#133)
* chore: add harness — tests, CI, protocol types, design invariants

Context layer:
- Rewrite AGENTS.md: new daemon architecture, 3-file command pattern,
  7 design invariants (INV-1 through INV-7), testing section
- Update protocol.ts: add tab/seq/cursor/since fields, update
  DaemonStatus, remove SSE types, widen TabInfo.tabId to string

Constraints layer:
- Unit tests: RingBuffer (14), TabState + TabStateManager (36)
- Contract tests: all 7 invariants verified (48)
- Protocol drift tests: real daemon HTTP response shape checks (13,
  skipped when Chrome unavailable)
- CI workflow: build + lint + test on push/PR to main
- Pre-commit hook: pnpm build && pnpm test via husky

GC layer:
- Harness scenarios doc: 6 reusable Agent workflow scenarios for
  periodic UX validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve lint errors caught by CI

- Remove unused imports (TabState, TabInfo, JsonObject, Response)
- Remove unused variable assignments in tests (r1-r4)
- Change `let items` to `const items` in command-dispatch
- Remove stale eslint-disable directive, use proper cast
- Exclude .d.ts files from eslint in daemon package

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Codex review findings

- cursor returns since threshold instead of 0 on empty results,
  preventing infinite re-fetch in incremental polling loops
- Guard shutdown against double-call (SIGINT + /shutdown race)
- Fix timer leak in discoverCdpPort when fetch fails

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update READMEs — remove Extension references, update architecture diagram

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 15:47:10 +08:00
Yanwu 62af1f89d7 refactor: unify daemon — CDP direct, remove Extension dependency (#132)
Merge daemon + cdp-monitor into a single CDP-direct backend. All CLI
and MCP commands now route through the daemon HTTP API instead of
direct CDP connections or Extension relay.

Key changes:
- Daemon: rewrite to hold persistent CDP WebSocket, handle all 30+
  commands via dispatchRequest, per-tab event state with RingBuffer,
  short tab IDs (4-char hex), global seq counter, incremental queries
  via since/cursor mechanism
- CLI: all commands route through daemon HTTP, --tab accepts string
  short IDs, add --since/--method/--status flags for network queries,
  fix tab close with --tab flag, fix invalid tab ID silent fallback
- MCP: tab parameter z.number() → z.string(), add browser_console
  and browser_errors tools, add since/method/status/limit params to
  browser_network, remove Extension dependency hints
- Shared: export additional types for daemon consumption

Bug fixes from harness testing:
- Invalid tab ID now errors instead of silent fallback
- tab_new works with zero existing tabs
- console.warn() correctly captured as type "warn"
- --tab flag works with tab close command

Closes #131

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 11:02:48 +08:00
Ted Li 45c1fbb385 fix(shared): avoid global crypto for request ids (#89) 2026-03-22 14:36:05 +08:00
Waffle 0a2e150238 chore(lint): add small-scope eslint checks 2026-03-19 16:37:41 +08:00
yan5xu fdb8288ce2 feat: add history, site recommend, site info, --jq, and Agent UX guide
New commands:
- `history search/domains` — query Chrome browsing history via chrome.history API
- `site recommend` — cross-reference history with adapters for personalized discovery
- `site info <name>` — expose full adapter @meta (args, example, domain)
- `--jq <expr>` — inline jq-like JSON filtering (inspired by gh CLI)

Agent UX improvements:
- --help grouped by intent, "开始使用" section first with site examples
- --help top hint explicitly guides Agents to use site commands over browser automation
- `open` command shows 💡 hint when the URL has available site adapters
- `--jq` auto-strips `.data.` prefix for Agent tolerance
- `site update` post-install nudge for site recommend
- AGENTS.md with full UX writing spec for contributors

Closes #21, closes #23, relates #24

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 12:02:04 +08:00
yan5xu 537e5536f9 feat(mcp): add MCP server for AI agent integration
New package @bb-browser/mcp exposes all browser commands as MCP tools:
browser_snapshot, browser_click, browser_fill, browser_type, browser_open,
browser_tab_list, browser_tab_new, browser_press, browser_scroll,
browser_eval, browser_network, browser_screenshot, browser_get,
browser_close, browser_hover, browser_wait

All tools support `tab` parameter for concurrent multi-tab operations.
Uses stdio transport, connects to daemon via HTTP (same as CLI).

Also adds missing fields to shared protocol types:
- Request: key, modifiers, direction, pixels, waitType, ms
- ResponseData: dataUrl

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 21:59:22 +08:00
yan5xu a2e1f29daa feat: network requests --with-body captures full request/response data
Via CDP Network.loadingFinished + Network.getResponseBody, capture:
- Request headers and body (POST data)
- Response headers, body, and mimeType
- Memory management: 256KB/response, 64KB/request, 8MB/tab total
- Eviction: oldest bodies removed first when over budget
- Graceful error handling: bodyError field for failed captures

This enables a new paradigm for site adapters: instead of reverse-engineering
API signing (like Xiaohongshu's X-s), just navigate to a page and read the
response bodies from requests the page already made with proper authentication.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 20:44:22 +08:00
yan5xu e66256a138 feat: 切换 snapshot 到 CDP Accessibility Tree
- 新增 ax-tree-formatter.ts:将 CDP AXNode[] 格式化为语义化 accessibility 树文本
- cdp-dom-service.ts:getSnapshot 改用 Accessibility.getFullAXTree,元素操作改用 backendDOMNodeId
- 支持新参数:--compact/-c(移除空结构节点)、--depth/-d(限制树深度)、--selector/-s(CSS 范围)
- link 内联 /url:,heading 标记 [level=N],重复元素标记 [nth=N]
- GitHub 页面 snapshot 从 56KB 降到 31KB(-45%)

Closes #4

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-13 19:55:34 +08:00
yan5xu 1acfb6031b feat: tab select/close 支持按 tabId 操作
- 协议 Request 新增 tabId 字段(优先于 index,向下兼容)
- Extension handleTabSelect/handleTabClose 支持 tabId 直接定位
- CLI 新增 tab select --id <tabId> 和 tab close --id <tabId> 语法
- 解决多调用方共用时 index 重编号导致的 tab 错位问题
2026-02-06 09:35:33 +08:00
yan5xu 4c7c64f8ba fix: 解决 MV3 Service Worker 休眠导致连接断开的问题
- 添加 chrome.alarms 保活机制(每 24 秒唤醒)
- 使用 chrome.storage.session 持久化 refs(防止休眠后丢失)
- 心跳间隔从 30s 改为 15s
- 添加 storage 和 alarms 权限
2026-01-30 18:12:42 +08:00
yan5xu f02b081f48 feat: 添加 trace 命令 - 录制用户操作 2026-01-30 15:58:09 +08:00
yan5xu 02edcb4c85 feat: 添加 network/console/errors 调试命令
- network requests [filter]: 查看网络请求
- network route <url> [--abort|--body]: 拦截/mock 请求
- network unroute [url]: 移除拦截规则
- network clear: 清空请求记录
- console: 查看控制台消息(log/info/warn/error)
- console --clear: 清空控制台
- errors: 查看 JS 异常
- errors --clear: 清空错误记录

扩展通过 CDP Network/Fetch/Runtime/Log 域实现监控和拦截
2026-01-30 12:34:33 +08:00
yan5xu 0052ec5c31 merge: feat/dialogs - 添加 dialog 命令 2026-01-30 10:36:55 +08:00
yan5xu fb726aec3c merge: feat/frames - 添加 frame 命令 2026-01-30 10:32:37 +08:00
yan5xu 394d7e7dc3 feat: 添加 frame 命令(切换 iframe) 2026-01-30 10:28:48 +08:00
yan5xu 8d6e6c1d75 feat: 添加 dialog 命令(accept/dismiss) 2026-01-30 10:26:52 +08:00
yan5xu a608124f06 feat: 添加 tab 命令(list/new/select/close) 2026-01-30 10:26:31 +08:00
yan5xu bbfda3fc62 feat: 添加 Phase 3 命令 (eval/type/check/uncheck/select) 2026-01-30 10:02:53 +08:00
yan5xu 7e6bf96aba feat: 添加 eval 命令 - 执行 JavaScript 2026-01-30 09:58:22 +08:00
yan5xu 601f4ae813 feat: 实现 bb-browser 核心功能
- CLI: open, snapshot, click, fill, status, stop 命令
- Extension: buildDomTree 移植, DOM 服务, SSE 通信
- Daemon: HTTP SSE 架构, 请求管理
- Shared: 通信协议类型定义

snapshot 支持两种模式:
- 默认完整树模式 (含文本节点)
- -i 交互元素模式 (只含可交互元素)

已验证: 推特/Reddit 登录态页面访问正常
2026-01-29 21:17:28 +08:00