254 Commits

Author SHA1 Message Date
yan5xu f3d0c8a4ca chore: release v0.13.0 v0.13.0 2026-05-26 20:34:42 +08:00
yan5xu efd9f0119a feat(streamer): auto-download pre-built bb-viewer binary from COS
- hub-bridge.ts: ensureStreamerBinary() checks local install, PATH,
  then auto-downloads platform-specific binary from COS
- Dockerfile: single-stage build, downloads pre-built bb-viewer and
  Chrome from COS instead of compiling Go in multi-stage

Pre-built binaries uploaded to COS:
  releases/bb-viewer/latest/bb-viewer-darwin-arm64
  releases/bb-viewer/latest/bb-viewer-linux-amd64
  releases/chrome/chrome-linux64.zip
2026-05-26 17:14:30 +08:00
yan5xu 93d1d44ac7 docs: update AGENTS.md for always-headed Chrome 2026-05-26 13:15:32 +08:00
yan5xu 4baa834e57 fix(chrome): always launch headed, drop --headless=new entirely
Headless mode lacks full GUI APIs on Linux and gets flagged by anti-bot
systems. On macOS, Chrome renders off-screen without a visible window
when there is no focused user session, so headed mode works everywhere.

Remove CHROME_HEADED env toggle — headed is now the only mode.
2026-05-26 13:15:16 +08:00
yan5xu eb790bfa38 docs: add Docker Xvfb mode, anti-bot findings, decoder notes to AGENTS.md 2026-05-26 13:04:48 +08:00
yan5xu ddbe72e207 fix(docker): use Xvfb + headed Chrome to bypass anti-bot detection
Linux --headless=new lacks full GUI APIs (WebGL, plugins, screen info)
that anti-bot systems like Google's use to detect automation. macOS
headless works because Chrome still links the full Cocoa/CoreGraphics
stack even in headless mode.

Changes:
- Dockerfile: add Xvfb, entrypoint starts virtual framebuffer before
  daemon; pre-install Chrome for Testing in image; use China mirrors
  for apt/Go/npm to speed up builds
- chrome.ts: CHROME_HEADED=1 env skips --headless=new and --disable-gpu
  flags, letting Chrome render on the Xvfb display

Video stream still uses CDP screencast → bb-viewer → WebRTC.
2026-05-26 13:00:22 +08:00
yan5xu 27490e96b5 fix(docker): multi-stage build for bb-viewer, add runtime libs 2026-05-26 10:48:54 +08:00
yan5xu 4ddcb141a6 fix: support CHROME_WINDOW_SIZE env, disable stealth, clean Dockerfile
- chrome.ts: read CHROME_WINDOW_SIZE env for Docker deployment
- cdp-connection.ts: disable stealth injection permanently (detectable)
- Dockerfile: remove .so deps, use static bb-viewer
2026-05-26 01:23:41 +08:00
yan5xu 4e52e7e20a fix: disable stealth injection, update Docker for full Chrome
Stealth injection (Emulation overrides, addScriptToEvaluateOnNewDocument)
is itself detectable by Google and other anti-bot systems. CDP alone
without stealth works fine for Google login.

- Disable stealth for all Chrome types (full Chrome doesn't need it,
  headless-shell is inherently detectable regardless)
- Update Dockerfile: static-linked bb-viewer, no .so dependencies
- Clean up Dockerfile for full Chrome for Testing
2026-05-26 01:15:19 +08:00
yan5xu 5dffaf9302 fix(stealth): only inject stealth for headless-shell, skip for full Chrome
Stealth injections (Emulation.setUserAgentOverride, setDeviceMetricsOverride,
Page.addScriptToEvaluateOnNewDocument) are themselves detectable by Google's
anti-bot systems. CDP alone is NOT detected — the stealth overrides were
causing the detection.

Now only inject stealth when running headless-shell (product starts with
'HeadlessChrome'). Full Chrome (headed or --headless=new) works fine
with just CDP, no stealth needed.
2026-05-26 00:55:00 +08:00
yan5xu 604a30402a fix(stealth): fix screen size, webdriver value, and plugin count
Three headless fingerprints that Google detects:

1. screen.width/height was 800x600 (headless default) — now overridden
   to 1920x1080 via Emulation.setDeviceMetricsOverride
2. navigator.webdriver was undefined (stealth bug) — normal Chrome
   returns false, not undefined
3. navigator.plugins.length was 3 — normal Chrome has 5 PDF-related
   plugins
2026-05-26 00:42:17 +08:00
yan5xu 12cb7ca64e fix(stealth): use browserCommand for version detection, not sessionCommand
Browser.getVersion is a browser-level CDP command, not a session-level
one. Using sessionCommand returned null/error, causing fallback to
hardcoded 149.0.7827.22 — creating a version mismatch with the real
Chrome (148.0.7778.179) that Google detects as bot fingerprint.

Switch to browserCommand which correctly returns the real product
version string.
2026-05-26 00:36:33 +08:00
yan5xu 22f2781de3 feat: Docker support + fix streamer spawn crash
- Add Dockerfile for containerized deployment
- chrome.ts: prefer system Chrome, auto-download Chrome for Testing
  (full browser, not headless-shell) when no system Chrome found
- hub-bridge: handle bb-viewer spawn ENOENT gracefully instead of
  crashing the entire daemon process
2026-05-25 23:10:23 +08:00
yan5xu c05bfd04c2 feat(chrome): prefer full Chrome over headless-shell, dynamic UA version
headless-shell is an stripped-down Chrome that lacks many browser APIs
(Bluetooth, USB, SharedWorker, etc.), making it easily detectable by
anti-bot systems. Switch to using full Chrome:

1. System Chrome: prefer already-installed Google Chrome/Chromium/Edge
2. Chrome for Testing: auto-download full browser if no system Chrome
3. headless-shell: removed as a fallback (no longer needed)

Also fix UA override to dynamically detect the real Chrome version from
Browser.getVersion, so User-Agent, Sec-Ch-Ua, and Sec-Ch-Ua-Full-Version
are always consistent — no more version mismatch fingerprint.
2026-05-25 22:57:42 +08:00
yan5xu 9ed872837e fix(stealth): align UA version with actual Chrome binary (149.0.7827.22)
Sec-Ch-Ua-Full-Version was leaking the real Chrome version (149) while
User-Agent and Sec-Ch-Ua claimed 148 — this version mismatch is a
strong anti-bot signal. Align all version strings to match the actual
headless-shell binary version.

Also fix Accept-Language format.
2026-05-25 22:45:38 +08:00
yan5xu 9d8c206bf1 feat(cdp): inject stealth script on daemon controlConn
Previously stealth (UA override, navigator.webdriver, canvas/WebGL/audio
fingerprint noise, chrome.runtime shim, etc.) was only injected by
bb-viewer's inputConn. The daemon's controlConn — used by site adapters,
CLI commands, and agent operations — had no stealth protection, exposing
HeadlessChrome UA and automation markers to anti-bot systems.

Now injects Emulation.setUserAgentOverride + Page.addScriptToEvaluateOnNewDocument
in attachAndEnable(), so every tab gets stealth from the moment it's
attached, regardless of whether a stream is active.
2026-05-25 22:34:38 +08:00
yan5xu 113f3b1e64 fix(hub-bridge): skip streamer call on stream.close when no streamer running
stream.close was timing out (15s) when no streamer process existed,
because it unconditionally sent an HTTP request to the streamer port.
Now returns {ok: true} immediately if streamer is not running.
2026-05-25 22:08:49 +08:00
yan5xu 403f29592f fix(view): allow tab switching when stream is disconnected
- Remove pointer-events:none from dimmed tab/url bar so tabs remain
  clickable even when not connected to a stream.
- Update activeTabId and re-render immediately on click for responsive
  UI, then async invoke stream.switch only if connected.
- Skip stream.switch call when disconnected to avoid errors.
2026-05-25 22:03:31 +08:00
yan5xu ff7d742aa4 fix(chrome): open about:blank on launch so daemon has a default tab v0.12.2 2026-05-25 20:41:39 +08:00
yan5xu 7c972fc1d9 fix: include web/ in npm package for Clip Web UI v0.12.1 2026-05-25 20:28:04 +08:00
yan5xu bc76ed80de chore: release v0.12.0 v0.12.0 2026-05-25 19:53:31 +08:00
yan5xu 6837152c8e feat(daemon): auto-download and manage Chrome headless-shell (#120)
Daemon now automatically downloads and launches Chrome headless-shell
when started. No need to install Chrome separately.

- Downloads from Pinix COS (unified source for all platforms)
- Supports mac-arm64, mac-x64, linux64, linux-arm64
- Stores at ~/.bb-browser/browser/chrome-headless-shell-<platform>/
- Manages Chrome lifecycle: start on daemon start, kill on shutdown
- --no-chrome flag to skip (use external Chrome)
- Tested on macOS ARM64, Docker linux/amd64, Docker linux/arm64
2026-05-25 19:38:58 +08:00
yan5xu f0b2100c32 fix(site-adapter): preload _helper.js before executing site adapters
Site adapters like twitter/search.js reference shared functions defined
in _helper.js (e.g. findTransactionIdGenerator). The runtime was not
loading this helper file, causing ReferenceError at runtime.

Now checks for _helper.js in the same directory as the adapter and
prepends it into the evaluation scope.
2026-05-25 19:13:31 +08:00
yan5xu d882756351 docs: update AGENTS.md with stream lifecycle and view.html heartbeat design 2026-05-24 13:45:56 +08:00
Yanwu e60412f5fd feat(view): heartbeat ping + idle timeout + visibility-based disconnect (#226)
* feat(view): heartbeat ping + idle timeout + visibility-based disconnect

- Send OpPing (0x08) heartbeat every 5s (active) / 10s (idle)
- Visibility change: hidden starts 30s idle countdown, visible cancels
- cleanup(reason): distinct overlay messages for user/idle/network disconnect
- Start heartbeat on DataChannel open, stop on cleanup

* fix(view): use polling instead of setTimeout for idle timeout in hidden tabs

Browsers throttle setTimeout in hidden tabs to >=1 minute (Chrome).
The 30s idle timeout was never firing because the timer was delayed.

Fix: record hiddenSince timestamp on visibilitychange, check elapsed
time on each heartbeat interval tick instead of relying on setTimeout.

* fix(view): fire-and-forget stream.close to prevent disconnect from hanging

disconnect() was awaiting invoke('stream.close') which could hang if
the server is slow or the connection is degraded. This blocked cleanup()
from running, so the UI stayed in 'connected' state and heartbeat
continued indefinitely.

Fix: fire stream.close without awaiting, then immediately cleanup.
2026-05-24 13:40:00 +08:00
yan5xu e60463d8a9 docs: update AGENTS.md for new architecture
- Three CDP connections (controlConn/inputConn/captureConn)
- Protocol 1 (CLI↔daemon) and Protocol 2 (daemon↔streamer)
- Unified command definitions
- --hub mode, no standalone provider
- --tab required policy
2026-05-24 10:27:05 +08:00
yan5xu c7d4d94c3d fix: unwrap streamer result in hub-bridge stream.* handlers
Streamer returns {result: {...}} but hub-bridge was passing the
entire envelope to Hub invoke. Frontend received
{result: {offer_sdp: ...}} instead of {offer_sdp: ...}, causing
offer_sdp to be undefined and video blackscreen.

Unwrap resp.result for stream.start/answer/close/switch.
2026-05-22 22:12:47 +08:00
yan5xu 057ab7872c refactor: unified command definitions (commands.ts)
Single source of truth for all command metadata:
- ParamDef with type, required, position, description
- CommandDef with method, group, description, requiresTab, params
- commandToJsonSchema() for Hub registration
- getCommand() / getCommandsByGroup() helpers

Hub registration now uses COMMANDS directly instead of
hardcoded arrays. Removed zod dependency from shared package.

CLI and daemon dispatch unchanged — they can consume COMMANDS
in a future refactor.
2026-05-22 20:24:45 +08:00
yan5xu 811ae49592 fix: daemon lifecycle — stop stale daemon on startup and CDP port mismatch (#224)
- stopDaemon(): 3-phase shutdown (HTTP → wait → SIGKILL)
- ensureDaemon(): detect CDP port mismatch, stop old daemon before respawn
- Daemon startup: cleanupStaleDaemon() kills orphan processes
- Fixes stale daemon persisting across provider/daemon restarts
2026-05-22 20:18:35 +08:00
yan5xu 7fe310a1c6 chore: remove standalone provider (bb-browser-provider.ts)
Daemon --hub mode replaces the standalone provider. All Hub
connection, clip registration, and invoke handling is now in
packages/daemon/src/hub-bridge.ts.

Also remove provider entry from tsup release build.
2026-05-22 19:46:35 +08:00
yan5xu 822d0788a4 refactor: rename view.* to stream.*, update Clip Web UI
Hub registration:
- view.start → stream.start
- view.answer → stream.answer
- view.close → stream.close
- Add stream.switch (switch streamer to different tab)

Clip Web UI (view.html):
- Tab management: tab_list, tab_new, close (standard daemon commands)
- Tab switching: stream.switch with tab short ID
- Navigation: open/back/forward/reload with --tab
- All nav commands pass activeTabId explicitly

New: getTabCdpUrl() resolves short tab ID → CDP WebSocket URL
2026-05-22 19:46:00 +08:00
yan5xu 9fdff2e0b3 refactor: update hub-bridge for Protocol 2 streamer API
- stream.start → /command {method:'connect', params:{cdpUrl}}
- stream.answer → /command {method:'answer', params:{...}}
- stream.close → /command {method:'stop'}
- Remove view.tab_*/navigate/back/forward/reload routing
- ensureStreamer() no longer passes --cdp-port
- Add getCurrentTabCdpUrl() to resolve CDP WebSocket URL
2026-05-22 19:32:50 +08:00
yan5xu 635443fe05 feat: Phase 4 — merge provider into daemon with --hub mode
Daemon can now connect to Pinix Hub directly via --hub flag:
  bb-browser daemon --hub https://hub.pinixai.com --hub-token xxx

New: packages/daemon/src/hub-bridge.ts (1042 lines)
- HubBridge class with gRPC ProviderStream connection
- Clip registration (browser clip + platform clips)
- Invoke handling calls dispatchRequest() directly (no HTTP round-trip)
- Streamer (bb-viewer) lifecycle management
- Clip Web UI serving
- TURN credential generation
- Data command file I/O

Key improvement: browser commands invoked via Hub are dispatched
directly in-process instead of HTTP POST to daemon, eliminating
one network hop.

bin/bb-browser-provider.ts kept as-is for backward compat.
2026-05-22 19:02:50 +08:00
yan5xu 8559b772c9 refactor: Phase 2 — new protocol format, command renames, --tab required
Protocol:
- action → method, success/data/error → result/error (mutually exclusive)
- Remove id field from Request/Response
- Error is now {message, hint} object, not plain string
- HTTP method filter renamed: method → httpMethod (avoids conflict)

Command renames:
- snapshot → snap
- refresh → reload
- Remove tab_select (use open --tab)
- Remove tab_close (use close --tab)

--tab required:
- Most commands require --tab, error with hint if missing
- Exceptions: open (optional), site (auto-match domain), tab list/new, daemon

Removed commands:
- fetch (use eval or site run)
- wait (agent polls snap)
- history (not browser control)

Other:
- get supports all 5 attributes (text/url/title/value/html)
- -1400 lines across 36 files
2026-05-22 18:45:35 +08:00
yan5xu c7bd48a900 feat: move site adapter execution from CLI to daemon (Phase 3)
- New: packages/daemon/src/site-adapter.ts — site adapter scanning,
  @meta parsing, domain-based tab matching, and JS evaluation via CDP
- Daemon: add site_run, site_list, site_info, site_search actions
- CLI: site commands now send HTTP to daemon instead of executing locally
- Provider: executeSiteCommand uses daemon site_run instead of shelling
  out to CLI subprocess (removes execFile dependency)
- Protocol: add siteName, siteArgs, query fields to Request type

Site adapter execution is now a single path: CLI/Provider → daemon
→ CDP evaluate. No more three-layer process nesting.
2026-05-22 17:44:58 +08:00
yan5xu 7bb8cd1624 refactor: Phase 1 cleanup — remove MCP, dead code, unify CDP port
- Remove packages/mcp/ entirely (Agent uses CLI or Hub invoke)
- Remove dead code: reload.ts (never routed)
- Unify default CDP port to 9222 (daemon was 19825, viewer was 9222)
- Remove --mcp CLI flag
- Update comments to reflect MCP removal
2026-05-22 17:16:42 +08:00
yan5xu 6f7aeefa9c fix(view): correct mouse button mapping and clean up dead code
Fix left-click being sent as middle-click: mapButton was converting
Web API button 0 (left) to 1, but bb-viewer uses 0-indexed buttons
(0=left, 1=middle, 2=right). Middle-click on links opens new tabs
in Chrome, which was the root cause of the 'links open in new tab'
bug.

Also remove unused URL bar composition tracking variables and
debug-only url_keydown logging.
2026-05-21 22:54:15 +08:00
yan5xu bf6ab49cf9 fix(view): prevent stuck modifier keys on remote Chrome
Three fixes for modifier keys getting stuck:
- URL input capture-phase keyup no longer stops propagation for
  modifier keys (Shift/Ctrl/Alt/Meta), allowing them to reach the
  document-level onKeyUp handler
- Clicking the video overlay sends keyUp for all modifiers to clear
  any stuck state from URL bar interactions
- Window focus event resets all modifiers to handle Cmd+Tab scenarios

Also add version indicator (v9) for debugging cache issues.
2026-05-21 22:45:20 +08:00
yan5xu 03f7604ae3 feat(view): route tab/nav commands through bb-viewer
Route tab_list, tab_switch, tab_new, tab_close, navigate, back,
forward, reload through bb-viewer API instead of daemon. This ensures
tab operations and video stream operate on the same Chrome instance
and CDP session.

Also fix URL bar: skip polling overwrite when input is focused,
and simplify Enter key handling to avoid IME false suppression.
2026-05-21 20:34:16 +08:00
yan5xu f73d03551a fix(provider): pass CDP_PORT to daemon when spawning
When CDP_PORT is set, forward it as --cdp-port to the daemon process
so both daemon (tab operations) and bb-viewer (screencast) connect
to the same Chrome instance. Without this, daemon defaults to 19825
(GUI Chrome) while bb-viewer uses the CDP_PORT value, causing tab
list and video stream to reference different Chrome instances.
2026-05-21 20:06:48 +08:00
Yanwu bad09debe1 fix(web): polish remote viewer UI (#223)
- Green connect button, red disconnect state
- Video overlay with centered status text when disconnected
- Input overlay with crosshair cursor for mouse capture
- Tab/URL bar dimmed when not connected
- Slimmer toolbar (32px), webkit scrollbar styling
- Status resolution aligned right
2026-05-21 18:30:43 +08:00
Yanwu 80e4e37056 feat(web): add tab bar, URL bar, and debug panel to remote viewer (#222)
- Tab bar: list/switch/create/close tabs via invoke, 5s polling
- URL bar: address input, back/forward/reload, IME-aware Enter
- Debug panel: event log with timestamps, toggle Ctrl+Shift+D, copy/clear
- Status bar: connection dot, resolution display, debug toggle
- Matches bb-viewer index.html dark theme styling
2026-05-21 18:24:48 +08:00
Yanwu 4815d776c1 feat: remote browser viewing via Clip Web UI (#220)
* feat(daemon): expose CDP host/port in daemon.json and /status

- Add cdpHost, cdpPort to DaemonInfo (optional in shared, required in daemon)
- Write cdpHost/cdpPort from discovered cdpEndpoint to daemon.json
- Return cdpHost/cdpPort in GET /status response
- Enables external tools (bb-viewer) to discover the Chrome CDP connection

* feat(provider): add view.start/answer/close commands + bb-viewer sidecar management

- Register view.start, view.answer, view.close commands in browser clip
- Viewer sidecar: auto-spawn bb-viewer on first view.start, kill on exit
- Read CDP info from daemon.json (cdpHost/cdpPort) for viewer config
- TURN config from env vars: TURN_URL, TURN_USER, TURN_CRED
- bb-viewer binary lookup: ~/.bb-browser/bin/bb-viewer or PATH fallback
- Sidecar health check with 10s startup timeout

* feat(provider): add Clip Web UI for remote browser viewing

- web/view.html: remote viewer page using @pinixai/core/web invoke()
  - WebRTC video with playoutDelayHint=0 for low latency
  - DataChannel binary input: mouse, keyboard, scroll, IME
  - X11 keysym mapping, composition handling
  - Connect/Disconnect with auto-cleanup
- provider: set hasWeb=true for browser clip
- provider: handle GetClipWebCommand to serve web/ files
  - MIME type detection, ETag, range request support
  - Path security validation

* feat(provider): TURN ephemeral credentials via TURN REST API

- Generate time-limited TURN credentials using HMAC-SHA1 shared secret
- Credentials valid for 24h, auto-generated at viewer sidecar spawn
- Config: TURN_URL + TURN_SECRET env vars (no user-facing config needed)
- Replaces static TURN_USER/TURN_CRED with secure ephemeral credentials

* fix(web): replace esm.sh import with direct /api/ fetch

@pinixai/core is a private package, not available on esm.sh.
Use direct fetch to /api/{command} which the Hub's subdomain
handler proxies to provider invoke.
2026-05-21 18:01:49 +08:00
github-actions[bot] 7ceb9e9ff5 chore(main): release bb-browser 0.11.6 (#214)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
bb-browser-v0.11.6
2026-05-11 18:01:40 +08:00
yan5xu ba86f0c38c feat(daemon): screenshot saves to clip data and returns pinix:// URI
Screenshot command now writes PNG to ~/.pinix/data/browser/screenshots/
and returns a pinix:// URI by default. Base64 dataUrl is only included
when explicitly requested (includeBase64 flag), keeping the response
lightweight for cross-clip consumption.

CLI screenshot command sets includeBase64 for local file saving.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-11 17:54:21 +08:00
yan5xu 0ddcca6b66 feat(provider): handle DataCommand for clip data protocol
bb-browser provider now processes DataCommand from the Hub, supporting
read/write/list/delete/stat on per-clip data directories (~/.pinix/data/).
This enables pinix://browser/... URIs to work for cross-clip file sharing.

Updated @pinixai/hub-client to ^0.2.0 for Data protocol types.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-11 17:47:52 +08:00
github-actions[bot] 46a8aff93a chore(main): release bb-browser 0.11.5 (#208)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
bb-browser-v0.11.5
2026-05-07 16:19:02 +08:00
yan5xu 1938ce46b1 fix(snapshot): clean up stale highlight overlays before each snapshot and screenshot
Highlights accumulated across multiple snapshot calls because the
cleanup function was commented out. Now buildDomTree clears previous
overlays at the start, and screenshot removes them before capture.

Closes #173

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 15:21:48 +08:00
github-actions[bot] 3494cb1d65 chore(main): release bb-browser 0.11.4 (#196)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
bb-browser-v0.11.4
2026-05-07 09:23:50 +08:00
Yanwu c0b4ff1691 fix(cli): daemon self-healing on CDP disconnect + cookie persistence (#207)
* fix(cli): daemon self-healing on CDP disconnect + cookie persistence

- ensureDaemon() now checks cdpConnected from /status response; when
  Chrome is killed and daemon loses its CDP WebSocket, the stale daemon
  is stopped and a fresh daemon+Chrome is spawned automatically instead
  of returning 503 on every command.

- daemon status command shows actionable hints: how to start when not
  running, how to stop when running, and a warning with recovery steps
  when CDP is disconnected.

- Add --use-mock-keychain to managed Chrome launch args so cookie
  encryption uses a fixed key instead of macOS Keychain. This fixes
  cookie persistence on headless/remote machines (e.g. Mac Mini via SSH)
  where Keychain access is unavailable (errSecInteractionNotAllowed).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): add `daemon start` subcommand for explicit background startup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-07 09:20:46 +08:00