105 Commits

Author SHA1 Message Date
jackwener ae4929e73d test(e2e): remove redundant --keyword flags from browser-public tests
Positional args don't need --name prefix. Replace all instances of
['cmd', 'search', '--keyword', 'VALUE'] with ['cmd', 'search', 'VALUE'].
2026-03-23 17:50:41 +08:00
jackwener 92408e7ee9 fix: improve E2E testing infrastructure
- buildMcpArgs: use CI env var instead of token detection (preserves local behavior)
- browser.test.ts: align tests with CI-based mode detection
- api-health.test.ts: remove duplicate xueqiu entry
- browser-public.test.ts: unify BBC test with tryBrowserCommand
- vitest.config.ts: default to unit tests only (E2E via explicit path)
- ci.yml: remove unnecessary needs:build for unit-test, use composite action
- e2e-headed.yml: use composite action, increase timeout to 20min
- Add .github/actions/setup-chrome composite action for shared CI steps
2026-03-16 17:33:25 +08:00
AlexYue 257e9ec4f8 feat: add E2E testing infrastructure with real Chrome in CI (#1)
# feat: add E2E testing infrastructure with real Chrome in CI

## What

Establish a comprehensive E2E testing framework for opencli using **real Chrome + xvfb virtual display** in GitHub Actions CI.

## Changes

### E2E Test Suite (~52 test cases)
- `public-commands.test.ts` — Public API commands (hackernews, v2ex)
- `browser-public.test.ts` — Browser commands for public data across all 18 sites (21 tests)
- `browser-auth.test.ts` — Graceful failure verification for login-required commands (14 tests)
- `management.test.ts` — Full coverage of management commands (list/validate/verify/version/help)
- `output-formats.test.ts` — Output format validation (json/yaml/csv/md)
- `smoke/api-health.test.ts` — Scheduled API health checks

### Auto-detect Browser Mode
- `buildMcpArgs` automatically selects mode based on `PLAYWRIGHT_MCP_EXTENSION_TOKEN`:
  - Token present → `--extension` (local user, connects to logged-in Chrome)
  - Token absent → standalone (CI launches its own browser)
- No extra environment variables needed

### CI Pipeline
- `e2e-headed.yml` — Real Chrome via `browser-actions/setup-chrome` + `xvfb-run` in headed mode
- `ci.yml` — build + unit-test (2 shards) + smoke-test (scheduled/manual)
- Browser commands that return empty data due to geo-blocking or bot detection gracefully warn + pass without blocking CI

### Documentation
- New `TESTING.md` — Architecture, coverage, local setup, how to add tests, CI explanation
- Updated `README.md` — Added Testing section with quick-start commands
2026-03-16 16:22:00 +08:00
jackwener 6f629260e7 feat: add interactive 'opencli setup' command with TUI checkbox
New zero-dependency TUI checkbox component (src/tui.ts) with:
  - ↑↓/jk navigation, Space toggle, Tab toggle+move
  - 'a' toggle all, Enter confirm, q/Esc cancel

New 'opencli setup' command (src/setup.ts) that:
  - Auto-discovers token from Chrome LevelDB extension
  - Shows interactive multi-select for config files
  - Displays tool names (Codex, Cursor, Claude Code, etc.)
  - Color-coded status (green=ok, yellow=mismatch, red=missing)
  - Applies changes only to selected files
2026-03-16 13:53:03 +08:00
jackwener 1e0b83bb84 feat(doctor): add Antigravity and Gemini CLI config paths
Add ~/.gemini/settings.json (Gemini CLI) and
~/.gemini/antigravity/mcp_config.json (Antigravity) to the
default MCP config scan list.
2026-03-16 13:45:02 +08:00
jackwener 1ff184e24d feat(doctor): add Claude Code and project .mcp.json config paths
Add ~/.claude.json (Claude Code user-scoped MCP config) and
.mcp.json (Claude Code project-scoped MCP config) to the
default config scan list.
2026-03-16 13:43:03 +08:00
jackwener 8e84145ccc feat(doctor): auto-discover extension token from Chrome LevelDB
Scan Chrome/Edge/Chromium localStorage LevelDB files to extract the
Playwright MCP Bridge auth-token directly from the extension's storage.
Uses a fast 'strings | grep' shell pipeline (~200ms) on macOS/Linux
with a pure-Node fallback for Windows.

The discovered token is now shown in 'opencli doctor' output and takes
priority as the recommended token when using '--fix'.
2026-03-16 13:38:48 +08:00
jackwener c77c8a8e3a feat: add Coupang search and add-to-cart adapters
Add browser-backed Coupang adapters for search and add-to-cart workflows.
- coupang search: multi-strategy data collection (API/JSON-LD/bootstrap/DOM), structured fields (price, rating, rocket, delivery), pagination and rocket filter support
- coupang add-to-cart: logged-in browser session reuse, stops before checkout
- coupang.ts: comprehensive data normalization layer with badge/rocket/delivery mapping
- browser-tab.ts: withTemporaryTab utility for isolated tab operations
- Unit tests for core normalization functions

Co-authored-by: CodeBBakGoSu <127713112+CodeBBakGoSu@users.noreply.github.com>
2026-03-16 13:30:42 +08:00
jackwener 9e024d4e46 0.5.2 v0.5.2 2026-03-16 13:22:18 +08:00
jackwener 34a9bff2b3 refactor: eliminate code duplication, improve type safety, add tests
- NEW: src/interceptor.ts — unified XHR/Fetch interceptor (was duplicated 3x)
- NEW: src/version.ts — centralized PKG_VERSION (was duplicated 2x)
- NEW: src/constants.ts — shared VOLATILE_PARAMS, FIELD_ROLES etc.
- NEW: src/engine.test.ts, src/registry.test.ts — 14 new unit tests

- browser.ts: use shared normalizeEval, interceptor, withTimeoutMs, PKG_VERSION
- intercept.ts, tap.ts: use shared interceptor generators
- cascade.ts: extract shared buildFetchProbeJs (90% dedup)
- engine.ts: use InternalCliCommand (no more 'as any' casts)
- executor.ts: remove all 15 'as StepHandler' type casts
- runtime.ts: add withTimeoutMs, IBrowserFactory interface
- registry.ts: add InternalCliCommand type for internal fields
- validate.ts: add pipeline step name validation
- output.ts: remove 'null as any' and '.filter(() => true)' hacks
- explore.ts, synthesize.ts: use shared constants
- docs: fix V2EX commands (3→6), SKILL.md version, verify example

Tests: 88 passed (was 74), tsc --noEmit: 0 errors
2026-03-16 13:22:12 +08:00
jackwener b0c51ddf19 fix: use Playwright MCP --executable-path flag (kebab-case)
Extract buildMcpArgs() helper and fix --executablePath → --executable-path
to match the Playwright MCP CLI's expected flag format.

Closes #16

Co-authored-by: KasumiChen <KasumiChen@users.noreply.github.com>
2026-03-16 12:58:18 +08:00
jackwener 141c2cf7d5 0.5.1 v0.5.1 2026-03-16 12:54:45 +08:00
jackwener 7ee0c9c96b fix: tab cleanup regex to match actual Playwright MCP tab format
The tab list from Playwright MCP uses '- N: (current) [title](url)' format,
but extractTabEntries only matched 'Tab N ...' format. This caused
_initialTabIdentities to always be empty, so tabs were never cleaned up.

Now supports both formats.
2026-03-16 12:54:44 +08:00
jackwener 76eefed83d docs: add opencli doctor hint after token setup 2026-03-16 12:47:59 +08:00
jackwener 08ae9c3fed chore: sync SKILL.md version to 0.5.0 2026-03-16 12:46:30 +08:00
jackwener 5a1581e67a 0.5.0 v0.5.0 2026-03-16 12:46:16 +08:00
jackwener 7a24e8d74b fix: add engines field for Node badge, sync SKILL.md version, fix zh-CN intro 2026-03-16 12:46:15 +08:00
jackwener 95fb841fc7 refactor: remove all CDP code, keep extension-only mode
- Remove CDP auto-discovery (discoverChromeEndpoint, isCdpApiAvailable, isPortReachable)
- Remove CDP readiness probe and autoAllowCdpDialog
- Remove forceExtension plumbing from registry, runtime, main, and adapters
- Remove CDP diagnostics from doctor (extractTokenViaCdp, remoteDebugging fields)
- Simplify connect() to always use --extension mode
- Remove CDP env vars (OPENCLI_USE_CDP, OPENCLI_CDP_ENDPOINT, OPENCLI_FORCE_EXTENSION)
- Update README.md, README.zh-CN.md, SKILL.md to remove CDP sections
- Update tests to remove CDP-related assertions

Current state preserved in extension-and-cdp branch.
2026-03-16 12:44:09 +08:00
jackwener 67d50191af fix: CDP mode hanging when using chrome://inspect remote debugging
- Add isCdpApiAvailable() to verify CDP HTTP JSON API before using endpoint
- Chrome's chrome://inspect#remote-debugging writes DevToolsActivePort but
  its CDP endpoint is incompatible with Playwright connectOverCDP (init
  succeeds but all tool calls hang silently)
- Add CDP readiness probe in connect() to catch unresponsive endpoints
- Skip tab cleanup in close() for CDP mode (no bridge tabs to clean)
- Add PlaywrightMCPMode tracking for extension vs CDP lifecycle policy
v0.4.6
2026-03-16 02:28:28 +08:00
jackwener 61a62f80c4 0.4.5 v0.4.5 2026-03-16 01:51:27 +08:00
jackwener f8491f01ae fix: add 5s timeout for initial tabs fetch during CDP connect
CDP mode on @playwright/mcp v0.0.68 hangs on browser_tabs calls.
This timeout prevents the entire connect() from blocking for 30s.
2026-03-16 01:51:22 +08:00
jackwener 4ec454530a 0.4.4 v0.4.4 2026-03-15 23:58:15 +08:00
jackwener e2192a2e8d docs: rename CLI-CREATOR → CLI-EXPLORER, add CLI-ONESHOT refs to READMEs
- Rename CLI-CREATOR.md → CLI-EXPLORER.md (explorer better reflects its purpose)
- Update all references across SKILL.md, README.md, README.zh-CN.md, CLI-ONESHOT.md
- Add quick-mode (CLI-ONESHOT) and full-mode (CLI-EXPLORER) entries in both READMEs
2026-03-15 23:58:10 +08:00
jackwener db8194337b Improve browser session lifecycle handling 2026-03-15 23:31:52 +08:00
jackwener 8e5a61458d docs: add CLI-ONESHOT.md — lightweight one-shot CLI generator guide
- CLI-ONESHOT.md: ~150 lines, 4-step flow for single-URL adapter generation
- Update SKILL.md: add quick-mode tip pointing to CLI-ONESHOT.md
- Update CLI-CREATOR.md: add tip pointing to lightweight alternative
2026-03-15 23:30:13 +08:00
jackwener dca908db90 docs: use bold AND to emphasize required config in zh-CN 2026-03-15 22:13:34 +08:00
jackwener 5a676b9f4b docs: clarify that both env var and MCP config are required for token 2026-03-15 22:12:59 +08:00
jackwener 797a392250 0.4.3 v0.4.3 2026-03-15 21:53:43 +08:00
jackwener 59a02d3a5d feat(twitter): implement like adapter 2026-03-15 21:53:42 +08:00
jackwener cef50dc8bf feat(doctor): print cli version in doctor report 2026-03-15 21:51:21 +08:00
jackwener 1aaafed2db feat(twitter): implement post, reply, delete, following, followers and notifications adapters 2026-03-15 21:42:48 +08:00
jackwener ee6669f45d feat(core): support YAML format output and optimize manifest builder 2026-03-15 21:05:36 +08:00
jackwener eb9050fadd feat(v2ex): add daily, me, and notifications session adapters 2026-03-15 20:59:39 +08:00
jackwener 53cd006a23 fix(boss): resolve limit=60 infinite loop and global CLI hangs
- search.ts: Added deduplication break condition so if the API returns duplicate pages, it stops instead of looping forever.
- search.ts: Removed unsafe Promise.race around page.goto which caused 'context closed' errors.
- browser.ts: Added robust static cleanup hooks to guarantee process and MCP lock destruction on crashed or ctrl+c'd runs.
2026-03-15 20:33:22 +08:00
jackwener c9710a656b feat(boss): add pagination loop to support fetching more than 15 results
- The API endpoint only returns 15 results per page.
- Added a loop to fetch multiple pages until the requested limit is reached.
- Added random 1-3 second delays between page fetches to avoid rate limits.
2026-03-15 20:19:30 +08:00
jackwener 4f570fa75e docs: add instruction to get token by clicking extension icon 2026-03-15 20:08:30 +08:00
jackwener 730285c0ca docs: set MCP Bridge as primary connection method, CDP as fallback 2026-03-15 20:07:32 +08:00
jackwener c5915d9ee3 docs: clarify that MCP token needs to be configured in both MCP config and env var 2026-03-15 20:06:32 +08:00
jackwener b7c8067bc9 docs: restore playwright MCP token configuration guide 2026-03-15 20:05:08 +08:00
jackwener c28ccd4b23 docs: huge improvement on README structure, badges and readability 2026-03-15 19:59:02 +08:00
jackwener 00f0ac9133 fix(boss): prioritize fresh browser cookie extraction over boss-cli cache 2026-03-15 19:58:46 +08:00
jackwener 64d655ae0d feat(boss): rewrite search adapter to use boss-cli native cookie extraction
- BOSS Zhipin aggressively detects CDP connections and kills pages
- Completely rewrote boss/search.ts to use browser: false (bypassing Playwright)
- Implemented native Python subprocess to extract cookies via browser-cookie3
- Falls back to boss-cli ~/.config/boss-cli/credential.json if available
- Uses direct Node.js fetch with anti-detection headers to avoid bans
2026-03-15 19:55:46 +08:00
jackwener 9c78ffdd39 chore: change license to BSD-3-Clause 2026-03-15 19:54:34 +08:00
jackwener a6e9c3b611 feat(browser): enhance CDP auto-discovery with port probe & opt-in priority
- Add TCP port reachability check to handle stale DevToolsActivePort files
- Make discoverChromeEndpoint() async with port probe before returning
- Change connection priority: extension mode is default, CDP is opt-in
  - OPENCLI_CDP_ENDPOINT: explicit CDP endpoint
  - OPENCLI_USE_CDP=1: enable DevToolsActivePort auto-discovery
  - Default: --extension mode (Playwright MCP Bridge)
- Add verbose logging for CDP discovery when -v flag is used
- Fix indentation inconsistency in connect() method
- Update README.md, README.zh-CN.md, SKILL.md with new connection docs
2026-03-15 19:44:27 +08:00
jackwener da155e96f0 feat: add forceExtension to bypass CDP detection for anti-bot sites
- Added forceExtension flag to CliCommand, CliOptions, browserSession, and PlaywrightMCP.connect()
- BOSS Zhipin detects CDP mode and drops connections; marked boss/search with forceExtension: true
- Also supports OPENCLI_FORCE_EXTENSION=1 env var for global override
- Fixed async discoverChromeEndpoint() placement outside Promise constructor
- Enhanced boss search with city names, experience, degree, salary, etc.
2026-03-15 19:39:39 +08:00
backtime1993 c20eaee393 feat(browser): Chrome 144+ auto-discovery via DevToolsActivePort
Chrome 144+ allows enabling remote debugging from chrome://inspect
without any command-line flags (--remote-debugging-port). Chrome writes
the active port and browser GUID to a DevToolsActivePort file in the
user data directory.

This change reads that file to auto-discover the CDP WebSocket endpoint,
eliminating the need for the Playwright MCP Bridge browser extension in
most cases.

Connection priority:
1. OPENCLI_CDP_ENDPOINT env var (manual override)
2. DevToolsActivePort auto-discovery (Chrome/Edge, cross-platform)
3. --extension mode fallback (original behavior)

Supports Windows, macOS, and Linux with Chrome, Edge, and Chromium.

Co-authored-by: kensei <backtime1993@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 19:34:14 +08:00
jackwener 15d8c45b95 feat(boss): add city name mapping, experience/degree/salary filters, page support
- 30+ Chinese city names → code mapping (e.g. --city 杭州)
- Experience filter: 应届/1-3年/3-5年/5-10年/10年以上
- Degree filter: 大专/本科/硕士/博士
- Salary filter: 3K以下/3-5K/5-10K/10-20K/20-50K/50K以上
- Page number support for pagination
- Area now shows city·district·business district
- Skills column added to output
2026-03-15 19:31:05 +08:00
jackwener 3ac5aa3023 docs: reduce excessive emoji usage across markdown files 2026-03-15 19:30:32 +08:00
jackwener 9a6e157af6 docs: add strong AI Agent prompts to read CLI-CREATOR.md before creating adapters 2026-03-15 19:28:05 +08:00
jackwener 9a45988197 v0.4.2 v0.4.2 2026-03-15 19:00:15 +08:00