Replace "Playwright MCP" with accurate terminology (browser bridge,
snapshot engine, etc.) in comments across the codebase. Functional
stealth code that detects/cleans __playwright globals is preserved.
* feat(browser): human-like delay system for anti-detection
Adds a framework-level delay/jitter system using log-normal distribution
to simulate natural browsing patterns, addressing issue #59 (P0).
- New `HumanDelay` class with configurable profiles (none/fast/moderate/cautious/stealth)
- Log-normal distribution for realistic delay variance (not uniform)
- Periodic "breaks" that simulate reading/thinking pauses
- Auto-injected between page.goto() navigations
- Configurable via OPENCLI_DELAY_PROFILE env var
- Boss search adapter migrated from hardcoded jitter to framework delay
- 10 unit tests covering all profiles and edge cases
Real-world validation against a major job board (cookie-authenticated,
aggressive bot detection):
| Scenario | Without jitter | With jitter |
|-----------------------|--------------------|--------------------|
| 50 detail pages | ✅ OK | ✅ OK |
| 200 detail pages | ❌ Banned (code 32) | ✅ OK |
| 850 requests over 5h | N/A (banned early) | ✅ Zero detection |
| 4-day sustained crawl | N/A | ✅ 1800+ records |
Closes#59
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: disable human delay in CI environment to prevent E2E timeouts
In CI environments (CI=true), resolveProfile() now defaults to the
'none' profile instead of 'moderate'. This prevents the 1-8s per-
navigation delay from causing E2E test timeouts (30s limit).
Users can override this by setting OPENCLI_DELAY_PROFILE explicitly.
---------
Co-authored-by: toolmanlab <toolmanlab@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(grok): preserve conversation across repeated ask calls (#330)
The adapter unconditionally navigated to grok.com/ on every invocation,
destroying the existing conversation URL even when --new was not passed.
Since the browser daemon already reuses the same Chrome tab, skipping
navigation lets the tab stay on the current chat thread.
- Only navigate to grok.com/ when --new is true or tab is not on grok.com
- Add tryStartFreshChat to the default path's --new branch (was dead code)
- Add isOnGrok helper with hostname-based domain matching
- Add unit tests for isOnGrok
* test(grok): add adapter to vitest project config
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The search command defined its argument as `query` but referenced
`args.keyword`, causing the search term to be undefined.
Closes#334
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add "Why opencli?" section and comparison guide (#238)
- Add "Why opencli?" section to README.md and README.zh-CN.md
(between Highlights and Prerequisites)
- Add docs/comparison.md with 5-scenario honest evaluation
- Add Comparison entry to VitePress sidebar
* docs: refine positioning — use approximate numbers, emphasize broad coverage
- Replace specific counts (300+, 55, 20+) with approximate descriptions
- Emphasize broad global + Chinese platform coverage instead of singling out Chinese sites
- Fix Firecrawl description to mention self-hosted option
- Replace "sub-second" / "milliseconds" with accurate "seconds" / "fast deterministic"
- Add testing and AI workflow to Further Reading links
- Add "easy to extend" point to strengths
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The favorite command was using up_mid: 0 which returns empty results. Now it correctly fetches the current user UID using getSelfUid().
Co-authored-by: 章晖 <zhanghui@MacBook-Pro.local>
* feat(linkedin): add timeline feed command
* test(linkedin): add timeline adapter unit tests
Add shape tests and utility function tests for the new timeline command.
Include linkedin in the vitest adapter project config.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The C2 fix in PR #337 added a null-page guard after lazy-loading TS
modules, but it threw unconditionally — breaking all browser:false
commands (bloomberg, apple-podcasts, google, yollomi, etc.) that
use func() with a null page. Guard now checks updated.browser !== false.
Also fixes apple-podcasts top E2E flake: when the command times out on
CI, stderr is empty and the guard didn't catch it.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add generic `web read` command for any URL → Markdown
Adds a new `opencli web read --url <any-url>` command that fetches any
web page and exports it as clean Markdown with optional image download.
Uses browser-side DOM heuristics for content extraction:
1. <article> element
2. [role="main"] element
3. <main> element
4. Largest text-dense block fallback
Pipes through the existing article-download pipeline (Turndown + image
localization), so it inherits code block handling, frontmatter generation,
and concurrent image downloading for free.
Tested on: Anthropic blog, OpenAI blog, general news sites.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: improve web read dedup for sites with duplicated DOM paragraphs
Anthropic's blog renders each paragraph twice (a normal version + a
line-broken animation version). The previous substring-based dedup
missed these because whitespace differences changed string lengths.
Fix: compare texts after stripping ALL whitespace, and keep the
version with more proper spacing (more spaces = better formatted).
Result on Anthropic blog: 98.4KB → 53.7KB (45% reduction).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Harrison <harrison@HarrisondeMacBook-Pro.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- I1: Log pre-navigation failures in debug mode instead of silently swallowing
- I2: Validate env var timeout values, fallback on NaN/negative
- I4: Guard against indexOf returning -1 for unknown strategies in cascade
- I5: Fix shouldReplaceManifestEntry returning true for same-type entries
- I6: Prevent infinite loop in parseTsArgsBlock cursor advancement
- I7: Skip redundant Page.enable calls in CDP goto
- I8: Fix wait({time:0}) being treated as falsy
- I10: Warn when cookiesFile path doesn't exist before fallback
- I11: Sanitize tab/newline chars in cookie name/value for Netscape format
- I12: Use DEFAULT_DAEMON_PORT constant instead of hardcoded port in error
- I15: Log npm install failures in plugin lifecycle instead of swallowing
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1. execution.ts: Guard lazy-loaded func commands against null page — if a
lazy module incorrectly requires browser context, throw a clear error
instead of a cryptic TypeError on page.goto().
2. daemon.ts: Fix readBody race condition — add aborted flag to prevent
req.destroy() from triggering both reject (via error) and resolve
(via end event) on the same Promise, which could process truncated data.
3. browser/cdp.ts: Prevent CDPBridge.connect() reentry — throw if already
connected instead of silently leaking the previous WebSocket and its
message handlers.
4. interceptor.ts: Store intercept pattern in a separate global variable
so subsequent installInterceptor calls with different patterns update
the match condition without being blocked by the patchGuard.
5. record.ts: Always call cleanupEnter() after Promise.race — previously
only called in the timeout path, leaving readline open when user pressed
Enter, potentially blocking process exit. Also removed unused enterRace.
6. generate.ts: Fix undefined entering String.includes() — when c.name is
undefined, toLowerCase() returns undefined which gets coerced to the
string "undefined" by includes(), causing false positive matches.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): harden against command injection and sandbox escape
1. cli.ts: Remove auto-discover of arbitrary system binaries via denylist.
Unknown commands now require explicit registration via `opencli register`.
The previous denylist approach was trivially bypassable (bash, curl, etc.).
2. template.ts: Protect evalJsExpr against prototype chain escape.
Block expressions containing constructor/prototype/__proto__/process/etc.
Deep-copy context objects to sever prototype chains before passing to
new Function().
3. external.ts: Expand shell operator detection in parseCommand to cover
$(), $, #, \n, \r — preventing command substitution and comment injection.
4. fetch.ts: Use JSON.stringify for HTTP method in browser evaluate() instead
of raw string interpolation, preventing JS injection via crafted method values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: harden security-sensitive execution paths
* chore: tighten template sandbox guard
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace inline sync chrome.debugger.detach() in onUpdated listener
with the shared async detach() function for consistent cleanup behavior
across all detach paths.
- Only override navigator.plugins when empty (don't replace real user
browser plugins with fakes)
- Replace Error.prepareStackTrace (V8/Node-only) with
Error.prototype.stack getter override that works in browser context
- Fix \\n escaping in template literal for stack trace split/join
- Dynamic cdc_ variable scan via getOwnPropertyNames instead of
hardcoded names
- Update tests to cover 7 patches
Add stealth.ts module that patches browser globals to hide automation
fingerprints when opencli controls a browser via CDP or daemon extension.
Patches applied:
- navigator.webdriver → undefined (CDP sets it to true)
- window.chrome stub (only if missing)
- navigator.plugins fake list (only if empty)
- navigator.languages guarantee (only if empty)
- Permissions.query normalization for notifications
- Cleanup __playwright/__puppeteer/cdc_* artifacts
CDP mode: stealth registered via Page.addScriptToEvaluateOnNewDocument
(runs before any page JS on every navigation).
Daemon mode: stealth injected via exec after navigation, with guard
flag to prevent double-injection.
- Add getErrorMessage() to errors.ts (used in 5 files)
- Add DEFAULT_DAEMON_PORT to constants.ts (used in 5 files)
- Reduces code duplication and improves maintainability
* fix: remove duplicate getErrorMessage import in discovery.ts
Squash merge left a duplicate import line causing TS2300 and oxc parse
errors in CI. Also clean up stale blank lines in discovery.ts and
execution.ts.
* fix(e2e): broaden xiaoyuzhou skip logic for overseas CI runners
The isExpectedChineseSiteRestriction function only matched FETCH_ERROR
with specific HTTP status codes. On overseas CI runners, xiaoyuzhou may
also return PARSE_ERROR (mangled HTML) or NOT_FOUND (geo-redirected
pages), causing false test failures. Now matches all CliError codes
from the adapter.
* fix(external): replace execSync with execFileSync to prevent command injection
* fix(review): preserve Windows external installs and restore docs build
* fix(review): preserve Windows external installs after rebase
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(yollomi): add new commands and update documentation in README files
- Added yollomi commands for generating images, videos, and editing capabilities.
- Updated README.md and README.zh-CN.md to include yollomi in the command list.
- Enhanced SKILL.md with yollomi-related tags and usage examples.
* feat(yollomi): add yollomi adapter to documentation
- Included yollomi in the VitePress configuration for browser adapters.
- Updated adapters index documentation to reflect yollomi's capabilities and commands.
* fix(yollomi): bug fixes, tests & improvements
- models.ts: add browser: false (no browser connection needed for hardcoded data)
- edit.ts: remove unused resolveImageInput import
- upload.ts: lower video upload limit from 100MB to 20MB (base64 OOM risk)
- generate.ts: improve file extension detection using URL.pathname
- upscale.ts: use choices for scale arg, improve extension detection
- object-remover.ts: make image/mask args positional
- Add yollomi models tests to public-commands.test.ts
- Add yollomi generate/video graceful-failure tests to browser-auth.test.ts
---------
Co-authored-by: anichikage <hanzhishuai@bytedance.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(record): add live recording command for API capture
- Add `opencli record <url>` command that injects fetch/XHR interceptors
into all tabs in the automation window, polls captured requests, and
auto-generates YAML candidate adapters
- Support multi-tab recording: new tabs discovered during polling are
automatically injected
- Add --timeout (default 60s) for agent-friendly non-blocking operation;
stops on Enter, timeout, or SIGINT — whichever comes first
- Fix idempotent re-injection: restores original fetch/XHR before
re-patching so guard flag no longer blocks subsequent record runs
- Add --poll interval option (default 2000ms)
- Expand SKILL.md with full Record Workflow section: interceptor
internals, page-type capture expectations, YAML→TS conversion guide,
and troubleshooting table
* fix(record): fix XHR listener leak, pathChain syntax error, readline hang & args interpolation
- XHR send(): add __rec_listener_added guard to prevent duplicate event
listeners when XHR is reused (abort → open → send)
- pathChain: when findArrayPath returns '' (root-level array), data access
is just 'data' not 'data?.' which was invalid JS syntax
- waitForEnter(): return cleanup fn so timeout path can close readline.Interface
preventing the process from hanging on stdin after auto-timeout
- buildRecordedYaml: replace search/page query param values with template
vars ({{args.keyword}}, {{args.page}}) so generated YAML actually uses
the declared args instead of hardcoding the recorded URL
---------
Co-authored-by: yee.wang <yee.wang@lazada.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
- docs/adapters/browser/xiaohongshu.md: fill in search command description
(was empty), update usage examples with keyword positional arg
- TESTING.md: update unit test count 31→32 (search.test.ts added in #298),
add xiaohongshu/search.test.ts to the adapter test file list
* Add weibo search command
* fix(weibo/search): correct domain to weibo.com, add browser: true, fill doc description
- Change domain from s.weibo.com to weibo.com so browser cookies are picked
up correctly (matches hot.ts which also uses weibo.com)
- Add browser: true for consistency with other browser-based adapters
- Add description for weibo search in adapter docs table
---------
Co-authored-by: 小小机器人 <14351708+little-little-robot@user.noreply.gitee.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(v2ex): add node, user, member, replies, nodes commands
Add 5 new public API commands to the v2ex adapter:
- node: browse topics by node name
- user: list topics by username
- member: show user profile
- replies: list topic replies
- nodes: list all nodes sorted by topic count
All commands use strategy: public, browser: false.
* test(v2ex): add E2E tests for node, user, member, replies, nodes commands
* docs(v2ex): update adapter docs with new commands
* fix(v2ex): address review findings - rate-limit guards, sort verification, docs
* docs(v2ex): update README command tables and add user example
* test(v2ex): improve test quality - soft guards, value assertions, smoke tests
- Replace isExpectedChineseSiteRestriction with if(code===0) soft guard
(V2EX is globally accessible; YAML fetch doesn't throw FETCH_ERROR)
- Add value assertions: member username===Livid, limit effectiveness
- Add smoke tests for node, member, replies, nodes commands
* fix(v2ex): add url field to node/user commands, add missing user smoke test
- Add url to node.yaml and user.yaml pipeline map steps and columns
(V2EX API provides item.url; improves usability for follow-up lookups)
- Add v2ex user smoke test (other 4 new commands all had smoke tests; user was missing)
- Update E2E assertions to verify url field in node/user results
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
jackwener