56 Commits

Author SHA1 Message Date
jackwener 9a45988197 v0.4.2 v0.4.2 2026-03-15 19:00:15 +08:00
jackwener 2d638e3a0a fix(manifest): extract TS adapter args at build time for proper CLI registration
The manifest builder now statically parses TS source files to extract
args, description, columns, domain, and strategy metadata. Previously
TS adapters had empty args in the manifest, causing commander to reject
their options (e.g. --keyword) before the lazy module could load.

Also includes the xiaohongshu search DOM fallback fix from earlier.

Co-authored-by: jack4world <jack4world@users.noreply.github.com>
2026-03-15 18:59:15 +08:00
jackwener b3850d9c53 fix(xiaohongshu): switch search from broken XHR interception to DOM extraction (Fixes #10)
Co-authored-by: jack4world <jack4world@users.noreply.github.com>
2026-03-15 18:55:29 +08:00
jackwener 91d7892516 docs(CLI-CREATOR): streamline 840→719 lines — remove redundancies, keep all value
- MCP Bridge: 6-step flowchart + step-by-step → compact table (-51 lines)
- Remove duplicate cascade code example (kept full template below)
- Verbose mode + output verification → 3-line compact form
- Step 5 + design philosophy → merged into concise section
2026-03-15 18:49:47 +08:00
jackwener 3f859b3238 docs(CLI-CREATOR): add SDK table, template-first guide, pagination pattern, success story
Four practical improvements based on real adapter development experience:
- Real success story: 5-minute following adapter walkthrough
- Step 2.5: template-first + Bilibili SDK reference table
- Pagination pattern with code template (pn+ps)
- Wbi vs fetchJson decision rule
2026-03-15 18:46:13 +08:00
jackwener 2fdab085a7 chore: release v0.4.1 v0.4.1 2026-03-15 18:41:14 +08:00
jackwener 7ec04c66cb docs: deep review fixes — sync command tables to actual 47 commands
- Command count 46 → 47 (added following)
- bilibili row: add 'following' command
- github row: remove non-existent 'trending', change mode to Public
- SKILL.md: add following, xueqiu hot/search commands
- Fix github section from 'trending=browser' to 'public'
2026-03-15 18:41:14 +08:00
jackwener f8ee50b776 feat(bilibili): add following adapter — get user's followed accounts
Discovered via Playwright MCP Bridge workflow (CLI-CREATOR.md):
 - Navigate to space.bilibili.com/{uid}/fans/follow
 - Sniff network: GET /x/relation/followings (Cookie-tier, no Wbi needed)
 - Verify API response: total=1342, fields: mid/uname/sign

Also fixed .d.ts loading bug in build-manifest.ts and engine.ts:
 - build-manifest now scans .js files in dist/ (not .ts)
 - engine.ts skips .d.js declaration stubs
2026-03-15 18:38:34 +08:00
jackwener 53b28637bb docs(CLI-CREATOR): add mandatory Playwright MCP Bridge workflow for AI agents
Critical addition: AI agents MUST use browser_navigate, browser_snapshot,
browser_network_requests, browser_click to discover APIs interactively.
Includes 6-step workflow, concrete tool examples, and common mistakes table.
This directly addresses the root cause of the subtitle API exploration failure.
2026-03-15 18:31:04 +08:00
jackwener ad9e9605fc feat: extreme architectural optimizations
Phase 1: Smart autoScroll with MutationObserver (single IPC, instant DOM change detection)
Phase 2: Build-time CLI manifest compiler + lazy-loading engine (instant cold start)
Phase 3: Batch IPC fetch (N browser fetches in single evaluate() call)
2026-03-15 18:25:58 +08:00
jackwener 0e1cea84b6 chore: release v0.4.0 v0.4.0 2026-03-15 18:16:27 +08:00
jackwener ee2e0a3bfc docs: update command tables to 46 commands, add subtitle/dynamic/ranking/frontpage/subreddit/profile/timeline/me/user, update SKILL.md version 2026-03-15 18:16:26 +08:00
jackwener 4fc4c4b92f fix: apply remaining code review fixes to subtitle.ts and explore.ts
- subtitle.ts: URL injection fix (JSON.stringify), --lang option,
  debug console.error removal, comment numbering
- explore.ts: clickLabels targeted clicking, empty-value penalty
  moved to scoreEndpoint(), removed console.log debug output
- Deleted stale test_subtitle.ts
2026-03-15 18:12:40 +08:00
jackwener a47d687513 perf: extreme architectural optimizations
Phase 1: Smart autoScroll with MutationObserver (single IPC, instant DOM change detection)
Phase 2: Build-time CLI manifest compiler + lazy-loading engine (instant cold start)
Phase 3: Batch IPC fetch (N browser fetches in single evaluate() call)
2026-03-15 18:12:36 +08:00
jackwener 2dacba8244 fix: code review improvements (#1-#8)
- #1 Fix URL injection in subtitle.ts via JSON.stringify
- #2 Remove debug console.error from production code
- #3 Delete stale test_subtitle.ts
- #4 Add --lang option for multi-language subtitle selection
- #5 Fix duplicate comment numbering (two '// 4.')
- #6 Add clickLabels targeted clicking + --click flag to explore
- #7 Move empty-value penalty into scoreEndpoint() (affects filtering)
- #8 Add cascade request code template to CLI-CREATOR.md
2026-03-15 18:09:59 +08:00
jackwener e64fc65554 feat: add Bilibili subtitle adapter with Wbi signing & explore fuzzing
- New: src/clis/bilibili/subtitle.ts - cascading BVID→CID→subtitle fetch
  using apiGet with automatic Wbi signature (w_rid)
- Enhanced: src/explore.ts - added INTERACT_FUZZ_JS for interactive
  API discovery via simulated clicks (--auto flag)
- Enhanced: src/explore.ts - empty-value score penalty for anti-bot detection
- Fixed: src/main.ts - restored explore command syntax, added --auto option
- Updated: CLI-CREATOR.md - cascading requests & anti-bot sections
2026-03-15 18:03:25 +08:00
jackwener 4e6d6bb81d chore: release v0.3.0 v0.3.0 2026-03-15 16:04:40 +08:00
jackwener 790c94d644 feat: Add Xueqiu site with 6 new commands and a Twitter bookmarks command, updating documentation. 2026-03-15 16:03:45 +08:00
jackwener aef110a8f8 feat(browser): support OPENCLI_BROWSER_EXECUTABLE_PATH env var to specify custom Playwright MCP browser path (Resolves #8) 2026-03-15 15:54:08 +08:00
jackwener 71f2f3d6e6 fix: integrate community bugfixes for PRs 1, 4, 6 and Issue 7
- Fix zhihu id truncation issue beyond safe integer
- Add json filter to template engine
- Escape evaluate args in browser scripts using json filter
- Resolve module not found bug for index.ts

Co-authored-by: 袁鑫 <eric.yuanxin@gmail.com>
Co-authored-by: zhenxing jiang <jiangzhx@gmail.com>
Co-authored-by: wangruotian <wangruotian@meituan.com>
Co-authored-by: 滕文证 <83223457+tenwz@users.noreply.github.com>
2026-03-15 15:43:33 +08:00
jackwener 829172fc59 feat: introduce browser interception and auto-scroll utilities, enable dynamic loading of TypeScript CLIs, and add a JSON template filter 2026-03-15 15:42:19 +08:00
jackwener 20f7d58dd6 feat: Add new CLIs for Bilibili, Reddit, Twitter, and Xiaohongshu, incorporating an intercept strategy for Twitter. 2026-03-15 15:33:30 +08:00
jackwener 92c23e8f9b docs: fix CLI creator guide links in READMEs 2026-03-15 08:59:13 +08:00
jackwener 70863fbdec Release v0.2.0 v0.2.0 2026-03-15 08:54:35 +08:00
jackwener 1361fe6c19 feat: release v0.2.0 with all new adapters and fixes 2026-03-15 08:54:35 +08:00
jackwener f19842c6b7 0.1.2 v0.1.2 2026-03-15 08:46:39 +08:00
jackwener 46ae0a0aa3 fix: test import path 2026-03-15 08:46:39 +08:00
jackwener 0fd9adf0f5 feat: add 8 new sites — bbc, weibo, boss, yahoo-finance, reuters, smzdm, ctrip, youtube
New adapters ported from bb-sites:
- bbc/news: BBC News RSS headlines (public API)
- weibo/hot: 微博热搜 (browser)
- boss/search: BOSS直聘搜索职位 (browser)
- yahoo-finance/quote: 股票行情 (browser)
- reuters/search: 路透社新闻搜索 (browser)
- smzdm/search: 什么值得买搜索好价 (browser)
- ctrip/search: 携程旅行搜索 (browser)
- youtube/search: YouTube视频搜索 (browser)

Also: build script adds clean-yaml step, docs updated to 28 commands / 16 sites
2026-03-15 08:46:13 +08:00
jackwener 862be02732 0.1.1 v0.1.1 2026-03-15 08:10:51 +08:00
jackwener 999d340a34 refactor: remove xiaohongshu me, add lessons learned, fix docs
- Remove xiaohongshu/me (profile page async loading too unreliable)
- CLI-CREATOR: add 4 lessons to 常见陷阱 (stale dist YAML, TS IIFE
  format, async polling, YAML vs TS threshold)
- Fix build: clean-yaml before copy to prevent stale overrides
- Fix docs: MCP token is configured via MCP config, not env var
- Update all command lists (README, README.zh-CN, SKILL.md)
2026-03-15 08:10:40 +08:00
jackwener fd94a5d27b fix: auto-close tabs after command, add login requirement reminders
- browser.ts: close all tabs opened during session (not just extension tabs)
  uses _initialTabCount to restore Chrome to pre-command state
- README/README.zh-CN: add ⚠️ login requirement warning with examples
- SKILL.md: add login note and tab auto-close mention
2026-03-15 07:57:12 +08:00
jackwener 129df91247 docs: fix site count from 13 to 8 in READMEs 2026-03-15 07:53:03 +08:00
jackwener 9df96db598 refactor: pipeline error propagation, me.yaml→me.ts, docs overhaul
Code:
- pipeline.ts: warn on stderr when step returns {error} object
- xiaohongshu/me.yaml → me.ts (evaluate code belongs in TS)
- index.ts: register xiaohongshu/me.ts

Docs:
- CLI-CREATOR: YAML vs TS decision tree, tap step docs with
  real example, 'explore store first' debugging workflow
- SKILL.md: add tap to Pipeline Steps table, update XHS commands
- README + README.zh-CN: update xiaohongshu command list
2026-03-15 07:51:20 +08:00
jackwener f26b906bd5 docs(CLI-CREATOR): add mandatory testing checklist and tap debugging guide
- build ≠ working: must run commands to verify
- added step-by-step testing checklist
- added tap step debugging tips for intercept strategy
2026-03-15 07:39:38 +08:00
jackwener 7bc670ee22 feat(xiaohongshu): add me + notifications, remove broken adapters
- me: personal profile info (navigate + evaluate from Pinia store)
- notifications: browse notifications by type (mentions/likes/connections)
  uses tap pipeline to call getNotification store action
- removed note, comments, user-posts (store action names unverified)
2026-03-15 07:38:53 +08:00
jackwener 9124dcc018 chore: clean up duplicate adapters
- Remove xiaohongshu/search.yaml (keep .ts with XHR interception)
- Remove zhihu/search.ts (keep .yaml, self-contained)
- Clean ~/.opencli/clis/ generated candidates (5 dirs)

24 commands across 8 sites, no duplicates
2026-03-15 07:30:35 +08:00
jackwener 0988fcb334 feat(xiaohongshu): add me, note, comments, user-posts commands
All 4 commands use YAML tap pipeline with Pinia store interception:
- me: personal profile info
- note --id: note detail
- comments --id: note comments
- user-posts --uid: user's published notes
2026-03-15 07:29:42 +08:00
jackwener 7c88c603c8 docs: add CLI-CREATOR.md + fix MCP token config instructions
- CLI-CREATOR.md: comprehensive adapter development guide with
  5-tier strategy decision tree, YAML/TS templates, XHR interception
  pattern, common pitfalls, and AI Agent auto-generation workflow
- Fix all docs: MCP Bridge connects via PLAYWRIGHT_MCP_EXTENSION_TOKEN
  in MCP config (not by clicking to approve)
2026-03-15 07:23:16 +08:00
jackwener 81c8ad522b docs: add npm install, update guide, Chinese README
- README.md: npm install -g as primary, update instructions, release guide
- README.zh-CN.md: full Chinese version with same structure
- SKILL.md: updated install section with npm global install
- Both READMEs link to each other
2026-03-15 07:20:27 +08:00
jackwener ba620f4498 feat: add tap pipeline step and probe command
- tap: declarative Store Action Bridge (Pinia/Vuex)
  - fetch + XHR dual interception, auto-cleanup
  - 4-line YAML replaces 30+ line JS monkey-patching
- probe: AI-powered site intelligence command
  - auto-discovers Pinia/Vuex stores and actions
  - correlates stores → API endpoints → capabilities
  - outputs stores.json artifact
- synthesize: generates tap-based pipelines for store-action sites
- xiaohongshu: feed.yaml and search.yaml using tap
- build: copy YAML adapters to dist/
2026-03-15 07:20:00 +08:00
jackwener d35d78d836 docs: polish README and SKILL for v0.1.0
README:
- Cleaner structure, accurate 25 commands / 13 sites
- Correct strategy labels (v2ex=public, not cookie)
- Explore artifacts documentation
- Better formatting and English/Chinese mix

SKILL:
- 5-tier authentication strategy table
- Public API YAML pattern example
- TS escape hatch guidance
- intercept pipeline step docs
- Troubleshooting section
2026-03-15 07:18:17 +08:00
jackwener 4b85ddd678 feat: add xiaohongshu search + v2ex topic + test fixes
- xiaohongshu/search.ts: Pinia store + XHR interception (from bb-sites)
- v2ex/topic.yaml: public API topic details
- Register xiaohongshu/search in index.ts
- 25 commands across 13 sites

Tested: bilibili hot/search ✅, zhihu hot ✅, v2ex hot/topic ✅,
hackernews top ✅, github search ✅
2026-03-15 07:16:03 +08:00
jackwener 93f4753d1c fix: critical MCP result parsing + V2EX strategy + extension tab cleanup
- browser.ts: strip '### Ran Playwright code' suffix from MCP evaluate results
  to correctly parse JSON data (was causing all evaluate-based commands to fail)
- browser.ts: auto-close chrome-extension:// tabs on session close
- pipeline.ts: add JSON string auto-parse fallback for evaluate step
- v2ex: add strategy:public + browser:false (public API, no browser needed)
- v2ex: simplify to use fetch step instead of evaluate

Tested: bilibili hot/search ✅, zhihu hot ✅, v2ex hot ✅,
hackernews top ✅, github search ✅
2026-03-15 07:14:07 +08:00
jackwener 1ee4919a24 docs: update README and SKILL with new features
- Fix directory path: ai-native-cli → opencli
- Add reddit, zhihu/question to command tables
- Add AI Agent workflow section with cascade command
- Add intercept pipeline step to reference
- Add reddit examples to SKILL.md
2026-03-15 07:01:01 +08:00
jackwener 36c4a3a480 feat: Deep Explore + Strategy Cascade + adapter migration
- Rewrite explore.ts: network interception, response body analysis,
  endpoint scoring, capability inference, framework detection
- Rewrite synthesize.ts: evaluate-based YAML generation,
  buildTemplatedUrl, volatile param stripping
- Add cascade.ts: automatic strategy probe (PUBLIC→COOKIE→HEADER)
- Upgrade zhihu/hot.yaml: better API with heat/answers
- Add zhihu/search.yaml, zhihu/question.ts, reddit/hot.yaml
- Migrate bilibili adapters: favorite, feed, history, user-videos
- 22 commands across 12 sites
2026-03-15 06:58:58 +08:00
jackwener c019f8c6f2 feat: add cascade strategy probe, intercept pipeline step, buildTemplatedUrl 2026-03-15 06:45:05 +08:00
jackwener f1160be8bb feat(synthesize): add buildTemplatedUrl with volatile param stripping v0.1.0 2026-03-15 06:41:20 +08:00
jackwener 932ab9097f fix: MCP extension mode, network output parsing, URL template generation
- Add --extension flag when spawning Playwright MCP server
- Fix findMcpServerPath to find @playwright/mcp/cli.js locally
- Fix parseNetworkOutput regex to handle [GET] url => [200] format
- Add buildTemplatedUrl: strips volatile params (w_rid/wts), templates
  search/limit/pagination params without URL encoding artifacts
- Add fileURLToPath import
2026-03-15 06:41:01 +08:00
jackwener 668421db3d fix: rename to @jackwener/opencli (npm name conflict with open-cli) 2026-03-15 06:38:16 +08:00
jackwener 5e01125dec feat: deep explore + enhanced synthesize + generate pipeline
- Rewrite explore.ts: network interception, response body schema analysis,
  framework detection, auto capability inference, endpoint scoring
- Rewrite synthesize.ts: generate YAML pipelines from capabilities with
  field mappings, templated URLs, and proper arg definitions
- Enhance generate.ts: goal normalization (with Chinese aliases),
  smart candidate selection, richer summary output
- Add Page.networkRequests(), consoleMessages(), scroll() to browser.ts
2026-03-15 06:34:09 +08:00