Commit Graph

106 Commits

Author SHA1 Message Date
jakevin 80eef46b4e refactor: monorepo adapter separation (clis/ at root) (#782)
* refactor: move adapters from src/clis/ to root clis/ for monorepo separation

Separates CLI adapters from the core runtime to prepare for independent
adapter distribution via postinstall fetch.

Key changes:
- Move src/clis/ → clis/ (adapters at repo root)
- Change tsconfig rootDir from "src" to "." so tsc compiles both
- Create root-level shim files (registry.ts, errors.ts, etc.) so adapter
  relative imports (../../registry.js) resolve correctly
- Update build-manifest.ts, main.ts paths for new dist/src/ structure
- Expand ensureUserCliCompatShims() to cover all adapter import targets
  (types, utils, logger, launcher, browser/*, download/*, pipeline/*)
- Add scripts/fetch-adapters.js postinstall for ~/.opencli/clis/ sync
- Update vitest.config.ts adapter test paths
- Add package.json files field to exclude adapters from npm package

Official adapter files are unconditionally overwritten on update;
user-created files not in the manifest are preserved.

* fix: add dist/clis/ and cli-manifest.json to npm files, harden fetch-adapters

- Add dist/clis/ and dist/cli-manifest.json to package.json files field
  so built-in adapters and manifest ship with the npm package
- Replace execSync with execFileSync to prevent command injection
- Add version check to skip redundant adapter fetches
- Track tmpRoot explicitly for reliable cleanup

* fix: address review blockers — manifest-based updates, global-only fetch, first-run fallback

1. Manifest-based update strategy:
   - Read old manifest to identify previously-official files
   - Clean up files removed upstream (in old manifest but not new)
   - User-created files (never in any manifest) remain untouched

2. Only run fetch-adapters on global install (npm_config_global=true)
   or explicit OPENCLI_FETCH=1, preventing heavy side effects for
   local/dev installs

3. First-run fallback in discovery.ts:
   - ensureUserAdapters() checks for adapter-manifest.json
   - If missing and ~/.opencli/clis/ is empty, spawns fetch-adapters.js
   - Guarantees adapters are available even with --ignore-scripts

* fix: remove OPENCLI_FETCH env var, use internal _OPENCLI_FIRST_RUN instead

* feat: also support OPENCLI_FETCH=1 for explicit adapter fetch trigger

* simplify: replace git clone with local copy from dist/clis/

Adapters already ship in the npm package (dist/clis/), so there's no
need to clone from GitHub. Copy directly from the installed package:

- Eliminates git, curl, tar dependencies
- No network calls in postinstall
- No timeout/offline issues
- Version always matches the installed CLI
- ~65 lines of clone/download code replaced by one cpSync loop
2026-04-05 01:46:36 +08:00
jakevin b2f1f58a1b chore: bump version to 1.6.2 (#775) 2026-04-04 21:28:37 +08:00
jakevin 6cdcb9dd51 fix: add prepare script so source installs trigger build (#729)
* fix: add prepare script so source installs trigger build

npm install from git (e.g. npm install github:jackwener/opencli) skips
prepublishOnly, so dist/ is never generated. The prepare hook runs on
git-based installs; the [ -d src ] guard skips it for registry installs.

* fix: include extension/dist in git so clone works out of the box

.gitignore had conflicting rules: line 3 tried to un-ignore extension/dist/
but line 26 re-ignored it. Remove the later rule so the built extension JS
is tracked in git — users can load the extension directly after clone.
2026-04-03 16:23:05 +08:00
jakevin 706e01dbca docs: fix outdated adapter counts, missing commands and adapters (#704)
* docs: fix outdated adapter counts, missing commands, and absent adapters

- Update version 1.6.0 → 1.6.1 in skills/opencli-usage/SKILL.md
- Update site count 70+ → 73+ across README.md, README.zh-CN.md,
  docs/comparison.md
- Remove non-existent adapters (kimi, deepseek, qwen) from SKILL.md
- Add missing commands for xiaohongshu (+note, comments, download,
  publish), weibo (+search, feed, user, me, post, comments),
  jike (+post, topic, user), linux-do (+hot, latest, category),
  doubao (+detail, history, meeting-summary, meeting-transcript),
  weread (+notebooks), chatgpt (+model), wikipedia (+random, trending),
  stackoverflow (+unanswered), producthunt (fix command list)
- Add entirely missing adapters: band, zsxq, bluesky, douyin, 36kr,
  ones, tieba, gemini, notebooklm, imdb, spotify, paperreview
- Update docs/adapters/index.md with same fixes
- Add opencli-operate to Related Skills section

* docs: second-pass audit fixes — deeper inconsistencies

Skills sub-files (browser.md, public-api.md):
- Remove phantom kimi/deepseek/qwen adapters (no src/clis/ dirs)
- Replace with real gemini and notebooklm sections
- Add missing weibo commands (search, feed, user, me, post, comments)
- Add missing xiaohongshu commands (note, comments, download, publish)
- Add missing doubao commands (detail, history, meeting-summary, meeting-transcript)
- Add 7 entirely missing adapter sections: bluesky, douyin, band, zsxq,
  tieba, 36kr, ones
- Fix producthunt: remove non-existent week/month/search, add hot/browse/posts
- Add wikipedia random and trending

SKILL.md command table:
- Add twitter `likes`, xueqiu `comments`, douban `movie-hot`/`book-hot`
- Add entirely missing `amazon` adapter
- Add linux-do `latest`
- Remove producthunt non-existent `search`

Individual adapter docs:
- docs/adapters/browser/weibo.md: add 5 missing commands
- docs/adapters/browser/doubao.md: add 4 missing commands
- docs/adapters/browser/wikipedia.md: add random and trending
- docs/adapters/browser/36kr.md: fix contradictory prerequisites
- docs/adapters/index.md: add twitter `likes`
- docs/developer/contributing.md: add missing `positional: true`
- package.json: fix description to include "Electron App"
2026-04-03 01:05:47 +08:00
jakevin 4d036a5364 chore: release v1.6.1 (#700) 2026-04-02 22:45:41 +08:00
jakevin aa3edfefc0 chore: release v1.6.0 (#697) 2026-04-02 22:14:23 +08:00
jakevin a21cc5e9f0 chore: release v1.5.9 (#678) 2026-04-02 13:49:10 +08:00
jakevin 1ddca55b4a chore: release v1.5.8 (#663) 2026-04-01 22:12:39 +08:00
jakevin c908d9cd47 chore: release v1.5.7 (#654) 2026-04-01 13:09:11 +08:00
jakevin 62e3c55993 chore(release): 1.5.6 (#596) 2026-03-30 13:27:19 +08:00
jakevin 812f27a05e chore(release): 1.5.5 (#565) 2026-03-28 22:19:59 +08:00
jakevin c9e29d9f22 chore(release): 1.5.4 (#558) 2026-03-28 20:22:24 +08:00
jakevin 4f13484aa3 chore(release): 1.5.3 (#533) 2026-03-28 11:02:09 +08:00
wangsl ea0cf4d0b0 fix(network): honor proxy env for node requests (#512)
* fix(network): honor proxy env for node requests

* fix(network): honor default ports in NO_PROXY

* refactor(network): normalize proxy config handling

* refactor(network): delegate proxy env handling to undici

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-28 01:20:39 +08:00
jakevin 9919f03aed chore(release): 1.5.2 (#523)
* chore(release): 1.5.2

* test(e2e): stabilize output format checks
2026-03-28 00:48:27 +08:00
jakevin f9f11e4b17 chore(release): 1.5.1 (#513) 2026-03-27 19:25:37 +08:00
jakevin 218ba918d9 chore: bump version to 1.5.0 (#482) 2026-03-27 03:57:05 +08:00
AlexYue 6d1fb6d219 feat(runtime): add Bun runtime compatibility (#459)
* feat(runtime): add runtime detection utility for Bun/Node.js

Add runtime-detect.ts module that detects whether opencli is running
under Bun or Node.js via globalThis.Bun check. Includes helper
functions for version string and label formatting.

Add corresponding unit tests that work correctly under both runtimes.

* feat(runtime): integrate Bun runtime support into CLI tooling

- doctor: show runtime label (e.g. 'node v22.13.0') in diagnostic output
- package.json: add dev:bun, start:bun, test:bun convenience scripts
- E2E helpers: support OPENCLI_TEST_RUNTIME env var for runtime selection

* ci: add Bun compatibility test job and document runtime support

- ci.yml: add bun-test job using oven-sh/setup-bun@v2
- README.md: update Prerequisites to mention Bun, add Runtime Support
  section with usage examples for dev:bun, start:bun, test:bun

* ci: pin Bun version in compatibility job

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 19:10:31 +08:00
jakevin 2bfd3eeeb3 chore: release v1.4.1 (#425) 2026-03-25 16:21:14 +08:00
jakevin 15369fa23c chore: release v1.4.0 (#413)
* chore: release v1.4.0

* docs: sync command references across SKILL.md, README, and docs

SKILL.md:
- Add 12 missing sites: apple-podcasts, arxiv, bloomberg, coupang,
  dictionary, doubao, jd, linkedin, pixiv, web, weixin, xiaoyuzhou
- Add 36 missing commands across 6 existing sites (twitter, hackernews,
  yollomi, xueqiu, linux-do, v2ex)

README (EN + zh-CN):
- Add linkedin timeline command

docs/:
- Add 13 missing adapters to vitepress sidebar navigation
- Add 6 missing adapters to docs/adapters/index.md overview table
- Update xueqiu commands with fund-holdings, fund-snapshot
2026-03-25 14:48:46 +08:00
dependabot[bot] 37c7ea41b8 chore(deps): bump typescript from 5.9.3 to 6.0.2 (#327)
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.9.3 to 6.0.2.
- [Release notes](https://github.com/microsoft/TypeScript/releases)
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.9.3...v6.0.2)

---
updated-dependencies:
- dependency-name: typescript
  dependency-version: 6.0.2
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 14:51:15 +08:00
jakevin 1b3f74cd6a test: focus adapter coverage on four priority sites (#339) 2026-03-24 12:03:26 +08:00
jakevin 167c7c784a chore: release v1.3.3 (#321) 2026-03-24 01:45:21 +08:00
jakevin 760b91e7b3 chore: release v1.3.2 (#317) 2026-03-24 01:05:47 +08:00
jakevin c9b3568594 chore: pre-release cleanup — fix dependencies, sync docs, reduce code duplication (#311)
* chore: pre-release cleanup — fix dependencies, sync docs, reduce code duplication

- fix: move @types/turndown from dependencies to devDependencies
- docs: backfill CHANGELOG for v1.2.0 through v1.3.1
- docs: remove internal release reminder from READMEs
- docs: update SKILL.md version to 1.3.1
- refactor: extract getErrorMessage() to errors.ts (was duplicated 5x)
- refactor: introduce CommandArgs type alias in registry.ts
- refactor: eliminate as-any in runtime.ts and cli.ts
- refactor: parallelize site directory scanning in discovery.ts
- refactor: add declare global for registry globalThis access
- fix: strengthen isBooleanRecord type guard in explore.ts
- fix: replace empty catch with log.debug in explore.ts
- fix: daemon now accepts timeout from request body
- chore: remove unused REGISTRY_KEY constant
- chore: update generate.ts TODO to stub annotation

* docs: add doubao-app desktop adapter doc (fixes docs-build dead link)
2026-03-24 00:33:40 +08:00
jakevin 2a52906ed2 fix: add turndown dependency to package.json (#288)
turndown and @types/turndown were used in article-download.ts and
zhihu/download.test.ts but never declared in package.json, causing
CI failures on fresh npm ci installs.
2026-03-23 14:46:08 +08:00
jakevin 788126198b chore: release v1.3.1 (#273) 2026-03-23 00:33:04 +08:00
jakevin 98ecfab8ad chore: bump version to 1.3.0 (#272)
* chore: bump version to 1.3.0

* chore: bump version to 1.3.0
2026-03-23 00:27:55 +08:00
jakevin c44bc62b60 chore: code cleanup + extension conflict troubleshooting (#260)
Cleanup:
- Remove redundant double-retry in resolveTabId (was retrying data: URI
  with the same data: URI)
- Fix stale comment (30s → 120s idle timeout)
- Remove verbose debug logging in resolveTabId
- Built extension is now smaller (16.66kB vs 17.18kB)

Extension conflict:
- Add hint to attach-failed error when chrome-extension:// URL is detected
- Add troubleshooting entry for extension conflicts (e.g. youmind, New Tab
  Override) to both README.md and README.zh-CN.md

Ref: #249
2026-03-22 22:18:22 +08:00
jakevin 7f57e76485 fix: treat empty tab URL as debuggable (fixes first-run doctor --live failure) (#259)
When a new automation window is created, the initial tab URL may be
empty briefly while Chrome loads the data: URI. isDebuggableUrl('') was
returning false, causing ensureAttached to reject the tab.

Fix: only reject known non-debuggable URLs (chrome://, chrome-extension://).
Empty/undefined URLs are now treated as debuggable since they represent
tabs still loading.

Also adds 200ms delay after window creation to let Chrome populate the
tab URL.
2026-03-22 22:10:30 +08:00
jakevin 3e91876d13 fix: replace all about:blank with data: URI to prevent New Tab Override interception (#257)
Root cause: getAutomationWindow and resolveTabId used about:blank which
New Tab Override extensions intercept immediately, replacing it with
chrome-extension:// URLs that cannot be debugged.

Changes:
- Window creation: about:blank → data:text/html
- reuseTab fallback: about:blank → data:text/html
- newTab handler: about:blank → data:text/html
- Added diagnostic logging to resolveTabId for debugging
- Synced extension version to 1.2.4

Ref: #249
2026-03-22 22:04:25 +08:00
jakevin 7c02588105 chore: bump version to 1.2.3 (#256) 2026-03-22 21:48:17 +08:00
jakevin e077ad2336 chore: bump version to 1.2.2 (#254) 2026-03-22 21:27:03 +08:00
jakevin 81384ede00 chore: bump version to 1.2.1 (#252) 2026-03-22 21:24:42 +08:00
jakevin 2d1b8c1e76 chore: prepare v1.2 release (#244) 2026-03-22 18:40:32 +08:00
jakevin 9921e5d696 remove broken desktop adapters (#221) 2026-03-22 04:14:07 +08:00
jakevin d831b04d48 feat(browser): advanced DOM snapshot engine with 13-layer pruning pipeline (#210)
Core Changes:
- New dom-snapshot.ts: 13-layer LLM-optimized DOM pruning engine
  - Tag filtering, SVG collapse, ad/noise detection
  - CSS visibility, viewport threshold, paint-order occlusion
  - Shadow DOM traversal, same-origin iframe extraction
  - BBox parent-child dedup, attribute whitelist + synthetic attrs
  - Table → markdown serialization
  - Incremental diff (mark new elements with *)
  - data-opencli-ref annotation for precise click/type targeting
  - Hidden interactive element hints (scroll-to-reveal)

New APIs:
- IPage.scrollTo(ref) — scroll to snapshot-identified elements
- IPage.getFormState() — extract all form fields as structured JSON
- scrollToRefJs(), getFormStateJs() — standalone JS generators

Integration:
- Page (daemon) + CDPPage (direct CDP): use new engine as primary
- dom-helpers click/type: 4-layer fallback (data-opencli-ref → data-ref → CSS → index)
- Exports from browser/index.ts barrel

Testing:
- 21 new tests for dom-snapshot engine
- All 283 tests pass (29 files, 1.07s)
- Split test scripts: npm test (unit only), npm run test:all (full)
2026-03-22 00:11:51 +08:00
AlexYue 1d39295f4b feat: plugin system (Stage 0-2)
* feat: plugin system (Stage 0-2)

- Stage 0: discoverPlugins() scans ~/.opencli/plugins/ at startup
- Stage 1: demo plugin repos (github-trending, hot-digest)
- Stage 2: opencli plugin install/uninstall/list commands
- package.json exports ./registry for TS plugin peerDep support
- 17 new/updated tests, tsc --noEmit clean

* fix: CDPBridge connect timeout unit mismatch (seconds vs ms)

opts.timeout is passed in seconds from runtime.ts but CDPBridge
was using it as milliseconds, causing instant timeout (30ms).

* feat: add registry-api public entry point for TS plugin peerDep support

- Add src/registry-api.ts: re-exports core registration API (cli, Strategy,
  getRegistry) without transitive side-effects, safe for plugin imports
- Update package.json exports: './registry' -> './dist/registry-api.js'
- Update src/registry.ts: use globalThis shared registry to ensure single
  instance across npm-linked plugin modules
- Update .gitignore for plugin-related artifacts

* fix: symlink host opencli into plugin node_modules on install

After npm install, replace the npm-installed @jackwener/opencli
with a symlink to the running host's package root. This ensures
TS plugins always resolve '@jackwener/opencli/registry' against
the host installation, avoiding version mismatches when the
published npm package lags behind.

* fix: transpile TS plugins to JS on install, deduplicate .ts/.js discovery

- installPlugin: after symlinking host opencli, transpile any .ts files
  to .js using esbuild from the host's node_modules/.bin/
- discoverPluginDir: skip .ts files when a .js sibling exists (production
  node cannot load .ts directly)
- scanPluginCommands: deduplicate basenames via Set to avoid showing
  'aggregate, aggregate' when both .ts and .js exist

* docs: add plugin system user guide

- New docs/guide/plugins.md covering:
  - Installation/uninstallation commands
  - Creating YAML plugins (zero-dep)
  - Creating TS plugins (with peerDep)
  - TS plugin install lifecycle (clone → deps → symlink → transpile)
  - Example plugins and troubleshooting
- Add Plugins to VitePress sidebar (EN + ZH)
- Link from getting-started.md Next Steps

* fix: address review issues in plugin system

- Security: replace execSync with execFileSync to prevent shell injection
- Replace deprecated npm --production with --omit=dev
- Tighten parseSource regex to [\w.-]+ to reject special chars
- Fix ZH sidebar plugin link (/guide/plugins → /zh/guide/plugins)
- Return plugin name from installPlugin() to avoid duplicated logic
- Use execFileSync for esbuild transpilation
- Fix misleading comment in linkHostOpencli

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-21 19:41:15 +08:00
jackwener 4e274a92b7 v1.1.1 2026-03-21 10:49:39 +08:00
jackwener 9a79501bfd 1.1.0 2026-03-20 20:37:30 +08:00
jakevin 4ab4f88bcd chore(main): release 1.0.6 (#128) 2026-03-20 16:25:30 +08:00
jackwener fafa990acd v1.0.5 2026-03-20 15:51:52 +08:00
jackwener 152cc48091 v1.0.4 2026-03-20 15:36:06 +08:00
AlexYue 31f58ae699 docs: add VitePress documentation site (#112)
- Add VitePress with full navigation, sidebar, i18n (en/zh), local search
- Create 50+ doc pages: guide, adapters (browser + desktop), developer, advanced
- Migrate content from README.md, CONTRIBUTING.md, TESTING.md, CDP.md, CLI-ELECTRON.md
- Migrate all 11 adapter READMEs to structured documentation
- Add new pages: architecture, yaml-adapter guide, ts-adapter guide, ai-workflow
- Add GitHub Actions workflow for deploying to GitHub Pages
- Add Chinese locale pages (getting-started, installation, browser-bridge, etc.)
- Add docs:dev, docs:build, docs:preview npm scripts
2026-03-20 14:11:56 +08:00
jackwener ee35ee723f chore: release version 1.0.3 2026-03-20 11:27:56 +08:00
jackwener 10af754c89 docs: align CDP release notes 2026-03-20 00:51:55 +08:00
jackwener c21250bd88 1.0.1 2026-03-20 00:29:10 +08:00
jackwener 48e277bd0b 1.0.0 2026-03-19 17:00:45 +08:00
jackwener 59c0d639a5 refactor: fix 9 issues from round 2 code review
Bug fixes:
- #1 /logs?level=error returned 404 — use pathname for route matching
- #2 Duplicate initialization — added 'initialized' guard flag

Should fix:
- #4 Added screenshot() to IPage interface
- #5 Graceful shutdown rejects pending requests before exit
- #6 Use process.execPath instead of 'npx tsx' for faster daemon spawn

Cleanup:
- #7 Removed duplicate 'browser' keyword in package.json
- #8 Removed unused normalizeEvaluateSource import from browser.ts
- #9 Changed dynamic import to static import in intercept.ts
- #10 Added explicit throw at end of sendCommand for clarity

61 tests pass (4 test files). Extension: 10.55KB.
2026-03-19 16:36:06 +08:00
jackwener b2fa7daf57 feat: replace @playwright/mcp with lightweight daemon + Chrome Extension
Architecture:
- Micro-daemon (HTTP + WebSocket bridge, ~190 lines, auto-start/idle-exit)
- Chrome MV3 Extension using chrome.debugger CDP (10KB build)
- 5 protocol actions: exec, navigate, tabs, cookies, screenshot
- All DOM ops via JS evaluate — no extension update needed for new features

Key features:
- CDP Runtime.evaluate for JS execution in page context
- Tab management, cookie access via Chrome APIs
- Auto-start daemon on cold boot, idle auto-exit (5min)
- Minimal permissions: debugger, tabs, cookies, activeTab, alarms

Tested: zhihu hot (14.3s), twitter timeline (9.3s)
2026-03-19 16:21:17 +08:00