Commit Graph

22 Commits

Author SHA1 Message Date
jakevin 0f3021a086 fix: relax extension version check, enable all adapter tests (#520)
* feat: zero onboarding, extension version check, and update notifier

- Fail-fast guard in execution.ts: when daemon is running but extension
  is not connected, immediately surface a setup guide instead of waiting
  for the 30s connect timeout

- Extension version handshake: extension sends `hello` with its version
  on WebSocket connect; daemon stores it and exposes via /status; CLI
  warns on mismatch in both execution path and `opencli doctor`

- `opencli doctor` now shows extension version inline and reports
  version mismatch as an actionable issue

- Non-blocking npm update checker: registers a process exit hook so the
  update notice appears after command output (same pattern as npm/gh/yarn);
  background fetch writes to ~/.opencli/update-check.json for next run

- postinstall: print Browser Bridge setup instructions after shell
  completion install for first-time global install users

Bug fixes caught in review:
- discover.ts: add AbortController timeout to checkDaemonStatus() fetch,
  move clearTimeout after res.json() to cover body streaming
- daemon.ts: clear extensionVersion and reject pending requests in
  ws.on('error') handler, not just ws.on('close')
- update-check.ts: skip update notice when process exits with non-zero
  code; read cache once at module load to avoid double disk I/O;
  guard isNewer() against NaN from pre-release version strings

* fix: relax extension version check to major-only in doctor, remove from hot path

* test: enable all adapter tests via wildcard glob, fix apple-podcasts url field

* fix: clearTimeout in finally block, reset extensionVersion on reconnect, fix e2e regex
2026-03-28 00:08:10 +08:00
Lr_2002 310e136a6c feat(paperreview): add paperreview.ai adapter (#464)
* feat(paperreview): add paperreview.ai adapter

* fix(cli): normalize boolean command options

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 22:25:30 +08:00
AstroHan 0773616c1e feat(imdb): add IMDb adapter with 6 commands (#472)
* feat(imdb): add IMDb adapter with 6 commands

Add a public IMDb adapter using browser-based JSON-LD and __NEXT_DATA__
extraction. All commands use Strategy.PUBLIC with browser: true.

Commands:
- imdb search <query> — search movies, TV shows, and people
- imdb title <id> — get movie/show details (Movie, TVSeries, TVEpisode, TVMiniseries, TVMovie, etc.)
- imdb top — IMDb Top 250 chart
- imdb trending — Most Popular Movies
- imdb person <id> — actor/director info with filmography
- imdb reviews <id> — user reviews (first page, max 25)

Shared utils: ID normalization, ISO 8601 duration formatting, locale
forcing, JSON-LD extraction (supports type array filtering), and
anti-bot challenge detection.

* review: harden imdb adapter loading and tests

* test: unblock PR CI on merge head

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 22:21:52 +08:00
槑囿脑袋 c75fea90ad feat(douban): add photo listing and download commands (#474)
* feat(douban): add photo listing and download commands

* refactor(douban): remove unreachable empty download branch

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 21:59:20 +08:00
Conn Ho 15b9bc8e0c feat(producthunt): add Product Hunt CLI adapter (#462)
* feat(producthunt): add Product Hunt CLI adapter

Add three commands:
- posts: RSS feed with optional category filter
- today: latest day's posts from feed
- hot: today's top posts with vote counts (browser INTERCEPT strategy)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(producthunt): add browse command for category best products

Browse top-rated products in any Product Hunt category (e.g. vibe-coding,
ai-agents, developer-tools) with name, tagline, and review count.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(producthunt): add adapter documentation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(producthunt): rebase on main and stabilize selectors

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 16:02:33 +08:00
Conn Ho 22399cee1a feat(36kr): add 36氪 CLI adapter (#461)
* feat(36kr): add 36氪 CLI adapter with 4 commands

- news: latest articles via public RSS feed (no browser needed), includes title/summary/date/url
- hot: trending articles via INTERCEPT strategy, supports --type renqi/zonghe/shoucang/catalog
- search: keyword search via INTERCEPT + DOM scraping
- article: fetch article detail (title/author/date/body) by ID or URL

Also adds vitest adapter project entry for 36kr tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(36kr): add adapter documentation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(36kr): use Shanghai hot-list dates and complete docs

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 15:34:46 +08:00
Xeron ed89157804 fix(jd): filter avif images only from pcpubliccms CDN (#453)
* feat(jd): add item adapter for JD.com product details

Support fetching:
- Product title, price, shop
- Product specifications (品牌, 型号, 规格参数 etc.)
- Main product images
- Detail images from product page

Usage: opencli jd item <sku>

* fix(jd): update test to expect avifImages column

* review: tighten jd item image contract

* fix: stabilize extension packaging and Chinese-site e2e

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 15:12:41 +08:00
Xiao Han 7348231b08 feat(twitter): add likes command (#448)
* feat(twitter): add likes command

* review: harden twitter likes query resolution

* refactor(twitter): share query id resolution

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 12:38:35 +08:00
glwlg 784bbc45f4 fix(xiaohongshu): improve image-text publish flow (#447)
* fix(xiaohongshu): improve image-text publish flow

Match visible 图文 tab labels instead of relying on narrow class selectors, fail early when the page is still on the video publish surface, and avoid injecting images into a generic file input. Add regression coverage for the image-text tab flow and the video-page failure case.

* test(xiaohongshu): include publish tests in adapter project

* fix(xiaohongshu): wait for image-text surface before upload

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 12:25:57 +08:00
tiaot33 e64046219d feat(linux-do): refactor adapters with unified feed, tags, user commands (#434)
* feat(linux-do): refactor adapters with unified feed, tags, user commands

- Replace hot/latest/category with unified `feed` command (tag/category/view routing)
- Add `tags`, `user-topics`, `user-posts` commands
- Add static data files for categories and tags lookup
- Fix error handling: use CliError subclasses instead of raw Error
- Fix Discourse API field mapping in search (tags, created)
- Add strategy: cookie to all YAML adapters
- Update docs and README command listings
- Update E2E tests for new command signatures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* review: resolve linux-do feed from live metadata

* fix: restore linux-do CI

* fix: harden linux-do compatibility

* refactor: stabilize linux-do command migration

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-26 00:45:34 +08:00
Conn Ho 824dc38aab fix(weread): restore positional book-id coverage (#433)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-25 23:58:40 +08:00
jakevin dba93c2739 fix(test): limit default e2e to bilibili/zhihu/v2ex, gate extended browser tests (#423)
Split browser-public.test.ts: core sites (bilibili, zhihu, v2ex) run
by default; all other 20+ site tests moved to browser-public-extended
and gated behind OPENCLI_E2E=1 to prevent AI agents from launching
dozens of browser instances.
2026-03-25 16:13:45 +08:00
jakevin 03d94ba2e1 chore: trim adapter test suite to bilibili, zhihu, v2ex only (#421)
Remove other adapter sites from vitest config to keep test runs
focused and avoid flaky failures from live site changes.
2026-03-25 16:01:15 +08:00
nianyi(likai) 39ca8330c5 feat(douyin): add Douyin creator center adapter (14 commands, 8-phase publish pipeline) (#416)
* feat(douyin): add Douyin creator center adapter (14 commands, 8-phase publish pipeline)

- publish: 8-phase pipeline (STS2 → TOS multipart upload w/ resume → ImageX cover → transcode poll → safety check → create_v2)
- draft: save as draft (phases 1-6 + is_draft:1, no timing)
- videos/drafts/delete/profile/update: content management
- hashtag (search/suggest/hot) / location / activities / collections / stats: discovery & analytics
- _shared: tos-upload (AWS Sig V4, multipart, resume), imagex-upload, transcode poller (encode=2), browser-fetch, sts2, creation-id, timing, text-extra
- 124 tests, tsc clean

* fix(douyin): accept unix timestamp strings

* docs(douyin): add browser adapter guide

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-25 15:36:07 +08:00
Ryan Tan d36a43e805 feat(pixiv): add Pixiv adapter (#403)
* feat(pixiv): add Pixiv adapter with 6 commands

Add support for Pixiv (pixiv.net) with the following commands:
- ranking: daily/weekly/monthly illustration rankings
- search: search illustrations by keyword/tag
- user: view artist profile info
- illusts: list illustrations by artist
- detail: view illustration details (tags, stats)
- download: download original-quality images

All commands use COOKIE strategy to reuse Chrome's logged-in session.
YAML adapters for simple API fetches (ranking, detail, user), TypeScript
for complex logic (search, illusts, download with Referer header).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(pixiv): add unit tests and E2E auth failure tests

- search.test.ts: auth error, result parsing, limit, empty results (4 tests)
- illusts.test.ts: auth error, empty user, two-step fetch, limit (4 tests)
- download.test.ts: auth error, no images, Referer header, partial failure (4 tests)
- Add pixiv to vitest adapter project include list
- Add 5 pixiv commands to E2E browser-auth graceful failure tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(pixiv): correct ranking API path and YAML arg naming

- ranking: use /ranking.php?format=json (not /ajax/ranking which 404s)
- ranking: fix JSON path from data.body.contents to data.contents
- user/detail: rename hyphenated args (user-id → uid, illust-id → id)
  to fix YAML template evaluation (dot access doesn't support hyphens)

All 6 commands verified working against live Pixiv API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(pixiv): use JSON.stringify to prevent code injection in page.evaluate

Address CodeRabbit review: all user inputs (query, userId, illustId,
idsParam) passed to page.evaluate are now serialized via JSON.stringify
instead of direct string interpolation, preventing code injection in
browser context.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(pixiv): address code review feedback

- ranking.yaml: add | json filter to page/limit args for defense-in-depth
- user.yaml: guard illusts/manga/novels with typeof check for robustness
- Extract shared createPageMock to test-utils.ts, deduplicate across 3 test files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(pixiv): use minimal page mock and add download E2E test

- test-utils.ts: slim down to minimal mock (goto, evaluate, getCookies)
  with overrides support, matching upstream's pragmatic mock style
- Add missing download command to E2E browser-auth graceful failure tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(pixiv): address all remaining CodeRabbit review comments

- detail.yaml: add url to columns to match description mentioning "URLs"
- All adapters: differentiate HTTP errors — 401/403 → AuthRequiredError,
  404 → "not found", others → generic "request failed (HTTP N)"
- Tests: use beforeAll to cache registry lookup, avoiding repeated reads
  from global singleton
- Tests: assert error type (AuthRequiredError) not just message content
- Tests: add dedicated test cases for non-auth errors (500) and 404

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(pixiv): add adapter docs and indexes

- Add pixiv.md documentation page under docs/adapters/browser/
- Update docs/adapters/index.md with pixiv entry
- Add Pixiv to sidebar in docs/.vitepress/config.mts
- Update README.md and README.zh-CN.md adapter tables
- Add pixiv to download support tables in both READMEs

Completes the documentation checklist for the pixiv adapter PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(pixiv): address code review findings

- Use CommandExecutionError instead of raw Error for HTTP failures
- Add page.goto() before page.evaluate() to establish browser context
- Fix search keyword double-encoding in URL construction
- Fix ranking.yaml using rating_count instead of illust_bookmark_count
- Throw on batch detail fetch failure instead of silent empty return
- Add beforeEach mock reset in download tests
- Add novels column to user.yaml output

Ensures pixiv adapter follows upstream CliError conventions and handles
edge cases correctly before submitting to upstream.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(pixiv): improve download description in READMEs

- Replace technical Referer header detail with user-facing description
- Describe what users care about: original quality and multi-page support

Technical details belong in code comments, not user-facing docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(pixiv): expand usage examples with all options

- Add ranking mode examples including R18 variants
- Add search filter examples (mode, order, pagination)
- Organize examples by command category for readability

Users need to know available options without reading source code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(pixiv): address second round of CodeRabbit review comments

- Validate illust-id is numeric to prevent path traversal
- Move URL parsing inside per-item try block for graceful error handling
- Add auth error handling for batch detail request (consistent with step 1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(pixiv): extract shared pixivFetch helper, add input validation & batch support

- Create utils.ts with pixivFetch() for unified navigate + fetch + error handling
- Refactor search.ts, illusts.ts, download.ts to use pixivFetch (DRY)
- Add user-id/illust-id numeric validation in TS adapters
- Add batch pagination in illusts.ts for limit > 48 (Pixiv server limit)
- Add comment explaining Pixiv search API dual keyword requirement
- Update tests: new invalid-ID test cases, aligned mock format with pixivFetch

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-25 12:52:21 +08:00
AstroHan 3a7a5e135b fix(grok): preserve conversation across repeated ask calls (#332)
* fix(grok): preserve conversation across repeated ask calls (#330)

The adapter unconditionally navigated to grok.com/ on every invocation,
destroying the existing conversation URL even when --new was not passed.
Since the browser daemon already reuses the same Chrome tab, skipping
navigation lets the tab stay on the current chat thread.

- Only navigate to grok.com/ when --new is true or tab is not on grok.com
- Add tryStartFreshChat to the default path's --new branch (was dead code)
- Add isOnGrok helper with hostname-based domain matching
- Add unit tests for isOnGrok

* test(grok): add adapter to vitest project config

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-24 19:19:40 +08:00
Piotr Yordanov 8869d3b457 feat(linkedin): add timeline feed command (#342)
* feat(linkedin): add timeline feed command

* test(linkedin): add timeline adapter unit tests

Add shape tests and utility function tests for the new timeline command.
Include linkedin in the vitest adapter project config.

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-24 15:54:51 +08:00
jakevin 1b3f74cd6a test: focus adapter coverage on four priority sites (#339) 2026-03-24 12:03:26 +08:00
jackwener e0395ce5ed test: make Vitest project order explicit
Add explicit group ordering for Vitest projects so unit tests run before e2e tests, while keeping the e2e ordering fix from PR #38.\n\nCo-authored-by: RbBtSn0w <hamiltonsnow@gmail.com>
2026-03-17 17:46:03 +08:00
jackwener fbec2f6f5d feat(snapshot): filter contentinfo subtrees, bilibili ad URLs, boilerplate buttons
- Add contentinfo to subtree-level noise filtering (biggest single win)
  - Reuters: 51% → 62%, Google: 57% → 70%, Netflix: 48% → 60%
- Add cm.bilibili.com/cm/api/fees/ ad URL pattern
- Add 广告 keyword to ad detection
- Add back-to-top / 回到顶部 boilerplate button filtering
- Unify ad/boilerplate/contentinfo into single subtree-skip mechanism
- Add vitest config and comprehensive test suite (33 tests)
- Fixture tests skip gracefully when snapshot files are absent

Bump to v0.7.1
2026-03-17 01:26:49 +08:00
jackwener 7eec7ce89f fix: restore tests/ in vitest include for CI compatibility
vitest run tests/e2e/ intersects the CLI path with include patterns,
so tests/ must be in the include glob for CI to find test files.
2026-03-16 17:37:48 +08:00
jackwener 0fd9adf0f5 feat: add 8 new sites — bbc, weibo, boss, yahoo-finance, reuters, smzdm, ctrip, youtube
New adapters ported from bb-sites:
- bbc/news: BBC News RSS headlines (public API)
- weibo/hot: 微博热搜 (browser)
- boss/search: BOSS直聘搜索职位 (browser)
- yahoo-finance/quote: 股票行情 (browser)
- reuters/search: 路透社新闻搜索 (browser)
- smzdm/search: 什么值得买搜索好价 (browser)
- ctrip/search: 携程旅行搜索 (browser)
- youtube/search: YouTube视频搜索 (browser)

Also: build script adds clean-yaml step, docs updated to 28 commands / 16 sites
2026-03-15 08:46:13 +08:00