Commit Graph

41 Commits

Author SHA1 Message Date
jakevin 4475d4efe3 feat(twitter): P1+P2+P3+P4+P5 — search filters, bookmark folders, engagement scoring, sibling dedupe + help docs (#1406)
Round 21 follow-up to #1400 (P0 write-action symmetry, merged `644d4517`). 5 features + help docs unified into one PR per WAWQAQ "全部合成一个 PR" directive.

## Scope

- **P1** (`cf10c098`): `twitter search` `--from / --has / --exclude / --product` filters, mapping to X `from:` / `filter:` / `-filter:` / `f=` operators; legacy `--filter top|live` preserved (--product win on conflict)
- **P2** (`a484a69a`): new `twitter bookmark-folders` + `bookmark-folder <id>`; X Premium GraphQL `bookmarkFoldersSlice` + `BookmarkFolderTimeline`; queryId 三层 fallback (placeholder.json → client-web bundle → pinned constants)
- **P3** (`f209f914`): `--top-by-engagement N` to 7 tweet-shaped read commands (search/timeline/likes/bookmarks/list-tweets/tweets/thread); single helper in `utils.js`; formula `likes×1 + retweets×3 + replies×2 + bookmarks×5 + log10(views+1)×0.5`; **N=0 reference equality no-op** → existing 157 twitter tests 0 churn
- **P4** (`89283fa0`): `TWITTER_BEARER_TOKEN` + composer image helpers extracted to `utils.js` (12 GraphQL adapter dedup); reply hardening; quote adds `--image`
- **P5** (`a3d10a48`): sibling article-scope helper extracted to `shared.js` (9 write commands reuse, dedup with #1400 P0 invariant)
- **docs** (`2a358d80`): help-doc precision (positional-omitted defaults + download/bookmarks/notifications/timeline/lists description thicken; concurrent #1401/#1403 wording preserved)

47 files / +2594/-470. Tests **96 → 216 (+120)**, manifest 798 → 801 (+3), typed-error-lint 190 → 189 (resolved 1 grandfathered sentinel).

## Iteration history (3 review fix commits on top of 6 author commits)

- `7f93779b` — codex-mini1 lead fix1: 3 blocker bundle (P5 host invariant + P2 safe-id + sentinel removal + P1 fallback fail-fast)
- `2a29ecc6` — codex-mini1 lead fix2: P3 help formula consistency (doc/help text matches actual `log10(views+1)×0.5`)
- `df4dcd76` — codex-mini1 lead fix3 (F-P-1 aux catch): P2 `bookmark-folder --limit` upfront validation (`Number(kwargs.limit ?? 20)` + reject non-positive/non-integer + regression `0/negative/fractional/NaN` + `page.goto` zero-call assert)

## 4 progressive blockers caught (codex-mini1 lead 3 rounds + F-P-1 aux 1 round)

1. **P5 host invariant gap** (lead): article-scope helper preserved exact `/status/<id>` path but ignored link host → off-domain `https://evil.com/alice/status/<target>` would satisfy `__twHasLinkToTarget`. Fixed: `https` + X/Twitter host or subdomain + exact `/status/<id>` or `/i/status/<id>` path; query/hash allowed; off-domain/host-suffix/non-https/path-suffix/substring-id rejected; JSDOM positive + 5 negative anchors.

2. **P2 listing→detail round-trip + sentinel** (lead): `bookmark-folders` accepted opaque IDs but `bookmark-folder <id>` only accepted numeric → round-trip broken; new `author: 'unknown'` sentinel created fabricated author URL. Fixed: `[A-Za-z0-9_-]+` opaque safe-id (rejects `/`, `?`, `%`, spaces) + `resolveTwitterQueryId()` sanitization for queryId resolution; sentinel removed → empty author + canonical `/i/status/<id>` URL.

3. **P1 fallback silent tab miss** (lead): pushState fail → fallback typing into search box, `clickProductTabIfNeeded()` silent return on tab not found → user `--product photos` silently degraded to Top results. Fixed: throw `CommandExecutionError` when requested `--product` tab cannot be selected + invalid `--from` / `--limit` upfront pre-nav reject + double-direction tests.

4. **P2 limit silent normalize** (aux): `const limit = kwargs.limit || 20` → `--limit 0` silent → 20; negative/non-integer pre-IO unchecked. Fixed: `Number(kwargs.limit ?? 20)` + require positive integer before `page.goto` + regression covers `0/negative/fractional/NaN` + `page.goto` zero-call.

## Cultural sediment (Round 21 audit checklist 7 rules / 6 dimensions)

This PR **immediately validated 4 of 7 rules** in review pipeline:
- (b) silent-clamp class — P1 fallback silent tab miss (silent semantic-downgrade) + P2 `|| 20` silent normalize
- (e) ID exact-not-substring — P5 host invariant (was only path-exact, not host-exact)
- (f) grandfathered-not-exempt — P5 helper-refactor boundary lost host invariant + P2 new adapter inherited grandfathered `'unknown'` sentinel
- (g) fallback-must-have-success-criterion — P1 fallback path missing post-condition assertion

7 rules / 6 dimensions:
- (a) cross-grep sibling URL pattern — structural
- (b) silent-clamp class — failure mode (input)
- (c) broad querySelector → article-scoping — scope
- (d) missing-validation early reject — boundary
- (e) ID exact-not-substring — identity
- (f) grandfathered-not-exempt (corollary: applies to new file + new helper-refactor boundary; not original-file line-edit) — time-axis
- (g) fallback-must-have-success-criterion (sub-rule g': fallback unit test must include post-condition assertion, not just "doesn't throw") — failure mode (output)

**Cross-PR validation 4-chain on meta-anchor "Structural exactness for identity matching"**:
- #1391 URL layer (`isFacebookAuthRedirectPath`: top-level anchor + `\.php` + `(/|$)` segment edge)
- #1392 URL parser layer (`parseGrokSessionId`: bare UUID exact / URL host-exact-or-subdomain + path-exact)
- #1400 DOM layer (article-scoping: status-id `/\/status\/${id}(?:\/|$)/` regex / segment-array exact)
- #1406 P5 helper-refactor boundary (full URL invariant in shared helper: host+path re-anchored after extraction)
- Common invariant: boundary-lock structural shape; **fuzzy match is silent-failure 温床**; lesson lifecycle = surface-shift not add-and-forget.

**Audit framework self-discipline**: each rule must have grep-able detection signal, otherwise rule degenerates to mantra. Framework is "7 rules + sub-instance pattern in new surface", not frozen 7 rules.

**Round 17 race-mitigation 第 9 连续 race-free execution**: standard alternation cadence (#1400 A 组 → #1406 B 组), lead final + aux final + `@pr-monitor squash?` trigger, pr-monitor proactive ack + serial squash, lead silent on closeout.

## Validation gates (final head `df4dcd76`)

Local: Twitter adapter tests `25 files / 216 tests`, focused P1/P2/P3/P5 tests `99/99`, `node --check` touched runtime, `npx tsc --noEmit`, `npm run build`, manifest 801 entries, typed-error-lint `189/189`, silent-column-drop `103/103`, doc-coverage `140/140`, docs:build clean, listing-id advisory `13` unchanged (wikipedia/trending residual non-Twitter), `git diff --check` clean.

GitHub: build×3 (ubuntu/macos/windows) SUCCESS, unit-test shards SUCCESS, bun-test SUCCESS, adapter-test SUCCESS, audit SUCCESS, doc-coverage SUCCESS, docs-build SUCCESS, smoke-test skipped, PR `CLEAN/MERGEABLE`.

Reviewers:
- Lead: @codex-mini1 (3 fix rounds, all caught proactively + amend P3 help consistency)
- Aux: @First-principles-1 (better-solution triangulation on P2 queryId 三层 fallback + P5 invariant + P3 N=0 reference no-op + caught P2 limit silent normalize)
- Author: @opencli-user (5-feature scope + 7-rule sediment co-author + corollary contributor)
2026-05-08 02:36:39 +08:00
jakevin b9b87a5c64 refactor(facebook/notifications): pipeline→func + typed errors + 7-col contract + runtime upfront limit (Phase 3 P5, #1391)
First Facebook adapter — Pattern C HTML scrape (lead 5 + author 4 = 7 endpoint family probe matrix dual-source negative evidence: graphql×3 / m.facebook redirect / login.php / checkpoint.php / fetch-patch / Messenger relay / ajax legacy 全 unauth 不可达, DOM walk over rendered notification rows + path-anchored auth detection 是当前 reviewable boundary).

Caller-visible delta: 3 cols (index/text/time) → 7 cols (+unread/+url/+notif_id/+notif_type).

[Bug fix] — 5 silent failures resolved
- silent-bad-shape: text.substring(0,150) → full body via per-row 'Mark as read' aria-label
- silent-bad-shape: time || '-' sentinel → string|null typed unknown
- silent-column-drop: unread badge / anchor href / notif_id / notif_t 暴露
- silent-empty-row: /login(.php)? + /checkpoint(.php)? redirect 返 [] → AuthRequiredError; empty/no-recoverable-text → EmptyResultError
- silent-clamp: limit 越界 silent clamp → ArgumentError (1-100), upfront before any navigation (navigateBefore: false)

[Structural refactor]
- pipeline → cli() func form + Strategy.COOKIE + navigateBefore: false (runtime upfront invariant 与 #1387 standard 拉齐)
- module-level pure exports: normalizeNotificationsLimit, stripMarkAsReadPrefix, stripAnchorChrome, parseNotifQuery, extractNotificationRowsFromDoc, isFacebookAuthRedirectPath, buildNotificationsScript
- Live IIFE 通过 \${fn.toString()} 嵌入 (dianping #1313 / hupu #1387 / xiaoe #1388 lineage)
- Locale 表 6 prefix / 4 badge label 显式列出
- AUTH_REQUIRED: sentinel → Node-side AuthRequiredError mapper

[Typed-error hardening]
- Path-anchored auth helper: isFacebookAuthRedirectPath(/^\/(?:login|checkpoint)(?:\.php)?(?:\/|\$)/i) — domain-invariant-first encoding (FB top-level auth-only invariant), 排除 /loginhelp /help/login /account/login/identify
- Three-layer navigateBefore=false invariant lock: registration assertion + manifest absence + executeCommand runtime page.goto-zero-call (test layer 与 invariant layer 完整对齐)
- Row-level silent-empty-row defense: anchor rows with no recoverable body text 直接 skip, 不 emit text:null success row

[Doc fix]
- docs/adapters/browser/facebook.md notifications enrichment + Output table (列类型 / null vs sentinel 语义) + auth/empty error contract
- Boy Scout audit: cross-checked profile / feed / search / marketplace-listings / marketplace-inbox 例 commands 与 args 定义一致

Tests
- notifications.test.js 39/39 + src/execution.test.ts 21/21
- Anti-pattern regression guards: not.toMatch(/text\.substring\(0,\s*150\)/) + not.toMatch(/time\s*\|\|/)
- JSDOM frozen-fixture (slim 13 lines, 0 blank): header listitem skip / full text / unread badge / query parsing / null time / blank-row skip / relative href absolute / 19-case auth path matrix
- typed-error-lint baseline 192 → 191 (silent-sentinel resolved 1)

Review iterations (4 head, A 组 codex-mini0 lead + First-principles-0 aux):
1. 052d2b18 (initial 29 tests) → 376cb50f (lead gate fix: Ubuntu lint + auth path-segment + anchor.href + 5 typed-error func tests)
2. 376cb50f0d6c1340 (pr-monitor grep cross-verify catch /login.php false-negative; lead 加 \\.php 边界)
3. 0d6c13403e5a5ff0 (opencli-user 19-case 实测 + lead 抽 named helper isFacebookAuthRedirectPath domain-invariant-first encoding + 2 row-shape silent-failure 顺手 catch)
4. 3e5a5ff036e44f73 (F-P-0 aux blocker: registry-injected navigateBefore 在 limit validation 之前 fire pre-nav 违反 #1387 upfront boundary; navigateBefore:false + 三层断言 registration/manifest/runtime executeCommand)

Closes #1391
2026-05-07 17:49:57 +08:00
jakevin 9a7dd44b3e refactor(tiktok): 6 read commands -> page-context API (Phase 3 P0, absorbs #1382)
Phase 3 P0: refactor 6 TikTok read commands (explore, following, friends, live, notifications, user) from DOM/network-intercept to TikTok web's own page-context API endpoints, sharing one helper boundary.

Helper boundary (clis/tiktok/utils.js):
- BROWSER_HELPERS: in-browser fetchJson + cleanText + asNumber (null/'' -> null preserve missing-vs-zero distinction) + cookie/msToken plumbing
- VIDEO_ITEM_NORMALIZER: normalize page-context item -> row shape
- assertTikTokApiSuccess(data, label): unify TikTok in-band envelope (status_code/statusCode != 0; code 8 or auth-looking message -> AUTH_REQUIRED; other -> upstream label API failed)
- throwTikTokPageContextError() (Node side): map browser-thrown errors -> AuthRequiredError / EmptyResultError / CommandExecutionError

Per command:
- explore: /api/recommend/item_list/ pagination, --limit upfront ArgumentError
- following: /api/user/list/ relationships
- friends: /api/user/list/ + cross-filter
- live: /api/live/discover/ feed
- notifications: /api/notice/multi/ (status 8 -> AUTH_REQUIRED)
- user (absorbed from #1382): secUid resolve via __UNIVERSAL_DATA_FOR_REHYDRATION__ -> /api/user/detail/, /api/post/item_list/ pagination, /api/search/general/full/ exact-author fallback. !secUid -> EmptyResultError (NOT AuthRequiredError; auth still covered by HTTP 401/403 + envelope status_code 8/auth-looking msg). source field = bootstrap | profile-api | search-fallback in row/columns/manifest/docs/tests.

Closes #1382 (absorbed; #1382 closed without separate merge per WAWQAQ direction).

Validation:
- clis/tiktok/ tests: 38/38
- typed-error-lint: 190/190
- silent-column-drop: 103/103
- doc-coverage: 140/140
- docs:build pass, manifest no drift
- GitHub gates: build x3 / unit x2 / bun / adapter-test / audit / doc-coverage / docs-build all SUCCESS, smoke skipped, MERGEABLE

Reviewers: codex-mini0 (lead, push 4 boundary fixes 18cdf930 -> a1f1ada4 -> 53499609 -> 276dce3b), First-principles-0 (aux, caught secUid auth-vs-empty boundary + verified 6 cmd integral helper boundary).
2026-05-07 16:15:30 +08:00
jakevin 67cde0e263 enrich(coupang): product detail cmd + replace silent clamp/sentinel/Error with typed errors (#1370)
* enrich(coupang): add product detail cmd + replace silent clamp/sentinel/Error with typed errors

Two enrichment changes plus three silent-failure fixes on top of existing
search / add-to-cart.

New cmd: coupang product
─────────────────────────
Pairs with search as the listing↔detail round-trip target. Reads a logged-in
product page and extracts a single canonical row with price, original_price,
discount_rate, rating, review_count, seller, brand, rocket, delivery_promise,
image_url, url. Three-source extractor (JSON-LD Product schema → bootstrap
globals → DOM) merged in priority order, mirroring the search.js pattern.

The columns use string|null typing — null means "upstream did not provide
this field on this product" (e.g. some items have no original_price).
Failures (login wall / page mismatch / page failed to render) raise typed
errors instead of silently returning empty rows, so callers can treat any
returned row as real data.

Search column shape: added product_id
─────────────────────────────────────
Listing must pair with detail by id. The data was already extracted by
normalizeSearchItem; only the columns array needed updating so the field
projects through to the rendered row. Per the listing-id-pairing convention
(PR #1297) the new column lets agents round-trip rows directly into
`coupang product` without re-scraping URLs.

Silent-failure fixes
────────────────────
1. search --limit silent clamp.
   Old: `Math.min(Math.max(Number(kwargs.limit||20),1),50)` silently
        rewrote `--limit 999` to 50 and `--limit 0` to 1.
   New: `parseLimitArg(raw, 20, 50)` throws ArgumentError on out-of-range
        / non-integer / negative input. Same convention as the typed-fail-fast
        memory & PR #1289.

2. search --page silent clamp.
   Old: `Math.max(Number(kwargs.page||1),1)` silently lifted negative pages.
   New: parsePageArg throws ArgumentError on non-positive input.

3. Generic `throw new Error(...)` → typed errors.
   - Empty query, unsupported --filter, missing --product-id/--url
     → ArgumentError
   - Login wall detection → AuthRequiredError('coupang.com', ...)
   - Empty result / filter-not-rendered → EmptyResultError
   - PRODUCT_MISMATCH / OPTION_REQUIRED / button-not-found / unknown
     ack failure (add-to-cart) → CommandExecutionError
   - The PRODUCT_MISMATCH and `actualProductId || 'unknown'` sentinel were
     also fixed (silent-sentinel was the audit hit there).

Coverage
────────
- 21 contract assertions in clis/coupang/coupang.test.js covering
  parseLimitArg / parsePageArg (no silent clamp), registry shape (search has
  product_id, product is read-class with expected columns, add-to-cart is
  write-class), and typed-error pre-flight rejections (empty query / bad
  filter / out-of-range limit & page / missing detail args).
- Manifest 763 → 764 (+1 entry: coupang/product).
- Audits: typed-error-lint 196 → 194 (resolved 2 silent-clamp/sentinel
  baseline entries; baseline updated). silent-column-drop 103/103 unchanged.

* fix(coupang): tighten product id and browser errors

* fix(coupang): require real product urls
2026-05-07 00:18:57 +08:00
jakevin dcaae37068 refactor(registry): remove dead adapter metadata (#1369)
* refactor(registry): remove dead adapter metadata

* docs(changelog): note header strategy removal
2026-05-06 23:49:39 +08:00
jakevin 69ee36f997 fix(linkedin): surface detail_error on --details (no silent catch / no silent empty) (#1363)
* fix(linkedin): surface detail_error on --details (no silent catch / no silent empty)

The previous --details enrichment path had two indistinguishable failure modes
that both produced `description: '', apply_url: ''`:

1. `if (!job.url)` early return — row had no jobId, so we couldn't navigate.
2. `} catch {}` — page.goto / page.evaluate threw (network, timeout, parse error).

Callers couldn't tell "upstream had no description" from "we failed to fetch",
and the catch swallowed every error without logging. For an enrichment that
costs one page navigation per row, silent failure is especially harmful — users
just see an empty cell with no way to debug.

Fix: replace empty strings with `null` for missing/failed rows, add a new
`detail_error` column (string|null) carrying a short typed reason:

  - 'no url'                — row had no jobId
  - 'fetch failed: <msg>'   — page.goto / page.evaluate threw
  - 'missing description'   — page loaded but body was empty
  - null                    — success

Every failure is also logged to stderr with the offending URL so debugging is
possible. Per-row failures still don't abort the batch (the original intent),
but they're now visible.

Tests: 13 new contract assertions in clis/linkedin/search.test.js covering
parseCsvArg, mapFilterValues (ArgumentError on unknown values), decodeLinkedinRedirect,
and 5 enrichJobDetails paths (no-url / goto-throw / empty-description / success /
multi-row-mixed). Added `export const __test__` for testability.

Audits clean: typed-error-lint 196/196, silent-column-drop 103/103.

* fix(linkedin): fail fast on auth walls
2026-05-06 22:57:09 +08:00
jakevin da2453cfbd enrich(reuters): article-detail + bug fixes (silent clamp, silent error envelope) (#1362)
* enrich(reuters): article-detail + bug fixes (silent clamp, silent error envelope)

Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #2.

- `reuters article-detail` — full article body + canonical metadata for a
  Reuters URL. Pairs with `reuters search` (use the `url` column to
  round-trip).

- **Silent clamp on `--limit` removed**: out-of-range values now raise
  `ArgumentError`. Validation happens before browser navigation.
- **Silent error envelope removed**: the in-page IIFE used to swallow
  `fetch` errors with `catch(e) {}` and return `{error: ...}`, then the node
  side did `if (!Array.isArray(data)) return [];`. Now:
  - in-page IIFE returns `{ ok, status, body, error? }` raw envelope
  - node side throws typed errors:
    - `CommandExecutionError` on in-page exception
    - `CliError(FETCH_ERROR)` on non-2xx upstream
    - `CommandExecutionError` on captcha HTML (200 + non-JSON body)
    - `EmptyResultError` on empty articles array
- **Empty query**: now `ARGUMENT_INVALID` instead of triggering an empty
  upstream call.
- **Column shape enriched**: previously dropped `section_path` and
  `authors` are now stable columns.

- `docs/adapters/browser/reuters.md`: full Commands / Columns / Error
  Behaviour section (was a 3-line stub).
- `docs/adapters/index.md`: add `article-detail` to the commands cell.

27 contract assertions across `parseLimit` / `mapSearchArticles` /
`mapArticleDetail` / `buildSearchScript` / `buildArticleDetailScript`
+ registry-level checks for both commands (Strategy, ARG validation
before nav, every typed-error path, success path).

- typed-error-lint: 196 → 195 (silent-clamp resolved on
  `clis/reuters/search.js:18`); baseline updated.
- silent-column-drop: 103 = 103 (unchanged).
- listing-id-pairing: advisory only (article-detail keys off `url`).

757 → 758 entries (+1 for `article-detail`).

* fix(reuters): type auth and fetch failures

* fix(reuters): preserve search detail round trip
2026-05-06 20:00:36 +08:00
jeff_woo f4637486b0 fix(twitter): rewrite followers command using DOM extraction (#1324)
- 重写 `clis/twitter/followers.js` 从 INTERCEPT (已坏) 改为 Strategy.UI DOM extraction
- bio 提取用 `data-testid$="-follow"` selector minus pattern (locale-independent button identification)
- drop `followers` column (DOM 不可靠) — manifest 同步 `strategy=ui` + columns `[screen_name,name,bio]`
- typed fail-fast: bad limit / empty user → ArgumentError;无登录 profile link → AuthRequiredError;followers link selector drift → SELECTOR typed error;空 followers → EmptyResultError;不再 silent `return []` / sentinel rows
- 恢复 `normalizeScreenName()` (opencli-user `b37f7ae3` 误删):`@elonmusk` / `/elonmusk` 不再走错路径
- `page.scroll('bottom')` no-op 改 `page.autoScroll({ times: 1, delayMs: 500 })`,避免 `--limit` 超首屏时 silent partial
- CI 全绿 (smoke-test SKIPPED);A 组 codex-mini0 lead green + First-principles-0 aux green on `dc8ac93`
- F-P-0 first-principles reflection: 长期更稳方向是 GraphQL helper (像 twitter/following),本 PR 是 INTERCEPT 已坏情况下的最小可用收口

Round 14 A 组对位收尾。
2026-05-06 00:42:10 +08:00
jakevin 4de1b42ab7 chore(convention): retire listing↔detail id pairing CI gate, keep advisory (#1316)
PR #1297 introduced a CI gate that fails when a site has both a listing
and a detail command but the listing rows don't carry an id-shaped column.
The gate came with a 10-entry EXEMPT map (topic-string trending,
profile-attribute rows, UI-only sessions, ...) where each exemption
recorded a "why this listing legitimately doesn't pair" reason.

By the same filter that closed PR #1311 (write-without-delete-pair gate):

  Is "listing should pair with detail" a *permanent* anti-pattern, or
  case-by-case business judgment?

It's case-by-case. Topic-string listings and profile-attribute rows
genuinely don't pair with a detail command. The fact that we needed an
EXEMPT map with 10 entries and individual reason strings is the smell —
it's not the rule winning, it's the rule failing. Forcing every adapter
PR to either add an id column or file an exemption was a higher cognitive
cost than the silent-loss bugs the rule actually catches.

Changes:

- .github/workflows/ci.yml — drop the "Check listing↔detail id pairing"
  step. Other gates (silent-column-drop, typed-error-lint) stay in place.
- package.json — rename the script from `check:listing-id-pairing` to
  `advise:listing-id-pairing` to make the advisory nature explicit.
- scripts/check-listing-id-pairing.mjs — drop the `--strict` flag and the
  EXEMPT map. The script now always exits 0 and prints an advisory report
  of listings that don't carry an id-shaped column. Reviewers/authors use
  it as guidance, not a gate.
- docs/conventions/listing-detail-id-pairing.md — rewrite from "MUST" to
  "soft convention". Adds an explicit "why advisory, not a gate" section
  that lists the legitimate non-pairing categories so future readers know
  the rule's boundary.
- docs/developer/ts-adapter.md — match the advisory tone in the
  adapter-author guidance.

The doc, the script, and the column patterns table all stay — agents and
adapter authors can still consult them. What's gone is the CI failure and
the per-PR exempt-list maintenance burden.

Net diff: -34 lines (gate + EXEMPT map removed, advisory-tone doc adds
a small "why advisory" section).
2026-05-05 00:34:18 +08:00
jakevin c4a1d2a91c fix(audit): reduce silent column drop false positives (#1315) 2026-05-05 00:29:20 +08:00
jakevin 5ad0b81d92 ci: gate new typed error lint violations
Adds a baseline CI gate for convention-audit typed-error lint findings. Also refreshes the silent-column-drop baseline for dianping changes already on main.
2026-05-04 23:45:36 +08:00
jakevin 46d0f24f57 ci: gate new silent column drops
Adds a baseline CI gate for convention-audit silent-column-drop findings so CI rejects only newly introduced table-output loss.
2026-05-04 23:24:13 +08:00
jakevin 0f806e9473 feat(dianping): browser adapter — search + shop on www.dianping.com (#1309)
* feat(dianping): browser adapter — search + shop on www.dianping.com

Adds two browser-mode adapters for the dianping (大众点评) PC site:

- `dianping search "<keyword>" --city <name|id> --limit <n>`: keyword
  shop/restaurant search. Returns rank, shop_id, name, rating, reviews,
  price, cuisine, district, url. shop_id round-trips into `dianping shop`.
- `dianping shop <shop_id>` (alias `detail`): shop detail sheet
  (field/value rows: name, rating, breakdown 口味/环境/服务/食材, reviews,
  price, rank, hours, address, subway, features, url).

Both use Strategy.COOKIE on www.dianping.com (the PC site renders search
SSR and does not require JS hydration). m.dianping.com is intentionally
crippled for non-mobile UAs, so it's not used.

Auth detection (utils.detectAuthOrEmpty) inspects both response text and
final URL for the Meituan Yoda captcha redirect (verify.meituan.com) and
the dianping login redirect; raises AuthRequiredError with the captcha
URL embedded so the user can clear it manually in the same profile.

Listing↔detail id pairing: search.shop_id → shop.<id>. Adds 'shop' to
DETAIL_NAMES in scripts/check-listing-id-pairing.mjs so the convention
gate scans this site (35 sites / 78 listings now covered).

* fix(dianping): harden browser failure classification

* fix(dianping): fail on partial missing shop ids
2026-05-04 23:00:38 +08:00
jakevin 0a85e73aa5 feat(convention): listing↔detail id pairing rule + CI gate (#1297)
* feat(convention): listing↔detail id pairing rule + CI gate

Adds a hard convention: when a site exposes both a listing-class command
(search / hot / top / recent / ...) and a detail-class command (read /
paper / article / view / ...), every listing row MUST surface an id-shaped
column whose value round-trips into the detail command. Without that, an
agent has no way to follow up on a listing row except re-searching by
title or scraping URLs out of band — both of which break the agent-native
contract.

What's in this PR

- docs/conventions/listing-detail-id-pairing.md — full rule, examples
  table, why-it-matters, what counts as id-shaped, exemption taxonomy,
  how to add an id column to a listing.
- scripts/check-listing-id-pairing.mjs — validator that reads
  cli-manifest.json, classifies each entry as listing / detail / other,
  and fails when a listing on a site that also has a read-detail command
  is missing an id-shaped column. Exemption allowlist records WHY each
  pair is exempt so future maintainers know what to verify.
- npm run check:listing-id-pairing — strict-mode wrapper.
- CI: new step in build job runs the validator after the manifest
  freshness check on Linux.
- docs/developer/ts-adapter.md — cross-link from the adapter authoring
  guide.
- docs/.vitepress/config.mts — sidebar entries for the new conventions
  section.

Fixes brought to zero violations

- 1688/search: add offer_id (already extracted, just surfaced)
- bluesky/user: add uri (AT URI round-trips into bluesky/thread)
- tieba/search: add id + url (thread_id already extracted)
- tieba/hot: add url (rows are topics, not threads — url is the
  best-effort round-trip handle, doc'd as such)

Exemptions (intentional, doc'd in EXEMPT map with rationale)

- nowcoder/hot, bluesky/trending, twitter/trending — listing rows are
  topic strings, not posts.
- lesswrong/user, reddit/user — rows are profile-attribute key/value
  pairs, addressed by the username arg.
- discord-app/search — desktop UI session, message ids not extractable.
- notion/search — Strategy.UI Quick Find, page ids not exposed in DOM.

Validator output after this PR: 32 sites scanned, 75 listings checked,
7 exempted, 0 violations.

* fix(convention): tighten listing id gate

* fix(convention): close url-derived id loophole
2026-05-04 20:54:14 +08:00
jakevin 79a15e8353 Remove unused OPENCLI_SKIP_FETCH env var (#987)
The adapter sync already has version caching (skips if same version)
and makes no network requests, so this opt-out flag adds no value.
2026-04-13 14:09:10 +08:00
jakevin 988ed19223 fix: clean up stale .yaml adapter files from older versions (#953) (#986)
* fix: clean up stale .yaml adapter files from older versions (#953)

Users upgrading from v1.6.x retain .yaml adapter files in
~/.opencli/clis/ that trigger "Ignoring YAML adapter" warnings on
every run. The hash-based sync only tracks .js files, so these
legacy .yaml files are never cleaned up.

Add a cleanup step (3b) that removes .yaml/.yml files from user
adapter directories when the corresponding site exists in the
official package (i.e., the site has been migrated to .js).

* fix(fetch-adapters): narrow stale yaml cleanup
2026-04-13 13:17:27 +08:00
jakevin 72bc86cf41 fix: code audit round 2 — safety, hot-reload, error diagnostics (#982)
* fix: code audit round 2 — pruneEmptyDirs, evaluateWithArgs, hot-reload, error cause chain

1. pruneEmptyDirs: use path.relative() instead of startsWith() to prevent
   false boundary matches on overlapping directory names
2. evaluateWithArgs: add safe evaluate method that auto-serializes args via
   JSON.stringify, preventing injection by design
3. Hot-reload: detect mtime changes on user adapter files in daemon mode,
   invalidate module cache so edits take effect without restart
4. toEnvelope: preserve error cause chain in verbose mode for better
   production debugging

* fix: address review feedback on code audit round 2

- pruneEmptyDirs: resolve() paths before relative() check
- evaluateWithArgs: validate keys are valid JS identifiers
- hot-reload: only bust ESM cache on reload, not first load
- toEnvelope: move cause serialization into toEnvelope itself
  so all consumers (AI agents, MCP tools) get cause chain
2026-04-13 09:36:53 +08:00
jakevin ffb61c51ea fix: address code audit findings (C1-C4, I1, I4, I6) (#981)
* fix: address code audit findings (C1-C4, I1, I4, I6)

Security:
- C1: Fix page.evaluate injection in browser type/select commands and
  6 adapter files by using JSON.stringify for user input interpolation
- C2: Close WebSocket on CDP connect timeout to prevent resource leak
- C3: Reject CDP connect promise on Page.enable failure instead of
  silently swallowing the error

Reliability:
- C4: Guard against corrupted adapter-manifest.json hashes to prevent
  false-positive override deletion
- I1: Throw on pre-navigation failure instead of warn-and-continue
- I4: Use Map<string, Promise<void>> for lazy module loading to prevent
  concurrent double-imports of the same adapter

Performance:
- I6: Replace O(n) registry alias cleanup with O(k) direct deletion

* fix: address self-review findings on PR #981

- C1: add quotes around CSS selector attribute values in browser
  type/select to match other commands (get text/value/attributes)
- C2: clear this._ws in timeout handler to prevent race with open event
- C4: refine corruption guard — treat null/undefined hashes as empty,
  only skip sync for truly invalid types (string, number, array)
2026-04-13 09:24:01 +08:00
jakevin 3c088da53e refactor: smart sync adapters — hash-based diff instead of full copy (#966)
* refactor: smart sync adapters instead of full copy (#sparse-override)

Replace unconditional full-copy of all adapters to ~/.opencli/clis/ with
hash-based smart sync that only copies files whose content has changed.

Changes:
- fetch-adapters.js: use SHA-256 content hashes to skip unchanged files;
  store per-file hashes in adapter-manifest.json
- discovery.ts: simplify ensureUserAdapters() to only create the directory
  (no longer triggers full copy on first run)
- main.ts: fix fast completion to check manifest file existence instead of
  directory existence (sparse override may have empty user dir)
- cli.ts: add `opencli adapter eject/reset/status` commands for managing
  local adapter overrides
- engine.test.ts: add tests for empty user dir and ensureUserAdapters

* fix: address review blockers — site-level sync + reset --all

1. Fix `adapter reset --all`: change <site> from required to optional
   argument so --all can be used without specifying a site name.

2. Change smart sync from file-level to site-level granularity:
   if any file in a site has changed upstream, overwrite the entire
   site directory. This matches the agreed product semantics — local
   modifications to any file in a site are replaced when upstream
   updates that site.

* fix: delete old site dir before writing updated adapter files

When a site has upstream changes, delete the entire site directory
first, then write the new version. This prevents stale files from
older versions lingering in the user directory.

* fix: reset --all preserves custom sites, only removes official overrides

Blocker 3 fix: reset --all now checks BUILTIN_CLIS to identify official
sites and only deletes those, preserving user-created custom sites.

* refactor: sparse sync deletes local overrides instead of copying new versions

Changed fetch-adapters.js semantics per team agreement:
- When an official site has upstream changes, DELETE the local override
  instead of copying the new version into ~/.opencli/clis/
- Runtime automatically falls back to package baseline
- ~/.opencli/clis/ becomes a true sparse override layer

* fix: reset <site> rejects custom sites, only allows official overrides

Single-site reset now checks BUILTIN_CLIS before deleting, matching
the same protection that reset --all already has.

* fix: reset <site> allows custom sites per product decision

Per @WAWQAQ: explicit single-site reset should work on custom sites too.
Differentiate messaging: official sites say "using official baseline",
custom sites say "removed custom site".

reset --all still only removes official overrides (bulk safety).

* fix: reset --all deletes all local sites including custom per product decision

Per @WAWQAQ: --all should clear the entire local working cache,
including custom sites. Single-site reset already handles both types.
2026-04-11 21:28:40 +08:00
jakevin 63b7b291ab fix: clean up stale .ts adapter files during upgrade (#948)
Older versions (pre-1.7.1) shipped adapters as .ts files. When users
upgrade to a .js-only version, the old .ts files are left orphaned in
~/.opencli/clis/. Add a cleanup step that removes .ts files when a
corresponding .js official adapter exists.
2026-04-11 12:36:02 +08:00
jakevin a9d21f3de0 fix: project hygiene — docs, lint, daemon restart (#942)
* fix: project hygiene — docs, lint, daemon restart, code fence

- Update Node version requirement from >= 20 to >= 21 in 7 doc files
  (README, README.zh-CN, installation guides, troubleshooting)
- Update adapter count from 79+ to 87+ in READMEs
- Remove duplicate `lint` script (identical to `typecheck`)
- Fix TESTING.md CI matrix: Node ['22'] instead of ['20', '22']
- Fix autofix SKILL.md code fence escaping (\``` → ~~~)
- Add daemon restart to postinstall so updated adapters are picked up
- Fix preuninstall to respect OPENCLI_DAEMON_PORT env var

* fix: align docs and skills with JS-first adapter contract

Adapters are now .js files (not .ts). Update all references across:
- README.md, README.zh-CN.md, CONTRIBUTING.md
- docs/guide/getting-started.md, docs/index.md
- skills/opencli-browser/SKILL.md, skills/opencli-explorer/SKILL.md

The runtime (discovery.ts) only loads .js from user clis/ directories,
and `opencli browser init` generates .js scaffolds. Documentation was
still teaching users to create .ts files.

* fix: update CI matrix to Node 22 only (drop Node 20)

package.json requires Node >= 21 (styleText dependency). The CI matrix
was still testing Node 20 which doesn't meet this requirement.

* fix: revert incorrect daemon restart from postinstall

The daemon (browser bridge) only handles CDP communication — it has no
knowledge of adapters. Adapter discovery, loading, and execution all
happen in the CLI process, which is fresh each invocation. The
_loadedModules cache in execution.ts is process-local and not a real
staleness concern. Remove the unnecessary restartDaemon() call.
2026-04-11 00:50:05 +08:00
jakevin f92f7571b4 chore: remove unused test-site.mjs script (#940)
Not referenced in package.json, CI, or documentation.
2026-04-11 00:19:02 +08:00
jakevin cc18ed67b7 fix: sync package-lock.json to unblock CI (#937)
* fix: sync package-lock.json with package.json dependencies

package-lock.json was missing @emnapi/core@1.9.2 and
@emnapi/runtime@1.9.2 (transitive deps of @emnapi/wasi-threads),
causing `npm ci` to fail on all CI jobs.

* fix: resolve remaining CI failures after TS-to-JS adapter migration

- vitest.config.ts: update adapter project include/exclude from .test.ts
  to .test.{ts,js} to match converted adapter test files
- check-doc-coverage.sh: skip adapter directories containing only utility
  files (prefixed with _), fixing false positive for clis/slock/
- linux-do/topic-content.test.js: fix hardcoded reference to topic.ts
  (now topic.js after PR #928 migration)
2026-04-10 23:29:09 +08:00
jakevin 2288cf7149 fix: clean up legacy shim files and stale tmp files on upgrade (#934)
* fix: clean up legacy shim files and stale tmp files on upgrade

Add cleanup steps to fetch-adapters.js that run on every version upgrade:

1. Remove legacy compat shim files from ~/.opencli/ (registry.js,
   errors.js, utils.js, etc.) that were created by an older approach
   using file:// re-exports. Current approach uses node_modules symlink.
   Only deletes files containing "export * from 'file://" to avoid
   removing user-created files.

2. Remove legacy compat shim directories (browser/, download/, errors/,
   etc.) using the same safety check.

3. Clean up stale .plugins.lock.json.tmp-* files left behind by
   crashed processes. These accumulate over time (108 found on one
   machine) and clutter ~/.opencli/.

* fix: check every file in legacy shim directories before deleting

Instead of checking only the first file and deleting the entire
directory, now checks each file individually and only deletes files
matching the shim pattern. Directory is removed only if empty after
individual file cleanup.
2026-04-10 18:58:44 +08:00
jakevin 2457002167 chore: remove migration residuals (mapDistToSource, clean-yaml) (#931)
- Remove mapDistToSource() from diagnostic.ts — mapped dist/clis/
  paths back to clis/ but dist/clis/ no longer exists after JS-first
  migration. The function always returned null.
- Simplify resolveAdapterSourcePath() to check candidates directly
  without the dead dist→source mapping detour.
- Delete scripts/clean-yaml.cjs — walked dist/clis/ to delete YAML
  files, but dist/clis/ no longer exists.
- Remove clean-yaml script entry from package.json.
2026-04-10 16:41:44 +08:00
jakevin d2974a9ff6 refactor(adapters): convert adapter layer from TypeScript to JavaScript (#928)
* refactor(adapters): convert adapter layer from TypeScript to JavaScript

Core framework stays TypeScript; adapter layer moves to JS-first.
Adapters are essentially "executable config + browser scripts" that
barely use TS features — this simplifies the build/distribution pipeline
by removing the dist/clis/ intermediate compilation step.

Changes:
- Convert all 753 adapter files in clis/ from .ts to .js
- Update tsconfig to exclude clis/ from compilation
- Simplify build-manifest to scan clis/*.js directly (no dist/clis/)
- Update discovery, main, fetch-adapters to load JS adapters from clis/
- Update generate-verified to output .js artifacts
- Update package.json files field: dist/clis/ → clis/
- Fix all test files for the .ts → .js transition

* fix(main): use findPackageRoot for BUILTIN_CLIS path

The previous relative path (../../clis from __dirname) only worked for
dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname
is <repo>/src — resolving to /clis instead of <repo>/clis.

Use findPackageRoot() which works for both dev and prod paths.
2026-04-10 14:52:18 +08:00
Kai 97a547c6c5 fix: avoid inserting completion config inside multi-line shell commands (#796)
* fix: avoid inserting completion config inside multi-line shell commands

The postinstall zshrc insertion logic splits backslash-continued blocks
(e.g. zinit stanzas) when it finds a compinit match inside them, which
breaks the user's shell config. Walk backward past continuation lines
so the insertion lands before the entire logical command.

* fix: append zsh completion to end of .zshrc instead of splicing

Replace the fragile compinit-searching splice logic with a simple
append, matching the strategy already used for bash. This avoids
breaking multi-line commands (e.g. zinit blocks with zicompinit).

Still detects existing compinit to avoid adding a duplicate call.

* fix: stop modifying shell rc files in postinstall

Replace the fragile .zshrc/.bashrc modification logic with a safer
approach: only write completion files and print setup instructions.

The previous approach tried to parse and splice into rc files, which
broke multi-line shell commands (e.g. zinit blocks with backslash
continuations matching /compinit/). Instead of attempting to fix the
parser, remove rc modification entirely — this matches the approach
used by rustup, homebrew, and other CLI tools.

Closes #788

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-05 17:39:23 +08:00
jakevin 639a31fc84 fix: review follow-ups for monorepo adapter separation (#783)
* fix: review follow-ups — better first-run log, OPENCLI_FETCH=1 skips version check

- Clarify first-run log message: "copying adapters (one-time setup)"
- Add comment explaining why scriptPath uses two levels of ../
- OPENCLI_FETCH=1 now bypasses version-skip to allow forced refresh

* fix: update doc-coverage script path after clis/ move

check-doc-coverage.sh still referenced src/clis/ after PR #782 moved
adapters to root clis/. This caused CI to fail with "0/1 documented".

* fix: resolve package root dynamically for symlink and first-run paths

The symlink at ~/.opencli/node_modules/@jackwener/opencli pointed to
dist/ instead of the package root in prod mode, breaking user TS CLIs
that import from '@jackwener/opencli/registry'.

The first-run scriptPath also resolved incorrectly in dev mode.

Extract findPackageRoot() that walks up to find package.json, fixing
both paths for dev (src/) and prod (dist/src/) layouts.
2026-04-05 02:25:53 +08:00
jakevin 80eef46b4e refactor: monorepo adapter separation (clis/ at root) (#782)
* refactor: move adapters from src/clis/ to root clis/ for monorepo separation

Separates CLI adapters from the core runtime to prepare for independent
adapter distribution via postinstall fetch.

Key changes:
- Move src/clis/ → clis/ (adapters at repo root)
- Change tsconfig rootDir from "src" to "." so tsc compiles both
- Create root-level shim files (registry.ts, errors.ts, etc.) so adapter
  relative imports (../../registry.js) resolve correctly
- Update build-manifest.ts, main.ts paths for new dist/src/ structure
- Expand ensureUserCliCompatShims() to cover all adapter import targets
  (types, utils, logger, launcher, browser/*, download/*, pipeline/*)
- Add scripts/fetch-adapters.js postinstall for ~/.opencli/clis/ sync
- Update vitest.config.ts adapter test paths
- Add package.json files field to exclude adapters from npm package

Official adapter files are unconditionally overwritten on update;
user-created files not in the manifest are preserved.

* fix: add dist/clis/ and cli-manifest.json to npm files, harden fetch-adapters

- Add dist/clis/ and dist/cli-manifest.json to package.json files field
  so built-in adapters and manifest ship with the npm package
- Replace execSync with execFileSync to prevent command injection
- Add version check to skip redundant adapter fetches
- Track tmpRoot explicitly for reliable cleanup

* fix: address review blockers — manifest-based updates, global-only fetch, first-run fallback

1. Manifest-based update strategy:
   - Read old manifest to identify previously-official files
   - Clean up files removed upstream (in old manifest but not new)
   - User-created files (never in any manifest) remain untouched

2. Only run fetch-adapters on global install (npm_config_global=true)
   or explicit OPENCLI_FETCH=1, preventing heavy side effects for
   local/dev installs

3. First-run fallback in discovery.ts:
   - ensureUserAdapters() checks for adapter-manifest.json
   - If missing and ~/.opencli/clis/ is empty, spawns fetch-adapters.js
   - Guarantees adapters are available even with --ignore-scripts

* fix: remove OPENCLI_FETCH env var, use internal _OPENCLI_FIRST_RUN instead

* feat: also support OPENCLI_FETCH=1 for explicit adapter fetch trigger

* simplify: replace git clone with local copy from dist/clis/

Adapters already ship in the npm package (dist/clis/), so there's no
need to clone from GitHub. Copy directly from the installed package:

- Eliminates git, curl, tar dependencies
- No network calls in postinstall
- No timeout/offline issues
- Version always matches the installed CLI
- ~65 lines of clone/download code replaced by one cpSync loop
2026-04-05 01:46:36 +08:00
Josh e18e0ed7a4 fix(browser): mention Chromium in Browser Bridge hints (#738) 2026-04-03 22:28:16 +08:00
bhutano 8e37f66e53 fix(spotify): follow-up fixes for token refresh, null guards and credentials guidance (#591)
* fix(spotify): fix token refresh, null guards, env parse, missing credentials guidance, postinstall template

* fix(spotify): restore credential guardrails

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-30 13:03:25 +08:00
bhutano 1ae1c82c4a feat(spotify): add Spotify playback adapter (#560)
* feat(spotify): add Spotify playback adapter

Adds a new adapter for controlling Spotify via the official Web API.
Uses Strategy.PUBLIC with OAuth2 — no browser session required.

Commands: auth, status, play, pause, next, prev, volume, search, queue, shuffle, repeat.
Credentials are loaded from ~/.opencli/spotify.env or environment variables.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(spotify): rename index.ts → spotify.ts and fix CliError calls

- Renamed src/clis/spotify/index.ts to spotify.ts so the build-manifest
  picks it up (index.js is intentionally excluded from manifest scanning)
- Fixed 4 CliError calls: constructor now requires (code, message, hint?)
  so each throw now passes an appropriate error code as first argument

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(spotify): fix token refresh corruption, env parse, null guards, validation

- refreshAccessToken: check res.ok before parsing; construct Tokens object
  directly instead of mutating loadTokens() result to avoid writing
  undefined/NaN on Spotify error responses; preserve existing refresh_token
  when Spotify omits it from the response
- loadEnv: split on first '=' only so values containing '=' are preserved
- SCOPES: remove write/library/top scopes not used by any command
- status: guard against data.item being null (active device but no track)
- volume: validate 0-100 range before API call
- auth: check tokenRes.ok on initial token exchange; add server.on('error')
  handler for EADDRINUSE; add 5-minute timeout with clearTimeout on close

* feat(postinstall): auto-create ~/.opencli/spotify.env template on install

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(spotify): guard null progress, podcast items, missing tracks data, corrupted tokens, invalid search limit

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(spotify): improve missing credentials error with step-by-step guidance

* fix(spotify): harden setup and add docs coverage

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-29 17:28:37 +08:00
jakevin 7617dff262 feat: zero onboarding, extension version check, and update notifier (#479)
* feat: zero onboarding, extension version check, and update notifier

- Fail-fast guard in execution.ts: when daemon is running but extension
  is not connected, immediately surface a setup guide instead of waiting
  for the 30s connect timeout

- Extension version handshake: extension sends `hello` with its version
  on WebSocket connect; daemon stores it and exposes via /status; CLI
  warns on mismatch in both execution path and `opencli doctor`

- `opencli doctor` now shows extension version inline and reports
  version mismatch as an actionable issue

- Non-blocking npm update checker: registers a process exit hook so the
  update notice appears after command output (same pattern as npm/gh/yarn);
  background fetch writes to ~/.opencli/update-check.json for next run

- postinstall: print Browser Bridge setup instructions after shell
  completion install for first-time global install users

Bug fixes caught in review:
- discover.ts: add AbortController timeout to checkDaemonStatus() fetch,
  move clearTimeout after res.json() to cover body streaming
- daemon.ts: clear extensionVersion and reject pending requests in
  ws.on('error') handler, not just ws.on('close')
- update-check.ts: skip update notice when process exits with non-zero
  code; read cache once at module load to avoid double disk I/O;
  guard isNewer() against NaN from pre-release version strings

* fix: reduce fail-fast timeout to 300ms and guard stderr.write in exit hook
2026-03-27 02:14:37 +08:00
jakevin 14672ddf9b refactor: simplify codebase by removing dead code, deduplicating types, and extracting shared desktop adapter commands (#360)
- Remove dead code: unused `promises` array in discovery.ts, unused `DEFAULT_BROWSER_SMOKE_TIMEOUT`, unused `checkFfmpeg()`, deprecated `PlaywrightMCP` alias
- Extract shared `YamlArgDefinition`/`YamlCliDefinition` into `yaml-schema.ts` (was duplicated in discovery.ts and build-manifest.ts)
- Unify `BrowserCookie` type: remove duplicate from download/index.ts, re-export from types.ts
- Create `_shared/desktop-commands.ts` with factory functions (makeScreenshotCommand, makeStatusCommand, makeNewCommand, makeDumpCommand), simplifying 11 adapter files from ~20-30 lines each to 3 lines
- Fix unnecessary dynamic imports in utils.ts

Net: +30 / -361 lines
2026-03-24 20:20:32 +08:00
jakevin 9921e5d696 remove broken desktop adapters (#221) 2026-03-22 04:14:07 +08:00
jakevin eeace115cb fix: harden external CLI hub — command injection, denylist, sync API, build-copy (#149) 2026-03-21 01:26:02 +08:00
AlexYue 15d3583c60 docs: add missing adapter docs, fix sidebar 404s, add doc-check CI (#140)
* docs: add missing adapter docs, fix sidebar 404s, add doc-check CI

- Add doc pages for 11 undocumented adapters: arxiv, barchart,
  chaoxing, grok, hf, jike, jimeng, linux-do, sinafinance,
  stackoverflow, weread, wikipedia
- Update adapters/index.md with all new adapter entries
- Update VitePress sidebar config with 12 new entries
- Remove broken zh/ sidebar refs (troubleshooting, testing)
- Add doc-check CI workflow (adapter coverage + build + link check)
- Add scripts/check-doc-coverage.sh for adapter doc enforcement
- Enhance PR template with adapter doc checklist

* fix(ci): use --root-dir instead of --base for lychee link checker

lychee v0.23 requires --base to be a URL or absolute path.
Use --root-dir for resolving root-relative links in local files.

* fix(ci): remove lychee link-check job, rely on VitePress build

VitePress links use extension-less paths (e.g. /adapters/browser/twitter)
which lychee cannot resolve. The docs-build job already catches all
broken internal links via VitePress dead link detection during build.
2026-03-20 22:08:38 +08:00
stometaverse c370bd0582 feat(xiaohongshu): add 4 creator analytics commands (creator-profile, creator-stats, creator-notes, creator-note-detail) (#49)
* feat(xiaohongshu): add 4 creator analytics commands

Add creator backend support for Xiaohongshu (小红书), enabling
creators to access their analytics data from the command line.

New commands:
- creator-profile: account info (followers, likes, creator level)
- creator-stats: 7-day/30-day overview (views, likes, collects,
  comments, shares, new followers) with daily trend data
- creator-notes: note list with per-note metrics from note manager
- creator-note-detail: single note analytics breakdown
  (organic vs promoted vs video traffic)

API discovery:
- /api/galaxy/creator/home/personal_info (cookie auth, 200 OK)
- /api/galaxy/creator/data/note_detail_new (cookie auth, 200 OK)
- /api/galaxy/creator/data/note_detail?note_id=xxx (cookie auth, 200 OK)
- Note manager DOM extraction for note list (bypasses v2 signature)

All endpoints verified working with real creator account.
Screenshots (redacted) included in docs/screenshots/.

Requires: Chrome logged into creator.xiaohongshu.com

* chore: remove screenshots from repo (will host externally for PR)

* review: fix creator analytics CLI integration

Co-authored-by: stone16 <stone2paul@gmail.com>

* test: add site-scoped test runner

Co-authored-by: stone16 <stone2paul@gmail.com>

* review: ignore publish timestamps in creator note metrics

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-03-18 22:27:25 +08:00
jackwener 68840fc85c refactor: P0 architecture improvements
- Split browser.ts (700 lines) into src/browser/ module (page, mcp, errors, discover, tabs, index)
- Add unified error handling: CliError base class + logger module
- Enable TypeScript strict mode, fix 12 type errors
- Extract inline build scripts to scripts/clean-yaml.cjs and copy-yaml.cjs
- All 178 unit tests pass, build produces 83 entries across 19 sites
2026-03-17 17:01:51 +08:00
jackwener 8263a06a85 fix(completion): insert fpath before compinit in .zshrc
The postinstall script was appending the fpath line at the end of .zshrc,
but compinit (called earlier by oh-my-zsh or directly) would have already
finished scanning. This caused zsh completion to silently fail for most
users.

Now the script detects the first compinit / oh-my-zsh source line and
inserts the fpath entry before it, ensuring completion works immediately.
2026-03-17 16:53:07 +08:00
jackwener de962eb5fb feat: support commands completion (#32)
Add full shell tab-completion for opencli, supporting Bash, Zsh, and Fish.

Co-authored-by: RinChanNOWWW <rin_chan_now@outlook.com>

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-17 16:33:19 +08:00