* enrich(coupang): add product detail cmd + replace silent clamp/sentinel/Error with typed errors
Two enrichment changes plus three silent-failure fixes on top of existing
search / add-to-cart.
New cmd: coupang product
─────────────────────────
Pairs with search as the listing↔detail round-trip target. Reads a logged-in
product page and extracts a single canonical row with price, original_price,
discount_rate, rating, review_count, seller, brand, rocket, delivery_promise,
image_url, url. Three-source extractor (JSON-LD Product schema → bootstrap
globals → DOM) merged in priority order, mirroring the search.js pattern.
The columns use string|null typing — null means "upstream did not provide
this field on this product" (e.g. some items have no original_price).
Failures (login wall / page mismatch / page failed to render) raise typed
errors instead of silently returning empty rows, so callers can treat any
returned row as real data.
Search column shape: added product_id
─────────────────────────────────────
Listing must pair with detail by id. The data was already extracted by
normalizeSearchItem; only the columns array needed updating so the field
projects through to the rendered row. Per the listing-id-pairing convention
(PR #1297) the new column lets agents round-trip rows directly into
`coupang product` without re-scraping URLs.
Silent-failure fixes
────────────────────
1. search --limit silent clamp.
Old: `Math.min(Math.max(Number(kwargs.limit||20),1),50)` silently
rewrote `--limit 999` to 50 and `--limit 0` to 1.
New: `parseLimitArg(raw, 20, 50)` throws ArgumentError on out-of-range
/ non-integer / negative input. Same convention as the typed-fail-fast
memory & PR #1289.
2. search --page silent clamp.
Old: `Math.max(Number(kwargs.page||1),1)` silently lifted negative pages.
New: parsePageArg throws ArgumentError on non-positive input.
3. Generic `throw new Error(...)` → typed errors.
- Empty query, unsupported --filter, missing --product-id/--url
→ ArgumentError
- Login wall detection → AuthRequiredError('coupang.com', ...)
- Empty result / filter-not-rendered → EmptyResultError
- PRODUCT_MISMATCH / OPTION_REQUIRED / button-not-found / unknown
ack failure (add-to-cart) → CommandExecutionError
- The PRODUCT_MISMATCH and `actualProductId || 'unknown'` sentinel were
also fixed (silent-sentinel was the audit hit there).
Coverage
────────
- 21 contract assertions in clis/coupang/coupang.test.js covering
parseLimitArg / parsePageArg (no silent clamp), registry shape (search has
product_id, product is read-class with expected columns, add-to-cart is
write-class), and typed-error pre-flight rejections (empty query / bad
filter / out-of-range limit & page / missing detail args).
- Manifest 763 → 764 (+1 entry: coupang/product).
- Audits: typed-error-lint 196 → 194 (resolved 2 silent-clamp/sentinel
baseline entries; baseline updated). silent-column-drop 103/103 unchanged.
* fix(coupang): tighten product id and browser errors
* fix(coupang): require real product urls
* refactor(linux-do): remove deprecated hot/category/latest compat shims
The three shims have been pure backward-compat wrappers since linux-do/feed
became the unified entrypoint. With no stable release commitment to preserve,
they are pure surface cost: 3 manifest entries, 3 deprecated branches in help
output, and a `buildLinuxDoCompatFooter` helper that exists only to feed them.
- delete clis/linux-do/{hot,category,latest}.js
- drop now-orphaned `buildLinuxDoCompatFooter` from feed.js and unexport
`executeLinuxDoFeed` (no external consumers remain)
- remove the Compatibility section in docs/adapters/browser/linux-do.md
- regenerate cli-manifest.json (-125 lines)
BREAKING CHANGE: `opencli linux-do hot|category|latest` are removed. Use
`opencli linux-do feed --view top --period <period>`,
`opencli linux-do feed --category <id-or-name>`, and
`opencli linux-do feed --view latest` instead.
* fix(linux-do): finish compat shim removal
* refactor(runtime): unify command timeout into a single --timeout arg
Drop the cli-level `timeoutSeconds` build-time ceiling field. A command
now opts into runtime-enforced timeouts purely by declaring an arg named
`timeout`; the user-facing `--timeout` value (its default or override)
is the single authoritative knob, used both by the adapter polling loop
and by the runtime ceiling (with a 30s padding for return + closeWindow
+ trace export).
Behavior:
- Browser commands without a `--timeout` arg fall back to
OPENCLI_BROWSER_COMMAND_TIMEOUT (default 60s, unchanged).
- Non-browser commands without a `--timeout` arg now run unbounded
rather than against the previously implicit `timeoutSeconds` cap.
Affected commands keep their old caps via newly added `--timeout` args.
- LLM adapters (gemini/claude/deepseek/doubao/qwen/yuanbao ask) keep
their current `--timeout` defaults; the runtime ceiling is now strictly
more generous (userTimeout + 30s vs. the previous 180s cap), so
`--timeout 600` actually buys 600s of polling rather than dying at 180s.
Closes the design discussion that started from PR #1227, which proposed
a per-site `OPENCLI_GEMINI_ASK_TIMEOUT` env var to work around the same
underlying mismatch.
* fix(timeout): wire --timeout arg into chatgpt/gemini image adapter polling
codex-coder review on PR #1364 caught that the new --timeout arg I added
to chatgpt/image and gemini/image only drove the runtime ceiling — the
adapter still hardcoded `const timeout = 120`, so users passing
--timeout 240/600 saw runtime allow 270s/630s but the adapter stop
polling at 120s. That recreated the same single-knob mismatch this PR
was meant to delete.
Also add the browser-path runWithTimeout assertion codex-coder flagged
as missing: a browser command with --timeout default=5 must call
runWithTimeout with timeout: 35; a browser command without --timeout
arg must fall back to DEFAULT_BROWSER_COMMAND_TIMEOUT.
Image adapters now read kwargs.timeout and reject non-positive-integer
values with ArgumentError (no silent fallback). chatgpt/image.test.js
updated to pass an explicit timeout when calling .func directly (the
test bypasses arg coercion).
* fix(runtime): reject invalid timeout ceilings
* fix(timeout): normalize timeout args to integer values
* fix(timeout): preserve remaining command ceilings
* fix(runtime): validate timeout before browser setup
* enrich(toutiao): hot board + bug fixes (silent column drop, partial render)
Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #3.
## New command
- `toutiao hot` (Strategy.PUBLIC, browser:false) — public homepage hot
board via the toutiao.com hot-event/hot-board endpoint. No login required.
Returns 8 stable columns (rank/id/title/query/hot_value/label/url/image).
## Bug fixes for `toutiao articles`
- **Silent column drop fixed**: `parseToutiaoArticlesText` previously
did `if (title && stats) push(...)`, silently dropping any row where
the stats span hadn't finished rendering by the time page.innerText
was read. Slow-render bugs were invisible — adapter looked "complete"
while writers saw extra rows in the dashboard. Partial rows now
surface with `null` stat columns.
- **Silent clamp on `--page` removed**: out-of-range / non-integer
values raise `ArgumentError` with explicit bounds [1, 4]. Same
validation reused by both `articles` and `hot` via `parseArticlesPage`
/ `parseHotLimit` in `utils.js`.
- **Empty result typed**: zero-row scrape now raises `EmptyResultError`
instead of returning `[]` silently (would otherwise look like a
legitimate "no articles" response).
## Refactor
- Parser logic extracted to `clis/toutiao/utils.js` (alongside hot-row
mapping, validators, and the hot-board URL constant).
- `articles.js` switches from declarative `pipeline:` to imperative
`func` form so `parseArticlesPage` validation can run before the
navigation step (declarative pipeline can't pre-validate args).
- Strategy is now explicit: `Strategy.COOKIE, browser: true` for
articles (creator dashboard is logged-in only).
## hot field map
`ClusterIdStr` (or numeric `ClusterId`) → id; `Title` → title;
`QueryWord` → query (falls back to title); `HotValue` → hot_value
(non-negative numeric, else null); `Label`, `Url`, `Image` →
respective columns. `pickImage` walks `Image.url` → first truthy
`Image.url_list[]`. Empty-title rows are dropped (returns null) before
ranks are densely re-assigned 1..N.
## Tests
29 contract assertions across `parseArticlesPage` / `parseHotLimit` /
`parseToutiaoArticlesText` / `mapHotRow` + registry-level shape checks
+ `hot` adapter func behaviour (typed errors / no silent clamp / fetch
failure paths / dense-rank).
## Audits
- typed-error-lint: 196 = 196 (unchanged baseline)
- silent-column-drop: 103 = 103 (unchanged baseline)
- listing-id-pairing: hot has `id` column (round-trippable when a
detail command lands later); advisory list unchanged.
## Manifest
757 → 758 entries (+1 for `hot`).
## Doc
- index.md: toutiao mode 🔐 → 🌐/🔐 (hot is public, articles is logged-in)
- toutiao.md: per-command mode/domain table + column docs + prerequisites
* fix(toutiao): tighten hot and articles contracts
* fix(linkedin): surface detail_error on --details (no silent catch / no silent empty)
The previous --details enrichment path had two indistinguishable failure modes
that both produced `description: '', apply_url: ''`:
1. `if (!job.url)` early return — row had no jobId, so we couldn't navigate.
2. `} catch {}` — page.goto / page.evaluate threw (network, timeout, parse error).
Callers couldn't tell "upstream had no description" from "we failed to fetch",
and the catch swallowed every error without logging. For an enrichment that
costs one page navigation per row, silent failure is especially harmful — users
just see an empty cell with no way to debug.
Fix: replace empty strings with `null` for missing/failed rows, add a new
`detail_error` column (string|null) carrying a short typed reason:
- 'no url' — row had no jobId
- 'fetch failed: <msg>' — page.goto / page.evaluate threw
- 'missing description' — page loaded but body was empty
- null — success
Every failure is also logged to stderr with the offending URL so debugging is
possible. Per-row failures still don't abort the batch (the original intent),
but they're now visible.
Tests: 13 new contract assertions in clis/linkedin/search.test.js covering
parseCsvArg, mapFilterValues (ArgumentError on unknown values), decodeLinkedinRedirect,
and 5 enrichJobDetails paths (no-url / goto-throw / empty-description / success /
multi-row-mixed). Added `export const __test__` for testability.
Audits clean: typed-error-lint 196/196, silent-column-drop 103/103.
* fix(linkedin): fail fast on auth walls
* enrich(ctrip): hotel-suggest + bug fixes (silent clamp, dropped columns, fake URL)
Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #1.
## New command
- `ctrip hotel-suggest` — surfaces hotel-context suggestions (cities,
business areas, individual hotels) via the same backing endpoint with
searchType=H. Distinct from `ctrip search` (searchType=D) which returns
destinations / scenic spots / railway stations.
## Bug fixes for `ctrip search`
- **Silent clamp on `--limit` removed**: out-of-range values (≤0, ≥51,
non-integer) now raise `ArgumentError` with explicit bounds rather than
silently snapping to [1, 50].
- **Silent column drop fixed**: previously the adapter discarded `id`,
`cityId`, `cityName`, `provinceName`, `countryName`, `lat`, `lon`, `eName`
and `displayType` from upstream rows. Now all are surfaced as stable
columns.
- **Fake URL fixed**: previously `url` was always `''`. Now constructs
canonical Ctrip URLs by `type` (City / Markland / Hotel / Zone / RailwayStation)
and returns `null` (no silent fabrication) for unknown types.
- **In-band error envelope typed**: `Result: false` payloads now surface
as `COMMAND_EXEC` (was previously not handled — adapter returned empty
rows).
## Doc fix
- `Mode: 🔐 Browser` → `🌐 Public` (search uses public API, no login)
- Add `hotel-suggest` to commands table in both `docs/adapters/index.md`
and `docs/adapters/browser/ctrip.md`.
## Coords picker
Mainland China rows ship `gdLat`/`gdLon` (gaode); international rows ship
`gLat`/`gLon` (wgs84). Adapter picks the first non-zero pair (zero is the
upstream sentinel for "missing"); returns `null` if all variants are zero.
## Tests
25 contract assertions across `parseLimit` / `pickCoords` / `buildUrl` /
`mapSuggestRow` + registry-level checks for both commands (Strategy /
shape parity / typed errors / no silent clamp).
## Audits
- typed-error-lint: 196 = 196 (unchanged baseline)
- silent-column-drop: 103 = 103 (unchanged baseline)
- listing-id-pairing: advisory only (search has `id` round-trip column)
## Manifest
757 → 758 entries (+1 for `hotel-suggest`).
* fix(ctrip): wrap suggest fetch and json failures
Closes#1334.
Exposes viewport overrides for `opencli browser screenshot` so an adapter or
ad-hoc shell user can render a page at a fixed width and capture the full
scrollable height. The ljg-card HTML to PNG pipeline use case.
Behavior:
- `--width W` only overrides device-metrics width; height is left unchanged.
- `--height H` only overrides height (ignored under `--full-page`).
- `--full-page` keeps the existing `captureBeyondViewport` shortcut.
- `--full-page --width W` first reflows at W, then re-overrides to (W, contentH)
so the captured image reflects the layout at the requested width.
- Override is always cleared in `finally`, including on capture failure.
* feat(deepseek): add detail and send commands for explicit conversation control
doubao already ships `detail <id>` and `send` for ID-explicit conversation
read/write; deepseek had only `read` (current page only) plus the
implicit-resume `ask`. Adding both gives users a stable handle when they
know the conversation ID, without going through `ask`'s resume detection
or its full prompt-then-wait pipeline.
`deepseek detail <id>`:
- parses a bare UUID or any URL containing `/a/chat/s/<id>`,
- rejects malformed input via `ArgumentError` before any browser
navigation,
- navigates to `https://chat.deepseek.com/a/chat/s/<id>` and returns
the visible message list,
- throws `EmptyResultError` when the conversation has no rendered
messages.
`deepseek send <id> <prompt>`:
- takes the conversation id as a required positional, because the
framework runs each browser command in an ephemeral per-command
workspace (a fresh tab) and there is no shared "current conversation"
across commands; the navigation must be explicit,
- drives input through CDP `Input.insertText` via `page.nativeType`,
mirroring the doubao adapter (#1278); `execCommand('insertText')` plus
a synthesised input event leaves the React-controlled state desynced
on a freshly-opened tab and the resulting click silently no-ops,
- keeps the verification loop inside the same `page.evaluate` so the
framework cannot close the tab mid-flight; counts user-class bubbles
by text-match (DeepSeek virtualises the message list, so a numeric
bubble-count check is unreliable),
- throws `CommandExecutionError` with a specific reason when the
textarea did not populate, the send button stayed disabled, the
bubble never settled, or the optimistic render rolled back during
a 3s settle window,
- treats "Promise was collected" from the post-click eval as success,
matching the existing pattern in `ask --file`.
Helper `parseDeepSeekConversationId` is exported from utils.js so the
same parser feeds both commands and round-trips the canonical lower-case
ID.
Tests:
- utils.test.js: 5 cases covering bare UUID, upper-case
normalisation, URL extraction with and without query string, empty /
null / whitespace input, and non-UUID rejection.
- detail.test.js: 5 cases covering registration, navigation +
message return, URL normalisation, ArgumentError before browser
navigation, and EmptyResultError on no-messages.
- send.test.js: 7 cases covering registration, ArgumentError on bad
id, full happy-path through nativeType + IIFE verification, the
textarea-mount timeout, missing nativeType helper, focus failure,
IIFE-reason translation to CommandExecutionError, and the
"Promise was collected" success path.
Manifest auto-regenerated to register both commands.
Live-verified end-to-end against my own DeepSeek session:
- `detail` returns the canonical message list for a bare UUID, parses
a full chat URL, and rejects malformed IDs before any browser
navigation,
- `send` lands the prompt as the latest user message in the target
conversation and gets an AI response back; reload of the
conversation page in a separate tab confirms the message persisted
server-side.
* docs(deepseek): document detail and send commands
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Keep the owned automation container window warm across lease release. Non-final owned leases close their tab; the final owned lease resets its tab to about:blank as a reusable placeholder. Update browser close wording to describe lease release rather than window closure.
Closes#1342.
`opencli deepseek ask` (without --new) used to resume the most recent
conversation by clicking the first `a[href*="/a/chat/s/"]` in DOM order
after a fixed 2-second wait.
Two bugs:
1. Pinned conversations sit in their own DOM section ("置顶") that
renders above "30 天内" and friends. Click-first-anchor lands on the
pinned thread, not the user's most recent. Reproduced live by
pinning a conversation through the sidebar context menu and
observing that the existing logic targets it instead of the most
recent non-pinned thread.
2. The 2s wait is fixed. On a slow network the sidebar has not
populated yet, the click is a no-op, and `ask` silently falls
through to the new-chat path. The user typed "follow up" and
a brand-new conversation gets created.
Replace the click-first-anchor + fixed wait with a new helper
`pickResumeUrl(page)` in utils.js that:
- polls the sidebar for up to 10s (5 attempts × 2s),
- identifies pinned anchors by a text-based check on the section
header (`/^\s*(置\s*顶|Pinned)\s*$/i`); DeepSeek's CSS-module
class names are randomized per build, so the text is the only
stable signal,
- returns the URL of the first non-pinned anchor (or falls back to
the first overall if every visible anchor is pinned),
- returns null if no anchor surfaces in time.
`ask.js` calls the helper and `page.goto`s the returned URL. When the
helper returns null, `ask` now throws a `CommandExecutionError`
instead of silently navigating to a fresh chat. The user gets a clear
"pass --new" hint and their prompt is never sent to a wrong target.
Tests:
- utils.test.js: 4 cases covering happy path, polling-then-success,
timeout returns null, and a structural assertion that the embedded
DOM walker uses text-based pinned detection.
- ask.test.js: replaced the prior "still selects model when no
conversation to resume" test (which exercised the silent
fall-through) with a fail-fast assertion. Updated the resume-success
test to mock the new helper.
* feat: 11 read adapters across 8 sites (dblp / steam / bbc / devto / lobsters / medium / coingecko / hf)
Round 2 of the adapter expansion sweep. All 11 commands hit public APIs (no
browser, no auth), follow the post-#1332 typed-error / no-silent-failure
discipline, and were live-verified against real endpoints.
New adapters:
- dblp/author : recent publications for one author (resolve PID by name, or pass --pid)
- steam/search : storefront name search (storesearch API)
- steam/app : single app detail (appdetails API; HTML entities decoded)
- bbc/topic : per-topic RSS (8 canonical BBC News feeds)
- devto/latest : /api/articles/latest with --page pagination
- lobsters/domain : stories from a specific source domain (/domains/<d>.json)
- medium/tag : tag RSS (description full-length, no silent truncation)
- coingecko/exchanges : trust score + 24h BTC volume leaderboard
- coingecko/categories : sector buckets with 6 sort options
- coingecko/global : aggregate market totals + BTC/ETH dominance
- hf/paper : single-paper detail by arXiv id (summary, ai_summary, ai_keywords, upvotes)
Also adds clis/steam/utils.js + clis/bbc/utils.js as shared helpers (HTML entity
decode, RSS parsing). All listings carry a round-trippable id where a detail
sibling exists; advise:listing-id-pairing reports zero new violations. typed-
error-lint and silent-column-drop gates both unchanged from baseline.
Manifest: 698 → 709 (+11 entries).
* fix: tighten adapter round2 contracts
* Add uisdc news adapter for CLI
Implements a CLI adapter for fetching the latest AI/design news from uisdc.com. Allows specifying the number of news items to return.
* feat(aibase): add aibase daily news adapter
This file implements a news adapter for AIbase that fetches the latest AI industry news and allows for configurable limits on the number of news items returned.
* fix(news): harden uisdc and aibase adapters
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Document the pattern for running opencli on a remote machine while keeping
the daemon and Chrome on the local machine. Reverse-tunnel local 19825
back to the remote (via SSH -R or frp) so the remote opencli still talks
to its own loopback and the daemon never leaves localhost.
Captures the rationale we landed on after reviewing #636: native
extension-to-remote-daemon support is deferred until the daemon protocol
gains authentication; in the meantime this is the safe, zero-code path
that achieves the same outcome.
* feat: add tiktok creator-videos command
TikTok Studio creator content list with views/likes/comments/saves/shares.
Hits the Studio item_list endpoint
(https://www.tiktok.com/tiktok/creator/manage/item_list/v1/?aid=1988) from a
logged-in /tiktokstudio/content session and pages with cursor until limit is
satisfied (server caps size at 50). Username for the resulting video URL is
extracted from the user_text= query param on play_addr / download_info entries,
falling back to scraping a[href*="/video/<id>"] from the Studio page DOM.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tiktok): regen manifest + replace silent-clamp with ArgumentError
- Regenerate cli-manifest.json (CI gate: must match `npm run build` output)
- Replace `Math.max(1, Number(args.limit) || 20)` and
`Math.min(Math.max(limit, 1), 50)` with an explicit positive-integer
guard + a server-cap-only ternary, per the silent-clamp guidance in
references/typed-errors.md (typed-error-lint baseline is unchanged)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tiktok): tighten creator videos contract
---------
Co-authored-by: root <root@example.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ feedback in #OpenCLI thread on the flat "Site adapters (112)" listing:
the bucket conflates real web sites (bilibili, dianping, ...) with desktop apps
(chatgpt-app, chatwise, codex, cursor, discord-app, doubao-app, antigravity, notion).
Group them so agents that fall back to --help can scan by category.
Three buckets, sourced from existing metadata only — no new adapter schema:
- External CLIs: passthrough binaries from loadExternalClis() (docker, gh, vercel, ...)
- App adapters: domain is `localhost` or any non-DNS string (no `.`)
- Site adapters: domain contains `.` (real DNS), or domain is unset (default)
The classifier is one line: `domain.includes('.') ? 'site' : 'app'`. Adapters
without a domain field default to site (most are public web scrapers like
arxiv / wikipedia / spotify / ...).
Verified against the live registry: 7 External CLIs, 8 App adapters
(antigravity, chatgpt-app, chatwise, codex, cursor, discord-app, doubao-app,
notion), 104 Site adapters.
Structured help (-f yaml/json) gains parallel `external_clis` / `app_adapters`
/ `site_adapters` keys; `commands` no longer leaks adapter names.
External CLIs are now hidden from the default Commands listing (mirrors how
site adapters were already filtered) and surfaced in their own section.
- Add clis/test-utils.js with standard createPageMock utility
- Migrate 11 test files to use shared utility (removes ~300 lines of duplication)
- Delete extension/src/cdp.test.ts dead skip test (chrome.scripting.executeScript removed from source)
- Remove clis/pixiv/test-utils.js (superseded by shared utility)
Codify the JSDOM-against-frozen-fixture pattern that PR #1313 introduced
for dianping (and that PR #1318 had to follow up to clean up). The skill
previously had no reference for this category of test, so authors of the
next adapter that hits silent-in-browser-DOM bugs would either reinvent
it or skip it.
Key conventions captured:
- **Mandatory awk 'NF>0' as the final step of fixture creation.** The
blank-line noise that PR #1318 removed (84.6% / 54.8% of file content
in dianping/{shop,search}.html) came from manually stripping
script/style content without collapsing the surrounding newlines.
Skipping this step is the silent quality regression that the next
fixture author would also hit.
- **Trim-to-minimum but never re-flow content.** Some bugs depend on
text-node adjacency without intervening whitespace
(dianping #1312 bug #2: rating "4.8" + reviews "21241条" fused as
"4.821241条"). Pretty-printing the meaningful mega-line would mask
the very condition the test is meant to catch.
- **Reverse-validate the regression guard.** "18/18 tests pass" only
proves agreement with the current implementation, not that the test
would have caught the original bug. Reintroducing the buggy variant
must make the test fail — otherwise the fixture is over-stripped or
the assertion is too loose.
- **__fixtures__/ is the documented exception** to the "no committed
HTML dumps" rule in the skill's "关键约定". Calling that out
explicitly because the rule otherwise reads as "all HTML in repo is
bad," which the dianping fixture pattern intentionally violates for
a real reason.
Background: WAWQAQ in #1313 follow-up thread (`#OpenCLI:36d2f65a`) asked
twice — first about the visible blank-line noise (→ PR #1318 cleanup),
then about the root cause and what should improve in the workflow itself.
This is the workflow improvement.
No new tooling / CI gate / lint introduced (B 1-week gate freeze still
applies). When a fifth fixture site adopts this pattern,
`opencli browser fixture-snapshot` automation can be revisited; until
then, runbook discipline + skill reference is the right scope.
dianping/__fixtures__/search.html and shop.html came out of page.content() with
hundreds of blank lines that JSDOM ignores during parsing — pure visual / disk
noise that bloats reviewer diff and obscures the meaningful DOM subtree the
fixture freezes.
search.html: 372 → 168 lines (-204 lines, -572 bytes)
shop.html: 39 → 6 lines (-33 lines, -64 bytes)
`awk 'NF>0'` keeps every line that has any non-whitespace character, so the
minified mega-line (where the rating-vs-reviews adjacency that triggers #1312
silent fusion lives) is preserved verbatim. dianping.test.js 18 tests still
pass, and reintroducing the buggy `headText.match(/(\d+)条/)` extractor still
makes the regression guard fail with the expected '821241条' (proving the
fusion-bug detection power is intact after the strip).
Per WAWQAQ feedback in #1313 thread; opus independently validated the same
approach before the cleanup.
PR #1312 fixed two silent in-browser DOM bugs that the existing mocked
`page.evaluate` tests could not catch:
1. shop title fallback split on ASCII `[]` while dianping renders
full-width `【】`, so `name` was always empty (or `"undefined"`).
2. headText `\s+` collapse fused rating "4.8" with reviews "21241条",
so a head-wide `/\d+条/` regex captured "4.821241" → 5.
Both bugs only surfaced on live verify; mocked-evaluate unit tests fed
pre-baked results to the func and the real DOM walk never ran.
Make the in-browser extractor logic testable in CI:
- clis/dianping/shop.js, clis/dianping/search.js: extract the IIFE
bodies into top-level `extractShopFields()` / `extractSearchRows()`
using bare `document` / `location`. The live adapters inject these
via `page.evaluate(\`(\${fn.toString()})()\`)` so behavior is
unchanged; both commands re-verified end-to-end against live
dianping (shop returns name=芈重山老火锅(五道口店), reviews=21241,
rating=4.8; search returns 3 result-shaped rows with correct ids).
- clis/dianping/__fixtures__/shop.html (3.4KB), search.html (8.4KB):
sanitized HTML snapshots — scripts/styles/iframes/comments stripped,
img src placeholdered, only structural attributes kept. Trimmed to
the minimum subtree needed to exercise the extractors (search keeps
3 of 15 li cards; shop keeps .shop-head + .desc-info + .review-title
plus full-width 【】 title and headText with the rating/reviews
fusion preserved).
- clis/dianping/dianping.test.js: add a fifth describe block —
"extractors against frozen HTML fixtures" — that loads the fixtures
via JSDOM, swaps `globalThis.document` / `globalThis.location`, and
asserts the post-fix behavior:
* shop: name=芈重山老火锅(五道口店), reviewsRaw=21241条, rating=4.8,
breakdown={口味:4.8,环境:4.8,服务:4.8,食材:4.9}, hours, rank, subway.
* search: 3 rows with correct shop_ids, names, reviewsRaw, priceRaw,
starClass; round-trip through parseReviewCount/parsePrice mappers
to lock in {rating:5.0,reviews:21231,price:109} et al.
* ok:false branches: shop fixture without `.shop-head`, search
fixture with empty `#shop-all-list`.
Manually verified the fixtures would catch the original bugs by running
buggy extractor variants against shop.html — ASCII-bracket fallback
returns `name="undefined"`, and head-wide `/\d+条/` returns `821241条`
(both fail the new assertions).
Test suite grows 14 → 18 passing tests; live verify of both commands
still produces correct output post-refactor.
Pattern intentionally limited to dianping as a reference point. If other
sites with in-browser DOM extraction encounter similar silent bugs, this
JSDOM-against-frozen-fixture pattern can be adopted per-site.
* docs(cases): add three researcher workflow examples
Add use cases under cases/ that exercise the recently-landed
researcher-friendly adapters:
- daily-rl-research-monitor.md uses arxiv recent + openreview venue
+ hf top to compress a morning paper-skim into one shell pipeline.
- find-paper-implementation.md chains arxiv search/paper + dblp
search + hf top + openreview search to map a paper's canonical
record, follow-ups, and community uptake.
- track-conference-papers.md walks openreview venue + reviews to
shortlist accepted papers and digest review threads in batch.
Each file is a real workflow built on commands from #1289 (arxiv
recent), #1294 (openreview), and #1299 (dblp).
* docs(cases): correct venue ids and forum example to ones that return data
The first revision used "ICLR.cc/2026/Conference" and "ICLR 2026 oral"
as venue strings. Both return EMPTY_RESULT today because the venue is
not open. Update each case to use natural-language venue text that
OpenReview currently exposes ("ICLR 2024 oral", "NeurIPS 2025 oral")
and a real forum id (KS8mIvetg2, "Proving Test Set Contamination in
Black-Box Language Models") in the reviews / paper drill-down. Note
the arxiv free-text-search ranking quirk so the worked DPO example
makes sense.
PR #1297 introduced a CI gate that fails when a site has both a listing
and a detail command but the listing rows don't carry an id-shaped column.
The gate came with a 10-entry EXEMPT map (topic-string trending,
profile-attribute rows, UI-only sessions, ...) where each exemption
recorded a "why this listing legitimately doesn't pair" reason.
By the same filter that closed PR #1311 (write-without-delete-pair gate):
Is "listing should pair with detail" a *permanent* anti-pattern, or
case-by-case business judgment?
It's case-by-case. Topic-string listings and profile-attribute rows
genuinely don't pair with a detail command. The fact that we needed an
EXEMPT map with 10 entries and individual reason strings is the smell —
it's not the rule winning, it's the rule failing. Forcing every adapter
PR to either add an id column or file an exemption was a higher cognitive
cost than the silent-loss bugs the rule actually catches.
Changes:
- .github/workflows/ci.yml — drop the "Check listing↔detail id pairing"
step. Other gates (silent-column-drop, typed-error-lint) stay in place.
- package.json — rename the script from `check:listing-id-pairing` to
`advise:listing-id-pairing` to make the advisory nature explicit.
- scripts/check-listing-id-pairing.mjs — drop the `--strict` flag and the
EXEMPT map. The script now always exits 0 and prints an advisory report
of listings that don't carry an id-shaped column. Reviewers/authors use
it as guidance, not a gate.
- docs/conventions/listing-detail-id-pairing.md — rewrite from "MUST" to
"soft convention". Adds an explicit "why advisory, not a gate" section
that lists the legitimate non-pairing categories so future readers know
the rule's boundary.
- docs/developer/ts-adapter.md — match the advisory tone in the
adapter-author guidance.
The doc, the script, and the column patterns table all stay — agents and
adapter authors can still consult them. What's gone is the CI failure and
the per-PR exempt-list maintenance burden.
Net diff: -34 lines (gate + EXEMPT map removed, advisory-tone doc adds
a small "why advisory" section).
Adds a baseline CI gate for convention-audit typed-error lint findings. Also refreshes the silent-column-drop baseline for dianping changes already on main.
The merged adapter had two silent in-browser bugs that the mocked-evaluate
unit tests don't catch — only live verify against www.dianping.com surfaces
them:
1. Shop name returned `undefined`. The fallback parsed `document.title` with
an ASCII-bracket split (`/[\\[\\]]/`) but dianping wraps the name in
full-width brackets `【芈重山老火锅(五道口店)】...`. Switch to a `【...】`
regex so the title fallback actually fires.
2. Reviews returned `5` instead of `21241`. The headText was whitespace-
collapsed to `★★★★★4.821241条...`, fusing the rating and review digits;
a head-wide `/\d+条/` then captured `4.821241` and rounded to `5`. Read
the dedicated `.reviews / .review-num` element ("21241条") instead, with
a `.review-title` "评价(<n>)" fallback.
* feat(dianping): browser adapter — search + shop on www.dianping.com
Adds two browser-mode adapters for the dianping (大众点评) PC site:
- `dianping search "<keyword>" --city <name|id> --limit <n>`: keyword
shop/restaurant search. Returns rank, shop_id, name, rating, reviews,
price, cuisine, district, url. shop_id round-trips into `dianping shop`.
- `dianping shop <shop_id>` (alias `detail`): shop detail sheet
(field/value rows: name, rating, breakdown 口味/环境/服务/食材, reviews,
price, rank, hours, address, subway, features, url).
Both use Strategy.COOKIE on www.dianping.com (the PC site renders search
SSR and does not require JS hydration). m.dianping.com is intentionally
crippled for non-mobile UAs, so it's not used.
Auth detection (utils.detectAuthOrEmpty) inspects both response text and
final URL for the Meituan Yoda captcha redirect (verify.meituan.com) and
the dianping login redirect; raises AuthRequiredError with the captcha
URL embedded so the user can clear it manually in the same profile.
Listing↔detail id pairing: search.shop_id → shop.<id>. Adds 'shop' to
DETAIL_NAMES in scripts/check-listing-id-pairing.mjs so the convention
gate scans this site (35 sites / 78 listings now covered).
* fix(dianping): harden browser failure classification
* fix(dianping): fail on partial missing shop ids
Adds opencli convention-audit for batch convention scanning, with structured output, strict mode, docs, and startup isolation from local user/plugin discovery.
jakevin