Wire up the standard browser-LLM command surface for Yuanbao, matching the
recently shipped chatgpt + claude + qwen baselines:
- status — login + current model + (agentId, convId) + URL
- read — render the visible conversation as User/Assistant rows
- detail — open `<agentId>/<convId>` and read its messages
- history — list sidebar conversations with stable IDs
- send — fire-and-forget, returns once the send button has been clicked
Refactor `ask.js` to share helpers (`sendYuanbaoMessage`, `normalizeBooleanFlag`)
with the new commands via `shared.js`, keeping the public ask behavior intact.
Notable bits:
- `parseYuanbaoSessionId` accepts only full chat URLs or `<agentId>/<convId>`
pairs — Yuanbao chat URLs encode both, and silently opening the wrong agent
on a bare UUID is a worse failure mode than throwing. URL regex anchored
with `(?:[/?#]|$)` so 37+ char tails reject rather than truncate.
- `sendYuanbaoMessage` polls the send button (up to 3s) for the React
re-render that drops `style__send-btn--disabled___*` after composer input —
a fixed wait raced the debounce and produced silent no-op clicks.
- `getYuanbaoMessageBubbles` uses `data-conv-id`/`data-conv-idx`/
`data-conv-speaker` attributes for stable per-turn identity (was relying
on innerHTML alone).
- Status surfaces both human label (`Yuanbao`) and `dt-model-id`
(`hunyuan_gpt_175B_0404`) — sentinel strings would silently look like a
real model name; null is the typed-unknown signal.
Verified: 25 unit tests pass; targeted live smoke for status/read/detail/
history/new/send + ask round-trip on yuanbao.tencent.com.
* feat(qwen): add detail command + fix stale message bubble selector
`getMessageBubbles` was matching `[data-msgid="<id>-question|answer"]` from an
older Qianwen frontend. The reshipped DOM no longer carries that attribute on
chat turns; `[data-message-id]` now lives on citation cards inside assistant
responses, so the old selector silently returned an empty list and `qwen read`
had been silently broken.
Rewire to walk `[data-chat-question-wrap]` and `[data-chat-answers-wrap]` in
DOM order (correct Q/A interleaving) and synthesize stable IDs from the
nearest sibling `data-req-id` so `waitForAnswer.seenAssistantId` and
read/ask/detail dedupe paths keep working. Verified live against an existing
conversation: 3 user turns + 3 assistant turns extracted; old selector
returned 0.
`qwen detail <id|url>`: open a specific conversation by ID or full chat URL,
poll up to 20s for the transcript to render, return Role/Text rows. Adds
`parseQianwenSessionId` (5 unit tests covering ID/URL parsing + ArgumentError
on malformed input). Reuses the same site-level browser session as `read`/
`ask` so consecutive calls continue in the same Qwen tab.
- clis/qwen/detail.js (new)
- clis/qwen/utils.js (parseQianwenSessionId + getMessageBubbles rewire)
- clis/qwen/utils.test.js (new)
- docs/adapters/browser/qwen.md (detail entry + options/columns)
- cli-manifest.json (regenerated)
* fix(qwen): anchor URL regex to reject 33+ hex tail truncation
codex-coder review on PR #1390 caught that
`https://www.qianwen.com/chat/<33+ hex>` would silently truncate to the
first 32 chars and open the wrong conversation. Adds end-of-input /
slash / query / fragment boundary to the URL match group and two new
unit-test cases (digit tail + letters tail) covering the truncation gap.
Add ChatGPT web ask/send/read/history/detail/new/status alongside existing image support. Tighten ChatGPT web helper selectors and typed error contracts, update docs/changelog, regenerate manifest, and seed local ChatGPT verify fixtures for ask/read.
* enrich(coupang): add product detail cmd + replace silent clamp/sentinel/Error with typed errors
Two enrichment changes plus three silent-failure fixes on top of existing
search / add-to-cart.
New cmd: coupang product
─────────────────────────
Pairs with search as the listing↔detail round-trip target. Reads a logged-in
product page and extracts a single canonical row with price, original_price,
discount_rate, rating, review_count, seller, brand, rocket, delivery_promise,
image_url, url. Three-source extractor (JSON-LD Product schema → bootstrap
globals → DOM) merged in priority order, mirroring the search.js pattern.
The columns use string|null typing — null means "upstream did not provide
this field on this product" (e.g. some items have no original_price).
Failures (login wall / page mismatch / page failed to render) raise typed
errors instead of silently returning empty rows, so callers can treat any
returned row as real data.
Search column shape: added product_id
─────────────────────────────────────
Listing must pair with detail by id. The data was already extracted by
normalizeSearchItem; only the columns array needed updating so the field
projects through to the rendered row. Per the listing-id-pairing convention
(PR #1297) the new column lets agents round-trip rows directly into
`coupang product` without re-scraping URLs.
Silent-failure fixes
────────────────────
1. search --limit silent clamp.
Old: `Math.min(Math.max(Number(kwargs.limit||20),1),50)` silently
rewrote `--limit 999` to 50 and `--limit 0` to 1.
New: `parseLimitArg(raw, 20, 50)` throws ArgumentError on out-of-range
/ non-integer / negative input. Same convention as the typed-fail-fast
memory & PR #1289.
2. search --page silent clamp.
Old: `Math.max(Number(kwargs.page||1),1)` silently lifted negative pages.
New: parsePageArg throws ArgumentError on non-positive input.
3. Generic `throw new Error(...)` → typed errors.
- Empty query, unsupported --filter, missing --product-id/--url
→ ArgumentError
- Login wall detection → AuthRequiredError('coupang.com', ...)
- Empty result / filter-not-rendered → EmptyResultError
- PRODUCT_MISMATCH / OPTION_REQUIRED / button-not-found / unknown
ack failure (add-to-cart) → CommandExecutionError
- The PRODUCT_MISMATCH and `actualProductId || 'unknown'` sentinel were
also fixed (silent-sentinel was the audit hit there).
Coverage
────────
- 21 contract assertions in clis/coupang/coupang.test.js covering
parseLimitArg / parsePageArg (no silent clamp), registry shape (search has
product_id, product is read-class with expected columns, add-to-cart is
write-class), and typed-error pre-flight rejections (empty query / bad
filter / out-of-range limit & page / missing detail args).
- Manifest 763 → 764 (+1 entry: coupang/product).
- Audits: typed-error-lint 196 → 194 (resolved 2 silent-clamp/sentinel
baseline entries; baseline updated). silent-column-drop 103/103 unchanged.
* fix(coupang): tighten product id and browser errors
* fix(coupang): require real product urls
* refactor(linux-do): remove deprecated hot/category/latest compat shims
The three shims have been pure backward-compat wrappers since linux-do/feed
became the unified entrypoint. With no stable release commitment to preserve,
they are pure surface cost: 3 manifest entries, 3 deprecated branches in help
output, and a `buildLinuxDoCompatFooter` helper that exists only to feed them.
- delete clis/linux-do/{hot,category,latest}.js
- drop now-orphaned `buildLinuxDoCompatFooter` from feed.js and unexport
`executeLinuxDoFeed` (no external consumers remain)
- remove the Compatibility section in docs/adapters/browser/linux-do.md
- regenerate cli-manifest.json (-125 lines)
BREAKING CHANGE: `opencli linux-do hot|category|latest` are removed. Use
`opencli linux-do feed --view top --period <period>`,
`opencli linux-do feed --category <id-or-name>`, and
`opencli linux-do feed --view latest` instead.
* fix(linux-do): finish compat shim removal
* enrich(toutiao): hot board + bug fixes (silent column drop, partial render)
Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #3.
## New command
- `toutiao hot` (Strategy.PUBLIC, browser:false) — public homepage hot
board via the toutiao.com hot-event/hot-board endpoint. No login required.
Returns 8 stable columns (rank/id/title/query/hot_value/label/url/image).
## Bug fixes for `toutiao articles`
- **Silent column drop fixed**: `parseToutiaoArticlesText` previously
did `if (title && stats) push(...)`, silently dropping any row where
the stats span hadn't finished rendering by the time page.innerText
was read. Slow-render bugs were invisible — adapter looked "complete"
while writers saw extra rows in the dashboard. Partial rows now
surface with `null` stat columns.
- **Silent clamp on `--page` removed**: out-of-range / non-integer
values raise `ArgumentError` with explicit bounds [1, 4]. Same
validation reused by both `articles` and `hot` via `parseArticlesPage`
/ `parseHotLimit` in `utils.js`.
- **Empty result typed**: zero-row scrape now raises `EmptyResultError`
instead of returning `[]` silently (would otherwise look like a
legitimate "no articles" response).
## Refactor
- Parser logic extracted to `clis/toutiao/utils.js` (alongside hot-row
mapping, validators, and the hot-board URL constant).
- `articles.js` switches from declarative `pipeline:` to imperative
`func` form so `parseArticlesPage` validation can run before the
navigation step (declarative pipeline can't pre-validate args).
- Strategy is now explicit: `Strategy.COOKIE, browser: true` for
articles (creator dashboard is logged-in only).
## hot field map
`ClusterIdStr` (or numeric `ClusterId`) → id; `Title` → title;
`QueryWord` → query (falls back to title); `HotValue` → hot_value
(non-negative numeric, else null); `Label`, `Url`, `Image` →
respective columns. `pickImage` walks `Image.url` → first truthy
`Image.url_list[]`. Empty-title rows are dropped (returns null) before
ranks are densely re-assigned 1..N.
## Tests
29 contract assertions across `parseArticlesPage` / `parseHotLimit` /
`parseToutiaoArticlesText` / `mapHotRow` + registry-level shape checks
+ `hot` adapter func behaviour (typed errors / no silent clamp / fetch
failure paths / dense-rank).
## Audits
- typed-error-lint: 196 = 196 (unchanged baseline)
- silent-column-drop: 103 = 103 (unchanged baseline)
- listing-id-pairing: hot has `id` column (round-trippable when a
detail command lands later); advisory list unchanged.
## Manifest
757 → 758 entries (+1 for `hot`).
## Doc
- index.md: toutiao mode 🔐 → 🌐/🔐 (hot is public, articles is logged-in)
- toutiao.md: per-command mode/domain table + column docs + prerequisites
* fix(toutiao): tighten hot and articles contracts
* fix(linkedin): surface detail_error on --details (no silent catch / no silent empty)
The previous --details enrichment path had two indistinguishable failure modes
that both produced `description: '', apply_url: ''`:
1. `if (!job.url)` early return — row had no jobId, so we couldn't navigate.
2. `} catch {}` — page.goto / page.evaluate threw (network, timeout, parse error).
Callers couldn't tell "upstream had no description" from "we failed to fetch",
and the catch swallowed every error without logging. For an enrichment that
costs one page navigation per row, silent failure is especially harmful — users
just see an empty cell with no way to debug.
Fix: replace empty strings with `null` for missing/failed rows, add a new
`detail_error` column (string|null) carrying a short typed reason:
- 'no url' — row had no jobId
- 'fetch failed: <msg>' — page.goto / page.evaluate threw
- 'missing description' — page loaded but body was empty
- null — success
Every failure is also logged to stderr with the offending URL so debugging is
possible. Per-row failures still don't abort the batch (the original intent),
but they're now visible.
Tests: 13 new contract assertions in clis/linkedin/search.test.js covering
parseCsvArg, mapFilterValues (ArgumentError on unknown values), decodeLinkedinRedirect,
and 5 enrichJobDetails paths (no-url / goto-throw / empty-description / success /
multi-row-mixed). Added `export const __test__` for testability.
Audits clean: typed-error-lint 196/196, silent-column-drop 103/103.
* fix(linkedin): fail fast on auth walls
* enrich(ctrip): hotel-suggest + bug fixes (silent clamp, dropped columns, fake URL)
Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #1.
## New command
- `ctrip hotel-suggest` — surfaces hotel-context suggestions (cities,
business areas, individual hotels) via the same backing endpoint with
searchType=H. Distinct from `ctrip search` (searchType=D) which returns
destinations / scenic spots / railway stations.
## Bug fixes for `ctrip search`
- **Silent clamp on `--limit` removed**: out-of-range values (≤0, ≥51,
non-integer) now raise `ArgumentError` with explicit bounds rather than
silently snapping to [1, 50].
- **Silent column drop fixed**: previously the adapter discarded `id`,
`cityId`, `cityName`, `provinceName`, `countryName`, `lat`, `lon`, `eName`
and `displayType` from upstream rows. Now all are surfaced as stable
columns.
- **Fake URL fixed**: previously `url` was always `''`. Now constructs
canonical Ctrip URLs by `type` (City / Markland / Hotel / Zone / RailwayStation)
and returns `null` (no silent fabrication) for unknown types.
- **In-band error envelope typed**: `Result: false` payloads now surface
as `COMMAND_EXEC` (was previously not handled — adapter returned empty
rows).
## Doc fix
- `Mode: 🔐 Browser` → `🌐 Public` (search uses public API, no login)
- Add `hotel-suggest` to commands table in both `docs/adapters/index.md`
and `docs/adapters/browser/ctrip.md`.
## Coords picker
Mainland China rows ship `gdLat`/`gdLon` (gaode); international rows ship
`gLat`/`gLon` (wgs84). Adapter picks the first non-zero pair (zero is the
upstream sentinel for "missing"); returns `null` if all variants are zero.
## Tests
25 contract assertions across `parseLimit` / `pickCoords` / `buildUrl` /
`mapSuggestRow` + registry-level checks for both commands (Strategy /
shape parity / typed errors / no silent clamp).
## Audits
- typed-error-lint: 196 = 196 (unchanged baseline)
- silent-column-drop: 103 = 103 (unchanged baseline)
- listing-id-pairing: advisory only (search has `id` round-trip column)
## Manifest
757 → 758 entries (+1 for `hotel-suggest`).
* fix(ctrip): wrap suggest fetch and json failures
* feat(deepseek): add detail and send commands for explicit conversation control
doubao already ships `detail <id>` and `send` for ID-explicit conversation
read/write; deepseek had only `read` (current page only) plus the
implicit-resume `ask`. Adding both gives users a stable handle when they
know the conversation ID, without going through `ask`'s resume detection
or its full prompt-then-wait pipeline.
`deepseek detail <id>`:
- parses a bare UUID or any URL containing `/a/chat/s/<id>`,
- rejects malformed input via `ArgumentError` before any browser
navigation,
- navigates to `https://chat.deepseek.com/a/chat/s/<id>` and returns
the visible message list,
- throws `EmptyResultError` when the conversation has no rendered
messages.
`deepseek send <id> <prompt>`:
- takes the conversation id as a required positional, because the
framework runs each browser command in an ephemeral per-command
workspace (a fresh tab) and there is no shared "current conversation"
across commands; the navigation must be explicit,
- drives input through CDP `Input.insertText` via `page.nativeType`,
mirroring the doubao adapter (#1278); `execCommand('insertText')` plus
a synthesised input event leaves the React-controlled state desynced
on a freshly-opened tab and the resulting click silently no-ops,
- keeps the verification loop inside the same `page.evaluate` so the
framework cannot close the tab mid-flight; counts user-class bubbles
by text-match (DeepSeek virtualises the message list, so a numeric
bubble-count check is unreliable),
- throws `CommandExecutionError` with a specific reason when the
textarea did not populate, the send button stayed disabled, the
bubble never settled, or the optimistic render rolled back during
a 3s settle window,
- treats "Promise was collected" from the post-click eval as success,
matching the existing pattern in `ask --file`.
Helper `parseDeepSeekConversationId` is exported from utils.js so the
same parser feeds both commands and round-trips the canonical lower-case
ID.
Tests:
- utils.test.js: 5 cases covering bare UUID, upper-case
normalisation, URL extraction with and without query string, empty /
null / whitespace input, and non-UUID rejection.
- detail.test.js: 5 cases covering registration, navigation +
message return, URL normalisation, ArgumentError before browser
navigation, and EmptyResultError on no-messages.
- send.test.js: 7 cases covering registration, ArgumentError on bad
id, full happy-path through nativeType + IIFE verification, the
textarea-mount timeout, missing nativeType helper, focus failure,
IIFE-reason translation to CommandExecutionError, and the
"Promise was collected" success path.
Manifest auto-regenerated to register both commands.
Live-verified end-to-end against my own DeepSeek session:
- `detail` returns the canonical message list for a bare UUID, parses
a full chat URL, and rejects malformed IDs before any browser
navigation,
- `send` lands the prompt as the latest user message in the target
conversation and gets an AI response back; reload of the
conversation page in a separate tab confirms the message persisted
server-side.
* docs(deepseek): document detail and send commands
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: 11 read adapters across 8 sites (dblp / steam / bbc / devto / lobsters / medium / coingecko / hf)
Round 2 of the adapter expansion sweep. All 11 commands hit public APIs (no
browser, no auth), follow the post-#1332 typed-error / no-silent-failure
discipline, and were live-verified against real endpoints.
New adapters:
- dblp/author : recent publications for one author (resolve PID by name, or pass --pid)
- steam/search : storefront name search (storesearch API)
- steam/app : single app detail (appdetails API; HTML entities decoded)
- bbc/topic : per-topic RSS (8 canonical BBC News feeds)
- devto/latest : /api/articles/latest with --page pagination
- lobsters/domain : stories from a specific source domain (/domains/<d>.json)
- medium/tag : tag RSS (description full-length, no silent truncation)
- coingecko/exchanges : trust score + 24h BTC volume leaderboard
- coingecko/categories : sector buckets with 6 sort options
- coingecko/global : aggregate market totals + BTC/ETH dominance
- hf/paper : single-paper detail by arXiv id (summary, ai_summary, ai_keywords, upvotes)
Also adds clis/steam/utils.js + clis/bbc/utils.js as shared helpers (HTML entity
decode, RSS parsing). All listings carry a round-trippable id where a detail
sibling exists; advise:listing-id-pairing reports zero new violations. typed-
error-lint and silent-column-drop gates both unchanged from baseline.
Manifest: 698 → 709 (+11 entries).
* fix: tighten adapter round2 contracts
* Add uisdc news adapter for CLI
Implements a CLI adapter for fetching the latest AI/design news from uisdc.com. Allows specifying the number of news items to return.
* feat(aibase): add aibase daily news adapter
This file implements a news adapter for AIbase that fetches the latest AI industry news and allows for configurable limits on the number of news items returned.
* fix(news): harden uisdc and aibase adapters
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Document the pattern for running opencli on a remote machine while keeping
the daemon and Chrome on the local machine. Reverse-tunnel local 19825
back to the remote (via SSH -R or frp) so the remote opencli still talks
to its own loopback and the daemon never leaves localhost.
Captures the rationale we landed on after reviewing #636: native
extension-to-remote-daemon support is deferred until the daemon protocol
gains authentication; in the meantime this is the safe, zero-code path
that achieves the same outcome.
* feat: add tiktok creator-videos command
TikTok Studio creator content list with views/likes/comments/saves/shares.
Hits the Studio item_list endpoint
(https://www.tiktok.com/tiktok/creator/manage/item_list/v1/?aid=1988) from a
logged-in /tiktokstudio/content session and pages with cursor until limit is
satisfied (server caps size at 50). Username for the resulting video URL is
extracted from the user_text= query param on play_addr / download_info entries,
falling back to scraping a[href*="/video/<id>"] from the Studio page DOM.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tiktok): regen manifest + replace silent-clamp with ArgumentError
- Regenerate cli-manifest.json (CI gate: must match `npm run build` output)
- Replace `Math.max(1, Number(args.limit) || 20)` and
`Math.min(Math.max(limit, 1), 50)` with an explicit positive-integer
guard + a server-cap-only ternary, per the silent-clamp guidance in
references/typed-errors.md (typed-error-lint baseline is unchanged)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tiktok): tighten creator videos contract
---------
Co-authored-by: root <root@example.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
PR #1297 introduced a CI gate that fails when a site has both a listing
and a detail command but the listing rows don't carry an id-shaped column.
The gate came with a 10-entry EXEMPT map (topic-string trending,
profile-attribute rows, UI-only sessions, ...) where each exemption
recorded a "why this listing legitimately doesn't pair" reason.
By the same filter that closed PR #1311 (write-without-delete-pair gate):
Is "listing should pair with detail" a *permanent* anti-pattern, or
case-by-case business judgment?
It's case-by-case. Topic-string listings and profile-attribute rows
genuinely don't pair with a detail command. The fact that we needed an
EXEMPT map with 10 entries and individual reason strings is the smell —
it's not the rule winning, it's the rule failing. Forcing every adapter
PR to either add an id column or file an exemption was a higher cognitive
cost than the silent-loss bugs the rule actually catches.
Changes:
- .github/workflows/ci.yml — drop the "Check listing↔detail id pairing"
step. Other gates (silent-column-drop, typed-error-lint) stay in place.
- package.json — rename the script from `check:listing-id-pairing` to
`advise:listing-id-pairing` to make the advisory nature explicit.
- scripts/check-listing-id-pairing.mjs — drop the `--strict` flag and the
EXEMPT map. The script now always exits 0 and prints an advisory report
of listings that don't carry an id-shaped column. Reviewers/authors use
it as guidance, not a gate.
- docs/conventions/listing-detail-id-pairing.md — rewrite from "MUST" to
"soft convention". Adds an explicit "why advisory, not a gate" section
that lists the legitimate non-pairing categories so future readers know
the rule's boundary.
- docs/developer/ts-adapter.md — match the advisory tone in the
adapter-author guidance.
The doc, the script, and the column patterns table all stay — agents and
adapter authors can still consult them. What's gone is the CI failure and
the per-PR exempt-list maintenance burden.
Net diff: -34 lines (gate + EXEMPT map removed, advisory-tone doc adds
a small "why advisory" section).
Adds a baseline CI gate for convention-audit typed-error lint findings. Also refreshes the silent-column-drop baseline for dianping changes already on main.
* feat(dianping): browser adapter — search + shop on www.dianping.com
Adds two browser-mode adapters for the dianping (大众点评) PC site:
- `dianping search "<keyword>" --city <name|id> --limit <n>`: keyword
shop/restaurant search. Returns rank, shop_id, name, rating, reviews,
price, cuisine, district, url. shop_id round-trips into `dianping shop`.
- `dianping shop <shop_id>` (alias `detail`): shop detail sheet
(field/value rows: name, rating, breakdown 口味/环境/服务/食材, reviews,
price, rank, hours, address, subway, features, url).
Both use Strategy.COOKIE on www.dianping.com (the PC site renders search
SSR and does not require JS hydration). m.dianping.com is intentionally
crippled for non-mobile UAs, so it's not used.
Auth detection (utils.detectAuthOrEmpty) inspects both response text and
final URL for the Meituan Yoda captcha redirect (verify.meituan.com) and
the dianping login redirect; raises AuthRequiredError with the captcha
URL embedded so the user can clear it manually in the same profile.
Listing↔detail id pairing: search.shop_id → shop.<id>. Adds 'shop' to
DETAIL_NAMES in scripts/check-listing-id-pairing.mjs so the convention
gate scans this site (35 sites / 78 listings now covered).
* fix(dianping): harden browser failure classification
* fix(dianping): fail on partial missing shop ids
Adds opencli convention-audit for batch convention scanning, with structured output, strict mode, docs, and startup isolation from local user/plugin discovery.
While auditing instagram/facebook/pixiv coverage gaps, found that pixiv
listings already extract `user_id` and construct `url` per row but drop
both fields from the table view (`columns` doesn't list them). The data
is in the row object — only the column projection was missing.
Per the listing↔detail id pairing convention (#1297), surface them so:
- `user_id` round-trips from `ranking` / `search` → `user` / `illusts`
- `url` is the canonical share link for every illust / user record
Changes:
- `ranking`: + user_id, + url
- `search`: + user_id, + url
- `illusts`: + url (user_id is the arg, no need to repeat per row)
- `user`: + url
No behavior change beyond the table view — JSON output already had these
fields, so existing scripts that consume `-f json` keep working.
* feat(dblp): public bibliography adapter — search + paper
Wraps the dblp.org public API:
- `dblp search <query>` → /search/publ/api JSON, projected into one row per hit
- `dblp paper <key>` → /rec/<key>.xml, parsed into a one-row record
Why dblp on top of arxiv/openreview: dblp is the largest, oldest CS
bibliography (7M+ entries) and the only one of the three that consistently
indexes pre-arXiv literature, journal articles, books, and theses. The
canonical record key (e.g. `conf/nips/VaswaniSPUJGKP17`) round-trips
cleanly between the two commands per the listing↔detail convention.
Implementation notes:
- No deps beyond the registry — XML parsed with conservative regexes,
same approach as the arxiv adapter.
- Polite User-Agent per dblp's API guidance; HTTP 429 mapped to a
CommandExecutionError with a "lower --limit" hint.
- Author homonym suffixes (`"Smith 0001"`) trimmed for clean output.
- 39 unit tests cover validators, XML extraction, both commands.
* fix(dblp): fail fast on API status envelopes
* feat(convention): listing↔detail id pairing rule + CI gate
Adds a hard convention: when a site exposes both a listing-class command
(search / hot / top / recent / ...) and a detail-class command (read /
paper / article / view / ...), every listing row MUST surface an id-shaped
column whose value round-trips into the detail command. Without that, an
agent has no way to follow up on a listing row except re-searching by
title or scraping URLs out of band — both of which break the agent-native
contract.
What's in this PR
- docs/conventions/listing-detail-id-pairing.md — full rule, examples
table, why-it-matters, what counts as id-shaped, exemption taxonomy,
how to add an id column to a listing.
- scripts/check-listing-id-pairing.mjs — validator that reads
cli-manifest.json, classifies each entry as listing / detail / other,
and fails when a listing on a site that also has a read-detail command
is missing an id-shaped column. Exemption allowlist records WHY each
pair is exempt so future maintainers know what to verify.
- npm run check:listing-id-pairing — strict-mode wrapper.
- CI: new step in build job runs the validator after the manifest
freshness check on Linux.
- docs/developer/ts-adapter.md — cross-link from the adapter authoring
guide.
- docs/.vitepress/config.mts — sidebar entries for the new conventions
section.
Fixes brought to zero violations
- 1688/search: add offer_id (already extracted, just surfaced)
- bluesky/user: add uri (AT URI round-trips into bluesky/thread)
- tieba/search: add id + url (thread_id already extracted)
- tieba/hot: add url (rows are topics, not threads — url is the
best-effort round-trip handle, doc'd as such)
Exemptions (intentional, doc'd in EXEMPT map with rationale)
- nowcoder/hot, bluesky/trending, twitter/trending — listing rows are
topic strings, not posts.
- lesswrong/user, reddit/user — rows are profile-attribute key/value
pairs, addressed by the username arg.
- discord-app/search — desktop UI session, message ids not extractable.
- notion/search — Strategy.UI Quick Find, page ids not exposed in DOM.
Validator output after this PR: 32 sites scanned, 75 listings checked,
7 exempted, 0 violations.
* fix(convention): tighten listing id gate
* fix(convention): close url-derived id loophole
* feat(indeed): add `search` and `job` adapters (US site)
Adds an Indeed adapter that fills the US job-search gap (alongside
existing 51job / boss-zhipin / linkedin coverage). Both commands run
through a real browser session because Indeed sits behind Cloudflare
and answers bare HTTP fetches with `403` + `cf-mitigated: challenge`.
## Commands
- `indeed search <query>` — keyword job search
- args: `query`, `--location`, `--fromage`, `--sort`, `--start`, `--limit`
- columns: `rank, id, title, company, location, salary, tags, url`
- `indeed job <jk>` (alias `detail`, `view`) — full job posting
- args: `id` (positional, the 16-char hex `jk` from `search`)
- columns: `id, title, company, location, salary, job_type, description, url`
## Listing↔detail id pairing
`search.id` is the Indeed `jk` (job key, 16-char lowercase hex). It feeds
directly into `indeed job <jk>`. Conforms to the listing↔detail id
pairing convention proposed in #1297.
## CF challenge handling
The adapter polls the result selectors for up to 15s after navigation,
giving the browser time to clear the Cloudflare interstitial. If the
challenge is still up after the wait, the adapter throws a
`CommandExecutionError` with a hint pointing the user at the connected
browser to clear it once. Subsequent calls reuse the warmed cookies via
`Strategy.COOKIE`, mirroring the v2ex / boss / linkedin patterns.
## Validation
`utils.js` keeps argument validation pure and unit-testable:
- `requireJobKey` rejects anything that isn't a 16-char lowercase hex
- `requireFromage` only accepts `1` / `3` / `7` / `14` (Indeed's enum)
- `requireSort` only accepts `relevance` / `date`
- `requireBoundedInt(limit, default=15, max=25)` — Indeed serves at most
one page (10 jobs/page); ArgumentError on out-of-range, no silent
clamping, per the typed-error feedback in #1289.
## Tests
18 unit tests in `clis/indeed/indeed.test.js` cover registration,
validators, URL builders, and DOM-card normalizers. Browser-driven
verification stays out of CI by design (CF challenge is interactive).
## Docs
- `docs/adapters/browser/indeed.md` — full adapter doc with prerequisite
CF-challenge notes and listing↔detail id pairing callout.
- Sidebar entry + adapter index row.
* fix(indeed): tighten timeout fail-fast and runtime tests
* fix(indeed): align readiness with search parser
* feat(openreview): add public adapter — search/venue/paper/reviews
OpenReview is the open peer-review platform used by ICLR / TMLR / COLM
and ML workshops. Its v2 API exposes everyone-readable submissions,
reviews, and decisions without auth, so all four commands run with
`browser: false`.
Commands:
- `openreview search <query>` — full-text search
- `openreview venue <venue>` — list submissions; accepts either a venue
display name (matched against `content.venue`, e.g. "ICLR 2024 oral")
or a full invitation id (e.g. "ICLR.cc/2025/Conference/-/Submission")
via `/-/` heuristic; supports offset pagination
- `openreview paper <id>` — single-paper detail with full abstract
- `openreview reviews <forum>` — paper + threaded reviews/decisions/
comments, ordered chronologically with paper lifted to row 0;
classifies notes via invitation tail (REVIEW / DECISION / REBUTTAL /
COMMENT / META_REVIEW / WITHDRAWAL); per-row truncation via
`--max-length` (min 200)
Listing IDs round-trip into `paper`/`reviews`. PDF URLs normalized to
absolute `https://openreview.net/pdf/...`. `pdate` falls back to
`cdate` when missing, formatted as `YYYY-MM-DD`.
All limits/offsets/ids fail-fast with typed errors (`ArgumentError`,
`EmptyResultError`, `CommandExecutionError`) — no silent clamping, no
empty-array fallbacks. fetch + json + non-2xx + 404 are wrapped so
network/API failures never look like empty results.
Tests: 23 unit tests covering the column contract, content extraction,
date/PDF normalization, invitation-vs-venue dispatch, error paths
(network/JSON/HTTP), pagination offset accounting, and the reviews
classifier + section joiner + truncation.
Live-verified against api2.openreview.net for search ("diffusion
model"), venue ("ICLR 2024 oral"), paper (KS8mIvetg2), and reviews on
that paper's full thread.
* fix(openreview): tighten error and review typing
* fix(openreview): stabilize review contracts
jakevin