Files
jackwener__opencli/clis/36kr/article.js
Benjamin Liu a50074d684 fix(adapters): drop silent-sentinel row fallbacks across 6 read commands (#1631)
* fix(adapters): drop silent-sentinel row fallbacks across 6 read commands

Continues the audit-baseline cleanup started in #1611 (lesswrong) and
the direction set by #1599 / #1603 / #1604. Replaces the
`silent-sentinel` row-data fallbacks (`'Unknown'` / `'-'` / `'unknown'`
that mask missing fields) with the empty-string signal so agents can
tell apart "field really has the value Unknown" from "upstream returned
no value".

Touched 6 read adapters, 10 baseline entries:
- wikipedia/trending: title, description
- 36kr/article: author, date, body
- xiaoyuzhou/download: podcast
- xiaoyuzhou/transcript: podcast
- zhihu/collection: dedup key + type field (the empty prefix still
  produces a unique-per-content dedup key, just without the `unknown:`
  noise)
- zhihu/download: author

Intentionally skipped (line-by-line audited):
- v2ex/me.js: `'Unknown'` is an in-band control-flow sentinel. Line 35
  initialises `let username = 'Unknown';`, line 41 uses
  `if (username === 'Unknown')` to trigger the profileEl fallback
  selector, line 75 uses the same check to raise the auth error.
  Empty would silently bypass both checks and return a row with an
  empty username as if auth succeeded.
- v2ex/daily.js: `'未知'` is user-facing 签到 success text in the
  rendered status message, not a row field. Empty would render a
  broken sentence.
- weibo/comments.js, weibo/feed.js: the sentinel sits inside an in-IIFE
  error-message string composition (`'API error: ' + (data.msg || 'unknown')`),
  not in a returned row. Empty would silently truncate diagnostic
  output. Both stay on baseline.

Verified live: `opencli wikipedia trending --limit 3` and `opencli 36kr
hot --limit 2` both return populated rows; the empty-string signal only
kicks in when the upstream value is actually missing.

* test(adapters): add empty-signal coverage for the cluster-2 sentinel swap

Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with focused unit tests that
mock the upstream to return null / missing fields and assert the row
surfaces an empty-string signal instead of the old fabricated
'Unknown' / '-' / 'unknown' sentinel.

Coverage:

- clis/wikipedia/trending.test.js (new): mocks wikiFetch to return
  three articles - one with both title + description populated, one
  with no title and no description, one with title only. Asserts the
  missing fields render as '' (was '-' before this PR).

- clis/36kr/article.test.js (new): mocks page.evaluate to return a
  scrape where title is present but author / date / body are empty.
  Asserts those three fields render as '' in the row pair output
  (was '-' before this PR). Also covers the NOT_FOUND and
  INVALID_ARGUMENT error paths that already existed.

- clis/zhihu/collection.test.js (+1 case): mocks the zhihu collection
  API to return an item with content.id but no content.type. Asserts
  type renders as '' (was 'unknown' before this PR); the new dedup
  key prefix is :id rather than unknown:id, semantically identical
  for dedup purposes.

The other three files in this PR (xiaoyuzhou/download,
xiaoyuzhou/transcript, zhihu/download) use the same `|| 'unknown'` ->
`|| ''` value swap with no downstream sentinel consumer. They are
covered by the same JS language semantics the three tests above
demonstrate.

* fix(adapters): fail typed on missing row identity

* fix(adapters): tighten sentinel row identity guards

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:35:14 +08:00

67 lines
2.8 KiB
JavaScript

/**
* 36kr article detail — INTERCEPT strategy.
*
* Fetches the full content of a 36kr article given its ID or URL.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
/** Extract article ID from a full URL or a bare numeric ID string */
function parseArticleId(input) {
const m = input.match(/\/p\/(\d+)/);
return m ? m[1] : input.replace(/\D/g, '');
}
cli({
site: '36kr',
name: 'article',
access: 'read',
description: '获取36氪文章正文内容',
domain: 'www.36kr.com',
strategy: Strategy.INTERCEPT,
args: [
{ name: 'id', positional: true, required: true, help: 'Article ID or full 36kr article URL' },
],
columns: ['field', 'value'],
func: async (page, args) => {
const articleId = parseArticleId(String(args.id ?? ''));
if (!articleId) {
throw new CliError('INVALID_ARGUMENT', 'Invalid article ID or URL');
}
await page.installInterceptor('36kr.com/api');
await page.goto(`https://www.36kr.com/p/${articleId}`);
await page.wait(5);
const data = await page.evaluate(`
(() => {
// Title: 36kr uses class "article-title" on h1
const title = document.querySelector('.article-title, h1')?.textContent?.trim() || '';
// Author: second .author-name (first is empty nav link, second has real name)
const authorEls = document.querySelectorAll('.author-name');
const author = Array.from(authorEls).map(el => el.textContent?.trim()).filter(Boolean)[0] || '';
// Date: 36kr uses class "title-icon-item item-time" for the publish date
const dateRaw = document.querySelector('.item-time')?.textContent?.trim() || '';
const date = dateRaw.replace(/^[·\s]+/, '').trim();
// Article body paragraphs
const bodyEls = document.querySelectorAll('[class*="article-content"] p, [class*="rich-text"] p, .article p');
const body = Array.from(bodyEls)
.map(el => el.textContent?.trim())
.filter(t => t && t.length > 10)
.join(' ')
.slice(0, 800);
return { title, author, date, body };
})()
`);
if (!data?.title) {
throw new CliError('NOT_FOUND', 'Article not found or failed to load', 'Check the article ID');
}
if (!data.body) {
throw new CliError('PARSE_ERROR', 'Article body not found', '36kr page loaded but no article body paragraphs were extracted');
}
return [
{ field: 'title', value: data.title },
{ field: 'author', value: data.author || '' },
{ field: 'date', value: data.date || '' },
{ field: 'url', value: `https://36kr.com/p/${articleId}` },
{ field: 'body', value: data.body || '' },
];
},
});