skills/jackwener__opencli
1
0
Fork 0
mirror of https://github.com/jackwener/OpenCLI.git synced 2026-09-14 18:25:42 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,151 Commits 348 Branches 114 Tags
ci/drop-e2e-from-pr-trigger
Commit Graph

6 Commits

Author SHA1 Message Date
jakevin ff7d741a4c perf(twitter): drop redundant goto+wait — framework auto pre-navs (PR C) (#1451) 
* perf(twitter): drop redundant goto+wait — framework auto pre-navs (PR C)

Twelve twitter read adapters did `await page.goto('https://x.com'); await
page.wait(2~3)` purely to establish cookie context for the subsequent
`document.cookie` read. After PR #1450 hoisted those reads to
`page.getCookies({url})` (which queries the CDP cookie store directly,
no navigation needed), the explicit goto+wait became dead.

The framework already pre-navigates to `https://${domain}` for any
adapter declaring `Strategy.COOKIE + domain` (`src/registry.ts:191`),
so the cookie store is populated before `func` runs. The 2-3s
`page.wait` was the slowest part of the redundant call.

Files (all read-only, all ct0/cookie-only):
- bookmark-folder / bookmark-folders / bookmarks
- following / likes / list-add / list-remove / list-tweets / lists
- thread / timeline / tweets

Out of scope (kept as-is): goto calls that navigate to a *specific*
URL needed for content/SPA shell — `trending` (`/explore/tabs/trending`),
`notifications` (`/home`), `article` (`/i/article/{id}`), `profile`
(`/${username}`), and `list-add` line 133 (`/${username}` for UI ops).

Verification:
- npx tsc --noEmit ✓
- npx vitest run clis/twitter → 216/216 ✓
- typed-error-lint 189/189, 0 new ✓
- silent-column-drop 103/103, 0 new ✓

* fix(twitter): keep list UI root navigation
 2026-05-11 01:24:07 +08:00
jakevin 60dbbd4baa perf(adapters): hoist cookie reads to page.getCookies (Tier 1, 25 files) (#1450) 
* perf: replace document.cookie reads with page.getCookies({domain}) (Tier 1 cookie API sweep)

Prior pattern in 25 adapter files round-tripped through `page.evaluate(\`document.cookie.split…\`)` to extract a single cookie value (CSRF token, session ID, etc.). CDP's `page.getCookies({domain})` reads the cookie store directly with zero JS-execution overhead.

Files touched (sites: twitter / linkedin / maimai / youtube):

- twitter (15): thread, timeline, list-add, bookmark-folders, following, list-tweets, bookmarks, list-remove, tweets, bookmark-folder, likes, lists, trending — direct 4-line replacement (cookie was outside `page.evaluate`); article, profile — hoisted ct0 read OUT of `page.evaluate` and threw `AuthRequiredError` upfront so unreachable in-evaluate auth branches got cleaned up too.

- linkedin/search.js — JSESSIONID was read inside the per-batch fetch loop's `page.evaluate`; hoisted once before the loop and pass `csrf` value into the template via `JSON.stringify`.

- maimai/search-talents.js — csrftoken cookie hoisted via getCookies; meta-tag fallback preserved inside `page.evaluate` (reached only when no cookie). Also converted the `page.evaluate(async (body) => …, body)` Playwright-style call to OpenCLI's template-string form so the helper actually runs.

- youtube — `SAPISID_HASH_FN` (used by like / unlike / subscribe / unsubscribe) reworked: sapisid is now passed in as a parameter; new `readYoutubeSapisid(page)` helper reads it via CDP. The HMAC-SHA1 compute still happens browser-side (Web Crypto), only the cookie read is hoisted.

Tests updated where mocks specifically referenced `document.cookie` (twitter following / bookmark-folder / bookmark-folders) to mock `getCookies` instead.

Verification:
- `npx tsc --noEmit` clean
- `npx vitest run clis/twitter clis/linkedin clis/youtube` → 264/264 pass
- typed-error-lint 189/189 (no new violations)
- silent-column-drop 103/103 (no new violations)

Scope notes (not in this PR):
- `goto + wait` redundancy and `browserSession: { reuse: 'site' }` rollout are scoped to follow-up PRs B and C per the #OpenCLI:3889b5cf thread plan.
- `document.cookie.match(...)` patterns (instagram 8 / xiaoe / qwen / hupu / tiktok / 1point3acres — ~13 files) are outside the original \`document.cookie.split\` audit scope and will follow as a Tier 1 expansion sweep.

* fix(adapters): read auth cookies by url scope
 2026-05-11 01:06:48 +08:00
jakevin 4475d4efe3 feat(twitter): P1+P2+P3+P4+P5 — search filters, bookmark folders, engagement scoring, sibling dedupe + help docs (#1406) 
Round 21 follow-up to #1400 (P0 write-action symmetry, merged `644d4517`). 5 features + help docs unified into one PR per WAWQAQ "全部合成一个 PR" directive.

## Scope

- **P1** (`cf10c098`): `twitter search` `--from / --has / --exclude / --product` filters, mapping to X `from:` / `filter:` / `-filter:` / `f=` operators; legacy `--filter top|live` preserved (--product win on conflict)
- **P2** (`a484a69a`): new `twitter bookmark-folders` + `bookmark-folder <id>`; X Premium GraphQL `bookmarkFoldersSlice` + `BookmarkFolderTimeline`; queryId 三层 fallback (placeholder.json → client-web bundle → pinned constants)
- **P3** (`f209f914`): `--top-by-engagement N` to 7 tweet-shaped read commands (search/timeline/likes/bookmarks/list-tweets/tweets/thread); single helper in `utils.js`; formula `likes×1 + retweets×3 + replies×2 + bookmarks×5 + log10(views+1)×0.5`; **N=0 reference equality no-op** → existing 157 twitter tests 0 churn
- **P4** (`89283fa0`): `TWITTER_BEARER_TOKEN` + composer image helpers extracted to `utils.js` (12 GraphQL adapter dedup); reply hardening; quote adds `--image`
- **P5** (`a3d10a48`): sibling article-scope helper extracted to `shared.js` (9 write commands reuse, dedup with #1400 P0 invariant)
- **docs** (`2a358d80`): help-doc precision (positional-omitted defaults + download/bookmarks/notifications/timeline/lists description thicken; concurrent #1401/#1403 wording preserved)

47 files / +2594/-470. Tests **96 → 216 (+120)**, manifest 798 → 801 (+3), typed-error-lint 190 → 189 (resolved 1 grandfathered sentinel).

## Iteration history (3 review fix commits on top of 6 author commits)

- `7f93779b` — codex-mini1 lead fix1: 3 blocker bundle (P5 host invariant + P2 safe-id + sentinel removal + P1 fallback fail-fast)
- `2a29ecc6` — codex-mini1 lead fix2: P3 help formula consistency (doc/help text matches actual `log10(views+1)×0.5`)
- `df4dcd76` — codex-mini1 lead fix3 (F-P-1 aux catch): P2 `bookmark-folder --limit` upfront validation (`Number(kwargs.limit ?? 20)` + reject non-positive/non-integer + regression `0/negative/fractional/NaN` + `page.goto` zero-call assert)

## 4 progressive blockers caught (codex-mini1 lead 3 rounds + F-P-1 aux 1 round)

1. **P5 host invariant gap** (lead): article-scope helper preserved exact `/status/<id>` path but ignored link host → off-domain `https://evil.com/alice/status/<target>` would satisfy `__twHasLinkToTarget`. Fixed: `https` + X/Twitter host or subdomain + exact `/status/<id>` or `/i/status/<id>` path; query/hash allowed; off-domain/host-suffix/non-https/path-suffix/substring-id rejected; JSDOM positive + 5 negative anchors.

2. **P2 listing→detail round-trip + sentinel** (lead): `bookmark-folders` accepted opaque IDs but `bookmark-folder <id>` only accepted numeric → round-trip broken; new `author: 'unknown'` sentinel created fabricated author URL. Fixed: `[A-Za-z0-9_-]+` opaque safe-id (rejects `/`, `?`, `%`, spaces) + `resolveTwitterQueryId()` sanitization for queryId resolution; sentinel removed → empty author + canonical `/i/status/<id>` URL.

3. **P1 fallback silent tab miss** (lead): pushState fail → fallback typing into search box, `clickProductTabIfNeeded()` silent return on tab not found → user `--product photos` silently degraded to Top results. Fixed: throw `CommandExecutionError` when requested `--product` tab cannot be selected + invalid `--from` / `--limit` upfront pre-nav reject + double-direction tests.

4. **P2 limit silent normalize** (aux): `const limit = kwargs.limit || 20` → `--limit 0` silent → 20; negative/non-integer pre-IO unchecked. Fixed: `Number(kwargs.limit ?? 20)` + require positive integer before `page.goto` + regression covers `0/negative/fractional/NaN` + `page.goto` zero-call.

## Cultural sediment (Round 21 audit checklist 7 rules / 6 dimensions)

This PR **immediately validated 4 of 7 rules** in review pipeline:
- (b) silent-clamp class — P1 fallback silent tab miss (silent semantic-downgrade) + P2 `|| 20` silent normalize
- (e) ID exact-not-substring — P5 host invariant (was only path-exact, not host-exact)
- (f) grandfathered-not-exempt — P5 helper-refactor boundary lost host invariant + P2 new adapter inherited grandfathered `'unknown'` sentinel
- (g) fallback-must-have-success-criterion — P1 fallback path missing post-condition assertion

7 rules / 6 dimensions:
- (a) cross-grep sibling URL pattern — structural
- (b) silent-clamp class — failure mode (input)
- (c) broad querySelector → article-scoping — scope
- (d) missing-validation early reject — boundary
- (e) ID exact-not-substring — identity
- (f) grandfathered-not-exempt (corollary: applies to new file + new helper-refactor boundary; not original-file line-edit) — time-axis
- (g) fallback-must-have-success-criterion (sub-rule g': fallback unit test must include post-condition assertion, not just "doesn't throw") — failure mode (output)

**Cross-PR validation 4-chain on meta-anchor "Structural exactness for identity matching"**:
- #1391 URL layer (`isFacebookAuthRedirectPath`: top-level anchor + `\.php` + `(/|$)` segment edge)
- #1392 URL parser layer (`parseGrokSessionId`: bare UUID exact / URL host-exact-or-subdomain + path-exact)
- #1400 DOM layer (article-scoping: status-id `/\/status\/${id}(?:\/|$)/` regex / segment-array exact)
- #1406 P5 helper-refactor boundary (full URL invariant in shared helper: host+path re-anchored after extraction)
- Common invariant: boundary-lock structural shape; **fuzzy match is silent-failure 温床**; lesson lifecycle = surface-shift not add-and-forget.

**Audit framework self-discipline**: each rule must have grep-able detection signal, otherwise rule degenerates to mantra. Framework is "7 rules + sub-instance pattern in new surface", not frozen 7 rules.

**Round 17 race-mitigation 第 9 连续 race-free execution**: standard alternation cadence (#1400 A 组 → #1406 B 组), lead final + aux final + `@pr-monitor squash?` trigger, pr-monitor proactive ack + serial squash, lead silent on closeout.

## Validation gates (final head `df4dcd76`)

Local: Twitter adapter tests `25 files / 216 tests`, focused P1/P2/P3/P5 tests `99/99`, `node --check` touched runtime, `npx tsc --noEmit`, `npm run build`, manifest 801 entries, typed-error-lint `189/189`, silent-column-drop `103/103`, doc-coverage `140/140`, docs:build clean, listing-id advisory `13` unchanged (wikipedia/trending residual non-Twitter), `git diff --check` clean.

GitHub: build×3 (ubuntu/macos/windows) SUCCESS, unit-test shards SUCCESS, bun-test SUCCESS, adapter-test SUCCESS, audit SUCCESS, doc-coverage SUCCESS, docs-build SUCCESS, smoke-test skipped, PR `CLEAN/MERGEABLE`.

Reviewers:
- Lead: @codex-mini1 (3 fix rounds, all caught proactively + amend P3 help consistency)
- Aux: @First-principles-1 (better-solution triangulation on P2 queryId 三层 fallback + P5 invariant + P3 N=0 reference no-op + caught P2 limit silent normalize)
- Author: @opencli-user (5-feature scope + 7-rule sediment co-author + corollary contributor)
 2026-05-08 02:36:39 +08:00
jakevin 407b559a83 feat(help): hard-gate empty positional help text + fix 18 offenders (#1403) 
Why
- `opencli twitter followers --help` rendered:
    Arguments:
      user
  with a blank trailing column. Both humans and agents could not
  recover the parameter's purpose without reading source. WAWQAQ
  surfaced this directly: "没有说明当后面的 followers [user] [options]
  如果都没填的时候，获取的是什么？"
- This is metadata completeness, not stylistic taste. Failing closed
  is the only way to keep the help surface trustworthy as adapters
  land.

What
- src/build-manifest.ts: add `findManifestMetadataIssues()` that flags
  any positional with empty / whitespace-only / missing `help`. Wired
  into `main()` after the import-failures gate; build aborts non-zero
  with a per-arg report (`site/cmd positional "name" (sourceFile)`).
- src/build-manifest.test.ts: cover the gate (positives + negatives,
  scoped strictly to positionals — named flags are intentionally
  out-of-scope).
- 18 adapter offenders (16 required + 2 optional) get explicit help
  text:
    twitter: followers/following/list-add/list-remove/list-tweets/
             search/thread
    reddit:  search/subreddit/user/user-comments/user-posts
    douyin:  stats/update
    bilibili: subtitle
    jike:    search
  Optional positionals (`twitter followers/following [user]`) now
  document the omit semantics — fetches the currently logged-in
  account.
- CHANGELOG: document the build gate and the offender list.

Out of scope (planned follow-ups)
- Semantic-quality advisory: optional positional help should also
  contain `default / omit / current / logged-in / required unless …`
  keywords. That belongs to the planned Arg metadata v2 work
  (`when_omitted / when_present / value_format` 3-field schema).
- Named-flag `help` quality. Named flags carry the flag name itself
  in help, so a missing `help` is not as opaque; if we want to gate
  those too, do it as a separate, intentional decision.

Validation
- `npm run build`            → 799 entries, clean.
- `npm run typecheck`        → clean.
- `npx vitest run --project unit --project adapter` → 257 + 4 files,
  all green (build-manifest 13 tests, manifest gate added).
- Smoke: temporarily reverted `followers.js` help to empty → build
  aborts with the exact `twitter/followers positional "user" (...)`
  line; restored, build is clean again.
- `npm run check:silent-column-drop` and `check:typed-error-lint`
  baselines unchanged.
 2026-05-08 01:55:19 +08:00
jakevin eea9ff8bfe feat(cli): add command access metadata (#1296) 2026-05-04 19:47:08 +08:00
Ocean bb21e7e831 feat(twitter): GraphQL-based lists + list-tweets + list-add/remove (#1076) 
* feat(twitter): rewrite lists via GraphQL + add list-tweets

The DOM-scraping / detail-click approach in PR #1053 remained fragile
against X's frequent overview-page rendering changes and slow (N+1 page
loads per list). Rewrite `twitter lists` to call
`ListsManagementPageTimeline` GraphQL directly — one request returns all
owned + subscribed lists with id/name/member_count/subscriber_count/mode.

Also add `twitter list-tweets <listId>` for pulling the tweet stream from
a list, completing the read-side chain (lists → pick an id → list-tweets).

- lists: drop positional `user` arg (GraphQL returns only logged-in
  user's lists), add `id` column, change followers to exact integer from
  subscriber_count.
- list-tweets: same GraphQL pattern as bookmarks/likes (BEARER + ct0 +
  dynamic queryId with static fallback + cursor pagination).
- Delete obsolete lists-parser.js and lists.d.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(twitter): add list-add / list-remove with Save-button commit

Two new commands to toggle list membership. X's list dialog uses a
"click-to-stage, Save-to-commit" pattern — clicking a row only updates
optimistic UI; the actual POST fires when the user clicks the top-right
"Save" button. Pressing ESC or the close-X silently cancels the change.

Implementation:
- Resolve listId → name via ListsManagementPageTimeline GraphQL, so we
  match the dialog row by name (dialog rows have no data-testid listId).
- Open profile page → DOM click "…" menu → "Add/remove from Lists".
- Scroll dialog to locate target row (virtualized list).
- page.nativeClick on row — trusted CDP Input.dispatchMouseEvent fires
  React's onclick, flips aria-checked (.click() alone does not suffice;
  X ignores non-trusted events for list mutations).
- page.nativeClick on the Save button — commits to server.
- Verify by re-fetching ListsManagementPageTimeline and diffing
  member_count: success only if N→N±1. No silent successes.

This fixes the pattern where batch `list-add` calls returned success for
every user but committed zero to the server (optimistic UI lied).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: stabilize twitter list manifest and query ids

* docs: add twitter list command discoverability

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
 2026-04-19 01:42:42 +08:00