mirror of
https://github.com/jackwener/OpenCLI.git
synced 2026-09-14 18:25:42 +08:00
jakevin
f594e500a8
* feat: AutoResearch framework + V2EX test suite (40 tasks) AutoResearch framework (Karpathy-style autonomous iteration): - engine.ts: 8-phase loop (review → modify → commit → verify → guard → decide → log) - config.ts: typed config + CLI parser + metric extraction - logger.ts: TSV append-only results log - commands/run.ts: main loop spawning Claude Code per iteration - commands/plan.ts: interactive config wizard - commands/fix.ts: auto-detect broken state, iteratively fix - commands/debug.ts: hypothesis-driven debugging for failing tasks V2EX test suite (5 layers, 40 tasks): - L1 Atomic (10): open, state, click, scroll, eval, back, wait - L2 Single Page (10): hot topics, node list, topic meta, pagination - L3 Multi-Step (10): click-read, navigate-node, tab-then-topic, pagination - L4 Write Ops (5): reply typing, favorite detection, form detection - L5 Complex Chain (5): cross-page collect, multi-node compare, full workflow Presets: operate-reliability, skill-quality, v2ex-reliability * test: V2EX test suite 60/60 — fix selectors, add harder tasks - Fix v2ex-collect-hot-authors selector (pathname-based member link detection) - Fix v2ex-wait-text judge (accept "appeared") - Fix trailing commas in eval step strings - Add 20 harder tasks: state+click interaction + long chain workflows - Baseline: 60/60 across all layers * feat: Zhihu test suite — 60 tasks across 8 layers, 60/60 passing Knowledge-intensive Chinese Q&A site (React SPA, lazy loading, complex DOM): - L1 Atomic (10): open, state, title, url, scroll, tab, back, wait, keys, screenshot - L2 Feed (8): feed titles, hot list, metrics, tabs, authors, content types, avatar, search - L3 Question (8): title, meta, answer, votes, buttons, descriptions, answer count - L4 Navigation (8): hot→question, feed→question, author profile, search, topic, user, back - L5 Write (6): upvote/follow/comment/bookmark/write-answer/share button detection - L6 Chain (8): read-answer-author, author-profile, multi-hot, search-then-read, scroll-answers - L7 Search (6): basic, people, topic, click-result, filter, back - L8 Complex (6): full workflow, deep author chain, cross-question, search-read, 3-page, scroll-deep Key fixes during development: - Zhihu search page needs 5s+ wait (SPA lazy loading) - Back navigation goes to about:blank (daemon init page), fixed with direct navigate - User profile answers page needs 4s wait for content - Broader selectors needed (h2 a instead of specific class names) * feat: combined eval-all runner + combined-reliability preset * experiment(operate): fix extract-npm-description + nav-click-link-example Round 1: Fix 2 remaining browse-tasks failures: - extract-npm-description: use generic <p> selector instead of class-based - nav-click-link-example: include URL in output (title is 'Example Domains', not 'IANA') * experiment(operate): fix bench-imdb-matrix — use broader selectors for year/rating Round 2: IMDB page selectors were too specific (data-testid changed). Use generic h1 for title, link text match for year, broader class match for rating. * experiment(operate): add edge cases + fix SPA navigation timing Round 3: Add 10 edge case tasks (5 V2EX + 5 Zhihu): - rapid-navigate: 3 consecutive opens - eval-after-click: verify URL changes after SPA click - scroll-and-extract: extract after deep scroll - structured extraction: multi-field JSON from dynamic content - lazy-load answers: scroll triggers more content Key finding: Zhihu SPA click() doesn't update location.pathname immediately. Use window.location.href = a.href for reliable navigation. V2EX: 65/65, Zhihu: 65/65, Browse: 59/59 = 189/189 * experiment(operate): add agent-style tasks using state+click+type (no eval for interaction) Round 4-5: Add 5 tasks that test the actual agent workflow: - agent-click-first-topic: find topic index via data-opencli-ref - agent-type-search: type into search using state index - agent-click-navigate-back: click by ref, verify navigation - agent-state-has-interactive: verify state output format - agent-state-after-scroll: verify scroll position in state V2EX: 70/70 tasks * fix: review fixes — extractVerdict, stderr, dead code - eval-skill.ts: remove dead TASKS_FILE variable (skill-tasks.yaml never existed) - eval-skill.ts: rewrite extractVerdict to use brace-counting JSON.parse instead of regex (handles escaped quotes in explanation) - eval-browse.ts: include stderr in runCommand error output for debuggability