Files
jackwener__opencli/.github/workflows/ci.yml
jakevin dc724262f2 feat: agent-native retrospective — analyze / verify guards / fixture content checks (#1133)
* feat: agent-native retrospective — analyze / verify guards / fixture content checks

Post-mortem on slow 1point3acres + 51job adapter sessions, consolidated
into one PR. Scope is "reduce uncertainty and catch silent failures"
— the two things that sink agent success rate on first-time adapters.

Changes:
- `browser analyze <url>` — one command returns pattern (A/B/C/D),
  anti-bot vendor (Aliyun/Cloudflare/Akamai/Geetest), nearest adapter,
  and a single-sentence recommended_next_step. Replaces the three-step
  open/wait/network recon loop when it can reach a confident verdict.
- `browser wait xhr <regex>` — poll for a specific XHR URL instead of
  blind `wait time N`, so SPA data-arrival barriers are deterministic.
- Fixture `mustNotContain` / `mustBeTruthy` — catch two silent-failure
  modes `notEmpty` misses: content contamination (sibling DOM bleed)
  and `|| 0` / `|| false` fallbacks.
- `browser verify` post-success site-memory check + `--strict-memory`
  — verify-green no longer hides the case where `~/.opencli/sites/`
  was never written back. Memory only materializes if authors write it.
- CI: guard that committed `cli-manifest.json` matches a fresh build.
  Main was already drifted (#1118 left stale ordering + a missing arg);
  this PR regenerates the manifest and will catch the next drift.

Docs (opencli-adapter-author + opencli-autofix skills):
- `success-rate-pitfalls.md` — 10 concrete silent-failure scenarios
  seen in real adapter sessions, each with defense via fixture /
  adapter patterns.
- `autofix` gains discipline rule #6: verify pattern failure means
  tighten the adapter, never loosen the fixture.
- `site-recon.md` leads with `browser analyze`; `api-discovery.md`
  adds a §0 covering WAF vendor detection and cross-subdomain CORS
  (the two gotchas that burned the 51job session).
- `wait time 3` → `wait time 2`, with `wait xhr` as the robust choice.

* fix: make output-dir defaults host-independent in manifest

Three adapters (chatgpt/image, gemini/image, instagram/download) baked
`path.join(os.homedir(), ...)` into the `default` field of their args.
The committed manifest therefore carried my personal `/Users/jakevin/...`
paths — which agents running on a different host saw as surprising
defaults. The drift guard I just added to CI caught it on the first run.

Runtime behavior is unchanged: each adapter still falls back to
`path.join(os.homedir(), …)` inside `func` when the kwarg is absent.
Only the displayed / registered default becomes a tilde-path.

* fix(cli): enforce strict-memory without fixture

* fix(browser): harden analyze and xhr guards

* fix(browser): fallback to interceptor buffer
2026-04-22 01:59:19 +08:00

156 lines
4.5 KiB
YAML

name: CI
on:
push:
branches: [main, dev]
pull_request:
branches: [main, dev]
schedule:
- cron: '0 8 * * 1' # Weekly Monday 08:00 UTC — smoke tests
workflow_dispatch:
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
# ── Fast gate: typecheck + build ──
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
with:
node-version: '22'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Type check
run: npx tsc --noEmit
- name: Build
run: npm run build
# Guard: committed cli-manifest.json must match the one build regenerates.
# Prevents silent drift where unrelated adapter entries vanish or change
# across PRs (agent hits unexpected manifest diff → surgical-merge churn).
- name: Check cli-manifest.json is up-to-date
if: runner.os == 'Linux'
shell: bash
run: |
if ! git diff --exit-code -- cli-manifest.json; then
echo "::error::cli-manifest.json is out of sync with the source. Run 'npm run build' and commit the result."
exit 1
fi
# ── Unit tests (vitest shard) ──
# PR: ubuntu + Node 22 only (fast feedback, 2 jobs).
# Push to main/dev: full matrix for cross-platform/cross-version coverage (12 jobs).
unit-test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && fromJSON('["ubuntu-latest","macos-latest","windows-latest"]') || fromJSON('["ubuntu-latest"]') }}
node-version: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && fromJSON('["22"]') || fromJSON('["22"]') }}
shard: [1, 2]
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run unit tests (Node ${{ matrix.node-version }}, shard ${{ matrix.shard }}/2)
run: npx vitest run --project unit --project extension --reporter=verbose --shard=${{ matrix.shard }}/2
# ── Bun compatibility check ──
bun-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.5
- uses: actions/setup-node@v6
with:
node-version: '22'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run unit tests under Bun
run: bun vitest run --project unit --reporter=verbose
# Adapter tests are pure unit tests — OS doesn't affect results.
adapter-test:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
with:
node-version: '22'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run focused adapter tests
run: npm run test:adapter -- --reporter=verbose
# ── Smoke tests (scheduled / manual only) ──
smoke-test:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
needs: build
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# NOTE: Windows excluded — browser-actions/setup-chrome hangs during
# Chrome MSI installation on Windows runners (known issue).
os: [ubuntu-latest, macos-latest]
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
with:
node-version: '22'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Setup Chrome
uses: ./.github/actions/setup-chrome
id: setup-chrome
- name: Build
run: npm run build
- name: Run smoke tests (Linux, via xvfb)
if: runner.os == 'Linux'
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
npx vitest run tests/smoke/ --reporter=verbose
- name: Run smoke tests (macOS / Windows)
if: runner.os != 'Linux'
run: npx vitest run tests/smoke/ --reporter=verbose
timeout-minutes: 15