* refactor: smart sync adapters instead of full copy (#sparse-override)
Replace unconditional full-copy of all adapters to ~/.opencli/clis/ with
hash-based smart sync that only copies files whose content has changed.
Changes:
- fetch-adapters.js: use SHA-256 content hashes to skip unchanged files;
store per-file hashes in adapter-manifest.json
- discovery.ts: simplify ensureUserAdapters() to only create the directory
(no longer triggers full copy on first run)
- main.ts: fix fast completion to check manifest file existence instead of
directory existence (sparse override may have empty user dir)
- cli.ts: add `opencli adapter eject/reset/status` commands for managing
local adapter overrides
- engine.test.ts: add tests for empty user dir and ensureUserAdapters
* fix: address review blockers — site-level sync + reset --all
1. Fix `adapter reset --all`: change <site> from required to optional
argument so --all can be used without specifying a site name.
2. Change smart sync from file-level to site-level granularity:
if any file in a site has changed upstream, overwrite the entire
site directory. This matches the agreed product semantics — local
modifications to any file in a site are replaced when upstream
updates that site.
* fix: delete old site dir before writing updated adapter files
When a site has upstream changes, delete the entire site directory
first, then write the new version. This prevents stale files from
older versions lingering in the user directory.
* fix: reset --all preserves custom sites, only removes official overrides
Blocker 3 fix: reset --all now checks BUILTIN_CLIS to identify official
sites and only deletes those, preserving user-created custom sites.
* refactor: sparse sync deletes local overrides instead of copying new versions
Changed fetch-adapters.js semantics per team agreement:
- When an official site has upstream changes, DELETE the local override
instead of copying the new version into ~/.opencli/clis/
- Runtime automatically falls back to package baseline
- ~/.opencli/clis/ becomes a true sparse override layer
* fix: reset <site> rejects custom sites, only allows official overrides
Single-site reset now checks BUILTIN_CLIS before deleting, matching
the same protection that reset --all already has.
* fix: reset <site> allows custom sites per product decision
Per @WAWQAQ: explicit single-site reset should work on custom sites too.
Differentiate messaging: official sites say "using official baseline",
custom sites say "removed custom site".
reset --all still only removes official overrides (bulk safety).
* fix: reset --all deletes all local sites including custom per product decision
Per @WAWQAQ: --all should clear the entire local working cache,
including custom sites. Single-site reset already handles both types.
Binance was the only adapter left in src/clis/ after the TS→JS
migration (PR #928). Move all 11 adapters and the test file to
clis/binance/, strip TypeScript syntax from the test, and switch
the test import to the @jackwener/opencli/pipeline package export.
log.debug() requires DEBUG=opencli to output, which means
DEBUG_SNAPSHOT=1 alone no longer shows snapshot fallback diagnostics.
Use process.stderr.write directly since the DEBUG_SNAPSHOT guard
already controls when this diagnostic fires.
Users who created custom .ts adapters in ~/.opencli/clis/ will see
their commands silently disappear after upgrading to the JS-only
version. Add an explicit warning so they know to convert to .js.
The alias resolution logic checked `!registry.has(target)` before
calling `registry.get(target)`, which always returned undefined.
Moreover, aliases registered as `site/alias` keys meant `registry.has`
returned true, skipping the block entirely. The canonical name was
never resolved, so `validate site/alias` silently checked 0 commands.
Simplify to always resolve via `registry.get(target)` which handles
both canonical keys and alias keys correctly.
* perf: P0 performance optimizations — VM context reuse, startup parallelization, stealth caching
1. Reuse VM sandbox context in pipeline template engine instead of creating
a new vm.createContext() on every expression evaluation. This eliminates
~0.3ms per call in map/filter loops over large arrays.
2. Cache sanitizeContext() results via WeakMap keyed by object reference.
In pipeline loops, `args` and `data` are the same object across all
iterations — the expensive JSON round-trip now runs only once per step.
3. Parallelize independent startup I/O: built-in CLI discovery now runs
concurrently with ensureUserCliCompatShims and ensureUserAdapters,
saving ~30-50ms on cold start.
4. Cache the stealth JS string (350 lines, pure static) after first
generation — every subsequent goto() reuses the cached string.
* fix: address review feedback on P0 perf optimizations
1. sanitizeContext: cache JSON string instead of parsed object to prevent
sandbox mutation from polluting subsequent calls
2. VM sandbox: clean non-whitelisted properties before each execution to
prevent cross-expression state leakage
3. Startup parallelization: document registry overwrite semantics and
confirm no shared-state race between parallel tasks
* refactor(validate): switch from YAML scanning to registry-based validation
The validate/verify commands only scanned YAML files, which are no
longer supported. Rewrite to validate commands from the in-memory
registry populated by discoverClis(), aligning with the JS-first
adapter architecture.
New checks: missing description, browser commands without domain,
pipeline step name typos, commands without func/pipeline, duplicate
arg names, and positional arg ordering.
* fix(validate): treat lazy-loaded commands as valid
Manifest-registered commands have _lazy=true and no func/pipeline
until execution time. Recognize this as a valid execution form.
* fix(validate): warn on empty registry, support alias targets
- Emit warning when registry is empty instead of silent PASS
- Resolve alias targets to canonical key before filtering
Strategy is a 5-value enum (PUBLIC/COOKIE/HEADER/INTERCEPT/UI) that
the execution path was reading at two points — resolvePreNav() and
shouldUseBrowserSession() — to make decisions that are already fully
expressible by the existing `browser` and `navigateBefore` fields.
This commit introduces normalizeCommand() inside registerCommand(),
which expands strategy into concrete runtime fields at registration
time. After normalization, execution code never reads cmd.strategy.
normalizeCommand expansion rules:
- strategy → browser: PUBLIC defaults to false, others to true.
Explicit browser value always wins.
- strategy + domain → navigateBefore:
· COOKIE/HEADER + domain → 'https://{domain}' (pre-navigate)
· Non-PUBLIC without domain → true (needs auth context, no URL)
· PUBLIC → undefined (no auth needed)
Explicit navigateBefore (false or string) always wins.
This matters because commands enter the registry from 4 sources
(cli(), manifest, generate-verified, tests), and previously only
cli() did strategy derivation. The other 3 constructed CliCommand
directly, leaving strategy as a runtime dependency. Now all sources
converge through registerCommand → normalizeCommand.
Changes:
- registry.ts: add normalizeCommand(); simplify cli() to delegate
all derivation to normalizeCommand via registerCommand()
- execution.ts: resolvePreNav() no longer reads strategy; just
reads the already-expanded navigateBefore field. Strategy import
removed.
- capabilityRouting.ts: shouldUseBrowserSession() checks
cmd.navigateBefore (truthy = needs browser session) instead of
cmd.strategy !== PUBLIC. Strategy import removed.
- discovery.ts: manifest path no longer hardcodes browser default;
delegates to normalizeCommand.
- capabilityRouting.test.ts: test now reflects normalized command
shape (navigateBefore: true for COOKIE without domain).
strategy is preserved as metadata on CliCommand — opencli list,
cascade probe, adapter generation, and documentation continue to
read it. Only the execution path stops consuming it.
- Remove mapDistToSource() from diagnostic.ts — mapped dist/clis/
paths back to clis/ but dist/clis/ no longer exists after JS-first
migration. The function always returned null.
- Simplify resolveAdapterSourcePath() to check candidates directly
without the dead dist→source mapping detour.
- Delete scripts/clean-yaml.cjs — walked dist/clis/ to delete YAML
files, but dist/clis/ no longer exists.
- Remove clean-yaml script entry from package.json.
1. candidateToJs: escape single quotes in site, name, domain, and arg
name/type fields to prevent syntax errors in generated JS adapters.
Previously only description and help fields were escaped.
2. diagnostic: pass network request body through redactText() to
prevent sensitive data (JWT, bearer tokens) from leaking into
repair context. responseBody/responsePreview already used
sanitizeCapturedValue which calls redactText, but the body field
only had truncation.
* refactor(adapters): convert adapter layer from TypeScript to JavaScript
Core framework stays TypeScript; adapter layer moves to JS-first.
Adapters are essentially "executable config + browser scripts" that
barely use TS features — this simplifies the build/distribution pipeline
by removing the dist/clis/ intermediate compilation step.
Changes:
- Convert all 753 adapter files in clis/ from .ts to .js
- Update tsconfig to exclude clis/ from compilation
- Simplify build-manifest to scan clis/*.js directly (no dist/clis/)
- Update discovery, main, fetch-adapters to load JS adapters from clis/
- Update generate-verified to output .js artifacts
- Update package.json files field: dist/clis/ → clis/
- Fix all test files for the .ts → .js transition
* fix(main): use findPackageRoot for BUILTIN_CLIS path
The previous relative path (../../clis from __dirname) only worked for
dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname
is <repo>/src — resolving to /clis instead of <repo>/clis.
Use findPackageRoot() which works for both dev and prod paths.
* fix(build-manifest): import compiled JS from dist/clis/ instead of raw TS
Node's type stripping does not rewrite '.js' → '.ts' in import
specifiers, so dynamically importing .ts source files fails whenever
they contain relative imports like './utils.js'.
Switch to scanning dist/clis/ for compiled .js files after tsc runs.
This eliminates all 268 "Cannot find module" warnings and increases
manifest entries from 254 to 532 (previously half were silently skipped).
* fix: write manifest to dist/cli-manifest.json where runtime expects it
The runtime resolves BUILTIN_CLIS to dist/clis/ (relative to
dist/src/main.js), so discoverClis() looks for manifest at
dist/cli-manifest.json. Previously it was written to the package root
where the runtime never found it — manifest was effectively unused,
always falling through to filesystem scanning.
* refactor(errors): unify error output as YAML envelope to stderr
Replace the 100+ line chalk renderError() switch-case with a single
YAML envelope output path. All errors now output a structured
{ok, error: {code, message, help, exitCode}} envelope to stderr,
regardless of TTY status.
This simplifies the error system from 5 mechanisms to 3:
1. Error Envelope (YAML → stderr) — unified error output
2. Exit codes (sysexits.h) — process exit semantics
3. Diagnostic (OPENCLI_DIAGNOSTIC=1) — autofix repair context
Removed: chalk error rendering, ERROR_ICONS map, classifyGenericError
regex classifier, BrowserConnectError-specific bridge status display.
Added: toEnvelope() utility, ErrorEnvelope type.
* refactor(errors): migrate adapters to throw CliError, update docs
- Migrate xueqiu adapters from return [{error,help}] to throw CliError
- xueqiu/utils.ts: fetchXueqiuJson now throws AuthRequiredError/
CommandExecutionError instead of returning {error, help} objects
- Remove resolveColumns error fallback from output.ts (no longer needed)
- Add verbose stack trace support to error envelope
- Add ADAPTER_LOAD to AutoFix hint trigger codes
- Update skill docs (adapter-templates, explorer, oneshot, advanced-patterns)
to recommend throw CliError pattern instead of return [{error, help}]
* fix: remove remaining dead error-forwarding in 4 xueqiu adapters + review fixes
- Remove `if ('error' in d) return [d]` from feed, hot, search, kline
(fetchXueqiuJson now throws, so these were dead code)
- Add `stack?: string` to ErrorEnvelope interface (removes type cast hack)
- Fix adapter-templates.md: use AuthRequiredError instead of plain Error
* fix: migrate barchart/quote and yahoo-finance/quote to throw CliError
Last two adapters that silently returned [] on error instead of
throwing CommandExecutionError.
* fix: self-review fixes — doc evaluate crash, error messages, kline consistency
- adapter-templates.md: getServerContext was throwing AuthRequiredError
inside a function serialized into page.evaluate() (browser has no
CliError). Reverted to return {error} sentinel + func() body throw.
- yahoo-finance/quote, barchart/quote: include symbol in fallback error msg
- xueqiu/kline: throw EmptyResultError instead of returning [] for
consistency with other xueqiu adapters
* docs(skills): add Tier 2.5 localStorage Bearer, SPA discovery, and test standards
From real-world experience building slock.ai CLI adapters:
- oneshot: add network-empty diagnosis, SPA baseURL bundle search, Tier 2.5
localStorage Bearer template (with multi-tenant X-Server-Id pattern),
updated auth quick-reference, file path note, opencli browser verify test flow
- explorer: add Tier 2.5 to decision tree and strategy table, update test section
with opencli browser verify + Done standard, fix Step 5 path to ~/.opencli/clis/,
add 4 new pitfall rows (SPA HTML, 400 context header, empty network, wrong dir)
* docs(skills): fix path conflict + add anti-change patterns from real adapters
Fix reviewer blocking issue:
- Remove the contradictory "~/.opencli/clis/" note that mixed user-local and
repo-contributor workflows; replace with explicit two-scenario callout in
Step 4, Step 5, pitfall table, and oneshot test section
- Template comments in oneshot restored to clis/<site>/<name>.ts (repo path)
Add "抗变更模式" section to explorer, based on opencli's own production code:
- Pattern 1: dynamic queryId discovery (twitter/shared.ts resolveTwitterQueryId)
— scan loaded JS bundle by operationName (stable) to find queryId (unstable)
- Pattern 2: semantic DOM priority fallback (web/read.ts)
— article > [role=main] > main > class-hint > body, pick largest text block
- Pattern 3: ordered selector array + timestamp comments (xiaohongshu/publish.ts)
— first-match wins, comment records UI version and observed attribute values
- Pattern 4: nullish-coalescing field multi-path (xiaohongshu/user-helpers.ts)
— covers camelCase/snake_case variants without assuming fixed key name
* docs(explorer): split SKILL.md into reference sub-documents
- Shrink main SKILL.md from 994 to 270 lines — core workflow only
- Extract all TS templates (Tier 1~4, pagination) to references/adapter-templates.md
- Add error handling standard: { error, remedy } pattern (remedy > hint)
- Add Tier 2.5 localStorage Bearer template with multi-tenant X-Server-Id example
- Extract cascading requests, tap debug, verbose mode, anti-change patterns to references/advanced-patterns.md
- Extract record workflow to references/record-workflow.md
* docs(skills): fix verify command — split by dev scenario
browser verify only reads ~/.opencli/clis/, not repo's clis/.
Split all verify instructions:
- Repo 贡献: npm run build + opencli <site> <cmd>
- 私人 adapter: opencli browser verify <site>/<name>
Fixes blocker in explorer:L209, L224 and oneshot:L286, L298
* docs(adapter-templates): add utils.ts extraction pattern for same-site adapters
* docs(skills): add decision matrix, stop conditions, sync comments
explorer: add path decision matrix before core workflow
oneshot: add explicit stop/switch conditions (when to escalate to explorer)
both: add keep-in-sync comment on the two-scenario verify block
* feat(slock): extract utils.ts + apply { error, help } pattern; docs: remedy→help
slock/utils.ts: new — getSlockContext(), resolveChannelId()
- Shared token + workspace resolution, no more 4-line duplication
- UUID regex (/^[0-9a-f]{8}-...$/) replaces fragile !includes('-')
- Returns { error, help } instead of throwing
tasks.ts / members.ts / send.ts:
- Import from utils.ts, remove all duplicated auth boilerplate
- All errors return [{ error, help }], no more throw
- members.ts: add limit arg (was unbounded before)
docs: rename remedy → help across all skill references
* refactor(adapters): migrate pipeline adapters to func() with { error, help } pattern
- slock: agents, channels, messages, servers now use getSlockContext/resolveChannelId
from utils.ts; error handling uses { error, help } return instead of bare throws
- linux-do: export fetchLinuxDoJson from feed.ts; migrate search, topic, categories,
tags, user-posts, user-topics from pipeline+throw to func() using fetchLinuxDoJson
- xueqiu: add utils.ts with fetchXueqiuJson helper; migrate hot, feed, search, stock,
watchlist, hot-stock, groups, kline, earnings-date from pipeline+throw to func()
* fix(output): show error rows in table/csv/markdown when columns declared
When a command declares columns (e.g. ['rank', 'title', 'value']) but
returns an error row ({ error, help }), the declared columns would
render empty cells. Now resolveColumns detects the error key and falls
back to the row's actual keys, making diagnostics visible in all output
formats.
* chore: remove slock adapters from this PR
Slock adapters should be in a separate PR, not bundled with the
adapter refactor and skill docs improvements.
* feat: auto-close adapter windows, add OPENCLI_WINDOW_FOCUSED, document config
1. Adapter commands now close the automation window immediately after
completion instead of waiting for the 30s idle timeout.
2. OPENCLI_WINDOW_FOCUSED=1 opens automation windows in the foreground
(useful for debugging). Default remains background.
3. Add Configuration section to README (EN/ZH) and opencli-usage skill
listing all stable user-facing environment variables.
* Fix OPENCLI_WINDOW_FOCUSED to be per-request, not frozen at daemon startup
Move env var read from daemon (startup-time constant) to CLI side
(sendCommandRaw), so it works correctly with the persistent daemon model.
Each request now reads the env var fresh and includes windowFocused in
the command payload.
* refactor: make daemon persistent, remove idle timeout
- Remove IdleManager and 4-hour idle auto-exit
- Daemon now stays alive until explicit shutdown or uninstall
- Add preuninstall hook for best-effort daemon cleanup on npm uninstall
- Update docs to reflect persistent daemon model
* fix: remove stale idle timeout references from code and docs
* refactor: remove daemon status/restart commands and lastCliRequestTime
- Remove `daemon status` and `daemon restart` CLI commands (doctor covers diagnostics)
- Remove `lastCliRequestTime` tracking (no longer needed without idle timeout)
- Keep only `daemon stop` as the explicit shutdown command
* Add AbortSignal.timeout(3s) to preuninstall shutdown fetch
Prevents npm uninstall from hanging if the daemon port accepts
connections but never responds.
* refactor: unify browser error classification and deduplicate retry logic
Replace two overlapping error classification systems with a single
classifyBrowserError() that returns retry advice (retryable + delayMs):
- Extension/daemon transient errors → retryable, 1500ms delay
- CDP target navigation errors → retryable, 200ms delay
- Non-transient errors → not retryable
Deduplicate sendCommand/sendCommandFull retry loop into sendCommandRaw,
making both public functions thin return-value wrappers.
* fix: add error kind to prevent page-level retry of extension errors
classifyBrowserError() now returns a `kind` field:
- extension-transient: retried by daemon-client only
- target-navigation: retried by page-level settle logic
- non-retryable: no retry
Page.goto() and Page.evaluate() now only settle-retry on
target-navigation, preventing extension/daemon errors from being
silently swallowed as settle noise.
Use Chrome CDP targetId (UUID) as the canonical page identity across
all layers (extension → daemon → CLI), demoting tabId to an
extension-internal routing detail.
- Add extension/src/identity.ts: bidirectional targetId ↔ tabId mapping
with lazy refresh via chrome.debugger.getTargets()
- Update protocol: Command.page and Result.page carry targetId
- Update background.ts: resolveCommandTabId() and pageScopedResult()
helpers; all page-scoped handlers return targetId
- Add sendCommandFull() to daemon-client for responses with page identity
- Update Page class: _page stores targetId, goto/selectTab extract it
- Update record.ts: injectedPages tracks by targetId
- Add extension tests to vitest config and CI test scripts
* perf: fast-path completion, version, and shell scripts to bypass full discovery
Lightweight commands (--get-completions, --version, completion <shell>) now
resolve before any heavy module loading. Key changes:
- New completion-fast.ts: manifest-based completion + shell script generators
with zero dependency on registry/discovery/cli modules
- main.ts: static imports replaced with dynamic import() for the full startup
path so the fast path never pays the cost of loading discovery, registry,
Commander, hooks, etc.
- USER_CLIS_DIR inlined to avoid importing the entire discovery module
- completion.ts: removed manifest functions (moved to completion-fast.ts),
now only used as fallback when manifest is unavailable
* fix: address review blockers from codex-mini0
1. --version fast path: only match when argv[0] is --version/-V,
not anywhere in argv. Prevents intercepting `opencli gh --version`
which should pass through to the subcommand.
2. Completion fast path: require ALL manifests to exist (hasAllManifests),
not just one. If user clis dir exists but has no manifest, fall back
to full discovery so user adapters aren't silently dropped.
If user clis dir doesn't exist at all, skip its manifest requirement
since there are no user adapters to miss.
* refactor(skills): merge opencli-generate into opencli-explorer
opencli-generate was a thin wrapper over generateVerifiedFromUrl,
essentially an internal pipeline orchestration. Merge its entry point
into opencli-explorer as the automated fast path, keeping one unified
skill for adapter creation.
- Delete skills/opencli-generate/SKILL.md
- Add automated generation tip to opencli-explorer SKILL.md
- Update README/README.zh-CN skill references
- Update skill-generate.ts comment
* fix(docs): fix dead link in yaml-adapter deprecation page
Change ../../CONTRIBUTING.md to ./contributing (VitePress internal link).
* refactor: remove version field from GenerateOutcome and EarlyHint
All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.
Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).
* refactor: migrate all 123 CLI adapters from YAML to TypeScript
Remove YAML as an adapter format entirely. All adapters now use
TypeScript with cli() from @jackwener/opencli/registry.
- Convert 123 YAML adapter files to TypeScript via batch script
- Remove YAML scanning from discovery.ts (registerYamlCli, yaml import)
- Remove scanYaml() and shouldReplaceManifestEntry() from build-manifest.ts
- Change synthesize.ts to output JSON candidates (internal format)
- Change generate-verified.ts to write .ts adapter files instead of .yaml
- Delete yaml-schema.ts (dead code) and scripts/yaml-to-ts.mjs (one-time tool)
- Update all tests to match new format
Closes discussion in #OpenCLI thread 47ddba82.
* fix: close YAML migration gaps in plugin scaffold, validation, and scan
- plugin-scaffold.ts: generate hello.ts (TS pipeline) instead of hello.yaml
- plugin.ts validatePluginStructure: no longer accept .yaml as valid command file
- plugin.ts scanPluginCommands: remove .yaml/.yml from scanned extensions
- discovery.ts: add explicit log.warn() when YAML files detected in clis/ or plugins/
- plugin.test.ts: update all test fixtures from .yaml to .js
- plugin-scaffold.test.ts: update hello.yaml references to hello.ts
- Delete dead src/yaml-schema.ts
Resolves PR #887 review blockers from @mbp-codex-pr0.
* refactor: complete YAML removal across docs, skills, record, and binance adapters
Code changes:
- record.ts: candidate output changed from .yaml (yaml.dump) to .json (JSON.stringify), removed js-yaml import
- src/clis/binance: convert all 11 YAML adapters to TypeScript cli() format
- binance/commands.test.ts: rewrite to use registry instead of yaml.load
- skill-generate.test.ts, diagnostic.test.ts: update mock paths from .yaml to .ts
- build-manifest.ts, synthesize.ts: update stale YAML comments
Documentation:
- README.md: remove .yaml from Dynamic Loader, fix plugin types, fix synthesize comment
- README.zh-CN.md: fix synthesize comment
- CONTRIBUTING.md: replace YAML Adapter section with Pipeline Adapter (TS), update arg examples
- docs/developer/yaml-adapter.md: replaced with deprecation redirect
- docs/developer/architecture.md: remove YAML pipeline references
- docs/developer/contributing.md: remove YAML adapter section
- docs/developer/ai-workflow.md: YAML → TS in synthesize description
- docs/guide/getting-started.md: remove .yaml from loader, update engine description
- docs/guide/plugins.md: remove YAML plugin option, update plugin types
- docs/index.md, docs/comparison.md: remove YAML adapter references
- docs/zh/guide/plugins.md: remove .yaml from scan description
Skills:
- opencli-explorer/SKILL.md: rewrite YAML vs TS decision tree to TS-only
- opencli-oneshot/SKILL.md: replace YAML templates with TS cli() templates
- opencli-generate/SKILL.md: YAML artifact path → TS artifact path
- opencli-usage/SKILL.md, plugins.md: update adapter format references
* fix: clean up remaining YAML adapter references in docs
- docs/zh/guide/plugins.md: replace YAML plugin example with TS pipeline
- docs/developer/testing.md: YAML Adapter heading → Adapter, remove validate line
- TESTING.md: same fix in root testing doc
- CONTRIBUTING.md: remove "YAML validation" comment
- docs/.vitepress/config.mts: mark YAML Adapter Guide as (Deprecated) in nav
- docs/advanced/download.md: remove "YAML Adapters" from pipeline step heading
All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.
Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).
* fix: use Strategy.PUBLIC enum in skill-generate test to fix typecheck regression
* feat: add P2 EarlyHint callback channel to generateVerifiedFromUrl
Add optional onEarlyHint callback for internal cost gating before verify stage.
- EarlyHint type: version, stage, continue, reason, confidence, candidate?
- 3 emit points: explore (viable/not), synthesize (candidate/not), cascade (auth/ok)
- candidate only on synthesize/cascade + continue:true (not on stop or explore)
- unsupported-required-args goes directly to P1 terminal, no P2 hint emitted
- 6 new tests covering all hint paths + guardrails
* docs: add opencli-generate skill spec (SKILL.md)
Captures A+B consensus from team discussion:
- Input: url + goal? (natural language intent hint)
- Output: SkillOutput with machine-readable fields + human message
- Decision tree: thin mapping from GenerateOutcome
- Guardrails: no re-orchestration, no auto-escalation, no new taxonomy
- P1/P2 boundary: P1 is single source of truth, P2 transparent to skill
* fix: address review nits on skill spec
- Make path explicitly optional in needs-human-check decision tree
- Add missing non-array-result message template
* feat: add GenerateOutcome → SkillOutput thin wrapper
Implements the skill mapping layer per opencli-generate SKILL.md:
- mapOutcomeToSkillOutput: thin translation from P1 contract to agent-facing output
- executeGenerateSkill: entry point accepting SkillInput (url + goal?)
- Message templates for all StopReason and EscalationReason values
- 8 tests covering all outcome paths and contract shape validation
* fix: prefer outcome.message for richer context in needs-human-check
When GenerateOutcome has a message (e.g. "required args: id"), use it
instead of the generic template, so the specific args info reaches the user.
- Add getDaemonHealth() returning 'stopped' | 'no-extension' | 'ready'
- Delete discover.ts (thin wrapper with no value)
- Bridge uses getDaemonHealth() + _pollUntilReady() (eliminates duplicate polling)
- Doctor simplified: live check auto-starts daemon; no-live mode does minimal
auto-start only when stopped (avoids misreporting idle-exit as failure)
- CommanderAdapter preserves error message/hint detail (not just generic title)
- All callers use single unified status entry point
* feat: add GitHub Trending, Binance, and Weather (Open Meteo) adapters
GitHub Trending (2 commands, browser mode):
- repos: trending repositories with stars, forks, language filter
- developers: trending developers with popular repos
Supports --since daily/weekly/monthly and --language filter
Binance (11 commands, public API via data-api.binance.vision):
- top: top trading pairs by 24h volume
- price: single pair 24h price stats
- prices: latest prices for all pairs
- ticker: 24h ticker statistics
- gainers: top gaining pairs by 24h change
- losers: top losing pairs by 24h change
- trades: recent trades for a pair
- depth: order book bid prices
- asks: order book ask prices
- klines: candlestick/kline data
- pairs: list active trading pairs
Weather / Open Meteo (11 commands, free public API, no key needed):
- current: current weather for a city
- forecast: daily forecast up to 16 days
- hourly: hourly forecast
- search: city geocoding lookup
- air: air quality index (simple)
- air-quality: detailed air quality (US/EU AQI, PM2.5, PM10, ozone, NO2, SO2)
- sunrise: sunrise/sunset times with UV index
- wind: detailed wind forecast with gusts and 80m altitude
- precipitation: rain/snow forecast with probability
- history: historical weather up to 92 past days
- compare: side-by-side weather comparison across cities
All 24 commands tested with live data. 258 existing tests pass.
* docs: add missing douban, sinablog, substack adapter documentation
* fix(binance): sort numeric metrics and filter active pairs
* chore: drop non-binance adapters from pr214
* chore: drop binance docs from pr214
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: add Self-Repair protocol for automatic adapter fixing
When an AI agent uses opencli and a command fails, the agent
automatically diagnoses the failure, fixes the adapter, and retries.
- Add CLAUDE.md with Self-Repair protocol (auto-loaded by Claude Code)
- Add designs/self-repair-protocol.md documenting the approach
- Update opencli-repair skill: add Safety Boundaries (AUTH/BROWSER → STOP,
sourcePath-only scope, max 3 rounds), fix AUTH_REQUIRED guidance
- Update opencli-usage skill: add Self-Repair section
Key design decisions:
- Repair target is always RepairContext.adapter.sourcePath (works for both
repo-local clis/ and user-local ~/.opencli/clis/)
- Only adapter files may be modified, never core src/
- Max 3 repair rounds per failure
- AUTH_REQUIRED and BROWSER_CONNECT are hard stops (report, don't modify)
* fix: align auth boundary and scope language across all documents
- Remove "Auth changed (AUTH_REQUIRED)" exploration section from
opencli-repair skill — contradicted the hard stop rule above it
- Update design doc: scope language matches repo-local + explicit skill
delivery model, not universal product behavior
- Update usage skill: reference sourcePath instead of "files under clis/"
* fix: replace remaining repo-relative clis/ paths with sourcePath in design doc
* refactor: rename opencli-repair to opencli-autofix, remove CLAUDE.md
CLAUDE.md was wrong — users don't work inside the opencli repo, and
the protocol shouldn't assume Claude Code. The skill is the portable
delivery mechanism for any AI agent.
- Rename skills/opencli-repair → skills/opencli-autofix
- Remove CLAUDE.md (not the right delivery mechanism)
- Update all references in usage skill and design doc
- Design doc rewritten to reflect skill-first approach
* fix: use sourcePath in example repair session
* feat: emit AutoFix hint on repairable adapter errors
When a command fails with a repairable error (SELECTOR, EMPTY_RESULT,
COMMAND_EXEC, or generic http/not-found), the error output now includes
a hint telling agents to re-run with OPENCLI_DIAGNOSTIC=1 for repair
context. This is the trigger mechanism that bridges the gap between
"command failed" and "agent enters autofix loop".
Non-repairable errors (AUTH_REQUIRED, BROWSER_CONNECT, ARGUMENT) do not
emit the hint — these require user action, not adapter fixes.
* fix: narrow AutoFix hint to adapter-drift errors only
Remove hint from CommandExecutionError (covers env/launcher/runtime
issues, not adapter drift) and generic http errors (often temporary
site issues). Keep hint only for SelectorError, EmptyResultError,
and generic not-found — clear adapter-drift signals.
When the Browser Bridge extension is older than the CLI, sending
'network-capture-start' to the daemon returns 'Unknown action',
causing explore and operate-open to crash with an unhandled error.
Wrap startNetworkCapture calls with .catch() so they degrade
gracefully — explore continues without network capture data, and
operate-open falls back to the JS interceptor injection.
- engine.ts: replace `git add -A` with scope-aware `execFileSync` to
stage only files matching config.scope globs, and guard against empty
scope degenerating into staging all files
- fix.ts: pass prompt via stdin `input` option instead of shell string
interpolation to prevent $, backtick, and other metacharacter expansion
- generate.ts: update stale comment that claimed unimplemented pipeline
steps (register, verify, Strategy Cascade)
* refactor: remove scoring heuristic, replace with noise filter + metadata
The scoring mechanism was a pre-LLM heuristic that compressed rich endpoint
metadata into a single number. Since this project is designed for AI Agents,
the agent can reason about structured metadata directly.
Changes:
- Remove scoreEndpoint/scoreRequest/scoreWriteRequest and all score fields
- Replace with isNoiseUrl() filter (tracking/beacon/pixel) + isUsefulEndpoint()
- Remove artificial confidence percentages (was score/20)
- Sort by itemCount (transparent, observable) instead of weighted score
- Endpoints now expose full structured metadata for agent consumption
- Net reduction: -43 lines
* fix: widen endpoint filter to keep single-object JSON and stats/metric URLs
- Remove stats/metric from noise pattern — these are often business APIs
- Relax isUsefulEndpoint to keep any JSON endpoint, not just arrays
(preserves /me, /profile, /detail and other single-object APIs)
* fix: add deterministic endpoint ordering for generate/synthesize path
The AI agent path doesn't need ranking, but generate/synthesize still
pick candidates[0] as default — this needs a stable, explainable order.
- Add endpointSortKey() with transparent observable signals: array items,
detected fields, API path patterns, query params
- Update synthesize chooseEndpoint fallback to use itemCount + field count
- Sort key is internal only; not exposed as score to external consumers
* refactor: extract shared scoring logic and consolidate time format utils
- Extract applyUrlScoreAdjustments() and scoreArrayResponse() to analysis.ts,
eliminating duplicated endpoint scoring between explore.ts and record.ts
- Consolidate formatDuration/formatUptime into a single formatDuration(ms)
in download/progress.ts, reused by commands/daemon.ts
* fix: preserve explore scoring semantics and round daemon uptime
- Revert explore.ts scoreEndpoint to original inline /api/ /x/ bonus
without record's tracking/analytics penalty (blocker from review)
- Math.round uptime*1000 to avoid floating-point noise in daemon status
* feat(operate): unify network capture + implement CDP consoleMessages
- operate open: start session capture before navigation (catches initial requests)
- operate network: prefer readNetworkCapture() over JS interceptor
- CDPPage: implement consoleMessages() via Runtime.consoleAPICalled
Part of #810
* fix(operate): use correct daemon/CDP entry field names for network capture
Daemon and CDP capture entries use responseStatus/responseContentType/
responsePreview (not status/contentType/responseBody). Fix the
normalization in operate network to match the actual entry shape from
extension/src/cdp.ts.
* fix(cdp): capture Runtime.exceptionThrown in consoleMessages
- Register Runtime.exceptionThrown handler to capture uncaught exceptions
as error-level messages (most valuable diagnostic signal)
- 'error' filter now returns both console.error() and warning/exception
entries, matching typical severity-based logging semantics
* feat(cdp): implement session-level network capture for CDPPage
Implements startNetworkCapture() and readNetworkCapture() on CDPPage using
CDP Network domain events. Updates explore.ts to prefer session capture
over Performance API networkRequests().
Closes part of #810
* fix(cdp): use Network.loadingFinished for reliable body capture
- Move getResponseBody call from responseReceived to loadingFinished,
matching the extension's implementation pattern
- Use extension-compatible entry shape (responseStatus, responseContentType,
responsePreview) instead of custom field names
- Remove unreliable 100ms sleep hack in readNetworkCapture()
- Align with extension/src/cdp.ts:419-437 for consistency
* fix(cdp): drain buffer on readNetworkCapture to match daemon contract
readNetworkCapture() must clear the buffer after reading, matching the
daemon Page's read-and-drain behavior. Without this, repeated reads
would return stale entries.
* fix(cdp): await in-flight body fetches before returning from readNetworkCapture
Track all pending getResponseBody promises and await them in
readNetworkCapture() before draining the buffer. This ensures
explore/diagnostic consumers always get entries with responsePreview
populated, not empty shells where the body fetch hasn't resolved yet.
* fix(explore): handle both legacy and capture entry field names
parseNetworkRequests now maps both shapes:
- Legacy: status, contentType, responseBody
- Capture (extension/CDP): responseStatus, responseContentType, responsePreview
Also clears _pendingBodyFetches on startNetworkCapture reset.
* fix: add safety boundaries to diagnostic output
- Redact sensitive headers (Authorization, Cookie, etc.) from network requests
- Redact sensitive URL query parameters (token, key, secret, etc.)
- Cap individual fields: snapshot (100K chars), adapter source (50K chars),
network requests (50 entries, 4K body each), stack trace (5K chars)
- Enforce 256KB total output budget with graceful degradation:
drops snapshot first, then page state entirely
- Export truncate/redactUrl helpers for testing
* fix: add free-text redaction for all diagnostic string channels
Addresses review feedback: snapshot, consoleErrors, error message/hint/stack
could contain inline secrets (Bearer tokens, JWTs, cookie values, token=value
patterns). All string channels now pass through redactText() before emission.
- Add redactText() with patterns for Bearer tokens, JWTs, cookie values,
and inline key=value secrets
- Apply redactText to: error.message, error.hint, error.stack,
page.snapshot, page.consoleErrors
- Add 6 new test cases for redactText and error message redaction
* fix: resolve adapter source path and add page state collection timeout
Fixes#808 items 1 and 3:
1. adapter.source was missing for all command types because buildRepairContext
only checked cmd._modulePath (set only for manifest lazy-loaded TS).
Now resolveAdapterSourcePath() checks cmd.source first, skips manifest:
pseudo-paths, and maps dist/clis/*.js back to source clis/*.ts.
3. collectPageState() had no timeout — a hung CDP connection would block
error propagation indefinitely. Now wrapped with 5s Promise.race timeout,
falling back to emitting diagnostic without page state.
* fix: track sourceFile in manifest for YAML adapter source resolution
YAML commands inlined in the manifest previously lost their original file
path, causing resolveAdapterSourcePath() to return undefined. Add
sourceFile field to ManifestEntry so discovery can reconstruct the
editable source path for both YAML and TS commands.
* feat: add structured diagnostic output for AI-driven adapter repair
When OPENCLI_DIAGNOSTIC=1 is set, failed commands emit a RepairContext
JSON to stderr containing the error, adapter source, and browser state
(DOM snapshot, network requests, console errors). AI Agents consume
this to diagnose and fix adapters when websites change.
Also adds the opencli-repair skill guide for AI Agents.
* fix: correct e2e test binary path to dist/src/main.js
The e2e helpers pointed to dist/main.js but the actual build output
is at dist/src/main.js (matching package.json "main" field). This
caused all e2e-headed tests to fail with "Cannot find module".
* fix: correct dist/main.js path in autoresearch scripts
* fix: emit diagnostic for pre-session browser failures
When browser connection fails before the session callback runs
(e.g., BrowserConnectError), the inner diagnostic catch never fires.
Use a flag to ensure the outer catch emits diagnostic as a fallback.
* test: tolerate unavailable Bloomberg RSS feeds in e2e
* test: skip flaky bloomberg businessweek e2e test
The Bloomberg Businessweek RSS feed is intermittently unavailable,
causing CI failures unrelated to code changes.
* revert: restore bloomberg businessweek e2e coverage
jakevin