mirror of
https://github.com/jackwener/OpenCLI.git
synced 2026-09-14 18:25:42 +08:00
main
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jakevin
|
55088bbb28 |
feat: 13 read adapters across 5 new sites + 4 extensions (round 3) (#1346)
New sites (8 commands):
- npm : search / package / downloads (registry.npmjs.org + api.npmjs.org)
- pypi : package / downloads (pypi.org + pypistats.org)
- crates : search / crate (crates.io)
- mdn : search (developer.mozilla.org)
- nvd : cve (services.nvd.nist.gov)
Extensions (5 commands; +1 dblp/author surfaced in index):
- hf : spaces (Hugging Face Spaces by likes / created_at / last_modified)
- dblp : venue (search dblp's venue registry by acronym/topic)
- coingecko : derivatives (perpetual / futures markets, 24h volume)
- stackoverflow : related (related questions for a given question id)
All commands hit public unauthenticated endpoints (Strategy.PUBLIC, browser:false),
typed-fail-fast on bad inputs (no silent fallback / clamp), and round-trip listing
ids into their detail commands where applicable.
Audits (all green vs baseline):
- typed-error-lint : 196 = 196 baseline, no new
- silent-column-drop : 103 = 103 baseline, no new
- listing-id-pairing : 13 advisory (was 12; +1 = dblp/venue with no
corresponding venue-detail command)
Doc coverage : 120/120 adapter dirs documented (+5 new doc pages, +4 updated)
Manifest : 722 entries (was 709; +13 commands)
Live verified:
- npm search react / npm package react / npm downloads react --period last-week
- npm downloads react --period 2025-01-01:2025-01-05
- pypi package requests / pypi downloads requests --period recent / overall
- crates search tokio / crates crate serde
- mdn search fetch
- nvd cve CVE-2021-44228
- hf spaces --limit 3
- dblp venue ICLR
- coingecko derivatives --limit 3
- stackoverflow related 79935770 --limit 3
- typed-error sanity: invalid CVE id, bad npm name, bad --period
|
||
|
jakevin
|
6f597a2a4b |
feat: 8 read adapters across 5 sites (arxiv / SO / coingecko / wikipedia / hf) (#1338)
* feat: add 13 read adapters across 6 sites (github / arxiv / SO / coingecko / wikipedia / hf)
New site:
- github: user, repo, search-repos, user-repos, releases (unauth REST API; 60 req/h IP limit)
Existing sites — gap-fill for high-traffic verticals:
- arxiv author (papers by author, newest first; au:"name" phrase match on the public Atom API)
- stackoverflow user / tag (Stack Exchange API 2.3, with HTML-entity decode for display names / titles)
- coingecko coin / trending (single-coin market detail; 24h trending search-volume)
- wikipedia page (full plain-text article extract; opt-in --paragraphs cap, no silent truncation)
- hf models / datasets (downloads/likes/trending/freshness sorted lists)
All adapters use Node-side func + typed errors per the post-#1332 convention:
- ArgumentError for invalid limit / bad enum / empty positional / malformed owner-repo
- EmptyResultError for genuinely-empty results (no silent return [])
- CommandExecutionError for upstream HTTP/JSON failures (rate limit / 5xx / parse)
- AuthRequiredError reserved for endpoints that genuinely refuse anonymous traffic
- No silent clamp on --limit; no sentinel rows; no scalar 'unknown' / '-' fallbacks
Audit gates locally green:
- check:typed-error-lint 196/196 (no new)
- check:silent-column-drop 103/103 (no new)
- check:doc-coverage --strict 113/113 (added github.md, extended 5 existing pages)
- advise:listing-id-pairing advisory only (+2 wikipedia entries: title is the
round-trippable key into wikipedia/page; not a gate)
* chore: drop github adapter set per WAWQAQ directive
WAWQAQ (#opencli-pr-review): "我们不需要GitHub的adapter，因为已经有GH了"
Removes the 5 github commands + utils + docs added in 664ed1aa
(github/user, github/repo, github/search-repos, github/releases,
github/user-repos). The remaining 8 read commands across 5 sites
(arxiv author, stackoverflow user/tag, coingecko coin/trending,
wikipedia page, hf models/datasets) are unaffected.
Audit gates re-checked:
- check:typed-error-lint: 196/196 (baseline unchanged)
- check:silent-column-drop: 103/103 (baseline unchanged)
- doc-coverage: 112/112 (one less site documented)
- advise:listing-id-pairing: 12 advisory (unchanged)
* fix(adapter-expansion): tighten id and currency contracts
|
||
|
jakevin
|
eea9ff8bfe | feat(cli): add command access metadata (#1296) | ||
|
jakevin
|
c1a4bd3b7e |
feat(stackoverflow): surface question_id on listings + new read <id> (#1293)
* feat(stackoverflow): surface question_id + metadata on listings, add `read <id>`
Agent-native gap: all 4 stackoverflow listings (`hot`, `search`,
`unanswered`, `bounties`) only emitted `[title, score, answers, url]`,
which means an agent could see a hot question but had no `id` to round-
trip into a body read, no `tags` to filter by topic, no `views` to gauge
demand, and no `is_answered` / `creation_date` / `author` to triage.
There also wasn't a `read` adapter, so reading a SO question through
opencli was impossible.
Listings (`hot` / `search` / `bounties` / `unanswered`):
- Add `rank`, `id` (question_id), `views`, `is_answered` (skipped on
`unanswered` since always false), `tags` (joined), `author`
(owner.display_name), `creation_date` columns.
- Pass `pagesize` to the upstream API instead of fetching the default
page and trimming locally.
New `stackoverflow read <id>`:
- 4-call fan-out against the public Stack Exchange API
(`/questions/{id}` + `/questions/{id}/comments` +
`/questions/{id}/answers` + batched `/answers/a;b;c/comments`).
- Returns `POST` + `Q-COMMENT` + `ANSWER` + `A-COMMENT` rows mirroring
the `hackernews read` and `lobsters read` shape.
- Accepted answer is always surfaced first and tagged `accepted='true'`;
remaining answers follow in descending vote order, capped by
`--answers-limit`.
- HTML body cleanup: tags stripped, `<pre><code>` preserved, `<code>`
inline-fenced, `<li>` rendered as `- `, comments indented with `> `.
- Entity decoding: a shared `decodeEntities` handles named (incl.
`…`/`©`/etc), decimal (`ö`), and hex (`'`)
forms, applied to both bodies AND `display_name` (otherwise users
like `Jonas Kölker` come through mojibaked).
- Typed fail-fast: `ArgumentError` for non-numeric id and
`--max-length < 100` (with no-fetch assertion); `EmptyResultError`
when `items` is empty; `CommandExecutionError` for HTTP non-2xx and
for Stack Exchange's in-band `error_id` envelopes (throttle / quota).
No silent clamps anywhere.
Tests: 14 vitest assertions
- 4 listing column-shape (incl. `unanswered` skipping `is_answered` and
`bounties` keeping its `bounty` column position)
- 10 read-adapter cases: registration / args / strategy + 3 typed-error
fail-fast paths (with no-fetch assertion on the pre-fetch ones) + the
full POST/Q-COMMENT/ANSWER/A-COMMENT row order with accepted-first +
the answer-comments fetch verified to batch ids semicolon-joined +
HTML entity decoding (named/decimal/hex) on both body and display_name
+ answers-limit honored when there are more answers than the cap.
Live verification:
- `stackoverflow hot --limit 2` → `id`/`tags`/`views`/`is_answered`/
`author` populated.
- `stackoverflow search "async await" --limit 1`,
`stackoverflow unanswered --limit 1` → same shape.
- `stackoverflow read 79935770` and the very-long classic question
`stackoverflow read 11227809 --answers-limit 1 --comments-limit 2`
→ produces the threaded POST/Q-COMMENT/ANSWER/A-COMMENT structure
with proper entity decoding (`Jonas Kölker` reads correctly).
- `stackoverflow read not-numeric` → exits with `ARGUMENT`.
- `stackoverflow read 999999999` → exits with `EMPTY_RESULT`.
* fix(stackoverflow): wrap fetch/json/coerce paths in typed errors
Apply the 3 lessons from PR #1292 (devto) review at merge time, before
B-group hits this PR:
1. CLI args may arrive as strings (e.g. `--max-length 50` → `'50'`).
The bare `Number.isInteger(value)` in `requirePositiveInt` /
`requireMinInt` would accept negative-but-coerced numbers and reject
string-form integers. Now the helpers `coerceInt` first then validate,
and the rejection message echoes the raw input via `JSON.stringify`.
2. `await fetch(url)` and `await res.json()` were not wrapped — a network
blip would surface as a raw `TypeError` and a maintenance HTML page
would surface as a raw `SyntaxError`. Both are now caught and rethrown
as `CommandExecutionError` with hints, matching the in-band error_id
path.
Tests: +3 cases (17 total)
- fetch network failure → CommandExecutionError
- malformed JSON body → CommandExecutionError
- string-form max-length "50" / "abc" rejected with ArgumentError before
fetching
* fix(stackoverflow): avoid partial read fanout
|
||
|
jakevin
|
d2974a9ff6 |
refactor(adapters): convert adapter layer from TypeScript to JavaScript (#928)
* refactor(adapters): convert adapter layer from TypeScript to JavaScript Core framework stays TypeScript; adapter layer moves to JS-first. Adapters are essentially "executable config + browser scripts" that barely use TS features — this simplifies the build/distribution pipeline by removing the dist/clis/ intermediate compilation step. Changes: - Convert all 753 adapter files in clis/ from .ts to .js - Update tsconfig to exclude clis/ from compilation - Simplify build-manifest to scan clis/*.js directly (no dist/clis/) - Update discovery, main, fetch-adapters to load JS adapters from clis/ - Update generate-verified to output .js artifacts - Update package.json files field: dist/clis/ → clis/ - Fix all test files for the .ts → .js transition * fix(main): use findPackageRoot for BUILTIN_CLIS path The previous relative path (../../clis from __dirname) only worked for dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname is <repo>/src — resolving to /clis instead of <repo>/clis. Use findPackageRoot() which works for both dev and prod paths. |
||
|
jakevin
|
70b1145b5e |
refactor: migrate all CLI adapters from YAML to TypeScript (#887)
* refactor: remove version field from GenerateOutcome and EarlyHint All consumers are in the same repo and evolve together — version field adds ceremony without practical value at this stage. Keeps schema_version in VerifiedArtifactMetadata (sidecar file format). * refactor: migrate all 123 CLI adapters from YAML to TypeScript Remove YAML as an adapter format entirely. All adapters now use TypeScript with cli() from @jackwener/opencli/registry. - Convert 123 YAML adapter files to TypeScript via batch script - Remove YAML scanning from discovery.ts (registerYamlCli, yaml import) - Remove scanYaml() and shouldReplaceManifestEntry() from build-manifest.ts - Change synthesize.ts to output JSON candidates (internal format) - Change generate-verified.ts to write .ts adapter files instead of .yaml - Delete yaml-schema.ts (dead code) and scripts/yaml-to-ts.mjs (one-time tool) - Update all tests to match new format Closes discussion in #OpenCLI thread 47ddba82. * fix: close YAML migration gaps in plugin scaffold, validation, and scan - plugin-scaffold.ts: generate hello.ts (TS pipeline) instead of hello.yaml - plugin.ts validatePluginStructure: no longer accept .yaml as valid command file - plugin.ts scanPluginCommands: remove .yaml/.yml from scanned extensions - discovery.ts: add explicit log.warn() when YAML files detected in clis/ or plugins/ - plugin.test.ts: update all test fixtures from .yaml to .js - plugin-scaffold.test.ts: update hello.yaml references to hello.ts - Delete dead src/yaml-schema.ts Resolves PR #887 review blockers from @mbp-codex-pr0. * refactor: complete YAML removal across docs, skills, record, and binance adapters Code changes: - record.ts: candidate output changed from .yaml (yaml.dump) to .json (JSON.stringify), removed js-yaml import - src/clis/binance: convert all 11 YAML adapters to TypeScript cli() format - binance/commands.test.ts: rewrite to use registry instead of yaml.load - skill-generate.test.ts, diagnostic.test.ts: update mock paths from .yaml to .ts - build-manifest.ts, synthesize.ts: update stale YAML comments Documentation: - README.md: remove .yaml from Dynamic Loader, fix plugin types, fix synthesize comment - README.zh-CN.md: fix synthesize comment - CONTRIBUTING.md: replace YAML Adapter section with Pipeline Adapter (TS), update arg examples - docs/developer/yaml-adapter.md: replaced with deprecation redirect - docs/developer/architecture.md: remove YAML pipeline references - docs/developer/contributing.md: remove YAML adapter section - docs/developer/ai-workflow.md: YAML → TS in synthesize description - docs/guide/getting-started.md: remove .yaml from loader, update engine description - docs/guide/plugins.md: remove YAML plugin option, update plugin types - docs/index.md, docs/comparison.md: remove YAML adapter references - docs/zh/guide/plugins.md: remove .yaml from scan description Skills: - opencli-explorer/SKILL.md: rewrite YAML vs TS decision tree to TS-only - opencli-oneshot/SKILL.md: replace YAML templates with TS cli() templates - opencli-generate/SKILL.md: YAML artifact path → TS artifact path - opencli-usage/SKILL.md, plugins.md: update adapter format references * fix: clean up remaining YAML adapter references in docs - docs/zh/guide/plugins.md: replace YAML plugin example with TS pipeline - docs/developer/testing.md: YAML Adapter heading → Adapter, remove validate line - TESTING.md: same fix in root testing doc - CONTRIBUTING.md: remove "YAML validation" comment - docs/.vitepress/config.mts: mark YAML Adapter Guide as (Deprecated) in nav - docs/advanced/download.md: remove "YAML Adapters" from pipeline step heading |
||
|
jakevin
|
80eef46b4e |
refactor: monorepo adapter separation (clis/ at root) (#782)
* refactor: move adapters from src/clis/ to root clis/ for monorepo separation Separates CLI adapters from the core runtime to prepare for independent adapter distribution via postinstall fetch. Key changes: - Move src/clis/ → clis/ (adapters at repo root) - Change tsconfig rootDir from "src" to "." so tsc compiles both - Create root-level shim files (registry.ts, errors.ts, etc.) so adapter relative imports (../../registry.js) resolve correctly - Update build-manifest.ts, main.ts paths for new dist/src/ structure - Expand ensureUserCliCompatShims() to cover all adapter import targets (types, utils, logger, launcher, browser/*, download/*, pipeline/*) - Add scripts/fetch-adapters.js postinstall for ~/.opencli/clis/ sync - Update vitest.config.ts adapter test paths - Add package.json files field to exclude adapters from npm package Official adapter files are unconditionally overwritten on update; user-created files not in the manifest are preserved. * fix: add dist/clis/ and cli-manifest.json to npm files, harden fetch-adapters - Add dist/clis/ and dist/cli-manifest.json to package.json files field so built-in adapters and manifest ship with the npm package - Replace execSync with execFileSync to prevent command injection - Add version check to skip redundant adapter fetches - Track tmpRoot explicitly for reliable cleanup * fix: address review blockers — manifest-based updates, global-only fetch, first-run fallback 1. Manifest-based update strategy: - Read old manifest to identify previously-official files - Clean up files removed upstream (in old manifest but not new) - User-created files (never in any manifest) remain untouched 2. Only run fetch-adapters on global install (npm_config_global=true) or explicit OPENCLI_FETCH=1, preventing heavy side effects for local/dev installs 3. First-run fallback in discovery.ts: - ensureUserAdapters() checks for adapter-manifest.json - If missing and ~/.opencli/clis/ is empty, spawns fetch-adapters.js - Guarantees adapters are available even with --ignore-scripts * fix: remove OPENCLI_FETCH env var, use internal _OPENCLI_FIRST_RUN instead * feat: also support OPENCLI_FETCH=1 for explicit adapter fetch trigger * simplify: replace git clone with local copy from dist/clis/ Adapters already ship in the npm package (dist/clis/), so there's no need to clone from GitHub. Copy directly from the installed package: - Eliminates git, curl, tar dependencies - No network calls in postinstall - No timeout/offline issues - Version always matches the installed CLI - ~65 lines of clone/download code replaced by one cpSync loop |