Commit Graph

6 Commits

Author SHA1 Message Date
jakevin a78ceb1602 feat: 11 read adapters across 8 sites (round 2) (#1345)
* feat: 11 read adapters across 8 sites (dblp / steam / bbc / devto / lobsters / medium / coingecko / hf)

Round 2 of the adapter expansion sweep. All 11 commands hit public APIs (no
browser, no auth), follow the post-#1332 typed-error / no-silent-failure
discipline, and were live-verified against real endpoints.

New adapters:
- dblp/author      : recent publications for one author (resolve PID by name, or pass --pid)
- steam/search     : storefront name search (storesearch API)
- steam/app        : single app detail (appdetails API; HTML entities decoded)
- bbc/topic        : per-topic RSS (8 canonical BBC News feeds)
- devto/latest     : /api/articles/latest with --page pagination
- lobsters/domain  : stories from a specific source domain (/domains/<d>.json)
- medium/tag       : tag RSS (description full-length, no silent truncation)
- coingecko/exchanges  : trust score + 24h BTC volume leaderboard
- coingecko/categories : sector buckets with 6 sort options
- coingecko/global     : aggregate market totals + BTC/ETH dominance
- hf/paper         : single-paper detail by arXiv id (summary, ai_summary, ai_keywords, upvotes)

Also adds clis/steam/utils.js + clis/bbc/utils.js as shared helpers (HTML entity
decode, RSS parsing). All listings carry a round-trippable id where a detail
sibling exists; advise:listing-id-pairing reports zero new violations. typed-
error-lint and silent-column-drop gates both unchanged from baseline.

Manifest: 698 → 709 (+11 entries).

* fix: tighten adapter round2 contracts
2026-05-06 12:46:14 +08:00
jakevin eea9ff8bfe feat(cli): add command access metadata (#1296) 2026-05-04 19:47:08 +08:00
jakevin 68485cc54e feat(devto): surface article id on listings + new read <id> (#1292)
* feat(devto): surface article id + published_at on listings, add `read <id>`

Agent-native gap: devto listings (`top`/`tag`/`user`) didn't include the
article `id`, so an agent couldn't round-trip from a listing into a body
read. They also dropped `reading_time` and `published_at`, which are cheap
signals the API gives you for free.

Changes:
- `top` / `tag` / `user`: add `id`, `reading_time`, `published_at` columns
  alongside existing rank/title/etc. `user` keeps its no-author shape since
  it's already user-scoped.
- New `devto read <id>`: hits `dev.to/api/articles/<id>` and returns one
  row with the article body (truncated by `--max-length`, default 20000,
  min 100). DEV.to's public API does not expose comments yet, so this is
  intentionally a single-row reader rather than a HN/lobsters-style
  threaded tree — if/when comments become public we can extend to
  POST + L0/L1.
- Typed fail-fast: `ArgumentError` for non-numeric id and for `--max-length`
  below 100; `EmptyResultError` on 404; `CommandExecutionError` for other
  non-2xx HTTP statuses. No silent clamps.
- Defensive tag normalization: the `/api/articles/<id>` endpoint returns
  `tag_list` as a comma-string and `tags` as an array (the opposite shape
  from listing endpoints). Caught this on live verification — both shapes
  now collapse to a comma-joined string.

Tests: 12 vitest assertions covering listing column shape (all 3) +
register/args/strategy + typed-error fail-fast paths + happy-path body
extraction + truncation marker + alternate tag_list shape.

Live verification: `devto top --limit 3` and `devto read 3602287` both
return the expected agent-native shape.

* fix(devto): harden article read contract
2026-05-04 18:55:14 +08:00
jakevin d2974a9ff6 refactor(adapters): convert adapter layer from TypeScript to JavaScript (#928)
* refactor(adapters): convert adapter layer from TypeScript to JavaScript

Core framework stays TypeScript; adapter layer moves to JS-first.
Adapters are essentially "executable config + browser scripts" that
barely use TS features — this simplifies the build/distribution pipeline
by removing the dist/clis/ intermediate compilation step.

Changes:
- Convert all 753 adapter files in clis/ from .ts to .js
- Update tsconfig to exclude clis/ from compilation
- Simplify build-manifest to scan clis/*.js directly (no dist/clis/)
- Update discovery, main, fetch-adapters to load JS adapters from clis/
- Update generate-verified to output .js artifacts
- Update package.json files field: dist/clis/ → clis/
- Fix all test files for the .ts → .js transition

* fix(main): use findPackageRoot for BUILTIN_CLIS path

The previous relative path (../../clis from __dirname) only worked for
dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname
is <repo>/src — resolving to /clis instead of <repo>/clis.

Use findPackageRoot() which works for both dev and prod paths.
2026-04-10 14:52:18 +08:00
jakevin 70b1145b5e refactor: migrate all CLI adapters from YAML to TypeScript (#887)
* refactor: remove version field from GenerateOutcome and EarlyHint

All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.

Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).

* refactor: migrate all 123 CLI adapters from YAML to TypeScript

Remove YAML as an adapter format entirely. All adapters now use
TypeScript with cli() from @jackwener/opencli/registry.

- Convert 123 YAML adapter files to TypeScript via batch script
- Remove YAML scanning from discovery.ts (registerYamlCli, yaml import)
- Remove scanYaml() and shouldReplaceManifestEntry() from build-manifest.ts
- Change synthesize.ts to output JSON candidates (internal format)
- Change generate-verified.ts to write .ts adapter files instead of .yaml
- Delete yaml-schema.ts (dead code) and scripts/yaml-to-ts.mjs (one-time tool)
- Update all tests to match new format

Closes discussion in #OpenCLI thread 47ddba82.

* fix: close YAML migration gaps in plugin scaffold, validation, and scan

- plugin-scaffold.ts: generate hello.ts (TS pipeline) instead of hello.yaml
- plugin.ts validatePluginStructure: no longer accept .yaml as valid command file
- plugin.ts scanPluginCommands: remove .yaml/.yml from scanned extensions
- discovery.ts: add explicit log.warn() when YAML files detected in clis/ or plugins/
- plugin.test.ts: update all test fixtures from .yaml to .js
- plugin-scaffold.test.ts: update hello.yaml references to hello.ts
- Delete dead src/yaml-schema.ts

Resolves PR #887 review blockers from @mbp-codex-pr0.

* refactor: complete YAML removal across docs, skills, record, and binance adapters

Code changes:
- record.ts: candidate output changed from .yaml (yaml.dump) to .json (JSON.stringify), removed js-yaml import
- src/clis/binance: convert all 11 YAML adapters to TypeScript cli() format
- binance/commands.test.ts: rewrite to use registry instead of yaml.load
- skill-generate.test.ts, diagnostic.test.ts: update mock paths from .yaml to .ts
- build-manifest.ts, synthesize.ts: update stale YAML comments

Documentation:
- README.md: remove .yaml from Dynamic Loader, fix plugin types, fix synthesize comment
- README.zh-CN.md: fix synthesize comment
- CONTRIBUTING.md: replace YAML Adapter section with Pipeline Adapter (TS), update arg examples
- docs/developer/yaml-adapter.md: replaced with deprecation redirect
- docs/developer/architecture.md: remove YAML pipeline references
- docs/developer/contributing.md: remove YAML adapter section
- docs/developer/ai-workflow.md: YAML → TS in synthesize description
- docs/guide/getting-started.md: remove .yaml from loader, update engine description
- docs/guide/plugins.md: remove YAML plugin option, update plugin types
- docs/index.md, docs/comparison.md: remove YAML adapter references
- docs/zh/guide/plugins.md: remove .yaml from scan description

Skills:
- opencli-explorer/SKILL.md: rewrite YAML vs TS decision tree to TS-only
- opencli-oneshot/SKILL.md: replace YAML templates with TS cli() templates
- opencli-generate/SKILL.md: YAML artifact path → TS artifact path
- opencli-usage/SKILL.md, plugins.md: update adapter format references

* fix: clean up remaining YAML adapter references in docs

- docs/zh/guide/plugins.md: replace YAML plugin example with TS pipeline
- docs/developer/testing.md: YAML Adapter heading → Adapter, remove validate line
- TESTING.md: same fix in root testing doc
- CONTRIBUTING.md: remove "YAML validation" comment
- docs/.vitepress/config.mts: mark YAML Adapter Guide as (Deprecated) in nav
- docs/advanced/download.md: remove "YAML Adapters" from pipeline step heading
2026-04-08 23:01:08 +08:00
jakevin 80eef46b4e refactor: monorepo adapter separation (clis/ at root) (#782)
* refactor: move adapters from src/clis/ to root clis/ for monorepo separation

Separates CLI adapters from the core runtime to prepare for independent
adapter distribution via postinstall fetch.

Key changes:
- Move src/clis/ → clis/ (adapters at repo root)
- Change tsconfig rootDir from "src" to "." so tsc compiles both
- Create root-level shim files (registry.ts, errors.ts, etc.) so adapter
  relative imports (../../registry.js) resolve correctly
- Update build-manifest.ts, main.ts paths for new dist/src/ structure
- Expand ensureUserCliCompatShims() to cover all adapter import targets
  (types, utils, logger, launcher, browser/*, download/*, pipeline/*)
- Add scripts/fetch-adapters.js postinstall for ~/.opencli/clis/ sync
- Update vitest.config.ts adapter test paths
- Add package.json files field to exclude adapters from npm package

Official adapter files are unconditionally overwritten on update;
user-created files not in the manifest are preserved.

* fix: add dist/clis/ and cli-manifest.json to npm files, harden fetch-adapters

- Add dist/clis/ and dist/cli-manifest.json to package.json files field
  so built-in adapters and manifest ship with the npm package
- Replace execSync with execFileSync to prevent command injection
- Add version check to skip redundant adapter fetches
- Track tmpRoot explicitly for reliable cleanup

* fix: address review blockers — manifest-based updates, global-only fetch, first-run fallback

1. Manifest-based update strategy:
   - Read old manifest to identify previously-official files
   - Clean up files removed upstream (in old manifest but not new)
   - User-created files (never in any manifest) remain untouched

2. Only run fetch-adapters on global install (npm_config_global=true)
   or explicit OPENCLI_FETCH=1, preventing heavy side effects for
   local/dev installs

3. First-run fallback in discovery.ts:
   - ensureUserAdapters() checks for adapter-manifest.json
   - If missing and ~/.opencli/clis/ is empty, spawns fetch-adapters.js
   - Guarantees adapters are available even with --ignore-scripts

* fix: remove OPENCLI_FETCH env var, use internal _OPENCLI_FIRST_RUN instead

* feat: also support OPENCLI_FETCH=1 for explicit adapter fetch trigger

* simplify: replace git clone with local copy from dist/clis/

Adapters already ship in the npm package (dist/clis/), so there's no
need to clone from GitHub. Copy directly from the installed package:

- Eliminates git, curl, tar dependencies
- No network calls in postinstall
- No timeout/offline issues
- Version always matches the installed CLI
- ~65 lines of clone/download code replaced by one cpSync loop
2026-04-05 01:46:36 +08:00