Commit Graph

11 Commits

Author SHA1 Message Date
jakevin a5a3248a77 refactor(linux-do): remove deprecated hot/category/latest compat shims (#1368)
* refactor(linux-do): remove deprecated hot/category/latest compat shims

The three shims have been pure backward-compat wrappers since linux-do/feed
became the unified entrypoint. With no stable release commitment to preserve,
they are pure surface cost: 3 manifest entries, 3 deprecated branches in help
output, and a `buildLinuxDoCompatFooter` helper that exists only to feed them.

- delete clis/linux-do/{hot,category,latest}.js
- drop now-orphaned `buildLinuxDoCompatFooter` from feed.js and unexport
  `executeLinuxDoFeed` (no external consumers remain)
- remove the Compatibility section in docs/adapters/browser/linux-do.md
- regenerate cli-manifest.json (-125 lines)

BREAKING CHANGE: `opencli linux-do hot|category|latest` are removed. Use
`opencli linux-do feed --view top --period <period>`,
`opencli linux-do feed --category <id-or-name>`, and
`opencli linux-do feed --view latest` instead.

* fix(linux-do): finish compat shim removal
2026-05-06 23:57:15 +08:00
jakevin ac94b75879 feat(1688/hupu/douban/linux-do): surface dropped ids on listings (#1302)
Round 7 — silent-drop sweep. Continues the listing→detail id-pairing
work from #1297. Each row was already extracting these ids/urls
internally; only the `columns` projection was missing, so they showed
up in `-f json` but never on the table view.

| Adapter            | Added columns                       |
|--------------------|-------------------------------------|
| `1688 search`      | `item_url`, `member_id`             |
| `hupu mentions`    | `tid`, `pid`, `url`                 |
| `douban photos`    | `photo_id`, `subject_id`            |
| `linux-do tags`    | `slug`                              |

Round-trip wins:
- `1688 search` → `1688 item <item_url>` (item_url is the canonical
  detail.1688.com URL); `1688 search` → `1688 store <member_id>`
- `hupu mentions` → `hupu detail <tid>` (and `pid` for the deep link)
- `douban photos` → tied back to the parent movie via `subject_id`
- `linux-do tags` → `linux-do feed --tag <slug>` (slug is the URL form)

No logic change — only the column array. JSON output unchanged.
Tests: 45/45 pass for the four affected sites.
2026-05-04 21:22:23 +08:00
jakevin eea9ff8bfe feat(cli): add command access metadata (#1296) 2026-05-04 19:47:08 +08:00
jakevin cc18ed67b7 fix: sync package-lock.json to unblock CI (#937)
* fix: sync package-lock.json with package.json dependencies

package-lock.json was missing @emnapi/core@1.9.2 and
@emnapi/runtime@1.9.2 (transitive deps of @emnapi/wasi-threads),
causing `npm ci` to fail on all CI jobs.

* fix: resolve remaining CI failures after TS-to-JS adapter migration

- vitest.config.ts: update adapter project include/exclude from .test.ts
  to .test.{ts,js} to match converted adapter test files
- check-doc-coverage.sh: skip adapter directories containing only utility
  files (prefixed with _), fixing false positive for clis/slock/
- linux-do/topic-content.test.js: fix hardcoded reference to topic.ts
  (now topic.js after PR #928 migration)
2026-04-10 23:29:09 +08:00
jakevin d2974a9ff6 refactor(adapters): convert adapter layer from TypeScript to JavaScript (#928)
* refactor(adapters): convert adapter layer from TypeScript to JavaScript

Core framework stays TypeScript; adapter layer moves to JS-first.
Adapters are essentially "executable config + browser scripts" that
barely use TS features — this simplifies the build/distribution pipeline
by removing the dist/clis/ intermediate compilation step.

Changes:
- Convert all 753 adapter files in clis/ from .ts to .js
- Update tsconfig to exclude clis/ from compilation
- Simplify build-manifest to scan clis/*.js directly (no dist/clis/)
- Update discovery, main, fetch-adapters to load JS adapters from clis/
- Update generate-verified to output .js artifacts
- Update package.json files field: dist/clis/ → clis/
- Fix all test files for the .ts → .js transition

* fix(main): use findPackageRoot for BUILTIN_CLIS path

The previous relative path (../../clis from __dirname) only worked for
dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname
is <repo>/src — resolving to /clis instead of <repo>/clis.

Use findPackageRoot() which works for both dev and prod paths.
2026-04-10 14:52:18 +08:00
jakevin 309dadcf46 refactor(adapters): migrate pipeline adapters to func() + { error, help } pattern; docs: skill improvements (#922)
* docs(skills): add Tier 2.5 localStorage Bearer, SPA discovery, and test standards

From real-world experience building slock.ai CLI adapters:

- oneshot: add network-empty diagnosis, SPA baseURL bundle search, Tier 2.5
  localStorage Bearer template (with multi-tenant X-Server-Id pattern),
  updated auth quick-reference, file path note, opencli browser verify test flow
- explorer: add Tier 2.5 to decision tree and strategy table, update test section
  with opencli browser verify + Done standard, fix Step 5 path to ~/.opencli/clis/,
  add 4 new pitfall rows (SPA HTML, 400 context header, empty network, wrong dir)

* docs(skills): fix path conflict + add anti-change patterns from real adapters

Fix reviewer blocking issue:
- Remove the contradictory "~/.opencli/clis/" note that mixed user-local and
  repo-contributor workflows; replace with explicit two-scenario callout in
  Step 4, Step 5, pitfall table, and oneshot test section
- Template comments in oneshot restored to clis/<site>/<name>.ts (repo path)

Add "抗变更模式" section to explorer, based on opencli's own production code:
- Pattern 1: dynamic queryId discovery (twitter/shared.ts resolveTwitterQueryId)
  — scan loaded JS bundle by operationName (stable) to find queryId (unstable)
- Pattern 2: semantic DOM priority fallback (web/read.ts)
  — article > [role=main] > main > class-hint > body, pick largest text block
- Pattern 3: ordered selector array + timestamp comments (xiaohongshu/publish.ts)
  — first-match wins, comment records UI version and observed attribute values
- Pattern 4: nullish-coalescing field multi-path (xiaohongshu/user-helpers.ts)
  — covers camelCase/snake_case variants without assuming fixed key name

* docs(explorer): split SKILL.md into reference sub-documents

- Shrink main SKILL.md from 994 to 270 lines — core workflow only
- Extract all TS templates (Tier 1~4, pagination) to references/adapter-templates.md
- Add error handling standard: { error, remedy } pattern (remedy > hint)
- Add Tier 2.5 localStorage Bearer template with multi-tenant X-Server-Id example
- Extract cascading requests, tap debug, verbose mode, anti-change patterns to references/advanced-patterns.md
- Extract record workflow to references/record-workflow.md

* docs(skills): fix verify command — split by dev scenario

browser verify only reads ~/.opencli/clis/, not repo's clis/.
Split all verify instructions:
- Repo 贡献: npm run build + opencli <site> <cmd>
- 私人 adapter: opencli browser verify <site>/<name>

Fixes blocker in explorer:L209, L224 and oneshot:L286, L298

* docs(adapter-templates): add utils.ts extraction pattern for same-site adapters

* docs(skills): add decision matrix, stop conditions, sync comments

explorer: add path decision matrix before core workflow
oneshot: add explicit stop/switch conditions (when to escalate to explorer)
both: add keep-in-sync comment on the two-scenario verify block

* feat(slock): extract utils.ts + apply { error, help } pattern; docs: remedy→help

slock/utils.ts: new — getSlockContext(), resolveChannelId()
  - Shared token + workspace resolution, no more 4-line duplication
  - UUID regex (/^[0-9a-f]{8}-...$/) replaces fragile !includes('-')
  - Returns { error, help } instead of throwing

tasks.ts / members.ts / send.ts:
  - Import from utils.ts, remove all duplicated auth boilerplate
  - All errors return [{ error, help }], no more throw
  - members.ts: add limit arg (was unbounded before)

docs: rename remedy → help across all skill references

* refactor(adapters): migrate pipeline adapters to func() with { error, help } pattern

- slock: agents, channels, messages, servers now use getSlockContext/resolveChannelId
  from utils.ts; error handling uses { error, help } return instead of bare throws
- linux-do: export fetchLinuxDoJson from feed.ts; migrate search, topic, categories,
  tags, user-posts, user-topics from pipeline+throw to func() using fetchLinuxDoJson
- xueqiu: add utils.ts with fetchXueqiuJson helper; migrate hot, feed, search, stock,
  watchlist, hot-stock, groups, kline, earnings-date from pipeline+throw to func()

* fix(output): show error rows in table/csv/markdown when columns declared

When a command declares columns (e.g. ['rank', 'title', 'value']) but
returns an error row ({ error, help }), the declared columns would
render empty cells. Now resolveColumns detects the error key and falls
back to the row's actual keys, making diagnostics visible in all output
formats.

* chore: remove slock adapters from this PR

Slock adapters should be in a separate PR, not bundled with the
adapter refactor and skill docs improvements.
2026-04-10 02:29:35 +08:00
jakevin 70b1145b5e refactor: migrate all CLI adapters from YAML to TypeScript (#887)
* refactor: remove version field from GenerateOutcome and EarlyHint

All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.

Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).

* refactor: migrate all 123 CLI adapters from YAML to TypeScript

Remove YAML as an adapter format entirely. All adapters now use
TypeScript with cli() from @jackwener/opencli/registry.

- Convert 123 YAML adapter files to TypeScript via batch script
- Remove YAML scanning from discovery.ts (registerYamlCli, yaml import)
- Remove scanYaml() and shouldReplaceManifestEntry() from build-manifest.ts
- Change synthesize.ts to output JSON candidates (internal format)
- Change generate-verified.ts to write .ts adapter files instead of .yaml
- Delete yaml-schema.ts (dead code) and scripts/yaml-to-ts.mjs (one-time tool)
- Update all tests to match new format

Closes discussion in #OpenCLI thread 47ddba82.

* fix: close YAML migration gaps in plugin scaffold, validation, and scan

- plugin-scaffold.ts: generate hello.ts (TS pipeline) instead of hello.yaml
- plugin.ts validatePluginStructure: no longer accept .yaml as valid command file
- plugin.ts scanPluginCommands: remove .yaml/.yml from scanned extensions
- discovery.ts: add explicit log.warn() when YAML files detected in clis/ or plugins/
- plugin.test.ts: update all test fixtures from .yaml to .js
- plugin-scaffold.test.ts: update hello.yaml references to hello.ts
- Delete dead src/yaml-schema.ts

Resolves PR #887 review blockers from @mbp-codex-pr0.

* refactor: complete YAML removal across docs, skills, record, and binance adapters

Code changes:
- record.ts: candidate output changed from .yaml (yaml.dump) to .json (JSON.stringify), removed js-yaml import
- src/clis/binance: convert all 11 YAML adapters to TypeScript cli() format
- binance/commands.test.ts: rewrite to use registry instead of yaml.load
- skill-generate.test.ts, diagnostic.test.ts: update mock paths from .yaml to .ts
- build-manifest.ts, synthesize.ts: update stale YAML comments

Documentation:
- README.md: remove .yaml from Dynamic Loader, fix plugin types, fix synthesize comment
- README.zh-CN.md: fix synthesize comment
- CONTRIBUTING.md: replace YAML Adapter section with Pipeline Adapter (TS), update arg examples
- docs/developer/yaml-adapter.md: replaced with deprecation redirect
- docs/developer/architecture.md: remove YAML pipeline references
- docs/developer/contributing.md: remove YAML adapter section
- docs/developer/ai-workflow.md: YAML → TS in synthesize description
- docs/guide/getting-started.md: remove .yaml from loader, update engine description
- docs/guide/plugins.md: remove YAML plugin option, update plugin types
- docs/index.md, docs/comparison.md: remove YAML adapter references
- docs/zh/guide/plugins.md: remove .yaml from scan description

Skills:
- opencli-explorer/SKILL.md: rewrite YAML vs TS decision tree to TS-only
- opencli-oneshot/SKILL.md: replace YAML templates with TS cli() templates
- opencli-generate/SKILL.md: YAML artifact path → TS artifact path
- opencli-usage/SKILL.md, plugins.md: update adapter format references

* fix: clean up remaining YAML adapter references in docs

- docs/zh/guide/plugins.md: replace YAML plugin example with TS pipeline
- docs/developer/testing.md: YAML Adapter heading → Adapter, remove validate line
- TESTING.md: same fix in root testing doc
- CONTRIBUTING.md: remove "YAML validation" comment
- docs/.vitepress/config.mts: mark YAML Adapter Guide as (Deprecated) in nav
- docs/advanced/download.md: remove "YAML Adapters" from pipeline step heading
2026-04-08 23:01:08 +08:00
tiaot33 5553300597 feat(linux-do): split topic content into a dedicated command (#821)
* feat(linux-do): split topic content into a dedicated command

Move the old main-post path out of linux-do topic so topic stays a summarized first-page reader while topic-content becomes the Markdown-focused entrypoint for full post bodies.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* feat(linux-do): update topic content handling to include YAML front matter

* fix(linux-do): update default output format to plain for topic-content rendering

* fix(linux-do): replace js-yaml with inline YAML serialization for topic-content

Adapters must only import node builtins, relative modules, or opencli
public APIs. Hand-roll the simple front matter serialization to remove
the third-party js-yaml dependency.

* fix(linux-do): refine YAML quoting to only escape colons followed by space

Colons in URLs (e.g. https://) are valid unquoted YAML values. Only
quote when a colon is followed by a space or appears at end of line.

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-06 02:59:56 +08:00
jakevin 4fe9a73ebc refactor: migrate adapter imports to package exports (#795)
* refactor: migrate adapter imports to package exports

Replace all relative imports (../../src/registry.js, ../../browser/cdp.js, etc.)
with package exports (@jackwener/opencli/registry, @jackwener/opencli/errors, etc.)
across all 484 adapter files.

This decouples adapter import resolution from directory structure:
- User CLIs in ~/.opencli/clis/ resolve via node_modules symlink
- Internal adapters resolve via Node.js self-referencing
- No more shim files needed for import resolution

Changes:
- package.json: add sub-path exports for all public modules
- clis/**: replace relative imports with @jackwener/opencli/...
- discovery.ts: simplify ensureUserCliCompatShims to symlink-only
- registry-api.ts: export CommandArgs type
- Remove root-level shim directories (browser/, download/, pipeline/)
- Remove shim entries from tsconfig.json include and package.json files

* test: add regression tests for package exports

Prevents regressions like #788/#791 by:
1. Scanning all adapter files for forbidden relative imports
   (../../src/, ../../browser/, etc.) — fails if any remain
2. Verifying every package.json export maps to an existing source file

18 new test cases.

* fix: use junction on Windows + broaden test patterns

- discovery.ts: use 'junction' symlink type on Windows (no admin required)
- package-exports.test.ts: generalize forbidden patterns to catch any
  depth of ../ traversal (not just ../../ and ../../../)

* fix: update stale vi.mock/importActual paths in adapter tests

Test files still used old relative paths for vi.mock() and
vi.importActual() calls. Updated 5 test files to use package exports.
Also broadened regression test patterns to catch mock/importActual paths.

* fix: use rm instead of unlink for symlink cleanup, add warn on failure

Addresses review feedback from Astro-Han:
- rm() handles both symlinks and stale directories (unlink fails on dirs)
- Log a warning when symlink creation fails instead of silent catch

* docs: update import examples to use package exports

Update all documentation, contributing guides, and skills to use
@jackwener/opencli/registry instead of ../../src/registry.js.

Without this, users following the docs would write adapters with
broken imports since the old shim files are no longer created.
2026-04-05 16:02:17 +08:00
jakevin 12176de1a5 refactor: simplify core modules (#784)
* refactor: simplify core modules — remove root shims, consolidate error classification, streamline cascade/interceptor, clean up synthesize

1. Remove root-level shim files (errors.ts, logger.ts, registry.ts, types.ts, utils.ts, launcher.ts) — update all ~840 adapter imports to reference src/ directly
2. Consolidate interceptor: reuse shared DISGUISE_FN in tap interceptor instead of reimplementing
3. Unify error classification: single ClassifiedError type with icon/exitCode/hint lookup table, eliminating duplicated pattern matching between resolveExitCode and renderError
4. Simplify cascade probe: replace repetitive switch cases with PROBE_OPTIONS lookup map
12. Clean up synthesize.ts: remove deprecated snake_case field aliases (recommended_args, recommended_columns, recommendedColumnsLegacy) and unnecessary constant aliases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update import paths in contributor docs and skill templates

Update all documentation and skill files to reference src/ directly,
matching the shim removal in the previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update new Gemini adapter imports to use src/ paths

Fix imports in newly added deep-research adapter files that were
still referencing the deleted root shim files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update xiaohongshu tests for /search_result/ URL change

Tests now expect /search_result/<id> for bare note IDs (matching
the note-helpers.ts change from PR #774) and updated empty-shell
hint assertion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update LessWrong and hupu adapter imports to use src/ paths

Fix imports in newly merged LessWrong and hupu/mentions adapter
files that were still referencing the deleted root shim files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(xueqiu): mock logger via src path

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 03:35:12 +08:00
jakevin 80eef46b4e refactor: monorepo adapter separation (clis/ at root) (#782)
* refactor: move adapters from src/clis/ to root clis/ for monorepo separation

Separates CLI adapters from the core runtime to prepare for independent
adapter distribution via postinstall fetch.

Key changes:
- Move src/clis/ → clis/ (adapters at repo root)
- Change tsconfig rootDir from "src" to "." so tsc compiles both
- Create root-level shim files (registry.ts, errors.ts, etc.) so adapter
  relative imports (../../registry.js) resolve correctly
- Update build-manifest.ts, main.ts paths for new dist/src/ structure
- Expand ensureUserCliCompatShims() to cover all adapter import targets
  (types, utils, logger, launcher, browser/*, download/*, pipeline/*)
- Add scripts/fetch-adapters.js postinstall for ~/.opencli/clis/ sync
- Update vitest.config.ts adapter test paths
- Add package.json files field to exclude adapters from npm package

Official adapter files are unconditionally overwritten on update;
user-created files not in the manifest are preserved.

* fix: add dist/clis/ and cli-manifest.json to npm files, harden fetch-adapters

- Add dist/clis/ and dist/cli-manifest.json to package.json files field
  so built-in adapters and manifest ship with the npm package
- Replace execSync with execFileSync to prevent command injection
- Add version check to skip redundant adapter fetches
- Track tmpRoot explicitly for reliable cleanup

* fix: address review blockers — manifest-based updates, global-only fetch, first-run fallback

1. Manifest-based update strategy:
   - Read old manifest to identify previously-official files
   - Clean up files removed upstream (in old manifest but not new)
   - User-created files (never in any manifest) remain untouched

2. Only run fetch-adapters on global install (npm_config_global=true)
   or explicit OPENCLI_FETCH=1, preventing heavy side effects for
   local/dev installs

3. First-run fallback in discovery.ts:
   - ensureUserAdapters() checks for adapter-manifest.json
   - If missing and ~/.opencli/clis/ is empty, spawns fetch-adapters.js
   - Guarantees adapters are available even with --ignore-scripts

* fix: remove OPENCLI_FETCH env var, use internal _OPENCLI_FIRST_RUN instead

* feat: also support OPENCLI_FETCH=1 for explicit adapter fetch trigger

* simplify: replace git clone with local copy from dist/clis/

Adapters already ship in the npm package (dist/clis/), so there's no
need to clone from GitHub. Copy directly from the installed package:

- Eliminates git, curl, tar dependencies
- No network calls in postinstall
- No timeout/offline issues
- Version always matches the installed CLI
- ~65 lines of clone/download code replaced by one cpSync loop
2026-04-05 01:46:36 +08:00