package-lock.json was accidentally committed in #206 (an npm install
artifact — this repo is pnpm-managed). bun prefers it over pnpm-lock.yaml
in release-binaries, and its migration fails under --frozen-lockfile.
Remove it and gitignore it.
setup.ts was formatted by prettier 3.9.6 from that same npm node_modules;
CI pins 3.7.4 via pnpm-lock.yaml. Reformatted with 3.7.4.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
After any setup skill install, check for an API key (env var or stored
credentials). Authenticated users see nothing. Otherwise: --browser runs
the browser login directly, an interactive terminal offers it (default
yes), and non-interactive runs print a one-line hint — never a prompt,
so agents, scripts, and CI are unaffected. The bundle flow defers the
offer until every step has run so it never asks twice.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
firecrawl setup developer-index # firecrawl- prefix optional
firecrawl setup firecrawl-scrape
Group subcommands keep precedence, so `setup build` means the group and
`setup firecrawl-build` reaches the skill.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One short command per skill family:
firecrawl setup core # primitives + index skills ("skills" stays as alias)
firecrawl setup build # app-integration skills (previously not installable via setup)
firecrawl setup workflows # unchanged
The build family had no install path since init dropped it; BUILD_SKILLS
mirrors the five build skills in the catalog. All groups install from the
catalog by name, so skills.sh counters stay consolidated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The catalog directory is being renamed from skills/cli/ to skills/core/ —
the family covers the Firecrawl primitives via CLI or MCP, not just the
CLI. Install selection is name-based, so slugs are unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review fixes for #203:
- setup's no-npx fallback called installSkillsNative(repo) without the
name filter, installing the entire catalog (build skills included)
instead of the selection every other install path uses
- drop the nonexistent --skills-only CLI flag from the developer-index
skill (skills="only" stays HTTP/MCP-only per cli-argv contract)
- note that the workflow retry hint reinstalls all workflow skills, not
the picked subset
- fix 'sturctural' typo in the research-index skill
They teach the firecrawl research and firecrawl developer CLI commands,
so this repo is their code home. Authored here, mirrored to the catalog
under skills/cli/, and included in the default init/setup CLI set.
The init retry hints point at setup skills/workflows, but those still
installed from the legacy repo list (including build skills init no
longer installs). Share the catalog selections between init and setup
so a retry reinstalls exactly what init attempted. Harden the sync
workflow: least-privilege token, SHA-pinned checkout.
Route all init skill installs through the firecrawl/skills catalog with
name-based --skill selection so install volume accrues to catalog
counters. Interactive init now offers a multi-select of the 16 workflow
skills; build skills are no longer installed (they target SDK
integration, not CLI users, and remain one command away). Promote
npx skills add firecrawl/skills and the contributor routing rule in the
README.
* Add developer-index and research-index as session skills.
They belong with the CLI, not the build skills repo. Developer hits use the live id prefix for kind; search-papers hits are paperId, primaryId, ids, title, abstract, and score.
* Align index session skills with the live wire and shorter trigger descriptions.
* Match developer and research index skills to the prod scrape/map shape.
Quick start, --help instead of option tables, and a Done when line, while keeping the live id-prefix and paperId response shapes.
The developer-search wire carries no type field; the kind is encoded in
the id prefix (doc:, issue:, pull_request:, readme:). The old item.type
read matched nothing and the kind label silently never rendered.
* fix(research): describe the real paper corpus in CLI help
Why: over 40 agent traces, 8/8 biomedical CLI runs read the root-help
research line ("Research arXiv papers and GitHub history"), concluded the
index was arXiv-only, and hand-scraped PubMed instead of ever running
`firecrawl research`. The index is ~43M abstracts, ~90% biomedical. The
group description is the single highest-leverage string: it renders both
in `firecrawl --help` and as the `firecrawl research --help` header.
Summary:
- research group description now names the corpus (PubMed/bioRxiv/medRxiv
plus arXiv), its scale, and routes agents away from hand-scraping.
- search-papers description drops the three "arXiv" claims and names the
multi-source id forms it actually returns; keeps the load-bearing
"run several distinct framings" guidance verbatim.
- search-papers --categories only claims what is true: arXiv-style
taxonomy labels, not applicable to biomedical records.
- search --categories disambiguates `research` (a ~web-domain filter)
from the paper index, pointing at `research search-papers`. The wire
value 'research' is unchanged.
- related-papers accepts pmid:/pmcid:/doi:/arxiv: forms in its docs, in
line with inspect-paper and read-paper.
- research examples now include biomedical ids, not arXiv only.
- cli-argv tests gain the research pair that mirrored `developer`, plus a
regression guard that root help names the biomedical corpus.
* docs(readme): route paper search to `research`, add its command section
Why: the README taught the wrong tool. "# Find research papers" was
followed by `firecrawl search --categories research`, which is only a
website filter over ordinary web results, not the paper index. README
also had a `###` section for every command except `research`, so the
index had no documentation surface at all, and the tagline omitted both
`research` and `developer`.
Summary:
- the "Find research papers" example now leads with `research
search-papers`; the `--categories research` example stays but is
labelled as the website filter it is.
- same disambiguation added to the category example block and the search
options table.
- new `### research` section beside `developer`: corpus and scale, the
explicit "not the same as --categories research" note, all five
subcommands, search-papers options, and workflow tips.
- line-3 tagline now mentions research papers and developer sources.
* chore: add research vocabulary to package metadata, bump to 1.20.0
Why: npm search and package listings carried no research, paper, or
biomedical vocabulary, so the paper index was undiscoverable from outside
the CLI too. Behavioural metadata change across help text, docs, and
skills -> minor bump.
Summary:
- package.json description names the paper index and its sources.
- keywords gain research, paper/literature search, biomedical, pubmed,
biorxiv, medrxiv, arxiv, life sciences.
- 1.19.31 -> 1.20.0.
- nfpm.yaml and homebrew/firecrawl-cli.rb carry the same one-line
description; both updated. The brew `desc` is kept under the 80-char
audit limit.
Note: homebrew/firecrawl-cli.rb `version` is 1.10.0 and is not written by
any release tooling in this repo; left as found rather than guessed at.
* docs(skills): give paper search a routing surface in the CLI skills
Why: agents load skills before they load `--help`. firecrawl-search had a
full "Developer search" section and nothing for research; its options
table listed `research` with no elaboration, so the naming collision with
the paper index was invisible. firecrawl-cli's routing table and "When to
Load References" omitted research entirely, so nothing ever pointed an
agent at `firecrawl research`.
Summary:
- firecrawl-search: new "## Paper search" section parallel to "##
Developer search" — states plainly that `--categories research` is not
the paper index, routes to `research search-papers`, and shows the
find/expand/verify loop with real id forms.
- firecrawl-search: `--categories` options row and "When to use" now
disambiguate the two; frontmatter description gains literature triggers
so the skill actually loads on a biomedical task.
- firecrawl-cli: research rows added to the routing table and to "When to
Load References", both stating the anti-pattern (hand-scraping PubMed).
* docs: stop pinning stale CLI versions in install commands
Why: cubic flagged the README install example pinning firecrawl-cli@1.19.6
while this PR ships 1.20.0. The pin was already 25 patches behind npm's
published 1.19.31, and the same stale pin appeared in five more places in
the bundled skills rules.
Summary: install and run commands now use @latest, matching the convention
the public docs already use in all 30 of their occurrences. That removes the
drift permanently rather than trading one stale pin for another — and avoids
pinning 1.20.0, which is not published yet and would break the documented
one-shot install until release. The illustrative `--status` sample output now
shows the version this PR ships.
Test Plan: pnpm test 422/422, prettier clean, no hard-pinned versions remain
in any markdown file.
The developer command called /v2/developer/search. That mount rejects a
keyless caller, so keyless developer search never reached the index. It
is also the path that may be withdrawn. Call /v2/search/developer, the
public path, which accepts a keyless caller.
The API returns 10 results by default, not 20. Correct the help text and
the README.
The developer index now has two surfaces, the same as research: the
`developer` category on `firecrawl search`, and a dedicated command.
The command mirrors the research commands. It calls GET
/v2/developer/search, keeps the keyless free-tier path, and renders the
ranked hits as markdown blocks with the passages clipped at 1200
characters; `--json` keeps the full envelope. It takes a query and
`--limit` only; the endpoint accepts no filters, and that narrow
surface is deliberate.
New tests cover the request URL, the readable and JSON output, and the
error path. Two argv tests prove the command appears in the root help
and parses, so a wrapper cannot drop it the way `executeSearch` once
dropped the category group.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C4vZ6XH65bUJDkLthyfmvC
`firecrawl search <query> --categories developer` searches an index built
for coding agents: GitHub issues, merged pull requests, repository
READMEs, and curated documentation sites.
The API already served the category. The CLI rejected the value, and
`executeSearch` copied only `web`, `images`, and `news` out of the
payload. A developer group therefore never reached the caller.
The category is an extra arm, not a filter on the web results, so the API
returns its hits in `data.developer`. The command copies that group
through, prints it under its own heading, and labels the web group when
both are present.
A matched passage runs to several KB. The readable output clips it at 500
characters and names `--json`, which keeps the full text. Without the
clip, three hits flood a terminal.
Add support for scoping init/setup to a single agent/harness. README documents one-shot usage. scripts/install.sh forwards installer args to `firecrawl init` (non-interactive) and runs `init --yes` when piped. init.ts: add interactive harness picker, detectInstalledAgentNames, and installRepoAcrossAgents to allow routing skills/workflows to a subset of detected agents or a single --agent. skills-native.ts: canonicalize per-agent global skills dirs (fixes several agent paths), add detectInstalledAgentNames helper. This enables one-shot, non-interactive, and interactive per-harness installs.
Research commands bypassed the SDK's typed methods and never set
integration, so requests were logged with integration=NULL. Append
integration=cli as a query param in getResearch, covering all five
subcommands in both keyed and keyless modes. Bump to 1.19.24.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJ3prpziToFecBhLGRiX6t
Allow monitor create/update to only read piped stdin when flags can't build a payload. Added allowImplicitStdin option to readJsonPayload and exported helpers hasCreateFlagPayload / hasUpdateFlagPayload to detect when flags are sufficient. Updated command actions to pass the allowImplicitStdin flag and extended tests to cover the stdin payload policy. Also bumped package version to 1.19.23.
- monitor create --help: a 'Target modes' section distinguishing single page,
URL batch, whole-site crawl, and web-search targets.
- On create, print the Cloud dashboard link + a run-now hint (interactive only,
so pipes/--output stay clean).
- SKILL.md: a target-mode table so agents pick the right flags.
- Bump version to 1.19.21.
Clarify messaging by changing references from 'search monitors' to 'web monitors'. Update the thrown error in buildCreateBody to "--goal is required for web monitors (--queries)" and adjust the corresponding unit test to expect the new wording. This makes it clearer that a goal is required when creating web monitors with queries (the API auto-enables the AI judge). Files modified: src/commands/monitor.ts, src/__tests__/commands/monitor.test.ts.
Search monitors (type:"search") are implemented in the API but undocumented.
Add --queries/--search-window/--max-results/--include-domains/--exclude-domains
flags, build the search target in buildCreateBody, and require --goal for search
monitors (the API mandates a non-empty goal). Includes tests.
Add native skills installation support and an interactive setup picker for launch flow. Introduces nativeSkills and quiet options, colorized/condensed output, and status summaries from the native installer (skills-native.ts). The launch command can now prompt to install MCP, skills, or both, and passes ALL_SKILL_REPOS to the skills installer; MCP/skills installs now run with quiet/native flags by default when appropriate (launch.ts, setup.ts). Tests updated to mock inquirer select, handle TTY changes, and assert the new behavior. Also remove the implicit default for the --yes launch option in the CLI (index.ts).
The notice wraps "Update available!" in ANSI codes on a TTY, so the
contiguous substring assertion failed in CI. Assert on the version
portion that survives colorization instead.
Claude-Session: https://claude.ai/code/session_016Szz5ZgWNULKbvNGqt8sdm
Add a skillsAgent property to several LaunchTarget entries (claude-code, codex, codex-app, opencode) so the launcher can associate skill installation with the correct agent. Update unit tests to assert installSkillsForAgent is called (or not called) where appropriate and add a test to verify skipping skills for opencode. Adjusted expectations/counts in existing tests to reflect the new behavior.
Bump CLI version and add YAML dependency; implement native MCP installers for Hermes and OpenClaw and a hosted Firecrawl MCP URL. Normalize agent aliases and extend setup/installMcp to handle special launchers (hermes, openclaw, and --agent all) and to invoke add-mcp with the hosted MCP URL and transport option. Add functions to write Hermes config and configure OpenClaw via its CLI, and add installSkillsForAgent helper. Extend launch command to recognize hermes/openclaw targets, run their MCP installers, optionally install skills (new --skip-skills flag), and launch with correct args. Update command help text and expand tests to cover these new behaviors.
Introduce a separate "Codex App" launch target (aliases: codex-app, codex-desktop, codex-gui) that uses the system "open" command with a macOS fallback (-a Codex). Add a supportsExtraArgs flag and make resolveLaunchCommand throw when extra arguments are passed to targets that don't accept them. Update CLI help to mention codex-app and add tests to verify MCP configuration, app launch behavior, and the error when extra args are provided.
Eric Ciarla