firecrawl setup developer-index # firecrawl- prefix optional
firecrawl setup firecrawl-scrape
Group subcommands keep precedence, so `setup build` means the group and
`setup firecrawl-build` reaches the skill.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One short command per skill family:
firecrawl setup core # primitives + index skills ("skills" stays as alias)
firecrawl setup build # app-integration skills (previously not installable via setup)
firecrawl setup workflows # unchanged
The build family had no install path since init dropped it; BUILD_SKILLS
mirrors the five build skills in the catalog. All groups install from the
catalog by name, so skills.sh counters stay consolidated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The catalog directory is being renamed from skills/cli/ to skills/core/ —
the family covers the Firecrawl primitives via CLI or MCP, not just the
CLI. Install selection is name-based, so slugs are unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review fixes for #203:
- setup's no-npx fallback called installSkillsNative(repo) without the
name filter, installing the entire catalog (build skills included)
instead of the selection every other install path uses
- drop the nonexistent --skills-only CLI flag from the developer-index
skill (skills="only" stays HTTP/MCP-only per cli-argv contract)
- note that the workflow retry hint reinstalls all workflow skills, not
the picked subset
- fix 'sturctural' typo in the research-index skill
They teach the firecrawl research and firecrawl developer CLI commands,
so this repo is their code home. Authored here, mirrored to the catalog
under skills/cli/, and included in the default init/setup CLI set.
The init retry hints point at setup skills/workflows, but those still
installed from the legacy repo list (including build skills init no
longer installs). Share the catalog selections between init and setup
so a retry reinstalls exactly what init attempted. Harden the sync
workflow: least-privilege token, SHA-pinned checkout.
Route all init skill installs through the firecrawl/skills catalog with
name-based --skill selection so install volume accrues to catalog
counters. Interactive init now offers a multi-select of the 16 workflow
skills; build skills are no longer installed (they target SDK
integration, not CLI users, and remain one command away). Promote
npx skills add firecrawl/skills and the contributor routing rule in the
README.
* Add developer-index and research-index as session skills.
They belong with the CLI, not the build skills repo. Developer hits use the live id prefix for kind; search-papers hits are paperId, primaryId, ids, title, abstract, and score.
* Align index session skills with the live wire and shorter trigger descriptions.
* Match developer and research index skills to the prod scrape/map shape.
Quick start, --help instead of option tables, and a Done when line, while keeping the live id-prefix and paperId response shapes.
- Replace cached CLI option tables with 'firecrawl <cmd> --help' pointers,
keeping only options the recipes actually use (all verified against src/)
- Delete 'When to use' sections that restated frontmatter descriptions
- Split firecrawl-monitor: lean recipe in SKILL.md, goal/query authoring in
goals.md, JSON change tracking in json-tracking.md, each behind a one-line
'read when' pointer
- Rewrite never/don't phrasing as positive instructions; keep genuine
opt-out and data guardrails
- Add exactly one 'Done when' completion criterion per skill
skills/**/*.md drops from 1045 to 837 lines (SKILL.md files: 1045 -> 723).
- Router SKILL.md 329 -> 141 lines: pure routing; monitor details,
install/auth, and search-feedback specifics now live only in their
sub-skills, with routing pointers left behind.
- Moved two unique monitor facts into firecrawl-monitor (markdown-only
diff.json parse-diff AST note; goal bullet about user-named noise).
- Resolved keyless contradiction: keyless free tier is real for
search/scrape/interact (src/utils/client.ts isKeylessMode; scrape and
search absent from AUTH_REQUIRED_COMMANDS in src/index.ts). Single
correct statement kept in rules/install.md, including which commands
prompt interactive login.
The developer-search wire carries no type field; the kind is encoded in
the id prefix (doc:, issue:, pull_request:, readme:). The old item.type
read matched nothing and the kind label silently never rendered.
Developer search passages are now shaped server-side (search#843 policy):
the CLI trusts passage_budget_applied responses and keeps a legacy local
cut only for older servers. Removes the short-lived --passage-budget flag
before it ever shipped in a release.
interact sends --code literally to the API with no client-side
prefixing, and the server-side bash environment requires invoking
agent-browser explicitly (per maintainer review). The earlier fix
correctly removed the nonexistent --language flag but wrongly dropped
the prefix, over-generalizing from browser execute's auto-prefix
logic, which never runs for interact.
The collapsed example only sent --missing-content, so substituting
--rating good violated the valuable-sources requirement and the
--silent call swallowed the HTTP 400, silently losing the refund.
Show both --valuable-sources and --missing-content so any rating
substitution satisfies the substantive-content rule.
A live --rating good default anchors copy-paste agents toward
reporting good and skews the feedback signal. Show a
<good|partial|bad> placeholder that cannot run until the agent
chooses, keep the rating-neutral --missing-content field as the
worked example, and let the rules list govern per-rating required
fields. Applied to both mirrored snippets.
Replace the three parallel rating examples (latterly commented
alternates) with a single guarded good-rating call. The rules list
above already defines per-rating field requirements and the
--missing-content section documents payload shapes, so the extra
examples duplicated both. Follows the default-with-escape-hatch
pattern and matches the mirrored firecrawl skill snippet.
All three rating calls were live sequential commands inside the guard,
so a literal copy-paste submitted three conflicting ratings for one
search id. Keep the good-rating call live and show partial/bad as
commented alternates, matching the "send exactly one" prose.
Update the .claude-plugin skill paths missed by the folder rename, and
move the three search-feedback examples inside the guard conditional so
copying any of them cannot bypass the zero-result check.
The installer copies each skill to ~/.agents/skills/<frontmatter-name>/,
and the router skill is named "firecrawl", so ../firecrawl-cli/SKILL.md
links 404ed after install. Rename the folder to skills/firecrawl/ so the
folder and frontmatter agree and cross-skill links resolve both in-repo
and installed.
A bare SEARCH_ID=$(jq -er ...) assignment does not stop the follow-on
search-feedback call when the guard fails (missing file or zero
results), so it fired with an empty id. Wrap the feedback call in the
guard conditional in both mirrored snippets.
Adversarial review against src/ surfaced wrong claims that survived the
first audit pass, in the same files this PR touches:
- scrape: -H is --html (raw HTML shortcut), not HTTP headers
- interact: --language does not exist; the CLI uses --node/--python/--bash
- download: the command is only registered as 'firecrawl x download'
(experimental), so all examples used a nonexistent invocation; the
'only these options' list omitted --lockdown; the wizard example is
TTY-only and blocks agents without -y
- search: spell out guard-failure behavior and split the missing-file
vs stale-file hazards
- scrape/agent: reword confusing multi-URL parenthetical and dedupe the
--wait tip
- cli: mirror the guarded jq search-id recipe so it doesn't drift from
the corrected search skill
Make download examples safe for non-interactive flows and correct agent JSON guidance in the README. Include skill Markdown in Prettier scripts so formatting drift is caught.
* fix(research): describe the real paper corpus in CLI help
Why: over 40 agent traces, 8/8 biomedical CLI runs read the root-help
research line ("Research arXiv papers and GitHub history"), concluded the
index was arXiv-only, and hand-scraped PubMed instead of ever running
`firecrawl research`. The index is ~43M abstracts, ~90% biomedical. The
group description is the single highest-leverage string: it renders both
in `firecrawl --help` and as the `firecrawl research --help` header.
Summary:
- research group description now names the corpus (PubMed/bioRxiv/medRxiv
plus arXiv), its scale, and routes agents away from hand-scraping.
- search-papers description drops the three "arXiv" claims and names the
multi-source id forms it actually returns; keeps the load-bearing
"run several distinct framings" guidance verbatim.
- search-papers --categories only claims what is true: arXiv-style
taxonomy labels, not applicable to biomedical records.
- search --categories disambiguates `research` (a ~web-domain filter)
from the paper index, pointing at `research search-papers`. The wire
value 'research' is unchanged.
- related-papers accepts pmid:/pmcid:/doi:/arxiv: forms in its docs, in
line with inspect-paper and read-paper.
- research examples now include biomedical ids, not arXiv only.
- cli-argv tests gain the research pair that mirrored `developer`, plus a
regression guard that root help names the biomedical corpus.
* docs(readme): route paper search to `research`, add its command section
Why: the README taught the wrong tool. "# Find research papers" was
followed by `firecrawl search --categories research`, which is only a
website filter over ordinary web results, not the paper index. README
also had a `###` section for every command except `research`, so the
index had no documentation surface at all, and the tagline omitted both
`research` and `developer`.
Summary:
- the "Find research papers" example now leads with `research
search-papers`; the `--categories research` example stays but is
labelled as the website filter it is.
- same disambiguation added to the category example block and the search
options table.
- new `### research` section beside `developer`: corpus and scale, the
explicit "not the same as --categories research" note, all five
subcommands, search-papers options, and workflow tips.
- line-3 tagline now mentions research papers and developer sources.
* chore: add research vocabulary to package metadata, bump to 1.20.0
Why: npm search and package listings carried no research, paper, or
biomedical vocabulary, so the paper index was undiscoverable from outside
the CLI too. Behavioural metadata change across help text, docs, and
skills -> minor bump.
Summary:
- package.json description names the paper index and its sources.
- keywords gain research, paper/literature search, biomedical, pubmed,
biorxiv, medrxiv, arxiv, life sciences.
- 1.19.31 -> 1.20.0.
- nfpm.yaml and homebrew/firecrawl-cli.rb carry the same one-line
description; both updated. The brew `desc` is kept under the 80-char
audit limit.
Note: homebrew/firecrawl-cli.rb `version` is 1.10.0 and is not written by
any release tooling in this repo; left as found rather than guessed at.
* docs(skills): give paper search a routing surface in the CLI skills
Why: agents load skills before they load `--help`. firecrawl-search had a
full "Developer search" section and nothing for research; its options
table listed `research` with no elaboration, so the naming collision with
the paper index was invisible. firecrawl-cli's routing table and "When to
Load References" omitted research entirely, so nothing ever pointed an
agent at `firecrawl research`.
Summary:
- firecrawl-search: new "## Paper search" section parallel to "##
Developer search" — states plainly that `--categories research` is not
the paper index, routes to `research search-papers`, and shows the
find/expand/verify loop with real id forms.
- firecrawl-search: `--categories` options row and "When to use" now
disambiguate the two; frontmatter description gains literature triggers
so the skill actually loads on a biomedical task.
- firecrawl-cli: research rows added to the routing table and to "When to
Load References", both stating the anti-pattern (hand-scraping PubMed).
* docs: stop pinning stale CLI versions in install commands
Why: cubic flagged the README install example pinning firecrawl-cli@1.19.6
while this PR ships 1.20.0. The pin was already 25 patches behind npm's
published 1.19.31, and the same stale pin appeared in five more places in
the bundled skills rules.
Summary: install and run commands now use @latest, matching the convention
the public docs already use in all 30 of their occurrences. That removes the
drift permanently rather than trading one stale pin for another — and avoids
pinning 1.20.0, which is not published yet and would break the documented
one-shot install until release. The illustrative `--status` sample output now
shows the version this PR ships.
Test Plan: pnpm test 422/422, prettier clean, no hard-pinned versions remain
in any markdown file.
Eric Ciarla