Commit Graph

48 Commits

Author SHA1 Message Date
Developers Digest 21cccc51cc Add Alexandria discovery and retry IDs to CLI 2026-09-10 16:47:19 -04:00
Himadri Mishra 5cf5c926d3 revert: remove managed MCP router cards (#153) 2026-07-12 02:28:13 +05:30
Himadri Mishra f7ce029608 feat: deliver managed MCP router cards (#152) 2026-07-12 02:17:50 +05:30
Developers Digest 792e637330 Merge remote-tracking branch 'origin/main' into check-for-updates 2026-06-19 11:59:52 -04:00
Developers Digest 911070a543 add make default web setup 2026-06-19 11:55:08 -04:00
mogery 928fbbff7d feat: add research CLI commands 2026-06-16 08:47:37 -07:00
mogery d2da1c874e chore: updates 2026-06-15 09:34:38 -07:00
Developers Digest df37ecc62d wire up setup defaults and add interactive per-harness control 2026-06-09 12:41:48 -04:00
Developers Digest 875f721dd6 harden web defaults config edits 2026-06-04 11:09:57 -04:00
Developers Digest eb12c048d5 add firecrawl web defaults setup 2026-06-04 11:00:21 -04:00
Developers Digest 14ba13a453 throttle update notice 2026-06-04 08:29:02 -04:00
Developers Digest 3c45bcd424 add startup update notice 2026-06-04 08:13:21 -04:00
Abimael Martell 48a913d5b8 feat: Add doctor command (#112)
* Add firecrawl doctor command (ENG-4883)

* Fix doctor support diagnostics

* Refine doctor credit label

* Polish doctor option handling

* Respect terminal color support in doctor

* Bump CLI version to 1.18.7
2026-06-01 12:12:40 -07:00
Abimael Martell b4b333c377 feat(scrape): add redact-pii flag (#124)
* Add redact-pii scrape flag

* Bump CLI version
2026-06-01 11:27:43 -07:00
Erik Engervall 69c74ee7f9 Refactor package dependencies to replace '@mendable/firecrawl-js' wit… (#118)
* Refactor package dependencies to replace '@mendable/firecrawl-js' with 'firecrawl' in package.json and update related imports across the codebase. This change ensures consistency and aligns with the new package naming convention.

* Update import statement for Firecrawl in client.ts to use named import for consistency with new package structure.
2026-05-29 08:39:58 -04:00
Abimael Martell d0145f4e62 feat: add --lockdown flag to scrape (#104)
* feat: add --lockdown-mode flag to scrape

* rename --lockdown-mode to --lockdown
2026-04-22 15:19:02 -07:00
Developers Digest bc2c6d93a8 Switch agent scaffold to web-agent repo
Update vendored scaffold references and default remote to use firecrawl/web-agent. Adjusted header comments in create.ts, create-flow.ts, credentials.ts, manifest.ts, scaffold.ts, and ui.ts, and changed DEFAULT_REMOTE in manifest.ts from 'firecrawl/firecrawl-agent' to 'firecrawl/web-agent' so the scaffolder clones the web-agent repo for templates.
2026-04-16 12:16:44 -04:00
Developers Digest 47af8daf5c chore: friendlier error when default clone fails 2026-04-15 13:58:19 -04:00
Developers Digest b7c16d0df6 feat: vendor agent scaffolder into firecrawl-cli
Replaces the `npx firecrawl-agent-cli` delegator with an in-process
scaffolder vendored from firecrawl/firecrawl-agent. No separate npm
package for the agent CLI — the root firecrawl-cli clones the public
agent repo at runtime to fetch templates + agent-core.

- Vendor under src/utils/agent-scaffold/ (manifest, scaffold,
  credentials, ui, create-flow). Upstream-tracked; keep in sync.
- manifest.ts: strip the "bundled next to CLI" branch (doesn't apply
  here), always clone the public repo. Nested-manifest fallback
  handles the current `.internal/cli/agent-manifest.json` location.
- Extract handleCreate from upstream init.ts; drop the commander
  wrapper since the root CLI owns the command surface.
- create.ts now lazy-imports the scaffolder so non-create paths stay
  snappy.

Smoke-tested end-to-end against a local agent repo via --from: next
template scaffolds correctly with agent-core, .env.local, and
expected file layout.
2026-04-15 13:56:17 -04:00
Nicolas 40b3fdd9a3 Nick: improvements 2026-03-23 17:17:03 -03:00
Nicolas c084d98486 Nick: 2026-03-23 12:54:22 -03:00
Nicolas 0bdf2b7426 Nick: 2026-03-22 20:25:44 -03:00
Developers Digest 7ba546f5ce add --query flag for query format support 2026-03-12 11:39:46 -04:00
Developers Digest c79f8c403f fix download to map from origin 2026-02-23 10:29:02 -05:00
Developers Digest 33f41bc022 Add dl command to map + scrape with options with app router style download organization 2026-02-19 17:58:08 -05:00
Nicolas 03d66babe7 Merge pull request #30 from firecrawl/browser
b
2026-02-16 19:17:11 -08:00
Developers Digest 0f1e6545ec browser init 2026-02-13 13:01:08 -05:00
Abimael Martell f2f2789625 feat(scrape): Implement location settings for scrapes (#29) 2026-02-12 15:15:31 -08:00
Abimael Martell 60aa8f6154 feat(scrape): Add --max-age option to Scrape (#26)
* feat(scrape): Add --max-age option to Scrape

* update comment

* update doc

* update doc
2026-02-06 23:09:23 -08:00
Abimael Martell 41927d7bd4 Implement agent command (#22)
* Add agent command for AI-powered web data extraction

Implement the agent endpoint that allows users to extract data from the web
using natural language prompts. Supports structured output via JSON schemas,
URL targeting, credit limits, and async job handling with polling.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add --model option to agent command

Support selecting between fire-1 (default) and fire-1-mini (faster, cheaper)
models for agent extraction tasks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update firecrawl-js SDK to 4.12.0 with model support

- Upgrade @mendable/firecrawl-js from 4.10.0 to 4.12.0
- Update model types to match SDK: spark-1-mini and spark-1-pro

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add spinner animation for agent command

- Add spinner utility for CLI feedback
- Show animated spinner while agent is starting and processing
- Display success/fail indicators when complete
- Remove --progress flag (spinner now always shown)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Handle Ctrl+C gracefully in agent command

When user interrupts with Ctrl+C while waiting for agent completion,
show helpful message with the command to check job status later.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Allow --wait when checking agent job status

Now you can use `firecrawl agent <job-id> --wait` to continuously
poll an existing job until it completes, with the same spinner
animation and Ctrl+C handling.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Improve error handling in agent command

- Stop spinner before showing error messages
- Extract detailed error messages from API responses
- Show full error details to help debug issues

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Extract error details from Firecrawl SDK errors

Show helpful error messages like "Invalid option: expected one of
spark-1-pro|spark-1-mini" instead of just "Bad Request".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Validate model option before API call

Show clear error message for invalid models instead of waiting
for API rejection.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add agent to skill

* only command for now

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 16:36:14 -08:00
Abimael Martell 8845a2fc08 Add support for local/custom API URLs (#20)
* Add support for local/custom API URLs

For custom API URLs (e.g., local development with Docker), the CLI now:
- Skips API key requirement when using non-cloud URLs
- Allows optional API key input during login
- Skips browser auth flow for custom URLs

This enables use with local Firecrawl instances that have
USE_DB_AUTHENTICATION=false.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add --api-url option to commands

Add --api-url flag at both global and command levels (scrape, crawl,
map, search, credit-usage) similar to --api-key. When a custom API URL
is provided, authentication is skipped allowing requests to local or
self-hosted Firecrawl instances without an API key.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: Add --api-url option documentation

Document the --api-url option for self-hosted and local development
use cases in the README, including examples for environment variables
and CI/CD usage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: Add --api-url to login and config examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:48:27 -08:00
Abimael Martell 96d0786b4f Fix job ID detection to support UUID v7 (#19)
Firecrawl returns UUID v7 job IDs, but the regex only matched UUID v4.
This caused `firecrawl crawl <job-id>` to fail with a URL validation error
instead of checking job status.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:26:16 -08:00
Nicolas 2925e4bc74 Update auth.ts 2026-01-22 13:29:42 -03:00
Nicolas 4279839428 Update auth.ts 2026-01-22 12:42:24 -03:00
Nicolas e8cc94b24d Nick: 2026-01-22 12:34:58 -03:00
Nicolas 424605ef33 Nick: infer output from file extension 2026-01-21 16:06:29 -03:00
Nicolas dc6fbf9f3c Nick: positional formats 2026-01-21 15:04:16 -03:00
Nicolas 9cadd3bb26 Nick: 2026-01-21 01:16:36 -03:00
Nicolas 25ead35b1f Nick: 2026-01-21 00:29:48 -03:00
Nicolas 4a3d047783 Nick: 2026-01-21 00:09:47 -03:00
nobita2041 c61bafcb62 Add multiple format support for scrape command (#8)
* Fix array format output for links and images

The links and images formats return arrays from Firecrawl API,
but writeOutput expected a string, causing endsWith() to fail.

Now converts arrays to newline-separated strings for CLI-friendly output.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add multiple format support for scrape command

- Allow comma-separated formats: --format markdown,links,images
- Single format outputs raw content (backward compatible)
- Multiple formats output JSON with all requested data
- Case-insensitive format input (rawHtml, RAWHTML both work)
- Add format validation with helpful error messages
- Update help text to clarify multiple format usage

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 13:41:32 -08:00
Abimael Martell 279ddc0d96 Add timing information to scrape (#7)
* Add timing information to scrape

* bump
2026-01-13 11:39:27 -08:00
Abimael Martell be2f61b919 Update Global Config and Entry Points (#6) 2026-01-09 13:41:24 -08:00
Abimael Martell 107bbc9811 feat: Add crawl and map commands (#3)
* feat: Add crawl and map commands

* bump version
2026-01-08 12:04:26 -08:00
Abimael Martell 283404c2dc Add CI Workflow and Code Formatting (#1)
* Add CI Workflow and Code Formatting

* update lockfile

* deprecation notice

* update formatting rule
2026-01-08 11:35:00 -08:00
Abimael Martell 6241325370 update config error 2026-01-08 11:10:39 -08:00
Abimael Martell d85d2b86f8 handle default output 2026-01-02 11:34:46 -08:00
Abimael Martell 30ec5fe79e add implementation 2026-01-02 11:20:40 -08:00