Commit Graph

244 Commits

Author SHA1 Message Date
Max Loffgren 55ece3bcae fix(cli): preserve codex config byte order mark 2026-09-01 00:06:13 -05:00
Max Loffgren 755a8a8a47 test(cli): align setup bundle after rebase 2026-08-31 16:23:22 -05:00
Max Loffgren 5c08fecd7a fix(cli): stop the web-provider step corrupting configs it edits
Four defects in the defaults writer, all reproduced before the fix.

The Codex insert anchored on the first line of the last root key rather than the
end of its value, so a multi-line root value had web_search spliced into the
middle of it. An inline array left the file unparseable while the run reported
success; a triple-quoted string silently corrupted the user's value and never
set web_search. The offset now comes from the end of the value, and the edited
document is re-parsed before it is written, matching how the MCP TOML writer
already guards itself.

Claude settings guarded the read of `permissions` against a non-object but not
the write, so `{"permissions": null}` parsed fine, previewed fine, was confirmed
by the user, and then threw out of jsonc-parser. A permissions value of the
wrong shape now has the whole key replaced, the same way the MCP JSON writer
handles a malformed servers map.

Both files ignored CODEX_HOME and CLAUDE_CONFIG_DIR, which this branch honours
everywhere else, so on a relocated config tree the server entry and the switch
landed in different places and the switch went to a file the agent never reads.

The step is an offer on top of the work the command was asked to do, so an
unreadable file or a permissions error no longer fails a run whose MCP write
already succeeded and was reported.

Also drops an em dash from an init picker label.
2026-08-31 16:23:22 -05:00
Max Loffgren 0f83a954bc fix(cli): raise the aggregate MCP failure after the web-provider step
When one selected agent failed and Claude Code or Codex succeeded, reporting
threw before the handover ran, so --defaults never reached the agent that was
configured. The step is already scoped to agents whose entry landed, so the
throw was acting as a safety gate it did not need to be.

Reporting now only reports. The aggregate failure moves to its own step that
runs after the handover, keeping both existing outcomes: quiet callers still
fail only when nothing landed, and an interactive run still fails when any
requested agent did not.
2026-08-31 16:23:22 -05:00
Max Loffgren 77521ec335 feat(cli): offer the web-provider handover from setup mcp instead of rules
setup mcp wrote instruction files telling each agent to prefer Firecrawl for web
work. A rule is only advice: the agent can still call its own web search, and it
cost a rule spec per agent, a marker-fenced writer for shared files, a launcher
workspace probe, and a five-state reporting surface.

Registering the server is now all setup mcp writes. In its place the command
offers a second step: hand its native web search and fetch over to Firecrawl.
Claude Code and Codex each document a switch for that, so those are the two
agents offered, and only when their server entry landed in the same run. An
agent left without a working Firecrawl server never has its own tools taken
away.

Turning off a built-in tool is a real behavior change, so it is confirmed:
the exact line for each config is printed before anything is written, a value
already set is reported and kept rather than replaced, and declining writes
nothing. -y stays server-only; automation opts in with --defaults and out with
--no-defaults. setup defaults still runs the same step on its own.

Removes the rule spec, the rule writers, the marker fence, the launcher
workspace lookup, and the dead local left over from resolving auth in
installMcpClients.
2026-08-31 16:23:22 -05:00
Max Loffgren 08d9f7d324 fix(cli): never override a web_search the user set, and confirm before writing
setup defaults matched web_search by line and rewrote whatever it found. A user
who had deliberately set web_search = "enabled" got it flipped to "disabled",
and the inline comment explaining why went with it. The already-disabled check
compared the whole line as a string, so web_search="disabled" without spaces was
rewritten too and reported as a change. A test asserted the overwrite, so it was
locked in rather than accidental.

web_search is now read from the TOML AST that this branch already parses for
Codex MCP config. A value the user set is reported and left alone; only an
absent key is written, and only the value this command writes is removed on
undo. A config.toml that does not parse is skipped rather than rewritten.

Claude settings are patched through the same JSONC editor used for MCP config,
so comments survive. Previously they were stripped to parse and lost on write.

Both writers can run as a dry run, so an interactive run now prints the exact
line each agent will receive and asks before touching anything. --yes and
non-interactive runs are unchanged. The closing line no longer claims an agent
was configured when its existing value was kept.
2026-08-31 16:23:22 -05:00
Max Loffgren 09a51381ee fix(cli): install MCP into the agents the user has, not all of them
init passed `--agent all` whenever FIRECRAWL_API_KEY was exported. That was a
workaround for a guard this branch removed, back when environment-backed setup
refused to run without a named agent, and `all` now means every supported
agent whether or not it is installed. On a machine with only Claude Code it
wrote six config files, including ~/.cursor/mcp.json and VS Code's, then
failed on OpenClaw and ended the run pointing at "firecrawl setup mcp". With
no agent named, detection runs instead and resolves the exported key on its
own, so the same machine gets one file and a clean finish.

The rule fence now pairs the last two markers rather than the first two. A
file carrying an odd marker, from a half-written run or a hand edit, paired
that stray one with our opening marker, so the next run replaced the span
between them and took the user's own text with it.

Also: `openclaw config get` gets the 8s timeout doctor's probe already has,
since its stderr is discarded and a wedged launcher would otherwise hang
setup with nothing on screen; a launcher result records the mode the run
configured instead of folding oauth in with keyless; and the `--agent` help no
longer advertises the environment-backed requirement that is gone.
2026-08-31 16:23:22 -05:00
Max Loffgren f0b9ce4e18 fix(cli): edit only Firecrawl's own tables when rewriting Codex config
upsertTomlServer replaced everything between the first Firecrawl table and
the last, so an unrelated [mcp_servers.other] sitting between the entry and
its env sub-table was deleted along with them. Each table is now removed on
its own, in reverse order so the earlier offsets stay valid in the mutated
string, and the replacement goes in where the first one started.

The first table keeps its terminating newline. The replacement block is
written without one, so a config whose next table follows immediately, which
is the documented stdio layout of [mcp_servers.firecrawl] above
[mcp_servers.firecrawl.env], came back as `url = "..."[mcp_servers.other]`
and failed to reparse. The user saw "could not parse existing config" about
a file that was valid TOML.

runClientCommand moves to its own module. Doctor's OpenClaw probe called
execFileSync directly, which cannot launch a .cmd shim, so on Windows it
reported the server unregistered for the exact reason the wrapper exists.
OpenClaw also counts as installed when the launcher is on PATH rather than
only when ~/.openclaw exists, which is what setup already detects.
2026-08-31 16:23:22 -05:00
Max Loffgren 6f7af7767a fix(cli): keep setup --yes and doctor honest when MCP is optional
Do not fail the default setup bundle after skills when no agents are
installed, and treat a recovered JSONC tree as registered when it still
contains Firecrawl.
2026-08-31 16:23:22 -05:00
Max Loffgren ec22426613 fix(cli): stop native MCP setup from writing unusable agent config
Parse Codex TOML by AST so quoted and BOM-prefixed tables are updated
in place, honor CODEX_HOME and HERMES_HOME, write VS Code rules to the
documented instructions path, and reject empty install-mode runs.
2026-08-31 16:23:22 -05:00
Max Loffgren 8662b953dd fix(cli): say when requested rules cannot be installed
Hermes has no global rule file, so --rules was going silent: clients only
reach unsupported after rules were requested, and the dropped line hid that.
2026-08-31 16:23:22 -05:00
Max Loffgren dc33fee7c4 fix(cli): tell quiet callers when only some agents were configured
Quiet mode fails only when nothing lands, so a run where one agent failed
resolved normally and init printed "✓ MCP server installed" directly under
its own ✗ line, then hid the "Add MCP" next step that would have fixed it.
installMcp now returns whether every targeted agent was configured, and init
claims success only on that; a partial run points at "firecrawl setup mcp"
and keeps the next step.

The OpenClaw workspace lookup also expanded any leading tilde against home,
so a `~other/ws` workspace, which names another account to a shell, wrote the
rule into $HOME/other/ws instead. Expansion is now limited to `~` and a `~/`
prefix, and reads the resolved home from the context the writers already
carry rather than calling os.homedir() again.
2026-08-31 16:23:22 -05:00
Max Loffgren b489e62c20 fix(cli): stop init from advertising MCP setup it just ran
The integrations checkbox named three editors, but MCP setup writes to every
agent it detects, so the list both undersold it and went stale as agents were
added. It now reads like the skills entry above it and names no agents.

printNextSteps also offered "Add MCP: firecrawl setup mcp" unconditionally,
including right below its own "✓ MCP server installed". stepIntegrations now
reports whether the install succeeded, and the line prints only for someone
who does not have it: skipping the integration or a failed install both keep
it.
2026-08-31 16:23:22 -05:00
Max Loffgren 682ccb7d34 fix(cli): drop the rules line for agents that have no rules mechanism
Hermes has no global rules file to own, so every run ended its block with "Rules not supported by this agent", a line about something the user never asked for and cannot act on. The skipped case still prints, because that one means a rule was requested and did not land.
2026-08-31 16:23:21 -05:00
Max Loffgren b5910e753f fix(cli): print a sign-in step only where an agent needs one
The oauth report prefixed every step with "Sign in" and then repeated the
verb inside the step itself, so VS Code read "Sign in sign in from the MCP
view in VS Code". Checking each agent against its own docs showed the
duplication was hiding worse problems: VS Code has no MCP view and runs
client registration itself, Cursor surfaces a needs-login control of its
own, OpenCode prompts on first use, and Claude Code shows a startup notice
for any server that answers 401, which the sign-in endpoint does. Those
four now print nothing.

The three that do need a command keep one, and the strings are corrected:
Codex leads with its own command but names the Authenticate action, since
the desktop app and the IDE extension share the config file this writes and
only the CLI needs the command. Hermes reloads config on a 30s timer that
cannot outlast an interactive flow, so it names the login command and says
to run it from a new terminal rather than claiming the browser opens by
itself. The footer no longer promises that every agent prompts you.
2026-08-31 16:18:16 -05:00
Max Loffgren 8f99e7935b fix(cli): skip OpenClaw rules when MCP fails and close restored launcher gaps
The Hermes/OpenClaw revert brought back unguarded launcher rule writes, PATH false positives, a Windows workspace probe that swallowed .cmd failures, and YAML setIn crashes on non-mapping mcp_servers.
2026-08-31 16:18:16 -05:00
Max Loffgren 54c14c6745 fix(cli): ask about rules for --agent all, launchers, and openclaw
Those three branches passed `yes: true` into installMcpClients to skip the
picker, but naming the targets already does that: the picker only runs when no
targets were selected, and each of these arrives with a list. What `yes` still
reached was the rules prompt, so `--agent cursor` asked whether to install rules
and `--agent all` silently decided no.

Dropping the flag leaves the picker skipped as before and lets the question be
asked. Nothing else moves: without a TTY the run is still non-interactive, `-y`
still declines rules, and `--rules` still forces them.
2026-08-31 16:18:16 -05:00
Max Loffgren 2453c661ca Revert "write MCP config only for agents that share one contract"
This reverts the scope reduction and everything built on it. The PR keeps the
harnesses main already supports and changes only how MCP gets installed:
Claude Code, Cursor, VS Code, Codex, OpenCode, Hermes Agent, and OpenClaw, with
native writers in place of a third-party installer subprocess.

Narrowing the set to the agents that share one contract is a real argument, but
it is a different change: it drops harnesses, so it has to move `firecrawl
launch` and the docs with it, and it should be judged on its own rather than
riding along with the writer swap. It comes back as "stop auto-configuring
Hermes and OpenClaw" if we want it.

So the Hermes YAML writer and the OpenClaw launcher are back, `launch`
configures both exactly as it does on main, and the URL-only path invented here
is gone rather than left as a second way an agent can be supported.

Kept from the reverted range, since none of it depends on the smaller set: the
auth-mode check ahead of every branch, rules not being written for an agent
whose server entry failed, a partial run failing the command, and .cursor/ in
.gitignore. The launcher isolation test now asserts both halves of that last
one: OpenClaw failing is reported, and Cursor beside it is still configured.
2026-08-31 16:18:16 -05:00
Max Loffgren 7007415464 fix(cli): indent the unwritten-agent lines with the rest of quiet output
init and launch render an indented block, and these lines sat flush against it.
2026-08-31 16:18:16 -05:00
Max Loffgren 381f8d23f9 fix(cli): keep launch and the agent flags honest about what MCP setup did
installMcp is shared, so making Hermes and OpenClaw URL-only reached callers
this branch never edited. Three consequences, all in that seam:

* `firecrawl launch hermes --install` printed the URL for an agent we do not
  configure and then said it was configured with Firecrawl MCP. Launch now
  claims MCP only when the run actually wrote it, which also covers --skip-mcp,
  where the same line was already claiming a write that never happened.
* Naming an agent we do not configure returned from the whole run, so
  `--agent hermes --cursor` printed the URL and left Cursor unconfigured. The
  URL is now reported for whichever flag named it and the rest of the run
  continues; it ends early only when nothing else was named.
* Detecting no agents at all is still an error, since nothing was configured
  and the run should not report success. It now names the server URL alongside
  the agent flags, so a machine we cannot detect still learns how to connect.

The launch target table keeps its `mcpAgent` entries for those two agents: the
call is what surfaces the URL, and dropping it would trade a false claim for
silence.
2026-08-31 16:18:16 -05:00
Max Loffgren bf3a02b8a7 fix(cli): keep the unwritten agents reported when a writer fails
Failing the run on a partial install swallowed the tail of `--agent all`: the
throw left before the Hermes and OpenClaw lines printed, and their URL is the
whole answer for those two, so the run that most needed to mention them was the
one that did not. The report now runs whether or not the writers threw.

The exit code is unchanged. Configuring what it can and then reporting failure
is the same contract as before, since per-agent isolation is about the other
agents still being configured, not about the command claiming success.
2026-08-31 16:18:16 -05:00
Max Loffgren d7cef7ee73 fix(cli): never leave a rule without the server it points at
A rule tells an agent to prefer firecrawl_search and firecrawl_scrape. It was
written even when the server entry for that agent had failed, so an agent could
be instructed to reach for tools it had no way to call. The rule now depends on
the entry landing. The dependency runs one way only: a failed rule still leaves
a working MCP server, which is what keeps the two writes separate.

A run that configured some of the chosen agents also exited zero, so a script
could not tell a partial install from a complete one. The summary already named
what failed; the exit code now agrees with it. Quiet mode is unchanged, since it
runs inside init and launch, which report their own outcome and continue.

Also ignores .cursor/, alongside the editor directories already listed. Nothing
from it was ever tracked, but it is the same kind of workspace artifact a broad
`git add` swept into the repo once already.
2026-08-31 16:18:16 -05:00
Max Loffgren b11d3c77de fix(cli): check the auth modes before any early return, drop the unused yaml dep
Two defects reported against the scope change:

* The URL-only branch returns before the writers run, and the mutual-exclusion
  check lived with the writers, so `--hermes --oauth --keyless` printed a
  sign-in URL and exited zero while `--cursor --oauth --keyless` was rejected.
  The check now runs at the top of installMcp, ahead of everything that reports
  or returns.
* Removing the Hermes writer took the last import of the yaml package with it.
  Nothing under src/ imports it now, so it is dropped from the manifest and the
  lockfile rather than shipped unused.

A third finding, that a test leaks FIRECRAWL_API_KEY into later tests, does not
hold: beforeEach deletes that variable and afterEach restores the original, so
each test starts without it. The assignment is also load-bearing where it is,
since the assertion it supports is that sign-in drops a key that was present.
2026-08-31 16:18:16 -05:00
Max Loffgren 9d65072bd4 refactor(cli): write MCP config only for agents that share one contract
setup mcp promises a global entry in a file we can parse, never a literal key,
plus an optional rule file we own. Claude Code, Cursor, VS Code, and Codex meet
that as registry data: the only difference is the file format. OpenCode is the
same kind of write with its own map key and header form, so it stays.

Hermes and OpenClaw were each a second product. Hermes needed a YAML writer, an
owner-only create mode, an extra entry field for sign-in, and had no global
rule file to own. OpenClaw needed a subprocess, a JSON5 config we cannot parse,
a workspace probe, a duplicate credential builder, and a rule that could only
be written if its AGENTS.md already existed. Neither shaped the product; both
shaped the code around them.

They stay supported and stop being written. `--hermes`, `--openclaw`, and their
names on --agent print the server URL and succeed, the way an agent we install
skills for but write no MCP config for already did. Skills and firecrawl launch
are untouched, and the URL still works for both.

Removing the two takes the whole launcher concept with them: the subprocess
runner and its Windows argv escaping, the second credential path, launcher
detection, the YAML writer, and the target/client split that existed only
because launchers were not clients. Five agents, one contract, one writer.
2026-08-31 16:18:16 -05:00
Max Loffgren 47e116bcc8 fix(cli): keep credentials off the sign-in endpoint
installOpenClawMcp read the stored key whenever keyless was not also set, so
calling it with sign-in alone produced an OAuth entry carrying an Authorization
header. OpenClaw ignores a static header once auth is oauth, so the result was
inert rather than harmful, but writing credential configuration into a sign-in
entry is wrong either way.

The CLI never reached this: setup derives keyless from the auth mode, and
sign-in is not env, so the header was already dropped. The helper is exported
though, and the credential helper beside it is written to be safe in isolation
for the same reason, so sign-in now drops the key in the config builder where
every caller passes through.
2026-08-31 16:18:16 -05:00
Max Loffgren 52e9843d5b fix(cli): keep user content and report rules accurately
Four defects reported against the sign-in commit, each confirmed against the
code before changing it:

* Replacing an empty `mcp_servers:` key dropped an inline comment sitting on
  it. The comment belongs to the null value being replaced; a block map has no
  inline slot on its key, so it now moves to the head of the section instead of
  disappearing with the node it was attached to. Preserved and relocated beats
  silently deleted.
* A marker-fenced rule section was written with LF regardless of the file it
  joined, so updating a CRLF AGENTS.md left mixed endings. The section now
  adopts the line endings of the file it is written into, which is how the
  JSON, TOML, and YAML writers already behave.
* The OpenClaw workspace was resolved from the environment and the documented
  defaults alone, so a workspace moved through config took the rule to a path
  the launcher never reads. OpenClaw is now asked where its workspace is, since
  its config is JSON5 and out of reach; the previous resolution stays as the
  fallback for when the CLI cannot answer.
* Declining rules reported OpenClaw as not supporting them, which stopped being
  true when it gained a rule. It reports skipped now, and unsupported is left
  for agents that genuinely have nowhere to put one.
2026-08-31 16:18:16 -05:00
Max Loffgren 5a90be096b feat(cli): add the browser sign-in lane to setup mcp
Setup could carry a key or run anonymously, but not sign in, so the one path
the docs present to a person at a terminal had no command behind it.
`--oauth` writes the sign-in endpoint instead of a credential, and each agent
starts the browser flow itself the first time it connects.

Sign-in is a different server URL rather than a different header, so it
replaces the credential rather than travelling beside it: an exported key is
ignored under --oauth and no Authorization header is written. The two are
separate endpoints, so combining --oauth with --keyless is rejected instead of
silently preferring one.

A URL alone is not enough for every agent. Hermes starts the flow only when the
entry carries `auth: oauth`, and OpenClaw ignores a static Authorization header
unless `auth: "oauth"` is set, which is also what gates its login command.
Those fields live beside the agent in the registry, next to the environment
syntax, and an agent without a verified sign-in shape gets no entry rather than
one that reports success and then exposes no tools.

No agent signs in during setup, and each one starts the flow differently, so
the summary prints the step per agent rather than one footer: `/mcp` in Claude
Code, `codex mcp login firecrawl`, Cursor Settings, and a browser on first use
for the rest.
2026-08-31 16:18:16 -05:00
Max Loffgren 2a97b76edc feat(cli): install the Firecrawl rule for OpenClaw
OpenClaw was reported as having no rules mechanism. It has one: the workspace
AGENTS.md is a bootstrap file that OpenClaw injects into the system prompt on
every turn, treats as instruction context, and passes down to sub-agent
sessions. It is workspace-level rather than project-level, so it is reachable
from a global setup, and the rule now goes there fenced by markers like any
file the user also writes to.

The workspace can move, so the path follows OPENCLAW_WORKSPACE_DIR and the
profile suffix before falling back to ~/.openclaw/workspace. An explicit
agents.defaults.workspace in the config wins over both, but that file is JSON5
and out of reach here; landing the rule in an unused workspace is inert, unlike
a misplaced server entry.

The rule is only written when that AGENTS.md already exists. OpenClaw seeds the
file with its own instructions on first run, and creating it first would leave
the user with our section instead of those.

Writing a rule and registering the server are separate concerns, so a launcher
can take one without giving up its own MCP registration. Hermes still gets no
rule: it reads AGENTS.md from the working directory and deliberately ignores
one in $HOME, and its only global context file is SOUL.md, which is the user's
agent identity rather than a place for tool routing.
2026-08-31 16:17:58 -05:00
Max Loffgren 9c2e06ae1b fix(cli): preserve YAML file details the document tree drops
Two defects in the new Hermes writer, both reported against the last commit:

* A server key with nothing under it, `mcp_servers:` on its own, parses as a
  null scalar. Setting a path through that refuses to descend, so setup failed
  on a valid config. The key has to be replaced with a collection node first;
  assigning a plain object raises the same error one level down, because the
  value is stored as-is rather than converted.
* Serialising the document tree drops a leading byte order mark and rewrites
  every line with LF. Both belong to the user's file, so they are captured from
  the input and restored on write, which is how the JSON and TOML writers
  already treat them.
2026-08-31 16:17:39 -05:00
Max Loffgren 09675bf321 feat(cli): configure Hermes Agent through the shared MCP engine
Hermes was classified as a launcher, which implied it owns its MCP config and
has to be shelled out to. It does not: it reads plain YAML from
~/.hermes/config.yaml under `mcp_servers`, with `url` and a `headers` mapping
for an HTTP server, and it expands `${VAR}` in any string value in a server
entry. All of that is documented by Nous Research and matches what we already
emit, so Hermes is a config-file client and now goes through the same engine as
the editors.

That fixes a real defect. The old writer round-tripped the file through
parse/stringify, so a hand-written config.yaml came back with every comment,
inline note, and blank line removed. Edits now go through the YAML document
tree, which keeps comments, key order, and formatting, and a file that does not
parse is reported as a per-agent failure instead of being rewritten. A config we
create is owner-only; one the user already has keeps the permissions they gave
it, rather than being chmod-ed on every run.

Hermes gets no rules. It reads AGENTS.md from the project directory and setup
only ever writes global config, so there is no global rule file to own, and the
summary says so rather than implying one was written.

OpenClaw stays a launcher, and the type now says why. Its config is JSON5, which
the JSONC editor we patch JSON with cannot read, so writing that file directly
would either corrupt it or refuse a valid config. `openclaw mcp set` is the
vendor-documented path and normalises the entry on the way in.
2026-08-31 16:17:39 -05:00
Max Loffgren 4e2fda2e20 fix(cli): keep an escaped quote from ending a TOML multi-line string
The scanner added in the previous commit closed a multi-line basic string at
the first `"""` it found, but a basic string honours backslash escapes, so
`\"""` is an escaped quote followed by two literal ones rather than the
terminator. Ending the string there marked the rest of it as ordinary config,
which is how a `[table]` written inside a string becomes a header the editor
will replace, taking the user's content with it.

Fence candidates are now skipped while the backslash run before them is odd.
An even run is a real terminator, since backslashes escape each other. Literal
strings are unaffected because they have no escapes at all.
2026-08-31 16:17:14 -05:00
Max Loffgren aed6b663ff fix(cli): close the remaining review findings on MCP setup
Four defects, each reported against this branch and each confirmed against
the code before changing it:

* `resolveMcpClientId` read `__proto__` and `constructor` off the alias
  object's prototype and returned something truthy, so those two names
  resolved to a bogus agent and crashed the run instead of being rejected
  as unknown. `toString` and the rest are already rejected because
  lowercasing them stops matching an inherited key.
* VS Code was detected only through `Code/User`, which is created on first
  launch. The picker offers detected agents alone, so an install that had
  not been launched was invisible rather than merely unselected. Detection
  now uses the same two markers doctor already uses.
* The TOML editor scanned lines for table headers with no awareness of
  multi-line strings, so a `[table]` written inside one was treated as a
  real header and the surrounding edit could corrupt the file. The scan
  now tracks string state, and a multi-line string left open at the end of
  the file is reported as a per-agent failure instead of being appended to,
  which matches how the JSON path already treats a config it cannot parse.
* `hasFirecrawlMcpEntry` matched `servers` and `mcp` at any depth, so an
  unrelated nested object holding a `firecrawl` property made doctor report
  the server as registered. Those two keys only ever sit at the root of the
  configs that are scanned; `mcpServers` keeps matching at any depth
  because Claude Code nests a per-project map under `projects`.

Full TOML validation is deliberately not attempted. It would need a parser
dependency, and a line-based validator would reject valid configs, since a
TOML array may legally span several lines.
2026-08-31 16:17:14 -05:00
Max Loffgren 0c999bd142 fix(cli): one stored-key rule, accurate agent aliases, no repo config artifacts
Three Firecrawl MCP configs were tracked at the repository root. Tests wrote
them into the working directory and a broad `git add` swept them in, so anyone
opening this checkout inherited MCP servers from the repo. Removed, and every
setup test now runs in its own working directory so project-relative writes
cannot reach the repository again.

A stored key behaved three different ways depending on how the target was
named: keyless for the boolean flags, keyless for `--agent hermes`, and a hard
abort for `--agent all`. The README documents keyless. `--agent all` now agrees
with the rest, and the launchers report through the same summary so the keyless
fallback is stated rather than implied by a bare installer log line.

`--agent launchers` was a synonym for every agent plus both launchers, which is
not what the name says. It now selects the launchers.

Launcher dispatch is exhaustive rather than treating anything that is not
Hermes as OpenClaw, and doctor recognises OpenCode's top-level `mcp` map, which
it previously reported as unregistered right after setup wrote it.
2026-08-31 16:17:14 -05:00
Max Loffgren 25850719da fix(cli): keep MCP setup global-only and keyless for stored launcher keys
Project scope fought the one-command-every-agent goal, and --agent hermes/openclaw aborted on a stored key while the boolean flags wrote keyless config.
2026-08-31 16:17:14 -05:00
Max Loffgren cef79d6449 fix(cli): keep skills-only agents from failing setup, and settle the scope flag
`firecrawl setup --yes --agent windsurf` installed skills and then aborted,
because MCP setup rejected a name it writes no config for. An agent we support
for skills but not for MCP is not an error: the run now finishes, skips the MCP
step, and prints the server URL so the user can wire it up themselves. A name
nothing supports is still rejected, so a typo does not silently do nothing.

Scope: global is the intended default, so that one command reaches every agent
surface rather than the current checkout alone. `--project` is the only scope
flag that means anything on setup. `-g` is accepted for existing scripts but
hidden from help and reported as deprecated when used, and the mutually
exclusive scope error it existed for is gone. `-g` is untouched on init and
launch.

Tests also pin USERPROFILE and APPDATA alongside HOME. os.homedir() reads
USERPROFILE on Windows, so the sandbox that keeps a test run away from the
developer's own agent config was doing nothing there.
2026-08-31 16:17:14 -05:00
Max Loffgren 0a93728d59 fix(cli): correct config-writing edge cases found in review
Six defects, four of them silent:

* Quiet mode returned before the total-failure check, so a run in which
  nothing was written resolved successfully. `firecrawl init` and
  `firecrawl launch` both use quiet mode and reported success regardless.
* The TOML writer split on "\n" only, so a config.toml with CRLF endings
  never matched its existing table and gained a duplicate one, leaving the
  file invalid and taking the rest of the user's Codex config with it.
* The TOML writer absorbed comment and blank lines directly above the next
  table into the replaced range and deleted them.
* A leading byte order mark was reported as a parse error even though the
  document parses, so a config written by a Windows editor was refused.
* The rule fence required "\n" after its marker, so a file converted to
  CRLF gained a second copy of the section instead of an updated one.
* `--agent all` reached only detected clients. It means every client, which
  is what the installer it replaced did.

The fence replacement now uses a function so nothing in the rule body can
be read as a replacement pattern. Line endings and byte order marks are
preserved on write rather than normalised away.
2026-08-31 16:17:14 -05:00
Max Loffgren 7214d42d6d feat(cli): drop Windsurf from MCP setup
Windsurf's remote entry shape is not settled: sources disagree on whether a
transport field is required and what its value should be, and one reports
streamable HTTP working only through a local proxy. A wrong entry does not
error, it reports success and then exposes no tools, so this stays out until
the shape can be confirmed against a live install.

With every supported agent now carrying a verified environment-reference
syntax and project-level config, the keyless-fallback and global-fallback
branches no longer have a case. Removing them rather than leaving unreachable
logic behind; they come back with the agent that needs them.
2026-08-31 16:17:14 -05:00
Max Loffgren 2c61608de7 feat(cli): drop Zed from MCP setup and show every agent in the picker
Zed's native remote MCP support is version-gated and its handling of
request headers is inconsistent across releases, so a written entry can
report success while the agent never connects. That reads as Firecrawl
being broken, which is worse than not offering the agent at all. Removing
it until the shape can be confirmed against a live install.

Also pins the picker page size to the number of agents. The default was
smaller than the list, so the last agent scrolled out of view.
2026-08-31 16:17:14 -05:00
Max Loffgren a07d01c0fb fix(cli): detect Hermes Agent by config directory only
The PATH lookup matched any executable named `hermes`, including an
unrelated JavaScript engine that ships with common toolchains, so the
picker pre-selected an agent the user did not have.

Detection now prefers a false negative to a false positive: every agent is
listed either way, so missing one costs a keystroke while pre-selecting a
missing one is misleading.

Also pins HOME and PATH for setup tests. Both feed agent detection, so
leaving the real ones visible made results depend on what happened to be
installed on the machine running the suite.
2026-08-31 16:17:14 -05:00
Max Loffgren 69b22103fb feat(cli): configure MCP natively across coding agents
Replace the subprocess installer in `setup mcp` with a built-in one that
detects installed agents, pre-selects them in a picker, and offers to install
rules telling those agents to prefer Firecrawl for web search and scraping.

Covers Claude Code, Cursor, VS Code, Codex, OpenCode, Windsurf, Zed, Hermes
Agent, and OpenClaw through per-agent flags, `--project` for project scope, and
`--rules` / `--no-rules` for scripted runs. `-y` stays MCP-only. The two
launchers were previously reachable only by flag, so a plain `setup mcp` never
offered them; they now sit in the picker alongside the editors, and because a
launcher shells out to a CLI, a missing binary is reported against that one
agent instead of ending the run.

Credential handling is unchanged in principle and stricter in reach: an API key
is never written as a literal. Knowing which agents were selected means each one
receives a reference to FIRECRAWL_API_KEY in the syntax it expands, so the setup
no longer has to refuse a run that omits --agent. Agents with no verified syntax
fall back to the keyless endpoint and say so rather than persisting a secret.

Config edits are surgical. JSON is patched through a JSONC-aware editor so
commented settings files parse at all and keep their comments, and TOML tables
are replaced along with any stale sub-tables left by a previous stdio entry.
Reruns are byte-identical.

Also gives every setup test a throwaway HOME and resets spawn mocks between
tests, since MCP setup now writes real config files and would otherwise rewrite
the developer's own agent settings; and teaches doctor about the `servers` and
`context_servers` keys so those registrations are recognized.
2026-08-31 16:17:14 -05:00
Karan Lokchandani 6a0fac5006 fix(developer): deduplicate URLs and clean separators 2026-08-26 10:59:32 +05:30
Eric Ciarla 4867dc9358 Fix skill-scoped installs installing the whole catalog
skills add --all means "ALL skills to all agents" and silently overrides
any --skill filter, so every scoped install (setup core/build/workflows,
setup <skill>, and init's selections) was installing all 33 catalog
skills. Scoped installs now pass --yes instead of --all: same promptless
install to every detected agent, but the --skill list is respected.

Verified: setup developer-index installs 1 skill, setup build installs 5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 11:06:25 -04:00
Eric Ciarla 3538f0d8a9 Fix CI: remove stray package-lock.json, reformat setup.ts with pinned prettier
package-lock.json was accidentally committed in #206 (an npm install
artifact — this repo is pnpm-managed). bun prefers it over pnpm-lock.yaml
in release-binaries, and its migration fails under --frozen-lockfile.
Remove it and gitignore it.

setup.ts was formatted by prettier 3.9.6 from that same npm node_modules;
CI pins 3.7.4 via pnpm-lock.yaml. Reformatted with 3.7.4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 10:56:31 -04:00
Eric Ciarla c110d2e610 Offer auth after skill installs without blocking automation
After any setup skill install, check for an API key (env var or stored
credentials). Authenticated users see nothing. Otherwise: --browser runs
the browser login directly, an interactive terminal offers it (default
yes), and non-interactive runs print a one-line hint — never a prompt,
so agents, scripts, and CI are unaffected. The bundle flow defers the
offer until every step has run so it never asks twice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 10:31:35 -04:00
Eric Ciarla d2f78c3f56 Accept a single catalog skill name as a setup subcommand
firecrawl setup developer-index   # firecrawl- prefix optional
firecrawl setup firecrawl-scrape

Group subcommands keep precedence, so `setup build` means the group and
`setup firecrawl-build` reaches the skill.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 10:31:35 -04:00
Eric Ciarla f6d5dcee88 Add setup core/build group subcommands
One short command per skill family:

  firecrawl setup core       # primitives + index skills ("skills" stays as alias)
  firecrawl setup build      # app-integration skills (previously not installable via setup)
  firecrawl setup workflows  # unchanged

The build family had no install path since init dropped it; BUILD_SKILLS
mirrors the five build skills in the catalog. All groups install from the
catalog by name, so skills.sh counters stay consolidated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 10:31:35 -04:00
Eric Ciarla f0adf5300b Point catalog sync at skills/core/
The catalog directory is being renamed from skills/cli/ to skills/core/ —
the family covers the Firecrawl primitives via CLI or MCP, not just the
CLI. Install selection is name-based, so slugs are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 08:51:11 -04:00
Chenxin Yan 285fb952c7 fix(setup): filter the no-npx native fallback to the selected skills
Review fixes for #203:
- setup's no-npx fallback called installSkillsNative(repo) without the
  name filter, installing the entire catalog (build skills included)
  instead of the selection every other install path uses
- drop the nonexistent --skills-only CLI flag from the developer-index
  skill (skills="only" stays HTTP/MCP-only per cli-argv contract)
- note that the workflow retry hint reinstalls all workflow skills, not
  the picked subset
- fix 'sturctural' typo in the research-index skill
2026-08-21 08:31:40 -04:00
Chenxin Yan 1603c24375 feat(skills): adopt the research/developer index skills as CLI skills
They teach the firecrawl research and firecrawl developer CLI commands,
so this repo is their code home. Authored here, mirrored to the catalog
under skills/cli/, and included in the default init/setup CLI set.
2026-08-21 08:31:40 -04:00
Chenxin Yan a4e7920aad fix(setup): align setup skills/workflows with init's catalog selections
The init retry hints point at setup skills/workflows, but those still
installed from the legacy repo list (including build skills init no
longer installs). Share the catalog selections between init and setup
so a retry reinstalls exactly what init attempted. Harden the sync
workflow: least-privilege token, SHA-pinned checkout.
2026-08-21 08:31:40 -04:00