* feat(mcp): compact server instructions in both eras + MCP-only help tool (#1833)
MCP-only clients got no workflow guidance: server/discover carried two
sentences, legacy initialize carried nothing, and the CLI guides
(agent-device --help, help <topic>) were unreachable over MCP.
- MCP_SERVER_INSTRUCTIONS: one MCP-phrased workflow card (<2 KB, the
Claude Code truncation limit) returned by server/discover and legacy
initialize alike.
- help tool, router-owned (not a command descriptor): no topic -> the CLI
decision card; topic -> agent-device help <topic|command> text, prefixed
with the one-line CLI->tool-property mapping; unknown topic -> isError
listing the topics. listCommandTools() stays descriptor-only for the AI
SDK; the router composes descriptors + help.
- Move src/cli/parser/cli-help{,-overview}.ts to src/cli-schema/ so
src/mcp (rank 3) can import the renderers without a layering back-edge
into src/cli (rank 6).
* fix(mcp): name terminal-only commands in help guides; colocate cli-help tests with their sources
- The MCP guide preamble claimed every `agent-device <command>` line is a
tool of that name; `help web` tells the reader to run `web setup` /
`web doctor` and no `web` tool exists. The preamble now lists the exact
CLI-only set (listCliCommandNames minus listMcpExposedCommandNames) —
derived, not scanned out of prose where `device`/`web` are ordinary
words. Regression: help web names `web` as terminal-only, and the listed
set equals the registry difference.
- cli-help-*.test.ts move from src/cli/parser/__tests__ to src/cli-schema/
to mirror the moved sources.
* perf(mcp): tighten the guide card, tool description, and preamble
Instructions card 1572 -> 1378 bytes (paid every session), tool
description and preamble trimmed, HELP_TOOL built once as a const.
Bundle delta vs main 3189 -> 2715 bytes; the remainder is the guide text
itself, which the bundle carried in no MCP-phrased form before.
* feat(bench): renderer-pinned samples, topic-coverage gate, error quizzes; trim skillgym to agentic checks
The help conformance bench's quoted CLI output is now sourced from
scripts/help-conformance-sample-outputs.mjs, and every sample is rebuilt
through the real production renderers (settle output formatters,
printHumanError, formatSnapshotText, refMutationAdmissionResponse) by
scripts/__tests__/help-conformance-sample-outputs.test.ts — a rendering or
message change fails deterministically instead of leaving the bench grading
against output the CLI no longer prints. This retires the fabricated
recoverable-failure envelope (production never throws a textual settle
timeout; that case is replaced by a real DEVICE_IN_USE recovery quiz).
Bench cases move to scripts/help-conformance-cases.mjs and are enumerated
against the help-topic registry: helpTopicIds() is exported from cli-help,
and scripts/__tests__/help-conformance-topic-coverage.test.ts fails when a
help topic has neither a bench case nor an explicit waiver. New case
families: error-envelope recovery quizzes (device-in-use, stale pinned ref,
ambiguous find match, app-not-installed) pinned to real error text, topic
coverage for tv/web/react-native/debugging/workflow, and a metamorphic twin
of the settled-diff quiz.
The skillgym smoke suite shrinks from 119 cases to the 5 that measure what
only an agentic runner can show: skill routing plus output interpretation
with a proven local CLI help probe (local-cli-help-policy). Its embedded
samples now import the same pinned constants, replacing hand-transcribed
output that had already drifted from the renderer. Knowledge checks belong
to the bench; live fixture behavior belongs to the iOS simulator e2e suite.
* review: drive error samples through the real producers; enforce local-help on the routing smoke
The DEVICE_IN_USE, AMBIGUOUS_MATCH, and APP_NOT_INSTALLED parity tests no
longer hand-author the producer message before rendering: each drives the
actual producer — buildDeviceInUseBySessionError (extracted in
session-open.ts and called by the handler), buildAmbiguousMatchError (now
exported from find.ts), and buildAppNotInstalledError (extracted in
app-resolution.ts and thrown by the resolver). Because each factory is
exported from its producer file and called by the production path, dropping
the production call would make it test-only and fail
check:production-exports — the wiring is gate-enforced, not conventional.
open-and-snapshot now sets requireLocalCliHelp and
allowOnlyLocalCliHelpCommands, so the 'skill plus local help' claim is
observed rather than assumed; without them the case can pass on model prior
alone.
Michał Pierzchała