Files
software-mansion__argent/packages/tool-server/test/tool-input-schema-contract.test.ts
filip131311 6177409c7c fix(tools): drop top-level JSON Schema combinators from tool input schemas (#782)
Fixes #773.

## The bug

`gesture-rotate` (`anyOf`), `flow-execute` and `flow-read-prerequisite`
(`oneOf`) hand-write an `inputSchema` that spreads a top-level
combinator onto the derived schema. The Anthropic Messages API rejects
that outright:

```
tools.N.custom.input_schema: input_schema does not support oneOf, allOf, or anyOf at the top level
```

The 400 fails the **whole request — every tool in it** — so any client
forwarding our schemas verbatim as `custom` tools is bricked entirely,
not just for these three tools. Reported from opencode.

## Why we never hit it

**Claude Code silently repairs it before sending.** Verified live: it
strips the combinator and re-expresses the rule as a prose line
prepended to the description. All of our E2E/QA runs through Claude
Code, so the defect was invisible from inside our own test loop.

Secondary reasons: the code is recent (#546, 2026-07-24 for
`gesture-rotate`; #568, 2026-08-05 for the two flow tools — before late
July no argent tool had a top-level combinator), and nothing in CI
asserted the shape of any tool's `inputSchema`.

## Why removing it is safe

The combinators never enforced anything. Nothing in argent validates
against the JSON Schema — the zod `.refine()`/`.superRefine()` does all
the rejecting, on every dispatch path (HTTP 400, MCP `isError`, CLI flag
error, internal registry dispatch), and always did. #546's stated
rationale was "to keep invalid calls rejected at the schema layer", but
that layer does not exist; the only JSON Schema evaluator in the repo
was a 5-line helper inside that PR's own test.

Verified equivalence before deleting: the removed `anyOf` accepts
exactly the same 8 radius shapes as the two refinements, and `oneOf` the
same 4 source combinations as the superRefine.

## What changed

- The three hand-written `inputSchema` consts are gone. No tool
hand-writes one now, so `Registry.registerTool` is the single emission
point for all 76 schemas.
- Each rule moves into the tool description, **worded differently per
tool** — `tool-description-quality.yml` triggers on these paths and
scores `disambiguation_score` (W=1.5), so identical prose on two tools
would work against the gate.
- `ToolDefinition.inputSchema` documents why it should stay underived.
- New per-tool contract test over the whole catalog: no top-level
`oneOf`/`allOf`/`anyOf` (API-rejected) and no top-level
`not`/`if`/`then`/`else`/`$ref`/`$schema` (blocked pre-emptively — the
natural way to re-encode a rejected combinator). Nested combinators stay
legal and in use: `tv-remote`'s `button` is a `z.union`.
- Catalog walk + expected count extracted to `test/helpers/catalog.ts`,
shared with `interaction-messages` so a new tool needs the count bumped
once, not twice.
- Six existing tests read `.inputSchema` off unregistered definitions
(populated only at registration) and now derive it.

## Not done by design

**No throw in `registerTool`.** `createRegistry()` runs at tool-server
bootstrap (`index.ts:182`), where a throw becomes an uncaught exception
→ `crashShutdown` → `exit 1`: a total outage, and `update-argent` is
itself a tool so it could not run to repair it. The catalog is static
and compiled in, so the contract test catches the same regressions
before merge — and it also covers `paste` and the Lens tools, which
never pass through `registerTool` at all.

**Deliberately a denylist, not an allowlist** of permitted top-level
keys: `additionalProperties` legitimately appears on schemas derived
from `.strict()` objects (`screenshot-diff`,
`stop-all-simulator-servers`).

## Follow-up left out

`gesture-rotate`'s refinements carry `path: []`, and
`deriveInvalidParams` drops empty-path issues — so every cross-field
rejection is telemetered with an empty `invalid_params`. Fixing it
changes CLI wording and starts populating that field, so it is not
riding in this PR.

## Verification

- 75 registered tools probed live: **zero** top-level combinators; the
three now advertise `type, properties, required` only.
- Negative test: re-introducing an `anyOf` fails the gate with
`gesture-rotate declares top-level anyOf`.
- `packages/registry` 90/90 pass; `packages/tool-server` 3952 pass with
**7 pre-existing failures** in `bind-failure-telemetry`,
`lens-relay-telemetry`, `startup-telemetry` and
`tool-fail-invalid-params-emission` — all failing identically on
pristine `fe5976857` (they die in `attachRegistryEventLogger`, unrelated
to schemas).
- Build, eslint, prettier clean; knip identical to baseline (60 unused
exports / 187 unused exported types both before and after).

Reaches users on the next npm publish; current version is 0.20.0.
2026-08-13 19:40:32 +02:00

43 lines
1.9 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { createRegistry } from "../src/utils/setup-registry";
import {
advertisedSchema,
definitionsById,
CLIENT_UNSAFE_TOP_LEVEL_KEYWORDS,
EXPECTED_TOOL_COUNT,
} from "./helpers/catalog";
// Three tools once shipped a hand-written inputSchema that spread a top-level
// oneOf/anyOf onto the derived schema, to express a cross-field rule zod cannot
// represent in JSON Schema. The Anthropic Messages API rejects that with a 400
// that fails the whole request — every tool in it — so the tools were unusable
// through any client forwarding our schemas verbatim (issue #773). Claude Code
// happens to strip the keyword before sending, which is why our own QA never
// saw it; this test is what does see it.
//
// This is a denylist, deliberately not an allowlist of permitted top-level
// keys: `additionalProperties` legitimately appears at the top level on the
// schemas derived from `.strict()` objects (screenshot-diff,
// stop-all-simulator-servers), and `required` is absent entirely on a schema
// with no required fields.
describe("advertised tool input schemas", () => {
const definitions = definitionsById(createRegistry());
it("covers the whole catalog", () => {
expect(definitions.size).toBe(EXPECTED_TOOL_COUNT);
});
for (const [id, definition] of definitions) {
it(`${id}: is a plain object schema with no top-level combinator`, () => {
const schema = advertisedSchema(definition);
expect(schema, `${id} declares neither zodSchema nor inputSchema`).not.toBeNull();
const offending = CLIENT_UNSAFE_TOP_LEVEL_KEYWORDS.filter((keyword) => keyword in schema!);
expect(offending, `${id} declares top-level ${offending.join("/")}`).toEqual([]);
expect(schema!.type, `${id}.type`).toBe("object");
expect(typeof schema!.properties, `${id}.properties`).toBe("object");
});
}
});