Two things this PR was missing, both now closed.
## The Claude SDK quickstarts are unblocked
#6618 put `showcase/integrations/claude-sdk-{python,typescript}` on
`createCopilotRuntimeHandler` with `mode: "single-route"`, at the **plain**
`route.ts` path. That dissolves the coupling that forced these two pages to be
reverted earlier: `verify-shell-docs.ts` asserts each page claims a starter file
at `src/app/api/copilotkit/route.ts` AND that the file exists in the extracted
starter. Single-route keeps that path, so the prose claims and
`requiredStarterFiles` are unchanged — only the fence bodies move to v2.
The two content assertions that pinned those pages to v1
(`ExperimentalEmptyAdapter`, `copilotRuntimeNextJSAppRouterEndpoint`) now
require `createCopilotRuntimeHandler`, the `/v2` entrypoint,
`mode: "single-route"` and a `POST` export. Mutation-checked: flipping the
fixture to `mode: "multi-route"` fails with
`app/api/copilotkit/route.ts missing single-route mode`.
## Snippet gating: 1 -> 20 route fences
I previously claimed the integration pages could not be doctested because of
path aliases and per-integration deps. **That was an assumption I never
checked, and it was wrong.** Of the 52 migrated route fences, 47 import nothing
project-relative; 36 are complete, self-standing routes. 27 pages were
eligible, 20 now hold a gated fence — each extracted and typechecked by
`tsc --noEmit` against real npm-installed packages in CI.
One fence per (page, title): `extract.ts` concatenates tagged blocks sharing a
title, so a second complete route on the same page would collide.
Mutation-checked on `snippets/integrations/langsmith/index.mdx`: restoring the
v1 import in the gated fence turns the run red (20 passed, 1 failed). My first
attempt at this check was a no-op — the pattern missed because the fence is
JSX-indented — and it "passed" misleadingly. The real check asserts the mutation
reached the extracted snippet before trusting the result.
### Harness changes this needed
- `extract.ts` now finds the nearest `doctest.json` by walking up to the docs
root, instead of looking only in the page's own directory. Otherwise gating
20 pages means ~20 duplicated dependency lists that then drift. A shared list
lives at `content/doctest.json`; `docs/integrations/langgraph/` keeps its own
(Python deps) and now also carries the TS deps its page needs.
- `run.ts` installs each dependency set **once**, into
`.doctest-output/.deps/<hash>`, and links it into every snippet sharing that
set. Per-snippet installs took **7:58** for 21 snippets, uncomfortably close
to the job's 15-minute timeout; shared installs take **0:45** cold. Different
dep sets still get separate stores, so this is a dedupe, not a merge.
### `@ag-ui/*` versions have to be pinned to what the runtime expects
Unpinned, the gated fences failed with `HttpAgent is not assignable to
AbstractAgent — separate declarations of a private property '_debug'`: npm
installs a newer `@ag-ui/client` than `@copilotkit/runtime` depends on, so two
`AbstractAgent` declarations collide. The sidecar pins `@ag-ui/client@0.0.57` and
`@ag-ui/core@0.0.57` to match `@copilotkit/runtime@1.68.3`.
## Seven fences are deliberately NOT gated
Un-tagged with the reason, rather than left failing or quietly dropped:
- `docs/auth.mdx`, `docs/premium/connect-your-runtime.mdx` — illustrative
fences referencing placeholders (`myAgent`, `verifyJwt`) that cannot compile
standalone by design.
- the four langgraph-family pages and
`snippets/self-hosting-copilot-runtime-langgraph-endpoint.mdx` — these hit
`LangGraphAgent is not assignable to AbstractAgent — separate declarations of
a private property '_debug'`, which pinning does not fix.
**That last one is a real pre-existing defect, not a migration regression.** I
reconstructed the v1 form of the langgraph quickstart snippet verbatim from
`origin/main` and typechecked it against the identical installed dependencies:
it fails with the same error. So these snippets have never typechecked against
published packages — worth filing separately. It is also what the ~220
`@ts-ignore` comments across `showcase/integrations` were papering over.
## Verified
doc-tests (cold, no cache) -> 21 passed, 0 failed in 0:45
mutation check (real, verified) -> 20 passed, 1 failed
vitest extract + verify-shell-docs -> 34 passed
showcase/shell-docs typecheck -> exit 0
showcase/shell-docs build -> exit 0
structural audit -> 21/21 pages, fence + JSX identical to HEAD
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`snippets/copilot-runtime.mdx` calls itself "the recommended way to use
CopilotKit" and taught the v1 plain-POST route. A reader following it got an
endpoint that answers 405 on `GET /api/copilotkit` and does not route
`GET /api/copilotkit/info` at all, so nothing could discover the runtime.
The route fence is now the shape proved in `examples/shadcn`: catch-all path,
`createCopilotEndpoint`, a runner, and both verbs exported. The three other
fences on the page that were titled with the plain route path are retitled to
the catch-all path, so no snippet claims a path its body contradicts.
Also documents the two things the shape actually requires, both of which were
found by running it:
- `npm install @copilotkit/runtime hono` — `hono` is a dependency of the
runtime rather than a peer, so `hono/vercel` resolves under npm's flat
layout but not under pnpm.
- `useSingleEndpoint={false}` on `<CopilotKit>`. The provider defaults it to
`true` (single-route transport), which posts to the bare `runtimeUrl`; a
multi-route runtime answers that with 404. The Angular provider defaults its
transport to `"auto"` and needs no equivalent change, so that snippet is
unchanged.
## The snippet is now gated in CI
`scripts/doc-tests` already extracts fences tagged `doctest=` and, for
`component`, runs `tsc --noEmit` against real npm-installed dependencies — but
it could not handle a fence whose title is a path, which is exactly what a
Next.js route handler's title is. Two defects blocked it, both fixed here:
- `extract.ts` wrote the title as a path without creating intermediate
directories, so a titled route fence died on ENOENT.
- `run.ts` derived the npm package name from the snippet directory, and a
catch-all leaf directory is literally `[[...slug]]`, which npm rejects as an
invalid name. It also looked for `doctest.json` only in the snippet's own
directory, while the sidecar is copied once per page — so a nested snippet
silently installed no dependencies and failed with "Cannot find module".
With those fixed, the canonical route snippet is tagged `doctest="component"`
and typechecked in CI against the published `@copilotkit/runtime` 1.68.3 and
`hono`. This is the gate the docs did not have: snippet correctness no longer
rests on review alone.
Verified:
pnpm tsx scripts/doc-tests/extract.ts -> 2 snippets extracted
pnpm tsx scripts/doc-tests/run.ts -> 2 passed, 0 failed
mutation check: restoring the v1 import in the fence turns the run red
(1 passed, 1 failed), so the gate fails on the regression it exists to catch
vitest scripts/doc-tests/__tests__/extract.test.ts -> 10 passed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add --validate-on-load to all aimock invocations (4 workflows/scripts
+ 13 integration docker-compose files)
- Replace hardcoded 2-file fixture list with dynamic discovery across
showcase/, examples/integrations/*/, scripts/doc-tests/ (16 fixtures)
- Add sanity check to prevent silent zero-test pass when discovery fails
- Extend showcase_validate.yml path filter to trigger on
examples/integrations/**/fixtures/** and scripts/doc-tests/fixtures/**
- Import and use ValidationResult type for callback parameters
- Fix scripts/doc-tests/fixtures/default.json to use { fixtures: [...] }
envelope shape
Benjamin Taylor