\`validate-fixture-tool-surface.ts\` resolves a demo's backend tool
surface by file name (\`<agentId>.py\` or \`<agentId>_agent.py\`); the
prior plural \`tool_rendering_agents.py\` matched no agentId, so the four
tool-rendering variants had empty tool surfaces and the validator flagged
"weather" fixture drift. Split into per-variant files
(tool_rendering_agent.py, tool_rendering_default_catchall_agent.py,
tool_rendering_custom_catchall_agent.py,
tool_rendering_reasoning_chain_agent.py) that share a new
tool_rendering_common.py for the get_weather / search_flights /
query_data implementations and the system prompt. \`tools=[...]\` is now
a literal at module scope per file so the validator's regex picks it up.
Verified locally: \`pnpm exec tsx validate-fixture-tool-surface.ts\` →
65 fixtures × 534 demos, no drift.
\`validate-fixture-tool-surface.ts\` parses backend tool surfaces with
\`/tools\s*=\s*\[/\` — that regex matches \`tools=[...]\` keyword-arg
syntax, not \`"tools": [...]\` dict-key syntax. The previous \`_build\`
helper passed tools through a kwargs dict, leaving the four
tool-rendering variants with empty tool surfaces and triggering aimock
fixture drift on the "weather in Tokyo" suggestion. Inline the four
constructors so \`tools=[get_weather, search_flights, query_data]\` is
visible to the regex.
The Windows-generated starter wrote \`from .tools import\` for files inside
agent/tools/, but the Linux generator emits \`from . import\` because
\`filePath.includes("/tools/")\` only matches forward-slash paths
(generate-starters.ts:494). Match the Linux output so the drift check
passes.
Pre-existing drift on main: showcase/packages/crewai-crews/src/agent_server.py
was migrated from BaseHTTPMiddleware to a raw ASGI splice (commits 72c79bf99,
6b1ccfa86, ecbc76872, f16ea4d96) without regenerating the starter, so the
drift check was failing on every PR including this one. Re-emit via
\`pnpm exec tsx showcase/scripts/generate-starters.ts --slug crewai-crews\`.
`generate-starters.ts` regenerates each starter from its source package; the
prior commits added 18 new agent modules + refactored agent_server.py in the
google-adk package, so the starter needed re-emission. Generated via
`pnpm exec tsx showcase/scripts/generate-starters.ts --slug google-adk`.
Fixes drift-check failure on this branch.
Backend: refactor src/agent_server.py from a single shared LlmAgent at /
to one ADKAgent per demo at /<agent_name>, driven by a new
src/agents/registry.py source-of-truth (with AgentSpec dataclass for
predict_state / emit_messages_snapshot kwargs). Adds 18 per-demo agent
modules covering reasoning, tool-rendering, HITL, shared-state,
multi-agent, A2UI dynamic + fixed-schema, BYOC, open-gen-ui, and others.
Frontend: src/app/api/copilotkit/route.ts now maps each agent name to its
matching backend path. Adds 6 dedicated runtime routes for demos that
need bespoke runtime config: copilotkit-auth (V2 createCopilotRuntimeHandler
with onRequest 401 gate), copilotkit-declarative-gen-ui and
copilotkit-a2ui-fixed-schema (a2ui.injectA2UITool: false), plus
copilotkit-byoc-{hashbrown,json-render} and copilotkit-ogui (shared by
open-gen-ui + advanced).
Manifest: declares 35 features (was 8). Dashboard wired count for
google-adk goes from 8/39 to 35/39, parity_tier minimal -> partial.
Reference (langgraph-python) is at 38/39.
Skipped (intentional): gen-ui-interrupt and interrupt-headless rely on
LangGraph's interrupt() primitive with no direct ADK equivalent;
mcp-apps left for follow-up.
## Summary
- Bundle all 9 D5 (e2e-deep) fixture files into
`showcase/aimock/d5-all.json` so aimock loads them at boot via GitHub
raw URL. **Root cause of 93/158 D5 red cells**: the D5 fixtures in
`showcase/ops/fixtures/d5/` were never loaded into aimock — probes hit
real LLMs producing non-deterministic responses that fail strict
assertions.
- Add new `hitl-steps` D5 feature type + probe script for the
`/demos/hitl` route, which renders a step-selection UI
(`StepsFeedback`), not a time-picker. The previous mapping (`hitl →
hitl-text-input`) drove the wrong interaction against the wrong page.
- Update `RAILWAY.md` with the new fixture URL and updated
`startCommand` (d5-all.json loads before feature-parity.json for match
precedence).
## Post-merge deploy step
After merge, the Railway `startCommand` for `showcase-aimock` must be
updated to include the new `--fixtures` URL:
```sh
node /app/dist/cli.js \
--proxy-only \
--provider-openai https://api.openai.com \
--provider-anthropic https://api.anthropic.com \
--provider-gemini https://generativelanguage.googleapis.com \
--fixtures https://raw.githubusercontent.com/CopilotKit/CopilotKit/main/showcase/aimock/d5-all.json \
--fixtures https://raw.githubusercontent.com/CopilotKit/CopilotKit/main/showcase/aimock/smoke.json \
--fixtures https://raw.githubusercontent.com/CopilotKit/CopilotKit/main/showcase/aimock/feature-parity.json \
--validate-on-load \
--host 0.0.0.0 \
--port 4010
```
Then restart the aimock service to pick up the new fixtures.
## Test plan
- [x] 100/100 D5 tests pass (11 test files)
- [x] New hitl-steps tests: 7/7 pass
- [x] Existing hitl-text-input tests: no regression (10/10)
- [x] d5-all.json validated: 23 fixtures, correct structure
- [ ] After deploy: trigger D5 probe cycle and verify red count drops
from 93 to near-zero
## Summary
Fixes the dashboard at https://dashboard.showcase.copilotkit.ai so every
`docs-og` / `docs-shell` cell that *has* a working docs target shows
green ✓ instead of red ✗.
The previous state: `docs-status.json` flagged virtually every feature
as `notfound` because `showcase/shared/feature-registry.json` pointed at
`https://docs.copilotkit.ai/features/<id>` URLs that all soft-404
(Next.js catch-all + `noindex` meta), and `shell_docs_url` paths like
`/docs/features/<id>` that don't exist in
`shell-docs/src/content/docs/`. Per-package `docs-links.json` overrides
covered ~8 features per integration but had `null` entries for many
cells, which `feature-grid.tsx`/`cell-pieces.tsx` render as red.
## What changed
- **`showcase/shared/feature-registry.json`** — replace dead URLs with
indexable docs.copilotkit.ai pages, rename legacy `shell_docs_url` →
canonical `shell_docs_path`, leave fields unset only for features with
no real docs page yet (auth/agent-config/voice/multimodal/byoc-* and
`kind: testing` features whose docs row is hidden).
- **14 existing `showcase/packages/<framework>/docs-links.json`** — fill
in every `null` with the closest framework-specific page (probed live
with `curl` for 200 + indexable, no `x-matched-path: /[[...slug]]`
catch-all) or a framework-agnostic indexable parent. Where neither a
scoped page nor a working agnostic alternative exists, leave null with a
`missing[]` rationale (honest red ✗).
- **New `showcase/packages/langroid/docs-links.json`** — placeholder
file declaring `features: {}` plus a rationale; every langroid demo with
documentation is covered by the feature-level registry, so per-cell
overrides aren't needed.
No code changes — only JSON sources of truth. The generated bundles
(`shell-dashboard/src/data/{registry,docs-status,catalog}.json` etc.)
are gitignored and rebuilt by `pnpm -C showcase/scripts
{generate-registry,probe-docs}` on dev/CI.
## Verification
- `pnpm -C showcase/scripts probe-docs` → `ok=61 / notfound=0 /
missing=19` across 40 features × 2 links (was effectively all-`notfound`
before).
- `pnpm -C showcase/scripts generate-registry` → all 18 manifests
validate, registry + catalog regenerate cleanly for shell-dashboard,
shell-docs, and shell-dojo.
- Dashboard cell simulation (mirrors the resolver in
`feature-grid.tsx:62-78` + `cell-pieces.tsx:51-62`):
- **docs-og: 344 green / 72 red**
- **docs-shell: 343 green / 73 red**
- **Total: 687 green / 145 red docs cells**
- Remaining reds are exclusively
`auth/voice/multimodal/agent-config/byoc-hashbrown/byoc-json-render` on
integrations whose framework docs don't have those pages — i.e. honest
"not yet documented" signaling.
## Test plan
- [ ] Wait for the dashboard's next build → confirm
`dashboard.showcase.copilotkit.ai` shows green ✓ on `docs-og` and
`docs-shell` for the listed cells.
- [ ] Spot-check a few clicks: e.g. ag2/agentic-chat docs-og (→
`/ag2/prebuilt-components`), strands/tool-rendering docs-shell (→
`/strands/unselected/generative-ui/tool-rendering`).
- [ ] Confirm no manifest validation regression in CI (`pnpm -C
showcase/scripts validate-manifests`).
D5 probes hit real LLMs instead of fixtures because the 8 D5 fixture
files in showcase/ops/fixtures/d5/ were never loaded into aimock.
Railway fetches fixtures from GitHub raw URLs at boot — bundle all
D5 fixtures into d5-all.json and document the updated startCommand
with the new URL appearing before feature-parity.json for match
precedence.
## Summary
Six small, independently-verifiable shell-docs cleanup commits. All
scoped to polish — no structural reorganization.
- **Sidebar title now shows the integration name** on framework-scoped
routes. Previously every `/<framework>/*` URL hardcoded `\"CopilotKit
Docs\"` at the top of the sidebar; now `/langgraph-python/*` reads
\"LangGraph (Python)\", `/built-in-agent/*` reads \"Built-in Agent\",
`/ms-agent-dotnet/*` reads \"MS Agent Framework (.NET)\", etc. Unscoped
routes (`/`, `/quickstart`) keep the \"CopilotKit Docs\" default.
- **Event Inspector + Hook Explorer added to the Troubleshooting
sidebar.** Both pages exist on disk but weren't in
`troubleshooting/meta.json`, making them unreachable from the sidebar —
only direct-URL navigation surfaced them. Inserted between Debug Mode
and Observability Connectors so the four debugging tools cluster
together.
- **Framework selector now reflects the stored choice on unscoped
pages.** The dropdown's display label was derived from the URL-derived
`framework` only. On unscoped pages where `framework` is null, it
reverted to the \"Pick an agentic backend\" placeholder even when
localStorage held the user's prior selection — so the choice appeared to
be forgotten on every return to the root. Falls back to
`storedFramework`, matching the precedence the Clear-Selection button
below already uses.
- **Wire the orphan `multi-agent/subagents` page into the App Control
nav.** `content/docs/multi-agent/subagents.mdx` is a real page with
`snippet_cell: subagents` and a lucide icon, but had no entry in any
`meta.json` — only direct-URL navigation reached it. Slotted into App
Control next to `frontend-tools` and `shared-state`.
- **Collapse legacy `/frontend-actions` into `/frontend-tools`.**
`content/docs/frontend-actions.mdx` was a 33-line stub from when the
feature was called \"Frontend Actions\" (the API is now exclusively
`useFrontendTool`). Deleted the orphan + added a 301 redirect in
`next.config.ts` so external links still land cleanly. The HTML anchor
IDs `id=\"frontend-actions-example\"` referenced from 12 framework
`frontend-tools.mdx` files are unaffected (anchors, not URLs).
- **Drop dormant per-framework threads + self-hosting stubs.** 24
byte-identical stub files across 12 framework trees: 12 ×
`integrations/<fw>/threads.mdx` (7L each, body `<Threads />`, all md5
`0371508d…`) and 12 × `integrations/<fw>/premium/self-hosting.mdx` (8L
each, body `<SelfHosting />`, all md5 `06d17f98…`). The
`buildFrameworkOverridesNav` filter already dropped them from the merged
sidebar (root wins), and the framework-scoped router falls back to root
MDX when no per-framework override exists — so deletion is invisible to
users. Net 187 lines gone. Path-exclusion patterns added to
`sync-docs-from-main.ts` so future upstream syncs don't resurrect them.
## Test plan
- [ ] `/langgraph-python/quickstart`,
`/langgraph-typescript/quickstart`, `/langgraph-fastapi/quickstart` show
\"LangGraph (Python)\", \"LangGraph (TypeScript)\", \"LangGraph
(FastAPI)\" respectively at the top of the sidebar.
- [ ] `/built-in-agent/quickstart` shows \"Built-in Agent\".
- [ ] `/mastra/quickstart` shows \"Mastra\".
- [ ] `/ms-agent-dotnet/auth` and `/ms-agent-python/auth` show \"MS
Agent Framework (.NET)\" and \"MS Agent Framework (Python)\".
- [ ] `/quickstart` (unscoped) keeps \"CopilotKit Docs\" — default
unchanged.
- [ ] Pick a framework, navigate back to `/` — the sidebar dropdown
shows the stored choice with violet active styling, not the \"Pick an
agentic backend\" placeholder.
- [ ] Use Clear Selection — dropdown reverts to placeholder; `framework`
is null and `storedFramework` is null.
- [ ] After a dev server restart (meta.json cache is module-scoped): the
Troubleshooting section in any docs page sidebar lists Common Issues →
Error Debugging → Debug Mode → **Event Inspector** → **Hook Explorer** →
Observability Connectors → migrate-* in that order.
- [ ] After a dev server restart: `/multi-agent/subagents` appears in
the App Control sidebar section (after Shared State) and renders with
the \"Sub-Agents\" title.
- [ ] After a dev server restart (next.config.ts redirects load on
startup): `curl -I http://localhost:3003/frontend-actions` returns `301`
with `Location: /frontend-tools`.
- [ ] `/frontend-tools` renders unchanged.
- [ ] All of these continue to render with multi-hundred-KB bodies after
stub deletion (routing fallback to root MDX): `/built-in-agent/threads`,
`/langgraph-python/threads`, `/mastra/threads`,
`/built-in-agent/premium/self-hosting`,
`/langgraph-python/premium/self-hosting`. Sidebar appearance unchanged —
these stubs were already filtered from the override nav.
Two follow-up fixes from the second CR pass:
1. crewai-crews/docs-links.json `subagents.og_docs_url` was set to
`https://docs.copilotkit.ai/crewai-crews/multi-agent-flows` in the
prior commit. That URL 308-redirects through
`/crewai-flows/multi-agent-flows` to `/crewai-flows` — the same
wrong-product overview the original `missing` rationale already
warned about. Reverting to null with the rationale restored;
honest red is better than a green badge that lands users on a
different product's docs.
2. langroid/docs-links.json `missing[0].reason` claimed "every demo
in langroid's manifest has a working feature-level og_docs_url +
shell_docs_path in feature-registry.json." That's false: `auth`
and `agent-config` are langroid demos with no registry entry, so
those cells render red. Corrected the rationale to acknowledge
the partial coverage and call out the honest-red intent for those
two cells.
No probe-state change (`ok=61 / notfound=0 / missing=19` unchanged
because crewai-crews subagents was never an integration that demos
that feature in any case — the override was dead-weight; the fix is
about not lying to a future reader who promotes it).
24 files in two byte-identical families across 12 integration trees:
- 12 `integrations/<fw>/threads.mdx` — 7 lines each, all md5 0371508d…,
body is just `<Threads />`.
- 12 `integrations/<fw>/premium/self-hosting.mdx` — 8 lines each, all
md5 06d17f98…, body is just `<SelfHosting />`.
Both render shared snippets the root-level pages already render. The
`buildFrameworkOverridesNav` filter dropped them from the merged
sidebar (root wins), and the framework-scoped router falls back to
root MDX when no per-framework override exists, so deletion is
invisible to users — `/built-in-agent/threads`,
`/langgraph-python/premium/self-hosting`, etc. continue to render
identically. Verified: each URL still returns 200 with full body
(340–465 KB) by curling the dev server before commit.
Add path-exclusion patterns to sync-docs-from-main.ts so future
upstream syncs don't resurrect them. Same shape as the BIA branch's
`(other)/` exclusion. Sample-tested the regexes against expected-
exclude and expected-keep paths — all classify correctly.
Audit covered every `integrations/<fw>/*.mdx` with body ≤ 2 non-blank
lines; no false-negative stubs of other shapes.
Code-review pass surfaced that `https://docs.copilotkit.ai/multi-agent`
is a Next.js SPA fallback (200 + `x-matched-path: /[[...slug]]` +
`<meta robots noindex>`) — the probe correctly flags it `notfound`,
but the per-package override path bypasses the probe and was
rendering green badges that 404 on click.
Replaces the URL across the 9 docs-links.json files where I introduced
it, plus the feature-registry fallback:
- Frameworks WITH an indexable framework root → use that:
ag2 → /ag2, agno → /agno, google-adk → /adk,
ms-agent-dotnet → /microsoft-agent-framework,
strands → /aws-strands.
- Frameworks without a real scoped docs page → null (honest red,
not lying-green): claude-sdk-python, claude-sdk-typescript,
spring-ai, langroid.
- Feature-registry `subagents.og_docs_url` → null. The `1 notfound`
in the prior probe summary is now `0 notfound / 19 missing` —
same total, but the registry no longer carries a known-bad URL.
Also rewrote `showcase/packages/langroid/docs-links.json` to align
with langroid's actual manifest demos: of the 8 entries the original
file declared, only 3 (`agentic-chat`, `gen-ui-agent`,
`shared-state-streaming`) matched langroid's manifest; the other 5
(`hitl-in-chat`, `tool-rendering`, `gen-ui-tool-based`,
`shared-state-read-write`, `subagents`) were dead weight (the demos
don't exist on langroid). The file is now `features: {}` with a
rationale — every langroid demo has a working feature-level entry in
the registry, so per-cell overrides aren't needed.
Updated stale "missing" rationale strings in claude-sdk-python,
claude-sdk-typescript, spring-ai, strands to match the new state.
Verification
- `pnpm -C showcase/scripts probe-docs` → `ok=61 / notfound=0 /
missing=19` across 40 features × 2 links.
- Dashboard cell simulation: still 687 docs cells green / 145 red.
No regression — the `/multi-agent` fix touched 8 dead-weight
overrides + 1 registry-fallback path that was already not
rendering.
content/docs/frontend-actions.mdx was a 33-line stub from when the
feature was called "Frontend Actions." The product API is now
exclusively useFrontendTool, the canonical page is /frontend-tools
(1967 bytes, snippet_cell-driven, in the App Control sidebar
section), and the orphan stub had no entry in any meta.json — it
only rendered if you typed /frontend-actions directly.
Per the IA analysis: "redirect or delete; pick one." Picked both:
delete the orphan file + add a 301 redirect /frontend-actions →
/frontend-tools so any external link (search results, blog posts,
customer docs) lands cleanly on the canonical page.
Other 'frontend-actions' references in the repo are stable HTML
anchor IDs (`id="frontend-actions-example"` in 12 integration
`frontend-tools.mdx` files) and external links to the legacy
docs.copilotkit.ai site — neither breaks.
content/docs/multi-agent/subagents.mdx is a real, well-written page
(snippet_cell: subagents, lucide/Users icon) covering the supervisor →
specialized-sub-agents pattern. It rendered at /multi-agent/subagents
but had no entry in any meta.json, so it was unreachable from the
sidebar — a true orphan.
Slot it into the existing App Control section (next to frontend-tools
and shared-state, where it fits conceptually as another agent-behavior
control). The structural IA restructure (sam-shell-docs-ia, PR 2) can
promote it to a dedicated '---Multi-Agent---' section later if the
JTBD reorganization warrants it; this commit just stops it from being
hidden.
The 'multi-agent-flows' references in three integration metas
(crewai-flows, llamaindex, pydantic-ai) point at framework-specific
files (e.g. integrations/crewai-flows/multi-agent-flows.mdx), not at
this top-level page — different topic, no overlap.
The sidebar framework dropdown derived its display label and
"current" highlight from the URL-derived `framework` only. On
unscoped pages (/, /quickstart, /threads, etc.) where `framework` is
null, that resolved to `undefined` and the dropdown reverted to its
"Pick an agentic backend" placeholder — even when localStorage held
the user's prior selection. Visually this looked like the user's
choice had been forgotten on every return to a root-scoped URL.
Fall back to `storedFramework` when `framework` is null. Same
precedence as the Clear-Selection button further down the file
(`framework || storedFramework`). Framework-scoped routes are
unaffected because `framework` is set there.
Both files exist on disk (event-inspector.mdx, hook-explorer.mdx) but
weren't in troubleshooting/meta.json's pages array, so they were
unreachable from the sidebar — only direct-URL navigation surfaced
them. buildNavTree iterates meta.pages exclusively when present
(filesystem auto-discovery only fires when meta.json is missing or
its pages field is missing), so the slugs were effectively orphans.
Inserted between debug-mode and observability-connectors so the four
in-IDE debugging tools (error-debugging → debug-mode → event-inspector
→ hook-explorer) cluster naturally before observability and the
migration-guide tail.
Note: dev server picks up the new entries on restart — readMeta in
lib/docs-render.tsx caches at module scope by design (comment notes
this is intentional).
The sidebar title on /<framework>/* routes was hardcoded to
"CopilotKit Docs" — a regression noted in the IA analysis where
upstream's `integrations/langgraph/meta.json` correctly rebrands the
sidebar via `title: "LangChain"` + `root: true`.
Pass the integration's display name from the registry (already in
scope at this call site for mergeFrameworkNav and other uses) instead
of the hardcoded string. Verified across slugs: /langgraph-python/*
shows "LangGraph (Python)", /built-in-agent/* shows "Built-in
Agent", /ms-agent-dotnet/* shows "MS Agent Framework (.NET)",
unscoped routes (e.g. /quickstart) keep "CopilotKit Docs" via the
DocsPageView default prop.
Replaces dead `https://docs.copilotkit.ai/features/<id>` URLs in the
shared feature registry with paths that actually resolve, so the
dashboard's docs-og / docs-shell badges stop falling back to red ✗
for every cell that has no per-package override.
What changed
- `showcase/shared/feature-registry.json`:
- rename legacy `shell_docs_url` → canonical `shell_docs_path`
(silences the probe's legacy-key warning)
- point `og_docs_url` at indexable docs.copilotkit.ai pages and
`shell_docs_path` at existing `.mdx` files in
`showcase/shell-docs/src/content/docs/`
- leave fields unset for features with no real docs page yet
(`auth`, `agent-config`, `voice`, `multimodal`, `byoc-hashbrown`,
`byoc-json-render`, plus the `kind: testing` features whose docs
row is hidden by `CellStatus`)
- `showcase/packages/<framework>/docs-links.json`: fill in `null`
entries with the closest framework-specific page that exists, or
fall back to a framework-agnostic root (`/multi-agent`,
`/shared-state`, etc.) where no scoped page exists. Files touched:
ag2, agno, claude-sdk-python, claude-sdk-typescript, crewai-crews,
google-adk, langgraph-python, llamaindex, mastra, ms-agent-dotnet,
ms-agent-python, pydantic-ai, spring-ai, strands.
- New: `showcase/packages/langroid/docs-links.json` (was missing
entirely; modeled on the spring-ai/strands pattern since no
langroid-scoped pages exist on docs.copilotkit.ai).
Coverage
- Probe (`pnpm -C showcase/scripts probe-docs`) goes from
every-feature-`notfound` to `ok=61 / notfound=1 / missing=18`
across 40 features × 2 links.
- Dashboard cell simulation: 687 docs cells green, 145 red.
Remaining reds are exclusively features with no real docs target
yet (auth/voice/multimodal/agent-config/byoc-* on integrations
whose framework docs don't have those pages).
No code changes — only the JSON sources of truth. The generated
artifacts (`shell-dashboard/src/data/{registry,docs-status,catalog}.json`,
`shell-{docs,dojo}/src/data/...`) are gitignored and rebuilt by
`pnpm -C showcase/scripts {generate-registry,probe-docs}` on dev/CI.
Alem Tuzlak