## Summary
The 6 beautiful-chat demos (spring-ai, strands, langroid, agno,
claude-sdk-typescript, claude-sdk-python) ship three identical
suggestion chips. Against the deployed aimock-backed showcase, all three
were broken:
| Suggestion | Symptom | Cause |
|---|---|---|
| Plan a 3-day Tokyo trip | Returned a generic "Hi there! I'm your
showcase assistant…" greeting | Substring `"hi"` matches inside
**arc*hi*tecture**, hijacked by the broad `userMessage: "hi"` fixture |
| Explain RAG like I'm 12 | aimock 4xx — `"No fixture matched"` | No
fixture |
| Draft a launch email | aimock 4xx — `"No fixture matched"` | No
fixture |
Verified locally against `showcase up spring-ai` in a headed browser —
all three now return on-topic content.
## Fix
Add three full-sentence fixtures before the broad `"hi"` matcher in
`feature-parity.json`. Aimock's matcher is substring + first-match-wins
by file order, so the long sentence matchers win first and the `"hi"`
fixture is never reached for these prompts. Each returns a plausible
markdown response (3-day Tokyo itinerary, open-book-test analogy,
3-paragraph launch email).
## Regression coverage
Replaced the 1-line beautiful-chat placeholders with a 4-test suite for
all 6 integrations:
- Page loads with heading + chat input
- Each suggestion's reply contains the expected keywords (`Day 1|Day
2|Day 3`, `open-book|retrieval|RAG`, `Subject:|co-pilot|launch`)
- Each test ALSO asserts `toHaveCount(0)` against `/I'm your showcase
assistant/i` — if the broad "hi" fixture re-broadens or the new fixtures
are reordered/removed, the tests fail with a useful message.
## Test plan
- [x] `validate-fixture-tool-surface` clean: 141 fixtures × 628 demos,
no drift
- [x] Manual headed-browser verification on `showcase up spring-ai` —
all 3 suggestions return their on-topic responses
- [ ] CI's `Validate Showcase` job stays green
- [ ] On-demand E2E (`/test-aimock <slug>`) passes for any of the 6
integrations
The 6 beautiful-chat demos (spring-ai, strands, langroid, agno,
claude-sdk-typescript, claude-sdk-python) ship three identical
suggestion chips: "Plan a 3-day Tokyo trip", "Explain RAG like I'm
12", and "Draft a launch email". Against the deployed aimock-backed
showcase, all three were broken:
- Tokyo trip: hijacked by the broad `userMessage: "hi"` fixture,
because the substring "hi" appears inside "arc**hi**tecture" in
the prompt. Returned a generic "Hi there! I'm your showcase
assistant..." greeting with nothing about Tokyo.
- RAG explain: no fixture matched, aimock returned an error.
- Launch email: same — no fixture, error.
Add three on-topic fixtures with the full suggestion sentence as
`userMessage` (effectively-exact substring match). Place them
before the broad "hi" fixture in the file so first-match-wins
routes each suggestion to the right response.
Add a `beautiful-chat.spec.ts` regression suite to all 6
integrations: send each suggestion, assert the right keywords
appear in the assistant reply ("Day 1/2/3" for Tokyo,
"open-book/RAG" for RAG, "Subject:/co-pilot" for email), AND
assert the hijacked greeting is absent. If the broad "hi" fixture
re-broadens or the new fixtures are reordered/removed, these
tests fail loudly.
Bug: in a single chat session, running both HITL booking flows
back-to-back (Alice 1:1 → then Sales call without refresh) used to
skip the time-picker on the second flow and jump straight to
"Booked ..." text.
Cause: confirmation fixtures were matched on `hasToolResult: true`,
which fires whenever the conversation has ANY tool message in
history. After the first flow finished, the second user message
short-circuited to a confirmation match before the second flow's
toolCall fixture (gated on `hasToolResult: false`) had a chance to
fire. The picker never rendered.
Fix: re-key the two confirmation fixtures on `toolCallId` (the
specific tool_call_id of the matching `book_call` invocation), which
only fires when the LAST conversation message is a tool result with
that id — exactly the moment we want the confirmation. Drop the
`hasToolResult: false` constraint on the toolCall fixtures so they
match a fresh user request regardless of prior tool history.
Add a back-to-back regression test to all 17 hitl-in-chat specs:
walk Alice flow to completion, then sales flow without refresh,
assert two `time-picker-card` elements rendered. If the multi-flow
regression returns, the second card never appears and the test
fails at `toHaveCount(2)`.
The hitl-in-chat demo ships in 17 integrations (langgraph-python plus
16 others — mastra, strands, ag2, agno, crewai-crews,
langgraph-typescript, langgraph-fastapi, pydantic-ai, llamaindex,
langroid, claude-sdk-python, claude-sdk-typescript, ms-agent-python,
ms-agent-dotnet, spring-ai, google-adk). All shipped placeholder e2e
specs that only checked the chat input was visible — none exercised
the actual booking flow.
Replace each with the full booking-flow spec written for
langgraph-python:
1. The "Schedule a 1:1 with Alice" suggestion renders the time-picker
card AND the Tokyo greeting is absent (regression guard against
the broad aimock `userMessage: "Alice"` matcher).
2. Picking a slot transitions to the picked-state card and produces
a "Booked … Alice" assistant follow-up.
3. The "Book a call with sales" suggestion runs the same flow with
the sales attendee.
Also add the matching aimock fixture pair for the sales suggestion
in feature-parity.json — without it, case 3 would only pass against
real OpenAI, not the aimock-backed CI deployments. The pair mirrors
the Alice fixture pair: `book_call` toolCall on first turn,
confirmation message after the picker resolves.
Per-integration coverage matters because each integration has its
own framework-specific HITL wiring (`useHumanInTheLoop` binding to
the agent, agent-side tool registration, run streaming protocol)
that can regress independently of the shared aimock fixture.
- Add dedicated tool-free voice agents for strands, llamaindex,
ms-agent-python (aimock returns tool calls when tools are registered,
which the adapters don't loop on)
- Add sample_agent alias to langgraph-typescript langgraph.json
(was only in dev-mode config)
- Add SampleAudioButton and voice route to google-adk
- Add sample.wav to agno, ms-agent-dotnet, ms-agent-python, google-adk
The custom _run_reasoning_agent handler had a bug where text messages
weren't rendered by the frontend despite the backend emitting correct
AG-UI events. The stock AGUI handler works with reasoning=False and
aimock fixtures — the D5 probe checks for reasoning keywords in the
transcript, not for REASONING_MESSAGE events specifically.
Locally verified: 28/29 D5 features pass (only auth fails — pre-existing
auth gate regression unrelated to this change).
The _run_reasoning_agent handler's fallback path (for aimock fixtures
that return plain text with "Reasoning:" prefix) was setting
answer_text="" which skipped emitting any TEXT_MESSAGE events.
CopilotKit requires a text message to render an assistant bubble in
the conversation view -- reasoning events alone produce no visible
DOM element that the D5 probe selectors can match, causing both
reasoning-display and tool-rendering-reasoning-chain to timeout
with 0 assistant messages.
Fix: set answer_text = full_text so the response is emitted as both
a reasoning message (for the ReasoningBlock slot) and a text message
(for the conversation transcript the probe reads).
- agno: add default agent alias + per-request runtime
- langgraph-fastapi: add default agent alias
- llamaindex: fix agent name mismatch (byoc_hashbrown → byoc-hashbrown-demo)
- mastra: create dedicated byocHashbrownAgent with hashbrown system prompt
(was using weatherAgent which produced plain text instead of JSON)
- ms-agent-dotnet: upgrade byoc page to V2 CopilotKit import
The D5 conversation runner detects assistant responses via
data-testid="copilot-assistant-message". The byoc-hashbrown demo
overrides the assistantMessage slot with a custom HashBrown renderer,
which dropped that attribute. Without it the harness sees 0 messages
and times out.
Replace gen-ui-interrupt and interrupt-headless "not supported" stubs
with working demos using useFrontendTool + async Promise pattern.
Backend agents use system prompt + tools=[] — CopilotKit runtime
routes tool calls to the frontend handler. Pattern proven by
ms-agent-python/dotnet, now extended to ag2, agno, built-in-agent,
claude-sdk-python, claude-sdk-typescript, crewai-crews, google-adk,
langroid, llamaindex, mastra, pydantic-ai, strands.
Recent feature commits added new dependencies to integration package.json
files (@copilotkit/voice, @hashbrownai/{core,react}, @json-render/{core,react})
and bumped Next.js from 15.4.10 to 15.5.15, but never regenerated the
corresponding package-lock.json. The Showcase Build & Deploy workflow runs
`npm ci --legacy-peer-deps` which strictly enforces lock sync, so every
deploy attempt has been failing at the install step. No new images have been
pushed to GHCR, so Railway services have stayed on stale code and any cell
added since each fw's last successful deploy iframes 404.
Regenerated all 18 lockfiles via `npm install --legacy-peer-deps
--package-lock-only --ignore-scripts` per integration. Verified each with
`npm ci --dry-run --legacy-peer-deps` — all clean.
Refs PDX-90.
The marker-insertion script in ac3885fe0 used a brace counter that
counted opening braces from the destructured function parameters as
the start of the function body, then matched the destructuring's
closing `}` as the body's close. The result on every fw was an
`@endregion[sample-audio-button]` jammed onto the same line as the
destructuring's `}`, with the actual function body falling outside the
region — broken structure plus a format violation (`}// @endregion` on
one line).
Fixes both: strips the broken inline endregion and appends a proper
@endregion marker at end-of-file (which is where the function actually
ends, since these files contain only the single SampleAudioButton
function below the imports + interface). 17 files restored.
Prior commit (878259e20) deployed sibling .snippet.* files for voice across
all 18 frameworks. That was the wrong call — siblings are a *fallback* for
demos that legitimately diverge from the canonical teaching shape. The
voice demos in 17 frameworks already match the canonical (V2 runtime +
TranscriptionService + sample-audio-button), so the right move is to tag
region markers on the real source.
Changes:
- 17 frameworks (everything except google-adk): add `@region[…]` markers
to actual demo source for `voice-runtime`, `transcription-service-guard`,
`voice-page`, `sample-audio-button`. 51 source files modified, no
behavioral changes — just `// @region[name]` / `// @endregion[name]`
comments wrapping existing code.
- crewai-crews/manifest.yaml: add `highlight:` block to the voice demo
with the route file path so the bundler picks up the runtime regions.
Every other framework already had this entry.
- 17 frameworks: delete the wrong sibling files (`voice-runtime.snippet.ts`
and `voice-frontend.snippet.tsx`) that 878259e20 created.
- google-adk: KEEP the two siblings — google-adk genuinely diverges
(uses the shared `/api/copilotkit` route rather than a dedicated
`/api/copilotkit-voice`), which is exactly when the sibling fallback
is the right answer.
Result: snippet audit B-docs-gap = 0; every framework's voice page
renders real demo code via `<Snippet>` refs. The 16 standard frameworks
pull from their actual route.ts / page.tsx / sample-audio-button.tsx;
google-adk pulls from its sibling.
The first pass of /voice.mdx had inline code blocks. Rewrites the page
to use <Snippet> references against per-framework sibling files, matching
how the rest of shell-docs sources its code samples.
- Two siblings per framework (×18 fws = 36 files):
- voice-runtime.snippet.ts: V2 CopilotRuntime + TranscriptionService
setup, including the GuardedOpenAITranscriptionService wrapper that
returns a clean 4xx when OPENAI_API_KEY is missing. Regions:
`voice-runtime`, `transcription-service-guard`.
- voice-frontend.snippet.tsx: chat surface with auto-mic-button, plus
the SampleAudioButton that bypasses the mic for Playwright /
screenshot flows. Regions: `voice-page`, `sample-audio-button`.
- /voice.mdx now uses 4 `<Snippet region="..." />` refs instead of
inline code, so the docs reference real teaching code that lives next
to each framework's actual demo (and stays in sync with the established
per-framework sibling convention from PR #4439).
The shell-docs `/generative-ui/a2ui/fixed-schema` page references the
regions `backend-schema-json-load` and `backend-render-operations` to
teach how the backend loads (or inlines) the A2UI schema and emits
render operations. 5 frameworks (ag2, agno, claude-sdk-python,
claude-sdk-typescript, langroid) ship working schema-loading demos but
hadn't tagged those region markers, so cells rendered a yellow
"missing snippet" box. built-in-agent has the same issue with its
schema-inline variant.
Per the established sibling convention (matching
`tool-rendering/render-flight-tool.snippet.tsx`), each framework now
ships a docs-only `a2ui-backend.snippet.{py,ts}` exposing both regions
with the canonical pattern. Zero changes to the actual demo source.
Files:
- 4 × `.snippet.py` (Python backends): ag2, agno, claude-sdk-python, langroid
- 1 × `.snippet.ts` (TypeScript backend, schema-loading): claude-sdk-typescript
- 1 × `.snippet.ts` (TypeScript backend, schema-inline): built-in-agent
Closes 12 B-docs-gap region refs (6 frameworks × 2 regions).
Adds two new manifest pattern flags (matching the existing
`interrupt_pattern` / `a2ui_pattern` convention) so the canonical
`/agent-config` and `/auth` shell-docs pages can gate their per-pattern
sections via `<WhenFrameworkHas>` and only render the implementation that
applies to the framework the user has selected.
- `agent_config_pattern: shared-state | runtime-properties | null`
- `runtime-properties` (1 fw): built-in-agent
- `shared-state` (17 fws): everything else that wires agent-config
- `auth_pattern: langgraph | ag2-context-variables | microsoft-agent-framework | runtime-onrequest | null`
- `langgraph` (3 fws): langgraph-python, langgraph-typescript, langgraph-fastapi
- `ag2-context-variables` (1 fw): ag2
- `microsoft-agent-framework` (2 fws): ms-agent-python, ms-agent-dotnet
- `runtime-onrequest` (12 fws): everything else
Also fills in the previously-missing `a2ui_pattern` flag on 6 frameworks
that have wired demos but were rendering near-empty doc pages because
none of the existing `<WhenFrameworkHas>` gates matched. Audit-driven:
ag2/agno/claude-sdk-{python,typescript}/langroid use schema-loading;
built-in-agent uses schema-inline.
Sweep across all `.snippet.*` files (existing + new in this branch) to
remove non-teaching content that distracts from the docs-page render.
Changes:
- 6 files (5 hitl + 1 tool-rendering): replace `(props: any)` +
`eslint-disable-next-line @typescript-eslint/no-explicit-any` with
proper structural prop types. Reads identical to the eye but no lint
suppression in the rendered snippet.
- 1 file (state-streaming-middleware.snippet.py): drop 2
`# type: ignore[name-defined]` markers. The stand-in identifiers
(`write_document`, `AgentState`) already read as docs-only references.
- 1 file (delegation-log-frontend.snippet.tsx, BIA): rewrite the in-region
JSDoc to be framework-agnostic. The file was ported from ag2 and still
named `AG2 sub-agent` + referenced `ReplyResult` / `ContextVariables`
in the BIA copy. Also drop a historical bug-fix note ("Per-status
color map…") that is irrelevant outside ag2's commit history.
- 2 files (use-rendered-messages.snippet.tsx, google-adk + llamaindex):
strip brittle internal-path references (`packages/react-core/src/v2/.../
CopilotChatMessageView.tsx:542-612`, `react-core/v2/components/chat/
CopilotChatToolCallsView.tsx`) that would rot within months. Replaced
with conceptual references to the public component name only.
No region markers changed; audit still reports B-docs-gap: 0.
The shell-docs `/human-in-the-loop` page teaches the booking pattern
(useHumanInTheLoop with a TimePickerCard rendering candidate slots)
via `<Snippet region="hitl-hook" />` and `<Snippet region="time-slots" />`.
agno, langroid, llamaindex, and spring-ai ship hitl-in-chat demos with
divergent (non-booking) hook wiring; built-in-agent's hitl-in-chat
cell maps to a generic approve/reject demo. Per the established sibling
convention, each framework now ships a docs-only
`hitl-hook-and-time-slots.snippet.tsx` exposing both regions with the
canonical booking shape.
Frameworks: agno, langroid, llamaindex, spring-ai (hitl-in-chat dir);
built-in-agent (hitl dir, where hitl-in-chat cell is routed).
Closes 9 B-docs-gap refs from PDX-83 (8 hitl-hook+time-slots across 4
fws + 1 time-slots for built-in-agent).
The shell-docs `/generative-ui/tool-based` page teaches the
`useComponent` bar-chart pattern via `<Snippet region="bar-chart-renderer" />`,
but 14 frameworks ship a haiku-generator demo that uses
`useFrontendTool` instead — a fundamentally different API. Per the
established sibling convention (matching `tool-rendering/render-flight-tool.snippet.tsx`),
each framework now ships a docs-only `bar-chart-renderer.snippet.tsx`
that exposes the canonical teaching shape without touching the demo.
Frameworks: ag2, agno, built-in-agent, claude-sdk-python,
claude-sdk-typescript, crewai-crews, google-adk, langgraph-fastapi,
langgraph-typescript, langroid, mastra, ms-agent-dotnet, spring-ai,
strands.
Closes 14 of the 45 remaining B-docs-gap refs from PDX-83.
- @region[runtime-mcpapps-config] on copilotkit-mcp-apps/route.ts
- @region[no-frontend-renderer-needed] on demos/mcp-apps/page.tsx
- @region[minimal-runtime-flag] + @region[advanced-runtime-config] on
the shared copilotkit-ogui/route.ts
- @region[minimal-provider-setup] on demos/open-gen-ui/page.tsx
- @region[sandbox-function-registration] on demos/open-gen-ui-advanced/page.tsx
Adds a dedicated A2UI dynamic-schema cell for the Agno integration:
new `a2ui_dynamic_agent.py` owns its own `generate_a2ui` tool that
reads the runtime-injected client catalog from
`run_context.session_state["copilotkit"]["context"]` and feeds it
to a secondary OpenAI client bound to `render_a2ui`. Result is an
`a2ui_operations` container the runtime A2UI middleware forwards
to the registered frontend catalog renderer.
Unlike the main agent's existing `generate_a2ui` (hardcoded
internal catalog), this dedicated agent stays in sync with whatever
catalog the frontend registers via `<CopilotKit a2ui={{ catalog }}>`.
Frontend ports `page.tsx` + Zod definitions / renderers / catalog
from the langgraph-python reference. Dedicated runtime route at
`/api/copilotkit-declarative-gen-ui` runs with
`injectA2UITool: false` so the agent owns the tool itself.
Adds a dedicated A2UI fixed-schema cell for the Agno integration:
backend `a2ui_fixed_agent.py` ships `flight_schema.json` +
`booked_schema.json` and a single `display_flight` tool that emits
an `a2ui_operations` container directly (no secondary LLM).
Frontend ports `page.tsx` + Zod definitions / renderers / catalog
from the langgraph-python reference. Dedicated runtime route at
`/api/copilotkit-a2ui-fixed-schema` runs the A2UI middleware with
`injectA2UITool: false` so the agent owns its rendering tool.
Beautiful Chat (simplified) — polished landing-style chat shell over the
shared agentic_chat agent with gradient theming, suggestion pills, and an
example dashboard surface (BarChart + PieChart re-used from byoc-json-
render). Mirrors the spring-ai simplified port; full A2UI demonstration
catalog is out of scope while Agno lacks an A2UI middleware adapter.
gen-ui-interrupt + interrupt-headless — stubbed as not_supported_features
in manifest.yaml. Agno has no graph-level interrupt primitive, so each
stub page documents the gap and links to the closest Agno-supported
pattern (hitl-in-chat for in-chat blocking, hitl-in-app for out-of-chat
blocking via async useFrontendTool).
Adds three feature-matrix entries to the Agno integration manifest:
- `cli-start` — informational copy-paste starter command card (no route).
- `gen-ui-tool-based` — frontend-only haiku-renderer demo that already ships
in `src/app/demos/gen-ui-tool-based/`; the manifest entry plus the
existing alias in `src/app/api/copilotkit/route.ts` (mainAgentNames)
surface it in the showcase.
- `hitl-in-chat-booking` — second cell for the booking flavor of the
time-picker HITL surface; reuses `src/app/demos/hitl-in-chat/` files and
the main agent's `book_call` external-execution tool.
PARITY_NOTES.md updated with a fourth-pass entry summarising the additions.
Five sibling features (`beautiful-chat`, `gen-ui-interrupt`,
`interrupt-headless`, `declarative-gen-ui`, `a2ui-fixed-schema`) remain
deferred for the reasons already documented in PARITY_NOTES.md — Agno's
AGUI adapter has no interrupt primitive, and the dedicated combined
runtimes (openGenerativeUI + a2ui injection + mcpApps) and the
CopilotKit-middleware-driven catalog injection that the langgraph-python
reference relies on have not been wired into Agno yet.
Add demo entries for hitl, hitl-in-app, hitl-in-chat, tool-rendering,
shared-state-read-write, and gen-ui-tool-based across 14 integrations.
Ensure every demo ID also appears in the features list so the showcase
matrix and D5 probes discover them correctly.
STATE_SNAPSHOT can deliver a Preferences object with interests undefined,
crashing .includes(), .filter(), and spread at 4 sites per file. Add
(value.interests ?? []) guards across all 17 integrations.
The packages/starters merge (PR #4351) eliminated starters as separate
deployable units. Remove the starter: block (path, name, description,
github_url, demo_url, clone_command) from all 17 integration manifests
to stop propagating stale showcase-starter-* Railway URLs through the
data pipeline.
Replace sys.path.insert hacks in Python agent files with direct
imports via symlinks to shared/{python,typescript}/tools.
Update Dockerfiles, entrypoints, and configs to support the new
symlink-based tool resolution. Add PARITY_NOTES for frameworks
that have known gaps.
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
Alem Tuzlak