Closes the demo↔probe coverage gap for /demos/{interrupt-headless,
shared-state-read, tool-rendering-reasoning-chain} so every demo
under langgraph-python (the north-star integration) now has a D5
probe writing to its own PocketBase cell — not relying on cross-
demo umbrella records.
New probes (multi-turn, mirroring the agentic-chat structure):
- d5-interrupt-headless: exercises useHeadlessInterrupt — chip
prompt → backend interrupt(...) → app-surface popup → slot pick
→ resume → assistant confirmation. Distinct from gen-ui-interrupt
(which uses inline useInterrupt).
- d5-tool-rendering-reasoning-chain: combines reasoning-block slot
+ per-tool renderer (WeatherCard, FlightListCard) on the same
chat surface. Catches a regression in either side.
- d5-shared-state-read: recipe-editor demo (neutral default agent,
no tools) — verifies recipe-card form mounts AND agent reads
shared state across turns. Drops the dual-claim that
d5-shared-state.ts had on `shared-state-read` (now write-only).
Driver retry-once (e2e-deep.ts):
Probes that fail with a transient class (`goto-error` /
`conversation-error`) AND took ≥2s on the first attempt now retry
once before recording red. Persistent assertion-style failures
(sub-2s) and intentional aborts/feature-timeouts skip retry —
retrying a deterministic mismatch just burns clock and obscures
the signal. Cuts ~10× the dashboard flap rate.
Plumbing:
- D5FeatureType enum: +interrupt-headless, +tool-rendering-reasoning-chain.
- REGISTRY_TO_D5 (harness) + CATALOG_TO_D5_KEY (dashboard) mirror
the new mappings; d5-mapping-drift test enforces this.
- LGP manifest features + demos entries + constraints allowlist.
- feature-registry.json: +shared-state-read.
- aimock d5-all.json: +2 shared-state-read fixtures (interrupt-
headless + tool-rendering-reasoning-chain reuse existing fixtures
that already match their chip prompts).
Tests: 1588/1588 harness vitest green. validate-fixture-tool-surface
clean (282 fixtures × 627 demos, no drift). Two pre-existing test
fixes folded in — d5-gen-ui-interrupt assertion mock updated to
match the current evaluate-poll resume signal; conversation-runner
preFill ordering test now asserts the actual deferred-cascade
contract instead of a stricter pre-preFill ban that the runner
never enforced.
Known follow-up (not in this PR): auth.spec.ts test #5 ("signing
back in re-mounts a fresh chat surface") fails on Railway — second
sign-in's "Hello again" never produces an assistant response. Looks
like a react-core/v2 ref-handling regression on <CopilotKit>
unmount/remount; deserves its own focused investigation.
Other integrations may flip red on the new probes — that's
expected. We're treating LGP as the template; cross-integration
parity follows in a separate wave.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the Bucket A framework fix and the fixture-correctness changes
needed to flip the remaining D4/D2 cells in the langgraph-python column
to D5. Full local D5 sweep is green (1 passed, 0 failed).
Framework: every Python `ToolMessage` constructed via `Command(update=...)`
now sets `name=` and `id=str(uuid.uuid4())`. Without these, @ag-ui/langgraph
synthesises TOOL_CALL_START events with `toolCallName: null` and
`parentMessageId: null`, which @ag-ui/client@0.0.53's Zod schema rejects;
the rejection is silently swallowed by `withAbortErrorHandling -> EMPTY`,
completing the SSE observable mid-stream so post-tool state never reaches
the consumer. Fix is applied across shared_state_streaming, shared_state_read_write,
gen_ui_agent, beautiful_chat, subagents (2 sites). Single-flag change in
a2ui_dynamic flips the secondary `_design_a2ui_surface` LLM call to
`streaming=True` so aimock's record/replay (SSE-only) sees it.
Fixtures (d5-all.json):
- toolCallId follow-ups for set_steps (3), display_flight, generate_a2ui (4),
schedule_meeting (2), generateSandboxedUi (7), and revenue chart so
multi-turn probes don't recurse into recursion-limit loops
- four hand-crafted secondary `_design_a2ui_surface` fixtures so A2UI
dynamic renders without a real LLM
- mcp-apps fixture rewritten to emit `create_view` tool call with a
minimal Excalidraw element payload; runtime middleware fetches the UI
resource and the iframe mounts
- AAPL and revenue `hasToolResult: true` follow-ups tightened to
`toolCallId` so they don't match cross-turn after prior turns' tool
results
- voice fast-path content-only fixture
- beautiful-chat-schedule-meeting first-turn fixture gains content so
the conversation runner sees an assistant message before the picker
click assertion
Probes: bumped per-card waitForSelector in d5-gen-ui-headless-complete from
15s to 60s — recharts ResponsiveContainer can be slow under 4 sequential
turns.
Shell-dojo: hide CLI Start Command from the dojo navigation via
`HIDDEN_DOJO_FEATURE_IDS`. Registry/manifests untouched so
harness/parity/dashboard still see it.
/langgraph-python/frontend-tools referenced regions
frontend-tool-registration and frontend-tool-handler which were missing
markers on the production demo. Added in-place @region markers on the
existing useFrontendTool block in
showcase/integrations/langgraph-python/src/app/demos/frontend-tools/page.tsx
since the demo is a clean teaching example.
/langgraph-python/custom-look-and-feel/slots referenced regions
register-welcome-slot, register-assistant-message-slot, and
register-disclaimer-slot. The chat-slots production demo registers ~12
slot overrides at once with `as unknown as typeof X` casts that obscure
the per-pattern teaching shape, so added a sibling
slot-overrides.snippet.tsx file (mirrors the llamaindex chat-slots
sibling pattern) with the three minimal teaching regions.
Verified by re-bundling demo-content.json, running shell-docs production
build, and curling both pages on a local server: zero Missing snippet
boxes, and the region code (change_background, CustomWelcomeScreen,
CustomDisclaimer, CustomAssistantMessage) renders on the pages.
Three classes of regression are now pinned:
1. Secondary-LLM tool name doesn't collide with the A2UI middleware's
default intercept list (`render_a2ui`). New
`src/agents/test_a2ui_internal_tools.py` parametrises over
`beautiful_chat._design_a2ui_surface`, `a2ui_dynamic._design_a2ui_surface`,
and `a2ui_fixed.display_flight` and asserts none match the
middleware's `a2uiToolNames` default. Catches accidental rename
reverts that would re-enable the bypass.
2. `generate_a2ui` force-pins the canonical `catalog_id` even when the
secondary LLM hallucinates a wrong one. The new test stubs
`ChatOpenAI` with a fake response carrying a bogus catalogId and
asserts the surface op carries the module's `CUSTOM_CATALOG_ID`.
3. `generate_a2ui` short-circuits with a clean error string when the
LLM emits a root component without a `component` field — never
feeds the renderer the partial tree that surfaced as the "Cannot
create component root without a type" infinite-loop.
7 unit tests, all green locally (`pytest src/agents/test_a2ui_internal_tools.py`).
E2E tests on the same fixes now also assert:
- No `A2UI render error: Catalog not found` banner on the page after
Beautiful Chat → Sales Dashboard, Declarative Gen UI → BarChart, and
A2UI Fixed Schema → Find SFO → JFK round-trips.
- No `Cannot create component … without a type` banner on the same
three pills.
- Exactly ONE flight card on A2UI Fixed Schema (was 6+ on deploy
pre-fix from the `display_flight` loop) — `Flight Details` count
pinned to 1, `Book flight` count pinned to 1.
- At most one ResponsiveContainer on the BarChart pill (loops would
stack multiple).
- At most two ResponsiveContainers on Beautiful Chat → Sales Dashboard
(one pie + one bar = single dashboard render).
Two follow-up fixes layered on the previous internal-tool rename:
(1) `a2ui_fixed.py` — fixed-schema demo infinite loop on deploy. The
`display_flight` tool returns the raw `a2ui.render(...)` JSON descriptor
as its tool result. gpt-4o-mini reads that opaque blob, can't tell the
flight was rendered, and re-calls `display_flight` indefinitely (visible
on the deployed showcase as 6+ duplicate flight cards stacked under
repeated assistant text). Local was just lucky.
Hardened the docstring + system prompt to spell out: the JSON return
value is the surface descriptor, the card is already rendered, do NOT
call again, reply with one short confirmation and stop.
(2) Rename `render_a2ui` → `_design_a2ui_surface` in shared and
langgraph-python parity copies of `tools/generate_a2ui.py` (+
`tools/__init__.py` re-export `RENDER_A2UI_TOOL_SCHEMA` →
`DESIGN_A2UI_SURFACE_TOOL_SCHEMA`), and in `showcase/shared/typescript/
tools/generate-a2ui.ts`. These shared helpers were the source-of-truth
for the secondary-LLM tool name across integrations; renaming here keeps
parity with the langgraph-python agents already renamed in
`beautiful_chat.py` / `a2ui_dynamic.py`. Other framework integrations
keep their own `render_a2ui` for now (separate parity sweep).
(3) `showcase/aimock/feature-parity.json` — added a sibling fixture
matching `toolName: "_design_a2ui_surface"` for the beautiful-chat Sales
Dashboard pill so the langgraph-python e2e suite still hits a
deterministic mock on Railway. The original `render_a2ui` fixture is
kept above it so other integrations whose secondary LLM still requests
`render_a2ui` continue to match.
(4) Comment update in `beautiful-chat.spec.ts` to name the new internal
tool.
The A2UI middleware (`@ag-ui/a2ui-middleware`) defaults `a2uiToolNames` to
`["render_a2ui"]` and synthesises ACTIVITY_SNAPSHOT events from the
streaming tool-call args of any matching call — using the LLM's RAW
catalogId and components verbatim, before the Python tool body has a
chance to validate or normalise.
Both `beautiful_chat.py` and `a2ui_dynamic.py` use a `generate_a2ui` tool
that internally invokes a secondary LLM bound to a structured-output
`render_a2ui` helper. Because the helper's name matched the middleware's
default intercept list, the secondary LLM's hallucinated catalogId
(`declarative-gen-ui-catalog` leaking into the beautiful-chat dashboard)
and malformed root components (no `component` field on KPI dashboard)
were emitted to the frontend bypass, surfacing as:
- "A2UI render error: Catalog not found: declarative-gen-ui-catalog" on
the beautiful-chat Sales Dashboard pill
- "A2UI render error: Cannot create component root without a type"
infinite-loop on the declarative-gen-ui KPI Dashboard pill
The earlier force-pin of `catalog_id` and the defensive component sweep
in `generate_a2ui` were correct but ran too late — they execute on the
tool-result, after the middleware has already fired surface events from
the streaming args.
Fix: rename the internal helper to `_design_a2ui_surface` (and update
`tool_choice`, prompt header, and module docstring) so it falls outside
the middleware's intercept list. The explicit `a2ui.render(...)` ops the
outer `generate_a2ui` returns are then the only path to the frontend,
and our Python validation layer is authoritative.
Verified locally: Beautiful Chat → Sales Dashboard pill renders Total
Revenue / New Customers / etc. with no errors.
## Summary
- Beautiful Chat **Search Flights** pill: restored
`_build_flight_components` so the agent emits a flat literal-children
FlightCard tree (the structural-children template form via
`flight_schema.json` + `update_data_model` doesn't expand correctly
through GenericBinder for our custom catalog — it was working until the
recent flagship-cell pass swapped it back). FlightCards render again
instead of falling through to the default tool card.
- Beautiful Chat **Sales Dashboard** pill: hardened `generate_a2ui` so
the secondary LLM gets explicit catalog-id + component-shape rules and
the resulting `catalog_id` is force-pinned to the registered catalog.
Kills the "Catalog not found: declarative-gen-ui-catalog" sibling-demo
hallucination.
- **Declarative Gen UI** (`a2ui_dynamic.py`): same `generate_a2ui`
hardening, plus a defensive sweep that drops malformed components and
bails clean when the LLM omits a typed root. Kills the "Cannot create
component root without a type" infinite-loop renderer error reported on
the KPI Dashboard pill.
Two files touched (`src/agents/beautiful_chat.py`,
`src/agents/a2ui_dynamic.py`); no frontend / runtime / package changes.
Existing aimock fixtures in `feature-parity.json` already use the
canonical catalogId and well-formed components, so e2e specs should pass
without fixture changes.
## Test plan
- [x] Verified locally: `langgraph_cli dev` + `next dev`, clicked
**Search Flights** on `/demos/beautiful-chat` — Delta DL 405 ($329) and
United UA 120 ($289) FlightCards rendered live with airline, route,
times, status, Select.
- [x] Verified locally: clicked **Show a KPI dashboard** on
`/demos/declarative-gen-ui` — REVENUE / SIGNUPS / CHURN metric tree
rendered cleanly, no "Cannot create component root without a type"
error.
- [ ] CI / `nx run @copilotkit/showcase-langgraph-python:e2e`
(beautiful-chat spec already authored against the literal-children
form).
Three regressions, two files. Without these the Search Flights pill falls
through to the default tool card instead of rendering FlightCards, the
Sales Dashboard pill errors with "Catalog not found:
declarative-gen-ui-catalog", and the Declarative Gen UI cell loops on
"Cannot create component root without a type."
beautiful_chat.py — search_flights:
- Restore _build_flight_components and have search_flights emit the flat
literal-children component tree it produces, instead of the
flight_schema.json structural-children template (Row.children =
{ componentId, path: "/flights" }) plus update_data_model. The
GenericBinder doesn't reliably expand the structural form for our
custom FlightCard catalog — sibling demos avoid the form for the same
reason. The literal form has been the working shape since it was
introduced; the recent flagship-cell pass swapped it back to the
schema/data-model form and broke fixed-schema rendering.
- Make Flight TypedDict permissive (total=False) and drop the required
id / statusIcon fields. langchain rejected calls when the LLM (or
aimock fixture) omitted those, surfacing the validation string as the
tool result and never producing a surface.
beautiful_chat.py + a2ui_dynamic.py — generate_a2ui:
- Prepend a hard-requirements header to the secondary LLM's prompt
pinning the canonical catalogId and the component-shape contract
(every entry — including root — must carry both `id` AND `component`).
With injectA2UITool: false the runtime context alone leaves the LLM
enough room to hallucinate sibling-demo catalog IDs (e.g.
declarative-gen-ui-catalog leaking into the beautiful-chat dashboard)
and to omit `component` on the root entry (the source of the
"Cannot create component root without a type" infinite loop).
- Force catalog_id to the module-level CUSTOM_CATALOG_ID after the tool
call so any residual hallucination still routes to the registered
frontend catalog.
- Drop malformed component entries before constructing the operations
list, and bail with a clean error string if no valid root survives —
fail-soft instead of looping the renderer.
The renderer-side definitions.ts/renderers.tsx changes from the same
flagship pass are kept as-is; their Row/Column children union
(string-array OR { componentId, path }) is forward-compatible and
doesn't hurt the literal-children path.
The voice demo's <CopilotKit> didn't pass enableInspector, so
shouldShowDevConsole(undefined) defaulted to isLocalhost(), which
auto-mounts <cpk-web-inspector> on any local Docker host
(localhost:3100 in showcase compose). The inspector overlay
intercepts pointer events on top of the voice sample-audio button,
so dev/D5 probe runs can't click it through Playwright.
Production isn't localhost, so the inspector never mounts there —
voice is D5 in prod and D4 locally for this reason alone. Set
enableInspector={false} explicitly so the demo behaves the same in
both environments.
Probe result: d5:langgraph-python/voice flips green.
## Summary
Closes the remaining yellow-box snippet gaps in shell-docs ahead of the
Tue 5/12 cutover. Replaces 5 individual one-PR-per-framework attempts
with a single consolidated PR.
## What's in here
**Region marker additions** (5 commits, comment-only changes — zero
runtime impact):
- `claude-sdk-typescript` `gen-ui-interrupt::backend-tool-call` —
sibling teaching file (no per-demo backend in claude-sdk-ts; mirrors
claude-sdk-python's Strategy B SYSTEM_PROMPT shape)
- `langroid` `gen-ui-interrupt::backend-tool-call` — in-place around the
existing `ScheduleMeetingTool` `ToolMessage` subclass (preserves
langroid's idiom — no forced LangGraph shape)
- `pydantic-ai` `gen-ui-tool-based::bar-chart-renderer` — sibling
teaching file (production demo is haiku-generator via `useFrontendTool`,
not a `useComponent` bar-chart)
- `spring-ai` `shared-state-streaming` (2 regions:
`frontend-use-coagent-state` + `state-streaming-middleware`) — in-place;
backend Java file already in `manifest.yaml` `highlight:`
- `strands` `gen-ui-interrupt::backend-tool-call` — in-place around the
canonical `@tool def schedule_meeting`
All follow the [Mastra Round
playbook](https://www.notion.so/34f3aa38185281c0b41dc6b6c63c89c6) for
in-place vs sibling decisions.
**Catalog flips** (2 commits, manifest-only):
- 11 non-LangGraph frameworks: move `shared-state-streaming` from
`features` to `not_supported_features` (ag2, agno,
claude-sdk-{python,typescript}, crewai-crews, langroid, llamaindex,
mastra, ms-agent-{dotnet,python}, pydantic-ai)
- 6 more without working demos: built-in-agent, google-adk, strands,
langgraph-typescript, langgraph-fastapi, spring-ai
- Per team's call 2026-05-07: state-streaming only works on LangGraph
variants currently. Anything without a working demo gets the blue
`UnsupportedBox` instead of yellow `Missing snippet` warnings.
Engineering can flip these back to `features` once impls land.
Final catalog state for `shared-state-streaming`: 17 unsupported, 1
wired (`langgraph-python` — canonical reference).
## Why now
This PR is cutover-blocking. Sam's bar for Tuesday: zero yellow boxes
anywhere in shell-docs. The current state had 33+ yellow boxes from the
post-2026-04-29 audit refresh; this PR drives that to zero (in
conjunction with the previously-merged PDX-122 fix in #4685).
## Verification
- `npx tsx showcase/scripts/bundle-demo-content.ts` — all newly-marked
regions resolve in `demo-content.json`
- `npx tsx showcase/scripts/generate-registry.ts` — catalog regenerates
clean with the manifest changes
- Generated `catalog.json` files are gitignored artifacts; not in this
diff
## Test plan
- [ ] Build shell-docs locally: `cd showcase/shell-docs && npm run
build` clean
- [ ] Visit a few `shared-state/streaming.mdx` pages on framework
variants — should render the blue "Not supported on X" box for the 17
unsupported frameworks
- [ ] Visit `langgraph-python/shared-state/streaming.mdx` — should still
render the working snippets
- [ ] Visit the 4 fixed B-cell pages
(claude-sdk-typescript/langroid/strands
`human-in-the-loop/useInterrupt.mdx`, pydantic-ai
`generative-ui/tool-based.mdx`, spring-ai `shared-state/streaming.mdx`)
— all snippets resolve, no yellow boxes
CR Round 3 final: my auth.spec.ts e2e was asserting on
[data-message-role="assistant"] which is the v1 react-ui RenderMessage
attribute. The auth demo uses v2 CopilotChat — its
CopilotChatAssistantMessage only emits data-testid="copilot-assistant-message".
The selector would never have matched and both tests would have
timed out at 30s when actually run.
Verified via grep:
- packages/react-core/src/v2/components/chat/CopilotChatAssistantMessage.tsx:192
emits data-testid="copilot-assistant-message" (no data-message-role)
- packages/react-ui/src/components/chat/messages/RenderMessage.tsx:32,41
emits data-message-role="user"/"assistant" (v1 path)
- All sibling specs in langgraph-python/tests/e2e/ correctly use
data-testid="copilot-assistant-message"
One-character switch from data-message-role to data-testid with the
canonical v2 testid value.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The auth demo refactor flipped its lifecycle: unauthenticated is now
the default state, <CopilotKit> only mounts after sign-in, and
sign-out unmounts the entire chat tree (instead of leaving stale
auth headers in a still-mounted chat). The old probe + e2e spec
chased a 401-error-banner surface that no longer exists in the new
demo, plus a brittle 500ms hardcoded `useEffect` flush wait.
Probe rewrite (`d5-auth.ts` + tests):
- Add `buildAuthPreFill` that clicks the SignInCard's sign-in button
before turn 1, then waits for the chat textarea to mount (proves
<CopilotKit> handshook with the runtime).
- `buildAuthAssertion` now clicks sign-out, then waits for SignInCard
to re-mount. The unmount marker IS the proof — no chat-send-and-401
dance is needed (or possible — there's no chat to send into).
- Drop the hardcoded 500ms setTimeout, the unauth-banner wait, and
the error-surface poll. None apply to the new flow.
E2E rewrite (`tests/e2e/auth.spec.ts`):
- "page loads unauthenticated with SignInCard visible"
- "signing in mounts the chat surface with AuthBanner"
- "authenticated send produces an assistant response"
- "signing out unmounts the chat tree and re-renders SignInCard"
- "signing back in re-mounts a fresh chat surface"
Fixture comment updated to reflect the new flow. The user message
("auth check turn 1") and content response are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The headless-complete refactor replaced the old 5-chip set
(Weather/AAPL/Highlight/Sketch/Largest continent) with a new 4-chip
set wired through `useConfigureSuggestions`/SuggestionBar:
Weather → "What's the weather in Tokyo?" → WeatherCard
Stock price → "What's the price of AAPL right now?" → StockCard
Highlight a note → "Highlight this note for me: 'ship the demo on Friday'."
→ HighlightNote
Revenue chart → "Show me a chart of revenue over the last six months."
→ ChartCard
Probe rewrite:
- Drop the wrapper-scoped chip selector (`[data-testid=
"headless-suggestions"] >> text=...`) — the new SuggestionBar
doesn't have that container testid. Click via plain `button >>
text="<chip title>"` instead.
- Per-turn assert the matching tool card testid mounts
(headless-weather-card, headless-stock-card, headless-highlight-card,
headless-revenue-chart) plus a distinguishing text token. Driven
off a TURN_EXPECTATIONS table so adding a chip is one entry.
- Drop the readMessagesText helper that scoped to a missing
`[data-testid="headless-complete-messages"]` container; collect
text by concatenating all `headless-message-assistant` bubbles.
Fixture rewrite:
- Drop the Excalidraw + Largest-continent entries (no longer in the
chip set). Weather is already covered by feature-parity.json.
- Add a Revenue chart fixture for `get_revenue_chart` with inline
data so ChartCard renders without backend.
- Stock + Highlight fixtures preserved (still match the new chip
messages via substring).
Also restore `data-message-role` attributes on the headless-complete
UserBubble + AssistantBubble so the runner's settle plateau cascade
can resolve them — matches the runner's documented headless-template
contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The headless-simple demo was refactored to a deliberately minimal
"two hooks, one shadcn shell" template — text-in/text-out only, no
gen-UI. The D5 type literal `gen-ui-headless` no longer described
what the probe tests, and the old probe (Profile-card useComponent
+ continent fallback) was asserting against UI that no longer exists.
Three coordinated changes:
1. Rename `gen-ui-headless` D5FeatureType to `headless-simple` so the
slug matches the demo. Updated d5-registry, REGISTRY_TO_D5,
CATALOG_TO_D5_KEY (dashboard), and dependent tests/comments.
`headless-complete` keeps its existing literal because that demo
still drives the full gen-UI surface.
2. Replace d5-gen-ui-headless.{ts,test.ts,fixture} with
d5-headless-simple.ts + headless-simple.json fixture. New probe
clicks the "Say hello in one short sentence." chip and asserts the
`[data-testid="headless-message-assistant"]` bubble mounts with
non-empty content.
3. Restore `data-message-role` attributes on the headless-simple
UserBubble + AssistantBubble. The runner's chat-input cascade
documents these as the headless-template contract; the refactor
dropped them, breaking the runner's settle plateau detection.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The chat-slots refactor replaced the explicit `data-testid="custom-assistant-message"`
attribute with a SlotMarker wrapper component. The probe still asserted
the old testid which no longer exists, so the cell stuck at D4.
Add `data-slot-label={label}` to SlotMarker's outer span — idiomatic
data attribute that mirrors the existing `label` prop and gives the
probe a stable contract without restoring the legacy testid pattern.
The marker was already passing label="MessageView.AssistantMessage";
this just surfaces it in the DOM.
Update d5-chat-slots probe + test to assert
`[data-slot-label="MessageView.AssistantMessage"]`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related changes targeting the fc=97 'Target page closed' D5 failure
on `beautiful-chat-toggle-theme`:
1. Probe (`_beautiful-chat-shared.ts:assertToggleTheme`) — replace the
manual 200ms `page.evaluate` poll over 30s with Playwright's
`waitForFunction`. The native polling is event-driven inside the
browser context, disconnects cleanly on page-close, and avoids
~150 round-trip evaluates per probe. The catch branch now checks
`page.isClosed()` first and surfaces a diagnostic message naming
the renderer-crash case explicitly, so operators don't have to
guess what 'Target page closed' meant.
2. Demo (`use-generative-ui-examples.tsx`) — drop `[theme, setTheme]`
from the `useFrontendTool` deps array. The handler reads `document`
directly and the setter is stable across renders, so the deps
array forced a re-registration after every theme flip. That race
could collide with an in-flight tool result and surface as a
renderer error during multi-turn sequences. Removing deps keeps
the tool registration stable for the duration of the conversation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Sam's clarification 2026-05-07: anything without a working state-streaming demo gets unsupported. Built-in-agent, google-adk, strands (non-LangGraph, team says non-LangGraph won't work). Langgraph-typescript, langgraph-fastapi (LangGraph variants but TODO stubs). Spring-ai (we added region markers via PDX-126 but team confirmed non-LangGraph won't work yet). Only langgraph-python (canonical reference) stays wired. Engineering can flip these back to features once impls land.
Per team decision 2026-05-07: state-streaming only works on LangGraph variants. Move shared-state-streaming from features to not_supported_features for ag2, agno, claude-sdk-python, claude-sdk-typescript, crewai-crews, langroid, llamaindex, mastra, ms-agent-dotnet, ms-agent-python, pydantic-ai. Catalog regenerator now classifies these cells as unsupported, so the docs render the blue UnsupportedBox instead of yellow Missing snippet warnings.
Add the two snippet regions referenced by /shared-state/streaming.mdx:
- frontend-use-coagent-state in src/app/demos/shared-state-streaming/page.tsx
- state-streaming-middleware in SharedStateStreamingController.java
Comment-only. Both markers wrap the conceptual block that maps the
write_document tool argument into state.document, mirroring the
LangGraph Python StateStreamingMiddleware reference.
Verified with `npx tsx showcase/scripts/bundle-demo-content.ts`:
spring-ai::shared-state-streaming: 4 files (4 highlighted) + README + 2 regions
Both regions resolve in the bundled demo-content.json.
Wrap the ScheduleMeetingTool ToolMessage subclass in
backend-tool-call region markers so the langroid useInterrupt
docs page can render the backend half of the cell. Comment-only
change. Langroid declares frontend tools as ToolMessage subclasses
that the AG-UI adapter intercepts client-side, so the region
captures the langroid-idiomatic shape rather than forcing the
langgraph-python @tool + Command pattern.
Adds a docs-only sibling teaching file so shell-docs useInterrupt MDX can
resolve its `<Snippet cell="gen-ui-interrupt" region="backend-tool-call" />`
reference. Production architecture diverges from langgraph-python: the
claude-sdk-typescript backend is a single shared Express agent server with
a generic system prompt, so there is no per-demo backend file to mark up
in place. The sibling exposes the canonical Strategy B shape (system
prompt instructing the model to call the frontend-registered
schedule_meeting tool), mirroring claude-sdk-python's interrupt_agent.py.
Comment-only / unimported file; zero runtime impact. Verified via
showcase/scripts/bundle-demo-content.ts: claude-sdk-typescript::gen-ui-interrupt
now bundles 3 regions (frontend-promise-handler, weather-tool-backend,
backend-tool-call).
_invoke_sub_agent's last-resort branch silently returned '' or a
Python repr like \"[{'type': 'text', ...}]\" for block-list content,
which the UI would render as a blank/garbled card. Return a stable
SUB_AGENT_EMPTY_SENTINEL ('<sub-agent produced no output>') instead
so the d5-subagents probe can match it against its boilerplate-marker
list and fail the genuine-pass test loudly when a sub-agent produces
no usable output.
The get_stock_price tool returned randint-based price/change every
call, so e2e specs asserting $338.37 / -2.96% would only pass when
aimock short-circuited the entire call. The Python tool body still
runs server-side under aimock — only the LLM call is mocked.
Mirror the deterministic-`value` pattern on roll_d20: accept optional
price_usd / change_pct arguments and echo them back when present.
Defaults to random mock data when the args are omitted, preserving
the legacy live-LLM behaviour.
The HELLO_LEADING phrase was the showcase-assistant catch-all
boilerplate ('I can help you with weather lookups...') that other
tests in this PR explicitly guard AGAINST. The dedicated d5-all.json
fixture for 'Say hello in one short sentence' now returns a distinct
non-boilerplate greeting; the spec asserts that distinct phrase, so a
fixture-priority misroute fails loudly instead of passing by accident.
The blitz removed the explicit useDefaultRenderTool() call expecting a
framework-level fallback to handle zero-hook registrations, but the
integration uses the published @copilotkit/react-core@1.56.5 which does
not yet ship that fallback. Without the call, useRenderToolCall has no
'*' renderer and tool calls render invisibly — the user only sees the
agent's final text summary instead of the OOTB tool card.
Restore the explicit useDefaultRenderTool() invocation. The framework
fallback (committed in this PR but inert until react-core publishes a
release that includes it) becomes a no-op once that ships.
## Summary
Six commits, batched into one PR for the May 12 docs cutover.
- `2c10c735d` Port upstream CLI walkthrough verbatim onto 8 framework
quickstarts (adk, agno, aws-strands, langgraph, llamaindex, mastra,
microsoft-agent-framework, pydantic-ai). Initial byte-perfect import
from upstream `docs/content/docs/integrations/<fw>/quickstart.mdx`.
- `f17c86867` Add `sample-audio-button` region marker on the
built-in-agent voice demo. Closes a B-cell snippet gap; the bundler now
reports 4 regions on `built-in-agent::voice`.
- `cebca9216` Remove AG-UI brand tab from desktop header for visual
parity with `docs.copilotkit.ai` (canonical has zero AG-UI elements in
the header).
- `abeb2792b` Also remove AG-UI link from mobile slide-out menu and drop
now-unused helpers (`AgUiIcon`, `AG_UI_LINKS`, `AG_UI_PREFIXES`, `Brand`
type, `activeBrandFromPath`, `usePathname`). Net 82-line cleanup of
`brand-nav.tsx`.
- `99d8ac7a5` Intermediate rewrite of the 8 quickstarts to use `npx
copilotkit@latest create --framework <id>` (after CLI accuracy audit
found `--intelligence` and `--framework` are mutually exclusive in
`copilotkit@2.0.3`). Also applies V2 canonical imports to the same 8
files: `<CopilotKit>` from `@copilotkit/react-core` (root), styles from
`@copilotkit/react-core/styles.css`, drop `@copilotkit/react-ui` from
npm install.
- `59a8a71a6` Final CLI-section shape: each quickstart now leads with
the upstream interactive walkthrough (`npx copilotkit@latest create`
plus a bulleted explanation of Project name / Enterprise Intelligence
Platform with sign-up link / Framework picker), followed by an "Or skip
the prompts and pin a framework directly" `--framework <id>` block.
LangGraph and Microsoft Agent Framework show both language variants in
the flag block.
## Test plan
- [ ] Visit each of the 8 framework quickstarts on the dev server.
Verify the CLI section shows the interactive walkthrough with the
EIP/sign-up bullet AND the `--framework <id>` flag block below it.
- [ ] LangGraph quickstart shows both `langgraph-py` and `langgraph-js`
flag commands. Microsoft Agent Framework shows both
`microsoft-agent-framework-dotnet` and `microsoft-agent-framework-py`.
- [ ] All 8 quickstarts: imports show `@copilotkit/react-core` (root),
styles from `@copilotkit/react-core/styles.css`. No `/v2` paths.
- [ ] Inspect shell-docs header at desktop and mobile (narrow viewport).
No AG-UI brand tab anywhere.
- [ ] Visit a built-in-agent voice page that consumes the
`sample-audio-button` region. The snippet renders.
Resolved d5-all.json conflict by appending B6's headless-simple/complete fixtures (Say hello, joke, fun fact, Highlight, chart) before the tool-rendering and open-gen-ui entries already on integration.
Adds production-code testids needed by the Phase-2B genuine D5 probes:
- frontend-tools/background.tsx: data-testid='frontend-tools-background'
+ data-background-value mirror so the probe can read the live gradient
off the DOM without computing styles.
- declarative-gen-ui/a2ui/renderers.tsx: data-testid for Card,
StatusBadge, Metric, PieChart, BarChart so per-pill probes can assert
the expected catalog component painted.
- a2ui-fixed-schema/a2ui/renderers.tsx: data-testid='a2ui-fixed-card'
on the Card override so the fixed-schema component-tree mount is
observable without keyword-matching the transcript.
Each cell's Layer 1 spec is already green; this is purely additive — no
spec rewrites.
All three tool-rendering cells now drive the suggestion pills directly
instead of typing free-form prompts that race fixture matchers:
- tool-rendering: 6 tests (page loads + 5 pills) — Weather in SF
asserts SF city + deterministic temp/humidity/wind; Find flights
asserts >=2 flight rows from the dedicated fixture; Stock price
asserts AAPL $338.37 / -2.96%; Roll a d20 asserts exactly 5 d20
cards with the 5th=20; Chain tools asserts weather+flights+d20
cards mount from a single pill click.
- tool-rendering-default-catchall: 6 tests asserting the OOTB default
tool-call renderer paints every tool call with data-testid=
copilot-tool-render plus data-tool-name. Branded sibling-cell
testids stay at zero. Test 6 asserts every card matches the
built-in renderer DOM signature.
- tool-rendering-custom-catchall: 6 tests asserting the same
custom-wildcard-card testid renders for every tool. Test 6 is the
cross-tool snapshot — every tool kind paints via the same shell.
Add stable testids and rendering surfaces for the three tool-rendering
cells so the e2e suite can distinguish each cell's strategy:
- tool-rendering: register useRenderTool for get_stock_price and
roll_d20; new StockCard / D20Card components with testids
stock-card / d20-card / stock-price / stock-change / d20-value.
Rename FlightListCard testid flight-list-card -> flights-card.
- tool-rendering-default-catchall: drop the custom shadcn
useDefaultRenderTool registration so the cell is truly zero
custom-render-hooks. The framework's built-in
DefaultToolCallRenderer now paints every tool call, with stable
data-testid='copilot-tool-render' wrapper plus data-tool-name,
data-args, and data-result attributes for inspection without
expanding the card.
- tool-rendering-custom-catchall: rename the wildcard renderer's
testids from custom-catchall-* to custom-wildcard-* so the cell
is distinguishable from the (now-OOTB) default-catchall demo.
- packages/react-core: when no per-tool / wildcard renderer is
registered, useRenderToolCall now falls back to the built-in
DefaultToolCallRenderer instead of returning null.
Replace the random roll_dice tool with roll_d20(value), which echoes
the LLM-supplied value back as the result. Aimock fixtures script the
five sequential calls returning [7, 14, 3, 19, 20] so the e2e suite can
assert exact values rather than rolling until 20 lands.
Update SYSTEM_PROMPT to allow multi-tool chaining when the user
explicitly asks for it (Chain tools pill emits 3 tool calls in one
turn).
hitl-in-app — 7 explicit tests (1 skip):
- Page-load and pill-render tests retained.
- New refund #12345 approve/reject pair asserts the deterministic
fixture leading phrases ("I am processing the $50 refund" vs
"refund request was not approved").
- New escalate #12347 approve/reject pair asserts "Escalated ticket
#12347" vs "Not escalated".
- The describe block runs in serial mode so the approve test
(sequenceIndex 0 in the fixture) always runs before the reject test
(sequenceIndex 1) for each pill.
- Downgrade #12346 stays skipped per spec (broken upstream as of
2026-05-07).
frontend-tools-async — 4 explicit tests:
- Page-load test asserts composer + 3 pills.
- Project-planning, auth, and reading pills each click and assert the
Notes DB card renders with the correct keyword heading and the
per-note testid rows that the async handler returned. Anti-regression
assertions catch the previous fixture-priority bugs (generic-plan
boilerplate, showcase-assistant catch-all).
- Reading pill locks the full canonical shape per spec test #4: keyword,
match count, note title, content lines, tag chip, and the assistant
narration leading phrase.
No production-code testid changes — the existing dialog and notes-card
testids cover every assertion. Fixture work lives in d5-all.json (prior
commit).
- Drop the 8 stale tests that asserted travel-planner shapes
(Current Itinerary / supervisor-indicator / .bg-gray-50). They
predated the supervisor + 3-subagent rewrite and could not catch
the 3 production bugs.
- New suite (5 tests, 0 skipped):
1) page loads with composer + 3 verbatim suggestion pills + 3
subagent role indicators (testid-based).
2-4) one test per pill (Write a blog post / Explain a topic /
Summarize a topic). Each clicks the pill, waits for all
three role-scoped subagent cards to reach data-status=
"complete", then asserts each card's subagent-result is
non-empty AND does not contain the showcase-assistant
boilerplate fragments. Test 4 also serves as a regression
gate on the delegations reducer fix — without it, that pill
returns HTTP 400 and the cards never reach complete.
5) clicks any pill, waits for terminal state, then asserts the
critic-card count is exactly 1 and stays at 1 with status
complete across a 5s dwell — catches any return of the
supervisor -> critic loop.
- Aimock fixtures: 3 verbatim pill chains (cold exposure training,
LLM tool calling, reusable rockets) added to both d5-all.json and
the harness source d5/mcp-subagents.json. Each chain drives the
full supervisor flow (turnIndex 0..3) plus three nested sub-agent
fixtures so every Researcher/Writer/Critic card surfaces real
prose instead of showcase boilerplate.
Drop the 5 cross-origin contentFrame() / page.on('console', ...) skipped
assertions across the two specs — sandbox=allow-scripts only blocks host
introspection of the iframe DOM, and console-spying on the host page
catches no inner-iframe logs. Replace with iframe-presence assertions:
each pill click must produce iframe[sandbox*='allow-scripts'] with a
non-empty srcdoc (or src) attribute. That is the load-bearing signal
that the open-generative-ui pipeline mounted SOMETHING.
Rewrite each suggestion message string as a short verbatim label that
doubles as a deterministic aimock fixture key (paired with the new
fixtures in showcase/aimock/d5-all.json). Drop pill-title parentheticals
per the cosmetic note in lgp-test-genuine-pass.md so titles read as
natural human prompts; keep the message field aligned with the fixture
key.
Final test counts: 5 minimal (page-load + 4 pill-iframe), 4 advanced
(page-load + 3 pill-iframe). All .skip() removed. The sandbox-function
round-trip (evaluateExpression / notifyHost) is intentionally not
asserted here — that requires a same-origin sandbox option or a
host-side spy on the runtime's sandbox-function-call event, both
deferred to a follow-up.
- subagent-activity-card: emit data-testid="subagent-card-<role>" on
each card wrapper (researcher | writer | critic), data-testid=
"subagent-result" on the Result content div, and data-testid=
"subagent-status" on the status pill. The previous testids
(subagent-activity-card, subagent-activity-result) collapsed across
roles, so the e2e suite couldn't count or content-assert per role.
- delegation-log: render a fixed row of 3 always-visible role
indicators (data-testid="subagent-indicator-<role>") so the page
exposes a stable hook for the load-state assertion regardless of
whether the supervisor has delegated yet.
- Annotate AgentState.delegations with operator.add reducer so concurrent
sub-agent emissions in one supervisor step accumulate instead of
raising INVALID_CONCURRENT_GRAPH_UPDATE (HTTP 400 on the Summarize pill).
- Update _delegation_update to return only the new entry (the reducer
concatenates) instead of echoing the full prior list, which would
duplicate entries each step under operator.add.
- Cap supervisor -> critique_agent loop at _MAX_CRITIQUE_ITERATIONS
(default 1). Re-entrant critique calls short-circuit with a finish-now
ToolMessage and do not append a second delegation, so the UI shows
exactly one critic card per supervisor run.
- _invoke_sub_agent now walks messages newest-first and returns the
first non-empty AIMessage content (handles list-of-content-blocks
shape too). Prevents the previous failure mode where a final empty
AIMessage made the card Result blank or echoed the showcase-assistant
intro.
- Strengthen supervisor system prompt: each sub-agent must be called
exactly once, with no further calls after critique returns.
Rewrites both headless specs to the pill-driven plan from
.claude/specs/lgp-test-genuine-pass.md (Family-1F).
headless-simple (4 tests):
1. page loads with custom composer + 3 pills visible
2. hello pill -> assistant bubble starts with the greeting leading phrase
3. joke pill -> assistant bubble contains the deterministic joke
4. fun fact pill -> assistant bubble contains 'Honey never spoils!'
headless-complete (5 tests):
1. page loads with custom composer + 4 pills visible
2. weather pill -> WeatherCard with Tokyo / Sunny / 68F + narration
3. AAPL pill -> StockCard with AAPL / $189.42 / +1.27% + narration
4. highlight pill -> HighlightNote with 'ship the demo on Friday' + narration
5. revenue chart pill -> ChartCard with 'Quarterly revenue', subtitle,
month labels Jan-Jun + narration
Each test asserts on the headless-specific testids introduced in the
preceding commit, so a regression that demotes the headless surface
back to the default CopilotChat surface fails every tool test. Each
pill exercises a different render-hook path so regressions surface
test-by-test.
No .skip() — all 9 tests are live.
Adds stable data-testid hooks on the hand-rolled headless chat surface
shared by headless-simple and headless-complete so e2e specs can target
the headless surface (and not the default CopilotChat surface) by
selector.
Shared:
- 'headless-message-assistant' on the custom assistant bubble
- 'headless-message-user' on the custom user bubble
- 'headless-composer' on the composer container
headless-complete only:
- 'headless-weather-card' on the WeatherCard rendered via useRenderTool
- 'headless-stock-card' on the StockCard
- 'headless-highlight-card' on the HighlightNote rendered via useComponent
- 'headless-revenue-chart' on the ChartCard
If the headless surface ever silently regresses to the default
CopilotChat surface, the headless-specific testids are absent and the
spec fails. Each tool-card testid is scoped per-component so a regression
in a single render hook fails only that test.
Replace the previous 4-test (2 active, 2 skipped) suite with a
5-test deterministic plan keyed off the demo's actual published
context defaults and pill verbatim prompts.
Tests:
1. page loads — context-card + composer render
2. edits propagate to JSON — type into name/timezone, JSON updates
3. "Who am I?" pill — assistant reply leads with "I see you're Atai"
and the identity card has name=Atai, timezone=America/Los_Angeles,
avatar text=A (defaults from page.tsx)
4. activity checkboxes default-checked — "viewed the pricing page"
and "watched the product demo video" are checked on first paint
5. "Suggest next steps" pill — assistant reply leads with "Since
you recently viewed the pricing page and watched the product
demo video"
Tests 3 and 5 rely on the deterministic aimock fixtures added in
the previous commit. Tests 4 uses the new activity-<slug> testids.
0 .skip() remaining.
Add four production-code testids on demo-layout.tsx:
- identity-name on the user-name display
- identity-timezone on the timezone display
- identity-avatar on the first-letter avatar circle
- activity-<slug> on each activity <label>, kebab-case of activity name
These let Playwright lock the identity card against the published
context defaults (Atai, America/Los_Angeles, A) and assert the
default-checked activity rows ("Viewed the pricing page", "Watched
the product demo video") without depending on assistant text.
Add two verbatim-prompt aimock fixtures to showcase/aimock/d5-all.json
for the demo's suggestion pills:
- "What do you know about me from my context?" leading phrase
"I see you're Atai, and you're in the America/Los_Angeles timezone.
Recently, you viewed the pricing page and watched the product demo
video."
- "Based on my recent activity, what should I try next?" leading phrase
"Since you recently viewed the pricing page and watched the product
demo video, ..."
Pinned to the verbatim message bodies so they don't contend with the
showcase-assistant catch-all fixture.
Tyler Slaton