mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Alem Tuzlak
ff4d7efa2a
The aggregated multi-turn probe from #4672 hit a CopilotKit v2 quirk on /demos/beautiful-chat: only the FIRST useComponent tool call in a conversation paints its component. Subsequent tool calls emit (the agent's followup content arrives) but the component never mounts. Reproduced cleanly without any frontend tool involvement — pie-chart turn 1 paints 5 svg circles in seconds, bar-chart turn 2 emits "Bar chart rendered above..." but paints zero recharts elements. The runner can't sidestep this from inside one conversation without a page.reload() between turns, which the structural Page type doesn't expose. Splitting into per-pill scripts means each probe gets its own browser launch — fresh page state, fresh conversation, no useComponent ordering pollution. CATALOG_TO_D5_KEY maps `beautiful-chat` to all listed literals; isD5Green requires every key green for the cell to advance to D5, and per-pill failure isolation surfaces in PB row names. Coverage in this PR (5 pills): - beautiful-chat-toggle-theme (frontend tool, html.dark flip) - beautiful-chat-pie-chart (controlled gen-UI useComponent) - beautiful-chat-bar-chart (controlled gen-UI useComponent) - beautiful-chat-search-flights (A2UI fixed-schema FlightCards) - beautiful-chat-schedule-meeting (HITL with slot-click resolution) All 5 verified locally end-to-end (5/5 pass against the local stack). Out of scope, intentionally (track in follow-up): - Excalidraw — depends on mcp.excalidraw.com reachability - Calculator — sandboxed iframe; dup of d5-gen-ui-open - Sales Dashboard — generate_a2ui → render_a2ui chain renders Metric labels but Row-bound charts don't paint recharts containers under aimock fixtures (live pill against same fixture chain shows the inverse symptom). Suggests aimock's non-progressive arg streaming differs from a live LLM in a way the A2UI binder is sensitive to. Needs separate aimock/binder investigation. - Task Manager — manage_todos dispatches and agent emits closing content, but StateStreamingMiddleware's state.todos propagation doesn't populate the App pane TodoList through aimock — same suspected root cause as Sales Dashboard. Architecture details: - _beautiful-chat-shared.ts factors DOM helpers + per-pill assertions, mirroring _hitl-shared.ts's pattern for an extended Page type with click() + a runtime guard - Each fixture uses unique D5-prefixed userMessage substrings; the multi-stage Schedule Meeting flow uses hasToolResult false→true for round disambiguation (no toolCallId leakage since each probe runs in its own fresh page session)
30 lines
1.3 KiB
JSON
30 lines
1.3 KiB
JSON
{
|
|
"_comment": "D5 fixture for /demos/beautiful-chat — Pie Chart pill. Direct pieChart toolCall (skipping query_data — payload supplies inline data so the chart renders without a query_data round-trip, matching feature-parity's 'revenue distribution by category' pattern). Tool payload mirrors the canonical 4-slice fixture so visual fingerprints match across runners.",
|
|
"fixtures": [
|
|
{
|
|
"match": {
|
|
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
|
"hasToolResult": false
|
|
},
|
|
"response": {
|
|
"toolCalls": [
|
|
{
|
|
"id": "call_d5_bc_pie_chart_001",
|
|
"name": "pieChart",
|
|
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
|
}
|
|
]
|
|
}
|
|
},
|
|
{
|
|
"match": {
|
|
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
|
"hasToolResult": true
|
|
},
|
|
"response": {
|
|
"content": "Pie chart rendered above — Electronics leads at $42k, followed by Clothing, Food, and Books."
|
|
}
|
|
}
|
|
]
|
|
}
|