`<CopilotKit agent="beautiful-chat">` routes the chat to agent id
"beautiful-chat", but ExampleCanvas called `useAgent()` with no args and
fell back to DEFAULT_AGENT_ID ("default"). The frontend's agent registry
tracks state per id, so `manage_todos` state-deltas from the chat run
landed on "beautiful-chat" and never reached the canvas's "default"
subscription — the Task Manager pill auto-flipped the panel to App mode
but the To Do column stayed empty. Drop the unused "default" alias from
the runtime route and pin the canvas to `useAgent({ agentId:
"beautiful-chat" })` so both halves share one ProxiedCopilotRuntimeAgent
instance. Adds a Playwright regression test asserting the 3 verbatim
todo titles render after the pill click, plus 3 aimock fixtures for the
multi-turn flow (enableAppMode -> manage_todos -> confirmation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three layers of regression guards for the runtime reasoning-role filter and
the chained demo behavior:
1. Runtime unit test — packages/runtime/.../run-message-filtering.test.ts:
- Verifies `LangGraphAgent.run` strips `role:"reasoning"` from
`input.messages` before delegating to super.run.
- Verifies user/assistant/system/tool messages pass through in order.
- Verifies empty + missing messages arrays are tolerated.
- Verifies pre-existing forwardedProps.streamSubgraphs default + override
behavior is preserved.
- 6/6 tests pass against the runtime package's vitest config.
2. D5 harness probe — showcase/harness/.../d5-tool-rendering-reasoning-chain.ts:
- Expanded from one chained turn (flights→weather) to all three chained
pills in a single thread (stocks AAPL→MSFT, dice d20→d6, flights→weather).
- This is the canonical multi-pill regression at the harness layer:
without the runtime reasoning-role filter, the second pill would crash
before the model was called.
- Each turn asserts the per-turn delta of reasoning-block mounts (idx+1),
the minimum card count for each tool group, and unique transcript
substrings that scope to that turn.
3. Playwright e2e spec — showcase/integrations/langgraph-python/tests/e2e/
tool-rendering-reasoning-chain.spec.ts:
- Mirrors the pattern of the sibling tool-rendering-default-catchall spec
(notably its multi-pill regression at lines 162-212).
- Page-loads test verifies the 3 pills mount and no cards leak from a
prior session.
- One test per chained pill (stocks, dice, flights+weather) asserts the
full chain renders with reasoning-block + correct per-tool cards +
narration matching the aimock fixture text.
- Sequential-pills regression test clicks all 3 pills in one thread,
asserts each chain renders independently AND the reasoning-block count
increases monotonically across turns.
Agent: extend `get_stock_price` to accept optional `price_usd` and
`change_pct` arguments (mirrors the basic tool-rendering agent's signature
introduced in #4770). The aimock fixtures script the chained AAPL/MSFT
comparison by passing deterministic prices via these args; without the
wider signature, pydantic rejects the tool call and the card never mounts.
The runtime unit test is the strongest guard — it would catch any
regression on the role-filter logic without depending on the full Docker
stack. The harness probe and Playwright spec catch end-to-end regressions
in the canonical CI environment.
Four independent showcase production bugs Alem reported, plus the
D5 multimodal harness regression they unblocked.
Shared-state-read-write: "Greet me" ("Say hi and introduce yourself.")
and "Plan a weekend" ("Suggest a weekend plan based on my interests.")
were matching the bare `hi` and `plan` catch-alls in feature-parity.json
and returning the generic showcase-assistant blurb / 5-step content plan
instead of shared-state-aware responses. Added pill-specific fixtures in
shared-state.json (mirrored into d5-all.json) so the longer userMessage
substrings win first-match-wins ahead of feature-parity.
Auth sign-out: signing out unmounted CopilotKit entirely and bounced
the user back to the SignInCard, so the demo never showcased the
runtime returning 401 — its whole point. The QA contract in
qa/auth.md spelled out the intended UX. Restored it: CopilotKit stays
mounted after the first sign-in, the AuthBanner flips to an amber
"Signed out — the agent will reject your messages" state with a
re-Sign-in button, and CopilotKit's `onError` callback drives a
`data-testid="auth-demo-error"` surface that displays the runtime's
401 the moment the user sends an unauthenticated message. Updated the
e2e spec to match (the old "SignInCard re-mounts after sign-out" test
pinned the regression).
Gen-ui-agent: the aimock fixture short-circuited the 7-step
progression spelled out in `gen_ui_agent.py`'s SYSTEM_PROMPT to a
single set_steps call with all three steps already `completed`, so
the InlineAgentStateCard rendered the final 3/3 state instantly with
no sequential pending → in_progress → completed animation.
Regenerated as a 7-leg toolCallId chain per pill (8 fixtures × 3
pills): seed leg keyed on userMessage with NO `hasToolResult` gate
(matching PR #4770's pattern — `hasToolResult: false` would block the
seed from firing on the second pill in a multi-pill session), then
six toolCallId-keyed transitions, then a final narration. Fixture
order: toolCallId legs FIRST so the most specific match wins.
Multimodal D5: the sample-attachment buttons auto-send via
`agent.addMessage + copilotkit.runAgent` (restored in PR #4761), but
the D5 harness still typed `input` + pressed Enter via the runner
after `preFill`, sending a second user message that competed with the
in-flight image upload — the v1 LangGraph runtime SSE stream got
tangled (browser DevTools showed `statusCode: pending` indefinitely)
and the assistant message never rendered. Added `skipSend?: boolean`
to ConversationTurn (distinct from `skipFill`, which still presses
Enter once the textarea has content) and switched d5-multimodal.ts to
`skipSend: true` with `responseTimeoutMs: 60_000` so the runner waits
on the assistant response without poking the chat further. Bumped the
PDF auto-prompt fixture in feature-parity.json to include the word
"document" so the existing `buildModalityAssertion("document")` check
still lands.
D5 result: 37 → 39 of 40 features passing. Only
`tool-rendering-reasoning-chain` remains and is a separate
agent/runtime bug (Tokyo Responses-API `reasoning` message survives
into the next turn's conversation history, runtime returns
`RUN_ERROR: "message role is not supported"`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The tool-rendering, frontend-tools-async, and hitl-in-app fixtures all gated
their first-leg (tool-emitting) vs. follow-up (narration) responses on
`hasToolResult: false/true` and/or `turnIndex`. Those constraints count the
*entire* thread, so once a user clicked a tool-using pill the thread already
contained tool messages and assistant turns and subsequent pill clicks fell
through to the wrong branch — d20 dropped from 5 rolls to 3, Chain tools
emitted no cards, query_notes returned narration without the Notes DB card,
and the second HITL pill never raised an approval dialog.
Re-key every follow-up fixture on the prior step's `toolCallId` (the matcher
checks `messages[last].tool_call_id`), drop the global `hasToolResult` gates
from the tool-emitting fixtures, and reorder so the toolCallId-specific
fixtures come first under first-match-wins. The d20 chain becomes a linear
toolCallId graph (`call_tr_d20_seq_001` → `_002` → … → `_005`), Chain tools
gets disambiguators for each of its three parallel tool_call_ids, and
Weather/AAPL/query_notes/HITL approve+reject branches all gate on the
specific request_user_approval / get_weather / query_notes / get_stock_price
id that landed last. userMessage matchers are unchanged.
Adds Playwright multi-pill regression tests to the four affected demos that
click every pill sequentially in one thread and assert the full card counts:
- tool-rendering-default-catchall: Find flights → 5 d20 rolls (with 20 last)
- tool-rendering-custom-catchall: 1 flights + 5 d20 + 3 chain = 9 cards
- frontend-tools-async: 3 NOTES DB cards with the right keyword per pill
- hitl-in-app: refund approve then escalate, each with its own dialog
The previous fixture regression (HTML+CSS only, no jsFunctions) slipped
past CI because the e2e suite only asserted "iframe mounts with non-empty
srcdoc" — which passes whether or not the iframe is interactive. Adds
two layers of guard so the same regression cannot land silently:
1. showcase/scripts/__tests__/open-gen-ui-advanced-fixtures.test.ts
(vitest, runs in showcase_validate on every PR): asserts each of the
three interactive fixture entries in d5-all.json ships jsFunctions
referencing the matching host bridge (evaluateExpression / notifyHost).
Catches "someone removed jsFunctions" at PR-time with no
infrastructure dependencies.
2. showcase/integrations/langgraph-python/tests/e2e/open-gen-ui-advanced.spec.ts
(playwright, runs in test_e2e-showcase-on-demand): adds three
round-trip tests that drive the in-iframe controls and assert the
host-side handler ran by capturing its console.log + verifying the
iframe output element reflects the host response. Catches "the
renderer fails to inject jsFunctions into the sandbox" too.
The e2e tests also switch the existing smoke tests off pill-click and
onto a textarea-driven fill+Enter path, following the same precedent as
commit 15db0bbf3 (gen-ui-headless-complete) — chip mounts diverge
between EmptyState and SuggestionBar surfaces, and Playwright's pill
click races React hydration. Using [data-testid="copilot-chat-textarea"]
with an explicit click + waitForLoadState("networkidle") makes the
suite reliable end-to-end (7/7 passing locally against the aimock-driven
stack).
## Summary
Re-lands the multimodal-attachments fix from #4584 (May 1, never merged)
onto current `main`, ported to the post-refactor file layout where
`page.tsx` was split into `legacy-converter-shim.tsx`,
`multimodal-chat.tsx`, and `file-to-data-attachment.ts`.
Auto-send was the visible regression: clicking **Try with sample image /
Try with sample PDF** only queued the attachment chip instead of sending
the canned prompt. This PR restores the full end-to-end behavior plus
five regression tests so it can't silently break again.
## What was broken and what changed
1. **Random uploads crashed with `Failed to fetch`.** aimock returned
HTTP 404 on no-match, the LangGraph SDK surfaced `NotFoundError`, the
AG-UI stream surfaced a `RUN_ERROR`, the demo crashed. → Added
`--proxy-only` + `--provider-openai https://api.openai.com` to the local
aimock command so unmatched user prompts fall through to real OpenAI
(mirrors Railway).
2. **Bundled-sample fixtures keyed on user-visible canned prompts.**
Auto-prompts are now natural and specific ("can you tell me what is in
this demo image/pdf I just attached") so they render cleanly as the user
message bubble AND can't collide with arbitrary user prompts — random
uploads phrase questions differently and fall through to the proxy.
3. **Sample buttons now auto-send via `useAgent`.** The previous
DataTransfer path queued the attachment via the chat's hidden file input
but required clicking send while the attachment was still uploading —
`CopilotChat.onSubmitInput` rejects submits during upload AND clears the
input regardless, so the canned prompt was eaten. Rewrite calls
`agent.addMessage(...)` + `copilotkit.runAgent({ agent })` directly with
the base64'd content part.
4. **PDF flattened text bled into the rendered user message.**
`_PdfFlattenMiddleware` ran in `before_model` and persisted the rewrite
to agent state. Switched to `wrap_model_call` so the PDF→text rewrite is
scoped to the model request only.
5. **Attachments doubled (and PDFs rendered as broken `<img>`).** The
`@ag-ui/langgraph` round-trip mis-tags PDFs as `image` and re-injects
the user's original modern part, doubling chips. Added
`dedupeUserMessageMedia` subscriber on `onMessagesSnapshotEvent` +
`onRunFinalized` to dedupe by `source.value` and re-key type from
mimeType. Also flipped `onRunInitialized` from REPLACE to APPEND so the
modern part stays for the UI alongside a legacy `binary` sibling for the
converter.
6. **Regression suite (`tests/e2e/multimodal.spec.ts`).** Five focused
tests, all pass against live local stack (15.4s):
- page loads with all expected affordances
- sample image: auto-sends, EXACTLY ONE `<img>`, assistant references
the logo
- sample PDF: auto-sends, EXACTLY ONE `DocumentAttachment` chip ("PDF"
label), NO `<img>`, no `[Attached document]` text bleed
- image then PDF in the same session: each message keeps its own single
chip
- PDF then image in the same session: symmetric
## Test plan
- [x] `showcase up langgraph-python` — both sample buttons auto-send;
image renders as `<img>`, PDF renders as PDF chip; random paperclip
uploads go through proxy
- [x] `BASE_URL=http://localhost:3100 CI=1 npx playwright test
multimodal.spec.ts` — **5 / 5 passing**
- [ ] Post-merge: e2e-deep cycle for langgraph-python multimodal cell
stays green
## Closes
Closes#4584.
The langgraph-python multimodal-attachments demo had a stack of bugs
that compounded each other. Fixing them required touching the local
docker-compose, the aimock fixtures, the LangChain middleware, the
client-side AG-UI shim, and the sample-attachment buttons. This
commit lands the full set together because they only make sense as
a unit — verified end-to-end against `showcase up langgraph-python`
in a headed browser. New e2e suite pins each regression.
Supersedes #4584 (the original fix from May 1 that never landed —
this is a fresh port onto the post-refactor file layout where
page.tsx is split into legacy-converter-shim.tsx, multimodal-chat.tsx,
file-to-data-attachment.ts).
What was broken and what changed:
1. Random uploads crashed with `Failed to fetch`. aimock returned
HTTP 404 on no-match, the LangGraph SDK surfaced `NotFoundError`,
the AG-UI stream surfaced a `RUN_ERROR`, the demo crashed.
Added `--proxy-only` + `--provider-openai https://api.openai.com`
to the local aimock command so unmatched user prompts fall through
to real OpenAI (mirrors the Railway aimock setup).
2. Bundled-sample fixtures keyed on user-visible canned prompts.
The auto-prompts are deliberately long, specific, and natural-
reading ("can you tell me what is in this demo image/pdf I just
attached") so they (a) render cleanly as the user message bubble,
and (b) can't collide with arbitrary user prompts — random
uploads phrase questions differently and fall through to the
proxy.
3. Sample buttons now auto-send via `useAgent`. The previous
DataTransfer-based path queued the attachment via the chat's
hidden file input, then required clicking send while the
attachment was still uploading — `CopilotChat.onSubmitInput`
rejects submits during upload AND clears the input regardless,
so the canned prompt was eaten. Rewrite to call
`agent.addMessage(...)` + `copilotkit.runAgent({ agent })`
directly with the base64'd content part, sidestepping the
upload race entirely.
4. PDF flattened text bled into the rendered user message.
`_PdfFlattenMiddleware` ran in `before_model` and returned
`{"messages": rewritten}`, which persisted to agent state. The
chat UI then rendered the `[Attached document]\n<pdf body>` text
part inline with the user prompt. Switched to `wrap_model_call`
so the PDF→text rewrite is scoped to the outgoing model request
only and never pollutes state.
5. Attachments doubled (and PDFs rendered as broken `<img>`). The
`@ag-ui/langgraph` round-trip translates outgoing `binary` parts
to LangChain `image_url` and incoming `image_url` back to `image`
AG-UI parts — regardless of mimeType, so PDFs came back as
`type: "image"` with `mimeType: "application/pdf"` and were
forced into `ImageAttachment`, where the load failed and the
chat showed two "Failed to load image" boxes. Plus the user's
original modern part survived alongside the round-tripped one,
doubling visible chips.
Added a `dedupeUserMessageMedia` subscriber on both
`onMessagesSnapshotEvent` and `onRunFinalized` to:
- dedupe media parts by `source.value` so the local + round-
tripped copy collapse to one chip
- re-key part `type` from `mimeType` so PDFs route to
`DocumentAttachment` (icon + filename) and images to
`ImageAttachment`.
Also flipped the `onRunInitialized` shim from REPLACE to APPEND
— keep the modern part for the UI AND emit a legacy `binary`
sibling for the converter.
6. Regression suite (`tests/e2e/multimodal.spec.ts`). Replaces the
pre-rewrite suite with five focused tests:
- page loads with all expected affordances
- sample image: auto-sends, EXACTLY ONE `<img>`, assistant
references the logo
- sample PDF: auto-sends, EXACTLY ONE `DocumentAttachment` chip
("PDF" label), NO `<img>`, no `[Attached document]` text bleed
- image then PDF in the same session: each message keeps its own
single chip, no cross-contamination
- PDF then image in the same session: symmetric
All 5 pass against the live local stack (15.4s).
The voice route's OpenAI client previously fell through to OPENAI_BASE_URL,
which docker-compose.local.yml sets to http://aimock:4010/v1. Aimock has a
catchall transcription fixture that returns "What is the weather in Tokyo?"
for every audio file, so the mic button always produced that phrase no
matter what the user actually said.
Pin baseURL to real OpenAI (overridable via OPENAI_TRANSCRIPTION_BASE_URL).
The sample-audio button stays as synchronous text injection — that's the
documented design, and what the e2e + d5 probe rely on.
Also:
- Tidy the sample button label ("Try a sample question" -> "Try a sample
audio") so the affordance matches what it does.
- Realign tests/e2e/voice.spec.ts with the shipped component (the
voice-sample-audio container testid and Sample: "..." caption it asserted
on never existed on HEAD) and add cold-start timeout headroom for the
mic-button render and the agent-flow test.
- Add "env": ".env" to langgraph.json so langgraph_cli dev picks up
OPENAI_API_KEY locally. Docker/Railway paths inject env vars directly so
this is a no-op there.
The specs and QA markdowns had drifted from the demos they describe.
This commit brings every test contract into line with the actual demo
source — eliminating false-greens, false-fails, and stale assertions.
False-fail spec assertions (would fail every run):
- `agentic-chat.spec.ts` — rewrote from the old `change_background` /
`weather-card` / `useAgentContext` flow that no longer exists. New
spec exercises the vanilla `<CopilotChat>` + three suggestion pills
contract the simplified demo actually exposes.
- `gen-ui-tool-based.spec.ts` — asserted on UI text ("Use the sidebar
to generate charts", "Chart Generator") that doesn't exist; switched
to suggestion-pill assertions and scoped the SVG check to inside the
assistant-message bubble (was matching CopilotChat's send-button
SVG).
- `agent-config.spec.ts` — asserted heading "Agent Config Object" but
the demo has "Agent Config".
- `multimodal.spec.ts` — asserted a non-existent "Multimodal
attachments" heading; switched to the `multimodal-demo-root` testid.
- `chat-slots.spec.ts` — asserted `[data-testid="custom-assistant-
message"]` and the bare text "slot" — neither exists. The actual
signal is `data-slot-label="MessageView.AssistantMessage"` from the
SlotMarker wrapper.
- `reasoning-default.spec.ts` — asserted `[data-testid="copilot-
reasoning-message"]` and `[data-message-role="reasoning"]`; neither
is emitted by `CopilotChatReasoningMessage`. Switched to the text-
based "Thinking…/Thought for…" header label.
False-green spec assertions (passed for the wrong reason):
- `shared-state-read.spec.ts` — was a complete false-green: asserted
on "Sales Pipeline", "Total Pipeline", "Active Deals" but the demo
has been a Recipe Editor for some time. Rewrote against the
recipe-card / ingredients-container / instructions-container testids.
- 11 specs (agent-config, beautiful-chat, frontend-tools-async,
gen-ui-tool-based, gen-ui-agent, gen-ui-interrupt, hitl-in-chat,
hitl-in-app, multimodal, readonly-state-agent-context, voice) used
`[data-role="assistant"]` to gate "agent responded" — but the v2
react-core bundle never emits that attribute (it ships
`data-testid="copilot-assistant-message"`). Mechanical sweep to the
correct testid.
- Deleted `shared-state-write.spec.ts` (route consolidated into
`shared-state-read-write` earlier on this branch — spec targeted a
removed demo) and `renderer-selector.spec.ts` (asserted on a radio-
pill UI that no longer exists; the four "Declarative UI" variants
are now separate manifest demos).
QA drift:
- `qa/gen-ui-tool-based.md` documented a "Haiku Generator" demo with
haiku-card / japanese-line / english-line / haiku-image testids — a
demo that doesn't exist anywhere on this branch. Rewrote to match
the chart-rendering demo's actual testids and pill prompts.
- `qa/chat-slots.md` referenced "Custom Slot" pill / "Welcome to the
Slots demo" heading / "This welcome card is rendered via the
welcomeScreen slot." body text — all of which the slot-wrappers
refactor on this branch removed. Updated to match the
`custom-welcome-message` sub-slot that's actually rendered. Also
fixed max-w-4xl → max-w-5xl to match the page.
- `qa/shared-state-read.md` said default instruction is "Preheat oven
to 350 F" but the source has "Preheat oven to 350°F (175°C)".
- `qa/agentic-chat.md` rewrote to match the simplified vanilla-chat
demo (the previous QA documented `change_background` / `WeatherCard`
flows that no longer exist).
- `qa/reasoning-default.md` cited `kind: "testing"` in feature-
registry.json for the `reasoning-default` entry; the registry entry
has no `kind` field. Rewrote without the false cross-file claim.
- Deleted 4 orphan QA files for demos that don't exist:
`agentic-chat-reasoning.md`, `hitl.md`, `hitl-in-chat-booking.md`,
`shared-state-write.md`.
- Renamed `qa/reasoning-default-render.md` → `qa/reasoning-default.md`
to match the manifest cell name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three classes of regression are now pinned:
1. Secondary-LLM tool name doesn't collide with the A2UI middleware's
default intercept list (`render_a2ui`). New
`src/agents/test_a2ui_internal_tools.py` parametrises over
`beautiful_chat._design_a2ui_surface`, `a2ui_dynamic._design_a2ui_surface`,
and `a2ui_fixed.display_flight` and asserts none match the
middleware's `a2uiToolNames` default. Catches accidental rename
reverts that would re-enable the bypass.
2. `generate_a2ui` force-pins the canonical `catalog_id` even when the
secondary LLM hallucinates a wrong one. The new test stubs
`ChatOpenAI` with a fake response carrying a bogus catalogId and
asserts the surface op carries the module's `CUSTOM_CATALOG_ID`.
3. `generate_a2ui` short-circuits with a clean error string when the
LLM emits a root component without a `component` field — never
feeds the renderer the partial tree that surfaced as the "Cannot
create component root without a type" infinite-loop.
7 unit tests, all green locally (`pytest src/agents/test_a2ui_internal_tools.py`).
E2E tests on the same fixes now also assert:
- No `A2UI render error: Catalog not found` banner on the page after
Beautiful Chat → Sales Dashboard, Declarative Gen UI → BarChart, and
A2UI Fixed Schema → Find SFO → JFK round-trips.
- No `Cannot create component … without a type` banner on the same
three pills.
- Exactly ONE flight card on A2UI Fixed Schema (was 6+ on deploy
pre-fix from the `display_flight` loop) — `Flight Details` count
pinned to 1, `Book flight` count pinned to 1.
- At most one ResponsiveContainer on the BarChart pill (loops would
stack multiple).
- At most two ResponsiveContainers on Beautiful Chat → Sales Dashboard
(one pie + one bar = single dashboard render).
Two follow-up fixes layered on the previous internal-tool rename:
(1) `a2ui_fixed.py` — fixed-schema demo infinite loop on deploy. The
`display_flight` tool returns the raw `a2ui.render(...)` JSON descriptor
as its tool result. gpt-4o-mini reads that opaque blob, can't tell the
flight was rendered, and re-calls `display_flight` indefinitely (visible
on the deployed showcase as 6+ duplicate flight cards stacked under
repeated assistant text). Local was just lucky.
Hardened the docstring + system prompt to spell out: the JSON return
value is the surface descriptor, the card is already rendered, do NOT
call again, reply with one short confirmation and stop.
(2) Rename `render_a2ui` → `_design_a2ui_surface` in shared and
langgraph-python parity copies of `tools/generate_a2ui.py` (+
`tools/__init__.py` re-export `RENDER_A2UI_TOOL_SCHEMA` →
`DESIGN_A2UI_SURFACE_TOOL_SCHEMA`), and in `showcase/shared/typescript/
tools/generate-a2ui.ts`. These shared helpers were the source-of-truth
for the secondary-LLM tool name across integrations; renaming here keeps
parity with the langgraph-python agents already renamed in
`beautiful_chat.py` / `a2ui_dynamic.py`. Other framework integrations
keep their own `render_a2ui` for now (separate parity sweep).
(3) `showcase/aimock/feature-parity.json` — added a sibling fixture
matching `toolName: "_design_a2ui_surface"` for the beautiful-chat Sales
Dashboard pill so the langgraph-python e2e suite still hits a
deterministic mock on Railway. The original `render_a2ui` fixture is
kept above it so other integrations whose secondary LLM still requests
`render_a2ui` continue to match.
(4) Comment update in `beautiful-chat.spec.ts` to name the new internal
tool.
CR Round 3 final: my auth.spec.ts e2e was asserting on
[data-message-role="assistant"] which is the v1 react-ui RenderMessage
attribute. The auth demo uses v2 CopilotChat — its
CopilotChatAssistantMessage only emits data-testid="copilot-assistant-message".
The selector would never have matched and both tests would have
timed out at 30s when actually run.
Verified via grep:
- packages/react-core/src/v2/components/chat/CopilotChatAssistantMessage.tsx:192
emits data-testid="copilot-assistant-message" (no data-message-role)
- packages/react-ui/src/components/chat/messages/RenderMessage.tsx:32,41
emits data-message-role="user"/"assistant" (v1 path)
- All sibling specs in langgraph-python/tests/e2e/ correctly use
data-testid="copilot-assistant-message"
One-character switch from data-message-role to data-testid with the
canonical v2 testid value.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The auth demo refactor flipped its lifecycle: unauthenticated is now
the default state, <CopilotKit> only mounts after sign-in, and
sign-out unmounts the entire chat tree (instead of leaving stale
auth headers in a still-mounted chat). The old probe + e2e spec
chased a 401-error-banner surface that no longer exists in the new
demo, plus a brittle 500ms hardcoded `useEffect` flush wait.
Probe rewrite (`d5-auth.ts` + tests):
- Add `buildAuthPreFill` that clicks the SignInCard's sign-in button
before turn 1, then waits for the chat textarea to mount (proves
<CopilotKit> handshook with the runtime).
- `buildAuthAssertion` now clicks sign-out, then waits for SignInCard
to re-mount. The unmount marker IS the proof — no chat-send-and-401
dance is needed (or possible — there's no chat to send into).
- Drop the hardcoded 500ms setTimeout, the unauth-banner wait, and
the error-surface poll. None apply to the new flow.
E2E rewrite (`tests/e2e/auth.spec.ts`):
- "page loads unauthenticated with SignInCard visible"
- "signing in mounts the chat surface with AuthBanner"
- "authenticated send produces an assistant response"
- "signing out unmounts the chat tree and re-renders SignInCard"
- "signing back in re-mounts a fresh chat surface"
Fixture comment updated to reflect the new flow. The user message
("auth check turn 1") and content response are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HELLO_LEADING phrase was the showcase-assistant catch-all
boilerplate ('I can help you with weather lookups...') that other
tests in this PR explicitly guard AGAINST. The dedicated d5-all.json
fixture for 'Say hello in one short sentence' now returns a distinct
non-boilerplate greeting; the spec asserts that distinct phrase, so a
fixture-priority misroute fails loudly instead of passing by accident.
Resolved d5-all.json conflict by appending B6's headless-simple/complete fixtures (Say hello, joke, fun fact, Highlight, chart) before the tool-rendering and open-gen-ui entries already on integration.
All three tool-rendering cells now drive the suggestion pills directly
instead of typing free-form prompts that race fixture matchers:
- tool-rendering: 6 tests (page loads + 5 pills) — Weather in SF
asserts SF city + deterministic temp/humidity/wind; Find flights
asserts >=2 flight rows from the dedicated fixture; Stock price
asserts AAPL $338.37 / -2.96%; Roll a d20 asserts exactly 5 d20
cards with the 5th=20; Chain tools asserts weather+flights+d20
cards mount from a single pill click.
- tool-rendering-default-catchall: 6 tests asserting the OOTB default
tool-call renderer paints every tool call with data-testid=
copilot-tool-render plus data-tool-name. Branded sibling-cell
testids stay at zero. Test 6 asserts every card matches the
built-in renderer DOM signature.
- tool-rendering-custom-catchall: 6 tests asserting the same
custom-wildcard-card testid renders for every tool. Test 6 is the
cross-tool snapshot — every tool kind paints via the same shell.
hitl-in-app — 7 explicit tests (1 skip):
- Page-load and pill-render tests retained.
- New refund #12345 approve/reject pair asserts the deterministic
fixture leading phrases ("I am processing the $50 refund" vs
"refund request was not approved").
- New escalate #12347 approve/reject pair asserts "Escalated ticket
#12347" vs "Not escalated".
- The describe block runs in serial mode so the approve test
(sequenceIndex 0 in the fixture) always runs before the reject test
(sequenceIndex 1) for each pill.
- Downgrade #12346 stays skipped per spec (broken upstream as of
2026-05-07).
frontend-tools-async — 4 explicit tests:
- Page-load test asserts composer + 3 pills.
- Project-planning, auth, and reading pills each click and assert the
Notes DB card renders with the correct keyword heading and the
per-note testid rows that the async handler returned. Anti-regression
assertions catch the previous fixture-priority bugs (generic-plan
boilerplate, showcase-assistant catch-all).
- Reading pill locks the full canonical shape per spec test #4: keyword,
match count, note title, content lines, tag chip, and the assistant
narration leading phrase.
No production-code testid changes — the existing dialog and notes-card
testids cover every assertion. Fixture work lives in d5-all.json (prior
commit).
- Drop the 8 stale tests that asserted travel-planner shapes
(Current Itinerary / supervisor-indicator / .bg-gray-50). They
predated the supervisor + 3-subagent rewrite and could not catch
the 3 production bugs.
- New suite (5 tests, 0 skipped):
1) page loads with composer + 3 verbatim suggestion pills + 3
subagent role indicators (testid-based).
2-4) one test per pill (Write a blog post / Explain a topic /
Summarize a topic). Each clicks the pill, waits for all
three role-scoped subagent cards to reach data-status=
"complete", then asserts each card's subagent-result is
non-empty AND does not contain the showcase-assistant
boilerplate fragments. Test 4 also serves as a regression
gate on the delegations reducer fix — without it, that pill
returns HTTP 400 and the cards never reach complete.
5) clicks any pill, waits for terminal state, then asserts the
critic-card count is exactly 1 and stays at 1 with status
complete across a 5s dwell — catches any return of the
supervisor -> critic loop.
- Aimock fixtures: 3 verbatim pill chains (cold exposure training,
LLM tool calling, reusable rockets) added to both d5-all.json and
the harness source d5/mcp-subagents.json. Each chain drives the
full supervisor flow (turnIndex 0..3) plus three nested sub-agent
fixtures so every Researcher/Writer/Critic card surfaces real
prose instead of showcase boilerplate.
Drop the 5 cross-origin contentFrame() / page.on('console', ...) skipped
assertions across the two specs — sandbox=allow-scripts only blocks host
introspection of the iframe DOM, and console-spying on the host page
catches no inner-iframe logs. Replace with iframe-presence assertions:
each pill click must produce iframe[sandbox*='allow-scripts'] with a
non-empty srcdoc (or src) attribute. That is the load-bearing signal
that the open-generative-ui pipeline mounted SOMETHING.
Rewrite each suggestion message string as a short verbatim label that
doubles as a deterministic aimock fixture key (paired with the new
fixtures in showcase/aimock/d5-all.json). Drop pill-title parentheticals
per the cosmetic note in lgp-test-genuine-pass.md so titles read as
natural human prompts; keep the message field aligned with the fixture
key.
Final test counts: 5 minimal (page-load + 4 pill-iframe), 4 advanced
(page-load + 3 pill-iframe). All .skip() removed. The sandbox-function
round-trip (evaluateExpression / notifyHost) is intentionally not
asserted here — that requires a same-origin sandbox option or a
host-side spy on the runtime's sandbox-function-call event, both
deferred to a follow-up.
Rewrites both headless specs to the pill-driven plan from
.claude/specs/lgp-test-genuine-pass.md (Family-1F).
headless-simple (4 tests):
1. page loads with custom composer + 3 pills visible
2. hello pill -> assistant bubble starts with the greeting leading phrase
3. joke pill -> assistant bubble contains the deterministic joke
4. fun fact pill -> assistant bubble contains 'Honey never spoils!'
headless-complete (5 tests):
1. page loads with custom composer + 4 pills visible
2. weather pill -> WeatherCard with Tokyo / Sunny / 68F + narration
3. AAPL pill -> StockCard with AAPL / $189.42 / +1.27% + narration
4. highlight pill -> HighlightNote with 'ship the demo on Friday' + narration
5. revenue chart pill -> ChartCard with 'Quarterly revenue', subtitle,
month labels Jan-Jun + narration
Each test asserts on the headless-specific testids introduced in the
preceding commit, so a regression that demotes the headless surface
back to the default CopilotChat surface fails every tool test. Each
pill exercises a different render-hook path so regressions surface
test-by-test.
No .skip() — all 9 tests are live.
Replace the previous 4-test (2 active, 2 skipped) suite with a
5-test deterministic plan keyed off the demo's actual published
context defaults and pill verbatim prompts.
Tests:
1. page loads — context-card + composer render
2. edits propagate to JSON — type into name/timezone, JSON updates
3. "Who am I?" pill — assistant reply leads with "I see you're Atai"
and the identity card has name=Atai, timezone=America/Los_Angeles,
avatar text=A (defaults from page.tsx)
4. activity checkboxes default-checked — "viewed the pricing page"
and "watched the product demo video" are checked on first paint
5. "Suggest next steps" pill — assistant reply leads with "Since
you recently viewed the pricing page and watched the product
demo video"
Tests 3 and 5 rely on the deterministic aimock fixtures added in
the previous commit. Tests 4 uses the new activity-<slug> testids.
0 .skip() remaining.
Cross-cutting changes that don't belong with any one demo: manifest +
landing-page tags, runtime route adjustments, e2e + QA notes that
follow the demo renames, and a few small cleanups.
Manifest (manifest.yaml + src/app/page.tsx tag labels):
- Naming convention: every demo uses `Thing: Subthing` (Generative UI:
Tool Rendering - Default / Custom Default / Specific; Open Generative
UI: Default / Advanced; Shared State: Streaming / Read + Write;
Reasoning: Default / Custom; Frontend Tools: In-App Actions / Async;
Human in the Loop: In-chat / In-App / Interrupt based; Chat
Customization: CSS / Slots; Headless UI: Simple / Complete).
- Retags: Auth → `platform`; HITL Step Selection + Interrupt-based →
`interactivity`; Reasoning Default + Custom → `chat-ui`; Generative
UI: Tools → `generative-ui`.
- Renames: Readonly State (Agent Context) → Frontend Context Sharing.
- HITL slot points at /demos/hitl-in-chat (working
useHumanInTheLoop+interrupt path) instead of the previous
/demos/hitl that had no backend `interrupt()` calls.
- Highlight paths corrected for the rebuilt headless demos (root-level
paths replaced with hooks/, chat/, tools/, attachments/ subdirs).
- Descriptions rewritten where they had drifted from the implementation
(gen-ui-agent: dropped useCoAgentStateRender claim;
headless-simple: shadcn primitives, not raw Tailwind;
headless-complete: enumerates the actual hooks wired).
Runtime / route:
- src/app/api/copilotkit/route.ts — 30 agents registered (incl. the
reasoning-custom rename from agentic-chat-reasoning).
- copilotkit-mcp-apps/route.ts replaced with [[...slug]]/route.ts so v2
subpath POSTs (/v2/agent/run) resolve.
- src/app/api/copilotkit-voice/[[...slug]]/route.ts — env var standardized
(was `AGENT_URL || LANGGRAPH_DEPLOYMENT_URL`, now matches the rest
of the showcase with just LANGGRAPH_DEPLOYMENT_URL); trailing `/`
removed from deploymentUrl.
Tests / QA:
- e2e specs renamed and paths updated for the demo renames.
- qa notes for a2ui-fixed-schema (booked-state checklist removed) and
byoc-json-render (Wave 4a residue removed).
- docs-links.json key renamed for reasoning-custom.
Cleanup:
- Removed remaining stub agent.py files in demo dirs (real graphs in
src/agents/); removed dead beautiful-chat/components/headless-chat.tsx
(zero importers); removed [A2UI-DEBUG] / [A2UI-RESPONSE] print
statements from beautiful_chat.py; gpt-5.4-mini → gpt-5-mini typo
fix in beautiful_chat.py:249 (would have 4xx'd every model call);
stripped iframe-restriction LLM-prompt copy bleed from
open-gen-ui-advanced suggestion titles.
The convention pass that ran across ~28 demos earlier in this branch is
already reflected in their per-demo commits — every page.tsx reads as
imports + provider + suggestions hook + JSX, with `useConfigureSuggestions`
extracted to a sibling suggestions.ts.
A pair of demos that exercise the same backend reasoning graph but
differ only in whether the frontend overrides the
`messageView.reasoningMessage` slot.
Backend (src/agents/reasoning_agent.py): uses a reasoning-capable OpenAI
model (gpt-5-mini by default, override via OPENAI_REASONING_MODEL) routed
through the Responses API so the model's chain-of-thought streams as
AG-UI REASONING_MESSAGE_* events with `role: "reasoning"`. The prompt
asks for a concrete physics answer, which reliably triggers reasoning;
meta-prompts like "show your reasoning step by step" produce no
reasoning summary because the model recognizes those as a request to
reveal chain-of-thought (which it refuses).
Frontend:
- reasoning-default/ — no slot override; built-in
CopilotChatReasoningMessage renders the "Thinking… / Thought for X"
header with an expandable content region.
- reasoning-custom/ — overrides `messageView.reasoningMessage` with a
ReasoningBlock (amber banner with `data-testid="reasoning-block"`).
The label flips from "Thinking…" while streaming to "Agent reasoning"
once the stream settles.
Suggestions live in their own files (per the page-as-entry-point
convention). Both demos share `agent="reasoning-default"` /
`agent="reasoning-custom"` against the same `reasoning_agent` graph,
registered in api/copilotkit/route.ts.
Removed:
- src/app/demos/agentic-chat-reasoning/ — replaced by reasoning-custom/
for naming clarity.
- src/app/demos/reasoning-default-render/ — earlier draft of the Default
demo with a slightly different page name.
- tests/e2e/agentic-chat-reasoning.spec.ts — replaced by
reasoning-custom.spec.ts.
The langgraph-python voice cell sat at D4 even when its d5-voice probe
row was green. Root cause: the dashboard's CATALOG_TO_D5_KEY mirror in
showcase/shell-dashboard/src/lib/live-status.ts was missing voice ->
["voice"], so computeMaxPossible capped voice at D4 regardless of probe
state. The harness REGISTRY_TO_D5 already had the entry; only the
dashboard mirror was out of sync.
Separately, the "Play sample" button used to fetch sample.wav and POST
it to /transcribe. With aimock that meant both the sample button AND
the mic returned the same canned response, which made it impossible to
demo the mic path locally without conflating the two affordances.
Reworked the button into a synchronous static-text injector
(onTranscribed(sampleText)) so:
- Sample button = deterministic test/demo affordance, no runtime calls.
- Mic = real Whisper transcription via /transcribe.
Synced across all 18 voice-enabled integrations. Phrase stays "What is
the weather in Tokyo?" so aimock's "weather in Tokyo" substring fixture
still matches.
Also adds the missing d5-voice.test.ts companion (every other d5-* probe
script has one) and trims the langgraph-python qa/voice.md + e2e steps
that depended on the now-removed async behavior.
## Summary
Adds hand-rolled persistent suggestion chips to the `headless-simple`
and `headless-complete` demos in the langgraph-python north-star,
propagates the same surface to the other 17 showcase integrations, and
adds a deterministic aimock fixture so a new chip-click e2e test
("Largest continent") rounds-trips against a stable `Asia is the largest
continent…` response across all 18 demos.
## What changed
**Phase 0 — north-star (commit `7cbc5ea8`)**
- `showcase/aimock/feature-parity.json` — new fixture: `What is the
largest continent?` → `Asia is the largest continent — about 30% of
Earth's land area, home to over 4.6 billion people.`
- `langgraph-python/src/app/demos/headless-{simple,complete}/page.tsx` —
refactor `send` / `handleSubmit` to accept `(override?: string)` so chip
clicks dispatch synchronously without a `setInput` round-trip; render a
persistent `<div data-testid="headless-suggestions">` chip row above the
composer with 5 canonical entries; remove the dead
`useConfigureSuggestions` call from headless-complete (it was
registering suggestions nothing rendered).
- `langgraph-python/tests/e2e/headless-{simple,complete}.spec.ts` —
append one new test in each spec asserting chip click → user message →
`Asia` reply.
**Phase 1 — parity propagation across 17 integrations (commit
`4882c61f`)**
- Spec files `headless-simple.spec.ts` and `headless-complete.spec.ts`
are now byte-identical to the north-star in every integration (10 tests
each = 5 simple + 5 complete; verified via `cmp` for all 34 spec files).
- The 5-entry `suggestions` const is byte-identical between every
integration's simple and complete demos.
- All 17 integrations now expose the same selector surface (canonical
headings, empty-state text, `data-testid="headless-complete-messages"`,
dynamic placeholder, `rounded-br-sm` user bubble, no CopilotChat-default
testids).
**Glue preserved per integration** (verified by post-blitz code review):
- `built-in-agent`: `<CopilotKitProvider runtimeUrl="/api/copilotkit"
useSingleEndpoint>` + `agentId: "default"`
- `google-adk` / `llamaindex`: `agentId: "headless_simple"` /
`"headless_complete"` (Python-style underscores)
- `claude-sdk-typescript`: headless-complete
`runtimeUrl="/api/copilotkit-headless-complete"`
- `spring-ai`: 70-line `deduplicateMessages` adapter workaround +
`useMemo` import preserved verbatim
- All `@region[...]` markers preserved in place
**Adapter-specific decisions worth flagging in review:**
- `google-adk` headless-complete: rewrote `message-list.tsx` from
`msg-user`/`msg-assistant`/`agent-thinking` testid scheme to the
canonical `headless-complete-messages` wrapper; rewrote `input-bar.tsx`
placeholder to canonical dynamic; added the missing subtitle and
empty-state hint
- `ms-agent-dotnet`: extracted inline composer to a new `input-bar.tsx`
to match north-star structure
- `llamaindex`, `ms-agent-python`: added the canonical empty-state hint
(was missing entirely)
- `agno`, `built-in-agent`, `crewai-crews`, `mastra`, `ms-agent-dotnet`,
`pydantic-ai`: replaced per-integration empty-state hint with the
canonical Excalidraw line — chosen for parity over per-integration
accuracy (some demos don't actually wire an Excalidraw tool; alignment
was the explicit goal)
## Verification
- `validate-parity.ts`: 18/18 packages pass, 0 MUST failures
- `aimock-fixtures` test suite: 18/18 pass
- aimock fixture probed directly: `What is the largest continent?`
returns the canonical Asia response
- Each propagation slot reported `tsc --noEmit` clean (0 new errors) +
`playwright --list` shows all 10 expected tests
- Code review (`pr-review-toolkit:code-reviewer`) on the full diff: 0
Critical / Important / Minor findings, 1 stylistic nit (north-star
`input-bar.tsx` `onSubmit` type contravariant-loose, harmless)
## What was NOT done
Live per-integration Playwright runs against rebuilt Docker images. The
17 containers would each need a no-cache rebuild (~5-15 min each = hours
total) and the canonical local-test path is `showcase test <slug>` per
the existing CLI / CI pipeline. Static + structural verification covers
the propagation pattern.
## Test plan
- [ ] Run `showcase test <slug>` (or equivalent CI job) for at least one
drift-heavy integration: `google-adk` (testid scheme rewrite),
`built-in-agent` (provider glue), `spring-ai` (dedup workaround),
`llamaindex` (added testid + empty-state)
- [ ] Run the existing per-integration Playwright suites for at least
the north-star (`langgraph-python`) to confirm the new chip test passes
against a real backend + aimock
- [ ] Confirm aimock fixture validation still passes after deploy
Search Flights and Sales Dashboard pills both produce visible surfaces
on the langgraph-python beautiful-chat demo. Three independent bugs were
masking each other:
- Flight TypedDict required `id` + `statusIcon`, which the aimock fixture
doesn't supply. langchain rejected the call with `flights.0.id: Field
required` and the agent surfaced the error string as the tool result.
Made the type permissive (only the fields `_build_flight_components`
reads need to be there).
- search_flights now expands flights into literal-children FlightCard
components server-side instead of relying on the structural-children
template form (the binder doesn't reliably expand it for our custom
catalog — sibling demos avoid the form for the same reason).
- Sales Dashboard pill went into a tool-call loop because the
userMessage+toolName fixtures matched both the initial call and the
post-tool turn. Hoisted the toolCallId fixture above them so the
follow-up turn returns content and breaks the loop.
Custom Row/Column reintroduced with `gap` support — the basic catalog's
versions ignore it, leaving cards squished. Children are array-of-strings
only (matches what the agent and fixture emit).
Two new e2e tests cover both pills end-to-end. 3s wait in beforeEach so
the v2 chat provider hydrates before the click dispatches. Full spec:
7/7 green.
## Summary
The `agentic-chat-reasoning` and `reasoning-default-render` cells in
`langgraph-python` and `langgraph-fastapi` never rendered any reasoning
content. Root cause: both agents were configured with `gpt-4o-mini` +
`use_responses_api=False`, so the underlying model produced no reasoning
content blocks and the Chat Completions API has no reasoning summary
surface in the first place. The frontend's `reasoningMessage` slot
stayed empty even though the cells are billed as reasoning demos.
This PR:
- Switches both agents (and their `tool_rendering_reasoning_chain`
siblings) to `gpt-5-mini` through the Responses API with
`reasoning={"effort":"medium","summary":"detailed"}`, mirroring the
`langgraph-typescript` and `pydantic-ai` agents that already worked.
Model is overridable via `OPENAI_REASONING_MODEL`.
- Updates the aimock `d5-all.json` fixture (and the matching harness
`reasoning-display.json`) to set the `reasoning` field on the `show your
reasoning step by step` match. Aimock now emits
`response.reasoning_summary_text.delta` events so the demo renders
deterministically without a real LLM call.
- Adds a `Show reasoning` `useConfigureSuggestions` pill on both
reasoning pages in both integrations so the demo is one click to
exercise.
- Tightens the `d5-reasoning-display` probe to also assert that a
reasoning-role message rendered (`[data-testid="reasoning-block"]` or
`[data-message-role="reasoning"]`), not just that the word "reasoning"
appears in the transcript.
- Un-skips the three streaming reasoning-block tests in
`agentic-chat-reasoning.spec.ts`, adds a suggestion-pill test, and
extends `reasoning-default-render.spec.ts` to cover the default
reasoning slot.
- Updates the `langgraph-python` QA doc to describe the new model +
Responses API setup and the pill flow.
Verified locally end-to-end: clicking the pill at
`/demos/agentic-chat-reasoning` renders the amber `ReasoningBlock` with
the fixture's reasoning text above the final answer bubble.
## Out of scope
Other integrations were audited and intentionally left alone:
- `langgraph-typescript`, `pydantic-ai` already use a reasoning model +
Responses API and work today.
- `agno`, `claude-sdk-python`, `ms-agent-python` use deliberate
workarounds (XML-tag reasoning + custom AGUI handler, Claude
extended-thinking deltas, `think` tool respectively) because their AG-UI
bridges either don't translate Responses-API reasoning items, run a
multi-call CoT loop incompatible with fixture replay, or don't emit
reasoning events at all.
- `llamaindex` uses `gpt-4.1` and surfaces reasoning inline as assistant
text. Its bridge (`llama-index-protocols-ag-ui`) does not translate
Responses-API reasoning items into AG-UI events; fixing that needs an
upstream patch and is out of scope here.
## Notes
Committed with `--no-verify` (explicit user request) — this worktree has
no `node_modules`, so the lefthook `test-and-check-packages` step
couldn't run locally. Changes are entirely under `showcase/` and CI runs
the same checks.
## Test plan
- [ ] CI fixture-validation passes on `showcase/aimock/d5-all.json`
- [ ] `showcase test langgraph-python --d5 --verbose` —
`reasoning-display` probe green (asserts `reasoning-block` selector +
keyword)
- [ ] `showcase test langgraph-fastapi --d5 --verbose` — same
- [ ] `nx run @copilotkit/showcase-langgraph-python:test:e2e -- --grep
reasoning` — un-skipped specs pass against the deployed Railway image
- [ ] Manual: visit `/demos/agentic-chat-reasoning` on a deployed
langgraph-python, click `Show reasoning`, confirm amber `REASONING —
Agent reasoning` block renders with italic step text above the final
answer bubble
- [ ] Manual: same on `/demos/reasoning-default-render`, confirm
CopilotKit's default `CopilotChatReasoningMessage` card renders
The agentic-chat-reasoning and reasoning-default-render cells in
langgraph-python and langgraph-fastapi were configured with
gpt-4o-mini + use_responses_api=False, which never produces AG-UI
REASONING_MESSAGE_* events: gpt-4o-mini is not a reasoning model and
the Chat Completions API does not surface reasoning summary items at
all. The frontend's reasoningMessage slot was rendering nothing,
even though the cells were billed as "reasoning" demos.
- Switch both reasoning agents to gpt-5-mini (override via
OPENAI_REASONING_MODEL) routed through the Responses API with
reasoning={"effort":"medium","summary":"detailed"} so the model's
chain of thought streams as content blocks that @ag-ui/langgraph
translates into REASONING_MESSAGE_* events.
- Update the aimock d5-all.json and harness reasoning-display.json
fixtures to include a "reasoning" field so aimock emits
response.reasoning_summary_text.delta SSE events deterministically
in CI without hitting a real LLM.
- Add a "Show reasoning" useConfigureSuggestions pill on both
reasoning demo pages so the user can trigger the fixture-matched
prompt with one click.
- Tighten the d5-reasoning-display probe: it now also asserts a
reasoning-role message rendered via [data-testid="reasoning-block"]
or [data-message-role="reasoning"], so a plain text response
containing the word "reasoning" no longer falsely passes.
- Un-skip the three streaming reasoning-block tests in
langgraph-python's agentic-chat-reasoning.spec.ts and add a
suggestion-pill test; expand the reasoning-default-render spec to
cover the default reasoning slot.
- Update the langgraph-python QA doc to describe the new model +
Responses API setup and the suggestion-pill flow.
## Summary
The hitl-in-chat demo's **"Schedule a 1:1 with Alice next week to review
Q2 goals."** suggestion was being intercepted by the broad `userMessage:
"Alice"` matcher used by the memory/context demo, which returns a
generic "Nice to meet you, Alice! I see you're in Tokyo — wonderful
city..." greeting. The HITL flow never fired and the user saw a
nonsensical reply.
Aimock's matcher uses `text.includes(match.userMessage)` (substring) +
first-fixture-wins by file order, so any message containing "Alice"
hijacked the suggestion before the HITL flow could trigger.
## Fix
Added a fixture pair earlier in `showcase/aimock/feature-parity.json`
with the **full suggestion sentence** as the matcher:
- `hasToolResult: false` → returns a `book_call` toolCall, letting the
frontend `useHumanInTheLoop` render the time-picker.
- `hasToolResult: true` → returns the booking confirmation message.
The substring-match-on-full-sentence is effectively exact — no other
realistic user message will contain that whole sentence — so the broad
`Alice` / `alice` fixtures stay scoped to the memory demo where the user
actually says "I'm Alice" or similar.
## Test plan
- [ ] Click "Schedule a 1:1 with Alice next week to review Q2 goals." in
the langgraph-python hitl-in-chat demo against an aimock-backed
deployment → expect the time-picker card to render and a booking
confirmation after picking a slot.
- [ ] The memory/context demo (where users type "I'm Alice") still gets
the Tokyo greeting — broad fixtures unchanged.
- [x] Pre-commit hooks pass (test, check-packages, commitlint).
Bug: in a single chat session, running both HITL booking flows
back-to-back (Alice 1:1 → then Sales call without refresh) used to
skip the time-picker on the second flow and jump straight to
"Booked ..." text.
Cause: confirmation fixtures were matched on `hasToolResult: true`,
which fires whenever the conversation has ANY tool message in
history. After the first flow finished, the second user message
short-circuited to a confirmation match before the second flow's
toolCall fixture (gated on `hasToolResult: false`) had a chance to
fire. The picker never rendered.
Fix: re-key the two confirmation fixtures on `toolCallId` (the
specific tool_call_id of the matching `book_call` invocation), which
only fires when the LAST conversation message is a tool result with
that id — exactly the moment we want the confirmation. Drop the
`hasToolResult: false` constraint on the toolCall fixtures so they
match a fresh user request regardless of prior tool history.
Add a back-to-back regression test to all 17 hitl-in-chat specs:
walk Alice flow to completion, then sales flow without refresh,
assert two `time-picker-card` elements rendered. If the multi-flow
regression returns, the second card never appears and the test
fails at `toHaveCount(2)`.
The hitl-in-chat demo ships in 17 integrations (langgraph-python plus
16 others — mastra, strands, ag2, agno, crewai-crews,
langgraph-typescript, langgraph-fastapi, pydantic-ai, llamaindex,
langroid, claude-sdk-python, claude-sdk-typescript, ms-agent-python,
ms-agent-dotnet, spring-ai, google-adk). All shipped placeholder e2e
specs that only checked the chat input was visible — none exercised
the actual booking flow.
Replace each with the full booking-flow spec written for
langgraph-python:
1. The "Schedule a 1:1 with Alice" suggestion renders the time-picker
card AND the Tokyo greeting is absent (regression guard against
the broad aimock `userMessage: "Alice"` matcher).
2. Picking a slot transitions to the picked-state card and produces
a "Booked … Alice" assistant follow-up.
3. The "Book a call with sales" suggestion runs the same flow with
the sales attendee.
Also add the matching aimock fixture pair for the sales suggestion
in feature-parity.json — without it, case 3 would only pass against
real OpenAI, not the aimock-backed CI deployments. The pair mirrors
the Alice fixture pair: `book_call` toolCall on first turn,
confirmation message after the picker resolves.
Per-integration coverage matters because each integration has its
own framework-specific HITL wiring (`useHumanInTheLoop` binding to
the agent, agent-side tool registration, run streaming protocol)
that can regress independently of the shared aimock fixture.
Pins the contract that the new full-sentence aimock fixture pair beats
the broad `userMessage: "Alice"` matcher:
1. Sending the suggestion `"Schedule a 1:1 with Alice next week to
review Q2 goals."` renders `[data-testid="time-picker-card"]`,
not the Tokyo greeting. The test explicitly asserts the Tokyo
greeting is absent — `toHaveCount(0)` against
`/Nice to meet you, Alice/i` — so any future broad-match
regression fails here loudly.
2. Clicking a slot transitions to `[data-testid="time-picker-picked"]`
and the assistant follow-up message contains "Booked ... Alice",
verifying the `hasToolResult: true` branch of the fixture pair
also wires through.
The langgraph-python gen-ui-agent demo was the only one of 18
integrations using the V1 `useCoAgentStateRender` hook. That hook
binds renders to messages via per-message claims, so each
state-changing tool call (each `set_steps` invocation) produced its
own card snapshot in the chat — a typical 3-step plan run pushed
~7+ stacked cards instead of one updating card.
Migrate the page to the canonical V2 pattern already used by every
other gen-ui-agent demo (mastra, strands, ag2, agno, crewai-crews,
langgraph-typescript, pydantic-ai, ...): subscribe to live state via
`useAgent` and render a single `InlineAgentStateCard` inside
`messageView.children`. The card now re-renders in place as state
streams — no per-message claims, no duplicates.
Also tighten the agent system prompt with an explicit numbered tool
sequence (1 plan + 6 transitions + final message) to make the
"step 3 stuck in_progress" tail-of-run failure less likely with
gpt-4o-mini. The UI is robust to a missed final transition either
way: when `agent.isRunning` flips to false, the card headlines
"All N steps complete" regardless of step.status.
Replace the stale e2e spec (which targeted a long-removed
`task-progress` test id) with one that pins the contract:
- exactly one `agent-state-card` rendered, even after the run
finishes
- every `agent-step` ends in `data-status="completed"`
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
Tyler Slaton