Two LGT-only test failures fixed:
1. reasoning-default: The demo page sends agent="reasoning-default" but
the LGT route.ts only registered "reasoning-default-render". Added the
missing "reasoning-default" -> "agentic-chat-reasoning" mapping (same
graph used by reasoning-custom and reasoning-default-render).
2. hitl-in-chat back-to-back: After the first HITL flow completes on
LGT, sending a second message immediately triggers a RUN_ERROR race
condition in the CopilotKit runtime ("Cannot send event type: The run
has already errored"). Root cause is the LangGraph TypeScript server
takes slightly longer to finalize thread state after the interrupt ->
resume -> confirmation cycle. Fix adds page.waitForLoadState
("networkidle") between flows so all in-flight SSE streams are closed
before the next message is sent. Applied to both LGP and LGT test
copies for consistency.
The specs and QA markdowns had drifted from the demos they describe.
This commit brings every test contract into line with the actual demo
source — eliminating false-greens, false-fails, and stale assertions.
False-fail spec assertions (would fail every run):
- `agentic-chat.spec.ts` — rewrote from the old `change_background` /
`weather-card` / `useAgentContext` flow that no longer exists. New
spec exercises the vanilla `<CopilotChat>` + three suggestion pills
contract the simplified demo actually exposes.
- `gen-ui-tool-based.spec.ts` — asserted on UI text ("Use the sidebar
to generate charts", "Chart Generator") that doesn't exist; switched
to suggestion-pill assertions and scoped the SVG check to inside the
assistant-message bubble (was matching CopilotChat's send-button
SVG).
- `agent-config.spec.ts` — asserted heading "Agent Config Object" but
the demo has "Agent Config".
- `multimodal.spec.ts` — asserted a non-existent "Multimodal
attachments" heading; switched to the `multimodal-demo-root` testid.
- `chat-slots.spec.ts` — asserted `[data-testid="custom-assistant-
message"]` and the bare text "slot" — neither exists. The actual
signal is `data-slot-label="MessageView.AssistantMessage"` from the
SlotMarker wrapper.
- `reasoning-default.spec.ts` — asserted `[data-testid="copilot-
reasoning-message"]` and `[data-message-role="reasoning"]`; neither
is emitted by `CopilotChatReasoningMessage`. Switched to the text-
based "Thinking…/Thought for…" header label.
False-green spec assertions (passed for the wrong reason):
- `shared-state-read.spec.ts` — was a complete false-green: asserted
on "Sales Pipeline", "Total Pipeline", "Active Deals" but the demo
has been a Recipe Editor for some time. Rewrote against the
recipe-card / ingredients-container / instructions-container testids.
- 11 specs (agent-config, beautiful-chat, frontend-tools-async,
gen-ui-tool-based, gen-ui-agent, gen-ui-interrupt, hitl-in-chat,
hitl-in-app, multimodal, readonly-state-agent-context, voice) used
`[data-role="assistant"]` to gate "agent responded" — but the v2
react-core bundle never emits that attribute (it ships
`data-testid="copilot-assistant-message"`). Mechanical sweep to the
correct testid.
- Deleted `shared-state-write.spec.ts` (route consolidated into
`shared-state-read-write` earlier on this branch — spec targeted a
removed demo) and `renderer-selector.spec.ts` (asserted on a radio-
pill UI that no longer exists; the four "Declarative UI" variants
are now separate manifest demos).
QA drift:
- `qa/gen-ui-tool-based.md` documented a "Haiku Generator" demo with
haiku-card / japanese-line / english-line / haiku-image testids — a
demo that doesn't exist anywhere on this branch. Rewrote to match
the chart-rendering demo's actual testids and pill prompts.
- `qa/chat-slots.md` referenced "Custom Slot" pill / "Welcome to the
Slots demo" heading / "This welcome card is rendered via the
welcomeScreen slot." body text — all of which the slot-wrappers
refactor on this branch removed. Updated to match the
`custom-welcome-message` sub-slot that's actually rendered. Also
fixed max-w-4xl → max-w-5xl to match the page.
- `qa/shared-state-read.md` said default instruction is "Preheat oven
to 350 F" but the source has "Preheat oven to 350°F (175°C)".
- `qa/agentic-chat.md` rewrote to match the simplified vanilla-chat
demo (the previous QA documented `change_background` / `WeatherCard`
flows that no longer exist).
- `qa/reasoning-default.md` cited `kind: "testing"` in feature-
registry.json for the `reasoning-default` entry; the registry entry
has no `kind` field. Rewrote without the false cross-file claim.
- Deleted 4 orphan QA files for demos that don't exist:
`agentic-chat-reasoning.md`, `hitl.md`, `hitl-in-chat-booking.md`,
`shared-state-write.md`.
- Renamed `qa/reasoning-default-render.md` → `qa/reasoning-default.md`
to match the manifest cell name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug: in a single chat session, running both HITL booking flows
back-to-back (Alice 1:1 → then Sales call without refresh) used to
skip the time-picker on the second flow and jump straight to
"Booked ..." text.
Cause: confirmation fixtures were matched on `hasToolResult: true`,
which fires whenever the conversation has ANY tool message in
history. After the first flow finished, the second user message
short-circuited to a confirmation match before the second flow's
toolCall fixture (gated on `hasToolResult: false`) had a chance to
fire. The picker never rendered.
Fix: re-key the two confirmation fixtures on `toolCallId` (the
specific tool_call_id of the matching `book_call` invocation), which
only fires when the LAST conversation message is a tool result with
that id — exactly the moment we want the confirmation. Drop the
`hasToolResult: false` constraint on the toolCall fixtures so they
match a fresh user request regardless of prior tool history.
Add a back-to-back regression test to all 17 hitl-in-chat specs:
walk Alice flow to completion, then sales flow without refresh,
assert two `time-picker-card` elements rendered. If the multi-flow
regression returns, the second card never appears and the test
fails at `toHaveCount(2)`.
The hitl-in-chat demo ships in 17 integrations (langgraph-python plus
16 others — mastra, strands, ag2, agno, crewai-crews,
langgraph-typescript, langgraph-fastapi, pydantic-ai, llamaindex,
langroid, claude-sdk-python, claude-sdk-typescript, ms-agent-python,
ms-agent-dotnet, spring-ai, google-adk). All shipped placeholder e2e
specs that only checked the chat input was visible — none exercised
the actual booking flow.
Replace each with the full booking-flow spec written for
langgraph-python:
1. The "Schedule a 1:1 with Alice" suggestion renders the time-picker
card AND the Tokyo greeting is absent (regression guard against
the broad aimock `userMessage: "Alice"` matcher).
2. Picking a slot transitions to the picked-state card and produces
a "Booked … Alice" assistant follow-up.
3. The "Book a call with sales" suggestion runs the same flow with
the sales attendee.
Also add the matching aimock fixture pair for the sales suggestion
in feature-parity.json — without it, case 3 would only pass against
real OpenAI, not the aimock-backed CI deployments. The pair mirrors
the Alice fixture pair: `book_call` toolCall on first turn,
confirmation message after the picker resolves.
Per-integration coverage matters because each integration has its
own framework-specific HITL wiring (`useHumanInTheLoop` binding to
the agent, agent-side tool registration, run streaming protocol)
that can regress independently of the shared aimock fixture.
Pins the contract that the new full-sentence aimock fixture pair beats
the broad `userMessage: "Alice"` matcher:
1. Sending the suggestion `"Schedule a 1:1 with Alice next week to
review Q2 goals."` renders `[data-testid="time-picker-card"]`,
not the Tokyo greeting. The test explicitly asserts the Tokyo
greeting is absent — `toHaveCount(0)` against
`/Nice to meet you, Alice/i` — so any future broad-match
regression fails here loudly.
2. Clicking a slot transitions to `[data-testid="time-picker-picked"]`
and the assistant follow-up message contains "Booked ... Alice",
verifying the `hasToolResult: true` branch of the fixture pair
also wires through.
Jordan Ritter