pydantic-ai had never received the fleet D6 parity sweep — its e2e specs and demo pages
were a pre-sweep, integration-specific set (only 3/26 suggestion files; missing canonical
demos; non-canonical byoc-*/agentic-chat-reasoning/reasoning-default-render variants).
Sitting at 69/109/2.
This change mirrors langgraph-python's canonical frontend (demos + specs + aimock
fixtures) into pydantic-ai, preserving pydantic-ai's Python backend untouched. The
per-demo agent.py files that pydantic-ai carries inside demo directories are preserved.
Changes:
- tests/e2e/: rsync LGP canonical 37-spec set over pydantic-ai (byte-identical). Removes
non-canonical byoc-hashbrown.spec.ts, byoc-json-render.spec.ts, shared-state-write.spec.ts.
Adds canonical declarative-hashbrown.spec.ts, declarative-json-render.spec.ts,
reasoning-custom.spec.ts, reasoning-default.spec.ts.
- src/app/demos/: rsync LGP demos over pydantic-ai. Removes non-canonical demos
(byoc-hashbrown, byoc-json-render, agentic-chat-reasoning, reasoning-default-render,
shared-state-write). Adds canonical demos (declarative-hashbrown, declarative-json-render,
reasoning-default, reasoning-custom) and the _shared/ helpers + demos/layout.tsx
pydantic-ai was missing. Restores pydantic-ai-specific agent.py files into the 9 demo
dirs that survived the mirror.
- src/app/demos/frontend-tools/page.tsx: patched agent slug from "frontend_tools" (LGP)
to "frontend-tools" (matches pydantic-ai's main route.ts registry).
- src/app/api/copilotkit-byoc-{hashbrown,json-render}/ renamed to copilotkit-declarative-*
to match the canonical frontend wiring. Internals still use HttpAgent against the
pydantic backend's /byoc_hashbrown/ + /byoc_json_render/ mounts (Python backend
untouched per scope). copilotkit-declarative-hashbrown/route.ts updates the registered
agent slug from "byoc-hashbrown-demo" to "declarative-hashbrown-demo" to match the
canonical demo. copilotkit-declarative-json-render/route.ts updates only the endpoint
path string (the agent slug "byoc_json_render" is the canonical LGP convention).
- src/app/api/copilotkit/route.ts: renamed reasoning agent registrations from
agentic-chat-reasoning + reasoning-default-render to reasoning-custom + reasoning-default
to match canonical demo slugs. Both still proxy to the same /reasoning/ backend mount.
- manifest.yaml: features[] + demos[] updated to reflect the canonical demo set
(byoc-* + agentic-chat-reasoning + reasoning-default-render removed; declarative-* +
reasoning-default + reasoning-custom added).
- aimock/d6/pydantic-ai/: added gen-ui-custom.json (mirrored from LGP with
context-swap + copiedFrom marker, per established fixture convention). Removed
orphan gen-ui-open-advanced.json (no LGP counterpart in the canonical set).
The Python backend (agent.py / src/agent_server.py / src/agents/) is unchanged.
Some pydantic-ai backend mounts continue to exist that the mirrored frontend no longer
references (e.g. /reasoning/ remains, the deleted demos' agent slugs are still
registered in route.ts but harmlessly orphaned) — these are intentional carry-overs
to avoid touching Python backend code per scope.
Each non-LGP integration carried its own drifted/stale copy of the e2e specs, causing
inconsistent behavior and noisy diffs across the fleet. Copied langgraph-python's canonical
specs verbatim across ~15 integrations (576 spec files total, SHA-1-verified identical to
LGP) so every integration runs the same assertions.
Also removed 2 orphan specs whose underlying demo pages do not exist:
- showcase/integrations/agno/tests/e2e/hitl-in-chat-booking.spec.ts
- showcase/integrations/built-in-agent/tests/e2e/shared-state-write.spec.ts
Integration-specific variant specs were intentionally left as-is: reasoning-default-render,
byoc-*, agentic-chat-reasoning, and shared-state-write where the demo exists. google-adk and
langgraph-typescript were already in parity from earlier commits and show no new changes.
Bug: in a single chat session, running both HITL booking flows
back-to-back (Alice 1:1 → then Sales call without refresh) used to
skip the time-picker on the second flow and jump straight to
"Booked ..." text.
Cause: confirmation fixtures were matched on `hasToolResult: true`,
which fires whenever the conversation has ANY tool message in
history. After the first flow finished, the second user message
short-circuited to a confirmation match before the second flow's
toolCall fixture (gated on `hasToolResult: false`) had a chance to
fire. The picker never rendered.
Fix: re-key the two confirmation fixtures on `toolCallId` (the
specific tool_call_id of the matching `book_call` invocation), which
only fires when the LAST conversation message is a tool result with
that id — exactly the moment we want the confirmation. Drop the
`hasToolResult: false` constraint on the toolCall fixtures so they
match a fresh user request regardless of prior tool history.
Add a back-to-back regression test to all 17 hitl-in-chat specs:
walk Alice flow to completion, then sales flow without refresh,
assert two `time-picker-card` elements rendered. If the multi-flow
regression returns, the second card never appears and the test
fails at `toHaveCount(2)`.
The hitl-in-chat demo ships in 17 integrations (langgraph-python plus
16 others — mastra, strands, ag2, agno, crewai-crews,
langgraph-typescript, langgraph-fastapi, pydantic-ai, llamaindex,
langroid, claude-sdk-python, claude-sdk-typescript, ms-agent-python,
ms-agent-dotnet, spring-ai, google-adk). All shipped placeholder e2e
specs that only checked the chat input was visible — none exercised
the actual booking flow.
Replace each with the full booking-flow spec written for
langgraph-python:
1. The "Schedule a 1:1 with Alice" suggestion renders the time-picker
card AND the Tokyo greeting is absent (regression guard against
the broad aimock `userMessage: "Alice"` matcher).
2. Picking a slot transitions to the picked-state card and produces
a "Booked … Alice" assistant follow-up.
3. The "Book a call with sales" suggestion runs the same flow with
the sales attendee.
Also add the matching aimock fixture pair for the sales suggestion
in feature-parity.json — without it, case 3 would only pass against
real OpenAI, not the aimock-backed CI deployments. The pair mirrors
the Alice fixture pair: `book_call` toolCall on first turn,
confirmation message after the picker resolves.
Per-integration coverage matters because each integration has its
own framework-specific HITL wiring (`useHumanInTheLoop` binding to
the agent, agent-side tool registration, run streaming protocol)
that can regress independently of the shared aimock fixture.
agentic-chat: strip extra frontend tools (change_background, get_weather) and
agent context that interfere with D5 3-turn conversation probe; use unified V2
imports from @copilotkit/react-core/v2
hitl-in-app: fix mixed V2/non-V2 CopilotKit import; portal approval dialog to
document.body via createPortal so D5 probe can reliably click Approve button
regardless of parent CSS stacking context
Update e2e tests to match simplified agentic-chat page (remove weather card and
background change assertions for features that no longer exist)
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
Jordan Ritter