Option A removed the two-stage server-side A2UI planner
(generate_a2ui_via_llm, _get_a2ui_llm, _resolve_a2ui_model, _A2uiError,
_A2uiErrorKind, _RENDER_A2UI_FUNCTION_SPEC, _RENDER_A2UI_TOOL_SPEC,
_a2ui_error). The CopilotKit JS runtime A2UIMiddleware now intercepts
generate_a2ui before it reaches Python and drives the render_a2ui LLM
pass itself.
test_generate_a2ui.py was importing the removed symbols at collection
time, causing an ImportError that failed CI's "Run showcase package
Python unit tests" step for Python 3.12.
Fix: remove all tests for the deleted two-stage planner infra; rewrite
GenerateA2UITool.handle() tests to the Option A contract (handle fires
only on middleware regression, returns structured error JSON, logs
ERROR); retain all tests that remain valid: _ToolErrorKind enum identity,
backend tool handle() happy+error paths, create_agent wiring, module
hygiene (no openai at load time, clean subprocess import).
The langroid `declarative-gen-ui` D6 cell was red at turn 1 with
`reason=surface-missing`: the demo was still on the pre-D6 (D5-era) shape
while the google-adk / strands siblings had moved to the two-stage
dynamic-schema A2UI pattern. Four independent defects each blocked the
surface from painting; all four are fixed here to bring langroid to
sibling parity.
Root causes (each verified against a live isolated stack):
1. Stale suggestions + fixtures. `suggestions.ts` still offered the old
D5 pills ("Show a KPI dashboard", "pie chart of sales by region", …)
and `aimock/d6/langroid/gen-ui-declarative.json` only mocked those old
prompts. The D6 driver sends the four current business-question pills
("Show me my sales dashboard for this quarter.", etc.). Re-authored both
to the four current prompts, mirroring the google-adk north-star
(outer `generate_a2ui` no-arg → inner forced `render_a2ui` → outer
narration, three fixtures per pill).
2. Required `context` on the outer tool. `GenerateA2UITool.context` was a
required pydantic field, so the mocked outer call's `arguments: {}`
raised `ValidationError` before the tool could run — no inner call, no
surface. Made it optional (default "") to match the no-arg sibling tools.
3. Legacy functions API hid the inner tool from aimock's matcher. The
inner planner call used langroid's `functions=`/`function_call=` (legacy
OpenAI) path; aimock's `toolName` matcher only inspects the modern
`tools[]` array, so the inner `render_a2ui` fixture never matched and the
call fell through to the outer `generate_a2ui` fixture (empty surface,
wrong catalogId). Switched the inner call to the modern
`tools=`/`tool_choice=` API via a new `_RENDER_A2UI_TOOL_SPEC`. The
response extractor already reads the modern `oai_tool_calls` path first.
4. Inner call could not be discriminated per pill. langroid has no
framework middleware to forward the run's conversation into the inner
call (unlike ag_ui_adk / ag_ui_strands), so its inner user message was a
fixed generic string across all four pills. Added an explicit
last-user-turn thread (ContextVar set by the adapter, consumed by the
planner) so the pill prompt rides as the inner `userMessage` — the same
discriminator the sibling fixtures rely on.
Renderer/catalog parity: added the missing `declarative-info-row` testid
to InfoRow (turn 4) and a full `DataTable` definition + renderer
(`declarative-data-table`, turn 2), plus `trendValue` on Metric. Added
`sales-context.ts` (byte-identical dataset + composition rules to the
strands/google-adk siblings) and wired it via `chat.tsx`.
Red-green (isolated control-plane, slot 16):
- RED (3 runs, pre-fix): `d6:langroid/gen-ui-declarative` red,
turn-1 surface-missing.
- GREEN (2 runs, post-fix, --rebuild): 4/4 turns pass. aimock journal
confirms all four pills' outer `generate_a2ui` + inner `render_a2ui`
calls return 200 and emit the correct `declarative-gen-ui-catalog`
surfaces. Backend SSE verified per turn: turn-1 4 metrics + pie + bar,
turn-2 DataTable, turn-4 7 InfoRows.
reasoning-default-render.spec.ts and shared-state-write.spec.ts navigate to
/demos/reasoning-default-render and /demos/shared-state-write respectively,
but neither demo directory exists in any integration (including the
langgraph-python gold reference) and neither is declared in any manifest.
They are stale, non-canonical leftovers from the #5127 page-mirror.
Removed from all baselines where present: reasoning-default-render from 8
(langgraph-fastapi never had it) and shared-state-write from all 9. The
parity validator stays green (0 fail) and langgraph-fastapi's prior
spec-under-coverage warning clears once its phantom spec is gone and the 5
canonical specs are added.
The 9 baseline integrations (ag2, agno, crewai-crews, langgraph-fastapi,
langroid, llamaindex, mastra, spring-ai, strands) were page-mirrored from
langgraph-python but the mirror omitted 5 LGP-canonical Playwright specs
whose demos are present on disk:
- declarative-hashbrown
- declarative-json-render
- reasoning-custom
- reasoning-default
- threadid-frontend-tool-roundtrip
Copied each spec verbatim (byte-identical) from langgraph-python, which the
baselines mirror. All 5 backing demo directories exist in every baseline.
The specs are framework-agnostic (navigate by route + testid), so no
per-integration edits are needed. Restores apples-to-apples spec parity.
Each non-LGP integration carried its own drifted/stale copy of the e2e specs, causing
inconsistent behavior and noisy diffs across the fleet. Copied langgraph-python's canonical
specs verbatim across ~15 integrations (576 spec files total, SHA-1-verified identical to
LGP) so every integration runs the same assertions.
Also removed 2 orphan specs whose underlying demo pages do not exist:
- showcase/integrations/agno/tests/e2e/hitl-in-chat-booking.spec.ts
- showcase/integrations/built-in-agent/tests/e2e/shared-state-write.spec.ts
Integration-specific variant specs were intentionally left as-is: reasoning-default-render,
byoc-*, agentic-chat-reasoning, and shared-state-write where the demo exists. google-adk and
langgraph-typescript were already in parity from earlier commits and show no new changes.
## Summary
The 6 beautiful-chat demos (spring-ai, strands, langroid, agno,
claude-sdk-typescript, claude-sdk-python) ship three identical
suggestion chips. Against the deployed aimock-backed showcase, all three
were broken:
| Suggestion | Symptom | Cause |
|---|---|---|
| Plan a 3-day Tokyo trip | Returned a generic "Hi there! I'm your
showcase assistant…" greeting | Substring `"hi"` matches inside
**arc*hi*tecture**, hijacked by the broad `userMessage: "hi"` fixture |
| Explain RAG like I'm 12 | aimock 4xx — `"No fixture matched"` | No
fixture |
| Draft a launch email | aimock 4xx — `"No fixture matched"` | No
fixture |
Verified locally against `showcase up spring-ai` in a headed browser —
all three now return on-topic content.
## Fix
Add three full-sentence fixtures before the broad `"hi"` matcher in
`feature-parity.json`. Aimock's matcher is substring + first-match-wins
by file order, so the long sentence matchers win first and the `"hi"`
fixture is never reached for these prompts. Each returns a plausible
markdown response (3-day Tokyo itinerary, open-book-test analogy,
3-paragraph launch email).
## Regression coverage
Replaced the 1-line beautiful-chat placeholders with a 4-test suite for
all 6 integrations:
- Page loads with heading + chat input
- Each suggestion's reply contains the expected keywords (`Day 1|Day
2|Day 3`, `open-book|retrieval|RAG`, `Subject:|co-pilot|launch`)
- Each test ALSO asserts `toHaveCount(0)` against `/I'm your showcase
assistant/i` — if the broad "hi" fixture re-broadens or the new fixtures
are reordered/removed, the tests fail with a useful message.
## Test plan
- [x] `validate-fixture-tool-surface` clean: 141 fixtures × 628 demos,
no drift
- [x] Manual headed-browser verification on `showcase up spring-ai` —
all 3 suggestions return their on-topic responses
- [ ] CI's `Validate Showcase` job stays green
- [ ] On-demand E2E (`/test-aimock <slug>`) passes for any of the 6
integrations
The 6 beautiful-chat demos (spring-ai, strands, langroid, agno,
claude-sdk-typescript, claude-sdk-python) ship three identical
suggestion chips: "Plan a 3-day Tokyo trip", "Explain RAG like I'm
12", and "Draft a launch email". Against the deployed aimock-backed
showcase, all three were broken:
- Tokyo trip: hijacked by the broad `userMessage: "hi"` fixture,
because the substring "hi" appears inside "arc**hi**tecture" in
the prompt. Returned a generic "Hi there! I'm your showcase
assistant..." greeting with nothing about Tokyo.
- RAG explain: no fixture matched, aimock returned an error.
- Launch email: same — no fixture, error.
Add three on-topic fixtures with the full suggestion sentence as
`userMessage` (effectively-exact substring match). Place them
before the broad "hi" fixture in the file so first-match-wins
routes each suggestion to the right response.
Add a `beautiful-chat.spec.ts` regression suite to all 6
integrations: send each suggestion, assert the right keywords
appear in the assistant reply ("Day 1/2/3" for Tokyo,
"open-book/RAG" for RAG, "Subject:/co-pilot" for email), AND
assert the hijacked greeting is absent. If the broad "hi" fixture
re-broadens or the new fixtures are reordered/removed, these
tests fail loudly.
Bug: in a single chat session, running both HITL booking flows
back-to-back (Alice 1:1 → then Sales call without refresh) used to
skip the time-picker on the second flow and jump straight to
"Booked ..." text.
Cause: confirmation fixtures were matched on `hasToolResult: true`,
which fires whenever the conversation has ANY tool message in
history. After the first flow finished, the second user message
short-circuited to a confirmation match before the second flow's
toolCall fixture (gated on `hasToolResult: false`) had a chance to
fire. The picker never rendered.
Fix: re-key the two confirmation fixtures on `toolCallId` (the
specific tool_call_id of the matching `book_call` invocation), which
only fires when the LAST conversation message is a tool result with
that id — exactly the moment we want the confirmation. Drop the
`hasToolResult: false` constraint on the toolCall fixtures so they
match a fresh user request regardless of prior tool history.
Add a back-to-back regression test to all 17 hitl-in-chat specs:
walk Alice flow to completion, then sales flow without refresh,
assert two `time-picker-card` elements rendered. If the multi-flow
regression returns, the second card never appears and the test
fails at `toHaveCount(2)`.
The hitl-in-chat demo ships in 17 integrations (langgraph-python plus
16 others — mastra, strands, ag2, agno, crewai-crews,
langgraph-typescript, langgraph-fastapi, pydantic-ai, llamaindex,
langroid, claude-sdk-python, claude-sdk-typescript, ms-agent-python,
ms-agent-dotnet, spring-ai, google-adk). All shipped placeholder e2e
specs that only checked the chat input was visible — none exercised
the actual booking flow.
Replace each with the full booking-flow spec written for
langgraph-python:
1. The "Schedule a 1:1 with Alice" suggestion renders the time-picker
card AND the Tokyo greeting is absent (regression guard against
the broad aimock `userMessage: "Alice"` matcher).
2. Picking a slot transitions to the picked-state card and produces
a "Booked … Alice" assistant follow-up.
3. The "Book a call with sales" suggestion runs the same flow with
the sales attendee.
Also add the matching aimock fixture pair for the sales suggestion
in feature-parity.json — without it, case 3 would only pass against
real OpenAI, not the aimock-backed CI deployments. The pair mirrors
the Alice fixture pair: `book_call` toolCall on first turn,
confirmation message after the picker resolves.
Per-integration coverage matters because each integration has its
own framework-specific HITL wiring (`useHumanInTheLoop` binding to
the agent, agent-side tool registration, run streaming protocol)
that can regress independently of the shared aimock fixture.
Two bugs introduced in 534cd1efa (D5 integration fixes) when the
langroid adapter switched backend tool results from TEXT_MESSAGE_*
triples to TOOL_CALL_RESULT events:
1. Two leftover `logger.warning("DEBUG ...")` calls in agui_adapter.py
that fired on every chat turn — exactly the noise pattern the
`test_plain_text_turn_does_not_warn` test was written to prevent.
2. Three stale tests in test_agui_adapter.py (only run on 3.11+, so
the failure was 3.12-only):
- test_backend_tool_execution_happy_path: still expected the old
TEXT_MESSAGE_START/CONTENT/END inner triple instead of a single
TOOL_CALL_RESULT event after TOOL_CALL_END.
- test_backend_tool_exception_returns_sanitized_error: still read
the sanitized error JSON from TEXT_MESSAGE_CONTENT.delta — it
now rides on TOOL_CALL_RESULT.content.
- test_plain_text_turn_does_not_warn: caught the DEBUG leak above.
Tests are skipped on Python 3.10 (langroid imports `typing.Self`),
which is why this only shows up on the 3.12 matrix entry.
Replace sys.path.insert hacks in Python agent files with direct
imports via symlinks to shared/{python,typescript}/tools.
Update Dockerfiles, entrypoints, and configs to support the new
symlink-based tool resolution. Add PARITY_NOTES for frameworks
that have known gaps.
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
Tyler Slaton