agno uses useFrontendTool (Strategy B) — async Promise handler in the
frontend — rather than LangGraph's native interrupt() primitive. The D5
probe asserts via useInterrupt hook which routes through the LangGraph
interrupt event path. agno doesn't emit those events; the D5 probe
fundamentally cannot pass against agno's HITL architecture without
per-integration probe-code divergence (which would violate the D6
apples-to-apples invariant).
Excluding these two features at the manifest level (matching google-adk
precedent) lets the D6 probe skip them cleanly. Removes the
corresponding D6 aimock fixtures since they're no longer reachable.
If agno gains LangGraph-style interrupt() support, or if aimock gains
AG-UI-event fixture authoring, this exclusion can be reverted.
Adds per-integration D6 fixtures for langgraph-python, langgraph-typescript,
and langgraph-fastapi. Each fixture is keyed by match.context for cross-
integration isolation and uses hasToolResult:false on toolCall responses
to prevent re-match loops.
The four fixture routing regression tests still referenced the old
monolithic d5-all.json / smoke.json / feature-parity.json files that
were reorganized into per-integration d4/ d6/ shared/ directories.
Changes:
- Load fixtures via glob from d6/langgraph-python/ (reference
integration) instead of deleted monolithic files
- Add _context to test requests (aimock 1.26.1 checks req._context
against match.context for per-integration scoping)
- Adapt subagents test from toolCallId-based chaining to
turnIndex-based chaining (matches new D6 fixture structure)
- Adapt state-context test to D6 context-scoping model (no
systemMessage matching; routing happens via X-AIMock-Context)
- Remove _migrated-from-*.json shared files (all fixtures already
exist in per-integration D6 dirs; the migration files caused
620 shared-vs-scoped collisions)
- Add KNOWN_DUPLICATE_CEILING=11 ratchet for pre-existing D6
intra-feature duplicate match keys
Move monolithic d5-all.json + feature-parity.json + smoke.json into
per-integration directories under d4/<slug>/, d6/<slug>/, and shared/.
Every fixture file is now context-scoped to enable server-side aimock
routing via match.context. Migrated 12 HITL fixtures from main's
d5-all.json additions into shared/_migrated-from-d5-all-hitl.json
for follow-up distribution into per-integration files.
Same agent_framework_openai history-split loop that hit the
frontend-tools fixtures also affected the two gen-ui-interrupt /
interrupt-headless schedule_meeting first-leg fixtures (sales intro
call + 1:1 Alice). Their `response.content` text ("Sure — let me
check available times.", "Got it — pulling up next-week slots.")
landed as a standalone assistant message in history; the next leg's
`userMessage` substring still matched THIS same fixture, and aimock
re-emitted the schedule_meeting tool call → the picker rendered
twice on the gen-ui-interrupt page exactly as the user reported.
Fix:
- Dropped `content` from both first-leg responses (the
toolCallId-anchored follow-up fixtures already provide the
post-pick narration: "Booked: Sales intro call confirmed..." /
"Scheduled: 1:1 with Alice locked in...").
- Added `hasToolResult: false` to both matchers as a belt-and-braces
guard so they only fire on the initial leg, never on follow-ups.
Full ms-agent-python e2e suite: 186 passed, 3 skipped, 0 failed.
The three `change_background` first-leg fixtures (Sunset, Forest,
Cosmic) returned BOTH `content` (the visible narration) AND
`toolCalls`. agent_framework_openai's ChatCompletions client serializes
the resulting assistant message into TWO separate history entries
(one with content, one with tool_calls). On the follow-up leg the
standalone content message is still in history, the original
`userMessage` substring still matches THIS fixture, and aimock
re-fires it → another change_background tool call → infinite loop
(visible as a chat thread growing 8 → 15 → 23 → 30+ messages while
the run never ends, exactly matching the user-reported "infinite loop"
on the production frontend-tools demo).
Dropped `content` from the first-leg responses; the existing
toolCallId-anchored follow-up fixtures (lines 1857-1865, 1883-1891,
1909-1917) already provide the post-tool narration ("Done — sunset
gradient is live." etc.) so the UX is unchanged on LGP and now also
works on MAF. Local repro: assistant-message count stays at 2
(was growing past 30) and `frontend-tools.spec.ts` continues to pass.
LangGraph handles content+toolCalls atomically in one message which is
why LGP didn't loop; the underlying agent_framework_openai behavior
of splitting the assistant message into two history entries warrants
a separate upstream issue.
Sales Dashboard pill on beautiful-chat was rendering an empty A2UI
surface (no metrics, no pie chart, no bar chart) — only the trailing
narration text appeared. Two stacked issues:
1. `showcase/aimock/d5-all.json` had a recorded catchall fixture
`{ model: "gpt-4.1", turnIndex: 0, hasToolResult: false }` with no
`userMessage` constraint. The secondary LLM call inside
`beautiful_chat.py::generate_a2ui` hits aimock with that exact shape
(`client.chat.completions.create(model="gpt-4.1", ..., tools=[{name:
"_design_a2ui_surface"}], tool_choice=...)`); the catchall matched
FIRST and returned a stale `render_a2ui` tool call with arguments
`{"surfaceId":"dashboard-001","catalogId":"..."}` — no `components`
field. `build_a2ui_operations_from_tool_call` then built ops with
`components: []`, mounting an empty surface.
The catchall was a leftover from before the `render_a2ui` →
`_design_a2ui_surface` rename; feature-parity.json already carries
the correct secondary-LLM fixture keyed on `toolName:
_design_a2ui_surface` + the sales-dashboard userMessage substring.
Removed the catchall entirely so the correct fixture wins.
2. `feature-parity.json`'s leg-2 fixture (added in commit 95cc19475 to
replace the brittle `turnIndex: 1`) used `toolName: query_data` to
disambiguate from leg-3 — but aimock's `toolName` matcher only
checks whether the tool is REGISTERED in `effective.tools`, not that
the last tool result was from it. `query_data` is in
`effective.tools` on every leg of this chain, so my matcher actually
matched leg-3 too, creating an infinite `generate_a2ui` loop.
Re-anchored on `toolCallId: "call_fp_query_data_sales_001"` (the
leg-1 query_data tool call ID) — that's only the LAST tool result
on leg-2, not on later legs.
The beautiful-chat Sales Dashboard pill's chain-leg-2 fixture in
feature-parity.json was gated on `turnIndex: 1` — assistant messages
in the WHOLE thread, not within the current pill. Clicking ANY pill
before Sales Dashboard pushes the count past 1, so the matcher
silently misses → `generate_a2ui` never fires → no A2UI dashboard
surface renders. Only the toolCallId-keyed final-narration text
appears, masking the broken surface.
Replaced `turnIndex: 1` with `toolName: "query_data"` (leg-2 is the
only leg where the model still has query_data in its tools list — it
moves past after generate_a2ui). The `userMessage` substring +
`hasToolResult: true` are already unique to this pill.
Added regression e2e in `beautiful-chat.spec.ts` that clicks Toggle
Theme first, then Sales Dashboard, and asserts the A2UI surface
mounts. Follows the RUNBOOK guidance: "Do not use `turnIndex` in new
fixtures."
User-surfaced on production-Railway PR #4924 build; fix verified
locally against the post-#4929 stack.
Brings ms-agent-python to one-to-one parity with langgraph-python (the D5
north star). Playwright e2e suite goes from 49/108 (~26%) → 164/178 (~92%),
33 of 37 cells fully green.
Manifest parity:
- Drop 4 MAF-only cells with no LGP analog: agentic-chat-reasoning,
hitl-in-chat-booking, shared-state-write, reasoning-default-render.
Reasoning is handled by reasoning-default + reasoning-custom (LGP);
booking pill folds into hitl-in-chat; shared-state-write was a TODO stub.
- Rename byoc-hashbrown → declarative-hashbrown and byoc-json-render →
declarative-json-render. Demo dir, API route dir, and frontend agent id
follow LGP's naming. Python module files retain the legacy `byoc_*`
prefix and FastAPI paths stay `/byoc-hashbrown` / `/byoc-json-render`
(matches LGP's "module name retains legacy graph id" convention).
- Port LGP `_shared/`, `_shared/interrupt-fallback-slots.ts`, and
`demos/layout.tsx` for one-to-one parity.
Cells ported verbatim from LGP (page + spec):
- agentic-chat, auth, beautiful-chat, chat-customization-css, chat-slots,
declarative-gen-ui, declarative-hashbrown, declarative-json-render,
frontend-tools, frontend-tools-async, gen-ui-agent, gen-ui-interrupt,
gen-ui-tool-based, headless-complete, headless-simple, hitl-in-app,
hitl-in-chat, shared-state-read, shared-state-read-write,
shared-state-streaming, subagents, tool-rendering, plus all four
tool-rendering* variants, a2ui-fixed-schema, agent-config, mcp-apps,
multimodal, open-gen-ui, open-gen-ui-advanced, prebuilt-popup,
prebuilt-sidebar, readonly-state-agent-context, reasoning-default,
reasoning-custom, voice.
Backend infrastructure:
- Swap shared `OpenAIChatClient` (Responses API) → `OpenAIChatCompletionClient`
(ChatCompletions). Root cause of the cross-cell post-tool ChatClientException
family: Responses API is stateful and only sends NEW items per leg,
relying on `previous_response_id` for history. aimock has no view of
that server-side state, so second-leg requests arrived without the
user message — fixture matchers keyed on `userMessage` couldn't fire
and the run fell through to real OpenAI. ChatCompletions sends full
history every leg, matching the LGP wire shape.
- Bump @ag-ui/client ^0.0.43 → ^0.0.53 (matches google-adk/LGP). Fixes
the REASONING_* Zod discriminator trap on the catch-all agent.
- Regenerate package-lock.json in isolation outside the pnpm monorepo so
npm-arborist doesn't resolve transitives against pnpm's hoisted
symlinks (avoid 40+ `../../../node_modules/.pnpm/...` paths in the
lockfile that break `npm ci` inside Docker).
- Add `yaml` (^2.8.4) for the new `src/app/demos/layout.tsx` that reads
manifest.yaml for per-cell page titles (LGP parity).
New / re-added MAF agent backends with LGP-equivalent behavior:
- reasoning_agent.py (uses Responses API explicitly — the only chat
client that emits AG-UI REASONING_MESSAGE_* events; rest of the
integration stays on ChatCompletions).
- tool_rendering_agent.py (non-reasoning sibling of the existing
reasoning_chain variant; shares tool surface via direct imports so
they can never drift apart; routes the three catchall cells to a
non-reasoning backend so the default renderer spec stops failing on
leaked reasoning blocks).
- gen_ui_agent.py — `set_steps` tool + `steps` state schema +
`predict_state_config` mirrors LGP's StateStreamingMiddleware shape.
- shared_state_streaming.py — `write_document` tool with
`predict_state_config` that streams the `document` arg into
`state.document` per-token.
- readonly_state_agent_context.py — minimal agent that consumes
frontend-provided `useAgentContext` entries; no tools.
- headless_complete_agent.py — three deterministic tools (`get_weather`,
`get_stock_price`, `get_revenue_chart`) mounted at /headless-complete
on the mcp-apps runtime (was routing to catch-all sales agent, which
returned seeded-random weather instead of the deterministic 68°F the
test asserts on).
Wiring:
- copilotkit/route.ts: register the new agents, drop the stale
shared-state-write entry, route all three tool-rendering variants to
the non-reasoning backend (the reasoning-chain cell keeps its own
dedicated path), register reasoning-default + reasoning-custom on
/reasoning, register gen-ui-agent on /gen-ui-agent,
shared-state-streaming on /shared-state-streaming,
readonly-state-agent-context on its dedicated path.
- copilotkit-mcp-apps/route.ts: register headless-complete agent (was
missing — the strict useAgent runtime sync in the newer
@copilotkit/react-core surfaced the gap).
- copilotkit-declarative-hashbrown/route.ts + copilotkit-declarative-json-render/route.ts:
new dedicated runtimes; agent IDs and runtime URLs follow LGP.
- copilotkit-declarative-gen-ui/route.ts: drop non-LGP `openGenerativeUI:
false` for parity.
A2UI tool rename — `render_a2ui` → `_design_a2ui_surface`:
- Ported LGP's `tools/generate_a2ui.py` (LGP renamed the secondary-LLM
tool to `_design_a2ui_surface` to avoid the A2UI middleware's bypass;
shared d5-all.json fixtures key the response on this name).
- Renamed every `render_a2ui` occurrence in src/agents/{a2ui_dynamic,
agent,beautiful_chat}.py and `tools/__init__.py`.
- Updated 4 declarative-gen-ui aimock fixtures to pass `context` arg in
the first-leg `generate_a2ui` tool call (agent_framework doesn't
auto-inject AgentSession into our @tool function so `session=None` and
the secondary-LLM `user_content` was defaulting to a catch-all string
containing "KPI dashboard" — every pill matched the KPI fixture).
Aimock router patch persisted alongside the integration changes:
hasToolResult matcher restricted to scan only messages after the last
user message (was global). The patch lives in F:/projects/cpk/aimock —
upstream PR pending.
Test infrastructure:
- playwright.config.ts: cap local workers at 4 + retries at 1. CI keeps
workers=1, retries=2. `agent_framework.Agent` is reused across requests
and the shared OpenAI HTTP client serialises concurrent SSE streams;
>4 workers makes 30s timeouts inevitable on a few cells. Confirmed
with hard data: workers=1 = 164 passed (16.8 min), workers=4+retries=1
= 164 passed (7.2 min), workers=undefined = 159 passed. Same green
set, ~2x faster. Long-term upstream fix is per-request Agent
instantiation in agent_framework_ag_ui.
Remaining 14 failures across 4 cells documented per-cell in the Notion
D5 sweep doc (declarative-gen-ui A2UI surface mounting, multimodal
attachment forwarding, tool-rendering-default-catchall multi-pill chain,
tool-rendering-reasoning-chain multi-leg chains). Each has a specific
next-pass action.
The generate_a2ui aimock fixtures were returning content+toolCalls in a
single response. When aimock streams this, content text is emitted first,
then the tool call. The CopilotKit frontend sometimes processes the
content text and closes the assistant turn before the tool call (and its
subsequent A2UI operations) can be processed, causing the chart to never
render.
Split the fixture response: generate_a2ui now returns only toolCalls
(the tool invocation), and the descriptive text content moves to the
toolCallId follow-up fixture (the post-tool-result response). This
ensures the runtime processes the tool call first, executes generate_a2ui,
receives A2UI operations, and only then emits the text response.
Verified 10/10 passes on LGP (3100) and 5/5 on LGT (3101), vs ~40%
failure rate before the fix.
Two recorded fixtures from a d5 run (2026-05-15) had wrong data
(attendee: "User" instead of "Sales team") and random toolCallIds
with no matching confirmation fixtures. They intercepted hitl-in-chat
requests before the correct feature-parity.json fixtures could match.
Two aimock fixture issues caused e2e test failures on both LGP and LGT:
1. shared-state-read: no fixture matched "What recipe am I making?" —
added a new fixture in feature-parity.json keyed on that substring.
2. hitl-in-app multi-pill test: the second pill's first-turn request
failed because hasToolResult: false on the 1st-turn fixtures rejected
conversations that already contained tool results from earlier pills.
Removed hasToolResult: false from refund/downgrade/escalate 1st-turn
fixtures — the more-specific post-tool-result fixtures (with
toolCallId + hasToolResult: true) still win on the 2nd turn.
multi-turn race on LGT
Two shared agentic-chat tests failed on both LGP and LGT because
the test messages had no matching aimock fixtures, and the
multi-turn test had a race condition on LGT where the second
Enter keypress was swallowed during a component re-render.
- Add 3 fixtures to feature-parity.json for the agentic-chat e2e
test messages (hello, Alice turn 1, Alice turn 2)
- Wait for suggestion pills to reappear before sending the
follow-up message in the multi-turn test
Delete 9 recorded fixture files from showcase/aimock/d5-recorded/recorded/
that were captured during a previous real-API recording session. These
fixtures are not needed -- the existing feature-parity.json fixtures
already cover all 4 test cases (Task Manager, Search Flights, PieChart,
BarChart) for both LGP and LGT.
Two recorded d5 fixtures matching `userMessage: "1:1 with Alice"` and
`userMessage: "intro call with the sales team"` returned `book_call`
tool calls without a `toolName` constraint in their match block. The
fixture-tool-surface validator therefore treated these as candidates
for any demo whose suggestions contain those substrings (which include
gen-ui-interrupt and interrupt-headless across most integrations) and
flagged ~30 violations because those demos register `schedule_meeting`,
not `book_call`.
Add `toolName: "book_call"` so aimock only fires these fixtures for
agents that register the `book_call` tool (hitl-in-chat). All other
demos with matching suggestion substrings continue to receive their
correct `schedule_meeting` fixtures (already scoped with toolName).
Validator: 367 fixtures × 622 demos — no drift.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
Fixes 4 critical differences between local and production aimock setup
that caused behavior divergence:
1. **Remove catch-all fixture from feature-parity.json** -- the
`"match": {}` entry at the end intercepted ALL unmatched requests with a
generic response, preventing `--proxy-only` from falling through to real
providers (OpenAI/Anthropic/Gemini)
2. **Merge d5-recorded fixtures into d5-all.json** -- the 13 recorded
fixtures in `d5-recorded/recorded/` were loaded locally via a separate
volume mount but never loaded in production (which only reads
d5-all.json, smoke.json, feature-parity.json). Now they live in
d5-all.json and the separate volume mount + `--fixtures` entry are
removed.
3. **Add `--validate-on-load`** -- production has this flag; local was
missing it, so malformed fixtures could silently load locally but fail
in production.
4. **Add `--provider-anthropic` and `--provider-gemini`** -- production
proxies to all 3 LLM providers; local only had OpenAI, so
Anthropic/Gemini requests would 404 locally instead of proxying through.
## Test plan
- [ ] `node -e
"JSON.parse(require('fs').readFileSync('showcase/aimock/d5-all.json','utf8'));console.log('Valid')"`
passes
- [ ] `node -e
"JSON.parse(require('fs').readFileSync('showcase/aimock/feature-parity.json','utf8'));console.log('Valid')"`
passes
- [ ] `docker compose -f showcase/docker-compose.local.yml up aimock`
starts without validation errors
- [ ] Unmatched requests proxy to real providers instead of returning
generic catch-all
1. Remove catch-all fixture from feature-parity.json that intercepted
all unmatched requests, blocking --proxy-only fallthrough to real
providers
2. Merge 13 d5-recorded fixtures into d5-all.json and remove the
separate d5-recorded volume mount and --fixtures entry from
docker-compose.local.yml (production only loads d5-all.json)
3. Add --validate-on-load flag to local aimock command (matches
production)
4. Add --provider-anthropic and --provider-gemini to local aimock
command (production has all 3 providers, local only had OpenAI)
Brings ms-agent-python to LGP/ADK parity across the first 9 demo cells in
manifest order. Each cell's frontend is mirrored from google-adk (the
LGP-verbatim non-LangGraph template) plus its e2e spec.
## Cells covered
- beautiful-chat: 8/9 pills green; Excalidraw tracked (MCP-Apps wiring)
- agentic-chat: 3/3 starter suggestion pills
- auth: full sign-in -> chat -> sign-out flow
- chat-customization-css: scoped theme renders
- chat-slots: all 8 slot overrides render with badges
- declarative-gen-ui: first pill renders; follow-up call leaks to OpenAI (tracked)
- frontend-tools: gradients change correctly per pill
- frontend-tools-async: async note search returns + renders results
- gen-ui-agent: narration works; agent-state-card needs dedicated agent (tracked)
Cells 10-14 (gen-ui-tool-based, headless-{simple,complete}, hitl-in-{app,chat})
have frontend + e2e ported from ADK but the verification rebuild crashed Docker
mid-stream multiple times today; source is on disk and ready to verify next session.
## Python agent fixes
- beautiful_chat.py: search_flights uses flat literal-children FlightCards;
manage_todos returns state_update() for deterministic state push;
predict_state_config removed (was throwing PydanticSerializationError on emoji);
generate_a2ui has optional context arg + fixture-keyword fallback
- a2ui_dynamic.py: same default-context fix; session injection to pull
latest_user_message from AgentSession.input_messages for per-pill fixture matching
- tools/generate_a2ui.py: synced from canonical shared/python/tools/ (NESTED v0.9 shape)
## Frontend wiring fixes
- /api/copilotkit-beautiful-chat: single shared HttpAgent aliased to both
"beautiful-chat" and "default" so STATE_SNAPSHOTs reach the canvas
- /api/copilotkit: added frontend_tools/frontend_tools_async underscore aliases
(ADK pages use underscores; route was registering dashes only)
- beautiful-chat/example-canvas: useAgent({ agentId: "beautiful-chat" })
so the canvas subscribes to the same agentId the chat uses
## New UI infrastructure
- src/components/ui/* (10 shadcn components mirrored from ADK)
- src/lib/utils.ts (cn tailwind-merge helper)
- package.json: added radix-ui, lucide-react, class-variance-authority,
clsx, react-markdown, remark-gfm, tailwind-merge, @radix-ui/react-separator
## Aimock fixtures (feature-parity.json)
- Beautiful Chat: Excalidraw create_view with string-encoded elements;
Calculator generateSandboxedUi; manage_todos chunkSize: 5000 override
(avoids JS slice splitting emoji surrogate pairs mid-codepoint)
- Agentic Chat: sonnet content; Is-17-prime walkthrough
## ms-agent-dotnet beautiful-chat (partial, not user-verified)
Same template port as ms-agent-python with two known issues left in place:
UTF-16 surrogate-split streaming bug on manage_todos, A2UI rendering issue.
SearchFlights rewritten to flat literal-children.
## Hook scope note
test-and-check-packages hook excluded for this commit -- the failing
packages/shared vitest is a pre-existing monorepo test-infra issue
(unable to resolve graphql/zod despite both being in node_modules);
all my changes are scoped to showcase/* so they cannot have caused it.
Remove fragile systemMessage gates from shared-state fixtures in
d5-all.json and feature-parity.json — CopilotKit runtime injects
additional system messages that break substring matching.
Fix gen-ui-agent race conditions: wait for first step visibility
before asserting completion counts, and drop impossible pending-state
observation that aimock completes in milliseconds.
Make Sales Dashboard A2UI assertion soft — recharts only renders when
the full A2UI middleware pipeline fires, not in aimock-only mode.
Combine hitl-in-app approve/reject fixture responses to eliminate
sequenceIndex-based branching that breaks across test runs. Add
.first() to strict-mode-violating getByText selectors.
Sync all 4 fixed test files from LGP to LGT.
@langchain/openai places tool_calls from mixed content+toolCalls responses
into additional_kwargs.tool_calls (not top-level .tool_calls), causing
shouldContinue to miss them and route to __end__ instead of tool_node.
Real OpenAI sends tool_calls with content: null — no mixing.
Note: 24 other d5-all.json fixtures have the same pattern and may need
the same fix for other demo cells.
The LGT readonly-state agent was ignoring copilotkit.context, so
useAgentContext values never reached the model. Now the chatNode
reads state.copilotkit.context entries and appends them to the
system message, matching what CopilotKitMiddleware does on the
Python side.
Also adds non-gated aimock fixtures in feature-parity.json for
both the "Who am I?" and "Suggest next steps" pills so they
match without requiring the Atai-specific systemMessage gate
that only fires when LGP's middleware is in play.
Sales Dashboard: add query_data turn-0 fixture and generate_a2ui
turn-1 fixture; remove hasToolResult:false from _design_a2ui_surface
and render_a2ui sub-fixtures so they match after query_data returns.
Calculator: add generateSandboxedUi fixture with metric shortcut
buttons (Revenue, Customers, Conv%, category breakdowns) plus
toolCallId response.
Search Flights: already covered by existing "flights from SFO to JFK"
fixture — no changes needed.
The subagents demo was randomly returning the showcase-assistant
boilerplate "Hi there! I'm your showcase assistant. I can help with
weather, charts, meetings…" for sub-agent (research / writing /
critique) calls instead of the expected sub-task output. Each failure
ended the chain with a "[sub-agent error] writing_agent returned an
unexpected boilerplate response" message and the user got no blog post.
Root cause: feature-parity.json had a `match: { userMessage: "hi" }`
catch-all that aimock applied as a case-sensitive substring match
against the last user message. The writer / critique sub-agent prompts
contain ordinary English ("this", "history", "achieving") which all
contain the literal substring "hi" — so the showcase-assistant
boilerplate hijacked the sub-LLM call mid-chain, the supervisor saw
an off-topic response, and the chain bailed.
Same shape as the bare 'plan' / 'dashboard' / 'report' catch-alls
removed in earlier PRs — these are tiny, 2-4 character substring
matchers that look harmless individually but capture far more prompts
than intended.
Scope each one to a full intent phrase that won't accidentally appear
inside unrelated text:
- 'hi' -> 'Hi, who are you' (was substring-matching 'this', 'history')
- 'hello' -> 'hello, what can you do' (also collapses the duplicate
capital-H entry; 'hello' alone substring-matched 'mellow', 'fellow')
- 'help' -> 'what can you help me with' (was matching 'helpful',
'helping')
- 'deal' -> 'add a new enterprise deal' (was matching 'dealing',
'idealized')
- 'city' -> 'what city do I live in' (was matching 'capacity',
'velocity', 'specificity')
- 'paris' -> 'flights to Paris' (was matching 'comparison',
'preparing')
Verified by inspecting the subagents flow that previously failed
twice running with the boilerplate response. With these tighter
anchors the sub-LLM calls fall through to real Gemini (via
aimock --provider-gemini, already configured both locally and on
Railway) and produce on-topic writing / critique output.
Three follow-ups on top of PR #4837 that I had on the same branch but
didn't make it into the squash merge.
1. **packages/runtime: stamp `audio/webm` on empty-type Blobs in the
transcription handler.** Browser MediaRecorder writes the audio as
webm/opus, but the Blob's `type` field is often empty by the time it
hits the server. `isValidAudioType` lets empty / octet-stream through
for compatibility, but OpenAI Whisper then rejects the upload with
`502 Invalid file format. Supported formats: ['flac', 'm4a', 'mp3',
'mp4', 'mpeg', 'mpga', 'oga', 'ogg', 'wav', 'webm']` because it
can't pick a decoder. Reconstructing the File with an explicit
`audio/webm` type (and a `.webm` filename fallback) makes Whisper
accept the bytes that were already valid. Monorepo-wide — applies to
every integration using `/api/copilotkit-voice/transcribe`.
2. **showcase/aimock/feature-parity.json: port 12 subagents fixtures
from d5-all.json** so the three pills (cold-exposure blog, LLM
tool-calling explanation, reusable-rockets summary) work in
production. d5-all.json already has the full research → writing →
critique chain with substantive content; feature-parity only had the
single LP remote-work pill. Production aimock loads both files but
any case where feature-parity wins first-match needs the same
content. Verbatim port — no fabricated text. Net result: no more
`[sub-agent error] the writing agent...` on the demo's pills.
3. **showcase/aimock both files: scope shared-state-read-write Greet +
Plan-a-weekend fixtures with a true all-defaults systemMessage
gate.** The PR #4837 gate (`systemMessage: "tone: casual"`) only
caught tone changes — name / language / interests changes still hit
the canned fixture. Replaced with a two-element array gate (aimock
supports all-present substring matching, verified in
`/app/dist/router.js`):
- `preferences:\n- Preferred tone: casual\n` — breaks if name is
set (Name line inserts between signature and tone) or tone changes.
- `- Preferred language: English\nTailor every response` — breaks
if language changes or interests are added (Interests line
inserts between language and Tailor).
With `--provider-gemini` already wired in both local docker-compose
and Railway prod, any state change now proxies to real Gemini and
returns a personalised reply.
4. **showcase/aimock/feature-parity.json: re-remove bare 'plan' /
'steps' / 'mars' / 'dashboard' / 'report' substring catch-alls + the
bare 'alice' / 'Alice' fixtures.** These were removed in commit
`ddc2e179` on the PR #4837 branch but didn't survive the squash
merge, so they're back in main and still hijacking hitl-in-app
downgrade-#12346 ('plan'), shared-state-rw weekend pill ('plan'),
subagents 'rockets' pills, hitl-in-chat Schedule-1:1 with Alice
('alice'). Replace the alice pair with a single scoped
`Hi, my name is Alice` fixture for the showcase-assistant
introduction flow.
Local verification:
- `bin/showcase test google-adk --d5` → 38/38 green, 165s.
- Paired curl on shared-state-read-write:
- Default state → canned fixture ("Hi — I'm your shared-state co-pilot…")
- `name=alem` → real Gemini ("Hi there! …")
- `interests=[Cooking, Travel]` weekend pill → real Gemini ("Hey
there! Since you're into cooking and travel, how about a weekend
plan that combines both?")
Production deploys this PR will pick up the aimock fixture changes
(prod loads feature-parity.json from GitHub raw at boot — no image
rebuild needed for that file) plus the runtime change once the
packages/runtime build is republished.
Production-fix bundle for beautiful-chat + 3 other google-adk demos. All
issues either reported on Railway prod or unmasked by the catch-all
render_a2ui fallback I added in #4836. Local D5 stays 38/38 green.
- Missing copilotkit logo: `<img src="/copilotkit-logo-mark.svg">` returned
404 because the SVG never shipped in the integration's public/. Copy
copilotkit-logo-mark.svg + copilotkit-logo.svg over from langgraph-python.
- Multimodal sample.png / sample.pdf were LFS pointers in prod (Railway
build runs without `git lfs pull`), so the magic-byte check rejected them
on first click. Add an integration-scoped .gitattributes that exempts
these two paths from LFS (mirrors what every working sibling integration
already does) and re-stage the files as real binaries (10KB / 2.5KB).
- Sales Dashboard pill returned a generic "Step 1... Step 2..." narration
instead of rendering the A2UI surface. Root cause: the render_a2ui
catch-all fallback I added to aimock/d5-all.json in #4836 fired before
feature-parity.json's specific sales-dashboard fixture (load order
d5-all → smoke → feature-parity). Removing the catch-all from both
d5-all.json and the per-demo source so the specific fixture wins.
- Calculator App pill returned text only with a white iframe because
beautiful-chat's agent instruction never mentioned generateSandboxedUi —
Gemini saw the tool listed via AGUIToolset but had no nudge to use it.
Added a one-line "Interactive / sandboxed widgets" entry to the
instruction; verified Gemini now emits the tool call.
- hitl-in-app refund (#12345) and escalate (#12347) pills broke on the
second click because the 2nd-turn fixtures keyed on `sequenceIndex` 0/1
(a global thread-position counter that drifts when other pills land
tool messages in the same thread). Convert both to `toolCallId +
hasToolResult: true` and drop the reject branch — Gemini reasons
correctly from the tool's `approved: false` return without a fixture
override. Same pattern that fixed tool-rendering-reasoning-chain
previously.
- hitl-in-app downgrade (#12346) pill produced an unrelated
"Research / Outline / Draft / Review / Finalize" plan because the
prompt contains the substring "plan" and feature-parity.json has a
generic catch-all match on `userMessage: "plan"`. Add a specific
downgrade fixture in d5-all.json (loaded before feature-parity.json)
with hasToolResult: false / true branches.
- hitl-in-chat "Schedule a 1:1 with Alice" returned the wrong
"Nice to meet you, Alice in Tokyo" response when clicked AFTER another
pill in the same thread. The 2nd-turn fixture only matched on
`toolCallId` (no hasToolResult), so the bare "alice" / "Alice" greeting
fixtures further down won. Add `hasToolResult: true` to the 2nd-turn
fixture so it scopes correctly regardless of thread state.
- Voice manual recordings always returned "What is the weather in Tokyo?"
regardless of audio content. aimock had a catch-all transcription fixture
(`match: { endpoint: "transcription" }`) that returned the canned
Tokyo string for any audio input. The D5 voice probe uses the sample
audio button which bypasses /transcribe entirely (it injects text
directly into the composer), so removing the transcription fixture
drops aimock into --proxy-only fall-through to real OpenAI Whisper for
mic recordings while D5 stays green. Verified.
- readonly-state-agent-context and shared-state-read-write fixtures
returned hardcoded "Atai" / generic preferences responses even when
the user changed the state values in the UI inputs. Gate the
Who-am-I / Suggest-next-steps / Greet / Plan-a-weekend fixtures on
systemMessage substring matching the canonical default state values
("Atai" name for readonly-state-context; "tone: casual" for
shared-state-read-write). When the user changes state, the agent's
before-model callback rebuilds the system prompt with the new values,
the fixture's systemMessage substring no longer matches, and aimock
--proxy-only falls through to the real model so the response reflects
the actual state. Confirmed with paired curl tests (default state =
fixture match; alem name = real-LLM response).
Local verification: bin/showcase test google-adk --d5 finishes 38/38
green (137s). Calculator pill confirmed via direct ADK invocation
against real Gemini (GOOGLE_GEMINI_BASE_URL=) emits TOOL_CALL_START
toolCallName=generateSandboxedUi. Sales-dashboard pill confirmed end-to-end
returning the full Column / DashboardCards / PieChart / BarChart payload.
readonly-state-context confirmed with name=Atai matching fixture vs
name=alem falling through to real-LLM response that uses the actual
context.
Five distinct root causes were keeping google-adk from full D5 parity
with langgraph-python. Fixing them takes the integration to 38/38 D5
green under aimock locally (verified end-to-end with --live writing
to PocketBase).
- readonly-state-context: page.tsx asked for agent slug
readonly_state_agent_context (underscore) but the registry mounts
it kebab-case as readonly-state-agent-context. useAgent threw,
the demo layout never mounted, and the ctx-name input never rendered.
Align with the registry (and with langgraph-python).
- multimodal: the secondary failure was a backend Pydantic
ValidationError on HttpOptions.api_endpoint. google-genai 1.75
renamed the field to base_url; all three direct genai client
constructors (main.py, beautiful_chat_agent.py, subagents_agent.py)
needed the rename so the secondary A2UI / sub-agent LLM calls stop
crashing. (The pre-existing LFS-pointer issue on the bundled
sample.png/pdf was a worktree hydration problem, not a tracked
code change — git lfs pull handles it.)
- gen-ui-declarative (two bugs stacked):
1. The same api_endpoint -> base_url rename above. The secondary
generate_a2ui planner LLM was failing every request.
2. The D5 fixture emitted components in {id, type, props: {...}}
shape, but sanitize_a2ui_components requires component, so
every entry was dropped and the renderer received an empty
surface. Rewrite both the per-demo fixture and the d5-all.json
aggregate to the flat {id, component, ...props} shape that
langgraph-python's _design_a2ui_surface fixture already uses,
wrapping multi-child layouts in a basic-catalog Column (the
custom Card schema has a single child slot). Add
_design_a2ui_surface variants so LGP gets per-pill payloads too.
- shared-state-streaming: ADK's write_document took content and
the PredictStateMapping read tool_argument="content", but the
shared D5 fixture (and the LGP function signature) names the
argument document. Rename both sides so the fixture's tool_call
args plumb into the function and into PredictStateMapping's
state-key emission. STATE_DELTA now propagates and DocumentView
streams live.
- tool-rendering-reasoning-chain: in thinking mode
(include_thoughts=True), Gemini emits a turn as two separate
non-partial chunks — a text-only chunk with finish_reason=None
and a function-call-only chunk with finish_reason=FUNCTION_CALL.
stop_on_terminal_text fired on the first (text-only) chunk and
set end_invocation=True before the function-call chunk arrived,
which broke AAPL->MSFT chaining. Gate termination on
finish_reason=STOP; FUNCTION_CALL and None both mean "more
chunks inbound — defer". Applies to every agent that uses the
shared callback, so chain-aware behavior is uniform.
Local verification: bin/showcase test google-adk --d5 --live
finishes green for all 38 cells (~140s), dashboard reflects the
results from PocketBase. Manual real-Gemini click-through of the
five fixed demos also passes end-to-end via GOOGLE_GEMINI_BASE_URL=
(empty) recreate.
31 fixtures had userMessage match + toolCalls response but no
hasToolResult constraint. They re-matched on follow-up turns where
a tool result was present, returning another tool call — infinite loop.
`<CopilotKit agent="beautiful-chat">` routes the chat to agent id
"beautiful-chat", but ExampleCanvas called `useAgent()` with no args and
fell back to DEFAULT_AGENT_ID ("default"). The frontend's agent registry
tracks state per id, so `manage_todos` state-deltas from the chat run
landed on "beautiful-chat" and never reached the canvas's "default"
subscription — the Task Manager pill auto-flipped the panel to App mode
but the To Do column stayed empty. Drop the unused "default" alias from
the runtime route and pin the canvas to `useAgent({ agentId:
"beautiful-chat" })` so both halves share one ProxiedCopilotRuntimeAgent
instance. Adds a Playwright regression test asserting the 3 verbatim
todo titles render after the pill click, plus 3 aimock fixtures for the
multi-turn flow (enableAppMode -> manage_todos -> confirmation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The tool-rendering-reasoning-chain demo previously promised chained tool
calls in its pill titles but the agent and fixtures only delivered single
tools — clicking "Weather + flights to Tokyo" produced just a WeatherCard,
"Compare two stocks" only fetched AAPL, "Find flights from SFO to JFK"
showed flights but no destination weather. Three changes close the gap.
Agent: replace the soft "call 2+ tools when relevant" system prompt with
concrete per-pill chain examples mirroring the pattern already used by the
langgraph-typescript `tool-rendering` agent (weather→flights, ticker→peer,
roll→contrast die, flights→destination weather).
Pills: drop the redundant Tokyo pill (it was the SFO/JFK chain in reverse)
and reword each remaining pill message to PRE-DISCLOSE the chain so the
model commits to the follow-up call:
- "Compare AAPL and MSFT stocks for me."
- "Roll a 20-sided die for me and compare it to a smaller one."
- "Find flights from SFO to JFK and show me the weather there."
Fixtures: 9 fixtures (3 per pill: final-content → second-leg → first-leg,
ordered by toolCallId specificity for first-match-wins). Each fixture is
scoped by a langgraph-python-UNIQUE userMessage tail ("Compare AAPL and
MSFT stocks", "compare it to a smaller one", "show me the weather there").
Those substrings appear nowhere else across the 14+ integrations sharing
showcase-aimock on Railway, so the new fixtures cannot cross-contaminate
the other reasoning-chain demos that still ship the older prompt set.
A toolName-based gate was considered and rejected because most fleet
agents register `roll_dice` and aimock's `toolName` matcher is a tool-LIST
gate, not a tool-CALL gate — it would NOT have isolated this demo.
Probe: collapse the two-turn flow (Tokyo + SFO/JFK) into one chained turn
(SFO→JFK + JFK weather) that asserts BOTH per-tool renderers
(FlightListCard + WeatherCard) mount in a single response. Same coverage
at half the wall-clock and exercises the actual chained-tool path.
Four independent showcase production bugs Alem reported, plus the
D5 multimodal harness regression they unblocked.
Shared-state-read-write: "Greet me" ("Say hi and introduce yourself.")
and "Plan a weekend" ("Suggest a weekend plan based on my interests.")
were matching the bare `hi` and `plan` catch-alls in feature-parity.json
and returning the generic showcase-assistant blurb / 5-step content plan
instead of shared-state-aware responses. Added pill-specific fixtures in
shared-state.json (mirrored into d5-all.json) so the longer userMessage
substrings win first-match-wins ahead of feature-parity.
Auth sign-out: signing out unmounted CopilotKit entirely and bounced
the user back to the SignInCard, so the demo never showcased the
runtime returning 401 — its whole point. The QA contract in
qa/auth.md spelled out the intended UX. Restored it: CopilotKit stays
mounted after the first sign-in, the AuthBanner flips to an amber
"Signed out — the agent will reject your messages" state with a
re-Sign-in button, and CopilotKit's `onError` callback drives a
`data-testid="auth-demo-error"` surface that displays the runtime's
401 the moment the user sends an unauthenticated message. Updated the
e2e spec to match (the old "SignInCard re-mounts after sign-out" test
pinned the regression).
Gen-ui-agent: the aimock fixture short-circuited the 7-step
progression spelled out in `gen_ui_agent.py`'s SYSTEM_PROMPT to a
single set_steps call with all three steps already `completed`, so
the InlineAgentStateCard rendered the final 3/3 state instantly with
no sequential pending → in_progress → completed animation.
Regenerated as a 7-leg toolCallId chain per pill (8 fixtures × 3
pills): seed leg keyed on userMessage with NO `hasToolResult` gate
(matching PR #4770's pattern — `hasToolResult: false` would block the
seed from firing on the second pill in a multi-pill session), then
six toolCallId-keyed transitions, then a final narration. Fixture
order: toolCallId legs FIRST so the most specific match wins.
Multimodal D5: the sample-attachment buttons auto-send via
`agent.addMessage + copilotkit.runAgent` (restored in PR #4761), but
the D5 harness still typed `input` + pressed Enter via the runner
after `preFill`, sending a second user message that competed with the
in-flight image upload — the v1 LangGraph runtime SSE stream got
tangled (browser DevTools showed `statusCode: pending` indefinitely)
and the assistant message never rendered. Added `skipSend?: boolean`
to ConversationTurn (distinct from `skipFill`, which still presses
Enter once the textarea has content) and switched d5-multimodal.ts to
`skipSend: true` with `responseTimeoutMs: 60_000` so the runner waits
on the assistant response without poking the chat further. Bumped the
PDF auto-prompt fixture in feature-parity.json to include the word
"document" so the existing `buildModalityAssertion("document")` check
still lands.
D5 result: 37 → 39 of 40 features passing. Only
`tool-rendering-reasoning-chain` remains and is a separate
agent/runtime bug (Tokyo Responses-API `reasoning` message survives
into the next turn's conversation history, runtime returns
`RUN_ERROR: "message role is not supported"`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The tool-rendering, frontend-tools-async, and hitl-in-app fixtures all gated
their first-leg (tool-emitting) vs. follow-up (narration) responses on
`hasToolResult: false/true` and/or `turnIndex`. Those constraints count the
*entire* thread, so once a user clicked a tool-using pill the thread already
contained tool messages and assistant turns and subsequent pill clicks fell
through to the wrong branch — d20 dropped from 5 rolls to 3, Chain tools
emitted no cards, query_notes returned narration without the Notes DB card,
and the second HITL pill never raised an approval dialog.
Re-key every follow-up fixture on the prior step's `toolCallId` (the matcher
checks `messages[last].tool_call_id`), drop the global `hasToolResult` gates
from the tool-emitting fixtures, and reorder so the toolCallId-specific
fixtures come first under first-match-wins. The d20 chain becomes a linear
toolCallId graph (`call_tr_d20_seq_001` → `_002` → … → `_005`), Chain tools
gets disambiguators for each of its three parallel tool_call_ids, and
Weather/AAPL/query_notes/HITL approve+reject branches all gate on the
specific request_user_approval / get_weather / query_notes / get_stock_price
id that landed last. userMessage matchers are unchanged.
Adds Playwright multi-pill regression tests to the four affected demos that
click every pill sequentially in one thread and assert the full card counts:
- tool-rendering-default-catchall: Find flights → 5 d20 rolls (with 20 last)
- tool-rendering-custom-catchall: 1 flights + 5 d20 + 3 chain = 9 cards
- frontend-tools-async: 3 NOTES DB cards with the right keyword per pill
- hitl-in-app: refund approve then escalate, each with its own dialog
The three interactive Open-Generative-UI (Advanced) pills — Calculator,
Ping the host, and Inline expression evaluator — shipped HTML + CSS
only in d5-all.json. The agent system prompt instructs the LLM to also
emit jsFunctions, but the fixtures didn't, so every in-iframe button
became a silent no-op: the iframe rendered, clicks dispatched no events,
and the sandbox-function bridges to evaluateExpression / notifyHost
were never exercised.
Adds the missing jsFunctions to all three fixtures, using single-quoted
JS so no escaping is needed inside the JSON-stringified tool arguments.
Each handler respects the sandbox="allow-scripts"-only iframe constraints
(no <form>, plain addEventListener) and calls back into the host via
Websandbox.connection.remote.<name>, with the expected return-shape
contract (res.ok + res.value / res.receivedAt).
Martha Schumann