Two independent fastapi divergences from the green north-star kept this red:
1. Missing chunkSize:9999 on the 8 tool-call fixtures. Under aimock's global
8-byte chunking the large tool-call args failed to JSON-parse in one piece,
so the AAPL->MSFT reasoning chain stopped at leg 1 (turns 1 & 2). Byte-align
to LGP (and the green mastra/langgraph-typescript siblings carry the same
pattern).
2. Fixture-pool shadowing broke turn 3. aimock pools all d4+d6 fixtures for a
context and matches by userMessage substring, first-match-wins in load order
(d4 before d6). fastapi had broad keys LGP doesn't:
- d4/chat.json: 'weather' -> 'weather in San Francisco', 'flights from SFO
to JFK' -> 'Find flights from SFO to JFK.' (period-terminated so it stops
being a substring of turn 3's 'Find flights from SFO to JFK and show me
the weather there.').
- tool-rendering-{custom,default}-catchall.json: bare 'Find flights' ->
'Find flights from SFO to JFK.' (now matches LGP's exact key).
Also byte-align the backend agent to LGP: scriptable get_stock_price signature,
the detailed chaining system prompt, model gpt-5.4, reasoning summary detailed.
D6 tool-rendering-reasoning-chain green (two real ~21s runs). Shared-fixture
regression all green: agentic-chat, tool-rendering, tool-rendering-custom-catchall,
tool-rendering-default-catchall, headless-complete (Tokyo-weather consumer).
gen-ui-agent had no dedicated backend graph: langgraph.json lacked a
gen_ui_agent entry and route.ts routed the name through the neutral-assistant
loop to sample_agent, which has no steps state or set_steps tool, so the
progress card never mounted (agent hit the default recursion limit of 25).
- Port LGP's gen_ui_agent.py (byte-identical) and register it in langgraph.json.
- route.ts: bind gen-ui-agent to createAgent("gen_ui_agent") and bake
assistantConfig.recursion_limit (default 100) into every LangGraphAgent —
the graph's Python with_config isn't visible to the server runs API, so the
multi-step set_steps walk overran 25. Mirrors langgraph-python.
- aimock: regenerate d6/gen-ui-agent.json from LGP (adds chunkSize:9999 on the
24 tool-call fixtures) and narrow the over-broad d4 chat.json "summarize" key
to "Summarize the sales pipeline" so it stops substring-shadowing the
competitor set_steps chain (and other summarize prompts). Matches LGP.
D6 langgraph-fastapi:gen-ui-agent now green (two real ~14s runs);
agent-config re-verified green (no regression).
headless-simple's "Give me a fun fact." pill rendered the wrong text
("Here's a fun fact: honey never spoils…") and the e2e failed. Root cause:
d4/mastra/chat.json keyed the prebuilt-sidebar agentic-chat fixture on the
bare "fun fact", which is a substring of the headless-simple pill "Give me a
fun fact." Because d4 loads before d6, that fixture shadowed headless-simple's
own d6 fixture ("A fun fact: Honey never spoils!").
Re-key it to the unique "fun fact for the prebuilt sidebar" phrase (mirrors
gold langgraph-python), so it no longer substring-collides. The prebuilt-sidebar
demo pill "Give me a fun fact." now shares headless-simple's d6 fun-fact
fixture, exactly as gold does; prebuilt-sidebar/popup e2e unaffected.
Verified (Node 22 + next start + aimock 1.37.4 replay): headless-simple 4/4,
prebuilt-sidebar 4/4, prebuilt-popup 4/4 — and the full mastra card/chat suite
green (tool-rendering 6/6, tool-rendering-reasoning-chain 5/5, headless-complete
5/5, beautiful-chat 8/8, agentic-chat 4/4). Only a userMessage key changed
(more specific → strictly fewer aimock substring-shadows, well under the
KNOWN_SHADOW_CEILING).
--no-verify: sparse showcase checkout has no monorepo lefthook/commitlint binaries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sales Dashboard (A2UI Dynamic) never produced a surface: the outer
generate_a2ui fixture carried empty {} args (generate_a2ui input
validation failed on the required 'messages' field) and the inner
render_a2ui fixture gated on context:mastra, which the secondary-LLM
request never carries. Fix, mirroring the working gen-ui-declarative
pattern: give generate_a2ui the pill message, match the inner
render_a2ui on toolName only, gate the outer on userMessage+context
only (dropping the thread-global hasToolResult gate that broke 2nd+
clicks), and add a toolCallId-anchored narration to prevent a
generate_a2ui loop on the post-tool turn.
Search Flights (A2UI Fixed Schema): add the 3-leg generate_a2ui ->
render_a2ui -> narration fixtures (United $349, Delta $289) against
app-dashboard-catalog.
Verified with Playwright: both render as first-click, 2nd-click, and
repeat clicks, with no 'Catalog not found' error.
The reasoning demos (reasoning-default, reasoning-custom,
tool-rendering-reasoning-chain) never rendered a reasoning message. Two
root causes, both showcase-side wiring (the @ag-ui/mastra bridge forwards
reasoning correctly):
1. Agent-not-found: the reasoning-default and reasoning-custom pages request
agent="reasoning-default"/"reasoning-custom", but the runtime registry
listed the stale feature keys "reasoning-default-render" and
"agentic-chat-reasoning" instead, so the chat never started. Register the
real agent names (the demoAgentNames parity test enforces this).
2. Non-reasoning model: all reasoning demos mapped to the default weatherAgent
(gpt-4o), which the OpenAI Responses API never emits reasoning-summary
items for, so the reasoning slot stayed dark. Add a dedicated reasoningAgent
(gpt-5-mini via OPENAI_REASONING_MODEL) with
providerOptions.openai.{reasoningEffort,reasoningSummary:"detailed"} on the
agent's default stream options, mirroring langgraph-python's reasoning_agent.
Map reasoning-default and reasoning-custom to it.
For tool-rendering-reasoning-chain, add a dedicated reasoningChainAgent that
registers the four chain tools (get_weather, search_flights, get_stock_price,
roll_dice, the latter new) under the exact tool-call names the aimock fixtures
emit, on the reasoning model, so Mastra executes each leg and the multi-turn
chain advances through its toolCallId-keyed fixtures to the closing narration.
Fixture parity fixes so the chain's gold toolCallId scheme wins on the
Responses path (matching langgraph-python):
- d4/mastra/chat.json: rename the broad "weather" and "flights from SFO to JFK"
probes to the non-colliding "_d4_unused_*" sentinels gold uses.
- d6/mastra/tool-rendering.json: the basic AAPL fixture used turnIndex:0, which
matched as a behind-count turn and stole the stock chain's later turns; switch
to hasToolResult:false (gold parity) so it only answers the no-tool-result turn.
Manifest features aligned to gold (reasoning-default, reasoning-custom). Removed
the stray agentic-chat-reasoning e2e spec (no page, no gold equivalent) and
renamed the reasoning QA docs to match the cell ids.
Verified via Playwright against the up --dev mastra container + aimock:
reasoning-default 2/2, reasoning-custom 7/7, tool-rendering-reasoning-chain 5/5.
The bare-substring 'weather' fixture in aimock/d4/llamaindex/chat.json emitted a
get_weather tool call with no toolName gate. The tool-free voice agent's prompt
"What is the weather in Tokyo?" (substring "weather") leaked into this fixture,
emitting a get_weather call the voice agent could never resolve, so the voice D6
cell hung (done-signal-missing, body stuck on get_weather/Running).
Add toolName:"get_weather" so the fixture only fires when the requesting agent
actually registers get_weather (mirrors the gate in d6 tool-rendering.json). The
tool-free voice request now falls through to voice.json's exact content match.
Local red->green proof (showcase test llamaindex:voice --d6 --direct):
- RED: done-signal-missing; body "What is the weather in Tokyo? get_weather Running"
- GREEN: assistant settled "The weather in Tokyo is currently 22C with partly
cloudy skies and light easterly winds."; 1 passed (3.0s)
Direct aimock probes confirm the gate: tool-free -> content; with get_weather tool
-> tool call still fires. Regression: tool-rendering D6 still green; headless-complete
weather turn still passes (uses its own gen-ui-headless-complete.json fixture).
(cherry picked from commit cf6ff7c08153367239437d6c4fff425d546eb245)
The bare 'summarize' userMessage in d4/llamaindex/chat.json substring-matched
the D6 gen-ui-agent pill 'Research our top competitor and summarize their
strengths and weaknesses.', returning the sales-pipeline text fixture instead
of the gen-ui-agent set_steps tool call. The competitor pill then produced
no/duplicate steps, failing d6:llamaindex. Narrow the match to the verbatim D4
toolbar probe 'Summarize the sales pipeline' (langgraph-python parity), which
no demo pill contains as a substring. D4 llamaindex stays green (the bare entry
was unused by any D4 cell).
(cherry picked from commit c23801c8b32d292cacf6fb2e7e2a68270eebaa84)
Strands mints UUID tool_call_ids for tool results (confirmed via the aimock
journal), so the langgraph-python toolCallId-keyed follow-up fixtures never
matched and the agent re-emitted the tool, looping (text-unstable) across
frontend-tools, gen-ui-agent, gen-ui-open, gen-ui-open-advanced,
gen-ui-headless-complete, reasoning-chain and the weather pills. Re-key the
affected multi-leg fixtures to the id- and thread-history-invariant
sequenceIndex pattern (as built-in-agent does), for both integrations.
Also:
- Add write_document poem/email/quantum fixtures for shared-state-streaming
(was a stale single fixture that 404d).
- Strip content from reasoning-chain tool legs (content+toolCalls in one
fixture is undefined behavior); reasoning rides toolCalls alone.
- Narrow the over-broad d4 summarize catch-all to "Summarize the sales
pipeline" so it stops shadowing gen-ui-agent competitor pill; matches
langgraph-python.
- Commit the real multimodal sample.png/sample.pdf (were git-LFS pointers
the harness could not resolve).
Bring the TypeScript AWS Strands integration to parity with the Python
strands sibling now that the @ag-ui/aws-strands TS adapter is confirmed to
support the same feature surface (per its examples/server):
- Restore A2UI: the declarative-gen-ui + a2ui-fixed-schema demos, their
routes, qa, specs, the @copilotkit/a2ui-renderer dep, beautiful-chat's
A2UI catalog, and the manifest entries (generative_ui / features / demos /
a2ui_pattern). manifest now matches strands-python feature-for-feature.
- Header forwarding: attach `x-aimock-context: strands-typescript` as a
static defaultHeader on the OpenAI client (model-factory + sub-agent
client) — the TS analog of the Python integration's _header_forwarding
shim — so aimock matches this integration's fixtures.
- aimock fixtures: add d6/strands-typescript + d4/strands-typescript
(ported from the Python sibling, context retargeted).
- playwright.config: X-AIMock-Context → strands-typescript.
Note: the raw tests/e2e Playwright suite is flaky and not a CI merge gate
(demo e2e / `/eval` D5 are comment-triggered, not required) — it fails the
same specs for strands-python too. The auto-gates (build, validate-
constraints, oxlint/oxfmt, unit) are green.
The injected/streamed a2ui fixtures all included catalogId, so aimock
replay never exercised the basic-catalog fallback that broke production
(real models omit catalogId per the tool-usage guide). Strip catalogId
from the langgraph-python sales-dashboard secondary-call fixtures and
hard-assert "Catalog not found" is absent outside the charts-rendered
soft branch, so the spec fails without a route defaultCatalogId.
Also repoint the on-demand e2e workflow at the d4/d5-recorded/d6/shared
fixture dirs — it still referenced feature-parity.json, deleted in the
1e66a5f8d fixture reorg, so every /test-aimock run died at aimock start.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same pattern as ms-agent-dotnet: full d4 chat.json rewrite + d6 mirrors. Dropped stale
turnIndex gates and broad shadow matchers. Default-catchall now green via page-level
shadcn-catchall-renderer. 11 residual: declarative-gen-ui charts, multimodal conveyance,
tool-rendering, reasoning-chain.
Full d4 chat.json rewrite + d6 mirrors. Dropped stale turnIndex gates and broad shadow
matchers that no longer reflect LGP-canonical conveyance. Default-catchall now green via
page-level shadcn-catchall-renderer (not react-core-gated). 5 residual: chat-slots, hitl,
interrupt-headless, readonly-state.
Mirrors the LGP first-match-wins fix on the langgraph-typescript side. Three broad
matchers in d4/langgraph-typescript/chat.json were shadowing d6 fixtures; narrowed
them so the d6 pills win.
Across the d6/langgraph-typescript suite: dropped fragile turnIndex:0 gates in favour
of hasToolResult:false for first-leg tool emissions, fixed em-dash escaping that broke
literal string matches in multi-pill threads, and added jsFunctions payloads to the
three sandboxed-ui fixtures (_from-feature-parity, headless-complete,
gen-ui-open-advanced) so the sandbox renderer has executable handlers.
LGT D6 now passes 184/1/2 locally. Residual 1 fail is the custom-catchall multi-pill
follow-up case; tracked separately.
Aimock fixture load order is shared -> d4 -> d6, and matching is first-match-wins. A
broad d4 langgraph-python chat fixture for search_flights was shadowing the d6
beautiful-chat fixture and preventing the intended response from firing. Neutered the
d4 matcher to a non-matching sentinel so the d6 fixture wins.
Also refreshed d6/tool-rendering-custom-catchall.json and d6/tool-rendering.json:
replaced fragile turnIndex:0 gates with hasToolResult:false so the AAPL first-leg
fixture fires correctly when D5 probes run a prior 'weather in Tokyo' pill in the
same thread (multi-pill turnIndex>=2).
LGP D6 now passes 185/0/2 locally (2 skips are the by-design mcp-apps iframe gap).
Refresh d6 fixtures across the rollout cohort: ag2, built-in-agent,
claude-sdk-{python,typescript}, crewai-crews, google-adk, langgraph-fastapi,
langgraph-typescript, langroid, llamaindex, mastra, ms-agent-{dotnet,python},
pydantic-ai, strands. Companion d4/{langgraph-typescript,mastra,ms-agent-dotnet}
chat.json refreshes. Add the missing ms-agent-python/gen-ui-custom.json
to bring the integration up to the standard pill set.
Also narrow aimock/shared/common.json's generic 'hello' fixture to
'hello world' so it no longer shadows D6 pills whose prompts contain
'hello' as a substring (e.g. langgraph-python headless-simple sends
'Say hello in one short sentence.'). 'hello world' is unused by any
current demo pill, so the fixture remains a manual-typing fallback
without poisoning fixture matching.
This is a mid-rollout snapshot — fixture coverage is uneven across
integrations and rides alongside the conveyance shims landed earlier
in this branch.
Refresh d6/langgraph-python fixtures (beautiful-chat, chat-slots,
frontend-tools-async, gen-ui-{agent,custom,declarative,interrupt,open},
headless-complete, hitl-in-app, interrupt-headless, shared-state-streaming,
tool-rendering, tool-rendering-reasoning-chain, _from-feature-parity) plus
d4/langgraph-python/chat.json. Combined with the conveyance shim landed
in this branch, LGP is fixture-complete at 180 pass / 5 fail / 2 skip.
The residual 5 fails trace to react-core/v2's custom-wildcard-renderer
bug surfaced through gen-ui-interrupt (shared useInterrupt hook misbehaves
on the second interrupt; cross-integration D6 blocker, not a fixture or
conveyance defect). The 2 skips are the by-design mcp-apps iframe gap.
Move monolithic d5-all.json + feature-parity.json + smoke.json into
per-integration directories under d4/<slug>/, d6/<slug>/, and shared/.
Every fixture file is now context-scoped to enable server-side aimock
routing via match.context. Migrated 12 HITL fixtures from main's
d5-all.json additions into shared/_migrated-from-d5-all-hitl.json
for follow-up distribution into per-integration files.
Mark