The D6 e2e-full probe for ms-agent-python:gen-ui-declarative failed at turn 1
with reason=surface-missing. Two root causes, both fixed:
1. Stale aimock fixture. The fixture still carried the old D5 pill prompts
(KPI/pie/bar/status) plus a lone outer generate_a2ui entry for the current
sales-dashboard prompt with no matching inner _design_a2ui_surface or
narration. The backend looped generate_a2ui to its invocation limit and
RUN_FINISHED was blocked while the tool call stayed active. Re-authored to
the current 4 VantageThreads sales prompts mirroring the llamaindex green
north-star for this backend shape: the outer generate_a2ui returns a
`context` steering phrase (the ms-agent-framework session does not surface
the latest user message to the secondary LLM), the inner
_design_a2ui_surface fixture matches that phrase (not the full prompt), and
hasToolResult discriminates outer vs narration. This also avoids the stale
render-a2ui "KPI dashboard" catch-all collision that was rendering the wrong
surface.
2. Renderer/catalog drift. ms-agent-python's declarative-gen-ui renderers.tsx
and definitions.ts lagged the green peers — missing the DataTable and
InfoRow components (declarative-data-table / declarative-info-row testids)
that turns 2 and 4 assert, plus Metric trendValue and the Row/Column/Text
gap overrides. Brought both files to parity with the langgraph-python /
google-adk green cluster.
Red-green proof on the real control-plane surface (SHOWCASE_ISO_SLOT=11,
--d6 --isolate):
- RED: d6:ms-agent-python/gen-ui-declarative = red (exit 1, turn 1 surface-missing)
- GREEN: d6:ms-agent-python/gen-ui-declarative = green (exit 0, 1 passed)
Visually verified all 4 turns via Playwright (header-injected to replicate the
harness x-aimock-context): turn 1 renders 4 KPI metrics + region pie + monthly
bar; turn 2 the rep-quota DataTable + attainment bar; turn 3 three at-risk
StatusBadges + KPI metrics; turn 4 seven InfoRow account facts + product-line
pie.
Mirrors LGP's `generate_a2ui` outer-emit fixture into 8 non-LGP slugs to close
the production 503 gap on the "Show me my sales dashboard for this quarter."
userMessage. Per /tmp/staging-journal-diff.md, aimock-staging returns 503 on
shape-B traffic (model=gpt-4.1, stream=true, tools=[generate_a2ui],
UA=AsyncOpenAI/Python) for all 8 slugs because no fixture matched.
Slugs patched (one entry each in d6/<slug>/gen-ui-declarative.json):
- llamaindex
- built-in-agent
- ag2
- langroid
- claude-sdk-typescript
- claude-sdk-python
- ms-agent-dotnet
- ms-agent-python
Each entry matches `userMessage` + `context: "<slug>"` and emits a
`generate_a2ui` toolcall with no args, mirroring LGP's sales-dashboard outer
entry. Per-slug unique toolCallId.
Red-green proof (local aimock, ghcr.io/copilotkit/aimock:latest):
RED (baseline main, 8 slugs): HTTP=404 no_fixture_match
GREEN (this branch, 8 slugs): HTTP=200 tool=generate_a2ui id=call_d6_decl_dash_outer_<slug>_001
LGP regression (baseline+fix): HTTP=200 (unchanged)
aimock fixture validation: 737/737 tests pass.
PR scope is intentionally narrow per CLAUDE.md "Scope PRs to flagged
findings": this closes ONE userMessage gap. Shape-A 503s (no tools key) and
other userMessage gaps remain as separate follow-ups.
Same pattern as ms-agent-dotnet: full d4 chat.json rewrite + d6 mirrors. Dropped stale
turnIndex gates and broad shadow matchers. Default-catchall now green via page-level
shadcn-catchall-renderer. 11 residual: declarative-gen-ui charts, multimodal conveyance,
tool-rendering, reasoning-chain.
Refresh d6 fixtures across the rollout cohort: ag2, built-in-agent,
claude-sdk-{python,typescript}, crewai-crews, google-adk, langgraph-fastapi,
langgraph-typescript, langroid, llamaindex, mastra, ms-agent-{dotnet,python},
pydantic-ai, strands. Companion d4/{langgraph-typescript,mastra,ms-agent-dotnet}
chat.json refreshes. Add the missing ms-agent-python/gen-ui-custom.json
to bring the integration up to the standard pill set.
Also narrow aimock/shared/common.json's generic 'hello' fixture to
'hello world' so it no longer shadows D6 pills whose prompts contain
'hello' as a substring (e.g. langgraph-python headless-simple sends
'Say hello in one short sentence.'). 'hello world' is unused by any
current demo pill, so the fixture remains a manual-typing fallback
without poisoning fixture matching.
This is a mid-rollout snapshot — fixture coverage is uneven across
integrations and rides alongside the conveyance shims landed earlier
in this branch.
Jordan Ritter