The claude-sdk-python gen-ui-declarative D6 cell went red with
reason=done-signal-missing: the aimock fixture still carried the legacy
D5 pills (KPI/pie/bar/status) plus a single stray hero `generate_a2ui`
entry, so the current 4-prompt sales-analyst driver (Show me my sales
dashboard / How are reps performing / accounts at risk / biggest account)
had no matching fixtures. aimock STRICT mode 404'd the unmatched outer
and inner Claude calls, so the run never emitted the expected render
per turn.
Two-part fix:
- Re-author aimock/d6/claude-sdk-python/gen-ui-declarative.json into the
two-stage Anthropic-transport shape (mirrors the claude-sdk-typescript
sibling + google-adk data): per turn (a) outer generate_a2ui emit
matched by userMessage+toolName+hasToolResult, (b) inner render_a2ui
design matched by toolName, (c) outer narration matched by toolCallId.
Covers all 4 current sales prompts.
- Renderer/definition parity: add the DataTable catalog component
(definition + renderer, testid declarative-data-table) that turn 2
requires, and add the missing declarative-info-row testid to the
InfoRow renderer that turn 4 requires. Both were absent on
claude-sdk-python (present on google-adk).
Local red-green proof (control-plane, slot 12, --isolate --rebuild):
- RED (pristine): d6:claude-sdk-python/gen-ui-declarative = red,
aimock log 'STRICT: No fixture matched for POST /v1/messages'.
- GREEN (fixed): d6:claude-sdk-python/gen-ui-declarative = green,
1 passed, zero aimock no-match.
Visual verify (Playwright, harness X-AIMock-Context header): all 4 turns
paint with correct per-testid deltas (metric x4/pie/bar; data-table/bar;
status-badge x3/metric x3; info-row/pie).
## Summary
- Productizes the Claude SDK Python and TypeScript showcase demos with
LangGraph-parity frontends.
- Wires the Claude demo backends through the official Claude Agent
SDK/AG-UI adapter paths using `claude-sonnet-4.6`.
- Keeps Claude integration docs hidden for this PR and excludes
generated/authored docs artifacts from scope.
## Why
The goal is to bring the productized LangGraph demo surface to Claude
Agents SDKs without publishing integration docs in this pass. This keeps
the PR focused on local showcase demos, runtime behavior, fixtures, and
validation support.
## How
- Ported the demo frontend surfaces and local shell-dojo support for
Claude SDK Python/TypeScript.
- Added official Claude SDK adapter/backend wiring plus real-Claude
local compose support.
- Updated Claude aimock fixtures and validation ratchets for the
expanded demo set.
- Set both Claude manifests to `docs_mode: hidden` and removed docs
setup/snippet artifacts from the PR scope.
Commit 7c3edca changed sample-attachment-buttons.tsx across all integrations
to auto-send via agent.addMessage with autoPrompt strings:
- "can you tell me what is in this demo image I just attached"
- "can you tell me what is in this demo pdf I just attached"
But the d5 harness fixture and all 19 d6 per-integration multimodal.json
fixtures still matched on the old strings:
- "describe the sample image"
- "summarize the sample document"
Aimock received requests with the new prompts, found no match, returned
a STRICT 404, and the agent emitted a streaming error back to the UI
(exact symptom: "An internal error has occurred while streaming events").
Also update agentic-chat.json across all 20 integrations (those files had
duplicate fallback entries for the old prompts) and fix split-fixtures.ts
to route the new strings to the "multimodal" feature bucket.
Local RED: ms-agent-python and crewai-crews both fail with fixture-miss
status=miss before this change.
Local GREEN: langgraph-typescript passes after this change (both turns
settle with "image" / "document" keywords confirmed in transcript).
Remaining failures after this fix are pre-existing Python backend issues
(ChatClientException on binary content parts in ms-agent-python; CrewAI
flow failure on binary content in crewai-crews) — unrelated to fixture
keys and tracked separately in the pydantic-ai multimodal work.
Mirrors LGP's `generate_a2ui` outer-emit fixture into 8 non-LGP slugs to close
the production 503 gap on the "Show me my sales dashboard for this quarter."
userMessage. Per /tmp/staging-journal-diff.md, aimock-staging returns 503 on
shape-B traffic (model=gpt-4.1, stream=true, tools=[generate_a2ui],
UA=AsyncOpenAI/Python) for all 8 slugs because no fixture matched.
Slugs patched (one entry each in d6/<slug>/gen-ui-declarative.json):
- llamaindex
- built-in-agent
- ag2
- langroid
- claude-sdk-typescript
- claude-sdk-python
- ms-agent-dotnet
- ms-agent-python
Each entry matches `userMessage` + `context: "<slug>"` and emits a
`generate_a2ui` toolcall with no args, mirroring LGP's sales-dashboard outer
entry. Per-slug unique toolCallId.
Red-green proof (local aimock, ghcr.io/copilotkit/aimock:latest):
RED (baseline main, 8 slugs): HTTP=404 no_fixture_match
GREEN (this branch, 8 slugs): HTTP=200 tool=generate_a2ui id=call_d6_decl_dash_outer_<slug>_001
LGP regression (baseline+fix): HTTP=200 (unchanged)
aimock fixture validation: 737/737 tests pass.
PR scope is intentionally narrow per CLAUDE.md "Scope PRs to flagged
findings": this closes ONE userMessage gap. Shape-A 503s (no tools key) and
other userMessage gaps remain as separate follow-ups.
Closed PR #5465 introduced cross-fixture leakage by stripping the
toolName discriminator on shared {userMessage, context} keys: with
aimock's alphabetical first-match-wins ordering,
tool-rendering-custom-catchall.json sorts before
tool-rendering-default-catchall.json, so default-catchall page requests
were served the custom file's content. The probe was structurally blind
because it asserted only DOM testids (copilot-tool-render +
data-tool-name=get_weather) — both fixtures emit get_weather, so the
testid signal passed regardless of which fixture won.
Real LGP-gold pattern is disjoint userMessages between default and
custom catchall fixtures (NOT shared keys discriminated by toolName).
This PR ports the LGP-gold pattern to the other 17 integrations and
adds page-text content assertions to both d5 catchall probes so this
class of regression can't recur silently:
- Probe prompts disjoint between default-catchall and custom-catchall
- Fixture userMessages updated to match the new disjoint prompts
- Page-text content assertions in both d5 catchall probes
(default negatively asserts the custom-catchall leak phrase;
custom positively asserts it)
Local gold-standard red-green proof captured:
- Step A: live staging Playwright baseline (RED-OF-RECORD)
- Step B: local control-plane on origin/main reproduces structural
fragility (probes pass at testid level, custom fixture wins on
default's userMessage path)
- Step C: local control-plane on this branch — both catchall probes
GREEN with the new content-asserting assertions
tool-rendering-default-catchall: pass=true (5443ms)
tool-rendering-custom-catchall: pass=true (8956ms)
cross-tool signature passed
LGP (langgraph-python) was already disjoint; it remains untouched.
PR #5459 added userMessage matchers ("forecast for Tokyo" + "current price
of AAPL") on the catchall fixtures, which flipped most integrations from
red to green. Six cells stayed red on staging because the FIXTURE SHAPE
itself was broken on the matched fixtures, not just the userMessage key.
Failing cells (all custom-catchall except strands default):
d5:agno/tool-rendering-custom-catchall
d5:llamaindex/tool-rendering-custom-catchall
d5:langroid/tool-rendering-custom-catchall
d5:claude-sdk-python/tool-rendering-custom-catchall
d5:strands/tool-rendering-custom-catchall
d5:strands/tool-rendering-default-catchall
PocketBase confirms the failure mode: turn 1 (Tokyo) completes; turn 2
(AAPL) times out at 30s or renders the wrong content. E.g. agno:
errorDesc: "timeout: assistant did not respond within 30000ms"
failure_turn: 2
turns_completed: 1 / 2
Root cause: the failing tool-emit fixtures used `hasToolResult: false`
(or `toolName: "<tool>"`) as their gate. aimock's hasToolResult check is
`messages.some(m => m.role === 'tool')` over the WHOLE thread — so once
turn 1's Tokyo tool result lands in the conversation, hasToolResult is
permanently true and `hasToolResult:false` can never match turn 2 → no
fixture → 30s timeout. `toolName:get_stock_price` likewise fails when an
integration backend doesn't forward the tool definition on turn 2.
The same trap is documented in
showcase/aimock/d6/langgraph-python/tool-rendering.json:
"_comment": "Gated on toolName:get_stock_price rather than
hasToolResult:false. The D5 tool-rendering-custom-catchall probe runs
'weather in Tokyo' first, which leaves a get_weather tool result in
the thread; the aimock router implements hasToolResult as
messages.some(m=>m.role==='tool'), so hasToolResult is permanently
true on the AAPL turn and a hasToolResult:false gate could never
match (→ no_fixture_match → 503 → 30s timeout)."
Fix: align all 6 files to the canonical pattern used by mastra/spring-
ai/built-in-agent on this probe:
1. toolCallId-keyed narration fixture FIRST
2. tool-emit fixture SECOND with ONLY `userMessage` + `context`
(no hasToolResult / toolName gate)
The toolCallId narration uses aimock's
`messages[last].role === 'tool' && tool_call_id === ...` check, so it
correctly wins on iteration 2 (post-tool-result) without being affected
by older turns' tool results. The tool-emit fixture matches turn 1 (last
message is user) and re-emits only when the narration above hasn't
matched.
Strands' two cells additionally needed REORDERING — they had the tool-
emit fixture before the toolCallId fixture, defeating first-match-wins.
Strands' staging backend was also returning 502 during testing; once it
recovers, the corrected fixtures should let the probe pass. The fixture
changes are necessary but may not be sufficient for strands if backend
remains down.
No probe-side, harness, or backend changes — pure fixture-content
alignment. Six fixture files modified; line totals: -87 / +68.
The previous 'weather in Tokyo' rename (388c69e68) was a substring of the
main tool-rendering chain pill prompt 'Chain a few tools in this single
turn: get the weather in Tokyo, search flights from SFO to Tokyo, and roll
a d20.' Because aimock loads fixtures alphabetically per integration dir
and uses substring match with first-match-wins, the catchall fixture
(loaded before tool-rendering.json) was intercepting chain pill matches
across 16 integrations.
Rename catchall fixture userMessage to 'forecast for Tokyo' — a phrase
not contained in any other pill prompt. Update the corresponding D5
catchall probe inputs in d5-tool-rendering-{default,custom}-catchall.ts
and the test assertions that pin those inputs.
Call-Site Enumeration: 'weather in Tokyo' remains intentionally in
page.tsx suggestions.ts pills (user-visible UX) and inside the chain
pill prompt itself — neither is in the substring-match path now.
The D5 e2e-deep probe for tool-rendering-{default,custom}-catchall sends
"weather in Tokyo" as the test input (harness/src/probes/scripts/
d5-tool-rendering-{default,custom}-catchall.ts). The fixture userMessage
matcher uses substring match. Main's catchall fixtures had a stale
"check Tokyo weather forecast" string that could not substring-match
the probe input, causing the fixture to miss and the probe to fall
through to the live LLM — surfacing as CV x red D4 on the dashboard.
Rename to the canonical "weather in Tokyo" string across affected
integrations' tool-rendering-{default,custom}-catchall.json files.
No agent or page.tsx changes; fixture content otherwise unchanged.
Adds the missing gen-ui-a2ui-fixed aimock fixture and reshapes the
declarative-gen-ui fixture so the claude-sdk-python D6 probes have
deterministic LLM replay for both gen-ui demos.
- SUPERSEDED annotations on unreachable recorded calculator entries (load-order shadowing is
the only guarantee; model gate is not a safety net)
- GOTCHAS sequenceIndex rewritten to per-X-Test-Id semantics with co-increment/eviction caveats
- hasToolResult paragraph corrected (omission = no gate, thread-global predicate)
- statelessness claim reconciled with sequence counters
- replace Websandbox host-bridge evaluateExpression (never registered in beautiful-chat) with
in-sandbox allowlist-gated Function() eval
- reorder specific calculator pair before generic (first-match-wins)
- allow exponential notation (eE) so chained evaluation of large results works
- drop thread-global hasToolResult gate that broke the pill after other tool-producing pills,
reorder toolCallId follow-ups before leg-1
- mint distinct tool_call_ids per repeat click via sequenceIndex variants + non-sequenced
fallback (fixes second-widget collapse)
- restore google-adk trailing newline
- document ordering invariants, gate tradeoffs, and sequence-counter scoping caveats in
fixture comments
Verified via local Playwright red-green (render, 7+8=15, chained exponent, interleaved pills,
repeat clicks).
The gen-ui-headless-complete probe
(showcase/harness/src/probes/scripts/d5-gen-ui-headless-complete.ts)
references fixtureFile "gen-ui-headless-complete.json", but that file
existed in no D6 slug, so the probe's first leg 503'd under strict.
Add gen-ui-headless-complete.json for all 18 D6 slugs, modeled on the
green langgraph-python headless-complete.json pattern: the four gen-UI
pills (weather/stock/highlight/revenue) with narration (toolCallId)
fixtures FIRST and toolcall (userMessage+context) fixtures AFTER, no
turnIndex gate, so every one of the probe's four sequential turns in one
chat thread matches regardless of prior assistant/tool history. Interrupt
fixtures are intentionally omitted (interrupt-headless is a separate
cluster item).
These 8 fixtures per slug share match keys with the pre-existing
headless-complete.json for the same context (the demos share pills and
are disambiguated at runtime by probe path), which raises the
aimock-fixtures collision-detection exact-duplicate count by 46. Bump
KNOWN_DUPLICATE_CEILING 230 -> 276 to match, consistent with how prior
per-integration feature fixtures bumped the baseline; the substring-shadow
ceiling is unchanged (no new shadows).
Validated: all 732 aimock-fixtures schema/collision tests pass, and a
local aimock --strict run returns 200 for each of the four pills across
turns (langgraph-python + pydantic-ai spearheads, plus spring-ai).
Refresh d6 fixtures across the rollout cohort: ag2, built-in-agent,
claude-sdk-{python,typescript}, crewai-crews, google-adk, langgraph-fastapi,
langgraph-typescript, langroid, llamaindex, mastra, ms-agent-{dotnet,python},
pydantic-ai, strands. Companion d4/{langgraph-typescript,mastra,ms-agent-dotnet}
chat.json refreshes. Add the missing ms-agent-python/gen-ui-custom.json
to bring the integration up to the standard pill set.
Also narrow aimock/shared/common.json's generic 'hello' fixture to
'hello world' so it no longer shadows D6 pills whose prompts contain
'hello' as a substring (e.g. langgraph-python headless-simple sends
'Say hello in one short sentence.'). 'hello world' is unused by any
current demo pill, so the fixture remains a manual-typing fallback
without poisoning fixture matching.
This is a mid-rollout snapshot — fixture coverage is uneven across
integrations and rides alongside the conveyance shims landed earlier
in this branch.
Move monolithic d5-all.json + feature-parity.json + smoke.json into
per-integration directories under d4/<slug>/, d6/<slug>/, and shared/.
Every fixture file is now context-scoped to enable server-side aimock
routing via match.context. Migrated 12 HITL fixtures from main's
d5-all.json additions into shared/_migrated-from-d5-all-hitl.json
for follow-up distribution into per-integration files.
Mark