Commit Graph

4 Commits

Author SHA1 Message Date
Mark 23cb3987fc fix(showcase): make multimodal fixtures factual and canonical 2026-08-16 20:33:11 -07:00
Jordan Ritter b2811f4feb fix(showcase): update multimodal fixture match keys to match actual autoPrompts
Commit 7c3edca changed sample-attachment-buttons.tsx across all integrations
to auto-send via agent.addMessage with autoPrompt strings:
  - "can you tell me what is in this demo image I just attached"
  - "can you tell me what is in this demo pdf I just attached"

But the d5 harness fixture and all 19 d6 per-integration multimodal.json
fixtures still matched on the old strings:
  - "describe the sample image"
  - "summarize the sample document"

Aimock received requests with the new prompts, found no match, returned
a STRICT 404, and the agent emitted a streaming error back to the UI
(exact symptom: "An internal error has occurred while streaming events").

Also update agentic-chat.json across all 20 integrations (those files had
duplicate fallback entries for the old prompts) and fix split-fixtures.ts
to route the new strings to the "multimodal" feature bucket.

Local RED: ms-agent-python and crewai-crews both fail with fixture-miss
  status=miss before this change.
Local GREEN: langgraph-typescript passes after this change (both turns
  settle with "image" / "document" keywords confirmed in transcript).

Remaining failures after this fix are pre-existing Python backend issues
(ChatClientException on binary content parts in ms-agent-python; CrewAI
flow failure on binary content in crewai-crews) — unrelated to fixture
keys and tracked separately in the pydantic-ai multimodal work.
2026-07-06 15:31:51 -07:00
Jordan Ritter c414fbad23 test(showcase): D6 per-integration aimock fixtures + shared narrowing
Refresh d6 fixtures across the rollout cohort: ag2, built-in-agent,
claude-sdk-{python,typescript}, crewai-crews, google-adk, langgraph-fastapi,
langgraph-typescript, langroid, llamaindex, mastra, ms-agent-{dotnet,python},
pydantic-ai, strands. Companion d4/{langgraph-typescript,mastra,ms-agent-dotnet}
chat.json refreshes. Add the missing ms-agent-python/gen-ui-custom.json
to bring the integration up to the standard pill set.

Also narrow aimock/shared/common.json's generic 'hello' fixture to
'hello world' so it no longer shadows D6 pills whose prompts contain
'hello' as a substring (e.g. langgraph-python headless-simple sends
'Say hello in one short sentence.'). 'hello world' is unused by any
current demo pill, so the fixture remains a manual-typing fallback
without poisoning fixture matching.

This is a mid-rollout snapshot — fixture coverage is uneven across
integrations and rides alongside the conveyance shims landed earlier
in this branch.
2026-05-29 16:16:12 -07:00
Jordan Ritter 1e66a5f8d2 feat(showcase-aimock): per-framework D4/D6 fixture reorg into d4/d6/shared dirs
Move monolithic d5-all.json + feature-parity.json + smoke.json into
per-integration directories under d4/<slug>/, d6/<slug>/, and shared/.
Every fixture file is now context-scoped to enable server-side aimock
routing via match.context. Migrated 12 HITL fixtures from main's
d5-all.json additions into shared/_migrated-from-d5-all-hitl.json
for follow-up distribution into per-integration files.
2026-05-26 11:25:30 -07:00