mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Jordan Ritter
b2811f4feb
Commit 7c3edca changed sample-attachment-buttons.tsx across all integrations
to auto-send via agent.addMessage with autoPrompt strings:
- "can you tell me what is in this demo image I just attached"
- "can you tell me what is in this demo pdf I just attached"
But the d5 harness fixture and all 19 d6 per-integration multimodal.json
fixtures still matched on the old strings:
- "describe the sample image"
- "summarize the sample document"
Aimock received requests with the new prompts, found no match, returned
a STRICT 404, and the agent emitted a streaming error back to the UI
(exact symptom: "An internal error has occurred while streaming events").
Also update agentic-chat.json across all 20 integrations (those files had
duplicate fallback entries for the old prompts) and fix split-fixtures.ts
to route the new strings to the "multimodal" feature bucket.
Local RED: ms-agent-python and crewai-crews both fail with fixture-miss
status=miss before this change.
Local GREEN: langgraph-typescript passes after this change (both turns
settle with "image" / "document" keywords confirmed in transcript).
Remaining failures after this fix are pre-existing Python backend issues
(ChatClientException on binary content parts in ms-agent-python; CrewAI
flow failure on binary content in crewai-crews) — unrelated to fixture
keys and tracked separately in the pydantic-ai multimodal work.
52 lines
2.3 KiB
JSON
52 lines
2.3 KiB
JSON
{
|
|
"_meta": {
|
|
"description": "D6 fixtures for strands / multimodal",
|
|
"sourceFile": "d5-multimodal.ts",
|
|
"created": "2026-05-22"
|
|
},
|
|
"fixtures": [
|
|
{
|
|
"_comment": "multimodal \u2014 image upload turn. The D5 script clicks the 'Try with sample image' button (auto-sends), then asserts the assistant transcript contains 'image'. The input text is a label only (skipSend:true in the script).",
|
|
"match": {
|
|
"userMessage": "can you tell me what is in this demo image I just attached",
|
|
"context": "strands"
|
|
},
|
|
"response": {
|
|
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. The colors form a gradient grid that confirms the binary payload round-tripped through the runtime without corruption."
|
|
}
|
|
},
|
|
{
|
|
"_comment": "multimodal \u2014 PDF upload turn. The D5 script clicks the 'Try with sample PDF' button (auto-sends), then asserts the assistant transcript contains 'document'.",
|
|
"match": {
|
|
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
|
"context": "strands"
|
|
},
|
|
"response": {
|
|
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was extracted and forwarded as content to the model. Receiving this response confirms the document upload path works end-to-end."
|
|
}
|
|
},
|
|
{
|
|
"_comment": "multimodal \u2014 fallback for image analysis when the auto-send injects a different message shape.",
|
|
"match": {
|
|
"userMessage": "image",
|
|
"turnIndex": 0,
|
|
"context": "strands"
|
|
},
|
|
"response": {
|
|
"content": "I can see the image you uploaded. It appears to be a test pattern used to validate the image processing pipeline in the multimodal demo."
|
|
}
|
|
},
|
|
{
|
|
"_comment": "multimodal \u2014 fallback for document analysis when the auto-send injects a different message shape.",
|
|
"match": {
|
|
"userMessage": "document",
|
|
"turnIndex": 0,
|
|
"context": "strands"
|
|
},
|
|
"response": {
|
|
"content": "I've reviewed the document you uploaded. It contains test content used to validate the PDF processing pipeline in the multimodal demo."
|
|
}
|
|
}
|
|
]
|
|
}
|