mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Jordan Ritter
b2811f4feb
Commit 7c3edca changed sample-attachment-buttons.tsx across all integrations
to auto-send via agent.addMessage with autoPrompt strings:
- "can you tell me what is in this demo image I just attached"
- "can you tell me what is in this demo pdf I just attached"
But the d5 harness fixture and all 19 d6 per-integration multimodal.json
fixtures still matched on the old strings:
- "describe the sample image"
- "summarize the sample document"
Aimock received requests with the new prompts, found no match, returned
a STRICT 404, and the agent emitted a streaming error back to the UI
(exact symptom: "An internal error has occurred while streaming events").
Also update agentic-chat.json across all 20 integrations (those files had
duplicate fallback entries for the old prompts) and fix split-fixtures.ts
to route the new strings to the "multimodal" feature bucket.
Local RED: ms-agent-python and crewai-crews both fail with fixture-miss
status=miss before this change.
Local GREEN: langgraph-typescript passes after this change (both turns
settle with "image" / "document" keywords confirmed in transcript).
Remaining failures after this fix are pre-existing Python backend issues
(ChatClientException on binary content parts in ms-agent-python; CrewAI
flow failure on binary content in crewai-crews) — unrelated to fixture
keys and tracked separately in the pydantic-ai multimodal work.
50 lines
2.3 KiB
JSON
50 lines
2.3 KiB
JSON
{
|
|
"_meta": {
|
|
"description": "D6 fixtures for langroid / multimodal",
|
|
"sourceScript": "d5-multimodal.ts",
|
|
"created": "2026-05-22"
|
|
},
|
|
"fixtures": [
|
|
{
|
|
"_comment": "Image sample turn \u2014 the D5 script clicks the sample-image button (which auto-sends). The assertion checks for 'image' keyword in the assistant transcript.",
|
|
"match": {
|
|
"userMessage": "can you tell me what is in this demo image I just attached",
|
|
"context": "langroid"
|
|
},
|
|
"response": {
|
|
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. Successful render of this response confirms the binary attachment round-tripped through the runtime."
|
|
}
|
|
},
|
|
{
|
|
"_comment": "PDF sample turn \u2014 the D5 script clicks the sample-pdf button (which auto-sends). The assertion checks for 'document' keyword in the assistant transcript.",
|
|
"match": {
|
|
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
|
"context": "langroid"
|
|
},
|
|
"response": {
|
|
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was flattened by pypdf on the Python side and forwarded as text content to the model. Receiving this response confirms the document upload path works."
|
|
}
|
|
},
|
|
{
|
|
"_comment": "Fallback for auto-sent image prompt \u2014 the sample button sends a canned user message that may differ from 'can you tell me what is in this demo image I just attached'. Match on 'image' substring to catch it.",
|
|
"match": {
|
|
"userMessage": "sample image",
|
|
"context": "langroid"
|
|
},
|
|
"response": {
|
|
"content": "The image shows a small test pattern \u2014 confirming the image upload pipeline round-tripped through the runtime successfully."
|
|
}
|
|
},
|
|
{
|
|
"_comment": "Fallback for auto-sent PDF prompt \u2014 match on 'sample' + 'pdf' or 'document' substring.",
|
|
"match": {
|
|
"userMessage": "sample pdf",
|
|
"context": "langroid"
|
|
},
|
|
"response": {
|
|
"content": "The document is a single-page PDF test fixture. The text was extracted and forwarded as content to the model, confirming the document upload path works end to end."
|
|
}
|
|
}
|
|
]
|
|
}
|