The langgraph-python multimodal-attachments demo had a stack of bugs
that compounded each other. Fixing them required touching the local
docker-compose, the aimock fixtures, the LangChain middleware, the
client-side AG-UI shim, and the sample-attachment buttons. This
commit lands the full set together because they only make sense as
a unit — verified end-to-end against `showcase up langgraph-python`
in a headed browser. New e2e suite pins each regression.
Supersedes #4584 (the original fix from May 1 that never landed —
this is a fresh port onto the post-refactor file layout where
page.tsx is split into legacy-converter-shim.tsx, multimodal-chat.tsx,
file-to-data-attachment.ts).
What was broken and what changed:
1. Random uploads crashed with `Failed to fetch`. aimock returned
HTTP 404 on no-match, the LangGraph SDK surfaced `NotFoundError`,
the AG-UI stream surfaced a `RUN_ERROR`, the demo crashed.
Added `--proxy-only` + `--provider-openai https://api.openai.com`
to the local aimock command so unmatched user prompts fall through
to real OpenAI (mirrors the Railway aimock setup).
2. Bundled-sample fixtures keyed on user-visible canned prompts.
The auto-prompts are deliberately long, specific, and natural-
reading ("can you tell me what is in this demo image/pdf I just
attached") so they (a) render cleanly as the user message bubble,
and (b) can't collide with arbitrary user prompts — random
uploads phrase questions differently and fall through to the
proxy.
3. Sample buttons now auto-send via `useAgent`. The previous
DataTransfer-based path queued the attachment via the chat's
hidden file input, then required clicking send while the
attachment was still uploading — `CopilotChat.onSubmitInput`
rejects submits during upload AND clears the input regardless,
so the canned prompt was eaten. Rewrite to call
`agent.addMessage(...)` + `copilotkit.runAgent({ agent })`
directly with the base64'd content part, sidestepping the
upload race entirely.
4. PDF flattened text bled into the rendered user message.
`_PdfFlattenMiddleware` ran in `before_model` and returned
`{"messages": rewritten}`, which persisted to agent state. The
chat UI then rendered the `[Attached document]\n<pdf body>` text
part inline with the user prompt. Switched to `wrap_model_call`
so the PDF→text rewrite is scoped to the outgoing model request
only and never pollutes state.
5. Attachments doubled (and PDFs rendered as broken `<img>`). The
`@ag-ui/langgraph` round-trip translates outgoing `binary` parts
to LangChain `image_url` and incoming `image_url` back to `image`
AG-UI parts — regardless of mimeType, so PDFs came back as
`type: "image"` with `mimeType: "application/pdf"` and were
forced into `ImageAttachment`, where the load failed and the
chat showed two "Failed to load image" boxes. Plus the user's
original modern part survived alongside the round-tripped one,
doubling visible chips.
Added a `dedupeUserMessageMedia` subscriber on both
`onMessagesSnapshotEvent` and `onRunFinalized` to:
- dedupe media parts by `source.value` so the local + round-
tripped copy collapse to one chip
- re-key part `type` from `mimeType` so PDFs route to
`DocumentAttachment` (icon + filename) and images to
`ImageAttachment`.
Also flipped the `onRunInitialized` shim from REPLACE to APPEND
— keep the modern part for the UI AND emit a legacy `binary`
sibling for the converter.
6. Regression suite (`tests/e2e/multimodal.spec.ts`). Replaces the
pre-rewrite suite with five focused tests:
- page loads with all expected affordances
- sample image: auto-sends, EXACTLY ONE `<img>`, assistant
references the logo
- sample PDF: auto-sends, EXACTLY ONE `DocumentAttachment` chip
("PDF" label), NO `<img>`, no `[Attached document]` text bleed
- image then PDF in the same session: each message keeps its own
single chip, no cross-contamination
- PDF then image in the same session: symmetric
All 5 pass against the live local stack (15.4s).
The specs and QA markdowns had drifted from the demos they describe.
This commit brings every test contract into line with the actual demo
source — eliminating false-greens, false-fails, and stale assertions.
False-fail spec assertions (would fail every run):
- `agentic-chat.spec.ts` — rewrote from the old `change_background` /
`weather-card` / `useAgentContext` flow that no longer exists. New
spec exercises the vanilla `<CopilotChat>` + three suggestion pills
contract the simplified demo actually exposes.
- `gen-ui-tool-based.spec.ts` — asserted on UI text ("Use the sidebar
to generate charts", "Chart Generator") that doesn't exist; switched
to suggestion-pill assertions and scoped the SVG check to inside the
assistant-message bubble (was matching CopilotChat's send-button
SVG).
- `agent-config.spec.ts` — asserted heading "Agent Config Object" but
the demo has "Agent Config".
- `multimodal.spec.ts` — asserted a non-existent "Multimodal
attachments" heading; switched to the `multimodal-demo-root` testid.
- `chat-slots.spec.ts` — asserted `[data-testid="custom-assistant-
message"]` and the bare text "slot" — neither exists. The actual
signal is `data-slot-label="MessageView.AssistantMessage"` from the
SlotMarker wrapper.
- `reasoning-default.spec.ts` — asserted `[data-testid="copilot-
reasoning-message"]` and `[data-message-role="reasoning"]`; neither
is emitted by `CopilotChatReasoningMessage`. Switched to the text-
based "Thinking…/Thought for…" header label.
False-green spec assertions (passed for the wrong reason):
- `shared-state-read.spec.ts` — was a complete false-green: asserted
on "Sales Pipeline", "Total Pipeline", "Active Deals" but the demo
has been a Recipe Editor for some time. Rewrote against the
recipe-card / ingredients-container / instructions-container testids.
- 11 specs (agent-config, beautiful-chat, frontend-tools-async,
gen-ui-tool-based, gen-ui-agent, gen-ui-interrupt, hitl-in-chat,
hitl-in-app, multimodal, readonly-state-agent-context, voice) used
`[data-role="assistant"]` to gate "agent responded" — but the v2
react-core bundle never emits that attribute (it ships
`data-testid="copilot-assistant-message"`). Mechanical sweep to the
correct testid.
- Deleted `shared-state-write.spec.ts` (route consolidated into
`shared-state-read-write` earlier on this branch — spec targeted a
removed demo) and `renderer-selector.spec.ts` (asserted on a radio-
pill UI that no longer exists; the four "Declarative UI" variants
are now separate manifest demos).
QA drift:
- `qa/gen-ui-tool-based.md` documented a "Haiku Generator" demo with
haiku-card / japanese-line / english-line / haiku-image testids — a
demo that doesn't exist anywhere on this branch. Rewrote to match
the chart-rendering demo's actual testids and pill prompts.
- `qa/chat-slots.md` referenced "Custom Slot" pill / "Welcome to the
Slots demo" heading / "This welcome card is rendered via the
welcomeScreen slot." body text — all of which the slot-wrappers
refactor on this branch removed. Updated to match the
`custom-welcome-message` sub-slot that's actually rendered. Also
fixed max-w-4xl → max-w-5xl to match the page.
- `qa/shared-state-read.md` said default instruction is "Preheat oven
to 350 F" but the source has "Preheat oven to 350°F (175°C)".
- `qa/agentic-chat.md` rewrote to match the simplified vanilla-chat
demo (the previous QA documented `change_background` / `WeatherCard`
flows that no longer exist).
- `qa/reasoning-default.md` cited `kind: "testing"` in feature-
registry.json for the `reasoning-default` entry; the registry entry
has no `kind` field. Rewrote without the false cross-file claim.
- Deleted 4 orphan QA files for demos that don't exist:
`agentic-chat-reasoning.md`, `hitl.md`, `hitl-in-chat-booking.md`,
`shared-state-write.md`.
- Renamed `qa/reasoning-default-render.md` → `qa/reasoning-default.md`
to match the manifest cell name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
github-actions[bot]