The langgraph-python multimodal-attachments demo had a stack of bugs
that compounded each other. Fixing them required touching the local
docker-compose, the aimock fixtures, the LangChain middleware, the
client-side AG-UI shim, and the sample-attachment buttons. This
commit lands the full set together because they only make sense as
a unit — verified end-to-end against `showcase up langgraph-python`
in a headed browser. New e2e suite pins each regression.
Supersedes #4584 (the original fix from May 1 that never landed —
this is a fresh port onto the post-refactor file layout where
page.tsx is split into legacy-converter-shim.tsx, multimodal-chat.tsx,
file-to-data-attachment.ts).
What was broken and what changed:
1. Random uploads crashed with `Failed to fetch`. aimock returned
HTTP 404 on no-match, the LangGraph SDK surfaced `NotFoundError`,
the AG-UI stream surfaced a `RUN_ERROR`, the demo crashed.
Added `--proxy-only` + `--provider-openai https://api.openai.com`
to the local aimock command so unmatched user prompts fall through
to real OpenAI (mirrors the Railway aimock setup).
2. Bundled-sample fixtures keyed on user-visible canned prompts.
The auto-prompts are deliberately long, specific, and natural-
reading ("can you tell me what is in this demo image/pdf I just
attached") so they (a) render cleanly as the user message bubble,
and (b) can't collide with arbitrary user prompts — random
uploads phrase questions differently and fall through to the
proxy.
3. Sample buttons now auto-send via `useAgent`. The previous
DataTransfer-based path queued the attachment via the chat's
hidden file input, then required clicking send while the
attachment was still uploading — `CopilotChat.onSubmitInput`
rejects submits during upload AND clears the input regardless,
so the canned prompt was eaten. Rewrite to call
`agent.addMessage(...)` + `copilotkit.runAgent({ agent })`
directly with the base64'd content part, sidestepping the
upload race entirely.
4. PDF flattened text bled into the rendered user message.
`_PdfFlattenMiddleware` ran in `before_model` and returned
`{"messages": rewritten}`, which persisted to agent state. The
chat UI then rendered the `[Attached document]\n<pdf body>` text
part inline with the user prompt. Switched to `wrap_model_call`
so the PDF→text rewrite is scoped to the outgoing model request
only and never pollutes state.
5. Attachments doubled (and PDFs rendered as broken `<img>`). The
`@ag-ui/langgraph` round-trip translates outgoing `binary` parts
to LangChain `image_url` and incoming `image_url` back to `image`
AG-UI parts — regardless of mimeType, so PDFs came back as
`type: "image"` with `mimeType: "application/pdf"` and were
forced into `ImageAttachment`, where the load failed and the
chat showed two "Failed to load image" boxes. Plus the user's
original modern part survived alongside the round-tripped one,
doubling visible chips.
Added a `dedupeUserMessageMedia` subscriber on both
`onMessagesSnapshotEvent` and `onRunFinalized` to:
- dedupe media parts by `source.value` so the local + round-
tripped copy collapse to one chip
- re-key part `type` from `mimeType` so PDFs route to
`DocumentAttachment` (icon + filename) and images to
`ImageAttachment`.
Also flipped the `onRunInitialized` shim from REPLACE to APPEND
— keep the modern part for the UI AND emit a legacy `binary`
sibling for the converter.
6. Regression suite (`tests/e2e/multimodal.spec.ts`). Replaces the
pre-rewrite suite with five focused tests:
- page loads with all expected affordances
- sample image: auto-sends, EXACTLY ONE `<img>`, assistant
references the logo
- sample PDF: auto-sends, EXACTLY ONE `DocumentAttachment` chip
("PDF" label), NO `<img>`, no `[Attached document]` text bleed
- image then PDF in the same session: each message keeps its own
single chip, no cross-contamination
- PDF then image in the same session: symmetric
All 5 pass against the live local stack (15.4s).
The branch's iteration arc went through several headless-demo
implementations (CSS Modules, AI Elements, prompt-kit, shadcn) and a
streaming-markdown experiment with `streamdown`. After the spec
narrowed to shadcn-only and `react-markdown` for assistant rendering,
those component libraries and their satellite packages were left
installed but unused.
Removed (zero importers across `src/`):
- @copilotkit/react-ui (v1 UI; the showcase consumes /v2 hooks only)
- @rive-app/react-webgl2 (animations, never wired)
- @streamdown/cjk, /code, /math, /mermaid + streamdown (replaced by
react-markdown for assistant content)
- @xyflow/react (graph viz, never wired)
- ai (Vercel AI SDK, was the AI Elements demo's transport)
- ansi-to-react (terminal output, never wired)
- marked (alternative markdown parser, never wired)
- media-chrome (media player UI, never wired)
- motion (animation library, was a prompt-kit demo dep)
- nanoid (we use crypto.randomUUID())
- react-jsx-parser (never wired)
- remark-breaks (markdown extension, never wired)
- shiki (syntax highlighting, never wired)
- tokenlens (never wired)
- use-stick-to-bottom (was an AI Elements dep)
- @radix-ui/react-use-controllable-state (no importers)
Kept everything actually imported: shadcn primitive surface intact
(class-variance-authority, cmdk, embla-carousel-react, lucide-react,
radix-ui umbrella + per-package @radix-ui/react-checkbox /
react-separator for beautiful-chat's local components), CopilotKit v2,
Hashbrown + json-render BYOC catalogs, recharts, react-markdown +
remark-gfm, openai (voice route), yaml (manifest parsing), zod.
Build still green at 54 routes. Lockfile updated. tsconfig.json
`jsx: "preserve"` — Next 15 reset this from `react-jsx` automatically
on build.
Foundational layer that the per-demo work in subsequent commits builds on.
- Manifest-driven landing page (src/app/page.tsx) — auto-generated grid
of demo cards from manifest.yaml, grouped by tag with explicit ordering
for chat-ui / interactivity / generative-ui / agent-state / multi-agent
/ headless / platform.
- Per-demo titles (src/middleware.ts + src/app/demos/layout.tsx).
- Diagnostic console gated to NODE_ENV=production in app/layout.tsx so
deployed showcases surface uncaught errors and iframe context.
- src/app/globals.css — Tailwind v4 @theme inline block that maps
showcase CSS variables into Tailwind theme tokens (without it, shadcn
utilities like bg-muted / text-foreground compile to nothing). body
intentionally NOT given width:100% so <CopilotSidebar /> can shrink
the document via marginInlineEnd.
- shadcn primitives under src/components/ui/ + src/lib/utils.ts.
- tsconfig.json — @/* path alias rooted at src/.
Removed:
- src/app/copilotkit-overrides.css (global override layer, superseded
by per-demo theming)
- src/app/api/copilotkit-mcp-apps/route.ts (replaced with
[[...slug]]/route.ts so v2 subpath POSTs like /v2/agent/run resolve)
Recent feature commits added new dependencies to integration package.json
files (@copilotkit/voice, @hashbrownai/{core,react}, @json-render/{core,react})
and bumped Next.js from 15.4.10 to 15.5.15, but never regenerated the
corresponding package-lock.json. The Showcase Build & Deploy workflow runs
`npm ci --legacy-peer-deps` which strictly enforces lock sync, so every
deploy attempt has been failing at the install step. No new images have been
pushed to GHCR, so Railway services have stayed on stale code and any cell
added since each fw's last successful deploy iframes 404.
Regenerated all 18 lockfiles via `npm install --legacy-peer-deps
--package-lock-only --ignore-scripts` per integration. Verified each with
`npm ci --dry-run --legacy-peer-deps` — all clean.
Refs PDX-90.
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
Alem Tuzlak