Files
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
..

CopilotKit - React Core

banner

Why CopilotKit?

  • Minutes to integrate - Get started quickly with our CLI
  • Framework agnostic - Works with React, Next.js, AGUI and more
  • Production-ready UI - Use customizable components or build with headless UI
  • Built-in security - Prompt injection protection
  • Open source - Full transparency and community-driven
class-support-ecosystem

🧑‍💻 Real life use cases

Deploy deeply-integrated AI assistants & agents that work alongside your users inside your applications.

headless-ui

🖥️ Code Samples

Drop in these building blocks and tailor them to your needs.

Build with Headless APIs and Pre-Built Components

// Headless UI with full control
const { visibleMessages, appendMessage, setMessages, ... } = useCopilotChat();

// Pre-built components with deep customization options (CSS + pass custom sub-components)
<CopilotPopup
  instructions={"You are assisting the user as best as you can. Answer in the best way possible given the data you have."}
  labels={{ title: "Popup Assistant", initial: "Need any help?" }}
/>
// Frontend actions + generative UI, with full streaming support
useCopilotAction({
  name: "appendToSpreadsheet",
  description: "Append rows to the current spreadsheet",
  parameters: [
    { name: "rows", type: "object[]", attributes: [{ name: "cells", type: "object[]", attributes: [{ name: "value", type: "string" }] }] }
  ],
  render: ({ status, args }) => <Spreadsheet data={canonicalSpreadsheetData(args.rows)} />,
  handler: ({ rows }) => setSpreadsheet({ ...spreadsheet, rows: [...spreadsheet.rows, ...canonicalSpreadsheetData(rows)] }),
});

Integrate In-App CoAgents with LangGraph

// Share state between app and agent
const { agentState } = useCoAgent({
  name: "basic_agent",
  initialState: { input: "NYC" }
});

// agentic generative UI
useCoAgentStateRender({
  name: "basic_agent",
  render: ({ state }) => <WeatherDisplay {...state.final_response} />,
});

// Human in the Loop (Approval)
useCopilotAction({
  name: "email_tool",
  parameters: [
    {
      name: "email_draft",
      type: "string",
      description: "The email content",
      required: true,
    },
  ],
  renderAndWaitForResponse: ({ args, status, respond }) => {
    return (
      <EmailConfirmation
        emailContent={args.email_draft || ""}
        isExecuting={status === "executing"}
        onCancel={() => respond?.({ approved: false })}
        onSend={() =>
          respond?.({
            approved: true,
            metadata: { sentAt: new Date().toISOString() },
          })
        }
      />
    );
  },
});
// intermediate agent state streaming (supports both LangGraph.js + LangGraph python)
const modifiedConfig = copilotKitCustomizeConfig(config, {
  emitIntermediateState: [
    {
      stateKey: "outline",
      tool: "set_outline",
      toolArgument: "outline",
    },
  ],
});
const response = await ChatOpenAI({ model: "gpt-4o" }).invoke(
  messages,
  modifiedConfig,
);

Documentation

To get started with CopilotKit, please check out the documentation.