mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
4b11b995a2
## Summary
Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.
### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)
### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts
### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section
### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)
### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51
### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`
## Test plan
- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
CopilotKit - React Core
✨ Why CopilotKit?
- Minutes to integrate - Get started quickly with our CLI
- Framework agnostic - Works with React, Next.js, AGUI and more
- Production-ready UI - Use customizable components or build with headless UI
- Built-in security - Prompt injection protection
- Open source - Full transparency and community-driven
🧑💻 Real life use cases
Deploy deeply-integrated AI assistants & agents that work alongside your users inside your applications.
🖥️ Code Samples
Drop in these building blocks and tailor them to your needs.
Build with Headless APIs and Pre-Built Components
// Headless UI with full control
const { visibleMessages, appendMessage, setMessages, ... } = useCopilotChat();
// Pre-built components with deep customization options (CSS + pass custom sub-components)
<CopilotPopup
instructions={"You are assisting the user as best as you can. Answer in the best way possible given the data you have."}
labels={{ title: "Popup Assistant", initial: "Need any help?" }}
/>
// Frontend actions + generative UI, with full streaming support
useCopilotAction({
name: "appendToSpreadsheet",
description: "Append rows to the current spreadsheet",
parameters: [
{ name: "rows", type: "object[]", attributes: [{ name: "cells", type: "object[]", attributes: [{ name: "value", type: "string" }] }] }
],
render: ({ status, args }) => <Spreadsheet data={canonicalSpreadsheetData(args.rows)} />,
handler: ({ rows }) => setSpreadsheet({ ...spreadsheet, rows: [...spreadsheet.rows, ...canonicalSpreadsheetData(rows)] }),
});
Integrate In-App CoAgents with LangGraph
// Share state between app and agent
const { agentState } = useCoAgent({
name: "basic_agent",
initialState: { input: "NYC" }
});
// agentic generative UI
useCoAgentStateRender({
name: "basic_agent",
render: ({ state }) => <WeatherDisplay {...state.final_response} />,
});
// Human in the Loop (Approval)
useCopilotAction({
name: "email_tool",
parameters: [
{
name: "email_draft",
type: "string",
description: "The email content",
required: true,
},
],
renderAndWaitForResponse: ({ args, status, respond }) => {
return (
<EmailConfirmation
emailContent={args.email_draft || ""}
isExecuting={status === "executing"}
onCancel={() => respond?.({ approved: false })}
onSend={() =>
respond?.({
approved: true,
metadata: { sentAt: new Date().toISOString() },
})
}
/>
);
},
});
// intermediate agent state streaming (supports both LangGraph.js + LangGraph python)
const modifiedConfig = copilotKitCustomizeConfig(config, {
emitIntermediateState: [
{
stateKey: "outline",
tool: "set_outline",
toolArgument: "outline",
},
],
});
const response = await ChatOpenAI({ model: "gpt-4o" }).invoke(
messages,
modifiedConfig,
);
🏆 Featured Examples
Documentation
To get started with CopilotKit, please check out the documentation.