Commit Graph

18 Commits

Author SHA1 Message Date
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Alem Tuzlak 773a4a6415 fix: format package.json files 2026-04-07 12:29:29 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00
Alem Tuzlak 136bfc1182 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-07 09:51:46 +02:00
Benjamin Taylor da2230ab32 feat(runtime): Add Segment tracking into v2 2026-04-06 16:19:52 -07:00
Alem Tuzlak 7de4bfa9cf chore: bump @ag-ui/core and @ag-ui/client to 0.0.50-alpha.0 2026-04-06 14:54:27 +02:00
Tyler Slaton 9c940dcaac Merge branch 'main' into claude/rebase-against-main-PGEp1 2026-04-05 21:46:05 -07:00
Maxim 208f9bc32f fix: restore .d.ts declaration files for legacy moduleResolution consumers
Copies .d.cts → .d.ts after each tsdown build so consumers on
moduleResolution "node" (e.g. DocuSign) can resolve types without
changing their tsconfig. Also points the "types" field back to .d.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:55:45 +02:00
Alem Tuzlak 9a6ee39d96 chore(runtime): update package config, CI workflows, and docs
- Add package.json exports for v2/{express,hono,node} subpaths
- Add elysia devDependency and tsdown entry points
- Exclude bun integration tests from vitest config
- Update CI workflows to include runtime-servers test job
- Add runtime-server-adapter docs page
- Add changeset for the fetch-based runtime feature
2026-04-03 18:41:27 +02:00
Tyler Slaton 4f1f132d41 fix: address issues with licensing, formatting and types (#3597) 2026-04-03 09:38:41 -07:00
Ran Shem Tov aae2c5cee1 chore: use latest ag-ui langgraph version 2026-04-03 14:57:45 +02:00
Tyler Slaton 9dc688be2e fix: stop barrel-exporting license-verifier through shared to avoid bundling Node crypto into client
@copilotkit/shared was doing `export * from "@copilotkit/license-verifier"`,
which pulled the entire license-verifier bundle (including Node's `crypto`)
into any client-side consumer. This caused "Module not found: Can't resolve
'crypto'" errors for users bundling @copilotkit/react-core for the browser.

Now shared only re-exports types (erased at compile time) and a client-safe
`createLicenseContextValue` inlined without crypto. Runtime imports
`createLicenseChecker` and `getLicenseWarningHeader` directly from
`@copilotkit/license-verifier`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:55:58 -07:00
Alem Tuzlak 401a27c981 fix: resolve rebase conflicts and format files
- Remove stale eslint-config-custom refs from package.json files
- Regenerate pnpm-lock.yaml
- Format agentcore and other files with oxfmt
2026-04-02 16:43:34 +02:00
Claude 0d9e3b88f6 chore(post-release): update version to 1.55.0-next.8
The previous publish workflow released 1.55.0-next.8 to npm but the
post-release commit failed to push to main (blocked by branch
protection). This applies the version bumps and pre.json updates that
the workflow would have committed, unblocking future releases.

https://claude.ai/code/session_01N8sLK9h9p6FdRjyHqzKzVV
2026-04-02 02:23:27 +00:00
github-actions[bot] a1edf59eae chore(post-release): update version to 1.55.0-next.7 2026-03-29 00:19:22 +00:00
Claude 1ceb963a84 Revert "fix: resolve changeset version errors from stale @copilotkitnext package refs"
This reverts commit 1f6ae0d629.
2026-03-28 17:04:55 -07:00
Claude 369266f1be fix: resolve changeset version errors from stale @copilotkitnext package refs
After the V1/V2 consolidation merge, several changesets still referenced
old @copilotkitnext/* package names that no longer exist. Fixed package
references and ran changeset version to apply pending version bumps.

https://claude.ai/code/session_01A5aahBWFTmLPR3xZSeh4SD
2026-03-28 17:04:55 -07:00
Tyler Slaton 96885b5959 refactor: consolidate V1/V2 packages into flat @copilotkit/* structure
Flatten all packages from packages/v1/* and packages/v2/* into packages/* —
every package now lives directly under the @copilotkit/ scope with no v1/v2
subdirectories.

- Move all v1 packages (react-core, react-ui, runtime, shared, etc.) from
  packages/v1/* to packages/*
- Absorb v2 react code into packages/react-core/src/v2/ (exported via /v2 subpath)
- Absorb v2 agent code into packages/runtime/src/agent/ (exported via /v2 subpath)
- Move v2 packages (core, angular, demo-agents, etc.) to packages/*
- Replace all @copilotkitnext/* imports with @copilotkit/* equivalents
- Keep @copilotkitnext/angular as the sole exception (angular remains on next)
- Update CI workflows, renovate config, release scripts for flat structure
- No public API surface changes — all exports fields are preserved

Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com>
Signed-off-by: Tyler Slaton <tyler@copilotkit.ai>
2026-03-28 16:45:10 -07:00