Commit Graph

50 Commits

Author SHA1 Message Date
github-actions[bot] 1bc4786759 chore: version packages (next) 2026-04-09 18:09:30 +00:00
github-actions[bot] 0093bdbaa4 chore: version packages 2026-04-09 02:26:42 +00:00
github-actions[bot] c2837a3001 chore: version packages (next) 2026-04-09 01:53:20 +00:00
github-actions[bot] 00cace6abf chore: version packages 2026-04-08 23:54:54 +00:00
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00
Alem Tuzlak 9ae7a97823 fix: address PR review feedback from Martha
Bugs fixed:
- v1 Chat.tsx: onUpload now reads result.type/result.value (was result.data/result.url)
- v1 Chat.tsx: metadata captured from onUpload and forwarded in InputContent parts
- v1 Chat.tsx: onUploadFailed callback now invoked at all three error paths
- useAttachments: consumeAttachments preserves in-flight uploads (was dropping them)

Should-fix addressed:
- Lightbox: vtCounter replaced with React.useId() (concurrent mode + SSR safe)
- Lightbox: SSR guard before createPortal(document.body)
- useBlobUrl: useEffect deps changed to scalar values (was object reference)
- Codemod: dynamic inputFileAccept expressions preserved (was silently dropped)
- Codemod: scope-awareness limitation documented in comment

Tests: 966/966 react-core, 19/19 codemod
2026-04-07 09:42:28 +02:00
Alem Tuzlak d7578216d3 refactor: extract useAttachments hook, memoize attachment renderers
- Extract all attachment logic from CopilotChat into reusable useAttachments hook
  (state, processFiles, drag/drop, paste, consumeAttachments, removeAttachment)
- Export useAttachments from hooks/index.ts for custom chat UI builders
- Memoize v2 CopilotChatAttachmentRenderer sub-components
  (ImageAttachment, AudioAttachment, VideoAttachment, DocumentAttachment)
- Memoize v1 AttachmentRenderer sub-components to match v2 structure,
  align props (content → filename)
- Guard process.env in attachment-utils.ts for browser environments
- Remove debug logging (existing.debug = true) from use-agent.tsx
2026-04-06 15:17:56 +02:00
Alem Tuzlak 0fd3ddf960 refactor(docs): create migration-guides section, move migration pages from troubleshooting
- New docs/(root)/migration-guides/ section with migrate-to-v2, migrate-attachments,
  migrate-to-1.10.X, migrate-to-1.8.2
- Troubleshooting now only contains error-debugging, observability-connectors, common-issues
- Update all @deprecated JSDoc URLs from /troubleshooting/ to /migration-guides/
- Update doc cross-links (multimodal-attachments page, snippet import)
- Move snippet to snippets/shared/migration-guides/
- Set minimum version to v1.56.0 (was vX.Y.Z placeholder)
2026-04-06 14:55:09 +02:00
Alem Tuzlak 89d39d639b fix: address code review findings — error handling, codemod correctness, deprecation versions
- Restore console.error as safety net in processFiles catch blocks (errors no longer
  silently vanish when onUploadFailed callback is not provided)
- Add try/catch to useBlobUrl atob() — gracefully handles malformed base64 instead of
  crashing the component tree
- Fix DocumentLightboxContent: use blobUrl instead of undefined src variable
- Add .catch() to View Transition API transition.finished promise
- Codemod: preserve actual imageUploadsEnabled value (false, dynamic expressions)
  instead of hardcoding true — adds 3 new test cases (18 total)
- Replace all placeholder @deprecated versions (v1.x.0) with @since 1.56.0
- Add missing @since to ImageRenderer, ImageRendererProps, AIMessage.image tags
2026-04-06 14:55:08 +02:00
Alem Tuzlak 18c7a6001d feat: multimodal attachments — UI polish, deprecations, docs, codemod
Attachment queue & previews:
- Image lightbox with View Transition API morph animation
- Video lightbox with native controls and play button overlay
- Document lightbox (PDF via blob URL, text inline, info card fallback)
- Drop zone overlay with upload icon
- Filename preservation via InputContent metadata
- Proper video thumbnail sizing and play/pause indicator
- Fix attachment queue positioning (max-w-3xl constraint)
- Padding between X button and content for audio/document cards
- Document filenames wrap instead of truncating

Attachments config:
- onUploadFailed callback for validation/upload errors (file-too-large, invalid-type, upload-failed)
- onUpload accepts sync or async returns
- AttachmentUploadResult discriminated union with explicit interfaces
- Metadata field on Attachment and onUpload return type

AG-UI version bump:
- Bump @ag-ui/client, @ag-ui/core, @ag-ui/encoder, @ag-ui/proto to 0.0.51
- Remove process.env Vite workaround (fixed upstream in 0.0.51)

Deprecation lifecycle:
- @deprecated JSDoc on all legacy image upload APIs
- ImageRenderer, ImageRendererProps, ImageUpload type, imageUploadsEnabled prop,
  inputFileAccept prop, ImageRenderer prop, AIMessage.image, ImageData
- Codemod at codemods/migrate-attachments.ts (15 tests)
- Migration guide updated with codemod instructions and new type shapes

Docs:
- New guide: docs/(root)/multimodal-attachments.mdx
- Updated migration guide with onUpload return type, metadata, codemod section
- Cross-links from prebuilt-components and migration guide
- Label change: "Add photos or files" → "Add attachments"

Tests:
- CopilotChat.attachments.test.tsx — 5 tests for onUploadFailed
- migrate-attachments codemod — 15 tests
2026-04-06 14:55:06 +02:00
Alem Tuzlak f6c15cdbfe refactor(shared): move attachment utilities to shared package 2026-04-06 14:54:42 +02:00
Alem Tuzlak 7c774e9eb6 refactor(shared): move attachment types to shared package 2026-04-06 14:54:41 +02:00
Alem Tuzlak 152d1ade96 style: format code with oxfmt 2026-04-06 14:54:40 +02:00
Alem Tuzlak 928c385516 fix(react-ui): add baseline error logging, thumbnail diagnostics, rejected file feedback, mimeType fallback 2026-04-06 14:54:39 +02:00
Alem Tuzlak 2c546e44b5 test(react-ui): add tests for attachment utility functions 2026-04-06 14:54:37 +02:00
Alem Tuzlak e683f9386c fix(react-ui): address code review findings — stale closures, error handling, type safety
- Fix stale closure in paste handler by using processFilesRef pattern
- Wrap processFiles calls in try/catch across paste, file upload, and drop handlers
- Add early return for empty/falsy accept string in matchesAcceptFilter
- Block send while attachments are still uploading
- Add id field to Attachment interface, replace _id hack with randomUUID()
- Add 10s timeout to generateVideoThumbnail to prevent infinite hanging
- Show placeholder div for uploading attachments instead of rendering incomplete source
- Include filename context in upload error messages
2026-04-06 14:54:36 +02:00
Alem Tuzlak 1d62da913d chore: add deprecation annotations and suppressable runtime warnings 2026-04-06 14:54:34 +02:00
Alem Tuzlak fc1cfa09d9 feat(react-ui): add attachments config, upload flow, drag-and-drop, and deprecation bridge 2026-04-06 14:54:33 +02:00
Alem Tuzlak ff236cfbe6 feat(react-ui): add AttachmentRenderer and update UserMessage for multimodal content 2026-04-06 14:54:32 +02:00
Alem Tuzlak c472f452f6 feat(react-ui): add AttachmentQueue component and styles 2026-04-06 14:54:31 +02:00
Alem Tuzlak 8bdb24ea87 feat(react-ui): add attachment types and utility functions 2026-04-06 14:54:30 +02:00
Maxim 208f9bc32f fix: restore .d.ts declaration files for legacy moduleResolution consumers
Copies .d.cts → .d.ts after each tsdown build so consumers on
moduleResolution "node" (e.g. DocuSign) can resolve types without
changing their tsconfig. Also points the "types" field back to .d.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:55:45 +02:00
Tyler Slaton 180b719d50 Revert "fix: add types conditions to package exports for proper TypeScript resolution"
This reverts commit 499bca09ad.
2026-04-03 09:42:01 -07:00
Tyler Slaton 499bca09ad fix: add types conditions to package exports for proper TypeScript resolution
All @copilotkit/* packages were missing type declarations in their exports
map. This meant TypeScript couldn't resolve types for subpath imports like
@copilotkit/react-core/v2 without manual tsconfig paths workarounds
pointing to dist .d.ts files (which no longer exist after the tsup to
tsdown migration).

Uses nested conditional exports to map .d.mts for ESM and .d.cts for CJS
consumers, satisfying both attw and TypeScript module resolution under
nodenext and bundler modes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:56:09 -07:00
Tyler Slaton 596b5cd1b5 Merge branch 'main' into fix/261-sidebar-full-height 2026-04-02 09:27:32 -07:00
Alem Tuzlak 401a27c981 fix: resolve rebase conflicts and format files
- Remove stale eslint-config-custom refs from package.json files
- Regenerate pnpm-lock.yaml
- Format agentcore and other files with oxfmt
2026-04-02 16:43:34 +02:00
Alem Tuzlak 79ce60c580 chore: migrate from eslint+prettier to oxlint+oxfmt
Replace eslint and prettier with oxlint and oxfmt for faster linting
and formatting across the monorepo. Remove all eslint and prettier
configs, dependencies, and related packages. Add .oxlintrc.json and
.oxfmtrc.json for the new tooling. Update CI workflows and lefthook
hooks accordingly. Reformat codebase with oxfmt.

https://claude.ai/code/session_01GMkSf29p78HuMR1mbXn8He
2026-04-02 16:39:05 +02:00
Claude 0d9e3b88f6 chore(post-release): update version to 1.55.0-next.8
The previous publish workflow released 1.55.0-next.8 to npm but the
post-release commit failed to push to main (blocked by branch
protection). This applies the version bumps and pre.json updates that
the workflow would have committed, unblocking future releases.

https://claude.ai/code/session_01N8sLK9h9p6FdRjyHqzKzVV
2026-04-02 02:23:27 +00:00
alex mitre 818e4e767b fix(sidebar): allow full-height children inside CopilotSidebar 2026-04-01 08:28:13 -07:00
github-actions[bot] a1edf59eae chore(post-release): update version to 1.55.0-next.7 2026-03-29 00:19:22 +00:00
Claude 1ceb963a84 Revert "fix: resolve changeset version errors from stale @copilotkitnext package refs"
This reverts commit 1f6ae0d629.
2026-03-28 17:04:55 -07:00
Claude 369266f1be fix: resolve changeset version errors from stale @copilotkitnext package refs
After the V1/V2 consolidation merge, several changesets still referenced
old @copilotkitnext/* package names that no longer exist. Fixed package
references and ran changeset version to apply pending version bumps.

https://claude.ai/code/session_01A5aahBWFTmLPR3xZSeh4SD
2026-03-28 17:04:55 -07:00
Tyler Slaton 96885b5959 refactor: consolidate V1/V2 packages into flat @copilotkit/* structure
Flatten all packages from packages/v1/* and packages/v2/* into packages/* —
every package now lives directly under the @copilotkit/ scope with no v1/v2
subdirectories.

- Move all v1 packages (react-core, react-ui, runtime, shared, etc.) from
  packages/v1/* to packages/*
- Absorb v2 react code into packages/react-core/src/v2/ (exported via /v2 subpath)
- Absorb v2 agent code into packages/runtime/src/agent/ (exported via /v2 subpath)
- Move v2 packages (core, angular, demo-agents, etc.) to packages/*
- Replace all @copilotkitnext/* imports with @copilotkit/* equivalents
- Keep @copilotkitnext/angular as the sole exception (angular remains on next)
- Update CI workflows, renovate config, release scripts for flat structure
- No public API surface changes — all exports fields are preserved

Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com>
Signed-off-by: Tyler Slaton <tyler@copilotkit.ai>
2026-03-28 16:45:10 -07:00
Atai Barkai 5f3727dde8 move to CopilotKit subdirectory
.
2023-08-22 23:09:55 -07:00
Atai Barkai 4010c99c7e split react-textarea into react-ui component 2023-08-22 22:50:57 -07:00
Atai Barkai 3debf56a56 tailwind —> tailwindcss 2023-08-20 03:52:17 -07:00
Atai Barkai 4211ae9953 tailwind: dependency —> dev dependency 2023-08-20 03:52:17 -07:00
Atai Barkai 4f19bd7bf8 publish packages 2023-07-18 23:32:25 -07:00
Atai Barkai 1befcaf145 upgrade ai package 2023-07-18 12:55:29 -07:00
Atai Barkai e4faea4adf breakpoint debugging potentially works better? 2023-07-17 12:43:04 -07:00
Atai Barkai 168adf236c debugging problem seems ACTUALLY fixed this time. fingers crossed 2023-07-17 12:43:04 -07:00
Atai Barkai 74261cd73b wip 2023-07-17 12:43:04 -07:00
Atai Barkai a4b3cc44d6 breakpoints in libs actually working
.

.
2023-07-17 12:43:04 -07:00
Atai Barkai 1f1c385464 add sourcemap to tsconfig and to tsup config 2023-07-15 08:13:58 -07:00
Atai Barkai 0c5e0512a5 minor copilot cosmetic change 2023-07-15 03:39:21 -07:00
Atai Barkai 40af06bb6b minor copilot optimizations 2023-07-15 03:26:17 -07:00
Atai Barkai 4e245a5a10 wip copilot integration 2023-07-14 17:04:28 -07:00
Atai Barkai 44c559061c move tooltip provider inside copilot sidebar ui provider 2023-07-14 06:28:12 -07:00
Atai Barkai 03164e91c7 v0.2 turbo refactor 2023-07-13 16:08:04 -07:00