Commit Graph

1601 Commits

Author SHA1 Message Date
Alem Tuzlak 136bfc1182 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-07 09:51:46 +02:00
Alem Tuzlak 9ae7a97823 fix: address PR review feedback from Martha
Bugs fixed:
- v1 Chat.tsx: onUpload now reads result.type/result.value (was result.data/result.url)
- v1 Chat.tsx: metadata captured from onUpload and forwarded in InputContent parts
- v1 Chat.tsx: onUploadFailed callback now invoked at all three error paths
- useAttachments: consumeAttachments preserves in-flight uploads (was dropping them)

Should-fix addressed:
- Lightbox: vtCounter replaced with React.useId() (concurrent mode + SSR safe)
- Lightbox: SSR guard before createPortal(document.body)
- useBlobUrl: useEffect deps changed to scalar values (was object reference)
- Codemod: dynamic inputFileAccept expressions preserved (was silently dropped)
- Codemod: scope-awareness limitation documented in comment

Tests: 966/966 react-core, 19/19 codemod
2026-04-07 09:42:28 +02:00
Benjamin Taylor 4094f30989 feat(runtime): Support configuring Threads lock TTL, prefix and heartbeat
feat(runtime): add defaults and max caps for thread lock TTL and heartbeat

lockTtlSeconds defaults to 20s (max 3600s / 1 hour) and
lockHeartbeatIntervalSeconds defaults to 15s (max 3000s / 50 minutes).
Both values are always set, so the heartbeat timer now starts
unconditionally and the paired-option validation is removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 17:17:55 -07:00
Benjamin Taylor 3dc6963956 fix(threads): Improve useThreads DX with clearer names and direct type hints 2026-04-06 16:50:46 -07:00
Benjamin Taylor 9413015aa6 fix(react-core): clear copy button timeout on unmount
The CopyButton setTimeout fires after the jsdom test environment is
torn down, causing an unhandled ReferenceError on Node 20. Track the
timer in a ref and clear it on cleanup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:19:52 -07:00
Benjamin Taylor da2230ab32 feat(runtime): Add Segment tracking into v2 2026-04-06 16:19:52 -07:00
Martha Schumann ccfff2d84b refactor(react-core): extract deduplicateMessages, add useMemo, expand test coverage
- Extract dedup logic into exported deduplicateMessages() pure function so
  it can be unit-tested directly
- Wrap with useMemo([messages]) to avoid allocating a new Map on every frame
- Collapse three-way branch to two-way (the two arms were identical)
- Use { ...existing, ...message, content } for a true merge so fields present
  only in an earlier occurrence are not silently dropped
- Expand comment to explain why || treats empty string as falsy
- Add deduplicateMessages unit tests: toolCalls assertion, reverse scenario,
  non-assistant keep-last behavior
- Rename "keeping the last occurrence" test — no longer accurate for assistant
  messages which now merge rather than keep-last

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:46:46 -07:00
Martha Schumann c89f42d311 fix(tests): type MCPAppsActivityRenderer test renderer with Zod schema
Replace `ReactActivityMessageRenderer<unknown>` + `(content as any).resourceUri`
with `ReactActivityMessageRenderer<z.infer<typeof MCPAppsActivityContentSchema>>`
so the renderer prop is properly typed and the `as any` cast is eliminated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:46:14 -07:00
Martha Schumann 8559608aeb style: fix oxfmt formatting in CopilotChatView 2026-04-06 10:59:37 -07:00
Martha Schumann e6539d7f24 fix(tests): update MockMCPProxyAgent.clone() to share isRunning and runAgent state
After the useRenderActivityMessage fix, MCPAppsActivityRenderer receives the
per-thread clone instead of the registry agent. The mock's clone() did not share
isRunning state, causing waitForAgentIdle(clone) to never resolve (emit() only
updated registry.isRunning). Also, tests that monkey-patch runAgent before
renderWithCopilotKit need the clone to delegate proxied MCP requests to the
registry (to pick up the monkey-patch) while running user-message flows on the
clone itself (so clone.messages is updated and rendered by CopilotKit).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 10:11:56 -07:00
Martha Schumann 1c156273a5 fix(react-core): address review feedback on perf PR
- Fingerprint all tool-call args on last message (not just last tool call)
  so parallel/non-last tool-call streaming is detected and triggers re-render
- Add dev warning when scroll container has clientHeight=0 so virtualization
  silently disabled (e.g. chat inside display:none tab) is surfaced early
- Use callback ref for non-autoScroll path in ScrollView to eliminate
  useLayoutEffect + eslint-disable; autoScroll path keeps useLayoutEffect
  since StickToBottom manages that ref internally

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 10:06:49 -07:00
Martha Kelly Schumann 2d4782f68d Merge branch 'main' into fix/a2ui-activity-thread-clone 2026-04-06 10:04:55 -07:00
Martha Schumann a92f43c1b3 test(react-core): add virtual path coverage to CopilotChatPerf tests
Adds a jsdom-compatible test that confirms the virtual code path activates
above VIRTUALIZE_THRESHOLD without requiring a real browser viewport. Mocks
clientHeight and getBoundingClientRect on the scroll element to pass both
CopilotChatMessageView's guard and TanStack Virtual's observeElementRect.
Asserts the virtual container div exists and rendered count < TOTAL.
Drains pending rAF callbacks before unmount to prevent spurious uncaught
exceptions after jsdom teardown.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 09:54:34 -07:00
Martha Kelly Schumann 4ebbbe5f2c Merge branch 'main' into claude/langgraph-tests-cleanup-1jVZ3 2026-04-06 09:46:46 -07:00
Martha Schumann 07b9ec13ec chore(a2ui): add /a2ui-demo page and DemoButtonAgent for manual fix validation
Adds a minimal demo at /a2ui-demo that uses the demo-button agent to
reproduce and validate the A2UI thread-clone bug fix. The agent renders
an A2UI surface with a Confirm button on first run; clicking it fires a
second run that emits a text confirmation message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 09:45:23 -07:00
Martha Kelly Schumann 90d99e857f Merge branch 'main' into feat/CPK-7190-chat-performance 2026-04-06 09:36:12 -07:00
Martha Kelly Schumann 6e444ecd31 Merge branch 'main' into fix/CPK-7154-agent-text-wiped-multiple-tool-calls 2026-04-06 09:23:51 -07:00
Martha Schumann 0263b06e2a fix(chat): remove scrollToIndex length dep, fix stale toolResultMap comment
scrollToIndex was firing on every streaming chunk (deduplicatedMessages.length
dep), forcibly yanking users to the bottom even when scrolled up to read
history. Removed the dep — use-stick-to-bottom handles streaming auto-scroll
via content height growth on the virtualizer's total-size div, same as the
flat path. scrollToIndex now only fires on virtual mode activation and thread
switches.

Also removed a stale comment referencing toolResultMap (removed two commits
ago) and replaced it with an accurate description of the linear scan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 09:15:26 -07:00
Ran Shemtov 88fef2afe3 Merge branch 'main' into claude/remove-endpoint-flag-xd3G1 2026-04-06 18:10:10 +02:00
Ran Shem Tov a9a4a3e4c2 fix: remove redundant type casts and inline fetch in transport auto-detection 2026-04-06 18:09:48 +02:00
Alem Tuzlak 53a2e9c6e3 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-06 17:30:11 +02:00
Alem Tuzlak 626c33bda2 fix: stabilize useAttachments hook with useCallback, add stability tests
- All callbacks (processFiles, handleFileUpload, handleDragOver, handleDragLeave,
  handleDrop, removeAttachment, consumeAttachments) are now wrapped in useCallback
  with empty deps — referentially stable across renders
- Config values read from configRef to avoid dep array changes
- consumeAttachments no-ops on empty queue (returns same state reference)
- 9 new tests: referential stability across re-renders, re-render counting
  (consumeAttachments on empty queue triggers zero re-renders), initial state,
  consumeAttachments behavior, removeAttachment no-op
2026-04-06 16:20:42 +02:00
Alem Tuzlak d5449f3b5d fix: restore JSON.stringify messages memo to fix streaming tool call re-renders 2026-04-06 16:15:13 +02:00
Martha Schumann c1a560ba85 fix(a2ui): pass per-thread clone to activity message renderers
useRenderActivityMessage was calling copilotkit.getAgent() directly,
always returning the registry agent. When a user clicked an A2UI button,
handleAction → runAgent executed on the registry agent — messages
accumulated there while CopilotChat displayed from the per-thread clone
(created by useAgent), so responses appeared to silently vanish.

Apply the same getThreadClone(registryAgent, threadId) ?? registryAgent
pattern already used in useRenderCustomMessages.

Adds a regression test that asserts the renderer receives the clone,
and confirms the test catches the bug when the fix is reverted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 07:01:22 -07:00
Alem Tuzlak d7578216d3 refactor: extract useAttachments hook, memoize attachment renderers
- Extract all attachment logic from CopilotChat into reusable useAttachments hook
  (state, processFiles, drag/drop, paste, consumeAttachments, removeAttachment)
- Export useAttachments from hooks/index.ts for custom chat UI builders
- Memoize v2 CopilotChatAttachmentRenderer sub-components
  (ImageAttachment, AudioAttachment, VideoAttachment, DocumentAttachment)
- Memoize v1 AttachmentRenderer sub-components to match v2 structure,
  align props (content → filename)
- Guard process.env in attachment-utils.ts for browser environments
- Remove debug logging (existing.debug = true) from use-agent.tsx
2026-04-06 15:17:56 +02:00
Alem Tuzlak 0fd3ddf960 refactor(docs): create migration-guides section, move migration pages from troubleshooting
- New docs/(root)/migration-guides/ section with migrate-to-v2, migrate-attachments,
  migrate-to-1.10.X, migrate-to-1.8.2
- Troubleshooting now only contains error-debugging, observability-connectors, common-issues
- Update all @deprecated JSDoc URLs from /troubleshooting/ to /migration-guides/
- Update doc cross-links (multimodal-attachments page, snippet import)
- Move snippet to snippets/shared/migration-guides/
- Set minimum version to v1.56.0 (was vX.Y.Z placeholder)
2026-04-06 14:55:09 +02:00
Alem Tuzlak 89d39d639b fix: address code review findings — error handling, codemod correctness, deprecation versions
- Restore console.error as safety net in processFiles catch blocks (errors no longer
  silently vanish when onUploadFailed callback is not provided)
- Add try/catch to useBlobUrl atob() — gracefully handles malformed base64 instead of
  crashing the component tree
- Fix DocumentLightboxContent: use blobUrl instead of undefined src variable
- Add .catch() to View Transition API transition.finished promise
- Codemod: preserve actual imageUploadsEnabled value (false, dynamic expressions)
  instead of hardcoding true — adds 3 new test cases (18 total)
- Replace all placeholder @deprecated versions (v1.x.0) with @since 1.56.0
- Add missing @since to ImageRenderer, ImageRendererProps, AIMessage.image tags
2026-04-06 14:55:08 +02:00
Alem Tuzlak 18c7a6001d feat: multimodal attachments — UI polish, deprecations, docs, codemod
Attachment queue & previews:
- Image lightbox with View Transition API morph animation
- Video lightbox with native controls and play button overlay
- Document lightbox (PDF via blob URL, text inline, info card fallback)
- Drop zone overlay with upload icon
- Filename preservation via InputContent metadata
- Proper video thumbnail sizing and play/pause indicator
- Fix attachment queue positioning (max-w-3xl constraint)
- Padding between X button and content for audio/document cards
- Document filenames wrap instead of truncating

Attachments config:
- onUploadFailed callback for validation/upload errors (file-too-large, invalid-type, upload-failed)
- onUpload accepts sync or async returns
- AttachmentUploadResult discriminated union with explicit interfaces
- Metadata field on Attachment and onUpload return type

AG-UI version bump:
- Bump @ag-ui/client, @ag-ui/core, @ag-ui/encoder, @ag-ui/proto to 0.0.51
- Remove process.env Vite workaround (fixed upstream in 0.0.51)

Deprecation lifecycle:
- @deprecated JSDoc on all legacy image upload APIs
- ImageRenderer, ImageRendererProps, ImageUpload type, imageUploadsEnabled prop,
  inputFileAccept prop, ImageRenderer prop, AIMessage.image, ImageData
- Codemod at codemods/migrate-attachments.ts (15 tests)
- Migration guide updated with codemod instructions and new type shapes

Docs:
- New guide: docs/(root)/multimodal-attachments.mdx
- Updated migration guide with onUpload return type, metadata, codemod section
- Cross-links from prebuilt-components and migration guide
- Label change: "Add photos or files" → "Add attachments"

Tests:
- CopilotChat.attachments.test.tsx — 5 tests for onUploadFailed
- migrate-attachments codemod — 15 tests
2026-04-06 14:55:06 +02:00
Alem Tuzlak 1603b63e33 style: format v2 multimodal changes 2026-04-06 14:54:46 +02:00
Alem Tuzlak cf0f8ad0b8 feat(react-core): export v2 attachment components and types 2026-04-06 14:54:45 +02:00
Alem Tuzlak c37d409bbf feat(react-core): add attachments prop, upload flow, drag-and-drop to v2 CopilotChat
Wire multimodal attachment support into the v2 CopilotChat component:
- Add AttachmentsConfig prop to CopilotChat for enabling file attachments
- Implement processFiles with accept filter, size validation, placeholder/ready lifecycle
- Support custom onUpload handlers and default base64 encoding
- Build InputContent[] when attachments accompany a message
- Add drag-and-drop handlers with visual feedback (dashed outline)
- Add clipboard paste handler for file items
- Render CopilotChatAttachmentQueue between scroll view and input
- Forward onAddFile through CopilotChatView to CopilotChatInput's AddMenuButton
- Omit internal attachment state props from CopilotChatProps public API
2026-04-06 14:54:44 +02:00
Alem Tuzlak ae1dc0ce4c feat(react-core): add v2 attachment renderer, update user message for multimodal, DRY shared utils 2026-04-06 14:54:43 +02:00
Alem Tuzlak 8f54f411ef feat(react-core): add v2 CopilotChatAttachmentQueue component 2026-04-06 14:54:42 +02:00
Alem Tuzlak f6c15cdbfe refactor(shared): move attachment utilities to shared package 2026-04-06 14:54:42 +02:00
Alem Tuzlak 7c774e9eb6 refactor(shared): move attachment types to shared package 2026-04-06 14:54:41 +02:00
Alem Tuzlak 152d1ade96 style: format code with oxfmt 2026-04-06 14:54:40 +02:00
Alem Tuzlak 928c385516 fix(react-ui): add baseline error logging, thumbnail diagnostics, rejected file feedback, mimeType fallback 2026-04-06 14:54:39 +02:00
Alem Tuzlak eeccbd3e36 fix(runtime): upgrade dropped content warnings to errors 2026-04-06 14:54:38 +02:00
Alem Tuzlak 2c546e44b5 test(react-ui): add tests for attachment utility functions 2026-04-06 14:54:37 +02:00
Alem Tuzlak e683f9386c fix(react-ui): address code review findings — stale closures, error handling, type safety
- Fix stale closure in paste handler by using processFilesRef pattern
- Wrap processFiles calls in try/catch across paste, file upload, and drop handlers
- Add early return for empty/falsy accept string in matchesAcceptFilter
- Block send while attachments are still uploading
- Add id field to Attachment interface, replace _id hack with randomUUID()
- Add 10s timeout to generateVideoThumbnail to prevent infinite hanging
- Show placeholder div for uploading attachments instead of rendering incomplete source
- Include filename context in upload error messages
2026-04-06 14:54:36 +02:00
Alem Tuzlak b6245d31fc fix(runtime): handle malformed URLs and unrecognized content parts defensively
Wrap all three new URL() call sites in try/catch so a malformed URL string
skips the part with a console.warn rather than crashing message conversion.
Add a default switch case to warn on unrecognized content part types.
Add a test covering the malformed-URL skip behaviour.
2026-04-06 14:54:36 +02:00
Alem Tuzlak 1d62da913d chore: add deprecation annotations and suppressable runtime warnings 2026-04-06 14:54:34 +02:00
Alem Tuzlak fc1cfa09d9 feat(react-ui): add attachments config, upload flow, drag-and-drop, and deprecation bridge 2026-04-06 14:54:33 +02:00
Alem Tuzlak ff236cfbe6 feat(react-ui): add AttachmentRenderer and update UserMessage for multimodal content 2026-04-06 14:54:32 +02:00
Alem Tuzlak c472f452f6 feat(react-ui): add AttachmentQueue component and styles 2026-04-06 14:54:31 +02:00
Alem Tuzlak 8bdb24ea87 feat(react-ui): add attachment types and utility functions 2026-04-06 14:54:30 +02:00
Alem Tuzlak c2c048f53f feat(runtime): support multimodal content in BuiltInAgent message conversion 2026-04-06 14:54:29 +02:00
Alem Tuzlak 0b05c3111d feat(shared): align UserMessage with AG-UI multimodal types
Re-export AG-UI multimodal input types (InputContent, TextInputPart, ImageInputPart,
AudioInputPart, VideoInputPart, DocumentInputPart, InputContentSource, etc.) from
@copilotkit/shared. UserMessage is now a plain pass-through of agui.UserMessage,
removing the CopilotKit-specific image?: ImageData extension. ImageData is kept with
@deprecated for backward compatibility with AIMessage.image.
2026-04-06 14:54:28 +02:00
Alem Tuzlak 7de4bfa9cf chore: bump @ag-ui/core and @ag-ui/client to 0.0.50-alpha.0 2026-04-06 14:54:27 +02:00
Ran Shemtov 12838ee879 Merge branch 'main' into claude/remove-endpoint-flag-xd3G1 2026-04-06 14:43:14 +02:00