Commit Graph

1601 Commits

Author SHA1 Message Date
Maxim 8a1f111602 Merge branch 'main' into perf/chat-input-cache-container-measurements 2026-04-08 19:38:34 +02:00
Maxim ad4271c266 test(react-core): extract shared helpers and document casts in cache tests
Extract renderAndWarmCache() and installBoundingRectSpy() helpers to
eliminate duplicated setup across 4 tests. Add justifying comments on
all necessary type casts (MockResizeObserver, CSSStyleDeclaration,
TextMetrics, canvas getContext overloads).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 19:22:33 +02:00
Maxim 35e35bb5bc fix(react-core): preserve cache across layout toggles and reject zero-width entries
Self-inflicted resizes (compact↔expanded) were clearing the cache before
the ignoreResizeRef guard, forcing a full re-measurement on the very next
keystroke. Move the guard ahead of invalidation so layout toggles keep the
cache warm — container dimensions don't change during these transitions.

Also reject compactWidth <= 0 from being cached, preventing a zero-width
entry from silently disabling text-width expansion until the next resize.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 18:54:25 +02:00
Maxim 84d531c1e3 fix: address review feedback — single observer, fresh font read, test coverage
- Merge two ResizeObserver instances into one; inspect entry.target to
  decide cache invalidation.  Eliminates the ignoreResizeRef race where
  two observers could consume the guard independently.
- Move textarea font read out of the cache into evaluateLayout so CSS/
  theme changes are picked up without a container resize.
- Return { compactWidth } | null from updateContainerCache; callers use
  the value directly instead of the two-step "call then check ref" pattern.
- Hoist cache invalidation above the ignoreResizeRef guard to remove
  duplicated containerCacheRef.current = null.
- Add zero-width guard in updateContainerCache to avoid caching
  compactWidth: 0 when the container is hidden.
- Replace vi.fn() as any ResizeObserver mock with a class-based
  MockResizeObserver that tracks observed targets, enabling per-target
  resize triggers in tests.
- Fix empty-font and null-canvas tests to assert data-layout="compact"
  (verifying the fallback path) instead of just non-null.
- Add tests: warm-cache keystroke perf (getBoundingClientRect not called),
  textarea-only vs container resize invalidation, ignoreResizeRef + cache
  invalidation during layout toggle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 18:39:22 +02:00
Alem Tuzlak 990acb46ab fix(runtime): address CR round 1 findings
- Fix all doc imports to use @copilotkit/runtime/v2 (symbols only
  exported from v2 entry point)
- Replace v1 copilotRuntimeNextJSAppRouterEndpoint with v2 pattern
  (CopilotRuntime + createCopilotEndpoint + InMemoryAgentRunner)
- Add warning callout on TanStack reasoning example (reasoning events
  not surfaced by TanStack converter)
- Fix randomUUID imports in converters: use @copilotkit/shared instead
  of Node.js crypto to match codebase convention
- Fix collectEvents timer leak in test-helpers.ts (add clearTimeout)
- Fix JSDoc/comments: convertInputToTanStackAI accurately describes
  role allowlist, TanStackInputResult.messages comment updated
2026-04-08 18:03:44 +02:00
Martha Kelly Schumann 04d73d1dca Merge branch 'main' into claude/langgraph-tests-cleanup-1jVZ3 2026-04-08 08:52:43 -07:00
Alem Tuzlak a8715b9631 style(runtime): fix formatting in agent test files 2026-04-08 17:40:47 +02:00
Alem Tuzlak 9b15a7c378 fix(runtime): address PR review feedback
- Close TOCTOU window: set abortController synchronously before Observable
  creation in classic run(), matching factory run() pattern (M3)
- Add concurrent run guard to classic run() (C1)
- Add threadId/runId to RUN_ERROR events in classic mode (I2)
- Check both 'output' and 'result' property names in classic tool-result (C2/M2)
- Add try/catch around JSON.stringify in classic tool-result (I4/M2)
- Add undefined guards before emitting state snapshot/delta events (I3)
- Guard assignToolsToAgents against factory-mode agent configs (M1)
- Improve error handling in classic error case with proper fallback (C3)
- Improve TanStack converter TODO comment with known gap details (M5)
- Document @ts-expect-error on clone() middlewares access (S1)
- Add JSDoc to AgentFactoryContext.abortController (M4)
- Fix duplicate import and clean up react-router example
- Update docs AgentFactoryContext reference
2026-04-08 17:32:50 +02:00
Tyler Slaton da8e58f1a1 Merge branch 'main' into fix/CPK-7154-agent-text-wiped-multiple-tool-calls 2026-04-08 08:25:46 -07:00
Alem Tuzlak 920640cb5a test(runtime): update Agent tests to use BuiltInAgent factory mode 2026-04-08 17:15:00 +02:00
Alem Tuzlak 802d20e95d refactor(runtime): remove Agent class, absorb into BuiltInAgent 2026-04-08 17:14:59 +02:00
Alem Tuzlak 3fcf2fbaf9 feat(runtime): add factory-path branch to BuiltInAgent.run()
- Add converter imports (convertAISDKStream, convertTanStackStream) to agent/index.ts
- Update canOverride() to return false for factory configs
- Add factory-path early return in run() that delegates to runFactory()
- Add private runFactory() method that handles aisdk/tanstack/custom factory configs
  with full lifecycle management (RUN_STARTED, RUN_FINISHED, RUN_ERROR, abort support)
- Fix copilot-runtime.ts: change BuiltInAgentConfiguration["tools"] indexed access
  types to BuiltInAgentClassicConfig["tools"] since tools only exist on classic configs
2026-04-08 17:14:58 +02:00
Alem Tuzlak d320a5217c feat(runtime): add factory config types to BuiltInAgentConfiguration union 2026-04-08 17:14:56 +02:00
Alem Tuzlak 7e69300537 fix(runtime): address CR loop round 1 findings
Source fixes:
- Fix TOCTOU race in concurrent run guard — set abortController
  synchronously before Observable creation, closing the window where
  two run() calls could both pass the guard
- Add JSON.stringify try/catch in TanStack TOOL_CALL_RESULT handler
  (matching existing aisdk.ts protection against circular refs)
- Use role allowlist (user/assistant/tool) instead of denylist in
  convertInputToTanStackAI to exclude activity/reasoning roles
- Change error+abort handler from break to return in aisdk converter
  to stop processing after error during abort
- Fix version-pinned comments (remove "AI SDK 5.0" reference)

Test fixes:
- Add timedOut flag to collectEventsIncludingErrors to distinguish
  hung observables from normal completion
- Add clearTimeout on success/error to prevent timer leaks
- Remove timing-dependent setTimeout in concurrent run guard test

Docs fixes:
- Split imports: copilotRuntimeNextJSAppRouterEndpoint from
  @copilotkit/runtime, Agent/converters from @copilotkit/runtime/v2
- Fix forwardedProps example: use CopilotKit properties prop
- Fix useCopilotChat → useCoAgent for state management reference
- Fix model syntax colon → slash (openai/gpt-4o)
2026-04-08 17:14:55 +02:00
Alem Tuzlak 4e33d42fb6 refactor(tests): test-hygiene fixes for Agent test files
- Replace Record<string, unknown> return types on TanStack mock chunk
  builders with inferred types via as const
- Remove dynamic imports in converter-aisdk error tests — use
  top-level collectEventsIncludingErrors import instead
- Remove redundant as RunAgentInput casts in converter-tanstack-input
  tests (createDefaultInput already returns the correct type)
- Add comments explaining necessary casts in createAgent overloads
  (TypeScript limitation with discriminated union narrowing)
2026-04-08 17:14:54 +02:00
Alem Tuzlak b344d08e17 fix(runtime): address code review feedback on Agent class
Critical fixes:
- Guard concurrent run() calls — throw if agent is already running
- Check both "output" and "result" properties on AI SDK tool-result events
  for compatibility with older AI SDK versions
- Handle undefined/missing error in AI SDK error events with proper
  Error wrapping instead of throwing undefined

Important fixes:
- Add TOOL_CALL_RESULT handling to TanStack converter (was silently dropped)
- Include threadId and runId in RUN_ERROR events for run correlation
- Guard state tool events (STATE_SNAPSHOT/STATE_DELTA) against undefined
  snapshot/delta fields before emitting
- Wrap JSON.stringify in tool-result with try/catch to handle circular refs
- Fix collectEventsIncludingErrors to distinguish error vs normal completion

Test coverage:
- Add tests for convertInputToTanStackAI (message filtering, tool call
  mapping, context injection, state serialization)
- Add tests for state management tool results in AI SDK converter
- Add tests for async factory (Promise-returning) paths
- Add tests for tool-result property compatibility (output vs result)
- Add tests for error event handling edge cases
- Add tests for concurrent run guard and RUN_ERROR correlation fields
- Replace flaky setTimeout-based abort test with signal-based sync

Other:
- Add exhaustiveness check (default: never) in config type switch
- Document clone() @ts-expect-error coupling with AbstractAgent.middlewares
- Fix docs: remove incorrect "v6" version reference, fix wrong hook name
2026-04-08 17:14:52 +02:00
Alem Tuzlak 051f55a7de style: apply prettier formatting to new Agent files 2026-04-08 17:14:49 +02:00
Alem Tuzlak 31919c4cf4 refactor(test): replace type casts with eventField in converter tests 2026-04-08 17:14:48 +02:00
Alem Tuzlak ebfd2c33fc refactor(test): remove all type casts from Agent test files 2026-04-08 17:14:47 +02:00
Alem Tuzlak 5f0a718932 feat(agent): add convertInputToTanStackAI utility for message/prompt conversion 2026-04-08 17:14:45 +02:00
Alem Tuzlak aae03a7ce0 test(agent): add custom converter (passthrough) tests 2026-04-08 17:13:58 +02:00
Alem Tuzlak 8b264f8735 test(agent): add core Agent class tests with parameterized lifecycle/abort/clone 2026-04-08 17:13:57 +02:00
Alem Tuzlak 4b544cc99b test(agent): add AI SDK converter tests 2026-04-08 17:13:55 +02:00
Alem Tuzlak dfcf032b6b test(agent): add TanStack AI converter tests 2026-04-08 17:13:54 +02:00
Alem Tuzlak c07aa13b9e test(agent): add shared test utilities for Agent class 2026-04-08 17:13:52 +02:00
Alem Tuzlak af9de02ee9 fix(agent): support async factory functions in Agent config types 2026-04-08 17:13:51 +02:00
Alem Tuzlak b9b5164033 feat(runtime): add universal Agent class with multi-backend support
Add Agent class extending AbstractAgent that accepts aisdk, tanstack,
or custom configurations via a discriminated union. Routes to the
appropriate converter, handles RUN_STARTED/FINISHED/ERROR lifecycle,
and supports clone/abortRun. Re-export Agent and converters from the
agent barrel.
2026-04-08 17:13:49 +02:00
Alem Tuzlak 3997203e47 feat(runtime): add TanStack AI stream converter and barrel export
Add convertTanStackStream async generator that converts TanStack AI
stream chunks (TEXT_MESSAGE_CONTENT, TOOL_CALL_START/ARGS/END) into
AG-UI events. Create converters/index.ts barrel that re-exports both
AI SDK and TanStack converters.
2026-04-08 17:13:48 +02:00
Alem Tuzlak ddc58b1e1a feat(runtime): extract AI SDK stream converter from BuiltInAgent
Extract the AI SDK fullStream event-mapping loop into a standalone
async generator (convertAISDKStream) that yields AG-UI BaseEvent
objects. This is a pure converter with no lifecycle events — the
caller handles RUN_STARTED/FINISHED/ERROR.
2026-04-08 17:13:46 +02:00
Maxim f35df00b16 test(react-core): improve test hygiene in container dimension cache tests
Extract shared helpers (DEFAULT_LAYOUT_OPTIONS, getLayoutGrid,
renderTypeAndExpectLayout) to eliminate duplication, add explanatory
comments for unavoidable `as any` casts, and add missing test for
canvas.getContext returning null.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:49:24 +02:00
Maxim 8da887c14c fix(react-core): harden container dimension cache in CopilotChatInput
- Split ResizeObserver into container vs textarea observers so textarea
  height changes (typing, manual resize) don't invalidate the dimension
  cache unnecessarily — only grid/button resizes invalidate it
- Invalidate cache on self-triggered resizes (compact↔expanded toggle)
  even when skipping re-evaluation, preventing stale dimensions
- Validate fontSize and fontFamily before constructing font fallback
  string, avoiding malformed CSS font values in Safari
- Add dev-mode console.warn when font resolution or canvas context
  fails, making silent measurement path failures diagnosable
- Update test ResizeObserver mock to support multiple observers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:36:47 +02:00
Maxim 27bde85b6f test(react-core): add tests for container dimension cache in CopilotChatInput
- Test canvas text measurement triggers expansion for long single-line text
- Test compact layout preserved when text fits within cached width
- Test cache invalidation on resize produces correct re-measurement
- Test empty font string is not cached (guard prevents invalid canvas state)
- Mock ResizeObserver to exercise cache invalidation path in jsdom
- Extend mockLayoutMetrics with getComputedStyle mocks for grid and textarea

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:10:10 +02:00
Maxim 8986a47220 fix(react-core): harden container dimension cache in CopilotChatInput
- Store compactWidth (pre-padding) in cache, compute compactInnerWidth
  at read time from live measurementsRef to avoid stale padding values
- Move cache invalidation after ignoreResize check so programmatic
  layout transitions don't needlessly destroy the cache
- Guard against empty/invalid font strings before writing to cache
- Fix comment inaccuracies: 2x (not 3x) getComputedStyle, lazily
  populated (not on mount), invalidated (not updated) on resize

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 22:56:12 +02:00
Maxim d9125fc076 perf(react-core): cache container dimensions in CopilotChatInput to reduce per-keystroke reflows
evaluateLayout runs on every keystroke via useLayoutEffect. Previously it
performed 3x getComputedStyle + 2x getBoundingClientRect on every call to
measure grid/button dimensions and the textarea font — none of which change
between keystrokes.

Cache these measurements in a ref (containerCacheRef), populated lazily on
first evaluateLayout call and invalidated when the ResizeObserver fires.
Per-keystroke reflows drop from 3-4 to 1 (the unavoidable adjustTextareaHeight).

Also removes a stale console.log("FOOBAR") debug line.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 22:24:17 +02:00
Martha Schumann 191fdc4999 fix(react-core): recover toolCalls when later chunk sends undefined; add test
The { ...existing, ...message } spread meant a later streaming chunk with
toolCalls: undefined would silently wipe accumulated tool calls, contradicting
the comment's claim that "latest toolCalls wins". Apply the same ?? recovery
logic to toolCalls as content already uses for the || fallback.

Add a test for the flip-side edge case: first occurrence has toolCalls, second
has non-empty content but undefined toolCalls — toolCalls must survive.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 12:21:56 -07:00
Martha Schumann 53b7fa206d chore: merge main into fix/CPK-7154-agent-text-wiped-multiple-tool-calls
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 12:10:59 -07:00
Martha Schumann 3e2b1a8915 fix(cli): bump tar to v7 to resolve npm deprecation warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 11:48:32 -07:00
Martha Schumann 1a61be49dc refactor(FOR-75): replace useJsonStable with useShallowStableRef; add unit tests and changeset
- Kill useJsonStable in CopilotChatConfigurationProvider — useShallowStableRef
  is now the single stabilization primitive for both messageView props and labels
- Harden useShallowStableRef with isPlainObject guard so arrays, Dates, and
  class instances are never shallow-compared (reference-only for non-plain objects)
- Remove redundant stableParentLabels — parentConfig?.labels is already stabilized
  by the parent provider's own useShallowStableRef
- Uninstall ts-deepmerge from react-core dependencies (no longer imported anywhere)
- Add unit tests for useShallowStableRef in slots.test.ts (5 tests)
- Add changeset (patch) describing the FOR-75 performance fix

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 11:00:19 -07:00
Martha Schumann 56cc194782 refactor(react-core): extract useShallowStableRef; fix Test 2 to guard labels regression
- Extract useShallowStableRef<T> into slots.tsx alongside shallowEqual —
  replaces the 20-line inline stabilization block in CopilotChat with two
  readable call sites, and covers suggestionView for free
- Apply useShallowStableRef to both messageView and suggestionView in CopilotChat
- Use post-hoc assignment for messageView to avoid an empty object allocation
  on every render when messageView is undefined
- Rewrite Test 2 (labels) to use LabelConsumerMessage which calls
  useCopilotChatConfiguration() directly — context consumers re-render when
  their context changes regardless of parent memo boundaries, making this a
  genuine regression guard for the labels fix (previous version tracked
  assistantRenderCount which was already protected by messageView stabilization)
- Replace `as any` on messageView with scoped cast to preserve structural
  type checking on the surrounding object
- Fix text/testid collision: CountingAssistantMessage now renders data-testid
  instead of a hardcoded string that matched the agent reply text

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 10:41:46 -07:00
Martha Schumann 35481074bd refactor(react-core): replace eslint-disable comments with useJsonStable hook
Replaces the manual JSON.stringify dep-key pattern (which required three
eslint-disable-next-line comments) with a useJsonStable utility that
stabilizes object references using JSON comparison. The dep array for
mergedLabels is now honest — it references the exact variables used inside
the useMemo callback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 10:30:26 -07:00
Martha Schumann cdbc3457cf fix(react-core): stabilize messageView/labels props to prevent message re-renders on keystroke
Inline `messageView` and `labels` objects passed to CopilotChat caused all
completed assistant messages to re-render on every keystroke because:

1. `ts-deepmerge.merge()` deep-clones its inputs, producing a new `messageView`
   reference on every render even when content is identical — defeating
   `MemoizedSlotWrapper`'s shallow equality check.

2. An inline `labels` object (new reference each render) invalidated the
   `mergedLabels` useMemo in `CopilotChatConfigurationProvider`, causing
   every `useCopilotChatConfiguration()` consumer to re-render on every keystroke.

3. The inline `onAddFile` arrow function was a new reference each render.

Fixes:
- Replace `merge()` with shallow spread; stabilize `messageView` via a ref +
  `shallowEqual` guard so the same object reference is returned when props
  are shallowly equal across renders.
- Use `useCallback` for `handleAddFile`.
- Add JSON.stringify-based dep keys in `CopilotChatConfigurationProvider` so
  `mergedLabels` is only recomputed when label values actually change.

Adds deterministic render-count regression tests (FOR-75) that fail on the
unfixed code and pass after the fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 09:55:34 -07:00
Martha Kelly Schumann 30b81395f6 Merge branch 'main' into feat/CPK-7190-chat-performance 2026-04-07 08:51:33 -07:00
Martha Schumann 82d58fae9b chore: merge main into feat/CPK-7190-chat-performance
Resolves conflict in CopilotChat.tsx — takes main's attachment-aware
messagesMemoKey fingerprint (scans all messages, avoids JSON.stringify
on base64 data) which subsumes our all-tool-calls fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 08:35:29 -07:00
Alem Tuzlak 39b5f671c0 Merge branch 'main' into claude/remove-endpoint-flag-xd3G1 2026-04-07 14:47:09 +02:00
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Ran Shemtov 0a262b3ad6 Merge branch 'main' into claude/remove-endpoint-flag-xd3G1 2026-04-07 12:55:33 +02:00
Ran Shem Tov 51bcc94988 chore: fix failing tests 2026-04-07 12:47:08 +02:00
Alem Tuzlak 773a4a6415 fix: format package.json files 2026-04-07 12:29:29 +02:00
Alem Tuzlak 0aca4e88bc fix: address review feedback — concurrent-safe consumeAttachments & scope-aware codemod
1. consumeAttachments: read from a ref mirror instead of side-effecting
   out of a setState updater, avoiding reliance on synchronous updater
   execution under React concurrent mode.

2. Codemod: skip declaration positions (variable, function, class, type,
   interface) and non-reference positions (object keys, member accesses).
   When a local declaration shadows the import name, only rename
   unambiguous type-position references to avoid corrupting unrelated code.
2026-04-07 12:14:14 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00