Commit Graph

82 Commits

Author SHA1 Message Date
Max Korp 5feeb21df1 test(runtime): add thread route coverage to fetch-router tests 2026-04-10 11:33:24 -07:00
Max Korp 58cc6ff55e fix(runtime): restore thread routes dropped during fetch-handler rewrite 2026-04-10 11:33:24 -07:00
Alem Tuzlak ff2093102c chore(lint): add new oxlint rules and auto-fix violations
Enable stricter oxlint rules for better code health:
- typescript/consistent-type-imports: enforce `import type` for type-only imports
- typescript/no-import-type-side-effects: prefer top-level type imports
- import/consistent-type-specifier-style: consistent type specifier placement
- typescript/no-unnecessary-type-assertion: bump to error
- react/self-closing-comp: enforce self-closing JSX components
- unicorn/prefer-optional-catch-binding: drop unused catch params
- eslint/no-useless-computed-key: simplify object keys
- unicorn/prefer-string-slice: prefer .slice() over .substring()
- unicorn/prefer-array-flat-map: prefer .flatMap() over .map().flat()

All existing violations auto-fixed via oxlint --fix.
2026-04-10 19:29:32 +02:00
Alem Tuzlak 672d3cf2e1 fix(runtime): preserve empty text parts in TanStack converter
Fix falsy check dropping valid empty-string text parts.
Add tests for unknown part types, null content, and
binary parts with neither data nor url.
2026-04-10 17:29:24 +02:00
Alem Tuzlak 4a69d7a8e3 feat(runtime): add multimodal support to TanStack converter
- Fix convertInputToTanStackAI silently dropping multimodal
  content (images, audio, video, documents) by converting
  AG-UI content parts to TanStack AI ContentPart format
- Switch react-router example from Hono server to a React
  Router resource route using createCopilotRuntimeHandler
- Both agents now use BuiltInAgent factory pattern
- Remove hono/react-router-hono-server deps from example
2026-04-10 17:15:57 +02:00
Tyler Slaton 214924c7f9 Merge branch 'main' into worktree-fancy-mapping-valley 2026-04-09 20:47:51 -07:00
Jordan Ritter 1b01725d40 test: add 18 MCP integration tests with aimock MCPMock
Phase 1 — BasicAgent mcpServers (8 tests):
- HTTP transport tool fetch, SSE error handling
- Tool call round-trip, client cleanup, unreachable server error
- Multiple servers merge tools, error-path cleanup, tool descriptions

Phase 2 — MCPAppsActivityRenderer proxy (5 tests):
- tools/call proxy round-trip, error handling
- ui/open-link handler + missing URL error
- Multiple independent activities

Phase 3 — MCPAppsMiddleware edge cases (5 tests):
- Middleware creation, tools/call proxy, resources/read proxy
- Non-proxied request delegation, wrong serverHash error

All tests use real HTTP connections to aimock MCPMock.
2026-04-09 14:58:32 -07:00
Jordan Ritter 55663ee69f chore(runtime): add @copilotkit/aimock devDependency for MCP integration tests 2026-04-09 14:54:07 -07:00
Alem Tuzlak 3dd17c1c5b Merge branch 'main' into worktree-fancy-mapping-valley 2026-04-09 20:40:27 +02:00
github-actions[bot] 1bc4786759 chore: version packages (next) 2026-04-09 18:09:30 +00:00
Tyler Slaton 62c604213c chore(a2ui-renderer): bump version for a2ui-middleware to 0.0.4
Signed-off-by: Tyler Slaton <tyler@copilotkit.ai>
2026-04-09 11:05:10 -07:00
github-actions[bot] 0093bdbaa4 chore: version packages 2026-04-09 02:26:42 +00:00
github-actions[bot] c2837a3001 chore: version packages (next) 2026-04-09 01:53:20 +00:00
Tyler Slaton 422f5f55e1 fix(open-gen-ui): correctly import clairinet
Signed-off-by: Tyler Slaton <tyler@copilotkit.ai>
2026-04-08 18:09:03 -07:00
github-actions[bot] 00cace6abf chore: version packages 2026-04-08 23:54:54 +00:00
Ben Taylor ca233aea20 Revert "fix: catch async errors in LangGraphAgent.run() Observable" 2026-04-08 18:40:52 -05:00
Max Korp c5d712f4cf test(runtime): add unit tests for LangGraphAgent.run() async error handling
Verify that async rejections from runAgentStream are routed through the
Observable error channel instead of becoming unhandled promise rejections.
2026-04-08 16:17:55 -07:00
Max Korp cffc4cafe0 fix: catch async errors in LangGraphAgent.run() Observable
The base class @ag-ui/langgraph creates an Observable that calls
runAgentStream() but ignores the returned Promise. Any async rejection
becomes an unhandled promise rejection that crashes the process.

Override run() to properly route async errors through the Observable
error channel so downstream catchError handlers can process them.
2026-04-08 16:17:55 -07:00
Max Korp 2142ecea99 fix: use plain UUID for thread-naming runs 2026-04-08 13:52:06 -07:00
Markus Ecker e6988014f8 feat(runtime): OpenGenerativeUI middleware + A2UI config cleanup
Add OpenGenerativeUIMiddleware (clarinet streaming JSON parser, held event
buffering, activity snapshot/delta conversion), remove a2ui boolean shorthand,
includeBasicCatalog defaults to false, before_agent SystemMessage ID reuse fix.
2026-04-08 12:50:18 -07:00
Alem Tuzlak 990acb46ab fix(runtime): address CR round 1 findings
- Fix all doc imports to use @copilotkit/runtime/v2 (symbols only
  exported from v2 entry point)
- Replace v1 copilotRuntimeNextJSAppRouterEndpoint with v2 pattern
  (CopilotRuntime + createCopilotEndpoint + InMemoryAgentRunner)
- Add warning callout on TanStack reasoning example (reasoning events
  not surfaced by TanStack converter)
- Fix randomUUID imports in converters: use @copilotkit/shared instead
  of Node.js crypto to match codebase convention
- Fix collectEvents timer leak in test-helpers.ts (add clearTimeout)
- Fix JSDoc/comments: convertInputToTanStackAI accurately describes
  role allowlist, TanStackInputResult.messages comment updated
2026-04-08 18:03:44 +02:00
Alem Tuzlak a8715b9631 style(runtime): fix formatting in agent test files 2026-04-08 17:40:47 +02:00
Alem Tuzlak 9b15a7c378 fix(runtime): address PR review feedback
- Close TOCTOU window: set abortController synchronously before Observable
  creation in classic run(), matching factory run() pattern (M3)
- Add concurrent run guard to classic run() (C1)
- Add threadId/runId to RUN_ERROR events in classic mode (I2)
- Check both 'output' and 'result' property names in classic tool-result (C2/M2)
- Add try/catch around JSON.stringify in classic tool-result (I4/M2)
- Add undefined guards before emitting state snapshot/delta events (I3)
- Guard assignToolsToAgents against factory-mode agent configs (M1)
- Improve error handling in classic error case with proper fallback (C3)
- Improve TanStack converter TODO comment with known gap details (M5)
- Document @ts-expect-error on clone() middlewares access (S1)
- Add JSDoc to AgentFactoryContext.abortController (M4)
- Fix duplicate import and clean up react-router example
- Update docs AgentFactoryContext reference
2026-04-08 17:32:50 +02:00
Alem Tuzlak 920640cb5a test(runtime): update Agent tests to use BuiltInAgent factory mode 2026-04-08 17:15:00 +02:00
Alem Tuzlak 802d20e95d refactor(runtime): remove Agent class, absorb into BuiltInAgent 2026-04-08 17:14:59 +02:00
Alem Tuzlak 3fcf2fbaf9 feat(runtime): add factory-path branch to BuiltInAgent.run()
- Add converter imports (convertAISDKStream, convertTanStackStream) to agent/index.ts
- Update canOverride() to return false for factory configs
- Add factory-path early return in run() that delegates to runFactory()
- Add private runFactory() method that handles aisdk/tanstack/custom factory configs
  with full lifecycle management (RUN_STARTED, RUN_FINISHED, RUN_ERROR, abort support)
- Fix copilot-runtime.ts: change BuiltInAgentConfiguration["tools"] indexed access
  types to BuiltInAgentClassicConfig["tools"] since tools only exist on classic configs
2026-04-08 17:14:58 +02:00
Alem Tuzlak d320a5217c feat(runtime): add factory config types to BuiltInAgentConfiguration union 2026-04-08 17:14:56 +02:00
Alem Tuzlak 7e69300537 fix(runtime): address CR loop round 1 findings
Source fixes:
- Fix TOCTOU race in concurrent run guard — set abortController
  synchronously before Observable creation, closing the window where
  two run() calls could both pass the guard
- Add JSON.stringify try/catch in TanStack TOOL_CALL_RESULT handler
  (matching existing aisdk.ts protection against circular refs)
- Use role allowlist (user/assistant/tool) instead of denylist in
  convertInputToTanStackAI to exclude activity/reasoning roles
- Change error+abort handler from break to return in aisdk converter
  to stop processing after error during abort
- Fix version-pinned comments (remove "AI SDK 5.0" reference)

Test fixes:
- Add timedOut flag to collectEventsIncludingErrors to distinguish
  hung observables from normal completion
- Add clearTimeout on success/error to prevent timer leaks
- Remove timing-dependent setTimeout in concurrent run guard test

Docs fixes:
- Split imports: copilotRuntimeNextJSAppRouterEndpoint from
  @copilotkit/runtime, Agent/converters from @copilotkit/runtime/v2
- Fix forwardedProps example: use CopilotKit properties prop
- Fix useCopilotChat → useCoAgent for state management reference
- Fix model syntax colon → slash (openai/gpt-4o)
2026-04-08 17:14:55 +02:00
Alem Tuzlak 4e33d42fb6 refactor(tests): test-hygiene fixes for Agent test files
- Replace Record<string, unknown> return types on TanStack mock chunk
  builders with inferred types via as const
- Remove dynamic imports in converter-aisdk error tests — use
  top-level collectEventsIncludingErrors import instead
- Remove redundant as RunAgentInput casts in converter-tanstack-input
  tests (createDefaultInput already returns the correct type)
- Add comments explaining necessary casts in createAgent overloads
  (TypeScript limitation with discriminated union narrowing)
2026-04-08 17:14:54 +02:00
Alem Tuzlak b344d08e17 fix(runtime): address code review feedback on Agent class
Critical fixes:
- Guard concurrent run() calls — throw if agent is already running
- Check both "output" and "result" properties on AI SDK tool-result events
  for compatibility with older AI SDK versions
- Handle undefined/missing error in AI SDK error events with proper
  Error wrapping instead of throwing undefined

Important fixes:
- Add TOOL_CALL_RESULT handling to TanStack converter (was silently dropped)
- Include threadId and runId in RUN_ERROR events for run correlation
- Guard state tool events (STATE_SNAPSHOT/STATE_DELTA) against undefined
  snapshot/delta fields before emitting
- Wrap JSON.stringify in tool-result with try/catch to handle circular refs
- Fix collectEventsIncludingErrors to distinguish error vs normal completion

Test coverage:
- Add tests for convertInputToTanStackAI (message filtering, tool call
  mapping, context injection, state serialization)
- Add tests for state management tool results in AI SDK converter
- Add tests for async factory (Promise-returning) paths
- Add tests for tool-result property compatibility (output vs result)
- Add tests for error event handling edge cases
- Add tests for concurrent run guard and RUN_ERROR correlation fields
- Replace flaky setTimeout-based abort test with signal-based sync

Other:
- Add exhaustiveness check (default: never) in config type switch
- Document clone() @ts-expect-error coupling with AbstractAgent.middlewares
- Fix docs: remove incorrect "v6" version reference, fix wrong hook name
2026-04-08 17:14:52 +02:00
Alem Tuzlak 051f55a7de style: apply prettier formatting to new Agent files 2026-04-08 17:14:49 +02:00
Alem Tuzlak 31919c4cf4 refactor(test): replace type casts with eventField in converter tests 2026-04-08 17:14:48 +02:00
Alem Tuzlak ebfd2c33fc refactor(test): remove all type casts from Agent test files 2026-04-08 17:14:47 +02:00
Alem Tuzlak 5f0a718932 feat(agent): add convertInputToTanStackAI utility for message/prompt conversion 2026-04-08 17:14:45 +02:00
Alem Tuzlak aae03a7ce0 test(agent): add custom converter (passthrough) tests 2026-04-08 17:13:58 +02:00
Alem Tuzlak 8b264f8735 test(agent): add core Agent class tests with parameterized lifecycle/abort/clone 2026-04-08 17:13:57 +02:00
Alem Tuzlak 4b544cc99b test(agent): add AI SDK converter tests 2026-04-08 17:13:55 +02:00
Alem Tuzlak dfcf032b6b test(agent): add TanStack AI converter tests 2026-04-08 17:13:54 +02:00
Alem Tuzlak c07aa13b9e test(agent): add shared test utilities for Agent class 2026-04-08 17:13:52 +02:00
Alem Tuzlak af9de02ee9 fix(agent): support async factory functions in Agent config types 2026-04-08 17:13:51 +02:00
Alem Tuzlak b9b5164033 feat(runtime): add universal Agent class with multi-backend support
Add Agent class extending AbstractAgent that accepts aisdk, tanstack,
or custom configurations via a discriminated union. Routes to the
appropriate converter, handles RUN_STARTED/FINISHED/ERROR lifecycle,
and supports clone/abortRun. Re-export Agent and converters from the
agent barrel.
2026-04-08 17:13:49 +02:00
Alem Tuzlak 3997203e47 feat(runtime): add TanStack AI stream converter and barrel export
Add convertTanStackStream async generator that converts TanStack AI
stream chunks (TEXT_MESSAGE_CONTENT, TOOL_CALL_START/ARGS/END) into
AG-UI events. Create converters/index.ts barrel that re-exports both
AI SDK and TanStack converters.
2026-04-08 17:13:48 +02:00
Alem Tuzlak ddc58b1e1a feat(runtime): extract AI SDK stream converter from BuiltInAgent
Extract the AI SDK fullStream event-mapping loop into a standalone
async generator (convertAISDKStream) that yields AG-UI BaseEvent
objects. This is a pure converter with no lifecycle events — the
caller handles RUN_STARTED/FINISHED/ERROR.
2026-04-08 17:13:46 +02:00
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Alem Tuzlak 773a4a6415 fix: format package.json files 2026-04-07 12:29:29 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00
Alem Tuzlak 136bfc1182 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-07 09:51:46 +02:00
Benjamin Taylor 4094f30989 feat(runtime): Support configuring Threads lock TTL, prefix and heartbeat
feat(runtime): add defaults and max caps for thread lock TTL and heartbeat

lockTtlSeconds defaults to 20s (max 3600s / 1 hour) and
lockHeartbeatIntervalSeconds defaults to 15s (max 3000s / 50 minutes).
Both values are always set, so the heartbeat timer now starts
unconditionally and the paired-option validation is removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 17:17:55 -07:00
Benjamin Taylor da2230ab32 feat(runtime): Add Segment tracking into v2 2026-04-06 16:19:52 -07:00
Alem Tuzlak 18c7a6001d feat: multimodal attachments — UI polish, deprecations, docs, codemod
Attachment queue & previews:
- Image lightbox with View Transition API morph animation
- Video lightbox with native controls and play button overlay
- Document lightbox (PDF via blob URL, text inline, info card fallback)
- Drop zone overlay with upload icon
- Filename preservation via InputContent metadata
- Proper video thumbnail sizing and play/pause indicator
- Fix attachment queue positioning (max-w-3xl constraint)
- Padding between X button and content for audio/document cards
- Document filenames wrap instead of truncating

Attachments config:
- onUploadFailed callback for validation/upload errors (file-too-large, invalid-type, upload-failed)
- onUpload accepts sync or async returns
- AttachmentUploadResult discriminated union with explicit interfaces
- Metadata field on Attachment and onUpload return type

AG-UI version bump:
- Bump @ag-ui/client, @ag-ui/core, @ag-ui/encoder, @ag-ui/proto to 0.0.51
- Remove process.env Vite workaround (fixed upstream in 0.0.51)

Deprecation lifecycle:
- @deprecated JSDoc on all legacy image upload APIs
- ImageRenderer, ImageRendererProps, ImageUpload type, imageUploadsEnabled prop,
  inputFileAccept prop, ImageRenderer prop, AIMessage.image, ImageData
- Codemod at codemods/migrate-attachments.ts (15 tests)
- Migration guide updated with codemod instructions and new type shapes

Docs:
- New guide: docs/(root)/multimodal-attachments.mdx
- Updated migration guide with onUpload return type, metadata, codemod section
- Cross-links from prebuilt-components and migration guide
- Label change: "Add photos or files" → "Add attachments"

Tests:
- CopilotChat.attachments.test.tsx — 5 tests for onUploadFailed
- migrate-attachments codemod — 15 tests
2026-04-06 14:55:06 +02:00