Commit Graph

362 Commits

Author SHA1 Message Date
Alem Tuzlak d320a5217c feat(runtime): add factory config types to BuiltInAgentConfiguration union 2026-04-08 17:14:56 +02:00
Alem Tuzlak 7e69300537 fix(runtime): address CR loop round 1 findings
Source fixes:
- Fix TOCTOU race in concurrent run guard — set abortController
  synchronously before Observable creation, closing the window where
  two run() calls could both pass the guard
- Add JSON.stringify try/catch in TanStack TOOL_CALL_RESULT handler
  (matching existing aisdk.ts protection against circular refs)
- Use role allowlist (user/assistant/tool) instead of denylist in
  convertInputToTanStackAI to exclude activity/reasoning roles
- Change error+abort handler from break to return in aisdk converter
  to stop processing after error during abort
- Fix version-pinned comments (remove "AI SDK 5.0" reference)

Test fixes:
- Add timedOut flag to collectEventsIncludingErrors to distinguish
  hung observables from normal completion
- Add clearTimeout on success/error to prevent timer leaks
- Remove timing-dependent setTimeout in concurrent run guard test

Docs fixes:
- Split imports: copilotRuntimeNextJSAppRouterEndpoint from
  @copilotkit/runtime, Agent/converters from @copilotkit/runtime/v2
- Fix forwardedProps example: use CopilotKit properties prop
- Fix useCopilotChat → useCoAgent for state management reference
- Fix model syntax colon → slash (openai/gpt-4o)
2026-04-08 17:14:55 +02:00
Alem Tuzlak 4e33d42fb6 refactor(tests): test-hygiene fixes for Agent test files
- Replace Record<string, unknown> return types on TanStack mock chunk
  builders with inferred types via as const
- Remove dynamic imports in converter-aisdk error tests — use
  top-level collectEventsIncludingErrors import instead
- Remove redundant as RunAgentInput casts in converter-tanstack-input
  tests (createDefaultInput already returns the correct type)
- Add comments explaining necessary casts in createAgent overloads
  (TypeScript limitation with discriminated union narrowing)
2026-04-08 17:14:54 +02:00
Alem Tuzlak b344d08e17 fix(runtime): address code review feedback on Agent class
Critical fixes:
- Guard concurrent run() calls — throw if agent is already running
- Check both "output" and "result" properties on AI SDK tool-result events
  for compatibility with older AI SDK versions
- Handle undefined/missing error in AI SDK error events with proper
  Error wrapping instead of throwing undefined

Important fixes:
- Add TOOL_CALL_RESULT handling to TanStack converter (was silently dropped)
- Include threadId and runId in RUN_ERROR events for run correlation
- Guard state tool events (STATE_SNAPSHOT/STATE_DELTA) against undefined
  snapshot/delta fields before emitting
- Wrap JSON.stringify in tool-result with try/catch to handle circular refs
- Fix collectEventsIncludingErrors to distinguish error vs normal completion

Test coverage:
- Add tests for convertInputToTanStackAI (message filtering, tool call
  mapping, context injection, state serialization)
- Add tests for state management tool results in AI SDK converter
- Add tests for async factory (Promise-returning) paths
- Add tests for tool-result property compatibility (output vs result)
- Add tests for error event handling edge cases
- Add tests for concurrent run guard and RUN_ERROR correlation fields
- Replace flaky setTimeout-based abort test with signal-based sync

Other:
- Add exhaustiveness check (default: never) in config type switch
- Document clone() @ts-expect-error coupling with AbstractAgent.middlewares
- Fix docs: remove incorrect "v6" version reference, fix wrong hook name
2026-04-08 17:14:52 +02:00
Alem Tuzlak 051f55a7de style: apply prettier formatting to new Agent files 2026-04-08 17:14:49 +02:00
Alem Tuzlak 31919c4cf4 refactor(test): replace type casts with eventField in converter tests 2026-04-08 17:14:48 +02:00
Alem Tuzlak ebfd2c33fc refactor(test): remove all type casts from Agent test files 2026-04-08 17:14:47 +02:00
Alem Tuzlak 5f0a718932 feat(agent): add convertInputToTanStackAI utility for message/prompt conversion 2026-04-08 17:14:45 +02:00
Alem Tuzlak aae03a7ce0 test(agent): add custom converter (passthrough) tests 2026-04-08 17:13:58 +02:00
Alem Tuzlak 8b264f8735 test(agent): add core Agent class tests with parameterized lifecycle/abort/clone 2026-04-08 17:13:57 +02:00
Alem Tuzlak 4b544cc99b test(agent): add AI SDK converter tests 2026-04-08 17:13:55 +02:00
Alem Tuzlak dfcf032b6b test(agent): add TanStack AI converter tests 2026-04-08 17:13:54 +02:00
Alem Tuzlak c07aa13b9e test(agent): add shared test utilities for Agent class 2026-04-08 17:13:52 +02:00
Alem Tuzlak af9de02ee9 fix(agent): support async factory functions in Agent config types 2026-04-08 17:13:51 +02:00
Alem Tuzlak b9b5164033 feat(runtime): add universal Agent class with multi-backend support
Add Agent class extending AbstractAgent that accepts aisdk, tanstack,
or custom configurations via a discriminated union. Routes to the
appropriate converter, handles RUN_STARTED/FINISHED/ERROR lifecycle,
and supports clone/abortRun. Re-export Agent and converters from the
agent barrel.
2026-04-08 17:13:49 +02:00
Alem Tuzlak 3997203e47 feat(runtime): add TanStack AI stream converter and barrel export
Add convertTanStackStream async generator that converts TanStack AI
stream chunks (TEXT_MESSAGE_CONTENT, TOOL_CALL_START/ARGS/END) into
AG-UI events. Create converters/index.ts barrel that re-exports both
AI SDK and TanStack converters.
2026-04-08 17:13:48 +02:00
Alem Tuzlak ddc58b1e1a feat(runtime): extract AI SDK stream converter from BuiltInAgent
Extract the AI SDK fullStream event-mapping loop into a standalone
async generator (convertAISDKStream) that yields AG-UI BaseEvent
objects. This is a pure converter with no lifecycle events — the
caller handles RUN_STARTED/FINISHED/ERROR.
2026-04-08 17:13:46 +02:00
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Alem Tuzlak 773a4a6415 fix: format package.json files 2026-04-07 12:29:29 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00
Alem Tuzlak 136bfc1182 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-07 09:51:46 +02:00
Benjamin Taylor 4094f30989 feat(runtime): Support configuring Threads lock TTL, prefix and heartbeat
feat(runtime): add defaults and max caps for thread lock TTL and heartbeat

lockTtlSeconds defaults to 20s (max 3600s / 1 hour) and
lockHeartbeatIntervalSeconds defaults to 15s (max 3000s / 50 minutes).
Both values are always set, so the heartbeat timer now starts
unconditionally and the paired-option validation is removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 17:17:55 -07:00
Benjamin Taylor da2230ab32 feat(runtime): Add Segment tracking into v2 2026-04-06 16:19:52 -07:00
Martha Kelly Schumann 4ebbbe5f2c Merge branch 'main' into claude/langgraph-tests-cleanup-1jVZ3 2026-04-06 09:46:46 -07:00
Alem Tuzlak 18c7a6001d feat: multimodal attachments — UI polish, deprecations, docs, codemod
Attachment queue & previews:
- Image lightbox with View Transition API morph animation
- Video lightbox with native controls and play button overlay
- Document lightbox (PDF via blob URL, text inline, info card fallback)
- Drop zone overlay with upload icon
- Filename preservation via InputContent metadata
- Proper video thumbnail sizing and play/pause indicator
- Fix attachment queue positioning (max-w-3xl constraint)
- Padding between X button and content for audio/document cards
- Document filenames wrap instead of truncating

Attachments config:
- onUploadFailed callback for validation/upload errors (file-too-large, invalid-type, upload-failed)
- onUpload accepts sync or async returns
- AttachmentUploadResult discriminated union with explicit interfaces
- Metadata field on Attachment and onUpload return type

AG-UI version bump:
- Bump @ag-ui/client, @ag-ui/core, @ag-ui/encoder, @ag-ui/proto to 0.0.51
- Remove process.env Vite workaround (fixed upstream in 0.0.51)

Deprecation lifecycle:
- @deprecated JSDoc on all legacy image upload APIs
- ImageRenderer, ImageRendererProps, ImageUpload type, imageUploadsEnabled prop,
  inputFileAccept prop, ImageRenderer prop, AIMessage.image, ImageData
- Codemod at codemods/migrate-attachments.ts (15 tests)
- Migration guide updated with codemod instructions and new type shapes

Docs:
- New guide: docs/(root)/multimodal-attachments.mdx
- Updated migration guide with onUpload return type, metadata, codemod section
- Cross-links from prebuilt-components and migration guide
- Label change: "Add photos or files" → "Add attachments"

Tests:
- CopilotChat.attachments.test.tsx — 5 tests for onUploadFailed
- migrate-attachments codemod — 15 tests
2026-04-06 14:55:06 +02:00
Alem Tuzlak 152d1ade96 style: format code with oxfmt 2026-04-06 14:54:40 +02:00
Alem Tuzlak eeccbd3e36 fix(runtime): upgrade dropped content warnings to errors 2026-04-06 14:54:38 +02:00
Alem Tuzlak b6245d31fc fix(runtime): handle malformed URLs and unrecognized content parts defensively
Wrap all three new URL() call sites in try/catch so a malformed URL string
skips the part with a console.warn rather than crashing message conversion.
Add a default switch case to warn on unrecognized content part types.
Add a test covering the malformed-URL skip behaviour.
2026-04-06 14:54:36 +02:00
Alem Tuzlak c2c048f53f feat(runtime): support multimodal content in BuiltInAgent message conversion 2026-04-06 14:54:29 +02:00
Alem Tuzlak 7de4bfa9cf chore: bump @ag-ui/core and @ag-ui/client to 0.0.50-alpha.0 2026-04-06 14:54:27 +02:00
Tyler Slaton 9c940dcaac Merge branch 'main' into claude/rebase-against-main-PGEp1 2026-04-05 21:46:05 -07:00
Claude 2171c7b855 test: address review feedback on LangGraph tests
- Python: extract track_parent_dispatches() context manager, eliminating
  repeated boilerplate; import AGUIBase directly instead of __bases__[0]
- Python: use named constants for "copilotkit_exit" and "Exit" strings
  with comments explaining why they're literals (source uses them too)
- TS runtime: extract withMockedParentMerge() helper using vi.spyOn for
  automatic cleanup instead of manual try/finally prototype patching
- TS runtime: add makeCustomEvent() factory to centralise as-any casts
- TS runtime: use CustomEventNames constants for all custom event tests
- TS runtime: fix misleading comment on empty string content test
- TS runtime: add callPrivate() docstring noting the any trade-off

https://claude.ai/code/session_01BPMn7zhadhapfyyD8kYeAH
2026-04-03 23:12:52 +00:00
Claude d849fae68b test: add LangGraph integration unit tests + dead code annotations
- Python SDK: 18 new tests for LangGraphAGUIAgent (custom event handling,
  emit filtering, state merging, copilotkit namespace)
- TypeScript SDK: 25 new tests for copilotkitCustomizeConfig and
  convertActionsToDynamicStructuredTools
- TypeScript Runtime: 27 new tests for event-source helpers
  (shouldEmitToolCall, getCurrentMessageId, getCurrentContent, etc.)
- TypeScript Runtime: expanded dispatch-event-filtering tests with
  custom event dispatch (manually_emit_message/tool_call/state, exit)
  and langGraphDefaultMergeState tests
- Dead code annotations: LangGraphAgent class and use_function_call=True
  branch annotated with TODO(ran-review) for Ran to verify

https://claude.ai/code/session_01BPMn7zhadhapfyyD8kYeAH
2026-04-03 22:44:09 +00:00
Maxim 208f9bc32f fix: restore .d.ts declaration files for legacy moduleResolution consumers
Copies .d.cts → .d.ts after each tsdown build so consumers on
moduleResolution "node" (e.g. DocuSign) can resolve types without
changing their tsconfig. Also points the "types" field back to .d.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:55:45 +02:00
Alem Tuzlak e77588a062 chore: fix formatting 2026-04-03 18:50:14 +02:00
Alem Tuzlak 5c7c894969 fix(runtime): address PR review findings for fetch-based runtime
- RouteInfo: discriminated union replaces interface with optional fields,
  eliminating 8 non-null assertions and enabling TypeScript narrowing
- Dispatch: consolidated dispatchMultiRoute and dispatchSingleRoute into
  single dispatchRoute function, removing parallel switch statements
- Hooks: multi-route 404/405 now throw Response to flow through onResponse
  hooks, matching single-route behavior
- Hono CORS: explicit `cors: true` default instead of relying on
  toFetchCorsConfig(undefined) returning {} (confusing truthy path)
- Tests: replaced 22 `platform as any` casts with MockIntelligencePlatform
  interface; removed unused CopilotKitIntelligence imports
2026-04-03 18:41:33 +02:00
Alem Tuzlak 9a6ee39d96 chore(runtime): update package config, CI workflows, and docs
- Add package.json exports for v2/{express,hono,node} subpaths
- Add elysia devDependency and tsdown entry points
- Exclude bun integration tests from vitest config
- Update CI workflows to include runtime-servers test job
- Add runtime-server-adapter docs page
- Add changeset for the fetch-based runtime feature
2026-04-03 18:41:27 +02:00
Alem Tuzlak 7355c9b380 test(runtime): add unit and integration tests for fetch-based runtime
Comprehensive test coverage for the new fetch handler layer:
- Unit tests for fetch-handler, fetch-cors, fetch-router, hooks
- Adapter tests for Express and Hono bridges
- Node.js native HTTP handler tests
- Integration test suites for single and multi-endpoint setups
  across Express, Hono, Node, and Bun/Elysia servers
- Edge case and backward compatibility tests
2026-04-03 18:41:26 +02:00
Alem Tuzlak 1a45d3cb3a feat(runtime): add Express, Hono, and Node.js adapters for fetch handler
Bridge the framework-agnostic fetch handler to specific frameworks:
- Express adapter via express-fetch-bridge.ts (converts req/res to fetch)
- Hono adapter (thin wrapper, Hono already uses Request/Response)
- Node.js native HTTP adapter via node-fetch-handler.ts
- Re-export entry points at @copilotkit/runtime/v2/{express,hono,node}

Simplifies existing Express and Hono endpoints by delegating to the
shared fetch core instead of duplicating routing and CORS logic.
2026-04-03 18:41:24 +02:00
Alem Tuzlak 971c3dc1b5 feat(runtime): add fetch-based core handler, CORS, hooks, and router
Introduce the framework-agnostic fetch handler layer:
- fetch-handler.ts: single Request→Response pipeline for all routes
- fetch-cors.ts: CORS preflight and response header handling
- fetch-router.ts: path-based routing for multi-endpoint setups
- hooks.ts: lifecycle hooks (onBeforeRequest, onAfterResponse, onError)

Also migrates handler error responses from Response.json() to
new Response(JSON.stringify(...)) for broader runtime compatibility
(Node.js <20, CF Workers, Deno).
2026-04-03 18:40:29 +02:00
Alem Tuzlak 83fffea5e8 refactor(runtime): move runtime internals into core/ directory
Move middleware-sse-parser.ts, middleware.ts, and runtime.ts into
a new core/ subdirectory to make room for the fetch-based handler
layer. Updates all import paths in handlers and existing tests.
2026-04-03 18:40:28 +02:00
Tyler Slaton 4f1f132d41 fix: address issues with licensing, formatting and types (#3597) 2026-04-03 09:38:41 -07:00
Ran Shem Tov aae2c5cee1 chore: use latest ag-ui langgraph version 2026-04-03 14:57:45 +02:00
Tyler Slaton ac54391d0f fix: only create license checker for Intelligence runtime, not SSE
createLicenseChecker was called unconditionally in BaseCopilotRuntime,
printing the "No license token configured" banner even for open-source
SSE-mode users who don't use any licensed features. Moved it to
CopilotIntelligenceRuntime only.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:56:31 -07:00
Tyler Slaton 9dc688be2e fix: stop barrel-exporting license-verifier through shared to avoid bundling Node crypto into client
@copilotkit/shared was doing `export * from "@copilotkit/license-verifier"`,
which pulled the entire license-verifier bundle (including Node's `crypto`)
into any client-side consumer. This caused "Module not found: Can't resolve
'crypto'" errors for users bundling @copilotkit/react-core for the browser.

Now shared only re-exports types (erased at compile time) and a client-safe
`createLicenseContextValue` inlined without crypto. Runtime imports
`createLicenseChecker` and `getLicenseWarningHeader` directly from
`@copilotkit/license-verifier`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:55:58 -07:00
Alem Tuzlak 401a27c981 fix: resolve rebase conflicts and format files
- Remove stale eslint-config-custom refs from package.json files
- Regenerate pnpm-lock.yaml
- Format agentcore and other files with oxfmt
2026-04-02 16:43:34 +02:00
Alem Tuzlak 79ce60c580 chore: migrate from eslint+prettier to oxlint+oxfmt
Replace eslint and prettier with oxlint and oxfmt for faster linting
and formatting across the monorepo. Remove all eslint and prettier
configs, dependencies, and related packages. Add .oxlintrc.json and
.oxfmtrc.json for the new tooling. Update CI workflows and lefthook
hooks accordingly. Reformat codebase with oxfmt.

https://claude.ai/code/session_01GMkSf29p78HuMR1mbXn8He
2026-04-02 16:39:05 +02:00
Claude 0d9e3b88f6 chore(post-release): update version to 1.55.0-next.8
The previous publish workflow released 1.55.0-next.8 to npm but the
post-release commit failed to push to main (blocked by branch
protection). This applies the version bumps and pre.json updates that
the workflow would have committed, unblocking future releases.

https://claude.ai/code/session_01N8sLK9h9p6FdRjyHqzKzVV
2026-04-02 02:23:27 +00:00
Alem Tuzlak 1c255f1381 feat(runtime): include agent capabilities in /info response 2026-04-01 18:27:53 +02:00
Alem Tuzlak 6c25fbd8dc feat(runtime): implement getCapabilities() on BuiltInAgent 2026-04-01 18:24:30 +02:00