Commit Graph

455 Commits

Author SHA1 Message Date
Alem Tuzlak 5f0a718932 feat(agent): add convertInputToTanStackAI utility for message/prompt conversion 2026-04-08 17:14:45 +02:00
Alem Tuzlak aae03a7ce0 test(agent): add custom converter (passthrough) tests 2026-04-08 17:13:58 +02:00
Alem Tuzlak 8b264f8735 test(agent): add core Agent class tests with parameterized lifecycle/abort/clone 2026-04-08 17:13:57 +02:00
Alem Tuzlak 4b544cc99b test(agent): add AI SDK converter tests 2026-04-08 17:13:55 +02:00
Alem Tuzlak dfcf032b6b test(agent): add TanStack AI converter tests 2026-04-08 17:13:54 +02:00
Alem Tuzlak c07aa13b9e test(agent): add shared test utilities for Agent class 2026-04-08 17:13:52 +02:00
Alem Tuzlak af9de02ee9 fix(agent): support async factory functions in Agent config types 2026-04-08 17:13:51 +02:00
Alem Tuzlak b9b5164033 feat(runtime): add universal Agent class with multi-backend support
Add Agent class extending AbstractAgent that accepts aisdk, tanstack,
or custom configurations via a discriminated union. Routes to the
appropriate converter, handles RUN_STARTED/FINISHED/ERROR lifecycle,
and supports clone/abortRun. Re-export Agent and converters from the
agent barrel.
2026-04-08 17:13:49 +02:00
Alem Tuzlak 3997203e47 feat(runtime): add TanStack AI stream converter and barrel export
Add convertTanStackStream async generator that converts TanStack AI
stream chunks (TEXT_MESSAGE_CONTENT, TOOL_CALL_START/ARGS/END) into
AG-UI events. Create converters/index.ts barrel that re-exports both
AI SDK and TanStack converters.
2026-04-08 17:13:48 +02:00
Alem Tuzlak ddc58b1e1a feat(runtime): extract AI SDK stream converter from BuiltInAgent
Extract the AI SDK fullStream event-mapping loop into a standalone
async generator (convertAISDKStream) that yields AG-UI BaseEvent
objects. This is a pure converter with no lifecycle events — the
caller handles RUN_STARTED/FINISHED/ERROR.
2026-04-08 17:13:46 +02:00
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Alem Tuzlak 773a4a6415 fix: format package.json files 2026-04-07 12:29:29 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00
Alem Tuzlak 136bfc1182 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-07 09:51:46 +02:00
Benjamin Taylor 4094f30989 feat(runtime): Support configuring Threads lock TTL, prefix and heartbeat
feat(runtime): add defaults and max caps for thread lock TTL and heartbeat

lockTtlSeconds defaults to 20s (max 3600s / 1 hour) and
lockHeartbeatIntervalSeconds defaults to 15s (max 3000s / 50 minutes).
Both values are always set, so the heartbeat timer now starts
unconditionally and the paired-option validation is removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 17:17:55 -07:00
Benjamin Taylor da2230ab32 feat(runtime): Add Segment tracking into v2 2026-04-06 16:19:52 -07:00
Martha Kelly Schumann 4ebbbe5f2c Merge branch 'main' into claude/langgraph-tests-cleanup-1jVZ3 2026-04-06 09:46:46 -07:00
Alem Tuzlak 18c7a6001d feat: multimodal attachments — UI polish, deprecations, docs, codemod
Attachment queue & previews:
- Image lightbox with View Transition API morph animation
- Video lightbox with native controls and play button overlay
- Document lightbox (PDF via blob URL, text inline, info card fallback)
- Drop zone overlay with upload icon
- Filename preservation via InputContent metadata
- Proper video thumbnail sizing and play/pause indicator
- Fix attachment queue positioning (max-w-3xl constraint)
- Padding between X button and content for audio/document cards
- Document filenames wrap instead of truncating

Attachments config:
- onUploadFailed callback for validation/upload errors (file-too-large, invalid-type, upload-failed)
- onUpload accepts sync or async returns
- AttachmentUploadResult discriminated union with explicit interfaces
- Metadata field on Attachment and onUpload return type

AG-UI version bump:
- Bump @ag-ui/client, @ag-ui/core, @ag-ui/encoder, @ag-ui/proto to 0.0.51
- Remove process.env Vite workaround (fixed upstream in 0.0.51)

Deprecation lifecycle:
- @deprecated JSDoc on all legacy image upload APIs
- ImageRenderer, ImageRendererProps, ImageUpload type, imageUploadsEnabled prop,
  inputFileAccept prop, ImageRenderer prop, AIMessage.image, ImageData
- Codemod at codemods/migrate-attachments.ts (15 tests)
- Migration guide updated with codemod instructions and new type shapes

Docs:
- New guide: docs/(root)/multimodal-attachments.mdx
- Updated migration guide with onUpload return type, metadata, codemod section
- Cross-links from prebuilt-components and migration guide
- Label change: "Add photos or files" → "Add attachments"

Tests:
- CopilotChat.attachments.test.tsx — 5 tests for onUploadFailed
- migrate-attachments codemod — 15 tests
2026-04-06 14:55:06 +02:00
Alem Tuzlak 152d1ade96 style: format code with oxfmt 2026-04-06 14:54:40 +02:00
Alem Tuzlak eeccbd3e36 fix(runtime): upgrade dropped content warnings to errors 2026-04-06 14:54:38 +02:00
Alem Tuzlak b6245d31fc fix(runtime): handle malformed URLs and unrecognized content parts defensively
Wrap all three new URL() call sites in try/catch so a malformed URL string
skips the part with a console.warn rather than crashing message conversion.
Add a default switch case to warn on unrecognized content part types.
Add a test covering the malformed-URL skip behaviour.
2026-04-06 14:54:36 +02:00
Alem Tuzlak c2c048f53f feat(runtime): support multimodal content in BuiltInAgent message conversion 2026-04-06 14:54:29 +02:00
Alem Tuzlak 7de4bfa9cf chore: bump @ag-ui/core and @ag-ui/client to 0.0.50-alpha.0 2026-04-06 14:54:27 +02:00
Tyler Slaton 9c940dcaac Merge branch 'main' into claude/rebase-against-main-PGEp1 2026-04-05 21:46:05 -07:00
Claude 2171c7b855 test: address review feedback on LangGraph tests
- Python: extract track_parent_dispatches() context manager, eliminating
  repeated boilerplate; import AGUIBase directly instead of __bases__[0]
- Python: use named constants for "copilotkit_exit" and "Exit" strings
  with comments explaining why they're literals (source uses them too)
- TS runtime: extract withMockedParentMerge() helper using vi.spyOn for
  automatic cleanup instead of manual try/finally prototype patching
- TS runtime: add makeCustomEvent() factory to centralise as-any casts
- TS runtime: use CustomEventNames constants for all custom event tests
- TS runtime: fix misleading comment on empty string content test
- TS runtime: add callPrivate() docstring noting the any trade-off

https://claude.ai/code/session_01BPMn7zhadhapfyyD8kYeAH
2026-04-03 23:12:52 +00:00
Claude d849fae68b test: add LangGraph integration unit tests + dead code annotations
- Python SDK: 18 new tests for LangGraphAGUIAgent (custom event handling,
  emit filtering, state merging, copilotkit namespace)
- TypeScript SDK: 25 new tests for copilotkitCustomizeConfig and
  convertActionsToDynamicStructuredTools
- TypeScript Runtime: 27 new tests for event-source helpers
  (shouldEmitToolCall, getCurrentMessageId, getCurrentContent, etc.)
- TypeScript Runtime: expanded dispatch-event-filtering tests with
  custom event dispatch (manually_emit_message/tool_call/state, exit)
  and langGraphDefaultMergeState tests
- Dead code annotations: LangGraphAgent class and use_function_call=True
  branch annotated with TODO(ran-review) for Ran to verify

https://claude.ai/code/session_01BPMn7zhadhapfyyD8kYeAH
2026-04-03 22:44:09 +00:00
Maxim 208f9bc32f fix: restore .d.ts declaration files for legacy moduleResolution consumers
Copies .d.cts → .d.ts after each tsdown build so consumers on
moduleResolution "node" (e.g. DocuSign) can resolve types without
changing their tsconfig. Also points the "types" field back to .d.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:55:45 +02:00
Alem Tuzlak e77588a062 chore: fix formatting 2026-04-03 18:50:14 +02:00
Alem Tuzlak 5c7c894969 fix(runtime): address PR review findings for fetch-based runtime
- RouteInfo: discriminated union replaces interface with optional fields,
  eliminating 8 non-null assertions and enabling TypeScript narrowing
- Dispatch: consolidated dispatchMultiRoute and dispatchSingleRoute into
  single dispatchRoute function, removing parallel switch statements
- Hooks: multi-route 404/405 now throw Response to flow through onResponse
  hooks, matching single-route behavior
- Hono CORS: explicit `cors: true` default instead of relying on
  toFetchCorsConfig(undefined) returning {} (confusing truthy path)
- Tests: replaced 22 `platform as any` casts with MockIntelligencePlatform
  interface; removed unused CopilotKitIntelligence imports
2026-04-03 18:41:33 +02:00
Alem Tuzlak 9a6ee39d96 chore(runtime): update package config, CI workflows, and docs
- Add package.json exports for v2/{express,hono,node} subpaths
- Add elysia devDependency and tsdown entry points
- Exclude bun integration tests from vitest config
- Update CI workflows to include runtime-servers test job
- Add runtime-server-adapter docs page
- Add changeset for the fetch-based runtime feature
2026-04-03 18:41:27 +02:00
Alem Tuzlak 7355c9b380 test(runtime): add unit and integration tests for fetch-based runtime
Comprehensive test coverage for the new fetch handler layer:
- Unit tests for fetch-handler, fetch-cors, fetch-router, hooks
- Adapter tests for Express and Hono bridges
- Node.js native HTTP handler tests
- Integration test suites for single and multi-endpoint setups
  across Express, Hono, Node, and Bun/Elysia servers
- Edge case and backward compatibility tests
2026-04-03 18:41:26 +02:00
Alem Tuzlak 1a45d3cb3a feat(runtime): add Express, Hono, and Node.js adapters for fetch handler
Bridge the framework-agnostic fetch handler to specific frameworks:
- Express adapter via express-fetch-bridge.ts (converts req/res to fetch)
- Hono adapter (thin wrapper, Hono already uses Request/Response)
- Node.js native HTTP adapter via node-fetch-handler.ts
- Re-export entry points at @copilotkit/runtime/v2/{express,hono,node}

Simplifies existing Express and Hono endpoints by delegating to the
shared fetch core instead of duplicating routing and CORS logic.
2026-04-03 18:41:24 +02:00
Alem Tuzlak 971c3dc1b5 feat(runtime): add fetch-based core handler, CORS, hooks, and router
Introduce the framework-agnostic fetch handler layer:
- fetch-handler.ts: single Request→Response pipeline for all routes
- fetch-cors.ts: CORS preflight and response header handling
- fetch-router.ts: path-based routing for multi-endpoint setups
- hooks.ts: lifecycle hooks (onBeforeRequest, onAfterResponse, onError)

Also migrates handler error responses from Response.json() to
new Response(JSON.stringify(...)) for broader runtime compatibility
(Node.js <20, CF Workers, Deno).
2026-04-03 18:40:29 +02:00
Alem Tuzlak 83fffea5e8 refactor(runtime): move runtime internals into core/ directory
Move middleware-sse-parser.ts, middleware.ts, and runtime.ts into
a new core/ subdirectory to make room for the fetch-based handler
layer. Updates all import paths in handlers and existing tests.
2026-04-03 18:40:28 +02:00
Tyler Slaton 4f1f132d41 fix: address issues with licensing, formatting and types (#3597) 2026-04-03 09:38:41 -07:00
Ran Shem Tov aae2c5cee1 chore: use latest ag-ui langgraph version 2026-04-03 14:57:45 +02:00
Tyler Slaton ac54391d0f fix: only create license checker for Intelligence runtime, not SSE
createLicenseChecker was called unconditionally in BaseCopilotRuntime,
printing the "No license token configured" banner even for open-source
SSE-mode users who don't use any licensed features. Moved it to
CopilotIntelligenceRuntime only.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:56:31 -07:00
Tyler Slaton 9dc688be2e fix: stop barrel-exporting license-verifier through shared to avoid bundling Node crypto into client
@copilotkit/shared was doing `export * from "@copilotkit/license-verifier"`,
which pulled the entire license-verifier bundle (including Node's `crypto`)
into any client-side consumer. This caused "Module not found: Can't resolve
'crypto'" errors for users bundling @copilotkit/react-core for the browser.

Now shared only re-exports types (erased at compile time) and a client-safe
`createLicenseContextValue` inlined without crypto. Runtime imports
`createLicenseChecker` and `getLicenseWarningHeader` directly from
`@copilotkit/license-verifier`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:55:58 -07:00
Alem Tuzlak 401a27c981 fix: resolve rebase conflicts and format files
- Remove stale eslint-config-custom refs from package.json files
- Regenerate pnpm-lock.yaml
- Format agentcore and other files with oxfmt
2026-04-02 16:43:34 +02:00
Alem Tuzlak 79ce60c580 chore: migrate from eslint+prettier to oxlint+oxfmt
Replace eslint and prettier with oxlint and oxfmt for faster linting
and formatting across the monorepo. Remove all eslint and prettier
configs, dependencies, and related packages. Add .oxlintrc.json and
.oxfmtrc.json for the new tooling. Update CI workflows and lefthook
hooks accordingly. Reformat codebase with oxfmt.

https://claude.ai/code/session_01GMkSf29p78HuMR1mbXn8He
2026-04-02 16:39:05 +02:00
Claude 0d9e3b88f6 chore(post-release): update version to 1.55.0-next.8
The previous publish workflow released 1.55.0-next.8 to npm but the
post-release commit failed to push to main (blocked by branch
protection). This applies the version bumps and pre.json updates that
the workflow would have committed, unblocking future releases.

https://claude.ai/code/session_01N8sLK9h9p6FdRjyHqzKzVV
2026-04-02 02:23:27 +00:00
Alem Tuzlak 1c255f1381 feat(runtime): include agent capabilities in /info response 2026-04-01 18:27:53 +02:00
Alem Tuzlak 6c25fbd8dc feat(runtime): implement getCapabilities() on BuiltInAgent 2026-04-01 18:24:30 +02:00
Alem Tuzlak 232c9eb994 chore: bump @ag-ui/core and @ag-ui/client to 0.0.49 2026-04-01 18:18:40 +02:00
Claude 5d612556d4 fix: address flaky CI unit tests
- Add fail-fast: false to unit test matrix strategies so one Node
  version failing doesn't cancel all other matrix jobs
- Add retry: 2 to runtime-client-gql vitest config (Nx confirmed
  this package's tests as flaky during parallel execution)
- Replace fragile setTimeout waits in middleware express tests with
  vi.waitFor() polling, which is resilient to CI timing variance

https://claude.ai/code/session_01W2Kb2HXsjZett3HdqM9zay
2026-04-01 11:50:56 +02:00
Martha Schumann 9060f2a930 fix(runtime): address code review feedback on reasoning lifecycle fix
- Call closeReasoningIfOpen() once before the switch instead of in every case branch
- Remove as-casts from closeReasoningIfOpen by using typed const variables
- Remove as-casts from new test code; use typed imports and unknown-safe casts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 08:14:45 -07:00
Martha Schumann 5aa901f122 test(agent): strengthen reasoning lifecycle test coverage
- Add cardinality assertions (toHaveLength(1)) to all 5 auto-close tests
  so duplicate REASONING_MESSAGE_END / REASONING_END events are caught
- Add missing RUN_FINISHED final-event assertion to abort auto-close test
- Add test for consecutive reasoning blocks with no reasoning-end between
  them (exercises closeReasoningIfOpen inside reasoning-start)
- Add test for exception thrown mid-stream (exercises catch block path)
- Add test for stream exhaustion without terminal event while reasoning is
  open (exercises !terminalEventEmitted fallback)
- Expand changeset description to cover all three behavioral fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 13:09:15 -07:00
Martha Schumann f78bb6d9b0 fix(agent): make reasoning-end idempotent, close reasoning in catch block
- reasoning-end case now delegates to closeReasoningIfOpen() so a late
  SDK-emitted reasoning-end is a no-op when auto-close already fired,
  preventing duplicate REASONING_MESSAGE_END + REASONING_END events
- Hoist isInReasoning / reasoningMessageId / closeReasoningIfOpen before
  the try block so the catch block can call closeReasoningIfOpen() --
  exceptions thrown mid-reasoning now emit proper lifecycle events before
  RUN_ERROR instead of leaking an unclosed reasoning state
- Remove dead (part as any).delta fallback in reasoning-delta case; the
  SDK always provides part.text in this branch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 13:09:14 -07:00
Martha Schumann fb63dc0cc9 fix(agent): regenerate reasoningMessageId for consecutive blocks with no SDK id
When closeReasoningIfOpen() auto-closes a prior block and a second
reasoning-start arrives with id="0" or no id, the handler was silently
reusing the previous reasoningMessageId. Mirror the text-start pattern:
always call randomUUID() when the provider id is falsy or "0".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 13:09:14 -07:00
Martha Schumann 0b76b2979d fix(agent): close reasoning on consecutive-start, fallback block, abort, and error paths
- Call closeReasoningIfOpen() in the reasoning-start handler before opening
  a new block, guarding against consecutive reasoning blocks without an
  intervening reasoning-end (e.g. Anthropic extended thinking with tools)
- Call closeReasoningIfOpen() in the !terminalEventEmitted fallback block
  so streams that end without a finish/abort/error event still close properly
- Add abort() test-helper and tests for abort + error cases with open reasoning
- Strengthen auto-close ordering assertions to explicitly verify
  REASONING_MESSAGE_END precedes REASONING_END in all three auto-close tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 13:09:14 -07:00