Commit Graph

318 Commits

Author SHA1 Message Date
Martha Kelly Schumann 04d73d1dca Merge branch 'main' into claude/langgraph-tests-cleanup-1jVZ3 2026-04-08 08:52:43 -07:00
Alem Tuzlak a8715b9631 style(runtime): fix formatting in agent test files 2026-04-08 17:40:47 +02:00
Alem Tuzlak 9b15a7c378 fix(runtime): address PR review feedback
- Close TOCTOU window: set abortController synchronously before Observable
  creation in classic run(), matching factory run() pattern (M3)
- Add concurrent run guard to classic run() (C1)
- Add threadId/runId to RUN_ERROR events in classic mode (I2)
- Check both 'output' and 'result' property names in classic tool-result (C2/M2)
- Add try/catch around JSON.stringify in classic tool-result (I4/M2)
- Add undefined guards before emitting state snapshot/delta events (I3)
- Guard assignToolsToAgents against factory-mode agent configs (M1)
- Improve error handling in classic error case with proper fallback (C3)
- Improve TanStack converter TODO comment with known gap details (M5)
- Document @ts-expect-error on clone() middlewares access (S1)
- Add JSDoc to AgentFactoryContext.abortController (M4)
- Fix duplicate import and clean up react-router example
- Update docs AgentFactoryContext reference
2026-04-08 17:32:50 +02:00
Alem Tuzlak 920640cb5a test(runtime): update Agent tests to use BuiltInAgent factory mode 2026-04-08 17:15:00 +02:00
Alem Tuzlak 802d20e95d refactor(runtime): remove Agent class, absorb into BuiltInAgent 2026-04-08 17:14:59 +02:00
Alem Tuzlak 3fcf2fbaf9 feat(runtime): add factory-path branch to BuiltInAgent.run()
- Add converter imports (convertAISDKStream, convertTanStackStream) to agent/index.ts
- Update canOverride() to return false for factory configs
- Add factory-path early return in run() that delegates to runFactory()
- Add private runFactory() method that handles aisdk/tanstack/custom factory configs
  with full lifecycle management (RUN_STARTED, RUN_FINISHED, RUN_ERROR, abort support)
- Fix copilot-runtime.ts: change BuiltInAgentConfiguration["tools"] indexed access
  types to BuiltInAgentClassicConfig["tools"] since tools only exist on classic configs
2026-04-08 17:14:58 +02:00
Alem Tuzlak d320a5217c feat(runtime): add factory config types to BuiltInAgentConfiguration union 2026-04-08 17:14:56 +02:00
Alem Tuzlak 7e69300537 fix(runtime): address CR loop round 1 findings
Source fixes:
- Fix TOCTOU race in concurrent run guard — set abortController
  synchronously before Observable creation, closing the window where
  two run() calls could both pass the guard
- Add JSON.stringify try/catch in TanStack TOOL_CALL_RESULT handler
  (matching existing aisdk.ts protection against circular refs)
- Use role allowlist (user/assistant/tool) instead of denylist in
  convertInputToTanStackAI to exclude activity/reasoning roles
- Change error+abort handler from break to return in aisdk converter
  to stop processing after error during abort
- Fix version-pinned comments (remove "AI SDK 5.0" reference)

Test fixes:
- Add timedOut flag to collectEventsIncludingErrors to distinguish
  hung observables from normal completion
- Add clearTimeout on success/error to prevent timer leaks
- Remove timing-dependent setTimeout in concurrent run guard test

Docs fixes:
- Split imports: copilotRuntimeNextJSAppRouterEndpoint from
  @copilotkit/runtime, Agent/converters from @copilotkit/runtime/v2
- Fix forwardedProps example: use CopilotKit properties prop
- Fix useCopilotChat → useCoAgent for state management reference
- Fix model syntax colon → slash (openai/gpt-4o)
2026-04-08 17:14:55 +02:00
Alem Tuzlak 4e33d42fb6 refactor(tests): test-hygiene fixes for Agent test files
- Replace Record<string, unknown> return types on TanStack mock chunk
  builders with inferred types via as const
- Remove dynamic imports in converter-aisdk error tests — use
  top-level collectEventsIncludingErrors import instead
- Remove redundant as RunAgentInput casts in converter-tanstack-input
  tests (createDefaultInput already returns the correct type)
- Add comments explaining necessary casts in createAgent overloads
  (TypeScript limitation with discriminated union narrowing)
2026-04-08 17:14:54 +02:00
Alem Tuzlak b344d08e17 fix(runtime): address code review feedback on Agent class
Critical fixes:
- Guard concurrent run() calls — throw if agent is already running
- Check both "output" and "result" properties on AI SDK tool-result events
  for compatibility with older AI SDK versions
- Handle undefined/missing error in AI SDK error events with proper
  Error wrapping instead of throwing undefined

Important fixes:
- Add TOOL_CALL_RESULT handling to TanStack converter (was silently dropped)
- Include threadId and runId in RUN_ERROR events for run correlation
- Guard state tool events (STATE_SNAPSHOT/STATE_DELTA) against undefined
  snapshot/delta fields before emitting
- Wrap JSON.stringify in tool-result with try/catch to handle circular refs
- Fix collectEventsIncludingErrors to distinguish error vs normal completion

Test coverage:
- Add tests for convertInputToTanStackAI (message filtering, tool call
  mapping, context injection, state serialization)
- Add tests for state management tool results in AI SDK converter
- Add tests for async factory (Promise-returning) paths
- Add tests for tool-result property compatibility (output vs result)
- Add tests for error event handling edge cases
- Add tests for concurrent run guard and RUN_ERROR correlation fields
- Replace flaky setTimeout-based abort test with signal-based sync

Other:
- Add exhaustiveness check (default: never) in config type switch
- Document clone() @ts-expect-error coupling with AbstractAgent.middlewares
- Fix docs: remove incorrect "v6" version reference, fix wrong hook name
2026-04-08 17:14:52 +02:00
Alem Tuzlak 051f55a7de style: apply prettier formatting to new Agent files 2026-04-08 17:14:49 +02:00
Alem Tuzlak 31919c4cf4 refactor(test): replace type casts with eventField in converter tests 2026-04-08 17:14:48 +02:00
Alem Tuzlak ebfd2c33fc refactor(test): remove all type casts from Agent test files 2026-04-08 17:14:47 +02:00
Alem Tuzlak 5f0a718932 feat(agent): add convertInputToTanStackAI utility for message/prompt conversion 2026-04-08 17:14:45 +02:00
Alem Tuzlak aae03a7ce0 test(agent): add custom converter (passthrough) tests 2026-04-08 17:13:58 +02:00
Alem Tuzlak 8b264f8735 test(agent): add core Agent class tests with parameterized lifecycle/abort/clone 2026-04-08 17:13:57 +02:00
Alem Tuzlak 4b544cc99b test(agent): add AI SDK converter tests 2026-04-08 17:13:55 +02:00
Alem Tuzlak dfcf032b6b test(agent): add TanStack AI converter tests 2026-04-08 17:13:54 +02:00
Alem Tuzlak c07aa13b9e test(agent): add shared test utilities for Agent class 2026-04-08 17:13:52 +02:00
Alem Tuzlak af9de02ee9 fix(agent): support async factory functions in Agent config types 2026-04-08 17:13:51 +02:00
Alem Tuzlak b9b5164033 feat(runtime): add universal Agent class with multi-backend support
Add Agent class extending AbstractAgent that accepts aisdk, tanstack,
or custom configurations via a discriminated union. Routes to the
appropriate converter, handles RUN_STARTED/FINISHED/ERROR lifecycle,
and supports clone/abortRun. Re-export Agent and converters from the
agent barrel.
2026-04-08 17:13:49 +02:00
Alem Tuzlak 3997203e47 feat(runtime): add TanStack AI stream converter and barrel export
Add convertTanStackStream async generator that converts TanStack AI
stream chunks (TEXT_MESSAGE_CONTENT, TOOL_CALL_START/ARGS/END) into
AG-UI events. Create converters/index.ts barrel that re-exports both
AI SDK and TanStack converters.
2026-04-08 17:13:48 +02:00
Alem Tuzlak ddc58b1e1a feat(runtime): extract AI SDK stream converter from BuiltInAgent
Extract the AI SDK fullStream event-mapping loop into a standalone
async generator (convertAISDKStream) that yields AG-UI BaseEvent
objects. This is a pure converter with no lifecycle events — the
caller handles RUN_STARTED/FINISHED/ERROR.
2026-04-08 17:13:46 +02:00
Alem Tuzlak 4b11b995a2 feat: multimodal attachments — images, audio, video, documents in CopilotChat (#3604)
## Summary

Adds full multimodal attachment support to CopilotKit's v2 chat
components. Users can send images, audio, video, and documents alongside
text messages to any AG-UI-compatible agent.

### Frontend
- **`attachments` prop** on `CopilotChat` — `{ enabled, accept, maxSize,
onUpload, onUploadFailed }`
- **`useAttachments` hook** — reusable, referentially stable hook for
custom chat UIs (all callbacks memoized, zero unnecessary re-renders)
- **Attachment queue** with lightbox previews (View Transition API),
video player, PDF/text document viewer
- **Drag-and-drop**, **clipboard paste**, and **file picker** upload
flows
- **Drop zone overlay** with visual indicator
- **Filename preservation** via `InputContent` metadata
- **`onUploadFailed` callback** with typed error reasons
(`file-too-large`, `invalid-type`, `upload-failed`)
- **`AttachmentUploadResult` discriminated union** for type-safe
`onUpload` returns
- Memoized attachment renderer components (v1 and v2)

### Backend
- `BuiltInAgent` properly converts multimodal `InputContent[]` to Vercel
AI SDK format
- Handles image, audio, video, document, and legacy binary content parts

### Deprecation lifecycle
- `@deprecated` JSDoc on all legacy image upload APIs (9 symbols) with
`@since 1.56.0`
- **Codemod** at `codemods/migrate-attachments.ts` — transforms props +
imports (18 tests)
- Migration guide moved to new **Migration Guides** docs section

### Docs
- New guide: `docs/(root)/multimodal-attachments.mdx`
- Updated migration guide with codemod instructions, new `onUpload`
type, metadata
- Cross-links from prebuilt-components page
- New **Migration Guides** section (moved from troubleshooting)

### AG-UI
- Bumped `@ag-ui/client`, `@ag-ui/core`, `@ag-ui/encoder`,
`@ag-ui/proto` to 0.0.51

### Example
- React Router example with dual agent setup (BuiltInAgent + TanStack
AI)
- Multimodal content properly forwarded to TanStack AI `chat()`

## Test plan

- [x] `useAttachments` hook stability tests — 9 tests (referential
stability, re-render counting, state defaults)
- [x] `CopilotChat.attachments` tests — 5 tests (onUploadFailed for
invalid-type, file-too-large, upload-failed, multiple rejections, valid
files)
- [x] Codemod tests — 18 tests (prop transforms, import renames,
idempotency, edge cases)
- [x] Full react-core suite — 966 tests passing
- [x] Tool call streaming tests pass (messagesMemoKey fingerprint fix)
2026-04-07 14:26:03 +02:00
Alem Tuzlak 773a4a6415 fix: format package.json files 2026-04-07 12:29:29 +02:00
Alem Tuzlak 1317e51302 fix: revert copy-dts script, use typesVersions for legacy TS consumers
Reverts the copy-dts.mjs approach from #3612 in favor of typesVersions
which is the standard Node/TS mechanism for resolving subpath types
under legacy moduleResolution: "node".
2026-04-07 11:39:15 +02:00
Alem Tuzlak 136bfc1182 Merge branch 'main' into feat/multimodal-builtin-agent 2026-04-07 09:51:46 +02:00
Benjamin Taylor 4094f30989 feat(runtime): Support configuring Threads lock TTL, prefix and heartbeat
feat(runtime): add defaults and max caps for thread lock TTL and heartbeat

lockTtlSeconds defaults to 20s (max 3600s / 1 hour) and
lockHeartbeatIntervalSeconds defaults to 15s (max 3000s / 50 minutes).
Both values are always set, so the heartbeat timer now starts
unconditionally and the paired-option validation is removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 17:17:55 -07:00
Benjamin Taylor da2230ab32 feat(runtime): Add Segment tracking into v2 2026-04-06 16:19:52 -07:00
Martha Kelly Schumann 4ebbbe5f2c Merge branch 'main' into claude/langgraph-tests-cleanup-1jVZ3 2026-04-06 09:46:46 -07:00
Alem Tuzlak 18c7a6001d feat: multimodal attachments — UI polish, deprecations, docs, codemod
Attachment queue & previews:
- Image lightbox with View Transition API morph animation
- Video lightbox with native controls and play button overlay
- Document lightbox (PDF via blob URL, text inline, info card fallback)
- Drop zone overlay with upload icon
- Filename preservation via InputContent metadata
- Proper video thumbnail sizing and play/pause indicator
- Fix attachment queue positioning (max-w-3xl constraint)
- Padding between X button and content for audio/document cards
- Document filenames wrap instead of truncating

Attachments config:
- onUploadFailed callback for validation/upload errors (file-too-large, invalid-type, upload-failed)
- onUpload accepts sync or async returns
- AttachmentUploadResult discriminated union with explicit interfaces
- Metadata field on Attachment and onUpload return type

AG-UI version bump:
- Bump @ag-ui/client, @ag-ui/core, @ag-ui/encoder, @ag-ui/proto to 0.0.51
- Remove process.env Vite workaround (fixed upstream in 0.0.51)

Deprecation lifecycle:
- @deprecated JSDoc on all legacy image upload APIs
- ImageRenderer, ImageRendererProps, ImageUpload type, imageUploadsEnabled prop,
  inputFileAccept prop, ImageRenderer prop, AIMessage.image, ImageData
- Codemod at codemods/migrate-attachments.ts (15 tests)
- Migration guide updated with codemod instructions and new type shapes

Docs:
- New guide: docs/(root)/multimodal-attachments.mdx
- Updated migration guide with onUpload return type, metadata, codemod section
- Cross-links from prebuilt-components and migration guide
- Label change: "Add photos or files" → "Add attachments"

Tests:
- CopilotChat.attachments.test.tsx — 5 tests for onUploadFailed
- migrate-attachments codemod — 15 tests
2026-04-06 14:55:06 +02:00
Alem Tuzlak 152d1ade96 style: format code with oxfmt 2026-04-06 14:54:40 +02:00
Alem Tuzlak eeccbd3e36 fix(runtime): upgrade dropped content warnings to errors 2026-04-06 14:54:38 +02:00
Alem Tuzlak b6245d31fc fix(runtime): handle malformed URLs and unrecognized content parts defensively
Wrap all three new URL() call sites in try/catch so a malformed URL string
skips the part with a console.warn rather than crashing message conversion.
Add a default switch case to warn on unrecognized content part types.
Add a test covering the malformed-URL skip behaviour.
2026-04-06 14:54:36 +02:00
Alem Tuzlak c2c048f53f feat(runtime): support multimodal content in BuiltInAgent message conversion 2026-04-06 14:54:29 +02:00
Alem Tuzlak 7de4bfa9cf chore: bump @ag-ui/core and @ag-ui/client to 0.0.50-alpha.0 2026-04-06 14:54:27 +02:00
Tyler Slaton 9c940dcaac Merge branch 'main' into claude/rebase-against-main-PGEp1 2026-04-05 21:46:05 -07:00
Claude 2171c7b855 test: address review feedback on LangGraph tests
- Python: extract track_parent_dispatches() context manager, eliminating
  repeated boilerplate; import AGUIBase directly instead of __bases__[0]
- Python: use named constants for "copilotkit_exit" and "Exit" strings
  with comments explaining why they're literals (source uses them too)
- TS runtime: extract withMockedParentMerge() helper using vi.spyOn for
  automatic cleanup instead of manual try/finally prototype patching
- TS runtime: add makeCustomEvent() factory to centralise as-any casts
- TS runtime: use CustomEventNames constants for all custom event tests
- TS runtime: fix misleading comment on empty string content test
- TS runtime: add callPrivate() docstring noting the any trade-off

https://claude.ai/code/session_01BPMn7zhadhapfyyD8kYeAH
2026-04-03 23:12:52 +00:00
Claude d849fae68b test: add LangGraph integration unit tests + dead code annotations
- Python SDK: 18 new tests for LangGraphAGUIAgent (custom event handling,
  emit filtering, state merging, copilotkit namespace)
- TypeScript SDK: 25 new tests for copilotkitCustomizeConfig and
  convertActionsToDynamicStructuredTools
- TypeScript Runtime: 27 new tests for event-source helpers
  (shouldEmitToolCall, getCurrentMessageId, getCurrentContent, etc.)
- TypeScript Runtime: expanded dispatch-event-filtering tests with
  custom event dispatch (manually_emit_message/tool_call/state, exit)
  and langGraphDefaultMergeState tests
- Dead code annotations: LangGraphAgent class and use_function_call=True
  branch annotated with TODO(ran-review) for Ran to verify

https://claude.ai/code/session_01BPMn7zhadhapfyyD8kYeAH
2026-04-03 22:44:09 +00:00
Maxim 208f9bc32f fix: restore .d.ts declaration files for legacy moduleResolution consumers
Copies .d.cts → .d.ts after each tsdown build so consumers on
moduleResolution "node" (e.g. DocuSign) can resolve types without
changing their tsconfig. Also points the "types" field back to .d.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:55:45 +02:00
Alem Tuzlak e77588a062 chore: fix formatting 2026-04-03 18:50:14 +02:00
Alem Tuzlak 5c7c894969 fix(runtime): address PR review findings for fetch-based runtime
- RouteInfo: discriminated union replaces interface with optional fields,
  eliminating 8 non-null assertions and enabling TypeScript narrowing
- Dispatch: consolidated dispatchMultiRoute and dispatchSingleRoute into
  single dispatchRoute function, removing parallel switch statements
- Hooks: multi-route 404/405 now throw Response to flow through onResponse
  hooks, matching single-route behavior
- Hono CORS: explicit `cors: true` default instead of relying on
  toFetchCorsConfig(undefined) returning {} (confusing truthy path)
- Tests: replaced 22 `platform as any` casts with MockIntelligencePlatform
  interface; removed unused CopilotKitIntelligence imports
2026-04-03 18:41:33 +02:00
Alem Tuzlak 9a6ee39d96 chore(runtime): update package config, CI workflows, and docs
- Add package.json exports for v2/{express,hono,node} subpaths
- Add elysia devDependency and tsdown entry points
- Exclude bun integration tests from vitest config
- Update CI workflows to include runtime-servers test job
- Add runtime-server-adapter docs page
- Add changeset for the fetch-based runtime feature
2026-04-03 18:41:27 +02:00
Alem Tuzlak 7355c9b380 test(runtime): add unit and integration tests for fetch-based runtime
Comprehensive test coverage for the new fetch handler layer:
- Unit tests for fetch-handler, fetch-cors, fetch-router, hooks
- Adapter tests for Express and Hono bridges
- Node.js native HTTP handler tests
- Integration test suites for single and multi-endpoint setups
  across Express, Hono, Node, and Bun/Elysia servers
- Edge case and backward compatibility tests
2026-04-03 18:41:26 +02:00
Alem Tuzlak 1a45d3cb3a feat(runtime): add Express, Hono, and Node.js adapters for fetch handler
Bridge the framework-agnostic fetch handler to specific frameworks:
- Express adapter via express-fetch-bridge.ts (converts req/res to fetch)
- Hono adapter (thin wrapper, Hono already uses Request/Response)
- Node.js native HTTP adapter via node-fetch-handler.ts
- Re-export entry points at @copilotkit/runtime/v2/{express,hono,node}

Simplifies existing Express and Hono endpoints by delegating to the
shared fetch core instead of duplicating routing and CORS logic.
2026-04-03 18:41:24 +02:00
Alem Tuzlak 971c3dc1b5 feat(runtime): add fetch-based core handler, CORS, hooks, and router
Introduce the framework-agnostic fetch handler layer:
- fetch-handler.ts: single Request→Response pipeline for all routes
- fetch-cors.ts: CORS preflight and response header handling
- fetch-router.ts: path-based routing for multi-endpoint setups
- hooks.ts: lifecycle hooks (onBeforeRequest, onAfterResponse, onError)

Also migrates handler error responses from Response.json() to
new Response(JSON.stringify(...)) for broader runtime compatibility
(Node.js <20, CF Workers, Deno).
2026-04-03 18:40:29 +02:00
Alem Tuzlak 83fffea5e8 refactor(runtime): move runtime internals into core/ directory
Move middleware-sse-parser.ts, middleware.ts, and runtime.ts into
a new core/ subdirectory to make room for the fetch-based handler
layer. Updates all import paths in handlers and existing tests.
2026-04-03 18:40:28 +02:00
Tyler Slaton 4f1f132d41 fix: address issues with licensing, formatting and types (#3597) 2026-04-03 09:38:41 -07:00
Ran Shem Tov aae2c5cee1 chore: use latest ag-ui langgraph version 2026-04-03 14:57:45 +02:00
Tyler Slaton ac54391d0f fix: only create license checker for Intelligence runtime, not SSE
createLicenseChecker was called unconditionally in BaseCopilotRuntime,
printing the "No license token configured" banner even for open-source
SSE-mode users who don't use any licensed features. Moved it to
CopilotIntelligenceRuntime only.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:56:31 -07:00