mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Mark
8a6d14b29a
Ports `showcase/integrations/pydantic-ai` — the last pydantic-ai surface still on v1 — to Pydantic AI v2. Refs #6364. Three commits plus a bot formatting fix, best reviewed separately. ## 1. `chore(showcase): port pydantic-ai integration to Pydantic AI v2` - **`requirements.txt`** → `pydantic-ai-slim[ag-ui,openai]==2.22.0`, `ag-ui-protocol==0.1.19`. Drops the `opentelemetry-api<1.44` ceiling from #6374; v2 resolves cleanly against otel 1.44.0, so the workaround is no longer needed. `starlette<1.0.0` is unchanged and satisfies v2's `>=0.46.2`. - **9 `StateDeps` imports** move from `pydantic_ai.ag_ui` (removed in v2) to `pydantic_ai.ui`. - **`agent_server.py`** — `Agent.to_ag_ui()` was removed in 2.0.0, so a `mount_agent()` helper builds the equivalent Starlette sub-app and mounts it. The shape is deliberately identical to what v1's `AGUIApp` produced — a Starlette app whose only route is `POST /`, named `run_agent` — so **all 19 mount paths behave exactly as before, trailing slashes included, and no TypeScript route file changes**. `deps` is constructed **per request**. v1's `run_ag_ui` did `deps = replace(deps, state=state)`, handing each run its own object; v2's adapter does `deps.state = state`, mutating what it is given. A single shared instance under v2 therefore lets concurrent runs overwrite each other's state mid-run. ## 2. `fix(showcase): apply the multimodal provider gate to v2 native content` v2's `AGUIAdapter.load_messages` converts AG-UI attachments to native content types *before* the model boundary; v1 delivered the raw AG-UI part dicts. `_NATIVE_CONTENT` listed `BinaryContent` as a flatten fixpoint, so under v2 inline attachments were waved straight through and the entire provider gate was skipped: - inline PDFs were no longer text-extracted, so raw bytes went to OpenAI - unsupported image subtypes (HEIC/SVG/TIFF) were no longer degraded and reached the provider as images, which fails the turn - missing-mime magic-byte sniffing never ran - `AudioUrl`/`VideoUrl` were neither fixpoints nor classifiable, so they hit the fail-loud raise `BinaryContent` is no longer a fixpoint. `_classify_native_content` maps native content onto the same `(kind, scheme, mime, value)` tuple the AG-UI classifier already produces, so **every existing gate applies unchanged** — no gate logic was rewritten. `audio/*` and `video/*` are named explicitly because `_kind_for` routes them to `"other"`, and a missing mime defaults to `"image"` so the sniffer runs. Net behaviour matches v1: a supported inline image still flattens to an `ImageUrl` data URI, which is why most of the suite went green without touching assertions. Five assertions did change. They checked that state-backing content was still AG-UI `InputContent`, which encoded v1's bridging. They now assert the flatten's output (`ImageUrl`) never appears in state — the leak they were written to guard. The adjacent identity and snapshot checks that prove non-mutation are untouched. ## 3. `fix(showcase): gate url-source content and correct the v1-parity claim` Adversarial review of the first two commits found the gate was only half fixed. `_NATIVE_CONTENT` still short-circuited `ImageUrl` and `DocumentUrl`, which v2 builds from unvetted client input, so url-source attachments bypassed the gate where v1 routed them through it: - an `image/heic` or `image/svg+xml` url reached the provider as `input_image`, which the Responses API rejects — failing the turn - an `audio/mpeg` document url reached it as `input_file` - a blank-mime inline PDF went to the image sniffer instead of text extraction, because `load_messages` collapses `ImageInputContent` and `DocumentInputContent` to the same bare `BinaryContent` and erases the modality v1 defaulted on Native content is now gated **before** the fixpoint check rather than instead of it. `_classify_native_content` returns a tuple only when the gate must act; `None` means provider-safe and falls through to the fixpoint, preserving object identity. `ImageUrl` is gated rather than rerouted so a provider-safe one keeps its identity and any explicit `_media_type`. It also corrected a false claim. The `mount_agent` docstring said routing *and* behaviour were unchanged. Routing is; model input is not. v2 defaults `manage_system_prompt='server'`, so each agent's `system_prompt=` now reaches the model. On v1 it never did — `_agent_graph` emitted system parts only `if not messages` and the AG-UI bridge always supplied history — so **18 of 19 agents had silently dead system prompts on main**. A/B on both versions with the same agent and request: v1 sends 0 system-prompt parts, v2 sends 1. The new behaviour is correct and kept; the docstring now says so. ## Verification Against pydantic-ai 2.22.0, in a venv built from this branch's `requirements.txt`: - **52/52 Python tests pass**, up from 42/52. `test_multimodal_content_mapping.py`'s `importorskip` pointed at the removed `pydantic_ai.ag_ui`, which would have skipped all 43 of its tests **green** under v2; it now targets `pydantic_ai.ui.ag_ui` and uses the public `AGUIAdapter.load_messages` in place of the v1 private helper. - **16/19 mounts** return `200 text/event-stream` with `RUN_STARTED … RUN_FINISHED` and no `RUN_ERROR`, driven through the real app with `TestClient` using trailing-slash URLs as the TS routes do. The other three (`/a2ui_dynamic`, `/beautiful_chat`, `/`) reach tool execution and then fail on a raw `OpenAI()` client constructed inside a tool, which the harness cannot intercept and aimock handles in CI. - **Per-request deps isolation** confirmed on `/shared_state_read_write`: state sent by one request does not appear in the next. `build-check (pydantic-ai)` is green on this branch, and because `requirements.txt` changed, the cached pip layer was invalidated — so that was a **genuine fresh resolve of pydantic-ai 2.22.0 inside the real Dockerfile**, not a cached pass. It also confirms dropping the `opentelemetry-api<1.44` ceiling is safe. ### D6 harness probes — run, with a baseline The behavioural gate is the shared harness D6 probes. No CI job runs them for showcase paths, so they were run locally on both this branch and `main`: | | main (v1) | this branch (v2) | |---|---|---| | passed | **33** / 36 | **34** / 36 | | `reasoning-display` | ✗ `no reasoning-role message rendered within 5000ms` | ✅ **passes** | | `gen-ui-agent` | ✗ `waitForTurnComplete … runStartCount=2, done-signal-missing` | ✗ identical error | | `shared-state-read` | ✗ `Strict mode: 1 candidate fixture(s) skipped by sequence/turn state` | ✗ identical error | ```bash cd showcase AIMOCK_URL_LOCAL=http://localhost:4010 bin/showcase test pydantic-ai --d6 --direct --rebuild --cycle --verbose ``` **The port takes D6 from 33/36 to 34/36.** The two remaining failures are pre-existing on `main` with byte-identical error strings — this branch neither causes nor fixes them, and both are tracked in #6381 rather than blocking here. `gen-ui-agent` is root-caused and is not fixture drift: that demo was never ported to pydantic-ai. `src/agents/gen_ui_agent.py` exists in llamaindex with a real `set_steps` tool but has no counterpart here, the route points at `/gen_ui_tool_based/` (the chart-viz agent), and `set_steps` is declared nowhere in the package. The fixture fabricates `set_steps` calls the backend cannot honour, so pydantic-ai rejects the unknown tool and exhausts its single retry. Confirmed live against real OpenAI: the cell returns plain text, which is correct for the code as written. `reasoning-display` going green is the notable behavioural gain, and it retires a documented v1 limitation. `PARITY_NOTES.md:91-97` justifies omitting the reasoning-message branch of `use-rendered-messages.tsx` on the grounds that "PydanticAI's AG-UI adapter does not emit reasoning content today" — true on v1, false on v2. (That block is stale on two further counts: it cites `@ag-ui/core@0.0.43` where `package.json` pins 0.0.57, and claims `ReasoningMessage` is not exported where it is imported at `reasoning-block.tsx:4`.) Correcting it is tracked on #6364. To be precise about what that proves: **v2 forwards reasoning content where v1 dropped it.** The probe supplies the reasoning channel via its fixture, so what is verified is the forwarding path — adapter → AG-UI stream → frontend renderer — end to end. Whether a given model actually emits a reasoning summary live is a separate matter and outside this port's control: it requires a native reasoning model (`reasoning_agent.py` defaults to `gpt-5`, overridable via `REASONING_MODEL`) and, for summary text, a verified OpenAI organisation. A live run here returned prose with no reasoning block, consistent with the org-verification gate rather than anything in the port. Also verified: the image builds from scratch on v2. Because `requirements.txt` changed, the cached pip layer was invalidated, so `build-check (pydantic-ai)` in CI was a genuine fresh resolve of pydantic-ai 2.22.0 inside the real Dockerfile — which also confirms dropping the `opentelemetry-api<1.44` ceiling is safe. ### CI gate coverage, for the record No CI job exercises this package's runtime behaviour on a PR, on this branch or on `main`: - `test / e2e / dojo` runs from the upstream `ag-ui` checkout (`ref: main`) against upstream example agents, and filters on `packages/**` / `sdk-python/**` - `test_showcase-frontend-matrix.yml` is dispatch-only and builds the integration from `base/` — a frozen-backend React baseline - `showcase_validate.yml` asserts `tests/e2e/` exists with a minimum spec count; it does not run it - the package's own `tests/e2e/` (37 files) is invoked by nothing — per `AGENTS.md` rule 1 the measuring test is the shared harness probe, so that layer is legacy ## Remaining for #6364 Two acceptance criteria are outstanding, which is why this says Refs rather than Closes: - the harness D6 value-test (`bin/showcase test pydantic-ai --d6 --rebuild`), which no CI gate runs for showcase paths - `PARITY_NOTES.md` has 6 version-dependent blocks, 4 of which were already inaccurate against the tree before this PR; left alone deliberately to keep this diff scoped ## Possible follow-up `multimodal_agent.py` still reaches into three private APIs (`pydantic_ai._run_context`, `pydantic_ai.models.wrapper`, `pydantic_ai.models.{ModelRequestParameters,StreamedResponse}`) and subclasses `WrapperModel`, overriding `request`/`count_tokens`/`request_stream`. v2 adds a supported alternative: `AbstractCapability.before_model_request`, which receives a `ModelRequestContext` carrying `messages` and `streaming`. Migrating would delete those private imports and ~85 lines. Deliberately not in this PR — it fixes nothing and would obscure the review.