## What does this PR do?
Closes the naming and documentation half of
[OSS-881](https://linear.app/copilotkit/issue/OSS-881). Paired with
CopilotKit/Intelligence#890, which adds `copilotkit verify` and tightens
the evaluation rubric.
### 1. One name for the Intelligence key
**Three** names for one value were live in CopilotKit's own
documentation, and following the wrong one with a CLI-provisioned
project yields an undefined key:
| Name | Where | Code readers |
| --- | --- | --- |
| `INTELLIGENCE_API_KEY` | what `copilotkit project select` writes; all
34 integration examples; the docs site | 34 |
| `COPILOTKIT_INTELLIGENCE_API_KEY` | 7 Channels package READMEs +
packaged skills | **0** |
| `COPILOTKIT_API_KEY` | `examples/slack`, `examples/teams`, and the
TSDoc on `CopilotKitIntelligence` itself | 2 |
`INTELLIGENCE_API_KEY` wins — it is the name the CLI provisions, and
changing it would break every scaffolded project in the wild.
- `COPILOTKIT_INTELLIGENCE_API_KEY` is **retired outright**. Nothing
ever read it, so there is nothing to keep compatible.
- `COPILOTKIT_API_KEY` stays **readable as a deprecated alias** in the
two examples that consume it, so an existing `.env` keeps working, and
is documented as deprecated everywhere it appears.
The third name was the worst placed: it was in the TSDoc on
`CopilotKitIntelligence`, which is what an IDE shows on hover.
This was not only untidy. The CLI's own `channels-preflight` accepts
`INTELLIGENCE_API_KEY` or `COPILOTKIT_API_KEY` — **not**
`COPILOTKIT_INTELLIGENCE_API_KEY`, the name the Channels READMEs told
people to set. So following a Channels README verbatim made `copilotkit
channels` warn that no runtime API key was present while the key sat
visibly in `.env`. After this PR the documented name is one preflight
accepts.
> [!NOTE]
> `NEXT_PUBLIC_COPILOTKIT_API_KEY` is a **different value** — the legacy
Copilot Cloud public key — and is deliberately left alone.
### 2. A real defect, not just naming skew
`skills/runtime/references/intelligence-mode.md` documented
`organizationId` as a `CopilotKitIntelligence` option, sourced from two
further env names (`COPILOTKIT_INTELLIGENCE_ORG_ID`,
`COPILOTKIT_ORG_ID`).
`CopilotKitIntelligenceConfig` has no such field — the copy-pasteable
sample it appeared in **would not compile**. Removed from the samples,
and the prose telling readers to fetch a value for it corrected. That
file is the only place those two names ever existed, which is very
likely why the failing validation run reported that "the runtime reads
`COPILOTKIT_INTELLIGENCE_API_KEY` and `COPILOTKIT_INTELLIGENCE_ORG_ID`".
### 3. Publish the Intelligence wiring
The wiring instructions existed only inside
`node_modules/@copilotkit/runtime/skills/`, and the only docs pages
mentioning `CopilotKitIntelligence` at all were the two Channels
frontends — so a developer on the plain web path had no page to reach it
from.
Adds **`/premium/connect-your-runtime`**: the wiring itself, how to
confirm the credential is actually consumed, the self-hosted
both-URLs-or-neither rule, and a troubleshooting table. Linked into both
navs, and the skills reference now points at the published page.
### 4. A guard so it cannot drift back
`scripts/validate-intelligence-env-names.ts` (`pnpm
check:intelligence-env-names`), wired to lefthook and a new workflow.
The workflow is **intentionally unfiltered**. The two workflows that
would otherwise cover this both filter: `plugin-skills-check` by
`paths:`, and `static/quality` by `paths-ignore: examples/**` — which is
exactly where the deprecated alias lives. Scoping the job would re-open
the hole it exists to close. Legitimate alias sites live in
`ALIAS_ALLOWLIST`.
## Related PRs and Issues
- [OSS-881](https://linear.app/copilotkit/issue/OSS-881) — needs
**both** PRs; neither closes it alone
- CopilotKit/Intelligence#890 — items 1 and 4 (`copilotkit verify` +
rubric contract 1.3.0)
## Verification
- Full lefthook pre-commit ran green: `check-plugin-skills`, `lint-fix`,
the new `check-intelligence-env-names`, and `test`/`publint`/`attw`
across **25 projects**.
- `examples/slack` `managed.test.ts` extended to cover **both** the
canonical name and the alias fallback, and proven non-vacuous — removing
the fallback turns the new test red.
- The drift guard proven non-vacuous the same way: reintroducing a
retired name fails it, exit 1.
- `oxfmt` and `oxlint` clean on every file touched (0 errors).
## Checklist
- [x] I have read the [Contribution
Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [x] If the PR changes or adds functionality, I have updated the
relevant documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Replaces the Codex CLI harness with **LangChain deep agents (Python)**:
sandboxed shell execution, parallel research subagents, no external
harness binary, our own API key.
All ten beats have been walked in a browser, and the demo now has its
own CI job. **Ready for review.**
## What banking's agent is now
```
banking gpt-5.4, temp 0
│ banking's own prompt · the browser's frontend tools · Intelligence
│ memory tools · render_report for the canvas
└─ expense-analyst gpt-5.6-sol, reasoning_effort=high
│ sandboxed shell (LocalShellBackend) · submit_expense_report
└─ merchant-researcher gpt-5.4, one per merchant
search_merchant (Tavily)
```
Six skins still run in-process as `BuiltInAgent`s. `banking` is an
`HttpAgent` pointed at `agent/`.
Each level exists because something about it must differ from its parent
— prompt, model, reasoning effort, tool set. A single agent had nowhere
to put any of that.
### Why the whole agent moved, not just the expense beat
The obvious design was a second agent id for the long-running beat,
leaving banking's `BuiltInAgent` alone. That was built first, and it
does not survive the requirement *"start the analysis, switch threads,
run other pills, come back to it."*
**Threads are scoped per agent.** `listThreads` takes `agentId` as a
required parameter, and measured against the running app the two lists
were disjoint — `banking` 46 threads, `banking-expenses` 10. The v2
runtime has no handoff mechanism, and `defineTool`'s `execute` takes
`(args)` with no emitter, so a tool cannot stream a multi-minute run
into the conversation. One conversation list means one agent.
### Why the beat is a subagent rather than prose in one prompt
It started as a section appended to banking's prompt. That left per-beat
configuration homeless (model, effort and recursion limit are all
agent-level, and there was one agent), and made banking's
~21,000-character rulebook ride every one of the ~20 model calls the run
makes — re-sending rules about markdown tables while the agent read a
CSV.
Reached as a `CompiledSubAgent`, because a raw `SubAgent` spec has no
`subagents` field and this one needs its own: the per-merchant fan-out
is a headline of the beat, and a flat subagent could only research
serially. A probe confirmed nesting survives — the analyst's `task`
dispatches, the researchers' `search_merchant` calls and the final
report tool all reach `astream_events`.
## The console
One CLI window in the transcript carries the whole run: narration, `$
execute`, `search "…"`, `→ merchant-researcher: …`, results, indented by
depth so a ten-way fan-out reads as a fan-out. Subagent narration is
suppressed from the conversation so the console is the single place the
harness is visible; banking's own replies still render normally.
`shell/subagents/subagent-activity.tsx` subscribes to the agent's
**event stream**. Reading `agent.messages` — the first design — was
wrong twice: messages materialise at the `MESSAGES_SNAPSHOT` (two per
run), so the pane sat still for minutes then filled at once; and
persisted messages carry no `subagentRunId`, so the harness's narration
could not be told from banking's reply. The fold is pure and idempotent,
so the same code serves the live subscription and a replay of the
thread's stored events when a conversation is reopened.
Three heuristics were deleted along the way, each replaced by identity
the protocol already carried:
| heuristic | replaced by |
|---|---|
| console anchored on "the first tool call" | the run's first `task`
call, from **message order** (durable; the event-derived version
rendered one console *per delegation* on a restored thread — six,
measured) |
| `CONSOLE_TOOL_NAMES` allowlist | presence of `subagentRunId` |
| `disable_streaming` on the researchers | the canary's per-lane state |
The message filter suppresses **prose** and keeps **tool calls**: an
agent routinely narrates and calls a tool in one message, and returning
`null` for the whole message hid the report card — the run looked
perfect and ended with nothing to show.
## Canary stack, contained to this app
The subagent surface only exists on the canary line, and a released
`@ag-ui/client` ≤ 0.0.57 rejects `SUBAGENT_*` events **in the HTTP
transport before any middleware runs**, killing the stream. So the demo
leaves the root pnpm workspace and ships its own lockfile:
- `@copilotkit/* 1.68.3-canary.1786716392`
- `@ag-ui/* 0.0.59-canary.1786716392.0`
- `ag-ui-langgraph 0.0.43.dev1786716392` → `ag-ui-protocol
0.1.20.dev1786716392`
That keeps an unreleased protocol out of every other package in the
monorepo.
A 1.62.2-based canary was tried first and **could not compile the app**:
`workspace:*` is not a version, so the demo had no recorded lower bound
on the CopilotKit API it needs, and pointing it at 1.62.2 silently
rewound that API five minors. `OpenGenerativeUIActivityRenderer` — a
public `/v2` export since ~1.63 — was the first thing to break, and
there was no reason to think it was the only one. Rebasing the canary on
1.68.3 collapsed that whole class of risk.
## Verified against the running app
Every row is a measurement, not a claim.
| | |
|---|---|
| Registry shape | `/info`: `banking` = HttpAgent, other six =
BuiltInAgent |
| Sandboxed execution | 8 `execute` calls; agent writes and runs its own
Python |
| Parallel subagents | 6 `SUBAGENT_STARTED`/`FINISHED` pairs, 10
`search_merchant` |
| Canvas beat | `render_report` → a2ui middleware emits `activityType:
a2ui-surface` |
| Frontend tools | given two, picks `showTransactions`, emits no result,
emits **no prose** — the prompt's restraint rule surviving the port |
| Memory (Intelligence) | run is handed `recall_memory`, `save_memory`,
`forget_memory`, `knowledge_base_shell` |
| HITL round-trip | tool call out, answer back in, agent continues |
| Durable background run | client disconnected at 8s; run finished
unattended; thread replayed |
| Thread restore | 61 messages persisted incl. the report tool (an
earlier flat-subagent attempt collapsed to 4) |
| Report correctness | 14 rows, 9 researched, 6 filed with ids read out
of real 201 bodies, totals reconciling against their own rows |
| Run duration | 86s reported vs 98s wall clock |
## Correctness bugs found by running it
Each produced a confident, complete-looking wrong answer rather than an
error — the characteristic failure of a multi-minute agentic beat.
- **Beat 3d was dead, and it took the whole thread with it.** The bet
against this one was right. `ag_ui_langgraph` routes *every* attachment
to the model as an `image_url` block, documents included, so the Q2
invoice was rejected before the first token: `400 Invalid MIME type.
Only image types are supported`. The exception is raised inside the
model node, which kills the SSE stream — the runtime sees `RUN_ERROR:
terminated` with no cause and the browser renders **nothing**: no error
bubble, no failed message. And the crashed run is still checkpointed, so
every later message on that thread replays the rejected content and dies
the same way. One click on the pill killed the conversation permanently;
only restarting the service cleared it. Fixed here by a
`wrap_model_call` middleware that rewrites those blocks into LangChain
standard `file` blocks, and upstream in
[ag-ui#2476](https://github.com/ag-ui-protocol/ag-ui/pull/2476) (both
adapters, plus the return leg so a non-image attachment survives
`MESSAGES_SNAPSHOT`). The middleware is a stopgap with its deletion
condition written into its header — this service installs the adapter
from PyPI, so the upstream fix cannot reach it until published.
- **Totals did not match their own rows.** Every per-row amount matched
the CSV while the headline totals came back $1.00 and $0.20 high: the
model authored them instead of adding them. The card prints the total
directly above the rows it is the sum of. Fixed structurally — derived
in `submit_expense_report`, removed as parameters.
- **`amount` arrived as a string** on all 14 rows, silently defeating
`toLocaleString(…currency…)` so it printed `842.10` with no currency.
- **`merchantKind` non-answers.** With no search tool the model wrote a
bare `"unclear"`; with Tavily live it hedges in prose (`"unknown (likely
bookbindery/bookshop retail, but not established for this exact
merchant)"`), which an exact-match filter passed into a 60-character
label glued to the merchant name. Now rejected on leading token, hedging
language, and a 40-character cap.
- **The run clock reported 333s for a two-minute run.** It took the
oldest open stamp across the process because the tool could not name its
own run; model calls *after* the report re-stamped it and the leftover
became the next run's start. Now keyed per run via an injected
`ToolRuntime`.
- **`graph.with_config({"recursion_limit"})` is silently dropped** by
the AG-UI adapter. The agent completed the entire analysis, streamed
every argument of the final report, then died at LangGraph's default of
25 supersteps.
- **A sync `wrap_model_call` under `astream`** surfaces as a bare
`RUN_ERROR: terminated`, cause only in the service log.
- **`emit_raw_events` defaults to `True`**, piggybacking LangChain's
internal events onto the stream: 27,950,261 bytes → 374,086 with it off,
identical report. Matters because the thread *persists* those events for
replay.
- **`gpt-5.6-sol` + `reasoning_effort` + function tools 400s** on
`/v1/chat/completions`; needs `use_responses_api=True`. The first model
probe missed it by binding no tools — a model probe for an agent has to
bind one.
## Upstream findings (reported separately, not fixed here)
1. **`@copilotkit/runtime` drops `subagentRunId` when persisting
messages.** 2888 of 3026 stream events carry it; 0 of 53 persisted
messages do. Reproduced with Intelligence removed entirely, so it is the
runtime's message shape rather than the platform store — and
`@copilotkit/runtime`'s dist contains no occurrence of the field, while
`@ag-ui/core`, `ag-ui-protocol` and `@copilotkit/core` all model it. One
field threaded through would let the console rebuild from message
history and delete the event-replay seeding here.
2. **`copilotkit` 0.1.95 × `ag-ui-langgraph` 0.0.43** — the FastAPI
endpoint calls `agent.clone()` per request; 0.0.43's base `clone()`
hard-passes three kwargs the SDK subclass does not accept, so **every
request 500s**. Verified with a minimal repro on stock classes and by
reading published wheels (0.0.41/0.0.42 are fine — the window is 0.0.43
only). `sdk-python` requires `>=0.0.42` unbounded, so fresh installs
break, and `examples/showcases/deep-agents{,-finance-erp,-job-search}`
are one `uv lock --upgrade` away. Now open as #6592 (`**kwargs`
passthrough + 9 regression tests incl. a forward-compat guard); it needs
a `0.1.96` bump to reach PyPI before `main.py`'s
`BankingAGUIAgent.clone()` workaround can go — and note `agent/uv.lock`
pins `copilotkit 0.1.95`, so removing the workaround is a re-lock as
well as a delete. That branch also fixes 4 pre-existing failures in the
sdk-python suite from the same root cause — **`test_unit-python-sdk` may
currently be red on `main`; worth checking independently.**
3. **Both LangGraph AG-UI adapters send non-image attachments as
`image_url`**, so a PDF, audio clip or video is rejected on the block
kind — see beat 3d in *Correctness bugs*. Fixed in
[ag-ui#2476](https://github.com/ag-ui-protocol/ag-ui/pull/2476); carried
here as a middleware until that publishes.
Also verified and **not** a problem: pnpm 10.33.4 still applies
`package.json` `pnpm.overrides` despite warning that it ignores them —
the ~70 root overrides including the security pins are live, nothing was
silently unpinned. (Migration to `pnpm-workspace.yaml` is worth doing
anyway, since the installed pnpm is 11.21.0 where the field genuinely is
dropped; branch exists, byte-identical lockfile.)
## What is not done
1. **Docs are stale.** `CLAUDE.md` still says `AgentRegistration` is `{
createAgent: () => BuiltInAgent }` and that the route "builds one
`BuiltInAgent` per registered skin"; the reskin skill says the same in
three places, and `templates.md` scaffolds it. The real type is `() =>
AbstractAgent`. This app has a standing rule that every change answers
whether the skill went stale. It did — the skill's launcher step is
fixed, the `BuiltInAgent` claims are not.
3. **Pre-release dependencies**, now on both halves: the JS canaries in
this app's own `pnpm-lock.yaml`, and `ag-ui-langgraph` /
`ag-ui-protocol` pinned `==` to the matching `.dev` canaries in
`agent/pyproject.toml`. Both move to stable when the subagent work
ships.
4. `pnpm test:e2e` has not been run.
### Closed since this PR was opened
- **The demo has its own CI job** —
`.github/workflows/test_reskinnable-demo.yml`. Nx discovers projects
*through* the pnpm workspace (no `workspaceLayout` in `nx.json`), so
leaving it also left the repo-wide `nx run-many -t build`
(`static_compat.yml`), `-t check-types` (`static_quality.yml`) and `-t
test` (`test_unit.yml`) sweeps, and both static workflows carry
`paths-ignore: ["examples/**"]` besides. One job runs the four gates
(lint, typecheck, 2460 unit tests, build); a second syncs
`agent/uv.lock` with `--frozen` and asserts the subagent *capability*
rather than a version string, so it keeps meaning something after the
pin moves.
The build gate is the one a developer can least run locally: `next
build` corrupts a concurrently running dev server's PostCSS/Turbopack
cache — measured, `globals.css` transforms to garbage, every route 500s,
and a dev-server restart does not clear it because the corruption is on
disk.
**Its first two runs failed, both usefully**, and both bugs were
pre-existing:
1. `pnpm/action-setup@v6.0.10` with `package_json_file:` pointed at this
app still installed the **root's** pnpm 10.33.4 rather than this app's
pinned 10.10.0 — the resolver that ignores `pnpm.overrides`, which is
the only place three of the five `@ag-ui/*` canaries are pinned. Caught
only because the job asserts `pnpm --version` against the pin; it would
otherwise have installed under the wrong resolver and stayed green. Now
activated through corepack, which reads the nearest `package.json`.
2. `pnpm install` in this directory **never installed this app**. pnpm
walks *up* for a workspace root, found the repo's, and installed all 70
monorepo projects (4645 packages) while leaving this directory with no
`node_modules` — the next command failing as `sh: 1: eslint: not found`.
So the app that ships its own lockfile was uninstallable by its own
documented instruction. Fixed by giving it its own
`pnpm-workspace.yaml`; `ignore-workspace=true` in an `.npmrc` does
**not** work (CLI-only in pnpm 10.10, measured).
- **The canary `overrides` now live in `pnpm-workspace.yaml`**, their
supported home, as a side effect of that fix. They no longer depend on
`packageManager: pnpm@10.10.0` being the version that still reads
`package.json` — which matters because `@ag-ui/core`, `@ag-ui/encoder`
and `@ag-ui/proto` are pinned nowhere else and a released `@ag-ui/core`
rejects `SUBAGENT_*` in the HTTP transport.
- **All ten beats walked in a browser**, including 3d — which was
broken, for the reason in *Correctness bugs* above.
- **`./run-demo.sh` starts a complete demo.** It now brings up the
Python agent between the compose wait and `pnpm dev`, guarded on
`/health` so a re-run reuses a live one, and dies with `(cd agent && uv
sync)` when the venv is missing. `./stop-demo.sh` had the mirror gap and
now stops it too — that one mattered more than it looks: the start
script *reuses* a live :8124, so an orphan surviving teardown is
silently adopted by the next cold start, serving whatever code it was
launched with.
- **The agent's Python deps are pinned and locked.** They were not, and
the JS half was — so a colleague's fresh `uv sync` resolved the
*release* `ag-ui-langgraph 0.0.43`, which does not accept
`emit_subagent_events` and ships no subagent symbols. That is this PR's
headline feature, and it would have failed silently: `main.py` sets the
flag as an attribute, so the assignment succeeds against an object
nobody reads and the service starts clean. Now `==` pins plus a
committed `agent/uv.lock`, verified by building a venv from only the
tracked files and replaying a captured Q2-with-PDF payload through it.
- **The README told people to install from the repo root** "as a
workspace package". It is deliberately not one, so a root install did
nothing for this app — and installing in the right place did nothing
either, until the `pnpm-workspace.yaml` above.
## Relationship to #6501
Based on `b94e4bfb5d`, the last commit before Codex appears, which is
**not on `main`**. So this PR carries **9 commits: the 4 here plus the 5
foundation commits it shares with #6501** — harness types + fixture,
prompt/workspace, the OFFSITE-to-fixture invariant guard, the
summary-shape/filing-contract fix, and `POST /transactions`. Whichever
merges first shrinks the other. #6501 and #6565 are deliberately
untouched.
## A demo-design question, not a bug
The fixture's merchant names are invented, so `Cardinal & Ash` — the
prompt's own worked example of *"could be a restaurant or a law firm,
find out"* — cannot be resolved by real web search and stays `unclear`
alongside `Bluebonnet Provisions`. The beat's headline claim is that the
agent researches every merchant; real merchant names in the CSV would
make that land harder.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01JRubZT6AS6LCGkcE2KzcfA
First run (32398188492) failed twice, both usefully:
* `pnpm/action-setup@v6.0.10` with `package_json_file:` pointed at this
app still installed the ROOT's pnpm 10.33.4 rather than the app's pinned
10.10.0, and it cannot take a `version:` alongside a `packageManager`
field. The version assertion caught it instead of the job silently
installing under a resolver that ignores the three override-only
`@ag-ui/*` canary pins. corepack reads the nearest package.json, so it
resolves the app's pin and honours its +sha512 hash. The pnpm store is
now cached explicitly, keyed on this app's lockfile, since setup-node's
implicit pnpm cache needs pnpm installed before it runs.
* The import smoke needs OPENAI_API_KEY. `main.py` calls `build_agent()`
at import time and that raises without it. A placeholder is enough —
constructing a ChatOpenAI does not validate the key or touch the network.
Nothing in CI built or type-checked this app. Leaving the root pnpm
workspace — which was right, it contains the canary line — also removed it
from every repo-wide sweep, because Nx discovers projects THROUGH the
workspace and there is no `workspaceLayout` in `nx.json`:
nx run-many -t build static_compat.yml does not see it
nx run-many -t check-types static_quality.yml does not see it
nx run-many -t test test_unit.yml does not see it
Both static workflows also carry `paths-ignore: ["examples/**"]`, so this
is a new workflow rather than an edit to either. It follows
`test_unit-showcase.yml`, which exists for the same reason applied to a
different directory.
Two jobs.
`gates` runs lint, typecheck, unit (2460 tests) and build, cheapest first,
inside the app because no root task reaches it. Measured locally at ~2
minutes of gates; the 20m budget is install headroom.
The build gate is the one that matters most and the one a developer can
least run: `next build` corrupts a concurrently running dev server's
PostCSS/Turbopack cache — measured, `globals.css` transforms to garbage
and every route 500s, and a dev-server restart does not clear it because
the corruption is on disk. CI is the only safe home for it. It needs no
env: the route constructs one agent per skin at module load and none of
them requires a key (banking's is an `HttpAgent` whose URL is never called
during a build).
`agent-resolve` syncs `agent/uv.lock` with `--frozen` and then asserts the
subagent surface is actually present. That is not hypothetical: until this
branch pinned it, `ag-ui-langgraph>=0.0.43` resolved the RELEASE, which
accepts no `emit_subagent_events` and exports no subagent symbols — the
demo's headline feature, failing silently because the flag is set as an
attribute on an object nobody reads. The assertion is on the CAPABILITY
rather than the version string, since a version assertion goes stale the
moment the pin moves and the surface is what must stay true.
One uncertainty is made into a gate rather than left to trust: this app
pins pnpm@10.10.0 while the root pins pnpm@10.33.4, and the older resolver
is load-bearing here — it still reads `pnpm.overrides` from package.json,
the only place `@ag-ui/core`, `@ag-ui/encoder` and `@ag-ui/proto` are
pinned to the canary (`@ag-ui/client` is a direct dependency; those three
are not). `pnpm/action-setup` is pointed at the app's package.json, and
because an unexpected action input is a WARNING in Actions rather than an
error, a step then compares `pnpm --version` against the pin and fails
loudly if they differ.
No `continue-on-error` and no `|| true` anywhere in the file.
Three names for one value were live in CopilotKit's own documentation, and
following the wrong one with a CLI-provisioned project yields an undefined
key:
- `INTELLIGENCE_API_KEY` — what `copilotkit project select` writes, used by
all 34 integration examples and the docs site.
- `COPILOTKIT_INTELLIGENCE_API_KEY` — the seven Channels package READMEs and
the packaged skills. Nothing ever read it.
- `COPILOTKIT_API_KEY` — the Slack and Teams examples, and the TSDoc on
`CopilotKitIntelligence` itself, which is what an IDE shows on hover.
`INTELLIGENCE_API_KEY` wins, because it is the name the CLI provisions and
changing it would break every scaffolded project in the wild.
`COPILOTKIT_INTELLIGENCE_API_KEY` is retired outright — no code read it.
`COPILOTKIT_API_KEY` stays readable as a deprecated alias in the two
examples that consume it, so an existing `.env` keeps working, and is
documented as deprecated everywhere it appears.
The skills reference also documented `organizationId`, sourced from a fourth
and fifth env name, as a `CopilotKitIntelligence` option. It is not one:
`CopilotKitIntelligenceConfig` has no such field, so the copy-pasteable
sample it appeared in would not compile. Removed from the samples, and the
prose that told readers to fetch a value for it corrected.
The Intelligence wiring itself was published only inside
`node_modules/@copilotkit/runtime/skills/`, and the only docs pages showing
`CopilotKitIntelligence` were the two Channels frontends — so a developer on
the plain web path had no page to reach it from. Adds
`/premium/connect-your-runtime`, which covers the wiring, how to confirm the
credential is actually consumed, and the self-hosted two-URL rule.
`scripts/validate-intelligence-env-names.ts` keeps this from drifting back.
It runs unfiltered in CI on purpose: the two workflows that would otherwise
cover it filter paths, and static/quality ignores `examples/**` — exactly
where the deprecated alias lives.
Every Playwright install in CI passed `--with-deps`, which runs `apt-get
update` before downloading the browser. apt on the runners cannot always
reach azure.archive.ubuntu.com; when it can't it retries for many minutes,
which is long enough to burn a job's whole `timeout-minutes` budget before
a single test runs. GitHub renders that kill as "The operation was
canceled", so it reads as a test failure rather than an infrastructure hang.
Chromium's system libraries are already present on the Ubuntu runner
images, and every one of these steps installs chromium only, so the browser
download is all they need. Six jobs lose their apt dependency:
test_unit, test_e2e-legacy-v1, test_e2e-showcase-on-demand,
test_showcase-frontend-matrix, showcase_eval and showcase_capture-previews.
Ports CopilotKit/website#529 to this repo.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ag-ui's dojo-e2e sets AG_UI_MOCK_WEATHER=1 because the backend
tool-rendering demos call the live open-meteo API, which rate-limits CI's
shared egress IPs and hangs the suites. Our copy of the workflow drifted
and never picked it up, so those suites hit the live API — a latent flake
source, and a trace-divergence source now that the dojo asserts golden
event traces.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dojo's new golden event-trace assertions (ag-ui #2392) compare
STATE_SNAPSHOT metadata exactly, and our e2e run was polluting it.
Injecting LANGSMITH_API_KEY makes langgraph-api force tracing on
(LANGSMITH_CONTROL_PLANE_API_KEY defaults to LANGSMITH_API_KEY, which
sets LANGSMITH_TRACING). The LangSmith client then merges every
LANGSMITH_*/LANGCHAIN_* env var into each run's metadata dict, and
langchain_core hands the tracer the same dict object the run config
streams out — so `langgraph dev`'s LANGSMITH_LANGGRAPH_API_VARIANT=local_dev
leaks into STATE_SNAPSHOT and every same-repo PR fails langgraph-python.
Upstream ag-ui runs these suites keyless against LLMock, which is why its
own CI is green on the same commit. Drop the key to restore parity; the
dojo never needed it, since OPENAI_BASE_URL already points at LLMock.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
## What
Adds a **LangSmith Platform** deploy guide to the CopilotKit docs,
modeled after the existing AWS AgentCore deploy page.
It's a self-contained, agent-side guide: deploy a **LangGraph** or
**Google ADK** agent to the LangSmith Platform, then point the
CopilotKit Runtime at it. LangSmith has no frontend-hosting offering, so
the guide covers only the agent side plus wiring the runtime.
## Pages
- **Canonical:** `deploy/langsmith.mdx` — renders in the Overview →
Deploy sidebar.
- **Per-framework wrappers** (thin, like AgentCore):
- `integrations/langgraph/deploy-langsmith.mdx` → `<Content
framework="langgraph" .../>`
- `integrations/adk/deploy-langsmith.mdx` → `<Content framework="adk"
.../>`
- Both registered in their `meta.json` under a new `---Deploy---`
section.
- **Shared walkthrough snippet:**
`snippets/integrations/langsmith/index.mdx` — single source of truth for
all three pages; framework-aware via the `Content` loader scope. Reuses
the existing `langgraph-platform-deployment-tabs` snippet for the "grab
your deployment URL" step.
## Structure (mirrors agentcore.mdx)
Intro → How it works (ASCII flow `Browser → CopilotKit Runtime →
LangSmith deployment → your agent`) → What you get → Quickstart
`<Steps>` inside a `<TailoredContent>` (deploy-new vs already-deployed)
→ `<Callout>`s for the API key/URL and the LangSmith docs authority →
framework tabs (LangGraph / Google ADK) for the deployable-app step →
Troubleshooting `<Accordions>` → What's next `<Cards>`.
## Registry glue
- Generalized the `Content` MDX component to accept an optional
`partial` prop (defaults to the AgentCore partial; existing AgentCore
wrappers unchanged).
- Registered a `LangGraphPlatformDeploymentTabs` stub so the existing
deployment-tabs snippet is reusable.
## Verification
- Commands/flags (`uv tool install langgraph-cli`, `langgraph new
--template new-langgraph-project-python`, `langgraph deploy
--name/--deployment-type dedicated`, deployment API URL) verified
against the live LangChain quickstart.
- ADK path (`pip install "deployments-wrap-sdk[google-adk]"`,
`saf_sdk.adk` `wrap()` + `LangsmithSessionService`, `langgraph.json`
export) verified against the live [Deploy Google ADK
agents](https://docs.langchain.com/langsmith/deploy-google-adk) guide.
- Runtime wiring (`LangGraphAgent` from `@copilotkit/runtime/langgraph`
with `deploymentUrl` / `graphId` / `langsmithApiKey`) matches the repo's
LangGraph quickstart.
- `oxfmt` (format) clean, `oxlint` exits 0, `tsc` clean; registry /
search-href / link-rewrite tests pass. (Pre-existing failures in this
worktree from an uninstalled `react-icons` and unfetched git-LFS assets
are unrelated.)
## Note (small extra)
The LangGraph `deploy-agentcore.mdx` wrapper already existed but was
orphaned (not in any `meta.json`). The new `---Deploy---` section
surfaces it alongside `deploy-langsmith`, matching how AWS Strands
already exposes it.
Ticket: GROW-540
## Summary
- extend the existing public-skill drift suite to validate maintained
setup assets against the generated public API manifest
- fail when a skill imports an unpublished CopilotKit package or
entrypoint, or a manifest-deprecated API
- run the guard in the existing plugin-skills workflow when skills or
the manifest change
## Why this matters
Coding agents copy these skill assets directly into user projects.
Mirror-sync tests prove that our duplicated skill files match, but they
do not prove that the examples still reference packages and APIs we
actually publish. A stale import can make CopilotKit fail at the first
install or build step, which is exactly the kind of failure that
prevents agents from choosing and successfully adopting us.
This PR adds the smallest deterministic guard for that risk. It reuses
our existing Vitest suite and canonical public API manifest; it does not
introduce an eval harness, run agents, score behavior, collect metrics,
add a provider, or add dependencies.
## Scope
This is package-contract validation, not behavioral evaluation. Broader
questions such as whether an agent follows a skill well, how many
attempts it needs, and whether the generated application behaves
correctly remain separate work and should start with a concrete decision
the deterministic checks cannot answer.
## Verification
- `pnpm exec vitest run scripts/__tests__/public-skill-drift.test.ts
scripts/__tests__/sync-plugin-skills.test.ts` (17 tests)
- `pnpm check:plugin-skills`
- `pnpm check:public-api-manifest`
- targeted TypeScript, oxfmt, and oxlint checks
- mutation check: replacing `BuiltInAgent` with deprecated `BasicAgent`
fails with the manifest-provided replacement
Linear: PDX-320
## What does this PR do?
`@copilotkit/react-native` maintained a **private tool-call render
registry** (`hooks/RenderToolContext.tsx`) alongside the canonical one
that `CopilotKitCoreReact` already provides — and which every React
Native app already ships, unused. This PR deletes the fork and points
React Native at the shared registry.
That fork caused three bugs:
| Bug | Symptom | Cause |
|---|---|---|
| **Tool renders never streamed** | A component registered with
`useRenderTool` / `useComponent` painted nothing until the tool call
completed | `CopilotChat` used `JSON.parse` on the argument buffer.
While a model writes a tool call that buffer is *invalid JSON by design*
— AG-UI delivers `TOOL_CALL_ARGS` deltas that are concatenated
client-side — so the parse threw on every delta, warned, and fell back
to `{}` |
| **`useComponent` rendered nowhere** | Silently, with no error | It
writes to core's registry; React Native's chat read React Native's
private `Map` |
| **Chat history degraded** | Navigating away from the registering
screen turned earlier tool calls into a `Called: <name>` placeholder |
The private `Map` deleted renderers on unmount; core deliberately keeps
them |
`@copilotkit/react-core` has used `partialJSONParse` on this path since
v2 shipped. React Native diverged because `useRenderToolCall` was
excluded from its re-exports on the stated grounds that it "depends on
DOM elements via `DefaultToolCallRenderer`" — a claim that was never
true of the hook itself. It was only ever reachable through the fat
`/v2` entry, whose weight is the real hazard (#4893). #5883 moved it
into `/v2/headless` on 2026-07-23; the exclusion comment was rewritten
the next day without revisiting the reason.
### What changed
- **One registry.** `useRenderTool` registers through `useFrontendTool`
into `CopilotKitCoreReact.renderToolCalls`. `CopilotChat` and any custom
surface consume react-core's `useRenderToolCall`.
- **Types are derived, not declared.** `RenderToolProps` is now
`React.ComponentProps<ReactToolCallRenderer<T>["render"]>`, so React
Native cannot drift from `ReactToolCallRenderer` — the contract every
registered renderer is actually invoked against. Change that contract
and `check-types` names every React Native renderer the change breaks.
React Native narrows only the *return* type to `ReactElement | null`,
which `FlatList`'s `renderItem` genuinely requires.
_Scope of that guarantee (corrected during review):_ it does **not**
extend to the type react-core publicly exports under the same name.
Web's `RenderToolProps<S>`
(`react-core/src/v2/hooks/use-render-tool.tsx`) is a separate
hand-declared union, generic over a schema, carrying arguments under
`parameters` (not `args`) and declaring `status` as string literals
rather than `ToolCallStatus` members. Both divergences are live today
and nothing type-checks them shut — the one place the shapes meet,
react-core's own bridge, compiles because a string-enum member is
assignable to its own literal type but not the reverse. Aligning web's
alias is a breaking web API change, filed separately.
- **`RenderToolContext.tsx` deleted** (−150 lines), along with 15 tests
that described the removed subsystem. One of them — `unregisters the
render function on unmount` — asserted the chat-history bug as a
requirement.
- **Two structural CI guards for #4893**, in opposite directions: a test
failing if any React Native source imports the fat `/v2` entry, and a
script failing if react-core's `/v2/headless` or `/v2/context` chunks
ever link shiki/mermaid/cytoscape/katex/streamdown. Both were verified
able to fail by deliberately introducing the regression. These are
*structural* assertions, not size budgets — `dev-docs/bundle-size.md`
freezes `limit` fields until OSS-122.
- **`react-native` added to the bundle-size glob**, which it had never
been in, plus a `size:headless` measurement.
React Native also gains capabilities it lacked: render props inferred
from your schema, `name`/`toolCallId` on render props, and `result` on
completed calls.
**Corrected during review — two capabilities this originally claimed are
not delivered:**
- **Wildcard (`"*"`) renderers do not work on React Native.** Because
`useRenderTool` routes through `useFrontendTool` (which calls
`addTool`), `name: "*"` registers a frontend tool literally named `*` —
advertised to the model, and colliding with core's separate
wildcard-executable-tool path. react-core's `useRenderTool` is
renderer-only and special-cases the wildcard; React Native's is not. The
guide now advises against it.
- **`followUp` (and `available`) are not forwarded**, and the handler's
`context` argument is dropped, so `stopAgent()`'s abort signal is
unreachable from an RN handler.
Both are tracked in § Known limitations for the follow-up that converges
React Native onto react-core's hooks — deleting RN's `useRenderTool` in
favour of re-exporting `useFrontendTool` (tool + renderer) and
react-core's `useRenderTool` (renderer-only, wildcard-capable). That is
an API change with its own migration note, so it is not in this PR.
### ⚠️ Breaking (in a minor)
`useRenderToolRegistry` and `RenderToolProvider` are **removed**. Both
are documented on the docs site, so this is a real break — see the
`BREAKING CHANGE:` footer on `db67ccf`, which is what the release notes
derive from, plus the rewritten reference pages.
```diff
- const registry = useRenderToolRegistry();
- const renderer = registry.get(toolCall.function.name);
- return renderer ? renderer({ args, status }) : null;
+ const renderToolCall = useRenderToolCall();
+ return renderToolCall({ toolCall });
```
Also note two semantic changes: `args` is `Partial<T>` **only** while
`status` is `"inProgress"`, and a render function is now captured at
registration — if it closes over changing values you must declare them
in `deps` (React Native previously refreshed the closure on every
render).
**Known limitation:** agent-scoped renderer resolution does not take
effect on React Native. `CopilotChatConfigurationProvider` is not in
RN's provider tree, so `agentId` always resolves to the default.
Renderers still resolve by name; two agents registering the same tool
name resolve arbitrarily. Filed separately.
### A data point worth recording
Adding `useRenderToolCall` to the measured headless entry moved the
bundle **92.8 kB → 92.7 kB**. Flat. The hook React Native spent months
not using was already inside the chunk every RN app resolves whole —
Metro doesn't tree-shake, so the fork never saved a byte. It cost them.
### Testing
- `@copilotkit/react-native`: **253 passing / 22 files** ·
`@copilotkit/react-core`: **1480 passing / 123 files** · `check-types`
and `build` green for both.
- Each of the three bugs has a deterministic test driving a real
`CopilotKitCoreReact` — no mocking of the code under test.
- Both #4893 guards carry mutation evidence: introduce the regression,
watch them fail, revert, watch them pass.
### Follow-up
`useRenderTool`'s JSDoc is split across two blocks, which orphans the
primary description from IDE hover (the `@param deps` warning still
surfaces). One-line fix, deliberately left out of the final fix wave.
## Related PRs and Issues
- **Supersedes #6346** (@davidmckayv) — its diagnoses were correct and
its test assertions are ported here, re-driven through the real registry
rather than a mocked local one. Credited via `Co-Authored-By` on
`4104bd1`.
- Addresses the React Native half of **#4893**.
- Builds on **#5883**, which created the lean `/v2/headless` entry this
PR consumes.
## Checklist
- [x] I have read the Contribution Guide
- [x] If the PR changes or adds functionality, I have updated the
relevant documentation
- [x] "Allow edits by maintainers" is checked
🤖 Generated with [Claude Code](https://claude.com/claude-code)
react-native was missing from static_bundle_size.yml's package glob, so its
dist/ has never been measured despite being the consumer most exposed to the
#4893 regression (Metro does not tree-shake). This adds coverage:
- Extend the compressed-size-action glob to include react-native.
- New scripts/measure-headless.mjs: an esbuild-driven gzip signal for the
@copilotkit/react-native/headless entry, mirroring react-core's
measure-copilotchat.mjs (stdin + resolveDir, gzip sum, job-summary output).
- Wire a build + measure step into the copilotchat-import-size CI job.
First baseline: @copilotkit/react-native/headless = 92.8 kB gzip
(esbuild regression signal, not a Metro figure).
No limit fields (Phase 1 policy — see dev-docs/bundle-size.md).
esbuild added as a react-native devDependency (^0.27.0, matching react-core);
the root ">=0.25.4" override keeps the monorepo on a single esbuild (0.27.3).
Two corrections to the drafted script, verified by running it:
- Fed the entry via esbuild stdin with resolveDir=pkgRoot; a temp-dir entry
cannot resolve @copilotkit/react-native/headless through workspace node_modules.
- Dropped useRenderToolCall from the import list — the RN headless surface
deliberately does not export it (DOM-dependent; see src/index.ts).
Co-Authored-By: Claude <noreply@anthropic.com>
## What does this PR do?
Community PRs keep arriving with `.changeset/*.md` files even though the
repo migrated off Changesets to conventional-commit-driven releases.
`.changeset/` has now been deleted from `main` twice (`5afa55f067` on
2026-06-16, `1e5ba689e0` on 2026-07-29) and **five open PRs carry
changeset files today** (#6287, #6289, #6290, #6292, #6346).
Three mechanisms keep feeding it:
1. **Stale forks.** `rodboev/CopilotKit`'s default branch still contains
10 of the pre-cleanup `.changeset/*.md` debris files. Three of the five
open PRs come from that fork — the contributor's agent opens the repo,
sees a directory full of changesets, and adds one more. (No
`config.json`, and `@changesets/cli` isn't installed anywhere, so these
are hand-written by agents, not CLI output.)
2. **Merging stale PRs re-seeds `main`.** The two files Tyler removed in
`1e5ba689e0` arrived via 2026-06-10-authored branches (#2910, #5360)
merged on 2026-07-25 — they sat on `main` for four days, and anyone who
forked in that window inherited the directory. His hunch in that commit
message was right.
3. **Convention inference, uncontradicted.** #6346 is from a branch in
this repo, where `.changeset/` does *not* exist, and it still has one.
The repo reads as a Changesets repo: pnpm workspace monorepo,
per-package `CHANGELOG.md` in Changesets' exact `### Patch Changes`
output format, `chore: release monorepo vX.Y.Z` release PRs. Nothing in
`CONTRIBUTING.md`, the PR template, `AGENTS.md`, `CLAUDE.md`, or
`.claude/docs/` said otherwise, so the guess was well-supported.
This PR closes all three off:
- **`CONTRIBUTING.md`** — new "Changelogs and releases — do not add a
changeset" section: we did use Changesets, `scripts/release/` now builds
changelogs from commit subjects, `.changeset/*.md` is inert, write a
good conventional commit subject instead, and leave versions/changelogs
to maintainers. Includes a note to rebase old forks.
- **`AGENTS.md` / `CLAUDE.md`** — the same rule as an Essentials bullet.
This is the highest-leverage change: the contributors doing this are
coding agents, and agents load these files automatically while mostly
not reading `CONTRIBUTING.md`.
- **`static / check binaries`** — fail the PR on added `.changeset/*`
files, so this stops depending on review catching it (which is what
failed in July and restarted the loop). Added to the existing
forbidden-files gate rather than a new workflow: it already runs on
every PR to `main`, is fork-safe (`contents: read`, no secrets), and has
exactly this `git diff --name-only origin/BASE...HEAD` + `VIOLATIONS`
shape. Filters on `--diff-filter=AM` so a PR that *deletes* stale
changesets still passes.
- **`.oxfmtrc.json`** — drop the ignore entry for
`.github/actions/changesets-action/src/run.ts`, a path that hasn't
existed for a long time. It was the last grep-visible "we use
changesets" signal in a root config file.
## Related PRs and Issues
- Follows up `1e5ba689e0` ("fix: remove all changesets"), whose commit
message asked for exactly this: a durable record of the decision that
future agents can find.
- Open PRs that would be caught by the new gate: #6287, #6289, #6290,
#6292, #6346.
## Testing
Docs + CI-config change, so verification focused on the guard.
`actionlint` was run on the workflow, then the step body was extracted
with `yq` and executed against real branches.
**Lint / parse:**
```
$ actionlint .github/workflows/static_check-binaries.yml
actionlint: clean
$ python3 -c "import json; json.load(open('.oxfmtrc.json'))" # oxfmtrc still valid JSON
oxfmtrc JSON OK
```
**True positive** — real head of #6292, via `yq
'.jobs.check-binaries.steps[1].run'` piped to bash with `BASE_REF=main`:
```
::error::Changeset files detected in PR:
.changeset/enable-mcp-apps-tool-filters.md
This repo no longer uses Changesets — releases are driven by conventional commit subjects (see scripts/release/).
Nothing reads .changeset/*.md. Delete these files and describe the change in your commit subject instead.
See the 'Changelogs and releases' section of CONTRIBUTING.md.
This PR contains files that should not be committed (see the errors above).
Please remove them and update your .gitignore if needed.
exit=1
```
**True negative** — same script on this branch, which has five changed
files and no changesets:
```
$ git diff --name-only origin/main...HEAD
.github/workflows/static_check-binaries.yml
.oxfmtrc.json
AGENTS.md
CLAUDE.md
CONTRIBUTING.md
$ BASE_REF=main bash step.sh
No binary artifacts or oversized files detected.
exit=0
```
**Delete-safety** — a commit that *removes* changesets must not be
punished. Using the real cleanup commit (`8806f668d1...1e5ba689e0`):
```
unfiltered: with --diff-filter=AM (what the gate uses):
.changeset/coalesce-...md (empty)
.changeset/fix-parallel-...md
```
Not verified locally: the gate firing in real GitHub Actions — that
needs this PR's own CI run (the `static / check binaries` check on this
PR exercises the true-negative path).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## What
Adds the CopilotKit side of the [telemetry event
registry](https://github.com/CopilotKit/oss-path-to-production/blob/main/docs/telemetry-registry-publish-roadmap.md):
tooling + CI that generate this repo's registry **fragments** and open
path-limited PRs into `CopilotKit/oss-path-to-production`, where the
reconciler folds them into `telemetry-events.json`.
Two surfaces, two mechanisms (per the surface-owns-its-extractor
design):
| Surface | Events | Extraction | Trigger |
|---|---|---|---|
| **runtime** | 5 `oss.runtime.*` | **bespoke catalog** — reads the
`AnalyticsEvents` type map (names + properties), scans `capture()` sites
for `call_sites`; **fails loud if the v1/v2 catalogs diverge** | stable
**monorepo** release (`on: release`, tag `vX.Y.Z`) |
| **docs** (`showcase/shell-docs`) | 11 | **callee mode** — inline
`posthog.capture("name", {…})` literals; drops `$`-reserved events |
push to `main` touching `showcase/shell-docs/src/**` (excluding
`src/content`) |
## Key properties
- **Content-gated.** The emitter leaves the target fragment
byte-for-byte untouched when the extracted event set is unchanged, so a
PR opens **only when telemetry actually changes** — no per-release /
per-commit churn.
- **Reconciled canonical in every PR.** Both workflows run the
registry's `pnpm reconcile` and commit `telemetry-events.json` alongside
the fragment, matching the registry's shipped emitters — a fragment-only
PR fails its `telemetry-reconcile` staleness gate.
- **Least-privilege cross-repo token.** No explicit `owner` (defaults to
the app installation's org) + bare `repositories:
oss-path-to-production` + `contents`/`pull-requests` write only; mint
gated on a job-level env var (GitHub rejects `secrets.*` in `if:`).
- **zizmor clean** at CI's `--min-severity low` (one `cache-poisoning`
suppression, justified in `.github/zizmor.yml`: the workflow configures
no cache and publishes a PR, not build artifacts).
## Files
- `scripts/telemetry/extract.ts` — pure extraction (callee scan +
catalog reader), deterministic output.
- `scripts/telemetry/emit-fragment.ts` — CLI: `--surface runtime|docs
--out <path>`, assembles + content-gates the fragment.
- `scripts/__tests__/telemetry-fragment.test.ts` — 13 unit tests
(fixtures) + a loose real-catalog drift smoke test.
- `.github/workflows/telemetry-{runtime,docs}-fragment.yml` — the two CI
jobs.
## Testing
Rebased onto `main` (`55aaad21a6`) and revalidated end-to-end on
2026-08-05 — the branch had fallen 1345 commits behind.
**Unit / static**
- `vitest run scripts/__tests__/telemetry-fragment.test.ts` → **13/13
passed**.
- `tsc --noEmit --strict --esModuleInterop` over both scripts →
**clean** (`scripts/` has no tsconfig, so this is the ad-hoc
invocation).
- `oxlint scripts/telemetry` → **0 warnings, 0 errors**; `oxfmt --check`
→ **all files correctly formatted**.
- `zizmor --min-severity low --config .github/zizmor.yml
.github/workflows` (CI's exact invocation) → **No findings to report**
(32 ignored, 233 suppressed).
**Runtime surface — mechanism proven against the live rebased tree**
```
$ tsx scripts/telemetry/emit-fragment.ts --surface runtime --out /tmp/CopilotKit.runtime.json
runtime: wrote 5 events → /tmp/CopilotKit.runtime.json (released_in runtime@1.66.2)
```
Diffed event-for-event against the registry's committed
`CopilotKit.runtime.json`: **semantically identical** (same 5 events,
same `call_sites`, same `properties_seen`) — the only difference is
ordering, since the emitter sorts alphabetically and the hand-seeded
fragment is in catalog-declaration order. Confirmed the reorder is a
no-op at the canonical level (see below), so the first automated run
opens one reordering PR with an empty `telemetry-events.json` diff and
is quiet thereafter.
Also confirmed the catalog is still complete on current `main`: the only
`oss.*` event literals anywhere under `packages/runtime/src` +
`packages/shared/src` are the 5 catalog entries (43/22/12/9/9
occurrences), so no untyped event is being silently dropped. Both v1 and
v2 catalogs remain byte-identical, so the divergence guard passes.
**Docs surface**
```
$ tsx scripts/telemetry/emit-fragment.ts --surface docs --out /tmp/CopilotKit.docs.json
docs: wrote 11 events → /tmp/CopilotKit.docs.json (released_in shell-docs@5855496103)
```
11 events (up from 7 when this PR was authored — the docs site grew):
`cli_command_copied`, `docs_conversion_clicked`,
`docs_conversion_copied`, `docs.framework_selected`,
`docs.frontend_selected`, `docs.journey_continued`,
`hero_command_copied`, `markdown_copied`, `open_in_llm_clicked`,
`talk_to_us_clicked`, `try_for_free_clicked`. `$pageview` correctly
dropped.
**End-to-end against the real registry**
Dropped both emitted fragments into a clean
`oss-path-to-production@main` worktree and ran its own `pnpm reconcile`:
- Both fragments **validate against `fragment.schema.json`** (ajv, via
the reconciler's loader).
- Reconcile succeeded; `telemetry-events.json` grew by 216 lines with 11
new `"surface": "docs"` observations.
- **Zero `oss.runtime.*` entries changed** — confirming the runtime
fragment's reordering has no canonical effect.
## Fixed during revalidation
- **`add-paths` bug in the docs workflow (would have failed on first
run).** It ran `pnpm reconcile` but listed only the fragment in
`add-paths`, so its PR would have landed a fresh fragment beside a stale
`telemetry-events.json` and tripped the registry's `telemetry-reconcile`
staleness gate — the exact failure the runtime workflow was already
fixed for. Verified against the registry's shipped emitters: every
automated fragment PR there (`website.corp` #232/#220, Intelligence
surfaces #228) carries `telemetry-events.json` alongside its fragment.
- **Stale action pins.** Refreshed to the SHAs `main` now uses
everywhere: `actions/checkout` v7, `actions/setup-node` v7.0.0,
`pnpm/action-setup` v6.0.10.
- **Over-broad docs trigger.** Narrowed from `showcase/shell-docs/**` to
the code under `src/**`, excluding `src/content/**` — 1012 MDX + 140
JSON prose files with zero `.ts`/`.tsx`, none of which can hold a
`posthog.capture` call site. Prose edits no longer fire a full monorepo
install.
- **zizmor justification accuracy.** `setup-node` v7 adds a
`package-manager-cache` input defaulting to `true`; per its `action.yml`
it engages only when `package.json` declares **npm**, and this repo
declares pnpm — so the workflow is still cacheless and the suppression
still holds. Noted inline.
## Prerequisite — now satisfied
The registry App secrets (`TELEMETRY_REGISTRY_APP_ID`,
`TELEMETRY_REGISTRY_APP_PRIVATE_KEY`) are configured on this repo (added
2026-07-09), and `app/copilotkit-telemetry-bot` is demonstrably
installed on `oss-path-to-production` — it has been opening fragment PRs
there from other surfaces (#232, #228, #220). No further setup needed.
## Not in this PR
- The registry-side seed of the **docs** surface. The docs fragment
first appears via this workflow's initial run, which now also carries
the reconciled canonical, so it lands green.
- The **web-inspector** surface, hand-seeded in the registry since this
PR was authored, remains manual. Automating it is a follow-up.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The repo migrated off @changesets/* to conventional-commit-driven releases
(scripts/release/ reads commit subjects from git log <lastTag>..HEAD), but
.changeset/ has been removed twice already (5afa55f067, 1e5ba689e0) and five
open PRs currently carry changeset files again. Two mechanisms keep feeding it:
contributor forks whose default branch still has the pre-cleanup .changeset/
debris, and plain convention inference — the repo reads as a Changesets repo
(pnpm monorepo, Changesets-formatted CHANGELOG.md files, "chore: release" PRs)
and nothing anywhere said otherwise.
- CONTRIBUTING.md: explain that we used Changesets, what replaced it, and what
to do instead (a good conventional commit subject).
- AGENTS.md / CLAUDE.md: same rule for coding agents, which author most of
these PRs and don't read CONTRIBUTING.md.
- static / check binaries: fail on added .changeset/* files, so this stops
depending on review catching it. Filters on added/modified only, so a PR
that deletes stale changesets still passes.
- .oxfmtrc.json: drop the ignore entry for the long-gone vendored
.github/actions/changesets-action, a stale "we use changesets" signal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Revalidation against current main (the branch was 1345 commits behind):
- docs workflow ran `pnpm reconcile` but `add-paths` listed only the
fragment, so its PR would land a fresh fragment beside a stale
telemetry-events.json and fail the registry's telemetry-reconcile
staleness gate — the exact failure the runtime workflow was already
fixed for. Verified against the registry's shipped emitters: every
automated fragment PR there (website.corp, Intelligence surfaces)
carries telemetry-events.json alongside its fragment.
- Refresh the action pins to the SHAs main now uses everywhere
(checkout v7, setup-node v7.0.0, pnpm/action-setup v6.0.10).
- Narrow the docs trigger to code under shell-docs/src, excluding
src/content (1000+ MDX/JSON prose files that cannot hold a
posthog.capture call site) so prose edits stop firing a full install.
- Note in the zizmor justification why setup-node v7's new
package-manager-cache auto-path still leaves this workflow cacheless
(it engages only for npm-declared repos; this one declares pnpm).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A fragment-only PR fails oss-path-to-production's telemetry-reconcile gate (it
recomputes telemetry-events.json and fails on staleness). After emitting each
fragment, install the registry's deps and run pnpm reconcile, then include
telemetry-events.json in the PR alongside the fragment — matching the Intelligence
CLI + surface-emitter pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cross-repo fragment PRs must be authored by the dedicated telemetry-registry
GitHub App that's installed on oss-path-to-production (the same App the
Intelligence CLI release workflow uses), not CopilotKit's DEVOPS_BOT release bot.
Switch both workflows to app-id/private-key from secrets.TELEMETRY_REGISTRY_APP_ID
/ TELEMETRY_REGISTRY_APP_PRIVATE_KEY and gate the mint on the App ID env var.
These secrets must be added to the CopilotKit repo (they currently live only on
Intelligence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/telemetry/ (emit-fragment.ts + extract.ts) and two CI workflows
that generate CopilotKit's telemetry-registry fragments and open path-limited
PRs into CopilotKit/oss-path-to-production:
- runtime (bespoke catalog): reads the AnalyticsEvents type map for event names
+ properties, scans capture() sites for call_sites, fails loud if the v1/v2
catalogs diverge. Triggered on stable monorepo release.
- docs (callee mode): extracts inline posthog.capture literals from
showcase/shell-docs (drops $-reserved events). Triggered on push to main
touching showcase/shell-docs/**.
Both are content-gated: the fragment is left untouched (and no PR opened) when
the event set is unchanged, so releases/edits don't churn the registry. Cross-
repo token follows the least-privilege recipe (no owner, bare repositories,
contents+PR write); mint gated on a job-level env var. zizmor clean (one
justified cache-poisoning suppression). 13 unit tests; tsc + oxlint clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
Adds a `react-version` matrix axis (**18**, **19**) to the unit-test
workflow so react-core, react-ui, and a2ui-renderer are exercised across
the full **supported peer range** (`^18 || ^19`), not just the
repo-default React 19.
This is a **reconstruction of the durable parts of #4221**
(@tylerslaton) onto current `main`. That PR went stale (~5,000 commits
behind, conflicting) and never landed. Rather than rebase it, this
rebuilds its design fresh — and deliberately **scopes to the supported
React range**: React 17 is dropped, because it is no longer a supported
peer version and carried ~80% of the original PR's complexity
(polyfills, `use-sync-external-store` source shims, `jsx-runtime`
aliases, a legacy `renderHook` fallback).
## What surfaced
Dropping R17 and validating R18 revealed a **latent React 18
incompatibility on current `main`**: the `window = {}` test pattern
crashes React 18's concurrent renderer with `"Should not already be
working."` mid-commit, which then corrupts the scheduler for the rest of
the file — **22 failures across 5 files** under React 18. (React 19
happens to tolerate the empty-window swap, so it was invisible until
now.)
The original PR fixed this but mislabeled it R17-only; it's actually
needed for R18, a *supported* version. So the matrix earned its keep on
day one.
Replacing `window = {}` with `stubWindowLocation()` is the load-bearing
fix — it resolves the crash cascade. Separately, **two** tests differ
under R18 purely in *render scheduling*, and are handled by narrow
version gates:
| Test | React 18 behavior | Why it's not a bug |
|---|---|---|
| `renderCustomMessages` → "executes multiple renderers in order" |
`executionOrder` is `["first", "first"]` | Renderer double-invoke.
`second` still never runs, which is the actual contract. |
| `use-human-in-the-loop` → `statusHistory` | `inProgress → executing →
inProgress → complete` | Transient backwards transition from extra
effect runs. Start, end, and the set of observed statuses are all still
correct. |
**No assertion tolerates a different state value.** An earlier revision
of this PR also relaxed the three-turn state-snapshot assertion to
accept `Turn: 2` on R18; @tylerslaton correctly flagged that as an
observable-behavior difference rather than a scheduling artifact.
Re-verified against a real 18.3.1 install — the strict `Turn: 3`
assertion passes **25/25** consecutive runs — so that gate was
unnecessary and has been removed (`a227f46a8`). The two gates above were
re-tested the same way and both genuinely reproduce.
## Changes
| File | What |
|---|---|
| `.github/workflows/test_unit.yml` | `react-version: ["18","19"]` axis.
R19 installs frozen; R18 overrides the root `pnpm.overrides` React
version and installs unfrozen. Adds a guard verifying the installed
React matches the matrix leg, and suffixes `NX_CI_EXECUTION_ID` with the
React version. Layered on top of the existing nx-affected selection
logic. |
| `test-helpers/stub-window-location.ts` *(new)* | Clears
`window.location` (so the localhost auto-open-inspector heuristic skips)
while keeping the real jsdom window — the safe replacement for `window =
{}`. |
| `use-agent-error-state`, `CopilotKitProvider.onError`,
`CopilotKitProvider.test` | Swap `window = {}` for
`stubWindowLocation()`. |
| `use-human-in-the-loop.e2e`, `renderCustomMessages.e2e` | Two
React-version-gated assertions, both **render-scheduling only** (see
table above). State assertions stay strict on every leg. |
No dependency or lockfile changes. None of the R17-only machinery from
#4221.
## CI cost
Full runs go from 3 legs (node 20/22/24) to **6** (node × react). On
PRs, nx-affected still scopes what actually builds/tests; the full 6×
only hits `workflow_dispatch` or when `test_unit.yml` itself changes (so
this PR runs all 6). This is the honest price of adding R18 coverage.
## Testing
Run locally in a worktree via the exact install-override logic the
workflow uses — `react`/`react-dom` → 18.3.1,
`@types/react`/`@types/react-dom` → `^18`, `@testing-library/react` →
`^14.3.1`, `streamdown>react` → 18.3.1, then `pnpm install
--no-frozen-lockfile`. Installed versions confirmed by resolving from
`packages/react-core` (18.3.1 / 19.2.3, `@testing-library/react` 14.3.1
on the R18 leg).
| Check | Result |
|---|---|
| react-core full suite @ React 18.3.1 | **117 files, 1433/1433
passing** ✓ |
| react-core full suite @ React 19.2.3 | **117 files, 1433/1433
passing** ✓ |
| Strict `Turn: 3` state-snapshot assertion @ R18, ×25 runs | **25 pass
/ 0 fail** — gate removed as unnecessary |
| `executionOrder` gate reverted to strict @ R18 | **fails**
(`['first','first']`) — gate justified |
| HITL `statusHistory` gate reverted to strict @ R18 | **fails** (extra
`inProgress`) — gate justified |
| `oxlint` (project-aware) | **0 warnings, 0 errors** — unchanged from
`main` |
| `oxfmt --check` | clean |
| Workflow YAML parse + lefthook commit hooks (lint-fix, package tests,
commitlint) | green |
Before the `window` fix, the R18 leg was **22 failing across 5 files**;
it is now fully green.
Credit to @tylerslaton for the original design in #4221, and for
catching the over-relaxed state assertion in review.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Brings the `setup-slack-channel` skill into the set that `copilotkit
skills install` distributes, and narrows both it and
`copilotkit-channels` so they stop matching the same request.
## Why
The skill was merged into
[CopilotKit/channels-sdk#7](https://github.com/CopilotKit/channels-sdk/pull/7),
which makes it reachable from a channels-sdk checkout and nowhere else.
`copilotkit skills install` reads `CopilotKit/CopilotKit/skills`, so
until the skill lives here, no CLI user can get it.
No CLI change is needed to pick it up: skill names are free-form
(`--skill` is validated by regex only, with no allowlist) and the
default install is `--skill *`.
## How
**The skill is copied from the merged channels-sdk source**, with one
addition: a `version: 1.0.0` frontmatter field, so it matches the other
standalone skills in this repo (every one of them carries a version;
only the package-mirrored skills omit it). The body, references, and
manifest asset are otherwise unchanged. It stays browser-first and makes
no reference to the `copilotkit channels` commands, per the call to
defer those to the web app until they have more real-world usage.
**Both descriptions are narrowed** so the trigger overlap is gone:
| Skill | Owns | Hands off |
| --- | --- | --- |
| `copilotkit-channels` | The code half — declaring, wiring, and
customising a Channel. Assumes the provider app exists | First-time
Slack setup → `setup-slack-channel` |
| `setup-slack-channel` | The provider half — the Slack app, its tokens,
attaching it to a Channel | Code questions → `copilotkit-channels` |
Before this, both matched "connect my agent to Slack" and the agent
picked whichever it read first. That collision was invisible in
channels-sdk, where `setup-slack-channel` is the only skill present; it
becomes live the moment both ship in the installed set.
## Notes for reviewers
- **Scope is Slack only.** A Teams sibling is deliberately a separate
pass.
- **No existing skill content changes** — the `copilotkit-channels` diff
is its frontmatter description and nothing else.
- Two known follow-ups are tracked and intentionally not addressed here:
[channels-sdk#9](https://github.com/CopilotKit/channels-sdk/issues/9)
(CLI-capability claims, deferred by decision) and
[channels-sdk#2](https://github.com/CopilotKit/channels-sdk/issues/2)
(`build-channels-bot` staleness, unrelated).
Refs CopilotKit/channels-sdk#10
## Added during the `main` merge
Resolving the conflict surfaced a second, unrelated problem that had to
be fixed for this PR to be safe to land.
`skills/setup-slack-channel` is a **standalone** skill — it has no
`packages/*/skills/` source. `scripts/sync-plugin-skills.ts` treats any
such directory as an orphan unless it is listed in
`RESERVED_LIFECYCLE_SLUGS`, which this one was not. Verified against the
pre-fix script:
- `pnpm check:plugin-skills` → exit 1, `orphan file(s) in mirror:
skills/setup-slack-channel`
- `pnpm sync:plugin-skills` (write mode) → **recursively deleted all 8
files of the new skill**
The `plugin-skills-check` workflow's path filter does not match
`skills/setup-slack-channel/**`, so this PR would not have caught it —
it would have gone red on the next unrelated PR that touched the script
or a package skill, or silently eaten the skill on the next sync run.
Fix is two lines: add the slug to `RESERVED_LIFECYCLE_SLUGS`, and move
the paired `size` assertion in
`scripts/__tests__/sync-plugin-skills.test.ts` from 9 to 10. The test
file already documents this exact hazard in a comment.
### Conflict resolution
The only conflict was the `copilotkit-channels` frontmatter description,
which #6320 rewrote in parallel. The two sides disagreed about Teams:
this branch said the skill "assumes the provider app already exists",
while #6320 established that Teams provider setup **is** this skill's
job because the CLI or dashboard wizard performs it. The resolution
keeps this branch's code-half framing and the `setup-slack-channel`
handoff, but scopes that handoff to *first-time Slack app creation* only
— so it contradicts neither #6320's Teams sections nor the Slack
provider troubleshooting that stays in this file. Took #6320's `version:
1.1.0`.
Addresses review on #6340.
Interactivity is no longer disabled on the managed path. The shared generator
emits `interactivity.is_enabled: true` with an Intelligence-hosted request URL
(Intelligence `libs/channels-setup/src/slack.ts:209-211`) and the ingress
handles `block_actions` (`apps/app-api/src/routes/channels-routes.ts:868`), so
HITL buttons and selects do fire. The skill's own bundled manifest asset already
said `is_enabled: true`, so the prose contradicted the file shipped beside it.
What is still undelivered is `slash_commands` (absent from the generator) and
`view_submission` (`apps/app-api/src/channels/slack-ingress.ts:1050`), so the
`onCommand` / `onModalSubmit` warnings stay. Corrected in all three places that
claimed otherwise, and the troubleshooting entry now tells the reader a dead
button is a real failure rather than a capability limit.
Browser-only framing is now a routing instruction rather than an architectural
claim, since `copilotkit channels add` does create the Channel and attach the
adapter. Same outcome, but it no longer contradicts `--help` for an agent that
was told to check it.
Trigger scope is narrowed in the frontmatter description instead of rewriting
Phase 0. The phases assume OpenTag conventions (`app/channel.tsx`, `app/env.ts`,
`INTELLIGENCE_CHANNEL_NAME`, an agent on port 8123), which are not what
`copilotkit init` scaffolds — naming that in the description keeps the skill from
firing on any "connect my agent to Slack" once it installs into customer repos.
Also widens the plugin-skills-check path filter to `skills/**` rather than
adding the one new slug. The orphan scan reads the whole mirror, so enumerating
individual directories is what let this PR's own blocker go untested and would
have armed it again for the next standalone skill.
`bun-version: latest` let a Bun release change module-resolution behaviour
between runs. 1.3.14 is the version the recent passing and failing runs both
resolved, so pin it and make the job reproducible.
Mike Ryan