2330 Commits

Author SHA1 Message Date
Mike Ryan dbf4a43b89 fix(channels): bound paced render latency 2026-07-29 09:00:19 -07:00
Mike Ryan 26385feca6 fix(channels): address realtime batch review feedback 2026-07-29 09:00:19 -07:00
Mike Ryan 62bc9df49d fix(channels): make render batches replay stable 2026-07-29 09:00:18 -07:00
Mike Ryan f84aa5ac64 fix(channels): back off render batch retries 2026-07-29 09:00:18 -07:00
Mike Ryan 35bf81fccb fix(channels): preserve frame-key message refs 2026-07-29 09:00:18 -07:00
Mike Ryan f0a9435bde test(channels-intelligence): bound delta-count latency 2026-07-29 09:00:18 -07:00
Mike Ryan ac1e51a0af fix(channels-intelligence): bound stalled render lanes 2026-07-29 09:00:17 -07:00
Mike Ryan 119f6e38d2 fix(channels-intelligence): retry immutable render batches 2026-07-29 09:00:17 -07:00
Mike Ryan f8d700c523 feat(channels-intelligence): batch managed render output 2026-07-29 09:00:17 -07:00
Maxim 50e0f939b9 fix(a2ui-renderer): hoist the export test's entry import out of the test body
Follow-up to 85223276fa, which fixed the ordering bug but introduced a timing
one. Importing the source entry inside the test body meant the dynamic import
transformed the whole renderer graph (lit, markdown-it, zod) while vitest's 5s
default testTimeout was running. On a warm local cache that took ~890ms and
passed; CI, cold, took 5039ms and failed:

  × filterCatalog package export > is reachable from the package entry 5039ms
    → Test timed out in 5000ms.

Moving the import to module scope pays that cost during collection, which is
not bounded by testTimeout, and leaves the assertion synchronous.

Verified under CI-like conditions this time — both vite caches moved aside AND
dist/ removed: 16/16 pass, and the test file goes from 5059ms to 4ms with the
graph cost showing up as collect 3.57s instead.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-29 16:11:20 +02:00
David McKay ce0576d7d1 Merge branch 'main' into fix/useagent-threadid 2026-07-29 06:50:17 -07:00
Maxim 85223276fa fix(a2ui-renderer): stop the filterCatalog export test depending on its own dist
The export test self-imported the package by name:

  const mod = await import("@copilotkit/a2ui-renderer");

which resolves through node_modules to dist/. Nothing guarantees dist/ exists
when the test runs: nx.json sets test.dependsOn to ["^build"], and the caret
means *dependencies'* builds, not the project's own. Under `nx run-many` the
scheduler ran a2ui-renderer:test before a2ui-renderer:build, so the import
failed with "Failed to resolve entry for package @copilotkit/a2ui-renderer".
Standalone runs passed only because an earlier build had left dist/ populated.

Nx reported the task as flaky, which it was not — it was order-dependent. The
same root cause failed check-types, since tsc resolves the dynamic import to
dist/index.d.mts.

Importing the source entry instead keeps what the test is actually for (that
src/index.ts -> react-renderer/index.ts -> ./filter-catalog re-export chain is
intact, so callers can import filterCatalog) while making it hermetic. The
package.json "exports" mapping is already covered by this package's publint and
attw targets.

Verified with dist/ deleted: 16/16 pass and check-types is clean. Both
`nx run-many -t test` and `-t check-types` across core, react-core, runtime,
web-inspector and a2ui-renderer now pass with --skip-nx-cache.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-29 15:36:14 +02:00
Maxim fa40e0424b Merge remote-tracking branch 'origin/blitz/glass-inspector/integration' into blitz/glass-inspector/integration 2026-07-29 15:28:16 +02:00
Alem Tuzlak 9d8916a9d9 fix(showcase): settle mcp-apps D5/D6 on the full iframe cascade
The `completeOnMount` gate added in d70d48a561 named the `mcp-app-iframe`
testid, which only Angular's `copilot-mcp-apps-widget` declared. react-core
and vue build the sandbox iframe imperatively with no testid, so every
React/Vue integration timed the turn out at 30s with
`reason=surface-missing` and never reached `assertIframePresent` — whose
`iframe[sandbox]` fallback would have passed. D5 + D6 `mcp-apps` went red on
all 18 integrations that support the feature (first_failure_at 2026-07-28
23:03Z) while the demos rendered correctly by hand.

Fixed on both sides of the contract:

- react-core and vue now set `data-testid="mcp-app-iframe"` and a `title` on
  the host-created iframe, matching Angular. Pinned by a test in each package.
- `completeOnMount` accepts CSS `selectors` alongside `testIds`, so the probe
  settles on the same cascade its module doc and assertion already use
  (`[data-testid="mcp-app-iframe"], iframe[sandbox]`). A comma-joined entry is
  one conjunctive surface whose branches `querySelectorAll` unions, so the
  delta/`minNewMounts` semantics are unchanged and `testIds` is now sugar for
  the equivalent selector. This half greens the fleet on the next sweep
  without waiting for a package release, since the integrations pin
  @copilotkit/react-core 1.61.2.

A spec naming no surface now throws instead of burning the turn budget and
reporting a misleading `surface-missing`.

Verified against live staging: after clicking the pill, the old gate matched
0 elements and the cascade matched 1 (the sandboxed iframe was there all
along). Also recorded in showcase/GOTCHAS.md.
2026-07-29 12:34:38 +02:00
Markus Ecker acdd9a9ef5 fix(react-core): enforce useAgent thread-scoping props at the type level
UseAgentProps left `agentId`, `runtimeAgentId`, and `threadId` independently
optional, so every unsafe or pointless combination compiled and only
`threadId`-without-`runtimeAgentId` was caught — at runtime, on render. Given
the bug this branch fixes was a silently-ignored `threadId` prop, the type is
where it should have been caught.

Split UseAgentProps into a base plus a two-branch union so exactly two shapes
are valid:

  useAgent()                                       // shared agent
  useAgent({ agentId })                            // shared agent
  useAgent({ agentId, runtimeAgentId, threadId })  // private agent, pinned thread

Every partial combination is now a compile error, each for its own reason.
`threadId` alone would scope a thread onto a shared singleton (the original
review finding). `runtimeAgentId` without `threadId` registers a private agent
that behaves like the shared one, minus nothing but a registration and a local
id to keep unique. And without an explicit `agentId`, the proxy registers under
the chat configuration's agentId or DEFAULT_AGENT_ID — ids that already belong
to real agents — which either throws `already registered` or silently shadows
one, depending on whether runtime discovery has landed.

The runtime throws stay as backstops for callers TypeScript doesn't cover (plain
JS, `as any`, props widened to `string | undefined` at a call boundary), and now
cover all three cases, each failing before any registration happens.

Adds use-agent-types.test.tsx pinning the compile-time half of the contract
across the accept/reject matrix, plus contract tests for the two new throws. The
`@ts-expect-error` on each deliberately-bad call doubles as an assertion that the
type still rejects it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 11:27:17 +02:00
tylerslaton 028a5adc9d chore: release monorepo v1.64.1 2026-07-28 17:48:23 -07:00
Ben Taylor a76c2bcce2 feat(channels-intelligence): measure render-frame push latency (refs OSS-648) (#6212)
## What this is now

Opt-in instrumentation for the channel egress path, plus a
replay-stability guard. **Measurement and safety only — no behaviour
change to how frames are sent.**

It started as a fix too. That fix is reverted here; see below.

## The problem it measures

A streaming reply pushes one render frame per AG-UI token and awaits
each durable acceptance receipt before sending the next, so reply
latency scales with token count. Nothing recorded that, so the cost was
invisible and got attributed to the Connector Outbox's 400ms poll
instead.

```bash
COPILOTKIT_CHANNELS_RENDER_METRICS=summary   # one summary per turn
COPILOTKIT_CHANNELS_RENDER_METRICS=frames    # also one line per frame
```

Unset means off and no collector is allocated. Output goes to the
transport `log` callback when configured, else the console.

## What it found, measured against production

Hosted api + realtime gateway, local runtime, fixture agent emitting
exactly 120 deltas 15ms apart:

```
pushMsMean: 478.43   pushMsP50: 463   pushMsP95: 589   pushBlockedPct: 99.67
```

**478ms per round trip against an ~85ms network RTT**, so roughly 390ms
of every push is server-side work — the `lookupClerkOrgId` query, the
off-cluster ops-api entitlement call, the read-model build, and the
Redis publish. Intelligence#627 removes those for `text_delta` frames.

At 478ms a push, a 120-token reply costs ~123 round trips ≈ **59s**.
That matches the original "sometimes many seconds" report.

## Why the fix was reverted

Coalescing adjacent text deltas worked — 123 frames to 4, a production
turn from ~59s to 3.4s — and broke replay.

A turn's id is a pure function of its delivery id (`turn_<deliveryId>`,
enforced by app-api) and the adapter resets the per-turn seq counter on
every dispatch, so a redelivery replays into the **same seq space**.
app-api treats that space as an idempotency key: same payload at a seq
is `duplicate_accepted`, a different payload is
`CHANNEL_RENDER_FRAME_CONFLICT`, which nacks the delivery. Merge
boundaries depend on wall-clock arrival, so a replay cuts them
differently:

| | attempt 1 (slow, merged) | attempt 2 (redelivery, faster) |
|---|---|---|
| seq 1–3 | *merged away, never pushed* | `"a"`, `"b"`, `"c"` → INSERT ✓
|
| seq 4 | `text_delta "abcd"` | `text_delta "d"` → **CONFLICT** ✗ |

Before coalescing, deltas were 1:1 with AG-UI events and a replay was
byte-identical. I verified both directions: the new test passes on the
pre-coalescing adapter and fails with coalescing. The regression was
mine.

`render-frame-replay.test.ts` stays as a standing guard — it passes with
frames 1:1 and fails for any change that makes a frame's payload at a
given seq depend on timing.

## The replacement

Batch the **transport**, don't merge frame **content**: send the
buffered frames as an array in one request. Same collapse in round
trips, `seq k` still carries `delta k` so replay stays byte-identical,
and seqs stay dense so the outbox can later require contiguity
(OSS-653).

That needs an app-api batch accept and a gateway envelope, so it ships
after launch with its own deploy ordering (app-api → gateway → SDK)
rather than two days before.

## Also found, filed separately

OSS-653 — the outbox advances an in-memory cursor to the highest seq it
read (`WHERE seq > cursor ORDER BY seq ASC`), so a frame accepted out of
order is skipped and never rendered. Pre-existing and independent of
this PR. Not durable corruption: thread history is rebuilt from all
acceptance rows ordered by seq, independent of that cursor, so threads
read back complete.

## Validation

```
nx run-many -t test check-types build --projects=@copilotkit/channels-intelligence --skip-nx-cache
```

216 tests pass (12 files), check-types and build clean, `oxlint` clean
on all changed files.

refs OSS-648
2026-07-28 18:39:41 -05:00
Benjamin Taylor 7845578550 docs(channels-intelligence): correct replay-test comments after the coalescing revert (refs OSS-648) 2026-07-28 18:17:16 -05:00
Benjamin Taylor 28ee17159d revert(channels-intelligence): drop frame coalescing, keep frames 1:1 with tokens (refs OSS-648)
Coalescing adjacent text deltas cut a 120-token reply from 123 frames to 4 and
took a production turn from an extrapolated ~59s to 3.4s, but it broke replay.

A turn's id is a pure function of its delivery id (turn_<deliveryId>, which
app-api enforces) and the adapter resets the per-turn seq counter on every
dispatch, so a redelivery replays into the same seq space. app-api treats that
space as an idempotency key: the same payload at a given seq is
duplicate_accepted, a different payload is CHANNEL_RENDER_FRAME_CONFLICT, which
nacks the delivery. Merge boundaries depend on wall-clock arrival, so a replay
cuts them differently and collides — attempt 1 stores "abcd" at seq 4, attempt 2
pushes "d" there. Before coalescing, deltas were 1:1 with AG-UI events and a
replay was byte-identical, so this worked; the regression is mine.

Keep the instrumentation, which is what found the problem and is what will
measure the replacement. Add the replay-stability test as a standing guard: it
passes with frames 1:1 and fails for any change that makes a frame's payload at a
given seq depend on timing.

The cost problem is real and unfixed. The replacement is to batch the TRANSPORT
(many frames per request) rather than merge frame CONTENT: it collapses the same
number of round trips, keeps seq k carrying delta k so replay stays
byte-identical, and keeps seqs dense so the outbox can later require contiguity
(OSS-653). That needs an app-api batch accept and a gateway envelope, so it ships
after launch with its own deploy ordering rather than two days before.
2026-07-28 18:16:45 -05:00
Benjamin Taylor 108f42d294 fix(web-inspector): keep the banner impression queue and stop guessing open segmentation
Both blockers from the third review.

**1. `banner_viewed` lost its pre-handshake queue.** On `main`,
`fetchAnnouncement()` stored `pendingBannerViewed` and the flush waited for
`connected`, discarding the event when the runtime reported `telemetryDisabled`.
Removing the deferral in the previous commit made the impression post directly,
so it now went out on the `false` placeholder that `telemetryDisabled` reads
before /info answers. That is a regression this PR introduced for an existing
event, distinct from the pre-existing `banner_clicked` race. The queue is
restored — widened from a single slot to a bounded array, since opening the panel
reveals the second surface and can happen before the first impression flushes —
and per-surface dedup is unchanged.

**2. `opened` recorded segmentation it could not know.** `licenseStatus` is
undefined and `runtimeMode` reads its `sse` default before the handshake
(`agent-registry.ts:77`), so an early open against an Intelligence runtime was
permanently filed as SSE. Rather than queue the open — a disconnected runtime
should not suppress a real user's open — `license_status` and `runtime_mode` are
now omitted unless the handshake has landed. An absent dimension is honest; a
wrong one corrupts the segmentation OSS-566 exists to provide. `runtime_url_type`
is config-derived and stays, since it is accurate immediately.

Also fixes a test that passed for the wrong reason: `HeaderMockCore` had no
`runtimeMode` at all, so a pre-handshake assertion about it read `undefined`
either way and would have gone green against the very bug it was meant to catch.
The mock now mirrors the real pre-handshake state (`runtimeMode: "sse"`,
`licenseStatus` absent). With that corrected, 3 of the 4 new cases fail against
the previous commit; the fourth guards the connected path.

Coverage: impression held then dropped on disabled, held then released on
allowed, open before the handshake omits both fields while keeping
`runtime_url_type`, and an open against a connected Intelligence runtime carries
`runtime_mode: intelligence` + `license_status: valid`.

Verified: web-inspector 115 passed, tsc clean, tsdown clean, oxfmt clean, oxlint
9 warnings all pre-existing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:11:13 -05:00
Mike Ryan ec0e496bc2 fix(showcase): restore Angular runtime parity 2026-07-28 15:25:16 -07:00
Benjamin Taylor 249282cf39 perf(channels-intelligence): coalesce adjacent text deltas into one frame (OSS-648)
A streaming reply emitted one render frame per AG-UI token and awaited each
durable acceptance receipt before sending the next, so reply latency scaled
with token count: a 100-token reply cost 100 serialized round trips. Against a
hosted API that is tens of seconds for a long reply, and it dominated every
other cost in the egress path.

Frames now land in a buffer that a single pump drains, folding adjacent
text deltas for the same message into one frame. The buffer only ever holds
what arrived while a push was already in flight, which is exactly the set the
old serial chain kept queued, so this waits on no timer and adds no latency.
Batch size self-tunes: the slower the round trip, the more deltas merge.

Ordering is preserved by keeping seq allocation at enqueue time, so a discrete
post or update interleaved mid-run still orders correctly against the run's own
frames. A merged frame carries the seq of the last delta it absorbed; skipped
seqs are never pushed, and the accepted high-water mark still advances. Only
adjacent same-message deltas merge, so a tool call or a second message ends the
run. Merged text is capped at 12k chars, under the platform's 40k delta limit
and within one Slack appendStream call.

Measured with the instrumentation from the previous commit, on a 40-token reply
with a 100ms round trip and tokens arriving every 10ms: 42 frames and 4242ms
before, 7 frames and 706ms after, with 40 deltas folded into 5 pushes and no
text lost.
2026-07-28 16:41:34 -05:00
tylerslaton c28c3fccad chore: release channels v0.4.0 2026-07-28 21:36:34 +00:00
Benjamin Taylor d651df8e5a feat(channels-intelligence): measure render-frame push latency (OSS-648)
Channel replies are slow in proportion to how many tokens they contain. The
run renderer pushes one render frame per AG-UI event and awaits its durable
acceptance receipt before sending the next, so a streaming reply costs one
round trip per token. Nothing recorded that, so the cost was invisible: no
frame counts, no per-push latency, no measure of how much of a turn was spent
waiting.

Add opt-in instrumentation at the renderer's serial push chain, the one point
every frame passes through, so it covers the HTTP render-accept transport and
the realtime-gateway transport with one seam. Set
COPILOTKIT_CHANNELS_RENDER_METRICS=summary for a per-turn summary, or =frames
to also log a line per frame. Unset means off and no collector is built.

The summary reports frame counts by kind, characters carried per text_delta
frame, total/mean/p50/p95/max push latency, and the share of the turn spent
blocked on receipts. charsPerTextFrame and pushBlockedPct are the two numbers
that show whether a batching change helped: one round trip per token reads as
a single-digit charsPerTextFrame and a pushBlockedPct near 100.

Measurement only. No change to frame ordering, sequencing, or delivery.
2026-07-28 16:29:34 -05:00
Tyler Slaton d2959daf1c Merge branch 'main' into release/publish/monorepo/v1.64.0 2026-07-28 14:21:24 -07:00
Benjamin Taylor 783fc89337 revert(telemetry): drop the CI opt-out work, keep only the new Inspector events
Narrows this PR to OSS-566 + OSS-568. The OSS-565 CI-suppression effort is
backed out entirely:

- the 10 workflows' `COPILOTKIT_TELEMETRY_DISABLED` / `DO_NOT_TRACK` env and the
  egress-block steps
- `examples/e2e/playwright.config.ts` env defaults
- the runtime/shared test-env manipulation and the `telemetry-opt-out-env`
  helper
- the `navigator.webdriver` check and its tests

Every reverted path is byte-identical to the merge-base — verified rather than
assumed, since `git checkout origin/main -- <path>` would otherwise have
imported main's newer content for files that had moved on.

One piece is deliberately kept, because it is not about CI: the jsdom egress
guard. These tests run with a real `fetch`, and the inspector's telemetry is
fire-and-forget, so any test touching a banner / threads / open path without
stubbing fetch POSTs a real `oss.inspector.*` event to the live sink — from
developer machines as well as CI. The announcement-dismissal tests did exactly
that before the guard, and the new `opened` / `banner_dismissed` tests hit the
same send path, so shipping the events without it would make our own dataset
dirtier. Its comments no longer cite OSS-565; the rationale stands on its own.

What remains: `oss.inspector.opened` with open-source attribution, the
`banner_viewed` surface split, and first-class `banner_dismissed`. No inspector
telemetry waits on the /info handshake.

Verified: web-inspector 112, runtime 1,760, shared 199 (the last two now on
main's own test files). tsc clean, oxfmt clean, oxlint 9 warnings all
pre-existing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 16:19:47 -05:00
Benjamin Taylor f717ad1764 fix(web-inspector,ci): suppress CI telemetry via automation flag and runner egress block
Replaces the /info-coupled approach with two mechanisms that need no handshake,
and backs the coupling out entirely.

The problem with gating on /info: `telemetryDisabled` only reaches the browser
through an async handshake, so either events race it (the hole) or they wait for
it (fail-closed). Waiting means an inspector whose runtime never connects — or
whose /info 404s, which sets status to `Error`, not `Connected` — goes silent for
the whole session. That penalizes real users with broken setups, who are exactly
the people most likely to have the inspector open.

Instead:

1. `navigator.webdriver` in `track()`. Spec'd, synchronous, available on the
   first line of script, and true under every Playwright launch — verified
   against this repo's Playwright 1.59.1 in both headless and headed mode. Needs
   no per-app env plumbing and covers pages served by deployed environments,
   which no CI-side variable could reach. Suppresses only on an explicit `true`:
   absent/falsy means "not automation", never "unknown, so stay quiet".

2. A job-wide egress block (`.github/scripts/block-telemetry-egress.sh`) pointing
   the sink at loopback for both address families, added to the 12 CI jobs that
   run or boot real CopilotKit code. Backstop for anything that reads neither the
   env nor the DOM. The frozen integration container gets the same via
   `--add-host`, since neither the workflow env nor the runner's hosts file
   crosses that boundary.

Backed out of the inspector: the egress gate and held queue, `pendingOpened`,
and `pendingBannerViewed` (the last pre-dating this PR). Inspector telemetry no
longer references the handshake at all — `index.ts` is net −94 lines and the
opt-out is now a single boolean check per call site, as it reads.

Two bugs this surfaced, both caught by existing or new tests rather than
shipped: removing the flush step also removed the only `telemetryDisabled` check
on the `banner_viewed` path, and a thread-refetch test counted calls on a stubbed
global `fetch`, so telemetry that now sends immediately inflated it — scoped to
the thread URLs it actually means.

Also corrects the workflow comments, which credited the /info chain for
browser-side suppression it never reliably provided.

Verified: web-inspector 116, runtime 1,760, shared 199. tsc clean, oxfmt clean,
oxlint back to the baseline 8 warnings (fixed a dead type import and a
non-capturing test helper I introduced). actionlint 344 findings before and
after, byte-identical modulo shifted line numbers. shellcheck clean. tsdown
build ships the check in dist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 16:02:33 -05:00
Tyler Slaton ca31c89545 Merge branch 'main' into release/publish/monorepo/v1.64.0 2026-07-28 13:50:06 -07:00
Tyler Slaton 4baff58cc0 test(channels-slack): cover initial streaming response post 2026-07-28 16:44:34 -04:00
Tyler Slaton d145d32f49 fix(channels-slack): remove duplicate thinking posts 2026-07-28 16:44:34 -04:00
Benjamin Taylor 95991084ff fix(web-inspector): gate all telemetry egress on the runtime handshake
The previous commit closed the pre-handshake hole for the three events this PR
touches, but every other inspector event still posted on the placeholder.
`AgentRegistry._telemetryDisabled` starts `false`, so the per-call-site
`if (this.core?.telemetryDisabled) return` check passes while disconnected and
the event goes out — reproduced with the threads tab, which emitted
`threads_tab_clicked` before /info could report the opt-out. 15 call sites
shared the flaw (threads tab, memories tab, view-state, CTAs, example tour).

Rather than queue at each of them, the gate now lives in `telemetry.ts` at the
single egress boundary: `track()` holds events while the runtime decision is
unknown, and the inspector calls `resolveTelemetryGate("allowed" | "denied")`
once the handshake lands. Nothing reaches the network without an explicit
allow.

That makes the per-event queues added in e7e8922 redundant, so
`pendingBannerViewed` and `pendingBannerInteractions` are gone (net −101 lines
in index.ts). `pendingOpened` stays: it exists to resolve `license_status` /
`runtime_mode` / `runtime_url_type` at flush time, which is a payload concern
rather than a deferral one. The gate resets on `detachFromCore` so a later core
starts from "unknown" instead of inheriting the previous runtime's decision.

Two fidelity details the refactor forced, both covered: held events carry the
timestamp of the interaction rather than of the release, and the opt-out is
re-checked at release in case the user opts out mid-handshake.

Trade worth stating: an inspector whose runtime never connects now emits
nothing. Consent cannot be established without the handshake, so silence is
the correct default.

Coverage: threads tab held then dropped on disabled / sent on allowed, plus a
gate unit suite (unknown holds, allowed releases in order, denied discards,
post-denial drops, queue bound, reset, opt-out precedence, event-time ts).
web-inspector 127 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 15:20:49 -05:00
Tyler Slaton b919b0e927 docs(runtime): direct Channels stay below the canonical/reliability layer by design (#6155)
## Problem

All five `OSS-599` references in
`packages/runtime/src/v2/runtime/core/channel-manager.ts` describe the
missing gateway/canonical/reliability wiring for **direct** Channels as
*"deferred"*:

> it is NOT wired into the Intelligence gateway/canonical/reliability
layer (deferred, OSS-599)

That reads as pending work — as though a direct Channel eventually
reaches managed parity.

**OSS-599 says the opposite.** Its boundary discipline places
run-correctness (canonical cross-surface history, fenced
outer-run/single-terminal, durable HITL-resume-across-restart, selection
pinning) and the reliability layer **Intelligence-side only**, and
states plainly that shipping an SDK-side equivalent *"collapses the
build-vs-buy moat"*. A direct Channel's ceiling is the SDK's in-process
run loop, permanently.

So the comments point the next reader at implementing precisely the
thing the ticket forbids.

## Change

Reword all five sites to say the boundary is by design, not pending:

| Site | Was | Now |
|---|---|---|
| `ChannelStatus` doc | "(deferred, OSS-599)" | "BY DESIGN, not a
deferral" + why, + "do not 'finish' this by pulling the layer into the
SDK" |
| `ChannelManager` class doc | "wiring … is deferred" | "stay below the
canonical/reliability layer by design" |
| `activate()` inline | "is deferred (OSS-599)" | "by design, not
pending work (OSS-599)" |
| `startDirectChannel` doc | "(deferred, OSS-599)" | "that boundary is
permanent, not a deferral" |
| direct-start log string | "wiring deferred (OSS-599)" | "stay below
the canonical/reliability layer by design (OSS-599)" |

Comments and one log string only. **No behavior change.**

## Testing

- **Log-string assertion preserved.** `channel-manager.test.ts:611-619`
asserts the direct-start breadcrumb contains `"direct adapter"` and
`"ɵruntime.start()"`. Both substrings survive the reword — only the
parenthetical changed.
- **Test run + control.** Ran `channel-manager.test.ts`,
`channel-manager-reconnect.test.ts`, and
`channel-activation-config.test.ts` in the worktree: `4 failed | 56
passed`. Ran the same suite on an **unmodified `origin/main`** worktree
as a control: `4 failed | 34 passed` — the *identical* four failures.

The four are a worktree artifact, not a regression:
`@copilotkit/channels` resolves to the outer checkout's pre-#6145 build,
which has no `ɵruntime`, so every "real direct transport" test fails
there. Same failures before and after the change; this diff adds none.
CI (with a correct install) is the real gate.
- **Formatted** with `oxfmt`.

## Notes

Pre-commit hooks were bypassed: `test-and-check-packages` runs `nx` in
the worktree, where `@copilotkit/core:build` fails for unrelated
environment reasons. The change is comments-only.

Follow-up, not in this PR: OSS-599 was written the day before Plan C
shipped, so its own framing ("a DIY runner is ~15 lines over
`ɵruntime.start()`", "a DIY runner gets this") is stale now that the DIY
path is removed. Its §2 response-policy and four-mode-binding scope is
unaffected. I'm leaving a reconcile note on the ticket.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-28 13:15:09 -07:00
Tyler Slaton ae6fe8ed59 Merge branch 'main' into tyler/oss-633-default-threadrunagent-prompt-from-messagetext 2026-07-28 12:50:20 -07:00
Tyler Slaton af8c981a1d fix(channels-core): avoid duplicate runAgent prompts 2026-07-28 15:21:33 -04:00
Benjamin Taylor e7e8922365 fix(web-inspector): gate banner interactions on the handshake and stamp opens at flush
Three review findings on the inspector-telemetry PR.

Banner interactions bypassed the runtime opt-out before the handshake.
`telemetryDisabled` reads `false` until /info answers, so a dismissal (or a
body click) handled while disconnected posted on that placeholder — even when
the runtime was about to report `telemetryDisabled: true`. `banner_clicked`
and `banner_dismissed` now defer behind the same connected gate as
`banner_viewed`. Dedup still happens at interaction time, so a double-click
queues one event rather than two, and the queue carries the same bound as the
others.

Queued opens preserved pre-handshake placeholder segmentation. `trackOpened`
snapshotted `licenseStatus` (undefined) and `runtimeMode` (its `sse` default)
at queue time, and the flush sent them unchanged — permanently attributing an
early open against an Intelligence runtime to SSE. Only the open *intent* is
queued now; `license_status` / `runtime_mode` / `runtime_url_type` are
resolved at flush time, once /info has landed.

Test env overrides left the opt-out cleared. Suites that must clear
`COPILOTKIT_TELEMETRY_DISABLED` / `DO_NOT_TRACK` now capture the originals and
restore them in `afterAll`, via a shared `telemetry-opt-out-env` helper. The
hoisted cases snapshot inline and return it, since hoisted callbacks run
before the imports they would otherwise need. Measured: under the repo's
default `isolate: true`, vitest forks a fresh process per file, so the delete
does not currently cross files (pids differ; a `--no-isolate` control shares
one pid and does leak). Restoring is the defensive fix, not a live-bug fix.

Coverage: pre-handshake dismissal dropped when the runtime then reports
disabled; sent when it allows; double-click dedup; and a queued open flushed
against an Intelligence runtime carrying `runtime_mode: intelligence` +
`license_status: valid`. Three of the four fail against the pre-fix code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 14:13:28 -05:00
Ben Taylor 8a8e52d9e9 fix(runtime): non-optional listener.channels and honest lifecycle docs (OSS-646) (#6207)
Closes OSS-646. Split out of OSS-641 as the unambiguous half. This PR
does **not** change when activation happens — whether the long-running
wrappers should auto-connect stays open on OSS-641.

## Why

`createCopilotRuntimeHandler` builds the `ChannelManager` but opens no
connection; activation is lazy, triggered by the first
`channels.ready()`. That is deliberate (`fbf35ac59`, OSS-473) —
Cloudflare/Next isolates freeze and recycle per request, so cold starts
would mint conflicting listeners. Two things were left inconsistent with
it:

1. `endpoints/node.ts` still documented the pre-`fbf35ac59` world — "the
same `ChannelsControl` surface the underlying fetch handler **activates
at creation time**" — and labelled the one required call as `//
Optional:`. That's the TSDoc developers and coding agents see in-editor,
and it contradicted every channel-package README. Same failure class as
OSS-634.
2. `68349bc1f` gave the fetch handler a branded overload so
`handler.channels.ready()` type-checks without `?.`, but the node
wrapper never got it — so every call site, including our own example and
all nine showcase docs pages, was written defensively.

### A live consequence, found en route

`examples/slack/app/managed.ts` never called `ready()`. It built the
runtime, mounted the listener, logged `[channel] started managed Channel
"…"`, and only ever called `stop()` — so since activation went lazy it
has connected nothing while reporting success. It was written against
exactly the creation-time model the TSDoc described. Fixed here, with a
regression assertion.

## What changed

- **Types** — `createCopilotNodeListener` gets the branded overload pair
mirroring `createCopilotRuntimeHandler`: a runtime with at least one
declared Channel yields non-optional `.channels`; `activateChannels:
false` and channel-less runtimes keep the optional shape. Adds
`NodeCopilotListenerWithChannels`; both listener types are now exported
from `@copilotkit/runtime/v2/node`.
- **Docs** — node/express/hono TSDoc corrected: creation opens no
connection, `ready()` is what activates, and it is required on a
long-running host. Same stale claim fixed in the three example comments
and `examples/slack/README.md` that repeated it.
- **Call sites** — `?.` dropped from `examples/slack`, `examples/teams`,
both READMEs, and the nine `showcase/shell-docs` channel pages.

## Deliberate scope choices, called out

- **Express/Hono keep an optional `.channels`.** Only their TSDoc is
corrected here. Their own type docs name Node as the lifecycle-owning
surface and attach `.channels` best-effort, so the branded overload is
Node-only for now; `endpoints-channels.test.ts` still uses `!` for those
two. Say the word if the overload should extend to them.
- **The non-optional shape requires a literal `channels` tuple**
(`readonly [Channel, ...Channel[]]`). A runtime built from a
dynamically-assembled `Channel[]` is unbranded and still needs `?.`. Now
stated in the node TSDoc.
- **`examples/slack/app/managed.ts` now exits nonzero if activation
fails**, where before it stayed up serving HTTP with nothing connected.
Intentional — fail loud, and it matches `index.ts`. Note that `ready()`
resolves for `setup_required`, so a declared-but-unprovisioned channel
still logs as started.
- **Signal handlers are registered before awaiting activation** in
`managed.ts`, so a Ctrl-C inside the 30s activation window still tears
the Channel down instead of hitting Node's default handler.

## Verification

- **Type contract, red → green:** the new `KeyIsRequired<typeof
listener, "channels">` assertion in `handler-channels-types.test.ts`
failed to compile before the overload (`error TS2344: Type 'false' does
not satisfy the constraint 'true'`) and passes after.
- **Example bug, red → green:** stashing only `managed.ts` fails the new
guard with `expected "vi.fn()" to be called once, but got 0 times`.
- **Strict-null proof:** `slack-example` and `teams-example` both `tsc
--noEmit` clean under `strict: true` with the `?.` removed. This matters
because the runtime package compiles with `strict: false`, so its own
type test can only probe the optionality modifier structurally.
- Runtime channel suites 54/54; slack example 63/63.
- **Coverage limit:** the `managed.ts` guard is mocked — it proves the
example *calls* `ready()` with a bound, not that a Channel connects.
Nothing in CI exercises a real gateway connect for these examples.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-28 14:11:53 -05:00
Maxim 0850c5e522 Merge remote-tracking branch 'origin/main' into blitz/glass-inspector/integration
Realigns the inspector/memory work onto the banking demo as it shipped in
#6136 (ChatGPT-style shell, gen-UI beats, durable-memory self-learning) and
#6202 (README refresh).

All six conflicts were the same collision: this branch removes the bespoke
Glass Engine inspector, while #6136 kept and rebuilt around it.

- run-handler.ts: kept both sides (our CopilotKitCoreCatalogComponent and
  main's MAX_FOLLOW_UP_DEPTH landed at the same spot).
- wrapper.tsx / layout.tsx: took main's rewritten provider tree and
  right-hand icon rail, minus the Glass Engine providers, pane, and
  telescope toggle. Also dropped main's `padClass` (it reserved space for
  the Glass pane and referenced a now-removed `glassActive`) and
  `<ProactiveNotice />` (main removed it; the import is already gone).
- memory-tab.tsx, lib/intelligence/memory.ts: confirmed the deletions.
  Their only remaining importers were the bespoke inspector and the
  banking-local /api/memories routes, all removed here. seed-memories.ts
  is unaffected: it POSTs to INTELLIGENCE_API_URL, not the local route.
- README.md: kept our product-inspector section over main's Glass Engine
  availability/activation prose, and documented the Capabilities tab.

Drive-by fixes to comment rot the migration created: user-id.ts and the
copilotkit route doc comments referenced the deleted Memory-panel proxies,
and the README pointed the presenter-reset control at the removed
telescope toggle.

Also replaces a literal NUL byte in capabilityKey() with a unicode escape.
The raw control character made tsc/grep/diff treat run-handler.ts as a
binary file, which hid this very merge's conflict markers from grep.
Behavior is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-28 21:10:21 +02:00
Benjamin Taylor a26990971e docs(runtime): note that the non-optional channels shape needs a literal channels tuple (OSS-646) 2026-07-28 13:26:05 -05:00
Ben Taylor 661ac447be feat(runtime): default the Intelligence platform URLs to the managed service (#6205)
Makes `new CopilotKitIntelligence({ apiKey })` the whole managed-service
setup. `apiUrl` and `wsUrl` were required on every construction, so the
two correct hosts had to be found and copied by hand — which is how an
agent came to invent them. Linear:
[OSS-638](https://linear.app/copilotkit/issue/OSS-638/add-default-rest-and-ws-api-urls-to-intelligence-sdk).

## The change

Both fields are now optional and default to CopilotKit's managed
platform (`https://api.intelligence.copilotkit.ai` /
`wss://realtime.intelligence.copilotkit.ai`). Overrides are unchanged
for self-hosted and non-production deployments, with two guards that the
previous required-field signature made unnecessary:

**A blank value counts as unset.** These URLs are almost always wired
from env vars, and a declared-but-empty variable
(`COPILOTKIT_INTELLIGENCE_URL=`, common in generated `.env` files and
container configs) arrives as `""`. Left as-is it slips past a `??`
default and produces host-relative requests — the failing test showed a
request to `/api/threads?...` with no host at all.

**Setting only one of the pair warns.** The API and realtime planes are
separate hosts, so a lone override silently pairs (say) a self-hosted
API with the managed gateway. That failure surfaces as a 30s hang rather
than an error, because the socket layer treats an unreachable host as a
retryable reconnect — so it is worth a log line at construction.

## Docs

Swept the copy-paste surfaces to the short form so nobody is handed URLs
to get wrong: channels quickstart/index/mcp + the 5 platform pages, the
7 channels package READMEs, the runtime and react-core skill references
(package and root copies), and the slack/teams examples with their
READMEs and `.env.example`s. The "separate hosts, override both or
neither" lesson now lives in the quickstart's override note and the SDK
JSDoc instead of in every snippet.

Two things beyond the mechanical sweep:

- `dev-docs/architecture/setup-intelligence.md` documented `apiUrl:
"https://your-intelligence-host/api"` and `wsUrl:
"wss://your-intelligence-host/socket"` — the exact double-`/api` and
fake-`/socket` suffixes the runtime skill reference warns against. Fixed
while here.
- The `CopilotKitIntelligence` class JSDoc sat above
`ThreadDeletedPayload`, so it never appeared on hover of the class.
Reattached — that is the discoverability half of the original complaint.

The integration examples (langgraph, mastra, adk, …) are deliberately
untouched: they pass `process.env.INTELLIGENCE_API_URL ??
"http://localhost:4201"` to target a local stack, where the managed
defaults would be wrong.

## Verification

Tests written first and watched fail — 5 on undefined URLs, then the
blank-URL case on the host-relative request.

- `client.test.ts` 54/54; full `runtime/src/v2` suite 1128/1128
- slack example 63/63, teams example 2/2, channels-intelligence 195/195
- `tsc --noEmit` clean on `packages/runtime`; both edited examples
typecheck clean against the new signature (verified by resolving
`@copilotkit/runtime` at the worktree source, since the symlinked
`node_modules` otherwise reports stale required-field types)
- oxlint 0 warnings / 0 errors on touched files, oxfmt clean

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-28 13:23:02 -05:00
Ben Taylor 73eaecdbe4 fix(channels): surface run_error on HTTP-fallback path and isolate ack failures (closes OSS-491) (#6034)
## What

Two SDK-side delivery / terminal-signal hardening fixes in the managed
channels
adapter (`packages/channels-intelligence/src/intelligence-adapter.ts`),
from the
pre-merge adversarial CR of #5972. Closes OSS-491.

## Scope correction (verified end-to-end during review)

The ticket originally framed Finding 1 as a live "self-hosted /
no-Outbox"
silent-failure. That premise is **wrong**, and I traced the full render
path to
confirm it:

- Every real deployment wires a render sink: the realtime gateway
launcher passes
one, and the config-free `intelligenceAdapter()` auto-wires
`HttpRenderEventSink`
  in `start()`. On that path a `run_error` is POSTed to app-api's
`/api/channels/deliveries/:id/render-events/accept` route, durably
persisted,
projected into canonical history as a `RUN_ERROR` terminal, **and** the
Connector
  Outbox posts a user-visible `⚠️ Agent error: …` to Slack.
- The Connector Outbox is the **same app-api binary** and runs in
**self-hosted / composite** deployments too (gated only by the channels
feature
flag, not deployment mode). Self-hosted handles `run_error` identically
— there
  is no server-side gap.
- The ad-hoc fallback sink in `renderSinkFor` — the code Finding 1
patches — is
reached **only** when a consumer injects both `source` and `egress` but
no
  `renderSink` (in-memory tests, and hypothetical BYO custom-transport
  integrations). It is not on any default or self-hosted path.

So Finding 1 is **defensive hardening of a fallback plus a lifecycle
fix**, not a
HIGH-severity production silent-failure (severity retriaged to Low on
the ticket).
Finding 2 is on the shared `dispatch()` path and does affect real
deployments.

## Finding 1 — `run_error` on the ad-hoc HTTP-fallback sink

`renderSinkFor`'s ad-hoc plain-text sink (used only when no `renderSink`
is wired)
had no `run_error` branch, so on that path a `run_error` was dropped,
the run
resolved, and `dispatch()` acked a text-less turn as success — silent,
nothing
logged. Now that sink logs the raw error via `config.log`, flushes any
buffered
partial text, and posts a generic user-visible error. Posted + logged +
**acked, not nacked**: a `run_error` is frequently a deterministic agent
failure
and the Channel path reprocesses redeliveries (`skipIngressDedup`), so
nacking
would livelock. A genuine egress failure of the error post itself still
nacks
(`emit` throws → `drain()` rejects).

Also — and this part **does** apply to the realtime path —
`onRunErrorEvent` now
calls `ensureRunStarted()`, so a `run_error` that precedes any text/tool
event
emits a preceding `run_started` (parity with the text/tool handlers; a
well-formed
run lifecycle on both paths). This is the one intentional realtime-path
change: a
text-less error now yields `[run_started, run_error, finalize]` instead
of
`[run_error, finalize]`. `run_error` **rendering** is unchanged — still
a frame to
the sink, never converted to an egress error post.

## Finding 2 — a failed `ack()` no longer nacks a completed turn

`ack()` sat inside the same `try` as `dispatchTo`, so an `ack()` network
failure
fell into the catch and nacked an already-processed turn with a
misleading reason;
with `skipIngressDedup`, the whole turn reran (duplicate agent run —
re-calls the
model, re-runs tools — even though egress op-id dedupe catches duplicate
posts).
Ack is now a distinct phase: a failed ack is logged, never nacked. This
kills the
**spurious** rerun — notably the common "ack landed server-side but the
response
was lost" case. A genuinely-failed ack still redelivers via lease expiry
(inherent
to at-least-once, unavoidable); the turn already posted its reply, so
only
delivery bookkeeping is delayed.

## Tests

TDD; the behavior tests were written failing first.

- Fallback text-less `run_error` ⇒ emits a user-visible error post
**and** logs it.
- `run_error` before any text/tool event ⇒ `run_started` precedes it.
- Realtime path ⇒ `run_error` still forwarded to the render sink,
**not** converted
  to an egress post (regression guard).
- Failed `ack()` on a successful turn ⇒ not nacked, no rerun, ack
failure logged.
- Failed error-post emit on the fallback path ⇒ `drain()` rejects
(nacks/retries).
- Buffered partial text flushed before the error post (order preserved).

## Verification

- `nx run @copilotkit/channels-intelligence:check-types` ✅
- `nx run @copilotkit/channels-intelligence:test` ✅ (all green)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-28 13:16:44 -05:00
Benjamin Taylor 3f0bbe4a7b feat(runtime): default the Intelligence platform URLs to the managed service
`CopilotKitIntelligence` required `apiUrl` and `wsUrl` on every construction,
so the two correct hosts had to be found and copied by hand — which is how an
agent came to invent them. Both now default to CopilotKit's managed platform,
making `new CopilotKitIntelligence({ apiKey })` the whole managed-service setup.

Overrides are unchanged for self-hosted and non-production deployments, with two
guards that the previous required-field signature made unnecessary:

- A blank value counts as unset. These URLs are usually wired from env vars, and
  a declared-but-empty variable arrives as `""`, which would otherwise produce
  host-relative requests instead of falling back to the managed platform.
- Setting only one of the pair warns. The API and realtime planes are separate
  hosts, so a lone override silently splits the client across two deployments —
  and that failure surfaces as a hang, not an error.

Sweeps the doc, skill, README, and example surfaces to the short form so the
copy-paste path no longer hands anyone URLs to get wrong, and reattaches the
`CopilotKitIntelligence` class JSDoc, which was orphaned above an interface and
so never appeared on hover.

Linear: OSS-638
2026-07-28 13:12:15 -05:00
Benjamin Taylor 956c6448a7 fix(runtime): non-optional listener.channels and honest lifecycle docs (OSS-646)
`createCopilotNodeListener` now mirrors `createCopilotRuntimeHandler`'s branded
overload pair, so a runtime with at least one declared Channel yields a listener
whose `.channels` is non-optional and the documented `listener.channels.ready()`
call type-checks with no `!` and no `?.`. `activateChannels: false` and
channel-less runtimes keep the optional shape. Both listener types are exported
from `@copilotkit/runtime/v2/node`.

Corrects TSDoc on the node, express, and hono wrappers that still claimed
activation happens "at creation time" and labelled `ready()` as optional — stale
since activation was deferred to make the Fetch handler serverless-safe. On a
long-running host that call is required, not optional.

Fixes a live consequence of that stale model: `examples/slack/app/managed.ts`
never called `ready()`, so it mounted a listener, logged "started managed
Channel", and connected nothing. Covered by a regression assertion.

Drops the now-unnecessary `?.` from the examples, READMEs, and channel docs, and
adds compile-time contracts for the listener shape alongside the existing
handler ones. The examples compile with `strict: true`, so they prove the
`?.`-free call under strict null checks, which the runtime package (strict:
false) cannot.
2026-07-28 13:11:55 -05:00
Benjamin Taylor a9a073e145 test(channels): cover fallback run_error egress-failure and partial-text ordering (OSS-491)
Close two assert-without-a-test gaps found in review of the OSS-491 fix:
a failed error-post emit on the fallback run_error path propagates (drain
rejects → the delivery nacks/retries rather than acking a dropped error),
and buffered partial text is flushed before the error post (order preserved,
no lost or duplicated output).
2026-07-28 13:04:25 -05:00
Benjamin Taylor fc0bb4f498 fix(channels): surface run_error on HTTP-fallback path and isolate ack failures (closes OSS-491)
Two pre-existing SDK delivery / terminal-signal defects in the managed
channels adapter, surfaced by the pre-merge adversarial CR of #5972 and
deliberately left out of that PR's scope. Both bite the HTTP-fallback /
self-hosted path (no Connector Outbox); the realtime + Outbox path is
unchanged.

Finding 1 (HIGH) — run_error swallowed on the HTTP-fallback render path,
delivery acked as SUCCESS. `renderSinkFor`'s ad-hoc plain-text sink had no
`run_error` branch, so an agent-side failure with no accompanying text (model
error, tool crash, upstream 5xx) resolved the run, drained, and acked the
delivery as success — no user-visible message and nothing logged ("bot goes
silent on every error"). Now the fallback sink logs the raw error via
`config.log`, flushes any buffered partial text, and posts a generic
user-visible error. Posted + logged + acked (not nacked) on purpose: a
run_error is frequently a deterministic agent failure and the Channel path
reprocesses redeliveries (skipIngressDedup), so nacking would livelock. A
genuine egress failure of the error post itself still nacks (emit throws →
drain rejects). Also: `onRunErrorEvent` now calls `ensureRunStarted()` so a
run_error that precedes any text/tool event still emits a preceding
run_started.

Finding 2 (LOW) — a failed ack() nacked an already-processed turn. `ack()`
sat inside the same try as `dispatchTo`, so an ack network failure fell into
the catch and nacked a completed turn with a misleading reason; because the
Channel path reprocesses redeliveries, the whole turn reran (duplicate agent
run / posts). Ack is now a distinct phase: a failed ack is logged, never
conflated with a turn failure and never nacked.
2026-07-28 13:04:25 -05:00
Tyler Slaton 2eba274a99 fix(channels-core): default runAgent prompt from inbound message 2026-07-28 13:58:11 -04:00
Benjamin Taylor fdcc0150ab fix(channels-intelligence): classify channel_not_declared as setup-required
A runtime that declares a managed channel the Intelligence project does not have
crashed startup: ready() rejected and the channel settled to error. Creating the
channel and attaching credentials is the setup step every new channel goes
through, so this was the first run of every managed channel, before the happy
path.

app-api had no declaration-response token for "the project has no channel by that
name", so it sent runtime_not_declared. That token means the inverse on a join —
a channel exists but no runtime declared it, while on a join the runtime is the
one declaring. classifyJoinError read it literally, which was correct, and so
treated a benign setup gap as a hard failure.

app-api now sends channel_not_declared for that case (Intelligence#615). Adding it
to SETUP_REQUIRED_CHANNEL_STATES lets the channel settle to setup_required and
ready() resolve. runtime_not_declared keeps its literal reading and stays a hard
error, as do runtime_conflict, platform_setup_failed, delivery_failed,
egress_failed and runtime_offline.

This also answers the TODO(gateway-owner) the classifier carried, which asked
whether runtime_not_declared was ever benign. It was, but not for the reason the
comment guessed: the server was overloading the token, not racing. Replaced the
TODO with what the answer turned out to be.

npx vitest run src/realtime-gateway.test.ts: 21 pass

33 failures elsewhere in this package are pre-existing on origin/main (a stale
@copilotkit/channels build missing the runtime symbol) and unrelated.
2026-07-28 12:49:37 -05:00
Benjamin Taylor 5c2faac569 test(runtime,shared): make telemetry tests control their own env
The OSS-565 workflow env vars leaked into the unit job and turned 17 tests
red across @copilotkit/shared and @copilotkit/runtime. Every one of them
assumed telemetry was enabled because the ambient environment happened to
have no opt-out set — they were asserting on the environment, not the code.
A developer with DO_NOT_TRACK exported would have seen the same failures.

- shared telemetry-client + runtime get-runtime-info: clear the opt-out vars
  per test (and restore), so each case sets the state it asserts on.
- The three license→sink integration suites need `vi.hoisted`, not a hook:
  the telemetry singleton latches `telemetryDisabled` at module-import time,
  which happens before any beforeEach runs.

Verified both with the CI env set and with it unset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 12:39:46 -05:00
Ben Taylor 3b9d2c27b5 fix(channels-intelligence): surface an unreachable gateway host instead of hanging the connect (#6191)
Closes OSS-623 (split out of OSS-621's acceptance item 3).

## The bug

A managed Channel whose `wsUrl` points at a wrong or unreachable host
sat in `connecting` for the caller's full settle budget and then
rejected with `did not settle in time` — no indication that the *host*
was the problem. A wrong **API** host fails fast with a clean HTTP
error; a wrong **WS** host hung.

Three failure modes collapsed into that one opaque timeout: DNS that
does not resolve, a refused/failed TCP or TLS connect, and a host that
serves HTTP with no Phoenix socket mounted at the endpoint (the prod
split-plane trap — `api.intelligence` and `realtime.intelligence` are
separate hosts).

## Why it hung

Two Phoenix behaviors compound, and the second is why the existing join
timeout did not save us:

1. `Socket` retries a failed **first** connect forever and reports
transport errors as reconnect churn, so nothing distinguishes "still
handshaking" from "this host will never work."
2. The join push's own `timeoutMs` is silently defused.
`Channel.onError` calls `joinPush.reset()` while the channel is joining
(phoenix 1.8.4 `channel.js`), which nulls `refEvent` and `off`s the
reply binding — the armed timeout then fires into a dead event name and
no `"timeout"` recHook ever runs.

Net: the connect await hung *indefinitely*; the 30s in the ticket was
purely `ChannelManager.ready({ timeoutMs })`'s deadline, which is why
the message was a stopwatch.

## The fix

**Watch the initial connect separately from reconnects**, keyed off
whether the socket has ever opened (phoenix passes exactly that as
`socket.onError`'s third argument, `establishedConnections`):

- **Never opened** → reject with a new `RealtimeGatewayUnreachableError`
naming the endpoint and the cause, and tear the socket down so it stops
retrying a host that will not answer.
- **Opened, then dropped** → unchanged; the existing `reconnecting` /
`gave_up` machine owns it.

**Get the cause from HTTP, because the socket does not have it.** This
is the part an adversarial review of the first commit corrected. Node's
global WebSocket (undici — the production default) dispatches the *same*
detail-free `ErrorEvent` for all three modes: `"Received network error
or non-101 status code."`, no OS code, an inner `Error` with nothing
more. Browsers are deliberately worse. So on the failure path only, we
probe the same origin **and path** over http(s) with `fetch`, which does
discriminate — verified against the real network:

| target | WebSocket says | `fetch` says |
|---|---|---|
| nonexistent host | network error or non-101 | `cause.code = ENOTFOUND`
|
| closed port | network error or non-101 | `cause.code = ECONNREFUSED` |
| `https://example.com/runner` | network error or non-101 | **HTTP 404**
|

A transport that *does* expose an OS code (the `ws` package, custom
transports) is trusted directly and skips the probe; a successful
connect issues no request at all.

**Failing fast vs. tolerating a boot race** is split by what the
diagnosis found:

- **The host does not resolve** → fail immediately. Measured against the
real network: **25ms**, message `the host does not resolve (getaddrinfo
ENOTFOUND …)`. This is the OSS-621 case (`realtime.copilotkit.ai` has no
DNS record at all).
- **Anything else** → wait out `connectTimeoutMs` (default **30s**,
plumbed through the launcher). A `ChannelManager` marks an activation
failure `error` **permanently, with no retry**, so a shorter window
would convert a rolling restart of the gateway into a dead Channel where
the old forever-retry would have recovered.

Per acceptance item 4, the error deliberately does **not** carry the
`SETUP_REQUIRED` marker. An unreachable host is a config problem, but
`setup_required` is a benign settled state that lets `ready()` resolve —
a typo'd `wsUrl` must reject loudly, not go quiet-green.

## What the messages look like now

```
realtime gateway unreachable: the socket never connected to wss://oss623-does-not-exist.copilotkit.ai/runner
  (the host does not resolve (getaddrinfo ENOTFOUND oss623-does-not-exist.copilotkit.ai))
  — check that the Intelligence wsUrl points at the gateway host          [25ms]

realtime gateway unreachable: the socket never connected to ws://127.0.0.1:45999/runner
  (the host refused the connection (connect ECONNREFUSED 127.0.0.1:45999))
  — check that the Intelligence wsUrl points at the gateway host          [full window]

realtime gateway unreachable: the socket never connected to wss://example.com/runner
  (the host answered HTTP 404 for https://example.com/runner, so it resolves and accepts
   connections — only the WebSocket handshake failed, so the wsUrl path may not be the
   gateway's socket endpoint)                                            [full window]
```

## Tests

Rebuilt around the shapes the real implementations emit, with the undici
shape as the default case (the first version's fake fed a `ws`-shaped
coded error undici never produces, which is what hid the dead code):

- The host does not resolve → fails in <1s despite a 5s window, probes
host **and path**, names the cause, carries `code ===
"GATEWAY_UNREACHABLE"` and the underlying `cause`, is not
`SETUP_REQUIRED`, socket closed.
- Refused → waits out the window, then names `ECONNREFUSED`.
- Host answers HTTP 404 → reported as a handshake/path problem, not a
missing host.
- A transport that exposes an OS code → fails fast with **no** probe
issued.
- A connect that hangs with no transport event at all → bounded by the
connect window (not a 60s join timeout), and the deadline itself starts
the probe so a cause is still reported.
- Neither transport nor probe yields a cause → still names the endpoint.
- **The shipped probe against a real closed loopback port** — no seam,
no DNS, no egress.
- A gateway that fails once and accepts on retry still connects (the
boot race).
- A healthy connect issues no diagnostic request.
- The connect deadline is disarmed after a successful connect — a later
drop is still a reconnect episode.

Before the fix, four of these hang until vitest's own 5s timeout — the
bug reproduced directly.

## Verification

`check-types` clean; full package suite 195/195; oxlint + oxfmt clean;
lefthook pre-commit (whole package suite + `check:packages`) green. All
three failure modes additionally verified end-to-end against the real
network and the real `fetch`, not just fakes.

## Known adjacent case, not in scope

A socket that opens and *then* drops before the initial join reply hits
the same defused-join-timeout behavior (2 above) and still relies on the
caller's settle deadline. That path can genuinely self-heal via
Phoenix's rejoin, so it keeps the retry; bounding it is a separate call
from OSS-623's "never connected once."

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-28 12:33:11 -05:00
Benjamin Taylor 749a8fb2b3 refactor(web-inspector): move the telemetry egress guard under src
Importing the factory from vitest.setup.ts tripped TS6059 (file outside
rootDir) in the package's typecheck. The helper now lives in
src/lib/testing/ — test-only, never referenced by the bundle entry, and
confirmed absent from dist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 11:54:56 -05:00
Benjamin Taylor 20369b990c fix(web-inspector): bound the pre-handshake telemetry queues
An open/close loop against a runtime that never connects would otherwise
queue indefinitely and burst on reconnect. Caps both pending queues at 20
events — this is a signal, not a ledger.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 11:53:27 -05:00