Two related Inspector-telemetry tickets: **OSS-566** and **OSS-568**.
## OSS-566 — explicit "Inspector opened" event
There was no event recording that the panel was opened. Opens could only
be inferred from in-panel activity (~1,655/90d, a floor) or from
`banner_clicked` cta=`body` (~511), which misses the common
floating-button path entirely.
Adds `oss.inspector.opened` with:
| property | values |
|---|---|
| `open_source` | `floating_button` \| `announcement_preview` |
| `has_unseen_announcement` | whether an announcement was on screen at
open time |
| `license_status` / `runtime_mode` / `runtime_url_type` | same
segmentation the threads events already carry |
| `package_name` / `package_version` / `inspector_distinct_id` | version
segmentation |
**Restoring a persisted-open panel deliberately does not count.**
Restore assigns `isOpen` directly instead of routing through
`openInspector()`, so page reloads — and every `next dev` hot reload —
stay out of the number.
## OSS-568 — banner surface + first-class dismissal
1. **`surface` on `banner_viewed`** — `collapsed_preview` (bubble on the
collapsed widget) vs `expanded_card` (card inside the opened panel),
stamped at fire time. Dedup is now per `(banner, surface)` instead of
per banner, so opening the panel records the card impression as its own
signal.
2. **`oss.inspector.banner_dismissed`** — emitted **in addition to**
`banner_clicked { cta: "dismiss" }`, not replacing it, so dashboards
reading the `cta` value keep working. Carries `surface` too, separating
"swatted the bubble away" from "dismissed the card after opening".
Both new events clear the sink's `oss.inspector.` prefix gate, so **no
telemetry-sink deploy is needed**.
## Testing
- **`packages/web-inspector` full suite — 112 passed (4 files)**, run
locally in the worktree:
```
✓ dev/css-raw-import.spec.ts (1 test) 1ms
✓ src/__tests__/telemetry-egress-guard.spec.ts (3 tests) 2ms
✓ src/lib/__tests__/telemetry.test.ts (28 tests) 8ms
✓ src/__tests__/web-inspector.spec.ts (80 tests) 890ms
Test Files 4 passed (4)
Tests 112 passed (112)
```
- **New coverage**: payload shape for `opened` / `banner_dismissed`,
incl. an allow-list assertion that no content/PII key can be added
accidentally; collapsed→expanded surface sequence on open; per-surface
dedup; open attribution for both sources; no event for an already-open
panel; no event for a restored-open panel; nothing emitted when the
runtime reports `telemetryDisabled`; an open still recorded while the
runtime is disconnected.
- **`tsc --noEmit`** on `@copilotkit/web-inspector`: clean (after
building `core` + `shared` dist in the worktree).
- **`tsdown` build**: succeeds; the test-only egress-guard helper is
**not** present in `dist/`.
- **`oxfmt --check`**: clean. **`oxlint`**: 9 warnings, all
pre-existing.
- `@copilotkit/runtime` (1,760) and `@copilotkit/shared` (199) also
green — both are back on main's own test files in this PR.
## A test-only egress guard rides along
`vitest.setup.ts` installs a fetch guard that swallows requests to the
telemetry sink. This is **not** CI plumbing — it is a prerequisite for
the new events. These tests run in jsdom, where a real `fetch` exists,
and inspector telemetry is fire-and-forget, so any test that drives a
banner / threads / open path without stubbing fetch POSTs a real
`oss.inspector.*` event to the live sink, from developer machines as
well as CI. The announcement-dismissal tests were already doing this;
the new `opened` / `banner_dismissed` tests hit the same send path. No
environment variable can prevent it, because the inspector's opt-out
arrives in the runtime's `/info` response and these tests never boot a
runtime.
## Not in scope
Suppressing telemetry from CI jobs that boot real apps (**OSS-565**) was
explored on this branch and removed. It needs a mechanism that does not
depend on the `/info` handshake — the env → `/info` → core chain is
asynchronous, so an early interaction beats it. That ticket stays open
and unaddressed here.
Closes OSS-566, OSS-568.
Both blockers from the third review.
**1. `banner_viewed` lost its pre-handshake queue.** On `main`,
`fetchAnnouncement()` stored `pendingBannerViewed` and the flush waited for
`connected`, discarding the event when the runtime reported `telemetryDisabled`.
Removing the deferral in the previous commit made the impression post directly,
so it now went out on the `false` placeholder that `telemetryDisabled` reads
before /info answers. That is a regression this PR introduced for an existing
event, distinct from the pre-existing `banner_clicked` race. The queue is
restored — widened from a single slot to a bounded array, since opening the panel
reveals the second surface and can happen before the first impression flushes —
and per-surface dedup is unchanged.
**2. `opened` recorded segmentation it could not know.** `licenseStatus` is
undefined and `runtimeMode` reads its `sse` default before the handshake
(`agent-registry.ts:77`), so an early open against an Intelligence runtime was
permanently filed as SSE. Rather than queue the open — a disconnected runtime
should not suppress a real user's open — `license_status` and `runtime_mode` are
now omitted unless the handshake has landed. An absent dimension is honest; a
wrong one corrupts the segmentation OSS-566 exists to provide. `runtime_url_type`
is config-derived and stays, since it is accurate immediately.
Also fixes a test that passed for the wrong reason: `HeaderMockCore` had no
`runtimeMode` at all, so a pre-handshake assertion about it read `undefined`
either way and would have gone green against the very bug it was meant to catch.
The mock now mirrors the real pre-handshake state (`runtimeMode: "sse"`,
`licenseStatus` absent). With that corrected, 3 of the 4 new cases fail against
the previous commit; the fourth guards the connected path.
Coverage: impression held then dropped on disabled, held then released on
allowed, open before the handshake omits both fields while keeping
`runtime_url_type`, and an open against a connected Intelligence runtime carries
`runtime_mode: intelligence` + `license_status: valid`.
Verified: web-inspector 115 passed, tsc clean, tsdown clean, oxfmt clean, oxlint
9 warnings all pre-existing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Narrows this PR to OSS-566 + OSS-568. The OSS-565 CI-suppression effort is
backed out entirely:
- the 10 workflows' `COPILOTKIT_TELEMETRY_DISABLED` / `DO_NOT_TRACK` env and the
egress-block steps
- `examples/e2e/playwright.config.ts` env defaults
- the runtime/shared test-env manipulation and the `telemetry-opt-out-env`
helper
- the `navigator.webdriver` check and its tests
Every reverted path is byte-identical to the merge-base — verified rather than
assumed, since `git checkout origin/main -- <path>` would otherwise have
imported main's newer content for files that had moved on.
One piece is deliberately kept, because it is not about CI: the jsdom egress
guard. These tests run with a real `fetch`, and the inspector's telemetry is
fire-and-forget, so any test touching a banner / threads / open path without
stubbing fetch POSTs a real `oss.inspector.*` event to the live sink — from
developer machines as well as CI. The announcement-dismissal tests did exactly
that before the guard, and the new `opened` / `banner_dismissed` tests hit the
same send path, so shipping the events without it would make our own dataset
dirtier. Its comments no longer cite OSS-565; the rationale stands on its own.
What remains: `oss.inspector.opened` with open-source attribution, the
`banner_viewed` surface split, and first-class `banner_dismissed`. No inspector
telemetry waits on the /info handshake.
Verified: web-inspector 112, runtime 1,760, shared 199 (the last two now on
main's own test files). tsc clean, oxfmt clean, oxlint 9 warnings all
pre-existing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the /info-coupled approach with two mechanisms that need no handshake,
and backs the coupling out entirely.
The problem with gating on /info: `telemetryDisabled` only reaches the browser
through an async handshake, so either events race it (the hole) or they wait for
it (fail-closed). Waiting means an inspector whose runtime never connects — or
whose /info 404s, which sets status to `Error`, not `Connected` — goes silent for
the whole session. That penalizes real users with broken setups, who are exactly
the people most likely to have the inspector open.
Instead:
1. `navigator.webdriver` in `track()`. Spec'd, synchronous, available on the
first line of script, and true under every Playwright launch — verified
against this repo's Playwright 1.59.1 in both headless and headed mode. Needs
no per-app env plumbing and covers pages served by deployed environments,
which no CI-side variable could reach. Suppresses only on an explicit `true`:
absent/falsy means "not automation", never "unknown, so stay quiet".
2. A job-wide egress block (`.github/scripts/block-telemetry-egress.sh`) pointing
the sink at loopback for both address families, added to the 12 CI jobs that
run or boot real CopilotKit code. Backstop for anything that reads neither the
env nor the DOM. The frozen integration container gets the same via
`--add-host`, since neither the workflow env nor the runner's hosts file
crosses that boundary.
Backed out of the inspector: the egress gate and held queue, `pendingOpened`,
and `pendingBannerViewed` (the last pre-dating this PR). Inspector telemetry no
longer references the handshake at all — `index.ts` is net −94 lines and the
opt-out is now a single boolean check per call site, as it reads.
Two bugs this surfaced, both caught by existing or new tests rather than
shipped: removing the flush step also removed the only `telemetryDisabled` check
on the `banner_viewed` path, and a thread-refetch test counted calls on a stubbed
global `fetch`, so telemetry that now sends immediately inflated it — scoped to
the thread URLs it actually means.
Also corrects the workflow comments, which credited the /info chain for
browser-side suppression it never reliably provided.
Verified: web-inspector 116, runtime 1,760, shared 199. tsc clean, oxfmt clean,
oxlint back to the baseline 8 warnings (fixed a dead type import and a
non-capturing test helper I introduced). actionlint 344 findings before and
after, byte-identical modulo shifted line numbers. shellcheck clean. tsdown
build ships the check in dist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit closed the pre-handshake hole for the three events this PR
touches, but every other inspector event still posted on the placeholder.
`AgentRegistry._telemetryDisabled` starts `false`, so the per-call-site
`if (this.core?.telemetryDisabled) return` check passes while disconnected and
the event goes out — reproduced with the threads tab, which emitted
`threads_tab_clicked` before /info could report the opt-out. 15 call sites
shared the flaw (threads tab, memories tab, view-state, CTAs, example tour).
Rather than queue at each of them, the gate now lives in `telemetry.ts` at the
single egress boundary: `track()` holds events while the runtime decision is
unknown, and the inspector calls `resolveTelemetryGate("allowed" | "denied")`
once the handshake lands. Nothing reaches the network without an explicit
allow.
That makes the per-event queues added in e7e8922 redundant, so
`pendingBannerViewed` and `pendingBannerInteractions` are gone (net −101 lines
in index.ts). `pendingOpened` stays: it exists to resolve `license_status` /
`runtime_mode` / `runtime_url_type` at flush time, which is a payload concern
rather than a deferral one. The gate resets on `detachFromCore` so a later core
starts from "unknown" instead of inheriting the previous runtime's decision.
Two fidelity details the refactor forced, both covered: held events carry the
timestamp of the interaction rather than of the release, and the opt-out is
re-checked at release in case the user opts out mid-handshake.
Trade worth stating: an inspector whose runtime never connects now emits
nothing. Consent cannot be established without the handshake, so silence is
the correct default.
Coverage: threads tab held then dropped on disabled / sent on allowed, plus a
gate unit suite (unknown holds, allowed releases in order, denied discards,
post-denial drops, queue bound, reset, opt-out precedence, event-time ts).
web-inspector 127 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three review findings on the inspector-telemetry PR.
Banner interactions bypassed the runtime opt-out before the handshake.
`telemetryDisabled` reads `false` until /info answers, so a dismissal (or a
body click) handled while disconnected posted on that placeholder — even when
the runtime was about to report `telemetryDisabled: true`. `banner_clicked`
and `banner_dismissed` now defer behind the same connected gate as
`banner_viewed`. Dedup still happens at interaction time, so a double-click
queues one event rather than two, and the queue carries the same bound as the
others.
Queued opens preserved pre-handshake placeholder segmentation. `trackOpened`
snapshotted `licenseStatus` (undefined) and `runtimeMode` (its `sse` default)
at queue time, and the flush sent them unchanged — permanently attributing an
early open against an Intelligence runtime to SSE. Only the open *intent* is
queued now; `license_status` / `runtime_mode` / `runtime_url_type` are
resolved at flush time, once /info has landed.
Test env overrides left the opt-out cleared. Suites that must clear
`COPILOTKIT_TELEMETRY_DISABLED` / `DO_NOT_TRACK` now capture the originals and
restore them in `afterAll`, via a shared `telemetry-opt-out-env` helper. The
hoisted cases snapshot inline and return it, since hoisted callbacks run
before the imports they would otherwise need. Measured: under the repo's
default `isolate: true`, vitest forks a fresh process per file, so the delete
does not currently cross files (pids differ; a `--no-isolate` control shares
one pid and does leak). Restoring is the defensive fix, not a live-bug fix.
Coverage: pre-handshake dismissal dropped when the runtime then reports
disabled; sent when it allows; double-click dedup; and a queued open flushed
against an Intelligence runtime carrying `runtime_mode: intelligence` +
`license_status: valid`. Three of the four fail against the pre-fix code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Realigns the inspector/memory work onto the banking demo as it shipped in
#6136 (ChatGPT-style shell, gen-UI beats, durable-memory self-learning) and
#6202 (README refresh).
All six conflicts were the same collision: this branch removes the bespoke
Glass Engine inspector, while #6136 kept and rebuilt around it.
- run-handler.ts: kept both sides (our CopilotKitCoreCatalogComponent and
main's MAX_FOLLOW_UP_DEPTH landed at the same spot).
- wrapper.tsx / layout.tsx: took main's rewritten provider tree and
right-hand icon rail, minus the Glass Engine providers, pane, and
telescope toggle. Also dropped main's `padClass` (it reserved space for
the Glass pane and referenced a now-removed `glassActive`) and
`<ProactiveNotice />` (main removed it; the import is already gone).
- memory-tab.tsx, lib/intelligence/memory.ts: confirmed the deletions.
Their only remaining importers were the bespoke inspector and the
banking-local /api/memories routes, all removed here. seed-memories.ts
is unaffected: it POSTs to INTELLIGENCE_API_URL, not the local route.
- README.md: kept our product-inspector section over main's Glass Engine
availability/activation prose, and documented the Capabilities tab.
Drive-by fixes to comment rot the migration created: user-id.ts and the
copilotkit route doc comments referenced the deleted Memory-panel proxies,
and the README pointed the presenter-reset control at the removed
telescope toggle.
Also replaces a literal NUL byte in capabilityKey() with a unicode escape.
The raw control character made tsc/grep/diff treat run-handler.ts as a
binary file, which hid this very merge's conflict markers from grep.
Behavior is unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
Importing the factory from vitest.setup.ts tripped TS6059 (file outside
rootDir) in the package's typecheck. The helper now lives in
src/lib/testing/ — test-only, never referenced by the bundle entry, and
confirmed absent from dist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An open/close loop against a runtime that never connects would otherwise
queue indefinitely and burst on reconnect. Caps both pending queues at 20
events — this is a signal, not a ledger.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OSS-565: no CI workflow disabled telemetry, so every job that runs or boots
real CopilotKit code emitted `oss.runtime.*` / `oss.inspector.*` events that
are indistinguishable from user activity in PostHog. Because the browser-side
events come from a real (headless) Chromium, the `$useragent != node`
heuristic can't filter them out after the fact — which is what made the
`banner_viewed` reach number untrustworthy.
Sets COPILOTKIT_TELEMETRY_DISABLED / DO_NOT_TRACK at the workflow level for
the jobs that execute app or runtime code (unit, integration, e2e-dojo,
e2e-legacy-v1, smoke-starter, showcase eval / frontend-matrix / on-demand).
The runtime reads them at startup and reports `telemetryDisabled` on /info,
which is also how the Inspector in a CI-booted app learns to stay quiet. The
frozen integration container gets them via explicit `--env` because
workflow-level env does not cross the container boundary, and the legacy-v1
Playwright webServer defaults them so local e2e runs are covered too.
The inspector's own unit tests can't be reached by an env var at all — its
telemetry keys off the /info handshake, not process.env — so they were
POSTing real events to the live sink on every run. A setup-file guard now
swallows requests to the sink and passes everything else through.
Out of scope, deliberately: showcase capture-previews drives Playwright
against DEPLOYED Railway apps, so no CI-side env can reach them; silencing
those would mean disabling telemetry on the deployed services, which would
also drop real visitors. Keep-alive needs nothing — /api/health is a static
route with no runtime and no browser.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OSS-566: there was no event recording that the Inspector panel was opened.
Opens could only be inferred from in-panel activity (a floor) or from
`banner_clicked` cta=body, which misses the floating-button path entirely.
Adds `oss.inspector.opened` with an `open_source` property, queued behind
the runtime handshake so an open is never reported for a runtime that has
telemetry disabled. Restoring a persisted-open panel deliberately does not
count — otherwise every reload / dev-server hot reload would be an "open".
OSS-568: `banner_viewed` fired once at fetch time and could not say which
of the two announcement surfaces the user actually saw — the bubble on the
collapsed widget or the card inside the opened panel. The event now carries
a `surface` stamped at fire time and dedups per (banner, surface), so
opening the panel records the card impression as its own signal. Dismissal
is also promoted to a first-class `oss.inspector.banner_dismissed` event,
emitted alongside the existing `banner_clicked { cta: "dismiss" }` so
dashboards reading the `cta` value keep working.
Both events pass the sink's `oss.inspector.` prefix gate, so no
telemetry-sink deploy is required.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Removes A3's optional "fired this session" capability-dot feature. Its
identity model was broken and unfixable without a redesign:
- Global tools never lit up: the tool-call subscriber keyed fired entries
as `${agentId}:${name}` with a real agentId, while a global tool's row
key is `:${name}`.
- Dots leaked across a core swap because firedCapabilities was never reset
in detachFromCore.
- A catalog component sharing a tool's name got a false dot (name-based
collision).
Rather than repair the fragile keying, the feature is removed entirely.
The Capabilities tab, its toggles, and enablement enforcement are
unaffected; _capabilitiesVersion is retained for toggle re-render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the single conflict in packages/web-inspector/src/index.ts by keeping
both additions: this branch's CpkMemoryList memory-tab element and main's
ɵCpkThreadDetails back-compat alias (independent top-level declarations).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the inspector's user-facing feature name from Memories to Learning:
the menu tab label and the enabled-state panel heading. The internal
"memories" menu key, the Brain icon, the REST wiring, and entry-level copy
(search box, empty states) are unchanged. Update the tab-presence test.
Ben Taylor