Commit Graph

12903 Commits

Author SHA1 Message Date
Tyler Slaton a95eb57da8 fix(docs): scope framework picker escape handling 2026-07-09 16:53:04 -07:00
Tyler Slaton 0b2863ee48 feat(docs): improve theme and search interactions 2026-07-09 15:51:00 -07:00
Tyler Slaton c67841df74 feat(docs): establish CopilotKit brand foundation 2026-07-09 14:50:31 -07:00
Ben Taylor bb7ad12075 feat(channels-intelligence): emit leaseToken on render-accept + complete (OSS-446) (#5899)
## Summary

SDK-emit half of **OSS-446** (lease-token fencing for hosted-bot
render/complete). The `fail` path already sends the delivery lease
token; **render-accept** and the **completion intent** did not — so
app-api fell back to the weaker instance-id + expiry check on those two
paths.

This makes the SDK send `leaseToken` on both, on **both transports**:

- **HTTP** — `HttpRenderEventSink.push` now includes `leaseToken`, via a
new `leaseTokenFor(deliveryId)` accessor on `HttpDeliverySource`
(mirrors the existing `scopeFor`). (`ack` already sent it.)
- **Phoenix** — `push` (render-accept) and `complete_requested` now
carry `leaseToken` from `DeliveryState` (`fail` already did).

## Why it's safe to ship now (ahead of the app-api "require" flip)

Verified end-to-end against the live Intelligence server:

- **Gateway** `validate_render_payload` allows `leaseToken` (optional,
`maybe_put_lease_token`) and forwards it via `accept_render_event`;
`validate_complete_payload` merges the payload through and forwards it
via `complete_delivery`.
- **app-api** fences render-accept (`$N IS NULL OR lease_token_hash =
$N`) and complete (`$N IS NULL OR ...`) **optionally** today — so
supplying the token starts fencing on it immediately, and omitting it
still works. No breakage; strictly a security improvement on the paths
that were previously unfenced.

## Scope

This is **half A** (SDK emit). **Half B** — flipping app-api's
render-accept + complete fences from optional to *required* and dropping
the instance-id/expiry fallback — is deferred until #511 (managed Teams)
settles `managed-bots/service.ts`, and follows from OSS-446's "require
*once the SDK sends it*" premise.

## Tests

- HTTP: render-accept POST asserts `leaseToken` from the claimed lease
flows into the body.
- Phoenix: render-accept + `complete_requested` payloads assert
`leaseToken`.
- Build + 95 tests green.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-09 16:30:39 -05:00
Benjamin Taylor 2bca275008 feat(channels-intelligence): emit leaseToken on render-accept + complete (OSS-446)
SDK-emit half of OSS-446 (lease-token fencing for hosted-bot render/complete).
The fail path already sends the lease token; render-accept and the completion
intent did not, so app-api fell back to instance-id + expiry there.

- HTTP: HttpRenderEventSink.push now includes leaseToken (new leaseTokenFor()
  bridge on HttpDeliverySource, mirroring scopeFor). (ack already sent it.)
- Phoenix: push (render-accept) and complete_requested now carry leaseToken
  from DeliveryState (fail already did).

Optional/forward-compatible: app-api + gateway already accept and fence on the
token when present (verified render/complete validators + fencing SQL), falling
back to the old check when absent — so this deploys safely ahead of the app-api
flip-to-required (OSS-446 half B), which waits on #511 settling managed-bots
service.ts.
2026-07-09 14:53:12 -05:00
Maxim 3c4da1affd feat(banking): data-bridged Open Generative UI (#5896)
## Summary

Adds **Open Generative UI (OGUI)** to the Northwind banking demo: the
agent can author sandboxed, interactive UI (rendered in an isolated
iframe) that pulls **real, read-only banking data** via sandbox-function
callbacks — so every figure it shows is real app data (fetched on
demand), never fabricated, and no secret ever crosses the boundary.

- **New `src/opengen/` module** — a stable, module-scope
`sandboxFunctions` array
(`getTransactions`/`getPolicies`/`getCards`/`getKpis`) whose handlers
read a module snapshot kept fresh by a headless `<SandboxDataSync/>`
mirroring the app's live (role-filtered) `useCreditCards` view. Handlers
return **projection DTOs** — `getCards` drops `pin`/`expiry`, guarded by
a no-leak unit test.
- **Shared over-limit derivation** — extracted to
`src/lib/over-limit.ts` so the chat readable, the A2UI report renderers,
and the OGUI sandbox all agree on which charges are over limit
(behavior-preserving).
- **OGUI enabled on both runtime paths** (Intelligence + OSS) with an
**artifact-type routing fence** in the agent prompt:
`generateSandboxedUi` only for interactive/custom UI, explicitly
excluded from reports/charts *even when the user says "build"* (protects
the existing "Build a spend report on the canvas" pill), and never part
of the teach/recall arc.
- **Provider wiring** — `openGenerativeUI={{ sandboxFunctions,
designSkill: NORTHWIND_DESIGN_SKILL }}` plus two OGUI-only pills ("Build
an interactive spend explorer", "Prototype a cash-flow what-if
calculator").
- **Deterministic routing guard** — `e2e/ogui-routing.spec.ts`
(dedicated OSS-mode config, isolated ports, no docker) pins both
boundary sides: the 5 visualization pills still route to their curated
tool (no iframe), and the 2 OGUI pills render an iframe. Also fixes
stale pill assertions in `smoke.spec.ts`.

Additive only — no changes to existing curated charts, the A2UI report
canvas, or the teach/recall arc beyond the behavior-preserving
over-limit extraction.

## Test Plan

- [x] Unit suite green (65 tests) — incl. `over-limit`,
`sandbox-functions` (no-`pin`/`expiry` leak + over-limit flag + KPI
counts)
- [x] `tsc --noEmit` clean · eslint clean · `nx build` success
- [x] OGUI routing e2e: **7 passed** (OSS mode, no docker) — curated
pills + boundary + OGUI pills
- [ ] **CI**: license-gated `smoke.spec.ts` + docker-backed
`memory-learning.spec.ts` (blocked locally: no license token /
Intelligence stack; must pass unchanged in CI)
- [ ] **Manual**: render "Build an interactive spend explorer", confirm
iframe figures match the dashboard in light/dark, and no PIN is ever
shown

## Notes

- Follow-up (out of scope): additional over-limit derivation copies
remain in `proactive-notice.tsx`, `transactions-list.tsx`,
`pending-approvals-chat.tsx` — candidates to migrate onto the new shared
helper.
- The deterministic e2e pins the tool call (aimock), so it proves the
tool-rendering plumbing and that curated surfaces still work — real LLM
routing is the manual check above.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-09 21:17:25 +02:00
David McKay a9c0910dea Merge branch 'main' into feat/banking-open-generative-ui 2026-07-09 12:07:23 -07:00
Maxim c1819f4932 test(banking): correct OGUI double-pill rationale comment
The .first() guard on the 'rendered on the canvas' handoff-pill assertion
was justified by an inaccurate comment (accumulation across exchanges). The
real cause is intra-turn: generateSandboxedUi has followUp:true, so aimock
re-serves the same fixture on the unchanged-userMessage follow-up turn in
replay -> a second identical pill. A terminating sequenceIndex follow-up
fixture was attempted but destabilized the suite (title-generation requests
substring-match the pill text and consume the sequence counter before the
real leg-1 turn), so the .first() guard remains. Test/fixtures only; no
source changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 20:34:29 +02:00
Tyler Slaton 704e31ee6a chore: release channels-slack v0.1.1 (#5903)
## Release channels-slack v0.1.1

**Scope:** `channels-slack` | **Bump:** `patch`

---

### How this release process works

1. **This PR was created automatically** by the "release / create-pr"
workflow.
   It bumped the `channels-slack` packages to `0.1.1`
   and generated AI-enhanced release notes.

2. **CI runs on this PR** — the full test suite (unit tests, lint, type
checks, build)
   must pass before merging. This is the review gate.

3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there
before merging.

4. **When this PR is merged**, the `release / publish` workflow
automatically:
   - Builds all packages
   - Publishes the `channels-slack` packages to npm at version `0.1.1`
   - Creates git tag `channels-slack/v0.1.1`
   - Creates a GitHub Release with the final release notes

### Before merging

- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)

---

> **Do not merge until CI is fully green.** The full test suite runs
automatically on this PR.
channels-slack/v0.1.1
2026-07-09 10:41:07 -07:00
tylerslaton 95ffa053d0 chore: release channels-slack v0.1.1 2026-07-09 17:38:30 +00:00
github-actions[bot] 87b6fa330d style: auto-fix formatting 2026-07-09 17:34:36 +00:00
Maxim cd7e49e467 test(banking): OGUI routing asserts the canvas surface 2026-07-09 19:27:57 +02:00
Maxim c541f1b32d feat(banking): render OGUI surfaces on the canvas with a chat handoff pill 2026-07-09 19:14:50 +02:00
Maxim 001b915af2 feat(banking): useOguiSurface reads the latest OGUI activity from the stream 2026-07-09 18:49:37 +02:00
Maxim 53cbeee311 feat(react-core): export OpenGenerativeUIRenderer from the public v2 barrel 2026-07-09 18:48:40 +02:00
Tyler Slaton 5cae8d4937 fix(docs): clean Claude generative UI snippets (#5890)
## Problem

Claude Agent SDK docs could render malformed or missing extracted
snippets across the generated Python and TypeScript integration docs.
The generative UI pages had stale duplicate regions and generic setup
leakage, while the custom look-and-feel reasoning and slots pages
referenced demo cells or regions that did not exist for every Claude
integration.

## Why

The docs pipeline treated accidental duplicate region names across files
as intentional multi-file regions, and several authored docs pages
drifted from the actual generated showcase demo IDs/regions. That left
some pages visually correct at a glance but broken when users opened
specific extracted code snippets.

## Fix

- Move the shared `bar-chart-renderer` regions to the complete
`useComponent` call and delete stale duplicate snippet files.
- Add an explicit duplicate-region guard and verifier coverage for
accidental cross-file region collisions.
- Add line-emphasis support for extracted `<Snippet>` blocks and setup
`<DemoCode>` output.
- Scope generative UI feature pages away from generic `agent-setup`
boilerplate.
- Repair the shared reasoning-messages docs to use the generated
`reasoning-default` and `reasoning-custom` demo cells.
- Add the missing Claude Python chat-slots teaching snippet regions and
keep both Claude slot snippets self-contained.
- Broad-audit both Claude integration docs locally, then targeted-audit
the repaired reasoning/slots pages in light and dark mode.
2026-07-09 09:42:56 -07:00
Alem Tuzlak d4a9bc3355 fix(react): resolve package types under bundler/node16/nodenext (#5264)
## Problem

The published declaration files for `@copilotkit/react-core`,
`@copilotkit/react-ui`, and `@copilotkit/react-textarea` contain imports
that TypeScript cannot resolve, so **`attw` (Are The Types Wrong)
reports `InternalResolutionError` across every resolution mode**
(`node10` / `node16` / `bundler`). In `@copilotkit/react-core` this was
being **masked in CI** by `--ignore-rules internal-resolution-error` on
the package's `attw` script — so the existing `check:packages` gate
looked green while consumers under `moduleResolution:
bundler`/`node16`/`nodenext` got broken types (the symptom reported in
#3324: `has no exported member 'useAgent'`, etc.).

Two distinct artifacts leaked into the emitted `.d.ts` / `.d.cts` /
`.d.mts` (neither affects the JS bundles):

1. **Side-effect CSS imports** — `import "./index.css"` is intentionally
kept in the JS so styles auto-load for bundler consumers, but
`rolldown-plugin-dts` also left it in the declarations, where TypeScript
can't resolve a `.css` as a typed module.
2. **Extensionless relative `./context` import** —
`@copilotkit/react-core/v2/headless` re-exports the externalized context
module; the JS bundle correctly externalizes it to
`@copilotkit/react-core/v2/context`, but the declaration kept the
relative `./context`, which is invalid in ESM declarations.

> Note: this is **not** the missing-`exports.types`-condition theory
from #3324. tsdown deliberately relies on co-located `.d.mts`/`.d.cts`
siblings; `@copilotkit/core` already resolves cleanly. The real defects
are the two leaked imports above.

## Fix

A small tsdown `build:done` hook post-processes the emitted declarations
**on disk** (after every format is written, so it catches both `.d.mts`
and `.d.cts`):

- strips side-effect CSS imports from declarations (JS keeps them);
- rewrites the relative `./context` import to the
`@copilotkit/react-core/v2/context` package path (matching how the JS
bundle externalizes it).

Also:
- **Removed the `--ignore-rules internal-resolution-error` band-aid**
from `react-core`'s `attw` script so the existing CI gate validates for
real.
- **Dropped the dead `codeSplitting` option** from the UMD configs —
tsdown never reads it (it's a rolldown-only key), and it was failing
`tsc` in the configs that type-check themselves. UMD output is unchanged
(single file).

## Verification

- All three packages build; **no CSS or relative-`./context` imports
remain in any declaration**, while the JS bundles still contain them
(styles auto-load preserved).
- `attw` + `publint` pass for all packages **with no suppression**
(`react-core`'s `/v2`, `/v2/headless`, `/v2/context` are green for
node16-cjs/esm/bundler).
- Unit tests pass.
- A standalone consumer project (real tarball install, `skipLibCheck:
false`) type-checks the public APIs — including `useAgent` /
`useFrontendTool` / `useConfigureSuggestions` — cleanly under **both
`bundler` and `nodenext`**, and the headless↔context class is nominally
identical.

## Out of scope (follow-ups)

- `@copilotkit/react-native`: its `--ignore-rules
internal-resolution-error` currently suppresses nothing (no IRE) and it
has a separate `NoResolution` flag.
- `@copilotkit/vue`: a large, genuine set of `.vue`/relative-import
declaration errors unrelated to this change.

Relates to #3324.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-09 18:42:07 +02:00
Sam Julien 3ad620137c docs: document CLI import command (#5824)
## Summary
- Add the new `copilotkit import` command to the shared CLI docs
rendered at `/cli` and integration CLI pages.
- Document ADK and LangGraph examples, scripted flags, source credential
env vars, `--dry-run`, and `--replace`.
- Update the shell-docs nav test expectation for de-duped `Threads`
placement in authored framework nav.

## Test Plan
- `cd showcase/shell-docs && npm run lint` (passes with existing
warnings)
- `cd showcase/shell-docs && npm run typecheck`
- `cd showcase/shell-docs && npm test`
- `cd showcase/shell-docs && npm run build`
2026-07-09 09:15:58 -07:00
Alem Tuzlak 52b957fda6 chore: release channels-teams v0.1.1 (#5898)
## Release channels-teams v0.1.1

**Scope:** `channels-teams` | **Bump:** `patch`

---

### How this release process works

1. **This PR was created automatically** by the "release / create-pr"
workflow.
   It bumped the `channels-teams` packages to `0.1.1`
   and generated AI-enhanced release notes.

2. **CI runs on this PR** — the full test suite (unit tests, lint, type
checks, build)
   must pass before merging. This is the review gate.

3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there
before merging.

4. **When this PR is merged**, the `release / publish` workflow
automatically:
   - Builds all packages
   - Publishes the `channels-teams` packages to npm at version `0.1.1`
   - Creates git tag `channels-teams/v0.1.1`
   - Creates a GitHub Release with the final release notes

### Before merging

- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)

---

> **Do not merge until CI is fully green.** The full test suite runs
automatically on this PR.
channels-teams/v0.1.1
2026-07-09 18:15:25 +02:00
tylerslaton 0653031fa2 chore: release channels-teams v0.1.1 2026-07-09 16:06:25 +00:00
Maximiliano Korp d1fbd583dd docs(shell-docs): document cli import command 2026-07-09 09:02:24 -07:00
Tyler Slaton 606afd8a13 feat(bot-teams): add ./render export for managed reuse (#5878)
## Summary

Adds a `./render` subpath export to `@copilotkit/bot-teams` surfacing
`renderAdaptiveCard`, `isPlainText`, `collectPlainText`,
`ADAPTIVE_CARD_CONTENT_TYPE`, and `createRunRenderer`, so the managed
(Intelligence-hosted) Teams egress path can reuse the adapter's Adaptive
Card rendering without deep-importing `dist/*`.

Paired with the managed Microsoft Teams work in Intelligence (OSS-441
slice ②): CopilotKit/Intelligence#511.

## Notes

- Additive only — a new `exports["./render"]` entry +
`src/render/index.ts` re-export barrel. No behavior change to existing
exports.
- Not strictly on the critical path yet: the managed runtime currently
resolves the renderer via a runtime deep-dist import of the published
package, so this export is the clean forward path rather than a hard
dependency.
- Coordinate the `bot-teams` → `channels-teams` rename (OSS-438) before
merge.
2026-07-09 08:58:43 -07:00
Alem Tuzlak f993c54bfb fix(bot-intelligence): align managed runtime delivery ownership (#5800)
## Problem

The Intelligence realtime loop exposed two SDK-side ownership gaps while
testing managed Coworkers against the Intelligence PR:

- Phoenix channel auth should use the SDK socket auth token path
expected by the gateway.
- Delivery handling needs to carry the app-api lease token and
authoritative delivery scope through render/fail/complete handling
instead of rebuilding ownership from local defaults.
- Runtime integration needs to pass the render sink into
`intelligenceAdapter` so managed runtimes stream rich render frames over
the realtime path.

## Why

The target Coworker path is websocket-first: Intelligence app-api owns
durable delivery state, realtime-gateway owns live transport, the SDK
receives leased delivery over Phoenix, streams render events, waits for
durable receipt coverage, and sends completion intent without taking
over app-api ack authority.

This PR is stacked on Alem's SDK realtime branch so the dependency trail
is explicit:

- Base SDK branch: `codex/oss-402-sdk-render-events`
- Intelligence PR: https://github.com/CopilotKit/Intelligence/pull/466
- Linear: OSS-402 / OSS-406

## Fix

- Use Phoenix socket `authToken` for the managed bot channel.
- Preserve `leaseToken` and delivery `scope` in
`PhoenixRealtimeTransport` state.
- Send fail/nack payloads with the correct lease token, delivery status,
and optional accepted-through pointer.
- Pass `renderSink` from `startManagedBots` into `intelligenceAdapter`.
- Add regression coverage for render-sink propagation and lease-token
fail payloads.

## Testing Methodology

Local verification used an external pnpm store to avoid repo-local
`.pnpm-store` churn:
`PNPM_CONFIG_STORE_DIR=/private/tmp/pnpm-store-codex`.

- `PNPM_CONFIG_STORE_DIR=/private/tmp/pnpm-store-codex
PNPM_CONFIG_CONFIRM_MODULES_PURGE=false corepack pnpm --dir
/private/tmp/copilotkit-oss-402-20260701 --filter
@copilotkit/bot-intelligence test`
  - Passed: 5 files, 48 tests.
- Commit hook also ran the package gate for
`@copilotkit/bot-intelligence`:
  - `test`: passed, 5 files / 48 tests.
  - `publint`: passed with repository URL suggestion only.
- `attw --pack . --profile esm-only`: passed with the existing ignored
CJS-to-ESM warning profile.
- `git diff --check -- packages/bot-intelligence/src/phoenix-channel.ts
packages/bot-intelligence/src/phoenix-transport.ts
packages/bot-intelligence/src/render-events.test.ts
packages/bot-intelligence/src/runtime.test.ts
packages/bot-intelligence/src/runtime.ts`
  - Passed.

Scope control: only the five `packages/bot-intelligence/src/*` files
above are committed. Existing local `pnpm-lock.yaml` and image/LFS dirt
in the SDK worktree were left unstaged and are not in this PR.
2026-07-09 17:56:38 +02:00
Maxim c54148412a test(banking): deterministic OGUI routing guard over the adjacency set
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 16:43:47 +02:00
Benjamin Taylor 58e561e2fa Merge origin/main into alem/oss-441-managed-teams (Bots→Channels rename)
Reconcile #5878 with the @copilotkit/bot*→@copilotkit/channels* rename (#5849):
- relocate the new render barrel (render/index.ts, render/index.test.ts) into
  packages/channels-teams; relative imports (./adaptive-card, ../event-renderer)
  resolve unchanged. package.json ./render export auto-merged.
- fix a stale @copilotkit/bot-teams comment ref to @copilotkit/channels-teams.
2026-07-09 08:44:19 -05:00
Benjamin Taylor b5dad876ab fix(channels-intelligence): make Phoenix delivery drops observable + cover new behaviors
Review follow-ups for the managed delivery-ownership change:
- add an optional `log` seam to PhoenixTransportConfig; the transport was
  otherwise silent, so the two new drop paths were invisible failure modes.
- log the leaseToken-missing drop distinctly (the gateway/SDK version-skew
  hazard: without it, every delivery silently re-loops on lease lapse) and the
  nack no-delivery-state drop, instead of bare returns.
- refresh TSDoc: toIngressEnvelope's new return shape + drop semantics, and the
  DeliveryState.leaseToken/scope fields.
- tests: leaseToken-required drop, nack no-state no-op, and per-delivery scope
  stamping on render + fail (the scope field previously had no coverage).
2026-07-09 08:10:44 -05:00
Benjamin Taylor 024446e5db Merge origin/main into codex/oss-402-sdk-handoff-fixes (freshen) 2026-07-09 08:07:37 -05:00
Tyler Slaton 66b5a58339 fix(docs): repair Claude custom look snippets 2026-07-08 21:39:49 -07:00
github-actions[bot] ae3ecc2cb7 style: auto-fix formatting 2026-07-09 03:39:16 +00:00
Tyler Slaton 0f5a916075 fix(docs): clean Claude generative UI snippets 2026-07-08 20:38:13 -07:00
Jordan Ritter 66502b5b3d fix(showcase-harness): D4 probe hardening follow-ups (budget-exhaustion, degraded-path, telemetry, coverage) (#5888)
Tracked follow-up to #5882 (D4 first-token SSE turn-complete fix). These
are the deferred **bucket-(b)** items from that PR's CR —
non-load-bearing polish/hardening on the D4 probe driver
(`showcase/harness/src/probes/drivers/d4-chat-roundtrip.ts` +
`.test.ts`). No re-architecture; each change is tight and scoped.

## Items

### 1. Budget-exhaustion retry guard (behavioral)
A late non-completion retry resend could floor its `type`/`press` action
timeout to ~1ms (when the remaining budget ≈ 0), throwing a
page-fault-shaped error. That throw was red-classified indistinguishably
from a real page fault — a spurious-red flap source. Fix: skip the
resend when the remaining budget is below `RETRY_MIN_BUDGET_MS` (750ms);
the stall then reds on its own terms.

**Red-green:** with the guard disabled, the doomed resend attempts a
second `type` (typeAttempts=2); with the guard, `type` fires exactly
once (typeAttempts=1). RED observed (2), GREEN observed (1).

### 2. Degraded-path floor when interceptor silently no-ops (behavioral)
When `sseAttachFailed` is true, the page-side turn-lifecycle globals
were never seeded, so the poll could only fall into the never-observed
branch and pin the deadline to the base `textPollTimeoutMs` floor —
reintroducing the slow-first-token false-red #5882 targets. Fix: consult
`sseAttachFailed` to widen the never-observed wait to the per-attempt
ceiling.

**Red-green:** degraded page + late-but-present token (800ms, base floor
bites at the ~500ms poll before the token) → RED (pre-fix, base-floor
fast-fail) vs GREEN (post-fix, widened to ceiling captures the token).
RED observed (`red`), GREEN observed (`green`). A genuinely-empty
degraded run still reds (over-correction guard).

### 3. Retry edge-header re-attribution (telemetry)
On a retry-rescued GREEN turn, `messageSendEdge` / `messageSendEdge` /
`lastMessagePostResp` stayed latched to the first (stalled) attempt,
mis-attributing `edge_interference_signal` / the DEBUG raw-byte sample.
Fix: re-arm the capture latches before the resend so the winning
attempt's response re-captures them.

**Focused test:** stalled attempt carries `cf-mitigated: stalled`,
winning resend `cf-mitigated: winning`; the final `probe.message.send`
boundary now carries `winning` (pre-fix it reported `stalled`).

### 4. Coverage (tests only, no prod change)
- FIFO-cap `CVDIAG_MAX_OUTSTANDING_STARTS_PER_URL` eviction backstop
(guard-verified: fails if the cap is removed).
- DEBUG-auto-disarm fail-closed negative test (disarmed → no raw-byte
capture).
- Alternate-content / raw-byte block SKIPPED on the container-success
(non-empty) path.
- Fixed the `makeLateTokenBrowser` shared-page state leak: each
`newContext().newPage()` now mints its own state, so L3 and L4 each run
an independent stall+retry cycle (was a singleton that leaked
`sendCount` from L3 into L4, so the L4 retry path was never genuinely
exercised).

### 5. `lastStoppedAtMs` documentation (cleanup)
Write-only in d4; documented that it is retained for shared-global
parity with the `attachSseInterceptor` global shape the d6 run-signal
snapshot mirrors, so it does not read as dead code.

## Verification
- `tsc --noEmit`: clean
- `tsc -p tsconfig.build.json` (build): clean
- Full harness vitest: **3213 passed, 0 failing** (72 in the d4 file,
all green)
- oxlint: **0 errors** (only pre-existing warnings, none new)
- Diff hygiene: only the driver + test file (no `repro_*` / `baseline`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)


---

## CR follow-up round (commit 13a636b67)

CR found the follow-up left `readTurnState()` error handling
**inconsistent** — the exact false-red/flap class this effort fights.
Addressed, plus completed two of the PR's own items.

### (a) Harmonized `readTurnState()` error handling
One guarded `safeReadTurnState()` wrapper now backs all three consumers
(`readBaseline`, `readTurnComplete`, `readDegraded`). A mid-poll
`readTurnState()` throw now means ONE thing everywhere: "no reliable
signal" → return a well-defined degraded sentinel (`sseAttachFailed:
true`, zeroed counters) → route onto the degraded **widen** path (not
base-floor fast-fail, not a spurious `level-error`), and it is
**observable** via a one-shot greppable marker. Previously:
`readDegraded` swallowed the throw into `false` (silent base-floor
false-red, no telemetry) while `readBaseline`/`readTurnComplete` let it
escape → generic `level-error` (spurious red). A genuinely-empty
degraded turn still reds at the widened ceiling (no masking).

### Folds
- **item-1 first-send cap:** guard the in-send `press` against
`SEND_PRESS_MIN_BUDGET_MS` so a near-hang `type` can't floor `press` to
~1ms and yield a generic `level-error`; classify distinctly as
`send-budget-exhausted`. Only `press` is guarded (`type` opens the
envelope) so a legitimately-small `pageTimeoutMs` still issues a healthy
first send.
- **item-3 null-header:** suppress the `finally`-block fallback
`probe.message.send` once a real-header boundary already fired, so a
retry whose winning resend lands no POST no longer emits a second
NULL-header boundary (mis-attributed `edge_interference_signal`).
- errorDesc JSDoc: added `abort` to the enumeration (zero-risk).

### Red-green (verbatim, against the real runLevel/readTurnComplete
path)
- **readTurnState throw:** RED `expected 'red' to be 'green'` (pre-fix
spurious red on a late-but-present token) → GREEN (degraded widen; late
token passes; genuinely-empty degraded still reds).
- **first-send cap:** RED `expected 'level-error' to be
'send-budget-exhausted'` → GREEN (distinct classification, no
1ms-floored `press`).
- **item-3 null-header:** RED `expected 2 to be 1` (second null-header
emit) → GREEN (single correct attribution).

### Gates
`tsc --noEmit` clean · `npm run build` clean · full d4 vitest **76/76
pass** · oxlint **0 errors** (warnings pre-existing). Diff: driver +
test only (no repro_*/baseline).
2026-07-08 18:35:36 -07:00
Jordan Ritter 9d9056db60 fix(showcase-harness): propagate level errorDesc to aggregate signal + reorder abort short-circuit
The aggregate e2e-smoke:<slug> red signal omitted errorDesc on the normal
return path, so an abort/timeout/send-budget-exhausted red that runLevel
RETURNS (not throws) showed on the PRIMARY dashboard tick as an unclassified
content-shaped red — only the side chat:/tools: rows kept the classifier.
Thread the failing level's errorDesc (L3 precedence, L4 fallback) onto the
aggregate so the primary tick matches the side row and the launcher-phase
abort path. Does not change red/green — only carries the classifier.

Also reorder the aborted-and-empty short-circuit ABOVE the alternate-content
/ raw-byte evaluate reads: an aborted run's page is tearing down, so those
reads were swallowed against a dead page and emitted an ambiguous empty
histogram. Non-aborted runs still perform the alternate-content salvage.
2026-07-08 18:27:14 -07:00
Maxim 2dfedb0769 test(banking): fix stale smoke-test pill assertions 2026-07-09 02:54:43 +02:00
Maxim b041d998bf feat(banking): register OGUI on the provider, mount data-sync, add OGUI pills 2026-07-09 02:42:10 +02:00
Jordan Ritter 80bd9469c4 fix(showcase-harness): classify mid-poll abort/timeout empty as abort, not content-red
A mid-poll abort — the external ctx.abortSignal firing, or the driver's
own hard-timeout landing during the first-token poll — makes runAttempt
return empty WITHOUT throwing. The retry loop breaks and control falls to
the clean-exit path, where the level was misclassified as a generic
content red ("empty assistant response", probe.exit outcome "err", no
errorDesc). That masqueraded a teardown/abort/timeout as a CONTENT
failure on the dashboard + CVDIAG.

Add an aborted-AND-empty guard before the content-red gate that
short-circuits to the same abort classification the other paths use
(errorDesc "abort", probe.exit outcome "timeout"). Discriminator is
abortSignal.aborted, not emptiness alone: a genuinely-completed-empty
turn (not aborted) stays the content-red "empty assistant response".
2026-07-08 17:39:15 -07:00
Maxim c2f0b0f14b feat(banking): enable OGUI on both runtimes and fence it in the prompt 2026-07-09 02:11:12 +02:00
Jordan Ritter 13a636b670 fix(showcase-harness): harmonize d4 readTurnState error handling + first-send cap + null-header re-attribution
Harmonize the three readTurnState() consumers in the d4 chat-roundtrip probe
through one guarded safeReadTurnState() wrapper so a mid-poll readTurnState()
throw is handled consistently everywhere: it means "no reliable signal" ->
degraded widen + observable telemetry, never a silent false-red (the prior
readDegraded swallow) nor a spurious level-error (the prior unguarded
readBaseline/readTurnComplete escape). A genuinely-empty degraded turn still
reds at the ceiling.

Folds completing the PR's own items:
- item-1 first-send cap: guard the in-send press against SEND_PRESS_MIN_BUDGET_MS
  so a near-hang type can't floor press to ~1ms and produce a generic
  level-error; classify distinctly as send-budget-exhausted. Only press is
  guarded (type opens the envelope), so a legitimately-small pageTimeoutMs still
  issues a healthy first send.
- item-3 null-header: suppress the finally-block fallback probe.message.send once
  a real-header boundary already fired, so a retry whose winning resend lands no
  POST no longer emits a second null-header boundary (mis-attributed
  edge_interference_signal).

Also add "abort" to the errorDesc JSDoc enumeration (zero-risk).

Red-green covered for all three behavioral items against the real
runLevel/readTurnComplete path with a faithful fake.
2026-07-08 17:09:51 -07:00
Jordan Ritter 6889fdb26a test(showcase-harness): cover D4 probe hardening follow-ups (red-green + coverage)
Tests for the #5882 bucket-(b) follow-up hardening:

- Budget-exhaustion retry guard (red-green): a near-exhausted-budget retry no
  longer attempts a doomed ~1ms-floored resend (type invoked exactly once).
- Degraded-path floor (red-green): a degraded page (sseAttachFailed, no
  completion signal) with a late-but-present token resolves GREEN instead of a
  base-floor false-red; a genuinely-empty degraded run still reds.
- Retry telemetry re-attribution (focused test): a retry-rescued GREEN turn
  records the winning attempt's edge headers on probe.message.send.
- Coverage: FIFO-cap CVDIAG_MAX_OUTSTANDING_STARTS_PER_URL eviction backstop;
  DEBUG-auto-disarm fail-closed (disarmed => no raw-byte capture);
  alternate-content / raw-byte block SKIPPED on the container-success path.
- Fake fix: makeLateTokenBrowser now mints per-page state so L3 and L4 each run
  an independent stall+retry cycle (was a shared-page singleton that leaked
  sendCount from L3 into L4, so the L4 retry path was never genuinely exercised).
2026-07-08 16:48:33 -07:00
Jordan Ritter 8c3e3aa381 fix(showcase-harness): harden D4 probe driver (budget-exhaustion, degraded-path, retry telemetry)
Follow-up to #5882 (bucket-(b) CR items). Three behavioral/telemetry fixes
plus one documentation clarification, all in d4-chat-roundtrip.ts:

- Budget-exhaustion retry guard: skip a non-completion retry resend when the
  remaining wall-clock budget is below RETRY_MIN_BUDGET_MS (750ms). A late
  resend previously floored its type/press action timeout to ~1ms, throwing a
  page-fault-shaped error that mis-classified the stall as a generic red — a
  spurious-red flap source. The stall now reds on its own terms.

- Degraded-path floor: when the SSE interceptor silently no-ops
  (sseAttachFailed), no completion signal ever arrives, so the poll could only
  fall into the never-observed branch and pin the deadline to the base floor —
  reintroducing the slow-first-token false-red #5882 targets. Consult
  sseAttachFailed to WIDEN the never-observed wait to the per-attempt ceiling so
  a late-but-present token on a degraded page is still captured.

- Retry edge-header re-attribution: on a retry-rescued turn, re-arm the
  message-POST edge-header capture (messageSendEdge / lastMessagePostResp /
  emitMessageSend latch) so probe.message.send / edge_interference_signal / the
  DEBUG raw-byte sample reflect the WINNING attempt, not the stalled first one.

- Document why lastStoppedAtMs is retained on the d4 TurnState (write-only in
  d4; part of the shared attachSseInterceptor global shape the d6 run-signal
  snapshot also mirrors) so it does not read as dead code.
2026-07-08 16:48:22 -07:00
Maxim 795b185d4f feat(banking): SandboxDataSync mirrors live view into OGUI snapshot 2026-07-09 01:39:38 +02:00
Jordan Ritter 725f98ff79 fix(showcase-harness): wait on SSE turn-complete signal for D4 first token (#5882)
## Root cause

D4's L4 flap was a **stalled turn, not a client render race**: aimock
served content but the page never rendered it (the real 20:16:52Z
failure). The turn stream never delivered a first token to the DOM
within the fixed `textPollTimeoutMs`, so the poll read the container
empty → spurious "empty assistant response" red.

The **prior fix (`ceae0c2c9`) was inert in production**. It keyed the
turn-complete decision off the `onSseEvent` Node-side seam, which the
real launchers never wire (Playwright has no per-SSE-event signal).
`sseObserved` therefore stayed `false` on the real path and the whole
extension collapsed to the pre-fix base budget floor. Its red-green used
a fake page that invoked the seam synthetically, so the deadness was
never caught.

## The fix (3 parts, mirrors `d6-all-pills.ts`)

1. **Wire the real completion signal.** `wirePlaywrightPage.goto` now
calls `attachSseInterceptor(page)` before navigation (injectable for
tests), seeding the page-side `__hk_runsFinished` /
`__hk_copilotRunning` turn-lifecycle globals at `document_start`. A new
`readTurnState()` `E2ePage` seam reads them via `page.evaluate`; the
first-token poll keys off **that** — the same transport-level
`RUN_FINISHED` + DOM run-stop edge d6 trusts — not the dead `onSseEvent`
seam.

2. **Fast-fail genuinely-empty turns.** With a real turn-complete edge,
a turn that completes with empty assistant text reds in ~`completion +
FIRST_TOKEN_GRACE_MS` (bounded by `FIRST_TOKEN_FAST_FAIL_MS` ≈ 15s)
instead of burning the flat 60s. A completed-empty turn still reds (no
masking) and is never retried.

3. **Retry-on-non-completion.** A turn **observed in-flight** that never
signals completion within budget (stalled/dropped stream) retries once
before red. **Never-observed** (dead/no-turn) runs stop at the base
floor with no retry. Total wall-clock is bounded by `pageTimeoutMs`
(per-attempt budget split), so the retry can't blow past the ceiling.

## CR findings resolved

- `abortSignal.aborted` checked **inside** the poll loop (was only at
level entry).
- Body-scrape fallback keeps `fromAssistantContainer=false` **and** no
longer clears `cvdiagResponseEmpty` — so a fallback-salvaged red can no
longer emit `terminal_outcome=ok`.
- Red/green never gated on `cvdiag` (telemetry-only); the `onSseEvent`
seam is documented telemetry-only.
- No-turn fallback budget capped by the per-attempt ceiling (bounded by
`pageTimeoutMs`).
- Fallback `tail.length > 20` floor and `split("\n")[0]` truncation
**removed** (they false-red'd short/multiline valid answers).
- Corrected stale "not started" / dead-seam comments.

## Real-surface red-green (NO fake page)

Real chromium + real `attachSseInterceptor` + real D4 driver against a
local fixture serving an SSE `/api/copilotkit` with an injected
first-token/stream delay (aimock-style). `sseObserved-on-real-path =
(runsFinished>0 || attrPresent)` is read from the actual page-side
globals.

**RED — pre-fix (interceptor UNWIRED), late-token (token@3s):**
```
MODE: unwired  scenario: late-token
L3(chat).state: red  summary: "empty assistant response"
elapsed_ms: 1444
sseObserved-on-real-path: false      <-- dead seam proven
```

**GREEN — post-fix (interceptor WIRED), same late-token:**
```
MODE: wired  scenario: late-token
L3(chat).state: green  summary: ""
elapsed_ms: 6333
LAST readTurnState on REAL path: {"runsFinished":0,"attrPresent":true,"sawRunningTrue":true,"runningNow":true}
sseObserved-on-real-path: true       <-- interceptor fires on real path
```

**completed-empty (WIRED) — still RED, fast-fail:**
```
L3(chat).state: red  summary: "empty assistant response"
elapsed_ms: 5340 (~2.5s/level, not the 60s ceiling)
LAST readTurnState on REAL path: {"runsFinished":1,"attrPresent":true,"sawRunningTrue":true,"runningNow":false}
sseObserved-on-real-path: true
```

**recoverable-stall (WIRED) — attempt 1 never completes → retry →
GREEN:**
```
L3(chat).state: green  summary: ""
elapsed_ms: 13423   (attempt 1 polls to per-attempt ceiling observed=true/complete=false, then resends; attempt 2 renders)
sseObserved-on-real-path: true
```

## Checks

- `tsc --noEmit`: clean
- `tsc -p tsconfig.build.json` (build): clean
- vitest `d4-chat-roundtrip.test.ts`: 56/56 pass (suite rewritten to
exercise the real `readTurnState` path + retry/fast-fail;
late-token→green, completed-empty→red, recoverable-stall→green via
retry, permanent-stall→red)
- oxlint: 0 errors (pre-existing warnings only)

Kept as a **draft**; do not merge.

---

## Follow-up (b572697ab): attempt-scoped completion — the dominant a1
false-red

CR flagged that the poll's `sseDone = runsFinished >= 1` read the
page-GLOBAL monotonic counter. A **prior run on the page**
(auto-greeting / initial-mount run) leaves `runsFinished >= 1` and
`sawRunningTrue` already latched **before the user's turn starts**, so
the poll thought THIS turn had already completed, saw the still-empty
container, and spuriously **fast-failed RED**. It only passed the old
tests because the fake reset per send.

### Fixes in this commit

1. **Completion is now ATTEMPT-SCOPED.** A per-attempt BASELINE
(`runsFinished` + `runStartCount`) is captured at send time; the turn is
complete only on a **new edge past the baseline** (`runsFinished >
baseline`, or a new `runStartCount` DOM run-start), never the
page-global `>= 1`. A fresh baseline is taken before each retry resend,
so a stale prior edge can't defeat the retry.
`TurnState`/`readTurnState` now surface `runStartCount` +
`lastStoppedAtMs` (the sse-interceptor already latches them), and the
grace window is stamped from the **real finished edge** rather than the
poll's local clock (fixes the ~500ms-short grace).
2. **Retry resend bounded by remaining budget.** The retry `sendTurn()`
type/press timeouts are capped to `hardCeiling` (were flat
`pageTimeoutMs`, which let a stalled resend push poll-phase wall-clock
~2x past the ceiling).
3. **Attach-fault is observable.** A failed `attachSseInterceptor` now
emits an `onAttachFault` marker + sets `TurnState.sseAttachFailed`, so a
silent regression to the inert base-floor path is detectable.

Folded (cheap): `probe.message.send` fallback moved into `finally`
(fires on nav/send throw); external `ctx.abortSignal` aborts labeled
`"abort"` (not `driver-error`) in the aggregate; corrected
fast-fail-floor / hardCeiling / poll-deadline doc comments.

### Real-surface RED→GREEN (real Chromium + real `attachSseInterceptor`
+ local SSE fixture)

The fixture fires a PRIOR run on mount (bumps the page-global counters),
then the user turn renders a first token at 2800ms — PAST the 2s grace
window. Driven through the REAL `createE2eSmokeDriver` +
`wirePlaywrightPage` (L3/chat, the every-service level).

**PRE-FIX (`sseDone = runsFinished >= 1`):**
```
a1-priorrun      L3.state: red   "empty assistant response"  elapsed 2141ms   <-- FALSE-RED (the bug)
recoverable      L3.state: red   "empty assistant response"  elapsed 2171ms   <-- retry defeated by stale prior edge
wired-late       L3.state: green                             elapsed 3136ms   (no prior run → no false completion)
completed-empty  L3.state: red   "empty assistant response"  elapsed 2156ms
```

**POST-FIX (attempt-scoped baseline):**
```
a1-priorrun      L3.state: green                             elapsed 3285ms   <-- fixed: prior run no longer false-reds
recoverable      L3.state: green                             elapsed 5192ms   <-- retry rescues (fresh baseline per resend)
wired-late       L3.state: green                             elapsed 3150ms
completed-empty  L3.state: red   "empty assistant response"  elapsed 2707ms   <-- fast-fail (~completion+grace, not the 9s ceiling)
sseObserved-on-real-path: true (all)   sseAttachFailed: false
```

### Checks
- `tsc --noEmit`: clean · `tsc -p tsconfig.build.json` (build): clean
- vitest `d4-chat-roundtrip.test.ts`: **63/63** (added: a1-regression
L3+L4, completed-empty-one-send, L3 grace/fast-fail/retry, attach-fault
telemetry)
- oxlint: **0 errors** (4 pre-existing warnings only)

Still a **draft**; do not merge.


---

## Follow-up (a): grace-window timestamp — Node-stamp completion instant

Four reviewers flagged the same line in `runAttempt`:

```ts
completeAtMs = stoppedAtMs > 0 ? stoppedAtMs : Date.now();
```

where `stoppedAtMs` came from the page-side
`readTurnState().lastStoppedAtMs`. Two defects in one line:

1. **Stale on SSE-only completion.** When the turn is detected complete
via `sseDone = runsFinished > baseline` but no fresh DOM stop-edge fires
for THIS turn, `lastStoppedAtMs` still holds a **stale prior-run** value
→ `graceEnd = staleStop + FIRST_TOKEN_GRACE_MS` lands in the past → the
~2s grace window collapses to the base floor → a late-but-present first
token **false-REDs**.
2. **Cross-clock skew.** `lastStoppedAtMs` is stamped on the
**browser-page** `Date.now()`, but `graceEnd`/deadline math runs on the
**Node** clock → page↔Node skew mis-sizes the window.

### Fix (single-point, simplifying)
Stamp `completeAtMs = Date.now()` (**Node**) at the first poll iteration
that observes THIS turn complete; `readTurnComplete` no longer threads
`stoppedAtMs`. Both clocks are now consistent; the stamp lags the true
finished edge by at most one 500ms poll interval, well inside
`FIRST_TOKEN_GRACE_MS`. Preserved: completed-empty still fast-reds;
base-floor and `hardCeiling` caps intact; no masking. `lastStoppedAtMs`
is retained on `TurnState` (still consumed by conversation-runner /
sse-interceptor / d6).

### Tightenings (net-negative source LOC: +54/−58 = **−4**)
- Extracted the attempt-0 baseline IIFE onto the existing `readBaseline`
helper (DRY; removes a drift hazard on the completion-scoping path).
- Fixed stale comments: fallback `emitMessageSend()` now runs in the
`finally`; eviction comment (rides the `onResponse` wiring, not "always
active"); `lastStoppedAtMs` doc.

### Red-green (mandatory)

**Unit — stale-`lastStoppedAtMs` grace collapse** (`SSE-only completion
with a STALE lastStoppedAtMs …`): prior finished run +
`sseOnlyStaleStop` + prior stop aged 5s + token 700ms after completion
(inside a healthy grace window).
```
RED   (pre-fix stamping: completeAtMs = staleStop):  expected 'red' to be 'green'   → RED
GREEN (Node-stamp fix):                              64/64 pass                      → GREEN
```

**Real-surface no-regression** (real chromium + `wirePlaywrightPage` →
real `attachSseInterceptor` + local SSE fixture,
`repro_d4_a1_realsurface.mts`):
```
a1-priorrun      aggregate green · L3 green                            elapsed 3292ms   (SSE-only-late, lastStoppedAtMs stale at completion)
completed-empty  aggregate red   · L3 "empty assistant response"      elapsed 2618ms   (fast-fail, not the 9s ceiling)
recoverable      aggregate green · L3 green                            elapsed 5161ms
wired-late       aggregate green · L3 green                            elapsed 3131ms
sseObserved-on-real-path: true (all)   sseAttachFailed: false
```

### Checks (this commit)
- `tsc --noEmit`: clean · build (`tsc -p tsconfig.build.json`): clean
- vitest `d4-chat-roundtrip.test.ts`: **64/64**
- oxlint: **0 errors** (4 pre-existing warnings only)
- commit: `7bb47e64d`

Still a **draft**; do not merge.


---

## Follow-up (commit `5047f7ba9`): make the SSE-only-stale grace guard
actually bite

Three reviewers noted that the `sseOnlyStaleStop` guard above did not,
in fact, guard the Node-stamp fix. Root cause in the fake
(`makeLateTokenBrowser.readTurnState()`): the driver takes the
per-attempt baseline via `readTurnState()` **before** the first send,
and at that point `lastSendAtMs===0` made `elapsed` (~epoch ms) exceed
`completeAt`, spuriously flipping `complete=true` at the **baseline**
read. That inflated the baseline `runsFinished` to `prior+1`, so the
current turn's real finished edge never rose **past** the baseline and
the driver's `sseDone = runsFinished > baseline.runsFinished` could
never fire. The SSE-only-stale-grace path was therefore never entered —
the test passed only because the token rendered directly (path (a)), so
it did **not** guard the Node-stamp fix.

**Test-only fix:** gate the fake's `complete` on `started` (a turn has
actually been sent). The pre-send baseline read is now a **true**
baseline (`runsFinished = prior + priorSendsDone`, no current-turn
finish); the finished edge is a genuine THIS-turn transition the driver
observes via `sseDone`; and the SSE-only completion path with a stale
`lastStoppedAtMs` is genuinely exercised.

### RED-on-revert proof (the guard bites)

Temporarily reverted the production stamp back to the pre-fix form
(`completeAtMs = stoppedAtMs > 0 ? stoppedAtMs : Date.now()`,
re-threading `stoppedAtMs` through `readTurnComplete`), ran the reworked
test, then restored the Node-stamp:

```
RED  (pre-fix revert: completeAtMs = stoppedAtMs):
  × SSE-only completion with a STALE lastStoppedAtMs … → GREEN
    AssertionError: expected 'red' to be 'green'
  Test Files  1 failed (1)

GREEN (Node-stamp restored: completeAtMs = Date.now()):
  ✓ SSE-only completion with a STALE lastStoppedAtMs … → GREEN
  Test Files  1 passed (1)
```

The production revert was **not committed** — final state has the
correct Node-stamp; the production diff in this commit is
**comment-only** (no logic change).

### Comment corrections (this commit)
- Completed-empty deadline comment: previously claimed "base floor
always respected"; corrected to state that
`Math.min(Math.max(baseBudgetEnd, graceEnd), fastFailEnd,
attemptCeiling)` intentionally clamps **below** the floor (fast-fail — a
completed-empty turn must red fast, not burn the base budget).
- `FIRST_TOKEN_FAST_FAIL_MS` doc: now spells out the full `Math.min`
term rather than only the `Math.max(base, grace)` cap.

### Checks (this commit)
- `tsc --noEmit`: clean · build (`tsc -p tsconfig.build.json`): clean
- vitest `d4-chat-roundtrip.test.ts`: **64/64**
- oxlint: **0 errors** (4 pre-existing warnings only)
- diff: driver (comments) + test file only; no `repro_*`/`baseline`
staged
- commit: `5047f7ba9`

Still a **draft**; do not merge.
2026-07-08 16:26:56 -07:00
Maxim d8b2da43ae feat(banking): read-only OGUI sandbox functions with projection DTOs 2026-07-09 01:20:14 +02:00
Jordan Ritter 5047f7ba9e test(showcase-harness): make D4 SSE-only-stale grace guard actually bite
The sseOnlyStaleStop guard in makeLateTokenBrowser was ineffective: the
driver reads the per-attempt baseline via readTurnState() BEFORE the
first send, and at that point lastSendAtMs===0 made elapsed (~epoch ms)
exceed completeAt, spuriously flipping complete=true at the baseline
read. That inflated the baseline runsFinished to prior+1, so the current
turn's real finished edge never rose PAST the baseline and the driver's
sseDone = runsFinished > baseline.runsFinished could never fire. The
SSE-only-stale-grace path was therefore never entered — the test passed
only because the token rendered directly, so it did NOT guard the
Node-stamp fix.

Gate the fake's complete on started (a turn has been sent) so the
pre-send baseline read is a TRUE baseline (runsFinished = prior +
priorSendsDone). The finished edge is now a genuine THIS-turn transition
the driver observes via sseDone, and the SSE-only completion path with a
stale lastStoppedAtMs is genuinely exercised.

Proof the guard now bites (temporary production revert, not committed):
- pre-fix stamp (completeAtMs = stoppedAtMs > 0 ? stoppedAtMs : now):
  test FAILS, expected 'red' to be 'green' (grace collapses).
- restored Node-stamp (completeAtMs = Date.now()): test PASSES.

Also corrects two production comments (3 reviewers flagged): the
completed-empty deadline comment claimed the base floor is always
respected, but Math.min(..., fastFailEnd, attemptCeiling) intentionally
clamps below the floor (fast-fail); and the FIRST_TOKEN_FAST_FAIL_MS doc
now spells out the full Math.min term. Comment-only, no logic change.
2026-07-08 16:18:44 -07:00
Maxim d1c573c6a2 refactor(banking): extract shared over-limit derivation into src/lib/over-limit 2026-07-09 01:08:38 +02:00
Jordan Ritter 7bb47e64df fix(showcase-harness): stamp D4 first-token grace from Node completion instant
On an SSE-only completion (turn detected complete via runsFinished>baseline
with no fresh DOM stop-edge for THIS turn), readTurnState().lastStoppedAtMs
still held a stale prior-run value, and it is stamped on the browser-page
clock while graceEnd/deadline math runs on the Node clock. Feeding it into
Node-clock arithmetic pushed graceEnd into the past and collapsed the
FIRST_TOKEN_GRACE_MS window to the base floor, false-REDing a late-but-present
first token.

Stamp completeAtMs from Date.now() (Node) at the first poll that observes
THIS turn complete; readTurnComplete no longer threads stoppedAtMs. Also DRY
the attempt-0 baseline onto the existing readBaseline helper and fix stale
comments (fallback emit is in finally; eviction rides the onResponse wiring;
lastStoppedAtMs doc). completed-empty still fast-reds; base floor and
hardCeiling caps preserved.

Adds a red-green unit test modelling an SSE-only completion with a stale
lastStoppedAtMs (grace collapses pre-fix, honored post-fix).
2026-07-08 16:02:19 -07:00
Jordan Ritter b572697ab2 fix(showcase-harness): scope D4 first-token completion per-attempt (was page-global false-red)
The first-token poll keyed turn-complete off the page-GLOBAL monotonic
`runsFinished >= 1` / latched `sawRunningTrue`. A PRIOR run on the page
(auto-greeting / initial-mount run) leaves those already satisfied when the
user's turn starts, so the poll treated THIS turn as already complete, saw the
still-empty container, and spuriously fast-failed RED — the a1 false-red.

Fix: capture a per-attempt BASELINE (`runsFinished` + `runStartCount`) at send
time and treat the turn complete only on a NEW edge past that baseline
(`runsFinished > baseline` / a new `runStartCount` DOM run-start). A fresh
baseline is taken before each retry resend, so a stale prior edge can no longer
defeat the retry. The grace window is now stamped from the REAL finished edge
(`lastStoppedAtMs`) rather than the poll's local clock (fixes the ~500ms-short
grace). `TurnState` / `readTurnState` are extended to surface `runStartCount`
and `lastStoppedAtMs` (the sse-interceptor already latches them).

Also:
- Bound the retry resend's type/press action timeouts by the remaining budget
  to `hardCeiling` (was flat `pageTimeoutMs`, letting a stalled resend push
  poll-phase wall-clock to ~2x past the ceiling).
- Surface an interceptor-attach fault (`wirePlaywrightPage.goto`) via an
  injectable `onAttachFault` marker + `TurnState.sseAttachFailed` so a silent
  regression to the inert base-floor path is detectable, not invisible.
- Move the `probe.message.send` fallback emit into the `finally` (idempotent)
  so it fires on nav/send throw paths too.
- Label external `ctx.abortSignal` aborts as `"abort"` (not `"driver-error"`)
  in the aggregate, matching the per-level classification.
- Correct the fast-fail-floor / hardCeiling / poll-deadline doc comments.

Tests: a1 regression (prior finished run + in-flight turn → not false-red) at
L3 and L4; completed-empty does exactly ONE send (retry does not fire); L3
coverage for the grace/fast-fail/retry path; attach-fault telemetry surfaces.
2026-07-08 15:45:52 -07:00
Martha Kelly Schumann 27ec110d6f Merge PR #5865 via QA Agent Pipeline
Auto-merged from Linear Needs Merge after approval and green CI.
2026-07-08 15:15:32 -07:00
Jordan Ritter 2f4a76943a fix(showcase-harness): wire real turn-complete signal for D4 first-token wait (was inert)
The prior D4 first-token fix (ceae0c2c9) was INERT in production: it keyed the
turn-complete decision off the `onSseEvent` Node-side seam, which the real
launchers never wire (Playwright has no per-SSE-event signal). `sseObserved`
therefore stayed false on the real path and the whole extension collapsed to the
pre-fix base budget floor. Its red-green used a fake page that invoked the seam
synthetically, so the deadness was never caught.

Root cause of the flap: a STALLED turn (RUN_FINISHED served by aimock but the
page never rendered it — the real 20:16:52Z failure), NOT a mere client render
race. So we need a real completion signal AND a retry for never-completed turns.

Three-part fix (mirrors d6-all-pills' production-wired signal):

1. Wire the real signal. `wirePlaywrightPage.goto` now calls
   `attachSseInterceptor(page)` before navigation (injectable for tests), seeding
   the page-side `__hk_runsFinished` / `__hk_copilotRunning` turn-lifecycle
   globals at document_start. A new `readTurnState()` E2ePage seam reads them via
   `page.evaluate`; the first-token poll keys off THAT — the same
   transport-level + DOM run-stop edge d6 trusts — not the dead onSseEvent seam.

2. Fast-fail genuinely-empty turns. With a real turn-complete edge, a turn that
   completes with empty assistant text reds in ~completion+grace (bounded by
   FIRST_TOKEN_FAST_FAIL_MS ~15s) instead of burning the flat 60s. A
   completed-empty turn still reds (no masking) and is never retried.

3. Retry-on-non-completion. A turn OBSERVED in-flight that never signals
   completion within budget (stalled/dropped stream) retries once before red.
   Never-observed (dead/no-turn) runs stop at the base floor, no retry. Total
   wall-clock is bounded by pageTimeoutMs (per-attempt budget split).

CR findings resolved: abortSignal.aborted checked inside the poll loop; the
body-scrape fallback keeps fromAssistantContainer=false AND no longer clears
cvdiagResponseEmpty, so a fallback-salvaged red can't emit terminal_outcome=ok;
red/green never gated on cvdiag (telemetry-only); the no-turn budget is capped by
the per-attempt ceiling; the fallback `tail.length>20` floor and
`split("\n")[0]` truncation removed (false-red on short/multiline answers);
stale "not started" / dead-seam comments corrected.

Real-surface red-green (real chromium + real attachSseInterceptor + real driver
against a local fixture serving SSE /api/copilotkit with injected stream delay):
- RED (pre-fix, interceptor unwired): late-token turn -> red "empty assistant
  response" in ~1.4s; sseObserved-on-real-path = FALSE (dead seam proven).
- GREEN (post-fix, interceptor wired): same late-token -> green;
  readTurnState on real path = {attrPresent:true,sawRunningTrue:true,...};
  sseObserved-on-real-path = TRUE.
- completed-empty (wired): still RED, fast-fail ~2.5s/level, runsFinished:1.
- recoverable-stall (wired): attempt 1 never completes -> retry -> green.

Unit suite (56 tests) rewritten to exercise the real readTurnState path plus the
retry/fast-fail behaviors; tsc + build + vitest all pass.
2026-07-08 15:15:28 -07:00
Martha Kelly Schumann 10ba63bd0e Merge PR #5835 via QA Agent Pipeline
Auto-merged from Linear Needs Merge after approval and green CI.
2026-07-08 15:15:08 -07:00
Martha Kelly Schumann 05443d1d63 Merge PR #5848 via QA Agent Pipeline
Auto-merged from Linear Needs Merge after approval and green CI.
2026-07-08 15:15:02 -07:00