mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
angular/v0.2.0
4451 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jordan Ritter
|
e5e7fc8b96 |
refactor(showcase): align complete/completed naming in D4 probe (T3)
readTurnComplete returned { observed, complete } while runAttempt returned
{ text, completed, observed } for the same concept. Rename the lower-churn
side (readTurnComplete's 'complete' → 'completed') so both use 'completed'.
Pure internal rename — the fields are local to runLevel, not exported and
not referenced by any test. Zero behavior change.
|
||
|
Jordan Ritter
|
041b4a0692 |
test(showcase): derive D4 guard probed-route set from probe SSOT (GB2)
The d4-probe-domdone-guard test hardcoded its probed-route set as a literal
["agentic-chat","tool-rendering"] parallel to the probe's own inline
route literals — a drift hazard if the probe starts driving a new D4 route.
Introduce D4_DEMO_ROUTE_AGENTIC_CHAT / D4_DEMO_ROUTE_TOOL_RENDERING and a
D4_PROBED_DEMO_ROUTES source-of-truth array in the probe, wire the two
runLevel demo/demoPath call sites (and hasToolRendering) to them, and have
the guard derive PROBED_ROUTES from that export. Add a superset assertion so
the derived set can never narrow coverage below {agentic-chat,tool-rendering}.
Behavior-preserving: the route strings are unchanged.
|
||
|
Jordan Ritter
|
3dc43b9ca9 |
feat(showcase): bring prod autoUpdates under drift-gate management
Prod autoUpdates is now "disabled" (was "unmanaged") for every service, so the drift gate enforces prod as well as staging. Paired with disabling autoUpdates on the live prod Railway services. Regenerates the SSOT JSON. |
||
|
Jordan Ritter
|
e0c7fd30ee |
fix(showcase): alert when an all-legs-cancelled build produced no successes
The notify-all-builds-failed and notify jobs keyed off a 'failure' rollup / bare failure(), so a build where every real service failed but one leg was cancelled (contention) rolled up to 'cancelled' and sent no alert — the same blind spot as the redeploy guard. Fire on any_success == 'false' (guarded by a status function so a user-cancelled run stays silent). Extends the guard test. |
||
|
Jordan Ritter
|
bd000f973b |
showcase: consolidate deploy onto CI-explicit path (guard fix, autoUpdates SSOT, drift gate, reconcile) (#6082)
## Showcase deploy-mechanism consolidation Consolidates the showcase Railway deploy path onto a single **CI-explicit** mechanism, so we can safely retire Railway's registry auto-watch (the source of the surprise "Service aimock upgraded to latest" emails). Design proposal: [Notion — Showcase Deploy-Mechanism Consolidation](https://app.notion.com/p/3a33aa38185281e4b64cc5bebde92d91). ### What & why The "aimock upgraded to latest" email was never a per-service config choice — it was a **CI bug** letting Railway's watcher win a race: a Renovate PR that only touches `showcase_build.yml` forces a full-fleet rebuild; the LFS `shell` leg gets cancelled under runner contention; and the `redeploy-staging` guard (`needs.build.result != 'cancelled'`) then skipped the CI redeploy for the **whole fleet**, orphaning aimock's fresh digest for Railway's watcher to pick up. The `autoUpdates` setting itself had also silently drifted (24 services `minor` / 17 none) — tracked in no SSOT, gated by nothing. ### The four changes (one commit each) 1. **`fix(showcase)` — the P0 guard bug.** Relax the `redeploy-staging` **and** `redeploy-staging-starters` guards so a cancelled sibling leg no longer skips the fleet's staging redeploy; they now redeploy the already-computed successful-service list. A guard-evaluation test reads the live workflow `if:` strings and models GitHub's matrix rollup. 2. **`feat(showcase)` — autoUpdates SSOT (per-env, staging-first).** Add a **per-env** `autoUpdates` policy to every service in `railway-envs.ts` — **staging: `disabled`** (enforced), **prod: `unmanaged`** (left exactly as-is until a later migration). Regenerate `railway-envs.generated.json`. CI-explicit redeploy becomes the single deploy path on staging. 3. **`feat(showcase)` — drift gate.** New CI gate fails when a live Railway service's `autoUpdates` diverges from the SSOT. Reads `Environment.config` (autoUpdates isn't on the typed `ServiceSource` output), **enforces managed (`disabled`) envs and skips `unmanaged` ones** (so prod is untouched), **fails closed per-env** on zero-checked, and skips cleanly on fork PRs with no Railway token. 4. **`feat(showcase)` — scheduled reconcile.** CI-owned self-heal (every 15m) comparing each staging service's deployed digest against GHCR `:latest`, re-running the staging redeploy for lagging services and alerting Slack. Invariant: **green ⟺ every in-scope service confirmed current**; any unconfirmed service (lag, digest error, dropped redeploy, empty scope, thrown redeploy) alerts and exits non-zero. ### Verification - Every behavior change carries red-green tests; **230 tests pass**, `tsc` clean, `oxfmt`/`oxlint` clean, generated JSON in sync, workflows parse. - Reviewed via a full CR loop (Tier 3, 5 rounds to convergence); the reconcile's fail-loud invariant was hardened across rounds (silent-green holes, stale-digest ordering, expansion false-positives, test hygiene). ### Rollout (staging-first) - **Staging is flipped live as part of this change** — `autoUpdates` disabled on all staging services (snapshot-first, verified only `autoUpdates` changed). The drift gate now enforces staging. - **Prod is untouched** — its `autoUpdates` stay exactly as-is and the gate marks prod `unmanaged` (skipped). Migrating prod is a deliberate follow-up (flip prod live + change prod SSOT `unmanaged`→`disabled` together) once we're comfortable with staging on the new mechanism. No transition window where anything is unguarded. ### Follow-ups (from CR, non-blocking) - Dedup the reconcile alert's `unconfirmed` list by service key (cosmetic double-listing; exit code already correct). - Harden the sibling `notify-all-builds-failed`/`notify` jobs against the same all-legs-cancelled rollup (pre-existing, in a job this PR doesn't touch). - Minor: `postSlackAlert` try/catch belt; a few added test assertions; comment/doc accuracy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jordan Ritter
|
6b4b1ee179 |
fix(showcase): gate D4 chat probe completion on domDone to close empty-response render race (#6081)
## Incident
At ~21:19Z on 2026-07-20 the prod showcase Feature Matrix showed the
`claude-sdk-python` column whole-column-red (✓0/✗37). It self-recovered
by ~21:30Z. Not infra, not staleness: `/api/health` was 200 throughout,
`commErrorKinds` was empty, and the image was current.
## Root cause
The shared D4 chat probe (`d4-chat-roundtrip.ts`) emitted ONE transient
red — `failureSummary: "empty assistant response"`, no
`errorClass`/`errorDesc`. Because the dashboard's `resolveD4` is
**slug-keyed**, that single `chat:<slug>` row gated all 37 feature cells
for the slug → the whole column went red off one flap.
The producer-side origin is a **render race**. `readTurnComplete()`
gated turn completion on `domDone || sseDone`. `sseDone` is the
transport-level `RUN_FINISHED` counter — a synchronous raw-byte SSE
parse that runs *outside* React and can fire at-or-before the React
commit that actually renders the assistant text. `domDone` (the
`data-copilot-running` true→false DOM edge) is coalesced into that
*same* React commit (`use-agent.tsx`
`batchedForceUpdate`/`queueMicrotask`). Under contention, gating on
`sseDone` let the fast-fail grace window elapse while the DOM was still
empty, redding a turn that was about to render correctly.
## The change
`complete: domDone || sseDone` → `complete: domDone`. The moment
`domDone` is true, the assistant text is already in the DOM (same
commit). `observed` still includes `sseDone`, so a turn whose attribute
never appears still gets the wider polling window rather than
fast-failing. `FIRST_TOKEN_GRACE_MS` and `NON_COMPLETION_RETRY_LIMIT`
are untouched. `domDone` is not a new signal — the probe already
observes that edge via `readTurnState`; this only changes which existing
signal `complete` trusts.
**Throughput-neutral:** `runAttempt` returns on a non-empty
`readAssistantText` *before* `readTurnComplete` is consulted, so the
happy path is unchanged. Only a completed-but-empty turn's
classification changes. A genuinely-empty completed turn still reds
(correct) — only the race false-positive is removed.
## Red → green proof (deterministic, on the real code path)
A new deterministic timing-sim regression (`sseLeadMs` fixture)
decouples `sseDone` from `domDone` *in time* and drives the real
`readTurnComplete`/`runAttempt`. Two cases: a TIMING SIM (sseDone at
send, text at 2500ms past the ~2000ms grace window) and a FORMULA PIN
(sseDone-only, token never arrives → asserts `sends === 2`, catching a
literal revert to `domDone || sseDone` via the retry-fired
discriminator, not a wall-clock threshold).
**RED — test hunks applied on top of UNFIXED probe code:**
```
FAIL src/probes/drivers/d4-chat-roundtrip.test.ts > d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > TIMING SIM: sseDone fires EARLY, domDone+text land LATE together → GREEN (RED pre-fix)
AssertionError: expected 'red' to be 'green' // Object.is equality
Expected: "green"
Received: "red"
❯ src/probes/drivers/d4-chat-roundtrip.test.ts:3195:21
FAIL src/probes/drivers/d4-chat-roundtrip.test.ts > d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > FORMULA PIN: sseDone alone must NOT flip `complete` — a turn that only ever fires sseDone stalls+retries (does NOT fast-fail as completed-empty)
AssertionError: expected 1 to be 2 // Object.is equality
- Expected 2
+ Received 1
❯ src/probes/drivers/d4-chat-roundtrip.test.ts:3229:19
Test Files 1 failed (1)
Tests 2 failed | 82 skipped (84)
```
**GREEN — probe fix hunk then applied, same tests re-run:**
```
✓ src/probes/drivers/d4-chat-roundtrip.test.ts (84 tests | 82 skipped) 9026ms
✓ d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > TIMING SIM: sseDone fires EARLY, domDone+text land LATE together → GREEN (RED pre-fix) 5017ms
✓ d4 render-race: sseDone-leads / domDone-lags turn completion (Fix B) > FORMULA PIN: sseDone alone must NOT flip `complete` — a turn that only ever fires sseDone stalls+retries (does NOT fast-fail as completed-empty) 4007ms
Test Files 1 passed (1)
Tests 2 passed | 82 skipped (84)
```
Full `d4-chat-roundtrip.test.ts` suite after the fix: **84 passed (84)**
— no regression.
## Real-probe forcing (Part B) — honest disclosure
The live race is low-rate and non-deterministic. The `--direct`
(direct-LLM) live-probe forcing path could **not** be stood up within
reasonable effort in this environment: Docker is running, but `--direct`
requires a real Anthropic API key that is not present here, and forcing
a probabilistic React-commit-timing race across ~300 reps on a
freshly-built `claude-sdk-python` integration stack is beyond a
reasonable bound for the bonus real-surface proof. It was **not** faked.
The deterministic timing-sim regression above exercises the real
`readTurnComplete`/`runAttempt` code path and is the gating proof.
**Recommendation:** a multi-tick staging dashboard watch post-merge to
confirm the flap does not recur.
## Scope note
This PR fixes the **producer-side** root cause. The **render-layer
de-amplifier** (a single transient D4 flap should degrade a slug's cells
to *amber*, not whole-column-red via slug-keyed `resolveD4` fan-out) is
owned separately by the dashboard-ladder redesign.
## CR follow-ups (post-review hardening)
Three small follow-ups from the 7-agent CR + adversarial verify (which
found zero live bugs in the Fix B core). The `complete: domDone` core is
untouched.
- **Regression-guard test (hardening).** New
`d4-probe-domdone-guard.test.ts` (source-level, TS AST). It asserts
every D4-probed demo page — `/demos/agentic-chat` for every integration,
`/demos/tool-rendering` where present — renders `<CopilotChat/>` in the
attribute-bearing **self-closing** form, not the children render-prop /
slot form. Only the self-closing form hits the `CopilotChatView` branch
that emits `data-testid="copilot-chat"` + `data-copilot-running`, which
the probe's `domDone` gate reads; the `if (children)` branch returns a
`display:contents` wrapper with neither, which would silently regress
the cell to the ~60s / double-send fallback. That regression is
invisible to the probe's own fake-injected unit tests, so it is guarded
structurally at CI time. Chose the source/AST approach over full page
render because the pages are Next.js `"use client"` app-router
entrypoints mounting a live `<CopilotKit runtimeUrl=.../>` provider +
framework agents (impractical to render faithfully in a harness unit
test).
**Red-green proof.** Temporarily mutated
`langgraph-python/agentic-chat/page.tsx` to the children render-prop
form → guard **RED** (`langgraph-python/agentic-chat … expected 0 to be
greater than or equal to 1`, 1 failed | 40 passed). Reverted the
mutation → **GREEN** (41 passed). Mutation fully reverted; net change is
the test only.
- **Comment factual fix.** In `d4-chat-roundtrip.ts` near the `complete:
domDone` gate: the comment said `observed` still includes `sseDone`
"(below)", but `observed` is defined *above*. Corrected "(below)" →
"(above)".
- **Timing-sim robustness pin.** The `TIMING SIM` test's discriminating
power depended on `FIRST_TOKEN_GRACE_MS (2000) < firstTokenDelayMs
(2500) < ceiling (~4000)`, encoded only in prose. Now imports the actual
`FIRST_TOKEN_GRACE_MS` source constant (newly exported) and asserts that
ordering, so raising the grace constant later fails the test loudly
instead of silently ceasing to discriminate. No existing assertion
weakened.
Quality: harness `d4-chat-roundtrip.test.ts` + new guard test green (125
tests); `tsc --noEmit` clean; `tsc -p tsconfig.build.json` clean; `oxfmt
--check` clean; `oxlint` warnings-only (no errors).
|
||
|
Jordan Ritter
|
eddfcc07e8 |
docs(showcase): fix stale cross-reference in D4 probe comment
The comment near the complete:domDone gate said `observed` still includes `sseDone` "(below)" — but `observed` is defined ABOVE the comment, not below. Correct "(below)" to "(above)". |
||
|
Jordan Ritter
|
4bf796b713 |
test(showcase): guard D4 probe domDone gate + pin first-token grace ordering
Add a source-level (TS AST) regression guard asserting every D4-probed demo page (/demos/agentic-chat always; /demos/tool-rendering where present) renders <CopilotChat/> in the attribute-bearing self-closing form, not the children render-prop form. The self-closing form is the CopilotChatView branch that emits data-testid="copilot-chat" + data-copilot-running, which the probe's domDone completion gate reads; the render-prop branch omits it, which would silently regress the cell to the ~60s/double-send fallback path. Invisible to the probe's own fake-injected unit tests, so guarded structurally at CI time. Also pin the TIMING SIM test's discriminating window against the actual FIRST_TOKEN_GRACE_MS source constant (now exported): assert FIRST_TOKEN_GRACE_MS < firstTokenDelayMs < attempt-0 ceiling so that raising the grace constant later fails the test loudly instead of silently ceasing to discriminate pre-fix vs post-fix. |
||
|
Jordan Ritter
|
a33e313df8 |
feat(showcase): make autoUpdates per-env for staging-first rollout
autoUpdates is now per-env: staging is enforced "disabled" while prod is "unmanaged" (the drift gate skips it) so prod stays untouched until a later migration. The gate enforces managed envs and skips unmanaged ones; the zero-checked floor applies only to managed envs. Regenerates the SSOT JSON. |
||
|
Tyler Slaton
|
50805ab47d |
docs: fix stale quickstart/link references and add MCP Codex setup (#6079)
## What & why Three independent documentation-accuracy fixes, batched into one PR. ### 1. Dead spec links + gen-ui page gaps (Closes #3975) `generative-ui-specs-overview.mdx` (rendered at `/whats-new/generative-ui-spec-support`) linked to the retired `/generative-ui/specs/<spec>` subgroup. Repointed to canonical destinations and added the frameworks list the issue asked for. **Verified live (HTTP status against docs.copilotkit.ai):** | Link | Before | After | | --- | --- | --- | | A2UI | `/generative-ui/specs/a2ui` (301 hop) | `/generative-ui/a2ui` → **200** | | MCP Apps | `/generative-ui/specs/mcp-apps` (301 hop) | `/generative-ui/mcp-apps` → **200** | | Open Generative UI (new) | — | `/generative-ui/open-generative-ui` → **200** | - **Supported Frameworks** list added — all 12 `/<slug>/quickstart` targets return **200** live (LangGraph Py/TS, Google ADK, MS Agent, AWS Strands, Mastra, PydanticAI, CrewAI, Agno, AG2, LlamaIndex, Claude Agent SDK, Deep Agents). - **Open-JSON-UI is intentionally NOT linked:** `/generative-ui/open-json-ui` is a placeholder pulled from the nav and redirected to `/generative-ui` on purpose (`next.config.ts` — `// AI-slop placeholder pulled from nav until properly authored`). Linking only the two specs that have live detail pages avoids sending readers to a redirect. Open-JSON-UI is still described in the comparison table on the page. - Open Generative UI is a CopilotKit capability (not an external spec), so it sits under "related capabilities." > Note: the issue/support-bot suggested a `/learn/...` path — there is no `/learn/` tree in shell-docs; the canonical homes are the flat `/generative-ui/<spec>` pages. ### 2. CLI `init` vs. existing app (Closes #2525) Maintainer resolution was "fix the docs." The original "`init` bootstraps your existing Next.js app" claim was already removed in the shell-docs migration (the reported `/direct-to-llm/guides/quickstart` now resolves to the Built-in Agent quickstart, which is fully manual). To remove the remaining ambiguity: - **Built-in Agent quickstart:** added an "Already have an app?" callout — existing apps skip `create-next-app`. - **CLI guide (`cli.mdx`):** clarified that `create` (aliased `init`) scaffolds a brand-new project in its own directory and does not detect/bootstrap an existing app; points to the manual install in the Quickstart. Verified against `copilotkit@latest` (4.3.0): `init --help` → *"Initialize a **new** CopilotKit project … before scaffolding"*, `-n, --name` *"names the local app **and its directory**"*, and `init`/`create` are aliases. ### 3. Codex setup for the MCP guide (Closes #2526) Added a **Codex** section to `mcp-server-setup.mdx` using the stdio `mcp-remote` bridge in `~/.codex/config.toml`, matching the page's existing command-based pattern (Cursor / Windsurf / Claude Desktop), plus the `codex mcp add` shortcut. Verified against the installed Codex CLI: `codex mcp --help` lists `add`/`list`/`get`/`remove`, and the `[mcp_servers.<name>]` table with `command`/`args` matches OpenAI's Codex config reference. Per the issue thread, the macOS `mcp-remote` port-blocking concern is **not** claimed to be solved — only the Codex config is documented. ## Testing - **#3975 (links):** curled every added/changed URL against the live docs — A2UI, MCP Apps, Open Generative UI, and all 12 framework quickstarts return **200**; confirmed `/generative-ui/open-json-ui` is a deliberate redirect (hence unlinked). Pre-existing `/ag-ui-protocol` and `/generative-ui` links are stable 301→200 and left as-is. - **#2525 (CLI):** ran `npx copilotkit@latest init --help` on the published `latest` (4.3.0) — confirmed new-directory scaffolding, no existing-app detection; verified `[Quickstart](/quickstart)` serves the manual-install page live (`create-next-app` + "Install CopilotKit packages"). - **#2526 (Codex):** ran `codex mcp --help` to confirm subcommands; new section reuses the file's existing `<Steps>`/fenced-code structure. `<Callout>` is a registered global MDX component (`src/lib/mdx-registry.tsx`), already used unimported on the Built-in Agent quickstart. - Docs-only; no code paths affected. Closes #3975 Closes #2525 Closes #2526 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jordan Ritter
|
077913e5b4 |
feat(showcase): scheduled staging reconcile with Slack self-heal
Adds a CI-owned reconcile (every 15m) that compares each staging service's deployed digest against GHCR :latest and re-runs the staging redeploy for lagging services, alerting Slack. The run is green only when every in-scope service is confirmed current; any unconfirmed service (lag, digest error, dropped redeploy, empty scope, or a thrown redeploy) alerts and exits non-zero. Exposes per-service redeploy records from redeploy-env for accurate per-service remediation confirmation. |
||
|
Jordan Ritter
|
b34debc02b |
feat(showcase): add autoUpdates drift gate against live Railway config
New CI gate fails when a live Railway service's autoUpdates diverges from the SSOT (every service must be disabled). Reads Environment.config (autoUpdates is not on the typed ServiceSource output), fails closed per-env when it verifies zero services, and skips cleanly on fork PRs that lack a Railway token. |
||
|
Jordan Ritter
|
2ecba43d2f |
feat(showcase): track autoUpdates in SSOT, disabled fleet-wide
autoUpdates was tracked nowhere and had drifted (24 services minor / 17 none). Add an explicit disabled autoUpdates policy to every service in railway-envs.ts and regenerate railway-envs.generated.json, making CI-explicit redeploy the single deploy path instead of Railway's registry auto-watch. |
||
|
Jordan Ritter
|
2d6883568e |
fix(showcase): don't skip staging redeploy when a build leg is cancelled
The redeploy-staging and redeploy-staging-starters jobs guarded on needs.build.result != 'cancelled', so a single cancelled matrix leg (e.g. the Git-LFS shell build under runner contention) skipped the whole fleet's staging redeploy even when the other 27 services built fine. Relax both guards to redeploy the already-computed successful-service list. Adds a guard-evaluation test that reads the live workflow if: strings and models GitHub's matrix rollup. |
||
|
Jordan Ritter
|
cae8e78ac8 |
fix(showcase): gate D4 chat probe completion on domDone to close empty-response render race
readTurnComplete() gated turn completion on `domDone || sseDone`. sseDone (the
transport-level RUN_FINISHED counter) is a synchronous raw-byte parse OUTSIDE
React and can fire at-or-before the React commit that renders the assistant
text; domDone is coalesced into that SAME commit (use-agent.tsx
batchedForceUpdate / queueMicrotask). Gating on sseDone let the fast-fail grace
window elapse with the DOM still empty under contention, redding a turn that was
about to render correctly ("empty assistant response").
Gate completion on domDone alone. observed still includes sseDone, so a turn
whose attribute never appears still gets the wider polling window. Throughput-
neutral: runAttempt returns on non-empty readAssistantText before
readTurnComplete is consulted, so the happy path is unchanged; a genuinely-empty
completed turn still reds.
|
||
|
Jordan Ritter
|
45898bd1b6 |
test(showcase): pin D4 chat probe sseDone-leads/domDone-lags render race
Add a deterministic timing-sim regression for the D4 chat-roundtrip probe render race. The sseLeadMs fixture decouples the transport RUN_FINISHED counter (sseDone) from the DOM run-stop edge (domDone) in time, so a turn whose sseDone fires early while domDone + assistant-text land later in the same React commit is exercised on the real readTurnComplete/runAttempt path. Two cases: a TIMING SIM (sseDone at send, text at 2500ms past the grace window) that reds pre-fix, and a FORMULA PIN (sseDone-only, token never arrives) asserting sends===2, catching a literal revert to domDone||sseDone via the retry-fired discriminator rather than a wall-clock threshold. |
||
|
Benjamin Taylor
|
7869d31e64 |
docs: drop unpublished Open-JSON-UI link, move Open Generative UI to related
CR: /generative-ui/open-json-ui is a deliberately-unpublished placeholder (next.config.ts redirects it to /generative-ui, pulled from nav until authored), so link only the two specs with live detail pages (A2UI, MCP Apps). Open Generative UI is a CopilotKit capability rather than an external spec, so it moves under related capabilities. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Benjamin Taylor
|
c9511cf4f9 |
docs: fix stale quickstart/link references and add MCP Codex setup
Fixes three independent documentation-accuracy issues: - #3975: repoint the retired /generative-ui/specs/<spec> links to the canonical flat /generative-ui/<spec> paths, add an Open Generative UI entry, and add a Supported Frameworks list on the Generative UI Spec Support page. - #2525: clarify that the CLI `create`/`init` scaffolds a brand-new project in its own directory and does not bootstrap an existing app; point existing-app users to manual installation. - #2526: add a Codex setup section to the MCP server guide using the stdio `mcp-remote` bridge, matching the page's existing pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jordan Ritter
|
305cfd494b |
feat(showcase): add outcome reaction to promote-notify init Slack message
Add an emoji reaction to the original promote-notify init message reflecting the net run outcome, so operators can see success/failure at a glance without opening the thread reply: success -> white_check_mark (checkmark) partial -> warning total -> x The live workflow calls reactions.add on the init post (guarded on a successful init post, warn-only on failure to mirror the thread reply). The dry-run harness emits the reaction it would add, using a byte-identical case mapping enforced by a new anti-drift bats guard. Adds bats coverage asserting the emitted reaction name per fixture. |
||
|
Jordan Ritter
|
999264f648 |
chore: bump aimock to 1.37.4 (multi-turn fixture matching fix) (#6071)
Bumps the showcase's `@copilotkit/aimock` dependency from `1.26.1` to **`1.37.4`** so the showcase's record/replay fixture stack picks up the multi-turn fixture-matching fix. **Why:** aimock v1.37.4 (aimock #319) fixes record/replay `hasToolResult` symmetry — recorded multi-turn fixtures now match on replay. The showcase records/replays LLM fixtures for its demo cells, so this pulls the fix into the showcase. **What changed** - `showcase/scripts/package.json`: `@copilotkit/aimock` `1.26.1` → `1.37.4` (exact pin, matching existing style) - `showcase/scripts/package-lock.json`: regenerated via `npm install --package-lock-only` (this lock is consumed by `npm ci` in the shell/shell-docs/shell-dashboard Dockerfiles) - `pnpm-lock.yaml`: regenerated via `pnpm install --lockfile-only` (showcase/scripts is a pnpm workspace member) The only version change is aimock (and its transitive tree in the npm lock). The minor eslint-config-next peer-resolution relabeling in `pnpm-lock.yaml` is benign normalization — no package version changes; both jiti variants already existed in the lock. **Validation gap:** this is a dependency bump only. Full validation (running the showcase multi-turn recording against 1.37.4 to confirm the recorded fixtures match on replay) requires the running showcase stack and is not exercised here. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jordan Ritter
|
107a17e493 |
fix(showcase): wire langroid declarative-gen-ui via Option A (JS-injected A2UI) (#6070)
## Summary Supersedes #6058's two-stage approach (outer `generate_a2ui` → secondary Python LLM call → `render_a2ui`), which severed across the prod streaming boundary. **Option A** (mirroring crewai-crews #6067): `injectA2UITool` defaults to `true`, so CopilotKit's `A2UIMiddleware` injects `render_a2ui` into `RunAgentInput.tools`. The langroid `agui_adapter.py` now merges those injected tools into its OpenAI call, so the LLM calls `render_a2ui` directly. The middleware intercepts the tool call stream, builds `a2ui_operations`, and fires `RUN_FINISHED` — no secondary Python LLM pass needed. ## Changes - **`agent.py`**: Remove ~550 lines of two-stage A2UI infrastructure (`generate_a2ui_via_llm`, `_a2ui_error`, `_resolve_a2ui_model`, `_get_a2ui_llm`, `_RENDER_A2UI_FUNCTION_SPEC`, etc.). Replace `GenerateA2UITool.handle` with a stub that logs loudly on regression (middleware should always intercept before reaching Python). - **`agui_adapter.py`**: Merge `run_input.tools` (AG-UI-injected) into the OpenAI tools list so `render_a2ui` is visible to the LLM. Remove `set_last_user_message` call (ContextVar no longer needed). - **`route.ts`**: Remove `injectA2UITool: false`; keep `defaultCatalogId` pin. - **`gen-ui-declarative.json`**: Replace 9 two-stage fixtures with 4 single-stage fixtures matching `toolName: render_a2ui` + `context: langroid`. ## Root cause of prior RED The langroid adapter builds its OpenAI tool list from `ALL_TOOLS` (Python-side registry) via `_get_openai_tools()`, which does NOT include `render_a2ui`. The `A2UIMiddleware` injects `render_a2ui` into `RunAgentInput.tools` at the AG-UI protocol level, but `agui_adapter.py` ignored `run_input.tools` entirely — so the LLM never saw `render_a2ui` in its tool list, never called it, and the fixture never matched. ## Red-green proof **RED** (from main, before changes): ``` ✗ d6:langroid/gen-ui-declarative red (0.0s) state=red 0 passed, 1 failed ``` **GREEN** (after this PR's changes, rebuild from worktree): ``` ✓ d6:langroid/gen-ui-declarative green (0.0s) 1 passed ✓ Tests passed for langroid:declarative-gen-ui ``` Test command: `bin/showcase test langroid:declarative-gen-ui --d6 --isolate --rebuild` ## Related - Supersedes #6058 (two-stage approach, now reverted in this integration) - Mirrors #6067 (crewai-crews Option A fix, same pattern) |
||
|
Jordan Ritter
|
683ce92c5b | chore: bump aimock to 1.37.4 (multi-turn fixture matching fix) | ||
|
Jordan Ritter
|
1196c7b637 |
test(showcase): align langroid test_generate_a2ui with Option A (JS-injected A2UI)
Option A removed the two-stage server-side A2UI planner (generate_a2ui_via_llm, _get_a2ui_llm, _resolve_a2ui_model, _A2uiError, _A2uiErrorKind, _RENDER_A2UI_FUNCTION_SPEC, _RENDER_A2UI_TOOL_SPEC, _a2ui_error). The CopilotKit JS runtime A2UIMiddleware now intercepts generate_a2ui before it reaches Python and drives the render_a2ui LLM pass itself. test_generate_a2ui.py was importing the removed symbols at collection time, causing an ImportError that failed CI's "Run showcase package Python unit tests" step for Python 3.12. Fix: remove all tests for the deleted two-stage planner infra; rewrite GenerateA2UITool.handle() tests to the Option A contract (handle fires only on middleware regression, returns structured error JSON, logs ERROR); retain all tests that remain valid: _ToolErrorKind enum identity, backend tool handle() happy+error paths, create_agent wiring, module hygiene (no openai at load time, clean subprocess import). |
||
|
Jordan Ritter
|
46f810779c |
fix(showcase): wire langroid declarative-gen-ui via Option A (JS-injected A2UI)
Supersedes #6058's two-stage approach (outer generate_a2ui → secondary Python LLM call → render_a2ui), which severed across the prod streaming boundary. Option A: `injectA2UITool` (default true) causes CopilotKit's A2UIMiddleware to inject `render_a2ui` into RunAgentInput.tools. The langroid agui_adapter now merges those injected tools into its OpenAI call so the LLM can call render_a2ui directly. The middleware intercepts the tool call stream, builds a2ui_operations, and fires RUN_FINISHED — no secondary Python LLM pass needed. Changes: - agent.py: remove ~550 lines of two-stage A2UI infrastructure (generate_a2ui_via_llm, _a2ui_error, _resolve_a2ui_model, _get_a2ui_llm, etc.); replace GenerateA2UITool.handle with a stub that logs loudly if middleware interception regresses - agui_adapter.py: merge run_input.tools (AG-UI-injected) into the OpenAI tools list so render_a2ui is visible to the LLM; remove set_last_user_message call - route.ts: remove injectA2UITool: false; keep defaultCatalogId pin - gen-ui-declarative.json: replace 9 two-stage fixtures with 4 single-stage fixtures matching toolName: render_a2ui + context: langroid (mirrors crewai-crews Option A) Mirrors the crewai-crews fix from #6067. |
||
|
Jordan Ritter
|
54eea138b8 |
fix(showcase): wire ag2 declarative-gen-ui via Option A (JS-injected A2UI) (#6069)
## Summary - **Route**: Drop `injectA2UITool: false` from `copilotkit-declarative-gen-ui/route.ts` — default `true` enables JS middleware injection - **Backend**: Replace the complex inner-LLM two-stage body in `a2ui_dynamic.py` with a fail-loud stub matching the crewai-crews Option A pattern (no more openai/AsyncOpenAI, no `_request_context`, no `tools/RENDER_A2UI_TOOL_SCHEMA`) - **Fixture**: Update `_meta` note and `_comment` fields to reflect Option A; fixture structure was already correct for aimock two-stage matching (outer `generate_a2ui` matched by `context:ag2`; inner `render_a2ui` matched by `toolName:render_a2ui`) ## Why Option A works AG2's AG-UI adapter has no Python-side A2UI injection. Option A routes the secondary LLM pass through the JS CopilotKit runtime middleware, which intercepts the agent's `generate_a2ui` toolcall, drives `render_a2ui` itself, synthesises the tool result, and fires `RUN_FINISHED`. This is the same pattern as the merged crewai-crews fix (#6067) and mirrors the green langgraph-python sibling. The previous two-stage backend approach failed under aimock because the backend's inner `AsyncOpenAI` call to `render_a2ui` bypassed aimock entirely (aimock only intercepts the frontend→backend path). ## Red → Green evidence **RED** (pristine, before changes): ``` ▸ Testing ag2:declarative-gen-ui (--d6)... ▸ Isolation active: project=showcase-iso19 slot=19 ✗ d6:ag2/gen-ui-declarative red (0.0s) state=red 0 passed, 1 failed ⚠ Tests failed for ag2:declarative-gen-ui (exit 1) ``` **GREEN** (after Option A changes): ``` ▸ Testing ag2:declarative-gen-ui (--d6)... ▸ Isolation active: project=showcase-iso20 slot=20 ✓ d6:ag2/gen-ui-declarative green (0.0s) 1 passed ✓ Tests passed for ag2:declarative-gen-ui ``` ## Files changed - `showcase/integrations/ag2/src/app/api/copilotkit-declarative-gen-ui/route.ts` — drop `injectA2UITool: false`, update header comment - `showcase/integrations/ag2/src/agents/a2ui_dynamic.py` — replace inner-LLM body with fail-loud stub - `showcase/aimock/d6/ag2/gen-ui-declarative.json` — update `_meta`/`_comment` for Option A 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Sam Julien
|
5a35379906 |
docs: add Thread & History Lifecycle guide (#5988)
## What & why
Adds a new framework-agnostic guide — **Thread & History Lifecycle**
(`docs/threads-lifecycle.mdx`) — that walks the full client-side
lifecycle of a conversation thread as one narrative. It fills the gap
between the [Headless Threads](/threads) how-to and the [Threads &
Persistence Architecture](/premium/threads-explained) explanation, and
directly answers a recurring cluster of support questions.
## Sections
- **How the threadId is created** — UUID v4, client-minted at mount, the
resolution precedence, and the remount-stability caveat (auto-minted ids
re-mint on remount / StrictMode; pass an explicit `threadId` for
continuity).
- **How history is restored** — explicit-`threadId` `connect()` replay
vs. manual `agent.setMessages(...)`; clarifies there is **no v2
`initialMessages`** and no v2 `useCopilotChat` (read via
`useAgent().agent.messages`).
- **Switching / starting threads** — `setActiveThreadId(id, { explicit
})` and `startNewThread()`, plus the prop-controlled no-op guard.
- **Creating a thread with your own API on first message** —
mint-up-front + `setActiveThreadId`/`threadId` prop as the robust path;
the headless `CopilotChatInput.onSubmitMessage` seam for submit-time
interception (noting the built-in `<CopilotChat>` overrides it).
- **CopilotKit threads vs. your framework's checkpointer** — two layers
correlated only by `threadId`; a LangGraph checkpointer creates
checkpoint tables, not a CopilotKit "threads" table.
- **MCP Apps activity & history** — activity messages are
frontend/middleware constructs (no server store); re-synthesize on
hydration.
- **v1 vs v2** disambiguation (incl. the two different `useThreads`
hooks).
## Addresses
Recurring thread-lifecycle questions: #4790, #4778, #5434, #2242, #5931.
(I'll close those pointing here once this lands.)
## Testing / accuracy
All referenced APIs verified present on `main`: `useThreads` (v2),
`useCopilotChatConfiguration` (`setActiveThreadId`/`startNewThread`),
`useAgent().agent.setMessages/addMessage`,
`CopilotChatInput.onSubmitMessage`, `<CopilotChat threadId>`, and the
`mcp-apps` activity type. All five cross-doc links resolve. Added to the
"Threads" group in `docs/meta.json`.
Note: written against the current v2 APIs — three details were corrected
against source during authoring (`startNewThread` not `createThread`;
`onSubmitMessage` is headless-only; no v2
`useCopilotChat`/`initialMessages`).
|
||
|
Jordan Ritter
|
0408f821a7 |
fix(showcase): wire ag2 declarative-gen-ui via Option A (JS-injected A2UI)
Remove the backend two-stage inner-LLM pattern (injectA2UITool:false + Python-side secondary openai call) in favour of Option A: the CopilotKit JS runtime middleware intercepts the agent's no-arg generate_a2ui toolcall and drives the render_a2ui secondary LLM pass itself, synthesising the tool result and firing RUN_FINISHED. Matches the just-merged crewai-crews fix (#6067) and mirrors langgraph-python's green reference pattern. Changes: - route.ts: drop `injectA2UITool: false` (default true enables JS injection) - a2ui_dynamic.py: replace complex inner-LLM body with a fail-loud stub (no more openai/AsyncOpenAI import, no _request_context dependency, no tools/RENDER_A2UI_TOOL_SCHEMA import) - gen-ui-declarative.json: update _meta note + _comment fields to reflect Option A (fixture structure was already correct for two-stage aimock matching; outer generate_a2ui matched by context:ag2, inner render_a2ui matched by toolName:render_a2ui) Red→Green: D6 control-plane harness confirmed red before (state=red, exit 1) and green after (1 passed, exit 0). |
||
|
Benjamin Taylor
|
6fba48faa9 |
docs(threads): reorder Threads nav and tighten lifecycle/architecture boundary
Addresses CR on the Thread & History Lifecycle guide (PR #5988): - Reorder the Threads navigation consistently across the root and all authored-framework meta.json files to: Overview, Threads Drawer, Headless Threads, Import Thread History, Threads & Persistence Architecture, Thread & History Lifecycle. Re-anchor the injected Architecture page before the Lifecycle page and update the nav-order test. - Tighten the two-page boundary: Architecture now owns platform behavior (persistence, replay, realtime sync, locks, failure modes) and defers client-side steps to Lifecycle; Lifecycle keeps only brief persistence context and links to Architecture for the deeper model. - Writing pass reducing heavy em-dash use in both pages, keeping em dashes only in link-gloss lists and table placeholders. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Jordan Ritter
|
f44cb57669 |
fix(showcase): wire crewai-crews declarative-gen-ui via Option A (JS-injected A2UI)
Switch the crewai-crews gen-ui-declarative cell from a broken Option B (Python-side injection, which has no mechanism in the ag_ui_crewai adapter) to Option A (JS-runtime-injected A2UI): - route.ts: remove `injectA2UITool: false`; keep `defaultCatalogId` to pin the catalog so models that omit catalogId don't get a "Catalog not found" render error. - declarative_gen_ui.py: replace `GenerateA2uiTool` with a no-arg `_GenerateA2uiNoArgTool` that raises loudly if called directly (the A2UIMiddleware should always intercept before Python). - definitions.ts + renderers.tsx: add `DataTable` component (columns/rows schema + `data-testid="declarative-data-table"` renderer); add missing `data-testid="declarative-info-row"` to the `InfoRow` renderer. Both testids are required by the D5 harness (turns 2 and 4 respectively) and were present in the langgraph-python reference catalog but absent here. - gen-ui-declarative.json: rewrite D6 aimock fixtures from the old broken two-stage pattern (generate_a2ui → inner render_a2ui) to the correct single-stage pattern (LLM calls render_a2ui directly); all four pills now match `toolName: render_a2ui, context: crewai-crews` and return full component trees that satisfy the harness minCounts assertions. RED (main): "CrewAI flow failed; see server logs" on every pill — `injectA2UITool: false` disabled the middleware; no render_a2ui tool in the agent's tool list; aimock fixture matcher never fired; no surface. GREEN (this branch): all 4 turns pass with assertions, 1 passed (0.0s). |
||
|
Ben Taylor
|
30cc551a9b |
docs(langgraph,crewai-flows): remove broken useCopilotContext example (#5821)
## What does this PR do? Removes the "Using setThreadId" example from the LangGraph and CrewAI Flows persistence docs. That example calls `useCopilotContext()`, which is a v1-only hook not exported from `@copilotkit/react-core/v2` — following the example as written throws a module resolution error for v2 users. The preceding "Dynamically Switching Threads" section on the same page already documents the correct, working pattern (plain React state + the `threadId` prop on `<CopilotKit>`), so removing the broken section doesn't leave a gap. ## Related PRs and Issues Closes #3860 ## Files changed - `showcase/shell-docs/src/content/docs/integrations/langgraph/advanced/persistence/loading-message-history.mdx` - `showcase/shell-docs/src/content/docs/integrations/crewai-flows/persistence/loading-message-history.mdx` ## Checklist - [x] I have read the [Contribution Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md) - [x] Docs-only change; no functionality updated - [x] Allow edits by maintainers |
||
|
Ben Taylor
|
e91ae56372 |
docs(shell-docs): emit canonical versioned URLs for reference pages in llms-full.txt (#5486)
## What does this PR do? `getAllLlmPages()` in `llm-text.ts` previously walked `src/content/reference/` directly and emitted reference pages at `reference/<slug>` (e.g. `reference/hooks/useCopilotAction`). The live site serves those pages at their versioned canonical URL — `/reference/v2/hooks/useCopilotAction` for the current v2 API — so `llms-full.txt` contained non-canonical source URLs that diverged from what users see in the browser. **Root cause:** The v2 API reference lives at the _root_ of `src/content/reference/` (no `v2/` subfolder), so a bare filesystem walk cannot distinguish v2 from older SDK versions. It emits `reference/hooks/foo` instead of the correct `reference/v2/hooks/foo`. **Fix:** Replace step 3 with an enumeration via `loadReferenceVersionItems` (which already knows the canonical URL per version) and `resolveReferencePage` (which resolves the content file path). This matches the URL scheme used by the `/reference/[...slug]` route handler. **Result:** - v2 hooks/components now appear at `reference/v2/hooks/...` in `llms-full.txt` - v1, react-native, core, and bot pages appear at their correct versioned prefixes - Version root index pages (`reference/v2`, `reference/v1`, ...) are included - The migration guide (`migrate/v2`) was already included via the docs walk (step 1 unchanged) ## Related PRs and Issues - Closes #3385 ## Checklist - I have read the Contribution Guide - If the PR changes or adds functionality, I have updated the relevant documentation - "Allow edits by maintainers" is checked |
||
|
Ben Taylor
|
5595cf76c6 |
fix(react-core): expose isReady from useAgent to guard agent subscriptions (#5000) (#6041)
## Summary Closes #5000. `useAgent` (v2) always returns a **fully-constructed** `AbstractAgent` — a *provisional* stand-in while the runtime is still connecting (or in an error state), swapped for the real agent once the `/info` sync resolves. The return type claimed `agent` was always the real agent, so consumers had **no way to tell the provisional instance from the real one**. One-time subscriptions registered during the provisional window (e.g. `onRunFinalized`) landed on the placeholder and missed events until the effect re-ran after the swap. This PR adds an **`isReady`** flag to the return value: - `false` — `agent` is provisional (runtime connecting / error) - `true` — `agent` is the real, runtime-synced (or locally-registered) instance This is exactly the API the issue requests in its *Expected Behavior*. It is **additive and backward compatible** — existing `const { agent } = useAgent()` callers are unaffected. ```tsx const { agent, isReady } = useAgent({ agentId }); useEffect(() => { if (!isReady) return; // only subscribe once the real agent is bound const sub = agent.subscribe({ onRunFinalized: (p) => console.log(p) }); return () => sub.unsubscribe(); }, [agent, isReady]); ``` ## On the original crash The crash reported in #5000 — `Cannot read properties of undefined (reading 'subscribers')` at `AbstractAgent.subscribe` — **no longer reproduces on `main`**. The provisional-agent work landed for #5533 / #5635 now guarantees `useAgent` always returns a fully-constructed `AbstractAgent`, so `subscribe()` is always safe to call. The added tests lock in that no-crash behavior. What remained unaddressed was the missing readiness signal, which this PR provides. ## Changes - **`packages/react-core/src/v2/hooks/use-agent.tsx`** — `useMemo` now returns `{ agent, isReady }`; real agent → `isReady: true`, provisional paths → `isReady: false`. Documented with JSDoc. - **`use-agent-subscribe-ready.test.tsx`** (new) — regression + behavior coverage: `subscribe()` does not throw while connecting (effect + during-render), `isReady` transitions `false → true` on sync and swaps the instance, local agent is ready immediately. - **`showcase/shell-docs/.../hooks/useAgent.mdx`** — signature + return-value docs updated; the *Event Subscription* example fixed (it used an empty `useEffect` dep array and never re-subscribed when the agent reference changed). ## Testing - New test file: 4/4 pass. - Full `react-core` v2 hooks suite: **35 files / 299 tests pass** (the `useMemo` return-shape change breaks nothing). - `tsc --noEmit` clean. ## Notes - Scope is React only, matching the issue. The Vue `useAgent` (`packages/vue`) is structured differently (reactive `shallowRef`, `agent` can be `null`); happy to add matching `isReady` as a follow-up if maintainers want cross-framework parity. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jordan Ritter
|
6cb3deb841 |
fix(showcase): aimock-wiring probe now covers the harness fleet (harness/harness-workers) (#6062)
## The incident this prevents
The showcase pays egress whenever a service reaches aimock over the
PUBLIC `*.up.railway.app` host instead of the free
`showcase-aimock.railway.internal:4010`. On STAGING, `harness-workers`
(the 6-replica probe fleet) had `OPENAI_BASE_URL` / `ANTHROPIC_BASE_URL`
/ `AIMOCK_URL` pointing at PUBLIC aimock, and `harness` had a public
`AIMOCK_URL` — together ~$657/mo of egress. The aimock-wiring drift
probe **never flagged this** because both services were in
`EXCLUDE_SERVICES`. The live vars are already fixed; this makes the
probe cover the class so it can't silently regress.
## Design choice + justification
Two facts constrained the fix:
1. `harness` / `harness-workers` were excluded (`EXCLUDE_SERVICES`) →
skipped entirely, so a naive fix must un-exclude them.
2. `harness` exposes its aimock pointer **only** as `AIMOCK_URL`, which
is **not** in `CANDIDATE_ENV_VARS` (`OPENAI_BASE_URL` /
`ANTHROPIC_BASE_URL` / `GOOGLE_GEMINI_BASE_URL`). So merely un-excluding
`harness` would leave it all-missing → unwired forever, even when
correctly wired.
Chosen approach — a dedicated **aimock-consumer** class:
- Remove `harness` / `harness-workers` from `EXCLUDE_SERVICES`.
- Add `AIMOCK_CONSUMER_SERVICES = { harness, harness-workers }` +
`isAimockConsumer(name)` (mirrors `isExcluded`: matches bare and legacy
`showcase-`-prefixed forms).
- Add `HARNESS_FLEET_CANDIDATE_ENV_VARS = [...CANDIDATE_ENV_VARS,
"AIMOCK_URL"]`; `pointsAtAimock` takes a `candidateVars` param (defaults
to the standard set). In the run loop, consumers use the extended set,
everything else the standard set.
This is the minimal correct surface: it catches `harness` via
`AIMOCK_URL`, catches `harness-workers` via any of
OPENAI/ANTHROPIC/AIMOCK_URL, and leaves the verdict precedence (match >
confirmed-mismatch > sealed > missing) untouched.
### Why `AIMOCK_URL` is scoped to the harness-fleet path (and safe)
Adding `AIMOCK_URL` to the **global** candidate set is not safe: a
regular demo backend that happens to expose `AIMOCK_URL` (pointed
anywhere) could then count as "wired" and **mask a missing real
`OPENAI_BASE_URL`/etc pointer**, hiding genuine drift. Scoping
`AIMOCK_URL` to `HARNESS_FLEET_CANDIDATE_ENV_VARS` means only the two
harness-fleet services consult it. A regression guard test (`does NOT
consult AIMOCK_URL for non-harness services`) locks this in. Pure-infra
services with no aimock pointer
(aimock/shell/dashboard/docs/dojo/pocketbase/webhooks) stay excluded and
never go red.
## Red → Green proof
Tests added in `aimock-wiring.test.ts`. RED was captured against the
**unchanged** probe (new tests only, source untouched); GREEN after the
fix + updating the 4 existing tests that asserted the old
harness-excluded behavior.
**RED** (new behavior tests fail on current code — harness fleet
excluded, so the incident is not flagged):
```
FAIL aimock-wiring.test.ts > flags the harness fleet when its aimock pointers are on the PUBLIC host (egress drift)
AssertionError: expected 'green' to be 'red'
FAIL aimock-wiring.test.ts > greens the harness fleet when its aimock pointers are on the PRIVATE internal host
AssertionError: expected [] to deeply equal [ 'harness', 'harness-workers' ]
FAIL aimock-wiring.test.ts > verifies `harness` via its only aimock pointer, AIMOCK_URL
AssertionError: expected 'green' to be 'red'
Test Files 1 failed (1)
Tests 3 failed | 44 passed (47)
```
**GREEN** (after the fix):
```
Test Files 1 passed (1)
Tests 47 passed (47)
```
Test coverage added:
- `flags the harness fleet when its aimock pointers are on the PUBLIC
host (egress drift)` — the exact incident → red.
- `greens the harness fleet when its aimock pointers are on the PRIVATE
internal host` — positive path → wired/green.
- `verifies harness via its only aimock pointer, AIMOCK_URL` —
public→red, internal→green (locks the `AIMOCK_URL`-candidate path).
- `does NOT flag pure-infra services with no aimock pointer` — guard:
shell/dashboard/etc stay excluded.
- `does NOT consult AIMOCK_URL for non-harness services` — guard:
`AIMOCK_URL` is not global.
## Local quality
- oxfmt (formatter) — clean on both files
- oxlint — 0 warnings, 0 errors
- `tsc --noEmit` (typecheck) — clean
- `tsc -p tsconfig.build.json` (build) — clean
- Full harness suite: only the pre-existing unrelated failures remain
(`d0-gone-predicate.test.ts`, `d5-mapping-drift.test.ts`, and
`cvdiag/staged-ts-scrub-parity.test.ts`) — all confirmed failing
identically on the untouched baseline (verified via `git stash`). This
PR adds **no** new failures.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01QCLub2Vb5Y56cPttSzkip1
|
||
|
Jordan Ritter
|
105a5a9d65 |
fix(showcase): aimock-wiring probe now covers the harness fleet (harness/harness-workers)
The aimock-wiring drift probe excluded `harness` and `harness-workers`, so when their aimock pointers drifted to the billed PUBLIC `*.up.railway.app` host instead of `showcase-aimock.railway.internal:4010` the probe never flagged it (~$657/mo egress on staging). Un-exclude the harness fleet and verify it as aimock consumers via a new AIMOCK_CONSUMER_SERVICES set. Consumers use HARNESS_FLEET_CANDIDATE_ENV_VARS (the standard OPENAI/ANTHROPIC/GEMINI candidates plus AIMOCK_URL), because `harness` exposes ONLY AIMOCK_URL as its aimock pointer. AIMOCK_URL is scoped to the harness-fleet path only, so a regular backend's stray AIMOCK_URL can't mask a missing real base-URL pointer. Pure-infra services (aimock/shell/ dashboard/docs/dojo/pocketbase/webhooks) stay excluded. |
||
|
Ben Taylor
|
0035a7e387 |
docs(mastra): clarify that interrupts are not supported, redirect to tool-based HITL (#5895)
## Summary Fixes FAC-64: Mastra Interrupts docs example fails on missing agentId and suspendPayload guard This PR rewrites the Mastra interrupt documentation to correctly reflect that **Mastra does not support native interrupt flow**. The framework lacks LangGraph-style `interrupt()` primitives and does not emit AG-UI interrupt events. ## Changes ### 📝 Documentation Updates 1. **`interrupt-flow.mdx`**: Completely rewritten to: - Add prominent warning callout that Mastra doesn't support interrupts - Explain why the interrupt pattern doesn't work with Mastra - Provide working alternative using `useHumanInTheLoop` - Include comparison table between interrupt-based and tool-based approaches - Redirect users to the tool-based HITL guide 2. **`index.mdx`**: Updated to: - Mark interrupt-based approach as "Not Supported" - Mark tool-based approach as "Supported" (the working pattern) - Reorder cards to prioritize the working approach ## Rationale The original docs documented `useInterrupt` with examples that would: - Fail with "Agent 'default' not found" (missing `agentId` parameter) - Fail with "Cannot destructure property 'action'" (incorrect payload access) - Silently fail (hook listens for events Mastra never emits) Research revealed: - `showcase/integrations/mastra/manifest.yaml` explicitly lists `gen-ui-interrupt` under `not_supported_features` - The actual working demo uses `useHumanInTheLoop`, not `useInterrupt` - Comments in the code confirm: "This framework has no LangGraph-style `interrupt()` primitive" ## Migration Path Users following the old docs can now: 1. See clear warning that interrupts aren't supported 2. Learn the correct `useHumanInTheLoop` pattern 3. Follow link to complete tool-based HITL guide with working examples ## Testing - ✅ Documentation changes only (no runtime code affected) - ✅ Verified redirect links work correctly - ✅ Checked against actual working implementation in `showcase/integrations/mastra/src/app/demos/gen-ui-interrupt/page.tsx` ## Related - Linear: FAC-64 - QA Report: Documented three specific runtime errors from the broken examples - Research: Identified Option B (rewrite for useHumanInTheLoop) as the correct approach |
||
|
Jordan Ritter
|
88138aefe0 | docs(showcase): clarify aimock-wiring unwired bucket and extractHostPort null-return comments | ||
|
Jordan Ritter
|
a9c11f4627 |
test(showcase): lock aimock-wiring host+port precedence, refresh stale comments
Non-behavioral cleanup + coverage pass for the aimock-wiring probe: - Correct comments that still described the old hostname-only matching or referenced removed helpers (normalizeUrl / extractHostname) to reflect the current host+port matching via extractHostPort, in both the probe and its test. - Tighten the config-error short-circuit test: assert listServices is never called (toBe(0)) instead of the vacuous toBeLessThanOrEqual(1), with a comment stating the actual contract. - Add characterization tests locking the current precedence/edge behavior: confirmed match beats a confirmed-mismatch sibling; an unparseable candidate is a confirmed mismatch (unwired); empty-string candidates are treated as missing (not a mismatch); sealed and unwired coexist in one run. No runtime behavior changes. |
||
|
Jordan Ritter
|
b2602cac04 |
fix(showcase): port-aware + confirmed-mismatch-beats-sealed in aimock-wiring probe
Coupled correctness fixes so real starter/backend drift can't hide now that the probe covers all 32 services (20 showcase-* + 12 starter-*): - c1: a CONFIRMED mismatch (a candidate set to a non-aimock host) now wins over a sealed sibling. Previously a var pointing at real api.openai.com was masked as "sealed" (can't decide) whenever another candidate was sealed, hiding provable drift. Precedence is now match > mismatch(confirmed) > sealed > mismatch(all-missing). - c2: matching compares host AND effective port, not host alone. Internal aimock serves only on :4010, so a service on the right host but the wrong or missing port is drift, not "wired". Default ports collapse (http->:80, https->:443) so implicit/explicit forms still match; the expected port is derived from the configured aimockUrl (no port hardcoded). - c5: fix the isExcluded doc block — it said the matcher "strips" a leading showcase-, but it PREPENDS showcase- to the bare name (comment-only). Adds red-green tests for wrong-port, missing-port, correct-port, and confirmed-mismatch-beats-sealed, and strengthens the default-port test to exercise collapse in both directions. |
||
|
Jordan Ritter
|
167ae3ad1b |
fix(showcase): aimock-wiring probe now covers starter-* services
The isExcluded() helper short-circuited every starter-* service out of the probe on the (stale) rationale that starters are not wired through aimock. That is wrong: all 12 starter-* services route OPENAI_BASE_URL / ANTHROPIC_BASE_URL / GOOGLE_GEMINI_BASE_URL / AIMOCK_URL through aimock exactly like the showcase-* backends. Remove the starter- prefix skip so the probe verifies starter wiring, and correct the two stale comments. Infra services in EXCLUDE_SERVICES are untouched. |
||
|
Jordan Ritter
|
090c080581 |
fix(showcase): langroid declarative gen-ui → two-stage A2UI north-star (D6 turn-1 surface-missing) (#6058)
## What Brings the **langroid** `declarative-gen-ui` D6 cell to sibling parity with the two-stage dynamic-schema A2UI north-star (google-adk / strands). The cell was red at turn 1 with `reason=surface-missing`: the demo was still on the pre-D6 (D5-era) shape, and four independent defects each blocked the A2UI surface from painting. Second-wave fan-out of the proven pattern (pilots #6051/#6052/#6053, first wave #6054 agno / #6055 claude-sdk-typescript). ## Root cause (four defects, each verified against a live isolated stack) 1. **Stale suggestions + fixtures.** `suggestions.ts` still offered the old D5 pills; the aimock fixture only mocked those. The D6 driver sends the four current business-question prompts. Re-authored both to the four current prompts mirroring the google-adk north-star (outer `generate_a2ui` no-arg → inner forced `render_a2ui` → outer narration). 2. **Required `context` on the outer tool.** `GenerateA2UITool.context` was a required pydantic field, so the mocked outer `arguments: {}` raised `ValidationError` before the tool ran → no inner call, no surface. Made optional (default `""`) to match the no-arg sibling tools. 3. **Legacy functions API hid the inner tool from aimock's matcher.** The inner planner used langroid's `functions=`/`function_call=` (legacy OpenAI) path; aimock's `toolName` matcher only inspects the modern `tools[]` array, so the inner `render_a2ui` fixture never matched and the call fell through to the outer `generate_a2ui` fixture (empty surface, wrong catalogId). Switched the inner call to the modern `tools=`/`tool_choice=` API. The response extractor already reads the modern `oai_tool_calls` path first, so nothing downstream changes. 4. **Inner call could not be discriminated per pill.** langroid has no framework middleware to forward the run's conversation into the inner call (unlike `ag_ui_adk` / `ag_ui_strands`), so its inner user message was a fixed generic string across all four pills. Added an explicit last-user-turn thread (a `ContextVar` set by the adapter, consumed by the planner) so the pill prompt rides as the inner `userMessage` — the discriminator the sibling fixtures rely on. **Renderer/catalog parity:** added the missing `declarative-info-row` testid on InfoRow (turn 4) and a full `DataTable` definition + renderer (`declarative-data-table`, turn 2), plus `trendValue` on Metric. Added `sales-context.ts` (byte-identical dataset + composition rules to the strands/google-adk siblings) and wired it via `chat.tsx`. Backend family mirrored: **google-adk / strands** — outer `generate_a2ui` (no args) + inner forced `render_a2ui`, `declarative-gen-ui-catalog`. ## Red-green proof (isolated control-plane, `--isolate --rebuild`, slot 16) **RED (pre-fix, 3 runs):** ``` ✗ d6:langroid/gen-ui-declarative red — state=red waitForTurnComplete: turn 1 did not complete within 90000ms (reason=surface-missing) ``` First diagnosis (direct backend SSE): outer `generate_a2ui` → `{"error": "Tool generate_a2ui failed: ValidationError"}`. After fixing that: outer succeeded but emitted `catalogId: copilotkit://app-dashboard-catalog` with `components: []` (inner never matched — legacy functions API). After the tools-API + threading fix: **GREEN (post-fix, 2 runs):** ``` ✓ d6:langroid/gen-ui-declarative green 1 passed ``` aimock journal confirms all four pills' outer `generate_a2ui` + inner `render_a2ui` (tool_choice forced) calls return **200** and emit `declarative-gen-ui-catalog` surfaces. Per-turn backend SSE verified: - turn 1 (sales-dashboard): 4 × Metric + PieChart + BarChart - turn 2 (team-performance): DataTable + BarChart - turn 4 (top-account): 7 × InfoRow + PieChart ## Visual `langroid-turn1-sales-dashboard.png` — live Playwright render (X-AIMock-Context: langroid): 4 metric tiles + revenue-by-region pie + monthly-revenue bar. Turns 2–4 surface renders are asserted and pass in the authoritative D6 run (conjunctive per-turn testid checks) and confirmed via backend SSE above. ## Notes - Draft: not for merge/promote (user-gated). - A concurrent `ms-agent-harness-dotnet` fan-out agent was hitting the shared local aimock during manual browser capture (interleaved 404s in the journal); it does not affect the isolated D6 result, which is the binding proof. --- **CI-driven follow-ups (in this same commit):** - Updated `integrations/langroid/tests/python/test_generate_a2ui.py` to assert the modern `tools=`/`tool_choice=` kwargs (was pinning the legacy `functions=`/`function_call=` API this fix intentionally replaced). 78 passed / 1 skipped locally. - Scoped the four inner `render_a2ui` fixtures to `context: langroid` (langroid forwards `x-aimock-context` to the inner planner call, unlike `ag_ui_adk`/`ag_ui_strands`) so they don't collide in the shared scope with the sibling integrations' identical inner keys — keeps the `aimock-fixtures` exact-duplicate ceiling at 297 (no bump). Full `aimock-fixtures.test.ts` suite: 837 passed. D6 re-run after this change: still green 4/4. |
||
|
Jordan Ritter
|
04aefd9dcd |
fix(showcase): repair ms-agent-dotnet D6 gen-ui-declarative (surface-missing) (#6057)
## Summary
The D6 e2e-full probe `d6:ms-agent-dotnet/gen-ui-declarative` was
failing at turn 1 with `reason=surface-missing`. Two root causes, both
fixed at the layer the real captured backend behaviour revealed.
### Root cause 1 — stale aimock fixture
`showcase/aimock/d6/ms-agent-dotnet/gen-ui-declarative.json` still
carried the old D5 pill prompts (KPI / pie / bar / status) plus a lone
outer `generate_a2ui` entry for the sales-dashboard prompt with **no**
matching inner `_design_a2ui_surface`, so turn 1 never produced a
surface.
Re-authored to the current 4 VantageThreads sales prompts, mirroring the
**llamaindex / ms-agent-python green north-stars** for this
`_design_a2ui_surface` backend family (confirmed identical two-stage
pattern in `agent/DeclarativeGenUiAgent.cs` +
`agent/A2uiSecondaryToolCaller.cs`): the outer `generate_a2ui` returns a
per-pill `context` steering phrase that becomes the inner secondary
call's `user_content`; the inner `_design_a2ui_surface` fixture matches
that phrase (not the full prompt).
**ms-agent-dotnet–specific discriminator.** Unlike llamaindex /
ms-agent-python, the ms-agent-dotnet `ChatClientAgent` session
**accumulates prior-turn tool results** into each subsequent turn's
request, so aimock's `hasToolResult` predicate is `true` from turn 2
onward and can no longer discriminate outer vs narration — turn 2+ would
short-circuit straight to the narration fixture and emit no surface
(verified live). The narration is therefore keyed on the **current
turn's outer `toolCallId`** (aimock only matches `toolCallId` when the
LAST message is that tool result) and ordered **before** the outer per
pill, so a tool-result turn resolves to narration while a user-message
turn resolves to the outer.
### Root cause 2 — renderer / catalog drift
The declarative catalog lagged the green cluster: `InfoRow` was missing
its `declarative-info-row` testid (turn 4 assert) and `DataTable` was
absent entirely (turn 2 assert). Added the testid and the `DataTable`
renderer + definition, matching the green cluster.
## Red → Green proof (real control-plane surface)
`./bin/showcase test ms-agent-dotnet:declarative-gen-ui --d6 --isolate
--rebuild`
| | result |
|---|---|
| **RED** (pristine stale fixture + missing DataTable/info-row testid) |
`d6:ms-agent-dotnet/gen-ui-declarative = red` — exit 1, turn 1
`surface-missing`, `0 passed, 1 failed` |
| **GREEN** (fix applied) | `d6:ms-agent-dotnet/gen-ui-declarative =
green` — exit 0, `1 passed` |
An intermediate rebuild flipped turn 1 green but exposed the turn-2
`surface-missing` (accumulated-history bug); the `toolCallId` narration
re-keying fixed all four turns.
## Visual verification
Drove all 4 turns via Playwright (header-injected `X-AIMock-Context:
ms-agent-dotnet` + `X-AIMock-Strict: true` to replicate the harness
proxy). Confirmed real painted surfaces:
- **turn 1 sales-dashboard**: 4 KPI metrics ($4.2M revenue, 186
customers, 31% win rate, $22.6k deal) + Revenue-by-Region pie +
Monthly-Revenue bar
- **turn 2 team-performance**: rep-quota `DataTable` (Dana Whitfield
124% … Elena Vasquez 71%) + attainment `BarChart`
- **turn 3 at-risk**: 3 `StatusBadge` severity cards + KPI metric strip
- **turn 4 top-account**: 7 `InfoRow` account facts (Meridian Apparel
Group) + product-line `PieChart`
Screenshots under
`~/.local/share/copilotkit/cr/2ndwave-shots/msdotnet-turn{1..4}-full.png`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Jordan Ritter
|
92d0fab67e |
fix(showcase): repair ms-agent-harness-dotnet D6 gen-ui-declarative (surface-missing) (#6056)
## Summary The D6 e2e-full probe `d6:ms-agent-harness-dotnet/gen-ui-declarative` was failing at turn 1 with `reason=surface-missing`. Three layers, each fixed at the layer the real captured request revealed. Part of the second-wave declarative fan-out (siblings #6051–#6055 / #6053 ms-agent-python). ### Root cause 1 — stale aimock fixture `showcase/aimock/d6/ms-agent-harness-dotnet/gen-ui-declarative.json` still carried the old D5 pill set (KPI / pie / bar / status prompts) with inner `_design_a2ui_surface` entries keyed on those stale prompts. The current driver sends four VantageThreads sales prompts, so turn 1 ("Show me my sales dashboard for this quarter.") had **no matching inner surface** — the backend looped `generate_a2ui` to its invocation limit and the frontend painted the stale KPI catch-all instead of the sales dashboard. Re-authored to the four current prompts, mirroring the **llamaindex green north-star** for this `_design_a2ui_surface` two-stage backend family (ms-agent-harness-dotnet's inner tool is `_design_a2ui_surface`, per `agent/DeclarativeGenUiAgent.cs` + `agent/A2uiSecondaryToolCaller.cs`). ### Root cause 2 — `hasToolResult` breaks the interleaved thread This is where the dotnet family diverges from the #6053 ms-agent-python template. ms-agent-python starts a fresh session per turn; the **.NET harness backend threads the FULL interleaved conversation**. `hasToolResult` is a thread-global predicate (see `showcase/GOTCHAS.md`), so once turn 1 leaves a `role:"tool"` message in the thread, every later pill's outer `generate_a2ui` call sees `hasToolResult:true` and matches the **narration** fixture instead of emitting the tool call → surface-missing on turns 2–4. (Reproduced live: turn 2's outer call, replayed with turn-1 history, returned the narration string instead of `generate_a2ui`.) Switched to the sanctioned interleaved-safe pattern: **narration keyed on this pill's outer `toolCallId`** (ordered before the outer), **outer keyed on `userMessage` only**. Verified all four pills resolve correctly through the full interleaved thread against live aimock. ### Root cause 3 — renderer / catalog drift `renderers.tsx` and `definitions.ts` lagged the green cluster — missing the `DataTable` and `InfoRow` components (the `declarative-data-table` / `declarative-info-row` testids that turns 2 and 4 assert), plus `Metric.trendValue` and the Row/Column/Text gap overrides. Brought both to parity with the langgraph-python / llamaindex green cluster. ## Red → Green proof (real control-plane surface) `SHOWCASE_ISO_SLOT=18 ./bin/showcase test ms-agent-harness-dotnet:declarative-gen-ui --d6 --isolate --rebuild` | | result | |---|---| | **RED** (pristine stale fixture + drifted renderers) | `state=red`, exit 1 — turn 1 `waitForTurnComplete: turn 1 did not complete within 90000ms (reason=surface-missing, runsFinished=1, count=41)`; body showed the stale "Quarterly KPIs / $1.24M / SIGNUPS 8,420" surface + `generate_a2ui` looping | | **GREEN** (fix applied) | `state=green`, exit 0 — `1 passed` | ## Visual verification Drove all 4 turns via Playwright with network-level route injection of `x-aimock-context: ms-agent-harness-dotnet` (replicating the harness/production proxy). Confirmed real painted surfaces: - **turn 1 sales-dashboard**: 4 KPI metrics ($4.2M revenue, 186 customers, 31% win rate, $22.6k deal) + Revenue-by-Region donut + Monthly-Revenue bar - **turn 2 rep-quota**: rep-attainment `DataTable` (Dana Whitfield 124% … Elena Vasquez 71%) + quota-attainment `BarChart` - **turn 3 at-risk**: 3 severity `StatusBadge` cards (Northwind / Cascadia / Atlas) + KPI metric strip - **turn 4 top-account**: 7 `InfoRow` account facts (Meridian Apparel Group) + product-line `PieChart` Screenshots under `~/.local/share/copilotkit/cr/2ndwave-shots/msharness-turn{1..4}-*.png`. |
||
|
Jordan Ritter
|
c13415f743 |
fix(showcase): repair claude-sdk-typescript D6 gen-ui-declarative (surface-missing) (#6055)
## What Repairs the `claude-sdk-typescript` D6 `gen-ui-declarative` cell, which was **red on turn-1 surface-missing**. The aimock fixture carried stale D5-era prompts and only a partial turn-1 outer entry, so the two-stage Anthropic A2UI flow (outer `generate_a2ui` → secondary `render_a2ui` → narration) never painted a surface for the four current sales-analyst pills. Fan-out of the proven pattern from the pilots (#6051 claude-sdk-python, #6052 mastra, #6053 ms-agent-python). This is the Anthropic two-stage family — `injectA2UITool: false`, backend owns `generate_a2ui` + secondary `render_a2ui`. ## Root cause `aimock/d6/claude-sdk-typescript/gen-ui-declarative.json` had the old D5 prompts (`Show me a quick KPI dashboard` / `pie chart of sales by region` / `bar chart of quarterly revenue` / `status report on system health`) plus one lone turn-1 outer entry for the new sales prompt. The driver's four current prompts had no complete triads, so aimock returned `STRICT: No fixture matched` and no surface mounted. ## Fix (3 layers) 1. **Fixture re-author** — 12 fixtures (4 pills × {outer `generate_a2ui`, inner `render_a2ui`, narration}) in the two-stage shape, mirroring the #6051 sibling + google-adk data. Render payloads mount the per-pill catalog components the driver asserts: Metric×4 + Pie + Bar (sales-dashboard); DataTable + Bar (team-performance); Metric×3 + StatusBadge×3 (at-risk); InfoRow + Pie (top-account). Render payloads are byte-identical to #6051. **Ordering/matcher fix vs a naive python mirror:** the CSTS runtime accumulates full conversation history across pills, so on turns 2-4 the outer `generate_a2ui` call carries prior pills' tool results and a `hasToolResult:false` matcher never fires. Each pill triad is ordered **narration (`toolCallId`) first** so it claims the last-role:tool calls, and the outer matcher drops `hasToolResult` and gates on `userMessage` + `toolName generate_a2ui` (last-role:user). Verified live via the aimock journal. 2. **InfoRow testid** — add `data-testid="declarative-info-row"` to the InfoRow renderer (turn-4 top-account parity; CSTS was missed by #6050). 3. **Suggestions refresh** — `suggestions.ts` had stale D5-era pill labels that emitted unmatched prompts (live 404 banner). Now the four sales-analyst pills, matching google-adk. ## Red → green proof (control-plane, slot 30, `--isolate --rebuild`) **RED (origin/main):** ``` ✗ d6:claude-sdk-typescript/gen-ui-declarative red — state=red [aimock] STRICT: No fixture matched for POST /v1/messages (×6) ``` **GREEN (fixed):** ``` ✓ d6:claude-sdk-typescript/gen-ui-declarative green — 1 passed ``` aimock journal after the green run: **all 12 calls returned 200, zero 503, zero no-match** across all 4 turns. ## Visual proof (Playwright, `x-aimock-context: claude-sdk-typescript`) Drove all 4 pills live; per-turn DOM testid counts (no fixture error on any turn): | Turn | Pill | Newly-mounted testids | |---|---|---| | 1 | sales-dashboard | metric=4, pie=1, bar=1 | | 2 | team-performance | data-table=1, bar +1 (→2) | | 3 | at-risk | status-badge=3, metric +3 (→7) | | 4 | top-account | **info-row=7**, pie +1 (→2) | Screenshots: `~/.local/share/copilotkit/cr/2ndwave-shots/csts-turn{1..4}-*.png`. ## Unit tests - `scripts/__tests__/aimock-fixtures.test.ts`: 837 passed (validates the new fixture shape). - `harness/src/probes/scripts/d5-gen-ui-declarative.test.ts`: 31 passed. ## Notes - Worktree tsc module-not-found / TS2322-ButtonProps noise is benign symlink noise; trust the PR's real `check-types` CI check. - Local red-green ran on slot 30 (slot 8 and several low slots were held by concurrent isolate stacks). |
||
|
Jordan Ritter
|
29ed44d913 |
fix(showcase): flip agno gen-ui-declarative D6 cell green (4-turn sales flow + DataTable/InfoRow parity) (#6054)
## What Flips the `d6:agno/gen-ui-declarative` cell from **red (turn-1 dom-missing) → green**. Fan-out of the proven 2nd-wave declarative fix pattern (pilots #6051 claude-sdk-python, #6052 mastra, #6053 ms-agent-python). ## Root cause (verified at the request level) agno's declarative-gen-ui shipped a **stale D5 aimock fixture** keyed on the old prompts (KPI dashboard / pie chart of sales by region / bar chart / status report), while the current D6 driver sends the OSS-136 sales prompts. Captured from the aimock journal on a RED run: agno's OUTER agent hit aimock with `tools=[generate_a2ui]`, `userMessage="Show me my sales dashboard for this quarter."`, `x-aimock-strict:true`, `context=agno` — the stale fixture matched **none**, aimock returned 503 (strict), the outer agent never emitted `generate_a2ui`, no surface rendered → turn-1 dom-missing → `state=red`. ## Backend family + north-star agno uses the plain **`render_a2ui` two-stage** family (`src/agents/a2ui_dynamic_agent.py`): an OUTER `generate_a2ui(context: str)` tool, then a forced-`render_a2ui` secondary call, then narration. Mirrored the **google-adk** green north-star (same VantageThreads surfaces + `declarative-gen-ui-catalog`; `render_a2ui` args copied verbatim). Key agno-specific wrinkle: the inner secondary call's **user message is hardcoded and identical across all four pills** ("Generate a dynamic A2UI dashboard based on the conversation."), so the inner `render_a2ui` fixtures cannot key on `userMessage`. They discriminate on `toolName:render_a2ui` + `context:agno` + a `systemMessage` substring equal to the per-pill context phrase the outer injects ("Conversation context:\n<context>"). aimock's CLI server uses substring matching, so this works; verified live against the journal. ## Fix layers - **Fixture** (`aimock/d6/agno/gen-ui-declarative.json`): 4 sales prompts × 3 calls (outer/inner/narration) = 12 fixtures. - **Renderers** (`.../a2ui/renderers.tsx`): `declarative-info-row` testid on InfoRow (turn 4) + new `DataTable` renderer with `declarative-data-table` testid (turn 2), mirroring google-adk. - **Definitions** (`.../a2ui/definitions.ts`): `DataTable` schema, `Metric.trendValue`, `z.unknown()` PrimaryButton action, refreshed descriptions. - **Backend** (`a2ui_dynamic_agent.py`): sales-analyst system prompt for live-mode steering. - **Test** (`scripts/__tests__/aimock-fixtures.test.ts`): `KNOWN_DUPLICATE_CEILING` 297→300 (+3) — the 4 inner render fixtures collapse to one `toolName=render_a2ui` matchKey (matchKey omits systemMessage/context) but aimock's router disambiguates them at runtime. ### Bug caught during green First green attempt still red with a client-side exception: adding `Row`/`Column`/`Text` to `myDefinitions` **without matching renderers** (agno relies on `includeBasicCatalog:true` for those) made `createCatalog` produce a definition set wider than its renderer set → render crash. Fixed by not declaring Row/Column/Text in definitions. ## Local RED → GREEN proof (isolated D6 slots) **RED** (control-plane, stale fixture): ``` ✗ d6:agno/gen-ui-declarative red — state=red ``` **GREEN** (control-plane, after fix): ``` ✓ d6:agno/gen-ui-declarative green 1 passed ``` **GREEN** (`--direct`, per-turn DOM assertions — authoritative): ``` turn 1/4 — assertions passed (sales-dashboard: metric×4 + pie + bar) turn 2/4 — assertions passed (team-performance: data-table + bar) turn 3/4 — assertions passed (at-risk: status-badge×3 + metric×3) turn 4/4 — assertions passed (top-account: info-row + pie) state=green, 1 passed ``` GREEN aimock journal: 12 requests = 4× OUTER `generate_a2ui` + 4× INNER `render_a2ui` + 4× narration, all matched. **Live Playwright visual** (all 4 surfaces painted, testids counted): turn1 metric×4/pie/bar, turn2 data-table×1/bar, turn3 status-badge×3/metric, turn4 info-row×7/pie. `aimock-fixtures.test.ts`: 837 passed after the ceiling bump. |
||
|
Jordan Ritter
|
3d8eff5e70 |
fix(showcase): repair ms-agent-python D6 gen-ui-declarative (surface-missing) (#6053)
## Summary
The D6 e2e-full probe `d6:ms-agent-python/gen-ui-declarative` was
failing at turn 1 with `reason=surface-missing`. Two independent root
causes, both fixed at the layer the real captured request revealed.
### Root cause 1 — stale aimock fixture
`showcase/aimock/d6/ms-agent-python/gen-ui-declarative.json` still
carried the old D5 pill prompts (KPI / pie / bar / status) plus a lone
outer `generate_a2ui` entry for the current sales-dashboard prompt with
**no** matching inner `_design_a2ui_surface` and **no** narration. The
backend looped `generate_a2ui` to its invocation limit and
`RUN_FINISHED` was blocked while the tool call stayed active (`Cannot
send 'RUN_FINISHED' while tool calls are still active`).
Re-authored to the current 4 VantageThreads sales prompts, mirroring the
**llamaindex green north-star** for this backend shape
(ms-agent-python's inner tool is `_design_a2ui_surface`, not
google-adk's `render_a2ui`):
- outer `generate_a2ui` returns a `context` steering phrase — the
ms-agent-framework session does not surface the latest user message to
the secondary LLM, so the phrase becomes the inner call's
`user_content`;
- the inner `_design_a2ui_surface` fixture matches that phrase (not the
full prompt);
- `hasToolResult` discriminates outer (false) vs narration (true).
This also eliminates the stale `render-a2ui.json` "KPI dashboard"
catch-all collision that was rendering the wrong (KPI) surface for the
sales prompt.
### Root cause 2 — renderer / catalog drift
`renderers.tsx` and `definitions.ts` for ms-agent-python's
declarative-gen-ui lagged the green cluster — missing the `DataTable`
and `InfoRow` components (the `declarative-data-table` /
`declarative-info-row` testids that turns 2 and 4 assert), plus
`Metric.trendValue` and the `Row`/`Column`/`Text` gap overrides. Brought
both files to parity with the langgraph-python / google-adk green
cluster.
## Red → Green proof (real control-plane surface)
`SHOWCASE_ISO_SLOT=11 ./bin/showcase test
ms-agent-python:declarative-gen-ui --d6 --isolate`
| | result |
|---|---|
| **RED** (pristine fixture + stale renderers) |
`d6:ms-agent-python/gen-ui-declarative = red` — exit 1, turn 1
`surface-missing` |
| **GREEN** (fix applied) | `d6:ms-agent-python/gen-ui-declarative =
green` — exit 0, `1 passed` |
## Visual verification
Drove all 4 turns via Playwright (header-injected `x-aimock-context:
ms-agent-python` to replicate the harness/production proxy). Confirmed
real painted surfaces:
- **turn 1 sales-dashboard**: 4 KPI metrics ($4.2M revenue, 186
customers, 31% win rate, $22.6k deal) + Revenue-by-Region pie +
Monthly-Revenue bar
- **turn 2 team-performance**: rep-quota `DataTable` + attainment
`BarChart`
- **turn 3 at-risk**: 3 `StatusBadge` severity cards + KPI metric strip
- **turn 4 top-account**: 7 `InfoRow` account facts + product-line
`PieChart`
Screenshots captured under
`~/.local/share/copilotkit/cr/2ndwave-shots/mspy-turn{1..4}-*.png`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Jordan Ritter
|
c01bde3759 |
fix(showcase): flip mastra gen-ui-declarative D6 cell green (#6052)
## What
Flips the `d6:mastra/gen-ui-declarative` showcase cell from **red** to
**green**.
The cell failed turn 1 with `reason=surface-missing` (the dom-missing
family). Reading the real failure surface (aimock logs + mastra
container logs + harness worker logs) exposed **three** distinct
defects, each fixed at its own layer.
## Root cause (empirically confirmed)
1. **Stale aimock fixture.** `aimock/d6/mastra/gen-ui-declarative.json`
carried the OLD D5 pill prompts (`"Show me a quick KPI dashboard"`,
`"pie chart of sales by region"`, ...) and the stale inner tool name
`_design_a2ui_surface`. The current D6 driver sends 4 different prompts,
so aimock matched **0** fixtures:
```
[aimock] STRICT: No fixture matched for POST /v1/responses (x2 = the
two-stage flow's outer + inner calls)
```
Nothing emitted → no render → `surface-missing`.
2. **Mastra outer-tool arg schema (mastra-specific).** Unlike the green
`google-adk` peer whose outer `generate_a2ui` takes `{}`, mastra's
`generateA2uiTool` (`integrations/mastra/src/mastra/tools/index.ts`) has
an `inputSchema` that **requires a `messages` array**. After
re-authoring the fixture to the green shape (which emits `generate_a2ui`
with `{}`), the mastra runtime rejected it:
```
Tool input validation failed for generate_a2ui — messages: Required.
Provided arguments: {}
```
The outer tool never executed → no `a2ui_operations` container →
`surface-missing` (narration bubble rendered, but no A2UI surface).
3. **Renderer testid parity.** The D6 probe DOM-asserts
`declarative-data-table` (turn 2) and `declarative-info-row` (turn 4).
Mastra's `renderers.tsx` had **no DataTable renderer at all** and its
InfoRow renderer **lacked the `data-testid`**; `definitions.ts` had no
DataTable definition. Turns 2 and 4 could never satisfy their
assertions.
## Fix (3 files)
- **`aimock/d6/mastra/gen-ui-declarative.json`** — re-authored to the
current 4 driver prompts + the green two-stage shape (outer
`generate_a2ui` + inner forced `render_a2ui`, `context: "mastra"`,
`catalogId: "declarative-gen-ui-catalog"`, per-pill narration). Each
outer `generate_a2ui` call now carries a valid `messages` array (mastra
schema requirement).
- **`integrations/mastra/.../a2ui/definitions.ts`** — added the
`DataTable` catalog definition (mirrors the green `google-adk` peer).
- **`integrations/mastra/.../a2ui/renderers.tsx`** — added
`data-testid="declarative-info-row"` to the InfoRow renderer, and added
a `DataTable` renderer carrying `data-testid="declarative-data-table"`.
## Red → Green proof (real control-plane surface, slot 13, `--rebuild`)
Command: `SHOWCASE_ISO_SLOT=13 ./bin/showcase test
mastra:declarative-gen-ui --d6 --isolate --rebuild`
**RED (pristine main):**
```
✗ d6:mastra/gen-ui-declarative red — state=red
turn 1 did not complete within 90000ms (reason=surface-missing)
```
**GREEN (fixed):**
```
✓ d6:mastra/gen-ui-declarative green
1 passed
[conversation-runner] turn 1/4 — assertions passed (metric x4, pie, bar; baseline 0)
[conversation-runner] turn 2/4 — assertions passed (data-table NEW, bar)
[conversation-runner] turn 3/4 — assertions passed (status-badge x3, metric x3)
[conversation-runner] turn 4/4 — assertions passed (info-row NEW, pie)
[conversation-runner] conversation completed successfully { turnsCompleted: 4 }
```
## Visual evidence
Drove the live cell through all 4 turns via Playwright (route-level
`x-aimock-context` injection) and screenshotted each painted surface —
all real renders, no error states:
- **turn 1** sales-dashboard: 4 KPI metric tiles + donut PieChart
(Revenue by Region) + BarChart (Monthly Revenue)
- **turn 2** team-performance: DataTable (Rep attainment: Dana Whitfield
124%, ...) — the new renderer
- **turn 3** at-risk: 3 severity cards each with a StatusBadge + 3 KPI
metric tiles
- **turn 4** top-account: Card of InfoRow facts (Owner/Region/ARR/...) +
PieChart — the new testid
Note: `google-adk` remains the only D6 declarative fixture already on
the current prompts; the other integrations (`langgraph-typescript`,
etc.) still carry the same stale-fixture shape and are a follow-up wave.
|
||
|
Jordan Ritter
|
45cdc02ee0 |
fix(showcase): re-author langroid declarative gen-ui to the two-stage A2UI north-star (D6 turn-1 surface-missing)
The langroid `declarative-gen-ui` D6 cell was red at turn 1 with
`reason=surface-missing`: the demo was still on the pre-D6 (D5-era) shape
while the google-adk / strands siblings had moved to the two-stage
dynamic-schema A2UI pattern. Four independent defects each blocked the
surface from painting; all four are fixed here to bring langroid to
sibling parity.
Root causes (each verified against a live isolated stack):
1. Stale suggestions + fixtures. `suggestions.ts` still offered the old
D5 pills ("Show a KPI dashboard", "pie chart of sales by region", …)
and `aimock/d6/langroid/gen-ui-declarative.json` only mocked those old
prompts. The D6 driver sends the four current business-question pills
("Show me my sales dashboard for this quarter.", etc.). Re-authored both
to the four current prompts, mirroring the google-adk north-star
(outer `generate_a2ui` no-arg → inner forced `render_a2ui` → outer
narration, three fixtures per pill).
2. Required `context` on the outer tool. `GenerateA2UITool.context` was a
required pydantic field, so the mocked outer call's `arguments: {}`
raised `ValidationError` before the tool could run — no inner call, no
surface. Made it optional (default "") to match the no-arg sibling tools.
3. Legacy functions API hid the inner tool from aimock's matcher. The
inner planner call used langroid's `functions=`/`function_call=` (legacy
OpenAI) path; aimock's `toolName` matcher only inspects the modern
`tools[]` array, so the inner `render_a2ui` fixture never matched and the
call fell through to the outer `generate_a2ui` fixture (empty surface,
wrong catalogId). Switched the inner call to the modern
`tools=`/`tool_choice=` API via a new `_RENDER_A2UI_TOOL_SPEC`. The
response extractor already reads the modern `oai_tool_calls` path first.
4. Inner call could not be discriminated per pill. langroid has no
framework middleware to forward the run's conversation into the inner
call (unlike ag_ui_adk / ag_ui_strands), so its inner user message was a
fixed generic string across all four pills. Added an explicit
last-user-turn thread (ContextVar set by the adapter, consumed by the
planner) so the pill prompt rides as the inner `userMessage` — the same
discriminator the sibling fixtures rely on.
Renderer/catalog parity: added the missing `declarative-info-row` testid
to InfoRow (turn 4) and a full `DataTable` definition + renderer
(`declarative-data-table`, turn 2), plus `trendValue` on Metric. Added
`sales-context.ts` (byte-identical dataset + composition rules to the
strands/google-adk siblings) and wired it via `chat.tsx`.
Red-green (isolated control-plane, slot 16):
- RED (3 runs, pre-fix): `d6:langroid/gen-ui-declarative` red,
turn-1 surface-missing.
- GREEN (2 runs, post-fix, --rebuild): 4/4 turns pass. aimock journal
confirms all four pills' outer `generate_a2ui` + inner `render_a2ui`
calls return 200 and emit the correct `declarative-gen-ui-catalog`
surfaces. Backend SSE verified per turn: turn-1 4 metrics + pie + bar,
turn-2 DataTable, turn-4 7 InfoRows.
|
||
|
Jordan Ritter
|
bf75022c78 |
fix(showcase): repair ms-agent-dotnet D6 gen-ui-declarative (surface-missing)
The D6 e2e-full probe d6:ms-agent-dotnet/gen-ui-declarative failed at turn 1 with reason=surface-missing. Two root causes fixed at the layer the real captured backend behaviour revealed. Root cause 1 - stale aimock fixture. The fixture still carried the old D5 pill prompts (KPI/pie/bar/status) plus a lone outer generate_a2ui entry for the sales-dashboard prompt with no matching inner _design_a2ui_surface, so turn 1 never produced a surface. Re-authored to the current 4 VantageThreads sales prompts mirroring the llamaindex/ms-agent-python green north-stars for this _design_a2ui_surface backend family (outer generate_a2ui returns a context steering phrase; the inner _design_a2ui_surface fixture matches that phrase). Unlike llamaindex/ms-agent-python, the ms-agent-dotnet ChatClientAgent session ACCUMULATES prior-turn tool results into each subsequent turn's request, so hasToolResult is true from turn 2 onward and cannot discriminate outer vs narration (turn 2+ would short-circuit straight to narration, no surface). The narration is therefore keyed on the CURRENT turn's outer toolCallId (aimock only matches toolCallId when the LAST message is that tool result) and ordered before the outer per pill so the tool-result turn resolves to narration while the user-message turn resolves to the outer. Root cause 2 - renderer/catalog drift. The declarative catalog lagged the green cluster: InfoRow was missing its declarative-info-row testid (turn 4) and DataTable was absent entirely (turn 2). Added the testid and the DataTable renderer + definition, matching the green cluster. Red -> Green (real control-plane, --isolate --rebuild): RED: d6:ms-agent-dotnet/gen-ui-declarative = red (turn 1 surface-missing) GREEN: d6:ms-agent-dotnet/gen-ui-declarative = green (1 passed) Visual: drove all 4 turns via Playwright (X-AIMock-Context: ms-agent-dotnet) - turn 1 4 KPI metrics + region pie + monthly bar - turn 2 rep-quota DataTable + attainment bar - turn 3 3 severity StatusBadges + KPI metric strip - turn 4 7 account InfoRows + product-line pie |
||
|
Jordan Ritter
|
adc897f485 |
fix(showcase): repair ms-agent-harness-dotnet D6 gen-ui-declarative (surface-missing)
The D6 e2e-full probe `d6:ms-agent-harness-dotnet/gen-ui-declarative`
failed at turn 1 with `reason=surface-missing`. Three layers, all fixed
at the layer the real captured request revealed.
Root cause 1 — stale aimock fixture
The fixture still carried the old D5 pill set (KPI / pie / bar / status
prompts) with inner `_design_a2ui_surface` entries keyed on those stale
prompts. The current driver sends four VantageThreads sales prompts, so
turn 1 ("Show me my sales dashboard for this quarter.") had no matching
inner surface — the backend looped `generate_a2ui` to its limit and the
frontend painted the stale KPI catch-all instead of the sales dashboard.
Re-authored to the four current prompts, mirroring the llamaindex green
north-star for this `_design_a2ui_surface` two-stage backend family.
Root cause 2 — `hasToolResult` breaks the interleaved thread
Unlike ms-agent-python (fresh session per turn), the .NET harness backend
threads the FULL interleaved conversation. `hasToolResult` is a
thread-global predicate (GOTCHAS.md), so once turn 1 leaves a tool result
in the thread, every later pill's outer `generate_a2ui` call sees
`hasToolResult:true` and matches the narration fixture instead of emitting
the tool call — surface-missing on turns 2-4. Switched to the sanctioned
interleaved-safe pattern: narration keyed on this pill's outer
`toolCallId` (ordered before the outer), outer keyed on `userMessage`
only.
Root cause 3 — renderer / catalog drift
`renderers.tsx` and `definitions.ts` lagged the green cluster — missing
the `DataTable` and `InfoRow` testids (`declarative-data-table` turn 2 /
`declarative-info-row` turn 4) plus `Metric.trendValue` and the
Row/Column/Text gap overrides. Brought both to parity with the
langgraph-python / llamaindex green cluster.
Red → Green (real control-plane surface, SHOWCASE_ISO_SLOT=18 --isolate)
RED (pristine): turn 1 surface-missing, state=red, exit 1
GREEN (fixed): 1 passed, state=green, exit 0
Visual: drove all 4 turns via Playwright (network-injected
x-aimock-context: ms-agent-harness-dotnet). Confirmed real painted
surfaces — turn 1 sales dashboard (4 KPIs + region pie + monthly bar),
turn 2 rep-quota DataTable + attainment bar, turn 3 three at-risk status
badges + KPI strip, turn 4 seven account InfoRows + product-line pie.
Screenshots under ~/.local/share/copilotkit/cr/2ndwave-shots/.
|
||
|
Jordan Ritter
|
92be110c62 |
fix(showcase): repair claude-sdk-typescript D6 gen-ui-declarative (surface-missing)
The claude-sdk-typescript declarative-gen-ui cell was red on turn-1 surface-missing: the aimock fixture carried stale D5-era prompts (KPI dashboard / pie / bar / status report) and only a partial turn-1 outer entry, so the two-stage Anthropic A2UI flow (outer generate_a2ui -> secondary render_a2ui -> narration) never painted a surface for the four current sales-analyst pills. Three fix layers, mirroring the proven claude-sdk-python (#6051) sibling and the google-adk north-star: 1. Re-author aimock/d6/claude-sdk-typescript/gen-ui-declarative.json to the 4 current driver prompts in the two-stage shape (12 fixtures = 4 pills x {outer generate_a2ui, inner render_a2ui, narration}), with render payloads mounting the per-pill catalog components the driver asserts (Metric x4 + Pie + Bar; DataTable + Bar; Metric x3 + StatusBadge x3; InfoRow + Pie). Render payloads are byte-identical to the #6051 data. Ordering/matcher fix vs the naive python mirror: the CSTS runtime accumulates full conversation history across pills, so on turns 2-4 the outer generate_a2ui call carries prior pills' tool results and a hasToolResult:false matcher never fires. Each pill triad is ordered narration (toolCallId) FIRST so it claims the last-role:tool calls, and the outer matcher drops hasToolResult and gates on userMessage + toolName generate_a2ui (last-role:user). 2. Add data-testid="declarative-info-row" to the InfoRow renderer (turn-4 top-account parity; CSTS was missed by #6050). 3. Refresh suggestions.ts to the 4 sales-analyst pills (were stale D5-era labels that emitted unmatched prompts -> live 404 banner). Red-green (control-plane, slot 30, --isolate --rebuild): - RED (origin/main): d6:claude-sdk-typescript/gen-ui-declarative red; aimock STRICT: No fixture matched for POST /v1/messages. - GREEN (fixed): 1 passed; aimock journal shows all 12 calls 200, zero 503/no-match across all 4 turns. Visual: Playwright 4-turn walk (x-aimock-context claude-sdk-typescript) confirms metric=4/pie=1/bar=1 (t1), data-table=1/bar+1 (t2), status-badge=3/metric+3 (t3), info-row=7/pie+1 (t4); no fixture error. Unit: aimock-fixtures 837 passed; d5-gen-ui-declarative 31 passed. |