Modern @a2ui/web_core and @angular/cdk subpath exports require bundler
resolution so angular:check-types can resolve the new A2UI imports.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align surface operation normalization and catalog test fixtures with the
RendererProps contract so the changed A2UI web-components pass tsc.
Co-authored-by: Cursor <cursoragent@cursor.com>
Remove the hand-written types entry from exports["."] so ng-packagr can
generate deterministic published package metadata without warnings.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Implemented audio transcription capabilities with error handling.
- Refactored CopilotChat component to utilize a directive for handling attachments.
- Improved CopilotChatReasoningMessage to manage streaming state and elapsed time more efficiently.
- Added new scroll view component for better message display and auto-scrolling behavior.
- Updated styles for A2UI surface components to enhance layout and scrolling.
- Enhanced tests for OpenGenerativeUIRenderer to ensure proper height measurement.
Apply the common Angular CLI schematics to code which has been added
by Soverius.
We can do a follow-up for the high amount of effects, but we have to be
careful. We need tests first to do a refactoring for that.
## Release monorepo v1.60.2
**Scope:** `monorepo` | **Bump:** `patch`
---
### How this release process works
1. **This PR was created automatically** by the "release / create-pr"
workflow.
It bumped the `monorepo` packages to `1.60.2`
and generated AI-enhanced release notes.
2. **CI runs on this PR** — the full test suite (unit tests, lint, type
checks, build)
must pass before merging. This is the review gate.
3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there
before merging.
4. **When this PR is merged**, the `release / publish` workflow
automatically:
- Builds all packages
- Publishes the `monorepo` packages to npm at version `1.60.2`
- Creates git tag `monorepo/v1.60.2`
- Creates a GitHub Release with the final release notes
### Before merging
- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)
---
> **Do not merge until CI is fully green.** The full test suite runs
automatically on this PR.
## Summary
- document the shell-docs hybrid docs architecture and `docs_mode`
meanings
- add a shell-docs README with npm local dev, validation commands, and
authoring recipes
- keep retired top-level `docs/` guidance intact while making shell-docs
the clear source of truth
## Verification
- `git diff --check -- .claude/docs/documentation.md
showcase/shell-docs/README.md`
- pre-commit: `nx run-many -t test --projects=packages/**` passed after
rerun; Nx flagged `@copilotkit/react-core:test` as flaky from an earlier
timeout
- pre-commit: `nx run-many -t publint,attw --projects=packages/**`
passed
## Notes
- Archive refs were created before this branch:
`archive/docs-save-do-not-prune` and `archive/docs-retired-2026-06-17`.
- This PR intentionally does not delete the retired top-level `docs/`
tree; that should be a follow-up cleanup PR.
examples/slack now starts a Slack bot and/or a Telegram bot from one platform-neutral app layer, env-conditional on which credentials are set. Neutralized Slack-specific rendering so the shared components work on both platforms: unicode glyphs instead of mrkdwn shortcodes, no Block Kit raw fallbacks, neutral context/tool wording. Migrated the Telegram e2e smoke harness and BotFather setup docs; removed the separate examples/telegram app.
grammY-based adapter implementing the PlatformAdapter interface. Long-polling by default with webhook/auto ingress opt-in. Topic-aware hybrid threading. HTML rendering with a plain-text format fallback, chunked-edit streaming, callback_query HITL, and inbound file ingestion. Ships a built-in Telegram user-lookup tool and platform context, plus a full unit test suite.
Add display-only generative-UI the copilot renders on request:
- showPendingApprovals — the dashboard's interactive approval table, in chat
- showSpendingTrend, showBudgetUsage, showSpendBreakdown, showIncomeVsExpenses —
brand-styled charts (dependency-free, hand-rolled SVG/CSS)
- showApprovalFlow — a diagram of how an over-limit charge gets cleared
Registered globally in copilot-context (so they work on any route) and routed
via the agent prompt.
- Cards page: the credit-card grid compressed each card (wrapping the holder /
valid-thru text) when the chat panel narrowed the content area. It's now a
horizontal-scroll row — each card holds a 300px minimum and grows to fill on
wide screens, scrolling instead of squishing.
- overflow-x:auto forces overflow-y:auto, which clipped the cards' soft drop
shadows; matching negative-margin/padding pairs give the shadows room inside
the scroll viewport without moving the cards.
- Chat: space stacked HITL/action cards so consecutive cards no longer share an
edge (CopilotKit stacks them with no row-gap).
## Summary
Harden the showcase harness producer against transient Railway-GQL 429 /
Cloudflare-WAF flaps. Today's incident: a ~25-min Cloudflare WAF
burst-block on `backboard.railway.com/graphql/v2` caused the producer's
catalog-enumerate to hard-fail every cron tick, zeroing out D4/D5/D6
writes and turning the entire staging dashboard red within one tick
window.
Three discrete behavior changes, one PR:
1. **Retry with exponential backoff in `source.enumerate`** — three
retries at 1s/4s/16s on HTTP 429, 5xx, Cloudflare 1015/1020/1022
markers, or transport-level errors. Does NOT retry on
`DiscoverySourceAuthError`, non-429 4xx, or schema errors
(operator-actionable, fail loud). Lives in
`showcase/harness/src/fleet/control-plane/catalog-enumerator.ts` (the
seam every family enumerator passes through).
2. **Cached-catalog fallback** — per-enumerator in-memory cache of the
last successful `services[]`. On persistent failure (all retries
exhausted) AND a cache present, the wrapper logs
`fleet.producer.enumerate-failed-using-cache` (warn, with `services`
count, `ageMs`, and `reason`) and returns the cached catalog. With NO
cache (fresh-boot first enumerate fails), the wrapper re-throws so the
producer's `enumerate-failed` short-circuit still runs — without a
catalog there's nothing to enqueue.
3. **3-tick family-silence threshold** —
`SILENCE_CONSECUTIVE_TICK_THRESHOLD = 3` layered ON TOP of the existing
`3 × period` elapsed-time gate. The silence alert now requires BOTH:
`now - lastSuccessAt > 3 × period` AND three consecutive evaluation
cycles observed silent. A single bad tick on a stale `lastSuccessAt` can
no longer page every family at once.
## Red-Green proof
**RED on main (`5a62acbf`)** — observed BEFORE the fix. The RED file
asserts the BUG (`calls === 1` after a 429 throw; `posts.length === 1`
after a single silent tick):
```
> vitest run src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts
RUN v3.2.4 .../showcase/harness
✓ src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts (2 tests) 3ms
✓ [BUG] one 429 + Cloudflare 1015 from the source aborts the whole enumerate (no retry)
✓ [BUG] silence alert posts on the FIRST silent evaluation tick (no consecutive-tick threshold)
Test Files 1 passed (1)
Tests 2 passed (2)
```
**GREEN on fix branch** — observed AFTER the fix (assertions inverted to
the fixed behavior; `calls === 4` after retries; `posts === []` until
the third silent tick):
```
> vitest run src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts src/fleet/control-plane/catalog-enumerator.test.ts src/fleet/control-plane/family-silence-monitor.test.ts
RUN v3.2.4 .../showcase/harness
✓ src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts (2 tests) 4ms
✓ src/fleet/control-plane/family-silence-monitor.test.ts (19 tests) 8ms
✓ src/fleet/control-plane/catalog-enumerator.test.ts (29 tests) 6ms
Test Files 3 passed (3)
Tests 50 passed (50)
```
Full harness suite — **131 files / 2812 tests pass** (`pnpm -F
@copilotkit/showcase-harness test`).
## Test plan
- [x] RED proof captured on `main` (single-attempt enumerate;
single-tick silence alert)
- [x] GREEN proof on this branch (retry to 4 calls; cached fallback;
3-tick threshold)
- [x] `pnpm -F @copilotkit/showcase-harness test` — 2812 passed
- [x] `pnpm -F @copilotkit/showcase-harness typecheck` — clean
- [x] `pnpm -F @copilotkit/showcase-harness build` — clean
- [x] `oxfmt --write` applied; `oxlint` clean
- [ ] CI green
- [ ] Staging redeploy verified via Railway CLI + Playwright dashboard
snapshot
## Files touched
- `showcase/harness/src/fleet/control-plane/catalog-enumerator.ts`
(+296, -3): retry+cache wrapper, exports `ENUMERATE_RETRY_BACKOFF_MS` +
`isRetryableEnumerateError` + `SleepFn`
- `showcase/harness/src/fleet/control-plane/catalog-enumerator.test.ts`
(+295): retry/cache/auth-not-retried/backoff-SSOT tests
- `showcase/harness/src/fleet/control-plane/family-silence-monitor.ts`
(+61): `SILENCE_CONSECUTIVE_TICK_THRESHOLD = 3`, per-family counter,
counter reset on healthy
-
`showcase/harness/src/fleet/control-plane/family-silence-monitor.test.ts`
(+218, -39): 3-tick threshold + counter-reset gate tests; updated
existing tests to advance through 3 silent ticks
-
`showcase/harness/src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts`
(new, +212): the literal RED→GREEN gate
## Operational notes
- No new env vars, feature flags, or backward-compat shims (per scope
directive).
- The cached-catalog warn surfaces in observability via
`fleet.producer.enumerate-failed-using-cache` (services count, ageMs,
reason).
- `SLACK_WEBHOOK_OSS_ALERTS` is intentionally unset (user config); not
touched.
Pre-existing repo-wide lefthook failures (`@copilotkit/core`,
`@copilotkit/runtime`, `@copilotkit/shared` etc.) reproduce on `main`
without my changes and are unrelated to harness code; harness-scoped
quality gates all passed before commit.
## Summary
Ports
[ag-ui-protocol/ag-ui#1914](https://github.com/ag-ui-protocol/ag-ui/pull/1914)
to CopilotKit — plus the two supporting guard files ag-ui already had:
- **`.github/workflows/canary.yml`** — discoverable **`canary /
publish`** `workflow_dispatch` orchestrator. Any maintainer can publish
a prerelease of the branch they're on straight from the Actions tab. It
is a thin orchestrator — it does **not** publish to npm itself:
1. Guards against `main` and non-branch refs.
2. Mints the devops-bot App token (app-id `1108748`,
`DEVOPS_BOT_PRIVATE_KEY`) with scoped `contents:write` +
`actions:write`.
3. Mirrors the dispatched ref to a unique
`canary/<slug>-<run_id>-<attempt>` branch via the GitHub API (no
checkout).
4. Dispatches **`publish-release.yml --ref canary/<slug> -f
mode=prerelease …`**, locates the run, and waits (`gh run watch
--exit-status` + explicit conclusion check).
5. Deletes the canary ref — status-gated (never yanks the ref under a
still-running delegated run) with a fresh cleanup token (90-min job
ceiling exceeds the 1h App-token TTL).
- **`scripts/release/verify-release-scope-dropdowns.sh`** — drift guard:
the hand-maintained `scope` dropdowns in `publish-release.yml` /
`stable-release.yml` / `canary.yml` must exactly match
`release.config.json`'s `.scopes` keys. Parsers fail loud and distinct
on structural changes instead of silently passing.
- **`.github/workflows/lint-release-workflows.yml`** — actionlint +
shellcheck + the dropdown-sync job over the release pipelines.
### Why a separate orchestrator (and not a flag in publish-release.yml)
- A GitHub Environment's deployment-branch policy is evaluated against
the ref a run is **triggered on** — not branches created mid-run. The
orchestrator exists to get the publish run *onto* a `canary/*` ref.
- `publish-release.yml` holds the **single npm OIDC trusted-publisher
binding**; a second publishing entry point would break OIDC for every
`@copilotkit/*` package. The orchestrator never touches npm.
- The cross-workflow dispatch uses the **App token, not `GITHUB_TOKEN`**
— `GITHUB_TOKEN`-authenticated events never start new workflow runs.
**Note:** the `npm` environment currently has *no* deployment-branch
policy, so the orchestrator is a convenience wrapper today. Tightening
the policy to `main` + `canary/*` + `release/publish/*` (matching
ag-ui's security posture) is being applied as repo configuration
alongside this PR — requires admin. This PR includes the prerequisite:
`publish-commit.yml` (pkg-pr-new) is removed from the `npm` environment,
since it runs on every PR and would be blocked by the policy (it
publishes to pkg.pr.new, not npm, and uses no environment secrets).
## Testing done
- Drift guard: positive run against all three real workflows; negative
tests (scope removed → drift FAIL with diff; bogus scope → FAIL; `case
"${SCOPE}"` quoting refactor → loud parser-degradation FAIL; whole case
block deleted → loud zero-block FAIL; quoted arm `"angular")` →
accepted; blank/comment lines inside `options:` → still parsed; prose
comments mentioning case/SCOPE/in → no false positive).
- `shellcheck` clean at all severities; `bash -n` on every workflow
`run:` block; YAML parses.
- 3 rounds of 7-agent code review converged to zero load-bearing
findings.
## ⚠️ Still to verify before first real use
- [ ] devops-bot App (id 1108748) has **Actions: write** — required for
the in-workflow `gh workflow run`. Safe first test: dispatch once with
`dry_run=true`.
- [ ] First `dry_run=false` run clears the `npm` environment end-to-end
via the App token once the deployment-branch policy is tightened.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Post-merge follow-ups (maintainer action required)
These need repo **admin** rights and must happen **in this order**:
1. **Merge this PR first.** `main`'s current `publish-commit.yml`
(pkg-pr-new) still sits in the `npm` environment and runs on every PR
touching `packages/**` — applying the branch policy before this PR lands
would block every snapshot publish. This PR removes that environment
association.
2. **Tighten the `npm` environment's deployment-branch policy** to
`main` + `canary/*` + `release/publish/*` (matching ag-ui). With an
admin-scoped token:
```bash
gh api --method PUT repos/CopilotKit/CopilotKit/environments/npm \
-F "deployment_branch_policy[protected_branches]=false" \
-F "deployment_branch_policy[custom_branch_policies]=true"
gh api --method POST
repos/CopilotKit/CopilotKit/environments/npm/deployment-branch-policies
-f name="main" -f type=branch
gh api --method POST
repos/CopilotKit/CopilotKit/environments/npm/deployment-branch-policies
-f name="canary/*" -f type=branch
gh api --method POST
repos/CopilotKit/CopilotKit/environments/npm/deployment-branch-policies
-f name="release/publish/*" -f type=branch
```
Or via UI: Settings → Environments → npm → Deployment branches and tags
→ "Selected branches and tags" → add the three patterns above.
Why these three: `main` covers stable `workflow_dispatch` retries and
`stable-release.yml`; `release/publish/*` covers the merged-release-PR
runs (the run's head branch is the release PR branch); `canary/*` covers
the orchestrator's delegated prerelease runs. After this, direct
`mode=prerelease` dispatches from arbitrary feature branches stop
working — the `canary / publish` orchestrator becomes the one-click path
(by design).
3. **Verify the devops-bot App (id `1108748`) has `Actions: write`**
(org/App settings). The orchestrator's `gh workflow run` dispatch 403s
without it. Safe end-to-end test, after step 2: Actions tab → **canary /
publish** → pick any feature branch, any scope, **`dry_run=true`** →
confirm the delegated `release / publish` run is created, watched, and
the `canary/*` ref is deleted afterward.
4. **First real canary** (`dry_run=false`) confirms the npm OIDC publish
clears the environment gate end-to-end on a `canary/*` ref.
Adds an integration-style test file that pins the BOTH layers of the
2026-06-17 Cloudflare-WAF-burst incident fix together:
- the enumerator retries 3× on a 429+Cloudflare-1015 burst before
bubbling (the original bug let one 429 abort the whole enumerate);
- the silence monitor requires THREE consecutive silent evaluation
cycles before posting an alert (the original bug fired on tick #1).
These assertions were the LITERAL red proof on `main`:
- on main both `it` blocks PASSED while asserting the buggy behavior
(calls === 1, posts.length === 1 after a single silent tick),
- on this branch the same gates re-pin the fixed behavior (calls === 4
after retries; posts === [] until the third silent tick).
Run-time output captured for the PR body confirms the inversion.
Layer a per-family consecutive-silent-tick counter ON TOP of the existing
3×period elapsed-time gate (`SILENCE_PERIOD_MULTIPLIER`). The silence
alert now requires BOTH:
- `now - lastSuccessAt > 3 × period` (existing elapsed-time gate), AND
- `SILENCE_CONSECUTIVE_TICK_THRESHOLD = 3` consecutive evaluation cycles
observed silent (new — the counter resets on any successful evaluation).
Without the new gate a single bad cron tick on a family whose
`lastSuccessAt` was already stale (e.g. after a long quiet window or a
deploy gap) tripped the alert immediately — the failure mode the
2026-06-17 Cloudflare-WAF-burst incident exposed where one ~25 min flap
on backboard.railway.com/graphql/v2 paged every family at once.
The counter is NOT incremented during boot grace, so a cold-start cycle
can't alone push it to threshold. The meta-alert (`family-silence-eval`)
path keeps its own clock and is unaffected. Existing tests advance
through three consecutive silent ticks before asserting the post.
Three-retry exponential backoff (1s/4s/16s) on `source.enumerate` against
Railway-GQL when the underlying error is transient (HTTP 429, 5xx, or a
Cloudflare 1015/1020/1022 WAF marker, or a transport-level reject). On
persistent failure, fall back to the last successful catalog from a
per-enumerator in-memory cache — LOUDLY logged via
`fleet.producer.enumerate-failed-using-cache` so the cache-use shows up
in observability.
A fresh-boot process with no cached entry preserves the current
hard-fail behavior (the producer's `enumerate-failed` short-circuit) —
without a catalog there is nothing to enqueue. Real config errors
(`DiscoverySourceAuthError`, non-429 4xx, schema rot) are NOT retried so
operator-actionable failures surface immediately.
Context: 2026-06-17 Cloudflare WAF burst-blocked
backboard.railway.com/graphql/v2 for ~25 min, hard-failing the producer
enumerate on every cron tick and zeroing out D4/D5/D6 writes — the
entire staging dashboard went red within one tick. Retries + cache ride
out the burst on the same tick and preserve job production across
longer outages.
Pre-existing repo-wide lefthook test failures (@copilotkit/core,
@copilotkit/runtime, @copilotkit/shared, etc.) are unrelated to the
harness; verified by stashing my changes and running the same hook on
main with identical failures. Harness suite (131 files / 2812 tests),
typecheck, and build pass on this branch.
- webhook: don't let GET / health shadow the verify handshake when WHATSAPP_PATH=/
- adapters: fail loud on a malformed PORT instead of binding NaN
- tests: cover the id-only senderContext branch, the 404-regression for the
health route, and the PORT guard
- docs: update README senderContext snippets to the 2-arg signature
Adds a cookbook recipe + runnable showcase that gives the Built-in Agent
OAuth-backed Arcade tools (Gmail, Google News) and renders Arcade's one-time
authorization step as a generative-UI "Connect" card in the chat.
- docs: showcase/shell-docs/src/content/docs/cookbook/arcade.mdx (+ meta.json, index card)
- app: examples/showcases/arcade-tools (Next.js App Router, single-route runtime)
Co-authored-by: Cursor <cursoragent@cursor.com>
- Add control-plane-run.test.ts (17 tests) covering buildLocalServicesJson,
expectedKeys, dedupeScopes, and runViaControlPlane error surfacing
- Export SlugScope, buildLocalServicesJson, expectedKeys, dedupeScopes
for unit-test coverage (factored inline dedup loop into dedupeScopes
helper at the same time)
- runViaControlPlane: surface scopeLabel (demo-aware) in the 0-enqueue
error instead of the bare-slug join, with an empty-targets guard so
the error never renders with a double-space gap
- runViaControlPlane: treat tick.enqueueFailures > 0 as fatal — partial
enqueue used to silently proceed and either mask missing cells or
hang the poll loop to timeout
- Eliminate a stray literal NUL byte in the source by switching the
dedup key separator to a \x00 escape
- lifecycle.up(): name the compose call (infra-up vs target rebuild)
in the health-fail error so an operator can tell which call left a
service unhealthy
(cherry picked from commit 9f35c64adfdf7f5ff2bf0a5ae57ed03818cea607)
CR Finding 1 (BLOCKER): d5-tool-rendering-default-catchall.ts used the
broken page.evaluate(fn, arg) second-arg form to pass the leak-phrase
needle into the browser-side closure. A11 proved empirically that the
arg arrives as undefined inside the closure, making 'if (needle)' guard
the entire leak-detection cascade as dead code — customLeakPhrasePresent
stayed false forever, rendering validateDefaultCatchall's leak branch
dead code as well. Mirrors the A11 fix on the sibling custom-catchall
probe by inlining the needle as a JS string literal inside the closure;
no page.evaluate(fn, arg) dependency at all. Both probes now share the
same inline-needle pattern and keep the canonical literal in lock-step
with their exported phrase constant.
CR Finding 2 (MAJOR): the A11 inline-needle fix on the sibling
custom-catchall probe had no regression test — fake Page.evaluate in
makePageReturning never executes the probe closure, so reverting the
fix would not be caught. Added regression tests that capture the
probe's function source via toString() and assert (a) the canonical
phrase appears as a literal inside the page.evaluate(...) closure and
(b) the closure takes no parameter / the evaluate call has no
second arg. Added the same coverage to default-catchall to protect
the new A25a fix.
CR Finding 3 (MAJOR): A7's requireContentPhrase=true branch in
validateCustomCatchall / assertCustomCatchall had zero coverage —
tests omitted the third arg and exercised only the default false
branch. Added coverage for the true branch (pass on phrase present,
fail on phrase absent, fail on phrase undefined, default-branch
preserved) plus assertCustomCatchall plumbing through the options
form. Also added coverage for default-catchall's customLeakPhrasePresent
branch in validateDefaultCatchall for symmetry.
Local proof:
- RED (fix reverted via git stash): 2 inline-needle regression tests
fail on d5-tool-rendering-default-catchall.test.ts
- GREEN (fix restored): 34/34 tests pass across both files
Out of scope (NOT touched this commit): showcase/harness/src/cli/
control-plane-run.ts and lifecycle.ts (A25b's scope).
(cherry picked from commit c409e3a8d99e16ad0bb05ee3c2e2051e5792049f)
Re-tier the showcase docs tree to be an agent entry point: README.md
opens with a 'when X, see Y' fanout table that routes to the right
procedural doc; each procedural doc gets a one-line tagline answering
'what does this answer'.
Consolidation:
- DELETE showcase/RUNBOOK.md — operational content merged into DEBUGGING.md
(Integration Patterns, Docker Compose Environment, Production Debugging,
Anti-Patterns, Aimock Fixture Deployment, Dev Iteration Speed). The
--isolate mechanics + CLI rules were already duplicated in DEBUGGING.md.
- DELETE showcase/QA-COVERAGE.md — per-demo coverage matrix + starter hero
matrix + probe depth + infra locations + gaps folded into TESTING.md as
the 'Per-Demo Coverage Matrix' section.
Taglines added (no behavioral change to content): TESTING.md, DEBUGGING.md,
GOTCHAS.md, INTEGRATION-CHECKLIST.md, STYLING-GUIDE.md, FRONTEND-STRATEGY.md,
RAILWAY.md, bin/README.md, aimock/README.md, aimock/RAILWAY.md,
harness/README.md, harness/docs/rotation-drill.md.
Cross-link fixups: FRONTEND-STRATEGY.md (was QA-COVERAGE.md →
TESTING.md#per-demo-coverage-matrix), TESTING.md (removed dangling RUNBOOK
companion reference), README.md (rewritten as fanout entry + retained
from-scratch setup + dashboard SOPs below the fanout).
PARITY_NOTES.md × 12 left alone (per-slug context, not redundant).
(cherry picked from commit 75c9d9755c9118c8abc1fa52deda2012b768cab1)
(cherry picked from commit b64189bae0fe2c9e3a5e3ca440013deb4121f23b)
New content:
- TESTING.md: add 10-step cell red→green SOP + bin/showcase test invocation
table (control-plane vs --direct, per-demo scoping matrix); retain
existing CI gating matrix below.
- GOTCHAS.md: add operational gotchas — aimock caches fixtures at container
startup (warm-slot reuse needs docker restart) + --isolate slot collisions
with foreign Docker projects.
- README.md: cross-link to TESTING.md SOP from CLI section; flesh out
--isolate / --direct in test options table; update use cases.
- RUNBOOK.md: update Verifying a Slug's D6 State to use auto-named --isolate;
note A21+A21b per-slug rebuild scoping; rewrite Fixture Matching to teach
picking the backend-id-invariant discriminator (turnIndex post-A12/A13/A20);
modernize Debugging Sequence to --isolate flow.
- DEBUGGING.md: lead with TESTING.md SOP cross-link; update Phase 1 to
--isolate canonical; soften turnIndex-only log-line description; note
aimock startup caching in Phase 5; switch Strategy 5 gold-standard check
to --isolate.
Pruned/updated stale claims (post-A11/A12/A13/A18/A20/A21/A21b):
- RUNBOOK.md "Do not use turnIndex in new fixtures" — turnIndex is now
the canonical backend-id-invariant alternative when toolCallId is fragile
(Anthropic / TanStack Responses API ID rewrites). Replaced with discriminator
selection guidance.
- RUNBOOK.md anti-pattern "NEVER use turnIndex" — replaced with NEVER
anchor on toolCallId strict equality against ID-rewriting backends, and
NEVER use --direct for value-tests.
- RUNBOOK.md bin/showcase test <slug> --d5 (no --isolate) as canonical SOP
— replaced with --isolate canonical, no manual name required.
- README.md --d5 option description claiming "subagents/tool-rendering/agentic-chat"
fixed slate — replaced with "defaults to agentic-chat representative; :demo
qualifier honored post-A18".
- DEBUGGING.md Phase 1 "showcase up aimock <slug> && showcase test <slug> --d5"
as primary — kept as legacy alternative; --isolate is now lead.
- DEBUGGING.md Phase 5 "fixtures baked into Docker image" — clarified that
aimock additionally caches fixtures in memory at startup (volume-mounted
isolated stack still requires docker restart for warm-slot edits).
- DEBUGGING.md Strategy 5 "showcase test langgraph-python --d5" — replaced
with :demo + --isolate so the gold-standard check exercises the same cell.
(cherry picked from commit 0e548455043396972f7fb5b96f8c0ea8abdf1d98)
(cherry picked from commit 592c02d392350d02cc5e17544e663a6605b8da65)
Mirrors A19b's BIA fix pattern for the Anthropic-family csdkts integration.
Root cause: csdkts uses Anthropic SDK which generates its own toolCallIds (toolu_*) rather than echoing aimock's prescribed call_d6_cc_*. The fixture's toolCallId-gated narration entries never matched on turn-2, causing fall-through to less-specific entries (or 503/no-match).
Fix: replace toolCallId discriminator with turnIndex (count of role:assistant messages). turnIndex is backend-id-invariant — it works regardless of how the backend rewrites tool_call_id values. Same shape as A19b BIA fix.
- Tokyo narration: toolCallId → turnIndex: 1
- AAPL narration: toolCallId → turnIndex: 3
- AAPL emit: added turnIndex: 2
- Tokyo emit: turnIndex: 0
response.content + canonical phrase ("rendered through the custom wildcard catchall") and response.toolCalls UNTOUCHED.
Verified locally on cr5495/fix-a20-csdkts-green at HEAD d178e6730 (post-A21b):
- /tmp/cr/a20v6-green-csdkts.log: 1 passed, INNER_EXIT=0
- iso2 slot, full infra healthy (aimock+pocketbase+dashboard+csdkts)
(cherry picked from commit e66e0eb0ce72c970348183eeb4f4b57c3f5b1d29)
A21 (a9114a831) regression: positional slug after 'up' restricted which services start (only target+depends_on came up; infra profile services never started). With concurrent sibling stack on same host ports, health checks crossed to foreign pocketbase → cell results misrouted → 0.0s red. Fix: split into 2 compose calls — (1) compose infra up -d, then (2) compose --profile slug up -d --build slug. Preserves A21's target-only rebuild + restores full infra stack.
(cherry picked from commit d71ff46edb4a7b0a89f89d8126fd5fe3d5c7d24e)
Root cause
----------
Both the shell `cmd_up` (showcase/bin/showcase line 98) and the harness
auto-start path (showcase/harness/src/cli/lifecycle.ts up() line 268) emit
`docker compose up -d --build` with NO positional service filter after `up`.
Under `docker compose` semantics, an unfiltered `--build` rebuilds EVERY
service in every active profile — for `--isolate` runs that means the
target slug PLUS aimock + pocketbase + dashboard + harness-pool-worker
all rebuild on every isolated test invocation.
Concurrent `--isolate` runs (one per `SHOWCASE_ISO_SLOT=<N>`) contend on
the shared BuildKit instance, serializing what should be parallel work.
This is the BuildKit-contention stall that has been blocking csdkts (A20)
when it runs alongside other concurrent isolated cells: a small slug
rebuild waits on every other slot's full-stack rebuild.
Fix
---
Scope `--build` to the targeted slug(s) by passing them as positional
service args AFTER `up`. Compose then rebuilds ONLY those services and
falls through to cached images for infra services. First-time bootstrap
still works because `--build` only FORCES a rebuild of services with an
existing image — missing images are built automatically by compose.
Two call sites:
- showcase/bin/showcase (user-facing `showcase up <slug>...`)
- showcase/harness/src/cli/lifecycle.ts (npx tsx test path, the one
actually triggered under `--isolate`)
When no slugs are provided (infra-only bring-up), the blanket `--build`
is preserved so first-time infra bootstrap rebuilds whatever is missing.
`--rebuild` regression: the `--rebuild` flag goes through `rebuild()`,
not `up()`, and that path is unchanged — it still force-rebuilds and
force-recreates every targeted slug as before.
Red→Green proof
---------------
3 new vitest cases in showcase/harness/src/cli/lifecycle.test.ts assert
the compose argv shape:
- up(["langgraph-python"]): slug positional after `up` (was: missing)
- up([]): no slug positional (infra-only blanket)
- up(["a","b"]): both slugs as positionals after `up`
Pre-fix: 2 failed | 4 passed (6)
Post-fix: 0 failed | 6 passed (6)
The compose-argv assertion is the deterministic proof of behavior
change. Wall-clock timing improvement on concurrent --isolate runs
follows mechanically from the compose semantics: with the fix, slot
N's BuildKit work no longer blocks slot M's because they only rebuild
their respective slug images, not the shared infra stack.
(cherry picked from commit 8a47dbb0d2a2fe72e21c09a824485e1bb5afc463)
Replace per-leg toolCallId pin with turnIndex (assistant-count) +
userMessage on the two narration fixtures, and add explicit turnIndex
to the AAPL-emit fixture, so first-match-wins partitions the four
request shapes BIA produces against the OpenAI Responses API.
Root cause:
- BIA uses @tanstack/ai-openai openaiText('gpt-4o') which calls
/v1/responses. aimock converts each /v1/responses request to a
chat-completions-shaped completionReq via responsesInputToMessages()
and matches with the same router. The matcher's toolCallId check is
strict equality against the last message's tool_call_id.
- BIA's TanStack runtime auto-generates tool_call_id at request time
(e.g. 'fc-fCgLtvquOtRpCJTM'), so the fixture-side literal
'call_d6_cc_weather_001' / 'call_d6_cc_stock_001' never matched.
Result: 503 STRICT no-fixture-match on the narration turns, BIA
agent looped on AAPL emit indefinitely.
Fix shape:
- Tokyo narration: toolCallId -> turnIndex: 1
- AAPL narration: toolCallId -> turnIndex: 3 (was off-by-one until I
accounted for the Tokyo-narration assistant message itself adding
to the assistant-count tally seen at AAPL emit time)
- AAPL emit: add turnIndex: 2 so first-match-wins partitions emit vs
narration on the second prompt's two turns
Verification (worktree wt-5495-a19-bia-record, slot iso6):
- RED: bin/showcase test built-in-agent:tool-rendering-custom-catchall
--d5 --isolate -> state=red, 0 passed/1 failed, INNER_EXIT=1
(aimock journal: 3x 503 'No fixture matched' on turn-2 narration
request; AAPL emit fixture matched repeatedly = infinite loop)
- GREEN: same command, post-fix and aimock-restart so the container
reloads the fixture -> state=green, 1 passed, INNER_EXIT=0; aimock
journal: 4 requests, all 200, clean progression
Tokyo-emit (asstCount=0) -> Tokyo-narrate (asstCount=1) ->
AAPL-emit (asstCount=2) -> AAPL-narrate (asstCount=3)
- LGP regression: bin/showcase test
langgraph-python:tool-rendering-custom-catchall --d5 --isolate ->
state=green, 1 passed, INNER_EXIT=0 (uses its own fixture under
aimock/d6/langgraph-python/ — untouched by this change)
Constraints honored: response.content and response.toolCalls
preserved verbatim; canonical narration phrases unchanged; only the
match keys (and their explanatory _comment fields) were modified;
other integrations' fixtures and the harness probe were not touched.
(cherry picked from commit 60d027a1376ba72309b5b3fcf94cc26c64757307)
Before A18, `bin/showcase test <slug>:<demo> --d5/--d6` routed through the
control-plane runner but `runViaControlPlane()` collapsed every TestTarget
to its bare slug:
const slugs = [...new Set(targets.map((t) => t.slug))];
`target.demo` was silently dropped, and:
- `buildLocalServicesJson` hardcoded `demos: ["agentic-chat"]` for d5 and
the full demo set for d6 — regardless of what the operator typed.
- `expectedKeys` always emitted the level's DEFAULT-scope key
(`d5:<slug>/agentic-chat` for d5, `d6:<slug>` aggregate for d6).
The net effect on d5: typing `built-in-agent:tool-rendering-custom-catchall
--d5` enqueued the agentic-chat representative, ran agentic-chat in the
worker fleet, and wrote the side row `d5:built-in-agent/agentic-chat` —
green in ~0.0s — while the dashboard reported PASS for a cell the run
never exercised. Validation was dishonest by construction; the CLI claim
of per-demo coverage was substituted with the default representative.
Fix (additive):
- Introduce a per-call SlugScope = { slug, demo? } and thread it through
`buildLocalServicesJson` + `expectedKeys` + `runViaControlPlane`.
- When `target.demo` is set:
* `LOCAL_SERVICES_JSON` synthesizes `demos: [<demo>]` (the worker's
d6 driver reads `input.demos` → `demosToFeatureTypes` → the closed
featureType set, so the matrix narrows to exactly that demo's
featureType(s)).
* `expectedKeys` translates the demo ID into its featureType(s) via
the same `REGISTRY_TO_D5` mapping the driver uses, and waits on
`<level>:<slug>/<featureType>` — NOT the default-scope key. An
unmappable demo throws (would otherwise hang to timeout).
- When `target.demo` is absent: zero behavioral change. d5 still
enqueues the agentic-chat representative and waits on
`d5-single-pill-e2e:<slug>` + `d5:<slug>/agentic-chat`; d6 still
enqueues the full demo set and waits on the `d6:<slug>` aggregate.
- CLI banner now prints the per-scope label (`<slug>:<demo>` qualifier
when present), mirroring the legacy direct path's labeling.
Repro (built-in-agent:tool-rendering-custom-catchall, --d5 --isolate):
BEFORE
Waiting for worker fleet to produce cells:
d5-single-pill-e2e:built-in-agent, d5:built-in-agent/agentic-chat
✓ 2 passed (0.0s) ← agentic-chat representative ran;
catchall never touched. False-positive.
AFTER
Waiting for worker fleet to produce cells:
d5:built-in-agent/tool-rendering-custom-catchall
✗ 1 failed: state=red ← catchall actually ran. Cell is RED at
this HEAD (BIA narration→DOM is a
separate task A12). Validation is now
HONEST.
Regression: bare-slug d5 default UNCHANGED:
built-in-agent --d5 →
d5-single-pill-e2e:built-in-agent, d5:built-in-agent/agentic-chat
✓ 2 passed (0.0s)
Files:
showcase/harness/src/cli/control-plane-run.ts (+96 -17)
- import demosToFeatureTypes
- SlugScope interface
- buildLocalServicesJson: per-slug demo override
- expectedKeys: per-demo featureType-derived keys; refuse-loud
on unmappable demo
- runViaControlPlane: (slug, demo) dedup; scope-aware banner/log
(cherry picked from commit 82ae79279415b7eaf91759df4d8bc1625816a8c3)
Pre-existing crewai backend defect: tool-rendering-custom-catchall agentId was routed to the shared LatestAiDevelopment ChatWithCrewFlow at /, with no get_weather/get_stock_price handlers. tool_result never returns, second LLM call never fires, toolCallId-gated narration fixture entries unreachable, A7 requireContentPhrase=true probe fails.
Fix: add get_stock_price_impl in src/agents/tool_rendering.py mirroring LGP-python tool shape; re-route 'tool-rendering-custom-catchall' agentId in src/app/api/copilotkit/route.ts to createAgent('/tool-rendering').
Local verification on cr5495/fix-a14-crewai-green:
- /tmp/cr/a14-green-crewai.log: 1 passed (6.8s), INNER_EXIT=0
- /tmp/cr/a14-green-lgp.log (LGP regression): 1 passed (9.0s), INNER_EXIT=0
(cherry picked from commit 208d90018a3fc41c263fd4514b104c78815f8303)
Rainer Hahnekamp