## Summary
- rolls the north-star Intelligence/Threads shell into
`examples/integrations/strands-python`
- wires the Strands route to use CopilotKit Intelligence when
`COPILOTKIT_LICENSE_TOKEN` is present, with in-memory fallback when
unlicensed
- documents the Intelligence env vars and removes the temporary
`strands-python` parity allowances
- adds a contract test for the migration pattern and stabilizes the
existing A2UI renderer test that blocked pre-commit
## Verification
- `pnpm exec vitest run
scripts/__tests__/integration-intelligence-migration.test.ts`
- `pnpm parity:verify --target=strands-python`
- `pnpm parity:check`
- `npm run build` in `examples/integrations/strands-python`
- `pnpm --dir packages/react-core exec vitest run
src/v2/__tests__/A2UIMessageRenderer.test.tsx`
- `NX_TUI=false pnpm nx run @copilotkit/react-core:test`
- `git diff --check`
- pre-commit hooks on both commits: `pnpm run test` and `pnpm run
check:packages`
## Manual smoke
- ran the local Intelligence stack from `test-signups4`
- started the Strands agent and Next UI on `localhost:3001`
- confirmed `/api/copilotkit/info` reported `mode: "intelligence"` and
`licenseStatus: "valid"`
- created a thread, asked the agent to enable app mode and add three
todos, then reloaded and selected the saved thread
- verified the restored thread showed the user prompt, tool calls,
assistant response, and the three persisted todos
Conflict resolutions:
- contract test: keep the rollout's parameterized version and add
strands-python to migratedIntegrations/appRoots (60/60 passing) in
place of her bespoke MIGRATED_INSTANCES file
- parity manifest: rollout's version with strands' three threads-shield
allowances removed (mirrors the langgraph-fastapi migration); parity
verify green — strands now 88 tracked files, zero drift
- package-lock: regenerated at 1.59.3 (a2ui-renderer stays 1.56.5,
the family-wide pin shared with the north-star)
Also rides: her react-core A2UIMessageRenderer test flake fix
(act -> waitFor), kept intentionally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- add env-gated CopilotKit Intelligence wiring to the shared AgentCore
Hono/Lambda runtime bridge
- wire the Vite frontend to REST Threads transport with the Threads
drawer, locked gate, and shared `threadId` chat/canvas context
- add local Docker env propagation for Intelligence vars and the Vite
client-safe Threads gate
- pin AgentCore frontend/runtime packages to the threads-capable
CopilotKit/AG-UI versions and add missing frontend peer deps needed for
a clean build
- extend the integration migration regression suite with
AgentCore-specific coverage
## Verification
- `pnpm exec vitest run
scripts/__tests__/integration-intelligence-migration.test.ts`
- `pnpm exec oxfmt --check
scripts/__tests__/integration-intelligence-migration.test.ts
examples/integrations/agentcore/frontend/src/components/chat/CopilotKit/index.tsx
examples/integrations/agentcore/frontend/src/components/threads-drawer/locked-state.tsx
examples/integrations/agentcore/frontend/src/components/threads-drawer/threads-drawer.tsx
examples/integrations/agentcore/frontend/vite.config.ts
examples/integrations/agentcore/infra-cdk/lambdas/copilotkit-runtime/src/runtime.ts`
- `npm run build` from `examples/integrations/agentcore/frontend`
- `npm run build` from
`examples/integrations/agentcore/infra-cdk/lambdas/copilotkit-runtime`
- pre-commit hook: `check-binaries`, `sync-lockfile`, `lint-fix`,
`test-and-check-packages`, `commitlint`
- local Intelligence stack: copied
`/Users/mothra/Projects/test-signups4/.env` into AgentCore's ignored
`docker/.env`, started `docker compose up -d --wait`, seeded
`demo-user`/`1_demo-user`
- local bridge smoke: `GET
http://localhost:3101/copilotkit/threads?agentId=default` returned `200`
- Playwright smoke with local generated `aws-exports.json` + seeded OIDC
storage: authenticated Vite app rendered, licensed Threads drawer
visible on desktop, mobile floating pill visible, no horizontal
overflow, reload preserved authenticated render
## Notes
- Full live AgentCore response verification was not completed because
this path requires a deployed AWS AgentCore/Cognito stack and real
AgentCore agent runtime. The browser smoke used fake OIDC storage and a
dummy agent URL, so it verified frontend/runtime Threads wiring but not
a real agent response.
- In the fake-auth local smoke, the Intelligence realtime websocket
returned `403`, while the REST Threads list rendered successfully. I did
not treat that as proof of a production websocket issue because the
smoke bypassed real Cognito/auth, but it is worth keeping an eye on in a
deployed AgentCore environment.
Conflict resolution: take the rollout's contract test and append the
agentcore describe block (CDK lambda runtime gate, Vite frontend with
import.meta.env gate, docker env wiring). Bump both agentcore
package.jsons 1.59.1 -> 1.59.3 + regen lockfiles. 54/54.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- add Intelligence Threads wiring for `examples/integrations/a2a-a2ui`
- add the reusable threads drawer UI and env-gated Intelligence runtime
config
- add a local A2UI v0.8 renderer so the demo renders the current A2A
operation payloads
- extend the migration verifier coverage for a2a-a2ui
## Verification
- `pnpm exec vitest run
scripts/__tests__/integration-intelligence-migration.test.ts`
- `npm run build` from `examples/integrations/a2a-a2ui`
- pre-commit hook: `pnpm run test && pnpm run check:packages` via
lefthook
Refs ENT-734.
Conflict resolution: take the rollout's parameterized contract test and
append the a2a-a2ui bespoke tests (namespaced helper, 1.59.3 pins).
Bump a2a-a2ui @copilotkit/* 1.59.1 -> 1.59.3 + regen lockfile. 49/49.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audit follow-up to #5232, which fixed the gen-ui-interrupt d6 fixture
leg mis-order for langgraph-python + langgraph-typescript. aimock's
matchFixture is first-match-wins in array order and the loader dedups
on userMessage, so a toolCallId resume-leg placed BEFORE its
toolName:schedule_meeting first-leg shadows the tool-emitting leg on
turn-2 — the interrupt never fires and time-picker-card never mounts.
Fleet audit of every showcase/aimock/d6/*/gen-ui-interrupt.json found
ag2 as the only remaining mis-ordered fixture (both toolCallId
resume-legs preceded their toolName first-legs). Reorder ag2 so each
pill's toolName first-leg precedes its toolCallId resume-leg, matching
the proven-correct mastra/pydantic-ai/langgraph pattern. Reorder only;
response payloads and match keys are unchanged.
All other gen-ui-interrupt-supporting integrations (built-in-agent,
claude-sdk-*, langgraph-fastapi, langroid, ms-agent-dotnet/python,
pydantic-ai, spring-ai, strands) were already correctly ordered.
- bump a2a-middleware, mcp-apps, agent-spec from 1.59.1 (their verified
pre-revert state) to 1.59.3 to match the starters
- regenerate package-lock.json for the 11 examples whose package.json
changed (drawer deps re-added on starters, version bumps on the three)
- update the migration contract test's version assertions to 1.59.3
(43/43 passing); oxfmt pass
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restores #5151 (north-star + batch 1 + crewai-flows + llamaindex),
#5196 (pydantic-ai), #5205 (a2a-middleware), #5211 (mcp-apps), reconciled
onto the current main baseline rather than the pre-revert tree:
- keep main's 1.59.3 pins, AGENT_URL normalization, default agent keys,
useConfigureSuggestions, available:false, useRenderTool
status/parameters API, call-time agent.state reads, and crewai-crews'
rebuilt page (not yet threads-migrated)
- graft the threads layer (drawer/gate/provider, env-gated route
intelligence block, next.config gate, env docs, drawer deps) on top
- drop threads-era sidebar suggestions props where main now registers
suggestions via useConfigureSuggestions (or omits them)
- fix the stale pydantic-ai doc link main reintroduced in ms-af-dotnet
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`bin/showcase test --rebuild` was either a silent no-op or an error
depending on container state:
- runner.run() only rebuilt services in `autoStarted` (those NOT already
running), so --rebuild against an already-up service did nothing and a
stale container was silently reused (the 36h-stale-image false-positive).
- lifecycle.rebuild([slug]) ran `compose --profile <slug> build` WITHOUT
`--profile infra`, leaving `aimock` undefined → "service <slug> depends
on undefined service aimock: invalid compose project" when the service
was down.
Fix both:
- runner.run() now rebuilds every targeted slug when --rebuild is set,
regardless of running state, and health-checks rebuilt-but-already-running
services before probing.
- rebuild() includes `--profile infra` alongside the slug profiles (mirrors
up()) and force-recreates the targeted containers so the freshly-built
image is actually adopted.
Adds red-green unit coverage asserting the infra profile and force-recreate
in the rebuild compose invocation.
## Summary
`setRuntimeTransport` was not idempotent on the **requested** transport
mode. Auto-detect resolves the requested `"auto"` to a concrete
transport (`"rest"`/`"single"`) and writes that back to
`_runtimeTransport`; the guard then compared against that **resolved**
value. So re-applying the same requested mode — which the provider
effect does on **every render** — compared unequal and **re-ran the
entire `/info` handshake**, rebuilding the runtime agents mid-session.
When that re-sync lands during a turn, `useAgent` hands the UI a
freshly-rebuilt (empty) agent for a render, **blanking the whole
transcript** (and any per-message UI bound to it — e.g. the intelligence
indicator) until it replays.
## Fix
Track the **requested** mode separately (`_requestedTransport`) and
guard on it. Re-applying an unchanged requested transport — including
`"auto"` after auto-detect has resolved it — is now a no-op, so no
redundant `/info` re-sync fires.
## Tests
`packages/core/src/__tests__/agent-registry-resync.test.ts`: re-applying
`"auto"` after auto-detect resolves it does **not** refetch `/info`.
## Scope / risk
Touches only `packages/core` (`core/agent-registry.ts` + the test). **No
public API change.** Genuine transport *changes* still re-sync exactly
as before; only redundant re-applications of the same requested mode are
skipped.
> **Verified live:** this idempotency guard *alone* eliminates the
mid-turn re-sync in the e-commerce intelligence demo — instrumentation
shows no `/info` handshake fires during a turn, and the
transcript/indicator no longer flickers. (An earlier draft also made the
re-sync itself non-destructive as defense-in-depth; that was dropped
because, with this guard, no mid-turn re-sync occurs for it to protect
against.)
The gen-ui-interrupt d6 probe drives two interrupt turns in one thread.
Turn-2 failed because the langgraph-python/typescript fixtures ordered each
pill's toolCallId resume-leg BEFORE its toolName first-leg. aimock dedups on
userMessage (first-match-wins), so the text-only resume-leg shadowed the
tool-emitting first-leg on the duplicate userMessage — the agent emitted a
plain text reply with no schedule_meeting tool call, the interrupt never
fired, and the time-picker-card never mounted.
Reorder each pill so the toolName:schedule_meeting first-leg precedes its
toolCallId resume-leg, matching the passing mastra/pydantic-ai ordering. The
resume-leg still only matches when the last message is its tool result
(toolCallId guard), so confirmation still works.
PR #5226 added the "Starter" pseudo-category row-group (StarterSection,
using resolveStarterRow/buildStarterBadge) only to CellMatrix in
cell-matrix.tsx. But the live DashboardPage renders the matrix tab via
FeatureGrid (feature-grid.tsx); CellMatrix is reached only through
cells-view.tsx, which no live route imports — it is dead/legacy. So the
Starter row shipped in code but could never render (the served prod bundle
had zero StarterSection/starter-row-/"no starter" strings).
Port the StarterSection into FeatureGrid: render the four fixed sub-rows
(health/agent/chat/interaction) after the feature categories, resolving
each cell via the existing S3 helpers (resolveStarterRow + buildStarterBadge,
5-state vocab) and structurally excluded from the feature rollup/column tally
(never calls renderCell/buildCellModel). Aligned to FeatureGrid's table
structure (Feature col + optional parity ref-depth spacer + categoryColSpan).
Remove the duplicate from the dead CellMatrix and move its dedicated tests
to feature-grid.test.tsx, which exercises the live registry-backed grid.
Render proof: production next build now bakes starter-row-/starter-cell-/
"no starter for this integration" into the served page.js (client + server)
bundles. Full dashboard suite green (815 passed, 1 skipped).
The updateProverb useFrontendTool passed the live `agent` object in its
deps array. react-core's use-frontend-tool does JSON.stringify(extraDeps),
which throws "Converting circular structure to JSON" on agent.activeRunDetach$
(an RxJS Subject), crashing the React tree so no assistant bubble renders.
`agent`/`agent.setState` are stable references; only `state` needs to be a
dep. Also bump the docker ag-ui-crewai override to >=0.2.0,<0.3.0 to match
agent/requirements.txt and the showcase backend.
Swap the boilerplate "Create Next App" / "Generated by create next app"
title and description in each starter layout for a per-framework
"<Framework> + CopilotKit Starter" title and matching description.
crewai-flows hardcoded the agent URL with no AGENT_URL env override; add
the fallback to match siblings. agno and llamaindex concatenate a path
suffix onto the base URL, so strip any trailing slash first to avoid a
double slash when AGENT_URL ends in "/".
The welcome message promised theme/proverb/weather tools, but the page
registered none of them (only a debug console.log of agent.state).
Mirror the crewai-flows sibling to wire setThemeColor, the proverbs
shared-state tool, and a weather render tool, remove the leftover
console.log effect, and drop the stray w-screen overflow.
The agno and llamaindex add_proverb frontend tools closed over a stale
`state` snapshot with no dependency array, so rapid successive adds
dropped earlier proverbs. Read agent.state at call time inside the
handler instead. Also drop the stray w-screen on the llamaindex page
that caused horizontal scrollbar overflow.
The smoke build path (docker/Dockerfile.agent) runs `uv sync` against
agent/pyproject.toml + agent/uv.lock, but those still pinned
ag-ui-langgraph==0.0.34 (and the lock still pinned copilotkit==0.1.87).
copilotkit 0.1.93 requires ag-ui-langgraph[fastapi]>=0.0.35, so uv sync
failed with "No solution found" and the langgraph-fastapi smoke job went
red. The prior fix only bumped the production Dockerfile's
`uv pip install`, which the smoke path does not use.
Bump pyproject to ag-ui-langgraph[fastapi]==0.0.37 and regenerate uv.lock
(copilotkit 0.1.87->0.1.93, ag-ui-langgraph 0.0.34->0.0.37). Verified by
building docker/Dockerfile.agent locally: uv sync now resolves and the
agent boots ("Uvicorn running on 0.0.0.0:8123" / "Application startup
complete") serving HTTP 200 on /health.
Also reconcile parity: strands-python's entrypoint.sh legitimately diverges
from the langgraph-python north-star (it boots the strands agent via
`uv run python main.py` and serves Next via `next start`, not serve.py +
standalone server.js). Add entrypoint.sh to strands-python's
allowedDivergence, mirroring how langgraph-js already declares its own
custom entrypoint. parity:check now reports 0 errors.
The entrypoint was copy-pasted from langgraph-python: it logged
'langgraph-python starter', ran 'python serve.py' (no serve.py exists in this
starter), and started Next via standalone 'node server.js' (this image is a
non-standalone .next + node_modules build, so server.js does not exist either).
The container crash-looped on boot with
"python: can't open file '/app/serve.py'".
Run the Strands agent's self-serving agent/main.py via its uv venv
(AGENT_PORT=8123) and serve the frontend with 'next start'. Verified by booting
the image: agent reaches 'Application startup complete' on :8123 and the
container serves HTTP 200 on /.
copilotkit 0.1.93 re-exports StateStreamingMiddleware/StateItem FROM
ag_ui_langgraph.middlewares.state_streaming, which only exists in
ag-ui-langgraph>=0.0.35. The Dockerfiles installed copilotkit with --no-deps
and pinned ag-ui-langgraph[fastapi]==0.0.22, so the middlewares submodule was
missing and the agent crash-looped at boot with
'ModuleNotFoundError: No module named ag_ui_langgraph.middlewares'.
Bump the pin to 0.0.37 (the version copilotkit 0.1.93 resolves with deps).
Verified by booting the langgraph-python image: the agent now reaches
'Application startup complete' / 'Uvicorn running on 0.0.0.0:8123' with no
ImportError, and the container serves HTTP 200 on /.
The strands-python frontend imports `showcaseConfig from "../../showcase.json"`
(via src/hooks/use-example-suggestions.tsx) but the Dockerfile frontend stage
never copied showcase.json into the build context, so `next build` fails with
"Module not found: Can't resolve '../../showcase.json'". Add the COPY step,
mirroring the langgraph-js/langgraph-python Dockerfiles that already copy it.
The @copilotkit/react-core v2 pin (1.59.3) was already correct and matches all
working v2 starters; no package.json change was needed.
The build-starters Docker images hardcoded copilotkit==0.1.78 in the
langgraph-python and langgraph-fastapi Dockerfiles, but their agents import
StateStreamingMiddleware and StateItem, which 0.1.78 does not export. This
crash-loops the Python agent on boot with an ImportError. Bump the hardcoded
pin to 0.1.93 (latest stable, re-exports both symbols, matches local
sdk-python). Align all three langgraph-family agent pyproject pins
(0.1.87 -> 0.1.93) for consistency and regenerate the strands uv.lock.
The d6-all-pills-e2e probe header was sized for a legacy "8 services x 4
features" single-round fleet (timeout_ms: 600000 / 10 min). Discovery now
enumerates ~18 showcase demo services, so at max_concurrency: 8 the fan-out
runs ceil(18/8) = 3 serialized rounds. Rounds 2-3 start late and execute under
CPU contention, stretching each service toward ~200s and blowing the 10-min
per-service budget — all 18 services went red with `driver timeout after
600000ms`.
Raise the outer cap to 1200000 (20 min), matching the sibling 18-service
e2e-demos probe (same fleet, same budget). 3 rounds x ~200s fits comfortably.
Timeout-only change: max_concurrency stays at 8, so D6 peak stays at 32
contexts (under BROWSER_POOL_MAX_CONTEXTS=40, leaving 8 for the offset D5
tick). Header comment rewritten to document the real fleet size and the
fan-out math.
showcase_build.yml's existing detect-changes job shared the workflow_dispatch
`service` input with the new detect-starter-changes job, so dispatching
`service=starter-<slug>` tripped detect-changes's fail-loud `exit 1` ("did
not match any entry in ALL_SERVICES") and reddened the run even though
build-starters published fine. Scope the showcase fail-loud to non-`starter-*`
inputs via a case statement (mirroring how the starter job scopes its own
fail-loud) so a `starter-*` dispatch resolves to an empty showcase matrix and
SKIPS; typo'd showcase service names still fail loud.
Restore the 6h `schedule` cron in test_smoke-starter.yml. The PR had removed
it, leaving NO post-merge floating-dependency breakage detector for starters
— and its harness-probe replacement depends on S5 Railway services that don't
exist yet. Keep the cron until S5 starter-service probing is confirmed live.
The unmapped-starter aggregate emitted `total: 0` while `failed` listed all
four levels — a broken count invariant. Set `total: STARTER_LEVELS.length`
to match. Add the two missing red-green unit tests: the `aborted` errorClass
branch (already-aborted ctx.abortSignal → levels classified `aborted`, not
`transport-error`) and the makeSideEmit writer-throw resilience (a writer
whose write() rejects is swallowed at error-level and must not fail the
aggregate tick).
Only counts (12) were asserted on each side, so a future slug rename on
one side would silently flip a column to grey "not-supported" while both
counts stayed 12. Add a dashboard-side test that fs-reads the harness
STARTER_TO_COLUMN value set (the producer-side remap lives in a separate
pnpm workspace, so it cannot be imported) and asserts SET-EQUALITY with the
dashboard STARTER_COLUMNS. Mirrors the harness starter-mapping-drift fs-read
pattern; reds if either side renames a column slug without the other.
The starter_smoke probe runs hourly (`schedule: "40 * * * *"`), but
STARTER_STALE_AFTER_MS was set to 13h with a comment claiming a 6h cadence
— ~13 missed hourly ticks before amber, defeating the intended two-miss
flip. Re-derive the window from the 1h probe period to 2.5h: strictly
> 2 periods (so two consecutive misses, last row ~3h old, flip amber) yet
< 3h (so a single missed/slow-wake tick, last row ~2h old, stays green,
absorbing a scale-to-zero cold-start). Reconcile both the staleness.ts and
starter_smoke.yml comments to the same hourly basis + 2.5h window in
lockstep (no more "6h" in the starter context). Extend the staleness tests
with explicit hourly-tick boundaries (1 miss → green, 2 misses → amber).
Also drop the inaccurate "hollow" from the not-supported ✗ comments — there
is no hollow render variant; the state renders as grey ✗ text.
showcase_build.yml: add detect-starter-changes + build-starters jobs building
each of the 12 starters from examples/integrations/<slug>/Dockerfile via Depot
(--platform linux/amd64) and pushing ghcr.io/copilotkit/starter-<slug>:latest +
:<sha>. The starter- prefix is disjoint from showcase-* so harness discovery
stays clean.
test_smoke-starter.yml: drop the 6h schedule cron (live signal now comes from
the harness probing deployed Railway services + the harness alert path); keep
the examples/integrations/** PR build-sanity gate + offline aimock smoke run.
Workflows slot (S4) of the starter-row-group spec (model B). Railway
provisioning (S5) is intentionally out of scope, gated on cost approval.
Builds the dashboard UI for the starter-smoke dimension: resolveStarterRow
(flat starter:<col>/<level> lookup) + buildStarterBadge implementing the full
5-state cell vocabulary (green ✓, red ✗ smoke-failed, amber ~ stale, gray ?
not-yet-run, grey ✗ not-supported keyed off the S1 mapping), the StarterSection
row-group, rollup-exclusion so starter rows don't skew column aggregates, and a
13h staleness window (> 2× the probe cadence).
UI slot (S3) of the starter-row-group spec (model B).
Adds the `starter_smoke` probe driver + config/probes/starter_smoke.yml that
fans out per-starter HTTP health/agent/chat/interaction levels, registers it
in the orchestrator, and exempts it from probe-config parity (its matrix
shape differs from the depth-dimension probes). Includes 13 driver unit tests.
Probe slot (S2) of the starter-row-group spec (model B): harness HTTP-probes
the deployed (sleepable) Railway starter services.
Introduces the `starter` probe dimension in harness types and the single
source of truth for the starter-slug→dashboard-column-slug remap
(STARTER_TO_COLUMN, starterToColumnSlug, STARTER_LEVELS) plus a drift-lint
test asserting every smoke-matrix starter is mapped or explicitly excluded
and every mapped column slug resolves to a real manifest directory.
Foundation slot (S1) of the starter-row-group spec (model B).
Commit 7ac3e59a5 ("D6 all-pills probe driver") replaced the
`e2eParityDriver` registration (kind `e2e_parity`) with `e2eFullDriver`
(kind `e2e_d6`) in BOTH branches of `registerAllProbeDrivers`. After that
commit NO driver registers kind `e2e_parity`, yet
`config/probes/e2e-parity.yml` still shipped.
The probe-loader hard-rejects any YAML whose `kind` has no registered
driver (`no driver registered for kind 'e2e_parity'`), so `e2e-parity.yml`
failed to load on every boot and was never scheduled — silently dropping
that probe family. The D6/parity dashboard dimension is now produced by the
`e2e_d6` (all-pills) driver, which emits the identical
`d6:<slug>/<featureType>` rows the dashboard reads by key prefix (plus a
`d6:<slug>` aggregate), so `e2e_parity` is genuinely superseded, not just
temporarily unregistered.
Delete the orphaned config so the loader no longer rejects it and the full
probe-config set loads clean. Add a loader test that exercises the REAL
shipped `config/probes` set against the REAL driver registry and asserts
every YAML loads with zero `probes.reload.failed` errors — the existing
orchestrator guard only checks a hardcoded kind list and never read the
on-disk YAMLs, so it missed this orphan.
RACE3b's post-shutdown `liveContextCount === 0` assertion is tautological —
shutdown() sets liveContextCount to 0 unconditionally — so it proves nothing.
Drop it; the load-bearing assertions remain (acquire rejects with the shutdown
sentinel, contextToBrowser.size === 0, the orphan context was closed).
Correct the RACE3a/RACE3b comments that claimed shutdown() awaits the tracked
orphan close (reverted): the straddled orphan is closed by the orphan guard's
fire-and-forget close and never lands in contextToBrowser.
The empty-drain shutdown case (inFlightRecycles + pendingLaunches both
empty) exits the drain loop and resolves BEFORE a parked open settles and
registers its close via closeContextTracked(), so the "shutdown resolved =>
everything closed" guarantee was never actually delivered and had no
discriminating coverage. Revert the openContextOn orphan guard's close back
to main's accepted fire-and-forget `void this.closeContext(...)` and remove
the closeContextTracked() helper; inFlightRecycles is touched only by the
recycle / self-heal paths again.
Also unify the shutdown-condition error string: shutdown()'s queued-waiter
rejection now says "BrowserPool is shut down" to match the at-entry guard and
the straddle-reject legs (launch/relaunch-failure strings are a different
condition and unchanged).
Item 1 (red-green for the REACHABLE new shutdown branches): RACE1 only
covered the openContextOn orphan-guard `isShutdown` term. RACE3 adds
coverage for the two branches it did not:
- RACE3a: a shifted waiter whose serve open straddles shutdown must be
REJECTED by serveNextWaiter's post-open `if (this.isShutdown)` leg
(not re-enqueued onto the cleared queue, not hung to timeout).
RED with that guard reverted (waiter never settles), GREEN with it.
- RACE3b: an acquire whose open straddles shutdown must REJECT via
acquire()'s OUTER-catch `if (this.isShutdown) throw` leg. RED with
that guard reverted (the acquire enqueues on the cleared queue and
the test times out), GREEN with it. Both also assert shutdown()
awaited the orphan close (item 2) — the straddled context is closed
and the pool maps are empty post-shutdown.
Item 5: relabel RACE2's comment + title so it is framed as a regression
lock for the PRE-EXISTING generation-guard exactly-once-rollback
invariant (passes against main unchanged), NOT a shutdown-race proof.
Also extends the `internals()` test accessor with `waiters` so the new
tests can assert the queue is empty (no stranded caller).
Item 2 (reviewer-7 MEDIUM): the openContextOn orphan guard's
`void this.closeContext(...)` under shutdown was fire-and-forget and
tracked in NO set that shutdown() drains, so shutdown() could resolve
while a context close was still in flight — violating the implied
"shutdown resolved => everything closed" contract upheld elsewhere via
the pendingLaunches / inFlightRecycles drains. Route the orphan close
through a new closeContextTracked() that registers the close promise in
inFlightRecycles (the SAME set shutdown's drain loop re-snapshots until
empty) and self-removes on settle. Reuses the existing tracked-set
mechanism — no new machinery; outside shutdown the add/delete is a
no-op since only shutdown's drain loop reads the set.
Item 1 decision (retry-leg guard): KEEP the acquire() transient-retry
leg's `if (this.isShutdown) throw`. Contrary to a reviewer note that it
is unreachable, it guards a DISTINCT, genuinely-reachable straddle
window: the outer catch already observed isShutdown===false, we
re-reserved and re-opened, and the pool can shut down WHILE the retry's
open is in flight (the orphan guard then throws into this catch).
Without it the acquire enqueues onto the already-cleared waiter queue
and hangs until timeout. Comment updated to assert that reachability
rather than imply it merely "mirrors" the outer guard.
Items 3 + 4 (serveNextWaiter straddle): correct the inaccurate comment
that claimed shutdown() "already rejected this waiter" — it cannot, the
waiter was shift()ed off this.waiters BEFORE shutdown's reject-loop ran,
so THIS branch is the sole settler of a shifted-then-straddled waiter.
Unify the shutdown error string on the at-entry guard's existing
"BrowserPool is shut down" (was "shutting down" only here).
A serveNextWaiter()/openContextOn() that shifted a waiter + reserved a
slot BEFORE shutdown could settle its newContext() AFTER shutdown()'s
close-pass. shutdown() drains inFlightRecycles + pendingLaunches, but a
fire-and-forget serve open is tracked by neither, so the freshly-opened
context landed in contextToBrowser/liveContexts on a torn-down pool — a
leaked context that is never closed. openContextOn's orphan guard
checked generation/recycling/isConnected but NOT isShutdown.
Fix: add an isShutdown term to openContextOn's post-await orphan guard
(treat shutdown like a recycle — close the just-opened context, roll
back the reservation, throw); bail serveNextWaiter before openContextOn
when isShutdown; and reject (not re-queue) a shifted waiter / straddling
acquire when the open throws under shutdown, so it cannot strand on the
already-cleared waiter queue. Preserves all #5185/#5221 behaviors.
Adds red-green coverage for the leak (RACE1) and a regression guard for
the acquire transient-retry -> concurrent-recycle -> orphan-guard
straddle (RACE2), which was verified to keep its reservation accounting
balanced via the existing generation guard (no overshoot).
The integration-demo parity-check CI job was red on main: langgraph-js,
strands-python, and langgraph-fastapi diverged from the langgraph-python
north-star on two tracked verbatim files.
Root cause is the partial revert in 3721e7b36 (Revert of #5151), NOT #5222.
The v2 API migration (23af69041) moved north-star and all instances to the
v2 surface. The revert rolled the *instances'* example-layout/index.tsx and
docker-route-override.ts back to the v1 API (@copilotkit/react-core,
@copilotkit/runtime) while leaving the north-star on v2
(@copilotkit/react-core/v2, @copilotkit/runtime/v2). That left the demos
internally inconsistent: each instance's real src/app/api/copilotkit route
already uses runtime/v2 on copilotkit 1.59.3, but its Docker route override
and example layout were stuck on v1.
Re-baseline forward by syncing the three instances to the v2 north-star via
`pnpm parity:sync --all`. Only the 5 drifting verbatim files change; no
package.json keys move (all instances already pin 1.59.3) and no
agent-surface or allowed-divergence files are touched. `pnpm parity:check`
is now green (0 errors across all instances).
## Summary
The showcase Railway promote CI flow could not promote the fleet: an
`all` promote was blocked end-to-end whenever a single service was red,
and the reported failure traced to `showcase-ag2` being chronically red
on staging.
Root causes and fixes:
- **ag2 crash-on-import (the red service).** `gen_ui_agent.py` carried
`from __future__ import annotations`, which stringified the `set_steps`
tool's `context_variables: ContextVariables` parameter into an
unresolved `ForwardRef`. AG2's tool-schema generation then raised
`PydanticUserError` at import time, so the process never came up and the
staging healthcheck failed on every deploy since 2026-05-31. Removed the
import (matching the working sibling agents) and added a regression test
that statically asserts the future-import stays absent
(version-independent) plus a live import check.
- **Promote loop was all-or-nothing.** The per-service loop ran under
`set -euo pipefail`, so the first failing service aborted the whole
`all` promote, leaving the rest unpromoted. Extracted the loop into
`showcase/scripts/promote-fleet.sh`, which attempts every service,
accumulates succeeded/failed sets, exits non-zero only after attempting
all, and exports `succeeded_csv`.
- **`verify-prod` defeated the best-effort design.** It was skipped on
any non-zero promote and verified the full requested set. It now runs
`if: !cancelled()` and scopes `--services` to the succeeded set.
- **Staging precondition blocked the fleet.**
`verify-staging-precondition` failed the whole `all` promote when any
one service was staging-red. It is now advisory — `promote` runs
regardless, and `bin/railway`'s per-service P2/P3 staging-green gates
authoritatively refuse red services while green services promote.
`notify` success keys on PROMOTE && PROD.
- **Regression tests now gate in CI.** Added a `shell-script-tests` job
(bats + shellcheck) to `showcase_validate.yml`, plus input-validation
hardening in the script (fail-loud on empty / all-empty CSV,
`RAILWAY_BIN` executability check, whitespace trim).
## Test plan
- [x] ag2 regression test passes in the 3.12 venv (`PYTHONPATH=".:src"
pytest tests/python/`) — 2/2
- [x] `promote-fleet.bats` — 12/12 (best-effort loop, succeeded_csv
export, empty/whitespace/missing-binary guards, digest forwarding)
- [x] `shellcheck promote-fleet.sh` clean; `actionlint` clean on both
workflows
- [ ] CI green on this PR
## Summary
Adds an automated, **concise** `#engr` Slack notification when a
CopilotKit release publishes (or fails) — modeled on the
internal-skills→#engr release pings, but **one message per release**,
not one per package. The monorepo publishes ~15 npm packages + angular +
the Python SDK; naive per-package posts would be spam, so a single post
summarizes the whole release.
- **New `notify` job** in `publish-release.yml` — runs after both lanes
(`always()`), gated to real release contexts (a `workflow_dispatch` or a
*merged* PR). Posts exactly one message via the existing org-wide
`SLACK_WEBHOOK_ENGR` secret, if-guarded so an unset webhook never breaks
the step. Suppressed entirely for canaries (`mode=prerelease`) and
dry-runs.
- **Release intent is computed in the notify job from the `github.event`
payload** (dispatch inputs, merged `release/publish/*` head ref, and the
PR changed-files API for the Python SDK) — *independent* of the build
jobs. The build jobs emit their intent signals after failure-prone
steps, so a build that dies early on a genuine release would otherwise
leave the alert empty. Computing intent independently closes that
silent-swallow class. Failure arms **page on uncertainty** (fail toward
paging, never toward silence).
- **Pure, unit-tested message builder**
(`scripts/release/lib/build-release-notification.ts`) with a CLI wrapper
that serializes to `$GITHUB_OUTPUT` (random-delimiter heredoc, fail-loud
when it can't emit). Exhaustive truth-table coverage: npm/PyPI success +
failure arms, canary/dry-run suppression, cancelled-is-neutral,
empty-version/empty-scope graceful rendering, package-count
pluralization.
- **Self-watchdog:** if the notify job's own machinery fails, a
best-effort Slack post fires so the failure isn't invisible. The intent
step runs **first** (before checkout/install) so its outputs survive an
infra failure and the watchdog can still fire.
Publish/PyPI logic (the build/publish/build-python/publish-python jobs)
is untouched.
## Review
Converged through a 2-round, 7-agent unbiased code review (zero
mandatory in-scope findings remaining; the one in-scope finding — the
self-watchdog gating on a later step's output — was fixed by relocating
the intent step). Pre-existing publish-lane issues surfaced by reviewers
(prerelease canary verify-step, stable-tag retry idempotency,
canary→PyPI gating) are **outside this diff** and documented separately.
## Test plan
- [x] `vitest run` on the builder + wrapper suites — **85 tests green**
(builder 40, wrapper 27, versions 18)
- [x] `actionlint .github/workflows/publish-release.yml` — zero new
findings (4 pre-existing style findings in untouched jobs)
- [x] Both secret-using Slack steps if-guarded on `env.SLACK_WEBHOOK !=
''`
- [ ] CI green on the PR
- [ ] Observe a real `#engr` post on the next stable release
The starter had two App Router dirs: a stale top-level app/ containing only
an /api/copilotkit route (with a wrong graphId "starterAgent") and the real
src/app/ with page.tsx + layout.tsx. At copilotkit 1.59.3 the build silently
dropped the / route from src/app (masked by next.config ignoreBuildErrors),
producing only /api/copilotkit/[[...slug]]. The app served GET / as 404, so the
docker healthcheck (GET / === 200) never passed and the smoke leg failed with
the app container unhealthy before tests ran.
Removing the duplicate top-level app/ lets Next build src/app unambiguously,
restoring the / route. Local docker-compose.test.yml smoke: app healthy in ~5s,
4 passed (@health, @agent, @chat, @interaction).
Ben Taylor