- BoundedToolCallingManagerConfig: use ConcurrentHashMap with atomic
compute() for per-turn counter; short-circuit null ChatOptions to
avoid shared null-key contamination; clear counter on delegate
exception + log before rethrow; make MAX iterations configurable via
copilotkit.tool.max-iterations (@Value); rename to
MAX_TOOL_ITERATIONS_BEFORE_RETURN_DIRECT for unambiguous semantics;
pin Spring-AI version in Javadoc and document invariant.
- WebClientConfig: keep static initializer as defensive belt-and-
suspenders but document entrypoint.sh JVM arg as authoritative path;
warn when a non-zero keepalive timeout is already set.
- application.properties: fail fast when OPENAI_API_KEY is unset via
${OPENAI_API_KEY:?…} placeholder.
- entrypoint.sh: drop bash 5.1+ PID args from 'wait -n'; log which
child (java/node) exited with which code; bump health-probe timeout
30s -> 60s to cover cold-start JVM warmup.
- Add JUnit 5 tests (spring-boot-starter-test, test scope): first/Nth
call, fresh-options reset, null-options safety, counter eviction on
cap and exception, HTTP/1.1 connector pin.
Address CR findings against the strands showcase package:
agent_server.py
- Replace throwaway lambda with named _disabled_instrument that returns
self instead of None (fluent callers no longer AttributeError).
- Add module docstring + inline warnings explaining the strict import
ordering invariant (instrumentor patch must precede ag_ui_strands /
strands imports).
- Add runtime assertion that the patch is actually installed.
- Annotate each # noqa: E402 with the concrete reason.
- Note that no upstream strands-agents issue has been filed yet.
agents/agent.py
- Wrap all module-level side effects (model init, agent construction,
per-thread dict patching) in build_showcase_agent() so import
failures are localized.
- Fail fast with RuntimeError when OPENAI_API_KEY is unset rather than
accepting an empty string silently.
- _ToolCallCapHook: protect _count with threading.Lock, document the
single-thread-at-a-time ag_ui_strands concurrency model, and emit a
logger.warning when the cap trips so operators see loops.
- _HookInjectingAgentDict: override update(), setdefault(), __ior__,
and __or__ in addition to __setitem__ so CPython's bulk-update C
paths (which bypass __setitem__) still route through hook injection.
- Preserve any pre-existing entries in _agents_by_thread when swapping
in the injecting dict (copy into the new dict first).
- Guard against double-injection on re-insert: before adding a cap
hook, check whether the Agent already has one attached.
- Narrow sales_state_from_args exception handler from bare Exception to
(json.JSONDecodeError, AttributeError, TypeError) and log a truncated
excerpt of the offending tool input at WARNING.
- Document that schedule_meeting duration is intentionally defaulted
in the showcase.
- Remove unused BaseModel / Field pydantic imports.
- Replace ASCII banner comments with section headers.
tests/python/
- New pytest suite exercising the cap hook counter semantics (fires at
max+1, resets on BeforeInvocationEvent, stop_event_loop sentinel),
the hook-injection dict (all mutation paths inject; existing entries
preserved; no double-injection on re-insert), and the threading
instrumentor patch (returns self, accepts arbitrary args, actually
replaces the method, agent_server module installs the patch).
- conftest.py stubs out strands / ag_ui_strands / uvicorn / dotenv so
the suite runs in environments where the heavy runtime deps aren't
installed.
- agui_adapter: compute thread_id once so RUN_STARTED/RUN_FINISHED never
disagree when the caller omits it.
- Narrow silent `except Exception: pass` sites, log parse/tool failures
with sanitized context, and drop the dead `agent.agent_response` path
(returns ChatDocument, not ToolMessage).
- Extract `_emit_text_block` helper DRYing the empty-delta guard, and a
`_parse_tool_args` helper with explicit warning on malformed input.
- Assert tool-class request-name uniqueness at import so collisions fail
loudly instead of silently shadowing.
- Replace `str(response)` fallback with documented empty default.
- Add unit tests covering the OAI tool-calls happy path, malformed args,
legacy function_call shape, empty-content skip, thread_id stability,
and tool-class uniqueness (red-green verified).
Spring-AI ignores the showcase-wide OPENAI_BASE_URL because its split
base-url + completions-path model would produce a double-/v1 against
aimock. Introduce SPRING_AI_OPENAI_BASE_URL carrying the host without
/v1 (e.g. http://aimock:4010), default to https://api.openai.com.
Additionally, pin WebClient's JDK HttpClient to HTTP/1.1 and disable
connection pooling:
- Without reactor-netty, Spring-AI's auto-configured WebClient sent
`Upgrade: h2c` on every cleartext request, which some fixtures (aimock)
reject with 404.
- Default JDK HttpClient pooling reused half-closed sockets between
fixture responses, tripping `Connection reset` on tool-result follow-ups.
L3 chat now passes locally against the stack. L4 tool-rendering still
needs additional work (tool-execution loop cap WIP in BoundedToolCallingManagerConfig).
The LLM could enter an infinite get_weather tool-call loop (300+ calls per
invocation) because strands has no upstream max-iterations knob, and
aimock's fuzzy fixture matching returns the same tool_call response
whenever the last user message contains the matched text. Direct OpenAI
usage can exhibit similar fixation under certain prompts.
Mitigations in agents/agent.py:
1. _ToolCallCapHook (HookProvider): counts tool calls per Agent
invocation (reset on BeforeInvocationEvent). On BeforeToolCallEvent
past the cap, sets event.cancel_tool so the tool returns a benign
error result. On AfterToolCallEvent past the cap, sets
invocation_state['request_state']['stop_event_loop'] = True which
strands' event_loop checks to terminate recursion.
2. ToolBehavior(stop_streaming_after_result=True) for get_weather: the
frontend weather card IS the response, so halting after the first
tool result also short-circuits any would-be recursion for the
tool-rendering demo.
3. _HookInjectingAgentDict: ag_ui_strands spawns a fresh Agent per
thread_id from the template WITHOUT copying hooks. Subclass the
per-thread dict and attach a fresh _ToolCallCapHook (stateful) on
every agent insertion.
Verified: L3 and L4 strands smoke tests pass 3x locally.
Langroid's OpenAI-backed LLM returns tool calls on
`response.oai_tool_calls` (OpenAI tools API) or `response.function_call`
(legacy function-calling API) with an empty `response.content`. The prior
adapter only inspected `response.content`, so pure tool-call turns
produced no TOOL_CALL events, the frontend never rendered the weather
card, and L4 (tool-rendering) smoke timed out waiting for an assistant
message.
Synthesize TOOL_CALL_START / TOOL_CALL_ARGS / TOOL_CALL_END events from
`oai_tool_calls` (and `function_call` as a fallback), preserving the
OpenAI-assigned call id and JSON-encoded arguments. For backend tools,
instantiate the matching ToolMessage subclass, execute its `.handle()`
off-thread, and stream the result back as a text message (skipping empty
deltas to stay schema-compliant). Frontend tool calls are forwarded as
events only, letting CopilotKit's client render them.
strands-agents 1.35.0 unconditionally calls ThreadingInstrumentor().instrument()
when constructing its Tracer (strands/telemetry/tracer.py). Combined with the
async model client dispatching work onto ThreadPoolExecutor, this wraps
ThreadPoolExecutor.submit such that it re-enters itself recursively, producing
RecursionError during tool-rendering requests and surfacing as an OpenAI
APIConnectionError.
Setting OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=threading does not help because
strands imports and instruments the class directly, bypassing the entry_point
autoloader. Monkey-patch the instrument() method to a no-op before strands is
imported.
Gemini 2.5-flash, given the SalesPipelineAgent's sales-heavy system prompt,
intermittently returned an empty response (FinishReason.STOP with no content
parts) for weather queries — about 70% of requests, causing the L4
tool-rendering smoke test to time out waiting for the assistant message.
Root causes and fixes:
1. System prompt priming: WEATHER guidance was buried after SALES TODOS.
Reordered to lead with a direct weather instruction ("When the user
asks about the weather, call the get_weather tool") and added an
explicit "ALWAYS provide a textual response after any tool call"
guard. The model now reliably invokes get_weather and emits a summary.
2. after_model_modifier over-termination: the old check ended the
invocation whenever parts[0].text was truthy, which fired on
streaming partial events and on mixed text+function_call responses,
stranding pending tool calls. Now skip partial events, and only
terminate when the response is role=model AND has text AND has no
pending function_call.
3. Added ADK_DISABLE_PROGRESSIVE_SSE_STREAMING=1 in entrypoint.sh to
suppress the ADK's progressive-SSE path, which aggregates cleaner
final events (removes the 'The last event is partial' warnings we
were seeing on every run).
Verified: 10/10 direct backend calls now return TOOL_CALL + TEXT_MESSAGE,
5/5 repeated L4 playwright runs pass, and L3 (agentic-chat /Hello/) still
passes.
SharedStateAgent forced ResponseFormat=json-schema for every request whenever
ag_ui_state was present in the options (which it always is for CopilotKit-
wrapped demos). For plain chat prompts like 'hello' the model dutifully
produced a SalesStateSnapshot shell that failed TryDeserialize, the code
hit yield break, and nothing was streamed to the client -- Playwright's L3
smoke waited 60s for [data-testid=copilot-assistant-message] and timed out.
Bypass the two-pass flow (run the agent normally) unless ag_ui_state.todos
is a non-empty array, i.e. there's actual sales pipeline data to sync. The
two-pass logic still runs for sales-pipeline demos that populate todos.
The AG-UI adapter was emitting TextMessageContentEvent with delta=""
whenever the Langroid response had empty content (e.g. a pure tool-call
turn) or a backend tool handler returned nothing. ag_ui.core validates
delta as min-length 1, so the event construction crashed with a pydantic
ValidationError and the SSE stream blew up mid-run, taking out the
tool-rendering L4 smoke test.
Guard both emission sites by skipping the entire
TEXT_MESSAGE_START/CONTENT/END block when the delta would be empty,
rather than padding with whitespace (which would leak a spurious blank
assistant bubble into the UI).
## Summary
Adds a one-command local smoke harness so the full 17-integration suite
can be exercised against Docker on the dev machine instead of Railway.
Useful when Railway is degraded (aimock OOM, rate limits, cold-start
drift) or when validating changes that haven't been deployed yet.
## Usage
```bash
# one-time
cp showcase/.env.example showcase/.env # fill in keys
pnpm --filter @showcase/e2e-smoke install
# full L1-L4 smoke
pnpm --filter @showcase/e2e-smoke smoke:local
# single level / keep containers up between runs
pnpm --filter @showcase/e2e-smoke smoke:local:L1
pnpm --filter @showcase/e2e-smoke smoke:local:keep
pnpm --filter @showcase/e2e-smoke smoke:local:nobuild
```
## What's in here
- **`docker-compose.local.yml`**: `aimock` added as 18th service →
integration containers reach `http://aimock:4010` on the compose
network, mirroring Railway's `showcase-aimock`.
- **`integration-smoke.spec.ts`**: `LOCAL_PORTS=1` env gates URL
rewriting from `https://showcase-<slug>-production.up.railway.app` →
`http://localhost:<port>` via `shared/local-ports.json`. Starters are
skipped under the flag because they're not in `local-ports.json`.
- **`scripts/smoke-local.sh`**: thin orchestrator — `build → up → wait
20s → playwright → down`. Flags: `--level=L1|L2|L3|L4`, `--keep`,
`--no-build`.
- **`tests/package.json`**: `pnpm smoke:local[:L1|:keep|:nobuild]`
wrappers.
- **`.env.example`**: documents optional
`OPENAI_BASE_URL`/`ANTHROPIC_BASE_URL` + `GitHubToken` (ms-agent-dotnet)
and `GOOGLE_API_KEY` (google-adk).
## Verification
Run locally against a fresh checkout of this branch:
- `LOCAL_PORTS=1 SMOKE_ALL=true npx playwright test integration-smoke
--grep @health` → **17/17 pass in 478ms**
- Full L1-L4 against the local stack → **42/51 pass** (9 failures in
L3/L4 for mastra, google-adk, ms-agent-dotnet, strands, langroid,
spring-ai — these are test-data / fixture gaps unrelated to this
infrastructure and will be filed separately)
- `docker compose -f showcase/docker-compose.local.yml config` validates
with 18 services
## Scope
Pure dev-ergonomics addition. No runtime behaviour changes in the
shipped containers. `LOCAL_PORTS` is opt-in; unset = existing
Railway-URL behaviour preserved.
## Test plan
- [ ] `Validate Showcase` CI still green (no package-source changes)
- [ ] No unrelated CI regressions
- [ ] Follow-up PR will investigate and fix the 9 L3/L4 failures
surfaced by local smoke
<!--
Thank you for sending the PR! We appreciate you spending the time to
work on these changes.
Help us understand your motivation by explaining why you decided to make
this change.
**Please PLEASE reach out to us first before starting any significant
work on new or existing features.**
By the time you've gotten here, you're looking at creating a pull
request so hopefully we're not too late.
We love community contributions! That said, we want to make sure we're
all on the same page before you start.
Investing a lot of time and effort just to find out it doesn't align
with the upstream project feels awful, and we don't want that to happen.
It also helps to make sure the work you're planning isn't already in
progress.
As described in our contributing guide, please file an issue first:
https://github.com/ag-ui-protocol/ag-ui/issues
Or, reach out to us on Discord: https://discord.com/invite/6dffbvGU3D
You can learn more about contributing to copilotkit here:
https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md
Happy contributing!
-->
## What does this PR do?
(Describe the changes introduced in this PR)
## Related PRs and Issues
- (Direct link to related PR or issue, if relevant)
## Checklist
- [ ] I have read the [Contribution
Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [ ] If the PR changes or adds functionality, I have updated the
relevant documentation
## Problem
vitest 3.2.4's hardcoded 60s birpc `onTaskUpdate` timeout
([vitest-dev/vitest#6129](https://github.com/vitest-dev/vitest/issues/6129))
was tripping `unit(20.x/22.x/24.x)` on CopilotKit showcase-scripts
tests. Root driver: cumulative RPC back-pressure from serial file
execution plus two oversized test files (`validate-pins.test.ts` 50.6s,
`audit.test.ts` 71.4s) individually approaching/exceeding the cliff.
Deterministic failure on #4068/#4018/#4079.
## Fix
- Split `validate-pins.test.ts` into 5 files (`cli`, `eacces`,
`parsers`, `r-scenarios`, `validate-all`) plus shared helpers. 50.6s →
12.8s.
- Split `audit.test.ts` into 3 files (`unit`, `audit-package`, `cli`)
plus shared helpers. 71.4s → ~16s.
- Isolated `bundle-demo-content.test.ts` / `generate-registry.test.ts` /
`create-integration.test.ts` from `.git/index.lock` races by serializing
every `restoreFromGitHead` invocation through a cross-process advisory
lock in `test-cleanup.ts`.
- Isolated `create-integration.test.ts` from the real
`showcase/packages/` tree. The generator now honors
`CREATE_INTEGRATION_{PACKAGES,WORKFLOWS}_DIR` env overrides; the test
runs entirely inside a per-suite `os.tmpdir()`.
- Flipped `fileParallelism: true` — safe now that `pool: 'forks'` gives
every file its own process AND FS isolation is in place.
## Results
- Full suite: 158s → 12s locally, 1061/1061 passing across 3 consecutive
`pnpm nx run @copilotkit/showcase-scripts:test --skip-nx-cache` runs.
- Zero `onTaskUpdate` timeouts, zero ENOENT, zero `index.lock`
contention across all 3 runs.
- Every parallel file gets a fresh 60s birpc budget, eliminating the
cumulative back-pressure cliff.
## Test plan
- [ ] CI green on `unit(20.x)`, `unit(22.x)`, `unit(24.x)`.
- [ ] Manual: `pnpm nx run @copilotkit/showcase-scripts:test
--skip-nx-cache` three times in a row — all 1061/1061, no timeout
warnings, no tree drift.
- test-cleanup.ts: `new Error(msg, { cause })` is ES2022; workspace lib is
ES2020 so the two-arg overload is missing. Replaced with an
`errorWithCause()` helper that assigns `.cause` after construction.
Runtime is identical (Node >=16.9); only the TS signature differs.
- test-cleanup.ts: retyped `SAFE_STDIO` as `StdioOptions` (still frozen at
runtime to keep `test-cleanup.test.ts` freeze assertion green) so
spreading `SAFE_EXEC_OPTS` into `execFileSync(..., opts)` no longer trips
the readonly-vs-mutable-array mismatch on `stdio` (fixes the error at
create-integration.test.ts:132).
- create-integration/index.ts: dropped unused `devCmd` local and unused
`args` parameter on `generateDemoPage` (+ call site); both were dead code
introduced during the parallel-isolation refactor.
- validate-pins.parsers.test.ts: annotated all `withTmp((tmp) => ...)`
callbacks as `(tmp: string)` for robustness under LSP module-resolution
glitches. Matches the contract in `validate-pins.shared.ts`.
Tests: 1061/1061 pass (`pnpm nx run @copilotkit/showcase-scripts:test`).
Prior reasons for `fileParallelism: false` are resolved:
- Env-var mutation races (VALIDATE_PINS_REPO_ROOT, SHOWCASE_AUDIT_ROOT,
VALIDATE_PARITY_REPO_ROOT) are moot under `pool: 'forks'` — every file
already gets its own node process with its own `process.env`.
- `.git/index.lock` races between suites that call `restoreFromGitHead`
are fixed by the cross-process lock in test-cleanup.ts.
- The create-integration vs generate-registry collision on
`showcase/packages/` is fixed by the create-integration tmpdir
isolation.
Under fork-per-file + parallel, each test file also gets a fresh 60s
birpc `onTaskUpdate` budget (vitest #6129), eliminating the cumulative
RPC back-pressure that tripped unit(20.x/22.x/24.x) on #4018/#4068/#4079.
Empirical: local full suite 158s → 12s, 1061/1061 passing across three
consecutive `--skip-nx-cache` runs with zero timeouts, zero ENOENT, zero
index.lock contention.
audit.test.ts is 3034 lines / 119 tests and on Node 22 CI its single-file
runtime grew from 36.7s (PR #4071) to 71.4s (PR #4081) — over the hardcoded
60s birpc onTaskUpdate RPC window (vitest #6129). Same cliff that motivated
the validate-pins split earlier in this PR.
Extract makeTmpTree, makeConfig, writePackage, makeExampleDir, anomalyStrings,
and the AUDIT_SCRIPT path constant into audit.shared.ts so the forthcoming
split files can share them without duplication. No behavior change — the
original audit.test.ts still re-declares its own local copies until the
split commit removes them.
Break the 3567-line validate-pins.test.ts into five smaller test files so
each one fits comfortably under vitest's hardcoded 60s birpc onTaskUpdate
RPC window (upstream vitest #6129: DEFAULT_TIMEOUT = 6e4 in the bundled
birpc). pool: 'forks' + fileParallelism: false already gives each file a
fresh 60s RPC budget; splitting ensures no single file is anywhere near
that cliff even when CI is slow.
Split buckets, chosen for logical cohesion and balanced subprocess load:
- validate-pins.parsers.test.ts - pure parsers (no validateAll, no subprocess)
- validate-pins.validate-all.test.ts - in-process validateAll + drift detection
- validate-pins.cli.test.ts - CLI subprocess exit codes
- validate-pins.eacces.test.ts - chmod/EACCES-routed subprocess tests
- validate-pins.r-scenarios.test.ts - R29/R33 regression scenarios
Test count is identical pre/post-split (134 tests). Behaviour-neutral
refactor: no test logic changes, only file boundaries + shared helper
imports.
Hoist tmpdir/write/withTmp helpers and FIXTURES_DIR/VALIDATE_PINS_SCRIPT
path constants out of validate-pins.test.ts so the forthcoming split files
can share them without duplication. Behaviour-neutral.
create-integration.test.ts invoked the real generator against
`showcase/packages/` and `.github/workflows/`, then healed the mutations
in afterEach. Under `fileParallelism: true` that collided with
generate-registry.test.ts (concurrent readdirSync of `showcase/packages/`
observed partial state → ENOENT) and with every suite that restored
workflow YAMLs from git (`.git/index.lock` contention).
Teach `create-integration/index.ts` to honor two env overrides —
`CREATE_INTEGRATION_PACKAGES_DIR` and `CREATE_INTEGRATION_WORKFLOWS_DIR`
— that redirect its writes to any directory. Production behavior
unchanged (defaults resolve to the real paths as before).
Rewrite the test to create a per-suite `os.tmpdir()`-backed root, seed
it with copies of the three real workflow YAMLs so the generator's
regex-based edits still match, and point both env vars there. The test
now never mutates a tracked file — no restorer, no git invocation, no
cross-suite shared state. generate-registry can scan real
`showcase/packages/` concurrently without collisions.
The regression-guard test still exercises the same cleanup semantics,
just against the tmpdir-backed baseline map instead of
`workflowRestorer.snapshotMap`.
`restoreFromGitHead` runs `git checkout HEAD -- <paths>` inside three
sibling suites (bundle-demo-content, generate-registry, create-integration)
plus concurrent `git` from the pre-commit hook. Every one of those grabs
`.git/index.lock` — parallel callers race for it and flake with
"fatal: Unable to create .git/index.lock: File exists".
Acquire a cross-process advisory lock (atomic `fs.mkdirSync` of
`/tmp/copilotkit-showcase-git-restore.lock`) around every git invocation
in this module: partition, pre-heal diff, checkout, post-heal diff. Held
for the entire sequence so intermediate state is consistent from the
caller's perspective. Stale locks (> 60s) are reaped before the wait loop
so a hard-killed previous run can't wedge subsequent runs.
Unblocks enabling vitest `fileParallelism: true` — the three consumer
suites can now run in parallel forks without stepping on each other's
git operations or on the pre-commit hook's.
## Problem
`showcase/scripts` test runs intermittently fail with birpc
`onTaskUpdate` timeouts, caused by an upstream bug in vitest 3.x's
subprocess reporter transport
([vitest-dev/vitest#8164](https://github.com/vitest-dev/vitest/issues/8164)).
Fix landed upstream in
[#8297](https://github.com/vitest-dev/vitest/pull/8297), shipped in
v4.0.0-beta.4+.
## Why this is safe
- **Blast radius is this one package.** Root `package.json` already
declares `"vitest": "^4.1.3"`. The only holdout was
`showcase/scripts/package.json` at `"vitest": "^3.0.0"`. All 10
subprocess-spawning test files in the monorepo live in
`showcase/scripts/__tests__/` — zero elsewhere — so this is the only
package actually affected by the bug.
- **Zero vitest API breaks in the diff.** Spot-checked: no `vi.mock`, no
`expect.extend`, no `.concurrent`, no snapshots, no coverage config, no
custom reporters. Only `vi.spyOn` (unchanged in v4) and standard
`describe`/`it`/`expect`.
- **Pool config unchanged.** `pool: 'forks'` + `fileParallelism` behave
identically in v3 and v4.
- **Lockfile diff is minimal.** Only `showcase/scripts` import
resolution changes + a new `@vitest/mocker@4.1.3` snapshot. No other
package was edited; no other package's resolution changed. vitest 3.2.4
and 2.1.9 continue to resolve side-by-side for the packages that still
pin them.
## Verification
3 consecutive `pnpm nx run @copilotkit/showcase-scripts:test
--skip-nx-cache` runs on Node 20.20.2:
| Run | Result | Wall time | Tests |
|-----|--------|-----------|-------|
| 1 | green | 34.45s | 1061/1061 |
| 2 | green | 29.64s | 1061/1061 |
| 3 | green | 29.46s | 1061/1061 |
Zero `onTaskUpdate` errors, zero unhandled exceptions across all three
runs.
## Test plan
- [ ] CI unit(20.x) green — this has been the red one on recent PRs due
to this exact bug
- [ ] CI unit(22.x) green
- [ ] CI unit(24.x) green
## Relationship to #4081
Complementary, not overlapping. #4081 splits / isolates the
showcase-scripts test suite as a workaround at the Nx/pool-config layer.
This PR fixes the root cause in the test runner itself. Once this
merges, the `onTaskUpdate` failures #4081 has been chasing should clear;
#4081 can then rebase on a clean main and its remaining work (the
split/isolation improvements) can be evaluated on its own merits rather
than as a bug workaround.
Fixes upstream birpc onTaskUpdate timeout (vitest-dev/vitest#8164, fixed
by #8297, v4-only). Root package.json already on ^4.1.3; this aligns
showcase/scripts, which held the only remaining ^3.0.0 pin and was
therefore the only package affected by the bug.
Local verification: 3 consecutive `pnpm nx run
@copilotkit/showcase-scripts:test --skip-nx-cache` runs on Node 20.20.2,
all green, 1061/1061 passing, zero onTaskUpdate errors, zero unhandled
exceptions (wall times: 34s / 30s / 29s).
## Why
CopilotKit has an unlimited Depot Startup plan, and standard GitHub
runners
(`ubuntu-latest`) have been amplifying the vitest birpc `onTaskUpdate`
timeout bug on subprocess-heavy unit-test suites (see #4081). Depot
runners
give us consistent runner quality and persistent caches for the test
workflows, matching what we already do on the showcase side.
## What
Flipped 5 workflows from `runs-on: ubuntu-latest` to
`runs-on: depot-ubuntu-24.04-4`, one commit per workflow:
- `.github/workflows/test_unit.yml`
- `.github/workflows/test_unit-python-sdk.yml`
- `.github/workflows/e2e_examples.yml`
- `.github/workflows/test_doc-examples.yml`
- `.github/workflows/test_runtime-servers.yml`
Each workflow also gets a top-level `permissions:` block with
`contents: read` + `id-token: write`, matching the pattern established
in
`showcase_validate.yml`. `id-token: write` is required for Depot OIDC
auth.
No test steps, timeouts, caching, or step order changed — the migration
is
strictly a runner flip + permissions block addition.
4-core (`depot-ubuntu-24.04-4`) was chosen to match the existing
precedent
in `showcase_validate.yml` / `showcase_drift-report.yml`. None of the
migrated jobs configure `--max-workers` or similar flags that would
justify
an 8-core runner; the win here is runner quality, not raw core count.
## Reference
Extends the pattern established by #4018, which shipped
`showcase_validate.yml` and `showcase_drift-report.yml` on Depot runners
earlier today.
## Verification
- `actionlint` diff-clean: all warnings on touched files are
pre-existing
(shellcheck info on unrelated lines) or the expected "unknown label"
warning on the Depot custom runner label — identical to the accepted
state of `showcase_validate.yml` on main.
- Depot OIDC permission block matches the pattern used in
`showcase_validate.yml`.
## Test plan
- [ ] CI green on this PR's own `unit (20.x / 22.x / 24.x)` matrix
- [ ] No new Depot OIDC errors in run logs
- [ ] `onTaskUpdate` timeout from #4081 no longer fires on the Depot
runs
Depot Startup plan (unlimited minutes) for consistent runner quality
across test workflows. Bun setup action is runner-agnostic on linux.
Extends the pattern from PR #4018.
Also adds the id-token: write permission required for Depot OIDC
auth, alongside contents: read for least-privilege defaults.
Depot Startup plan (unlimited minutes) for consistent runner quality
across test workflows. Extends the pattern from PR #4018.
Also adds the id-token: write permission required for Depot OIDC
auth, alongside contents: read for least-privilege defaults.
Depot Startup plan (unlimited minutes) for consistent runner quality
across test workflows. Playwright/chromium install is runner-agnostic
on linux. Extends the pattern from PR #4018.
Also adds the id-token: write permission required for Depot OIDC
auth, alongside contents: read for least-privilege defaults.
Depot Startup plan (unlimited minutes) for consistent runner quality
across unit-test workflows. Extends the pattern from PR #4018.
Also adds the id-token: write permission required for Depot OIDC
auth, alongside contents: read for least-privilege defaults.
Depot Startup plan (unlimited minutes) resolves the vitest birpc
onTaskUpdate timeouts on subprocess-heavy suites that standard
ubuntu-latest runners were amplifying. Extends the pattern from
PR #4018 (showcase_validate.yml, showcase_drift-report.yml).
Also adds the id-token: write permission required for Depot OIDC
auth, alongside contents: read for least-privilege defaults.
Pick up the split generative-ui categories, new cells (chat-customization-css,
tool-rendering-default/custom-catchall, tool-rendering-frontend-tools), and
the flat `files` shape (backend files merged in from manifest highlight:).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Shell /code viewer now builds a recursive file tree with core-only
(★ highlighted) and show-all-files toggle via ?view=all; collapses the
legacy flat files + backend_files arrays into one tree
- bundle-demo-content: strict mode — errors on missing highlight paths;
drop backend_files field; pull in external backend files referenced
by highlight: (column-relative paths) alongside demo-folder contents;
stable page-first ordering
- Update tests to reflect new column-relative filename shape
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- tool-rendering: per-tool WeatherCard + FlightListCard + wildcard
CustomCatchallRenderer (primary variant of 3)
- NEW tool-rendering-default-catchall: no frontend renderers, relies on
CopilotKit's built-in default catch-all
- NEW tool-rendering-custom-catchall: single wildcard renderer via
useDefaultRenderTool
- NEW tool-rendering-frontend-tools: useFrontendTool defines get_weather
client-side; render callback uses `args` (not `parameters`)
- NEW chat-customization-css cell: page.tsx + scoped theme.css
- hitl-in-chat: time-picker-card.tsx + page.tsx using useHumanInTheLoop
with book_call + topic/attendee args
- gen-ui-interrupt: time-picker-card.tsx + page.tsx using useInterrupt
against the schedule_meeting interrupt payload
- a2ui-fixed-schema: update page.tsx comment to point at new backend
- Delete old hitl/ folder
- Update api/copilotkit/route.ts to register new agent names
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- a2ui_fixed: load schemas from JSON via a2ui.load_schema (schemas in
src/agents/a2ui_schemas/); adds search_flight tool with airline + price
- a2ui_dynamic: secondary-LLM bind_tools([render_a2ui], tool_choice=...)
pattern; primary LLM only decides when to render
- tool_rendering_agent: expand to 4 mock tools (get_weather, search_flights,
get_stock_price, roll_dice) — shared by all 3 tool-rendering variants
- interrupt_agent: port schedule_meeting(topic, attendee) time-picker
interrupt payload
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Split generative-ui category into 4 (controlled/declarative/open/
operational), add chat-customization-css + tool-rendering-frontend-tools,
replace old tool-rendering-status/-result IDs with default-catchall /
custom-catchall. Port langgraph-python manifest features + highlight:
paths (column-relative to preserve pre-existing Docker structure).
Update tests for new category/feature counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NIT 3: collapse the per-leg jq pipeline in the notify step. The
previous version went `.jobs // [] | map(filter) | map(project) | .[]`
— three full passes over the list plus a flatten. Rewrite as a single
`.jobs[]? | select | "…"` stream: functionally identical, noticeably
easier to skim. No observable output change.
MEDIUM 3: keep the template/ exclusion cross-referenced in both places
it is enforced — validate-workflow-starters.ts EXCLUDED_DIRS and
showcase_smoke-monitor.yml's per-slug continue. If a future non-service
sibling (e.g. shared/, docs/) gets added, the pair of comments makes
it obvious that both sites need the new entry.
`cut -c1-200` truncated mid-slug, producing reader-hostile output
like `starter-claude-sdk-pyth` or `succeeded: starter-cl`. Replace
with a `truncate_csv` helper that iterates items, appends until the
next item would blow the budget, then emits `…` to signal truncation.
Applied to three list-cap sites (services summary, succeeded_list,
failed_list). Falls back to a hard character cut only when the very
first item is already over budget, so a single pathological 250-char
slug still emits something rather than an empty string.
Helper is inlined per step because YAML run: blocks don't share shell
functions. Comment cross-references steps.legs so the two copies stay
recognisably linked.
Replace the regex-over-YAML options-block scanner with a real YAML
parse + typed navigation down on.workflow_dispatch.inputs.service.
options. The previous regex depended on a brittle terminator
(`^\S | \n\s*\w+:\s*\n\s{6,}\w+:|\nconcurrency:`) that
would silently break under trivial reformats — adding a top-level key
after `on:` or reordering `concurrency:` would have produced a
false-positive pass.
ALL_SERVICES stays regex-scanned because its matrix is embedded as a
bash heredoc inside a run: step, not as a YAML sub-structure. We do
locate the step via YAML (jobs.detect-changes.steps[*] with run body
referencing ALL_SERVICES) and only then scan `"dispatch_name":"X"`
occurrences inside that step's run body. Cannot JSON.parse the matrix
directly — it interpolates ${{ github.sha }} expressions that aren't
valid JSON pre-execution.
NIT 2: drop the dual `import.meta.url === \`file://${argv[1]}\``
branch and keep only the canonical `fileURLToPath(import.meta.url)`
comparison. The file:// form was belt-and-suspenders for a
pre-fileURLToPath era of Node; Node 20+ handles the modern form
uniformly.
Verified:
- All 12 validator tests pass (substring-spoof, missing-from-both,
template-excluded, empty-starters, missing-workflow).
- Real showcase_deploy.yml returns OK: all 17 starter(s) registered.
Every peer validator under showcase/scripts/ already has a matching
__tests__ suite — this one shipped without. Model the new suite on
validate-parity.test.ts: per-test tmpdir fixtures + CLI subprocess
runs gated by VALIDATE_WORKFLOW_STARTERS_REPO_ROOT.
Extend the existing env-var override to re-home STARTERS_DIR as well
(previously it only redirected .github/). One env var, one root,
matches the pattern used by the other validators.
Coverage:
- happy path (all slugs registered, exit 0)
- slug missing from workflow_dispatch options only (exit 1)
- slug missing from ALL_SERVICES matrix only (exit 1)
- slug missing from both (exit 1, both sources named)
- empty showcase/starters/ (exit 3, refuses trivial pass)
- template/ excluded (not flagged as missing)
- substring-spoof: starter-ag2 missing vs starter-ag2-extended present
— regression guard for the word-boundary anchor
- showcase_deploy.yml absent (exit 3)
All 12 specs pass against the existing regex-based checks. The
substring-spoof case in particular pins the word-boundary contract so
a later refactor can't silently lose it.
Jordan Ritter