Two bugs introduced in 534cd1efa (D5 integration fixes) when the
langroid adapter switched backend tool results from TEXT_MESSAGE_*
triples to TOOL_CALL_RESULT events:
1. Two leftover `logger.warning("DEBUG ...")` calls in agui_adapter.py
that fired on every chat turn — exactly the noise pattern the
`test_plain_text_turn_does_not_warn` test was written to prevent.
2. Three stale tests in test_agui_adapter.py (only run on 3.11+, so
the failure was 3.12-only):
- test_backend_tool_execution_happy_path: still expected the old
TEXT_MESSAGE_START/CONTENT/END inner triple instead of a single
TOOL_CALL_RESULT event after TOOL_CALL_END.
- test_backend_tool_exception_returns_sanitized_error: still read
the sanitized error JSON from TEXT_MESSAGE_CONTENT.delta — it
now rides on TOOL_CALL_RESULT.content.
- test_plain_text_turn_does_not_warn: caught the DEBUG leak above.
Tests are skipped on Python 3.10 (langroid imports `typing.Self`),
which is why this only shows up on the 3.12 matrix entry.
Both BYOC demo pills were getting eaten by older generic substring
fixtures ("dashboard", "revenue by category as a pie chart", "monthly
expenses as a bar chart") — the json-render pills came back as foreign
tool calls (render_pie_chart) and the hashbrown sales pill came back as
the "Working on your request..." placeholder, leaving both renderers
empty.
Add a dedicated fixture per pill keyed on the full pill prompt sentence,
placed before the conflicting generic rules so first-fixture-wins lands
on the right one. Each fixture returns the exact content shape the
demo's renderer expects:
byoc-json-render -> {root, elements} flat spec for <Renderer />
byoc-hashbrown -> {ui:[{<tag>:{props:{...}}}]} for useJsonParser
Removes two pre-existing duplicates that had been added at the bottom of
the file but never matched because the conflicting generic rules came
first.
Pre-existing drift on main: count held at 134, hash shifted (one
FAIL healed, another regressed). Captured the new sorted-FAIL hash
locally with the same algorithm CI uses (sort -u | shasum -a 256)
and updated showcase/scripts/fail-baseline.json to match.
Two bugs introduced in 534cd1efa (D5 integration fixes) when the
langroid adapter switched backend tool results from TEXT_MESSAGE_*
triples to TOOL_CALL_RESULT events:
1. Two leftover `logger.warning("DEBUG ...")` calls in agui_adapter.py
that fired on every chat turn — exactly the noise pattern the
`test_plain_text_turn_does_not_warn` test was written to prevent.
2. Three stale tests in test_agui_adapter.py (only run on 3.11+, so
the failure was 3.12-only):
- test_backend_tool_execution_happy_path: still expected the old
TEXT_MESSAGE_START/CONTENT/END inner triple instead of a single
TOOL_CALL_RESULT event after TOOL_CALL_END.
- test_backend_tool_exception_returns_sanitized_error: still read
the sanitized error JSON from TEXT_MESSAGE_CONTENT.delta — it
now rides on TOOL_CALL_RESULT.content.
- test_plain_text_turn_does_not_warn: caught the DEBUG leak above.
Tests are skipped on Python 3.10 (langroid imports `typing.Self`),
which is why this only shows up on the 3.12 matrix entry.
Beautiful Chat suggestion clicks looped forever because feature-parity.json
tool-calling fixtures lacked an `id` and a paired `toolCallId` followup.
After the agent ran the tool and re-prompted aimock, the same userMessage
substring matched again and the same toolCall was returned indefinitely.
Added explicit ids to 10 broken fixtures (pieChart, barChart, render_*_chart,
scheduleTime, search_flights, toggleTheme) and 11 paired toolCallId
followups returning content summaries — same convention the file already
uses for show_card, weather, etc.
Beautiful Chat layout also showed a black/white split and a broken logo on
the 8 integrations using the full ExampleLayout pattern (crewai-crews,
langgraph-fastapi, langgraph-python, langgraph-typescript, mastra,
ms-agent-dotnet, ms-agent-python, pydantic-ai). Two issues:
1. globals.css hardcoded `body { background: #fafaf9 }` and never defined
the brand tokens (--background, --foreground, --card, --primary, …) that
the layout, mode-toggle, todo card/column, and chart components reference
via Tailwind 4 arbitrary values. ThemeProvider was also adding `dark` to
<html> from system preference, so CopilotKit's chat went dark while body
stayed cream.
2. example-layout/index.tsx renders <img src="/copilotkit-logo.svg" /> but
the file did not exist in any integration's public/.
Added the full token set (light + dark) under :root and :root.dark/.dark,
registered the Tailwind 4 dark variant, switched body to var(--background)
/var(--foreground), and copied copilotkit-logo.svg + copilotkit-logo-mark.svg
into each integration's public/ from examples/integrations/langgraph-python.
Two bugs introduced in 534cd1efa (D5 integration fixes) when the
langroid adapter switched backend tool results from TEXT_MESSAGE_*
triples to TOOL_CALL_RESULT events:
1. Two leftover `logger.warning("DEBUG ...")` calls in agui_adapter.py
that fired on every chat turn — exactly the noise pattern the
`test_plain_text_turn_does_not_warn` test was written to prevent.
2. Three stale tests in test_agui_adapter.py (only run on 3.11+, so
the failure was 3.12-only):
- test_backend_tool_execution_happy_path: still expected the old
TEXT_MESSAGE_START/CONTENT/END inner triple instead of a single
TOOL_CALL_RESULT event after TOOL_CALL_END.
- test_backend_tool_exception_returns_sanitized_error: still read
the sanitized error JSON from TEXT_MESSAGE_CONTENT.delta — it
now rides on TOOL_CALL_RESULT.content.
- test_plain_text_turn_does_not_warn: caught the DEBUG leak above.
Tests are skipped on Python 3.10 (langroid imports `typing.Self`),
which is why this only shows up on the 3.12 matrix entry.
Pre-existing drift on main: count held at 134, hash shifted (one
FAIL healed, another regressed). Captured the new sorted-FAIL hash
locally with the same algorithm CI uses (sort -u | shasum -a 256)
and updated showcase/scripts/fail-baseline.json to match.
Drop gen-ui-interrupt and interrupt-headless demo entries from
manifest.yaml for langroid and claude-sdk-typescript, and delete
the orphan stub page.tsx + README.md files for those routes. Both
features stay listed in not_supported_features so the dashboard
keeps showing the no-entry icon with hover tooltip.
Aligns these two integrations with the convention used by the other
12 integrations that mark these features unsupported (ag2, agno,
built-in-agent, claude-sdk-python, crewai-crews, google-adk,
llamaindex, mastra, pydantic-ai, spring-ai, strands): icon-only
with tooltip rather than a Demo / Code row that links to a stub
page explaining why there is nothing to demo.
Three D5 probe failures fixed:
tool-rendering: get_weather was a backend tool so the workflow emitted
TOOL_CALL_CHUNK but no TOOL_CALL_RESULT, leaving useRenderTool stuck in
loading state forever. Move get_weather to frontend_tools so CopilotKit
manages the tool call lifecycle. Add ToolCallResultWorkflowEvent (which
subclasses ToolCallEndWorkflowEvent to pass the AG_UI_EVENTS isinstance
filter) and override aggregate_tool_calls to emit TOOL_CALL_RESULT for
render-only tools. The WeatherCard component now always renders with
data-testid="weather-card" even during loading, instead of switching to
a separate div that lacks the testid.
gen-ui-headless: the shared agent was missing a show_card frontend tool
stub, so the workflow never emitted TOOL_CALL_CHUNK for it. Add the
stub and register it in frontend_tools.
hitl-steps: human_in_the_loop was routed to the hitl-in-chat specialized
agent (which only has book_call), not the shared agent (which has
generate_task_steps). Move human_in_the_loop to sharedAgentNames so it
routes through the default agent. Introduce render_only_tool_names set
to distinguish render-only tools (get_weather) from interactive tools
(generate_task_steps, book_call, show_card) — only render-only tools
get premature TOOL_CALL_RESULT; interactive tools let CopilotKit manage
the result lifecycle to keep buttons enabled.
The gen-ui-headless D5 probe had two bugs:
1. esbuild's keepNames transform injects __name() wrappers around const
arrow functions. When those functions run inside page.evaluate() in
the browser context, __name is not defined. Replace with string-based
new Function() construction using function declarations.
2. findFirstNonTrivial and readChildCountForSelector used querySelector
which only returns the first match. On headless chat pages, the first
assistant message div is an empty wrapper (children=0), causing the
cascade to skip the selector even when a later match contains the
rendered gen-UI component. Switch to querySelectorAll and iterate all
matches.
Spring AI's OpenAiChatModel.internalStream() auto-executes tool calls
through the global ToolCallingManager, which only knows about backend
tools. When the model calls frontend tools (generate_task_steps,
show_card, etc.) injected by the CopilotKit runtime, the
StaticToolCallbackResolver returns null and DefaultToolCallingManager
throws IllegalStateException, crashing the stream.
Two fixes:
1. StreamingToolAgent Phase 1 (streaming): set
internalToolExecutionEnabled=false via OpenAiChatOptions so Spring AI
detects tool_calls in the stream without attempting execution. Phase 2
(.call()) keeps internal execution enabled so Spring AI's built-in
loop handles backend tools.
2. BoundedToolCallingManagerConfig: wrap StaticToolCallbackResolver with
LenientToolCallbackResolver that returns a FrontendToolPlaceholder
for unknown tools instead of letting the resolver return null. This
prevents the IllegalStateException during Phase 2 when the model
calls frontend tools.
Also reorder AG-UI event emission so tool call events are emitted
BEFORE textMessageEnd, which is required for the frontend's
useRenderTool to see them while the message is still open.
Recovers agentic-chat, gen-ui-custom, and gen-ui-headless D5 tests
from 0/11 to 3/11. Remaining failures are pre-existing issues in
other controllers (subagents, shared-state, hitl).
Seven D5 features fail because built-in-agent demo pages use different
tool names and hook types than the LangGraph-Python reference agent
that the D5 fixtures were recorded against.
- tool-rendering: switch from useComponent("weather") to
useRenderTool("get_weather") with correct {parameters, result, status}
shape
- gen-ui-tool-based: rename useComponent("haiku") to
useComponent("generate_haiku"), fix HaikuCard to accept args as
direct props instead of {status, result, parameters}
- hitl-in-chat: create /demos/hitl-in-chat route (was missing); reuses
TimePickerCard from hitl-in-chat-booking with book_call tool
- shared-state: add set_notes server tool to baseServerTools so the
fixture's set_notes call has a backend handler
- subagents: rename delegate_to_planner/researcher to
research_agent/writing_agent/critique_agent matching LGP tool names
- subagents page: update useComponent registrations and DelegationCard
for the renamed tools
The hitl-in-app agent was registered with a trailing slash in the URL
(`/hitl-in-app/`), while the FastAPI backend mounts the endpoint at
`/hitl-in-app` (no trailing slash). FastAPI's default redirect_slashes
behavior returns a 307 redirect for POST requests to the trailing-slash
variant, and the AG-UI HttpAgent does not follow POST redirects. This
caused the agent to appear completely unresponsive — the D5
hitl-approve-deny probe timed out at 60s with zero assistant messages.
Use the shared `createAgent()` helper (which does not append a trailing
slash) for consistency with every other agent registration in the file.
Install the pkg.pr.new build of @copilotkit/runtime from PR #4482 to
validate the runFinished fix that stops duplicate TOOL_CALL_END events
in convertTanStackStream after TanStack's RUN_FINISHED.
When a dashboard cell has no probe data for a depth (D5, D6), render
the dimension name with CSS line-through instead of appending a "?"
label. Strikethrough communicates "this depth doesn't exist / hasn't
run" more clearly than "D5 ?".
Changes:
- Badge component: when label is "?", render name with line-through
styling and suppress the question mark
- CellDrilldown BadgeRow: render "n/a" with line-through instead of "?"
- Update cell-drilldown test to verify strikethrough rendering
PR #4464 only applied the upstream bug fixes to hitl_in_chat_agent.py.
The same three bugs (duplicate tool-call rendering, missing
parent_message_id, incorrect tool-result message roles) affect the
hitl_in_app_agent and the main agent router.
Switch both to use FixedAGUIChatWorkflow via workflow_factory so all
llamaindex demo features get the corrected AG-UI event stream.
## Summary
Adds comprehensive `console.debug` logging to the D5 probe execution
pipeline so production probe failures are diagnosable from logs alone,
without requiring reproduction.
**conversation-runner.ts** (the core multi-turn conversation engine):
- Logs conversation start/end with turn count and settle configuration
- Logs chat input selector cascade resolution -- each selector miss and
the final hit
- Logs per-turn lifecycle: message send, assistant settle wait, message
count changes, assertion execution, and failures with elapsed times
- Logs the fillAndVerifySend retry loop with attempt counts and user
message baseline vs current counts
- Logs waitForAssistantSettled polling with count changes, periodic
status during long waits (~every 5s), and timeout details
**e2e-deep.ts** (the driver that orchestrates per-feature runs):
- Logs page navigation URL and result
- Logs React hydration wait start, success, and timeout
- Logs conversation turn count after buildTurns
- Logs conversation success/failure with diagnostic capture summary
**All 9 D5 scripts** (agentic-chat, gen-ui-custom, gen-ui-headless,
hitl-approve-deny, hitl-steps, hitl-text-input, mcp-subagents,
shared-state, tool-rendering):
- Logs expected tokens/fragments and actual assistant text at each
assertion checkpoint
- Logs DOM element discovery (matched selectors, child counts, card
structure)
- Logs polling progress for long-running assertions (tool card probe,
chained reply fragments)
All logging uses `console.debug(...)` which is suppressed at INFO level
in production but available when operators set `LOG_LEVEL=debug` for
investigation.
## Test plan
- [x] All 1372 existing tests pass (`cd showcase/harness && npx vitest
run` -- 70 test files, 0 failures)
- [x] Debug output is visible in test stdout (vitest captures it
per-test)
- [x] No functional changes -- all logging is additive `console.debug`
calls
Instrument the conversation runner, e2e-deep driver, and all D5 scripts
with console.debug logging at every decision point so production probe
failures are diagnosable without reproduction.
conversation-runner.ts:
- Log conversation start/end with turn count and settle config
- Log chat input selector cascade resolution (each miss and final hit)
- Log per-turn lifecycle: message send, settle wait, count changes,
assertions, and failures with elapsed times
- Log fillAndVerifySend retry loop with attempt counts and user message
baseline/current counts
- Log waitForAssistantSettled polling with count changes, periodic
status during long waits, and timeout details
e2e-deep.ts (runFeature):
- Log page navigation URL and result
- Log React hydration wait (start, success, timeout)
- Log conversation turn count after buildTurns
- Log conversation success/failure with diagnostics
All D5 scripts (d5-agentic-chat, d5-gen-ui-custom, d5-gen-ui-headless,
d5-hitl-approve-deny, d5-hitl-steps, d5-hitl-text-input,
d5-mcp-subagents, d5-shared-state, d5-tool-rendering):
- Log expected tokens and actual assistant text at each assertion
- Log DOM element discovery (selectors matched, child counts)
- Log polling progress for long-running assertions (tool card probe,
chained reply fragments)
readLastAssistantText was reading textContent from the entire assistant
message wrapper, which includes rendered tool-component output (SVG
chart labels like "Electronics42,000Clothing28,000...") in addition to
the actual chat text. This caused gen-ui-custom probes to fail token
assertions because the extracted text was the rendered PieChart content
instead of the follow-up narration.
Scope the extraction to the prose/markdown child div inside the
canonical CopilotKit message structure, which contains only the
assistant's text response. Falls back gracefully when the expected
DOM structure isn't present (headless/custom composers).
Also adds debug logging so production traces show which element was
selected and what text was extracted.
TanStack's chat() engine runs a multi-turn agent loop: after the model
returns tool calls, it tries to execute them via processToolCalls().
Frontend-only tools (like render_pie_chart) are unknown to TanStack, so
executeToolCalls() treats them as errors and buildToolResultChunks()
re-emits TOOL_CALL_END without a preceding TOOL_CALL_START. The ag-ui
verify middleware rejects this duplicate.
Fix: track a runFinished flag in convertTanStackStream and discard all
events after the first RUN_FINISHED, which marks the boundary between
the streaming pass and TanStack's internal tool execution loop.
Also adds built-in-agent to docker-compose.local.yml and local-ports.json.
Spring AI's ChatClient.stream() does NOT auto-execute tool callbacks
(only .call() has a built-in tool execution loop). The stock AG-UI
SpringAIAgent uses .stream() exclusively, so tools are never invoked
and the CopilotKit runtime re-invokes the agent infinitely.
Replace SpringAIAgent with a custom StreamingToolAgent that:
- Phase 1: streams text via .stream() for real-time UX delivery
- Phase 2: if tool calls detected, re-invokes via .call() with
tool callbacks so Spring AI's internal loop handles execution
- Emits proper AG-UI tool call events via wrapper callbacks
Also fix Jackson deserialization crashes from CopilotKit runtime
messages with unrecognised roles (activity, reasoning) by disabling
FAIL_ON_INVALID_SUBTYPE, with null-filtering at controller boundaries
to prevent NPEs in LocalAgent.combineMessages().
Affects all spring-ai endpoints: agentic_chat, agent-config,
a2ui-fixed-schema, subagents, shared-state-read-write.
## Summary
- Replace `SpringAIAgent` (streaming `.stream()`) with custom
`SyncAgent` (synchronous `.call()`) for all spring-ai showcase
controllers
- Add `JacksonConfig` to handle unknown AG-UI message roles (activity,
reasoning) gracefully
- Fix the root cause of spring-ai D5 timeouts on tool-rendering,
hitl-steps, gen-ui, mcp-apps, and shared-state features
## Why
Spring AI's streaming mode does NOT auto-execute tool callbacks. The
model returns `tool_calls` in the SSE stream but the tools are never
invoked. The AG-UI event stream emits `TOOL_CALL_START/ARGS/END` without
`TOOL_CALL_RESULT`, causing the CopilotKit runtime to re-invoke the
agent in an infinite loop (and eventual timeout).
By using `ChatClient.call()`, Spring AI's internal tool-execution loop
runs tools automatically. `SyncAgent` wraps each backend tool callback
to capture AG-UI events during auto-invocation, and registers frontend
tools as pass-through callbacks.
The `JacksonConfig` addresses a secondary issue: the AG-UI Java SDK's
`MessageMixin` only maps 5 roles (assistant, developer, user, system,
tool). Messages with roles like "reasoning" cause
`InvalidTypeIdException` during deserialization. The config overrides
the mixin with `defaultImpl = UnknownMessage.class` so unknown roles
deserialize harmlessly.
## Test plan
- [ ] Docker build succeeds (verified locally)
- [ ] D5 tool-rendering passes
- [ ] D5 subagents passes
- [ ] D5 hitl-steps passes
- [ ] D5 mcp-apps passes
- [ ] D5 shared-state passes
SpringAIAgent from the AG-UI Java SDK uses streaming (.stream()) for
Spring AI's ChatClient. In streaming mode, Spring AI does NOT auto-execute
tool callbacks -- the model returns tool_calls but the tools are never
invoked. This causes the CopilotKit runtime to see incomplete tool cycles
(TOOL_CALL_START/ARGS/END without TOOL_CALL_RESULT) and re-invoke the
agent in an infinite loop, producing the D5 timeouts on tool-rendering,
hitl-steps, gen-ui, and mcp-apps features.
SyncAgent uses ChatClient.call() instead, which runs Spring AI's internal
tool-execution loop synchronously. Backend tools are auto-invoked and
their results are captured as AG-UI events (TOOL_CALL_START/ARGS/END/
RESULT). Frontend tools from the CopilotKit runtime are registered as
pass-through callbacks that emit start/args/end events without a result,
letting the runtime handle frontend execution.
Also adds JacksonConfig to handle unknown message roles (activity,
reasoning) that the AG-UI Java SDK doesn't model. Without this, Jackson
throws InvalidTypeIdException on deserialization, causing request failures
before any controller logic runs.
Changes:
- Add SyncAgent.java: LocalAgent subclass using .call() with event-
capturing tool callback wrappers
- Add JacksonConfig.java: lenient MessageMixin with defaultImpl for
unknown roles
- Update AgentConfig to return SyncAgent bean instead of SpringAIAgent
- Update AgentController to inject SyncAgent instead of SpringAIAgent
- Update A2uiFixedSchemaController to build SyncAgent per request
- Update AgentConfigController to build SyncAgent per request
## Summary
- Fixes `ERR_MODULE_NOT_FOUND: Cannot find package 'tsx'` crash in the
local langgraph-typescript container
- Root cause: the host bind mount `src:/app/src` clobbers
`src/agent/node_modules` that was installed during the Docker build
- Fix: anonymous volume `/app/src/agent/node_modules` preserves the
image's copy through the bind mount overlay
- google-adk D5 intermittent failures are pool starvation (already
addressed by #4460); no code change needed
## Test plan
- [x] Verified the compose container crash-loops before the fix
- [x] Verified the container starts healthy after the fix
- [x] Confirmed the production GHCR image works correctly (the issue
is compose-local only)
The host bind mount `src:/app/src` in docker-compose.local.yml overlays
the container's /app/src directory, clobbering src/agent/node_modules
that was installed during the Docker build. This causes `node --import tsx`
to fail with ERR_MODULE_NOT_FOUND at container startup.
Add an anonymous volume for /app/src/agent/node_modules so Docker
preserves the image's copy even when the parent directory is bind-mounted.
The LlamaIndex AGUIChatWorkflow has three bugs that caused the
hitl-text-input test to fail:
1. ToolCallChunkWorkflowEvent emitted without parent_message_id,
causing the AG-UI client to create a duplicate assistant message
(one from MESSAGES_SNAPSHOT, one from dechunked TOOL_CALL_START).
The duplicate produced 8 time-picker buttons instead of 4.
2. _snapshot_messages embeds toolCalls in the MESSAGES_SNAPSHOT AND
emits separate TOOL_CALL_CHUNK events, so the client registers
the same tool call twice on the same message.
3. ag_ui_message_to_llama_index_message converts AG-UI ToolMessages
to ChatMessage(role='user') instead of role='tool', so the second-
leg OpenAI API call has no tool-result message and aimock's
hasToolResult matcher fails.
Fix: subclass AGUIChatWorkflow (FixedAGUIChatWorkflow) that:
- Assigns a stable ID to assistant messages and passes it as
parent_message_id on ToolCallChunk events
- Strips toolCalls from MESSAGES_SNAPSHOT (letting TOOL_CALL_CHUNK
be the sole mechanism)
- Corrects tool-result message role from 'user' to MessageRole.TOOL
so the OpenAI API receives proper tool-result messages
Log pool stats and context counts when the abort signal fires and
force-releases a browser, so pool starvation events are visible in
Railway logs for diagnosis.
When the probe invoker's timeout fires, Promise.race returns the timeout
result but the driver promise keeps running with a pooled browser held.
The browser is not released back to the pool until the driver eventually
finishes (up to 5 minutes later). Since pool size (4) equals
max_concurrency (4), orphaned browsers accumulate across ticks and cause
cascading starvation -- each 15-minute probe run has fewer available
browsers until the pool is completely exhausted.
The fix wires the abort signal through to createPooledE2eDeepLauncher.
When abort fires, the launcher force-closes open browser contexts and
releases the browser back to the pool immediately. A forceReleased flag
prevents the driver's finally block from double-releasing.
## Summary
- **Restores the eval system** accidentally deleted by PR #4449 ("D5
all-green" session removed it as "superseded by D5 depth probes" while
unaware #4448 had just merged)
- **Per-test JSON granularity** — eval runner now injects
`--reporter=list,json` + `PLAYWRIGHT_JSON_OUTPUT_NAME` when spawning
test subprocesses, reads JSON file after exit for per-test results,
falls back to exit-code when absent
- **Slug-keyed baselines** — `transformHarnessResponse` restructured to
key by integration slug (via `summary.services[]`) instead of probe ID,
matching `collectResults` format so `computeRegressions` works against
harness-prod baselines
## Context
PR #4448 added the eval system (5 CR rounds, 8 bugs fixed, 35+ agent
reviews). PR #4449 (a concurrent session) deleted it. This PR restores
the CR-converged code and adds two follow-up improvements from the eval
plan.
## Test plan
- [x] `npx nx run @copilotkit/showcase-harness:test` — 1375 tests pass
(70 files)
- [x] All 8 CR fixes from #4448 verified present in restored code
- [ ] CI green
Production showcase-aimock was running a week-old image because fixture
file changes in showcase/aimock/ did not trigger a CI rebuild. This adds
showcase-aimock to the Build & Deploy workflow matrix so it auto-deploys
on merge, creates a thin Dockerfile that bakes fixture files into the
image, documents the local-vs-production parity requirement in
docker-compose.local.yml, and adds an aimock fixture deployment section
to the RUNBOOK.
Runner: inject --reporter=list,json and PLAYWRIGHT_JSON_OUTPUT_NAME
when spawning test subprocesses; read JSON file after exit for per-
test granularity; fall back to stdout then exit-code.
Baseline: restructure transformHarnessResponse to iterate
summary.services[] and produce slug-keyed results with _status
sub-keys matching collectResults format. Worst-status-wins when
same slug appears in multiple probes.
PR #4449 removed the eval tier system while unaware #4448 had just
merged. Restoring orchestrator, matrix, scope, config, GHA workflow,
and CLI wiring with all 8 CR fixes intact.
Alem Tuzlak