Commit Graph

14 Commits

Author SHA1 Message Date
Alem Tuzlak a3586fb62a fix(showcase): revert reasoning field on aimock d5 reasoning fixture
PR #4579 added a `reasoning` field to the "show your reasoning step by
step" fixture so aimock would emit response.reasoning_summary_* deltas
for the OpenAI Responses API path. Side effect: aimock's Chat
Completions handler also emits non-standard `reasoning_content` deltas
(DeepSeek/Qwen-style) ahead of the role/content chunks. Many
integrations' OpenAI client adapters don't expect those deltas and
either hang or fail to parse the stream — manifesting as "assistant
did not respond within 30000ms" across most reasoning cells in
production.

Restore the original content-only fixture. The langgraph-python /
langgraph-fastapi agent fixes from #4579 still work against real
OpenAI (gpt-5-mini + Responses API streams real reasoning summaries),
but the aimock-driven path no longer exercises the role-reasoning
render — keyword-only assertion in the d5 probe handles that.
2026-05-01 14:07:21 +02:00
Alem Tuzlak dca1b9894d fix(showcase): emit reasoning events in langgraph-python and langgraph-fastapi
The agentic-chat-reasoning and reasoning-default-render cells in
langgraph-python and langgraph-fastapi were configured with
gpt-4o-mini + use_responses_api=False, which never produces AG-UI
REASONING_MESSAGE_* events: gpt-4o-mini is not a reasoning model and
the Chat Completions API does not surface reasoning summary items at
all. The frontend's reasoningMessage slot was rendering nothing,
even though the cells were billed as "reasoning" demos.

- Switch both reasoning agents to gpt-5-mini (override via
  OPENAI_REASONING_MODEL) routed through the Responses API with
  reasoning={"effort":"medium","summary":"detailed"} so the model's
  chain of thought streams as content blocks that @ag-ui/langgraph
  translates into REASONING_MESSAGE_* events.
- Update the aimock d5-all.json and harness reasoning-display.json
  fixtures to include a "reasoning" field so aimock emits
  response.reasoning_summary_text.delta SSE events deterministically
  in CI without hitting a real LLM.
- Add a "Show reasoning" useConfigureSuggestions pill on both
  reasoning demo pages so the user can trigger the fixture-matched
  prompt with one click.
- Tighten the d5-reasoning-display probe: it now also asserts a
  reasoning-role message rendered via [data-testid="reasoning-block"]
  or [data-message-role="reasoning"], so a plain text response
  containing the word "reasoning" no longer falsely passes.
- Un-skip the three streaming reasoning-block tests in
  langgraph-python's agentic-chat-reasoning.spec.ts and add a
  suggestion-pill test; expand the reasoning-default-render spec to
  cover the default reasoning slot.
- Update the langgraph-python QA doc to describe the new model +
  Responses API setup and the suggestion-pill flow.
2026-05-01 13:30:28 +02:00
Alem Tuzlak 0eb81b1406 fix(showcase): unbreak agent-config + byoc D5, reaching 31/31 green
agent-config: drop the AgentConfigLangGraphAgent subclass and use plain
LangGraphAgent. The subclass repacked CopilotKit provider properties
into forwardedProps.config.configurable.properties so the Python graph
could read them via RunnableConfig.configurable.properties — but
@ag-ui/langgraph@0.0.31 builds the LangGraph SDK request as
{ ..., config, context: { ...input.context, ...config.configurable } }
which merges configurable INTO context. LangGraph 0.6.0+ then rejects
with HTTP 400 'Cannot specify both configurable and context' on every
chat round-trip. Net effect: chat sent the user message, runtime 400'd,
no assistant response ever rendered. Removing the subclass unbreaks
the round-trip; the Python agent falls back to its DEFAULT_* constants
so the demo's frontend toggles no longer steer the system prompt
(known regression, tracked separately pending @ag-ui/langgraph fix
that decouples context from configurable).

byoc:
- D5 probe now sends the 'Sales dashboard' pill prompt (matches the
  fixtures added in main:f0a89b843 in feature-parity.json) instead of
  the previous generic 'render a byoc hashbrown' prompt that had no
  matching JSON-shaped fixture. Removed the now-obsolete byoc.json D5
  fixture file and regenerated the d5-all.json bundle (52 -> 50
  fixtures).
- Added data-testid='copilot-assistant-message' + data-message-role=
  'assistant' to the byoc-hashbrown and byoc-json-render renderer
  wrapper divs. The CopilotChat default assistantMessage slot includes
  these markers; overriding the slot with a custom JSON-rendering
  component dropped them, so the e2e-deep conversation runner's
  settle-detection cascade (which counts these selectors) never saw
  the response and timed out at 30s. Re-attaching the markers is a
  purely additive change that doesn't affect the renderers'
  behavior.
- D5 byoc assertion now waits for [data-testid='metric-card'] AND a
  chart (bar-chart or pie-chart) to render — a structural check on
  the BYOC contract output, not a transcript-keyword check that the
  custom renderer would never produce.

E2E status: 31/31 passing locally against
./bin/showcase up langgraph-python aimock with this branch's bundle.
2026-04-30 15:11:19 +02:00
Alem Tuzlak 203da612db feat(showcase): D5 scripts for interrupt + BYOC families (B6) 2026-04-30 13:23:26 +02:00
Alem Tuzlak 56974444f8 feat(showcase): D5 scripts for gen-UI family (B5) 2026-04-30 13:22:00 +02:00
Alem Tuzlak b9246586b5 feat(showcase): D5 scripts for state family (B4) 2026-04-30 13:20:20 +02:00
Alem Tuzlak 465c9b697a feat(showcase): D5 scripts for frontend-tools + reasoning families (B3) 2026-04-30 13:19:15 +02:00
Alem Tuzlak 7f3d458cdf feat(showcase): D5 scripts for platform family (B2: auth, multimodal, agent-config) 2026-04-30 13:17:01 +02:00
Alem Tuzlak a5f347e88f feat(showcase): D5 scripts for chat-surface family (B1) 2026-04-30 13:12:16 +02:00
Jordan Ritter 8d0bc3cd93 fix(showcase): migrate D5 fixtures from turnIndex to hasToolResult
hasToolResult checks whether tool-role messages exist in the request,
correctly disambiguating multi-turn fixtures regardless of the backend
AG-UI implementation's re-invocation behavior.
2026-04-29 19:40:10 -07:00
Jordan Ritter 6a4c25a24d fix: add nested sub-agent fixtures for D5 mcp-subagents demo
Add 3 fixtures for the independent LLM calls made by each sub-agent
tool (research_agent, writing_agent, critique_agent) so the demo works
without --proxy-only mode. Interleave them with supervisor fixtures for
readability. Rebuild d5-all.json bundle.
2026-04-28 20:45:00 -07:00
Jordan Ritter a2bc64692f fix(showcase): migrate D5 fixtures from toolCallId to turnIndex
aimock v1.16.0 ships native turnIndex matching — counts assistant
messages in the request's message array instead of relying on
toolCallId in the last tool message. Stateless, concurrent-safe,
fixture-ordering-insensitive.

Removes all toolCallId match criteria from 9 D5 fixture files (8
source + 1 bundle), adds turnIndex values, and reorders fixtures
to natural 0→N reading order.
2026-04-28 18:10:25 -07:00
Jordan Ritter bd32bcff78 fix(showcase): replace sequenceIndex with toolCallId in D5 fixtures
aimock sequenceIndex counter is server-global, causing fixture match
failures when multiple integrations run concurrently. Replace with
stateless toolCallId matching.
2026-04-28 16:55:46 -07:00
Jordan Ritter 0522b7fa41 refactor(showcase): rename showcase/ops → showcase/harness
The monitoring/alerting service is a test harness (probes, assertions,
alerting), not an operations service. Rename the directory, package
name (@copilotkit/showcase-ops → @copilotkit/showcase-harness), all
internal references (Dockerfile, Prometheus metric prefix
showcase_ops_ → showcase_harness_, orchestrator log messages, probe
YAML nameExcludes, test fixtures), and regenerate pnpm-lock.yaml.

Wire protocol names (X-Ops-* headers) and shell-dashboard internal
API naming (OPS_BASE_URL, ops-api.ts) are intentionally unchanged —
they are stable contracts between sender and receiver.
2026-04-28 13:48:12 -07:00