Files
Tyler Slaton 04d8008ea7 fix(showcase/langgraph-python): unbreak shared-state pills, auth sign-out, gen-ui-agent progression, multimodal D5
Four independent showcase production bugs Alem reported, plus the
D5 multimodal harness regression they unblocked.

Shared-state-read-write: "Greet me" ("Say hi and introduce yourself.")
and "Plan a weekend" ("Suggest a weekend plan based on my interests.")
were matching the bare `hi` and `plan` catch-alls in feature-parity.json
and returning the generic showcase-assistant blurb / 5-step content plan
instead of shared-state-aware responses. Added pill-specific fixtures in
shared-state.json (mirrored into d5-all.json) so the longer userMessage
substrings win first-match-wins ahead of feature-parity.

Auth sign-out: signing out unmounted CopilotKit entirely and bounced
the user back to the SignInCard, so the demo never showcased the
runtime returning 401 — its whole point. The QA contract in
qa/auth.md spelled out the intended UX. Restored it: CopilotKit stays
mounted after the first sign-in, the AuthBanner flips to an amber
"Signed out — the agent will reject your messages" state with a
re-Sign-in button, and CopilotKit's `onError` callback drives a
`data-testid="auth-demo-error"` surface that displays the runtime's
401 the moment the user sends an unauthenticated message. Updated the
e2e spec to match (the old "SignInCard re-mounts after sign-out" test
pinned the regression).

Gen-ui-agent: the aimock fixture short-circuited the 7-step
progression spelled out in `gen_ui_agent.py`'s SYSTEM_PROMPT to a
single set_steps call with all three steps already `completed`, so
the InlineAgentStateCard rendered the final 3/3 state instantly with
no sequential pending → in_progress → completed animation.
Regenerated as a 7-leg toolCallId chain per pill (8 fixtures × 3
pills): seed leg keyed on userMessage with NO `hasToolResult` gate
(matching PR #4770's pattern — `hasToolResult: false` would block the
seed from firing on the second pill in a multi-pill session), then
six toolCallId-keyed transitions, then a final narration. Fixture
order: toolCallId legs FIRST so the most specific match wins.

Multimodal D5: the sample-attachment buttons auto-send via
`agent.addMessage + copilotkit.runAgent` (restored in PR #4761), but
the D5 harness still typed `input` + pressed Enter via the runner
after `preFill`, sending a second user message that competed with the
in-flight image upload — the v1 LangGraph runtime SSE stream got
tangled (browser DevTools showed `statusCode: pending` indefinitely)
and the assistant message never rendered. Added `skipSend?: boolean`
to ConversationTurn (distinct from `skipFill`, which still presses
Enter once the textarea has content) and switched d5-multimodal.ts to
`skipSend: true` with `responseTimeoutMs: 60_000` so the runner waits
on the assistant response without poking the chat further. Bumped the
PDF auto-prompt fixture in feature-parity.json to include the word
"document" so the existing `buildModalityAssertion("document")` check
still lands.

D5 result: 37 → 39 of 40 features passing. Only
`tool-rendering-reasoning-chain` remains and is a separate
agent/runtime bug (Tokyo Responses-API `reasoning` message survives
into the next turn's conversation history, runtime returns
`RUN_ERROR: "message role is not supported"`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 22:16:22 -07:00

79 lines
2.9 KiB
TypeScript

import { test, expect } from "@playwright/test";
// Shared State (Read + Write) — Preferences + Scratch Pad demo. The page
// publishes `agent.state.preferences` via setState, and the agent writes
// back via its `set_notes` tool. All three pills must hit
// shared-state-specific fixtures rather than falling through to the bare
// "hi" / "plan" catch-alls in feature-parity.json.
test.describe("Shared State (Read + Write)", () => {
test.beforeEach(async ({ page }) => {
await page.goto("/demos/shared-state-read-write");
});
test("preferences panel and agent scratch pad both mount", async ({
page,
}) => {
await expect(page.getByText("Your preferences")).toBeVisible({
timeout: 15000,
});
await expect(page.getByText("Agent Scratch pad")).toBeVisible({
timeout: 15000,
});
});
test("starter suggestions render", async ({ page }) => {
for (const title of ["Greet me", "Remember something", "Plan a weekend"]) {
await expect(page.getByRole("button", { name: title })).toBeVisible({
timeout: 15000,
});
}
});
// Regression for the wrong-fixture-fallthrough bug:
// The Greet me pill ("Say hi and introduce yourself.") was matching
// feature-parity.json's bare `userMessage: "hi"` fixture and replying
// with the generic showcase-assistant blurb ("Hi there! I'm your
// showcase assistant. I can help with weather, charts, meetings…").
// Likewise, Plan a weekend ("Suggest a weekend plan based on my
// interests.") was matching the bare `userMessage: "plan"` fixture and
// replying with a generic 5-step content-marketing plan. Fix: add
// d5-all.json fixtures with longer substring matchers that win first.
test("Greet me pill returns a shared-state-aware greeting, not the generic showcase blurb", async ({
page,
}) => {
test.setTimeout(120_000);
await page.getByRole("button", { name: /^Greet me$/i }).click();
const assistantMessage = page
.locator('[data-testid="copilot-assistant-message"]')
.last();
await expect(assistantMessage).toContainText(/shared-state co-pilot/i, {
timeout: 60_000,
});
// Negative assertion: the wrong-fixture response is gone.
await expect(assistantMessage).not.toContainText(
/showcase assistant\. I can help with weather, charts/i,
);
});
test("Plan a weekend pill returns an interests-aware plan, not the generic content-marketing plan", async ({
page,
}) => {
test.setTimeout(120_000);
await page.getByRole("button", { name: /^Plan a weekend$/i }).click();
const assistantMessage = page
.locator('[data-testid="copilot-assistant-message"]')
.last();
await expect(assistantMessage).toContainText(/interests panel/i, {
timeout: 60_000,
});
// Negative assertion: the wrong-fixture response is gone.
await expect(assistantMessage).not.toContainText(
/Research the topic.*Outline key points/is,
);
});
});