Files
Tyler Slaton 04d8008ea7 fix(showcase/langgraph-python): unbreak shared-state pills, auth sign-out, gen-ui-agent progression, multimodal D5
Four independent showcase production bugs Alem reported, plus the
D5 multimodal harness regression they unblocked.

Shared-state-read-write: "Greet me" ("Say hi and introduce yourself.")
and "Plan a weekend" ("Suggest a weekend plan based on my interests.")
were matching the bare `hi` and `plan` catch-alls in feature-parity.json
and returning the generic showcase-assistant blurb / 5-step content plan
instead of shared-state-aware responses. Added pill-specific fixtures in
shared-state.json (mirrored into d5-all.json) so the longer userMessage
substrings win first-match-wins ahead of feature-parity.

Auth sign-out: signing out unmounted CopilotKit entirely and bounced
the user back to the SignInCard, so the demo never showcased the
runtime returning 401 — its whole point. The QA contract in
qa/auth.md spelled out the intended UX. Restored it: CopilotKit stays
mounted after the first sign-in, the AuthBanner flips to an amber
"Signed out — the agent will reject your messages" state with a
re-Sign-in button, and CopilotKit's `onError` callback drives a
`data-testid="auth-demo-error"` surface that displays the runtime's
401 the moment the user sends an unauthenticated message. Updated the
e2e spec to match (the old "SignInCard re-mounts after sign-out" test
pinned the regression).

Gen-ui-agent: the aimock fixture short-circuited the 7-step
progression spelled out in `gen_ui_agent.py`'s SYSTEM_PROMPT to a
single set_steps call with all three steps already `completed`, so
the InlineAgentStateCard rendered the final 3/3 state instantly with
no sequential pending → in_progress → completed animation.
Regenerated as a 7-leg toolCallId chain per pill (8 fixtures × 3
pills): seed leg keyed on userMessage with NO `hasToolResult` gate
(matching PR #4770's pattern — `hasToolResult: false` would block the
seed from firing on the second pill in a multi-pill session), then
six toolCallId-keyed transitions, then a final narration. Fixture
order: toolCallId legs FIRST so the most specific match wins.

Multimodal D5: the sample-attachment buttons auto-send via
`agent.addMessage + copilotkit.runAgent` (restored in PR #4761), but
the D5 harness still typed `input` + pressed Enter via the runner
after `preFill`, sending a second user message that competed with the
in-flight image upload — the v1 LangGraph runtime SSE stream got
tangled (browser DevTools showed `statusCode: pending` indefinitely)
and the assistant message never rendered. Added `skipSend?: boolean`
to ConversationTurn (distinct from `skipFill`, which still presses
Enter once the textarea has content) and switched d5-multimodal.ts to
`skipSend: true` with `responseTimeoutMs: 60_000` so the runner waits
on the assistant response without poking the chat further. Bumped the
PDF auto-prompt fixture in feature-parity.json to include the word
"document" so the existing `buildModalityAssertion("document")` check
still lands.

D5 result: 37 → 39 of 40 features passing. Only
`tool-rendering-reasoning-chain` remains and is a separate
agent/runtime bug (Tokyo Responses-API `reasoning` message survives
into the next turn's conversation history, runtime returns
`RUN_ERROR: "message role is not supported"`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 22:16:22 -07:00

154 lines
5.8 KiB
TypeScript

import { test, expect } from "@playwright/test";
/**
* Auth demo lifecycle. The demo defaults to UNAUTHENTICATED on first
* paint and renders SignInCard. After sign-in, <CopilotKit> mounts with
* the bearer header attached and the chat boots. After sign-out the
* chat STAYS MOUNTED (now with no Authorization header) so the user can
* actually watch the runtime reject an unauthenticated send — that is
* the whole point of the demo. The AuthBanner flips between green
* (authenticated) and amber (signed-out) variants. Only a full page
* reload resets to the SignInCard first-paint state.
*/
test.describe("Authentication", () => {
test.beforeEach(async ({ page }) => {
await page.goto("/demos/auth");
});
test("page loads unauthenticated with SignInCard visible", async ({
page,
}) => {
await expect(
page.locator('[data-testid="auth-sign-in-card"]'),
).toBeVisible();
await expect(
page.locator('[data-testid="auth-sign-in-button"]'),
).toBeEnabled();
await expect(page.locator('[data-testid="auth-demo-token"]')).toBeVisible();
// Chat surface and AuthBanner only render after the first sign-in.
await expect(page.locator('[data-testid="auth-banner"]')).toHaveCount(0);
await expect(page.getByPlaceholder("Type a message")).toHaveCount(0);
});
test("signing in mounts the chat surface with AuthBanner", async ({
page,
}) => {
await page.locator('[data-testid="auth-sign-in-button"]').click();
const banner = page.locator('[data-testid="auth-banner"]');
await expect(banner).toBeVisible();
await expect(banner).toHaveAttribute("data-authenticated", "true");
await expect(page.locator('[data-testid="auth-status"]')).toContainText(
"Signed in",
);
await expect(
page.locator('[data-testid="auth-sign-out-button"]'),
).toBeEnabled();
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
// SignInCard is gone once we're authenticated.
await expect(page.locator('[data-testid="auth-sign-in-card"]')).toHaveCount(
0,
);
});
test("authenticated send produces an assistant response", async ({
page,
}) => {
await page.locator('[data-testid="auth-sign-in-button"]').click();
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
const input = page.getByPlaceholder("Type a message");
await input.fill("Hello");
await input.press("Enter");
await expect(
page.locator('[data-testid="copilot-assistant-message"]').first(),
).toBeVisible({ timeout: 30000 });
});
test("signing out flips the banner amber and keeps the chat surface mounted", async ({
page,
}) => {
await page.locator('[data-testid="auth-sign-in-button"]').click();
await expect(
page.locator('[data-testid="auth-sign-out-button"]'),
).toBeVisible();
await page.locator('[data-testid="auth-sign-out-button"]').click();
// Banner flips to the amber variant. SignInCard does NOT come back —
// the demo would never get to showcase the rejection if it did.
const banner = page.locator('[data-testid="auth-banner"]');
await expect(banner).toBeVisible();
await expect(banner).toHaveAttribute("data-authenticated", "false");
await expect(page.locator('[data-testid="auth-status"]')).toContainText(
"Signed out",
);
await expect(
page.locator('[data-testid="auth-authenticate-button"]'),
).toBeVisible();
await expect(
page.locator('[data-testid="auth-sign-out-button"]'),
).toHaveCount(0);
await expect(page.locator('[data-testid="auth-sign-in-card"]')).toHaveCount(
0,
);
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
});
test("unauthenticated send surfaces a 401 error without crashing the page", async ({
page,
}) => {
await page.locator('[data-testid="auth-sign-in-button"]').click();
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
await page.locator('[data-testid="auth-sign-out-button"]').click();
await expect(
page.locator('[data-testid="auth-authenticate-button"]'),
).toBeVisible();
// After sign-out, the next send must surface the rejection — not
// produce an assistant response and not white-screen the page.
const input = page.getByPlaceholder("Type a message");
await input.fill("Hello again");
await input.press("Enter");
const errorSurface = page.locator('[data-testid="auth-demo-error"]');
await expect(errorSurface).toBeVisible({ timeout: 15000 });
// Page chrome stays visible — no white-screen.
await expect(page.locator('[data-testid="auth-banner"]')).toBeVisible();
await expect(
page.locator('[data-testid="copilot-assistant-message"]'),
).toHaveCount(0);
});
test("re-signing in from the amber banner clears the error and resumes chat", async ({
page,
}) => {
await page.locator('[data-testid="auth-sign-in-button"]').click();
await expect(
page.locator('[data-testid="auth-sign-out-button"]'),
).toBeVisible();
await page.locator('[data-testid="auth-sign-out-button"]').click();
await expect(
page.locator('[data-testid="auth-authenticate-button"]'),
).toBeVisible();
await page.locator('[data-testid="auth-authenticate-button"]').click();
const banner = page.locator('[data-testid="auth-banner"]');
await expect(banner).toHaveAttribute("data-authenticated", "true", {
timeout: 5000,
});
await expect(page.locator('[data-testid="auth-demo-error"]')).toHaveCount(
0,
);
// Chat is responsive again.
const input = page.getByPlaceholder("Type a message");
await input.fill("Hello again");
await input.press("Enter");
await expect(
page.locator('[data-testid="copilot-assistant-message"]').first(),
).toBeVisible({ timeout: 30000 });
});
});