mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
04d8008ea7
Four independent showcase production bugs Alem reported, plus the
D5 multimodal harness regression they unblocked.
Shared-state-read-write: "Greet me" ("Say hi and introduce yourself.")
and "Plan a weekend" ("Suggest a weekend plan based on my interests.")
were matching the bare `hi` and `plan` catch-alls in feature-parity.json
and returning the generic showcase-assistant blurb / 5-step content plan
instead of shared-state-aware responses. Added pill-specific fixtures in
shared-state.json (mirrored into d5-all.json) so the longer userMessage
substrings win first-match-wins ahead of feature-parity.
Auth sign-out: signing out unmounted CopilotKit entirely and bounced
the user back to the SignInCard, so the demo never showcased the
runtime returning 401 — its whole point. The QA contract in
qa/auth.md spelled out the intended UX. Restored it: CopilotKit stays
mounted after the first sign-in, the AuthBanner flips to an amber
"Signed out — the agent will reject your messages" state with a
re-Sign-in button, and CopilotKit's `onError` callback drives a
`data-testid="auth-demo-error"` surface that displays the runtime's
401 the moment the user sends an unauthenticated message. Updated the
e2e spec to match (the old "SignInCard re-mounts after sign-out" test
pinned the regression).
Gen-ui-agent: the aimock fixture short-circuited the 7-step
progression spelled out in `gen_ui_agent.py`'s SYSTEM_PROMPT to a
single set_steps call with all three steps already `completed`, so
the InlineAgentStateCard rendered the final 3/3 state instantly with
no sequential pending → in_progress → completed animation.
Regenerated as a 7-leg toolCallId chain per pill (8 fixtures × 3
pills): seed leg keyed on userMessage with NO `hasToolResult` gate
(matching PR #4770's pattern — `hasToolResult: false` would block the
seed from firing on the second pill in a multi-pill session), then
six toolCallId-keyed transitions, then a final narration. Fixture
order: toolCallId legs FIRST so the most specific match wins.
Multimodal D5: the sample-attachment buttons auto-send via
`agent.addMessage + copilotkit.runAgent` (restored in PR #4761), but
the D5 harness still typed `input` + pressed Enter via the runner
after `preFill`, sending a second user message that competed with the
in-flight image upload — the v1 LangGraph runtime SSE stream got
tangled (browser DevTools showed `statusCode: pending` indefinitely)
and the assistant message never rendered. Added `skipSend?: boolean`
to ConversationTurn (distinct from `skipFill`, which still presses
Enter once the textarea has content) and switched d5-multimodal.ts to
`skipSend: true` with `responseTimeoutMs: 60_000` so the runner waits
on the assistant response without poking the chat further. Bumped the
PDF auto-prompt fixture in feature-parity.json to include the word
"document" so the existing `buildModalityAssertion("document")` check
still lands.
D5 result: 37 → 39 of 40 features passing. Only
`tool-rendering-reasoning-chain` remains and is a separate
agent/runtime bug (Tokyo Responses-API `reasoning` message survives
into the next turn's conversation history, runtime returns
`RUN_ERROR: "message role is not supported"`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
154 lines
5.8 KiB
TypeScript
154 lines
5.8 KiB
TypeScript
import { test, expect } from "@playwright/test";
|
|
|
|
/**
|
|
* Auth demo lifecycle. The demo defaults to UNAUTHENTICATED on first
|
|
* paint and renders SignInCard. After sign-in, <CopilotKit> mounts with
|
|
* the bearer header attached and the chat boots. After sign-out the
|
|
* chat STAYS MOUNTED (now with no Authorization header) so the user can
|
|
* actually watch the runtime reject an unauthenticated send — that is
|
|
* the whole point of the demo. The AuthBanner flips between green
|
|
* (authenticated) and amber (signed-out) variants. Only a full page
|
|
* reload resets to the SignInCard first-paint state.
|
|
*/
|
|
test.describe("Authentication", () => {
|
|
test.beforeEach(async ({ page }) => {
|
|
await page.goto("/demos/auth");
|
|
});
|
|
|
|
test("page loads unauthenticated with SignInCard visible", async ({
|
|
page,
|
|
}) => {
|
|
await expect(
|
|
page.locator('[data-testid="auth-sign-in-card"]'),
|
|
).toBeVisible();
|
|
await expect(
|
|
page.locator('[data-testid="auth-sign-in-button"]'),
|
|
).toBeEnabled();
|
|
await expect(page.locator('[data-testid="auth-demo-token"]')).toBeVisible();
|
|
// Chat surface and AuthBanner only render after the first sign-in.
|
|
await expect(page.locator('[data-testid="auth-banner"]')).toHaveCount(0);
|
|
await expect(page.getByPlaceholder("Type a message")).toHaveCount(0);
|
|
});
|
|
|
|
test("signing in mounts the chat surface with AuthBanner", async ({
|
|
page,
|
|
}) => {
|
|
await page.locator('[data-testid="auth-sign-in-button"]').click();
|
|
|
|
const banner = page.locator('[data-testid="auth-banner"]');
|
|
await expect(banner).toBeVisible();
|
|
await expect(banner).toHaveAttribute("data-authenticated", "true");
|
|
await expect(page.locator('[data-testid="auth-status"]')).toContainText(
|
|
"Signed in",
|
|
);
|
|
await expect(
|
|
page.locator('[data-testid="auth-sign-out-button"]'),
|
|
).toBeEnabled();
|
|
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
|
|
// SignInCard is gone once we're authenticated.
|
|
await expect(page.locator('[data-testid="auth-sign-in-card"]')).toHaveCount(
|
|
0,
|
|
);
|
|
});
|
|
|
|
test("authenticated send produces an assistant response", async ({
|
|
page,
|
|
}) => {
|
|
await page.locator('[data-testid="auth-sign-in-button"]').click();
|
|
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
|
|
|
|
const input = page.getByPlaceholder("Type a message");
|
|
await input.fill("Hello");
|
|
await input.press("Enter");
|
|
|
|
await expect(
|
|
page.locator('[data-testid="copilot-assistant-message"]').first(),
|
|
).toBeVisible({ timeout: 30000 });
|
|
});
|
|
|
|
test("signing out flips the banner amber and keeps the chat surface mounted", async ({
|
|
page,
|
|
}) => {
|
|
await page.locator('[data-testid="auth-sign-in-button"]').click();
|
|
await expect(
|
|
page.locator('[data-testid="auth-sign-out-button"]'),
|
|
).toBeVisible();
|
|
|
|
await page.locator('[data-testid="auth-sign-out-button"]').click();
|
|
|
|
// Banner flips to the amber variant. SignInCard does NOT come back —
|
|
// the demo would never get to showcase the rejection if it did.
|
|
const banner = page.locator('[data-testid="auth-banner"]');
|
|
await expect(banner).toBeVisible();
|
|
await expect(banner).toHaveAttribute("data-authenticated", "false");
|
|
await expect(page.locator('[data-testid="auth-status"]')).toContainText(
|
|
"Signed out",
|
|
);
|
|
await expect(
|
|
page.locator('[data-testid="auth-authenticate-button"]'),
|
|
).toBeVisible();
|
|
await expect(
|
|
page.locator('[data-testid="auth-sign-out-button"]'),
|
|
).toHaveCount(0);
|
|
await expect(page.locator('[data-testid="auth-sign-in-card"]')).toHaveCount(
|
|
0,
|
|
);
|
|
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
|
|
});
|
|
|
|
test("unauthenticated send surfaces a 401 error without crashing the page", async ({
|
|
page,
|
|
}) => {
|
|
await page.locator('[data-testid="auth-sign-in-button"]').click();
|
|
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
|
|
await page.locator('[data-testid="auth-sign-out-button"]').click();
|
|
await expect(
|
|
page.locator('[data-testid="auth-authenticate-button"]'),
|
|
).toBeVisible();
|
|
|
|
// After sign-out, the next send must surface the rejection — not
|
|
// produce an assistant response and not white-screen the page.
|
|
const input = page.getByPlaceholder("Type a message");
|
|
await input.fill("Hello again");
|
|
await input.press("Enter");
|
|
|
|
const errorSurface = page.locator('[data-testid="auth-demo-error"]');
|
|
await expect(errorSurface).toBeVisible({ timeout: 15000 });
|
|
// Page chrome stays visible — no white-screen.
|
|
await expect(page.locator('[data-testid="auth-banner"]')).toBeVisible();
|
|
await expect(
|
|
page.locator('[data-testid="copilot-assistant-message"]'),
|
|
).toHaveCount(0);
|
|
});
|
|
|
|
test("re-signing in from the amber banner clears the error and resumes chat", async ({
|
|
page,
|
|
}) => {
|
|
await page.locator('[data-testid="auth-sign-in-button"]').click();
|
|
await expect(
|
|
page.locator('[data-testid="auth-sign-out-button"]'),
|
|
).toBeVisible();
|
|
await page.locator('[data-testid="auth-sign-out-button"]').click();
|
|
await expect(
|
|
page.locator('[data-testid="auth-authenticate-button"]'),
|
|
).toBeVisible();
|
|
|
|
await page.locator('[data-testid="auth-authenticate-button"]').click();
|
|
const banner = page.locator('[data-testid="auth-banner"]');
|
|
await expect(banner).toHaveAttribute("data-authenticated", "true", {
|
|
timeout: 5000,
|
|
});
|
|
await expect(page.locator('[data-testid="auth-demo-error"]')).toHaveCount(
|
|
0,
|
|
);
|
|
|
|
// Chat is responsive again.
|
|
const input = page.getByPlaceholder("Type a message");
|
|
await input.fill("Hello again");
|
|
await input.press("Enter");
|
|
await expect(
|
|
page.locator('[data-testid="copilot-assistant-message"]').first(),
|
|
).toBeVisible({ timeout: 30000 });
|
|
});
|
|
});
|