mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Alem Tuzlak
7f9da6919b
## Summary The 6 beautiful-chat demos (spring-ai, strands, langroid, agno, claude-sdk-typescript, claude-sdk-python) ship three identical suggestion chips. Against the deployed aimock-backed showcase, all three were broken: | Suggestion | Symptom | Cause | |---|---|---| | Plan a 3-day Tokyo trip | Returned a generic "Hi there! I'm your showcase assistant…" greeting | Substring `"hi"` matches inside **arc*hi*tecture**, hijacked by the broad `userMessage: "hi"` fixture | | Explain RAG like I'm 12 | aimock 4xx — `"No fixture matched"` | No fixture | | Draft a launch email | aimock 4xx — `"No fixture matched"` | No fixture | Verified locally against `showcase up spring-ai` in a headed browser — all three now return on-topic content. ## Fix Add three full-sentence fixtures before the broad `"hi"` matcher in `feature-parity.json`. Aimock's matcher is substring + first-match-wins by file order, so the long sentence matchers win first and the `"hi"` fixture is never reached for these prompts. Each returns a plausible markdown response (3-day Tokyo itinerary, open-book-test analogy, 3-paragraph launch email). ## Regression coverage Replaced the 1-line beautiful-chat placeholders with a 4-test suite for all 6 integrations: - Page loads with heading + chat input - Each suggestion's reply contains the expected keywords (`Day 1|Day 2|Day 3`, `open-book|retrieval|RAG`, `Subject:|co-pilot|launch`) - Each test ALSO asserts `toHaveCount(0)` against `/I'm your showcase assistant/i` — if the broad "hi" fixture re-broadens or the new fixtures are reordered/removed, the tests fail with a useful message. ## Test plan - [x] `validate-fixture-tool-surface` clean: 141 fixtures × 628 demos, no drift - [x] Manual headed-browser verification on `showcase up spring-ai` — all 3 suggestions return their on-topic responses - [ ] CI's `Validate Showcase` job stays green - [ ] On-demand E2E (`/test-aimock <slug>`) passes for any of the 6 integrations