Commit Graph

14 Commits

Author SHA1 Message Date
Ran Shem Tov 620cf00974 fix(showcase): green strands + strands-typescript D6 cells via id-invariant fixtures + real multimodal assets
Strands mints UUID tool_call_ids for tool results (confirmed via the aimock
journal), so the langgraph-python toolCallId-keyed follow-up fixtures never
matched and the agent re-emitted the tool, looping (text-unstable) across
frontend-tools, gen-ui-agent, gen-ui-open, gen-ui-open-advanced,
gen-ui-headless-complete, reasoning-chain and the weather pills. Re-key the
affected multi-leg fixtures to the id- and thread-history-invariant
sequenceIndex pattern (as built-in-agent does), for both integrations.

Also:
- Add write_document poem/email/quantum fixtures for shared-state-streaming
  (was a stale single fixture that 404d).
- Strip content from reasoning-chain tool legs (content+toolCalls in one
  fixture is undefined behavior); reasoning rides toolCalls alone.
- Narrow the over-broad d4 summarize catch-all to "Summarize the sales
  pipeline" so it stops shadowing gen-ui-agent competitor pill; matches
  langgraph-python.
- Commit the real multimodal sample.png/sample.pdf (were git-LFS pointers
  the harness could not resolve).
2026-06-23 19:46:43 -07:00
Ran Shem Tov 8768c8a7e7 Merge remote-tracking branch 'origin/main' into claude/trusting-babbage-f4d48a 2026-06-22 16:11:21 +02:00
Ran Shem Tov 851dc1e90d feat(showcase): strands-typescript parity with strands-python (A2UI, aimock fixtures, header forwarding)
Bring the TypeScript AWS Strands integration to parity with the Python
strands sibling now that the @ag-ui/aws-strands TS adapter is confirmed to
support the same feature surface (per its examples/server):

- Restore A2UI: the declarative-gen-ui + a2ui-fixed-schema demos, their
  routes, qa, specs, the @copilotkit/a2ui-renderer dep, beautiful-chat's
  A2UI catalog, and the manifest entries (generative_ui / features / demos /
  a2ui_pattern). manifest now matches strands-python feature-for-feature.
- Header forwarding: attach `x-aimock-context: strands-typescript` as a
  static defaultHeader on the OpenAI client (model-factory + sub-agent
  client) — the TS analog of the Python integration's _header_forwarding
  shim — so aimock matches this integration's fixtures.
- aimock fixtures: add d6/strands-typescript + d4/strands-typescript
  (ported from the Python sibling, context retargeted).
- playwright.config: X-AIMock-Context → strands-typescript.

Note: the raw tests/e2e Playwright suite is flaky and not a CI merge gate
(demo e2e / `/eval` D5 are comment-triggered, not required) — it fails the
same specs for strands-python too. The auto-gates (build, validate-
constraints, oxlint/oxfmt, unit) are green.
2026-06-22 14:36:04 +02:00
Jordan Ritter e10df0b489 feat(showcase): add ms-agent-harness-dotnet d6/d4 aimock fixtures 2026-06-19 02:28:33 -07:00
Mark Fogle 53801f8e04 test(showcase): make sales-dashboard e2e reproduce real-model catalogId omission
The injected/streamed a2ui fixtures all included catalogId, so aimock
replay never exercised the basic-catalog fallback that broke production
(real models omit catalogId per the tool-usage guide). Strip catalogId
from the langgraph-python sales-dashboard secondary-call fixtures and
hard-assert "Catalog not found" is absent outside the charts-rendered
soft branch, so the spec fails without a route defaultCatalogId.

Also repoint the on-demand e2e workflow at the d4/d5-recorded/d6/shared
fixture dirs — it still referenced feature-parity.json, deleted in the
1e66a5f8d fixture reorg, so every /test-aimock run died at aimock start.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 18:47:16 +00:00
Jordan Ritter 95f726a408 fix(showcase/aimock): consolidate content-red fixture fixes — d4 broad-matcher narrowing, d6 per-family content-shape alignment, chunkSize emoji guard, threadid context gates 2026-06-12 08:00:05 -07:00
Jordan Ritter 47d3b594ce fix(showcase): align ms-agent-python D6 fixtures to LGP (152/33 → 174/11)
Same pattern as ms-agent-dotnet: full d4 chat.json rewrite + d6 mirrors. Dropped stale
turnIndex gates and broad shadow matchers. Default-catchall now green via page-level
shadcn-catchall-renderer. 11 residual: declarative-gen-ui charts, multimodal conveyance,
tool-rendering, reasoning-chain.
2026-05-30 09:53:35 -07:00
Jordan Ritter afa4cf0cf2 fix(showcase): align ms-agent-dotnet D6 fixtures to LGP (155/27 → 177/5)
Full d4 chat.json rewrite + d6 mirrors. Dropped stale turnIndex gates and broad shadow
matchers that no longer reflect LGP-canonical conveyance. Default-catchall now green via
page-level shadcn-catchall-renderer (not react-core-gated). 5 residual: chat-slots, hitl,
interrupt-headless, readonly-state.
2026-05-30 09:53:25 -07:00
Jordan Ritter a49ba1ef74 fix(showcase): align google-adk D6 fixtures to LGP contract
Narrowed d4 shadow matchers, removed whole-thread gates, render_a2ui inner tool
name, root chunkSize for emoji surrogate pairs, sentinel-disabled the duplicate
open-gen-ui-advanced matcher.
2026-05-29 23:59:41 -07:00
Jordan Ritter e8f38c6129 fix(showcase): narrow LGT D6 d4 shadows + wire sandboxed-ui jsFunctions
Mirrors the LGP first-match-wins fix on the langgraph-typescript side. Three broad
matchers in d4/langgraph-typescript/chat.json were shadowing d6 fixtures; narrowed
them so the d6 pills win.

Across the d6/langgraph-typescript suite: dropped fragile turnIndex:0 gates in favour
of hasToolResult:false for first-leg tool emissions, fixed em-dash escaping that broke
literal string matches in multi-pill threads, and added jsFunctions payloads to the
three sandboxed-ui fixtures (_from-feature-parity, headless-complete,
gen-ui-open-advanced) so the sandbox renderer has executable handlers.

LGT D6 now passes 184/1/2 locally. Residual 1 fail is the custom-catchall multi-pill
follow-up case; tracked separately.
2026-05-29 21:08:05 -07:00
Jordan Ritter 508bea0e88 fix(showcase): unblock LGP D6 beautiful-chat + custom-catchall via d4 matcher narrowing
Aimock fixture load order is shared -> d4 -> d6, and matching is first-match-wins. A
broad d4 langgraph-python chat fixture for search_flights was shadowing the d6
beautiful-chat fixture and preventing the intended response from firing. Neutered the
d4 matcher to a non-matching sentinel so the d6 fixture wins.

Also refreshed d6/tool-rendering-custom-catchall.json and d6/tool-rendering.json:
replaced fragile turnIndex:0 gates with hasToolResult:false so the AAPL first-leg
fixture fires correctly when D5 probes run a prior 'weather in Tokyo' pill in the
same thread (multi-pill turnIndex>=2).

LGP D6 now passes 185/0/2 locally (2 skips are the by-design mcp-apps iframe gap).
2026-05-29 21:08:05 -07:00
Jordan Ritter c414fbad23 test(showcase): D6 per-integration aimock fixtures + shared narrowing
Refresh d6 fixtures across the rollout cohort: ag2, built-in-agent,
claude-sdk-{python,typescript}, crewai-crews, google-adk, langgraph-fastapi,
langgraph-typescript, langroid, llamaindex, mastra, ms-agent-{dotnet,python},
pydantic-ai, strands. Companion d4/{langgraph-typescript,mastra,ms-agent-dotnet}
chat.json refreshes. Add the missing ms-agent-python/gen-ui-custom.json
to bring the integration up to the standard pill set.

Also narrow aimock/shared/common.json's generic 'hello' fixture to
'hello world' so it no longer shadows D6 pills whose prompts contain
'hello' as a substring (e.g. langgraph-python headless-simple sends
'Say hello in one short sentence.'). 'hello world' is unused by any
current demo pill, so the fixture remains a manual-typing fallback
without poisoning fixture matching.

This is a mid-rollout snapshot — fixture coverage is uneven across
integrations and rides alongside the conveyance shims landed earlier
in this branch.
2026-05-29 16:16:12 -07:00
Jordan Ritter 5f21584b3a test(showcase): D6 langgraph-python fixtures — drive to 180/5/2
Refresh d6/langgraph-python fixtures (beautiful-chat, chat-slots,
frontend-tools-async, gen-ui-{agent,custom,declarative,interrupt,open},
headless-complete, hitl-in-app, interrupt-headless, shared-state-streaming,
tool-rendering, tool-rendering-reasoning-chain, _from-feature-parity) plus
d4/langgraph-python/chat.json. Combined with the conveyance shim landed
in this branch, LGP is fixture-complete at 180 pass / 5 fail / 2 skip.

The residual 5 fails trace to react-core/v2's custom-wildcard-renderer
bug surfaced through gen-ui-interrupt (shared useInterrupt hook misbehaves
on the second interrupt; cross-integration D6 blocker, not a fixture or
conveyance defect). The 2 skips are the by-design mcp-apps iframe gap.
2026-05-29 16:15:57 -07:00
Jordan Ritter 1e66a5f8d2 feat(showcase-aimock): per-framework D4/D6 fixture reorg into d4/d6/shared dirs
Move monolithic d5-all.json + feature-parity.json + smoke.json into
per-integration directories under d4/<slug>/, d6/<slug>/, and shared/.
Every fixture file is now context-scoped to enable server-side aimock
routing via match.context. Migrated 12 HITL fixtures from main's
d5-all.json additions into shared/_migrated-from-d5-all-hitl.json
for follow-up distribution into per-integration files.
2026-05-26 11:25:30 -07:00