Commit Graph

3 Commits

Author SHA1 Message Date
Lukas Moschitz a19ed403cf fix(showcase): port langgraph-fastapi tool-rendering-reasoning-chain to parity
Two independent fastapi divergences from the green north-star kept this red:

1. Missing chunkSize:9999 on the 8 tool-call fixtures. Under aimock's global
   8-byte chunking the large tool-call args failed to JSON-parse in one piece,
   so the AAPL->MSFT reasoning chain stopped at leg 1 (turns 1 & 2). Byte-align
   to LGP (and the green mastra/langgraph-typescript siblings carry the same
   pattern).

2. Fixture-pool shadowing broke turn 3. aimock pools all d4+d6 fixtures for a
   context and matches by userMessage substring, first-match-wins in load order
   (d4 before d6). fastapi had broad keys LGP doesn't:
   - d4/chat.json: 'weather' -> 'weather in San Francisco', 'flights from SFO
     to JFK' -> 'Find flights from SFO to JFK.' (period-terminated so it stops
     being a substring of turn 3's 'Find flights from SFO to JFK and show me
     the weather there.').
   - tool-rendering-{custom,default}-catchall.json: bare 'Find flights' ->
     'Find flights from SFO to JFK.' (now matches LGP's exact key).

Also byte-align the backend agent to LGP: scriptable get_stock_price signature,
the detailed chaining system prompt, model gpt-5.4, reasoning summary detailed.

D6 tool-rendering-reasoning-chain green (two real ~21s runs). Shared-fixture
regression all green: agentic-chat, tool-rendering, tool-rendering-custom-catchall,
tool-rendering-default-catchall, headless-complete (Tokyo-weather consumer).
2026-07-27 11:01:18 +02:00
Jordan Ritter 07e4998798 fix(showcase/aimock): add forecast-Tokyo + AAPL catchall fixtures to flip remaining D5 reds
The catchall userMessage rename (#5453) only renamed STALE strings to 'forecast for Tokyo'. Integrations whose catchall fixtures used PILL-ALIGNED userMessages ('What's the weather in San Francisco?', 'Find flights from SFO to JFK.', 'Chain a few tools in this single turn', etc.) had no 'forecast for Tokyo' matcher to begin with — so the D5 catchall probe still fell through to live LLM and the cells stayed red on staging.

This PR ADDS the canonical 'forecast for Tokyo' (default+custom catchall) and 'What's the current price of AAPL?' (custom catchall only) emit+narrate fixture pairs to each integration that was missing them. Existing pill-aligned fixtures are preserved (pure prepend at the start of the fixtures array; first-match-wins means new fixtures match the D5 probe inputs without colliding with existing pill prompts).

Integrations touched:
- langgraph-typescript: both default + custom
- langgraph-fastapi: both default + custom
- google-adk: custom only (default already green)
- ms-agent-dotnet: custom only (default already green)

After merge, fleet-cp's e2e-deep probe will re-run within ≤6h staleness window and flip cells GREEN.
2026-06-15 15:00:53 -07:00
Jordan Ritter 69c964aca2 feat(showcase-aimock): D6 fixtures for LangGraph-based integrations
Adds per-integration D6 fixtures for langgraph-python, langgraph-typescript,
and langgraph-fastapi. Each fixture is keyed by match.context for cross-
integration isolation and uses hasToolResult:false on toolCall responses
to prevent re-match loops.
2026-05-26 14:02:17 -07:00