Replace single-file volume mounts with directory mounts (shared/, d4/,
d6/) so new fixture files are picked up without editing compose files.
Remove --proxy-only flag to catch fixture gaps instead of silently
falling through to real API. Add LANGGRAPH_HTTP env var from main.
Adds a record/replay loop on top of the existing local Docker stack so
each langgraph-python D5 demo can capture real-LLM responses once and
replay them deterministically thereafter. One fewer source of D5 flake
per demo: no more 'tests pass with real OpenAI / fail with aimock'
because the prompts have no fixture coverage.
What lands here:
- showcase/docker-compose.record.yml — overlays aimock with --record
+ --provider-openai/anthropic and a writable mount for the recording
dir. Drops the baseline d5-all.json/feature-parity.json/smoke.json
loads so prompts that already match a stale fixture can still proxy
through to the real provider.
- showcase/docker-compose.replay.yml — same writable mount, no
--record, no provider URLs; layers the per-demo fixtures alongside
the baseline ones for normal probe runs.
- showcase/scripts/record-d5-fixtures.mjs — orchestrator. For each
demo (catalog feature ID), drops any prior consolidated <slug>.json,
restarts aimock to clear in-memory recorded fixtures, snapshots
recorded/, runs the d5 probe through pnpm exec tsx, then merges
every per-call file written under recorded/ into a single
showcase/aimock/d5-recorded/<slug>.json (one fixture per LLM turn,
in chronological order).
- showcase/aimock/d5-recorded/<slug>.json × 6 — initial recordings
for the still-red bucket-C cells: beautiful-chat (8), gen-ui-interrupt
(2), gen-ui-tool-based (1), headless-complete (1), reasoning-custom
(1), tool-rendering-default-catchall (2). 15 fixtures total.
- showcase/aimock/d5-recorded/.gitignore — keeps the per-call
recorded/ scratch dir out of the tree (orchestrator deletes per-call
files after consolidation, this just guards against re-runs).
D5 impact: replaying the recordings flips
d5:langgraph-python/gen-ui-custom (catalog gen-ui-tool-based) from red
to green deterministically. The other five demos still fail their UI
or assertion-side checks, but their LLM-side responses are now fixed,
so the remaining work is probe/UI fixes against a stable baseline
rather than flake hunting.
Aimock recorder requires a one-line patch (turnIndex + hasToolResult
on each recorded fixture's match) for multi-turn flows to record
correctly. Upstream fix proposed for @copilotkit/aimock; until it
ships, the orchestrator probes for the patch and aborts loudly with
the missing-fields message rather than silently producing single-turn
fixtures. See the script's header comment for the exact patch payload.
Ran Shem Tov