Bumps exact-duplicate ceiling 11 → 230 and substring-shadow ceiling
126 → 151. The D6 per-integration fixtures naturally share match keys
with pre-existing demo fixtures in the same context scope, disambiguated
at runtime by the active demo/probe path.
The four fixture routing regression tests still referenced the old
monolithic d5-all.json / smoke.json / feature-parity.json files that
were reorganized into per-integration d4/ d6/ shared/ directories.
Changes:
- Load fixtures via glob from d6/langgraph-python/ (reference
integration) instead of deleted monolithic files
- Add _context to test requests (aimock 1.26.1 checks req._context
against match.context for per-integration scoping)
- Adapt subagents test from toolCallId-based chaining to
turnIndex-based chaining (matches new D6 fixture structure)
- Adapt state-context test to D6 context-scoping model (no
systemMessage matching; routing happens via X-AIMock-Context)
- Remove _migrated-from-*.json shared files (all fixtures already
exist in per-integration D6 dirs; the migration files caused
620 shared-vs-scoped collisions)
- Add KNOWN_DUPLICATE_CEILING=11 ratchet for pre-existing D6
intra-feature duplicate match keys
Add fixture analysis/split/merge scripts for the D6 restructure,
update aimock fixture collision detection tests for the new directory
layout, update GOTCHAS.md and QA-COVERAGE.md with D6 notes, and
add --d6 flag support to the showcase test CLI.
- Add --validate-on-load to all aimock invocations (4 workflows/scripts
+ 13 integration docker-compose files)
- Replace hardcoded 2-file fixture list with dynamic discovery across
showcase/, examples/integrations/*/, scripts/doc-tests/ (16 fixtures)
- Add sanity check to prevent silent zero-test pass when discovery fails
- Extend showcase_validate.yml path filter to trigger on
examples/integrations/**/fixtures/** and scripts/doc-tests/fixtures/**
- Import and use ValidationResult type for callback parameters
- Fix scripts/doc-tests/fixtures/default.json to use { fixtures: [...] }
envelope shape
Follow-up to #3971. aimock supports fixture schema validation at startup via
--validate-on-load, but it's opt-in. The showcase Dockerfile did not pass
the flag, so fixtures with unrecognized response keys (e.g. "text" instead
of "content") loaded silently and only failed at request time with HTTP 500.
That's what crashed crewai-crews on startup.
Changes:
- showcase/aimock/Dockerfile: pass --validate-on-load so broken fixtures
fail the container boot, not individual requests.
- showcase/scripts/__tests__/aimock-fixtures.test.ts: new vitest spec that
loads feature-parity.json and smoke.json via @copilotkit/aimock's
loadFixtureFile + validateFixtures and asserts zero errors. Runs as part
of the existing showcase-validate CI workflow.
- showcase/scripts/package.json: add @copilotkit/aimock dependency for the
validator import.
Verified red-green: with the pre-#3971 broken "text" fixtures, validateFixtures
flags 5 errors; post-#3971 it returns zero. Docker red-green: container with
an intentionally broken fixture fails to start with "Validation failed: 1
error(s)" and non-zero exit.
Jordan Ritter