copilotkit__copilotkit/showcase/TESTING.md

# Test-Gating Matrix

This matrix documents which CI workflows fire on which triggers, what they test, and whether they gate merges. Companion to [`QA-COVERAGE.md`](./QA-COVERAGE.md) -- that document tracks per-demo coverage; this one tracks per-workflow gating.

Scope: all testing-related workflows (unit, integration, e2e, smoke) across the monorepo. Data read directly from `.github/workflows/*.yml` on the current branch.

## Matrix

| Workflow file                                       | Name (CI UI)                      | Trigger                                                                                     | Path filter                                                                            | Required? | What it tests                                                                                    |
| --------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------ |
| `.github/workflows/test_unit.yml`                   | test / unit                       | push (main), pull_request (main), workflow_dispatch                                         | paths-ignore: `docs/**`, `README.md`, `examples/**`                                    | No        | Vitest unit suite across Node 20/22/24 for all TS packages                                       |
| `.github/workflows/test_unit-python-sdk.yml`        | test / unit / python-sdk          | push (main), pull_request (main)                                                            | `sdk-python/**`, this workflow                                                         | No        | pytest against `sdk-python/` under Python 3.12 + Poetry                                          |
| `.github/workflows/test_integration-runtime.yml`    | test / integration / runtime      | push (main), pull_request (main), workflow_dispatch                                         | `packages/runtime/**`, this workflow                                                   | No        | Runtime server integration tests (Node, possibly others)                                         |
| `.github/workflows/test_integration-docs.yml`       | test / integration / docs         | push (main), pull_request                                                                   | `docs/**`                                                                              | No        | Extracts code blocks from docs, runs them against aimock (model-name + doc-test)                 |
| `.github/workflows/test_e2e-dojo.yml`               | test / e2e / dojo                 | push (main), pull_request (main), workflow_dispatch                                         | `packages/**`, `sdk-python/**`, this workflow, `.changeset`                            | No        | ag-ui dojo end-to-end matrix on Depot runners                                                    |
| `.github/workflows/test_e2e-legacy-v1.yml`          | test / e2e / legacy-v1            | push (main), pull_request (main), workflow_dispatch                                         | `examples/**`, this workflow, `.changeset`                                             | No        | Legacy v1.x examples (form-filling, travel, research-canvas, chat-with-your-data, state-machine) |
| `.github/workflows/showcase_validate.yml`           | Showcase: Validate                | push (main), pull_request                                                                   | `showcase/**`, `examples/integrations/**/fixtures/**`, `scripts/doc-tests/fixtures/**` | No        | Build-pipeline Vitest + manifest/registry validation + shell build                               |
| `.github/workflows/test_e2e-showcase-on-demand.yml` | test / e2e / showcase / on-demand | issue_comment (`/test-aimock`), workflow_dispatch                                           | n/a (comment-gated)                                                                    | No        | aimock-backed Playwright E2E on demand per-package                                               |
| `.github/workflows/test_smoke-starter.yml`          | test / smoke / starter            | schedule (`0 */6 * * *`), workflow_run (publish / release), pull_request, workflow_dispatch | `examples/integrations/**`, this workflow                                              | No        | Docker-compose smoke for 12 starter integrations (build + curl)                                  |
| `.github/workflows/test_smoke-starter-deployed.yml` | test / smoke / starter-deployed   | schedule (`0 */6 * * *`), workflow_run (Showcase: Build & Deploy), workflow_dispatch        | n/a (scheduled / post-deploy)                                                          | No        | Playwright E2E against live deployed starter URLs (@starter-health/-agent/-chat)                 |

## Which tests run on a typical PR?

- `packages/**` (runtime/SDK): `test / unit`, `test / integration`, `test / e2e / dojo`, `static / quality`, `static / check binaries`, plus `static / danger` if `packages/sdk-js/src/langgraph.ts` is touched.
- `sdk-python/**`: `test / unit / python-sdk`, `test / e2e / dojo`, `static / check binaries`, plus `static / danger` if `copilotkit/langgraph_agent.py` is touched. `test / unit` also fires (paths-ignore does not exclude sdk-python).
- `showcase/**`: `Showcase: Validate`, `test / unit` (paths-ignore does not exclude showcase), `static / quality`, `static / check binaries`. No Playwright E2E runs automatically -- comment `/test-aimock` on the PR to trigger `test / e2e / showcase / on-demand`.
- `examples/**` (legacy v1.x): `test / e2e / legacy-v1`, `static / check binaries`. `test / unit` is excluded via paths-ignore.
- `examples/integrations/**` (starters): `test / smoke / starter` (Docker), `Showcase: Validate` (for fixtures only), `static / check binaries`.
- `docs/**`: `test / integration / docs` only. `test / unit` and `static / quality` are excluded via paths-ignore.
- `.github/workflows/**`: each workflow that lists its own path in its trigger runs (most do). No single "workflows changed" catch-all.

## Required status checks

None of the workflows above are enforced as required status checks. The active `PROTECT_OUR_MAIN` ruleset on `main` requires zero status contexts -- merges are gated only by review approval, not by CI outcome.

The legacy classic branch protection `required_status_checks.contexts` array contains stale entries (e.g. `test / unit`, `Showcase: Validate`) that appear in GitHub's API responses but are not evaluated by the active ruleset. If you see a CI workflow marked as "required" in an older doc or script, treat that as ghost data: nothing in GitHub's current enforcement path consumes it.

Practical consequence: a red CI run does not block merge. Reviewers must eyeball `gh pr checks` before approving.

## Footnotes

- `test / unit` matrix is Node 20/22/24; the other workflows pin a single Node version each (22 most common).
- `test / e2e / dojo` uses Depot runners (`depot-ubuntu-24.04`); all others use standard GitHub runners.
- `test / smoke / starter` and `test / smoke / starter-deployed` both run every 6h; the former validates Docker-build integrity of `examples/integrations/`, the latter validates the deployed Railway services.
- `workflow_run` triggers fire after another workflow completes -- they do not gate the triggering PR, they run post-merge.