This PR:
- builds on merged PR #4107
- adds `examples-manifest.json`: 93 runnable entrypoints across both
SDKs with tiers (unattended / provisioned / bounded / excluded),
per-entry timeouts, env contracts, and labeled readiness markers for
OAuth flows
- adds `harness/run.mjs`: sweeps entries against staging with per-entry
Composio traces, negative controls (garbage credentials must fail), a
`selftest`, and a baseline/candidate client swap for parity runs
- rejects every non-staging `COMPOSIO_BASE_URL` before the harness or
provisioning script can send credentials
- adds `--llm mock`: LLM examples run against a pinned
`@copilotkit/aimock` server with scripted tool-call fixtures
(`harness/llm-mock/`) — full sweeps spend zero model tokens while
Composio tool execution stays live on staging; 46/48 LLM entries pass
mocked, and the two `@openai/agents` entries are marked `llmMock:false`
- adds `examples-live.yml`: nightly gc-provision → mock sweep →
four-entry live LLM canary; dispatchable per language, client, entry
subset, and LLM mode, and bound to the `staging` GitHub environment
- accepts separate TypeScript tarball and Python wheel inputs for
candidate sweeps, restores exact pre-run workspace files on failure, and
rejects empty or partial parity comparisons
The configured `COMPOSIO_API_KEY` must belong to a dedicated, disposable
staging project. The workflow becomes dispatchable after it exists on
the default branch.
## Context
Examples historically went stale because nothing executed them. PR #4107
makes them runnable and provisionable; this PR adds the machinery that
runs them on a schedule and validates the Stainless→self-managed client
swap via baseline/candidate parity sweeps.
```mermaid
graph TD
M[examples-manifest.json] --> R[harness/run.mjs sweep]
P[examples-provision.mjs --gc] -->|COMPOSIO_EXAMPLES_* ids| R
R -->|tier 1-3 entries| E[TS + Python examples]
A[aimock fixtures] -->|--llm mock| E
E -->|live tool execution + traces| C[Composio staging]
R --> N[nightly live canary: 4 entries]
```
## Verification
- `env -u COMPOSIO_BASE_URL node harness/run.mjs selftest`
- `pnpm test:examples` against a clean tracked snapshot
- manifest integrity: 93 unique entry IDs and 93 existing files
- JavaScript, Python, JSON, YAML, and embedded Bash syntax checks
- `pnpm validate:changesets`
## Summary
- Repairs runnable TypeScript and Python examples for current backend
requirements, including authenticated MCP endpoints, current transports,
valid tool identifiers, provider limits, and resource uniqueness.
- Replaces placeholder resource IDs with explicit `COMPOSIO_EXAMPLES_*`
configuration and makes failed examples exit loudly.
- Adds `scripts/examples-provision.mjs` as an idempotent provisioning
check for a disposable examples project.
## Scope
- This PR no longer changes the Python SDK runtime or generated-client
dependency.
- Python remains pinned to the published `composio-client==1.43.0` in
`pyproject.toml`, `setup.py`, and `uv.lock`.
- The owned 2.x client integration is deferred until that client
completes its release guarantees and is explicitly published.
- The scheduled live workflow and manifest remain deferred until their
runner is tracked.
## Verification
- `make chk`
- `make tst` (`927 passed, 33 skipped`)
- Hosted checks rerun against commit `12691aca7`.
Alberto Schiabel