mirror of
https://github.com/ComposioHQ/composio.git
synced 2026-09-22 11:46:35 +08:00
Alberto Schiabel
71ec523d25
This PR: - builds on merged PR #4107 - adds `examples-manifest.json`: 93 runnable entrypoints across both SDKs with tiers (unattended / provisioned / bounded / excluded), per-entry timeouts, env contracts, and labeled readiness markers for OAuth flows - adds `harness/run.mjs`: sweeps entries against staging with per-entry Composio traces, negative controls (garbage credentials must fail), a `selftest`, and a baseline/candidate client swap for parity runs - rejects every non-staging `COMPOSIO_BASE_URL` before the harness or provisioning script can send credentials - adds `--llm mock`: LLM examples run against a pinned `@copilotkit/aimock` server with scripted tool-call fixtures (`harness/llm-mock/`) — full sweeps spend zero model tokens while Composio tool execution stays live on staging; 46/48 LLM entries pass mocked, and the two `@openai/agents` entries are marked `llmMock:false` - adds `examples-live.yml`: nightly gc-provision → mock sweep → four-entry live LLM canary; dispatchable per language, client, entry subset, and LLM mode, and bound to the `staging` GitHub environment - accepts separate TypeScript tarball and Python wheel inputs for candidate sweeps, restores exact pre-run workspace files on failure, and rejects empty or partial parity comparisons The configured `COMPOSIO_API_KEY` must belong to a dedicated, disposable staging project. The workflow becomes dispatchable after it exists on the default branch. ## Context Examples historically went stale because nothing executed them. PR #4107 makes them runnable and provisionable; this PR adds the machinery that runs them on a schedule and validates the Stainless→self-managed client swap via baseline/candidate parity sweeps. ```mermaid graph TD M[examples-manifest.json] --> R[harness/run.mjs sweep] P[examples-provision.mjs --gc] -->|COMPOSIO_EXAMPLES_* ids| R R -->|tier 1-3 entries| E[TS + Python examples] A[aimock fixtures] -->|--llm mock| E E -->|live tool execution + traces| C[Composio staging] R --> N[nightly live canary: 4 entries] ``` ## Verification - `env -u COMPOSIO_BASE_URL node harness/run.mjs selftest` - `pnpm test:examples` against a clean tracked snapshot - manifest integrity: 93 unique entry IDs and 93 existing files - JavaScript, Python, JSON, YAML, and embedded Bash syntax checks - `pnpm validate:changesets`
26 lines
609 B
JavaScript
26 lines
609 B
JavaScript
export const STAGING_BASE_URL = 'https://staging-backend.composio.dev';
|
|
|
|
export const requireStagingBaseUrl = (
|
|
value = process.env.COMPOSIO_BASE_URL ?? STAGING_BASE_URL
|
|
) => {
|
|
let url;
|
|
try {
|
|
url = new URL(value);
|
|
} catch {
|
|
throw new Error(`refusing invalid COMPOSIO_BASE_URL: ${value}`);
|
|
}
|
|
|
|
const isStagingRoot =
|
|
url.origin === STAGING_BASE_URL &&
|
|
url.pathname === '/' &&
|
|
!url.search &&
|
|
!url.hash &&
|
|
!url.username &&
|
|
!url.password;
|
|
if (!isStagingRoot) {
|
|
throw new Error(`refusing non-staging COMPOSIO_BASE_URL: ${value}`);
|
|
}
|
|
|
|
return STAGING_BASE_URL;
|
|
};
|