Alberto Schiabel
6e0a9db3c7
This PR: - fixes [UXE-233](https://linear.app/composio/issue/UXE-233/docs-should-explicitly-direct-agents-to-use-v31-apis) - makes the agent-facing Markdown channels publish concrete REST v3.1 base URLs and endpoint tables while preserving the supported v3.0 reference tree - centralizes `REST_VERSION_GUIDANCE`, `TOOL_VERSION_GUIDANCE`, and raw-spec path matching in `lib/api-version-guidance.ts` - renders `ApiBaseUrl` and `ApiEndpointsTable` in authored MDX and adds an explicit version pointer to generated OpenAPI operation Markdown - separates current and legacy REST references in `llms.txt`, excludes v3.0 page bodies from `llms-full.txt`, and adds v3.1 selection guidance to Context7 - validates serialized `ApiEndpointsTable` payloads with Zod before generation while preserving forward-compatible fields - addresses review feedback for the renamed authentication page, SDK-reference pointer scope, generator validation behavior, and stale OpenAPI tool-version descriptions ## Context REST v3.0 is superseded but remains supported for existing integrations. This PR changes what new agent-generated code discovers first; it does not require existing v3.0 callers to migrate. Authenticated read-only probes against the deployed API confirmed that the affected v3 endpoints default to `00000000_00`, while their v3.1 counterparts default to `latest`. `POST /tools/scopes/required` is available only on v3.1 and defaults to `latest`. This PR does not move public URLs. A future `/reference/v3/` to `/reference/v3.0/` migration remains separate because it has independent compatibility and search-indexing risk. ## Verification - `bun test tests/static/` - `bun run build` - `bun run test:integration` - `bun run types:check` - `bun run lint` ## Review follow-up The version-default guidance is intentionally limited to the five verified tool endpoints. v3.1 is a structural superset of v3, so this PR does not claim route parity. Static coverage rejects broad non-tool parity wording in both the shared guidance and Context7 rules.
OpenAPI Scripts
fetch-with-retry.ts
Shared fetchWithRetry helper used by the data-generation scripts
(generate-toolkits.ts, generate-meta-tools.ts). It wraps the global fetch
with rate-limit-aware retry/backoff: it retries 429 and transient 5xx
responses, honors the Retry-After header when present, otherwise falls back to
exponential backoff with jitter, and caps attempts so CI still fails fast when
the backend is genuinely down.
This matters because generate-toolkits.ts issues ~6500 requests per run
(a few catalog pages + 3 per toolkit across a ~2.1k catalog), which can exceed
the backend request limit.
Before this helper, runs failed with 429, and generate-meta-tools.ts — which
runs immediately after — inherited the exhausted rate-limit window.
Toolkit versions
generate-toolkits.ts owns the complete toolkit catalog and always sources it
from the production API. For a version-only repair, run the narrower generator:
COMPOSIO_API_KEY=... bun run generate:toolkit-versions
Both paths share toolkit-versions.ts, so they fetch and apply changelog values
with identical semantics: a toolkit missing from the production changelog gets
version: null. Any COMPOSIO_API_BASE override must normalize to
https://backend.composio.dev/api/v3; non-production sources fail before a
request is made.
fetch-openapi.mjs
Fetches the Composio OpenAPI spec and filters it for use in Fumadocs API reference documentation.
Usage
bun run scripts/fetch-openapi.mjs
This outputs public/openapi.json which is used by lib/openapi.ts.
Why Filtering is Needed
The raw OpenAPI spec from https://backend.composio.dev/api/v3/openapi.json has issues that break documentation generators:
- Endpoints with multiple tags - Causes duplicate entries in sidebar
- Internal endpoints exposed - CLI, Admin, Profiling endpoints shouldn't be in public docs
See OPENAPI_IMPROVEMENTS.md in the fumadocs root for planned fixes to the spec itself.
What Gets Filtered
Ignored Paths
These endpoints are completely removed:
/api/v3/mcp/validate/{uuid}/api/v3/cli/get-session/api/v3/cli/create-session/api/v3/auth/session/logout
Ignored Tags
Endpoints with only these tags are removed:
CLIAdminProfiling
Duplicate Prevention
If an endpoint has multiple tags, only the first tag is kept. This prevents the same endpoint appearing in multiple sidebar sections.
Configuration
The script uses environment variables:
| Variable | Default | Description |
|---|---|---|
OPENAPI_SPEC_URL |
https://backend.composio.dev/api/v3/openapi.json |
Source OpenAPI spec URL |
For staging deployments, set:
OPENAPI_SPEC_URL=https://staging.composio.dev/api/v3/openapi.json