This PR:
- Closes https://github.com/ComposioHQ/composio/issues/4343
- stamps eve's durable callback descriptors on every tool `EveProvider`
wraps, via the new internal `withDurableClosure(closure, callback)`
helper — eve only stamps them on `defineTool` calls its build transform
finds in the agent's own source, which never runs on this package inside
`node_modules`, so eve discarded the whole resolver result and the agent
silently lost every Composio tool
- persists `{ slug, binding }` per callback, where `binding` is an id
minted per `wrapTools` call and prefixed with a per-process token, and
re-attaches it to that resolve's Composio executor through a
module-level binding map. `executeTool` is bound to one Composio
session, so a slug-only closure would have routed a call to whichever
session resolved last; sessions for different users share one provider,
and eve's callback registry is keyed by tool name only, so the map lives
at module level rather than on the instance
- covers `execute` and, when `needsApproval` is set, `approvalRequest`;
the descriptor key is the global-registry symbol
`Symbol.for('eve:durable-dynamic-callback')`, so no eve internal is
imported and the stamp is inert on eve versions that predate the
contract
- adds 9 regression tests: descriptor presence and shape,
JSON-serializability of the closure, replay of execute and approval from
the closure alone, per-resolve executor isolation when sessions share a
provider, hooks of the producing provider on replay, the unknown-slug
and unknown-binding errors, that two fresh module instances never mint
the same binding id, and one suite that loads eve 0.52.1's own
`validateDurableDynamicToolCallbacks`, `replayDynamicTools`, and
callback registry from the installed package to validate and replay a
wrapped tool end to end. Before the change eve threw `Dynamic tool "..."
callback "execute" does not have a durable descriptor`
## Context
The reporter hit this on eve 0.50 as `non-serializable capture`; 0.52.1
reports the same root cause as a missing descriptor. eve exports no
public durable-callback helper (tracked at vercel/eve#2967), so the
provider stamps the descriptor itself rather than pinning users to an
older eve.
Bindings are kept for the life of the process: eve can resume a parked
call at any time. A binding lives only in the process that resolved the
tools, so a call parked across a restart cannot be replayed; the
per-process token in the id makes the stale closure fail the lookup
loudly instead of matching whichever resolve reused its counter value in
the new process. Growth is one entry per `session.tools()` resolve.
Docs (`/docs/providers/eve`) now state the contract, the restart limit,
and the real reason the `step.started` resolver runs each step
(principal re-evaluation and retry, cached per session).
Also unblocks `Docs - Tests` on this branch: the catalog refresh in
#4330 renamed Stripe's triggers, so the Stripe knowledge-base guide
cited two dead slugs and the corpus verifier failed for any PR touching
docs. The guide now cites only the renamed slug the catalog lists, and
`generate-toolkits.ts` fetches trigger types with `limit=1000` so the
catalog stops truncating every toolkit to its first 20 triggers.
https://claude.ai/code/session_019wRk1S4Z6V6FWr4UsybGvR
EOF -R ComposioHQ/composio
This PR:
- fixes
[UXE-233](https://linear.app/composio/issue/UXE-233/docs-should-explicitly-direct-agents-to-use-v31-apis)
- makes the agent-facing Markdown channels publish concrete REST v3.1
base URLs and endpoint tables while preserving the supported v3.0
reference tree
- centralizes `REST_VERSION_GUIDANCE`, `TOOL_VERSION_GUIDANCE`, and
raw-spec path matching in `lib/api-version-guidance.ts`
- renders `ApiBaseUrl` and `ApiEndpointsTable` in authored MDX and adds
an explicit version pointer to generated OpenAPI operation Markdown
- separates current and legacy REST references in `llms.txt`, excludes
v3.0 page bodies from `llms-full.txt`, and adds v3.1 selection guidance
to Context7
- validates serialized `ApiEndpointsTable` payloads with Zod before
generation while preserving forward-compatible fields
- addresses review feedback for the renamed authentication page,
SDK-reference pointer scope, generator validation behavior, and stale
OpenAPI tool-version descriptions
## Context
REST v3.0 is superseded but remains supported for existing integrations.
This PR changes what new agent-generated code discovers first; it does
not require existing v3.0 callers to migrate.
Authenticated read-only probes against the deployed API confirmed that
the affected v3 endpoints default to `00000000_00`, while their v3.1
counterparts default to `latest`. `POST /tools/scopes/required` is
available only on v3.1 and defaults to `latest`.
This PR does not move public URLs. A future `/reference/v3/` to
`/reference/v3.0/` migration remains separate because it has independent
compatibility and search-indexing risk.
## Verification
- `bun test tests/static/`
- `bun run build`
- `bun run test:integration`
- `bun run types:check`
- `bun run lint`
## Review follow-up
The version-default guidance is intentionally limited to the five
verified tool endpoints. v3.1 is a structural superset of v3, so this PR
does not claim route parity. Static coverage rejects broad non-tool
parity wording in both the shared guidance and Context7 rules.
This PR:
- builds on top of https://github.com/ComposioHQ/composio/pull/3966
- enables `typescript/no-explicit-any` (error, `fixToUnknown`) for docs
in `docs/.oxlintrc.json` and removes every remaining explicit `any` in
docs code
- parses untyped/external data once at the boundary with zod v4 schemas
and lets `z.infer` types flow downstream — no hand-rolled `'x' in obj`
guard chains, no `as`-casts of untyped page data, no
`docs/lib/unknown-value.ts`
- adds domain schema modules: `docs/lib/toolkit-schema.ts` (recursive
JSON-Schema node, raw tool/trigger payloads, list envelopes) and
`docs/lib/reference-page-data.ts` (fumadocs reference page data for both
reference routes)
- rewrites `validate-links.ts`, `generate-toolkits.ts`,
`generate-meta-tools.ts`, the `llms.mdx` route, and
`deprecated-api-sidebar.tsx` on those schemas with identical validation
outcomes
- carries the pure typing improvements from the earlier attempt (typed
reference source in `source.ts`, `LLMPage`/`PageLike`, typed
`ClientLogo[]` in `logo-bar.tsx`, component `any` removals)
- adds `tests/static/toolkit-schema.test.ts` and
`tests/static/generate-toolkits.test.ts` pinning the transform shapes;
no changeset (docs is not published)
## Context
Second of the three-PR split of #3958, replacing its rejected
structural-guard approach with zod schemas at the data boundaries.
Verified with `bun run lint`, `bun run types:check`, `bun test
tests/static/` (125 pass), `bun run lint:links`, and a full `next
build`.
## Review follow-up
Addressed the regressions reported in [the Zod boundary
re-review](https://github.com/ComposioHQ/composio/pull/3967#issuecomment-5105279224):
scalar JSON Schema enums and auth defaults are preserved as strings,
malformed toolkit page envelopes abort generation, and the pre-refactor
empty-string fallbacks are restored. Regression coverage lives in
`tests/static/toolkit-schema.test.ts` and
`tests/static/generate-toolkits.test.ts`.
Verified on `2f26c40be` with `bun test tests/static/` (125 pass), `bun
run types:check`, and `bun run lint` (0 errors; existing warnings only).
This PR:
- fixes a regression introduced by
https://github.com/ComposioHQ/composio/pull/3956: fumadocs-openapi 11's
`groupBy: 'tag'` silently skips operations whose tag is not declared in
the document's top-level `tags` array, which 404'd all 16 Projects and
Organization Management operation pages (e.g.
`/reference/api-reference/projects/postProjectUsageSummary`) on both
v3.1 and v3, and dropped them from the sitemap and search index
- adds `declareOperationTags()` (`docs/lib/openapi-tags.ts`), applied at
spec load time in `lib/openapi.ts` (covers build-time-fetched specs) and
at sync time in `scripts/fetch-openapi.mjs`, which now also warns when
the upstream payload omits tags; the checked-in specs are normalized
accordingly (purely additive)
- adds CI guards in `tests/static/api-reference-routes.test.ts`: spec
invariants (every visible operation has a tag and `operationId`),
bidirectional spec-vs-generated-routes set equality through the
production loader path, a negative test pinning fumadocs' silent-drop
behavior (fails when upstream fixes it, signaling the workaround can be
retired), and validation that every `ApiEndpointsTable` quick-link href
resolves to a generated page
- extends `Docs - Check Links` with a nightly (02:30 UTC) + manual
external-URL sweep via a new `lint:links:external` script; the validator
only fails on evidence a link is dead (404/410 or repeated network
errors, with UA/timeout/GET-fallback/retry and per-URL caching) and
scheduled failures file a deduplicated tracking issue; PR runs keep the
fast internal-only check
- fixes six dead external links the first sweep found (moved Claude
Agent SDK docs, stale dashboard settings URL ×2, a `master`-branch path
now pinned to tag `0.5.0+post.1`, and two removed `ComposioHQ` example
repos whose mentions were dropped — their code is embedded in the pages
via `RepoBrowser`)
## Context
fumadocs-openapi 10 generated a page for any tag string found on an
operation; v11 requires the tag to be declared top-level and skips
silently otherwise (`preset-auto.js`: `builder.fromTagName(tag)` →
`continue`, no warning). The backend spec generator omits `Projects`,
`Organization Management`, and `Invite Codes` from `tags`, so their
operation pages vanished while the checked-in MDX tag landing pages kept
rendering with dead quick links. With the normalizer, the generated URL
set is byte-identical to the pre-#3956 set (234 routes, verified via
`getReferenceSource().getPages()` diff); no revert needed.
Existing checks missed this because `lint:links` only extracts markdown
links and `Card` hrefs (the dead links live in `ApiEndpointsTable`'s
array prop) and validates against the same loader that shrank, and the
integration suite samples fixed routes under tags that survived. The new
completeness guard derives expectations from the specs themselves, so
routine data syncs don't churn it.
## Summary
Implements the docs half of **PLEN-2793** — renders a typed public
reference for the payloads Composio delivers to customer webhook URLs,
under **API Reference → Webhook Events**.
Pairs with platform PR ComposioHQ/platform#11389, which generates the
spec.
## What's here
- **`docs/public/openapi-webhooks.json`** — the generated OpenAPI
**3.1** spec (top-level `webhooks`: `composio.trigger.message`,
`composio.connected_account.expired`, `composio.trigger.disabled`),
produced from the webhook payload Zod schemas in Apollo (SSOT). Finishes
the earlier untracked WIP: adds the missing `trigger.disabled` event and
reuses shared schemas as `$ref` (`WebhookEventEnvelope`,
`ConnectedAccountDetailed`).
- **`docs/lib/openapi.ts`** — adds the spec as a **second input** to the
v3.1 reference source. Fumadocs (`fumadocs-openapi`) renders the
`webhooks` block natively; operations group under the "Webhook Events"
tag at `api-reference/webhook-events/*`.
- **`webhook-events/index.mdx`** — landing page listing the three
events, cross-linked to the Webhook Subscriptions API and signature
verification.
## Why a separate 3.1 spec
`webhooks` is a 3.1-only top-level field; the main `openapi.json` is
3.0. A separate file keeps the main spec (and its sync pipeline)
untouched and avoids the docs' union-normalisation pass. See the
platform PR for the full rationale.
## Verified locally
- [x] `bun scripts/validate-links.ts` — **0 errors** (the generated
`handle_composio_*` webhook pages resolve; no collision with the
hand-authored `index.mdx`)
- [x] `bun test tests/static/` — 33 pass / 0 fail
- [x] `bun run types:check` — clean
## Notes
- Setup, subscription, and signature-verification docs already exist
(`setting-up-triggers/*`) and are cross-linked, not rewritten.
- Follow-up (optional): auto-sync the webhooks spec from Apollo on prod
deploy (currently committed by hand), mirroring the main-spec
`docs-update-data` pipeline.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: jkomyno <alberto@composio.dev>
## Summary
Deprecated REST endpoints now surface the existing `Legacy` badge in
generated API-reference indexes, with endpoint-specific tooltip copy
instead of the Sessions-specific default. The generator behavior is
covered end to end for both v3 and v3.1, including rendered table
output.
This branch also restores CI compatibility with the organization Actions
policy: workflow tool versions still come from `mise.toml`, but
installation uses the approved, SHA-pinned setup actions. Secret
scanning pins the repaired organization reusable workflow from
ComposioHQ/.github#13.
## Changes
- Read the OpenAPI `deprecated` flag and emit `legacy: true` only for
deprecated operations.
- Render `LegacyBadge` on affected endpoint rows with accurate lifecycle
tooltip copy.
- Regenerate the affected `files` and `connected-accounts` indexes for
v3 and v3.1.
- Cover the real generator output, active-operation omission, badge
count, and tooltip through a focused regression test.
- Replace disallowed transitive Actions dependencies while retaining
`mise.toml` as the single tool-version source.
Endpoint detail pages continue to use `fumadocs-openapi`'s built-in
deprecated marker. The indexed endpoints are `GET /files/list` and `POST
/connected_accounts/{nanoid}/refresh`; internal operations remain
filtered from the docs.
## Spec data
The committed OpenAPI snapshots lagged the live backend for `POST
/connected_accounts/{nanoid}/refresh`. Both snapshots now carry its
current summary, description, and `deprecated: true`; the next
`fetch-openapi.mjs` run will preserve that state from the live spec.
## Validation
- `bun run test`: 25 passed, 0 failed.
- `bun run types:check`: passed.
- Focused ESLint and generated-index drift checks: clean.
- Composite-action and workflow YAML parsed successfully; extracted tool
pins match `mise.toml`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: jkomyno <alberto@composio.dev>
## What this fixes
This is a follow-up to #3770, not a second root-cause fix.
#3770 moved the docs data workflow from staging to production,
centralized the production API URL, removed staging hosts from the
committed data, and added the hostname guard. The committed toolkit
catalog still retained staging-derived `version` values, however,
because that PR intentionally did not regenerate the full catalog. After
#3770 merged, the scheduled production regeneration began failing with
`401 Unauthorized`: the repository's existing `COMPOSIO_API_KEY` secret
is staging-scoped.
The customer-visible result was that nearly every toolkit page showed
the internal staging version `20260703_00`; Gmail's production version
was `20260702_01`.
## Changes
- Correct every `version` in `docs/public/data/toolkits.json` from the
production toolkit changelog. Toolkits absent from that changelog
receive `null`, matching the full generator's semantics. No other JSON
field changes.
- Move production changelog fetching and version application into shared
`toolkit-versions.ts` logic used by the full catalog generator.
- Add `bun run generate:toolkit-versions` as the narrow, reproducible
generator for version-only repairs.
- Reject any non-production `COMPOSIO_API_BASE` in the toolkit and
meta-tool generators before a request is made.
- Keep the version-distribution check as a smoke signal for the known
whole-catalog staging-bump pattern, while testing the production source
boundary separately. The distribution heuristic is no longer described
as proof of provenance.
- Fail before writing when the production changelog response is
malformed or contains no versions.
## CI policy compatibility
- Replace the enterprise-blocked mise action with allowlisted tool setup
actions while continuing to resolve exact versions from mise.lock.
Install the existing pinned mise CLI release through a checksum-verified
repository script for lock freshness and preinstall validation.
- Run the existing GitHub Advanced Security alert check locally and
notify Slack through the already-allowlisted Slack action, avoiding the
central workflow dependency rejected by the enterprise action policy.
## Verification
- `bun test tests/static/` — 30 passed.
- Targeted ESLint for every changed script/test — passed.
- `bun run types:check` — passed.
- `bun run build` — passed.
- Explicit staging override of `generate-toolkits.ts` — rejected before
network access.
- Verified the JSON data change remains version-only; toolkit ordering,
tools, triggers, descriptions, and counts are unchanged.
## Remaining deployment action
An administrator still needs to replace `COMPOSIO_API_KEY` with a
production-scoped key. The scheduled `docs-update-data` workflow is
correctly pinned to production and therefore fails loudly with the
current staging credential instead of republishing staging data. Once
the secret is corrected, the normal full-catalog generator remains the
authoritative refresh path.
Triggered by: abhishek@composio.dev | Source: slack
Session: https://zen.corp.composio.io/dashboard/#/chat/zen-3a77f73eb146
---------
Co-authored-by: Zen Agent <zen@composio.dev>
Co-authored-by: abhishek <abhishek@composio.dev>
Co-authored-by: jkomyno <alberto@composio.dev>
## What was wrong
The API reference docs were shipping staging URLs to users. There were
two separate leaks, both from the same source: the scheduled
`docs-update-data` workflow fetches the OpenAPI specs and toolkit data
from staging every 5 hours and auto-commits them.
The first leak was the curl base URL. `servers[0].url` in `openapi.json`
and `openapi-v3.json` rendered as `http://staging-apollo.composio.dev`
in every curl example. That is what
https://github.com/ComposioHQ/composio/pull/3761 tried to fix.
The second leak was in the toolkit data. `toolkits.json` carried 269
`https://staging-backend.composio.dev/api/v1/auth-apps/add` default
values, surfaced in the white-labeling and auth-config docs. #3761 never
touched this file.
## Which PR caused it
Karan asked on Slack whether we could pin down the PR that introduced
this. We can. It was https://github.com/ComposioHQ/composio/pull/3426
(commit `a3e58a421`, merged 2026-07-03), which flipped `servers[0].url`
from `https://backend.composio.dev` / `PRODUCTION API` to
`http://staging-apollo.composio.dev` / `STAGING API` in both specs.
It was not a hand-written change. #3426 is itself an auto-generated PR
from this same `docs-update-data` workflow, opened by
`github-actions[bot]` on the `docs/auto-update-data` branch. The
workflow fetched from staging, staging serves the staging server URL in
its spec, and the value landed in the committed files where nobody
caught it inside a large auto-generated diff. That is the reason the
real fix belongs in the generator and the workflow, not in a one-off
edit to the JSON.
## Why #3761 was not enough
#3761 pinned `spec.servers` to production inside `fetch-openapi.mjs`.
That closed the first leak, but three things stayed open.
It only covered the OpenAPI specs. The 269 staging hosts in
`toolkits.json` come from a different generator, `generate-toolkits.ts`,
and stayed live on the docs site.
It added no CI guard. The fix was a single line in a generator with
nothing asserting it. A later refactor that dropped or reordered that
line would republish staging on the next 5-hour regeneration, which is
precisely the recurring failure that produced the original report.
And it scrubbed the symptom rather than the source. The workflow kept
fetching from staging; the pin just rewrote one field afterward.
## The fix
I addressed it at four layers, so no single regression brings staging
back.
**Source.** `docs-update-data.yml` no longer overrides the base URL to
staging. The generators default to production, so the docs reflect
production.
**Generators.** The production URL now lives in one place,
`docs/scripts/production-api.mjs`. The toolkit and meta-tools generators
sanitize any staging host to production before writing, so a staging
fetch can no longer republish staging.
**Committed data.** I rewrote the 269 staging hosts already in
`toolkits.json` to production.
**Guard.** `docs/tests/static/production-urls.test.ts` asserts the
OpenAPI `servers` is production and scans both specs and every
`public/data/*.json` for any `staging-*.composio.dev` host. It is an
independent oracle: it hardcodes the expected value and deliberately
does not import the generator constants, so a wrong edit there fails CI
instead of moving both sides together. I verified it catches both the
original `staging-apollo` and the `staging-backend` leaks.
## One thing to confirm before merge
`COMPOSIO_API_KEY` is paired with `COMPOSIO_BASE_URL_STAGING` in every
other workflow, so it is most likely a staging key. If that is the case,
provision a production-capable key before the next scheduled run.
Otherwise that run fails on auth, which is loud and safe, rather than
silently republishing staging. The generator sanitizer and the guard
keep the output correct regardless of which environment the fetch hits,
so there is no risk in the interim.
## Testing
- `bun test tests/static/` passes 21 of 21 (16 existing, 5 new).
- The shared module loads and all three generators build under bun.
- No `staging-*.composio.dev` host remains under `docs/public/`.
- `docs-update-data.yml` re-validated as valid YAML.
# Description
The curl examples in the API reference docs pointed at **staging**
instead of production.
Reported (Slack): *"Curl in API reference docs points to staging-apollo
instead of backend.composio.dev."*
**Root cause:** the API reference (fumadocs, `docs/lib/openapi.ts`)
renders curl snippets using `servers[0].url` from the committed specs
`docs/public/openapi.json` (v3.1) and `docs/public/openapi-v3.json`
(v3.0). Both had:
```json
"servers": [{ "url": "http://staging-apollo.composio.dev", "description": "STAGING API" }]
```
This is a **recurring** regression, not a one-off stale file. The
scheduled workflow **`.github/workflows/docs-update-data.yml`** (`cron:
'0 */5 * * *'`, plus on Apollo production deploys) deliberately points
`OPENAPI_SPEC_URL` at **staging** (`COMPOSIO_BASE_URL_STAGING`, with a
guard that *fails* if it's production), runs `bun run
scripts/fetch-openapi.mjs`, and auto-commits the regenerated specs via
PR. Staging's spec serves `servers[0].url =
http://staging-apollo.composio.dev` (verified: `GET
https://staging-backend.composio.dev/api/v3{,.1}/openapi.json`), and
`fetch-openapi.mjs` did not override `servers` — so every ~5h the
staging server URL got re-committed (the most recent spec commit
`a3e58a421 docs: update ... API spec ...` came from exactly this
workflow).
The live production backend and the platform-committed apollo spec both
correctly serve `https://backend.composio.dev / PRODUCTION API`.
**Changes:**
- `docs/public/openapi.json` + `docs/public/openapi-v3.json`: set
`servers[0]` to `https://backend.composio.dev` / `PRODUCTION API`.
- `docs/scripts/fetch-openapi.mjs`: pin `spec.servers` to the production
URL in `postProcessSpec`. This is the **durable** fix — because the
update-data workflow fetches from staging, a JSON-only edit would be
reverted within ~5h; pinning in the generator forces the published curl
base URL to production regardless of which environment the source spec
was fetched from.
# How did I test this PR
- **Verified the servers value** in both committed specs (valid JSON):
- `node -e "require('./public/openapi.json').servers"` →
`[{"url":"https://backend.composio.dev","description":"PRODUCTION
API"}]`
- same for `public/openapi-v3.json`
- **Zero residual** `staging-apollo` references under `docs/public`,
`docs/scripts`, `docs/lib`.
- **Confirmed the recurrence mechanism:** read `docs-update-data.yml`
(staging fetch + auto-PR) and confirmed
`staging-backend.composio.dev/api/v3{,.1}/openapi.json` returns
`http://staging-apollo.composio.dev`, while live prod +
`apps/apollo/openapi.json` return `https://backend.composio.dev`. The
`postProcessSpec` pin overrides this in the generator.
- **Generator:** `node --check docs/scripts/fetch-openapi.mjs` → syntax
OK.
- **Docs static tests** (the "Docs - Tests" CI check): `bun test
tests/static/` → 16 pass, 0 fail.
- **CI on this commit:** Docs - Tests ✓, Docs - TypeScript Code
Validation ✓, Docs - Check Links ✓, Secrets Detection ✓, Vercel preview
✓.
- **Codex review:** no correctness issues found.
# Security
- **Trivy** (`fs --scanners vuln,secret --severity CRITICAL,HIGH,MEDIUM
--ignore-unfixed`) on all three changed files → clean (no
vulnerabilities, no secrets).
- **Socket** (dependency/supply-chain): N/A — no dependency or lockfile
changes.
- Note: `security/snyk (Composio)` shows failing, but it's a
**pre-existing repo-wide failure** (errors identically on PRs
#3756–#3761; this change touches no dependencies).
Triggered by: palash@composio.dev | Source: slack
Session: https://zen.corp.composio.io/dashboard/#/chat/zen-e559fb27fb16
Co-authored-by: Zen Agent <zen@composio.dev>
Co-authored-by: palash <palash@composio.dev>
This PR:
- fixes the `docs.composio.dev/toolkits` search returning 0 results for
toolkits like Workday that exist in the catalog
- paginates `fetchToolkits()` in `generate-toolkits.ts` by following
`next_cursor`: `GET /api/v3/toolkits` silently caps `limit` at 1000 per
page and defaults to usage ordering, so the previous single-request
fetch shipped only the top-1000 toolkits — about half of the ~2.1k
catalog
- dedupes across pages by slug (pages can overlap when the catalog
shifts between cursor fetches) and throws instead of silently truncating
if the catalog ever outgrows the page guard, mirroring the dashboard's
`listToolkits` procedure
- fixes the `docs-update-data.yml` workflow, broken since the
sessions-first docs rewrite
(https://github.com/ComposioHQ/composio/pull/3637) moved the generated
meta-tools MDX to `docs/content/toolkits/meta-tools/`:
`create-pull-request` aborted on the stale
`docs/content/reference/meta-tools/` pathspec, so every scheduled data
sync since then failed at the Create Pull Request step and docs data
went stale
- updates the request-volume notes in `fetch-with-retry.ts` and
`scripts/README.md` (~3000 → ~6500 requests per run); the existing 429
backoff already absorbs the larger run
## Context
The toolkits landing page filters a build-time snapshot
(`public/data/toolkits-list.json`) client-side, so every toolkit missing
from the snapshot is unsearchable and its detail page has no data. The
snapshot regenerates via the scheduled `docs-update-data.yml` workflow;
with both fixes in place the automated data PR picks up the full catalog
and the "Browse N toolkits" count grows from 1000 to the real catalog
size.
Verified offline by running the script against a stubbed paginated
backend (1500 toolkits across 2 pages): all pages are fetched, ordering
is preserved, no duplicates. `bun run types:check` and eslint pass. A
manual `workflow_dispatch` of the fixed workflow on this branch produced
the full-catalog data PR.
## Summary
- Refresh the checked-in v3.1 and v3 OpenAPI snapshots from the live
backend specs using `bun run scripts/fetch-openapi.mjs`.
- Regenerate API reference index pages with `bun run
generate:api-index`.
- Surfaces `DELETE /api/v3.1/tool_router/session/{session_id}` in the
Tool Router API reference.
- Includes the other upstream generated spec changes from the same
auto-fetch, including generated API Keys and Organization Management
section indexes.
## Verification
- `bun run scripts/fetch-openapi.mjs`
- `bun run generate:api-index`
- `git diff --check`
- `bun run build`
## Notes before merge
- This is intentionally the full auto-generated OpenAPI refresh, not the
surgical two-file patch.
- The generated backend OpenAPI still contains some `internal` wording
in descriptions; left as-is to preserve the auto-generated output.
- Confirm the newly generated API Keys / Organization Management
sections are intended to publish with this refresh.
---------
Co-authored-by: jkomyno <alberto@composio.dev>
## Summary
- Switch the docs Eve agent from the AI Gateway `openai/gpt-5.4-mini`
string to an Inception Labs Mercury 2 OpenAI-compatible chat model.
- Keep tool calling on the chat-completions path and pass Mercury's
`reasoning_effort=medium` through the AI SDK OpenAI adapter.
- Add `DOCS_AGENT_MODEL_FLOW` so the same agent can run either `mercury`
or the old AI Gateway flow for eval comparisons.
- Add docs-agent eve evals covering grounded docs answers, docs
retrieval, citations, and account-specific support refusal.
- Replace the docs-agent retriever with an in-process BM25-style lexical
ranker that returns bounded full content for the top results, so Mercury
gets rich context in one fast tool call instead of a serial
`search_docs` → `read_doc` round trip.
- Precompute BM25 term counts/document frequencies into the generated
`agent/lib/docs-index.ts` snapshot at build time, removing deployed
cold-start corpus construction while keeping retrieval in-process.
- Add opt-in search perf logging (`DOCS_AGENT_SEARCH_PERF_LOG=1`,
optional `DOCS_AGENT_SEARCH_LOG_QUERY=1`) with timings for tokenization,
corpus load/cache, ranking, hydration, total duration, corpus source,
and top URLs.
- Add `eval:agent` and `eval:agent:flows` scripts; `eval:agent:flows`
can run local model-flow comparisons or remote target comparisons via
`DOCS_AGENT_EVAL_TARGETS`.
- Add `INCEPTION_API_KEY` / optional Mercury and gateway model knobs to
`docs/.env.example`, and move `@ai-sdk/openai` to runtime dependencies
for the agent import.
## Notes
- This is intentionally an experiment to see how Mercury's diffusion
model behaves with Eve tool calling (`search_docs` and `read_doc`).
- Preview/runtime environments need `INCEPTION_API_KEY`;
`INCEPTION_MODEL` and `INCEPTION_BASE_URL` are optional overrides.
- The custom fetch prevents accidentally falling back to
`OPENAI_API_KEY` against Inception's endpoint.
- The docs search is lexical/in-memory, not vector search. The slow path
was mostly serial model/tool round trips and cold index construction,
not embedding lookup.
- The generated BM25 snapshot is process-local once loaded: warm for the
lifetime of the running Node/Vercel function instance, and reset on cold
starts, redeploys, or process restarts. The expensive term-count corpus
is now built at docs build time.
- Perf logs omit raw user queries by default; set
`DOCS_AGENT_SEARCH_LOG_QUERY=1` only when you explicitly want raw
query/term logging.
- Local A/B-style eval run:
```bash
DOCS_AGENT_EVAL_FLOWS=gateway,mercury bun run eval:agent:flows --
--strict
```
- Live target comparison:
```bash
DOCS_AGENT_EVAL_TARGETS=baseline=https://<prod>,mercury=https://<preview>
bun run eval:agent:flows -- --strict
```
## Tests
- `bunx eslint scripts/build-agent-index.ts agent/lib/docs.ts
agent/tools/search_docs.ts`
- `bun scripts/build-agent-index.ts` (wrote 133 pages + 1000 toolkits +
1139 BM25 rows)
- `DOCS_AGENT_SEARCH_PERF_LOG=1 EVE_FORCE_BUNDLE=1 bun -e "const
tool=(await import('./agent/tools/search_docs.ts?log=' +
Date.now())).default; await tool.execute({query:'create a session with
github tools'}); await tool.execute({query:'auth config connected
account'});"` (logs cold and warm timing JSON)
- `EVE_FORCE_BUNDLE=1 bun -e "const tool=(await
import('./agent/tools/search_docs.ts?bundle=' + Date.now())).default;
const started=performance.now(); const r=await
tool.execute({query:'create a session with github tools'});
console.log(r.retrieval, r.results[0].url, r.results[0].content.length,
Math.round(performance.now()-started)+'ms');"` (precomputed bundle path,
~12ms)
- `bun -e "const tool=(await import('./agent/tools/search_docs.ts?live='
+ Date.now())).default; const started=performance.now(); const r=await
tool.execute({query:'create a session with github tools'});
console.log(r.retrieval, r.results[0].url, r.results[0].content.length,
Math.round(performance.now()-started)+'ms');"` (live-content path,
~34ms)
- `EVE_FORCE_BUNDLE=1 bun -e "const tool=(await
import('./agent/tools/search_docs.ts')).default; await
tool.execute({query:'create a session with github tools'}); const
started=performance.now(); const r=await tool.execute({query:'auth
config connected account'}); console.log(r.results[0].url,
r.results[0].content.length,
Math.round(performance.now()-started)+'ms');"` (warm path ~2ms)
-
`PATH=/Users/cryogenicplanet/.vite-plus/js_runtime/node/24.15.0/bin:$PATH
./node_modules/.bin/eve info --json` (reports `status: ready`, `model:
inception/mercury-2`, `errors: 0`)
- `DOCS_AGENT_MODEL_FLOW=gateway
PATH=/Users/cryogenicplanet/.vite-plus/js_runtime/node/24.15.0/bin:$PATH
./node_modules/.bin/eve info --json` (reports `status: ready`, `model:
openai/gpt-5.4-mini`, `errors: 0`)
-
`PATH=/Users/cryogenicplanet/.vite-plus/js_runtime/node/24.15.0/bin:$PATH
./node_modules/.bin/eve eval --list`
-
`PATH=/Users/cryogenicplanet/.vite-plus/js_runtime/node/24.15.0/bin:$PATH
bun scripts/eval-agent-flows.ts --list`
- `bun test tests/static/` (16 passed)
- `bun run types:check` currently fails on existing docs type-generation
errors in `app/(home)/docs/changelog/[...slug]/page.tsx`,
`app/(home)/examples/[[...slug]]/page.tsx`,
`app/(home)/toolkits/[[...slug]]/page.tsx`,
`app/llms.mdx/[[...slug]]/route.ts`, `lib/search-index.ts`, and
`lib/source.ts`; no new eval or `docs/agent/agent.ts` errors were
reported.
## Not run
- Real live model evals, because this local environment does not have
`INCEPTION_API_KEY` or AI Gateway credentials.
## Latest update
- Added default eager docs retrieval in the Eve HTTP channel: the server
runs the same BM25 search on the user's message before the first model
step and injects the results as one-turn context.
- Kept `search_docs` and `read_doc` available so Mercury can still
search/read more when the eager context is weak, ambiguous, or missing.
- Added `DOCS_AGENT_EAGER_SEARCH=0` as an escape hatch and labeled perf
logs with `invocation: "eager_context" | "tool"`.
- Updated the loading copy from “Searching the docs…” to “Thinking with
the docs…” so UI latency is not attributed solely to the search call.
## Latest tests
- `bun run lint -- agent/channels/eve.ts agent/tools/search_docs.ts
agent/lib/docs-search.ts components/eve-chat.tsx
evals/docs-agent/grounded-answers.eval.ts`
-
`PATH=/Users/cryogenicplanet/.vite-plus/js_runtime/node/24.15.0/bin:$PATH
node_modules/eve/bin/eve.js info --json` (reports `status: ready`,
`errors: 0`)
- `DOCS_AGENT_SEARCH_PERF_LOG=1 EVE_FORCE_BUNDLE=1 bun -e "import {
searchDocs } from './agent/lib/docs-search'; const r = searchDocs('How
do I create a session in Composio? Keep it brief.', { invocation:
'eager_context' }); console.log(JSON.stringify({count:r.results.length,
top:r.results[0]?.url, content: !!r.results[0]?.content}, null, 2));"`
- `DOCS_AGENT_SEARCH_PERF_LOG=1 bun -e "import { searchDocs } from
'./agent/lib/docs-search'; searchDocs('How do I create a session in
Composio? Keep it brief.', { invocation: 'eager_context' });
searchDocs('How do I create a session in Composio? Keep it brief.', {
invocation: 'tool' });"`
- `bun run types:check` still fails only on the pre-existing docs
type-generation issues listed above.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integration branch for the next docs release: a **sessions-first
documentation rewrite** — new and rewritten guides, example pages,
interactive components, and docs tooling — plus the supporting SDK
changes that the new docs describe.
The bulk of this PR is docs (~24k lines across ~150 commits); the SDK
changes (~5k lines) back the new guides.
## Documentation (the bulk)
- **Sessions-first restructure** — reorganized navigation and section
structure (incl. the "Sandbox (prev workbench)" section), with
v3-reorganization redirects so old URLs keep resolving.
- **Rewritten core guides** — quickstart, configuring sessions, triggers
(creating + subscribing to events), proxy-execute, toolkits
enable/disable, and common FAQ, rewritten in the house voice.
- **New example pages** — local-sandbox PR reviewer, daily standup bot,
and slack bot, with runnable build-ups.
- **New interactive components & diagrams** — triggers flow animation,
manage-connections visual, connection-refresh visual, and the
terminal-kit components.
- **Docs tooling** — a docs-graph link-graph connectivity checker,
search reprioritization (deprioritize legacy pages), and SDK-reference
regeneration.
## Supporting SDK changes
**`@composio/core` → 0.13.0 (minor)**
- `composio.sessions.create()` as the first-class sessions API
(`composio.create()` kept as an alias).
- **MCP is opt-in:** default `create()` / `use()` return native-tool
sessions (`SessionWithoutMcp`); pass `{ mcp: true }` to surface
`session.mcp`. _Migration: read `session.mcp` only after creating with
`{ mcp: true }`._
- `session.sandbox` is the canonical resolved config;
`session.workbench` kept as a deprecated alias. `sandbox` is the
preferred session-config key (`workbench` still accepted).
- `connectedAccounts.updateAcl()` graduated from experimental (alias
kept).
- `triggers.parse()` (parse + optionally verify an incoming webhook) and
`triggers.setWebhookSubscription()`.
**`@composio/experimental` → minor** — local-workbench helpers moved
onto the `@composio/experimental/workbench` subpath (out of
`@composio/core/experimental`), keeping the ~14 KB embedded Python
helper out of core. Plus the experimental Pi provider.
**`@composio/slim` → minor.**
**Python → 0.17.0** — mirrors the TS surface: `composio.sessions` mount
(`tool_router` deprecated), `triggers.parse()` /
`set_webhook_subscription()`, the `sandbox` config key, and
`connected_accounts.update_acl()`.
## Review response (#3664)
Addressed the `@composio/core` review:
- **Security:** `triggers.parse()` no longer fails open — a
present-but-empty `verifySecret` (e.g. unset `COMPOSIO_WEBHOOK_SECRET`)
now throws instead of silently skipping verification; omitting it stays
an explicit opt-out (both SDKs).
- Removed snake_case leakage from `transformWebhookSubscription` (+ the
index signature that allowed it).
- **Removed** the TS-only `connectedAccounts.link()` toolkit
auto-resolve (shipped with cancellability / orphaned-auth-config bugs
and was effectively undocumented; to be reintroduced properly later).
- Unified Python error types on `ValidationError`; added `mcp=True`
Python tests; fixed runtime-portability + error-type test assertions.
- Polished deprecation messages; fixed the backwards `/experimental`
`@deprecated` note and the `SessionWithMcp` JSDoc.
## Testing
- **TS:** `@composio/core` + `@composio/experimental` typecheck pass;
vitest green for the touched suites.
- **Python:** `test_tool_router.py` + `test_triggers.py` pass (161
tests).
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Kshitij Jhunjhunwala <kj@composio.dev>
Co-authored-by: Malay Vasa <malayvasa@gmail.com>
Co-authored-by: Sarah Simionescu <sarah@composio.dev>
Co-authored-by: Kshitij Jhunjhunwala <113939507+KJ-11@users.noreply.github.com>
This PR:
- adds a shared `fetchWithRetry` helper for the docs data-sync scripts
that retries `429` and transient 5xx responses, honoring `Retry-After`
(with jitter) and falling back to exponential backoff
- routes every request in `generate-toolkits.ts` and
`generate-meta-tools.ts` through it, so the `Docs - Update Data`
workflow rides out the staging rate limit (2000 req/min) instead of
hard-failing
- lowers toolkit fetch `batchSize` 10 → 5 (~30 → ~15 concurrent
requests) to soften burst pressure on the rate limit
- caps retry attempts so CI still fails fast when the backend is
genuinely down
- fixes the current `429 Too Many Requests` failure in `Generate meta
tools reference`, which inherited the rate-limit window exhausted by the
high-volume `generate-toolkits.ts` run
- verified with `bun run types:check` and an offline retry/backoff
behavior test (real end-to-end run needs the staging key in CI: `gh
workflow run docs-update-data.yml`)
- follow-up (separate PR): reduce per-toolkit request volume in
`generate-toolkits.ts` — left out here because it needs consumer and
search-quality validation
## Summary
- Re-apply the Algolia docs search migration after the previous PR was
merged and reverted.
- Keep the existing Fumadocs search UI while using Algolia API when
search keys are configured, with `/api/search` fallback for local
development and tests.
- Add a first-party Algolia index builder/sync script that creates
section-sized docs records from MDX/OpenAPI/toolkit/changelog content,
configures index relevance settings, and replaces index objects without
relying on Algolia Crawler.
- Add Algolia Insights view/click events and a terminal search relevance
test script.
- Clean search breadcrumbs so results show labels like `Toolkit` and
`Cookbook` instead of duplicated `toolkits > Gmail` formatting.
## Tests
- `cd docs && bun run types:check`
- `cd docs && bun run sync:search --dry-run`
- `cd docs && bunx eslint components/custom-search-dialog.tsx
lib/search-index.ts scripts/sync-algolia-search.ts
scripts/test-algolia-search.ts`
## Notes
- Live Algolia sync requires `ALGOLIA_ADMIN_API_KEY`.
- Live search relevance tests require `ALGOLIA_SEARCH_API_KEY` or
`NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY`.
## Summary
- Switch docs search dialog to Algolia when public Algolia env vars are
configured, with local `/api/search` fallback for development/tests
- Add a shared docs search index builder plus `bun run sync:search` to
publish records to Algolia
- Add a GitHub Actions workflow to dry-run and sync the Algolia index on
`next` docs changes when secrets are configured
- Document required Algolia env vars and sync command
## Tests
- `cd docs && bun run types:check`
- `cd docs && bun run sync:search --dry-run`
- `cd docs && bun test tests/static/`
- `cd docs && bunx eslint components/custom-search-dialog.tsx
lib/search-index.ts scripts/sync-algolia-search.ts`
- `cd docs && bun run build`
## Notes
- `bun run lint` still fails on existing unrelated repo-wide lint errors
in files outside this change (for example `app/global-error.tsx`,
`components/ask-ai-button.tsx`, `components/version-selector.tsx`). The
changed search files pass targeted ESLint.
- Production search requires `NEXT_PUBLIC_ALGOLIA_APP_ID`,
`NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY`, and optionally
`NEXT_PUBLIC_ALGOLIA_INDEX_NAME` (default `composio_docs`). Index
syncing requires `ALGOLIA_APP_ID`/`ALGOLIA_ADMIN_API_KEY` secrets.
generate-api-index.ts iterates every tag in spec.tags but used to `continue`
silently when a tag had zero operations in both v3.1 and v3.0 specs. After a
tag rename (e.g. Webhooks → Webhook Endpoints + Webhook Subscriptions), the
old tag dir still has an index.mdx from a prior run that points at the old
operationIds. The auto-update PR never includes a deletion for it, so the
phantom category lingers in the docs.
Replace the silent skip with an rmSync of any stale index.mdx for that tag
in both content/reference/api-reference/ and content/reference/v3/api-reference/.
Next auto-update run picks up the orphan and removes it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Re-fetch OpenAPI spec which now includes v3.1 tool endpoints (hermes#8990)
- Update fetch-openapi.mjs to remove older API versions when a newer one
exists for the same endpoint path (e.g. v3 hidden when v3.1 available)
- Update generate-api-index.ts with the same version deduplication logic
- Regenerate API reference index pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a programmatic Meta Tools reference under Reference > Meta Tools,
powered by data fetched from the Tool Router API.
- Generator script fetches tool schemas from API, produces JSON + MDX
- CI auto-updates via docs-update-data.yml on Apollo deploys
- Individual tool pages show tags, input parameters, and response schemas
- .md endpoint renders full parameter details for LLM consumers
- Updated existing docs to include COMPOSIO_GET_TOOL_SCHEMAS
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Surface the `composioManagedAuthSchemes` field from the API on toolkit
pages so users know upfront whether they can use Composio's managed
OAuth or need to provide their own credentials.
- Add `composioManagedAuthSchemes` to Toolkit type and generate script
- Show badge next to Authentication Details heading (OAuth toolkits only)
- Include managed app status in markdown/LLM export and toolkits index
- Regenerate toolkits.json with managed auth data (114/980 toolkits)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OpenAPI spec now uses `x-internal` as a tag on internal endpoints.
Instead of maintaining a hardcoded list of paths to ignore, filter any
operation tagged with `x-internal` automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces custom regex slugify with github-slugger to correctly handle
Unicode characters and duplicate heading suffixes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Exclude scripts/preload.ts from tsconfig (Bun-only preload script)
- Add explicit PageOf type annotation to fix implicit any
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use resolve() to normalize paths before deduplication so absolute paths
from Fumadocs sources match relative paths from glob.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix /toolkits/introduction → /toolkits in troubleshooting pages
- Remove broken relative links to source files in cookbooks
- Remove unnecessary "See Authenticating Users" link in errors.mdx
- Add dynamic toolkit slug validation from toolkits.json to link checker
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The link checker was using the static referenceSource which only
includes MDX pages, missing all OpenAPI-generated API endpoint pages.
Switch to getReferenceSource() which merges both MDX and OpenAPI pages.
Eliminates 67 false positives from API reference index pages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Moves FAQ markdown files from content/toolkit-faq/ into
content/toolkits/faq/ so they live alongside other toolkit content.
The link checker now scans all .md files under content/ generically
instead of special-casing the FAQ directory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
data.toc is unavailable when running outside the Next.js runtime (via
bun scripts), so fragment links were all flagged as invalid. Parse
headings from raw markdown content instead, with a fallback chain:
data.toc -> getText('raw') -> readFile.
Also adds bunfig.toml with fumadocs-mdx bun preload plugin (recommended
by fumadocs docs for running scripts outside Next.js).
Reduces false positives from 134 errors to 90 real broken links.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The populate keys in validate-links.ts were missing the (home) route
group prefix, so next-validate-link could never match them against the
actual file paths from app/(home)/. This meant zero URLs were registered
and all internal links passed validation silently.
Also adds toolkit FAQ markdown files to the link checker scope since
they are loaded outside of Fumadocs sources.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The generate-toolkits script was missing the `toolkit_versions=latest`
query parameter when fetching tools and triggers. This caused MCP
toolkits (e.g. Granola) to show 0 tools, and older toolkits to
undercount (Gmail 23→40, Notion 28→46).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only keep slug, name, logo, category, toolCount, triggerCount in the
light JSON file used by the landing page client bundle. Reduces
toolkits-list.json from ~814KB to ~103KB. Full toolkits.json for
detail pages is unchanged.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When both operationId and summary are undefined, use the same
method+path fallback for both to avoid empty/broken URLs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
fumadocs-openapi uses operationId (e.g., getToolRouterSessionBySessionId)
not slugified summaries for page URLs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Each endpoint now shows:
- HTTP method (GET, POST, PATCH, DELETE)
- Full path with parameters
- Linked title
- Full description from OpenAPI spec
Much more useful for LLMs to understand the API.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Brendan O'Leary