Alberto Schiabel
82896858ec
This PR: - adds a shared `fetchWithRetry` helper for the docs data-sync scripts that retries `429` and transient 5xx responses, honoring `Retry-After` (with jitter) and falling back to exponential backoff - routes every request in `generate-toolkits.ts` and `generate-meta-tools.ts` through it, so the `Docs - Update Data` workflow rides out the staging rate limit (2000 req/min) instead of hard-failing - lowers toolkit fetch `batchSize` 10 → 5 (~30 → ~15 concurrent requests) to soften burst pressure on the rate limit - caps retry attempts so CI still fails fast when the backend is genuinely down - fixes the current `429 Too Many Requests` failure in `Generate meta tools reference`, which inherited the rate-limit window exhausted by the high-volume `generate-toolkits.ts` run - verified with `bun run types:check` and an offline retry/backoff behavior test (real end-to-end run needs the staging key in CI: `gh workflow run docs-update-data.yml`) - follow-up (separate PR): reduce per-toolkit request volume in `generate-toolkits.ts` — left out here because it needs consumer and search-quality validation
Composio Docs
Documentation site for Composio, built with Fumadocs.
Getting Started
bun install
bun run dev
Open http://localhost:3000.
Project Structure
docs/
├── app/ # Next.js app router
├── content/ # MDX content
│ ├── docs/
│ ├── examples/
│ ├── changelog/
│ └── reference/
├── components/ # React components
├── lib/ # Utilities
└── public/ # Static assets
Adding Content
Create an .mdx file in content/, add frontmatter, then add to meta.json:
---
title: Page Title
description: Brief description
---
Content here...
Components
<Tabs items={['Python', 'TypeScript']}>
<Tab value="Python">...</Tab>
<Tab value="TypeScript">...</Tab>
</Tabs>
<Callout type="info">Note</Callout>
<Cards>
<Card title="Title" href="/path" />
</Cards>
Sidebar
Each folder has meta.json for ordering:
{
"pages": ["page-one", "page-two"]
}
TypeScript Code Blocks
All TypeScript code blocks in MDX files are type-checked at build time using Twoslash. This ensures docs stay in sync with the SDK.
- Use
// @noErrorsto skip checking for partial snippets - Use
// ---cut---to hide setup code from output - Run
bun run buildlocally to validate before pushing
See CLAUDE.md for detailed patterns and troubleshooting.
Search
Docs search uses Algolia when NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY is set; otherwise it falls back to the local Fumadocs /api/search endpoint for development and tests. The sync script builds a first-party index from MDX/OpenAPI/toolkit data (no crawler required), splits long pages into section-sized records, configures searchable attributes/custom ranking/distinct, requests clickAnalytics, and sends search result view/click events with search-insights.
NEXT_PUBLIC_ALGOLIA_APP_ID=62HI9PQZ1L # optional; default shown
NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY=...
NEXT_PUBLIC_ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages # optional; default shown
Sync the Algolia index with an admin key:
ALGOLIA_APP_ID=62HI9PQZ1L
ALGOLIA_ADMIN_API_KEY=...
ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages bun run sync:search
Preview the generated records or test live relevance:
bun run sync:search --dry-run --samples
ALGOLIA_SEARCH_API_KEY=... bun run test:search "oauth auth config" "gmail send email"
Commands
| Command | Description |
|---|---|
bun run dev |
Dev server |
bun run build |
Production build (validates TS code blocks) |
bun run types:check |
Type check |
bun run sync:search |
Sync docs search records to Algolia |
bun run test:search |
Query the configured Algolia index from the terminal |