## Summary
- Adds a `warn` callout under the quickstart's Python install step: the
current package is `composio`; `composio-core` and `composio-openai` are
the legacy v1 SDK — do not install them.
- Names the legacy package names explicitly in the migration guide's
intro, so searches for those names land on the migration path.
## Why (measured)
From the docs-agent-eval transcripts (202 agent runs, waves 4–5): **85%
of agents on prod docs and 63% on the current IA attempt a
legacy-package install first**, then discover the mistake, uninstall,
and redo — **+64s (+14%) mean build time**. The failure mode is visible
verbatim in transcripts: after reading the quickstart, agents search
"how to install **composio-core or composio** python package" — the docs
never disambiguated. LLM training priors will keep suggesting the old
names; the install step is where the correction lands.
## Deliberately out of scope
- Package-registry fixes (PyPI DEPRECATED banner on composio-core,
`python_requires>=3.10` on composio — currently installs on 3.9 then
crashes on import) — SDK-side, being raised separately.
- A machine-readable current-versions manifest — follow-up proposal.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Soumya Medapati <soumyamedapati@mac.local.meter>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Spotted while fixing the provider-dependency failures in #4221.
## Problem
`python/examples/tool_router/tool_router_mcp.py` guarded a missing
credential like this:
```python
api_key = os.environ.get("COMPOSIO_API_KEY")
if not api_key:
print("Error: COMPOSIO_API_KEY environment variable not set")
print("Please set it using: export COMPOSIO_API_KEY='your_api_key'")
return
```
`return` exits the coroutine normally, so `asyncio.run(main())`
completes and the process exits **0**. The live-examples harness scores
entries by exit code, so a run that did nothing at all was recorded
**green** — the worst kind of failure for a canary, since it reports
success while testing nothing.
## Fix
Raise instead, using the `require_env` helper the sibling examples
already use (`examples/tool_router/preload.py`):
```python
def require_env(name: str) -> str:
value = os.environ.get(name)
if not value:
raise RuntimeError(f"Set {name} before running this example.")
return value
```
## Verification
Replaying the harness's own invocation with the variable unset:
```
$ unset COMPOSIO_API_KEY
$ uv run --project python --with 'openai-agents>=0.19' \
--with ./python/providers/openai_agents --with ./python \
python python/examples/tool_router/tool_router_mcp.py
before: exit=0 # silently "passes"
after: exit=1 # RuntimeError: Set COMPOSIO_API_KEY before running this example.
```
`ruff check`, `ruff format --check`, and `nox -s chk_examples` all pass.
## Scope
I checked whether other examples share the pattern.
`examples/tool_router/authorize.py` prints the same message, but calls
`exit(1)` at module level rather than returning from a coroutine, so it
already fails correctly — left alone. This was the only instance.
## Summary
Automated refresh of the toolkit slugs the CLI knows without asking
the API, generated by
`ts/packages/cli/scripts/generate-toolkit-slugs.ts`.
Toolkits added since the last refresh currently cost users one
toolkit-list fetch (~2 s) the first time they run one of that
toolkit's tools. Merging this makes them free.
The generator refuses to write a list that is short, malformed, or
missing staple toolkits, so a bad fetch opens no PR at all.
Co-authored-by: jkomyno <12381818+jkomyno@users.noreply.github.com>
This PR:
- unpins the examples harness from the staging host: `COMPOSIO_BASE_URL`
now selects the backend and still defaults to staging, with the
structural checks kept (https, bare root, no
path/query/fragment/credentials) — `examples-live.yml` sets staging
explicitly, so CI is unchanged
- renames `harness/staging-backend.mjs` to `harness/backend-url.mjs` and
`requireStagingBaseUrl` to `resolveBackendBaseUrl`, and relaxes the
matching hard-exit in `harness/trace-py/sitecustomize.py`
- fixes the Python candidate swap: `relaxPyClientPin()` drops the
project's exact `composio-client==` pin for the duration of a candidate
sweep, so `COMPOSIO_CLIENT_WHEEL` can win over it, and
`python/pyproject.toml` + `uv.lock` are restored from a pre-run snapshot
afterwards
- adds `harness/README.md` covering backend selection, the
candidate-swap flow for both languages, and how to tell a real parity
pass from a vacuous one
- ignores `.artifacts/`, where every sweep writes its results, traces,
and downloaded client
- extends `harness/run.mjs selftest` with eleven checks over the new
backend-URL behaviour and the pin relaxation; the suite passes locally
## Context
Found while producing a Python examples parity run for the
`composio-client` 2.x bump, which lives on a separate branch. Both
problems blocked that run and neither is specific to it.
The staging pin made a comparison against any other project impossible
without editing the harness in place.
The Python candidate swap was worse than blocking, because it failed
quietly. A candidate sweep layers the wheel onto every entry with `uv
run --with`; seven of the twenty-two Python entries also install the
local `./python` project through `pyWith`, and uv cannot satisfy that
project's exact pin and the candidate wheel together:
```
No solution found when resolving `--with` dependencies:
Because only composio==0.20.0 is available and composio==0.20.0 depends on
composio-client==<pinned>, [...] your requirements are unsatisfiable.
```
Those entries went red in under a tenth of a second for a packaging
reason rather than a client one, and `parity.mjs` only compares entries
green on both sides — so the comparison silently shrank to fifteen
entries and still reported a clean pass. With the pin relaxed, the same
sweep runs 22/22 green and compares all twenty-two.
https://claude.ai/code/session_015Bx2BaSwuq2cxgJSRtAxag
Follow-up to the live-sweep findings in #4135. Independent of it — this
targets `next` directly.
## Python entries could never resolve their provider imports
`harness/run.mjs` runs each Python entry as `uv run --project python
[--with <dep>]... python <file>`. The base environment is the `python`
project only, so the provider packages are present just when an entry
names them in `pyWith`. Six entries import a provider and declared none,
and have failed with `ModuleNotFoundError` every time the sweep has run
since it landed on 2026-08-14:
| entry | imports | had |
| --- | --- | --- |
| `py/tool_router/files` | `composio_openai` | no `pyWith` |
| `py/tool_router/preload` | `composio_openai_agents` |
`openai-agents>=0.19` only |
| `py/tool_router/direct_tools_preset` | `composio_openai_agents` |
`openai-agents>=0.19` only |
| `py/tool_router/tools` | `composio_openai_agents` |
`openai-agents>=0.19` only |
| `py/tool_router/tool_router_mcp` | `composio_openai_agents` |
`openai-agents>=0.19` only |
| `py/custom_tools_agent_test` | `composio_openai_agents` |
`openai-agents>=0.19` only |
Those entries declared the upstream `openai-agents` SDK but not the
Composio provider built on it.
### Why local paths, not package names
A bare `--with composio-openai-agents` resolves from **PyPI**, so the
sweep would exercise the published provider rather than this repo's —
and it drags the published `composio` in with it. Declaring
`./python/providers/<name>` plus `./python` keeps both local. Verified
in the resulting overlay:
```
composio : file:///…/composio2/python
composio-openai : file:///…/composio2/python/providers/openai
```
`py/tool_router/claude_agent` already had `./python` but named its
provider, so it was testing the **published**
`composio-claude-agent-sdk` against the repo's `composio` (confirmed:
that dist has no `direct_url.json`). Pointed at the local path too, so
every entry now tests repo code under one convention.
## Deleting `ts/connected-accounts/magic-flow-demo`
- Not substantively touched since **2025-06-20** — ~14 months. The only
later commits are a model-version bump and its same-day revert, which
net to zero.
- It calls an endpoint the backend has retired. The sweep gets `400`:
*"Creating connections on this endpoint for Composio-managed OAuth auth
configs is no longer supported. Use POST /api/v3/connected_accounts/link
instead."*
- It could not pass unattended even if that were fixed:
`waitForConnection()` blocks until a human completes a browser OAuth
grant.
Removed the source, its manifest entry, and its README bullet. Rewriting
it onto `/connected_accounts/link` would be a genuinely new example
rather than a repair, so it seemed better to drop it than to carry a
broken one.
## Verified against the live staging backend
Not just locally — these entries were **executed** against
`staging-backend.composio.dev` (`llm=mock`, zero model spend):
```
provisioned state: complete
sweep 20260823224239-lk9413-baseline-mock: 5 entries
✓ py/tool_router/toolkits (5.8s)
✓ py/tool_router/preload (11.1s)
✓ py/tool_router/direct_tools_preset (11.2s)
✓ py/tool_router/files (12.3s)
✓ py/auth_configs (3.9s)
sweep: 5 green / 0 red / 0 skipped
```
All three previously-broken entries now pass end to end; `toolkits` and
`auth_configs` ran as unchanged controls. As far as I can tell from the
workflow history, this is the first time any of these examples has
completed green — the nightly has failed at `Verify provisioned project
state` on every run since it was introduced.
The run needed a branch carrying both these fixes and #4135's scoped
provisioning, since `next`'s provisioning tries to create a
`googledrive` auth config and dies before sweeping anything. That branch
was throwaway and is deleted.
Other checks: `node harness/run.mjs selftest` passes, `pnpm run
test:examples` validates 20 packages, the connected-accounts package
typechecks, and all 92 remaining manifest entries point at files that
exist.
### Still unproven
The other three entries fixed here — `tools`, `tool_router_mcp`,
`custom_tools_agent_test` — need github/gmail connected accounts, which
staging does not yet have. They get past their imports but cannot be run
end to end until those grants exist. `py/modifiers` is `tier=X`,
excluded by design.
## Related
`py/tool_router/tool_router_mcp` also exits **0** when
`COMPOSIO_API_KEY` is unset, so the harness would score it green while
it did nothing. Fixed separately in #4222.
## Problem
A catalog pin or lockfile edit swaps what every package resolves,
without touching a file under `ts/**` or `python/**`. The path filters
on our suites did not account for that, so a dependency bump could skip
the very checks meant to catch it.
This is not hypothetical. In #4135, bumping `@composio/client` from
`0.1.0-alpha.76` to `2.0.0-rc.5` — a one-line change in
`pnpm-workspace.yaml` — ran **no TypeScript E2E at all**. That client
adds a credential-transport guard that refuses plain-HTTP credentials to
non-loopback hosts, which breaks `@e2e-tests/cli-toolkits-list` (the
harness serves its mock backend at
`http://host.docker.internal:${port}`). The PR was green throughout; the
breakage only surfaced when the suite was dispatched by hand.
`ts.test.yml` ran on that PR purely by luck, because the changeset file
matched `.changeset/**`.
## Change
Add the dependency-resolution inputs to every path-filtered SDK workflow
that lacked them:
| workflow | added |
| --- | --- |
| `ts.test-e2e.yml` | `pnpm-workspace.yaml`, `pnpm-lock.yaml` |
| `ts.test.yml` | `pnpm-workspace.yaml`, `pnpm-lock.yaml` |
| `ts.typecheck.yml` | `pnpm-workspace.yaml`, `pnpm-lock.yaml` |
| `ts.examples.yml` | `pnpm-workspace.yaml`, `pnpm-lock.yaml` |
| `py.check.yaml` | `python/pyproject.toml`, `uv.lock` |
| `py.test.yml` | `uv.lock` |
`ts.audit.yml` and `ts.build.yml` already listed both pnpm files and are
unchanged. `ts.release.yml` and `ts.examples-nightly.yml` have no path
filters.
`py.check.yaml` was the widest gap on the Python side: it matched only
`python/**/*.py`, so the `composio-client` pin bump in #4135 triggered
no ruff or mypy run.
## Verification
Every touched file still parses as YAML, and each of the six now matches
on the dependency inputs. The trade is more CI on lockfile-only changes,
which seems clearly worth it against a silent client swap.
The sensitive-file-upload denylist matched deny segments only against the
symlink-resolved path. That catches a benign name pointing at a secret
(`~/innocent -> ~/.aws/creds`, which the existing test covers) but misses the
inverse: a sensitive directory that is itself a symlink to a target without the
denied segment. `~/.claude -> /state/claude` resolves to `/state/claude/...`,
which contains no `.claude` segment, so the denylist let it through — and
`.claude` is on the list precisely because it "may contain API keys".
That layout is not exotic. Dotfile managers (chezmoi, stow, yadm) and
containerised home directories produce it routinely; Composio's own agent
sandbox image does. For those users the control was silently inactive.
The repository's own test asserted the blocked behaviour and has been failing
wherever `~/.claude` is a symlink. CI never caught it because `~/.claude` does
not exist on the runners, so no realpath happens and the written path keeps its
`.claude` segment.
Match both the written and the resolved path, for directory segments and for
the basename check, since either one can be the one carrying the denied name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The release build downloads four `@zed-industries/codex-acp-<platform>`
tarballs itself, because pnpm will only install the one matching the runner's
platform and the archive has to name all four. Those bytes were extracted and
copied, executable, into the published CLI archive without ever being checked
against a known hash — the one dependency in the release that got none of the
integrity verification `pnpm install --frozen-lockfile` applies to everything
else.
`pnpm-lock.yaml` already pins a sha512 for each of them, so the fix is to use
it. The hash is read from the lockfile rather than the registry's own
`dist.integrity`, since whoever can serve a tarball can serve the metadata
vouching for it, and it is looked up before the download so a package the
lockfile never resolved aborts the build instead of shipping unverified bytes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This PR:
- refreshes the baked CLI toolkit catalog from 1,070 to 1,381 slugs
- updates the snapshot timestamp so release builds no longer warn about
the old catalog
- routes the scheduled production refresh through
`COMPOSIO_DOCS_API_KEY`
- preserves the shared `COMPOSIO_API_KEY` contract for staging SDK and
example suites
- verifies the generated catalog contract with its focused Vitest suite
- verifies the workflow file with Prettier and `git diff --check`
## Problem
`composio upgrade` from `0.3.4-beta.351` to `0.4.0-beta.359` looked hung
for several minutes:
```
◐ New version available: @composio/cli@0.4.0-beta.359 (current: @composio/cli@0.3.4-beta.351). Downloading.
```
Two independent defects behind that.
**No feedback.** `upgrade-binary.ts` printed that message once and then
said nothing until the download finished. The only byte-level signal was
a `logDebug`, invisible in a normal run, because the body was read with
a single buffered `response.arrayBuffer`.
**Mostly wasted payload.** The archive is 338 MB, and ~651 MB of what it
unpacks is `codex-acp` binaries for the three platforms the host cannot
execute. Measured on a darwin-arm64 install:
```
~/.composio 967 MB total
codex/darwin-arm64 180 MB ← the only one this Mac can run
codex/darwin-x64 190 MB ┐
codex/linux-arm64 208 MB ├─ 620 MB of dead weight
codex/linux-x64 222 MB ┘
```
## Changes
**Progress reporting.** The response is streamed and reported on a 250
ms interval: `Downloading... 42% (142.0 MB / 338.0 MB)`. The total comes
from the release asset's `size` (newly decoded — the field was being
dropped), falling back to `content-length`, and falling back again to a
plain byte count so an unknown size degrades instead of failing.
**Half the payload.** Each archive now carries real bytes only for the
`codex-acp` binary its own platform can execute.
The other three paths cannot simply be dropped — that is exactly the
break #4186 just repaired. A CLI released before 2026-08-18 verifies a
downloaded package against all four codex-acp paths and refuses one
missing any of them. So they ship as **empty placeholders**: the
existence check passes at zero bytes, and since no host ever executes a
foreign codex-acp, the placeholder is never read. Once no supported
client performs that check, placeholders become plain omissions —
`archiveCompanionEntries` is where that switch lives.
As a guard, codex adapter resolution now requires a non-empty file, so a
zero-byte binary resolves as absent and falls through to the existing
bundled → PATH → npx chain rather than trying to exec it.
Expected effect: download 338 MB → ~165 MB, install footprint 967 MB →
~347 MB.
## Verification
- Full CLI suite: **122 files, 1262 passed, 1 skipped**
- `pnpm --filter @composio/cli typecheck` — clean
- The existing `upgrade-binary` download tests run against a real HTTP
server and pass unchanged, covering the buffered → streamed rewrite
- New tests: progress formatting (known total, unknown total, zero
total, overshoot) and `archiveCompanionEntries` (all four paths named,
exactly one copied, three placeholdered, portable assets copied)
- `pnpm validate:changesets` — passes; note lands in
`ts/packages/cli/CHANGELOG.md`
Not verifiable locally: actual archive sizes and a real upgrade against
a pre-2026-08-18 client. Both need a beta build — worth confirming on
the beta cut from this branch before it goes near a stable promotion.
Independent of the `0.4.0` release in flight; `0.4.0-beta.359` is
unaffected.
https://claude.ai/code/session_01JkwtxPHfobZxzvAqe53x62
Addresses the `pnpm audit --prod` warning that has been riding along on
recent CLI PRs: **`extract-zip` unvalidated symlink path traversal**,
[GHSA-jmr9-qjv8-65gv](https://github.com/advisories/GHSA-jmr9-qjv8-65gv)
/ CVE-2026-56876, high.
## There is no version to upgrade to
`pnpm audit` renders "Patched versions >=2.0.2", which reads like a bump
would fix it. It would not:
- `npm view extract-zip versions` ends at **2.0.1** — no 2.0.2 was ever
published
- the advisory itself records `"first_patched_version": null`
So `pnpm audit --fix --prod` cannot resolve this, and neither can a
version constraint.
## What the flaw actually is
I reproduced it rather than working from the summary. `extract-zip`
2.0.1 **does** block writes that traverse a symlink — an archive with
`link -> ../../ESCAPED` plus `link/pwned.txt` is refused with `Out of
bound path ... while processing file link/pwned.txt`, and nothing lands
outside the target.
What it does not do is validate the symlink itself. An archive whose
only entry is `escape-abs -> /tmp/ABSOLUTE-TARGET` extracts cleanly and
plants that dangling symlink in the output directory. Per the advisory:
*"Depending on how extract-zip is used, an attacker could read or write
to arbitrary files."* Every call site here extracts an archive and then
reads or copies files back out of the tree, which is exactly the shape
that turns a planted symlink into an arbitrary-path read.
## Why not swap the library
`#4183` moved tar extraction to `Bun.Archive`, so that was the obvious
candidate. It does not work: **`Bun.Archive` cannot read zip archives at
all.** A benign zip fails with `ReadError`, same as the malicious ones —
those "refusals" were read failures, not security. Verified before
ruling it out.
## Why not fix it upstream
Because it already is, three times over, and none of it has moved:
| PR | Date | Targets |
| --- | --- | --- |
|
[max-mapper/extract-zip#158](https://github.com/max-mapper/extract-zip/pull/158)
| 2026-08-14 | GHSA-jmr9-qjv8-65gv, this advisory |
| [#160](https://github.com/max-mapper/extract-zip/pull/160) |
2026-08-17 | CVE-2026-19693, symlink at the final path component |
| [#161](https://github.com/max-mapper/extract-zip/pull/161) |
2026-08-20 | CVE-2026-56876 again, and bumps the version to 2.0.2 |
All three are mergeable with zero review comments. The repository's last
commit to `master` was **August 2021**; 2.0.1 shipped June 2020. [Issue
#159](https://github.com/max-mapper/extract-zip/issues/159), a
downstream consumer asking whether a patched release is planned, has
three replies — every one from another stranded consumer, none from a
maintainer.
A fourth PR would not be a contribution. And our implementation is the
wrong shape to upstream regardless: refusing *every* symlink is right
for an application whose archives never carry one, but a breaking change
for a general-purpose library, where the correct fix is validating that
a link's resolved target stays inside the extraction root — which is
what #158 and #161 already do. Upstream even tracks "do not extract
symlinks" as a separate [feature
request](https://github.com/max-mapper/extract-zip/issues/140), open
since 2023.
So this is a deliberate local workaround on a timeline we control. The
helper's doc comment records the tracking PRs and the condition for
deleting it: a published release that actually carries the fix.
## The fix
No archive either package extracts — release binaries, companion assets,
skills, sidecar bundles — legitimately contains a symlink. So the entry
type is refused outright, via `extract-zip`'s own `onEntry` hook, which
runs **before** the entry is written. A rejected archive plants nothing.
All four call sites now go through it:
| package | call site |
| --- | --- |
| `cli` | `services/upgrade-binary.ts` |
| `cli` | `services/run-companion-modules.ts` |
| `cli` | `effects/install-skill.ts` |
| `cli-local-tools` | `src/bundled-binaries.ts` |
Each package carries its own copy of the helper rather than sharing one.
`run-companion-modules.ts` is bundled as a standalone companion module
and imports nothing from `@composio/cli-local-tools`; routing it through
that package to share ~15 lines would pull the local-tools tree into
those bundles.
## Verification
Committed three fixture archives (827 bytes total) and tested against
real zips, not mocks:
- benign archive extracts, contents intact
- `symlink-absolute.zip` (`-> /tmp/ABSOLUTE-TARGET`) refused, output
directory left empty
- `symlink-relative.zip` (`-> ../../outside`) refused, output directory
left empty
- `isSymlinkZipEntry` unit-tested across symlink, regular file,
directory, and Windows-authored entries with no Unix mode
**Mutation-checked:** neutering the predicate makes exactly the three
symlink tests fail and leaves the other four green, so the suite
genuinely detects the regression rather than passing by construction.
Suites: `@composio/cli` 123 files / 1263 passed,
`@composio/cli-local-tools` 7 files / 20 passed. Typecheck clean on
both. `pnpm validate:changesets` passes; note lands in
`ts/packages/cli/CHANGELOG.md`.
`extract-zip` stays as the dependency — it is still the only working zip
reader here, and with symlink entries refused the advisory's vector no
longer applies to this codebase.
https://claude.ai/code/session_01JkwtxPHfobZxzvAqe53x62
> ### ⚠️ Breaking change
>
> `proxy_execute()` now returns a dict instead of the generated
`SessionProxyExecuteResponse` model. Every caller since `py@0.11.4` that
reads the result with attribute access breaks at runtime with
`AttributeError`.
>
> ```python
> # before
> response.status
>
> # after
> response["status"]
> ```
>
> `data`, `headers`, and `binary_data` follow the same rule. No version
bump or changelog entry ships in this PR. That omission is deliberate,
so the release call stays explicit. Details below.
## Summary
Builds on @AseemPrasad's #4163, which spotted a real problem. Python's
`proxy_execute()` returns the generated client's
`SessionProxyExecuteResponse` directly, while TypeScript's
`proxyExecute()` projects onto a curated shape. Returning the generated
model leaks a regenerated artifact into a public SDK return type.
This PR keeps that fix and resolves the review findings on top. #4163's
commit is preserved with its original authorship. The commits on top
carry the correction and the review fixes.
## What changed relative to #4163
| | #4163 | Here |
|---|---|---|
| Key casing | `binaryData`, `contentType`, `expiresAt` | `binary_data`,
`content_type`, `expires_at` |
| `status` type | declared `int`, returned `200.0` | declared `int`,
returns `200` |
| Test doubles | `SimpleNamespace` | real `SessionProxyExecuteResponse`
/ `BinaryData` |
| `mypy` | fails `nox -s chk` | clean |
| Docs | 3 snippets left broken | fixed |
**Casing.** Python public APIs use snake_case and TypeScript public APIs
use camelCase. The fields and their meanings match across SDKs, and the
spelling follows each language. `session.delete()` already works this
way (`session_id` in Python, `sessionId` in TypeScript), and so does
`RemoteFile` (`expires_at` / `expiresAt`).
**`status` and `size` are narrowed to `int`.** The generated model types
both as `float` and pydantic coerces, so a response read straight off it
renders `200.0` where TypeScript renders `200`. #4163 declared `int` but
still returned `200.0`. That mismatch also failed `nox -s chk`:
```
composio/core/models/session_context.py:56: error: Incompatible types
(expression has type "float", TypedDict item "status" has type "int") [typeddict-item]
```
**Tests use the real generated models again.** `SimpleNamespace` accepts
any attribute name and any type, so it silently tolerates a client
regeneration that renames or retypes a field. It was also what hid the
`float` coercion, since `assert result == {"status": 200}` passes
against `200.0`. The suite now asserts the narrowed types directly. This
matters ahead of the `composio-client` 2.x migration, which types every
response field as `Any` and removes type checking on this projection
entirely. The tests become the only remaining check.
**Simplification.** The projection folds into `proxy_execute_impl`, so
both entry points are a single call rather than an impl-then-normalize
pair. `response.binary_data` is read directly instead of through
`getattr(..., None)`. The defensive default could never fire on a typed
response, but it made mypy infer `Any` and stop checking the projection.
**Docs.** Three Python snippets that read the result as attributes are
fixed, and the response-shape table gets a per-language column. The
follow-up commit also marks `headers` and `data` as nullable in that
table, replaces the "returns the upstream response verbatim" claim with
what the projection actually does, and documents that `expires_at` can
be absent in TypeScript and `None` in Python.
## Breaking change
The method has shipped since `py@0.11.4`. Both directions of the old
access pattern were already inconsistent in the repo.
`python/examples/custom_tools_agent_test.py:95` does `res["status"]`,
which raises `TypeError` on `next` today and is fixed by this PR. The
doc snippets did attribute access and are updated here.
No changelog entry and no version bump are included. That is deliberate,
so the release call stays explicit rather than implied by the merge.
## How Has This Been Tested?
```bash
cd python
mypy --config-file config/mypy.ini composio/ tests/ # clean
ruff check --config config/ruff.toml composio/ tests/ # clean
pytest tests/ # 1336 passed, 33 skipped
```
`ruff format` was run with the repo's pinned toolchain.
## Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Refactor/Chore
- [ ] Documentation
- [x] Breaking change
## Checklist
- [x] I ran linters/tests locally and they passed
- [x] I updated documentation as needed
- [x] I added tests or explain why not applicable
- [ ] I added a changeset if this change affects published packages. Not
applicable: `AGENTS.md` reserves changesets for published TypeScript
packages
https://claude.ai/code/session_01GsD8zvAhrjFwk144oWkD9K
---------
Co-authored-by: AseemPrasad <aseemprasad0520@gmail.com>
Co-authored-by: Kshitij Jhunjhunwala <113939507+KJ-11@users.noreply.github.com>
## What
Points all instructional docs (and the Python example) for custom OAuth
credentials to the **v1** callback URL:
`https://backend.composio.dev/api/v1/auth-apps/add`
...replacing the previously documented
`https://backend.composio.dev/api/v3.1/toolkits/auth/callback`.
## Why
The backend returns the **v1** URL as the default redirect URI for every
toolkit (it's what flows into the generated
`docs/public/data/toolkits.json`), and Consumer product expects v1 — but
Consumer doesn't surface the callback URL, so users go to the docs, find
the **v3.1** URL, and whitelist the wrong one in their OAuth app.
Flagged as a top confusion in Go Live calls. Fixes
[UXE-261](https://linear.app/composio/issue/UXE-261/update-docs-to-point-to-v1-callback-url).
## Files changed
- `docs/.../authentication/programmatic-auth-configs.mdx` (ticket's
primary target)
- `docs/.../authentication/white-labeling-authentication.mdx`
- `docs/.../auth-configuration/white-labeling.mdx`
- `docs/.../auth-configuration/custom-auth-configs.mdx`
- `python/examples/auth_configs.py`
## Deliberately NOT changed
- **`docs/.../migration-guide/new-sdk.mdx`** — states *"The callback URL
is now v3… The previous URL was v1"*. It documents the v1→v3 migration
history; flipping it to v1 would make it self-contradictory. Left as-is.
- **`docs/public/data/toolkits.json` / `ts/docs/api/toolkits.md`** —
generated from the backend API, already v1. No manual edit.
## ⚠️ Open question worth confirming
The migration guide says v3 is the *current* callback and v1 is
*previous*. This PR points developers *back* to v1 to match Consumer.
That's a real backend/product inconsistency, not just a docs typo —
worth confirming with @rahul / @abhishek that v1 is genuinely the URL
Consumer expects (and whether the migration guide messaging should be
revisited separately).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Consumer product and the backend-provided default redirect URI use the
v1 callback (api/v1/auth-apps/add), but the developer docs instructed the
v3.1 URL (api/v3.1/toolkits/auth/callback). Users copied the wrong URL
into their OAuth app allow lists. Align all instructional docs + the
Python example with v1.
The migration guide (migration-guide/new-sdk.mdx) is left unchanged: it
documents the v1->v3 history and flipping it would make it contradictory.
Fixes UXE-261.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
Knowledge search indexes pages as section-level records. Exact and
keyword-only retrieval returned those chunks directly, so a query such
as `gmail` could show repeated cards and promote raw subsection names
without their page context. Markdown comments inside fenced code blocks
could also be mistaken for headings, producing labels such as `Using
array format`.
This makes keyword result finalization consistent with hybrid search,
gives each match a clear page → section → excerpt hierarchy, and keeps
code examples searchable under their real document headings.
## Changes
- Deduplicate keyword-only results by canonical page while preserving
keyword rank order
- Fill the result list from the full candidate set after deduplication
- Promote exact titles and identities above incidental body mentions on
the exact-match fast path
- Keep the page title primary and show a distinct matched section as
supporting context
- Normalize Markdown in displayed section labels and remove repeated
prefixes or dangling separators
- Ignore heading-like comments inside fenced code when building section
records while retaining the code in searchable content
- Rebuild the semantic artifact from the corrected section boundaries
- Add regressions for ranking, result fill, rendered card behavior, and
fenced-code parsing
## Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Refactor/Chore
- [x] Documentation
- [ ] Breaking change
## How Has This Been Tested?
- `cd docs && bun run test` — 472 passed, 0 failed
- `bun run types:check`
- `bun run lint`
- `bun run lint:links` — 0 errors
- `bun run check:kb-semantic` — artifact current at 876 records
- `bun run build` — 2,116 static pages generated
- Local API and rendered-page smokes for `gmail`, `user_bob`, and `using
array format`
## Screenshots (if applicable)
The Vercel preview generated for this PR can be used to verify the
search result cards.
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [x] I updated documentation as needed
- [x] I added tests or explain why not applicable
- [x] Changeset not required (docs-only change)
## Additional context
The semantic artifact changed from 1,033 to 876 records because 157 fake
document sections created from comments inside fenced code blocks were
removed. Their code content remains indexed under the correct parent
sections.
The repo's existing `COMPOSIO_ORG_API_KEY` secret is used for something
else. This points the docs-agent-eval check at a **dedicated** secret +
env var name (`COMPOSIO_ORG_API_KEY_FOR_CODING_AGENT_EVAL`) end to end,
so the two never get confused. Engine pin bumped to the version that
reads the new name (falls back to the old for local dev). No bare
`COMPOSIO_ORG_API_KEY` reference remains in this workflow.
The composio repo already has a COMPOSIO_ORG_API_KEY secret used for
something else. To avoid any confusion, the docs-eval check now uses a
dedicated secret AND env var name end to end (secret ref, env var, and
validate step), and pins the engine (19f6417) that reads that name.
Nothing in this workflow references the bare COMPOSIO_ORG_API_KEY anymore.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
## Summary
Launches the Support Knowledge Base MVP at `/kb` and makes its search a
unified public support surface across canonical docs, KB guides,
toolkits, examples, API reference pages, changelog entries, and OAuth
guides.
Internal context: [Support Knowledge Platform
PRD](https://app.notion.com/p/composio/Support-Knowledge-Platform-PRD-3bcf261a6dfe816297f9feeb5398a9af)
## Changes
- Publishes 124 reviewed resources from the reconciled public-only
`ComposioHQ/support-knowledge` snapshot with canonical guide, topic,
toolkit, browse, sitemap, and LLM-readable routes.
- Adds keyword-first unified retrieval over all public source classes.
Strong exact titles and tool slugs return without an embedding; weaker
natural-language queries can fuse keyword results with semantic docs/KB
matches.
- Uses a checked-in format-v2 semantic artifact with 1,033 public
docs/KB records. No vector database or Algolia NeuralSearch is
introduced.
- Preserves compatibility with the deployed Algolia keyword index while
retaining a full local fallback and branch-current docs/KB overlays for
previews.
- Server-renders initial `/kb/search` results, removes the client
hydration request waterfall, and shows concise mixed-source labels
without the redundant `Guide` badge.
- Publishes current Composio For You setup instructions for 17
client/setup categories, including pinned public onboarding media from
the dashboard repository.
- Adds semantic timeout, rate/cost/concurrency protection, structured
degradation logging, and PostHog query/outcome analytics with credential
redaction and stage timings.
- Routes the existing public `composio` skill through the unified
endpoint when current docs and CLI/tool schemas do not answer a support
question; no separate support skill is introduced.
- Keeps scheduled/dispatch-based public snapshot refresh automation and
expands privacy validation to all publishable bodies and metadata.
## Type of change
- [ ] Bug fix
- [x] New feature
- [ ] Refactor/Chore
- [x] Documentation
- [ ] Breaking change
## How Has This Been Tested?
- `cd docs && bun run check:kb-semantic` — artifact current, 1,033
records
- `cd docs && bun run test` — 464 passed, 0 failed (27,240 assertions)
- `cd docs && bun run types:check` — passed
- `cd docs && bun run lint` — exit 0 with existing repository warnings
- `cd docs && bun run lint:links` — 0 errors
- `cd docs && bun run build` — optimized production build succeeded
- `node ts/scripts/validate-agent-skills.mjs` — 18 canonical skills
validated
- `node ts/scripts/test-skill-routing.mjs` — 18 routing probes passed
- [End-to-end agent + skill smoke
eval](https://github.com/ComposioHQ/composio/pull/4159#issuecomment-5364319453)
— 6 pass / 1 partial across seven representative support questions;
no-skill control was acceptable on roughly 3/7
- Five representative API and server-rendered search smoke queries
passed; exact action slugs skipped semantic retrieval while
natural-language queries used hybrid mode
- Final independent review completed; its one important citation finding
was fixed before push
- `git diff --check`
## Preview
- [Knowledge
Base](https://docs-git-codex-support-knowledge-mvp.preview.composio.dev/kb)
- [Unified
search](https://docs-git-codex-support-knowledge-mvp.preview.composio.dev/kb/search?q=how+to+connect+to+claude)
- [Toolkit
directory](https://docs-git-codex-support-knowledge-mvp.preview.composio.dev/kb/toolkits)
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [x] I updated documentation as needed
- [x] I added tests or explain why not applicable
- [x] I added a changeset if this change affects published packages (not
applicable)
Bumps the engine pin so the live check includes the fix for the
false-PASS the first live smoke (#4200) exposed: when provisioning fails
(invalid org key) the eval ran 0 agents but reported ✓ PASS. Engine now
returns ERROR for 0-run / infrastructure crashes. Separately, the org
key secret (COMPOSIO_ORG_API_KEY) in this repo is invalid (401) and
needs re-adding.
Engine now reports ERROR (not PASS) when 0 agents ran / provisioning fails.
Surfaced by the first live smoke on #4200.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
## Security fixes for the docs-agent-eval caller (Cursor Bugbot
findings)
Follow-up to #4158 (merged). Cursor Bugbot flagged two real issues on
the App-token caller; both fixed here. Two other findings were already
resolved in the merged version (noted below).
**Fixed:**
- **App token left on disk (Medium):** check out the private engine with
`persist-credentials: false`. Otherwise `actions/checkout` writes the
GitHub App installation token into `engine/.git/config`, and the eval
then runs untrusted agent code with filesystem access on the same runner
— which could read that token (read access to the private engine repo).
The token is only needed for the one-time fetch.
- **Non-docs previews start evals (Medium):** the `deployment_status`
trigger now requires the `environment_url` to start with `https://docs-`
(matching the label path's `docs-*` rule), so a non-docs
`*.preview.composio.dev` deployment can't start or cancel a docs eval.
**Already resolved in #4158 (stale Bugbot findings from the pre-App
design):**
- "Unrelated labels cancel evals" — the route job's `startsWith(…,
'run-docs-tests')` guard skips unrelated `labeled` events before
anything enters the eval concurrency group.
- "Eval token cannot read deployments" — `deployments: read` is set on
both jobs.
Informational check, non-blocking. No behavior change beyond the two
hardening fixes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- App token left on disk (Medium): checkout the private engine with
persist-credentials: false, so the App installation token is never written
to engine/.git/config where the eval's agent steps (filesystem access on the
same runner) could read it. Token is only needed for the one-time fetch.
- Non-docs previews start evals (Medium): the deployment_status trigger now
requires the environment_url to start with https://docs- (matching the
label path's docs-* rule), so a non-docs *.preview.composio.dev deployment
can't kick off or cancel a docs eval.
Already-resolved findings (stale, from the pre-App design): "unrelated labels
cancel evals" (route job's label guard skips unrelated labels before the
concurrency group) and "token cannot read deployments" (deployments: read is
set on both jobs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
## What this adds
A **label-triggered, non-blocking CI check** that tests whether a docs
PR's proposed content can still guide a fresh coding agent to working
Composio code — before merge, against the PR's own Vercel preview.
Add the **`run-docs-tests`** label to a docs PR →
1. A **route job** decides if and for which PR to run: label present +
PR author is an org member/collaborator. Two entry paths so either order
works — label added after the preview built (`pull_request` events), or
preview built after the label (`deployment_status` events, which also
cover new pushes; `synchronize` is deliberately not used).
2. A deterministic router classifies the change: **no-eval** (assets
only) / **Smoke** (1 agent) / **Deep** (5 agents — auth, install,
env-var, code-sample, endpoint, or navigation changes).
`run-docs-tests-deep` forces Deep.
3. The PR's exact-HEAD preview URL is resolved from **GitHub's
deployment records** (written by the Vercel↔GitHub integration, read
with the per-run `GITHUB_TOKEN`) — **no Vercel credentials anywhere**
(decision 2026-08-19). Isolated agents get it served through a rewrite
proxy — they believe they're on docs.composio.dev.
4. Each agent (fresh env, ephemeral Composio project, torn down after)
attempts the weather-toolkit task by following the docs.
5. An **independent judge** verdicts every run before teardown —
re-executes the agent's script with the run's live key and scans for the
v1 anti-patterns (hardcoded slugs, legacy packages, manual execution,
/examples cheating). The agent's own success claim is recorded but never
trusted; the PR comment shows L1·L2·L3 per run and any judge-overruled
self-reports.
6. Result lands here as a PR comment: **PASS / FAIL / ERROR**, run
counts, teardown status, and the agents' own docs-friction feedback. New
pushes supersede stale runs (`cancel-in-progress`).
**This file is a router + one SHA-pinned call** — all eval logic lives
in
[ComposioHQ/docs-agent-eval-ci](https://github.com/ComposioHQ/docs-agent-eval-ci)
(reusable workflow, SHA-pinned per org policy). See its
[ARCHITECTURE.md](https://github.com/ComposioHQ/docs-agent-eval-ci/blob/main/ARCHITECTURE.md)
for the full component map and design-spec compliance audit.
## Evidence it works (sandbox runs)
- End-to-end green Smoke, auto-routed:
[run](https://github.com/ComposioHQ/docs-agent-eval-ci/actions/runs/32056739696)
(PR #4125 → smoke → PASS 1/1)
- End-to-end green Deep, auto-routed:
[run](https://github.com/ComposioHQ/docs-agent-eval-ci/actions/runs/32057336110)
(PR #4136 → deep via auth-docs signal → PASS 5/5, incl. agent feedback
reproducing known docs findings)
- The exact `workflow_call` mechanism this shim uses:
[run](https://github.com/ComposioHQ/docs-agent-eval-ci/actions/runs/32059660256)
- Receipts-based preview resolution live-verified on PR #4174 (exact-SHA
→ snapshot URL); fork PRs (no Vercel build) correctly produce the
explained no-preview ERROR path
- Builder (DeepSeek V4 Flash) calibrated 16/16 against production docs
before any CI verdicts
## ⚠️ Before merge (repo admin)
1. ~~Secrets~~ **Done 2026-08-19**: `COMPOSIO_ORG_API_KEY` +
`DEEPSEEK_API_KEY` exist in this repo's Actions secrets. (`VERCEL_TOKEN`
is no longer needed — removed from the design.) Team-owned replacements
for the individually-owned keys remain part of the rollout (current
owner offboards Aug 31).
2. **Engine repo visibility**: `docs-agent-eval-ci` is currently
*internal*; a public repo cannot call an internal reusable workflow, so
it must be flipped public (pre-publication scrub done) **before** this
merges.
3. **Pin bump**: after the engine's `no-vercel-credentials` PR merges,
the `uses:` SHA here gets re-pinned to the engine's merge commit.
Labels `run-docs-tests` / `run-docs-tests-deep` are already created.
## Non-goals (V1)
Informational only — **do not mark this check required**. No merge
blocking, no statistical thresholds, no LLM routing; scenario is the
weather-toolkit eval only.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
## Update 2026-08-21: keep the engine PRIVATE via a GitHub App
Design change (Sarah): the engine repo stays **private** instead of
going public. A public repo can't `uses:` a private repo's reusable
workflow, so the `eval` job now runs the engine **inline**:
1. mints a short-lived **GitHub App token**
(`actions/create-github-app-token`, SHA-pinned) scoped read-only to
`docs-agent-eval-ci`
2. checks the private engine out with it (`path: engine/`)
3. runs the engine on the runner (`route_pr.py` → `run_full_eval.py`);
result comment + check status still posted with composio's built-in
`GITHUB_TOKEN`
**⚠️ This supersedes the earlier public-reusable-workflow design in this
PR — needs a fresh review of the `eval` job.**
New repo secrets required (added): `DOCS_EVAL_APP_ID`,
`DOCS_EVAL_APP_PRIVATE_KEY`, plus existing `COMPOSIO_ORG_API_KEY`,
`DEEPSEEK_API_KEY`. No Vercel credentials.
Before this can be tested: the GitHub App must be **installed** on
ComposioHQ with **Contents: Read** on `docs-agent-eval-ci`. Since the
check only fires from the default branch, testing happens after this
merges (informational check, non-blocking, safe to merge to test).
Keeps the engine repo private (Sarah 2026-08-21). A public repo can't
`uses:` a private repo's reusable workflow, so the eval job now:
1. mints a short-lived GitHub App token (create-github-app-token, SHA-pinned)
2. checks out the private engine repo with it (path: engine/)
3. runs the engine inline (route_pr.py -> run_full_eval.py) on the runner
Result comment + check status still posted with the built-in GITHUB_TOKEN
(composio's own; App token is read-only on the engine repo).
Needs repo secrets: DOCS_EVAL_APP_ID, DOCS_EVAL_APP_PRIVATE_KEY (added),
plus existing COMPOSIO_ORG_API_KEY, DEEPSEEK_API_KEY.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Picks up the two-phase eval (vague build + surprise probe), the independent
judge (PTY-driven probe, no self-reported evidence, full A-series + access
DQs + build-phase anticipation log), and the trimmed CI watchdog ceilings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
## What this fixes
A custom MCP toolkit registered from the consumer dashboard could appear in the dashboard/Tool Router but not in `composio search`.
The CLI was receiving the custom toolkit from Tool Router search, then trying to fetch its schema through the legacy `/api/v3/tools/{slug}` endpoint. That endpoint does not serve dashboard-owned `CUSTOM_*` toolkits, so it returned 404 and the CLI could not complete the search result.
## How it fixes it
- For remote `CUSTOM_*` toolkit slugs with an inline `input_schema`, the CLI uses the schema already returned by Tool Router search.
- For `CUSTOM_*` results that contain only a `schemaRef`, the CLI calls Tool Router's `COMPOSIO_GET_TOOL_SCHEMAS` meta-tool in the same session and uses the returned schema.
- Both paths store the schema through the existing local tool-definition cache path, so normal execution can use it.
- Managed Composio toolkits keep the existing schema-fetch path.
- Existing cache behavior is unchanged.
## Reproduction
We reproduced this against a production dashboard-created DeepWiki custom MCP:
- Before the fix, a cold-cache CLI search reached the custom toolkit but failed on the legacy schema lookup with 404.
- With the fix, unfiltered `composio search` returned the custom MCP tools.
- Executing the returned custom tool succeeded and returned the DeepWiki response.
- Regression coverage also verifies the schema-reference path does not use the legacy endpoint.
## Scope
This fixes the `composio search` → normal execute path for dashboard-created custom MCP toolkits.
This PR does not change:
- CLI cache TTLs or refresh behavior
- Dashboard-to-CLI cache invalidation
- The separate schema paths used by `composio tools info` or `composio execute --dry-run`
## Validation
- Focused CLI tests: 21 passed
- Full CLI test suite: 1,256 passed, 1 skipped
- Source and test typechecks passed
- Focused Oxlint passed
- Production reproduction passed
Engine now includes the independent per-run judge (full v1 rubric,
smoke+deep tested) and receipts-based preview resolution.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
## Summary
Automated sync of backend data into the docs site. Triggered by:
`schedule`.
## What changed
- **Toolkit catalog** (`docs/public/data/toolkits.json`,
`toolkits-list.json`) — refreshed list of available toolkits, auth
schemes, and tools from the backend API
- **OpenAPI specs** (`docs/public/openapi.json`,
`docs/public/openapi-v3.json`, `docs/public/openapi-webhooks.json`) —
latest v3.1 and v3.0 API specifications plus the webhook-events spec,
fetched from production
- **API reference pages** (`docs/content/reference/api-reference/`,
`docs/content/reference/v3/api-reference/`) — regenerated index pages
for both API versions
- **Meta tools reference** (`docs/public/data/meta-tools.json`,
`docs/content/toolkits/meta-tools/*.mdx`) — updated meta tool schemas
and reference docs
Co-authored-by: Sushmithamallesh <19796925+Sushmithamallesh@users.noreply.github.com>
Co-authored-by: Sushmitha Mallesh <sushdec6@gmail.com>
jkomyno