- Log the full pusher:subscription_error payload (type, error, status)
instead of a flattened String(data.error) so operators can tell auth
failures from permission failures.
- Log the subscription success message only when Pusher dispatches
pusher:subscription_succeeded, not when subscribe() returns.
- Extend regression coverage for both behaviors.
Applies review findings #2 and #3 from the PR #4448 review.
## Summary
`transformToolkitRetrieveResponse` projects `auth_config_details[]` key
by key and silently dropped
three keys the pinned client declares. `is_secret` and
`legacy_template_name` are snake_case, so zod
stripped them; `auth_hint_url` was never listed.
The API documents `is_secret` as "Clients use it to decide whether to
mask the input". Without it a
consumer rendering its own credential form cannot tell an API key from a
shop subdomain, so a field holding a secret is not reported as one.
## Changes
- Map auth fields explicitly; add `isSecret`, `legacyTemplateName` and
`authHintUrl` to the schemas
- Spread optional keys conditionally, so a key the API omits stays
absent rather than `undefined`,
which would beat a caller's fallback
- Add a runtime test over all four field groups, plus a type-level test
that fails the build on an
unlisted client key in any group. It catches new wire keys, not an
assignment removed from the
mapper; the runtime test pins that for the keys known today
- Add a patch changeset for `@composio/core`
## Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Refactor/Chore
- [ ] Documentation
- [ ] Breaking change
## How Has This Been Tested?
- `pnpm --filter @composio/core test`: 55 files, 1288 passed, 2 expected
fail; typecheck and
`validate:changesets` exit 0
- `pnpm lint:packages`: exit 0 with 54 pre-existing warnings, one in a
file this PR touches; no new
warning on added lines
- The runtime test fails 5 of its 7 cases on `origin/next`; the two
green on both are the regression
guards. The type-level test was checked against all four client
interfaces
- Node 24.18.0; `mise.toml` pins 24.17.0, unavailable here, so it was
not run under it. Not
run: the root workspace suite (its `test:toolchain` needs Bun), Docker
E2E, the Python suite
## Screenshots (if applicable)
N/A
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed, with the exceptions
listed above
- [x] I updated documentation as needed (none needed: the new keys carry
TSDoc; the SDK reference is
generated from unchanged signatures)
- [x] I added tests or explain why not applicable
- [x] I added a changeset if this change affects published packages
## Additional context
Left out on purpose: `user_visible` and `required_scopes`, in the spec
but not the pinned client;
`deprecated_auth_provider_details`, which the client marks deprecated;
and the keys this transformer
drops outside `auth_config_details`, all pre-existing and adjacent.
One behavior change: the eager mapping now throws if an auth config
detail arrives without its
`auth_config_creation` group, where the pass-through still returned the
connection fields. The
client declares both groups required and non-nullable, so this affects
only out-of-contract
responses.
## Summary
Auto-generated Python SDK reference docs from `python/composio/`.
Regenerates pages at `docs/content/reference/sdk-reference/python/` to
reflect changes in the Python package's public API (new methods, updated
signatures, changed types).
Replacing the pass-through with an eager projection changed the failure mode for
an auth config detail that omits a field group. Passing the group through left
`undefined` in place, which zod rejected as a handled validation error;
`transformToolkitAuthFieldGroup` instead reads `group.required`, so the same
response now throws a `TypeError` before validation runs and crashes
`toolkits.get()`.
Default a missing or null group, and a missing list inside a group, to empty
lists. This repo's docs pipeline already assumes that shape: the schemas in
`docs/lib/toolkit-api.ts` carry `.catch({ required: [], optional: [] })` on the
same fields. Normalizing also keeps the other group usable when only one is
absent, which is typically the one the caller asked for, where a validation
error would have returned nothing.
Reported by greptile-apps on #4411.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Atomic Agent is a local-first agent (CLI and TUI) that ships a built-in
Composio integration: it connects to the hosted tool router over
Streamable HTTP MCP, so tools are registered at startup as
mcp.composio.*.
Setup is done from the Integrations tab or by adding COMPOSIO_API_KEY to
the agent's .env, so the entry documents that flow rather than the CLI
install used by other terminal agents.
- parse :raises Exc: docstring fields (plus Sphinx synonyms) into a
structured Raises section instead of leaking raw directives into the
Returns description
- normalize inline reST in all rendered prose: roles (:class:, :func:,
:meth:, ...) honor ~ short-name semantics; double-backtick literals
become single-backtick inline code
- regenerate docs/content/reference/sdk-reference/python/ pages
- add regression tests for raises parsing and reST normalization
Flagged by Greptile on the legacy-repo auto-PR (ComposioHQ/composio#4479).
Fixes for four `composio` CLI issues found while using the released
binary (v0.4.0). Each item below was reproduced from source on `next`
before the change and re-run after it. All commands were run with stdin
from `/dev/null`.
## 1. Bare `--stream` on `composio listen` is rejected
**Repro (before):**
```
$ composio listen GMAIL_NEW_GMAIL_MESSAGE --stream
Received unknown argument: '--stream='
<usage text>
exit=1
```
The command's own EXAMPLES block shows `composio listen
GMAIL_NEW_GMAIL_MESSAGE -p @trigger.json --stream`, and
`--stream=.data.text` / `--stream '.data.text'` both work.
**Cause:** `normalizeListenStreamFlag` in `src/commands/index.ts`
rewrites a valueless `--stream` to `--stream=`. `@effect/cli` only
recognises `--flag=value` when the value is non-empty (`FLAG_REGEX =
/^(--[^=]+)(?:=(.+))?$/`), so `--stream=` is treated as an unknown
argument.
**Change:** rewrite bare `--stream` to the two tokens `--stream ''`
instead. The listen handler already treats an empty path as "stream the
whole payload", so no handler change is needed.
**After:** `composio listen GMAIL_NEW_GMAIL_MESSAGE --stream` and `...
--stream --max-events 1` both parse and proceed to trigger creation.
Tests added in `listen.cmd.test.ts` cover bare `--stream` at the end of
argv and followed by another option.
## 2. Help examples reference a tool slug that no longer exists
**Repro (before):**
```
$ composio --help full | grep SLACK_
SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL -d '{ channel: "general", text: "Hello" }'
$ composio execute SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL --dry-run -d '{ channel: "general", text: "Hello team" }'
services/HttpServerError
Caused by: 404 {"error":{"message":"Tool SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL not found","code":2401,"slug":"Tool_ToolNotFound",...}}
exit=1
```
The current slug is `SLACK_SEND_MESSAGE`. Its input schema (`composio
tools info SLACK_SEND_MESSAGE`) has `channel`, `markdown_text`,
`blocks`, `thread_ts`, ... and no `text` key.
**Change:** the three occurrences in `src/commands/root-help.ts`
(execute examples, the `run` script example, and the root help snippet)
now use `SLACK_SEND_MESSAGE` with `markdown_text`. No other file in the
repo references the old slug.
## 3. `composio tools info <bad-slug>` prints nothing and exits 0
**Repro (before):**
```
$ composio tools info BOGUS_TOOL_XYZ; echo exit=$?
exit=0
$ composio tools info BOGUS_TOOL_XYZ 2>/dev/null | wc -c
0
```
`tools info` catches the 404 with the shared `handleHttpServerError`
helper, which logs through `ui.log.error` (rendered only when stderr is
a TTY) and returns a fallback value, so the command completes
successfully with no output when piped.
**Change:** `src/effects/handle-http-error.ts` now sets
`process.exitCode = 1` (the convention used elsewhere in the CLI, e.g.
`toolkits search`) and, when stderr is not a TTY, writes the error
message plainly via `ui.error` so it is still visible. The decorated
log/hint/suggestion flow is unchanged. This helper is shared by the
`info`/`create`/`enable`/`disable`/`status` commands for tools,
triggers, auth configs and connected accounts, all of which return early
on the same failure path, so they now exit non-zero on an API error as
well.
**After:**
```
$ composio tools info BOGUS_TOOL_XYZ; echo exit=$?
Tool "BOGUS_TOOL_XYZ" not found.
exit=1
```
Tests added in `tools.info.cmd.test.ts` assert the exit code and message
for an unknown slug, and that a valid slug leaves the exit code
untouched.
## 4. `composio listen <unknown-slug>` reports a missing connection
instead of an unknown slug
**Repro (before):**
```
$ composio listen BOGUS_SLUG_XYZ
commands/ListenCommandError • No active connected account found for toolkit "bogus" and consumer user "...". Run `composio link bogus` first.
exit=1
```
The toolkit is inferred from the slug prefix and the connected-account
lookup runs without checking that the trigger type exists, so a typo
sends the user to link an account for a toolkit that does not exist.
**Change:** in `src/commands/listen.cmd.ts`, when no active connected
account matches, the command now retrieves the trigger type first. A 404
fails with a new `unknown_trigger` reason; any other outcome falls
through to the existing connected-account error. The same lookup runs
when creating the temporary trigger fails, because an active account for
the inferred toolkit (e.g. `GMAIL_NEW_GMAIL_MESAGE` with Gmail linked)
skips the first check; a 404 there replaces the generic `create_trigger`
error. The happy path makes no additional request.
**After:**
```
$ composio listen BOGUS_SLUG_XYZ
commands/ListenCommandError • Unknown trigger slug "BOGUS_SLUG_XYZ". List available slugs with `composio triggers list <toolkit>`.
exit=1
```
The test layer gained a `triggersTypes.retrieve` mock (the base client
would otherwise hit the network), and `listen.cmd.test.ts` covers the
unknown-slug case both without an account and with an active account for
the inferred toolkit (the test layer's `triggerInstances.upsert` can now
reject unknown slugs via `triggersData.rejectUnknownTriggerSlugs`). The
existing missing-connection test now provides a
`GMAIL_NEW_GMAIL_MESSAGE` trigger-type fixture so it still exercises the
connected-account branch.
## Also: `listen` example used a trigger with an empty config
While updating the help examples above, the `-p` example `composio
listen SLACK_RECEIVE_MESSAGE -p '{ trigger_config: { channel: "C123" }
}'` was replaced. `composio triggers info SLACK_RECEIVE_MESSAGE` returns
`"config": {"properties": {}, ...}`, so that example configures nothing.
`SLACK_CHANNEL_MESSAGE_RECEIVED` takes `channel_id`, so the example now
uses that trigger and key. Same one-line change in
`skills-src/composio-cli/index.ts`, which is the source for the
generated CLI skill.
## Changelog
All four fixes are recorded under Unreleased in
`ts/packages/cli/CHANGELOG.md` (the CLI is excluded from Changesets).
## Verification
- `pnpm typecheck` (src + test) in `ts/packages/cli`
- `pnpm test` in `ts/packages/cli` (validate:skills,
validate:boundaries, vitest)
- `oxlint` and `prettier --check` on the changed files
- Live re-runs of each repro above against the API from source
- listen: replace `Effect.catchAll`, which no longer exists in
effect@4.0.0-rc.112, with `Effect.result` and narrow the lookup failure
with `NotFoundError` from @composio/client instead of a structural
status check.
- tests: `it.scoped` is gone from @effect/vitest v4 (`it.effect` already
provides a Scope); switch the new cases over and assert the exit code on
`triggers info` too, so the shared handler's contract is covered by more
than one command.
- handle-http-error: also print the hint plainly when stderr is not a TTY.
- changelog: record the four user-visible fixes.
Claude-Session: https://claude.ai/code/session_01SvfF1wniMJsBDo5PaBjm2Y
This PR:
- Closes PRDE-1620
- rewrites the `tools.getInput` JSDoc example to pass the required
`text` field and read the generated `arguments` / `error` (it passed
`userId` and read a nonexistent `schema`)
- rewrites the `tools.proxyExecute` JSDoc example with the real flat
shape (`endpoint`, `method`, `connectedAccountId`, `parameters`) instead
of `toolkitSlug` / `userId` / `data`
- documents that a relative proxy `endpoint` is appended to the
toolkit's base URL, which can already include a path (Google Calendar:
pass `/users/me/calendarList`, not
`/calendar/v3/users/me/calendarList`), in the JSDoc,
`tools-direct/executing-tools`, and `extending-sessions/proxy-execute`
- adds a "Pin a version" section to `tools-direct/executing-tools` with
Python and TypeScript examples that pass `version`, and shows the lookup
via `tools.getRawComposioToolBySlug(...).version` in the `tools.execute`
JSDoc
- drops TypeScript comments that pointed at
`toolkits.get(...).meta.version`, which doesn't exist on the TypeScript
toolkit meta type
- regenerates `sdk-reference/typescript/tools.mdx` with `pnpm --filter
@composio/core generate:docs` and adds a `@composio/core` patch
changeset
Checks: `pnpm --filter @composio/core typecheck` passes, docs static
tests `execute-version`, `content`, and `dashboard-links` pass, and
every new TypeScript snippet type-checks against `@composio/core`.
## Summary
`composio execute` made eight backend requests. Two of them were the
same list of the user's connected accounts, fetched by two code paths
that cannot see each other. It is fetched once now. Interleaved A/B
against #4469, compiled binaries, 15 runs each on a small response:
1735ms to 1636ms best, 1934ms to 1845ms median. That is one round trip
(~140ms) off the critical path.
Results are identical to before in every case. The picker derives its
toolkit subset from the shared list with the exact semantics of its old
query, and falls back to that query when the shared list is truncated.
Fifth PR in the stack. Stacked on #4469; review #4463, #4464, #4468 and
#4469 first. #4483 builds on this one.
## Changes
1. `src/utils/memoize-in-process.ts` (new). Memoizes an Effect per key
for the process lifetime, shares one run between concurrent callers, and
drops a failure, defect or interruption so the next caller retries.
2. `listActiveConnectedAccounts` in `connected-account-selection.ts`:
the unfiltered `GET /connected_accounts` for a user, memoized by client
identity and user id. It fails with the raw rejection, and each caller
wraps that in its own error. `resolveToolRouterSessionConnections` reads
from it when it has no toolkit filter, which is the execute path. With a
filter it keeps its own request.
3. `resolveConnectedAccountForToolkit` used to issue its own request,
toolkit-filtered, `limit: 100`. It now derives that from the shared
list: same slug match, server order preserved, first 100. If the shared
list has a `next_cursor` or `total_items` above what it holds, the
toolkit's accounts may sit past the cut, so the original filtered
request runs instead.
4. `get_latest_version` goes through the same memo. The definition
refresh fetched it twice with identical headers on the stale path; that
is one request now. The executor's own version lookup sends no org or
project headers and stays a separate request. Scoping it would change
which definition it resolves under, which is a semantics decision, not a
perf one.
What does not change: the request list on a normal execute is now
`project/resolve`, `connected_accounts`, `get_latest_version` twice,
`consumer/config`, `session`, `execute`. Error messages are unchanged;
the fallback passes the raw rejection through so the picker's message
reads as before.
## Type of change
- [ ] Bug fix
- [ ] New feature
- [x] Refactor/Chore
- [ ] Documentation
- [ ] Breaking change
## How Has This Been Tested?
Bun 1.4.1+4661e494f, Node 24.20.0, pnpm 11.8.0, linux-x64.
1. `cd ts/packages/cli && pnpm run typecheck && pnpm run
validate:boundaries`
2. `pnpm exec vitest run`: 131 files, 1341 passed, 1 skipped (whole
stack). New tests cover the memo sharing one run per key and retrying
after a failure or a defect. The execute suite already covers account
selection with and without a selector and passes unchanged. The test
layer builds a fresh client per test, so the client-keyed memo does not
bleed between tests.
3. Request count: hooked `fetch` while running `execute
HACKERNEWS_GET_ITEM_WITH_ID` from source. `connected_accounts` appears
once, the rest of the list as before.
4. Timing: `pnpm build:binary`, then the interleaved A/B above against
#4469's binary. A second round of 12 gave 1788 to 1621ms best, 2045 to
1963ms median.
## Screenshots (if applicable)
Not applicable.
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [ ] I updated documentation as needed
- [x] I added tests or explain why not applicable
- [ ] I added a changeset if this change affects published packages
No docs describe the request sequence. `@composio/cli` is private, so no
changeset.
## Additional context
The rest of an execute, from the same trace: `project/resolve` 140 to
640ms with no cache, `tool_router/session` 385 to 655ms created per
invocation, and the execute call itself 500 to 730ms. The
connected-account cache in `consumer-short-term-cache.ts` would take
`connected_accounts` off the path entirely, but
`DISABLE_CONNECTED_ACCOUNT_CACHE` defaults to on, and enabling it fails
no-auth toolkits with "not connected" because the cached list does not
include them. Both are separate changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01Wx9gEjuiHux2weiHjdNcDs
Whether `total_items` honors the `statuses` filter is a server detail. A
count that did not would make the shared list look truncated on every call
and quietly route the account picker onto the second request forever. Take
a page as complete when it is shorter than the requested size and carries
no cursor, and cover both the short-page and full-page cases.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
The derived toolkit filter lower-cased only, while the grouping helpers in
the same module trim first and the fallback sent the raw slug to the server.
A padded `--toolkit` value could therefore match or miss depending on how
many accounts the user has. Trim once, compare with the shared normalizer,
and send the trimmed slug on the fallback. Drop the optional chaining on
fields the client types as required.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
`resolveCommandProject` gives `composio execute` the consumer project's
nano id, while the cache refresh asked `getFor` for a client keyed on the
long id. Two clients meant two memo keys, so the connected-account list
was still fetched twice per consumer execute. Use the same id on both.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
The memoized `get_latest_version` is keyed on slug, org, and project. The
command's version check passed the resolved project while the executor's
schema lookup passed nothing, so the two never shared a key and
`composio execute` still issued both requests. Hand the executor the
project scope the command resolved.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
The user context is live state that `login` fills in during the same
process. Memoizing the `null` answered before that would skip the version
check for that tool for the rest of the run. Resolve the key outside the
memo and cache only real lookups.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
A memoized tool version or connected-account list lives for the whole
process, so within one vitest file the first case that lets the lookup
succeed would hand its answer to every later case sharing the key. Expose
`clear()` on each memo plus a process-wide `clearInProcessMemos`, and call
the latter from the shared vitest setup before every test.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
`Effect.cached` makes the first caller the owner of the shared run, so
interrupting that caller stores the interrupt in the cell and replays it to
every fiber already waiting on it. On `composio execute --parallel` two specs
sharing a slug could take the second one down when the first one's
`Effect.all` failed on the tool-detail fetch.
Run the memoized effect on a detached fiber that completes a Deferred, so a
caller's interruption cancels only its own wait.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
memoizeInProcess evicted a key only on typed failures, but Effect.cached stores
the whole Exit, so a defect or an interrupted first caller was replayed for the
rest of the process. Evict on any failure cause.
ActiveConnectedAccountsListError was only ever unwrapped to its cause by both
callers, so the shared list now fails with the raw rejection.
`composio execute` listed the user's connected accounts twice on every
call: once toolkit-filtered by the account picker, once unfiltered by
session creation. The two code paths cannot see each other. The
unfiltered list is now fetched once per process and shared, and the
picker derives its toolkit subset from it: same slug match, server order
kept, first 100, exactly what its own query returned. If the shared list
is truncated (more active accounts than one page), the picker falls back
to its original request, so results are identical in every case.
Interleaved A/B against the parent commit, compiled binaries, 15 runs
each on a small response: 1735 -> 1636ms best, 1934 -> 1845ms median.
One fewer request on the critical path, ~140ms.
`memoizeInProcess` memoizes an Effect per key for the process lifetime,
shares one run between concurrent callers, and drops failures so the next
caller retries. It also covers `get_latest_version`, which the definition
refresh fetched twice with the same key on the stale path. The executor's
own version lookup stays unscoped and separate; scoping it to org and
project would change which definition it resolves, so that is left as is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wx9gEjuiHux2weiHjdNcDs
## Summary
- Restore automatic plugin setup for detected supported agent hosts
during CLI installation. #4015 unintentionally changed
`COMPOSIO_INSTALL_PLUGINS` from `1` to `0`; this restores the existing
`setup --target auto --yes --if-present` path.
- Preserve `COMPOSIO_INSTALL_PLUGINS=0` and `--no-plugins`, including
flag precedence over an explicit `1`. Update installer help and docs.
- Cover the genuinely unset default, both opt-outs, and plugin failure
recovery in the hermetic installer harness.
## Validation
- Regression test failed against the old default: `FAIL: sh plugin setup
enabled by default`.
- `bash test/install-sh-release-resolution.test.sh`: passed under sh and
dash.
- `bash test/install-sh-atomic-replace.test.sh`: passed under sh and
dash.
- `bun test docs/tests/static/content.test.ts`: 7 passed.
- Shell syntax checks and `git diff --check`: passed.
## Known verification limitations
- ShellCheck is unavailable locally; the existing installer CI runs it.
- Tests use a fake CLI; no native plugins were installed on this
machine. Full docs build and Docker install E2E were not run locally.
Shell configuration, authentication, and plugin implementation are
unchanged.
## Summary
Security evaluations interpreted connected-account credential fields as
a token-export path and conflated keeping tokens out of application code
with keeping them outside Composio's custody. Add an architecture guide
that explains those boundaries and distinguishes hosted custody from
private deployments and customer-managed keys.
## Changes
- Add `/docs/security/token-custody` with an OAuth/execution diagram,
Python and TypeScript Proxy Execute examples, default API redaction,
execution paths, deployment options, and authorization and retention
responsibilities.
- Replace the connected-account credential-printing examples with
guidance on redacted authentication state and server-side execution.
- Add the guide to security navigation and link it from the security and
authentication pages.
## Type of change
- [x] Documentation
## How Has This Been Tested?
From `docs/`:
- `bun run build`: passed, including TypeScript checking and static page
generation. An existing Turbopack file-tracing warning remains.
- `bun test tests/static/content.test.ts tests/static/navigation.test.ts
tests/static/dashboard-links.test.ts
tests/static/product-navigation.test.ts`: 24 passed.
- `bun run lint:links`: no errors.
- `bun run lint`: passed with warnings in unchanged files.
Also compiled all changed MDX files, validated the page route and
Mermaid diagram, and verified the SDK examples against their method
signatures and parsed the Python example for syntax errors. No live
provider request was made.
## Checklist
- [x] Linters and focused tests passed locally.
- [x] Documentation updated.
- [x] Existing documentation checks cover this content-only change; no
new tests needed.
- [x] No changeset needed because no published package changes.
## Additional context
Addresses Gauge action `cmtvykziy00300jxgg1i6j149`. Inspected details,
insights, logs, and diffs for both evidence runs:
- [Arcade
comparison](https://agents.withgauge.com/composio-aclx/runs/cmtvo4yfd01gk0iuq0rdco72u):
inferred an unmasking path from `account.state.val` without evidence
that the API returned usable tokens.
- [Nango
comparison](https://agents.withgauge.com/composio-aclx/runs/cmtvo51xm01i20iuq6s4emlq3):
treated application credential isolation and ownership of the token
store as the same requirement.
Deployment claims link to current Composio sources, including the
published customer-managed key architecture. This PR documents behavior;
it does not change credential handling.
## Summary
`composio --version`: 288ms to 199ms. Peak RSS: 97.8MB to 77.3MB.
Executable: 85.9MB to 79.7MB. Every command benefits.
A compiled Bun binary parses its whole embedded bundle before running
any JS. #4468 stopped the TypeScript compiler and the tokenizer rank
table from being evaluated at startup, but they were still parsed every
time. The compiler was 44% of the executable's JavaScript, the o200k
table another 28%. Both now ship as files next to the executable and
load on demand.
Fourth PR in the stack. Stacked on #4468; review #4463, #4464 and #4468
first. #4475 builds on this one.
Bun 1.4.1+4661e494f, linux-x64, best of 15, telemetry disabled, both
binaries built in the same session:
| | before (#4468) | after |
|---|---|---|
| `composio --version` | 288ms | 199ms |
| `composio tools execute --help` | 287ms | 202ms |
| peak RSS | 97.8MB | 77.3MB |
| executable | 85.9MB | 79.7MB |
| executable JS, minified | 8.3MB | 2.1MB |
Across the whole stack, from `next`: `--version` 612ms to 184ms, peak
RSS 167MB to 78MB, executable 95.8MB to 79.7MB.
`composio execute` end to end, against the live backend with a logged-in
CLI, best of 7 for the small response and best of 5 for the large one.
Tool: `HACKERNEWS_GET_ITEM_WITH_ID` (no connected account needed) and
`HACKERNEWS_GET_LATEST_POSTS`. "Tail" is the time from the
`execute.tool_call.end` perf event to process exit.
| | `next` | #4468 | this PR |
|---|---|---|---|
| 1.6KB response, wall | 2431ms | 1857ms | 1761ms |
| 1.6KB response, tail | 294ms | 12ms | 11ms |
| 35KB response, wall | 2665ms | 2210ms | 2165ms |
| 35KB response, tail | 322ms | 329ms | 353ms |
The stack removes ~670ms from a small execute: ~430ms of startup and
~280ms of tokenizer construction that no longer happens. The large
response keeps its ~330ms tail because past 10KB the tokenizer is still
built; this PR adds ~20ms there for the on-demand parse of the encoder
file. The remaining ~1.7s is network the stack does not touch: DNS and
TLS to the backend, the preflight round trips before `tool_call.start`,
and the session create plus execute pair. Wall times move by ±150ms
between runs because of that; the tail column is the stable one.
## Changes
1. `generation-runtime.mjs` carries `src/generation/*`, the `composio
run` source rewrites, `typescript`, `@composio/ts-builders` and
`openapi-typescript`. `generate ts`, `generate py` and `run` load it
with the new `loadInstalledCompanionModule`. From a source checkout the
loader resolves the `.ts` file next to `run-companion-modules.ts`, so
tests and `bun run src/bin.ts` need no build step. The specifier is
computed at runtime on purpose; a literal `import('./x')` gets folded
back into the executable. Before importing, a packaged install runs the
self-repair download only if that companion's own files (its wrapper and
what the wrapper imports) are missing, so a different missing file
cannot block it. The repair has to come first, because Bun keeps a
failed or already-loaded import in its module registry. A file that
fails to import, or lacks one of the exports its caller names, is a
typed `RunCompanionRepairError` asking to reinstall, not a crash. Both
companions are also tsdown entries, so the `dist/` build resolves them.
2. `execute-output-encoder-runtime.mjs` carries `js-tiktoken/lite` plus
the rank table. `execute` loads it only past the 10KB byte gate from
#4463, and never for executes started by `composio run`. If it cannot be
loaded, even after the self-repair download, `execute` estimates the
token count from the byte length (about four bytes per token) instead of
failing a tool call that already succeeded. The estimate can undercount,
so such a response is always stored as a file rather than printed
inline.
3. Both join `RUN_COMPANION_MODULE_BASENAMES`, the mechanism `composio
run` already uses for its helpers, so build, release packaging, install
and upgrade verification, and the self-repair download pick them up
unchanged. The three hand-maintained uninstall lists and the upgrade E2E
fixture gain the two file names.
4. A companion bundles its own copy of `effect`, and a fiber cannot run
primitives from another copy. So nothing Effect-shaped crosses the
boundary. The generation companion exposes plain promises and returns
failures as values. `src/generation/errors.ts` rebuilds them as the
CLI's own error classes with fields and stack intact.
5. `src/constants.ts` imported `constants` from `@composio/core`'s root
entry for two strings and two URLs, which evaluated the whole SDK at
startup (~25ms, mostly zod schemas). The values are inlined and a test
pins them to core's. `tool-file-uploads.ts` imports its three core
helpers on the upload path instead of at module scope.
6. Build guard. After building the companions, the build bundles
`src/bin.ts` once more with the release build's `DEBUG_OVERRIDE_*` env
inlining, defines, `NODE_ENV=production` and syntax minification
(whitespace is kept, so the per-module path comments it reads survive),
and fails if the executable's graph reaches `typescript`, `js-tiktoken`,
`src/generation/*` or a companion entry. Checked that it fires on a
stray static import. `@composio/core`'s root entry is not on the list:
it is still bundled behind the file-upload path's dynamic import (see
Additional context), so the guard cannot exclude it.
`test/src/commands/startup-imports.test.ts` forbids the same modules
when the command tree loads from source.
What changes for users:
- A damaged install (companion file missing) now affects `generate` the
way it already affected `run`: self-repair from the release archive,
then an error. A large `execute` also attempts the repair, and if that
fails it stores the response with a byte-based token estimate rather
than failing. Responses under 10KB never touch the encoder. `--version`
and everything else are unaffected.
- `composio upgrade` from a binary older than this PR copies only the
companion files that binary knows about. The first `generate`, `run` or
large `execute` on the new version then restores the two new files
through the self-repair download.
- `execute` responses over 10KB pay ~20ms more after
`execute.tool_call.end` (351 to 374ms), the on-demand parse of the 2.2MB
encoder file. Under 10KB, unchanged.
- Errors from generation are rebuilt instances. Same class, tag, fields,
message and stack; different object identity.
Generated output is byte-identical to #4468 for `generate ts`, `generate
ts --transpiled` and `generate py`. The 11-invocation help/error diff
from #4468 is identical.
Found on the way: `assertBundledRuntimeFiles` blanked string literals to
same-length runs of spaces, and the import patterns' `^\s*` then
backtracked quadratically over the compiler's embedded lib strings. The
build hung for over ten minutes. String bodies are dropped now. The
check has also never matched a specifier, since the specifiers it looks
for are the strings it removes. Left as is, because a corrected version
flags false positives in `run-subagent-output-mcp`.
## Type of change
- [ ] Bug fix
- [ ] New feature
- [x] Refactor/Chore
- [ ] Documentation
- [ ] Breaking change
## How Has This Been Tested?
Bun 1.4.1+4661e494f, Node 24.20.0, pnpm 11.8.0, linux-x64.
1. `cd ts/packages/cli && pnpm run typecheck && pnpm run
validate:boundaries && pnpm run validate:skills`
2. `pnpm exec vitest run`: 129 files, 1335 passed, 1 skipped. New tests
cover the mirrored constants, error rehydration and outcome lifting, and
the loader resolving both companions from source.
3. `pnpm build:binary`, then against `dist/composio`: `generate ts`,
`generate ts --transpiled` and `generate py` diffed against #4468's
binary, `run` with a trailing expression, `execute` with 1.6KB and 35KB
responses, and the damaged-install cases with files deleted from
`dist/`.
4. Docker E2E on this branch: `upgrade` 2 pass, `run` 8 pass, `version`
9 pass, `install` 7 pass on bash and 5 pass on zsh. The install runs
used a fixture built the way CI builds it (`build:binary:cross`,
`build:binary:package`, `build:binary:checksums`), which also confirms
the release zip carries both new files.
5. `bun run test/release-workflow.test.ts` at the repo root, for the
synced uninstall lists.
6. Follow-up commit (encoder fallback, typed load failure, graph-check
and tsdown fixes): `pnpm run typecheck`, `validate:boundaries` and
oxlint pass. The execute, companion-loader, constants,
generation-runtime, `run` and `generate` suites pass (177 passed, 1
skipped), including new tests for the estimate when the encoder cannot
load and for the typed load failure. The fallback test fails without the
fix. `pnpm build` emits both companions, and `pnpm build:binary` passes
the graph check.
7. Review follow-ups (companion repair scoped to the requested module
and run before the import, required-export check, stored output when the
token count is an estimate, graph check using the release build inputs,
startup-imports list): `pnpm run typecheck`, prettier and oxlint pass.
Full `pnpm exec vitest run` on the CLI package: 1340 passed, 1 skipped,
1 failure in `analytics.dispatch.test.ts`, which this PR does not touch
and which fails 1 run in 3 on its own. After the last loader change, the
companion-loader, execute, `run`, `generate`, generation-runtime,
startup-imports and upgrade suites pass (198 passed, 1 skipped). The new
fallback test for a response whose estimate is under the threshold fails
without its fix. `bun run ./scripts/build-companion-modules.ts` passes
the updated graph check.
## Screenshots (if applicable)
Not applicable.
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [x] I updated documentation as needed
- [x] I added tests or explain why not applicable
- [ ] I added a changeset if this change affects published packages
`@composio/cli` is private, so no changeset. Docs: the uninstall lists
and the code generation section of `ts/packages/cli/AGENTS.md`.
## Additional context
`openai` and `pusher-js` (~0.5MB minified) are still in the executable.
Only `@composio/core`'s root entry reaches them, and the two upload
guards have no lighter subpath export. A
`@composio/core/utils/file-upload-guard` entry would remove them; that
is a core package change.
Companion files carry no version stamp. The loader checks that a
companion has every export its caller uses, so a file from another
release missing one fails with a reinstall error. A file from another
version with the same exports still loads as is; checking `APP_VERSION`
after import would catch that, and it is not done here.
The remaining ~95ms of module evaluation is a long tail of eager Schema
and command definitions across `src/commands`, `src/services`, `effect`
and `src/models`, not one dependency.
`composio execute` stored a response in a session file once js-tiktoken
counted more than 10,000 o200k tokens. Nothing reads that count as a number,
and o200k is not the tokenizer of the model reading the output, so the exact
count only moved the cutoff while costing the 2.3MB rank table, ~330ms after
large tool calls, and a separately shipped companion file with its own repair
path.
Responses over 40,000 UTF-8 bytes are stored now. The summary keeps
`tokenCount`, now estimated at four bytes per token, and gains `sizeBytes`.
The execute-output-encoder-runtime companion, js-tiktoken, and their build,
packaging and test entries are removed. The uninstall lists keep the
companion's file name so installs that shipped it can still remove it.
Bun keeps a failed or already-loaded import in its module registry, so
importing a companion again after a repair can still see the missing
dependency or the old module. Repair now runs before the single import, and
only when the requested companion's wrapper or its import graph is missing,
so unrelated missing companions still cannot block it.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
Brendan O'Leary