`composio execute` stored a response in a session file once js-tiktoken
counted more than 10,000 o200k tokens. Nothing reads that count as a number,
and o200k is not the tokenizer of the model reading the output, so the exact
count only moved the cutoff while costing the 2.3MB rank table, ~330ms after
large tool calls, and a separately shipped companion file with its own repair
path.
Responses over 40,000 UTF-8 bytes are stored now. The summary keeps
`tokenCount`, now estimated at four bytes per token, and gains `sizeBytes`.
The execute-output-encoder-runtime companion, js-tiktoken, and their build,
packaging and test entries are removed. The uninstall lists keep the
companion's file name so installs that shipped it can still remove it.
- Import an in-process companion before checking the rest of the set, so a
missing unrelated companion cannot fail generate or run through an offline
repair. Repair runs only when the requested module fails to load.
- Check each companion's required exports, so a file left by another release
fails with a typed reinstall error instead of calling a missing export.
- When the tokenizer cannot load, store any response past the byte pre-filter.
The four-bytes-per-token estimate can undercount, so it no longer keeps a
response inline.
- Bundle the executable graph check with the release build's env inlining,
defines, NODE_ENV and syntax minification.
Claude-Session: https://claude.ai/code/session_01MTb47vexN35pLGsSZwBmrJ
A large execute measured its output after the tool call had already succeeded,
so a missing encoder companion whose repair download failed turned a successful
call into a failed command. Fall back to a byte-based token estimate instead.
Load companion modules with Effect.tryPromise so an unloadable file is a typed
RunCompanionRepairError rather than a defect, and add the two companions as
tsdown entries so the dist build can resolve them.
The executable graph check listed @composio/core's root entry with a pattern
that could never match Bun's relative module paths. The root entry is still
bundled behind the file-upload dynamic import, so drop that entry and correct
the constants comment.
`composio --version` goes from 288ms to 199ms, peak RSS from 97.8MB to
77.3MB, and the executable from 85.9MB to 79.7MB. Every command benefits.
A compiled Bun binary parses its whole embedded bundle before the first
line of JavaScript runs, and #4468 had already made sure the TypeScript
compiler and the tokenizer rank table were never *evaluated* unless
`generate`, `run`, or a large `execute` response needed them. They were
still *parsed* on every start: the compiler alone was 44% of the
executable's JavaScript and the o200k rank table another 28%, so
`--version` spent ~75ms reading code it could never call.
Both now ship as companion modules next to the executable, through the
mechanism `composio run` already uses for its own runtime helpers:
- `generation-runtime.mjs` carries `src/generation/*`, the `composio run`
source rewrites, `typescript`, `@composio/ts-builders` and
`openapi-typescript`. `generate ts`, `generate py` and `run` load it
with `loadInstalledCompanionModule`; from a source checkout the loader
resolves the `.ts` next to `run-companion-modules.ts` instead, so tests
and `bun run src/bin.ts` need no build step.
- `execute-output-encoder-runtime.mjs` carries `js-tiktoken/lite` and the
rank table. `execute` loads it only once a response exceeds the 10KB
byte pre-filter.
A companion bundles its own copy of `effect`, and a fiber cannot run
primitives built by another copy of the runtime, so nothing Effect-shaped
crosses the boundary: the generation companion exposes plain functions
and promises, runs its pipelines on its own runtime, and returns failures
as values that `src/generation/errors.ts` rebuilds as the CLI's own error
classes, stack included. Generated output is byte-identical to #4468 for
`generate ts`, `generate ts --transpiled` and `generate py`.
Both modules join `RUN_COMPANION_MODULE_BASENAMES`, so the build, release
packaging, install verification, `upgrade` and the self-repair download
pick them up unchanged. The three hand-maintained uninstall lists and the
upgrade E2E fixture gain the two file names.
Two smaller startup costs go with it:
- `src/constants.ts` imported `constants` from `@composio/core`'s root
entry for two strings and two URLs, which evaluated the whole SDK at
startup (~25ms of module-scope work, mostly zod schemas). The four
values are spelled out and pinned to core's by a test.
- `tool-file-uploads.ts` imported three core helpers at module scope that
only a file upload reaches; they are imported on that path now.
The binary build gains a guard: after bundling the companions it bundles
`src/bin.ts` once more unminified and fails if the executable's graph
reaches `typescript`, `js-tiktoken`, core's root entry, `src/generation/*`
or a companion entry. Without it a stray static import would put the
compiler back into the executable with nothing to notice.
Building also surfaced that `assertBundledRuntimeFiles` blanked string
literals to same-length runs of spaces, which made the import patterns'
`^\s*` backtrack quadratically across the compiler's multi-megabyte
embedded lib strings and stalled the build for over ten minutes. String
bodies are dropped now. (The check itself has never matched a specifier,
since the specifiers it looks for are the string literals it removes;
that is left as it was.)
Measured on the pinned toolchain, Bun 1.4.1+4661e494f, linux-x64, best
of 15, telemetry disabled, both binaries built in the same session:
composio --version 288ms -> 199ms
tools execute --help 287ms -> 202ms
peak RSS 97.8MB -> 77.3MB
executable 85.9MB -> 79.7MB
executable JavaScript 8.3MB -> 2.1MB (minified)
The `execute` tail after `execute.tool_call.end` is unchanged for
responses under 10KB (~10ms) and ~20ms slower above it (351 -> 374ms),
which is the on-demand parse of the 2.2MB encoder companion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wx9gEjuiHux2weiHjdNcDs
This PR:
- routes Composio-prefixed runtime configuration through `APP_CONFIG`
and normalizes unprefixed host values through `HOST_CONFIG` in
`app-config.ts`
- removes `loadOptionalAppConfig`-style fallback loading so optional
strings, booleans, and CLI flag precedence resolve at the configuration
boundary
- removes every executable `process.env` reference from CLI source;
child commands inherit the ambient environment through
`@effect/platform` and receive explicit normalized overrides
- preserves arbitrary `CODEX_*`, `CLAUDE_*`, and `OPENCLAW_*` detection
through Effect config enumeration without exposing the raw environment
map
- propagates telemetry debug state through worker arguments instead of
mutating the process environment
- replaces removable filesystem, OS, subprocess, and error-handling
exceptions with Effect services and typed control flow
- documents supported and internal CLI environment variables, including
their `user_data.json` and `config.json` mappings
- reduces registered CLI lint boundaries from 45 to 4; the retained
entries are the detached-spawn, Node OS, synchronous preload filesystem,
and MD5 checksum implementations
- verifies the result with 1,201 passing CLI tests (1 skipped), strict
Oxlint, boundary validation, root typechecking across 19 packages, the
CLI build, and real `composio run` child-process checks
This PR:
- fixes
https://github.com/ComposioHQ/composio/actions/runs/30535159056/job/90850055191
by keeping checksum generation independent of unbuilt CLI runtime
packages
- extracts the shared Effect teardown into a dependency-light module
while preserving existing script imports
- adds an executable release regression that generates a checksum
manifest from a clean fixture
- verifies the fix with `pnpm test:release-workflow`, `pnpm
build:packages`, CLI typecheck, the full CLI test suite, and `pnpm
validate:changesets`
PostHog's own term for the public write-only phc_* credential is the
project API key; the old name read like a secret. Covers the env var,
the build-time bake define, the baked constant, and the workflow env.
The GitHub repo variable itself must be renamed manually.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback (KJ):
- Strip the added explanatory comments across dispatch.ts, constants.ts,
_shared.ts, build-binary.ts and the workflow; keep only pre-existing ones.
- Shorten the CHANGELOG entry to a single line.
- Revert the .gitignore addition.
Bugbot findings:
- Already-authenticated installs never stitched (events stayed anonymous after
upgrade, since only `login` persisted the id). Backfill now runs from any
command that already resolves a session: whoami, init, connected-accounts
link, plus the existing login paths.
- `composio login --agent` skipped the stitch; the agent branch now resolves
session info best-effort and links.
- `composio logout` left `apollo_user_id` behind, so later events kept
attributing to the previous person. Added clearApolloIdentityForAnalytics.
- A failed alias was never retried, because any persisted `apollo_user_id`
counted as "already aliased" — including when nothing was ever enqueued (no
project key / telemetry off). Track `aliased_apollo_user_id` separately and
only record it once the enqueue actually happened.
Tests: alias retry after a skipped enqueue, and logout reverting to install_id.
Full suite 956 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hygiene pass on the direct-to-PostHog telemetry: condense verbose block
comments in dispatch.ts/constants.ts.
Source the public write-only PostHog project key from a build-time
COMPOSIO_POSTHOG_KEY env var, baked into the binary via `bun build --define`
(scripts/_shared.ts posthogBakeArgs), instead of a committed source literal.
This mirrors the dashboard's env-driven NEXT_PUBLIC_POSTHOG_TOKEN pattern and,
for this OSS repo, keeps the key out of public git history and out of fork/local
builds (they compile a no-op binary). `--define` is used rather than a second
`--env` because Bun honors only one `--env` and it would clobber the existing
DEBUG_OVERRIDE_* inline. CI passes the key as an Actions variable on the
cross-compile step; empty -> telemetry no-ops.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- Add experimental CLI local tools behind `local_tools` (beta-on,
stable-off unless `composio config experimental local_tools on`).
- Add reusable `@composio/cli-local-tools` package: declarations,
metadata, readiness, command/MCP/native/FFI runtimes, bundled-binary
resolution, and Tool Router search/execute/schema integration.
- Add first-class local toolkits for:
- `BEEPER_IMESSAGE` backed by `imessage-cli` from the ComposioHQ
`platform-imessage` fork.
- `CHROME_DEVTOOLS` backed by the official `chrome-devtools-mcp@0.24.0`
npm/npx package.
- `PEEKABOO` backed by upstream Peekaboo CLI.
- Remove generated native executables from git: Beeper and Peekaboo
binaries are now ignored, pinned via git submodules, and generated
during CLI binary release jobs.
- Update CLI binary workflow so Linux artifacts skip native sidecars,
while macOS artifacts run on macOS runners and build only the matching
`darwin-x64` or `darwin-arm64` Beeper/Peekaboo sidecars before
packaging.
## Native binary policy
- No generated `imessage-cli` or `peekaboo` binary blobs are committed
in this PR history.
- Beeper source is pinned as
`ts/packages/cli-local-tools/vendor/platform-imessage` at
`364445a1b3089ad9fe293d5951efe160c5677c42` from
`https://github.com/ComposioHQ/platform-imessage`.
- Peekaboo source is pinned as
`ts/packages/cli-local-tools/vendor/peekaboo` at
`31e66e8d02656141d18f60bf3b46b24c2b9bc785` from
`https://github.com/steipete/Peekaboo`.
- Build commands:
- `pnpm --filter @composio/cli-local-tools build:local-tool-binaries --
--target darwin-arm64`
- `pnpm --filter @composio/cli-local-tools build:local-tool-binaries --
--target darwin-x64`
- Chrome DevTools is an npm/npx integration and is pinned as a dev
dependency (`chrome-devtools-mcp@0.24.0`) instead of using native
sidecars.
## Validation
- `pnpm --filter @composio/cli-local-tools build:local-tool-binaries --
--target darwin-arm64`
- `pnpm --filter @composio/cli-local-tools build:peekaboo -- --target
darwin-x64`
- `pnpm --filter @composio/cli-local-tools build:beeper-imessage --
--target darwin-x64`
- `pnpm --filter @composio/cli-local-tools typecheck:tsc`
- `pnpm --filter @composio/cli-local-tools test`
- `COMPOSIO_REAL_LOCAL_TOOLS_TESTS=1 pnpm --filter
@composio/cli-local-tools test -- src/toolkits/beeper-imessage.test.ts
src/toolkits/peekaboo.test.ts`
- `pnpm --filter @composio/cli typecheck:tsc`
- `pnpm --filter @composio/cli build`
- `COMPOSIO_CACHE_DIR=<tmp-config-with-local_tools-enabled> bun
ts/packages/cli/dist/bin.mjs local-tools doctor --json`
- `pnpm turbo build`
- Verified rewritten branch has no >1MB objects relative to
`origin/next` and no native executable paths in the PR diff.
## Notes / risks
- Beeper iMessage and Peekaboo are macOS-only and depend on local
permissions.
- Peekaboo requires macOS 15+; GUI tools require Screen Recording,
Accessibility, and sometimes Automation.
- Beeper send/reaction behavior can still depend on upstream Messages
automation reliability.
- Chrome DevTools can inspect/modify browser contents; avoid sensitive
browser sessions.
## Summary
- bundle the shipped `composio run` companion services with direct
`Bun.build(..., target: "bun")` output instead of reusing the shared
`dist` graph
- rebuild the Claude ACP support files as Bun-bundled runtime assets and
invalidate the cached adapter layout so stale raw files are not reused
- add runtime validation for shipped support files, wire the MCP helper
through the workspace `@composio/json-schema-to-zod` package, and cover
the static-import expectation in tests
## Why
Standalone binaries were still relying on external runtime dependencies
from companion support files. On a clean machine this surfaced as
failures like `Cannot find package 'decompress'` from the generated
companion modules. The fix is to make every shipped support file
self-contained for Bun and package exactly that tree into the binary
archives.
## Validation
- `pnpm run typecheck`
- `pnpm exec vitest run
test/src/services/run-subagent-output-mcp.test.ts
test/src/services/run-subagent-acp.test.ts`
- `pnpm run build:binary:all`
- `pnpm run build:binary:package`
- unpacked `dist/binaries/composio-darwin-aarch64.zip` and verified
`experimental_subAgent(..., { target: "codex", schema: ... })`
- unpacked `dist/binaries/composio-darwin-aarch64.zip` and verified
`experimental_subAgent(..., { target: "claude", schema: ... })`
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- replace runtime MCP SDK path resolution with static imports in the
structured-output helper
- ensure the helper works when shipped next to the standalone CLI binary
without repo-local node_modules
## Verification
- pnpm --filter @composio/cli run typecheck:src
- pnpm --filter @composio/cli test
test/src/services/run-subagent-acp.test.ts
- built and executed the bundled helper from a temp directory outside
the repo
- Only test binaries when target is `bun-linux-x64` since ARM64 binaries
can't execute on the x64 CI runner.
- Extract duplicated `teardown` function from 8 scripts into
`scripts/_shared.ts`.
- Fix `build-binary.ts` silent exit on failure (Effect.logInfo →
Effect.fail), matching the fix already applied to build-binary-cross.
jkomyno