This PR:
- wraps `pysher.Pusher` in `_ComposioPusher`, whose channel-auth POST
carries a `(5, 15)` connect/read timeout and raises
`TriggerSubscriptionAuthError` (a `TriggerSubscriptionError`) on a
transport failure, a non-200, or a response without an `auth` token —
pysher 1.0.8 sent it with no timeout and turned a non-200 into a bare
`AssertionError` on the websocket thread, on every (re)subscribe
- keeps that POST a plain `requests.post(..., timeout=...)` rather than
routing it through `safe_request`: the endpoint is built from the
configured Composio API base URL, a fixed trusted host, not a value from
a response, and the SSRF guard would refuse a local dev base URL
- validates `pusher_cluster` against `^[a-z0-9-]+$` (non-empty, at most
64 chars) before pysher formats it into `ws-{cluster}.pusher.com`,
raising `InvalidPusherClusterError` that names the shape violation
without echoing the value
- replaces the `unittest.mock.MagicMock` stand-in for pysher's
connection logger with a dedicated `logging.Logger` (`NullHandler`,
`propagate=False`, disabled), so `unittest` leaves the runtime import
graph while raw frames stay out of user logs; a test asserts the module
source no longer mentions `unittest`
- strips `Authorization`, `Proxy-Authorization`, and `Cookie` from the
next hop when `ssrfSafeFetch` or `safe_request` follows a redirect to a
different origin; same-origin hops keep them. Manual redirect following
bypasses both `fetch`'s cross-origin rule and `requests`'
`rebuild_auth`, so neither guard applied it before — the gap #4387 left
out
- `@composio/slim` has no mirrored source (its build copies
`core/dist`), so the changeset covers `@composio/core` and
`@composio/slim` as patches
Verified with `pytest tests/test_triggers.py tests/test_url_safety.py
tests/test_path_join_guardrail.py` (192 passed), `ruff check` / `ruff
format --check` on the changed files, `mypy --config-file
config/mypy.ini` on the three changed modules with the noxfile's stub
pins (no issues), `vitest run test/utils/ssrfGuard.test.ts` in
`@composio/core` (42 passed), `pnpm typecheck` at the root (14 tasks
successful), and `oxlint` + `prettier --check` on the changed TypeScript
files.
https://claude.ai/code/session_016ZuBv7JhVdSYTLYcTy2VJr
This PR:
- builds on top of https://github.com/ComposioHQ/composio/pull/4271,
whose commit it carries unchanged
- applies the Fetch standard's redirect method/body rules in **both**
SSRF guards via `_redirect_rewrite` / `redirectRewrite`: a `303` retries
as a bodiless request, a `301`/`302` does the same for a `POST`, and
`307`/`308` replay both
- narrows `ssrfSafeFetch` to the five statuses the Fetch standard calls
a redirect, so a `304` or `305` carrying a `Location` is returned to the
caller instead of followed — Python already used
`_REDIRECT_STATUS_CODES`
- drops `params` after the first hop in `safe_request`, since `Location`
carries the query for the target it names and re-appending handed a
query-string credential to a target that never asked for one
- purges the union of the Fetch `request-body-header` set and the two
`requests` also drops, identically on both sides
- blocks the IPv6 transition ranges the TypeScript CIDR list missed —
6to4 `2002::/16`, Teredo and the rest of `2001::/23`, local-use NAT64
`64:ff9b:1::/48`, `100::/64`, `2001:db8::/32`, site-local `fec0::/10` —
and the IPv4/IPv6 multicast and `192.88.99.0/24` ranges Python's
`is_global` missed
## Context
Both guards follow redirects by hand so every hop is revalidated against
the address blocklist. That also means neither inherits the method and
body rewriting `fetch` and `requests` would have done, so an upload
answered with a `303` was replayed — payload and all — at a result URL
that expects a GET.
https://github.com/ComposioHQ/composio/pull/4271 landed that rule in
Python only, which left the two SDKs disagreeing on the same wire
behavior. Reviewing for that divergence surfaced the redirect-status
set, the `params` replay, and the address-blocklist gaps above.
`2002:7f00:1::` is 6to4 for `127.0.0.1`, and it passed the TypeScript
guard as a public address.
Verified with `pytest python/tests/test_url_safety.py` (56 passed) and
`vitest run` in `@composio/core` (54 files, 1280 passed), plus `ruff`,
`tsc --noEmit`, `oxlint` and `prettier`. Fail-before confirmed: 10 of
the new TypeScript cases and 5 of the new Python cases fail against the
unmodified guards.
Two known gaps are deliberately left out, each deserving its own change:
neither guard strips `Authorization`/`Cookie` on a cross-origin
redirect, and a non-seekable Python body is re-sent exhausted on a `307`
where TypeScript throws a bare `TypeError` on a consumed
`ReadableStream`.
https://claude.ai/code/session_01SB3ZJdvoqBcRrWb2toWVrX
---------
Co-authored-by: ump45nose <52391318+ump45nose@users.noreply.github.com>
## Summary
- preserve boolean, empty, null, type-array, enum, const, and
scalar-constraint semantics across every Python conversion entry point
- intersect Zod enum and const values with declared types and
constraints, including compound JSON values
- default unversioned exact validation to Draft 7 and apply inclusive
and numeric exclusive bounds independently
- run one byte-identical corpus through Python, Zod, and Effect so
accepted and rejected inputs stay aligned
- keep exact JSON Schema acceptance separate from Pydantic default
materialization
## Review follow-up (second push)
- Python: exact Draft 7 acceptance now wraps all three entry points
(`json_schema_to_pydantic_type`, `json_schema_to_model`,
`pydantic_model_from_param_schema`), so they can no longer disagree
- Python: draft-4 boolean `exclusiveMinimum`/`exclusiveMaximum` (OpenAPI
3.0 style) no longer crash conversion — exact validation falls back to
Draft 4, and the library input is translated to the numeric spelling
- Python: ECMA-only regex patterns (look-around) no longer crash
pydantic model builds — Rust-incompatible patterns fall back to Python
`re`
- Python: type arrays with sibling constraints no longer raise
`TypeError` on valid input — constraints are scoped per member before
the library sees them
- Python: integral floats satisfy `integer`, `const` intersects `enum`,
annotation-only schemas accept anything, and an optional property with
an empty `enum` tolerates absence
- Zod: typeless scalar constraints apply per instance type, and string
lengths count Unicode code points instead of UTF-16 code units
- Effect: draft-4 boolean exclusive bounds are enforced instead of
silently ignored
- `multipleOf` uses decimal scaling in all three converters (declared
`divergesFromJsonSchema` on the corpus case)
- shared corpus grows by 13 primitive cases; new property-based tests
check acceptance against real Draft 7 oracles (hypothesis + `jsonschema`
in Python, fast-check + Ajv in TypeScript)
## Verification
- Python `make chk` (ruff + mypy)
- Python pytest: 1,572 passed (5 langchain-extra tests need an env this
sandbox lacks; unchanged from base)
- `@composio/json-schema-to-zod`: 187 passed incl. 300-run fast-check
property test; typecheck + build
- `@composio/json-schema-to-effect-schema`: 133 passed; typecheck
- `@composio/core` corpus ingress tests: 61 passed
- shared Python/TypeScript corpus files are byte-identical
(shasum-verified)
- `git diff --check`
## Contributor context
This replaces four narrow proposals after independent local
reproduction:
- [#4301](https://github.com/ComposioHQ/composio/pull/4301) ·
[Glen](https://app.tryglen.com/ComposioHQ/composio/pull/4301)
- [#4302](https://github.com/ComposioHQ/composio/pull/4302) ·
[Glen](https://app.tryglen.com/ComposioHQ/composio/pull/4302)
- [#4303](https://github.com/ComposioHQ/composio/pull/4303) ·
[Glen](https://app.tryglen.com/ComposioHQ/composio/pull/4303)
- [#4307](https://github.com/ComposioHQ/composio/pull/4307) ·
[Glen](https://app.tryglen.com/ComposioHQ/composio/pull/4307)
---------
Co-authored-by: simpleqt <89645338+simpleqt@users.noreply.github.com>
## Problem
`composio.utils.mimetypes.guess()` maps `.jpg` to `"image/jpg"`:
```python
".jpe": "image/jpeg",
".jpeg": "image/jpeg",
".jpg": "image/jpg", # <- non-standard
```
`image/jpg` is not a registered IANA media type. It is inconsistent
with:
- its own sibling extensions `.jpe`, `.jpeg`, and `.jfif-bnl`, which all
map to `image/jpeg`
- Python's standard library: `mimetypes.guess_type("x.jpg")` returns
`image/jpeg`
- IANA, which registers `image/jpeg`
`guess()` feeds the `Content-Type` used for presigned uploads
(`_upload_to_presigned_url`), so `.jpg` files are uploaded and stored
with a non-standard content type.
## Fix
Map `.jpg` to `image/jpeg`. One line.
The reverse `_MIME_TO_EXT` map keeps its `"image/jpg": "jpg"` entry on
purpose: the SDK should still accept `image/jpg` when a server sends it
in a `Content-Type` header (liberal in what it accepts, strict in what
it produces).
## Tests
Added coverage in `tests/test_mimetypes.py`: `.jpg`/`.jpeg` in the
known-extensions table plus a dedicated test documenting that `.jpg`
resolves to `image/jpeg`. Verified the new tests fail on the old value
and pass after the fix; the full `test_mimetypes.py` (35 tests) passes.
## Summary
- write CLI user data, pending login sessions, and agent identities
through one atomic `0600` helper
- repair `0644` credential files created by older CLI versions before
reading them
- redact credential-shaped structured values from CLI user-context
diagnostics
- redact secret-shaped text at both TypeScript and Python SDK log-output
boundaries, including Pusher `auth` responses and exception tracebacks
- preserve Python logger compatibility: errors remain untruncated,
disabled levels remain lazy, and malformed placeholders cannot expose
arguments
## Local reproduction
Under the normal `022` umask, `next` created a plaintext credential file
with mode `0644`. The pre-fix CLI user-context and TypeScript SDK debug
paths also emitted sentinel credentials. The private atomic writer
changes an existing `0644` target to `0600`, and the upgrade tests now
prove all three legacy credential files are tightened without changing
their contents.
## Verification
- CLI permission upgrade tests: 31 passed across user data, pending
login, and agent identity paths
- CLI source and test typechecks passed
- TypeScript core logging, redaction, and Pusher tests: 17 passed
- TypeScript core source and type-test typechecks passed
- Python logging regression tests: 5 passed
- focused Ruff, Prettier, Oxlint, and `git diff --check` passed
The focused CLI runner needed a temporary local alias for the
pre-existing missing `#ssrf_guard` mapping in the CLI Vitest config. The
alias was removed after verification and is not part of this PR.
## Contributor context
Credit to **Syed Anas Mohiuddin**, independent security researcher, for
reporting the legacy CLI credential-file permission issue.
Supersedes [#4300](https://github.com/ComposioHQ/composio/pull/4300) ·
[Glen review](https://app.tryglen.com/ComposioHQ/composio/pull/4300).
The implementation also covers agent credentials, retains atomic writes,
and applies redaction at the shared SDK logging boundary.
to_strict_json_schema reports the same constructs as the TypeScript
implementation (tuple items, boolean subschemas, malformed properties,
oneOf beside anyOf, conditional and dependency keywords), accepts a root
typed ["object"], leaves enum/const that already include null alone, and
omit_null_tool_arguments follows the composition branch matching the
argument's shape.
Co-authored-by: AseemPrasad <aseemprasad0520@gmail.com>
Claude-Session: https://claude.ai/code/session_01TDrxCHn2hg51HmxVstSUgs
to_strict_json_schema iterated a string-valued required character by
character, so a malformed required kept same-named properties non-nullable
while the TypeScript implementation treats it as absent and widens every
property. Both SDKs now agree.
Co-authored-by: AseemPrasad <aseemprasad0520@gmail.com>
Claude-Session: https://claude.ai/code/session_01TDrxCHn2hg51HmxVstSUgs
OpenAI structured outputs support local $ref pointers, including recursive
definitions, so toStrictJsonSchema no longer inlines them: $defs and
definitions are normalized where they are declared, an optional $ref
property is widened with an anyOf null branch, and external or dangling
$refs are reported as unsupported. omitNullToolArguments follows local
$refs when deciding whether a null is accepted. The Vercel provider still
inlines definitions before converting to Zod, which does not follow $ref.
Co-authored-by: AseemPrasad <aseemprasad0520@gmail.com>
Claude-Session: https://claude.ai/code/session_01TDrxCHn2hg51HmxVstSUgs
The strict flag now calls the base initializer (schema_config kwargs keep
working), emits strict on the wrapped tool, and mirrors the TypeScript
pipeline: optional parameters become required-nullable, unsupported
schemas downgrade the tool to non-strict, and null arguments the tool
schema rejects are dropped before execution.
Co-authored-by: AseemPrasad <aseemprasad0520@gmail.com>
Claude-Session: https://claude.ai/code/session_01TDrxCHn2hg51HmxVstSUgs
Fixes#4151.
## The problem
Both SDKs validated a URL by resolving its hostname, and then handed the
*hostname* to the HTTP client, which resolved it again when it opened
the socket. Two lookups, two answers: a short-TTL record under an
attacker's control answers publicly for the check and with
`169.254.169.254`, `127.0.0.1`, or RFC 1918 space for the connect. The
guard passes and the connection lands inside the network — classic
TOCTOU DNS rebinding, documented in both modules until now as a known
residual.
```mermaid
sequenceDiagram
participant SDK
participant DNS as Attacker DNS
participant Meta as 169.254.169.254
Note over SDK,Meta: before
SDK->>DNS: resolve evil.example.com (validate)
DNS-->>SDK: 93.184.216.34 — passes the guard
SDK->>DNS: resolve evil.example.com (connect)
DNS-->>SDK: 169.254.169.254
SDK->>Meta: GET /latest/meta-data/…
Meta-->>SDK: credentials
```
## The fix
Resolve once, validate every answer, then connect to the address that
was validated. There is no second lookup left to rebind.
- **Python** — `safe_get` / `safe_request` mount a transport adapter
that swaps the connect target for the duration of the socket connect
only. The `Host` header and TLS SNI keep the hostname, so certificate
verification is unchanged; rewriting `conn._dns_host` for the whole
connection would have sent `Host: <ip>` and offered the IP as SNI,
failing against every real origin. Every fetch call site now goes
through those two helpers, so no `requests.get` sits next to a bare
check any more:
- `_files.py::_fetch_file_from_url`,
`_files.py::FileDownloadable.download`
- `tool_router_session_files.py::_fetch_url_bytes`
- `safe_request`, per redirect hop
- **TypeScript** — `assertSafeFetchTarget` returns the validated address
and `ssrfSafeFetch` hands `fetch` a dispatcher pinned to it, re-pinned
per redirect hop. The dispatcher goes to the runtime's own `fetch`, so
callers that stub `globalThis.fetch` keep working. The pinned `lookup`
answers both shapes Node calls it with — the address *list* it uses for
Happy Eyeballs, and the single `(address, family)` it uses when
`autoSelectFamily` is off — since answering in the wrong shape is
rejected as an invalid address.
- A fail-closed peer assertion runs on the Python side before a byte is
written to the socket — redundant while pinning works, and a tripwire if
a urllib3 upgrade ever breaks it.
- `workerd` is unchanged: it already fails closed for user-supplied
URLs.
Redirect *validation* already existed in both SDKs (`safe_request` /
`ssrfSafeFetch`); what was missing was re-pinning each hop.
## Tests
The existing suites could not express this bug: they mock both the
resolver and the HTTP client, so check and use are the same mock. The
new tests use real sockets.
- `python/tests/test_url_safety_pinning.py` — two loopback servers and a
resolver that answers the first lookup with one endpoint and every later
one with another, which is what a short-TTL rebinding record does.
Asserts the rebound endpoint receives **zero** connections, and that
`Host` still carries the hostname. Both tests fail on `next` and pass
here.
- `ts/packages/core/test/utils/pinnedDispatcher.node.test.ts` — a real
server plus a hostname under `.invalid`, which RFC 2606 guarantees never
resolves. A request that arrives proves the connect used the pinned
address and never consulted DNS. The third case shows the contrast:
unpinned, the same fetch cannot resolve at all.
- `ssrfGuard.test.ts` gains assertions that each hop is pinned to that
hop's own validated address.
- `pinnedDispatcher.node.test.ts` also pins with
`setDefaultAutoSelectFamily(false)`, which is the branch Node takes for
the single-address callback.
## Notes
- Supersedes #4157, which diagnosed this correctly. Its post-response
peer check turned out not to hold: with an HTTP/1.0 or `Connection:
close` server, urllib3 detaches the socket (`conn.sock is None`) while
`r.content` still returns the full body, so the check fails open exactly
where exfiltration succeeds. That is why the assertion here runs at
connect time instead.
- The Python package now declares `urllib3>=2` directly. `url_safety`
imports it for `NameResolutionError`, which only exists from 2.0, and
the pinning adapter reaches into 2.x connection internals; `requests`
alone allows 1.x, where `import composio` would have failed outright.
- `@composio/core` gains an `undici` dependency, pinned to `^7`: undici
8 dispatchers are rejected by the `fetch` in every Node version this
package supports (22/24/25, verified). The real-socket test runs on the
full CI matrix, so a future incompatibility fails loudly instead of
silently un-pinning.
- `undici` is imported on first pinned request rather than at module
load: importing it installs a process-wide global dispatcher, which
would have handed the host application's own unrelated `fetch` calls
this package's undici merely because it imported `@composio/core`.
- Residuals, now documented in the modules:
- Requests routed through an environment proxy keep the pre-flight check
only. The proxy resolves the hostname itself and the SDK cannot see or
pin that resolution.
- A process that does perform a pinned fetch still ends up on this
package's `Agent` if nothing had claimed the global dispatcher slot yet.
undici defines that slot non-configurable, so it cannot be handed back —
assigning `undefined` leaves the runtime's own `fetch` asserting on a
missing dispatcher.
This PR:
- fixes#4153
- supersedes https://github.com/ComposioHQ/composio/pull/4154 (fork PR
authored against a pre-#4146 tree; its tests patch `requests.put`, which
`upload()` no longer calls after #4146, so they fail on merge)
- adds `parse_content_length()` in `composio.utils.url_safety`, shared
by both URL fetch helpers (`_files.py::_fetch_file_from_url` and
`tool_router_session_files.py::_fetch_url_bytes`), so a
malformed/negative remote `Content-Length` degrades to unknown size
under the streamed byte count instead of raising a raw `ValueError`
- converges the file and bytes upload paths onto
`_upload_to_presigned_url()`: one PUT that sends the `Content-Type` the
presign request was signed with and raises `ErrorUploadingFile` carrying
the HTTP status (a 403 no longer collapses into a path-only error)
- single-sources the presign wire shape via
`_request_presigned_upload()`; `from_path()` forwards the exact mimetype
it minted, so signed and sent content types cannot drift
- mirrors the TypeScript SDK: `uploadFileToS3` funnels path/URL/File
inputs through one uploader that always sends `Content-Type` and throws
on non-2xx, and `readResponseBodyWithLimit` trusts `Content-Length` only
as a hint
- adds tests against the `safe_request` seam
(`test_file_upload_robustness.py`, plus one malformed-header case for
`RemoteFile.buffer()`)
## Context
Both defects in #4153 were symptoms of two parallel upload paths
drifting: the bytes path sent `Content-Type` and raised with the status,
the file path did neither. Patching the symptom in place (as #4154
proposed) would have left three presigned PUT sites and two error
contracts in the tree; this PR deletes the drift vector instead. Full
suite green (1320 passed), `make chk` and `make snt` clean; Python-only
change, so no changeset per `AGENTS.md`.
---------
Co-authored-by: Mustaqeem66 <265153888+Mustaqeem66@users.noreply.github.com>
This PR:
- builds on top of https://github.com/ComposioHQ/composio/pull/4144
- routes every fetch whose URL comes from an API response through the
SSRF guards that already existed — ten sinks, five per SDK: the
tool-execution download, both S3 presigned uploads,
`RemoteFile.buffer()`/`blob()`, and the Tool Router session file upload
- adds `safe_request()` (Python), which follows redirects itself and
re-validates each hop, so a validated URL cannot 302 into private space
and an S3 307 region redirect still works
- makes `RemoteFile.buffer()` (Python) share `_fetch_url_bytes` with the
user-supplied-URL path instead of duplicating it — it previously read
`response.content` with no size cap, no redirect control, and no target
validation
- adds `ssrfSafeFetchWhereSupported` (TypeScript): the full guard on
Node, a plain `fetch` on workerd, so Tool Router session file transfers
keep working in edge runtimes rather than failing closed
- documents DNS rebinding as a known residual in both guards
- adds tests that assert the guard *runs* — blocked URL, sink never
called, nothing written — rather than that a transfer succeeds
## Context
`python/AGENTS.md` states the trust boundary: every field of an API
response is untrusted input, because the backend may be compromised or
the connection MITM'd. Both SDKs enforced that for URLs a *user* passes
in (`composio.utils.url_safety`, `ssrfGuard.node.ts`) and left the URLs
an API *response* supplies unguarded — backwards relative to the stated
model. A response naming an internal address turned the SDK into a
request proxy for it, and the fetched bytes were written to disk or
returned to the caller, typically into an LLM context.
`RemoteFile.buffer()` was the weakest of the ten: four lines above it,
`_fetch_from_url` validated its target, refused redirects, and streamed
against a 100 MiB cap; `buffer()` did none of the three. The only
difference between them was which side of the trust boundary the URL
arrived from.
Three decisions worth review:
- **The guard is unconditional.** Presigned URLs are public, so nothing
legitimate should resolve to private space. There is no escape hatch,
and no new configuration surface.
- **The tool-execution download stays uncapped.** It streams straight to
disk, so `_MAX_RESPONSE_SIZE` — a memory-exhaustion bound — does not
apply, and tool attachments legitimately exceed it.
`RemoteFile.buffer()` *is* capped, because it buffers in memory.
- **Uploads follow redirects with re-validation rather than refusing
them**, since S3 can answer a PUT with a 307 region redirect. Downloads
keep `allow_redirects=False`, matching the existing fetch paths.
Python now matches the TypeScript guard's per-hop re-validation, which
was the better of the two implementations.
Verified locally: `make chk` and `make tst` clean on the Python side;
`pnpm -C packages/core test` 1095 passed, typecheck and lint clean.
This PR:
- centralizes filesystem path construction for API-provided slugs and
filenames in `composio.utils.safe_path`
- rejects traversal, Windows-invalid names, invalid Unicode, and
overlong encoded filenames before creating directories or writing files
- normalizes trusted roots consistently and routes both Python download
paths through the shared helpers
- adds a fail-closed AST guard for new dynamic path construction,
including direct `Path(...)` calls
- isolates provider initialization from the real home directory and
makes the home-write guard report changes without deleting them
- removes the obsolete download filename wrapper
## Verification
- `pytest -q`: 1,248 passed, 47 skipped
- repository-configured Ruff checks and formatting passed for every
changed Python file
- targeted mypy checks passed for the changed helpers and tests
The dynamic-key reference walker descended into every nested dict, so a
$ref-shaped value stored under const, default, enum, or examples was
treated as a schema reference and raised while the tool was wrapped.
Restrict the walk to Draft 7 schema positions.
## Summary
Telemetry error text is redacted before it leaves the process, but the
key/value rule only matches when the separator follows the key name
directly:
```
\b(authorization|api[-_]?key|...|pwd)\b(\s*[:=]+\s*)(["']?)([^\s"',}&]+)\3
```
In JSON — and in a Python `dict` repr — the key's own closing quote sits
between the name and the colon, so `{"api_key": "..."}` never matches
and the value is sent verbatim. That is the shape error messages usually
carry: an API error envelope, a rejected request body echoed back, or an
f-string interpolating a config dict. Both SDKs share the pattern and
both are affected.
The fix moves the optional quote into the separator group
(`(["']?\s*[:=]+\s*)`). The key name, separator, and quoting are all
preserved in the output; only the value is replaced.
Measured with `redact_sensitive_text` / `redactSensitiveText` directly,
before and after:
| input | before | after |
|---|---|---|
| `api_key=sk-1` | redacted | redacted |
| `api_key: "sk-1"` | redacted | redacted |
| `x-api-key: sk-live-hdr` | redacted | redacted |
| `{"api_key": "sk-live-abc"}` | **leaks** | `{"api_key": "[REDACTED]"}`
|
| `{"api_key":"sk-live-abc"}` | **leaks** | redacted |
| `{"api_key" : "sk-live-abc"}` | **leaks** | redacted |
| `{"refresh_token":"rt-abc.def-123"}` | **leaks** | redacted |
| `{"x-api-key":"sk-hdr","user":"bob"}` | **leaks** | redacted,
`"user":"bob"` kept |
| `{'client_secret': 'cs-live-abc'}` | **leaks** | redacted |
| `the password field is required` | untouched | untouched |
| `no separator here "api_key" and nothing else` | untouched | untouched
|
The end-to-end path in Python is `composio/core/models/base.py:60-61`,
which passes `str(e)` and `traceback.format_exc()` through the redactor
into the telemetry `error` field. A tool failure whose message
interpolates the rejected request body — `Error executing tool: request
body was rejected: {"toolkit": "GMAIL", "arguments": {"api_key": "...",
...}}` — leaked the customer's key today.
## Changes
- `python/composio/utils/redaction.py`,
`ts/packages/core/src/telemetry/redact.ts`: allow a quote between the
key name and the separator.
- Tests on both sides for JSON, minified JSON, spaced-colon JSON, dict
repr, the realistic serialized-payload shape, key/quote preservation,
and no-false-positive cases.
- Changeset for `@composio/core`.
## Type of change
- [x] Bug fix
## How Has This Been Tested?
Node 24.13.0 / pnpm 11.8.0 / Python 3.10.20 (uv), Windows.
- `uv run pytest tests/test_redaction.py` — 13 passed. With
`redaction.py` reverted and the new tests kept, 9 of them fail; the 4
that still pass are the pre-existing test plus the three
no-false-positive controls.
- `npx vitest run test/telemetry/redact.test.ts` in `ts/packages/core` —
10 passed. With `redact.ts` reverted, the 3 new cases fail.
- Full Python suite: 913 passed, same 4 failures as on `next` unchanged
(3 are `ModuleNotFoundError: composio_langchain` from providers not
installed in my env; 1 is `test_empty_name_safe_fails_at_write_time`,
which expects `IsADirectoryError` but Windows raises `PermissionError`
when opening a directory). Collection 944 → 956, exactly the 12 tests
added.
- Full `@composio/core` vitest: 798 passed / 2 failed, byte-identical to
the unchanged baseline (794 passed / same 2 failed — both Windows
path-and-symlink cases). 11 test files fail to load in both runs because
I installed with `--ignore-scripts`; the repo's `preinstall` hook
doesn't run in my shell.
- `ruff format --check` and `ruff check` clean on the two Python files;
`prettier --write` applied to the two TypeScript files.
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [ ] I updated documentation as needed — no user-facing behavior change
to document
- [x] I added tests or explain why not applicable
- [x] I added a changeset if this change affects published packages
## Additional context
This is defence-in-depth, so it stays deliberately narrow: same denylist
of key names, same value character class, no new rules. Bare-prefix
secrets (`sk-...`, `ghp_...`) and PEM blocks are still not matched by
either SDK — worth a separate look, but widening the pattern is a
different risk trade-off than closing a shape the rule already intends
to cover.
`json_schema_to_model` and `json_schema_to_pydantic_type` now share one
compiled object policy, so provider-facing Pydantic models stop silently
discarding valid free-form arguments and stop ignoring `patternProperties` and
explicit `additionalProperties` controls. Accepted dynamic keys are written back
into `__pydantic_extra__`, which is what `LangchainProvider.wrap_tool` re-reads
with `getattr`.
Acceptance and rejection are asserted from the shared cross-SDK corpus, so the
Python entry points cannot diverge from the TypeScript converters.
### Summary
Preserves JSON Schema boolean-schema semantics throughout the Python
schema converter:
- an unsatisfiable `allOf` now propagates through nested combiners
instead of being dropped;
- impossible array items, object properties, and local definition
references remain rejecting;
- composed impossible schemas use a Pydantic type that rejects every
value, including JSON `null`;
- the existing `json_schema_to_pydantic_type(False) is None`
compatibility behavior remains unchanged.
### Problem
The original fix handled only a literal `false` directly inside one
`allOf`. Recursive filtering still interpreted the same `None` value as
both "discard this union branch" and "the whole schema is impossible."
That widened nested `allOf` schemas to `str`, removed impossible
properties/items/definitions, and allowed JSON `null` because Pydantic
treats a bare `None` annotation as `NoneType`.
### Fix
- Add a private unsatisfiable marker with Pydantic core- and JSON-schema
hooks.
- Drop that marker only from `anyOf`/`oneOf`; propagate it through
`allOf`.
- Preserve rejecting semantics for nested object properties, array
items, and local `$defs`/`definitions` references.
- Emit `{"not": {}}` from the rejecting Pydantic type.
- Correct the related empty-`allOf` and all-false-`anyOf` expectations.
### Testing
- `uv run pytest tests/test_schema_converter.py
tests/test_schema_parser.py -q` - 106 passed
- `uv run pytest -m schema -q` - 71 passed
- `make tst` - 928 passed
- `make chk` - Ruff and mypy passed across core, providers, tests, and
scripts
Regression coverage includes direct and nested `allOf`, JSON `null`,
empty arrays with impossible items, optional and required impossible
properties, local definition references, generated JSON Schema, and
shared model consumers.
### Note
#3876 adds coverage for the same function's `anyOf` handling but does
not change this `allOf` propagation path.
---------
Co-authored-by: jkomyno <alberto@composio.dev>
This PR:
- builds on top of https://github.com/ComposioHQ/composio/pull/3823
- carries forward the Requests canonicalization hardening proposed in
https://github.com/ComposioHQ/composio/pull/3810
- resolves and validates the prepared hostname that Requests will
actually connect to
- blocks backslash parser differentials and rejects malformed ports
before DNS resolution
- adds focused URL-safety coverage while preserving both existing upload
entry points
Validation:
- `uv run --frozen pytest tests/test_url_safety.py tests/test_files.py
tests/test_tool_router_session_files.py -q` (177 passed)
- `make chk`
`_type_to_parameter` in `python/composio/utils/openapi.py` assumed a
property's
`type` was a scalar string. JSON Schema Draft 2020-12 and OpenAPI 3.1
can express
a nullable field as a list of types instead, for example
`{"type": ["string", "null"]}`. Passing that list to the scalar
membership check
raised `TypeError: unhashable type: 'list'`, which could break tool
signature
generation in providers such as `google_adk`.
## Changes
- Handle list-valued `type` definitions as unions.
- Re-run every member through `_type_to_parameter` with a copy of the
full
property schema and only `type` replaced. This preserves sibling fields
such
as an array's `items`, so `{"type": ["array", "null"], "items": ...}`
resolves
to `Optional[List[...]]`.
- Preserve scalar error behavior for invalid member types: an unknown
type in a
list raises `InvalidSchemaError` instead of silently becoming `Any`.
- Keep the existing empty-list fallback to `Any`.
- Add regression coverage for nullable arrays and scalar/list
unknown-type
parity alongside the existing list-valued type cases.
## Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Refactor/Chore
- [ ] Documentation
- [ ] Breaking change
## Verification
From `python/`:
- `uv run pytest tests/test_openapi.py -q` — 18 passed.
- `make chk` — Ruff and mypy passed across the Python SDK, providers,
tests,
examples, and scripts.
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [ ] I updated documentation as needed (not applicable: internal
helper)
- [x] I added regression tests
- [ ] I added a changeset (not applicable: Python-only change)
---------
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: jkomyno <alberto@composio.dev>
Co-authored-by: jkomyno <alberto@composio.dev>
This PR:
- ports the TypeScript SDK's public-network guard to Python URL file
uploads
- rejects non-HTTP(S), private, loopback, link-local, reserved, and
mixed DNS targets before a request is made
- applies the guard to both `FileUploadable.from_url()` and Tool Router
session-file URL uploads
- redacts URL queries, Authorization credentials, and secret-like
key/value pairs before error telemetry leaves the process
- adds focused SSRF and redaction regressions alongside the existing
file-upload coverage
Validation:
- `uv run --frozen pytest tests/test_url_safety.py
tests/test_redaction.py tests/test_files.py -q`
- `uv run --frozen ruff check …`
- `uv run --frozen ruff format --check …`
This PR:
- builds on top of https://github.com/ComposioHQ/composio/pull/3754
- normalizes list-valued JSON Schema `type` fields before scalar type
lookups
- applies the handling to direct fields and `anyOf`/`oneOf` options
- adds regressions for nullable, single, mixed, unknown, and
combiner-nested type lists
- credits @anxkhn as a Git co-author
## Context
JSON Schema Draft 2020-12 and OpenAPI 3.1 allow `type` to be an array.
The previous parser passed that list to a dictionary lookup, raising
`TypeError: unhashable type: 'list'`. This replacement covers the direct
case from #3754 and the same shape nested in a combiner.
---------
Co-authored-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
## Summary
Fixes `json_schema_to_model` when a top-level JSON schema omits `title`
or sets it to `null`. This is the remaining direct-helper path;
anonymous nested objects from #2435 are already handled by
`schema_converter.py`.
- falls back to `GeneratedModel` only when the title is `None`,
preserving intentionally empty string titles
- adds regression coverage for missing, `null`, and empty titles, model
construction, and required-field validation
## Validation
- `ruff format --check python/composio/utils/shared.py
python/tests/test_schema_parser.py`
- `ruff check python/composio/utils/shared.py
python/tests/test_schema_parser.py`
- `PYTHONPATH="$PWD/python"
/Users/jkomyno/work/composio/composio2/.venv/bin/pytest -q
python/tests/test_schema_parser.py`
This is a follow-up to #2435; it intentionally does not close that
already-resolved issue.
Co-authored-by: jkomyno <alberto@composio.dev>
Co-authored-by: Alberto Schiabel <jkomyno@users.noreply.github.com>
## Summary
Toolkit version maps are keyed by **normalized (lowercase) slugs** on
the *write* side — env vars (`COMPOSIO_TOOLKIT_VERSION_*`) and
user-supplied dicts both get their keys lowercased when the map is
built. But the *lookup* read the map with the **raw slug**. A pin
configured under a different casing (e.g. `{ "GitHub": "20250101_00" }`
or `COMPOSIO_TOOLKIT_VERSION_GITHUB` looked up as `GitHub`) therefore
missed the map and silently fell back to `"latest"`, which
`Tools.execute` then rejects with a version-required error — discarding
the user's explicit pin.
Rather than patch only the read side, this **centralizes the
normalization rule** into a single helper per SDK and routes **both**
the map-building (write) and lookup (read) paths through it, so the two
sides can never drift apart again:
- **Python** — `normalize_toolkit_slug()` in
`python/composio/utils/toolkit_version.py`
- **TypeScript** — `normalizeToolkitSlug()` in
`ts/packages/core/src/utils/toolkitVersion.ts`, imported by
`getToolkitVersionsFromEnv` in `sdk.ts`
Behavior is now identical across the two SDKs. This **supersedes
#3759**, which patched only the Python read side and left TS diverging.
## Backend verification
Confirmed against the backend (staging, **1403 toolkits across 15
pages**): every toolkit `slug` is already lowercase (the mixed-case
`name` field is display-only). So the *primary* execute path — which
resolves via `tool.toolkit.slug` — was never broken in practice. This
change:
- **hardens** the direct-util and mixed-case-config paths (a user
passing `{ "GitHub": "…" }` now resolves as expected), and
- **removes a latent cross-SDK divergence**: the TS suite previously
asserted `getToolkitVersion` was *case-sensitive* (`versions.test.ts`),
locking in the bug for an input the production code path can't actually
produce. That test is flipped to assert case-insensitive resolution.
## Tests
- **Python** (`python/tests/test_toolkit_version.py`): case-insensitive
lookup via `get_toolkit_versions` round-trip, and against a raw
pre-normalized map. `18 passed`.
- **TypeScript** (`ts/packages/core/test/core/versions.test.ts`):
replaced the case-sensitivity assertion with case-insensitive resolution
+ a write→read round-trip of a mixed-case user pin. Full core suite `997
passed`.
- Typecheck (`tsc --noEmit`) clean; `ruff` clean.
## Notes
- Changeset added: `@composio/core` patch.
- No public API change; the fix is behavioral (case-insensitive) plus an
internal shared helper.
### Summary
`function_signature_from_jsonschema` in
`python/composio/utils/openapi.py` builds the `__signature__` for tools
wrapped by the `google_adk` provider
(`providers/google_adk/composio_google_adk/provider.py:66`). It already
falls back to `Any` for a top-level property with no `type`, but the
nested combiner helpers do not, so real combiner shapes crash at wrap
time:
- a `oneOf`/`anyOf` option with no `type` key -> `KeyError: 'type'`
- an empty `oneOf`/`anyOf` list -> `TypeError: Cannot take a Union of no
types`
- an empty `allOf` list -> `KeyError: 'type'` (merges to `{}`)
This extends the existing "default to Any" intent to nested options.
Follow-up to #3708, which fixed the same bug family in `shared.py` and
noted this `openapi.py` case for a separate PR.
Fixes #
### Changes
- `_type_to_parameter`: read `schema.get("type")` instead of
`schema["type"]`; when the type is missing, return `t.Any` (mirrors the
top-level fallback). An unknown *string* type still raises
`InvalidSchemaError`.
- `_handle_composite_type`: return `t.Any` for an empty option list
instead of building an empty `Union`. Empty `allOf` merges to `{}` and
resolves to `Any` via the above.
- New `python/tests/test_openapi.py` (11 tests): this builder had none.
Asserts the five former crashes now resolve to `Any`, and that existing
shapes are unchanged.
### Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Refactor/Chore
- [ ] Documentation
- [ ] Breaking change
### How Has This Been Tested?
From `python/`:
```
uv run pytest tests/test_openapi.py -q # 11 passed
make chk # ruff + mypy -> clean
```
To confirm the tests guard the fix, the five former crashes (typeless
`anyOf`/`oneOf` option, empty `anyOf`/`oneOf`/`allOf`) raise on the
unmodified file and now resolve to `Any`; regressions cover
`anyOf:[str,null]` -> `Optional[str]`, a 4-member union, enum, array,
object, and a typeless property.
### Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [ ] I updated documentation as needed (internal util)
- [x] I added tests (new test_openapi.py)
- [ ] I added a changeset (Python-only, not needed)
Co-authored-by: Alberto Schiabel <jkomyno@users.noreply.github.com>
## Summary
`get_signature_format_from_schema_params` in
`python/composio/utils/shared.py` builds the `__signature__` for the
tool wrappers used by the langchain, langgraph, llamaindex, and autogen
providers. Its `oneOf`/`anyOf` handling hardcoded a ladder for exactly
1, 2, or 3 union members and raised `ValueError("Invalid 'oneOf'
schema")` for any union with four or more options. It also indexed
`PYDANTIC_TYPE_TO_PYTHON_TYPE[ptype.get("type")]` directly, so any
combiner option missing a `type` key produced `None` and raised
`KeyError(None)`.
Both shapes are valid JSON Schema and appear in real tool input schemas,
so a tool with a 4-way union, or a combiner option without an explicit
`type`, makes those four providers raise at tool-wrap time. This
replaces the ladder with a single map + `functools.reduce` path that
mirrors the existing `_build_union_from_options` in
`schema_converter.py`, keeping the 1/2/3-member and nullable `[type,
null]` outputs identical to before.
Fixes #
## Changes
- `python/composio/utils/shared.py`: in
`get_signature_format_from_schema_params`, replace the 1/2/3-member
`oneOf`/`anyOf` ladder (which raised `ValueError` for 4+ members) with a
map-each-option-to-a-Python-type + `functools.reduce` Union build that
supports any member count. Unknown/missing option types map to
`typing.Any` instead of raising `KeyError`. Adds `from functools import
reduce` and removes three now-unnecessary `# type: ignore` comments.
- `python/tests/test_schema_parser.py`: add
`TestGetSignatureFormatFromSchemaParams` (8 cases) covering 4- and
5-member unions, an `anyOf` option missing `type`, an all-typeless
`anyOf`, and regressions for the previously supported single/2/3-member
and nullable `[type, null]` shapes.
## Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Refactor/Chore
- [ ] Documentation
- [ ] Breaking change
## How Has This Been Tested?
From `python/`:
```
uv run pytest tests/test_schema_parser.py::TestGetSignatureFormatFromSchemaParams -v # 8 passed
uv run pytest tests/test_schema_parser.py -q # 75 passed
make chk # ruff + mypy -> clean, no new ignores
```
To confirm the tests guard the fix, the new test class was also run
against the unmodified `shared.py`: 4 of the 8 fail there
(`ValueError`/`KeyError` on the 4+/typeless cases), and all 8 pass with
this change. The nullable `anyOf [type, null]` case still resolves to
`Union[str, Any, NoneType]` (its pre-existing behavior); collapsing
`null` into `Optional` lives in `schema_converter` and is intentionally
left out of this minimal fix, with the test documenting current
behavior.
## Screenshots (if applicable)
N/A
## Checklist
- [x] I have read the Code of Conduct and this PR adheres to it
- [x] I ran linters/tests locally and they passed
- [ ] I updated documentation as needed
- [x] I added tests or explain why not applicable
- [ ] I added a changeset if this change affects published packages
## Additional context
No linked issue: this is a self-identified, reproduced bug in the older
schema->signature path. The fix deliberately mirrors
`schema_converter._build_union_from_options` so the two schema paths
agree on N-ary union handling. The same bug family exists in
`python/composio/utils/openapi.py`
(`function_signature_from_jsonschema`, used by the google_adk provider)
but is kept out of this PR to stay single-purpose; happy to follow up on
it separately.
This PR:
- supersedes https://github.com/ComposioHQ/composio/pull/3625,
https://github.com/ComposioHQ/composio/pull/3629, and the Python
SDK/provider-side fix proposed by
https://github.com/ComposioHQ/composio/pull/3504
- adds `alias_tool_input_schema` / `restore_tool_arguments` for
provider-visible schemas and backend argument restoration
- keeps `substitute_reserved_python_keywords` /
`reinstate_reserved_python_keywords` as compatibility wrappers
- preserves the existing Python keyword alias style (`from` ->
`from_rs`), aliases invalid Python parameter names like `$top`, avoids
leading-underscore aliases so Pydantic-backed providers can build
models, and caps aliases at 64 characters for Anthropic-style tool
schema validators
- dereferences internal `$ref` / `$defs` before aliasing so referenced
object properties are exposed through the same safe provider-visible
names
- restores Gemini manual `handle_response` arguments with the per-tool
alias map before execution
- wraps Google ADK tools with aliased callable signatures and restores
original backend argument names
- wraps Anthropic and Claude Agent SDK tool schemas with the same shared
alias map and restores original backend argument names before execution
- fixes the package-local Claude Agent SDK provider tests so they run
without `pytest-asyncio` and keep using the provider-visible schema
conversion hook
- adds dependency-light core tests for helper behavior, `$ref` aliasing,
Pydantic model compatibility, Gemini manual response, Google ADK
wrapping, Anthropic wrapping, and Claude Agent SDK wrapping
Local validation:
- `uv run --project python pytest
python/tests/test_tool_schema_aliasing.py
python/tests/test_schema_parser.py python/tests/test_json_schema.py
python/tests/test_imports.py -q` -> 112 passed
- `PYTHONPATH=python uv run --project python --with-editable
./python/providers/claude_agent_sdk pytest
python/providers/claude_agent_sdk/tests/test_provider.py -q
--timeout=20` -> 16 passed
- `uv run --project python --with-editable ./python/providers/langchain
pytest --ignore-glob='python/tests/test_type_inference*.py'
python/tests/ -q` -> 742 passed, 31 skipped
- `uv run --project python --with-editable ./python --with-editable
./python/providers/langchain python <LangChain $top wrap smoke>` ->
wrapped args schema field `param_top`
- `uv run --project python --with-editable ./python --with-editable
./python/providers/langgraph python <LangGraph $top wrap smoke>` ->
wrapped args schema field `param_top`
- `uv run --project python ruff check --config python/config/ruff.toml
python/composio/utils/shared.py
python/tests/test_tool_schema_aliasing.py` -> passed
- `uv run --project python ruff format --check --config
python/config/ruff.toml python/composio/utils/shared.py
python/tests/test_tool_schema_aliasing.py` -> passed
- `git diff --check` -> passed
Note: #3500's hosted MCP emission-layer report is already closed; this
PR covers the Python SDK/provider surface rather than changing hosted
MCP schema emission.
## Problem
Tools whose parameter schemas express their file fields through a
`$ref`/`$defs` indirection lose auto file handling. With
`dangerously_allow_auto_upload_download_files=True`, a
`file_downloadable` output behind a `$ref` is never downloaded (the raw
`{name, mimetype, s3url}` object is passed through), and a
`file_uploadable` input behind a `$ref` is never staged — silently, with
no error.
`GMAIL_GET_ATTACHMENT` is the canonical shape — the flag sits two `$ref`
hops deep:
```
data → $ref → #/$defs/GetAttachmentResponse → file → $ref → #/$defs/FileDownloadable (file_downloadable: true)
```
**Root cause:** the hand-rolled walkers in
`composio/core/models/_files.py` (`_has_file_property`,
`_substitute_file_upload_value`, `_substitute_file_download_value`, …)
recurse through `properties`/`anyOf`/`oneOf`/`allOf`/`items` but **never
dereference `$ref`**, so a flagged node reachable only through a
reference is invisible to them.
Fixes https://github.com/ComposioHQ/composio/issues/3506
## Approach
Rather than teach each walker to dereference — which means threading a
root schema through ~10 methods, re-deriving `$ref` handling in each,
and risking unbounded recursion on cyclic schemas — this introduces a
single **`dereference_json_schema`** utility and inlines references
**once at the `FileHelper` boundary** (`process_file_uploadable_schema`,
`substitute_file_uploads`, `substitute_file_downloads`). The walkers
stay reference-agnostic.
This mirrors the TypeScript SDK's `dereferenceJsonSchema` (the
counterpart fix in #3566 for the TS twin of this bug, #3307) so both
SDKs share one behavioral contract. Resolving once at the boundary also
gives every walker `$ref` support for free — including keywords they
never special-cased (`additionalProperties`, `patternProperties`,
`prefixItems`, `not`), because the resolver walks containers
reflectively.
## What's in the utility (`composio/utils/json_schema.py`)
Faithful port of the TS guards:
- **Cycle safety** — breaks both `$ref` cycles and live-object identity
cycles with a permissive `{type: object, additionalProperties: true}`
sentinel. (A per-node lazy resolver without threaded cycle tracking
infinite-loops on a self-referential schema; there's a regression test
for exactly this.)
- **Depth caps** — `MAX_REF_CHAIN_DEPTH=100` / `MAX_NODE_DEPTH=512`
raise a typed `JSONSchemaRefResolutionError` instead of exhausting the
interpreter (with a `RecursionError` backstop converted to the same
typed error).
- **Sibling-keyword merge** — Draft 2020-12 semantics (siblings win on
collision), so a `description`/`default` next to a `$ref` survives.
- **External-ref passthrough** — `http(s)://` refs are left untouched
and logged once for audit.
- **`sentinel` mode** — a dangling `$ref` (an API schema that emits
`$ref` with no `$defs` block,
https://github.com/ComposioHQ/composio/issues/3307) degrades to the
permissive sentinel + an LLM-visible hint, instead of aborting the tool
call. The `FileHelper` uses this mode for API-sourced schemas.
- **Non-mutating** — returns a new schema; `$defs`/`definitions` are
stripped from the inlined root.
## Tests
- `python/tests/test_json_schema.py` — 30 cases mirroring the TS
`jsonSchema.test.ts` contract (chains, reflective container walk, legacy
`definitions`, sibling merge, cycles, depth caps, pointer-escape
decoding, array-index pointers, full `sentinel`-mode suite).
- `python/tests/test_files.py` — end-to-end regressions through the
public methods: download/upload behind `$ref`/`$defs` (the #3506 repro),
the LLM-facing input transform, graceful degradation of a dangling
`$ref`, and a **self-referential schema that must not recurse
infinitely**.
- `nox -s chk` (ruff + mypy) green; `nox -s tst` for the file/execution
suites green.
## Notes
Supersedes #3545 (third-party PR that fixed the same bug by threading
`root_schema` through each walker). That diagnosis was correct; this
takes the centralized route consistent with the TS SDK and adds the
cycle/depth guards. No changeset — Python-package change.
## Summary
Models — and some MCP transports — occasionally emit tool-call arguments
as a **JSON string instead of an object/dict**. The most visible trigger
is `COMPOSIO_MULTI_EXECUTE_TOOL` on the Vercel AI SDK, where streaming
fails with:
> `messages.3.content.1.tool_use.input: Input should be a valid
dictionary`
Until now only a handful of providers guarded against this, each with
its own slightly different inline check, leaving most providers
vulnerable and behaviour inconsistent across the SDK.
This PR centralizes the coercion into **one helper per language** and
routes **every** provider through it, in both the TypeScript and Python
SDKs.
Closes https://github.com/ComposioHQ/composio/issues/2406
## What changed
**TypeScript** — new `normalizeToolArguments` in `@composio/core`
(exported), used by every provider:
`vercel`, `cloudflare`, `openai-agents`, `openai` (ChatCompletions +
Responses), `anthropic`, `google`, `langchain`, `llamaindex`,
`claude-agent-sdk`, `mastra`.
**Python** — new `normalize_tool_arguments` in `composio.utils.shared`,
used by every provider:
`openai`, `openai-responses`, `anthropic`, `google`, `langchain`,
`langgraph`, `crewai`, `autogen`, `llamaindex`, `gemini`, `google-adk`,
`openai-agents`, `claude-agent-sdk`.
Shared semantics (identical in both languages):
| Input | Result |
| --- | --- |
| object / dict | returned unchanged |
| JSON string | parsed to object |
| empty / whitespace string | `{}` |
| `null` / `undefined` / `None` | `{}` |
| array, primitive, unparseable string, JSON that isn't an object |
**typed error** (`ComposioInvalidToolArgumentsError` / `InvalidParams`)
with the original parse error as cause |
The typed error replaces the previous grab-bag of behaviours: a raw
`SyntaxError`/`JSONDecodeError`, or — worse — silently forwarding a
malformed string downstream.
## Why this supersedes the open PRs
This consolidates and extends three open PRs that each addressed a slice
of the problem inconsistently. Their authors are credited as co-authors
on the relevant commits:
- **#3489** (LlamaIndex + Claude Agent SDK, TS) — @srijanarya
- **#3438** (Anthropic, Google, LangChain, TS) — @aptsalt
- **#3437** (Google ADK empty schemas + name fix, Python) —
@pragnyanramtha — its empty-`input_parameters` / missing-description
handling and the `gemini` → `google_adk` provider-name fix are folded in
here.
Compared to the three combined, this PR additionally: covers **every**
provider in **both** SDKs (not a subset of one), defines a single source
of truth instead of per-provider snippets, normalizes empty/`null`
payloads to `{}`, and raises an actionable typed error instead of
leaking `SyntaxError` or forwarding a bad string.
## Tests
- Exhaustive unit tests for both helpers (object passthrough,
JSON-string parse, empty/null → `{}`, malformed/non-object → typed
error).
- Per-provider regression tests across the touched TypeScript providers
(string path, malformed-string path, empty-payload path).
- Full `@composio/core` + touched-provider TS suites pass; `typecheck`
and `lint` clean. Python `ruff` clean and new test green.
## Changeset
Patch bump for all affected TypeScript packages (`@composio/core` + the
providers). Python follows its own versioning, so no changeset there.
---------
Co-authored-by: srijanarya <74669415+srijanarya@users.noreply.github.com>
Co-authored-by: Deepak Singh Kandari <deepaksinghkandari07@gmail.com>
Co-authored-by: Pragnyan Ramtha <pragnyanramtha@gmail.com>
The `return t.Optional[t.Any] if has_null else str` line introduced in
630038ab8 returns a union of `<typing special form>` and `type[str]`,
which mypy can't reconcile with the function's declared `t.Type` return.
Same `# type: ignore` pattern already used three other times in this
function (lines 280, 285, 287) for the identical reason — `t.Optional`
isn't a `type[T]`, it's a typing special form.
Unblocks Checks/common-checks on this PR and any other PR cut after
630038ab8 landed on next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Automatic tool file upload/download for `file_uploadable` fields is
**off by default** in TypeScript and Python. Callers must explicitly opt
in, and uploads from local paths are constrained by a fail-closed
allowlist.
## Changes
- **Removed (breaking):** `autoUploadDownloadFiles` (TS) /
`auto_upload_download_files` (Python) — the legacy default-on flag is
gone, not just deprecated.
- **New opt-in:** `dangerouslyAllowAutoUploadDownloadFiles` (TS) /
`dangerously_allow_auto_upload_download_files` (Python). When `true`,
`tools.get(...)` collapses `file_uploadable` schemas to `{ type:
'string', format: 'path' }` and the SDK stages local paths/URLs at
execute time.
- **New:** `fileUploadDirs?: string[] | false` — fail-closed allowlist
for local upload paths. `undefined` → `[<home>/.composio/temp]`; `false`
→ reject all local paths (URLs / `File` objects unaffected); explicit
`string[]` replaces the default. Components are matched on a path
boundary after `realpath`.
- **New:** `fileDownloadDir?: string` — directory where
`file_downloadable` results are staged.
- **New:** `beforeFileUpload` hook receives `source: 'path' | 'url' |
'file'` (TS) / `'path' | 'url'` (Python) so it can branch on input type.
- **New (TS):** when auto-upload is **off** and an LLM-driven
`tools.execute` is called against a tool with `file_uploadable` inputs,
the SDK emits a one-shot warning per tool slug pointing at
`composio.files.upload()` for manual staging.
## Migration
To restore previous behavior:
```ts
new Composio({
apiKey: process.env.COMPOSIO_API_KEY!,
dangerouslyAllowAutoUploadDownloadFiles: true,
// Optional: tighten the allowlist beyond the default ~/.composio/temp
fileUploadDirs: ['/srv/uploads'],
});
```
```python
Composio(api_key="...", dangerously_allow_auto_upload_download_files=True)
```
If you previously passed the legacy flag, remove it. There is no
transitional warning — TS and Python both reject the unknown property at
the type/keyword-arg level.
## Versioning
| Package | Bump |
| ------- | ---- |
| `@composio/core` | minor |
| `composio` (Python) | minor |
| Other `@composio/*` packages | patch (via changesets
`updateInternalDependencies: "patch"`) |
See
`docs/content/changelog/04-24-26-legacy-auto-upload-config-removal.mdx`
for the full migration writeup.
## Summary
Security-hardening for automatic file upload in `@composio/core` (patch
release per changeset).
### Changes
- **Default denylist** for local paths before auto-upload /
`files.upload`: blocks common credential directories (e.g. `.ssh`,
`.aws`) and credential-like filenames (e.g. `.env`, default SSH private
keys). Resolves symlinks when the path exists.
- **Config:** `sensitiveFileUploadProtection`,
`fileUploadPathDenySegments` on `Composio`.
- **`beforeFileUpload`** hook (e.g. with `composio.tools.get` /
`tools.execute`): rewrite path, return `false` to abort, or throw.
- **Errors:** `ComposioSensitiveFilePathBlockedError`,
`ComposioFileUploadAbortedError`; file modifier errors exported from
`@composio/core` errors entry.
- **Changeset:** patch bump for `@composio/core`.
### Notes
- URLs and `File` blobs are not subject to the path denylist
(unchanged).
- Opt out of path checks only if required:
`sensitiveFileUploadProtection: false`.
### Tests
- `pnpm test` in `ts/packages/core` (799 tests) passed locally before
commit.
Made with [Cursor](https://cursor.com)
## Summary
Fixes#2459
When executing `DROPBOX_SEARCH_FILE_OR_FOLDER` with the optional
`options` parameter, the SDK crashes with `KeyError: 'type'` before the
API request is even sent. This happens because
`pydantic_model_from_param_schema()` uses direct dictionary access
(`prop_info["type"]`, `prop_info["title"]`) on schema entries that may
not have a top-level `type` key — nested object schemas can use `anyOf`,
`allOf`, `oneOf`, or `$ref` instead.
### Changes in `python/composio/utils/shared.py`
- `prop_info["type"]` → `prop_info.get("type")` — prevents `KeyError` on
schemas without `type`
- `prop_info["title"]` → `prop_info.get("title", prop_name)` — falls
back to property name when `title` is absent
- `FALLBACK_VALUES[prop_type]` → `FALLBACK_VALUES.get(prop_type)` —
handles `None` prop_type gracefully
- When `prop_type is None` (schema uses combiners instead of `type`),
delegates to `json_schema_to_pydantic_type()` which already handles
`anyOf`/`allOf`/`oneOf` correctly
- Removed stale `:raises KeyError` from docstring since the function no
longer raises it
### Before
```python
prop_type = prop_info["type"] # KeyError if schema uses anyOf/allOf/$ref
prop_title = prop_info["title"] # KeyError if title is absent
prop_default = prop_info.get("default", FALLBACK_VALUES[prop_type]) # KeyError if prop_type not in FALLBACK_VALUES
```
### After
```python
prop_type = prop_info.get("type")
prop_title = prop_info.get("title", prop_name).replace(" ", "")
prop_default = prop_info.get("default", FALLBACK_VALUES.get(prop_type))
if prop_type is not None and prop_type in PYDANTIC_TYPE_TO_PYTHON_TYPE and prop_type not in CONTAINER_TYPE:
signature_prop_type = PYDANTIC_TYPE_TO_PYTHON_TYPE[prop_type]
elif prop_type is None:
# Delegate to json_schema_to_pydantic_type which handles all combiners
signature_prop_type = t.cast(t.Type, json_schema_to_pydantic_type(json_schema=prop_info))
else:
signature_prop_type = pydantic_model_from_param_schema(prop_info)
```
This fix is generic — it handles all cases where a schema property lacks
a `type` key, not just the Dropbox case.
**Note: This PR was authored by Claude (AI), operated by
@maxwellcalkin.**
Alberto Schiabel
