## Problem
Tools whose parameter schemas express their file fields through a
`$ref`/`$defs` indirection lose auto file handling. With
`dangerously_allow_auto_upload_download_files=True`, a
`file_downloadable` output behind a `$ref` is never downloaded (the raw
`{name, mimetype, s3url}` object is passed through), and a
`file_uploadable` input behind a `$ref` is never staged — silently, with
no error.
`GMAIL_GET_ATTACHMENT` is the canonical shape — the flag sits two `$ref`
hops deep:
```
data → $ref → #/$defs/GetAttachmentResponse → file → $ref → #/$defs/FileDownloadable (file_downloadable: true)
```
**Root cause:** the hand-rolled walkers in
`composio/core/models/_files.py` (`_has_file_property`,
`_substitute_file_upload_value`, `_substitute_file_download_value`, …)
recurse through `properties`/`anyOf`/`oneOf`/`allOf`/`items` but **never
dereference `$ref`**, so a flagged node reachable only through a
reference is invisible to them.
Fixes https://github.com/ComposioHQ/composio/issues/3506
## Approach
Rather than teach each walker to dereference — which means threading a
root schema through ~10 methods, re-deriving `$ref` handling in each,
and risking unbounded recursion on cyclic schemas — this introduces a
single **`dereference_json_schema`** utility and inlines references
**once at the `FileHelper` boundary** (`process_file_uploadable_schema`,
`substitute_file_uploads`, `substitute_file_downloads`). The walkers
stay reference-agnostic.
This mirrors the TypeScript SDK's `dereferenceJsonSchema` (the
counterpart fix in #3566 for the TS twin of this bug, #3307) so both
SDKs share one behavioral contract. Resolving once at the boundary also
gives every walker `$ref` support for free — including keywords they
never special-cased (`additionalProperties`, `patternProperties`,
`prefixItems`, `not`), because the resolver walks containers
reflectively.
## What's in the utility (`composio/utils/json_schema.py`)
Faithful port of the TS guards:
- **Cycle safety** — breaks both `$ref` cycles and live-object identity
cycles with a permissive `{type: object, additionalProperties: true}`
sentinel. (A per-node lazy resolver without threaded cycle tracking
infinite-loops on a self-referential schema; there's a regression test
for exactly this.)
- **Depth caps** — `MAX_REF_CHAIN_DEPTH=100` / `MAX_NODE_DEPTH=512`
raise a typed `JSONSchemaRefResolutionError` instead of exhausting the
interpreter (with a `RecursionError` backstop converted to the same
typed error).
- **Sibling-keyword merge** — Draft 2020-12 semantics (siblings win on
collision), so a `description`/`default` next to a `$ref` survives.
- **External-ref passthrough** — `http(s)://` refs are left untouched
and logged once for audit.
- **`sentinel` mode** — a dangling `$ref` (an API schema that emits
`$ref` with no `$defs` block,
https://github.com/ComposioHQ/composio/issues/3307) degrades to the
permissive sentinel + an LLM-visible hint, instead of aborting the tool
call. The `FileHelper` uses this mode for API-sourced schemas.
- **Non-mutating** — returns a new schema; `$defs`/`definitions` are
stripped from the inlined root.
## Tests
- `python/tests/test_json_schema.py` — 30 cases mirroring the TS
`jsonSchema.test.ts` contract (chains, reflective container walk, legacy
`definitions`, sibling merge, cycles, depth caps, pointer-escape
decoding, array-index pointers, full `sentinel`-mode suite).
- `python/tests/test_files.py` — end-to-end regressions through the
public methods: download/upload behind `$ref`/`$defs` (the #3506 repro),
the LLM-facing input transform, graceful degradation of a dangling
`$ref`, and a **self-referential schema that must not recurse
infinitely**.
- `nox -s chk` (ruff + mypy) green; `nox -s tst` for the file/execution
suites green.
## Notes
Supersedes #3545 (third-party PR that fixed the same bug by threading
`root_schema` through each walker). That diagnosis was correct; this
takes the centralized route consistent with the TS SDK and adds the
cycle/depth guards. No changeset — Python-package change.
Alberto Schiabel