build_raw_request kept the Content-Length inherited from the captured
request, so replaying a modified body (repeat_request) emitted a request
whose declared length did not match the body — truncating the payload or
stalling the target. Drop any inherited Content-Length (case-insensitively)
and recompute it from the body actually being sent.
Adds tests covering a lengthened body, an emptied body, and the
no-inherited-header path.
Fixes#814
Co-authored-by: thejesh23 <thejesh23@users.noreply.github.com>
* fix(proxy,tooling): serialize+reconnect Caido client, actionable HTTPQL errors, sandbox tool guidance
Addresses the top recurring agent tool-call failures observed in telemetry:
- proxy: the shared Caido client had no locking or reconnect, so concurrent
agent calls raced ("Transport is already connected") and a dead transport
poisoned the rest of the run ("Connector is closed"/"Server disconnected").
Add an asyncio lock + bounded reconnect in caido_api.call_with_client (sandbox
path) and a scan-wide caido_lock in the run context that host-side proxy tools
hold around every call. Deterministic errors are not retried.
- proxy: list_requests now returns Caido's exact parser message, echoes the
offending query, and includes a corrected-syntax hint so agents self-correct
instead of retrying a broken HTTPQL filter.
- shell/prompt: document that write_stdin requires a process started with
tty=true; nudge toward writing Python to a file over deeply-nested one-liners;
note the venv pre-installs common libs.
- agent-browser: distinguish daemon/connection failures (run doctor, don't loop)
from malformed commands; invoke directly (no sh -c wrapper).
- containers: use POSIX '.' instead of the bashism 'source' in generated rc
files (fixes 'sh: source: not found'); add file + xxd and pre-install
requests/httpx/beautifulsoup4/lxml/pyjwt/cryptography in the sandbox venv.
- tests: cover proxy serialization/reconnect/no-retry and HTTPQL errors.
* fix(proxy): host-side reconnect, close stale clients, don't retry mutations
Addresses Greptile review on the reconnect logic:
- Host path had no reconnect: a dead shared context client (Caido restart /
network blip) previously disabled proxy tools for the rest of the scan. Add
SharedCaidoClient, a serialized reconnect-safe holder stored once per scan in
the run context and shared across agents. On a dead transport it rebuilds via
reconnect_caido, which re-selects the SAME Caido project (preserving captured
traffic) instead of creating a new empty one.
- Don't repeat completed mutations: call_with_client / SharedCaidoClient.call
take idempotent=. Reads retry once on reconnect; replay + scope
create/update/delete heal the client but re-raise instead of risking a
double-apply.
- Don't leak replaced clients: the stale client is aclose()d (best-effort) on
every reconnect.
- Extend tests to cover close-on-reconnect, non-idempotent re-raise, and the
SharedCaidoClient holder.
* fix(proxy): close replacement Caido client when project.select fails
Addresses Greptile P1: in reconnect_caido (and bootstrap_caido) a successful
connect() followed by a failing project.select()/create() discarded the
connected client without closing it, so a missing/unavailable project could
leak a transport on every retry. Close the client before re-raising.
---------
Co-authored-by: Alex Schapiro <bearsyankees@gmail.com>
Five rounds of sweep across the tree. Net ~544 lines removed.
Removed:
- Section-divider banners and one-line section labels (# Display
utilities, # ----- list_requests -----, # CVSS breakdown, etc.).
- Module-level prose docstrings on internal modules. Kept one-line
summaries; trimmed multi-paragraph narration about SDK/Strix
responsibility splits, cache strategies, three-source precedence.
- Internal-helper docstrings that just restate the function name —
caido_api helpers (caido_url, get_client, view_request, etc.),
settings-class one-liners (LLMSettings, RuntimeSettings, ...),
UI helper docstrings.
- Args/Returns blocks on non-LLM-facing internal helpers
(build_strix_agent, render_system_prompt, create_or_reuse,
bootstrap_caido) — kept only the genuinely non-obvious params.
- Internal-history phrasing — "Mirrors main-branch shape",
"pre-SDK harness", "previous lookup matched no attribute".
- Narrative comments inside function bodies that explained what the
next line does, design rationale obvious from the surrounding code,
or "we used to..." asides.
- Trailing periods on every error-string literal across the tool tree.
- Duplicated roundtripTime quirk comment (kept the LLM-facing copy in
tools/proxy/tools.py).
Kept (every one names an upstream bug, vendored-code provenance, or
non-obvious data quirk):
- core/runner.py: SDK replay-with-empty-initial-input + on_agent_end
lifecycle gap.
- runtime/docker_client.py: VERBATIM COPY block of the upstream
_create_container body, pinned to SDK v0.14.6.
- runtime/session_manager.py: NO_PROXY for agent-browser CDP loopback.
- tools/proxy/caido_api.py: generated-pydantic Request.raw quirk,
replay double-history pitfall.
- tools/proxy/tools.py: Caido roundtripTime=0 quirk for proxy
captures.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four passes of audit-and-patch on the tool surface, condensed.
Tool API shape:
- Todo tools collapse to a single list-based form (one arg per tool,
always a list, no dual-mode validator). Result-field names line up
across the family — created_count / updated_count / marked_count /
deleted_count, and _mark returns a single "marked" key plus the new
status instead of marked_done / marked_pending.
- list_notes splits the overloaded total_count into filtered_count
(matches) and total_count (grand total), matching list_todos. All
three notes mutations now echo total_count and note_id.
- finish_scan drops the machine-code error strings; a single human
"error" key carries the reason on every failure path.
- scope_rules delete echoes a message so the renderer's success
branch has something to surface.
Failure-key unification: every tool now uses {"success": False,
"error": "..."} on failure paths. Touched thinking, web_search,
reporting, and finish. Trailing periods on error strings swept clean
across the whole tool tree.
Tool prompts (docstring re-imports vs main):
- create_vulnerability_report re-imports the CWE reference catalog,
multi-part fix rules, fix_before/fix_after PR-suggestion mechanics,
the COMMON MISTAKES list, the informational-vs-actionable
distinction, and file-path examples.
- web_search re-imports concrete example queries.
- list_sitemap docstring fixed hasDescendants -> has_descendants
(the camelCase reference never matched our snake_case schema).
- create_agent.skills description "Comma-separated" -> "List of".
- factory.py module docstring no longer claims there's no runtime
skill-loading tool. agents_graph module docstring lists stop_agent.
- system_prompt nudges loading the matching skill before guessing
payloads or syntax from memory.
TUI:
- proxy_renderer was reading stale field names from the pre-SDK
schema (requests / total_count / statusCode / matches /
showing_lines); now reads entries / page_info / status_code / hits
/ page+total_lines. Three proxy operations were rendering empty
before this.
- Idle-pane placeholder text trimmed to "Loading...".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Re-add list_sitemap and view_sitemap_entry from main, ported to the
new caido-sdk-client layout via raw GraphQL queries (the typed SDK
doesn't expose sitemap operations, but the Caido server still
supports sitemapRootEntries / sitemapDescendantEntries / sitemapEntry).
Wired through caido_api (sandbox-importable helpers), the host-side
@function_tool wrappers, factory _BASE_TOOLS, the system prompt, the
python skill doc, and the public proxy docs.
While threading these through, lock down the output contract across
every proxy tool so the model sees one consistent shape:
- All tools wrap success/failure in {"success": bool, "error"?: str}
- Canonical field names: status_code, length, roundtrip_ms (omitted
when 0), is_tls, has_descendants. snake_case everywhere on output;
camelCase stays only on the input side where it's the GraphQL
schema.
- repeat_request now returns a structured response that matches
list_requests' response_summary shape (parse_raw_response parses
the raw bytes into status_code / length / headers / body), with
body capped at 8KB and a body_truncated flag so the model knows
when to fetch the full body via view_request.
- RepeatRequestRenderer was reading non-existent top-level keys
(status_code, response_time_ms, body) and silently displaying
nothing useful — now reads the structured response shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
send_request was a thin wrapper over the Caido Replay API that the model
could replicate with a one-liner `curl` via exec_command. The sandbox's
HTTP_PROXY env captures all such traffic for free, so the tool was
adding bugs (duplicate dispatch, dropped responses) without adding
capability. Removed across factory, tools module, sandbox-importable
caido_api helper, TUI renderer, prompt template, skill doc, and public
docs. repeat_request stays — it operates on captured request IDs with
structured modifications, which curl can't replicate cleanly.
Three caido-sdk-client workarounds that were hitting us through both
send_request and repeat_request:
- replay_send_raw used to pass CreateReplaySessionFromRaw to
sessions.create(), which seeds a stored entry server-side, then
called send() — producing two history rows per call. Empty-create +
send produces one dispatched request.
- The same helper read result.entry.response_raw, an attribute that
doesn't exist on ReplayEntry, so response bytes were silently
dropped. Fixed to walk result.entry.response.raw with proper None
guards.
- get_request_with_client passed include_request_raw / include_response_raw
based on the requested part, but the SDK's generated pydantic models
declare raw as required even though the GraphQL fragment makes it
conditional via @include. Passing False crashed view_request with a
pydantic validation error. Always request both raw bodies; the caller
picks which to surface.
Also wrapped replay.send() in asyncio.wait_for(30s) so a stalled Caido
dispatch (notably loopback targets that don't route cleanly through the
sandbox proxy) fails fast with a model-readable error instead of
hanging the agent until the function_tool 120s budget expires.
Finally, list_requests now omits the roundtrip_ms field when Caido
reports 0 — proxy-captured unscoped traffic consistently reports 0
while scoped/replay traffic carries real measurements, so the absence
of the field is now informative ("Caido didn't measure this") rather
than misleading ("this request took 0ms").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
devin-ai-integration[bot]