Commit Graph

18 Commits

Author SHA1 Message Date
Ahmed Allam 53e4658d88 fix(todo): stop a todo plan failing on priority or duplicates (#1021) 2026-08-08 15:18:48 -07:00
0xallam 8414c59557 Strip narrative comments and module/helper docstrings
Five rounds of sweep across the tree. Net ~544 lines removed.

Removed:
- Section-divider banners and one-line section labels (# Display
  utilities, # ----- list_requests -----, # CVSS breakdown, etc.).
- Module-level prose docstrings on internal modules. Kept one-line
  summaries; trimmed multi-paragraph narration about SDK/Strix
  responsibility splits, cache strategies, three-source precedence.
- Internal-helper docstrings that just restate the function name —
  caido_api helpers (caido_url, get_client, view_request, etc.),
  settings-class one-liners (LLMSettings, RuntimeSettings, ...),
  UI helper docstrings.
- Args/Returns blocks on non-LLM-facing internal helpers
  (build_strix_agent, render_system_prompt, create_or_reuse,
  bootstrap_caido) — kept only the genuinely non-obvious params.
- Internal-history phrasing — "Mirrors main-branch shape",
  "pre-SDK harness", "previous lookup matched no attribute".
- Narrative comments inside function bodies that explained what the
  next line does, design rationale obvious from the surrounding code,
  or "we used to..." asides.
- Trailing periods on every error-string literal across the tool tree.
- Duplicated roundtripTime quirk comment (kept the LLM-facing copy in
  tools/proxy/tools.py).

Kept (every one names an upstream bug, vendored-code provenance, or
non-obvious data quirk):
- core/runner.py: SDK replay-with-empty-initial-input + on_agent_end
  lifecycle gap.
- runtime/docker_client.py: VERBATIM COPY block of the upstream
  _create_container body, pinned to SDK v0.14.6.
- runtime/session_manager.py: NO_PROXY for agent-browser CDP loopback.
- tools/proxy/caido_api.py: generated-pydantic Request.raw quirk,
  replay double-history pitfall.
- tools/proxy/tools.py: Caido roundtripTime=0 quirk for proxy
  captures.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 14:02:40 -07:00
0xallam 054eedf53f Tighten tool surface consistency
Four passes of audit-and-patch on the tool surface, condensed.

Tool API shape:
- Todo tools collapse to a single list-based form (one arg per tool,
  always a list, no dual-mode validator). Result-field names line up
  across the family — created_count / updated_count / marked_count /
  deleted_count, and _mark returns a single "marked" key plus the new
  status instead of marked_done / marked_pending.
- list_notes splits the overloaded total_count into filtered_count
  (matches) and total_count (grand total), matching list_todos. All
  three notes mutations now echo total_count and note_id.
- finish_scan drops the machine-code error strings; a single human
  "error" key carries the reason on every failure path.
- scope_rules delete echoes a message so the renderer's success
  branch has something to surface.

Failure-key unification: every tool now uses {"success": False,
"error": "..."} on failure paths. Touched thinking, web_search,
reporting, and finish. Trailing periods on error strings swept clean
across the whole tool tree.

Tool prompts (docstring re-imports vs main):
- create_vulnerability_report re-imports the CWE reference catalog,
  multi-part fix rules, fix_before/fix_after PR-suggestion mechanics,
  the COMMON MISTAKES list, the informational-vs-actionable
  distinction, and file-path examples.
- web_search re-imports concrete example queries.
- list_sitemap docstring fixed hasDescendants -> has_descendants
  (the camelCase reference never matched our snake_case schema).
- create_agent.skills description "Comma-separated" -> "List of".
- factory.py module docstring no longer claims there's no runtime
  skill-loading tool. agents_graph module docstring lists stop_agent.
- system_prompt nudges loading the matching skill before guessing
  payloads or syntax from memory.

TUI:
- proxy_renderer was reading stale field names from the pre-SDK
  schema (requests / total_count / statusCode / matches /
  showing_lines); now reads entries / page_info / status_code / hits
  / page+total_lines. Three proxy operations were rendering empty
  before this.
- Idle-pane placeholder text trimmed to "Loading...".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 12:16:47 -07:00
0xallam 763df86f17 Collapse todo tools to a single list-based form
create_todo / update_todo / mark_todo_done / mark_todo_pending /
delete_todo used to accept either a single-item form (title, todo_id,
…) or a bulk form (todos, updates, todo_ids), reject the call if the
agent set both, and explain the rule in the docstring. The agent kept
tripping the validator. Drop the single-item form everywhere — each
tool now takes one list arg. Single calls just pass a one-item list.

While the API was being reshaped, line the result schemas up:
created_count replaces the lone "count", _mark returns a single
"marked" key plus new_status instead of marked_done / marked_pending,
and list_todos splits the overloaded total_count into filtered_count
(matches) and total_count (grand total) so a filtered call no longer
hides the real size.

Docstrings now spell out each item's fields with required/optional
and the legal status / priority values, plus a worked example.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 23:51:08 -07:00
0xallam bdc3c5470e Tighten todo + think tool contracts
Reject ambiguous calls in todo tools that previously combined the
single-target params and the bulk-array param (e.g. create_todo with
both `title` and `todos` would silently create N+1 items). Each tool
now errors with a mode-specific hint pointing the model at the
appropriate form. Also drop the meaningless char-count from `think`'s
success message — the model already knows what it wrote.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:58:09 -07:00
0xallam af826e1281 refactor: consolidate run state layout 2026-04-26 15:01:35 -07:00
0xallam 5fd2a64562 fix(persistence): close all 9 gaps from the resume audit
Three critical correctness fixes + six TUI/audit/UX fixes from the
parallel-agent audit. All changes verified by an end-to-end smoke
that builds, persists, and re-hydrates state across two simulated
process boundaries.

Critical (resume integrity):

1. ``bus.cancel_descendants_graceful`` now calls ``_maybe_snapshot``
   after mutating the ``stopping`` set. Previously, a process crash
   between user-initiated graceful-stop and the next finalize lost
   the stop signal — respawned agents would run forever instead of
   exiting. ``_respawn_subagents`` also gains a guard that skips
   agents in ``stopping`` so a previously-cancelled agent is not
   resurrected on resume.

2. ``Tracer.hydrate_from_run_dir`` now **raises** on corrupt
   ``vulnerabilities.json`` instead of swallowing the exception. The
   prior behaviour silently reset ``vulnerability_reports`` to empty,
   so the next ``add_vulnerability_report`` would allocate ``vuln-0001``
   and overwrite the prior MD on disk — silent data loss.

3. ``--instruction`` passed on resume now reaches the model. The CLI
   captures whether the user explicitly passed an instruction
   (``args.user_explicit_instruction``) before ``_load_resume_state``
   loads the persisted one. ``run_strix_scan`` reads
   ``scan_config["resume_instruction"]`` and, on resume, sends the
   new instruction to root's bus inbox before calling
   ``run_with_continuation`` (which uses ``initial_input=[]`` for SDK
   replay). The inject filter surfaces it on the next turn.

4. ``--resume X`` errors loudly when ``scan_state.json`` exists but
   ``bus.json`` doesn't. Previously this silently fresh-started in
   the same dir, confusing the user who explicitly asked to resume.

TUI / audit / UX:

5. ``Tracer.hydrate_from_run_dir`` now reads ``bus.json`` too and
   pre-populates ``tracer.agents`` from the snapshot's ``statuses`` /
   ``names`` / ``parent_of``. Before this, the TUI tree on resume
   showed only currently-running agents; completed/crashed children
   from the prior run were invisible.

6. ``Tracer.hydrate_from_run_dir`` also seeds ``self._llm_stats`` from
   ``bus.stats_live + bus.stats_completed`` so the resume's footer
   shows cumulative tokens / requests across the prior run plus the
   resume segment, instead of resetting to zero.

7. ``Tracer.save_run_data`` now also writes ``run_metadata.json``
   (start_time, run_id, run_name, targets, status), and
   ``hydrate_from_run_dir`` restores ``start_time`` from it. Prior
   behaviour reset start_time to ``now()`` on every Tracer init,
   breaking the final report's duration calc on resumed scans.

8. Per-agent todos persist to ``{run_dir}/todos.json`` (atomic write
   on every CRUD). ``hydrate_todos_from_disk`` (called from
   ``run_strix_scan``) reloads them so respawned subagents find
   their lists intact. Previously, the module-level
   ``_todos_storage`` was lost on every process restart.

9. ``_load_resume_state`` validates each ``cloned_repo_path`` from
   the persisted ``scan_state.json`` still exists on disk. Previously
   a deleted clone dir would let the resume proceed with an empty
   source tree, with agents silently scanning nothing.

Bonus: ``bus.finalize`` no longer pops ``parent_of`` and ``names``
for finalized agents. Routing protection (don't accept ``send`` to
finalized agents) comes from the ``statuses[id]`` terminal-state
check in ``send`` itself, so dropping those keys was overzealous and
made completed children invisible in ``view_agent_graph`` and the
TUI tree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 00:57:52 -07:00
0xallam b3f7cfd040 refactor: nuke `strix_tool` shim + dead package re-exports
``@strix_tool`` was passing through every kwarg to ``@function_tool``
with the same defaults — zero Strix-specific value-add. The docstring
also still claimed terminal/browser/python tools opted into
``timeout_behavior="raise_exception"``, but those tools were all
deleted in the recent migrations.

- Replace 30 ``@strix_tool(...)`` callsites with ``@function_tool(...)``.
- Inline ``dump_tool_result(x)`` as ``json.dumps(x, ensure_ascii=False,
  default=str)`` at all 64 callsites — no helper.
- Delete ``strix/tools/_decorator.py``.

Drive-by: gut dead package re-exports.

- ``strix/{agents,orchestration,tools}/__init__.py`` re-exported
  symbols nobody imports via the package — every consumer uses deep
  paths (``from strix.agents.factory import build_strix_agent``).
- The 8 ``strix/tools/<sub>/__init__.py`` re-exports only fed the
  splat ``from .agents_graph import *`` etc. in the parent package
  init, which is also gone now.
- Reduced to docstrings (or empty) so ``import strix.tools`` doesn't
  drag every tool's transitive deps in eagerly.

Drive-by: drop dead helpers in ``runtime.session_manager``
(``cached_scan_ids``, ``_reset_cache_for_tests``) — zero callers since
``tests/`` was nuked in ``a6d578c``.

Verified all tool timeouts preserved (think=10, list_requests=120,
finish_scan=60, web_search=330) and ruff/mypy at baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 15:17:46 -07:00
0xallam 49c38de3b2 refactor: dedupe `_dump` helper, collapse retry-policy plumbing, scrub test scars
Tools:
- Add a single ``dump_tool_result`` helper in ``tools/_decorator.py``
  and remove the eight identical ``_dump`` definitions from
  ``proxy/tools.py``, ``file_edit/tools.py``, ``python/tool.py``,
  ``terminal/tool.py``, ``todo/tools.py``, ``browser/tool.py``,
  ``notes/tools.py``, ``agents_graph/tools.py``. Imports trimmed.
  Net -50 LoC across the tool modules.

run_config_factory:
- Inline the four retry-policy plumbing pieces
  (``_RETRYABLE_HTTP_STATUSES``, ``_DEFAULT_MAX_RETRIES``,
  ``_DEFAULT_BACKOFF``, ``_default_retry_policy()``) into a single
  module-level ``_DEFAULT_RETRY`` ``ModelRetrySettings`` literal. The
  inputs were never overridden and the helper had one caller.

Tests:
- Drop migration scars from ``tests/test_run_config_factory.py``
  (``Phase 1`` / ``C1`` / ``C11`` / ``C21`` / ``HARNESS_WIKI`` / ``AUDIT``
  references). Replace the ``_RETRYABLE_HTTP_STATUSES``-touching test
  with a ``retry.policy is not None`` smoke check now that the constant
  has been inlined.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 12:54:44 -07:00
0xallam d959fe2163 refactor: collapse dual stat buckets, prune unused params, kill dead helpers
Tracer:
- Collapse the ``live`` / ``completed`` LLM stat buckets into one
  flat dict. The ``completed`` bucket was only ever written by tests
  — production never moved stats across, and ``get_total_llm_stats``
  always summed both for display.
- Drop ``record_llm_usage(agent_id=...)``: argument was unused, and
  the per-call ``bucket=`` knob is gone with the buckets.

run_config_factory:
- Drop unused ``parallel_tool_calls``, ``tool_choice`` parameters
  from ``make_run_config`` — no caller ever overrode them.
- Drop ``agent_name`` from ``make_agent_context`` — set into the
  context dict but no consumer ever read it; the bus's ``names`` map
  is the source of truth.

Wire reasoning_effort through:
- ``Config.get("strix_reasoning_effort")`` is now actually plumbed
  to ``make_run_config`` from ``entry.py``. Previously the env var
  was advertised but never consumed.

Multi-agent graph tools:
- Replace six copies of
  ``inner = ctx.context if isinstance(ctx.context, dict) else {}``
  with a single ``_ctx(ctx)`` helper.

Todo tools:
- Lift the duplicated ``priority_order`` / ``status_order`` dicts
  to module-level ``_PRIORITY_RANK`` / ``_STATUS_RANK`` and replace
  both inline sort lambdas with ``_todo_sort_key``.

Notes tools:
- Delete ``append_note_content`` (and its test): docstring claimed
  it was for an "agents-graph wiki-update hook on agent_finish" that
  was never wired up. Pure dead public API.

Style:
- Drop the ``del ctx`` no-ops from notes / reporting / web_search
  tools. ``ARG001`` is already silenced project-wide for tool
  modules; the ``del`` was cargo-culted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 12:44:48 -07:00
0xallam e4be5f9588 docs: restore tool guidance into docstrings, drop prompt tool-format boilerplate
Port the prose guidance that previously lived in the deleted
*_actions_schema.xml files into per-tool docstrings, so the SDK's
auto-generated function schema carries the same domain knowledge
(HTTPQL syntax, Caido sitemap kinds, browser persistence/JS rules,
agent specialization caps, customer-facing report rules, CVSS/CWE
guidance, etc.) without any custom prompt scaffolding.

Strip the <tool_usage> block from system_prompt.jinja — XML format
guidance, the "CRITICAL RULES" 0-8 list, and the </function>
closing-tag reminder all contradicted the SDK's native JSON
function-calling protocol.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:48:41 -07:00
0xallam dc9b9f5f9c refactor: inline non-sandbox actions, strip registry, drop schemas
Cleanup pass after the migration:

#1 Inline ``*_actions.py`` into wrapper ``tool[s].py`` for the
non-sandbox tools (think, todo, notes, reporting, web_search,
finish_scan). One file per tool family now. Helpers + public
function bodies live alongside the ``@strix_tool``-decorated
wrappers that call them.

For notes, the sync helpers are renamed to ``_create_note_impl`` /
``_list_notes_impl`` / etc. so the public names ``create_note`` /
``list_notes`` / etc. can be the FunctionTool instances the agent
factory imports. ``append_note_content`` (used by the agents-graph
wiki-update hook) calls the impl helpers directly.

#2 Delete ``strix/tools/_state_adapter.py``. The ``AgentStateAdapter``
shim only existed to feed legacy ``*_actions.py`` functions a
``state.agent_id`` they could read. With the actions inlined, the
wrappers read ``ctx.context['agent_id']`` directly.

#3 Strip ``strix/tools/registry.py`` from ~250 LOC to ~110.
Deleted: XML schema loading, ``_parse_param_schema``,
``get_tools_prompt``, ``get_tool_param_schema``, ``needs_agent_state``,
``should_execute_in_sandbox``, ``validate_tool_availability`` — all
for the host-side legacy dispatcher path. Kept the ``register_tool``
decorator (sandbox side), ``get_tool_by_name``, ``get_tool_names``,
``tools`` list, ``clear_registry``.

The Jinja prompt template's ``{{ get_tools_prompt() }}`` injection
is dropped — the SDK auto-generates tool descriptions from function
signatures, so the legacy XML tool block was redundant and stale.

#4 Delete every ``*_actions_schema.xml`` (12 files). They were read
by the now-removed ``_load_xml_schema`` to build the legacy prompt's
tool descriptions. No consumer remains.

Side fixes:
- ``reporting_renderer.py`` updated to import ``_parse_*_xml`` from
  the new location with leading underscore.
- ``test_local_tools.py``, ``test_notes_jsonl_concurrency.py``,
  ``test_notes_wiki.py`` updated to point at the new module paths
  and call the ``_*_impl`` sync helpers.

Tests: 279/279 passing. ~1500 LOC of action files moved into the
tool wrappers; ~140 LOC of registry boilerplate removed; ~400 lines
of dead XML deleted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:26:02 -07:00
0xallam d8881498ee refactor: nuke legacy harness, drop sdk_ prefixes
The SDK harness is the only path now; legacy host-side code is gone.
File names no longer carry the ``sdk_`` distinction.

Deleted legacy host-side modules:
- strix/agents/StrixAgent/ (template moved to strix/agents/prompts/)
- strix/agents/base_agent.py, state.py
- strix/llm/llm.py, config.py
- strix/runtime/docker_runtime.py, runtime.py
- strix/tools/executor.py, agents_graph/agents_graph_actions.py
- strix/interface/sdk_dispatch.py + the env-flag dispatch in cli.py

Renamed (drop ``sdk_`` prefix):
- strix/sdk_entry.py → strix/entry.py
- strix/agents/sdk_factory.py → strix/agents/factory.py
- strix/agents/sdk_prompt.py → strix/agents/prompt.py
- strix/tools/<x>/<x>_sdk_tool[s].py → strix/tools/<x>/tool[s].py
- strix/tools/_legacy_adapter.py → strix/tools/_state_adapter.py
- ``_legacy`` aliases inside the wrappers → ``_impl``

CLI + TUI now call ``run_strix_scan`` directly — they build the
sandbox image / sources_path locally and rely on
``session_manager.cleanup`` (called inside ``run_strix_scan``'s finally)
for teardown. Three TUI handlers that reached into legacy multi-agent
globals (``_agent_instances``, ``send_user_message_to_agent``,
``stop_agent``) are now no-ops with a TODO; reconnecting them to the
``AgentMessageBus`` is a follow-up.

Tracer.get_total_llm_stats no longer reaches into the deleted
``agents_graph_actions`` globals — the orchestration hooks now feed the
tracer via ``Tracer.record_llm_usage`` (live + completed buckets).
finish_scan's ``_check_active_agents`` and load_skill's runtime
``_agent_instances`` reach-in are no-op stubs; the
``AgentMessageBus`` is the source of truth post-migration.

llm/utils.py rewritten to keep only the streaming-parser helpers
(``normalize_tool_format``, ``parse_tool_invocations``,
``fix_incomplete_tool_call``, ``format_tool_call``, ``clean_content``).
``STRIX_MODEL_MAP`` moved to ``llm/multi_provider_setup.py`` (its only
remaining caller).

Per-file ruff ignores added for legacy interface modules (TUI / main /
CLI / utils / streaming_parser / tool_components) and tracer.py —
pre-existing PLC0415/BLE001/PLR0915 patterns are out of scope.

Tests: 287/287 passing. Renamed test files to drop ``sdk_`` prefix.
``test_tracer.py::test_get_total_llm_stats_aggregates_live_and_completed``
rewritten to feed ``Tracer.record_llm_usage`` instead of legacy globals.
Test file annotations added so pre-commit's strict mypy passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:30:23 -07:00
0xallam 6e5d96af34 feat(migration): phase 2.1-2.3 — sandbox dispatch + thin slice tool wrappers
Phase 2.1 — sandbox dispatch helper:
- strix/tools/_sandbox_dispatch.py: post_to_sandbox() centralizes the
  host->container HTTP wire format. Connect=10s, read=150s timeouts mirror
  legacy executor.py. 50 MB response cap (C18) prevents OOM from a runaway
  tool. All errors surface as {"error": str} so the model can recover
  instead of the run dying.

Phase 2.2 — C6 lock-protected JSONL writes:
- strix/tools/notes/notes_actions.py: notes.jsonl appends are now wrapped
  in _notes_lock so concurrent agents can't interleave half-written lines.
  Regression test in test_notes_jsonl_concurrency.py verifies 1000 parallel
  writes produce exactly 1000 valid JSON lines.

Phase 2.3 — thin-slice SDK wrappers (think + todo + notes):
- strix/tools/_legacy_adapter.py: LegacyAgentStateAdapter shim — exposes
  just enough surface (.agent_id) for legacy tools that close over
  agent_state, sourced from ctx.context['agent_id'].
- strix/tools/thinking/thinking_sdk_tools.py: 1 tool (think).
- strix/tools/todo/todo_sdk_tools.py: 6 tools (create/list/update/done/
  pending/delete) with bulk-form preserved.
- strix/tools/notes/notes_sdk_tools.py: 5 tools (create/list/get/update/
  delete) with asyncio.to_thread around the lock-protected file I/O.

Tests: 22 new tests pass (10 sandbox dispatch + 2 concurrency + 10 SDK
local). Full suite still green.

Per-file ruff ignores added for SDK wrapper files: TC002 (RunContextWrapper
must be runtime-importable because the SDK calls get_type_hints() to
derive the JSON schema) and PLR0911 (sandbox dispatch's 10 short-circuit
returns are intentional, each a distinct documented failure mode).

Refs: PLAYBOOK.md §3.4, AUDIT_R3.md C6/C18.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 00:13:34 -07:00
0xallam 78b6c26652 enhance todo tool prompt 2025-12-15 10:26:59 -08:00
0xallam 2899021a21 chore(todo): encourage batched todo operations
Strengthen schema guidance to batch todo creation, status updates, and completions while reducing unnecessary list refreshes to cut tool-call volume.
2025-12-15 07:41:33 -08:00
0xallam 37c8cffbe3 feat(tools): add bulk operations support to todo tools
- update_todo: add `updates` param for bulk updates in one call
- mark_todo_done: add `todo_ids` param to mark multiple todos done
- mark_todo_pending: add `todo_ids` param to mark multiple pending
- delete_todo: add `todo_ids` param to delete multiple todos
- Increase todo renderer display limit from 10 to 25
- Maintains backward compatibility with single-ID usage
- Update prompts to keep todos short-horizon and dynamic
2025-12-14 20:31:33 -08:00
Ahmed Allam 2b926c733b feat(tools): add dedicated todo tool for agent task tracking (#196)
- Add new todo tool with create, list, update, mark_done, mark_pending, delete actions
- Each subagent has isolated todo storage keyed by agent_id
- Support bulk todo creation via JSON array or bullet list
- Add TUI renderers for all todo actions with status markers
- Update notes tool to remove priority and todo-related functionality
- Add task tracking guidance to StrixAgent system prompt
- Fix instruction file error handling in CLI
2025-12-14 22:16:02 +04:00