Commit Graph

272 Commits

Author SHA1 Message Date
MagMueller 215572442d fix: sanitize evaluate source history 2026-09-04 17:20:03 -07:00
MagMueller 5145852597 fix: retain evaluate source in agent history 2026-09-04 17:20:03 -07:00
MagMueller 571947dce9 fix: preserve evaluate JavaScript source 2026-09-04 17:20:03 -07:00
MagMueller d828d83d7a docs(tools): list jpeg in binary image support 2026-09-04 12:52:09 -07:00
MagMueller 257bcfd5f2 filesystem: let write_file create tiny binary images (png/gif/jpg/webp) from base64
Upload-validation flows were unwinnable: the agent's file tools are text-only, so
'upload a photo/gif' tasks could never be attempted. write_file now accepts small
image extensions where content is the base64 of a valid image (a 1x1 PNG is ~92
base64 chars, ~23 tokens); the bytes are decoded and written to disk so the file
is a real, uploadable image.

- Base64BinaryFile writes decoded BYTES (not base64 text) to disk; read() returns
  a '[binary png file, N bytes]' stub so base64 never enters the prompt (describe()
  runs every step).
- Strict base64 decode: invalid content is rejected with an error and no file is
  created, so a corrupt 'image' can't slip through upload's file_size>0 check.
- Registered in _file_types (so upload_file.get_file resolves it by basename) and
  from_state (so restored sessions keep it); pruned from UNSUPPORTED_BINARY_EXTENSIONS.
- Text write_file, PDF/DOCX rendering, and other binary rejections unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-04 12:51:18 -07:00
Magnus Müller f3a6becbf0 Merge branch 'main' into fix/switch-tab-error-reporting 2026-09-02 17:34:56 -07:00
郑耀翔 9bfb1d3258 fix(registry): replace sensitive placeholders in tuples 2026-08-28 09:30:24 +08:00
Faseeh c4e314072e address review feedback: preserve real cause via raise_if_any, fix pyright
- event_result(raise_if_any=True) so a SwitchTabEvent handler failure
  surfaces its actual cause through the except block instead of
  becoming a generic 'produced no result' message (only one handler
  is ever registered for SwitchTabEvent, so this is safe).
- Test file used a dynamically-built ActionModel that pyright can't
  see the fields of; switched to a statically declared _TabActionModel
  subclass, matching the pattern used elsewhere in this file.
2026-08-19 23:02:33 +05:00
Faseeh c45b850231 fix(tools): report failed tab switches as errors instead of silent success
switch() returned a non-error ActionResult on both of its failure paths
(a stale/unknown tab_id, and a SwitchTabEvent that produced no result),
so callers had no way to tell a failed switch from a real one. The false
'Switched to tab #...' claim was written into long_term_memory, so
subsequent agent steps reasoned from a tab that was never actually
reached.

Raise BrowserError on both failure paths instead, following the same
convention upload_file already uses in this file. The error message
preserves the actual underlying cause instead of a generic string, so
ActionResult.error carries actionable information back to the agent.

Adds regression tests covering the failing tab_id case and confirming
the happy path is unaffected.
2026-08-19 22:46:24 +05:00
Saurav Panda 40010b3680 Merge branch 'main' into fix/issue-5365-coordinate-click-exception 2026-08-03 15:38:26 -07:00
santosh-agebold c9dae1b6bf fix: include exception details in coordinate click and add missing error logs 2026-08-03 13:13:40 -07:00
Magnus Müller f7d8b34272 Merge branch 'main' into fix/search-google-action-alias 2026-08-02 19:15:34 -07:00
LHMQ878 ff8f6f10d5 fix(tools): restore SearchGoogleAction back-compat alias
`browser_use/tools/views.py:57` reads

    # Backward compatibility alias
    SearchAction = SearchAction

which is a no-op self-assignment, so the pre-rename name is not exported.
`SearchGoogleAction` (present through 0.7.x) has been unimportable since
`SearchAction` was introduced in 0.9.0 — `from browser_use.tools.views
import SearchGoogleAction` raises ImportError.

The sibling alias two blocks down does it correctly
(`GoToUrlAction = NavigateAction`), and the test suite still refers to the
old name in a comment (`# Test SearchGoogleAction`), so the alias was
clearly intended.

Also add a regression test asserting both aliases resolve to their current
classes.
2026-07-28 17:07:26 +08:00
abhay-codes07 5ebfd97ecc fix(registry): don't expose domain-restricted actions on an empty URL
ActionRegistry._match_domains returned True when the URL was empty (`not url`),
so a domain-restricted action was offered on a blank/unknown page URL. This
failed open and was inconsistent with the `page_url is None` path, which hides
restricted actions. An empty `page_url` is reachable (a freshly created
about:blank target has `target.url == ''`), so a domain-scoped action could be
exposed on the wrong context.

Only unrestricted actions (`domains is None`) short-circuit to available; a
restricted action with an unknown/empty URL now fails closed.

Add regression tests.
2026-07-12 14:45:24 +05:30
Saurav Panda 782535c34f Keep plain BrowserErrors recoverable: only structured ones bypass the generic handler
An unconditional re-raise sent BrowserErrors without long_term_memory (e.g.
upload_file's failure paths) into Tools.act's handle_browser_error, which
re-raises exactly those — escaping act() as an exception where callers
previously got a recoverable ActionResult(error=...).

Guard the bypass on long_term_memory being present (the exact condition
handle_browser_error formats without re-raising; short_term_memory alone
would still re-raise), and flatten plain BrowserErrors to RuntimeError as
before. Regression test covers the plain-BrowserError path through
tools.act.
2026-07-06 15:04:37 -07:00
Saurav Panda f30c3952c9 Preserve BrowserError's structured memory through execute_action
Registry.execute_action's catch-all handler flattened BrowserError into a
generic 'Error executing action ...' RuntimeError, destroying the structured
short_term_memory/long_term_memory the error carries to steer the LLM's next
action (e.g. the list of available dropdown options when clicking a select).
The 'except BrowserError' branch in Tools.act that formats those memories
into an ActionResult was dead code for any action that let a BrowserError
propagate (upload_file, dropdown_options via event_result, extraction
handlers).

Re-raise BrowserError before the generic handlers so handle_browser_error
becomes the single formatting point again.
2026-07-06 14:52:39 -07:00
Saurav Panda b88fd70481 fix(tools): truncate long URLs in save_as_pdf footer so page numbers stay visible
A flex item defaults to min-width:auto and won't shrink below its content,
so a long/unbroken URL in the default footer template overflowed and pushed
the page / totalPages span off the printable area. Give the url span
min-width:0 + overflow ellipsis so it truncates, and flex-shrink:0 on the
page-count span so it always renders. Adds a long-URL regression test.
2026-06-15 13:54:38 +05:30
Saurav Panda ad8f3a4dd7 feat(tools): print page metadata in save_as_pdf header/footer
The save_as_pdf action now renders page metadata into the PDF margins by
default, matching Chrome's Print dialog: the date in the header and the page
URL plus page numbers in the footer.

- Add display_header_footer (default True), header_template, and
  footer_template params to SaveAsPdfAction.
- Pass displayHeaderFooter + header/footer templates and explicit margins to
  CDP Page.printToPDF so the metadata has room to render (Chrome clips it
  otherwise, and defaults header/footer font-size to 0px).
- Default templates show the date / URL + page numbers; callers can override
  with custom HTML or disable entirely for a clean PDF.
2026-06-11 08:29:41 +05:30
Laith Weinberger 8bca742aba support claude fable 5
- fable for coord clicks
- explicit support
- model refusal response
- fable pricing with 5m/1h cache writes and inference geo multiplier
- anthropic fallback, thinking, output config, and inference geo request support
2026-06-09 13:22:04 -07:00
MagMueller af9d406419 Revert "fix(tools): refuse evaluate() on restricted browser profiles (#4871)"
This reverts commit d6a87c0961, reversing
changes made to eeff4d1984.
2026-05-23 12:13:38 -07:00
Saurav Panda e5f9b46f95 refactor(tools): tighten evaluate() guard comment
Drop the running narrative; the function name + the one-line why is
enough. Error message stays actionable but no longer reiterates the
threat model.

No behavior change; 6/6 tests pass.
2026-05-18 18:42:29 -07:00
Saurav Panda 8f2967a5c0 fix(tools): include prohibited_domains in evaluate guard
SecurityWatchdog enforces prohibited_domains as a navigation restriction
alongside allowed_domains. Without prohibited_domains in the evaluate()
guard, a profile using only the deny-list could still load an allowed page
and have the agent fetch() into a blocked domain via JS.

Add prohibited_domains to the restriction check.
2026-05-18 18:37:12 -07:00
Saurav Panda a7ce680948 fix(tools): refuse evaluate() on restricted browser profiles
The agent's evaluate() action called Runtime.evaluate directly through
CDP. SecurityWatchdog only subscribes to navigation events, so JS
running inside an already-allowed page could fetch() arbitrary internal
URLs, read cookies and localStorage from any allowed origin's context,
and otherwise act as if allowed_domains / block_ip_addresses weren't
configured.

When a profile has allowed_domains or block_ip_addresses set, the
operator has signalled the agent is constrained — exposing an unmediated
JS evaluation primitive contradicts that signal. Refuse evaluate()
outright on such profiles; agents that need JS can run on an unrestricted
profile.

Empty allowed_domains=[] is treated as 'no restriction' elsewhere in the
codebase (e.g. SecurityWatchdog); evaluate() behaves consistently and
does not refuse in that case.
2026-05-18 18:30:04 -07:00
Saurav Panda 65a377c20f fix(tools): only rewrite upload path to FileSystem on local sessions
Addresses codex review on #4865.

For remote (`is_local=False`) sessions, `params.path` is meant to reference
a file on the remote machine. A coincidental basename collision with a
local FileSystem-managed file (e.g. `/tmp/note.md` colliding with a local
managed `note.md`) would silently rewrite the upload to point at the local
file, uploading the wrong file with no indication to the agent.

Gate the FileSystem rewrite on `browser_session.is_local`. On remote
sessions, fall through to the existing pass-through branch that allows
remote-accessible absolute paths.

Add a regression test that exercises the remote-session basename-collision
case and asserts the local FileSystem path never appears in the resolved
upload path.
2026-05-18 15:05:08 -07:00
Saurav Panda a209c5ed38 fix(tools): contain upload_file path inside FileSystem dir
GHSA-j9hj-92j8-jv9h.

The `upload_file` action constructed the absolute upload path by joining
`file_system.get_dir()` with the agent-controlled `params.path`. Because
`FileSystem.get_file()` matches by basename (`os.path.basename` first), an
agent-controlled path like `../note.md` would:

1. Pass `get_file()` lookup if a file named `note.md` exists in the FileSystem.
2. Be naively joined to `data_dir`, producing `data_dir/../note.md` — which
   resolves to a sibling file outside the FileSystem directory.
3. Be uploaded to the browser as the resolved (escaped) file, surfacing
   arbitrary contents from outside `browseruse_agent_data` to whatever file
   input was targeted.

Use the FileSystem-owned `file_obj.full_name` for the join. Add a
`os.path.realpath` containment check as defense in depth; if it ever resolves
outside `data_dir`, refuse the upload.
2026-05-18 13:43:34 -07:00
Saurav Panda 32416bb48c address codex review: validate tools.act(action_timeout=...) override
P2 codex comment on 9a09c4d7: the public `action_timeout` parameter on
tools.act() skipped the same defensive validation that the env-var path
already had. Passing nan made every action time out instantly; inf /
<=0 disabled the guard entirely. Either mode silently defeats the safety
this module exists to provide, especially for callers sourcing timeouts
from runtime config.

Extracted _coerce_valid_action_timeout() (pairs with _parse_env_action_
timeout) and routed the override through it. None / nan / inf /
non-positive all fall back to the env-derived default with a warning.

New test_act_rejects_invalid_action_timeout_override asserts the
fallback by passing bad values and verifying the fast handler actually
executes to completion (which wouldn't happen if nan → immediate
timeout or if inf → hang would leak through).
2026-04-20 17:55:59 -07:00
Saurav Panda d2985dcab9 review: reject non-finite timeouts + restore module after reload tests
Two more issues from automated review on #4711:

1. (P2, Codex) float() accepts 'nan' and 'inf' — both parse successfully
   and bypass the fallback path. 'nan' makes asyncio.wait_for time out
   immediately for every action; 'inf' effectively disables the hang
   guard. Extracted the parse into _parse_env_action_timeout() which
   rejects non-finite and non-positive values (including 0 and negatives)
   with a warning + fallback.

2. (P2, Cubic) The previous reload test left browser_use.tools.service
   pinned at _DEFAULT_ACTION_TIMEOUT_S=45.0 (the last monkeypatch value),
   which would leak into any later test in the same worker. Added a
   _restore_service_module fixture that pops the env var and reloads
   cleanly on teardown.

Expanded test coverage to include 'nan', 'NaN', 'inf', '-inf', '0', '-5'
alongside the existing '' / 'abc' cases — all fall back to 180s.
2026-04-20 15:45:36 -07:00
Saurav Panda 1488a39b7f address PR review: raise default cap + tolerate bad env values
Two issues flagged by automated review on #4711:

1. (P1, Codex) The 90s default was *below* the extract action's intentional
   120s page_extraction_llm.ainvoke timeout (tools/service.py:1096,1172).
   Slow-but-valid extractions against large pages would be truncated into
   timeout errors — a regression. Raised default to 180s, which sits above
   that 120s inner cap with grace.

2. (P2, Cubic + Codex) float(os.getenv('BROWSER_USE_ACTION_TIMEOUT_S', '90'))
   ran at import time. An empty or non-numeric value (common with env
   templating) raised ValueError and prevented browser_use.tools.service
   from importing at all — turning a config typo into a process-wide
   startup failure. Wrapped in try/except with a warning and fallback to
   the hardcoded 180s default.

Tests:
- test_default_action_timeout_accommodates_extract_action — pins the
  default >= 150s so future edits can't silently regress extract.
- test_malformed_env_timeout_does_not_break_import — reloads the module
  with empty / non-numeric env values and asserts it falls back cleanly,
  plus verifies a valid numeric env value still takes effect.
2026-04-20 15:36:15 -07:00
Saurav Panda ce81ada89a fix(tools): enforce per-action timeout to prevent hung event handlers
Individual CDP calls like Page.navigate() have their own 20s timeouts, but
the surrounding event-bus plumbing (await event, event_result()) does not.
When a cloud browser's CDP WebSocket goes silent mid-session, agent handlers
hang indefinitely — agents never emit a step, any outer watchdog eventually
fires, and the run returns with zero history.

Observed in practice: a 170k-task collector run produced 1,090 empty-history
traces (21% of output). 100% hit the 240s outer watchdog; median 582s, max
2214s. Cloud HTTP layer was clean (all 200/201) — hang was entirely in CDP.

Wrap registry.execute_action in asyncio.wait_for with a configurable per-
action cap (default 90s, BROWSER_USE_ACTION_TIMEOUT_S env var or
tools.act(action_timeout=...)). On timeout, the action returns
ActionResult(error=...) so the agent can record the step and recover.

New tests/ci/test_action_timeout.py covers both hung and fast handlers.
Existing tools.act tests (test_multi_act_guards, test_action_blank_page)
still pass.
2026-04-20 15:22:30 -07:00
Laith Weinberger 4476f6e16e fix input clear fallbacks and clarify clear-then-type behavior 2026-04-15 17:31:04 -04:00
Saurav Panda 4a65aea6a9 Merge branch 'main' into auto-switch-new-tab 2026-03-21 17:17:32 -07:00
Laith Weinberger 43b5e4ce1d rm code agent 2026-03-21 02:05:42 -04:00
Saurav Panda e7a05cbe86 fix: auto-switch to new tabs after click and add checkbox toggle fallback
- AGI-569: after any click that opens a new tab, automatically dispatch
  SwitchTabEvent so the agent lands on the new page immediately instead
  of requiring a manual switch step (~877 occurrences)

- AGI-548: for <input type="checkbox/radio">, capture checked state
  before the CDP mouse click and verify it toggled afterward; if
  unchanged (custom-styled or shadow-DOM-backed inputs), fall back to
  JS element.click() and report the final checked state in metadata
  (~1,241 occurrences)
2026-03-20 12:43:58 -07:00
ShawnPana 91c02012c3 merge origin/main into cli, resolve agent.py delete conflict 2026-03-19 21:06:38 -07:00
Saurav Panda c53ff656f5 check root before llm_representation fallback 2026-03-19 20:17:46 -07:00
Saurav Panda 6d86be3a6b fix: detect skeleton screens and retry navigation for blank SPA pages 2026-03-19 18:48:47 -07:00
ShawnPana 694a111fad add upload command to CLI, extract find_file_input_near_element to BrowserSession
- Add `browser-use upload <index> <path>` command for uploading files to
  file input elements via the CLI
- Extract find_file_input_near_element from nested closures in tools/service.py
  to a reusable method on BrowserSession, deduplicating two copies
- Add BrowserWrapper.upload() for the Python REPL
- Resolve file paths to absolute on the client side before sending to daemon
- Update SKILL.md files and README with upload command docs
2026-03-19 17:02:34 -07:00
Saurav Panda cc1cf2b1c7 Merge branch 'main' into fix-image-url-stripping 2026-03-19 15:34:27 -07:00
Saurav Panda 9445a9ff9e fix image URLs stripped from extracted markdown 2026-03-19 15:29:06 -07:00
Saurav Panda cb75bf846a Merge branch 'main' into fix-compaction-hallucination 2026-03-18 17:54:14 -07:00
Saurav Panda cc3cc8fea7 Merge branch 'main' into fix-done-action-hallucination 2026-03-18 16:11:31 -07:00
Saurav Panda 179d526786 added better data grounding in done action 2026-03-18 16:08:00 -07:00
Saurav Panda 0dea9480ee Merge branch 'main' into fix-compaction-hallucination 2026-03-17 15:43:11 -07:00
Saurav Panda 02349f1d9c fix: prevent compaction hallucination 2026-03-17 15:39:35 -07:00
Saurav Panda 73d64d7bb5 remove read long content action 2026-03-12 12:44:10 -07:00
Saurav Panda 990e0ce828 Merge branch 'main' into fix-duplicate-extraction 2026-03-10 16:22:10 -07:00
Saurav Panda ce25884751 fix: prevent duplicate items in multi-page extraction via already_collected parameter 2026-03-10 16:12:46 -07:00
Saurav Panda 53b0239677 recompute the url scheme after refresh 2026-03-10 15:56:21 -07:00
Saurav Panda 1d9049fcad fix: detect empty DOM after navigation and retry before reporting failur 2026-03-10 15:40:19 -07:00
Saurav Panda 95215d6211 Add save_as_pdf agent action for CDP-based page-to-PDF export 2026-02-24 18:35:34 -08:00