Commit Graph

49 Commits

Author SHA1 Message Date
Larsen Cundric 49fb16b693 fix(BrowserError): drop trailing 'during: None' when no event is attached
BrowserError.__str__ unconditionally appended 'during: <while_handling_event>',
so any error with details set but no event (e.g. ScrollToText 'Text not found',
click failures raised from action handlers) renders as

    Text not found: "X" ({'text': 'X'}) during: None

The 'during: None' tail is logging noise that surfaces ~1.6k times/day in
prod (Datadog: status:error env:production '"\'\}) during: None"', 24h).

Build the string from the parts that are actually populated, instead of
two mutually-exclusive branches that always print one of them.
2026-05-05 16:04:59 +00:00
Magnus Müller 9e93a92ba4 Include popup msg into browser state 2025-10-29 21:18:20 -07:00
Magnus Müller b678f27a3e Migration from highlight index to backendnode id 2025-10-21 21:02:28 -07:00
Magnus Müller 9caa248316 Merge code7 branch changes 2025-10-21 18:55:54 -07:00
mertunsall 09c27fc2cc dont default 2025-08-30 21:08:26 +02:00
mertunsall b177f3166e better exception handling from browser events -> LLM 2025-08-30 20:50:42 +02:00
Nick Sweeting 140346b99e fix clicking and typing in iframes 2025-08-19 18:15:04 -07:00
Nick Sweeting 76512fbfdd fix Failed to build DOM tree: cannot reuse already awaited coroutine 2025-08-16 17:58:45 -07:00
Nick Sweeting 8d86832095 fix SwitchTab by using TargetID for all tab_ids 2025-08-15 16:45:27 -07:00
Nick Sweeting 7a0889cf20 formatting 2025-08-12 04:14:53 -07:00
Nick Sweeting d1e569eb31 raise BrowserError exceptions instead of returning success=False 2025-08-12 04:14:53 -07:00
Nick Sweeting 328a4b5be4 Merge branch 'feature/new-dom-extraction-layer' into local-remote-split 2025-08-05 03:12:48 -07:00
Nick Sweeting 032f314a41 Merge branch 'main' into local-remote-split 2025-08-04 19:44:05 -07:00
Gregor Žunič 286b5ec801 Merge commit '90fa344b599d55c2984bd926161acfecbb681d8c' into feature/new-dom-extraction-layer 2025-08-02 14:34:20 -07:00
Gregor Žunič 0aa3d54233 Merge commit '9b882faa10c54be5a870cc5d8abd6ddb19d0682f' into feature/new-dom-extraction-layer 2025-08-02 11:41:28 -07:00
mertunsall 9bd0036223 separate agent history and state 2025-08-02 15:48:49 +01:00
Nick Sweeting 2b12eb6542 wip 2025-07-31 03:31:32 -07:00
Nick Sweeting ae479d6330 add storage state events and show recent events summary to llm 2025-07-31 00:54:53 -07:00
Nick Sweeting 09b99c8afc tell the LLM when page loading was aborted so it knows to use wait tool 2025-07-29 13:16:46 -07:00
Nick Sweeting c86cd96274 Merge branch 'main' into claude-issue-2497-20250721-0633 2025-07-24 22:20:15 -07:00
Cursor Agent 0778166109 Add PDF viewer detection and handling in browser state summary
Co-authored-by: mailmertunsal <mailmertunsal@gmail.com>
2025-07-24 15:44:19 +00:00
claude[bot] ba12148b19 refactor: consolidate PLACEHOLDER_4PX_SCREENSHOT constant
- Move 4px placeholder screenshot constant to browser_use.browser.views
- Update all references to use the single definition
- Fix all lint errors and formatting issues

Co-authored-by: Nick Sweeting <pirate@users.noreply.github.com>
2025-07-21 22:28:01 +00:00
Gregor Žunič 7bf17064f6 started migration to pure cdp 2025-07-11 14:19:22 +02:00
Gregor Žunič 779196d74c nicer types 2025-07-10 22:28:53 +02:00
Gregor Žunič 8a2eda1828 wip, started migrating to pure cdp 2025-07-09 12:22:21 +02:00
Daniel T. c5a3c0cf2e Refines dynamic page state calculations
Replaces static page statistics with dynamic calculations for
viewport and scroll position metrics. Simplifies code logic in
scroll handling by removing unnecessary defaults for page
scrolling. Improves readability and reliability of page
scroll operations by requiring explicit num_pages parameter.

Enhances page-view consistency and user interaction handling.
2025-07-07 21:22:12 +02:00
Daniel T. 92bd24f1a0 Standardizes string quotes and cleans up whitespace
Replaces double quotes with single quotes for consistency in
string declarations. Removes unnecessary trailing whitespace
across various files to improve code readability and maintain
a uniform style.
2025-07-07 18:57:30 +02:00
Daniel T. bf878db2ab Enhances page info in browser state
Adds comprehensive page information including viewport and total
page dimensions, detailed scroll position, and page statistics.
Updates message prompts to utilize this data, improving content
presentation and potential user actions.

Relates to enhanced scrolling feature.
2025-07-07 18:47:50 +02:00
Nick Sweeting a17ee667d8 dont repr screenshot in terminal 2025-05-22 07:01:10 -07:00
Nick Sweeting 86ec6a78b0 new BrowserSession and BrowserProfile design, renamed BrowserState to BrowserStateSummary and more 2025-05-20 02:31:54 -07:00
Nick Sweeting 23252192a0 remove non-functional group tabs code 2025-05-04 19:40:27 +08:00
Nick Sweeting 2be4ba4f70 more pyupgrade changes 2025-05-02 20:50:21 +08:00
Nick Sweeting fb6fa259a8 apply ruff safe fixes 2025-03-28 18:11:36 -07:00
Nick Sweeting d95527ab27 Merge branch 'main' into enhancement/add_tabGroups_chrome 2025-03-25 16:34:08 -07:00
Nick Sweeting 73369f910c fix discovery of elements that have both shadow and non shadow child nodes 2025-03-24 14:28:00 -07:00
SwapnilSonker 92fe9212a9 #544 issue resolved , used chrometabs api 2025-02-08 22:48:46 +05:30
magmueller e7c49bfcac Error handling for restricted domains 2025-01-23 16:53:23 -08:00
magmueller 66ea4a3509 Restric urls 2025-01-23 15:44:14 -08:00
magmueller 2bd6e20953 Include how many pixels are above / below 2025-01-20 10:12:12 -08:00
magmueller 60fbbcae93 Split HistoryTreeProcessor in views and service 2024-12-03 09:26:17 +01:00
magmueller c581a1b251 First draft multi-action runs 2024-12-01 21:00:22 +01:00
magmueller 9dfc1f3faf Seperate class for browserhistory - fixed type errors 2024-11-28 23:59:49 +01:00
magmueller 519881a7b3 Rerun works 2024-11-28 21:10:25 +01:00
magmueller de7e38b75c History is saved and loaded 2024-11-28 19:37:18 +01:00
Gregor Žunič 829ed138fd clicking version - still missing iframe/shadow elements penetration 2024-11-25 21:45:05 +01:00
Gregor Žunič b0c390f2c0 fixed multi tab management, clicking timeouts, general bugfixes 2024-11-20 17:33:49 +01:00
Gregor Žunič 155f49acc4 started the switch from selenium to playwright 2024-11-18 20:38:00 +01:00
Gregor Žunič 89c63fdd63 Added custom actions registry and fixed extraction layer (#20)
* Validator

* Test mind2web

* Cleaned up logger

* Pytest logger

* Cleaned up logger

* Disable flag for human input

* Multiple clicks per button

* Multiple clicks per button

* More structured system prompt

* Fields with description

* System prompt example

* One logger

* Cleaner logging

* Log step in step function

* Fix critical clicking error - wrong argument used

* Improved thought process of agent

* Improve system prompt

* Remove human input message

* Custome action registration

* Pydantic model for custom actions

* Pydantic model for custome output

* Runs through, model outputs functions, but not called yet

* Work in progress - description for custome actions

* Description works, but schema not yet

* Model can call the right action - but is not executed

* Seperate is_controller_action  and is_custom_action

* Works! Model can call custom function

* Use registry for action, but result is not feed back to model

* Include result in messages

* Works with custom function - but typing is not correct

* Renamed registry

* First test cases

* Captcha tests

* Pydantic for tests

* Improve prompts for multy step

* System prompt structure

* Handle errors like validation error

* Refactor error handling in agent

* Refactor error handling in agent

* Improved logging

* Update view

* Fix click parameter to index

* Simplify dynamic actions

* Use run instead of step

* Rename history

* Rename AgentService to Agent

* Rename ControllerService to Controller

* Pytest file

* Rename get state

* Rename BrowserService

* reversed dom extraction recursion to while

* Rename use_vision

* Rename use_vision

* reversed dom tree items and made browser less anoying

* Renaming and fixing type errors

* Renamed class names for agent

* updated requirements

* Update prompt

* Action registration works for user and controller

* Fix done call by returning ActionResult

* Fix if result is none

* Rename AgentOutput and ActionModel

* Improved prompt Passes 6/8 tests from test_agent_actions

* Calculate token cost

* Improve display

* Simplified logger

* Test function calling

* created super simple xpath extraction algo

* Tests logging

* tiny fixes to dom extraction

* Remove test

* Dont log number of clicks

* Pytest file

* merged per element js checks

* Check if driver is still open

* super fast processing

* fixed agent planning and stuff

* Fix example

* Fix example

* Improve error

* Improved error correction

* New line for step

* small type error fixes

* Test for pydantic

* Fix line

* Removed sample

* fixed readme and examples

---------

Co-authored-by: magmueller <mamagnus00@gmail.com>
2024-11-15 21:42:02 +01:00
Gregor Žunič 68201df624 src -> browser_use 2024-11-06 18:18:00 +01:00