Commit Graph

124 Commits

Author SHA1 Message Date
laithrw a511ba2f1e Merge branch 'main' into 3619 2026-02-03 17:32:46 -05:00
Saurav Panda e66eac8a37 Improve Markdown Chunker 2026-02-02 00:05:09 -08:00
Saurav Panda 6878bd1ca4 handle autocomplete fields 2026-01-30 11:04:00 -08:00
Laith Weinberger f3f82689dd fix docstring incorrectly referencing y_position instead of pages + show generic hint 2026-01-25 13:19:30 -05:00
Laith Weinberger 9027991bae when detecting iframe elements, tell LLM exactly how far to scroll to reach each
Get pixel location of each iframe element that's out of the viewport, convert from pixels to number of page lengths, and provide that to the LLM as context
2026-01-25 13:11:37 -05:00
Laith Weinberger eb8e1c1d59 fix(#3619): add hidden content hints for iframes in DOM context
Elements in iframes beyond the 1000px viewport threshold are filtered from selector_map, causing agents to scroll blindly to find them. Add hints to the LLM context indicating hidden content
2026-01-25 12:19:57 -05:00
Laith Weinberger 1794849886 don't mutate page
instead of writing markers to DOM, collect element references in JS, resolve them to internal Chrome node IDs via CDP, and store those IDs in Python
2026-01-24 16:36:14 -05:00
Saurav Panda 2c06eabf5f Added ax-name fallback for history rerun. 2026-01-06 12:07:10 -08:00
mertunsall 1779e9717d improve hashing of elements using cascading hash 2025-12-17 00:55:27 -08:00
Saurav Panda e9c047be92 include ax_name for rerun history element hash 2025-12-16 17:05:48 -08:00
Magnus Müller 4f9ba966cb Expected date format + scroll container 2025-10-30 11:32:27 -07:00
Magnus Müller acac902a87 data-date-forma 2025-10-30 00:02:33 -07:00
Magnus Müller f1e3fbcd0c More forms for datepicker 2025-10-29 20:17:20 -07:00
Magnus Müller 8ee357495c Include accept 2025-10-29 20:01:11 -07:00
Magnus Müller b678f27a3e Migration from highlight index to backendnode id 2025-10-21 21:02:28 -07:00
Magnus Müller 9caa248316 Merge code7 branch changes 2025-10-21 18:55:54 -07:00
Magnus Müller 730019471d Fix: filter shadow roots from children nodes, ensuring clean separation in the DOM tree structure. Update EnhancedDOMTreeNode to make a copy of children_nodes to prevent mutation of the original list. 2025-10-13 19:36:13 -07:00
Magnus Müller 90e27e07d1 Works 2025-09-18 11:10:57 -07:00
Magnus Müller 6244c97876 Include information about dom metadata in prompt 2025-09-17 11:32:11 -07:00
Magnus Müller 30eb8fe36f Linter 2025-09-17 11:07:05 -07:00
Magnus Müller ca469caa31 The __hash__ method inefficiently recreates the static_attributes set on every call. 2025-09-17 11:06:31 -07:00
Magnus Müller 5d6c199f4f Update browser_use/dom/views.py
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2025-09-17 11:04:31 -07:00
Magnus Müller 48c75ffd3d Update browser_use/dom/views.py
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2025-09-17 11:04:23 -07:00
Magnus Müller 40ce0d382d Enhance static attributes list in EnhancedDOMTreeNode for improved DOM serialization 2025-09-17 09:51:34 -07:00
Magnus Müller 9618835c97 fix element hash 2025-09-17 09:48:17 -07:00
Magnus Müller d63292b3f2 Fix bounds handling in DOMInteractedElement to return None if bounds are not set; ensure proper closure of agent service after execution. 2025-09-16 22:38:49 -07:00
Magnus Müller 064c5ba016 Fix rerun with inital actions 2025-09-16 21:46:45 -07:00
Gregor Žunič c2eede8e20 paint order filtering 2025-09-07 20:56:48 -07:00
Gregor Žunič b141034ff3 Merge branch 'main' into c/better-frameset-detection 2025-08-30 11:15:58 -07:00
Magnus Müller a31ae9dffb Refactor bounding box drawing and enhance LLM text representation
- Removed the `_raise_if_stopped_or_paused` method from the `Agent` class to streamline functionality.
- Updated `draw_enhanced_bounding_box_with_text` to improve font size and padding for better visibility of index boxes.
- Introduced `get_meaningful_text_for_llm` method in `EnhancedDOMTreeNode` to provide more relevant text for LLM processing.
- Enhanced `llm_representation` method with observability features for better debugging.
2025-08-29 21:30:11 -07:00
Gregor Žunič 0913a8a345 fixed FRAME elements 2025-08-29 20:51:39 -07:00
Nick Sweeting 8d86832095 fix SwitchTab by using TargetID for all tab_ids 2025-08-15 16:45:27 -07:00
Magnus Müller e15a8a216e hot-fix/scroll 2025-08-12 23:03:09 -07:00
Magnus Müller 403803d48d Added ax attributes like checked 2025-08-12 17:27:06 -07:00
Magnus Müller c88599a749 Merge commit 'd8f147fac486f26cf951a88d8b0b8f6d2a31fb02' into local-remote-split-early-error 2025-08-12 15:11:55 -07:00
Mert Unsal 3211ae4ab8 Mert/reduce extracted elems (#2659)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Reduced the number of extracted interactive elements by filtering out
child nodes fully contained within certain parent element bounds,
improving accuracy and reducing noise in the serialized DOM output.

- **Refactors**
- Updated clickable element detection to exclude most SVG elements
unless they have explicit interactive attributes.
- Added bounding box filtering logic to skip redundant child elements
inside propagating parents like buttons and links.
- Marked excluded nodes in the tree for easier debugging and
serialization.

<!-- End of auto-generated description by cubic. -->
2025-08-12 20:34:18 +01:00
Magnus Müller d8f147fac4 Merge remote-tracking branch 'origin/local-remote-split' into local-remote-split-scroll-info 2025-08-12 12:32:45 -07:00
Magnus Müller d086e77ecb Scroll info works 2025-08-12 12:27:21 -07:00
Gregor Žunič a29d4ac8bd reverted multi-act back to branch_path_hashes 2025-08-12 12:04:23 -07:00
mertunsall cd71662ab8 some optimizations to extraction 2025-08-12 19:39:09 +01:00
Magnus Müller cb4d206a48 Update empty DOM tree message to indicate potential loading delay 2025-08-10 18:03:28 -07:00
Magnus Müller 9dc4f07233 Increase tag text from 25 to 100 2025-08-09 17:46:36 -07:00
Nick Sweeting db4c4e9b49 fix duplicate typed keystrokes 2025-08-08 04:13:44 -07:00
Nick Sweeting e7fb8aa376 try js click before quads click 2025-08-08 03:37:04 -07:00
Nick Sweeting 4826041154 new cdp_session singleton for current page 2025-08-07 21:48:57 -07:00
Nick Sweeting a8817ff65f click is working again 2025-08-07 14:43:58 -07:00
Nick Sweeting 058b20d7e7 fix click return values 2025-08-07 14:32:55 -07:00
Nick Sweeting 936f7d7d35 more playwright tearout 2025-08-06 04:06:34 -07:00
Nick Sweeting 13ce936128 add session.py cdp helpers 2025-08-05 18:30:53 -07:00
Nick Sweeting 46dd9bd584 wip 2025-08-05 17:45:48 -07:00