- Reduced sleep duration in the watchdog from 0.5s to 0.25s for efficiency.
- Added logic to track if a new tab was opened and included this information in the returned click metadata.
- Updated ClickElementAction to handle optional Ctrl key behavior and reflect new tab opening in memory logging.
- Revised system prompt descriptions to clarify the provision of screenshots and bounding boxes for interactive elements.
- Enhanced bounding box positioning logic in `python_highlights.py` to improve visibility and avoid content blocking.
- Updated screenshot logging to reflect dynamic filename usage.
- Updated .gitignore to exclude PNG files.
- Refactored the screenshot saving logic in `python_highlights.py` to use an environment variable for the filename and added async file writing for improved performance.
- Adjusted font size and padding to fixed values for consistent appearance across different screen sizes.
- Updated the instantiation of `DomService` to include `cross_origin_iframes` parameter from the browser session profile, improving its configuration for handling cross-origin iframes.
- Increased font size in `draw_enhanced_bounding_box_with_text` from 30 to 36 for improved readability.
- Adjusted padding from 8 to 6 to accommodate the larger font while maintaining visual clarity.
- Enhanced text positioning logic to prevent clipping and ensure proper alignment within the bounding box.
- Removed the `_raise_if_stopped_or_paused` method from the `Agent` class to streamline functionality.
- Updated `draw_enhanced_bounding_box_with_text` to improve font size and padding for better visibility of index boxes.
- Introduced `get_meaningful_text_for_llm` method in `EnhancedDOMTreeNode` to provide more relevant text for LLM processing.
- Enhanced `llm_representation` method with observability features for better debugging.
- Introduced `filter_highlight_ids` option in `BrowserProfile`, `BrowserSession`, and `create_highlighted_screenshot` to control the visibility of element IDs based on text length.
- Updated `draw_enhanced_bounding_box_with_text` to utilize the new filtering logic for enhanced visual clarity.
- Changed the color of text areas from yellow to orange for improved visibility in the UI.
- Increased font size in `draw_enhanced_bounding_box_with_text` from 35 to 40 for better readability.
- Adjusted padding in `draw_bounding_box_with_text` from 3 to 5 to enhance spacing around text elements.
- Eliminated the `display_highlights_on_screen` option from `BrowserProfile` and `BrowserSession` to streamline the highlighting functionality.
- Updated the `draw_enhanced_bounding_box_with_text` function to increase font size for better visibility and reduced padding for a more compact layout.
- Removed associated screenshot overlay logic in `DOMWatchdog` to reflect the removal of the display highlights feature.
- Updated `draw_enhanced_bounding_box_with_text` function to draw dashed bounding boxes for better visibility.
- Increased the size of index text and padding for improved prominence.
- Adjusted text positioning logic to ensure proper placement within or outside the bounding box based on available space.
- Enhanced background rectangle drawing for better contrast and visibility.
- Introduced `display_highlights_on_screen` option in `BrowserProfile` and `BrowserSession` to control the visibility of highlights directly on the browser screen.
- Implemented `draw_enhanced_bounding_box_with_text` function to improve the visual representation of highlighted elements with larger indices and solid borders.
- Updated screenshot overlay logic in `DOMWatchdog` to utilize the new highlighting feature, ensuring better visibility of interactive elements during browser sessions.
- Updated key code mappings for special characters to reflect correct usage with modifiers.
- Enhanced text field clearing method to use platform-specific modifiers (Cmd for macOS, Ctrl for others) for a more human-like interaction.
- Removed unnecessary `windowsVirtualKeyCode` assignments for printable characters to prevent incorrect virtual key code usage.
These changes improve the accuracy of character input handling and enhance the robustness of text field interactions.
- Added type hint for CDPSession in the _focus_element_simple method.
- Enhanced logging for focus attempts, including exception details.
- Reduced sleep duration in scrollIntoViewIfNeeded for better performance.
- Updated text clearing logic to ensure it only occurs after successful focus.
These changes enhance the robustness of element interaction and improve debugging capabilities.
- Added logging for new elements detected during actions in the Agent class.
- Implemented a human-like text field clearing method in DefaultActionWatchdog, utilizing Ctrl+A and Backspace.
- Improved focus handling for label elements, ensuring they are only interactive if they do not have a 'for' attribute.
- Updated clickable element detection logic to account for labels pointing to inputs.
These changes improve the robustness of user interactions and enhance debugging capabilities.
Magnus Müller