- Add ClickCoordinateEvent to handle coordinate-based clicking via CDP
- Replace broken page.mouse.click() with CDP Input.dispatchMouseEvent
- Add safety checks for file inputs, select elements, and print dialogs
- Add get_dom_element_at_coordinates() to BrowserSession for element detection
- Add force parameter to ClickElementAction to optionally skip safety checks
- Update _click_by_coordinate to dispatch ClickCoordinateEvent instead of using mouse
This fixes coordinate clicking functionality by using CDP directly instead of
the broken Actor mouse object, matching the implementation from mert/everything_coordinates.
Previously, when browser launch succeeded after retry using a temporary
directory, the code immediately deleted ALL temporary directories including
the one actively being used by the running browser process. This caused
browser crashes and data corruption when Chrome tried to write to the
deleted user_data_dir during normal operation.
The fix:
- Only clean up unused temporary directories after successful launch
- Preserve the currently used temporary directory until browser cleanup
- Update _temp_dirs_to_cleanup to track only the in-use directory for
proper cleanup during BrowserKillEvent
Updated the logging level for session management methods in BrowserSession to debug for more granular logging. This change applies to the reset, kill, and stop methods, enhancing the clarity of log outputs during browser session operations without cluttering the info logs.
- Added defensive checks in SessionManager to skip target attachment and focus recovery if the browser is shutting down (i.e., _cdp_client_root is None).
- Introduced a connection lock in BrowserSession to prevent concurrent connection attempts, ensuring safe CDP client management.
- Improved logging for session reset and connection handling, providing clearer insights into the browser session state during operations.
- Enhanced error handling when stopping the CDP client and clearing session manager state on initialization failures.
These changes improve the robustness and reliability of session management in the browser environment.
- Adjusted the structure of event handling for 'networkIdle' and 'load' events to improve readability and maintainability.
- Ensured that logging for page readiness is consistently executed within the event handling logic.
- Updated the `create_task_with_error_handling` calls in `DOMWatchdog` to include the `suppress_exceptions` parameter, improving error management during DOM tree building and screenshot capturing.
- This change allows for better handling of exceptions without interrupting the flow of task execution.
- Improved the `create_task_with_error_handling` function to allow for better exception logging and retrieval based on the `suppress_exceptions` parameter.
- Updated `SessionManager` to implement event-driven recovery for stale agent focus, replacing polling with efficient event handling.
- Refactored session retrieval methods to ensure focus validation and recovery are handled automatically.
- Enhanced logging for recovery processes and session management to provide clearer insights into state changes and errors.
- Adjusted various watchdogs to utilize the new session management methods, ensuring they correctly handle focus validation and session retrieval.
- Updated cdp-use dependency to version 1.4.4 for improved functionality.
- Increased max WebSocket frame size to 200MB in BrowserSession to accommodate large DOMs.
Mert Unsal