uses CDP Runtime.evaluate with includeCommandLineAPI to access getEventListeners() and mark elements with click/mousedown/pointerdown listeners before DOM snapshot capture
- Added 'row', 'cell', 'gridcell' to interactive_roles in ClickableElementDetector
- Added 'row', 'cell', 'gridcell' to interactive_ax_roles in ClickableElementDetector
- Verified logic with isolated test case
- Changed the type of the all_frames parameter to allow for None, enabling lazy fetching of cross-origin iframes only when necessary.
- Updated comments to clarify the behavior of all_frames during DOM tree construction.
- Improved the `create_task_with_error_handling` function to allow for better exception logging and retrieval based on the `suppress_exceptions` parameter.
- Updated `SessionManager` to implement event-driven recovery for stale agent focus, replacing polling with efficient event handling.
- Refactored session retrieval methods to ensure focus validation and recovery are handled automatically.
- Enhanced logging for recovery processes and session management to provide clearer insights into state changes and errors.
- Adjusted various watchdogs to utilize the new session management methods, ensuring they correctly handle focus validation and session retrieval.
- Eliminated the caching of target types in SessionManager, simplifying the management of target information.
- Updated various components to directly access target types from the Target objects, enhancing clarity and reducing redundancy.
- Adjusted related methods in BrowserSession, CrashWatchdog, and DomService to reflect the changes in target type retrieval.
- Updated DOMWatchdog to provide a detailed hierarchical breakdown of timing metrics for various operations, improving clarity in performance logging.
- Enhanced DomService to calculate and log specific timing metrics for iframe scrolling, CDP calls, and snapshot processing, contributing to more accurate performance tracking.
- Updated DOMWatchdog to exclude specific sub-timings from total tracked time to avoid double-counting.
- Enhanced DomService to calculate and log overhead timings for both get_dom_tree and get_serialized_dom_tree operations, improving accuracy in performance metrics.
- Introduced a utility function `create_task_with_error_handling` to manage asyncio tasks with proper exception logging and suppression options.
- Updated multiple components, including `SessionManager`, `BrowserSession`, and various watchdogs, to utilize the new error handling mechanism for background tasks.
- Enhanced overall robustness of asynchronous operations by ensuring exceptions are logged and handled appropriately, improving maintainability and debugging capabilities.
- Updated DOM tree retrieval methods to lazily fetch frame hierarchy only when necessary, improving performance for pages without cross-origin iframes.
- Adjusted method signatures to reflect optional frame fetching, enhancing flexibility in DOM service operations.
- Streamlined timing metrics by setting placeholders for timing logs, preparing for future performance enhancements.
- Updated DOM tree retrieval methods to include pre-fetched frame hierarchy, reducing redundant CDP calls and improving performance.
- Adjusted timing logs to provide detailed breakdowns of DOM tree construction, including total time and individual component timings.
- Removed commented-out code related to Python-based highlighting, streamlining the DOMWatchdog class.
- Reduced wait time in DOMWatchdog for page stability, enhancing responsiveness during DOM builds.
- Replaced instances of `agent_focus` with `agent_focus_target_id` across various components to ensure consistent target management.
- Enhanced error handling and logging in `Watchdog` classes to reflect the new target ID approach.
- Improved session retrieval methods to streamline interactions with the `SessionManager`, ensuring better performance and reliability.
- Enhanced the DOMTreeSerializer to include standardized placeholder formats for date, time, datetime-local, month, and week input types. This ensures LLMs utilize the correct ISO 8601 formats while programmatically setting values, improving user experience and consistency in date/time handling.
- Removed unnecessary boolean attributes from the serialized output for cleaner attribute management.
Laith Weinberger