Commit Graph

836 Commits

Author SHA1 Message Date
Magnus Müller 9899262186 Add additional debug logging in evaluate_tasks.py to track task execution flow and subprocess outputs. Enhance error reporting by capturing and displaying full stdout and stderr for better troubleshooting during agent execution. 2025-06-07 12:48:04 +02:00
Magnus Müller 54226c7b51 Add detailed debug logging in evaluate_tasks.py to capture agent execution steps and outputs. Enhance error reporting for subprocess failures and improve output handling when no agent output is provided. 2025-06-07 12:43:51 +02:00
Magnus Müller 592b9afbce Refactor evaluate_tasks.py to run agent tasks in separate subprocesses, enhancing isolation and error handling. Introduce argparse for task selection and improve logging management during task execution. 2025-06-07 12:37:07 +02:00
Magnus Müller 8246bcd299 Refactor run_task in evaluate_tasks.py to create a dedicated BrowserSession for each agent, improving session management and ensuring headless execution with --no-sandbox argument. 2025-06-07 11:58:01 +02:00
Magnus Müller 38cfa86738 Update browser_use_pip.yaml to simplify output requirements and refactor run_task in evaluate_tasks.py to remove shared profile parameter, enhancing browser session management with a dedicated profile. 2025-06-07 11:50:30 +02:00
Magnus Müller bf13dbb452 Refactor run_task function in evaluate_tasks.py to accept a shared profile for consistent browser sessions across tasks 2025-06-07 11:40:19 +02:00
Magnus Müller d50581e87c Remove --no-sandbox argument from BrowserProfile in evaluate_tasks.py to simplify configuration 2025-06-07 11:28:46 +02:00
Magnus Müller df14af7f00 Add --no-sandbox argument to BrowserProfile in evaluate_tasks.py for enhanced security during task execution 2025-06-07 11:20:26 +02:00
Magnus Müller b29dbe5f2a Remove keep_alive option from BrowserProfile in evaluate_tasks.py to streamline session management 2025-06-07 11:10:51 +02:00
Magnus Müller 576519ee40 Enhance CI workflow by adding agent tasks evaluation step and updating evaluate_tasks.py to output evaluation results 2025-06-07 10:59:24 +02:00
Magnus Müller 412904dc65 Add keep_alive option to BrowserProfile in evaluate_tasks.py for improved session persistence 2025-06-07 10:53:58 +02:00
Magnus Müller 3666d2b077 Add agent tasks evaluation script and update CI workflow to include it 2025-06-07 10:49:03 +02:00
Magnus Müller 9ef3d6198a Update test_agent_real_tasks to enable headless mode for BrowserProfile, enhancing browser session management 2025-06-07 10:13:18 +02:00
Magnus Müller b04d110d57 Refactor test_agent_real_tasks to utilize BrowserProfile for session management and improve browser session handling 2025-06-07 10:05:19 +02:00
Magnus Müller 6def4c24f1 Update test_agent_real_tasks to use temporary user data directory and disable headless mode 2025-06-07 09:25:36 +02:00
Magnus Müller 074169f80f Merge branch 'main' into tests/eval 2025-06-07 09:15:41 +02:00
Nick Sweeting 11b980b3ae fixture scoping 2025-06-06 19:26:29 -07:00
Nick Sweeting f94d56910f consistent test fixture scope, fix race condition on shutdown, improve logging 2025-06-06 19:21:51 -07:00
Magnus Müller 67fbcf46a5 Update test_agent_real_tasks to use None for user_data_dir in BrowserSession 2025-06-07 01:16:52 +02:00
Magnus Müller 25c96bcaca Add README for contributing agent tasks with guidelines and testing instructions 2025-06-07 00:51:32 +02:00
Magnus Müller 2c355ce6d9 Add new agent task tests for Amazon laptop search, browser-use pip installation, and Cloudflare captcha solving 2025-06-07 00:49:39 +02:00
Magnus Müller 80dbad0811 Refactor test_agent_real_task to use temporary browser session and improve judge context handling 2025-06-07 00:46:58 +02:00
Magnus Müller 80fd5e97e8 Add tests for agent real tasks using pytest and ChatOpenAI 2025-06-07 00:46:43 +02:00
Nick Sweeting 8504bc4c7b parallelize playwright tests using loop scope=session and pytest-xdist 2025-06-06 02:03:25 -07:00
Nick Sweeting 9b3dd67dad stop playwright during browser teardown 2025-06-06 01:23:47 -07:00
Nick Sweeting 683a97e475 try and tweak test_registry to fix deadlock tests 2025-06-06 01:01:31 -07:00
Nick Sweeting 819859c006 add BrowserSession.kill method 2025-06-05 23:44:23 -07:00
Nick Sweeting e1aa19bf5b fix test deadlocks by setting max 30sec timeout on launch and close 2025-06-05 23:35:25 -07:00
Nick Sweeting 743e357f46 fix test deadlocks 2025-06-05 23:29:05 -07:00
Nick Sweeting b07e838543 fix downloads tests 2025-06-05 23:16:58 -07:00
Nick Sweeting 02b4f2c5de fix tests by using temp dirs and headless for all tests 2025-06-05 23:12:32 -07:00
Nick Sweeting 8306a01a02 cleaner agent and browser session logging on connect and disconnect 2025-06-05 22:21:42 -07:00
Nick Sweeting 2a39dfa1c5 use self.logger to shorten log lines 2025-06-05 21:21:10 -07:00
Nick Sweeting 23d3bf4205 rename .emit to .dispatch and prefix event params to prevent clashes 2025-06-05 21:04:26 -07:00
Nick Sweeting f3c5672c46 better circular loop prevention for eventbus 2025-06-05 20:23:41 -07:00
Nick Sweeting 52b6b4b1cb simplify eventbus methods 2025-06-05 19:43:43 -07:00
Nick Sweeting 57e1d6ebe4 remove dead code 2025-06-05 19:01:11 -07:00
Nick Sweeting 9da6c485b7 implement new EventBus module with cloud_events to send on agent steps 2025-06-05 18:51:22 -07:00
Nick Sweeting cf6a0d9fbf improve chrome launch error handling when user_data_dir is bad 2025-06-05 18:22:02 -07:00
Nick Sweeting 1f32315031 fix fixture scopes in tests 2025-06-05 03:27:16 -07:00
Nick Sweeting f1c15d25ee use reusable pytest endpoint 2025-06-05 03:02:04 -07:00
Nick Sweeting c5cc142a4c fix cookies relative path test 2025-06-05 02:28:17 -07:00
Nick Sweeting 8e06ab5018 update tests for BrowserSession.start 2025-06-05 02:24:34 -07:00
Nick Sweeting 767a75e2bf rename downloads_dir to downloads_path 2025-06-05 02:17:57 -07:00
tinkerer-shubh 9b93a9fa6e fix: add missing await keyword to asyncio.sleep() in extraction test - Fixes asyncio.sleep(1) that was missing await keyword on line 85 - Without await, the sleep coroutine was never executed, causing no delay - This improves test reliability by ensuring proper 1-second wait after page navigation 2025-05-31 17:01:02 +05:30
Nick Sweeting fe0d206877 fix tests 2025-05-30 03:57:10 -07:00
Nick Sweeting 280e5eca62 fix old tests 2025-05-27 22:31:32 -07:00
Nick Sweeting 6cb1dbfac8 migrate old style tests 2025-05-27 20:50:41 -07:00
Nick Sweeting f5bb3828a0 re-add support for cookies_file param 2025-05-27 20:50:31 -07:00
Nick Sweeting aed2fa27bb add tests for 2025-05-27 17:09:18 -07:00