Commit Graph

8176 Commits

Author SHA1 Message Date
Magnus Müller 7d34f9c930 Fix framework events not triggering (#3639)
Can't pass an IIFE to `Runtime.callFunctionOn`; doing so fails with a
syntax error.

This caused some inputs to clear after the agent blurred them due to
framework events never firing.

## Summary
Fixes framework events not triggering by replacing the IIFE with a
regular function for Runtime.callFunctionOn, preventing syntax errors
and ensuring change/blur events fire correctly. Inputs no longer clear
after the agent blurs them.

## Changes
- Replace IIFE with a function executed via objectId to avoid CDP syntax
error
- Add success/failure logs to help diagnose event triggering

## Based on
Original PR: #3367 by @sophiabits
Merged with latest main branch

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Replace IIFE with a regular function in `Runtime.callFunctionOn` to
reliably trigger framework events and add success/failure logging.
> 
> - **Watchdogs
(`browser_use/browser/watchdogs/default_action_watchdog.py`)**:
> - Replace IIFE with a regular `function()` for
`Runtime.callFunctionOn` in `_trigger_framework_events` to avoid syntax
errors and ensure events dispatch.
> - Add debug/warning logs indicating success or failure of framework
event triggering.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
d301f629fd. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes framework events not firing by avoiding a syntax error in CDP
Runtime.callFunctionOn. Change/blur events now trigger, and inputs no
longer clear after blur.

- **Bug Fixes**
- Execute a regular function via objectId instead of an IIFE in
_trigger_framework_events.
  - Add success/failure logs to diagnose event triggering.

<sup>Written for commit d301f629fd.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-20 23:12:08 -08:00
Magnus Müller d301f629fd Merge main into fix-framework-events 2025-11-20 23:09:23 -08:00
Pyry Takala 8bb61d906a Fix: Preserve in-use temporary user_data_dir during browser launch
Previously, when browser launch succeeded after retry using a temporary
directory, the code immediately deleted ALL temporary directories including
the one actively being used by the running browser process. This caused
browser crashes and data corruption when Chrome tried to write to the
deleted user_data_dir during normal operation.

The fix:
- Only clean up unused temporary directories after successful launch
- Preserve the currently used temporary directory until browser cleanup
- Update _temp_dirs_to_cleanup to track only the in-use directory for
  proper cleanup during BrowserKillEvent
2025-11-21 02:39:10 +00:00
Mert Unsal 49b58a3a29 BU model fix (#3637)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Restricts auto-configured LLM screenshot size to only `claude-sonnet`
models, removing the `browser-use` model check.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5d525b7e60. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Restricts auto-configuration of llm_screenshot_size to Claude Sonnet
models only. Prevents incorrect screenshot sizing for other models that
include "browser-use" in their name.

- **Bug Fixes**
- Removed the "browser-use" substring check; auto-config now triggers
only when model_name.startswith('claude-sonnet').

<sup>Written for commit 5d525b7e60.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-20 17:52:56 -08:00
Pyry Takala 5a3d340ec4 fix: add type ignores for pyright in cache tests
Content blocks are dictionaries at runtime despite type annotations
indicating Pydantic models. Added type: ignore comments to allow
dictionary-style .get() access which works correctly at runtime.
2025-11-21 01:12:56 +00:00
Pyry Takala 3787380829 fix: apply cache_control only to last block in Anthropic messages
Previously, cache_control was applied to every content block and tool call
in a message, which could exceed Anthropic's limit of 4 cache breakpoints
per request. This fix ensures cache_control is only applied to the last
block in each message type:
- Last text block in UserMessage/AssistantMessage content
- Last tool_use block when AssistantMessage has tool calls
- Only the last overall block when AssistantMessage has both content and tools

This prevents exceeding the cache breakpoint limit while maintaining
correct caching behavior.
2025-11-21 01:04:12 +00:00
mertunsall 5d525b7e60 BU model fix 2025-11-20 14:07:05 -08:00
Shubham Singh 515a42bdac Merge branch 'main' into fix/3620-javascript-quote-fixer-regex 2025-11-21 01:37:29 +05:30
kalil0321 ebfbefcfa9 Merge branch 'main' into attempt-fix-real-browser 2025-11-20 18:42:06 +01:00
Saurav Panda 7d5828edad fixed rerun_history (#3635)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Fixes the rerun_history example by correcting the ChatOpenAI import
path, updating the model to gpt-4.1-mini, and simplifying summary-LLM
usage to avoid errors when rerunning histories.

- **Bug Fixes**
  - Use from browser_use.llm import ChatOpenAI.
- Update example model to gpt-4.1-mini and remove outdated provider
snippets.
  - Clarify that summary_llm=None uses the agent’s LLM by default.

<sup>Written for commit 7811c62377.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-20 09:01:33 -08:00
Saurav Panda 7811c62377 fixed rerun_history 2025-11-20 22:22:31 +05:30
Saurav Panda 8d57da7f58 feat: added AI summary for rerun task (#3634)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds an automated rerun summary that analyzes the final screenshot and
step outcomes, then reports success and completion status. Supports an
optional summary_llm to use a specific model; defaults to the agent’s
model.

- **New Features**
- Generates a summary at the end of rerun_history using screenshot +
execution stats; appended as the last ActionResult.
- Adds RerunSummaryAction with summary, success, and completion_status.
- Introduces get_rerun_summary_prompt/message and
_generate_rerun_summary with structured output and text fallback.
- Updates the rerun example and adds tests covering success, errors,
fallback, and statistics.

- **Migration**
- rerun_history now returns an extra final ActionResult containing the
summary; handle results[-1] accordingly.
- Use summary_llm to override the model used for the summary when
needed.

<sup>Written for commit 5e356108f9.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-20 08:36:17 -08:00
Saurav Panda 5e356108f9 feat: added AI summary for rerun task 2025-11-20 21:40:30 +05:30
Saurav Panda 26606632cc add-variable-substitution-in-rerun (#3631)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Add variable substitution to agent history reruns so you can replay
workflows with different data. Also fix step_interval to represent step
duration and improve delay logging during replay.

- **New Features**
- load_and_rerun accepts a variables dict to replace detected values
(e.g., email, names, dates) across actions, nested dicts, and lists
without mutating the original history.
  - Example updated to show reruns with original and substituted values.
  - Comprehensive unit tests added for substitution behavior.

- **Bug Fixes**
- Correct step_interval calculation to use step_end_time -
step_start_time (step duration).
- Rerun logs now show the delay source and format (ms/s) for clarity;
tests updated to match the new duration semantics.

<sup>Written for commit d243616550.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-20 05:24:26 -08:00
Saurav Panda d243616550 Updated test_step_interval_calculation() 2025-11-20 18:30:55 +05:30
Saurav Panda 8948c47416 add-variable-substitution-in-rerun 2025-11-20 17:54:32 +05:30
kalil0321 0aa531e006 Merge branch 'main' into kalil0321/demo-mode 2025-11-20 11:44:14 +01:00
Saurav Panda 2407f249e5 added variable detection from agent history (#3628)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds automatic variable detection from agent history to identify
reusable inputs (emails, names, dates, etc.) and surface them via a
simple API for reruns and integrations.
Exposes Agent.detect_variables(), returning a map of DetectedVariable
with type/format hints.

- **New Features**
- Added variable_detector.py to detect variables using element
attributes and value patterns.
- Introduced Agent.detect_variables() to analyze current history and
return detected variables.
  - Added DetectedVariable and VariableMetadata models in agent.views.
- Updated examples/features/rerun_history.py to print detected variables
before rerun.
- Added unit tests covering attribute-based and pattern-based detection,
deduping, and history parsing.

<sup>Written for commit e57f7ac14f.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-20 01:31:31 -08:00
Saurav Panda e57f7ac14f added variable detection from agent history 2025-11-20 14:39:32 +05:30
Shubham Singh e5d0c666e5 Merge branch 'main' into fix/3620-javascript-quote-fixer-regex 2025-11-20 13:11:51 +05:30
Saurav Panda ffda72e0a3 fix: prevent KeyError crash in OpenAI reasoning models (#3540)
Fix KeyError that occurs when using OpenAI reasoning models (o1, o3,
etc.) with temperature=None or frequency_penalty=None.

The code was using 'del' to remove keys that may not exist in
model_params dictionary. Changed to use .pop() with default value for
safe key removal.

Fixes crash scenario:
- ChatOpenAI(model='o1-mini', temperature=None) -> KeyError

Changes:
- Replace 'del model_params[key]' with 'model_params.pop(key, None)'





<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Prevent KeyError crashes when using OpenAI reasoning models by safely
removing temperature and frequency_penalty from model_params with
.pop(). Calls like ChatOpenAI(model='o1-mini', temperature=None) no
longer crash.

<sup>Written for commit 0263f05a7e.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-19 22:30:43 -08:00
kalil0321 7242b6cedf Merge branch 'main' into kalil0321/demo-mode 2025-11-20 07:19:50 +01:00
kalil0321 ac6c7e7a48 Merge branch 'main' into attempt-fix-real-browser 2025-11-20 07:19:00 +01:00
Saurav Panda 0263f05a7e Merge branch 'main' into fix/openai-reasoning-model-keyerror 2025-11-19 21:48:46 -08:00
Saurav Panda 006e2e8c46 feat: added step_interval calculation for Agent history rerun (#3625)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds per-step wait time tracking for Agent history and uses it during
reruns to match original pacing. Each step now records step_interval,
and reruns respect it with a sensible fallback.

- **New Features**
- Added step_interval to StepMetadata to capture time between
consecutive steps (None for the first step).
- rerun_history uses recorded step_interval for per-step delays; delay
is applied before executing each step and falls back to
delay_between_actions when missing.
- Added tests for calculation, serialization, and backward
compatibility.

- **Migration**
- No changes required. Old histories load with step_interval defaulting
to None.

<sup>Written for commit c5d763a8dd.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-19 21:32:52 -08:00
Saurav Panda c5d763a8dd wait before execution in execute_history_step 2025-11-20 10:48:03 +05:30
Saurav Panda 8f8713b001 feat: added step_interval calculation for history 2025-11-20 10:28:28 +05:30
kalil0321 0f85870e13 Merge branch 'main' into attempt-fix-real-browser 2025-11-20 01:51:05 +01:00
kalil0321 8ec6592f1b Merge branch 'main' into kalil0321/demo-mode 2025-11-20 01:50:25 +01:00
Shubham Singh ed12b02d55 Merge branch 'main' into fix/3620-javascript-quote-fixer-regex 2025-11-19 12:33:28 +05:30
shubham-021 813c8518e7 fix/regex-pattern-for-quote-fixer 2025-11-19 12:21:08 +05:30
Alexander Yue 2dd80a1612 add the captcha and impossible flag in AgentEvent (#3622)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds two telemetry flags to AgentEvent to record when a CAPTCHA is
reached and when a task is judged impossible. Updates the judge_trace
example to use ChatBrowserUse and print the judgement when available.

- **New Features**
- Telemetry: Added judge_reached_captcha and judge_impossible_task to
AgentTelemetryEvent and populate them during agent event logging.
- Example: Switched judge_trace to ChatBrowserUse (local base_url) and
print judgement if present.

<sup>Written for commit e62129ba93.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-18 22:07:47 -08:00
Alezander9 e62129ba93 add the captcha and impossible flag in AgentEvent 2025-11-18 21:29:52 -08:00
Mert Unsal 4b67de3d36 fix: Add default=True to ScrollAction.down field to handle LLM respon… (#3602)
…ses missing the field

- Fixes validation error when LLM returns {'scroll': {'pages': 3}}
without 'down' field
- Defaults to True (scroll down) when 'down' is missing, which is the
most common case
- Still supports explicit down=False for scrolling up
- Also fixes CDP client compatibility with older cdp-use versions by
making max_ws_frame_size optional

Fixes validation errors with AWS Bedrock Claude and Gemini models











<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Set ScrollAction.down default to True to prevent validation errors when
LLMs omit the field and keep scroll-down as the common behavior.

- **Bug Fixes**
- Handle {'scroll': {'pages': N}} without 'down'; defaults to scrolling
down, explicit down=False still works.
  - Updated action description to clarify defaults and parameter ranges.

<sup>Written for commit 72233ecc75.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-18 16:29:23 -08:00
Mert Unsal ece9901b83 Merge branch 'main' into fix/scroll-action-default-down-field 2025-11-18 16:28:06 -08:00
Mert Unsal 745ce1686f Record click coordinates in agent's system (#3621)
Record click coordinates in the agent's coordinate system in action
results to improve consistency with agent's reasoning.

When the agent performs a click based on coordinates, the recorded
`click_x` and `click_y` in the action result and memory now reflect the
coordinates in the agent's internal coordinate system (LLM screenshot
size), rather than the browser's viewport coordinates. This ensures that
when the agent reviews its history, the recorded coordinates match the
system it uses for its own perception and planning, making its reasoning
more consistent and traceable. The actual click execution still
correctly uses viewport coordinates.

---
[Slack
Thread](https://browser-use.slack.com/archives/D092QUQDC56/p1763508779761389?thread_ts=1763508779.761389&cid=D092QUQDC56)

<a
href="https://cursor.com/background-agent?bcId=bc-6424d14d-4d74-46a3-b42e-569307f122f7"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-6424d14d-4d74-46a3-b42e-569307f122f7"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>





<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Log click_x and click_y using the agent’s internal coordinate system
(LLM screenshot) instead of the browser viewport, so the agent’s history
matches its perception and reasoning. Click execution remains in
viewport coordinates; only memory and metadata were updated.

<sup>Written for commit 93dc7a378d.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
2025-11-18 16:26:50 -08:00
Cursor Agent 93dc7a378d Changes made by Agent
Co-authored-by: mailmertunsal <mailmertunsal@gmail.com>
2025-11-18 23:41:22 +00:00
Cursor Agent 73c6c7d76b Changes made by Agent
Co-authored-by: mailmertunsal <mailmertunsal@gmail.com>
2025-11-18 23:37:25 +00:00
kalil0321 1b472fe7be trigger CI 2025-11-18 23:13:05 +01:00
kalil0321 fd2c1f2c25 Merge branch 'main' into kalil0321/demo-mode 2025-11-18 22:32:54 +01:00
Mert Unsal 8a8ef967b2 more timeout for gemini 3 pro (#3618) 2025-11-18 11:54:57 -08:00
mertunsall 67e2d779d8 more timeout for gemini 3 pro 2025-11-18 11:54:23 -08:00
kalil0321 50bafbd9aa Change log level from info to debug for session data clearing and browser session reset messages 2025-11-18 18:52:43 +01:00
kalil0321 574b9e3472 Address cubic issue 2025-11-18 18:49:04 +01:00
kalil0321 9960e5d87b Merge branch 'kalil0321/demo-mode' of https://github.com/kalil0321/browser-use into kalil0321/demo-mode 2025-11-18 18:25:41 +01:00
kalil0321 1de03304e1 Address cubic 2025-11-18 18:25:38 +01:00
kalil0321 919372daf1 Merge branch 'main' into kalil0321/demo-mode 2025-11-18 17:59:34 +01:00
kalil0321 85a866e700 Fix some UI inconsistencies 2025-11-18 17:59:23 +01:00
kalil0321 e032a0742b Merge branch 'main' into attempt-fix-real-browser 2025-11-18 17:39:25 +01:00
kalil0321 19294a56be codespell ignore 2025-11-18 17:36:55 +01:00