## Problem
History rerun was failing when CSS classes changed between recording and
replay. The `element_hash` includes `class` attribute, but classes like
`gw-focus`, `gw-animation--focus`, `hover`, etc. change dynamically
based on UI state.
**Example:**
- Recording: `class="gw-animation--focus gw-focus"` → hash A
- Replay: `class="gw-focus"` → hash B
- Result: **Match fails**, rerun breaks
## Solution
Implemented a **cascading element matching strategy** with 3 levels:
| Level | Name | Description | Confidence |
|-------|------|-------------|------------|
| 1 | EXACT | Full element_hash match | 100% |
| 2 | STABLE | Hash with dynamic classes filtered | 98% |
| 3 | XPATH | XPath string match | 95% |
### Key Changes
1. **Added `filter_dynamic_classes()`** - Filters out transient CSS
patterns:
- `focus`, `hover`, `active`, `selected`, `disabled`
- `animation`, `transition`, `loading`
- `expanded`, `collapsed`, `visible`, `hidden`, etc.
2. **Added `compute_stable_hash()`** to `EnhancedDOMTreeNode` - Single
source of truth for stable hash computation
3. **Added `stable_hash` field** to `DOMInteractedElement` - Computed at
save time from the source `EnhancedDOMTreeNode`
4. **Updated `_update_action_indices()`** - Tries each match level in
order until success
### Backward Compatibility
- Old history files without `stable_hash` field load fine (defaults to
`None`)
- Falls back to XPATH matching for old files
### Verification
```
NOT_WORKING class: "gw-animation--focus gw-focus"
WORKING class: "gw-focus"
After filter_dynamic_classes:
NOT_WORKING: ""
WORKING: ""
EXACT level: ❌ Different hashes
STABLE level: ✅ Same hash (13488107035447772127)
```
## Files Changed
- `browser_use/dom/views.py` - Added MatchLevel, DYNAMIC_CLASS_PATTERNS,
filter_dynamic_classes, compute_stable_hash, stable_hash field
- `browser_use/agent/service.py` - Updated _update_action_indices with
cascading logic
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Stabilizes history rerun by using cascading element matching when CSS
classes change between record and replay. Filters transient classes to
compute a stable hash and falls back to XPath when needed.
- **Bug Fixes**
- Cascading matching: EXACT hash → STABLE hash (dynamic classes
filtered) → XPATH.
- Added filter_dynamic_classes and compute_stable_hash; stored
stable_hash in DOMInteractedElement.
- Backward compatible: old histories load; XPath used when stable_hash
is missing.
- Improved logs show match level; action indices update when the element
moves.
<sup>Written for commit bc9b51055b.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
playwright 1.57.0 installs chromium to $playwright_path/chromium-*/chrome-linux64/, causing the existing paths to fail to find the browser. globing chrome-linux* will allow watchdog to find playwright installed 32-bit and 64-bit chromium.
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Improve rerun history matching by including accessibility name in the
element hash and adding an XPath fallback. This reduces mismatches and
fixes action index rebuilds from older history files.
- **Bug Fixes**
- Element hash now includes ax_name to better distinguish similar
elements.
- Matching strategy tries element_hash first, then falls back to XPath
for legacy history.
- DOMInteractedElement now captures and serializes ax_name for
persistence.
<sup>Written for commit 1c57f2a1b2.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
Fixes#3755
### Problem
When switching to a newly opened tab and closing background tabs, a new
blank tab was incorrectly created.
### Cause
In
[aboutblank_watchdog.py](https://github.com/browser-use/browser-use/blob/cfccc38d52aa853cd4b9b0c7b181e24ce518185e/browser_use/browser/watchdogs/aboutblank_watchdog.py#L72),
the condition `len(page_targets) <= 1` triggered blank tab creation even
when an active tab remained.
### Fix
Changed condition from `<= 1` to `< 1` on line 72, ensuring blank tabs
are only created when no tabs remain.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Stops creating a blank tab when closing background tabs. A new
about:blank tab is only created when no tabs remain.
- **Bug Fixes**
- Updated aboutblank_watchdog check from len(page_targets) <= 1 to < 1.
- Prevents extra blank tab after switching to a new tab and closing
background tabs.
<sup>Written for commit b0fe15aeb2.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds support for OSS Browser Use models via 'browser-use/*' names and
includes an example. Also sends the selected model and optional
session_id in API requests (sticky routing), and normalizes 'bu-latest'
to 'bu-1-0'.
- **New Features**
- Accept models that start with 'browser-use/' (e.g.,
'browser-use/bu-30b-a3b-preview').
- Added example script at examples/models/bu_oss.py showing OSS model
usage.
- **Bug Fixes**
- Include 'model' in the request payload so the API uses the selected
model.
<sup>Written for commit 781f6eb95e.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds a max_step_interval to cap delays between replayed actions,
preventing long pauses from saved step intervals and making reruns more
predictable.
- **New Features**
- Added max_step_interval (default: 5s) to rerun_history and
load_and_rerun; caps saved step_interval when replaying history.
- Improved logs and docs: show when a delay is capped and clarified
kwargs (delay_between_actions, max_step_interval, summary_llm,
ai_step_llm).
<sup>Written for commit a0996ba39e.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
## Summary
- Only disable `use_vision` for grok-3 variants and grok-code models
- grok-2 and grok-4 models now retain vision support
## Test plan
- [x] pyright passes
- [x] vision-related CI tests pass
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fix XAI vision gating to apply only to unsupported models. use_vision is
now disabled only for grok-3 variants and grok-code; grok-2 and grok-4
keep vision enabled.
<sup>Written for commit cba65772f8.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
## Summary: Improvement of lazy import
Users who don't use OpenAI provider (e.g., Anthropic, Google, Groq
users) were paying ~0.88s startup penalty due to openai SDK being
imported at module level. This PR eliminates that unnecessary overhead.
Additionally, reportlab was imported at module level in
`filesystem/file_system.py` even though PDF generation is rarely used.
By moving the import inside `sync_to_disk_sync()`, users who don't
generate PDFs avoid the unnecessary import cost.
## Modified files
- **agent/views.py**: Move `RateLimitError` import inside
`format_error()` function to avoid loading openai SDK at module level
- **llm/messages.py**: Replace `openai.BaseModel` with
`pydantic.BaseModel` directly to remove unnecessary openai dependency
- **filesystem/file_system.py**: Move reportlab imports inside
`sync_to_disk_sync()` to avoid startup cost when PDF generation is not
used
- **utils.py**: Convert `OpenAIBadRequestError` and
`GroqBadRequestError` to lazy loaders to avoid loading SDKs at module
level
## Performance Improvement (average of 20 iterations)
| Module | current main | PR (improved lazyimport) | Improvement |
|--------|-------------|-----------|-------------|
| `agent/views.py` | 2.272s | 1.619s | **28.7% faster** |
| `ChatAnthropic` | 1.377s | 0.894s | **35.1% faster** |
### Key Changes
- Removed OpenAI SDK loading when using non-OpenAI providers
- reportlab is only loaded when PDF generation is actually needed
- No behavioral changes - only import timing affected
## Test plan
- [x] `uv run ruff check` passes
- [x] `uv run pyright` passes (0 errors)
- [x] Runtime verification - lazy loading works correctly
- [x] Benchmark comparison against current main branch
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Reduce startup latency by lazily importing OpenAI and reportlab,
removing module-level SDK dependencies. Non-OpenAI providers avoid ~0.9s
overhead with no behavior changes.
- **Refactors**
- Lazy import openai.RateLimitError inside format_error
(agent/views.py).
- Replace openai.BaseModel with pydantic.BaseModel (llm/messages.py).
- Lazy import reportlab inside sync_to_disk_sync
(filesystem/file_system.py).
- Add lazy loaders for OpenAI/Groq BadRequestError (utils.py).
- Results: agent/views import 28.7% faster; ChatAnthropic path 35.1%
faster; reportlab loads only when generating PDFs.
<sup>Written for commit 38703d5ad7.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
- agent/views.py: Move RateLimitError import inside format_error() function
to avoid loading openai SDK (~800ms) at module level
- llm/messages.py: Replace openai.BaseModel with pydantic.BaseModel directly
to remove unnecessary openai dependency
- filesystem/file_system.py: Move reportlab imports inside sync_to_disk_sync()
to avoid ~40ms startup cost when PDF generation is not used
- utils.py: Convert OpenAIBadRequestError and GroqBadRequestError to lazy
loaders to avoid loading SDKs at module level
This improves import time for users who don't use OpenAI provider,
especially when using Anthropic, Google, or other providers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously all grok models had vision disabled. Only grok-3 variants
(grok-3, grok-3-mini, grok-3-fast) and grok-code models don't support
vision. grok-2 and grok-4 models now correctly retain vision support.
# Fix: `output_model_schema` not working with `@sandbox`
## Problem
When using `output_model_schema` with an Agent inside a `@sandbox`
decorated function, the `history.structured_output` property always
returned `None`.
```python
@sandbox()
async def my_task(browser: Browser) -> AgentHistoryList[MyModel]:
agent = Agent(task="...", browser=browser, llm=llm, output_model_schema=MyModel)
return await agent.run()
result = await my_task()
result.structured_output # Always None ❌
```
## Root Cause
`AgentHistoryList` stores the output schema in `_output_model_schema` —
a **Pydantic private attribute** (underscore prefix). Private attributes
are **not serialized** when converting to JSON.
When sandbox executes remotely:
1. Agent runs and sets `history._output_model_schema = MyOutputModel` ✅
2. Result is serialized to JSON for transport → **private attribute
dropped** ❌
3. Client reconstructs `AgentHistoryList` but `_output_model_schema` is
`None`
4. `structured_output` property returns `None` (schema missing)
## Solution
### 1. Added `get_structured_output(output_model)` method
Users can explicitly pass their schema to parse the result:
```python
result.get_structured_output(MyModel) # Works even without type annotation
```
### 2. Enhanced sandbox parsing for Pydantic generics
When return type is `AgentHistoryList[MyModel]`, the sandbox now:
- Detects Pydantic generic metadata via `__pydantic_generic_metadata__`
- Extracts the type parameter (`MyModel`)
- Auto-sets `_output_model_schema` on the reconstructed object
### 3. Fixed import extraction for Pydantic generics
The sandbox's import extraction wasn't detecting `AgentHistoryList` from
return types like `AgentHistoryList[MyModel]`, causing `NameError` on
remote execution.
## Usage
### Option 1: Generic return type (recommended)
```python
@sandbox()
async def task(browser: Browser) -> AgentHistoryList[MyModel]:
agent = Agent(task="...", browser=browser, llm=llm, output_model_schema=MyModel)
return await agent.run()
result = await task()
result.structured_output # ✅ Works automatically
```
### Option 2: Non-generic return type with explicit method
```python
@sandbox()
async def task(browser: Browser) -> AgentHistoryList:
agent = Agent(task="...", browser=browser, llm=llm, output_model_schema=MyModel)
return await agent.run()
result = await task()
result.structured_output # None (expected)
result.get_structured_output(MyModel) # ✅ Works
```
## Files Changed
- `browser_use/agent/views.py` - Added `get_structured_output()` method
- `browser_use/sandbox/sandbox.py` - Enhanced
`_parse_with_type_annotation()` and `_get_imports_used_in_function()`
for Pydantic generics
- `tests/ci/test_sandbox_structured_output.py` - Unit tests
- `examples/sandbox/structured_output.py` - Example demonstrating both
approaches
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes structured output when agents run inside @sandbox.
structured_output now works with AgentHistoryList[Model], and a new
helper lets you parse results without type annotations.
- **Bug Fixes**
- Preserve output model schema in sandbox results by detecting Pydantic
generics and setting it on AgentHistoryList.
- Improve import extraction to include generic types like
AgentHistoryList[MyModel] and prevent NameError.
- Enhance _parse_with_type_annotation to reconstruct AgentHistoryList[T]
and set _output_model_schema correctly.
- **New Features**
- Added AgentHistoryList.get_structured_output(schema) to parse the
final result explicitly when no generic return type is used.
<sup>Written for commit 9b894150d5.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Enable coordinate-based clicking only for supported models, keeping
clicks index-only by default. This reduces misclicks and keeps the API
simpler.
- **New Features**
- Auto-enable coordinate clicking for models matching: claude-sonnet-4,
claude-opus-4, gemini-3-pro, and browser-use/*.
- Added set_coordinate_clicking(enabled) to toggle coordinate support
and re-register the click action accordingly.
- New ClickElementActionIndexOnly model for index-only clicks;
coordinate and index handlers are retained and shared.
<sup>Written for commit 8c9e3b272c.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
Re-enable sensitive data filtering for state messages to prevent real
sensitive values from action results leaking into LLM messages.
State messages, specifically the `agent_history_description`, include
action results which contain actual sensitive values (e.g., passwords)
after they've been replaced from placeholders during action execution.
The `_filter_sensitive_data` method was previously not called for state
messages, leading to these real values being sent to the LLM. This PR
ensures that state messages are properly filtered, replacing real
sensitive data with placeholder tags before being sent to the LLM. New
integration tests confirm the end-to-end filtering.
---
[Slack
Thread](https://browser-use.slack.com/archives/D092QUQDC56/p1765410795078139?thread_ts=1765410795.078139&cid=D092QUQDC56)
<a
href="https://cursor.com/background-agent?bcId=bc-97cd8546-fd39-4e4f-84af-8882815d705b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a> <a
href="https://cursor.com/agents?id=bc-97cd8546-fd39-4e4f-84af-8882815d705b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Restored sensitive data filtering for state messages so real values from
action results never reach the LLM. Sensitive values like passwords and
API keys are replaced with placeholder tags in
agent_history_description.
- **Bug Fixes**
- Filter state messages in MessageManager when sensitive_data is
present; system/context messages remain unfiltered.
- Replace real values in action results with <secret>...</secret>
placeholders before sending to the LLM.
- Added integration tests covering global and domain-specific formats
(e.g., password, api_key) to prevent leaks.
<sup>Written for commit 201b990591.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
This change ensures that sensitive data in action results is filtered before being included in state messages sent to the LLM. This prevents accidental leakage of private information. New tests are added to verify this functionality.
Co-authored-by: mailmertunsal <mailmertunsal@gmail.com>
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Fixed ChatOpenAI import path in actor docs to
browser_use.llm.openai.chat so code examples import correctly.
<sup>Written for commit ab48cc51df.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Add support for browser-use fine-tuned models with simplified,
JSON-first system prompts, and reorganize prompt templates into a
dedicated package for reliable loading and packaging.
- **New Features**
- Auto-detect browser-use models (model name contains "browser-use/")
and select simplified prompts for thinking, no-thinking, and flash
modes.
- Plumb is_browser_use_model through MessageManager to load the correct
template.
- **Refactors**
- Moved all prompt templates to browser_use/agent/system_prompts and
load via importlib.resources for package-safe access.
- Updated pyproject to include the new prompt directory and remove old
file paths.
<sup>Written for commit 110b96dc8d.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Corrected the documented default for max_actions_per_step to 3 to match
current behavior. Cleaned minor formatting in AGENTS.md (removed
trailing spaces and fixed a tips blockquote).
<sup>Written for commit 2e887d0076.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
Mert Unsal
