<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Adds a max_step_interval to cap delays between replayed actions,
preventing long pauses from saved step intervals and making reruns more
predictable.
- **New Features**
- Added max_step_interval (default: 5s) to rerun_history and
load_and_rerun; caps saved step_interval when replaying history.
- Improved logs and docs: show when a delay is capped and clarified
kwargs (delay_between_actions, max_step_interval, summary_llm,
ai_step_llm).
<sup>Written for commit a0996ba39e.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
## Summary
- Only disable `use_vision` for grok-3 variants and grok-code models
- grok-2 and grok-4 models now retain vision support
## Test plan
- [x] pyright passes
- [x] vision-related CI tests pass
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fix XAI vision gating to apply only to unsupported models. use_vision is
now disabled only for grok-3 variants and grok-code; grok-2 and grok-4
keep vision enabled.
<sup>Written for commit cba65772f8.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
## Summary: Improvement of lazy import
Users who don't use OpenAI provider (e.g., Anthropic, Google, Groq
users) were paying ~0.88s startup penalty due to openai SDK being
imported at module level. This PR eliminates that unnecessary overhead.
Additionally, reportlab was imported at module level in
`filesystem/file_system.py` even though PDF generation is rarely used.
By moving the import inside `sync_to_disk_sync()`, users who don't
generate PDFs avoid the unnecessary import cost.
## Modified files
- **agent/views.py**: Move `RateLimitError` import inside
`format_error()` function to avoid loading openai SDK at module level
- **llm/messages.py**: Replace `openai.BaseModel` with
`pydantic.BaseModel` directly to remove unnecessary openai dependency
- **filesystem/file_system.py**: Move reportlab imports inside
`sync_to_disk_sync()` to avoid startup cost when PDF generation is not
used
- **utils.py**: Convert `OpenAIBadRequestError` and
`GroqBadRequestError` to lazy loaders to avoid loading SDKs at module
level
## Performance Improvement (average of 20 iterations)
| Module | current main | PR (improved lazyimport) | Improvement |
|--------|-------------|-----------|-------------|
| `agent/views.py` | 2.272s | 1.619s | **28.7% faster** |
| `ChatAnthropic` | 1.377s | 0.894s | **35.1% faster** |
### Key Changes
- Removed OpenAI SDK loading when using non-OpenAI providers
- reportlab is only loaded when PDF generation is actually needed
- No behavioral changes - only import timing affected
## Test plan
- [x] `uv run ruff check` passes
- [x] `uv run pyright` passes (0 errors)
- [x] Runtime verification - lazy loading works correctly
- [x] Benchmark comparison against current main branch
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Reduce startup latency by lazily importing OpenAI and reportlab,
removing module-level SDK dependencies. Non-OpenAI providers avoid ~0.9s
overhead with no behavior changes.
- **Refactors**
- Lazy import openai.RateLimitError inside format_error
(agent/views.py).
- Replace openai.BaseModel with pydantic.BaseModel (llm/messages.py).
- Lazy import reportlab inside sync_to_disk_sync
(filesystem/file_system.py).
- Add lazy loaders for OpenAI/Groq BadRequestError (utils.py).
- Results: agent/views import 28.7% faster; ChatAnthropic path 35.1%
faster; reportlab loads only when generating PDFs.
<sup>Written for commit 38703d5ad7.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
- agent/views.py: Move RateLimitError import inside format_error() function
to avoid loading openai SDK (~800ms) at module level
- llm/messages.py: Replace openai.BaseModel with pydantic.BaseModel directly
to remove unnecessary openai dependency
- filesystem/file_system.py: Move reportlab imports inside sync_to_disk_sync()
to avoid ~40ms startup cost when PDF generation is not used
- utils.py: Convert OpenAIBadRequestError and GroqBadRequestError to lazy
loaders to avoid loading SDKs at module level
This improves import time for users who don't use OpenAI provider,
especially when using Anthropic, Google, or other providers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously all grok models had vision disabled. Only grok-3 variants
(grok-3, grok-3-mini, grok-3-fast) and grok-code models don't support
vision. grok-2 and grok-4 models now correctly retain vision support.
# Fix: `output_model_schema` not working with `@sandbox`
## Problem
When using `output_model_schema` with an Agent inside a `@sandbox`
decorated function, the `history.structured_output` property always
returned `None`.
```python
@sandbox()
async def my_task(browser: Browser) -> AgentHistoryList[MyModel]:
agent = Agent(task="...", browser=browser, llm=llm, output_model_schema=MyModel)
return await agent.run()
result = await my_task()
result.structured_output # Always None ❌
```
## Root Cause
`AgentHistoryList` stores the output schema in `_output_model_schema` —
a **Pydantic private attribute** (underscore prefix). Private attributes
are **not serialized** when converting to JSON.
When sandbox executes remotely:
1. Agent runs and sets `history._output_model_schema = MyOutputModel` ✅
2. Result is serialized to JSON for transport → **private attribute
dropped** ❌
3. Client reconstructs `AgentHistoryList` but `_output_model_schema` is
`None`
4. `structured_output` property returns `None` (schema missing)
## Solution
### 1. Added `get_structured_output(output_model)` method
Users can explicitly pass their schema to parse the result:
```python
result.get_structured_output(MyModel) # Works even without type annotation
```
### 2. Enhanced sandbox parsing for Pydantic generics
When return type is `AgentHistoryList[MyModel]`, the sandbox now:
- Detects Pydantic generic metadata via `__pydantic_generic_metadata__`
- Extracts the type parameter (`MyModel`)
- Auto-sets `_output_model_schema` on the reconstructed object
### 3. Fixed import extraction for Pydantic generics
The sandbox's import extraction wasn't detecting `AgentHistoryList` from
return types like `AgentHistoryList[MyModel]`, causing `NameError` on
remote execution.
## Usage
### Option 1: Generic return type (recommended)
```python
@sandbox()
async def task(browser: Browser) -> AgentHistoryList[MyModel]:
agent = Agent(task="...", browser=browser, llm=llm, output_model_schema=MyModel)
return await agent.run()
result = await task()
result.structured_output # ✅ Works automatically
```
### Option 2: Non-generic return type with explicit method
```python
@sandbox()
async def task(browser: Browser) -> AgentHistoryList:
agent = Agent(task="...", browser=browser, llm=llm, output_model_schema=MyModel)
return await agent.run()
result = await task()
result.structured_output # None (expected)
result.get_structured_output(MyModel) # ✅ Works
```
## Files Changed
- `browser_use/agent/views.py` - Added `get_structured_output()` method
- `browser_use/sandbox/sandbox.py` - Enhanced
`_parse_with_type_annotation()` and `_get_imports_used_in_function()`
for Pydantic generics
- `tests/ci/test_sandbox_structured_output.py` - Unit tests
- `examples/sandbox/structured_output.py` - Example demonstrating both
approaches
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes structured output when agents run inside @sandbox.
structured_output now works with AgentHistoryList[Model], and a new
helper lets you parse results without type annotations.
- **Bug Fixes**
- Preserve output model schema in sandbox results by detecting Pydantic
generics and setting it on AgentHistoryList.
- Improve import extraction to include generic types like
AgentHistoryList[MyModel] and prevent NameError.
- Enhance _parse_with_type_annotation to reconstruct AgentHistoryList[T]
and set _output_model_schema correctly.
- **New Features**
- Added AgentHistoryList.get_structured_output(schema) to parse the
final result explicitly when no generic return type is used.
<sup>Written for commit 9b894150d5.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Enable coordinate-based clicking only for supported models, keeping
clicks index-only by default. This reduces misclicks and keeps the API
simpler.
- **New Features**
- Auto-enable coordinate clicking for models matching: claude-sonnet-4,
claude-opus-4, gemini-3-pro, and browser-use/*.
- Added set_coordinate_clicking(enabled) to toggle coordinate support
and re-register the click action accordingly.
- New ClickElementActionIndexOnly model for index-only clicks;
coordinate and index handlers are retained and shared.
<sup>Written for commit 8c9e3b272c.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
Re-enable sensitive data filtering for state messages to prevent real
sensitive values from action results leaking into LLM messages.
State messages, specifically the `agent_history_description`, include
action results which contain actual sensitive values (e.g., passwords)
after they've been replaced from placeholders during action execution.
The `_filter_sensitive_data` method was previously not called for state
messages, leading to these real values being sent to the LLM. This PR
ensures that state messages are properly filtered, replacing real
sensitive data with placeholder tags before being sent to the LLM. New
integration tests confirm the end-to-end filtering.
---
[Slack
Thread](https://browser-use.slack.com/archives/D092QUQDC56/p1765410795078139?thread_ts=1765410795.078139&cid=D092QUQDC56)
<a
href="https://cursor.com/background-agent?bcId=bc-97cd8546-fd39-4e4f-84af-8882815d705b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a> <a
href="https://cursor.com/agents?id=bc-97cd8546-fd39-4e4f-84af-8882815d705b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Restored sensitive data filtering for state messages so real values from
action results never reach the LLM. Sensitive values like passwords and
API keys are replaced with placeholder tags in
agent_history_description.
- **Bug Fixes**
- Filter state messages in MessageManager when sensitive_data is
present; system/context messages remain unfiltered.
- Replace real values in action results with <secret>...</secret>
placeholders before sending to the LLM.
- Added integration tests covering global and domain-specific formats
(e.g., password, api_key) to prevent leaks.
<sup>Written for commit 201b990591.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
This change ensures that sensitive data in action results is filtered before being included in state messages sent to the LLM. This prevents accidental leakage of private information. New tests are added to verify this functionality.
Co-authored-by: mailmertunsal <mailmertunsal@gmail.com>
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Fixed ChatOpenAI import path in actor docs to
browser_use.llm.openai.chat so code examples import correctly.
<sup>Written for commit ab48cc51df.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Add support for browser-use fine-tuned models with simplified,
JSON-first system prompts, and reorganize prompt templates into a
dedicated package for reliable loading and packaging.
- **New Features**
- Auto-detect browser-use models (model name contains "browser-use/")
and select simplified prompts for thinking, no-thinking, and flash
modes.
- Plumb is_browser_use_model through MessageManager to load the correct
template.
- **Refactors**
- Moved all prompt templates to browser_use/agent/system_prompts and
load via importlib.resources for package-safe access.
- Updated pyproject to include the new prompt directory and remove old
file paths.
<sup>Written for commit 110b96dc8d.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Corrected the documented default for max_actions_per_step to 3 to match
current behavior. Cleaned minor formatting in AGENTS.md (removed
trailing spaces and fixed a tips blockquote).
<sup>Written for commit 2e887d0076.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Integrates Browser Use API skills into the agent, auto-registering them
as actions with Pydantic-validated parameters and structured outputs.
Adds cookie-aware execution and surfaces unavailable skills (missing
cookies) in the agent prompt; requires BROWSER_USE_API_KEY and passing
skills (or skill_ids) to enable skills.
- **New Features**
- Adds browser_use/skills package with SkillService (fetch/execute),
schema converters, and SDK view wrappers.
- Agent accepts skills or skill_ids (supports '*'); 'skills' takes
precedence and they cannot be set together; lazily loads skills,
registers them as actions with clean slugs, rebuilds action models, and
closes the service on shutdown.
- Validates parameters against skill schemas; fills cookie values from
browser cookies; returns clear errors for missing cookies; supports
output schema conversion.
- Shows "Unavailable Skills" info in the state message when required
cookies are missing.
- Updates BrowserSession.cookies to return all cookies via CDP
Storage.getCookies; removes the urls parameter.
- **Dependencies**
- Adds browser-use-sdk>=2.0.12.
<sup>Written for commit 22b37f3c11.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Fixes structured output when tools are provided by initializing tools
first and auto-detecting the output model from tools. Ensures the agent
uses the correct schema, with a warning if the agent and tools models
differ.
- **Bug Fixes**
- Initialize tools before task/LLM setup to allow schema detection.
- Use the tools’ output model when output_model_schema isn’t set; warn
on mismatch (agent schema takes precedence).
- ToolsService now stores and exposes the output model to keep done
action registration consistent.
<sup>Written for commit 1489d55210.
Summary will update automatically on new commits.</sup>
<!-- End of auto-generated description by cubic. -->
mertunsall
