Commit Graph

733 Commits

Author SHA1 Message Date
Magnus Müller b78b3fcebd Merge branch 'main' into fix/session-handler-registration 2026-09-03 09:00:42 -07:00
Magnus Müller 30b5aca5b0 Merge branch 'main' into fix/issue-5366 2026-08-28 02:25:57 -07:00
Matthew Elliot 2f573908ca fix(browser): clear cookies via Storage domain, not Network
BrowserSession.clear_cookies() sends Network.clearBrowserCookies on the
root CDP client, which always fails:

  RuntimeError: {'code': -32601,
                 'message': "'Network.clearBrowserCookies' wasn't found"}

Network is a per-target domain and is not dispatchable without a session
attachment; the root client has none. Storage is browser-level, and the
cookies() getter two lines above already uses Storage.getCookies() there.

Measured against a live CDP endpoint:

  Network.clearBrowserCookies()                  REJECTED -32601
  Storage.clearCookies()                         OK
  Storage.clearCookies(session_id=...)           OK
  Network.clearBrowserCookies(session_id=...)    OK

The private _cdp_clear_cookies() already uses Storage.clearCookies with a
session_id, though its docstring still refers to Network.clearBrowserCookies
- this looks like the public method was missed when that one was fixed.
2026-08-27 13:06:53 +02:00
Shantanav 2c3162069e Fix reconnect drop handling during reconnect
Record WebSocket drops that occur during an in-flight reconnect and schedule a retry after the reconnect finishes.\n\nFixes #5366
2026-08-03 23:20:34 +05:30
MagMueller 05b68eb567 test(browser): verify state recovery after timeout 2026-07-24 12:30:06 -07:00
MagMueller e7cc40f5ad fix(browser): expose minimal state after capture timeout 2026-07-24 11:51:16 -07:00
MagMueller 245cdd8dfc Revert "fix(browser): recover with cached state after timeout"
This reverts commit bd45e4c22a.
2026-07-24 10:38:26 -07:00
MagMueller b7ac967ea5 Revert "fix(browser): bound cached state recovery attempts"
This reverts commit 394ccc157e.
2026-07-24 10:38:26 -07:00
MagMueller 394ccc157e fix(browser): bound cached state recovery attempts 2026-07-23 21:59:58 -07:00
MagMueller bd45e4c22a fix(browser): recover with cached state after timeout 2026-07-23 21:59:58 -07:00
MagMueller 5a4b02c4bd Keep selector indices unique across CDP sessions 2026-07-23 15:44:03 -07:00
lisa0314 554ce8f99e Restore session handlers after event bus reset 2026-07-15 10:07:05 +08:00
Saurav Panda b4b6868232 Handle same-document navigations without burning the readiness timeout
Page.navigate omits loaderId for same-document navigations (#fragment,
History API), and Chrome emits no new load/DOMContentLoaded lifecycle events
for them — the navigation is already committed when Page.navigate returns.
The stale-event timestamp guard would otherwise reject all buffered events
and burn the full readiness timeout.

Short-circuit when loaderId is absent, and simplify the stale-event guard
(the no-navigation-id case can no longer reach it). Regression test drains
the previous load's trailing networkIdle first so a stale event can't
accidentally satisfy the wait.
2026-07-04 04:27:18 +08:00
Saurav Panda c5f0fa767c Fix navigation readiness detection: per-target lifecycle event storage
Navigation waits polled a per-session event deque whose feeding handler was
registered per-session on cdp-use's single-slot event registry. Any later
target attach replaced the handler, freezing existing tabs' deques with only
pre-navigation events, so every navigation on those tabs burned the full
readiness timeout (3s same-domain / 8s cross-domain) and then proceeded on a
page in unknown load state.

- Store lifecycle events per target_id in SessionManager, fed by ONE global
  Page.lifecycleEvent handler registered in start_monitoring() and routed by
  session_id; buffers are freed on target removal
- _navigate_and_wait reads the per-target buffer and now returns a timeout
  status string instead of swallowing readiness timeouts;
  on_NavigateToUrlEvent surfaces it via NavigationCompleteEvent.loading_status
- Skip loaderId-less lifecycle events that predate the current navigation
- Drop unused CDPSession._lifecycle_lock

Deterministic regression test: navigating tab A after opening tab B took
exactly the 3s fallback timeout before this fix, <0.5s after.
2026-07-03 22:48:20 +08:00
Laith Weinberger 53c3ba2e72 Fix browser use CLI review issues 2026-06-29 14:00:13 +08:00
Laith Weinberger f768a06cfe new browser use CLI; new functions for deleted CLI methods 2026-06-29 13:10:55 +08:00
Saurav Panda 5123e4fc08 style: tighten ResilientEventBus docstring and comments 2026-06-26 15:56:17 +08:00
Saurav Panda 9c1db02dd3 fix(browser): preserve EventBus_ name prefix on ResilientEventBus
bubus derives the default bus name from the class name, so the
ResilientEventBus default factory changed session bus names from
EventBus_* to ResilientEventBus_*, breaking the EventBus_ prefix contract
asserted in tests/ci/browser/test_session_start.py. Default the subclass
name back to EventBus_<id> when none is given (explicit names still honored).
2026-06-26 15:48:59 +08:00
Saurav Panda 16c2745a37 fix(browser): tolerate stepping a torn-down event bus on warm-Lambda resume
The V2 worker reuses a keep_alive BrowserSession across warm Lambda
invocations. At the end of a run Agent.close() stops the session's event
bus and nulls out its async primitives (event_queue / _on_idle) to release
the event loop. On resume the worker can step() the bus before any
dispatch() restarts it, and stock bubus EventBus.step() asserts
"EventBus._start() must be called before step()" in that state — crashing
the run deterministically so the task dead-letters after max receives
(~1,600+ occurrences over 2 days).

Wrap the session's bus in a ResilientEventBus subclass whose step() and
wait_until_idle() are safe no-ops when the bus has not been started,
instead of asserting. The nulling stays (it's what lets the next dispatch()
recreate a fresh queue and _start() the bus), so a later dispatch() still
restarts the bus and processes events normally.

Fixes ENG-5280.
2026-06-26 13:28:53 +08:00
LarsenCundric 9a6510a85c fix: respect proxy_country_code=None to disable proxy in cloud mode
When a user passes proxy_country_code=None to BrowserSession, they
intend to disable the proxy. Previously, None was indistinguishable
from "not passed" because both used None as the default value. The
server then applied its default (US proxy) instead of disabling it.

Introduces an _UNSET sentinel to distinguish "not passed" (server
default) from "explicitly None" (no proxy). When proxy_country_code
is explicitly set to None, it is now included in the CreateBrowserRequest
and sent as null in the API body, which the server correctly interprets
as "disable proxy".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-18 11:23:41 -07:00
Saurav Panda 9a09c4d7dc fix(cdp): timeout-wrap CDPClient.send_raw to break silent WebSocket hangs
cdp_use.CDPClient.send_raw awaits a future that only resolves when the
browser sends a response with a matching message id. There is no timeout
on that await. Against the cloud browser service, the failure mode we
observed is: WebSocket stays alive at the TCP/keepalive layer (proxy
keeps pong-ing our pings), but the browser upstream is dead / unhealthy
and never sends any CDP response. send_raw's future never resolves, and
every higher-level timeout in browser-use (session.start's 15s connect
guard, agent.step_timeout, tools.act's action timeout) relies on
eventually getting a response — so they all wait forever too.

Evidence from a 170k-task collector run: 1,090 empty-history traces,
100% hit the 240s outer watchdog, median duration 582s, max 2214s, with
cloud HTTP layer clean throughout (all 200/201). One sample showed
/json/version returning 200 OK and then 5 minutes of total silence on
the WebSocket before forced stop — classic silent-hang.

Fix: add TimeoutWrappedCDPClient, a thin subclass of cdp_use.CDPClient
that wraps send_raw in asyncio.wait_for(timeout=cdp_request_timeout_s).
Any CDP method that doesn't respond within the cap raises plain
TimeoutError, which propagates through existing `except TimeoutError`
handlers in session.py / tools/service.py. Uses the same defensive env
parse pattern as BROWSER_USE_ACTION_TIMEOUT_S — rejects empty /
non-numeric / nan / inf / non-positive values with a warning fallback.

Default is 60s: generous for slow operations like Page.captureScreenshot
or Page.printToPDF on heavy pages, but well below the 180s step timeout
and any typical outer watchdog. Override via BROWSER_USE_CDP_TIMEOUT_S.

Wired into both CDPClient construction sites in session.py (initial
connect + reconnect path). All 17 existing real-browser tests
(test_action_blank_page, test_multi_act_guards) still pass.
2026-04-20 17:40:32 -07:00
laithrw 76604913ad Merge branch 'main' into fix/browser-session-close 2026-04-12 17:45:44 -04:00
Laith Weinberger c1eb87a35f close alias for BrowserSession stop
thousands of users have attempted to use close, so why not add it
2026-04-12 17:43:04 -04:00
Alezander9 76569995fd Improve OSS-to-cloud conversion: UTM tracking, better error messages, and cloud nudges
- Add UTM params to all cloud-bound links across README, CLI, and error messages
- Rewrite README Open Source vs Cloud section: position cloud browsers as
  recommended pairing for OSS users, remove separate Use Both section
- Rewrite error messages for use_cloud=True and ChatBrowserUse() to clearly
  state what is wrong and what to do next
- Add missing URLs: invalid API key now links to key page, insufficient
  credits now links to billing page
- Add cloud browser nudge on captcha detection (logger.warning)
- Add cloud browser nudge on local browser launch failure
2026-04-08 22:05:50 -07:00
LarsenCundric 5e644981e8 fix: close CloudBrowserClient httpx pool on session stop to prevent memory leak
BrowserSession.on_BrowserStopEvent calls stop_browser() but never calls
_cloud_browser_client.close(), leaving the httpx connection pool alive.
On Lambda provisioned concurrency, these pools accumulate across
invocations — memory climbs from ~1.3GB to the 3GB ceiling over hours,
triggering OOM kills (21 Runtime.ExitError crashes in 6 hours observed
in production).

Changes:
- Call _cloud_browser_client.close() in on_BrowserStopEvent after
  stop_browser completes (in a finally block so it runs even if
  stop_browser fails)
- Make CloudBrowserClient.close() idempotent (check is_closed before
  calling aclose) so it's safe to call multiple times

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 11:19:43 -07:00
laithrw 4c883feb7c Merge branch 'main' into fix/cloud-session-stop-cdp-reconnect 2026-03-25 19:34:25 -04:00
laithrw a2313d9665 Merge branch 'main' into feature/cdp-navigation-timeout 2026-03-25 17:30:42 -04:00
Laith Weinberger e44ce7ce73 fix linux profile detection for chromium users 2026-03-24 18:14:22 -04:00
revanthkumar96 1c4182d4ec Add nav_timeout parameter to control CDP Page.navigate() timeout
- Add nav_timeout parameter to navigate_to_url_and_wait() method
- Pass nav_timeout from NavigateToUrlEvent to control CDP navigation timeout
- Fix issue where TIMEOUT_NavigateToUrlEvent env var only controlled outer timeout but CDP call had hardcoded 20s timeout
- Add comprehensive test case to verify env var controls CDP timeout
2026-03-24 11:19:34 +05:30
Saurav Panda 03de83a069 Merge branch 'main' into iframe-click-issue 2026-03-21 15:57:02 -07:00
Saurav Panda 3b8f2457b1 fix: index elements inside cross-origin iframes reliably 2026-03-21 15:54:35 -07:00
shawn pana a2c2fa33aa Merge branch 'main' into cli 2026-03-20 09:34:45 -07:00
reformedot 2675c52e65 Enhance User-Agent header handling in BrowserSession
- Updated the BrowserSession class to ensure the User-Agent header is set correctly for both local and remote connections.
- Introduced a utility function to retrieve the browser-use version and incorporated it into the User-Agent string.
- Ensured headers are consistently handled as dictionaries to avoid potential issues with None values.

This change improves the reliability of the CDP client communication by providing a proper User-Agent, which can be crucial for certain web interactions.
2026-03-19 23:33:10 -07:00
ShawnPana 91c02012c3 merge origin/main into cli, resolve agent.py delete conflict 2026-03-19 21:06:38 -07:00
ShawnPana 694a111fad add upload command to CLI, extract find_file_input_near_element to BrowserSession
- Add `browser-use upload <index> <path>` command for uploading files to
  file input elements via the CLI
- Extract find_file_input_near_element from nested closures in tools/service.py
  to a reusable method on BrowserSession, deduplicating two copies
- Add BrowserWrapper.upload() for the Python REPL
- Resolve file paths to absolute on the client side before sending to daemon
- Update SKILL.md files and README with upload command docs
2026-03-19 17:02:34 -07:00
Saurav Panda 151de628e0 fixed the highlighting check logic 2026-03-16 16:56:54 -07:00
Saurav Panda f8c729d468 fix(browser): prevent remove_highlights() from blocking screenshot handler on slow pages 2026-03-16 10:05:53 -07:00
Saurav Panda 2344df3fac fix: pass through timeout_ms and clip params in event handlers 2026-03-07 14:34:08 -08:00
Laith Weinberger b630ae596e lint 2026-03-06 11:37:38 -05:00
Laith Weinberger b051fe2a7a don't explicitly pass newWindow: false to CDP createTarget 2026-03-06 11:28:36 -05:00
Saurav Panda 83908a7de1 Merge branch 'main' into ph-launch 2026-03-02 16:40:22 -08:00
Saurav Panda 10b5d1cd88 fix: allow cross-origin iframe targets with empty URLs through _is_valid_target() 2026-03-02 16:36:39 -08:00
Laith Weinberger 8ebe4385ed fix: preserve non-ASCII chars in all remaining JSON file outputs 2026-03-01 14:19:38 -05:00
Laith Weinberger 9932f06500 Merge main into fix/issue-4050-windows-proxy and resolve conflicts 2026-03-01 11:22:28 -05:00
laithrw 744c2be4ae Merge branch 'main' into issue-3523-fix 2026-02-26 21:05:37 -05:00
Laith Weinberger c8bfc86440 rm unused imports 2026-02-26 15:55:36 -05:00
Laith Weinberger 89817de334 Fix proxy auth attach handler overwriting tab tracking 2026-02-26 15:48:38 -05:00
iafisch 405d9ec624 fix: stop cloud sessions for cdp_url reconnect flows 2026-02-25 08:07:52 -05:00
Larsen Cundrič 2503c5f159 Merge branch 'main' into larsen/add-timeouts-to-browser-connects 2026-02-24 18:19:42 -08:00
LarsenCundric 505d5df406 Add timeouts to browser connects and waits 2026-02-24 18:13:28 -08:00