Commit Graph

714 Commits

Author SHA1 Message Date
LarsenCundric 9a6510a85c fix: respect proxy_country_code=None to disable proxy in cloud mode
When a user passes proxy_country_code=None to BrowserSession, they
intend to disable the proxy. Previously, None was indistinguishable
from "not passed" because both used None as the default value. The
server then applied its default (US proxy) instead of disabling it.

Introduces an _UNSET sentinel to distinguish "not passed" (server
default) from "explicitly None" (no proxy). When proxy_country_code
is explicitly set to None, it is now included in the CreateBrowserRequest
and sent as null in the API body, which the server correctly interprets
as "disable proxy".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-18 11:23:41 -07:00
Saurav Panda 9a09c4d7dc fix(cdp): timeout-wrap CDPClient.send_raw to break silent WebSocket hangs
cdp_use.CDPClient.send_raw awaits a future that only resolves when the
browser sends a response with a matching message id. There is no timeout
on that await. Against the cloud browser service, the failure mode we
observed is: WebSocket stays alive at the TCP/keepalive layer (proxy
keeps pong-ing our pings), but the browser upstream is dead / unhealthy
and never sends any CDP response. send_raw's future never resolves, and
every higher-level timeout in browser-use (session.start's 15s connect
guard, agent.step_timeout, tools.act's action timeout) relies on
eventually getting a response — so they all wait forever too.

Evidence from a 170k-task collector run: 1,090 empty-history traces,
100% hit the 240s outer watchdog, median duration 582s, max 2214s, with
cloud HTTP layer clean throughout (all 200/201). One sample showed
/json/version returning 200 OK and then 5 minutes of total silence on
the WebSocket before forced stop — classic silent-hang.

Fix: add TimeoutWrappedCDPClient, a thin subclass of cdp_use.CDPClient
that wraps send_raw in asyncio.wait_for(timeout=cdp_request_timeout_s).
Any CDP method that doesn't respond within the cap raises plain
TimeoutError, which propagates through existing `except TimeoutError`
handlers in session.py / tools/service.py. Uses the same defensive env
parse pattern as BROWSER_USE_ACTION_TIMEOUT_S — rejects empty /
non-numeric / nan / inf / non-positive values with a warning fallback.

Default is 60s: generous for slow operations like Page.captureScreenshot
or Page.printToPDF on heavy pages, but well below the 180s step timeout
and any typical outer watchdog. Override via BROWSER_USE_CDP_TIMEOUT_S.

Wired into both CDPClient construction sites in session.py (initial
connect + reconnect path). All 17 existing real-browser tests
(test_action_blank_page, test_multi_act_guards) still pass.
2026-04-20 17:40:32 -07:00
laithrw 76604913ad Merge branch 'main' into fix/browser-session-close 2026-04-12 17:45:44 -04:00
Laith Weinberger c1eb87a35f close alias for BrowserSession stop
thousands of users have attempted to use close, so why not add it
2026-04-12 17:43:04 -04:00
Alezander9 76569995fd Improve OSS-to-cloud conversion: UTM tracking, better error messages, and cloud nudges
- Add UTM params to all cloud-bound links across README, CLI, and error messages
- Rewrite README Open Source vs Cloud section: position cloud browsers as
  recommended pairing for OSS users, remove separate Use Both section
- Rewrite error messages for use_cloud=True and ChatBrowserUse() to clearly
  state what is wrong and what to do next
- Add missing URLs: invalid API key now links to key page, insufficient
  credits now links to billing page
- Add cloud browser nudge on captcha detection (logger.warning)
- Add cloud browser nudge on local browser launch failure
2026-04-08 22:05:50 -07:00
LarsenCundric 5e644981e8 fix: close CloudBrowserClient httpx pool on session stop to prevent memory leak
BrowserSession.on_BrowserStopEvent calls stop_browser() but never calls
_cloud_browser_client.close(), leaving the httpx connection pool alive.
On Lambda provisioned concurrency, these pools accumulate across
invocations — memory climbs from ~1.3GB to the 3GB ceiling over hours,
triggering OOM kills (21 Runtime.ExitError crashes in 6 hours observed
in production).

Changes:
- Call _cloud_browser_client.close() in on_BrowserStopEvent after
  stop_browser completes (in a finally block so it runs even if
  stop_browser fails)
- Make CloudBrowserClient.close() idempotent (check is_closed before
  calling aclose) so it's safe to call multiple times

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 11:19:43 -07:00
laithrw 4c883feb7c Merge branch 'main' into fix/cloud-session-stop-cdp-reconnect 2026-03-25 19:34:25 -04:00
laithrw a2313d9665 Merge branch 'main' into feature/cdp-navigation-timeout 2026-03-25 17:30:42 -04:00
Laith Weinberger e44ce7ce73 fix linux profile detection for chromium users 2026-03-24 18:14:22 -04:00
revanthkumar96 1c4182d4ec Add nav_timeout parameter to control CDP Page.navigate() timeout
- Add nav_timeout parameter to navigate_to_url_and_wait() method
- Pass nav_timeout from NavigateToUrlEvent to control CDP navigation timeout
- Fix issue where TIMEOUT_NavigateToUrlEvent env var only controlled outer timeout but CDP call had hardcoded 20s timeout
- Add comprehensive test case to verify env var controls CDP timeout
2026-03-24 11:19:34 +05:30
Saurav Panda 03de83a069 Merge branch 'main' into iframe-click-issue 2026-03-21 15:57:02 -07:00
Saurav Panda 3b8f2457b1 fix: index elements inside cross-origin iframes reliably 2026-03-21 15:54:35 -07:00
shawn pana a2c2fa33aa Merge branch 'main' into cli 2026-03-20 09:34:45 -07:00
reformedot 2675c52e65 Enhance User-Agent header handling in BrowserSession
- Updated the BrowserSession class to ensure the User-Agent header is set correctly for both local and remote connections.
- Introduced a utility function to retrieve the browser-use version and incorporated it into the User-Agent string.
- Ensured headers are consistently handled as dictionaries to avoid potential issues with None values.

This change improves the reliability of the CDP client communication by providing a proper User-Agent, which can be crucial for certain web interactions.
2026-03-19 23:33:10 -07:00
ShawnPana 91c02012c3 merge origin/main into cli, resolve agent.py delete conflict 2026-03-19 21:06:38 -07:00
ShawnPana 694a111fad add upload command to CLI, extract find_file_input_near_element to BrowserSession
- Add `browser-use upload <index> <path>` command for uploading files to
  file input elements via the CLI
- Extract find_file_input_near_element from nested closures in tools/service.py
  to a reusable method on BrowserSession, deduplicating two copies
- Add BrowserWrapper.upload() for the Python REPL
- Resolve file paths to absolute on the client side before sending to daemon
- Update SKILL.md files and README with upload command docs
2026-03-19 17:02:34 -07:00
Saurav Panda 151de628e0 fixed the highlighting check logic 2026-03-16 16:56:54 -07:00
Saurav Panda f8c729d468 fix(browser): prevent remove_highlights() from blocking screenshot handler on slow pages 2026-03-16 10:05:53 -07:00
Saurav Panda 2344df3fac fix: pass through timeout_ms and clip params in event handlers 2026-03-07 14:34:08 -08:00
Laith Weinberger b630ae596e lint 2026-03-06 11:37:38 -05:00
Laith Weinberger b051fe2a7a don't explicitly pass newWindow: false to CDP createTarget 2026-03-06 11:28:36 -05:00
Saurav Panda 83908a7de1 Merge branch 'main' into ph-launch 2026-03-02 16:40:22 -08:00
Saurav Panda 10b5d1cd88 fix: allow cross-origin iframe targets with empty URLs through _is_valid_target() 2026-03-02 16:36:39 -08:00
Laith Weinberger 8ebe4385ed fix: preserve non-ASCII chars in all remaining JSON file outputs 2026-03-01 14:19:38 -05:00
Laith Weinberger 9932f06500 Merge main into fix/issue-4050-windows-proxy and resolve conflicts 2026-03-01 11:22:28 -05:00
laithrw 744c2be4ae Merge branch 'main' into issue-3523-fix 2026-02-26 21:05:37 -05:00
Laith Weinberger c8bfc86440 rm unused imports 2026-02-26 15:55:36 -05:00
Laith Weinberger 89817de334 Fix proxy auth attach handler overwriting tab tracking 2026-02-26 15:48:38 -05:00
iafisch 405d9ec624 fix: stop cloud sessions for cdp_url reconnect flows 2026-02-25 08:07:52 -05:00
Larsen Cundrič 2503c5f159 Merge branch 'main' into larsen/add-timeouts-to-browser-connects 2026-02-24 18:19:42 -08:00
LarsenCundric 505d5df406 Add timeouts to browser connects and waits 2026-02-24 18:13:28 -08:00
Saurav Panda 08bf06f734 Merge branch 'main' into websocket-connection-issues 2026-02-24 17:18:33 -08:00
Saurav Panda 719d0361e5 Fix reconnection wait timeout to cover full auto-reconnect window
The circuit breaker and agent error handler were using a hardcoded 20s
timeout, but the worst-case auto-reconnect window is ~48s (3 attempts *
15s + 7s delays). Handlers would raise ConnectionError while reconnection
was still in progress. Now both use BrowserSession.RECONNECT_WAIT_TIMEOUT
(54s) derived from the actual reconnect parameters.
2026-02-23 17:47:52 -08:00
Aitor c0528bec2d Merge branch 'main' into feat/add-captcha-solver-handling 2026-02-24 02:19:25 +01:00
Saurav Panda ea23b45daa Merge branch 'main' into websocket-connection-issues 2026-02-23 17:14:57 -08:00
Saurav Panda abd0326a6f Add WebSocket reconnection for remote browser CDP connections 2026-02-23 16:34:21 -08:00
Saurav Panda e50588f3c7 fixed styling issues 2026-02-23 15:24:40 -08:00
reformedot ed28e50806 feat: added automatic captcha solver handling via CAPTCHA watchdog 2026-02-23 10:51:02 -08:00
Saurav Panda 6713afc636 updated _navigate_and_wait 2026-02-21 19:45:54 -08:00
laithrw 2249e75fd8 Merge branch 'main' into improving-real-browser 2026-02-20 19:28:06 -05:00
LarsenCundric c9232a63ed Fix 2026-02-20 12:12:05 -08:00
LarsenCundric c67c31932e Fix 2026-02-20 10:57:00 -08:00
Laith Weinberger 4b81a4212d fix(storage_state): typing + format cleanup 2026-02-18 14:37:20 -05:00
Aitor 802f17cf91 Merge branch 'main' into feat/add-custom-headers-support-and-example 2026-02-18 06:22:42 +01:00
reformedot de659f9768 feat: add custom headres support + add example 2026-02-17 21:15:11 -08:00
Laith Weinberger fa51754895 fix(storage_state): await restore, normalize session cookies, scope storage to origin 2026-02-14 13:23:10 -05:00
Chase Xu d96fff3816 fix: only disable proxy for localhost CDP requests
Address review feedback: trust_env=False now only applies to localhost/127.0.0.1/::1.
Remote CDP URLs will still respect HTTP_PROXY/HTTPS_PROXY environment variables.
2026-02-09 20:55:30 -06:00
Chase Xu 2bd5211eb0 fix: disable proxy for localhost CDP requests (#4050)
On Windows with HTTP_PROXY/HTTPS_PROXY env vars set, httpx respects
these for localhost requests, causing 502 responses when querying
the CDP /json/version endpoint.

Adding trust_env=False prevents httpx from using proxy env vars,
ensuring localhost CDP communication works regardless of system
proxy configuration.
2026-02-09 20:55:30 -06:00
Laith Weinberger d8aef72702 lint 2026-01-27 11:58:25 -05:00
Laith Weinberger 77160b7d79 feat: add Browser.from_system_chrome() for real browser auto-detection
- auto-detects chrome executable and user data dir on macos/windows/linux
- auto-selects first available profile if none specified
- added Browser.list_chrome_profiles() to see available profiles
- updated examples to prompt user for profile selection
- fixed missing browser.stop() in save_cookies.py
2026-01-27 11:52:48 -05:00