Page.navigate omits loaderId for same-document navigations (#fragment,
History API), and Chrome emits no new load/DOMContentLoaded lifecycle events
for them — the navigation is already committed when Page.navigate returns.
The stale-event timestamp guard would otherwise reject all buffered events
and burn the full readiness timeout.
Short-circuit when loaderId is absent, and simplify the stale-event guard
(the no-navigation-id case can no longer reach it). Regression test drains
the previous load's trailing networkIdle first so a stale event can't
accidentally satisfy the wait.
Navigation waits polled a per-session event deque whose feeding handler was
registered per-session on cdp-use's single-slot event registry. Any later
target attach replaced the handler, freezing existing tabs' deques with only
pre-navigation events, so every navigation on those tabs burned the full
readiness timeout (3s same-domain / 8s cross-domain) and then proceeded on a
page in unknown load state.
- Store lifecycle events per target_id in SessionManager, fed by ONE global
Page.lifecycleEvent handler registered in start_monitoring() and routed by
session_id; buffers are freed on target removal
- _navigate_and_wait reads the per-target buffer and now returns a timeout
status string instead of swallowing readiness timeouts;
on_NavigateToUrlEvent surfaces it via NavigationCompleteEvent.loading_status
- Skip loaderId-less lifecycle events that predate the current navigation
- Drop unused CDPSession._lifecycle_lock
Deterministic regression test: navigating tab A after opening tab B took
exactly the 3s fallback timeout before this fix, <0.5s after.
bubus derives the default bus name from the class name, so the
ResilientEventBus default factory changed session bus names from
EventBus_* to ResilientEventBus_*, breaking the EventBus_ prefix contract
asserted in tests/ci/browser/test_session_start.py. Default the subclass
name back to EventBus_<id> when none is given (explicit names still honored).
The V2 worker reuses a keep_alive BrowserSession across warm Lambda
invocations. At the end of a run Agent.close() stops the session's event
bus and nulls out its async primitives (event_queue / _on_idle) to release
the event loop. On resume the worker can step() the bus before any
dispatch() restarts it, and stock bubus EventBus.step() asserts
"EventBus._start() must be called before step()" in that state — crashing
the run deterministically so the task dead-letters after max receives
(~1,600+ occurrences over 2 days).
Wrap the session's bus in a ResilientEventBus subclass whose step() and
wait_until_idle() are safe no-ops when the bus has not been started,
instead of asserting. The nulling stays (it's what lets the next dispatch()
recreate a fresh queue and _start() the bus), so a later dispatch() still
restarts the bus and processes events normally.
Fixes ENG-5280.
Drop the per-call-site rationale text; the helper names already convey
what's happening, and the WHY lives in the commit history. Shrink both
helper docstrings to one line.
No behavior change; 16/16 tests still pass.
Condense the running narrative across four prior fix commits into a single
docstring summarizing what canonicalization the function mirrors and the
never-throw invariant. Drop the per-step justifications — those live in
the commit messages where they belong.
No behavior change; 40/40 tests still pass.
Addresses third P1 codex review on #4866.
Per RFC 3490 / UTS46, four code points act as label separators in IDNA
processing — `.` (U+002E), `。` (U+3002 IDEOGRAPHIC FULL STOP),
`．` (U+FF0E FULLWIDTH FULL STOP), and `｡` (U+FF61 HALFWIDTH IDEOGRAPHIC
FULL STOP). WHATWG URL parsing folds all four to `.` before resolution,
so `http://127。0。0。1/` and `http://127｡0｡0｡1/` reach 127.0.0.1.
NFKC handles only U+FF0E and partially maps U+FF61 → U+3002, leaving the
IPv4 parser unable to match the dominant ideographic-dot form. Explicitly
replace U+3002 and U+FF61 with `.` after NFKC.
Test `test_idna_dot_separators_blocked` covers all four dot variants
both at `_is_url_allowed` and `_is_ip_address` levels, plus a combined
case with circled digits (`①②⑦。⓪。⓪。①`).
Addresses second P1 codex review on #4866.
WHATWG URL canonicalization maps fullwidth digits (`１２７.０.０.１`),
circled digits (`①②⑦.⓪.⓪.①`), and Unicode-prefixed hex forms
(`０x7f000001`) to ASCII IPv4 literals (`127.0.0.1` and `0x7f000001`)
before resolution. The classifier saw only the original string and
returned False, so `block_ip_addresses=True` was still bypassable by
re-encoding the IP with any equivalent Unicode digit variant.
Add a `unicodedata.normalize('NFKC', ...)` step after percent-decoding
and before parsing. NFKC handles all the bot's example forms and is
stdlib-only (no IDNA dependency). Wrapped in try/except to preserve the
never-throw invariant.
Tests:
- test_unicode_normalized_ipv4_blocked covers fullwidth, fullwidth+ASCII
hex, and circled-digit forms at both `_is_url_allowed` and
`_is_ip_address` boundaries.
- test_idn_domains_not_misclassified_as_ip is a false-positive guard
ensuring legitimate IDN domains (`café.example`, `日本.example`, their
punycode equivalents) remain classified as domains.
Addresses second codex review on #4866 (P1).
Chromium percent-decodes the host component before applying its IPv4
parser, so a URL like `http://%30x7f000001/` (decodes to `0x7f000001`,
i.e. 127.0.0.1) or `http://%31%32%37.0.0.1/` (decodes to `127.0.0.1`)
still reaches the IP address despite `block_ip_addresses=True`. Without
decoding, `_is_ip_address` sees the literal `%`-encoded string and
returns False — bypassing the block.
Call `urllib.parse.unquote` on the host before passing it to both
`ipaddress.ip_address` and `socket.inet_aton`. Wrap in try/except to
preserve the never-throw invariant for the classifier.
Adds two regression tests:
- test_percent_encoded_ipv4_blocked covers mixed (`%30x7f000001`), fully
encoded canonical (`%31%32%37.0.0.1`), and fully encoded decimal
(`%32%31%33%30%37%30%36%34%33%33`) bypass forms.
- test_malformed_percent_encoding_does_not_crash covers lone `%`, `%zz`,
`%2` — `unquote` leaves these as-is and the classifier must not throw.
Addresses codex review on #4866.
`socket.inet_aton` raises `UnicodeEncodeError` (not `OSError`) for
hostnames containing lone surrogates — common in URLs produced by
URL-decoding malformed UTF-8. The earlier patch only caught `OSError`,
so `_is_ip_address('\udcff')` would propagate the exception through
`_is_url_allowed` and crash the navigation security check whenever
`block_ip_addresses=True`.
Restore the original code's defensive `except Exception` posture for
both `ipaddress.ip_address` and `socket.inet_aton`. The classifier
should never throw — it returns True (recognized IP) or False (not a
recognizable IP); downstream domain-allowlist handling then applies.
Regression test covers lone-surrogate hostnames in
`test_malformed_unicode_hostnames_do_not_crash_classifier`.
GHSA-rv9j-wqjp-2fv4 (critical), GHSA-66xh-g88g-2h8j, GHSA-hpr4-fqgr-xhj9.
`DownloadsWatchdog` joined attacker-controlled filenames from CDP
(`Page.downloadWillBegin.suggestedFilename`) and `Content-Disposition`
headers directly into the configured `downloads_path`. Strings like
`../../escape.bin` or `/etc/shadow.bak` would `os.path.join` outside the
downloads directory, writing the fetched bytes (also attacker-controlled
— the response body is the exploit content) to an arbitrary location
with the agent's process privileges.
`download_file_from_url` triggers passively for any
`Content-Disposition: attachment` response, so this is reachable from any
visited site — `allowed_domains` does not mitigate it.
Add two private helpers on DownloadsWatchdog:
- `_sanitize_download_filename(name)`: keep only the basename, normalize
Windows separators, strip null bytes, fall back to `'download'` for
empty / pure-traversal inputs.
- `_is_path_contained(path, dir)`: realpath containment check for the
on-disk sinks.
Wire the sanitizer at every attacker-controlled filename ingress:
- `download_will_begin_handler` (CDP suggestedFilename → cache + events)
- `_handle_cdp_download` (same field, separate path)
- Network-monitor Content-Disposition parser
- `download_file_from_url` (suggested_filename argument)
- `_handle_download` (Playwright `download.suggested_filename`)
Wire the containment check at every on-disk write site:
- `download_file_from_url` write
- `_handle_download` (Playwright save_as path)
- `trigger_pdf_download` write (defense in depth — already basename'd)
GHSA-xrfv-gg9f-wwjp, GHSA-g27c-8gp4-28cv.
`SecurityWatchdog._is_ip_address` only recognized IP strings that
`ipaddress.ip_address()` accepts — i.e. the canonical dotted-quad form
(`127.0.0.1`) and full IPv6. Chromium and the kernel resolver, however,
also accept several non-standard IPv4 representations:
http://2130706433/ → 127.0.0.1 (decimal int)
http://0x7f000001/ → 127.0.0.1 (hex)
http://0177.0.0.1/ → 127.0.0.1 (octal)
http://127.1/ → 127.0.0.1 (short-form)
http://127.0.1/ → 127.0.0.1 (short-form)
`block_ip_addresses=True` was therefore trivially bypassed by re-encoding
the IP in any of these forms.
Fall back to `socket.inet_aton` after `ipaddress.ip_address()` fails — it
accepts the same liberal IPv4 forms the kernel resolver does, so the
classifier matches the browser's behavior.
The existing `test_ipv4_lookalike_domains_allowed` test was codifying the
buggy behavior for `1.2.3` (which IS a short-form IPv4 == 1.2.0.3).
Removed that assertion and added a dedicated `TestNonStandardIPv4Representations`
class covering decimal/hex/octal/short-form blocking, lookalike-domain
non-interference, and the interaction with `allowed_domains`.
When a user passes proxy_country_code=None to BrowserSession, they
intend to disable the proxy. Previously, None was indistinguishable
from "not passed" because both used None as the default value. The
server then applied its default (US proxy) instead of disabling it.
Introduces an _UNSET sentinel to distinguish "not passed" (server
default) from "explicitly None" (no proxy). When proxy_country_code
is explicitly set to None, it is now included in the CreateBrowserRequest
and sent as null in the API body, which the server correctly interprets
as "disable proxy".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ord() raises TypeError when called on a multi-character string. Several
Unicode characters expand on case conversion ('ß'.upper() == 'SS',
'ffi'.upper() == 'FFI', 'ʼn'.upper() == 'ʼN'). Typing any of these caused
'Failed to input text via CDP: TypeError: ord() expected a character,
but string of length 2 found' and aborted the action.
Fall back to the original character's code point when upper()/lower()
returns more than one code point. The Windows VK code is only an
approximation anyway; the 'text' field passed to dispatchKeyEvent
carries the actual character.
BrowserError.__str__ unconditionally appended 'during: <while_handling_event>',
so any error with details set but no event (e.g. ScrollToText 'Text not found',
click failures raised from action handlers) renders as
Text not found: "X" ({'text': 'X'}) during: None
The 'during: None' tail is logging noise that surfaces ~1.6k times/day in
prod (Datadog: status:error env:production '"\'\}) during: None"', 24h).
Build the string from the parts that are actually populated, instead of
two mutually-exclusive branches that always print one of them.
Two P2 comments from cubic on 9a09c4d7:
1. TimeoutWrappedCDPClient.__init__ trusted its cdp_request_timeout_s arg
blindly. nan / inf / <=0 would either make every CDP call time out
immediately (nan) or disable the guard (inf / <=0) — same defensive
gap we already fixed for the env-var path. Extracted _coerce_valid_
timeout() that mirrors _parse_env_cdp_timeout's validation; constructor
now routes through it, so both entry points are equally safe.
2. test_send_raw_times_out_on_silent_server used an inline copy of the
wrapper logic rather than the real TimeoutWrappedCDPClient.send_raw.
A regression in the production method — e.g. accidentally removing
the asyncio.wait_for — would not fail the test. Rewrote to:
- Construct via __new__ (skip CDPClient.__init__'s WebSocket setup)
- unittest.mock.patch the parent CDPClient.send_raw with a hanging
coroutine
- Call the real TimeoutWrappedCDPClient.send_raw, which does
super().send_raw(...) → our patched stub
- Assert it raises TimeoutError within the cap
Also added test_send_raw_passes_through_when_fast (fast-path regression
guard) and test_constructor_rejects_invalid_timeout (validation for
fix#1). All 14 tests in the timeout suite pass locally.
cdp_use.CDPClient.send_raw awaits a future that only resolves when the
browser sends a response with a matching message id. There is no timeout
on that await. Against the cloud browser service, the failure mode we
observed is: WebSocket stays alive at the TCP/keepalive layer (proxy
keeps pong-ing our pings), but the browser upstream is dead / unhealthy
and never sends any CDP response. send_raw's future never resolves, and
every higher-level timeout in browser-use (session.start's 15s connect
guard, agent.step_timeout, tools.act's action timeout) relies on
eventually getting a response — so they all wait forever too.
Evidence from a 170k-task collector run: 1,090 empty-history traces,
100% hit the 240s outer watchdog, median duration 582s, max 2214s, with
cloud HTTP layer clean throughout (all 200/201). One sample showed
/json/version returning 200 OK and then 5 minutes of total silence on
the WebSocket before forced stop — classic silent-hang.
Fix: add TimeoutWrappedCDPClient, a thin subclass of cdp_use.CDPClient
that wraps send_raw in asyncio.wait_for(timeout=cdp_request_timeout_s).
Any CDP method that doesn't respond within the cap raises plain
TimeoutError, which propagates through existing `except TimeoutError`
handlers in session.py / tools/service.py. Uses the same defensive env
parse pattern as BROWSER_USE_ACTION_TIMEOUT_S — rejects empty /
non-numeric / nan / inf / non-positive values with a warning fallback.
Default is 60s: generous for slow operations like Page.captureScreenshot
or Page.printToPDF on heavy pages, but well below the 180s step timeout
and any typical outer watchdog. Override via BROWSER_USE_CDP_TIMEOUT_S.
Wired into both CDPClient construction sites in session.py (initial
connect + reconnect path). All 17 existing real-browser tests
(test_action_blank_page, test_multi_act_guards) still pass.
- `on_BrowserConnectedEvent` now catches `RuntimeError` from
`start_recording()` so sessions with `record_video_dir` configured but
missing `[video]` extras (or a viewport that can't be sized) keep
starting — prior graceful-degradation behavior is restored.
- Lazy `RecordingWatchdog` in the CLI handler now calls
`attach_to_session()`, so `AgentFocusChangedEvent` / `BrowserStopEvent`
handlers are wired correctly if the session dispatches them.
- Daemon shutdown finalizes any in-progress recording before tearing the
browser down, preventing truncated MP4s on `close`, idle timeout, or
signal-driven exit.
- Added regression test that monkeypatches `start_recording` to raise and
asserts `on_BrowserConnectedEvent` swallows it without breaking startup.
Closes#4533.
- `RecordingWatchdog` gains public `start_recording(path, size?, framerate?)`,
`stop_recording() -> Path`, and `is_recording`; the existing
`BrowserConnectedEvent`/`BrowserStopEvent` path is refactored to use them,
so profile-driven recording behavior is unchanged.
- `browser-use record start <path>` / `record stop` / `record status`
subcommands wired through argparse, daemon dispatch, and the browser
command handler. `record stop` prints the saved file path so it can be
captured programmatically, matching the issue's requested UX. Works with
`--session NAME` via the existing named-daemon infrastructure.
- The CLI's `CLIBrowserSession` intentionally skips watchdogs; the handler
lazily instantiates `RecordingWatchdog` on first `record start` so CLI
recording doesn't pay the watchdog-setup cost for non-recording sessions.
- Output format is `.mp4` (libx264) since that's what the existing
`VideoRecorderService` encodes; optional dependency gate is unchanged
(`pip install "browser-use[video]"`).
- New `tests/ci/test_action_record.py` exercises the full stack against a
real headless browser + `pytest-httpserver`, verifying decodable MP4
output, double-start rejection, stop-without-start no-op, that the
existing `profile.record_video_dir` flow still works, and the argparse /
dispatch wiring.
- Add UTM params to all cloud-bound links across README, CLI, and error messages
- Rewrite README Open Source vs Cloud section: position cloud browsers as
recommended pairing for OSS users, remove separate Use Both section
- Rewrite error messages for use_cloud=True and ChatBrowserUse() to clearly
state what is wrong and what to do next
- Add missing URLs: invalid API key now links to key page, insufficient
credits now links to billing page
- Add cloud browser nudge on captcha detection (logger.warning)
- Add cloud browser nudge on local browser launch failure
Library keeps recording off by default. CLI reads cloud_connect_recording
from config (defaults True). Users can disable with:
browser-use config set cloud_connect_recording false
On slow CI machines, _get_pending_network_requests() can hang for 15s+
when Chrome is busy loading/rendering after a navigation. This silently
eats into the 30s BrowserStateRequestEvent budget, leaving insufficient
time for the actual DOM capture — causing 5 consecutive timeouts and
agent termination.
Observed on eBay search results in eval runs: DOMWatchdog started but
produced zero log output for 15 seconds before the timeout killed it.
The pending network check was the first await after the URL log.
cloud connect reads cloud_connect_proxy and cloud_connect_timeout from
~/.browser-use/config.json. Recording always enabled via enableRecording
default on CreateBrowserRequest. No CLI flags — edit config for custom
settings, use cloud v2 REST for full control.
BrowserSession.on_BrowserStopEvent calls stop_browser() but never calls
_cloud_browser_client.close(), leaving the httpx connection pool alive.
On Lambda provisioned concurrency, these pools accumulate across
invocations — memory climbs from ~1.3GB to the 3GB ceiling over hours,
triggering OOM kills (21 Runtime.ExitError crashes in 6 hours observed
in production).
Changes:
- Call _cloud_browser_client.close() in on_BrowserStopEvent after
stop_browser completes (in a finally block so it runs even if
stop_browser fails)
- Make CloudBrowserClient.close() idempotent (check is_closed before
calling aclose) so it's safe to call multiple times
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add nav_timeout parameter to navigate_to_url_and_wait() method
- Pass nav_timeout from NavigateToUrlEvent to control CDP navigation timeout
- Fix issue where TIMEOUT_NavigateToUrlEvent env var only controlled outer timeout but CDP call had hardcoded 20s timeout
- Add comprehensive test case to verify env var controls CDP timeout
Saurav Panda