BrowserSession.clear_cookies() sends Network.clearBrowserCookies on the
root CDP client, which always fails:
RuntimeError: {'code': -32601,
'message': "'Network.clearBrowserCookies' wasn't found"}
Network is a per-target domain and is not dispatchable without a session
attachment; the root client has none. Storage is browser-level, and the
cookies() getter two lines above already uses Storage.getCookies() there.
Measured against a live CDP endpoint:
Network.clearBrowserCookies() REJECTED -32601
Storage.clearCookies() OK
Storage.clearCookies(session_id=...) OK
Network.clearBrowserCookies(session_id=...) OK
The private _cdp_clear_cookies() already uses Storage.clearCookies with a
session_id, though its docstring still refers to Network.clearBrowserCookies
- this looks like the public method was missed when that one was fixed.
- Read BROWSER_USE_HEADLESS env var via default_factory in BrowserProfile
- Preserve fallback to display detection when env var is unset
- Add unit tests covering env var parsing and overrides
Fixes#5420
On a remote browser, the downloadProgress(completed) handler only dispatched FileDownloadedEvent on the event bus and never invoked the registered _download_complete_callbacks. The local branch does this inside _track_download(), so DefaultActionWatchdog._execute_click_with_download_detection waited on its on_download_complete asyncio.Event until the 30s timeout even though the download had finished.
Call the complete callbacks (with the same complete_info shape _track_download uses) right before dispatching the event in the remote branch. Adds a regression test that drives the registered downloadProgress handler and asserts the callback fires; the test fails on the unpatched code.
BrowserProfile.storage_state accepts str | Path | dict, but the
StorageStateWatchdog event handlers stringified the profile value
unconditionally. A dict became "{'cookies': [...]}", failed the
os.path.exists() check in _load_storage_state, and was silently
dropped, so cookies never reached the browser. The save path had the
same bug: the stringified dict was used as a file path, creating
junk directories named after the dict repr.
Preserve dicts through the event handlers, apply them directly in
_load_storage_state, and report a sentinel path in
StorageStateLoadedEvent instead of the repr (which would leak cookie
values). The existing dict guard in _save_storage_state is now
reachable via the event path.
Closes#4257
Page.navigate omits loaderId for same-document navigations (#fragment,
History API), and Chrome emits no new load/DOMContentLoaded lifecycle events
for them — the navigation is already committed when Page.navigate returns.
The stale-event timestamp guard would otherwise reject all buffered events
and burn the full readiness timeout.
Short-circuit when loaderId is absent, and simplify the stale-event guard
(the no-navigation-id case can no longer reach it). Regression test drains
the previous load's trailing networkIdle first so a stale event can't
accidentally satisfy the wait.
Navigation waits polled a per-session event deque whose feeding handler was
registered per-session on cdp-use's single-slot event registry. Any later
target attach replaced the handler, freezing existing tabs' deques with only
pre-navigation events, so every navigation on those tabs burned the full
readiness timeout (3s same-domain / 8s cross-domain) and then proceeded on a
page in unknown load state.
- Store lifecycle events per target_id in SessionManager, fed by ONE global
Page.lifecycleEvent handler registered in start_monitoring() and routed by
session_id; buffers are freed on target removal
- _navigate_and_wait reads the per-target buffer and now returns a timeout
status string instead of swallowing readiness timeouts;
on_NavigateToUrlEvent surfaces it via NavigationCompleteEvent.loading_status
- Skip loaderId-less lifecycle events that predate the current navigation
- Drop unused CDPSession._lifecycle_lock
Deterministic regression test: navigating tab A after opening tab B took
exactly the 3s fallback timeout before this fix, <0.5s after.
bubus derives the default bus name from the class name, so the
ResilientEventBus default factory changed session bus names from
EventBus_* to ResilientEventBus_*, breaking the EventBus_ prefix contract
asserted in tests/ci/browser/test_session_start.py. Default the subclass
name back to EventBus_<id> when none is given (explicit names still honored).
The V2 worker reuses a keep_alive BrowserSession across warm Lambda
invocations. At the end of a run Agent.close() stops the session's event
bus and nulls out its async primitives (event_queue / _on_idle) to release
the event loop. On resume the worker can step() the bus before any
dispatch() restarts it, and stock bubus EventBus.step() asserts
"EventBus._start() must be called before step()" in that state — crashing
the run deterministically so the task dead-letters after max receives
(~1,600+ occurrences over 2 days).
Wrap the session's bus in a ResilientEventBus subclass whose step() and
wait_until_idle() are safe no-ops when the bus has not been started,
instead of asserting. The nulling stays (it's what lets the next dispatch()
recreate a fresh queue and _start() the bus), so a later dispatch() still
restarts the bus and processes events normally.
Fixes ENG-5280.
Drop the per-call-site rationale text; the helper names already convey
what's happening, and the WHY lives in the commit history. Shrink both
helper docstrings to one line.
No behavior change; 16/16 tests still pass.
Condense the running narrative across four prior fix commits into a single
docstring summarizing what canonicalization the function mirrors and the
never-throw invariant. Drop the per-step justifications — those live in
the commit messages where they belong.
No behavior change; 40/40 tests still pass.
Addresses third P1 codex review on #4866.
Per RFC 3490 / UTS46, four code points act as label separators in IDNA
processing — `.` (U+002E), `。` (U+3002 IDEOGRAPHIC FULL STOP),
`．` (U+FF0E FULLWIDTH FULL STOP), and `｡` (U+FF61 HALFWIDTH IDEOGRAPHIC
FULL STOP). WHATWG URL parsing folds all four to `.` before resolution,
so `http://127。0。0。1/` and `http://127｡0｡0｡1/` reach 127.0.0.1.
NFKC handles only U+FF0E and partially maps U+FF61 → U+3002, leaving the
IPv4 parser unable to match the dominant ideographic-dot form. Explicitly
replace U+3002 and U+FF61 with `.` after NFKC.
Test `test_idna_dot_separators_blocked` covers all four dot variants
both at `_is_url_allowed` and `_is_ip_address` levels, plus a combined
case with circled digits (`①②⑦。⓪。⓪。①`).
Magnus Müller