Drop the per-call-site rationale text; the helper names already convey
what's happening, and the WHY lives in the commit history. Shrink both
helper docstrings to one line.
No behavior change; 16/16 tests still pass.
Condense the running narrative across four prior fix commits into a single
docstring summarizing what canonicalization the function mirrors and the
never-throw invariant. Drop the per-step justifications — those live in
the commit messages where they belong.
No behavior change; 40/40 tests still pass.
Addresses third P1 codex review on #4866.
Per RFC 3490 / UTS46, four code points act as label separators in IDNA
processing — `.` (U+002E), `。` (U+3002 IDEOGRAPHIC FULL STOP),
`．` (U+FF0E FULLWIDTH FULL STOP), and `｡` (U+FF61 HALFWIDTH IDEOGRAPHIC
FULL STOP). WHATWG URL parsing folds all four to `.` before resolution,
so `http://127。0。0。1/` and `http://127｡0｡0｡1/` reach 127.0.0.1.
NFKC handles only U+FF0E and partially maps U+FF61 → U+3002, leaving the
IPv4 parser unable to match the dominant ideographic-dot form. Explicitly
replace U+3002 and U+FF61 with `.` after NFKC.
Test `test_idna_dot_separators_blocked` covers all four dot variants
both at `_is_url_allowed` and `_is_ip_address` levels, plus a combined
case with circled digits (`①②⑦。⓪。⓪。①`).
Addresses second P1 codex review on #4866.
WHATWG URL canonicalization maps fullwidth digits (`１２７.０.０.１`),
circled digits (`①②⑦.⓪.⓪.①`), and Unicode-prefixed hex forms
(`０x7f000001`) to ASCII IPv4 literals (`127.0.0.1` and `0x7f000001`)
before resolution. The classifier saw only the original string and
returned False, so `block_ip_addresses=True` was still bypassable by
re-encoding the IP with any equivalent Unicode digit variant.
Add a `unicodedata.normalize('NFKC', ...)` step after percent-decoding
and before parsing. NFKC handles all the bot's example forms and is
stdlib-only (no IDNA dependency). Wrapped in try/except to preserve the
never-throw invariant.
Tests:
- test_unicode_normalized_ipv4_blocked covers fullwidth, fullwidth+ASCII
hex, and circled-digit forms at both `_is_url_allowed` and
`_is_ip_address` boundaries.
- test_idn_domains_not_misclassified_as_ip is a false-positive guard
ensuring legitimate IDN domains (`café.example`, `日本.example`, their
punycode equivalents) remain classified as domains.
Addresses second codex review on #4866 (P1).
Chromium percent-decodes the host component before applying its IPv4
parser, so a URL like `http://%30x7f000001/` (decodes to `0x7f000001`,
i.e. 127.0.0.1) or `http://%31%32%37.0.0.1/` (decodes to `127.0.0.1`)
still reaches the IP address despite `block_ip_addresses=True`. Without
decoding, `_is_ip_address` sees the literal `%`-encoded string and
returns False — bypassing the block.
Call `urllib.parse.unquote` on the host before passing it to both
`ipaddress.ip_address` and `socket.inet_aton`. Wrap in try/except to
preserve the never-throw invariant for the classifier.
Adds two regression tests:
- test_percent_encoded_ipv4_blocked covers mixed (`%30x7f000001`), fully
encoded canonical (`%31%32%37.0.0.1`), and fully encoded decimal
(`%32%31%33%30%37%30%36%34%33%33`) bypass forms.
- test_malformed_percent_encoding_does_not_crash covers lone `%`, `%zz`,
`%2` — `unquote` leaves these as-is and the classifier must not throw.
Addresses codex review on #4866.
`socket.inet_aton` raises `UnicodeEncodeError` (not `OSError`) for
hostnames containing lone surrogates — common in URLs produced by
URL-decoding malformed UTF-8. The earlier patch only caught `OSError`,
so `_is_ip_address('\udcff')` would propagate the exception through
`_is_url_allowed` and crash the navigation security check whenever
`block_ip_addresses=True`.
Restore the original code's defensive `except Exception` posture for
both `ipaddress.ip_address` and `socket.inet_aton`. The classifier
should never throw — it returns True (recognized IP) or False (not a
recognizable IP); downstream domain-allowlist handling then applies.
Regression test covers lone-surrogate hostnames in
`test_malformed_unicode_hostnames_do_not_crash_classifier`.
GHSA-rv9j-wqjp-2fv4 (critical), GHSA-66xh-g88g-2h8j, GHSA-hpr4-fqgr-xhj9.
`DownloadsWatchdog` joined attacker-controlled filenames from CDP
(`Page.downloadWillBegin.suggestedFilename`) and `Content-Disposition`
headers directly into the configured `downloads_path`. Strings like
`../../escape.bin` or `/etc/shadow.bak` would `os.path.join` outside the
downloads directory, writing the fetched bytes (also attacker-controlled
— the response body is the exploit content) to an arbitrary location
with the agent's process privileges.
`download_file_from_url` triggers passively for any
`Content-Disposition: attachment` response, so this is reachable from any
visited site — `allowed_domains` does not mitigate it.
Add two private helpers on DownloadsWatchdog:
- `_sanitize_download_filename(name)`: keep only the basename, normalize
Windows separators, strip null bytes, fall back to `'download'` for
empty / pure-traversal inputs.
- `_is_path_contained(path, dir)`: realpath containment check for the
on-disk sinks.
Wire the sanitizer at every attacker-controlled filename ingress:
- `download_will_begin_handler` (CDP suggestedFilename → cache + events)
- `_handle_cdp_download` (same field, separate path)
- Network-monitor Content-Disposition parser
- `download_file_from_url` (suggested_filename argument)
- `_handle_download` (Playwright `download.suggested_filename`)
Wire the containment check at every on-disk write site:
- `download_file_from_url` write
- `_handle_download` (Playwright save_as path)
- `trigger_pdf_download` write (defense in depth — already basename'd)
GHSA-xrfv-gg9f-wwjp, GHSA-g27c-8gp4-28cv.
`SecurityWatchdog._is_ip_address` only recognized IP strings that
`ipaddress.ip_address()` accepts — i.e. the canonical dotted-quad form
(`127.0.0.1`) and full IPv6. Chromium and the kernel resolver, however,
also accept several non-standard IPv4 representations:
http://2130706433/ → 127.0.0.1 (decimal int)
http://0x7f000001/ → 127.0.0.1 (hex)
http://0177.0.0.1/ → 127.0.0.1 (octal)
http://127.1/ → 127.0.0.1 (short-form)
http://127.0.1/ → 127.0.0.1 (short-form)
`block_ip_addresses=True` was therefore trivially bypassed by re-encoding
the IP in any of these forms.
Fall back to `socket.inet_aton` after `ipaddress.ip_address()` fails — it
accepts the same liberal IPv4 forms the kernel resolver does, so the
classifier matches the browser's behavior.
The existing `test_ipv4_lookalike_domains_allowed` test was codifying the
buggy behavior for `1.2.3` (which IS a short-form IPv4 == 1.2.0.3).
Removed that assertion and added a dedicated `TestNonStandardIPv4Representations`
class covering decimal/hex/octal/short-form blocking, lookalike-domain
non-interference, and the interaction with `allowed_domains`.
ord() raises TypeError when called on a multi-character string. Several
Unicode characters expand on case conversion ('ß'.upper() == 'SS',
'ffi'.upper() == 'FFI', 'ʼn'.upper() == 'ʼN'). Typing any of these caused
'Failed to input text via CDP: TypeError: ord() expected a character,
but string of length 2 found' and aborted the action.
Fall back to the original character's code point when upper()/lower()
returns more than one code point. The Windows VK code is only an
approximation anyway; the 'text' field passed to dispatchKeyEvent
carries the actual character.
- `on_BrowserConnectedEvent` now catches `RuntimeError` from
`start_recording()` so sessions with `record_video_dir` configured but
missing `[video]` extras (or a viewport that can't be sized) keep
starting — prior graceful-degradation behavior is restored.
- Lazy `RecordingWatchdog` in the CLI handler now calls
`attach_to_session()`, so `AgentFocusChangedEvent` / `BrowserStopEvent`
handlers are wired correctly if the session dispatches them.
- Daemon shutdown finalizes any in-progress recording before tearing the
browser down, preventing truncated MP4s on `close`, idle timeout, or
signal-driven exit.
- Added regression test that monkeypatches `start_recording` to raise and
asserts `on_BrowserConnectedEvent` swallows it without breaking startup.
Closes#4533.
- `RecordingWatchdog` gains public `start_recording(path, size?, framerate?)`,
`stop_recording() -> Path`, and `is_recording`; the existing
`BrowserConnectedEvent`/`BrowserStopEvent` path is refactored to use them,
so profile-driven recording behavior is unchanged.
- `browser-use record start <path>` / `record stop` / `record status`
subcommands wired through argparse, daemon dispatch, and the browser
command handler. `record stop` prints the saved file path so it can be
captured programmatically, matching the issue's requested UX. Works with
`--session NAME` via the existing named-daemon infrastructure.
- The CLI's `CLIBrowserSession` intentionally skips watchdogs; the handler
lazily instantiates `RecordingWatchdog` on first `record start` so CLI
recording doesn't pay the watchdog-setup cost for non-recording sessions.
- Output format is `.mp4` (libx264) since that's what the existing
`VideoRecorderService` encodes; optional dependency gate is unchanged
(`pip install "browser-use[video]"`).
- New `tests/ci/test_action_record.py` exercises the full stack against a
real headless browser + `pytest-httpserver`, verifying decodable MP4
output, double-start rejection, stop-without-start no-op, that the
existing `profile.record_video_dir` flow still works, and the argparse /
dispatch wiring.
On slow CI machines, _get_pending_network_requests() can hang for 15s+
when Chrome is busy loading/rendering after a navigation. This silently
eats into the 30s BrowserStateRequestEvent budget, leaving insufficient
time for the actual DOM capture — causing 5 consecutive timeouts and
agent termination.
Observed on eBay search results in eval runs: DOMWatchdog started but
produced zero log output for 15 seconds before the timeout killed it.
The pending network check was the first await after the URL log.
- AGI-569: after any click that opens a new tab, automatically dispatch
SwitchTabEvent so the agent lands on the new page immediately instead
of requiring a manual switch step (~877 occurrences)
- AGI-548: for <input type="checkbox/radio">, capture checked state
before the CDP mouse click and verify it toggled afterward; if
unchanged (custom-styled or shadow-DOM-backed inputs), fall back to
JS element.click() and report the final checked state in metadata
(~1,241 occurrences)
- Introduced type aliases for captcha results to improve clarity and maintainability.
- Updated event handler signatures to use specific event types from the CDP module.
- Ensured consistent typing for captcha result attributes within the CaptchaWaitResult class.
- Added assertion for timeout handling in the captcha waiting logic.
Magnus Müller