Addresses third P1 codex review on #4866.
Per RFC 3490 / UTS46, four code points act as label separators in IDNA
processing — `.` (U+002E), `。` (U+3002 IDEOGRAPHIC FULL STOP),
`．` (U+FF0E FULLWIDTH FULL STOP), and `｡` (U+FF61 HALFWIDTH IDEOGRAPHIC
FULL STOP). WHATWG URL parsing folds all four to `.` before resolution,
so `http://127。0。0。1/` and `http://127｡0｡0｡1/` reach 127.0.0.1.
NFKC handles only U+FF0E and partially maps U+FF61 → U+3002, leaving the
IPv4 parser unable to match the dominant ideographic-dot form. Explicitly
replace U+3002 and U+FF61 with `.` after NFKC.
Test `test_idna_dot_separators_blocked` covers all four dot variants
both at `_is_url_allowed` and `_is_ip_address` levels, plus a combined
case with circled digits (`①②⑦。⓪。⓪。①`).
Addresses second P1 codex review on #4866.
WHATWG URL canonicalization maps fullwidth digits (`１２７.０.０.１`),
circled digits (`①②⑦.⓪.⓪.①`), and Unicode-prefixed hex forms
(`０x7f000001`) to ASCII IPv4 literals (`127.0.0.1` and `0x7f000001`)
before resolution. The classifier saw only the original string and
returned False, so `block_ip_addresses=True` was still bypassable by
re-encoding the IP with any equivalent Unicode digit variant.
Add a `unicodedata.normalize('NFKC', ...)` step after percent-decoding
and before parsing. NFKC handles all the bot's example forms and is
stdlib-only (no IDNA dependency). Wrapped in try/except to preserve the
never-throw invariant.
Tests:
- test_unicode_normalized_ipv4_blocked covers fullwidth, fullwidth+ASCII
hex, and circled-digit forms at both `_is_url_allowed` and
`_is_ip_address` boundaries.
- test_idn_domains_not_misclassified_as_ip is a false-positive guard
ensuring legitimate IDN domains (`café.example`, `日本.example`, their
punycode equivalents) remain classified as domains.
Addresses second codex review on #4866 (P1).
Chromium percent-decodes the host component before applying its IPv4
parser, so a URL like `http://%30x7f000001/` (decodes to `0x7f000001`,
i.e. 127.0.0.1) or `http://%31%32%37.0.0.1/` (decodes to `127.0.0.1`)
still reaches the IP address despite `block_ip_addresses=True`. Without
decoding, `_is_ip_address` sees the literal `%`-encoded string and
returns False — bypassing the block.
Call `urllib.parse.unquote` on the host before passing it to both
`ipaddress.ip_address` and `socket.inet_aton`. Wrap in try/except to
preserve the never-throw invariant for the classifier.
Adds two regression tests:
- test_percent_encoded_ipv4_blocked covers mixed (`%30x7f000001`), fully
encoded canonical (`%31%32%37.0.0.1`), and fully encoded decimal
(`%32%31%33%30%37%30%36%34%33%33`) bypass forms.
- test_malformed_percent_encoding_does_not_crash covers lone `%`, `%zz`,
`%2` — `unquote` leaves these as-is and the classifier must not throw.
Addresses codex review on #4866.
`socket.inet_aton` raises `UnicodeEncodeError` (not `OSError`) for
hostnames containing lone surrogates — common in URLs produced by
URL-decoding malformed UTF-8. The earlier patch only caught `OSError`,
so `_is_ip_address('\udcff')` would propagate the exception through
`_is_url_allowed` and crash the navigation security check whenever
`block_ip_addresses=True`.
Restore the original code's defensive `except Exception` posture for
both `ipaddress.ip_address` and `socket.inet_aton`. The classifier
should never throw — it returns True (recognized IP) or False (not a
recognizable IP); downstream domain-allowlist handling then applies.
Regression test covers lone-surrogate hostnames in
`test_malformed_unicode_hostnames_do_not_crash_classifier`.
GHSA-xrfv-gg9f-wwjp, GHSA-g27c-8gp4-28cv.
`SecurityWatchdog._is_ip_address` only recognized IP strings that
`ipaddress.ip_address()` accepts — i.e. the canonical dotted-quad form
(`127.0.0.1`) and full IPv6. Chromium and the kernel resolver, however,
also accept several non-standard IPv4 representations:
http://2130706433/ → 127.0.0.1 (decimal int)
http://0x7f000001/ → 127.0.0.1 (hex)
http://0177.0.0.1/ → 127.0.0.1 (octal)
http://127.1/ → 127.0.0.1 (short-form)
http://127.0.1/ → 127.0.0.1 (short-form)
`block_ip_addresses=True` was therefore trivially bypassed by re-encoding
the IP in any of these forms.
Fall back to `socket.inet_aton` after `ipaddress.ip_address()` fails — it
accepts the same liberal IPv4 forms the kernel resolver does, so the
classifier matches the browser's behavior.
The existing `test_ipv4_lookalike_domains_allowed` test was codifying the
buggy behavior for `1.2.3` (which IS a short-form IPv4 == 1.2.0.3).
Removed that assertion and added a dedicated `TestNonStandardIPv4Representations`
class covering decimal/hex/octal/short-form blocking, lookalike-domain
non-interference, and the interaction with `allowed_domains`.
Saurav Panda