Conversation
Host list entries can be URL-style IPv6 literals such as "||[2001:db8::1]^": the brackets are stripped and the address is emitted as is. Brackets around anything else stay rejected. Domain names in all three entry types (host list, hosts file, wildcard) go through one IDNA profile: the UTS 0xERR0R#46 lookup mapping without the STD3 and label validity checks. This replaces the ToUnicode/ToASCII sequence, which kept mixed punycode/Unicode names such as "имяенн.010.xn--p1acf" in Unicode and then rejected them, and encoded uppercase Unicode ("MÜNCHEN.de") into a punycode form no query can match. Hosts file names and wildcards skipped the conversion. Malformed punycode labels, Base64-like junk and other invalid entries fail as before, with the same messages, line positions and error limit. Invalid UTF-8 now fails IDNA instead of being encoded into a dead label.
An empty "xn--" payload decodes to nothing and a label made only of ignored code points (soft hyphen, zero-width space) maps to nothing, so "*.com.xn--" came out as "*.com." with no parse error. The wildcard cache folds the trailing dot away and the rule then matched every .com name. Allowlists take the same path. The shared conversion now rejects an entry whose mapping has fewer non-empty labels than the input, counting the ideographic and fullwidth full stops as separators since the mapping turns them into dots. This covers host list, hosts file and wildcard entries. Trailing root dots and dot-like separators in valid names keep working. Wildcard suffixes are validated like plain entries, which closes "*..com" and "*." widening to every name. The unmarshal fuzzer checks that no accepted entry has fewer non-empty labels than the field it came from, and a parser-to-cache test asserts the malformed forms cannot match example.com.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Trailing-dot IDN entries do not match hosts-file or blocking lookups consistently.
Pull request overview
Adds bracketed IPv6 parsing and UTS #46 IDN normalization across list and hosts-file handling.
Changes:
- Supports bracketed IPv6 literals.
- Normalizes IDNs and validates wildcard entries.
- Adds parser, fuzz, golden, and cache integration coverage.
File summaries
| File | Reviewed change |
|---|---|
lists/parsers/testdata/fuzz/FuzzHostsUnmarshalText/b421722db68f7e5c |
Adds a fuzz regression corpus entry. |
lists/parsers/hosts.go |
Implements IPv6 unwrapping and IDNA normalization. |
lists/parsers/hosts_test.go |
Tests valid and invalid normalized entries. |
lists/parsers/hosts_funcs_test.go |
Extends reference, fuzz, and invariant coverage. |
cache/stringcache/testdata/golden/edge_cases.txt |
Adds malformed wildcard and IDNA fixtures. |
cache/stringcache/list_scope_test.go |
Verifies malformed entries cannot widen cache scope. |
Review details
Suppressed comments (2)
lists/parsers/hosts.go:186
- Because
toASCIIpreserves a trailing root dot, this now accepts0.0.0.0 münchen.example.de.asxn--mnchen-3ya.example.de..HostsFileResolverstores that exact key, bututil.ExtractDomainOnlyremoves the query's final dot before lookup, so the new IDN hosts-file entry never resolves. Normalize the stored key or lookup input consistently and add resolver coverage.
host, err := toASCII(string(field))
if err != nil {
return err
}
lists/parsers/hosts.go:283
- IDNA mapping intentionally preserves trailing root dots, but blocking lookups pass
util.ExtractDomain(question)to the list cache with that dot removed. A newly accepted entry such asmünchen.example.de.is therefore cached asxn--mnchen-3ya.example.de.and never matches DNS queries. Normalize root-dot handling at the cache boundary and cover this path with a blocking lookup test.
host, err = toASCII(host)
if err != nil {
return "", err
}
- Files reviewed: 6/6 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2261 +/- ##
==========================================
+ Coverage 88.09% 88.16% +0.07%
==========================================
Files 126 126
Lines 9967 10003 +36
==========================================
+ Hits 8780 8819 +39
+ Misses 923 921 -2
+ Partials 264 263 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Lookups strip the trailing dot from the query name (util.ExtractDomain), but the parsers kept it. An entry written as "example.com." went into the string cache or the hosts file resolver with the dot and matched no query. The wildcard cache trims the dot itself, so the gap only hit plain entries and hosts file names. The parsers now drop the dot after validation, which keeps "example.com.." rejected. Tests cover the parser tables, the fuzz invariants, the parser-to-cache scope check and a hosts file resolver lookup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Blocky rejects bracketed IPv6 literals in host lists and mishandles some internationalized domain names (IDNs). Valid names can fail to parse or produce entries that do not match DNS queries.
This PR accepts
[2001:db8::1]and ABP-style entries such as||[2001:db8::1]^. It applies one UTS #46 lookup mapping to host-list entries, hosts-file names and aliases, and wildcard entries. The mapping converts IDNs to the ASCII form DNS clients query.[2001:db8::1]2001:db8::1MÜNCHEN.example.dexn--mnchen-3ya.example.deимяенн.010.xn--p1acfxn--e1afmfa9h.010.xn--p1acf0.0.0.0 münchen.example.dexn--mnchen-3ya.example.de*.münchen.example.de*.xn--mnchen-3ya.example.deThe mixed Unicode/punycode example above comes from #1039.
The parser rejects entries whose labels disappear during IDNA mapping. Without this guard,
*.com.xn--can collapse to*.com.and make the wildcard cache match every.comname. Invalid wildcard suffixes such as*..comand*.now fail before caching.Regex handling and the ASCII fast path stay unchanged. Blocky keeps accepting trailing root dots, underscores and leading or trailing hyphens. Invalid entries count toward
maxErrorsPerSource, with their line numbers in the error messages.Changes
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[List entry] --> B{Entry type} B -->|Regex| C[Preserve as written] B -->|Bracketed IPv6| D[Validate and strip brackets] B -->|Host or wildcard| E[Apply IDNA mapping when needed] E --> F{Any label vanished?} F -->|Yes| G[Reject with parse error] F -->|No| H[Validate domain or wildcard suffix] H --> I[Emit normalized cache entry] D --> I C --> I