Accept internationalized domain names - #3011
Open
DL6ER wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates FTL’s shared valid_domain() character validation (used by config validation, API inputs, and list parsing) to stop rejecting internationalized domain names (IDN) that contain non-ASCII bytes, allowing names like äste.com to pass validation so downstream IDN handling can proceed.
Changes:
- Broaden
valid_domain_char[]to accept non-ASCII bytes (0x80–0xFF) in domain strings. - Update the surrounding comment to explain why non-ASCII bytes are permitted (IDN/libidn2 context).
Suppressed comments (1)
src/tools/gravity-parseList.c:43
- Allowing the entire 0x80–0xFF range will treat bytes that can never occur in valid UTF-8 (e.g., 0xC0–0xC1 and 0xF5–0xFF) as valid domain characters. This can let corrupted/non-UTF-8 input through validation and get stored/propagated as a “valid” domain. Consider excluding the always-invalid UTF-8 byte values while still allowing IDN U-label UTF-8 bytes.
static const unsigned char valid_domain_char[256] = {
['a' ... 'z'] = 1, ['A' ... 'Z'] = 1, ['0' ... '9'] = 1,
['-'] = 1, ['.'] = 1, ['_'] = 1,
[0x80 ... 0xff] = 1,
};
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+35
to
+38
| // Domain must not contain any character other than [a-zA-Z0-9.-_] and the bytes | ||
| // an internationalized name is written with. We build dnsmasq with libidn2 and | ||
| // it converts those to punycode itself, so rejecting them here only refused | ||
| // names the resolver handles perfectly well. |
DL6ER
force-pushed
the
fix/idn-domain-validation
branch
from
August 9, 2026 04:40
6e07366 to
f855224
Compare
`valid_domain()` rejected every byte above 0x7f, so `äste.com` was refused while its punycode form `xn--ste-pla.com` passed. We build the embedded dnsmasq with libidn2 and it converts such names itself, so our own validator was the only thing standing in the way. Such a name reaches us as UTF-8, and the bytes are only accepted where they form a sequence a decoder will take. `utf8_sequence_len()` applies the ranges of RFC 3629, which rules out overlong encodings, UTF-16 surrogates and everything above U+10FFFF - simply marking 0x80-0xff valid in the lookup table would have let arbitrary binary through as a domain name. Its result was compared against a reference decoder over all 67 million one- to four-byte inputs, with no disagreement. Signed-off-by: DL6ER <dl6er@dl6er.de>
DL6ER
force-pushed
the
fix/idn-domain-validation
branch
from
August 9, 2026 09:42
f855224 to
06c4288
Compare
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this implement/fix?
valid_domain()rejects every byte above0x7f, so an internationalized name likeäste.comis refused while its punycode formxn--ste-pla.comis accepted. We build the embedded dnsmasq withHAVE_LIBIDN2and link libidn2, so it converts such names itself - our own validator was the only thing standing in the way.Such a name reaches us as UTF-8, and the bytes are accepted only where they form a sequence a decoder will take.
utf8_sequence_len()applies the ranges of RFC 3629, so overlong encodings, UTF-16 surrogates and anything above U+10FFFF stay refused. Simply marking0x80-0xffvalid in the lookup table would have been shorter, but a per-byte table cannot express that, and it would have let arbitrary binary through as a domain name.This affects everything going through
valid_domain(), i.e.,dns.hosts,dns.domain.name,webserver.domain, the reverse servers and the list endpoints.dns.cnameRecordsanddns.hostRecordvalidate differently and already accepted such names.The length limits are unchanged, and the ASCII characters that were invalid before still are:
ex!ample.com,ex/ample.comand anything containing a newline remain rejected.How to test the change during review
or the same through
PATCH /api/config, which answered400withinvalid hostname ("äste.com")before and200now.xn--ste-pla.comand plain ASCII names keep working, and1.1.1.1 ex!ample.comis still refused.test/run.shcovers both directions: the new Internationalized domain names are accepted, malformed UTF-8 is not case accepts two-, three- and four-byte names and refuses an overlong encoding, a surrogate, a codepoint above U+10FFFF, a truncated sequence and a stray continuation byte.utf8_sequence_len()was additionally compared against a reference decoder over all 67 million one- to four-byte inputs, with no disagreement.Additional information
Related issue or feature (if applicable): N/A
Pull request in docs with documentation (if applicable): N/A
Checklist:
developmentbranch.