Built-in Regex terms are not JSON-quoted inside containers (list[types.email] generates [a@b.com])

@Sanjays2402

Describe the issue as clearly as possible:

Built-in Regex terms from outlines.types are not JSON-quoted when they appear inside a container, so list[types.email] generates [a@b.com] instead of ["a@b.com"]. The result does not parse as JSON.

This came out of review on #1961, which quotes date/time/datetime in containers. That fix is keyed to the terms python_types_to_terms returns for a Python type. But the same function has if isinstance(ptype, Term): return ptype at the top, so a built-in term handed in directly lands in the same branch of _ensure_json_quoted and comes back bare. Credit to @Sanjays2402 for spotting it.

Dict keys are already fine, since _ensure_json_quoted is called there with quote_regex=True. The gap is list items and dict values.

There are 29 built-in Regex terms. Three of them, date/time/datetime, are #1961. The other 26 fall into four groups.

Already correct, 1 term: string is "[^"]*", so it carries its own quotes and list[types.string] already matches ["abc"].

Bare is the right shape, 6 terms: integer, number, boolean, digit, latitude, longitude. Quoting these would break them.

Valid JSON strings once quoted. These are the real bug, 14 terms: email, uuid4, ipv4, ipv6, mac_address, semver, slug, hex_color, hex_str, credit_card, char, bic, e164, iban.

Quoting is necessary but not sufficient, 5 terms: newline ((\r\n|\r|\n)), whitespace (\s), sentence ([A-Z].*\s*[.!?]), paragraph (same, ending in \n+). Three different treatments hide in this bucket, so a fix that handles one and assumes it covers the rest will be wrong. Counting the characters JSON must escape (", the backslash, and U+0000 through U+001F) that each pattern admits anywhere inside a match:

newline      2    '\n' '\r'
whitespace   9    '\t' '\n' '\x0b' '\x0c' '\r' '\x1c' '\x1d' '\x1e' '\x1f'
sentence     34   all of them
paragraph    34   all of them

For newline and whitespace, escaping control characters is enough. For sentence and paragraph it is not, because .* also admits the backslash and a bare quote, and those two fail differently from a control character:

sentence matches 'A\.'   quote-only gives ["A\."]   json.loads -> Invalid \escape: line 1 column 4 (char 3)
sentence matches 'A".'   quote-only gives ["A"."]   json.loads -> Expecting ',' delimiter: line 1 column 5 (char 4)

So those two want whatever json.dumps does to a string, not an escape list assembled by hand. string stays out of the bucket entirely: "[^"]*" needs both quotes, so it never fullmatches a lone quote. Credit to @ErenAta16 for working the three cases apart.

isbn belongs in this bucket too, for a different reason (credit to @ErenAta16 for catching it). Its pattern carries four $ anchors inside lookaheads, so once it sits inside \[(...)\] the alternation is unsatisfiable and nothing matches list[isbn] at all, quoted or bare. Dropping the anchors pins them as the cause:

Regex(types.isbn.pattern.replace('$', ''))   in a list, bare    '[9783161484100]'    -> True

That line is False with the anchors in place. The quoted form stays False either way, since nothing on main quotes isbn, so isbn needs both the anchor fix and the quoting fix.

Note the predictor is anchor semantics, not the $ character. email has two $ characters inside a character class and list[email] matches [a@b.com] fine. Dropping the anchors changes what bare isbn accepts, so it needs the same maintainer call as the other four.

These failures are not all loud. json.loads reads a lone control character between brackets as whitespace:

json.loads('[\n]')     []
json.loads('[\t]')     []
json.loads('["a"]')    ['a']

So list[types.newline] generates text a parser accepts as an empty array. The caller asked for one element, the generation produced one, and the parse yields zero, with no exception raised anywhere. That is the worst case in this issue, worse than the JSONDecodeError rows, because nothing surfaces.

So the naive fix of growing the temporal tuple one type at a time is wrong twice over: it misses 14 terms today, and it would quietly mis-signal the other 5 as fixed.

One design question for a maintainer before anyone writes this. The cleanest shape I can see is to invert the rule: identity-match a short denylist of JSON-scalar built-ins and quote every other built-in term, leaving user-supplied Regex alone as it is today. That stops the list growing for new string-shaped types, and only a new numeric type would ever need adding. Whether the 4 escaping terms should be excluded from that, or fixed in the same pass, is the part worth a decision, keeping in mind that two of them need control-character escaping and two need full string escaping.

Steps/code to reproduce the bug:

import re
import outlines.types as types
from outlines.types.dsl import to_regex, python_types_to_terms

def matches(annotation, sample):
    return bool(re.fullmatch(to_regex(python_types_to_terms(annotation)), sample))

# list items
print(matches(list[types.email], '[a@b.com]'))      # True, not valid JSON
print(matches(list[types.email], '["a@b.com"]'))    # False, valid JSON rejected

# dict values
print(matches(dict[str, types.email], '{"k":a@b.com}'))    # True

# dict keys already handled by quote_regex=True
print(matches(dict[types.email, str], '{"a@b.com":"v"}'))  # True

Swap types.email for uuid4, ipv4, ipv6, mac_address, semver, slug, hex_color, hex_str, credit_card, char, bic, e164 or iban for the same result. isbn behaves differently: nothing matches, see the anchor note above.

Expected result:

Containers holding a string-shaped built-in term generate valid JSON:

    list[types.email]        matches ["a@b.com"]  and rejects [a@b.com]
    dict[str, types.email]   matches {"k":"a@b.com"}

Numeric and boolean built-ins keep their current bare output:

    list[int]                matches [5]
    list[bool]               matches [True]

On that last line: types.boolean is (True|False), so list[bool] matches [True] and rejects [true]. Python literals, not JSON. latitude and longitude have the sibling problem, a leading + that JSON rejects, so list[types.latitude] matches [+45.5] and json.loads refuses it. Both are separate bugs from the quoting one and not in scope here. The scalar denylist keeps all three bare either way, so neither changes the shape proposed above.

Error message:

No exception. The generated regex silently accepts non-JSON output.

Outlines/Python version information:

Version information

main @ be2cd15, re-verified on main @ 7d06847 (also reproduces on the #1961 branch)

Context for the issue:

Happy to open the PR once there is a call on the two questions above: whether to invert to a scalar denylist, and what to do about the 4 control-character terms.

docs(types): state the actual scope of container JSON-quoting

Confirmed against #1961's branch, and one of the terms on your list has a second problem that quoting won't fix.

Matching [value] and ["value"] against to_regex(python_types_to_terms(List[term])) on that branch:

term	`[v]`	`["v"]`	expected
`email`	✓	✗	quoted
`uuid4`	✓	✗	quoted
`ipv4`	✓	✗	quoted
`ipv6`	✓	✗	quoted
`mac_address`	✓	✗	quoted
`isbn`	✗	✗	quoted
`integer`	✓	✗	bare ✓
`number`	✓	✗	bare ✓
`boolean`	✓	✗	bare ✓
`digit`	✓	✗	bare ✓

So the split you describe holds: the four JSON scalars are already right, and the string-valued terms come through bare.

isbn matches nothing at all. Not the bare form, not the quoted form, not even []:

list[isbn] vs '[9783161484100]'    -> False
list[isbn] vs '["9783161484100"]'  -> False
list[isbn] vs '[]'                 -> False

The pattern is fine standalone — 9783161484100, 978-3-16-148410-0 and 0-306-40615-2 all match to_regex(python_types_to_terms(isbn)) directly. It breaks when embedded, because the term carries four $ anchors inside its lookaheads:

(?:ISBN(?:-1[03])?:? )?(?=[0-9X]{10}$|(?=(?:[0-9]+[- ]){3})[- 0-9X]{13}$|97[89][0-9]{10}$|...

Once that sits inside \[(...)\] the $ can't match, so the whole alternation is unsatisfiable. This is the same on main, so it predates #1961 rather than coming from it.

Worth separating from the quoting work: adding quotes around isbn still leaves list[isbn] generating nothing. It needs the anchors dropped from the term itself, and that changes what bare isbn accepts, so it's a different call.

For contrast, email also contains $ characters but they sit inside a character class rather than acting as anchors, and list[email] matches [a@b.com] fine — so a plain "does the pattern contain $" sweep would flag the wrong set.

docs(types): correct the built-in carve-out, pin the direct-term route

Note standalone use in comments and pin the e164 upper bound

Your table reproduces exactly on main @ 7d06847, and re-running it turned up two things I got wrong in the issue body. Both are now fixed above.

I said 24 built-in Regex terms. There are 29:

bic boolean char credit_card date datetime digit e164 email hex_color hex_str
iban integer ipv4 ipv6 isbn latitude longitude mac_address newline number
paragraph semver sentence slug string time uuid4 whitespace

The six I missed change the buckets. bic, e164 and iban are string-shaped and come through bare, so the real bug is 14 terms, not 11. latitude and longitude are numeric-shaped and belong with integer/number. And string is "[^"]*", which carries its own quotes, so list[types.string] already matches ["abc"] and needs nothing.

My anchor evidence was half wrong. I wrote that stripping the $ anchors fixes both the bare and the quoted form. Only bare flips:

list[Regex(isbn.pattern.replace('$',''))]  '[9783161484100]'    -> True
list[Regex(isbn.pattern.replace('$',''))]  '["9783161484100"]'  -> False

Obvious in hindsight, since nothing on main quotes isbn, so removing an anchor was never going to add a quote. The conclusion holds, the anchors are what makes the pattern unsatisfiable, but isbn needs both fixes rather than one, and I had recorded it as one.

Your [] line is the one piece I'd keep out of the isbn column. It isn't isbn-specific:

list[int]        '[]'  -> False
list[str]        '[]'  -> False
list[isbn]       '[]'  -> False

_handle_list builds \[(item)(,\ (item))*\], which requires at least one element, so no list[T] matches [] on main.

One more that came out of checking latitude: its pattern allows a leading +, so list[types.latitude] matches [+45.5] and json.loads rejects it. Same shape as the types.boolean (True|False) problem. Neither is a quoting bug and neither is in scope here, but they both sit under the scalar denylist, so whoever takes that design call should know they're there.

The two questions for a maintainer are unchanged: invert to a scalar denylist or keep growing an allowlist, and whether the four control-character terms get escaping in the same pass or a separate issue.

fix(types): classify iban/bic/e164 as quoted and latitude/longitude a…

You are right about the [] row and I was wrong to file it under isbn. Confirmed on outlines 1.3.3:

list[int]          '[]' -> False
list[str]          '[]' -> False
list[types.isbn]   '[]' -> False
tuple[int, ...]    '()' -> False

to_regex(list[int]) = \[([+-]?(0|[1-9][0-9]*))(,\ ([+-]?(0|[1-9][0-9]*)))*\]

_handle_list emits String("["), the item, KleeneStar([", ", item]), String("]"), so the first item is mandatory and the empty collection is unreachable for every element type. Nothing isbn-specific about it, exactly as you say.

Worth connecting rather than leaving it loose: that is already filed as #1940, with a fix open at #1941 that makes the first item optional for List[T] and for variadic Tuple[T, ...]. So it does not need a row in your issue at all, and it is not waiting on the quoting design call either. I should have linked it instead of listing it.

Your two scalar-shape cases both reproduce, and they are worth keeping visible for whoever takes the denylist decision:

list[types.latitude ] matches [+45.5]  -> True   json.loads FAILS (JSONDecodeError)
list[types.boolean  ] matches [True]   -> True   json.loads FAILS (JSONDecodeError)
list[types.longitude] matches [-73.6]  -> True   parses

longitude is the useful control there: the leading + is what breaks latitude, not the fact that it is a coordinate, so this is a per-pattern property rather than a per-category one. That is an argument for the denylist framing over the allowlist, because a category-shaped rule would have put latitude and longitude in the same bucket and got one of them wrong.

The count reproduces too, 29 rather than 24:

bic boolean char credit_card date datetime digit e164 email hex_color hex_str
iban integer ipv4 ipv6 isbn latitude longitude mac_address newline number
paragraph semver sentence slug string time uuid4 whitespace

On the anchor correction: agreed, and the general form of it is that the anchors and the quoting are independent defects that happen to co-occur on the string-shaped terms. Removing $ makes a bare pattern satisfiable inside a container; it cannot add the quotes that make the result parse. isbn needs both, which is what your rerun shows.

Given #1940 covers the empty-collection row, the two open questions really are just the two you name: denylist versus allowlist, and whether the four control-character terms get escaped in the same pass. The latitude/longitude split above is the strongest argument I have seen for the first one.

Checked #1940 and #1941. Both open, and #1941 does make the first item optional for List[T] and variadic Tuple[T, ...]. Agreed, the empty-collection row belongs there and not here.

Your scalar-shape table reproduces on main @ 7d06847:

list[types.latitude ] matches [+45.5]  -> True   json.loads FAILS (JSONDecodeError)
list[types.boolean  ] matches [True]   -> True   json.loads FAILS (JSONDecodeError)
list[types.longitude] matches [-73.6]  -> True   parses

On your second open question, the control-character terms, I have numbers now instead of a guess. Probing all 29 Regex terms for characters JSON has to escape (", \, and U+0000 through U+001F) finds three that accept one bare:

newline       ['\n', '\r']
string        ['"']
whitespace    ['\n', '\r', '\t', '\x0b', '\x0c', '\x1c', '\x1d', '\x1e', '\x1f']

paragraph and sentence miss that probe but hit it on real samples, since paragraph ends in \n+ and sentence allows \s* before its terminator. Running each term's sample through what a quote-only pass would emit:

newline     '["\n"]'          -> FAILS: Invalid control character at: line 1 column 3 (char 2)
whitespace  '["\t"]'          -> FAILS: Invalid control character at: line 1 column 3 (char 2)
paragraph   '["Hi there.\n"]' -> FAILS: Invalid control character at: line 1 column 12 (char 11)
sentence    '["Hi\tthere."]'  -> FAILS: Invalid control character at: line 1 column 5 (char 4)
char        '["a"]'           -> parses

So it is four terms that need escaping, not the three the character probe finds, and quoting them without escaping still leaves invalid JSON. char is the control: quoting alone is enough there.

string is the strongest case for the denylist. Its pattern is "[^"]*", already self-quoting, so a blanket pass double-wraps it:

string      '[""a""]'  -> FAILS: Expecting ',' delimiter: line 1 column 4 (char 3)

One more for whoever takes the call. newline and whitespace inside a container do not fail loudly today:

json.loads('[\n]') = []
json.loads('[\t]') = []

The matched value is swallowed as JSON insignificant whitespace and you get an empty list back, so constraining to List[types.newline] yields output that parses into something the type never described. Quieter than the isbn row, same root cause, worth taking in the same pass.

Thanks for actually running it, that answers the question I could not settle from reading. I reproduced your probe against the patterns on main and the counts line up, with one difference worth pulling out because it changes what the fix has to do.

Your single-character probe and mine agree:

newline      '\n' '\r'
whitespace   '\t' '\n' '\x0b' '\x0c' '\r' '\x1c' '\x1d' '\x1e' '\x1f'

string sits outside that probe for the reason you gave: "[^"]*" needs both quotes, so a lone " never fullmatches, and its problem is the self-quoting double-wrap rather than an unescaped character.

Where I get a different number is sentence and paragraph. You reached them through real samples and found control characters. Asking instead "which of the characters JSON must escape can appear anywhere inside a match" gives:

newline      2 characters
whitespace   9 characters
sentence     34 characters
paragraph    34 characters

34 is all of them: ", the backslash, and U+0000 through U+001F. .* in [A-Z].*\s*[.!?] admits every one, so those two are not a control-character case that a targeted fix will close.

The backslash is the one I would put in the issue, because it fails differently and a control-character fix will not catch it:

sentence matches 'A\.'      quote-only -> ["A\."]   FAILS: Invalid \escape: line 1 column 4 (char 3)

That is Invalid \escape, not Invalid control character. Same for a bare quote: sentence matches A"., which quote-only turns into ["A"."]. So for sentence and paragraph the right transform is whatever json.dumps does to a string, not an escape list assembled by hand. For newline and whitespace a control-character escape is genuinely sufficient, and for string the answer is to leave it alone. Three different treatments, which is probably worth stating in the issue so nobody implements one and assumes it covers the rest.

Your swallowing point reproduces and I think it is the sharpest thing in the thread:

json.loads('[\n]') = []
json.loads('[\t]') = []
json.loads('["a"]') = ['a']

List[types.newline] generates text that a parser reads as an empty array. That is worse than the loud failures, because nothing surfaces: the caller asked for one element, the generation produced one, and the parse yields zero. Agreed it belongs in the same pass, and I would lead the issue with it rather than with the isbn row, since a reader who sees only JSONDecodeError examples will assume the failure mode is always loud.

Agreed too on the empty-collection row belonging to #1940 / #1941 rather than here.

Ran your counts against main at 7d06847 and they hold, including the two I had lumped together:

newline      2
whitespace   9
sentence     34
paragraph    34

The backslash case reproduces with your error, and the bare quote fails a third way:

sentence matches 'A\.'   quote-only ["A\."]   Invalid \escape: line 1 column 4 (char 3)
sentence matches 'A".'   quote-only ["A"."]   Expecting ',' delimiter: line 1 column 5 (char 4)

json.loads('[\n]') and json.loads('[\t]') both give [] here too.

I have rewritten the fourth bucket in the issue to carry the three treatments separately rather than describing them as one control-character problem, and moved the silent-swallow case up so it reads before the naive-fix paragraph. Your name is on both. The string row now says why it is out of scope, since the double-wrap is a different bug from an unescaped character.

The maintainer question is unchanged and still the only thing holding a PR: invert to a scalar denylist or keep growing the allowlist, and whether the four escaping terms come along in the same pass now that two of them need json.dumps and not an escape table.

Thanks for re-running it and for splitting the fourth bucket, that reads much better as three treatments.

On the open question, since I have the numbers in front of me I will give an opinion rather than leave it hanging: denylist.

Of the 29 built-in terms, the ones needing anything other than plain quoting are string (skip, self-quoting), newline and whitespace (escape control characters), and sentence and paragraph (full json.dumps). That is five exceptions against 24 that want the same ordinary treatment. An allowlist has to be extended every time a term is added and fails open when someone forgets, which is exactly how the current bug arrived. A denylist fails the other way: a new term gets quoted like everything else, and if it turns out to need escaping that is a visible bug rather than silent invalid JSON.

The .*-shaped terms are the argument for it. sentence admits all 34 characters JSON must escape, so any future term built on .* or .+ has the same property, and an allowlist would not know to include it.

I would also not block the whole thing on the escaping half. Quoting the 24 straightforward terms is separable and fixes the reported case; sentence and paragraph can follow, since they need json.dumps rather than the same mechanism. Landing the easy 24 first makes the remaining five a small, well-described change instead of one PR that has to be right about everything at once.

Worth saying plainly for whoever picks this up: the queue there is long at the moment, so if you want this to move I would put the denylist choice in the PR description as a stated decision with the reasoning above, rather than wait for the call. It is much easier to review "I chose X because Y, tell me if you want Z" than an open question.

Denylist it is, and I will put the reasoning in the PR description as a stated decision rather than wait, as you suggest.

Concretely, _ensure_json_quoted grows a scalar set: integer, number, boolean, digit, latitude, longitude, plus string, which already carries its own quotes. Every other built-in Regex term gets wrapped. User-supplied Regex stays untouched, same as today.

That leaves your four escaping terms quoted but not correct, and I want that to be explicit rather than look like an oversight. Quoting still moves two of them in the right direction, because it converts the silent failure into a loud one:

json.loads('[\n]')      []                                          # main today, list[types.newline]
json.loads('["\n"]')    Invalid control character at: line 1 column 3   # after quoting

sentence and paragraph fail loudly either way, so quoting costs them nothing and leaves the json.dumps pass as a change to one branch instead of a rewrite.

isbn gets quoted with the rest and nothing observable changes, since its four $ anchors inside lookaheads make the alternation unsatisfiable once it sits inside \[(...)\]. Dropping the anchors is still a separate call, because it changes what bare isbn accepts.

So the PR fixes 17 terms, moves newline and whitespace from a silent bad parse to a raised error, and leaves sentence, paragraph and isbn written up as follow-ups. It also subsumes #1961, which only covered date/time/datetime; I will say so there and close it if the maintainers prefer the wider shape.

Tests will pin the six scalar terms staying bare, so a later denylist edit that over-quotes gets caught rather than shipping.

Ran your three claims rather than taking them, and all three hold exactly.

The isbn one first, because it turned out to be more interesting than a footnote. Using the pattern from src/outlines/types/__init__.py:258:

bare isbn against "0306406152"        True
inside [ ... ]   unquoted             False
inside [ "..." ] quoted               False

So you are right that quoting changes nothing observable for it. What that also shows is that list[isbn] is already unsatisfiable on main today, before this PR touches anything. The four $ anchors sit inside lookaheads, and once the term is embedded with a ] after it, no input can satisfy them. That is a silent total failure rather than a cosmetic wart, so the anchor follow-up is fixing a live bug, not tidying. Still a separate call from this PR, since dropping them changes what bare isbn accepts, but worth writing up with that framing rather than as a nicety.

The newline pair reproduces to the character:

'[\n]'      json.loads -> []
'["\n"]'    json.loads -> JSONDecodeError: Invalid control character at: line 1 column 3 (char 2)

Which is the argument for the change made concrete. [] is the worst possible outcome, because a caller gets a well-formed empty list and no signal that the model produced something the schema could not carry. Trading that for a raised error is a strict improvement even though neither is the eventual right answer.

On the denylist: agreed, and putting the reasoning in the description as a stated decision rather than leaving it implicit is the part that matters. A scalar set is a small, readable rule, and pinning the six bare terms in tests is what stops a later edit quietly over-quoting them. That test is doing the same job as the reasoning, one layer down.

No objection to any of the shape here.

Navigation Menu

Repository navigation

Built-in Regex terms are not JSON-quoted inside containers (list[types.email] generates [a@b.com]) #1962

Describe the issue as clearly as possible:

Steps/code to reproduce the bug:

Expected result:

Error message:

Outlines/Python version information:

Context for the issue:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Built-in Regex terms are not JSON-quoted inside containers (list[types.email] generates [a@b.com]) #1962

Description

Describe the issue as clearly as possible:

Steps/code to reproduce the bug:

Expected result:

Error message:

Outlines/Python version information:

Context for the issue:

Activity

ErenAta16 commented on Jul 31, 2026

bharadwaj-pendyala commented on Aug 8, 2026

ErenAta16 commented on Aug 12, 2026

bharadwaj-pendyala commented on Aug 16, 2026

ErenAta16 commented on Aug 16, 2026

bharadwaj-pendyala commented on Aug 17, 2026

ErenAta16 commented on Aug 17, 2026

bharadwaj-pendyala commented on Aug 17, 2026

ErenAta16 commented on Aug 21, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions