Skip to content

Fix optionality of schemas#4769

Merged
colinhacks merged 7 commits into
colinhacks:mainfrom
samchungy:mark-catch-as-input-optional
Jul 3, 2025
Merged

Fix optionality of schemas#4769
colinhacks merged 7 commits into
colinhacks:mainfrom
samchungy:mark-catch-as-input-optional

Conversation

@samchungy

@samchungy samchungy commented Jun 25, 2025

Copy link
Copy Markdown
Contributor

Resolves #4768

Changes:

  • Marks .catch() schemas as input optional.
  • Marks .never() schemas as input and output optional.
  • Marks .undefined() schemas as input and output optional.
  • Changes .undefined() output to align with .never(). At the moment it outputs null which isn't strictly correct. The Zod Schema would never be able to accept null. We essentially want them to send us nothing right? As undefined doesn't exist we can treat it the same as .never()
  • Marks z.union() schemas with optional members as input and output optional.

@samchungy samchungy changed the title Mark catch as input optional Fix optionality Jun 25, 2025
@samchungy samchungy changed the title Fix optionality Fix optionality of schemas Jun 25, 2025
Comment thread packages/zod/src/v4/core/schemas.ts Outdated
export const $ZodNever: core.$constructor<$ZodNever> = /*@__PURE__*/ core.$constructor("$ZodNever", (inst, def) => {
$ZodType.init(inst, def);

inst._zod.optin = "optional";

@colinhacks colinhacks Jul 3, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My current mental model:

  • optin should be "optional" if undefined will pass validation
  • optout should be "optional" if the schema might return undefined (with exceptions for unknowable stuff like transforms)

So it seems like ZodNever should be false for both. I read this comment and still don't understand the rationale. What am I missing?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import Ajv from 'ajv';

const ajv = new Ajv();

const jsonSchema = {
  type: 'object',
  properties: {
    a: {
      not: {},
    },
    b: {
      type: 'string',
    },
  },
  required: ['a', 'b'],
  additionalProperties: false,
};

console.log(
  ajv.validate(jsonSchema, {
    b: 'test',
  }),
);

// false;

Just trying to placate validators to allow parsing those JSON Schemas with valid objects. This should pass right?

@samchungy samchungy Jul 3, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a isn't required:

console.log(
  ajv.validate(jsonSchema, {
    a: 'bar',
    b: 'test',
  }),
);

// false ✅ 

console.log(
  ajv.validate(jsonSchema, {
    b: 'test',
  }),
);

// true ✅ 

@colinhacks colinhacks Jul 4, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm not a fan. This doesn't work in Zod

z.object({
  a: z.never(),
}).parse({}); // ❌ 

And it doesn't work in TypeScript:

Screenshot 2025-07-03 at 17 10 05

The only reason to use z.never() is if you really just want validation to always fail, so making this change to get AJV to pass validation isn't compelling imo. This goes against Zod's internal concept of optin and optout.

Of course there's no reason why z.toJSONSchema needs to use those flags. It can use an orthogonal mechanism to determine "JSON Schema" optionality. It would take some reworking but it wouldn't be super hard to implement a isOptional utility; you could use isTransforming as a starting point. Probably not worth it but if we keep encountering tension between the "Zod way" and the "JSON Schema way" then that's always an option.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, cool. I guess the workaround here is you can always go z.never().optional() if you need the behaviour. Sounds good to me.

@colinhacks

Copy link
Copy Markdown
Owner

The key reason why things are the they way the are is that Zod is trying to distinguish between these two types:

type A = { a?: string }
type B = { a: string | undefined }

I was trying to avoid marking a type as "runtime-optional" (inst._zod.optin = "optional") unless it was also marked as "statically optional" (optin: "optional" in the internals interface). But I suppose it's not a big deal.

I rolled back the changes to ZodNever because I'm still dubious, but I wanted to get this merged. We can continue that discussion if you like, but I think I'm unlikely to be convinced.

@colinhacks colinhacks merged commit 36fe14e into colinhacks:main Jul 3, 2025
4 checks passed
colinhacks added a commit to Cyjin-jani/zod that referenced this pull request Apr 29, 2026
Bundled into colinhacks#4769 alongside the legitimate `optin = "optional"` fix
for colinhacks#4768 without separate justification. The test added in the same
commit literally has `// z.undefined should NOT be optional` directly
above an assertion saying it is.

`optout` is read by the object parser's absent-key error-swallow path
(colinhacks#5589) and the tuple parser's `optStart`. Marking `z.undefined()`
optout-optional conflates "value type is undefined" with "key may be
absent in inferred output," which is upstream of the confusion in
colinhacks#5654 and colinhacks#5661.

Inference doesn't move: `\$ZodUndefinedInternals` doesn't promote
`optin`/`optout` to required fields, so `z.object({ a: z.undefined() })`
already infers as `{ a: undefined }` regardless. `optin = "optional"`
stays — that one is the JSON-schema `required`-array fix from colinhacks#4768.
colinhacks added a commit that referenced this pull request Apr 29, 2026
* fix(v4): apply trailing tuple defaults via optout flag

Tuple elements with `.default()` / `.prefault()` were silently dropped
when the input array was shorter than the tuple, because the parser
skipped any item past `input.length` whose slot was within `optStart`.

Gate the skip on `optout === "optional"` instead. That distinguishes
schemas that produce `undefined` for missing input (`.optional()`,
`z.undefined()`) — for which skipping is a no-op — from schemas that
produce a defined value. `optout` already bubbles through `nullable` /
`readonly` / `catch` / `pipe` / `union`, so chains like
`.default("x").nullable()` are covered without a type-name allowlist.

Closes #5229.

* test(v4): match tuple `_input` shape under tsc latest

`expectTypeOf<...>().toEqualTypeOf<[string, string?]>()` rejects the
inferred `[string, (string | undefined)?]` under TypeScript >=6 because
the optional element widens to `T | undefined` in the source position.

* fix(v4): keep tuple result dense when an optional precedes a default

The previous fix for #5229 left a sparse hole when an `.optional()`
slot sat between the input boundary and a later `.default()` — e.g.
`tuple([s, s.optional(), s.default("z")]).parse(["a"])` produced
`["a", <empty>, "z"]`, which fails `1 in r`, JSON-serializes to
`null`, and skips iteration.

Walk trailing items from the end to find the highest index whose
schema fills missing input (`optout !== "optional"`). Run every slot
up to that point so `.optional()` items reached while padding for a
later default produce explicit `undefined`. When the tail is purely
optional, the length-shortening behavior is preserved.

* refactor(v4): mirror $ZodObject in $ZodTuple parser

Drop the bespoke `runUntil` reverse-find scan in favor of the same
shape used by `$ZodObject`: run every item with `value: input[i]`
(undefined past the input boundary), let each schema decide what
undefined means, and have `handleTupleResult` swallow errors from
absent optional-out slots — the tuple-index analog of
`handlePropertyResult`'s `key in input` check.

A small post-loop trim drops trailing slots that produced `undefined`
for absent input (preserves the existing length-shortening behaviour
for purely-optional tails like `[s, s.optional()] / [a]`).

Behaviour identical to the previous fix on every case in the suite,
but the parser now reads alongside the object parser instead of
introducing a second, structurally-identical reverse-find boundary.

* fix(v4): break tuple parsing on first absent-optional rejection

When a tuple slot past `optStart` rejects `undefined` (e.g. an
`.optional()` chain with a refine that bans `undefined`), swallow the
issue and truncate the result there. Critically, also stop processing
later items so subsequent `.default()` slots do NOT materialize on top
of an already-malformed tail.

Previously the parser swallowed the absent-optional error but kept
running, letting a later default produce a value at an index past the
"missing" slot and yielding e.g. `["alpha", undefined, "d"]` for input
`["alpha"]`. Now the result is `["alpha"]`, matching the array-analog
of $ZodObject's absent-optional-key behaviour.

Implementation: parse all items in parallel, collect into an indexed
results array, then iterate in order during finalize — break on the
first absent-optional rejection, then run the trailing-undefined trim
so optional slots between the last real input and the rejected slot
also collapse away.

* test(v4): cover multi-trailing optional tuple cases

Lock in that:
- multiple trailing `.optional()` elements still trim back to the input
  length (we don't fill the tail with literal `undefined`s)
- explicit `undefined` inside `input.length` IS preserved
- trailing optionals after a default that fires are still trimmed

* refactor(v4): hoist tuple finalize into top-level handleTupleResults

The post-processing for tuple parse results (in-order walk with
break-on-absent-optional-error, then trailing-undefined trim) was
defined as a closure inside the parse hot path. Hoist it to a
top-level `handleTupleResults` helper, mirroring the `handle*Result`
convention already used for objects and rest items.

Also document why `optStart` is intentionally NOT consulted in
finalize: it's an input-length concern handled by the `too_small`
precheck at the top of parse. Output shaping uses `optout` instead so
that a `.default()` tail item — which sits inside the optStart region
but materializes a defined value — is correctly preserved rather than
dropped or swallowed.

Adds a regression test that explicit `undefined` inside the input is
preserved even when the element schema produces `undefined` as a
valid output (e.g. `z.string().or(z.undefined())`,
`z.string().optional()`, `z.undefined()`). The trim's
\`i >= input.length\` floor is what guards this.

* fix: drop `z.undefined()`'s `optout = "optional"`

Bundled into #4769 alongside the legitimate `optin = "optional"` fix
for #4768 without separate justification. The test added in the same
commit literally has `// z.undefined should NOT be optional` directly
above an assertion saying it is.

`optout` is read by the object parser's absent-key error-swallow path
(#5589) and the tuple parser's `optStart`. Marking `z.undefined()`
optout-optional conflates "value type is undefined" with "key may be
absent in inferred output," which is upstream of the confusion in
#5654 and #5661.

Inference doesn't move: `\$ZodUndefinedInternals` doesn't promote
`optin`/`optout` to required fields, so `z.object({ a: z.undefined() })`
already infers as `{ a: undefined }` regardless. `optin = "optional"`
stays — that one is the JSON-schema `required`-array fix from #4768.

* fix: drop `z.undefined()`'s `optin = "optional"` too

Same conflation as the previous commit, on the input side. The motivating
JSON-schema bug from #4768 is `.catch()`-specific; the analogous extension
to `z.undefined()` was opportunistic. Under strict semantics
`z.object({ a: z.undefined() })` infers as `{ a: undefined }` (required
key) and the JSON-schema `required` array should agree. Runtime stays
permissive — `z.undefined().parse(undefined)` succeeds whether the source
was an absent key or an explicit `undefined`.

* test(v4): adapt tuple tests to z.undefined()'s required-input semantics

After dropping z.undefined()'s `optin`/`optout = "optional"` flags, a
trailing `z.undefined()` slot is required input — omitting it triggers
`too_small` rather than trimming. Update the assertions accordingly,
and lock in the error shape with inline snapshots since the precheck
abort behaviour (single `too_small`, no element-level errors) is the
exact piece worth pinning.

Also tighten the "required slot fails past input length" test to
snapshot the issue list instead of just `success === false`, so the
shape — single `too_small` from the precheck rather than e.g. an
`invalid_type` from the per-item run — is unambiguous.

* fix(v4): reject absent object keys when optin is required

Object parsing should only treat key absence as acceptable when the input-side optionality flag says so. Keep optout for output shaping, but stop letting schemas that merely accept or catch undefined make a required key disappear.

---------

Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v4(toJSONSchema): .catch() should be marked as optional?

2 participants