fix(v4): align object and tuple optionality handling#5661
Conversation
There was a problem hiding this comment.
Medium priority — The fix correctly addresses the reported issue for .default() but misses .prefault(), which has identical default-providing semantics. This is a real edge case that users could hit.
Tuple elements with `.default()` / `.prefault()` were silently dropped
when the input array was shorter than the tuple, because the parser
skipped any item past `input.length` whose slot was within `optStart`.
Gate the skip on `optout === "optional"` instead. That distinguishes
schemas that produce `undefined` for missing input (`.optional()`,
`z.undefined()`) — for which skipping is a no-op — from schemas that
produce a defined value. `optout` already bubbles through `nullable` /
`readonly` / `catch` / `pipe` / `union`, so chains like
`.default("x").nullable()` are covered without a type-name allowlist.
Closes colinhacks#5229.
f252406 to
cf31678
Compare
`expectTypeOf<...>().toEqualTypeOf<[string, string?]>()` rejects the inferred `[string, (string | undefined)?]` under TypeScript >=6 because the optional element widens to `T | undefined` in the source position.
|
TL;DR — Fixes #5229 by restructuring the tuple parser to materialize Key changes
Summary | 5 files | 11 commits | base: Tuple default-value resolution for missing trailing elements
The key insight is that The trailing-trim loop floors at
Object property handling: require
|
The previous fix for colinhacks#5229 left a sparse hole when an `.optional()` slot sat between the input boundary and a later `.default()` — e.g. `tuple([s, s.optional(), s.default("z")]).parse(["a"])` produced `["a", <empty>, "z"]`, which fails `1 in r`, JSON-serializes to `null`, and skips iteration. Walk trailing items from the end to find the highest index whose schema fills missing input (`optout !== "optional"`). Run every slot up to that point so `.optional()` items reached while padding for a later default produce explicit `undefined`. When the tail is purely optional, the length-shortening behavior is preserved.
Drop the bespoke `runUntil` reverse-find scan in favor of the same shape used by `$ZodObject`: run every item with `value: input[i]` (undefined past the input boundary), let each schema decide what undefined means, and have `handleTupleResult` swallow errors from absent optional-out slots — the tuple-index analog of `handlePropertyResult`'s `key in input` check. A small post-loop trim drops trailing slots that produced `undefined` for absent input (preserves the existing length-shortening behaviour for purely-optional tails like `[s, s.optional()] / [a]`). Behaviour identical to the previous fix on every case in the suite, but the parser now reads alongside the object parser instead of introducing a second, structurally-identical reverse-find boundary.
When a tuple slot past `optStart` rejects `undefined` (e.g. an `.optional()` chain with a refine that bans `undefined`), swallow the issue and truncate the result there. Critically, also stop processing later items so subsequent `.default()` slots do NOT materialize on top of an already-malformed tail. Previously the parser swallowed the absent-optional error but kept running, letting a later default produce a value at an index past the "missing" slot and yielding e.g. `["alpha", undefined, "d"]` for input `["alpha"]`. Now the result is `["alpha"]`, matching the array-analog of $ZodObject's absent-optional-key behaviour. Implementation: parse all items in parallel, collect into an indexed results array, then iterate in order during finalize — break on the first absent-optional rejection, then run the trailing-undefined trim so optional slots between the last real input and the rejected slot also collapse away.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Restructured tuple parsing into a two-phase collect-then-finalize architecture: item schemas now always run (collecting results into
itemResults[]), thenfinalize()walks results in order, breaking on the first absent-optional rejection and truncating subsequent slots — preventing later defaults from materializing after a rejected position - Removed the
handleTupleResultintegration for item schemas (kept only for rest elements) and replaced the old inline skip logic (i >= input.length && i >= optStart && item._zod.def.type !== "default") with the more generalisOptionalOut && !isPresentcheck in finalize - Added comprehensive edge-case tests: absent-optional rejection with truncation, async variant, dense array assertions for interleaved optional/default, and a guard test confirming required slots still surface issues
- Added
.prefault()parity test
Prior review feedback:
-
.prefault()support — addressed (the new architecture handles it generically viaoptoutrather than checkingdef.type) -
.prefault()test coverage — addressed (line 189)
Claude Opus | 𝕏
| // tuple is malformed at that index and any later defaults must NOT fire. | ||
| const itemResults: ParsePayload[] = new Array(items.length); | ||
| for (let i = 0; i < items.length; i++) { | ||
| const r = items[i]._zod.run({ value: input[i], issues: [] }, ctx); |
There was a problem hiding this comment.
Minor performance note: the old code skipped absent optional schemas entirely (continue), while the new code runs every schema unconditionally — including purely-optional trailing slots that just pass undefined through. This is necessary for .default()/.prefault() to work, but means schemas with expensive refinements on optional trailing elements now do unnecessary work when absent. For typical use this is negligible, but worth being aware of if anyone reports perf regressions on tuples with many trailing optional + refine combinations.
Not blocking — the correctness gain outweighs the marginal cost.
Lock in that: - multiple trailing `.optional()` elements still trim back to the input length (we don't fill the tail with literal `undefined`s) - explicit `undefined` inside `input.length` IS preserved - trailing optionals after a default that fires are still trimmed
The post-processing for tuple parse results (in-order walk with break-on-absent-optional-error, then trailing-undefined trim) was defined as a closure inside the parse hot path. Hoist it to a top-level `handleTupleResults` helper, mirroring the `handle*Result` convention already used for objects and rest items. Also document why `optStart` is intentionally NOT consulted in finalize: it's an input-length concern handled by the `too_small` precheck at the top of parse. Output shaping uses `optout` instead so that a `.default()` tail item — which sits inside the optStart region but materializes a defined value — is correctly preserved rather than dropped or swallowed. Adds a regression test that explicit `undefined` inside the input is preserved even when the element schema produces `undefined` as a valid output (e.g. `z.string().or(z.undefined())`, `z.string().optional()`, `z.undefined()`). The trim's \`i >= input.length\` floor is what guards this.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Extracted the inline
finalizeclosure into a standalonehandleTupleResultsfunction — pure refactor, no behavioral change, improves readability - Added edge-case test ("tuple preserves explicit undefined inside input even for optional-out schemas") verifying that explicit
undefinedwithininput.lengthsurvives the trailing-trim loop for.or(z.undefined()),.optional(),z.undefined(), and mid-tuple positions - Updated comments to reference
handleTupleResultsand explain thei >= input.lengthfloor in the trailing-trim loop
Claude Opus | 𝕏
Bundled into colinhacks#4769 alongside the legitimate `optin = "optional"` fix for colinhacks#4768 without separate justification. The test added in the same commit literally has `// z.undefined should NOT be optional` directly above an assertion saying it is. `optout` is read by the object parser's absent-key error-swallow path (colinhacks#5589) and the tuple parser's `optStart`. Marking `z.undefined()` optout-optional conflates "value type is undefined" with "key may be absent in inferred output," which is upstream of the confusion in colinhacks#5654 and colinhacks#5661. Inference doesn't move: `\$ZodUndefinedInternals` doesn't promote `optin`/`optout` to required fields, so `z.object({ a: z.undefined() })` already infers as `{ a: undefined }` regardless. `optin = "optional"` stays — that one is the JSON-schema `required`-array fix from colinhacks#4768.
Same conflation as the previous commit, on the input side. The motivating JSON-schema bug from colinhacks#4768 is `.catch()`-specific; the analogous extension to `z.undefined()` was opportunistic. Under strict semantics `z.object({ a: z.undefined() })` infers as `{ a: undefined }` (required key) and the JSON-schema `required` array should agree. Runtime stays permissive — `z.undefined().parse(undefined)` succeeds whether the source was an absent key or an explicit `undefined`.
After dropping z.undefined()'s `optin`/`optout = "optional"` flags, a trailing `z.undefined()` slot is required input — omitting it triggers `too_small` rather than trimming. Update the assertions accordingly, and lock in the error shape with inline snapshots since the precheck abort behaviour (single `too_small`, no element-level errors) is the exact piece worth pinning. Also tighten the "required slot fails past input length" test to snapshot the issue list instead of just `success === false`, so the shape — single `too_small` from the precheck rather than e.g. an `invalid_type` from the per-item run — is unambiguous.
There was a problem hiding this comment.
Important
The z.undefined() optionality removal is semantically correct but is a breaking change that extends beyond the original tuple-defaults fix. Users relying on z.undefined() or z.union([..., z.undefined()]) to infer key/slot optionality in objects will now get validation errors for absent keys. This should be an intentional decision by a maintainer — flagging for awareness.
Reviewed the following changes:
- Removed
optin/optout = "optional"from$ZodUndefined, makingz.undefined()a pure value constraint ("must be undefined") rather than an optionality marker ("slot can be absent") — aligns withz.void()andz.null()which also don't set optionality - Updated
optional.test.tsassertions:z.undefined()andz.union([z.string(), z.undefined()])now haveoptin/optoutofundefined(not"optional") - Updated tuple tests:
z.tuple([z.string(), z.undefined()]).parse(["alpha"])now correctly produces atoo_smallerror (slot is required), explicitundefinedstill passes
Claude Opus | 𝕏
| @@ -1358,8 +1358,6 @@ export const $ZodUndefined: core.$constructor<$ZodUndefined> = /*@__PURE__*/ cor | |||
| $ZodType.init(inst, def); | |||
| inst._zod.pattern = regexes.undefined; | |||
| inst._zod.values = new Set([undefined]); | |||
There was a problem hiding this comment.
This is the right semantic distinction — z.undefined() is a type constraint ("value must be undefined"), not an optionality marker ("key/slot can be absent"). Consistent with z.void() and z.null() which also don't set optionality.
However, this is a user-visible breaking change: z.object({ key: z.undefined() }) will now require the key to be present (with an explicit undefined value), and z.union([z.string(), z.undefined()]) will no longer make object keys optional. Worth calling out in a changelog or migration note if this PR lands.
Object parsing should only treat key absence as acceptable when the input-side optionality flag says so. Keep optout for output shaping, but stop letting schemas that merely accept or catch undefined make a required key disappear.
|
Landed in Zod 4.4 |
`z.object({ a: z.preprocess(fn, T) }).parse({})` worked in 4.3 (the fn
ran with `undefined`, produced a value, the inner schema validated it)
but started failing on absent keys after #5661 tightened the object
parser. Users commonly use preprocess to inject pre-parse defaults for
fields that may be missing — that pattern broke silently in 4.4.
Restore by marking $ZodPreprocess as `optin === "optional"`, telling
`$ZodObject` that absent keys are legal here. The fn then runs with
`undefined` exactly as it did in 4.3.
To preserve the long-stable behavior of `preprocess(fn, T).optional()
.parse(undefined)` returning `undefined` (true in both 4.3 and 4.4 for
multi-year compatibility), have `$ZodTransform` set the `fallback`
payload flag on every invocation. `$ZodOptional` already clobbers a
result with `fallback === true` when its input was `undefined`, so the
outer optional keeps short-circuiting to `undefined` even though the
transform now runs underneath.
z.object({ a: z.preprocess(v => v ?? "X", z.string()) }).parse({})
// 4.3: { a: "X" }
// 4.4: FAIL (regression)
// now: { a: "X" }
z.preprocess(v => v ?? "X", z.string()).optional().parse(undefined)
// 4.3 + 4.4: undefined
// now: undefined (preserved)
Drops the `optin` defer-to-inner from #5929, but the same outcome
holds: when inner is `.optional()`, preprocess still accepts absent
keys (`optin === "optional"` either way).
…n absent keys (#5941) * fix(v4): propagate fallback flag through pipe boundaries `$ZodCatch` sets a payload flag when its `catchValue` substitutes so an outer `$ZodOptional` can clobber the recovery value with `undefined` (per #5939). But `handlePipeResult` was building a fresh payload for the right side of the pipe without copying the flag, so any chain like `catch().transform()...optional()` lost it — `optional` couldn't tell the inner had recovered, and surfaced the catch value instead of clobbering. Propagate the flag through pipe handoffs, alongside `value`/`issues`. Also rename `caught` to `fallback`: a slightly broader name that describes the consumer contract ("override me if you have a better value when input was undefined") rather than the producer ("catch fired me"). Internal-only; no public API surface. * fix(v4): restore preprocess handling for absent object keys `z.object({ a: z.preprocess(fn, T) }).parse({})` worked in 4.3 (the fn ran with `undefined`, produced a value, the inner schema validated it) but started failing on absent keys after #5661 tightened the object parser. Users commonly use preprocess to inject pre-parse defaults for fields that may be missing — that pattern broke silently in 4.4. Restore by marking $ZodPreprocess as `optin === "optional"`, telling `$ZodObject` that absent keys are legal here. The fn then runs with `undefined` exactly as it did in 4.3. To preserve the long-stable behavior of `preprocess(fn, T).optional() .parse(undefined)` returning `undefined` (true in both 4.3 and 4.4 for multi-year compatibility), have `$ZodTransform` set the `fallback` payload flag on every invocation. `$ZodOptional` already clobbers a result with `fallback === true` when its input was `undefined`, so the outer optional keeps short-circuiting to `undefined` even though the transform now runs underneath. z.object({ a: z.preprocess(v => v ?? "X", z.string()) }).parse({}) // 4.3: { a: "X" } // 4.4: FAIL (regression) // now: { a: "X" } z.preprocess(v => v ?? "X", z.string()).optional().parse(undefined) // 4.3 + 4.4: undefined // now: undefined (preserved) Drops the `optin` defer-to-inner from #5929, but the same outcome holds: when inner is `.optional()`, preprocess still accepts absent keys (`optin === "optional"` either way). * fix(v4): generalize optin=optional from preprocess to transform Promotes the "user-written input handler accepts absence" signal from $ZodPreprocess to $ZodTransform. Any schema with a transform fn at its input boundary (preprocess, standalone z.transform) now declares optin="optional" at runtime. Effects: - preprocess inherits optin="optional" via pipe.optin = transform.optin (same outcome as the previous commit's explicit override; preprocess loses both its optin and optout overrides since pipe already does the optout defer) - standalone z.transform(fn) now accepts absent object keys - z.string().transform(fn): unchanged (pipe.optin = string.optin = undefined; transform on the OUT side doesn't drive optin) - z.unknown().transform(fn).pipe(A): unchanged (pipe.optin = unknown. optin = undefined) The static type stays unchanged — transform's interface doesn't declare optin, so this only sets the runtime value, mirroring the catch pattern. Captures the "flexible inputs, strict outputs" design principle: schemas with a user-written escape hatch (catch's recovery, transform's fn) accept undefined at runtime even when the static type declares the input as required. After this, $ZodPreprocess is a near-empty marker subtype — the constructor body is just $ZodPipe.init(inst, def), kept for type narrowing and traits identity. * docs(wiki): add internal reference for v4 optionality semantics Captures the current state of optin/optout/fallback, who sets each, who reads each, the static/runtime divergence pattern, and walked- through cases for the gnarly interactions (catch+optional, default vs catch vs preprocess vs transform under optional, etc.). Also documents the "flexible inputs, strict outputs" design principle that motivates the runtime/static optin divergence on $ZodCatch and $ZodTransform: schemas with a user-written escape hatch accept undefined at runtime while keeping their declared input type strict. Internal-only doc; not published. * docs(wiki): explain why unknown.transform.pipe stays strict Adds the explicit contrast between z.preprocess(fn, T) (= pipe(transform, T), accepts absent) and z.unknown().transform(fn).pipe(T) (= pipe(pipe( unknown, transform), T), rejects absent). The two look structurally similar but only the leading position drives optin, and z.unknown() isn't input-optional. Also drops the stale "prototype only" caveat from the standalone z.transform(fn) walkthrough — the runtime optin=optional move from preprocess to transform is now a real part of this branch, not a prototype.
## Background `validateUIMessages` rejects assistant messages containing an `output-error` tool part with Zod 4.4 due to colinhacks/zod#5661 ## Summary Before Zod 4.4, `z.unknown()` treated a missing key the same as `undefined`, but now it requires `optional`. A similar fix was applied in #14894. In the AI SDK `process-ui-message-stream` writes errored tool parts with `input: undefined` on both the static-tool and `dynamic-tool` branches. Therefore errored tool calls started failing the validation. ## Manual Verification Verified using a bun patch that errored tools can validate successfully on our backend. ## Checklist - [x] All commits are signed (PRs with unsigned commits cannot be merged) - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review)
## Background `validateUIMessages` rejects assistant messages containing an `output-error` tool part with Zod 4.4 due to colinhacks/zod#5661 ## Summary Before Zod 4.4, `z.unknown()` treated a missing key the same as `undefined`, but now it requires `optional`. A similar fix was applied in #14894. In the AI SDK `process-ui-message-stream` writes errored tool parts with `input: undefined` on both the static-tool and `dynamic-tool` branches. Therefore errored tool calls started failing the validation. ## Manual Verification Verified using a bun patch that errored tools can validate successfully on our backend. ## Checklist - [x] All commits are signed (PRs with unsigned commits cannot be merged) - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review)
## Background `validateUIMessages` rejects assistant messages containing an `output-error` tool part with Zod 4.4 due to colinhacks/zod#5661 ## Summary Before Zod 4.4, `z.unknown()` treated a missing key the same as `undefined`, but now it requires `optional`. A similar fix was applied in #14894. In the AI SDK `process-ui-message-stream` writes errored tool parts with `input: undefined` on both the static-tool and `dynamic-tool` branches. Therefore errored tool calls started failing the validation. ## Manual Verification Verified using a bun patch that errored tools can validate successfully on our backend. ## Checklist - [x] All commits are signed (PRs with unsigned commits cannot be merged) - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review)
It now aligns object and tuple handling around the internal
optin/optoutsplit:optinfor input-length validation andoptoutfor output shaping, so trailing.default()/.prefault()elements materialize while purely optional tails still trim away.optin === "optional"before treating an absent key as valid. Schemas likez.undefined(),z.union([z.string(), z.undefined()]), and required.catch()fields no longer make a missing object key acceptable just because parsingundefinedcan succeed or be recovered.optoutcontinues to control whether absent optional output is omitted, including the.optional()/.exactOptional()cases.The net effect is that absence has to be explicitly represented by input-side optionality. Defaults still bubble through missing object keys and tuple slots; bare
.catch()still recovers explicit invalid values, but no longer acts like a missing-key default unless the field is also optional.Summary
Fixes #5229
When parsing tuples with missing trailing elements that have
.default()values, the default values were not being applied. Instead, the elements were simply skipped.Before (As-is)
After (To-be)
Changes
Modified the tuple parsing logic to run the schema for default type elements even when the input array is shorter, allowing $ZodDefault to apply its default value.
File: packages/zod/src/v4/core/schemas.ts
Test Plan
✅ Added test case for tuple with default elements in tuple.test.ts
✅ All existing tests pass
Verified Issue Examples
All examples from #5229 now work correctly: