fix(v4): generalize optin/fallback to transform; restore preprocess on absent keys#5941
Conversation
`$ZodCatch` sets a payload flag when its `catchValue` substitutes so an outer `$ZodOptional` can clobber the recovery value with `undefined` (per #5939). But `handlePipeResult` was building a fresh payload for the right side of the pipe without copying the flag, so any chain like `catch().transform()...optional()` lost it — `optional` couldn't tell the inner had recovered, and surfaced the catch value instead of clobbering. Propagate the flag through pipe handoffs, alongside `value`/`issues`. Also rename `caught` to `fallback`: a slightly broader name that describes the consumer contract ("override me if you have a better value when input was undefined") rather than the producer ("catch fired me"). Internal-only; no public API surface.
|
TL;DR — Two related cleanups around the Key changes
Summary | 5 files | 5 commits | base:
|
There was a problem hiding this comment.
Reviewed — no issues found.
Claude Opus | 𝕏
`z.object({ a: z.preprocess(fn, T) }).parse({})` worked in 4.3 (the fn
ran with `undefined`, produced a value, the inner schema validated it)
but started failing on absent keys after #5661 tightened the object
parser. Users commonly use preprocess to inject pre-parse defaults for
fields that may be missing — that pattern broke silently in 4.4.
Restore by marking $ZodPreprocess as `optin === "optional"`, telling
`$ZodObject` that absent keys are legal here. The fn then runs with
`undefined` exactly as it did in 4.3.
To preserve the long-stable behavior of `preprocess(fn, T).optional()
.parse(undefined)` returning `undefined` (true in both 4.3 and 4.4 for
multi-year compatibility), have `$ZodTransform` set the `fallback`
payload flag on every invocation. `$ZodOptional` already clobbers a
result with `fallback === true` when its input was `undefined`, so the
outer optional keeps short-circuiting to `undefined` even though the
transform now runs underneath.
z.object({ a: z.preprocess(v => v ?? "X", z.string()) }).parse({})
// 4.3: { a: "X" }
// 4.4: FAIL (regression)
// now: { a: "X" }
z.preprocess(v => v ?? "X", z.string()).optional().parse(undefined)
// 4.3 + 4.4: undefined
// now: undefined (preserved)
Drops the `optin` defer-to-inner from #5929, but the same outcome
holds: when inner is `.optional()`, preprocess still accepts absent
keys (`optin === "optional"` either way).
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Restored
preprocesshandling for absent object keys by pinning$ZodPreprocess.optinto"optional"(4.3 parity). - Marked every
$ZodTransforminvocation withpayload.fallback = true(both core and classic, sync and async) so outer$ZodOptionalclobbers transformed output when input wasundefined. - Added regression coverage in
preprocess.test.tsandcatch.test.tsfor direct, pipe, and object-key shapes. - Updated the
fallbackJSDoc onParsePayloadto reflect the new producer.
Claude Opus | 𝕏
Promotes the "user-written input handler accepts absence" signal from $ZodPreprocess to $ZodTransform. Any schema with a transform fn at its input boundary (preprocess, standalone z.transform) now declares optin="optional" at runtime. Effects: - preprocess inherits optin="optional" via pipe.optin = transform.optin (same outcome as the previous commit's explicit override; preprocess loses both its optin and optout overrides since pipe already does the optout defer) - standalone z.transform(fn) now accepts absent object keys - z.string().transform(fn): unchanged (pipe.optin = string.optin = undefined; transform on the OUT side doesn't drive optin) - z.unknown().transform(fn).pipe(A): unchanged (pipe.optin = unknown. optin = undefined) The static type stays unchanged — transform's interface doesn't declare optin, so this only sets the runtime value, mirroring the catch pattern. Captures the "flexible inputs, strict outputs" design principle: schemas with a user-written escape hatch (catch's recovery, transform's fn) accept undefined at runtime even when the static type declares the input as required. After this, $ZodPreprocess is a near-empty marker subtype — the constructor body is just $ZodPipe.init(inst, def), kept for type narrowing and traits identity.
Captures the current state of optin/optout/fallback, who sets each, who reads each, the static/runtime divergence pattern, and walked- through cases for the gnarly interactions (catch+optional, default vs catch vs preprocess vs transform under optional, etc.). Also documents the "flexible inputs, strict outputs" design principle that motivates the runtime/static optin divergence on $ZodCatch and $ZodTransform: schemas with a user-written escape hatch accept undefined at runtime while keeping their declared input type strict. Internal-only doc; not published.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Moved
optin = "optional"from$ZodPreprocess/$ZodPipeinto$ZodTransform, so any transform at a pipe'sinside (including preprocess) inherits optin naturally through the existing lazy pipe defer. - Dropped the now-redundant explicit
defineLazyofoptin/optoutin$ZodPreprocess— pipe's defaults produce equivalent runtime behavior sincedef.inis a transform. - Accepted the documented side effect that standalone
z.transform(fn)now accepts absent object keys at runtime (small blast radius, consistent with the catch/preprocess pattern of runtime accepting absence without widening the static input type). - Added
wiki/optionality.mdas an internal reference coveringoptin/optout/fallbackproducers and consumers, the static/runtime divergence rationale, and walked-through cases — repo-root only, not published viapackages/zod.
Claude Opus | 𝕏
Adds the explicit contrast between z.preprocess(fn, T) (= pipe(transform, T), accepts absent) and z.unknown().transform(fn).pipe(T) (= pipe(pipe( unknown, transform), T), rejects absent). The two look structurally similar but only the leading position drives optin, and z.unknown() isn't input-optional. Also drops the stale "prototype only" caveat from the standalone z.transform(fn) walkthrough — the runtime optin=optional move from preprocess to transform is now a real part of this branch, not a prototype.
Builds on #5939 to clean up the optional / catch / transform / preprocess interaction. Three independent changes that share the same internal mechanism (a payload
fallbackflag and anoptin === "optional"declaration), plus an internal wiki doc capturing the design.1. Restore preprocess handling for absent object keys (4.3 parity)
z.object({ a: z.preprocess(fn, T) }).parse({})worked in 4.3 — the fn ran withundefined, produced a value, the inner schema validated it. After #5661 tightened the object parser, this started failing on absent keys, breaking a common pattern of usingpreprocessto inject pre-parse defaults for missing fields:2. Generalize
optin = "optional"from preprocess to transformThe "user-written input handler accepts absence" signal moves from
$ZodPreprocessto$ZodTransform. Any schema with a transform fn at its input boundary now declaresoptin === "optional"at runtime. Effects:z.transform(fn)now accepts absent object keys (NEW; small population, transform standalone is rarely used in object property positions).z.string().transform(fn): unchanged (pipe.optin = string.optin = undefined; transform on the OUT side doesn't drive optin).z.unknown().transform(fn).pipe(A): unchanged.The static type stays unchanged — transform's interface doesn't declare optin, so this only sets the runtime value, mirroring the catch pattern. After this,
$ZodPreprocessis a near-empty marker subtype kept for type narrowing.To preserve the long-stable
preprocess(...).optional().parse(undefined) === undefinedbehavior (true in both 4.3 and 4.4),$ZodTransformnow sets thefallbackpayload flag on every invocation.$ZodOptionalalready clobbers a result withfallback === truewhen its input wasundefined.3. Propagate
fallbackthrough pipe boundariesThe flag
$ZodCatchset whencatchValuesubstitutes (per #5939) was being dropped at pipe boundaries.handlePipeResultallocated a fresh payload that copied onlyvalueandissues, so any chain shaped likecatch().transform()…optional()lost the flag andoptionalcouldn't tell the inner had recovered.Also renames the field from
caught→fallback. The new name describes the consumer contract — "outer wrapper, override me if you have a better value when input was undefined" — rather than the specific producer ($ZodCatch). Internal-only,@internal-marked, no public surface.4. Internal wiki doc
wiki/optionality.mdcaptures the current state ofoptin/optout/fallback, who sets each, who reads each, the static/runtime divergence pattern, walked-through cases for the gnarly interactions, and the "flexible inputs, strict outputs" design principle. Internal-only; not published.Design principle
The static/runtime divergence on catch and transform expresses a deliberate stance: schemas with a user-written escape hatch (catch's recovery, transform's fn) accept
undefinedat runtime even when the static type declares the input as required. Schemas without an escape hatch (coerce, plainstring/number,unknown,any, etc.) stay strict — users opt in explicitly via.optional()or.default(...)when they want absence accepted.Follows up on #5937 / #5939.