Skip to content

fix(v4): generalize optin/fallback to transform; restore preprocess on absent keys#5941

Merged
colinhacks merged 5 commits into
mainfrom
fix-fallback-flag-and-preprocess
May 4, 2026
Merged

fix(v4): generalize optin/fallback to transform; restore preprocess on absent keys#5941
colinhacks merged 5 commits into
mainfrom
fix-fallback-flag-and-preprocess

Conversation

@colinhacks

@colinhacks colinhacks commented May 4, 2026

Copy link
Copy Markdown
Owner

Builds on #5939 to clean up the optional / catch / transform / preprocess interaction. Three independent changes that share the same internal mechanism (a payload fallback flag and an optin === "optional" declaration), plus an internal wiki doc capturing the design.

1. Restore preprocess handling for absent object keys (4.3 parity)

z.object({ a: z.preprocess(fn, T) }).parse({}) worked in 4.3 — the fn ran with undefined, produced a value, the inner schema validated it. After #5661 tightened the object parser, this started failing on absent keys, breaking a common pattern of using preprocess to inject pre-parse defaults for missing fields:

z.object({
  area: z.preprocess((v) => (v ? String(v).split(",") : []), z.array(z.string())),
}).parse({})
// 4.3:  { area: [] }
// 4.4:  FAIL
// now:  { area: [] }

2. Generalize optin = "optional" from preprocess to transform

The "user-written input handler accepts absence" signal moves from $ZodPreprocess to $ZodTransform. Any schema with a transform fn at its input boundary now declares optin === "optional" at runtime. Effects:

  • preprocess inherits optin via pipe (in side = transform). Same outcome as setting it on preprocess explicitly.
  • Standalone z.transform(fn) now accepts absent object keys (NEW; small population, transform standalone is rarely used in object property positions).
  • z.string().transform(fn): unchanged (pipe.optin = string.optin = undefined; transform on the OUT side doesn't drive optin).
  • z.unknown().transform(fn).pipe(A): unchanged.

The static type stays unchanged — transform's interface doesn't declare optin, so this only sets the runtime value, mirroring the catch pattern. After this, $ZodPreprocess is a near-empty marker subtype kept for type narrowing.

To preserve the long-stable preprocess(...).optional().parse(undefined) === undefined behavior (true in both 4.3 and 4.4), $ZodTransform now sets the fallback payload flag on every invocation. $ZodOptional already clobbers a result with fallback === true when its input was undefined.

z.preprocess((v) => v ?? "X", z.string()).optional().parse(undefined)
// 4.3 + 4.4:  undefined
// now:        undefined  (preserved)

3. Propagate fallback through pipe boundaries

The flag $ZodCatch set when catchValue substitutes (per #5939) was being dropped at pipe boundaries. handlePipeResult allocated a fresh payload that copied only value and issues, so any chain shaped like catch().transform()…optional() lost the flag and optional couldn't tell the inner had recovered.

z.string().catch("X").transform((s) => s + "!").optional().parse(undefined)
// before: "X!"
// now:    undefined

Also renames the field from caughtfallback. The new name describes the consumer contract — "outer wrapper, override me if you have a better value when input was undefined" — rather than the specific producer ($ZodCatch). Internal-only, @internal-marked, no public surface.

4. Internal wiki doc

wiki/optionality.md captures the current state of optin / optout / fallback, who sets each, who reads each, the static/runtime divergence pattern, walked-through cases for the gnarly interactions, and the "flexible inputs, strict outputs" design principle. Internal-only; not published.

Design principle

The static/runtime divergence on catch and transform expresses a deliberate stance: schemas with a user-written escape hatch (catch's recovery, transform's fn) accept undefined at runtime even when the static type declares the input as required. Schemas without an escape hatch (coerce, plain string/number, unknown, any, etc.) stay strict — users opt in explicitly via .optional() or .default(...) when they want absence accepted.

Follows up on #5937 / #5939.

`$ZodCatch` sets a payload flag when its `catchValue` substitutes so an
outer `$ZodOptional` can clobber the recovery value with `undefined`
(per #5939). But `handlePipeResult` was building a fresh payload for the
right side of the pipe without copying the flag, so any chain like
`catch().transform()...optional()` lost it — `optional` couldn't tell
the inner had recovered, and surfaced the catch value instead of
clobbering.

Propagate the flag through pipe handoffs, alongside `value`/`issues`.
Also rename `caught` to `fallback`: a slightly broader name that
describes the consumer contract ("override me if you have a better
value when input was undefined") rather than the producer ("catch fired
me"). Internal-only; no public API surface.
@pullfrog

pullfrog Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

TL;DR — Two related cleanups around the optional/catch/transform interaction. Restores 4.3 behavior where z.preprocess(fn, T) accepts absent object keys, and propagates the internal fallback flag through pipe boundaries so that optional() wrapping a catch().transform()/.pipe(...) chain still clobbers the recovered value with undefined on missing input. Also ships an internal wiki/optionality.md reference covering the full optin/optout/fallback machinery.

Key changes

  • Restore preprocess on absent object keys$ZodTransform now sets optin = "optional" at runtime, which $ZodPreprocess inherits through pipe, so z.object({ a: z.preprocess(fn, T) }).parse({}) runs fn(undefined) as it did pre-fix(v4): align object and tuple optionality handling #5661.
  • Preserve preprocess(...).optional().parse(undefined) === undefined$ZodTransform stamps payload.fallback = true on every invocation so an outer $ZodOptional still short-circuits to undefined even though the inner transform ran.
  • Carry the flag through handlePipeResult — the fresh payload passed to the right-hand schema now includes fallback alongside value and issues.
  • Rename caughtfallback on ParsePayload — describes the consumer contract ("outer wrapper, override me if you have a better value") rather than the specific producer; updated at $ZodCatch set-sites and the $ZodOptional read-site.
  • Drop $ZodPreprocess constructor body — the defineLazy overrides for optin/optout are gone; pipe inheritance from $ZodTransform handles both axes.
  • Internal wiki/optionality.md — reference doc covering optin × optout × fallback, who sets and reads each, pipe propagation, static/runtime divergence, a full case walkthrough, and why z.unknown().transform(fn).pipe(T) stays strict on absent input.
  • Regression tests — covers preprocess on absent keys with and without optional, plus optional() over catch().transform()/.pipe() chains, multi-transform chains, and object-property membership.

Summary | 5 files | 5 commits | base: mainfix-fallback-flag-and-preprocess


preprocess accepts absent object keys again

Before: After #5661 tightened the object parser, z.object({ a: z.preprocess(fn, T) }).parse({}) started failing on absent keys, breaking the common pattern of using preprocess to inject pre-parse defaults for missing fields.
After: $ZodTransform declares optin = "optional" at runtime; $ZodPreprocess inherits that through its in side of the pipe, so $ZodObject treats absent keys as legal and fn runs with undefined exactly as it did in 4.3.

The fix is now expressed at the transform level rather than as a preprocess-specific override, which means z.transform(fn) itself also accepts absent input — a consistent rule ("any schema with a user-written escape hatch accepts undefined at runtime"). The old defineLazy overrides on $ZodPreprocess are gone; pipe inheritance does the work.

To keep the multi-year-stable preprocess(fn, T).optional().parse(undefined) === undefined behavior, $ZodTransform also flags every invocation as fallback, which $ZodOptional already knows to clobber when input was undefined.

core/schemas.ts · preprocess.test.ts


Fallback flag survives pipe boundaries

Before: handlePipeResult allocated { value, issues } for the right side, dropping the flag $ZodCatch set when catchValue substitutes, so z.string().catch("X").transform((s) => s + "!").optional().parse(undefined) returned "X!" instead of undefined.
After: the right-side payload is seeded with fallback: left.fallback, so $ZodOptional still sees the flag after any number of transforms or explicit .pipe() hops and clobbers the fallback on undefined input.

Any chain shaped like catch().transform()…optional() or catch().pipe(...)…optional() was losing the signal that the inner had recovered, so optional surfaced the catch value instead of undefined. Fix is a one-line propagation in handlePipeResult.

core/schemas.ts · catch.test.ts


caughtfallback rename

Before: The flag was named after its one producer ($ZodCatch), coupling the contract to a specific schema type.
After: The name describes what consumers should do — treat the value as a fallback that an outer wrapper may override when input was undefined. Now set by both $ZodCatch and $ZodTransform.

Field is @internal, so there's no public surface change. Follows up on #5937 / #5939.

core/schemas.ts · classic/schemas.ts


Internal optionality reference

Before: The optin / optout / fallback / aborted signals lived only in code and scattered PR threads (#5661, #5917, #5929, #5937, #5939, #5941).
After: wiki/optionality.md documents the three runtime signals, who sets and reads each, the static/runtime divergence pattern (catch, transform, preprocess), pipe propagation, a concrete case walkthrough, and why z.unknown().transform(fn).pipe(T) stays strict on absent input despite containing a transform.

Meant as an internal reference rather than user-facing docs — frames the design principle ("flexible inputs, strict outputs at runtime") and the rule of thumb for which schema kinds accept absent input.

wiki/optionality.md

Pullfrog  | View workflow run | via Pullfrog | Using Claude Opus𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no issues found.

Pullfrog  | View workflow run | Using Claude Opus𝕏

`z.object({ a: z.preprocess(fn, T) }).parse({})` worked in 4.3 (the fn
ran with `undefined`, produced a value, the inner schema validated it)
but started failing on absent keys after #5661 tightened the object
parser. Users commonly use preprocess to inject pre-parse defaults for
fields that may be missing — that pattern broke silently in 4.4.

Restore by marking $ZodPreprocess as `optin === "optional"`, telling
`$ZodObject` that absent keys are legal here. The fn then runs with
`undefined` exactly as it did in 4.3.

To preserve the long-stable behavior of `preprocess(fn, T).optional()
.parse(undefined)` returning `undefined` (true in both 4.3 and 4.4 for
multi-year compatibility), have `$ZodTransform` set the `fallback`
payload flag on every invocation. `$ZodOptional` already clobbers a
result with `fallback === true` when its input was `undefined`, so the
outer optional keeps short-circuiting to `undefined` even though the
transform now runs underneath.

  z.object({ a: z.preprocess(v => v ?? "X", z.string()) }).parse({})
  // 4.3:  { a: "X" }
  // 4.4:  FAIL  (regression)
  // now:  { a: "X" }

  z.preprocess(v => v ?? "X", z.string()).optional().parse(undefined)
  // 4.3 + 4.4:  undefined
  // now:        undefined  (preserved)

Drops the `optin` defer-to-inner from #5929, but the same outcome
holds: when inner is `.optional()`, preprocess still accepts absent
keys (`optin === "optional"` either way).
@colinhacks colinhacks changed the title fix(v4): propagate fallback flag through pipe boundaries fix(v4): restore preprocess on absent keys + propagate fallback through pipes May 4, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues. Reviewed the following changes:

  • Restored preprocess handling for absent object keys by pinning $ZodPreprocess.optin to "optional" (4.3 parity).
  • Marked every $ZodTransform invocation with payload.fallback = true (both core and classic, sync and async) so outer $ZodOptional clobbers transformed output when input was undefined.
  • Added regression coverage in preprocess.test.ts and catch.test.ts for direct, pipe, and object-key shapes.
  • Updated the fallback JSDoc on ParsePayload to reflect the new producer.

Pullfrog  | View workflow run | Using Claude Opus𝕏

colinhacks added 2 commits May 3, 2026 23:47
Promotes the "user-written input handler accepts absence" signal from
$ZodPreprocess to $ZodTransform. Any schema with a transform fn at its
input boundary (preprocess, standalone z.transform) now declares
optin="optional" at runtime.

Effects:
- preprocess inherits optin="optional" via pipe.optin = transform.optin
  (same outcome as the previous commit's explicit override; preprocess
  loses both its optin and optout overrides since pipe already does the
  optout defer)
- standalone z.transform(fn) now accepts absent object keys
- z.string().transform(fn): unchanged (pipe.optin = string.optin =
  undefined; transform on the OUT side doesn't drive optin)
- z.unknown().transform(fn).pipe(A): unchanged (pipe.optin = unknown.
  optin = undefined)

The static type stays unchanged — transform's interface doesn't
declare optin, so this only sets the runtime value, mirroring the
catch pattern. Captures the "flexible inputs, strict outputs" design
principle: schemas with a user-written escape hatch (catch's recovery,
transform's fn) accept undefined at runtime even when the static type
declares the input as required.

After this, $ZodPreprocess is a near-empty marker subtype — the
constructor body is just $ZodPipe.init(inst, def), kept for type
narrowing and traits identity.
Captures the current state of optin/optout/fallback, who sets each,
who reads each, the static/runtime divergence pattern, and walked-
through cases for the gnarly interactions (catch+optional, default
vs catch vs preprocess vs transform under optional, etc.).

Also documents the "flexible inputs, strict outputs" design principle
that motivates the runtime/static optin divergence on $ZodCatch and
$ZodTransform: schemas with a user-written escape hatch accept
undefined at runtime while keeping their declared input type strict.

Internal-only doc; not published.
@colinhacks colinhacks changed the title fix(v4): restore preprocess on absent keys + propagate fallback through pipes fix(v4): generalize optin/fallback to transform; restore preprocess on absent keys May 4, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues. Reviewed the following changes:

  • Moved optin = "optional" from $ZodPreprocess/$ZodPipe into $ZodTransform, so any transform at a pipe's in side (including preprocess) inherits optin naturally through the existing lazy pipe defer.
  • Dropped the now-redundant explicit defineLazy of optin/optout in $ZodPreprocess — pipe's defaults produce equivalent runtime behavior since def.in is a transform.
  • Accepted the documented side effect that standalone z.transform(fn) now accepts absent object keys at runtime (small blast radius, consistent with the catch/preprocess pattern of runtime accepting absence without widening the static input type).
  • Added wiki/optionality.md as an internal reference covering optin/optout/fallback producers and consumers, the static/runtime divergence rationale, and walked-through cases — repo-root only, not published via packages/zod.

Pullfrog  | View workflow run | Using Claude Opus𝕏

Adds the explicit contrast between z.preprocess(fn, T) (= pipe(transform,
T), accepts absent) and z.unknown().transform(fn).pipe(T) (= pipe(pipe(
unknown, transform), T), rejects absent). The two look structurally
similar but only the leading position drives optin, and z.unknown()
isn't input-optional.

Also drops the stale "prototype only" caveat from the standalone
z.transform(fn) walkthrough — the runtime optin=optional move from
preprocess to transform is now a real part of this branch, not a
prototype.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant