Describe the operator
This proposes an update to the existing random-number operators — starting with
RandomUniform as the pilot — that makes them optionally deterministic via an opt-in,
fully specified PRNG, while keeping the default behavior unchanged.
Problem. The random operators (RandomUniform, RandomNormal, RandomUniformLike,
RandomNormalLike, Bernoulli, Multinomial) are non-deterministic per spec: even with
seed set, results legitimately differ across implementations. As a consequence, their
conformance status is today either unmeasurable or mismeasured:
- Five of the six random operators have no backend node tests at all
(RandomUniform, RandomNormal, RandomUniformLike, RandomNormalLike,
Multinomial) — no expected output can be defined for a non-deterministic operator.
The ONNX Backend Scoreboard derives its
per-operator status from the node test suite (onnx/backend/test/report/coverage.py
marks an operator "Passed" only if it appears in at least one passing test), so these
five operators are reported as failed/unsupported for every backend, regardless of
actual support — in the current scoreboard results, onnxruntime shows all five as
"Failed!".
- The one random operator that does have node tests,
Bernoulli, has tests that are
effectively unsatisfiable. Their expected outputs are draws from numpy's RNG; the
test source itself notes that an implementation "may not produce the same values in
different implementations even if a seed is specified". ONNX's own reference-backend
suite excludes them for exactly this reason
(onnx/test/test_backend_reference.py: "The following tests cannot pass because they
consists in generating random number"), and on the scoreboard onnxruntime fails them.
- The same root cause can also hide behind a green status:
Dropout is reported as
"Passed" because the per-operator metric from (1) only requires one passing test
containing the operator, and Dropout's deterministic inference-mode tests pass — while
its actual random behavior fails every test that exercises it (for onnxruntime, all
four test_training_dropout* cases with a non-zero ratio fail, since their expected
outputs are numpy RNG draws no conforming backend is required to reproduce). Red for
RandomUniform, green for Dropout — same underlying gap, and neither color reflects
actual support.
- Beyond testing, reproducibility is a real user requirement (debugging, regression
testing, safety-critical certification workflows), and today it cannot be expressed in
a portable ONNX model at all.
With an opt-in, fully specified generator, the random operators can be covered by the
standard backend test suite with bit-exact expected outputs for the first time — the
attached implementation adds the first-ever node tests for RandomUniform — and their
support becomes genuinely measurable on the scoreboard instead of structurally "failed".
Proposed mechanism (new opset version of RandomUniform; same pattern intended for the
other five random operators in follow-ups):
- New string attribute
generator, default "unspecified":
"unspecified": the PRNG remains implementation-defined — explicitly no determinism
guarantee, not even across runs of the same implementation. This is exactly the
current behavior, so the change is fully backward compatible. (The value is named
"unspecified" rather than "default" so it states the semantics and stays accurate
if a future opset ever recommends a different generator.)
"philox4x32_10": the fully specified Philox-4x32-10 counter-based generator
(Salmon et al., "Parallel random numbers: as easy as 1, 2, 3", SC'11, standard
constants). Given the same seed, every conforming implementation must produce
bit-identical results. The value set is deliberately extensible for future
algorithms.
- New INT attribute
seed_int64, required in deterministic mode: its two's-complement bits
are the unsigned 64-bit Philox key. A dedicated attribute is needed because ONNX
attributes have exactly one fixed type and the legacy seed is float32, whose 24-bit
significand cannot address a 64-bit key space (precedent: Constant's
value_int/value_float family). The float seed is forbidden alongside a
deterministic generator (enforced by shape inference) to avoid silent ambiguity.
- New optional int64 scalar input
offset (default 0): it keys Philox counter words
c2/c3 while the block index lives in c0/c1, so streams of different offsets
never overlap, regardless of output size.
- This is what enables streaming inference and testing at the same time: feeding a
different offset per run (a host-side counter, an initializer advanced at checkpoint
time, or a Loop-carried value) yields fresh yet reproducible values per run, with
every individual run replayable; a constant or absent offset reproduces the same
values, which is what makes the operator testable.
- No hidden state, by design. A "continue where the last run stopped" mode was
considered and rejected: a model run must stay a pure function of its inputs — hidden
RNG state would be undefined under concurrent runs, session cloning, and replay, and
would destroy the testability this proposal is about. Reset-vs-continue is therefore a
wiring decision around the explicit offset input, following the explicit-state
precedent of CausalConvWithState (past_state/present_state) and the Attention
KV cache.
- A
next_offset output (offset + 1) was also considered and rejected: unlike
present_state/KV-cache outputs, it would carry no information — the next offset is
trivially computable by a host counter, an Add node, or a loop-carried update — and,
since every offset value already selects an independent stream, it would over-specify a
chaining protocol where any non-repeating scheme works.
- All arithmetic happens in the output data type — an implementation never needs
double-precision math unless dtype is double. (The exact per-dtype bit mapping is
fully specified in the operator documentation included in the linked PR.)
- With
generator="philox4x32_10", the random number generated for element i is a pure
function of (seed_int64, offset, i) — the counter-based design makes the
specification order-independent and parallelizable by construction: all elements can
be computed independently, in any order, on any device. This property is specific to
Philox (and one of the main reasons for choosing it); it is not a blanket requirement on
future generator values.
Why Philox-4x32-10? We surveyed PRNG adoption across ML frameworks before choosing:
Philox-4x32-10 is the de-facto standard for deterministic random generation in ML — it is
the algorithm behind JAX's jax.random, TensorFlow's stateless RNGs
(tf.random.stateless_* / tf.random.Generator), and PyTorch's CUDA RNG (cuRAND
Philox4x32-10); NumPy ships a Philox bit generator as well.
Can this operator be constructed using existing onnx operators?
No. Pseudo-random bit generation (64-bit modular multiplies, word shuffles, bit
extraction) is not expressible as an ONNX function body, and this is an update to an
existing operator rather than a new one. The "unspecified" mode keeps full backward
compatibility; the version converter can upgrade 27→28 unconditionally and downgrade 28→27
whenever the deterministic features are unused.
Is this operator used by any model currently? Which one?
RandomUniform and its siblings are long-standing ONNX operators used e.g. for noise
injection, sampling layers, and Monte-Carlo-style inference. The deterministic mode does
not target a single model but the conformance-testing gap described above: with it,
the random operators get satisfiable node tests for the first time, and their support
becomes genuinely measurable.
Are you willing to contribute it? (Y/N)
Y. A complete, working implementation exists and can be submitted as a PR immediately:
emmtrix#3
It contains:
- the full spec text in the operator documentation: self-contained Philox round
function, key schedule, counter mapping, per-dtype value derivation
- schema and shape-inference validation (generator whitelist,
seed_int64 required
with a deterministic generator, float seed forbidden alongside it, scalar offset)
- 27↔28 version-converter adapters (upgrade unconditional; downgrade whenever the
deterministic features are unused)
- a vectorized reference implementation, verified against the authoritative reference
values: the three philox4x32 10 known-answer vectors from the Random123 distribution
(the algorithm authors' published counter/key → expected-output pairs, hardcoded inline
in a unit test) and, independently, the word stream produced by the canonical Random123
implementation for this operator's specific counter mapping
- the first-ever backend node tests for
RandomUniform — 8 cases covering
multi-block/partial-block stress, n-D shapes, all four float types, and offset
streaming; each case documents its test intent in a docstring that is published into
docs/TestCoverage.md
- expected outputs generated by an independent inline Philox implementation in the
test case, cross-checking the reference runtime bit-exactly
- shape inference, version converter, and evaluator tests, including statelessness
(two runs are bit-identical), element independence (smaller tensors are prefixes of
larger ones), and stream disjointness across offsets
Notes
- Rollout plan: once the mechanism is accepted for
RandomUniform, the same
generator/seed_int64/offset pattern is intended for RandomNormal,
RandomUniformLike, RandomNormalLike, Bernoulli, and Multinomial — which would
also allow replacing Bernoulli's unsatisfiable node tests and covering Dropout's random
training mode via its function-body expansion. The shared validation helper, attribute
documentation, version-converter adapter, and the Philox implementation in the reference
runtime are already factored for reuse.
- Future generator algorithms can be added as new attribute values in later opsets without
breaking existing models.
Describe the operator
This proposes an update to the existing random-number operators — starting with
RandomUniformas the pilot — that makes them optionally deterministic via an opt-in,fully specified PRNG, while keeping the default behavior unchanged.
Problem. The random operators (
RandomUniform,RandomNormal,RandomUniformLike,RandomNormalLike,Bernoulli,Multinomial) are non-deterministic per spec: even withseedset, results legitimately differ across implementations. As a consequence, theirconformance status is today either unmeasurable or mismeasured:
(
RandomUniform,RandomNormal,RandomUniformLike,RandomNormalLike,Multinomial) — no expected output can be defined for a non-deterministic operator.The ONNX Backend Scoreboard derives its
per-operator status from the node test suite (
onnx/backend/test/report/coverage.pymarks an operator "Passed" only if it appears in at least one passing test), so these
five operators are reported as failed/unsupported for every backend, regardless of
actual support — in the current scoreboard results, onnxruntime shows all five as
"Failed!".
Bernoulli, has tests that areeffectively unsatisfiable. Their expected outputs are draws from numpy's RNG; the
test source itself notes that an implementation "may not produce the same values in
different implementations even if a seed is specified". ONNX's own reference-backend
suite excludes them for exactly this reason
(
onnx/test/test_backend_reference.py: "The following tests cannot pass because theyconsists in generating random number"), and on the scoreboard onnxruntime fails them.
Dropoutis reported as"Passed" because the per-operator metric from (1) only requires one passing test
containing the operator, and Dropout's deterministic inference-mode tests pass — while
its actual random behavior fails every test that exercises it (for onnxruntime, all
four
test_training_dropout*cases with a non-zero ratio fail, since their expectedoutputs are numpy RNG draws no conforming backend is required to reproduce). Red for
RandomUniform, green forDropout— same underlying gap, and neither color reflectsactual support.
testing, safety-critical certification workflows), and today it cannot be expressed in
a portable ONNX model at all.
With an opt-in, fully specified generator, the random operators can be covered by the
standard backend test suite with bit-exact expected outputs for the first time — the
attached implementation adds the first-ever node tests for
RandomUniform— and theirsupport becomes genuinely measurable on the scoreboard instead of structurally "failed".
Proposed mechanism (new opset version of
RandomUniform; same pattern intended for theother five random operators in follow-ups):
generator, default"unspecified":"unspecified": the PRNG remains implementation-defined — explicitly no determinismguarantee, not even across runs of the same implementation. This is exactly the
current behavior, so the change is fully backward compatible. (The value is named
"unspecified"rather than"default"so it states the semantics and stays accurateif a future opset ever recommends a different generator.)
"philox4x32_10": the fully specified Philox-4x32-10 counter-based generator(Salmon et al., "Parallel random numbers: as easy as 1, 2, 3", SC'11, standard
constants). Given the same seed, every conforming implementation must produce
bit-identical results. The value set is deliberately extensible for future
algorithms.
seed_int64, required in deterministic mode: its two's-complement bitsare the unsigned 64-bit Philox key. A dedicated attribute is needed because ONNX
attributes have exactly one fixed type and the legacy
seedis float32, whose 24-bitsignificand cannot address a 64-bit key space (precedent:
Constant'svalue_int/value_floatfamily). The floatseedis forbidden alongside adeterministic generator (enforced by shape inference) to avoid silent ambiguity.
offset(default 0): it keys Philox counter wordsc2/c3while the block index lives inc0/c1, so streams of different offsetsnever overlap, regardless of output size.
different offset per run (a host-side counter, an initializer advanced at checkpoint
time, or a
Loop-carried value) yields fresh yet reproducible values per run, withevery individual run replayable; a constant or absent offset reproduces the same
values, which is what makes the operator testable.
considered and rejected: a model run must stay a pure function of its inputs — hidden
RNG state would be undefined under concurrent runs, session cloning, and replay, and
would destroy the testability this proposal is about. Reset-vs-continue is therefore a
wiring decision around the explicit
offsetinput, following the explicit-stateprecedent of
CausalConvWithState(past_state/present_state) and theAttentionKV cache.
next_offsetoutput (offset + 1) was also considered and rejected: unlikepresent_state/KV-cache outputs, it would carry no information — the next offset istrivially computable by a host counter, an
Addnode, or a loop-carried update — and,since every offset value already selects an independent stream, it would over-specify a
chaining protocol where any non-repeating scheme works.
double-precision math unless
dtypeis double. (The exact per-dtype bit mapping isfully specified in the operator documentation included in the linked PR.)
generator="philox4x32_10", the random number generated for elementiis a purefunction of (
seed_int64,offset,i) — the counter-based design makes thespecification order-independent and parallelizable by construction: all elements can
be computed independently, in any order, on any device. This property is specific to
Philox (and one of the main reasons for choosing it); it is not a blanket requirement on
future generator values.
Why Philox-4x32-10? We surveyed PRNG adoption across ML frameworks before choosing:
Philox-4x32-10 is the de-facto standard for deterministic random generation in ML — it is
the algorithm behind JAX's
jax.random, TensorFlow's stateless RNGs(
tf.random.stateless_*/tf.random.Generator), and PyTorch's CUDA RNG (cuRANDPhilox4x32-10); NumPy ships a Philox bit generator as well.Can this operator be constructed using existing onnx operators?
No. Pseudo-random bit generation (64-bit modular multiplies, word shuffles, bit
extraction) is not expressible as an ONNX function body, and this is an update to an
existing operator rather than a new one. The
"unspecified"mode keeps full backwardcompatibility; the version converter can upgrade 27→28 unconditionally and downgrade 28→27
whenever the deterministic features are unused.
Is this operator used by any model currently? Which one?
RandomUniformand its siblings are long-standing ONNX operators used e.g. for noiseinjection, sampling layers, and Monte-Carlo-style inference. The deterministic mode does
not target a single model but the conformance-testing gap described above: with it,
the random operators get satisfiable node tests for the first time, and their support
becomes genuinely measurable.
Are you willing to contribute it? (Y/N)
Y. A complete, working implementation exists and can be submitted as a PR immediately:
emmtrix#3
It contains:
function, key schedule, counter mapping, per-dtype value derivation
seed_int64requiredwith a deterministic generator, float
seedforbidden alongside it, scalaroffset)deterministic features are unused)
values: the three
philox4x32 10known-answer vectors from the Random123 distribution(the algorithm authors' published counter/key → expected-output pairs, hardcoded inline
in a unit test) and, independently, the word stream produced by the canonical Random123
implementation for this operator's specific counter mapping
RandomUniform— 8 cases coveringmulti-block/partial-block stress, n-D shapes, all four float types, and offset
streaming; each case documents its test intent in a docstring that is published into
docs/TestCoverage.mdtest case, cross-checking the reference runtime bit-exactly
(two runs are bit-identical), element independence (smaller tensors are prefixes of
larger ones), and stream disjointness across offsets
Notes
RandomUniform, the samegenerator/seed_int64/offsetpattern is intended forRandomNormal,RandomUniformLike,RandomNormalLike,Bernoulli, andMultinomial— which wouldalso allow replacing Bernoulli's unsatisfiable node tests and covering Dropout's random
training mode via its function-body expansion. The shared validation helper, attribute
documentation, version-converter adapter, and the Philox implementation in the reference
runtime are already factored for reuse.
breaking existing models.