Skip to content

[Proposal] Optional deterministic generator mode for the random operators (starting with RandomUniform) #8161

Description

@strimo378

Describe the operator

This proposes an update to the existing random-number operators — starting with
RandomUniform as the pilot — that makes them optionally deterministic via an opt-in,
fully specified PRNG, while keeping the default behavior unchanged.

Problem. The random operators (RandomUniform, RandomNormal, RandomUniformLike,
RandomNormalLike, Bernoulli, Multinomial) are non-deterministic per spec: even with
seed set, results legitimately differ across implementations. As a consequence, their
conformance status is today either unmeasurable or mismeasured:

  1. Five of the six random operators have no backend node tests at all
    (RandomUniform, RandomNormal, RandomUniformLike, RandomNormalLike,
    Multinomial) — no expected output can be defined for a non-deterministic operator.
    The ONNX Backend Scoreboard derives its
    per-operator status from the node test suite (onnx/backend/test/report/coverage.py
    marks an operator "Passed" only if it appears in at least one passing test), so these
    five operators are reported as failed/unsupported for every backend, regardless of
    actual support — in the current scoreboard results, onnxruntime shows all five as
    "Failed!".
  2. The one random operator that does have node tests, Bernoulli, has tests that are
    effectively unsatisfiable.
    Their expected outputs are draws from numpy's RNG; the
    test source itself notes that an implementation "may not produce the same values in
    different implementations even if a seed is specified". ONNX's own reference-backend
    suite excludes them for exactly this reason
    (onnx/test/test_backend_reference.py: "The following tests cannot pass because they
    consists in generating random number"
    ), and on the scoreboard onnxruntime fails them.
  3. The same root cause can also hide behind a green status: Dropout is reported as
    "Passed" because the per-operator metric from (1) only requires one passing test
    containing the operator, and Dropout's deterministic inference-mode tests pass — while
    its actual random behavior fails every test that exercises it (for onnxruntime, all
    four test_training_dropout* cases with a non-zero ratio fail, since their expected
    outputs are numpy RNG draws no conforming backend is required to reproduce). Red for
    RandomUniform, green for Dropout — same underlying gap, and neither color reflects
    actual support.
  4. Beyond testing, reproducibility is a real user requirement (debugging, regression
    testing, safety-critical certification workflows), and today it cannot be expressed in
    a portable ONNX model at all.

With an opt-in, fully specified generator, the random operators can be covered by the
standard backend test suite with bit-exact expected outputs for the first time — the
attached implementation adds the first-ever node tests for RandomUniform — and their
support becomes genuinely measurable on the scoreboard instead of structurally "failed".

Proposed mechanism (new opset version of RandomUniform; same pattern intended for the
other five random operators in follow-ups):

  • New string attribute generator, default "unspecified":
    • "unspecified": the PRNG remains implementation-defined — explicitly no determinism
      guarantee
      , not even across runs of the same implementation. This is exactly the
      current behavior, so the change is fully backward compatible. (The value is named
      "unspecified" rather than "default" so it states the semantics and stays accurate
      if a future opset ever recommends a different generator.)
    • "philox4x32_10": the fully specified Philox-4x32-10 counter-based generator
      (Salmon et al., "Parallel random numbers: as easy as 1, 2, 3", SC'11, standard
      constants). Given the same seed, every conforming implementation must produce
      bit-identical results. The value set is deliberately extensible for future
      algorithms.
  • New INT attribute seed_int64, required in deterministic mode: its two's-complement bits
    are the unsigned 64-bit Philox key. A dedicated attribute is needed because ONNX
    attributes have exactly one fixed type and the legacy seed is float32, whose 24-bit
    significand cannot address a 64-bit key space (precedent: Constant's
    value_int/value_float family). The float seed is forbidden alongside a
    deterministic generator (enforced by shape inference) to avoid silent ambiguity.
  • New optional int64 scalar input offset (default 0): it keys Philox counter words
    c2/c3 while the block index lives in c0/c1, so streams of different offsets
    never overlap
    , regardless of output size.
  • This is what enables streaming inference and testing at the same time: feeding a
    different offset per run (a host-side counter, an initializer advanced at checkpoint
    time, or a Loop-carried value) yields fresh yet reproducible values per run, with
    every individual run replayable; a constant or absent offset reproduces the same
    values, which is what makes the operator testable.
  • No hidden state, by design. A "continue where the last run stopped" mode was
    considered and rejected: a model run must stay a pure function of its inputs — hidden
    RNG state would be undefined under concurrent runs, session cloning, and replay, and
    would destroy the testability this proposal is about. Reset-vs-continue is therefore a
    wiring decision around the explicit offset input, following the explicit-state
    precedent of CausalConvWithState (past_state/present_state) and the Attention
    KV cache.
  • A next_offset output (offset + 1) was also considered and rejected: unlike
    present_state/KV-cache outputs, it would carry no information — the next offset is
    trivially computable by a host counter, an Add node, or a loop-carried update — and,
    since every offset value already selects an independent stream, it would over-specify a
    chaining protocol where any non-repeating scheme works.
  • All arithmetic happens in the output data type — an implementation never needs
    double-precision math unless dtype is double. (The exact per-dtype bit mapping is
    fully specified in the operator documentation included in the linked PR.)
  • With generator="philox4x32_10", the random number generated for element i is a pure
    function of (seed_int64, offset, i) — the counter-based design makes the
    specification order-independent and parallelizable by construction: all elements can
    be computed independently, in any order, on any device. This property is specific to
    Philox (and one of the main reasons for choosing it); it is not a blanket requirement on
    future generator values.

Why Philox-4x32-10? We surveyed PRNG adoption across ML frameworks before choosing:
Philox-4x32-10 is the de-facto standard for deterministic random generation in ML — it is
the algorithm behind JAX's jax.random, TensorFlow's stateless RNGs
(tf.random.stateless_* / tf.random.Generator), and PyTorch's CUDA RNG (cuRAND
Philox4x32-10); NumPy ships a Philox bit generator as well.

Can this operator be constructed using existing onnx operators?

No. Pseudo-random bit generation (64-bit modular multiplies, word shuffles, bit
extraction) is not expressible as an ONNX function body, and this is an update to an
existing operator rather than a new one. The "unspecified" mode keeps full backward
compatibility; the version converter can upgrade 27→28 unconditionally and downgrade 28→27
whenever the deterministic features are unused.

Is this operator used by any model currently? Which one?

RandomUniform and its siblings are long-standing ONNX operators used e.g. for noise
injection, sampling layers, and Monte-Carlo-style inference. The deterministic mode does
not target a single model but the conformance-testing gap described above: with it,
the random operators get satisfiable node tests for the first time, and their support
becomes genuinely measurable.

Are you willing to contribute it? (Y/N)

Y. A complete, working implementation exists and can be submitted as a PR immediately:
emmtrix#3

It contains:

  • the full spec text in the operator documentation: self-contained Philox round
    function, key schedule, counter mapping, per-dtype value derivation
  • schema and shape-inference validation (generator whitelist, seed_int64 required
    with a deterministic generator, float seed forbidden alongside it, scalar offset)
  • 27↔28 version-converter adapters (upgrade unconditional; downgrade whenever the
    deterministic features are unused)
  • a vectorized reference implementation, verified against the authoritative reference
    values: the three philox4x32 10 known-answer vectors from the Random123 distribution
    (the algorithm authors' published counter/key → expected-output pairs, hardcoded inline
    in a unit test) and, independently, the word stream produced by the canonical Random123
    implementation for this operator's specific counter mapping
  • the first-ever backend node tests for RandomUniform — 8 cases covering
    multi-block/partial-block stress, n-D shapes, all four float types, and offset
    streaming; each case documents its test intent in a docstring that is published into
    docs/TestCoverage.md
  • expected outputs generated by an independent inline Philox implementation in the
    test case, cross-checking the reference runtime bit-exactly
  • shape inference, version converter, and evaluator tests, including statelessness
    (two runs are bit-identical), element independence (smaller tensors are prefixes of
    larger ones), and stream disjointness across offsets

Notes

  • Rollout plan: once the mechanism is accepted for RandomUniform, the same
    generator/seed_int64/offset pattern is intended for RandomNormal,
    RandomUniformLike, RandomNormalLike, Bernoulli, and Multinomial — which would
    also allow replacing Bernoulli's unsatisfiable node tests and covering Dropout's random
    training mode via its function-body expansion. The shared validation helper, attribute
    documentation, version-converter adapter, and the Philox implementation in the reference
    runtime are already factored for reuse.
  • Future generator algorithms can be added as new attribute values in later opsets without
    breaking existing models.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions