Skip to content

Random.NextBytes is very slightly biased against 0xFF #41457

Closed

Description

@Clockwork-Muse

NextBytes(byte[]) is implemented like so:

public virtual void NextBytes(byte[] buffer)
{
    if (buffer == null) throw new ArgumentNullException(nameof(buffer));
    for (int i = 0; i < buffer.Length; i++)
    {
        buffer[i] = (byte)InternalSample();
    }
}

(side note: why no unchecked on the cast...?)

The Span<byte> version has the same effective implementation if Next() has not been overridden.

The problem is that InternalSample() returns values in the range [0, int.MaxValue) - that is, int.MaxValue is not included. This skews the distribution of possible values for bytes:

  • 0x00-0xFE - 8388608 values
  • 0xFF - 8388607 values

(Sample generation program)

The catch is that I'm not sure that this is fixable, given that any fix for this would be a breaking change in the behavior, and we've been reluctant to do so for other parts of Random.

Activity

  1. Dotnet-GitSync-Bot commented on Aug 27, 2020

    @Dotnet-GitSync-Bot
    Collaborator

    I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

  2. GrabYourPitchforks commented on Aug 27, 2020

    @GrabYourPitchforks
    Member

    Possible dupe of #23198.

  3. Clockwork-Muse commented on Aug 27, 2020

    @Clockwork-Muse
    ContributorAuthor

    @GrabYourPitchforks - I mean, that issue is about the fact that there's a deviation in the algorithm used to generate what InternalSample() returns.

    This one is more about how there's a slight bias due to the fact that InternalSample() doesn't return int.MaxValue. That is, even if there was no deviation in the algorithm, there would still be this issue.

    Although yes, it may end up being a dupe due to the resolution. I just wanted this explicitly called out.

  4. added this to the Future milestone on Aug 27, 2020
  5. removed
    untriagedNew issue has not been triaged by the area owner
    on Aug 27, 2020
  6. GrabYourPitchforks commented on Aug 31, 2020

    @GrabYourPitchforks
    Member

    Ah, yes, you're right in that it's not an exact dupe. But the resolution would be the same. Either they're all resolved as "by design", or they're fixed all at once.

  7. ghost added
    in-prThere is an active PR which will close this issue when it is merged
    on Jan 20, 2021
  8. ghost removed
    in-prThere is an active PR which will close this issue when it is merged
    on Jan 21, 2021
  9. Clockwork-Muse commented on Jan 21, 2021

    @Clockwork-Muse
    ContributorAuthor

    Note to future readers:

    The fix implemented at this time only affects randoms created without an explicit seed. Due to compatibility concerns, randoms initialized with a seed retain the old behavior.

  10. ghost locked as resolved and limited conversation to collaborators on Feb 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions