Skip to content

++fl overflows to infinity in directed rounding modes (%z/%d/%u); IEEE 754 requires saturation #7426

Description

@sigilante

++fl (and so ++ff, ++rs, ++rd, ++rh, ++rq) overflows to infinity in every rounding mode. IEEE 754 §7.4 requires directed modes to saturate: round-toward-zero never overflows, round-down never overflows a positive result, round-up never overflows a negative one. The result must be the largest finite value of the correct sign.

Repro (urbit eval, calling ++ff directly so vere's rs jets do not run):

> `@ux`(~(mul ff [[8 23 --127] %z]) `@rs`0x7f7f.ffff .2)
0x7f80.0000                                  :: Hoon: +inf
> `@ux`(~(add ff [[8 23 --127] %d]) `@rs`0x7f7f.ffff `@rs`0x7f7f.ffff)
0x7f80.0000                                  :: Hoon: +inf

Expected: 0x7f7f.ffff in both cases. Berkeley SoftFloat and hardware agree. The jetted door already gives the correct answer, so today the jet and the Hoon disagree:

> `@ux`(~(mul rs %z) `@rs`0x7f7f.ffff .2)
0x7f7f.ffff                                  :: vere jet (SoftFloat)

Cause. ++lug in ++fl (hoon.hoon, "central rounding mechanism") ends with

?:  =(den %i)  [%f & a]
?:  =((cmp:si emx e.a) -1)  [%i &]  [%f & a]      ::  enforce max. exp

The max-exponent check ignores the rounding mode t. Any exact result whose truncated exponent exceeds emx (magnitude at least 2^(emax+1)) becomes [%i &]. Results within one ulp of the largest finite value take the earlier path and are handled correctly, which is why this is easy to miss.

Fix sketch. In that final check, when e.a exceeds emx, return [%f & lfn] for %fl and %sm (the toward-zero directions after swr sign handling) and keep [%i &] for %ce, %lg, %ne, %na, %nt. %ne/%na/%nt are correct to give inf: nearest modes round any value at or above MAX + ulp/2 to infinity.

Found while writing a pure-Rust SoftFloat for NockApp jets (numerics), where the jet must be bit-identical to the Nock. Related: %a in ++fl is a directed round-away-from-zero (ceiling on magnitude, no tie handling), which differs from what vere's _set_rounding maps it to (softfloat_round_near_maxMag); no shipped door admits %a, so that one is latent.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions