LLVM can reorder rounding-mode-sensitive float operations outside rounding mode changes in SSE

[pcwalton]

In release mode, this code can break:

    /// Converts these packed floats to integers via the floor function.
    #[inline]
    pub fn floor_to_i32x4(self) -> I32x4 {
        unsafe {
            x86::_MM_SET_ROUNDING_MODE(x86::_MM_ROUND_DOWN);
            let value = I32x4(x86::_mm_cvtps_epi32(self.0));
            x86::_MM_SET_ROUNDING_MODE(x86::_MM_ROUND_NEAREST);
            value
        }
    }

Because value can get reordered after all the rounding mode switches. I'm not sure what the right thing to do here is.

[pcwalton]

Apparently LLVM is working on fixing this: https://twitter.com/stephentyrone/status/1261470760624893953

[RalfJung]

I seem to recall language along the lines of "switching rounding modes during Rust execution is UB" or so... but maybe what I remember is outdated? Cc @hanna-kruppe @Gankra

[hanna-kruppe]

I don't think there's any formal lang team decision on the matter but "it's UB" is probably the most succinct and conservative way to describe what can happen today (and as long as rustc emits fadd, fmul, etc. IR operations) when code messes with the default floating point environment.

[RalfJung]

Ah I see, so in particular there is no way to ensure that no other FP instructions from elsewhere in the code are moved into the "differently rounded block of code" in the OP.

@pcwalton I am not sure what your use-case is, but do you have any way to ensure problems like the above do not arise?

Also I feel like we shouldn't expose "change rounding mode" intrinsics to begin with if they cannot actually be used correctly...

[RalfJung]

rust-lang/rfcs#3514 would resolve this issue as not-a-bug: calling _MM_SET_ROUNDING_MODE to change the rounding mode is UB; rustc may assume that the rounding mode is always in its default state.