Test suite assumes that cos, exp, and other functions have deterministic precision

[RalfJung]

After landing rust-lang/rust#138062, the portable-simd test suite no longer passes in Miri:

0.005650   ---- f32::cos::lanes_1 stdout ----
  0.000026   
  0.000010   thread 'f32::cos::lanes_1' panicked at /home/runner/work/miri-test-libstd/miri-test-libstd/rust-src-patched/library/portable-simd/crates/test_helpers/src/lib.rs:126:43:
  0.000008   called `Result::unwrap()` on an `Err` value: Fail(Reason("assertion failed: `(left == right)` \n  left: `[-0.7418637 (bf3deac8)]`,\n right: `[-0.7418636 (bf3deac6)]` at crates/test_helpers/src/lib.rs:183"), [-207806100.0])
  0.000008   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  0.000007   note: in Miri, you may have to set `MIRIFLAGS=-Zmiri-env-forward=RUST_BACKTRACE` for the environment variable to have an effect
  0.000007   
  0.000007   ---- f32::cos::lanes_2 stdout ----
  0.000007   
  0.000007   thread 'f32::cos::lanes_2' panicked at /home/runner/work/miri-test-libstd/miri-test-libstd/rust-src-patched/library/portable-simd/crates/test_helpers/src/lib.rs:126:43:
  0.000007   called `Result::unwrap()` on an `Err` value: Fail(Reason("assertion failed: `(left == right)` \n  left: `[-0.7418637 (bf3deac8), 1.0 (3f800000)]`,\n right: `[-0.74186367 (bf3deac7), 1.0 (3f800000)]` at crates/test_helpers/src/lib.rs:183"), [-207806100.0, 0.0])

  0.000022   failures:
  0.070860       f32::cos::lanes_1
  0.014170       f32::cos::lanes_2
  0.013913       f32::cos::lanes_3
  0.015623       f32::cos::lanes_6
  0.013689       f32::exp2::lanes_1
  0.014103       f32::exp2::lanes_2
  0.015393       f32::exp2::lanes_3
  0.013566       f32::exp2::lanes_6
  0.013604       f32::exp::lanes_1
  0.015736       f32::exp::lanes_2
  0.013619       f32::exp::lanes_3
  0.013681       f32::exp::lanes_6
  0.013655       f32::ln::lanes_1
  0.015216       f32::ln::lanes_2
  0.013688       f32::ln::lanes_3
  0.013658       f32::ln::lanes_6
  0.015251       f32::log10::lanes_1
  0.013628       f32::log10::lanes_2
  0.013611       f32::log10::lanes_3
  0.015248       f32::log10::lanes_6
  0.013703       f32::log2::lanes_1
  0.013625       f32::log2::lanes_2
  0.015322       f32::log2::lanes_3
  0.014007       f32::log2::lanes_6
  0.013735       f32::log::lanes_2
  0.015216       f32::log::lanes_3
  0.013565       f32::log::lanes_6
  0.013636       f32::sin::lanes_1
  0.015283       f32::sin::lanes_2
  0.014103       f32::sin::lanes_3
  0.013599       f32::sin::lanes_6
  0.015191       f64::cos::lanes_1
  0.013988       f64::cos::lanes_2
  0.013582       f64::cos::lanes_3
  0.013634       f64::cos::lanes_6
  0.015589       f64::exp2::lanes_1
  0.013691       f64::exp2::lanes_2
  0.013704       f64::exp2::lanes_3
  0.015679       f64::exp2::lanes_6
  0.013647       f64::exp::lanes_1
  0.014061       f64::exp::lanes_2
  0.015406       f64::exp::lanes_3
  0.013673       f64::exp::lanes_6
  0.013610       f64::ln::lanes_1
  0.015338       f64::ln::lanes_2
  0.013550       f64::ln::lanes_3
  0.013579       f64::ln::lanes_6
  0.015199       f64::log10::lanes_1
  0.013582       f64::log10::lanes_2
  0.013587       f64::log10::lanes_3
  0.015058       f64::log10::lanes_6
  0.013520       f64::log2::lanes_1
  0.013474       f64::log2::lanes_2
  0.013490       f64::log2::lanes_3
  0.014898       f64::log2::lanes_6
  0.013661       f64::log::lanes_1
  0.013527       f64::log::lanes_2
  0.014919       f64::log::lanes_3
  0.013641       f64::log::lanes_6
  0.013585       f64::sin::lanes_1
  0.015452       f64::sin::lanes_2
  0.013729       f64::sin::lanes_3
  0.013687       f64::sin::lanes_6

These functions do not have guaranteed precision, so results should probably by compared with assert_approx_ex instead of assert_eq, at least when running in Miri.

Cc @LorrensP-2158466

[workingjubilee]

...okay for most of those I see what you're saying but:

log2?

[RalfJung]

Quoting https://doc.rust-lang.org/nightly/std/primitive.f32.html#method.log2

The precision of this function is non-deterministic. This means it varies by platform, Rust version, and can even differ within the same execution from one invocation to the next.

Miri just implements what the docs say...

Cc @tgross35

[workingjubilee]

For most of our multi-width test suite, we are interested in assert_eq! because we are precisely interested in the cases of non-equivalence, as it is actually rare.

[tgross35]

Even for log2 musl (and our) implementations have up to 1ulp error (haven't checked glibc), presumably we have to allow for that if it is ever used as a fallback. This winds up with a similar problem to the standard library testsuite; asserting exact equality in our internal tests helps us be the first to know about implementation regressions rather than hiding them in a tolerance, as Jubilee mentioned. But for anything user-facing like doctest should be accounting for the possible imprecision.

Not sure what the best solution is here either. Maybe #[cfg(miri)] could enable an overriden assert_eq! that adds a tolerance?

[LorrensP-2158466]

asserting exact equality in our internal tests helps us be the first to know about implementation regressions

I agree with that statement.

For example, this change in rust-lang/rust#138062:

- assert_biteq!(1.0f64.powi(1), 1.0);
+ assert_approx_eq!(1.0f64.powi(1), 1.0);

It was made because the C23 standard doesn't specify pown(1.0, 1) = 1.

I mean, I get why this is expected. And maybe we should rethink those tests as well? I just don't exactly know how to do this without doing this in some places:

if cfg!(miri) { ... } else { ... };

But, glad to know we're catching some things :).