Algorithms for Stochastically Rounded Elementary Arithmetic Operations in IEEE 754 Floating-Point Arithmetic

Abstract

We present algorithms for performing the five elementary arithmetic operations ( <inline-formula><tex-math notation="LaTeX">$+$</tex-math></inline-formula> , <inline-formula><tex-math notation="LaTeX">$-$</tex-math></inline-formula> , ×, <inline-formula><tex-math notation="LaTeX">$\div$</tex-math></inline-formula> , and <inline-formula><tex-math notation="LaTeX">$\sqrt{\phantom{x}}$</tex-math></inline-formula> ) in floating point arithmetic with stochastic rounding, and demonstrate the value of these algorithms by discussing various applications where stochastic rounding is beneficial. The algorithms require that the hardware be compliant with the IEEE 754 floating-point standard and that a floating-point pseudorandom number generator be available. The goal of these techniques is to emulate stochastic rounding when the underlying hardware does not support this rounding mode, as is the case for most existing CPUs and GPUs. By simulating stochastic rounding in software, one has the possibility to explore the behavior of this rounding mode and develop new algorithms even without having access to hardware implementing stochastic rounding—once such hardware becomes available, it suffices to replace the proposed algorithms by calls to the corresponding hardware routines. When stochastically rounding double precision operations, the algorithms we propose are between 7.3 and 19 times faster than the implementations that use the GNU MPFR library to simulate extended precision. We test our algorithms on various tasks, including summation algorithms and solvers for ordinary differential equations, where stochastic rounding is expected to bring advantages.

Topics & Concepts