A survey of numerical linear algebra methods utilizing mixed-precision arithmetic
Ahmad Abdelfattah, Hartwig Anzt, Erik G. Boman, Erin Carson, Terry Cojean, Jack Dongarra, Alyson Fox, Mark Gates, Nicholas J. Higham, Xiaoye Sherry Li, Jennifer Loe, Piotr Łuszczek, Srikara Pranesh, Siva Rajamanickam, Tobias Ribizel, Barry Smith, Kasia Świrydowicz, Stephen Thomas, Stanimire Tomov, Yaohung M. Tsai, Ulrike Meier Yang
Abstract
The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.