Pushing the Limits of Narrow Precision Inferencing at Cloud Scale with Microsoft Floating Point
Bita Darvish Rouhani, Daniel Lo, Ritchie Zhao, Ming Liu, Jeremy Fowers, Kalin Ovtcharov, Anna Vinogradsky, Sarah Massengill, Lita Yang, Ray Bittner, Alessandro Forin, Haishan Zhu, Taesik Na, Prerak Patel, Shuai Che, Lok Chand Koppaka, Xia Song, Subhojit Som, Kaustav Das, T Saurabh, Steve Reinhardt, Sitaram Lanka, Eric S. Chung, Doug Burger
2020Neural Information Processing Systems54 citations
Abstract
In this paper, we explore the limits of Microsoft Floating Point (MSFP), a new class of datatypes developed for production cloud-scale inferencing on custom hardware. Through the co-evolution of hardware design and algorithms, MSFP achieves accuracy comparable to or better than industry standards Bfloat16 and INT8 at 3x and 4x lower cost, respectively. MSFP incurs negligible impact to accuracy (
Topics & Concepts
Computer scienceCloud computingPoint (geometry)Class (philosophy)Scale (ratio)Floating pointDouble-precision floating-point formatPoint cloudProduction (economics)Operating systemComputer hardwareEmbedded systemArtificial intelligenceMathematicsMacroeconomicsPhysicsGeometryQuantum mechanicsEconomicsParallel Computing and Optimization TechniquesBig Data and Digital EconomyNumerical Methods and Algorithms