MLPerf: An Industry Standard Benchmark Suite for Machine Learning Performance
Peter Mattson, Vijay Janapa Reddi, Christine Cheng, Cody Coleman, Greg Diamos, David Kanter, Paulius Micikevicius, David E. Patterson, Guenther Schmuelling, Hanlin Tang, Gu-Yeon Wei, Carole-Jean Wu
Abstract
In this article, we describe the design choices behind MLPerf, a machine learning performance benchmark that has become an industry standard. The first two rounds of the MLPerf Training benchmark helped drive improvements to software-stack performance and scalability, showing a 1.3× speedup in the top 16-chip results despite higher quality targets and a 5.5× increase in system scale. The first round of MLPerf Inference received over 500 benchmark results from 14 different organizations, showing growing adoption.
Topics & Concepts
Benchmark (surveying)Computer scienceScalabilitySuiteSpeedupInferenceMachine learningArtificial intelligenceParallel computingDatabaseHistoryGeodesyArchaeologyGeographyFerroelectric and Negative Capacitance DevicesMachine Learning and Data ClassificationAdversarial Robustness in Machine Learning