Accelerating ML Workloads using GPU Tensor Cores: The Good, the Bad, and the Ugly
Bagus Hanindhito, Lizy K. John
Abstract
Machine Learning (ML) workloads generally contain a significant amount of matrix computations; hence, hardware accelerators for ML have been incorporating support for matrix accelerators. With the popularity of GPUs as hardware accelerators for ML, specialized matrix accelerators are embedded into GPUs (e.g., Tensor Cores on NVIDIA GPUs) to significantly improve the performance and energy efficiency of ML workloads. NVIDIA Tensor Cores and other matrix accelerators have been designed to support General Matrix-Matrix Multiplication (GEMM) for many data types. While previous research has demonstrated impressive performance gains with Tensor Cores, they primarily focused on Convolutional Neural Networks (CNNs).
Topics & Concepts
Computer scienceMatrix multiplicationTensor (intrinsic definition)Parallel computingMatrix (chemical analysis)Computational scienceConvolutional neural networkComputationSupercomputerHardware accelerationField-programmable gate arrayComputer hardwareArtificial intelligenceAlgorithmQuantumMathematicsPhysicsComposite materialQuantum mechanicsMaterials sciencePure mathematicsTensor decomposition and applicationsParallel Computing and Optimization TechniquesStochastic Gradient Optimization Techniques