Hardware Compute Partitioning on NVIDIA GPUs

Joshua Bakita, James H. Anderson

202336 citationsDOI

Abstract

Embedded and autonomous systems are increasingly integrating AI/ML features, often enabled by a hardware accelerator such as a GPU. As these workloads become increasingly demanding, but size, weight, power, and cost constraints remain unyielding, ways to increase GPU capacity are an urgent need. In this work, we provide a means by which to spatially partition the computing units of NVIDIA GPUs transparently, allowing of tidled capacity to be reclaimed via safe and efficient GPU sharing. Our approach works on any NVIDIA GPU since 2013, and can be applied via our easy-to-use, user-space library titled libsmctrl. We back the design of our system with deep investigations into the hardware scheduling pipeline of NVIDIA GPUs. We provide guidelines for the use of our system, and demonstrate it via an object detection case study using YOLOv2.

Topics & Concepts

Computer sciencePartition (number theory)Pipeline (software)General-purpose computing on graphics processing unitsHardware accelerationParallel computingScheduling (production processes)Symmetric multiprocessor systemDesign space explorationCUDAComputer architectureEmbedded systemField-programmable gate arrayOperating systemGraphicsEconomicsOperations managementMathematicsCombinatoricsAdvanced Neural Network ApplicationsParallel Computing and Optimization TechniquesCCD and CMOS Imaging Sensors