Understanding and Mitigating Hardware Failures in Deep Learning Training Systems
Yi He, Mike Hutton, Steven H. Chan, Robert De Gruijl, Rama Govindaraju, Nishant Patil, Yanjing Li
Abstract
Deep neural network (DNN) training workloads are increasingly susceptible to hardware failures in datacenters. For example, Google experienced "mysterious, difficult to identify problems" in their TPU training systems due to hardware failures [7]. Although these particular problems were subsequently corrected through significant efforts, they have raised the urgency of addressing the growing challenges emerging from hardware failures impacting many DNN training workloads.
Topics & Concepts
Computer scienceTraining (meteorology)Deep learningDeep neural networksArtificial neural networkArtificial intelligenceEmbedded systemComputer architecturePhysicsMeteorologyRadiation Effects in ElectronicsAdversarial Robustness in Machine LearningVLSI and Analog Circuit Testing