Exposing Reliability Degradation and Mitigation in Approximate DNNs Under Permanent Faults
Ayesha Siddique, Khaza Anuarul Hoque
Abstract
Approximate computing is known for enhancing deep neural network accelerators’ energy efficiency by introducing inexactness with a tolerable accuracy loss. However, small accuracy variations may increase the sensitivity of these accelerators toward undesired subtle disturbances, such as permanent faults. The impact of permanent faults in accurate deep neural network (AccDNN) accelerators has been thoroughly investigated in the literature. Conversely, the impact of permanent faults and their mitigation in approximate DNN (AxDNN) accelerators is vastly underexplored. Toward this, we first present an extensive fault resilience analysis of approximate multilayer perceptrons (MLPs) and convolutional neural networks (CNNs) using the state-of-the-art Evoapprox8b multipliers in graphic processing unit (GPU) and tensor processing unit (TPU) accelerators. Then, we propose a novel fault mitigation method, i.e., fault-aware retuning of weights (Fal-reTune). Fal-reTune retunes the weights using a weight mapping function in the presence of faults for improved classification accuracy. To evaluate the fault resilience and the effectiveness of our proposed mitigation method, we used the most widely used MNIST, Fashion-MNIST, and CIFAR10 datasets. Our results demonstrate that the permanent faults exacerbate the accuracy loss in AxDNNs compared with the AccDNN accelerators. For instance, a permanent fault in AxDNNs can lead to 56% accuracy loss, whereas the same faulty bit can lead to only 4% accuracy loss in AccDNN accelerators. We empirically show that our proposed Fal-reTune mitigation method improves the performance of AxDNNs up to 98%, even with fault rates up to 50%. Furthermore, we observe that the fault resilience in AxDNNs is orthogonal to their energy efficiency.