Towards Improving Robustness of Deep Neural Networks to Adversarial Perturbations
Sajjad Amini, Shahrokh Ghaemmaghami
Abstract
Deep neural networks have presented superlative performance in many machine learning based perception and recognition tasks, where they have even outperformed human precision in some applications. However, it has been found that human perception system is much more robust to adversarial perturbation, as compared to these artificial networks. It has been shown that a deep architecture with a lower Lipschitz constant can generalize better and tolerate higher level of adversarial perturbation. Smooth regularization has been proposed to control the Lipschitz constant of a deep architecture and in this work, we show how a deep convolutional neural network (CNN), based on non-smooth regularization of convolution and fully connected layers, can present enhanced generalization and robustness to adversarial perturbation, simultaneously. We propose two non-smooth regularizers that present specific features for adversarial samples with different levels of signal-to-noise ratios. The regularizers build direct interconnections for the weight matrices in each layer, through which they control the Lipschitz constant of architecture and improve the consistency of input-output mapping of the network. This leads to more reliable and interpretable network mapping and reduces abrupt changes in the networks output. We develop an efficient algorithm to solve the non-smooth learning problems, which presents a gradual complexity addition property. Our simulation results over three benchmark datasets signify the superiority of the proposed formulations over previously reported methods for improving the robustness of deep architecture, towards human robustness to adversarial samples.