PADL: Privacy-Aware and Asynchronous Deep Learning for IoT Applications
Xiaoyuan Liu, Hongwei Li, Guowen Xu, Sen Liu, Zhe Liu, Rongxing Lu
Abstract
As a promising data-driven technology, deep learning has been widely employed in a variety of Internet-of-Things (IoT) applications. Examples include automated navigation, telemedicine, and smart home. To protect the data privacy of deep-learning-based IoT applications, a few privacy-preserving approaches have also been exploited, designed, and implemented in various scenarios. However, state-of-the-art works are still defective in accuracy, efficiency, and functionality. In this article, we propose the privacy-aware and asynchronous deep-learning-assisted IoT applications (PADL), a privacy-aware and asynchronous deep learning framework that enables multiple data collecting sites to collaboratively train deep neural networks (DNNs), while keeping the confidentiality of private data to each other. Specifically, we first design a layerwise importance propagation (LIP) algorithm to quantify the importance of the model's weights held by each site. Then, we present the customized perturbation mechanism, a precise combination of the LIP algorithm and differential privacy mechanism, which helps to make optimal tradeoffs between the availability and privacy of local models. Furthermore, to fully use the computing resources of all sites, for the first time, we propose an advanced asynchronous optimization (AAO) protocol to perform global updates without waiting. Theoretical analysis shows that the PADL is robust to extreme collusion even with only one reliable site while supporting lock-free optimization. Finally, extensive experiments conducted on real-world data sets using TensorFlow library show that the PADL outperforms the existing systems in terms of efficiency and prediction accuracy.