Model Protection: Real-Time Privacy-Preserving Inference Service for Model Privacy at the Edge
Jiahui Hou, Huiqi Liu, Yunxin Liu, Yu Wang, Peng‐Jun Wan, Xiang‐Yang Li
Abstract
Major cloud service providers with well-equipped infrastructure, experienced machine learning (ML) expertise, and enriched training datasets are building ML-as-a-Service (MLaaS) systems, in which clients can query ML-based prediction services with their data. Instead of moving private data to the cloud, in this work, we design, implement, and evaluate a novel secure ML system to enable MLaaS on edge devices. To protect the proprietary ML models on edge devices from revealing to the clients while maintaining a real-time inference is challenging. Existing privacy-preserving ML techniques can hardly satisfy real-time requirements. In our solution, we employ a secure enclave (e.g., SGX) to offer security and provide better efficiency than cryptographic techniques. However, the enclave alone cannot achieve real-time capability due to its limited capacity. We observe that the ML model imposes a severe accuracy degradation when adding noise to a few model weights. Based on this, we design a suite of novel solutions to optimize the performance of secure enclave-based inference service at the edge by enclosing only <inline-formula><tex-math notation="LaTeX">$1\%$</tex-math></inline-formula> computation within secure enclaves. Our work can achieve up to a <inline-formula><tex-math notation="LaTeX">$7.8\times$</tex-math></inline-formula> increase in efficiency and a <inline-formula><tex-math notation="LaTeX">$27\times$</tex-math></inline-formula> reduction in memory usage compared to the state-of-the-art.