LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
Xiaohu Tang, Yang Wang, Ting Cao, Li Lyna Zhang, Qi Chen, Deng Cai, Yunxin Liu, Mao Yang
Abstract
On-device Deep Neural Network (DNN) inference consumes significant computing resources and development efforts. To alleviate that, we propose LUT-NN, the first system to empower inference by table lookup, to reduce inference cost. LUT-NN learns the typical features for each operator, named centroid, and precompute the results for these centroids to save in lookup tables. During inference, the results of the closest centroids with the inputs can be read directly from the table, as the approximated outputs without computations.
Topics & Concepts
Lookup tableInferenceCentroidTable (database)Computer scienceArtificial neural networkArtificial intelligenceMachine learningComputationData miningPattern recognition (psychology)AlgorithmProgramming languageAdvanced Neural Network ApplicationsMachine Learning and Data ClassificationAnomaly Detection Techniques and Applications