Compressing deep neural networks by matrix product operators
Ze-Feng Gao, Song Cheng, Rong-Qiang He, Z. Y. Xie, Hui-Hai Zhao, Zhong-Yi Lu, Tao Xiang
Abstract
The authors propose a representation of the linear transformations in deep neural networks in terms of matrix product operators developed in quantum physics. The authors showcase their approach in forward neural networks, where both the fully-connected layers and the entire convolutional layers are transformed to this representation, and show that the prediction accuracy can be reached at the same level by using less free parameters
Topics & Concepts
Artificial neural networkConvolutional neural networkComputer scienceProduct (mathematics)Representation (politics)AlgorithmArtificial intelligenceMatrix (chemical analysis)Matrix multiplicationDeep learningDeep neural networksLinear mapOperator (biology)MathematicsPattern recognition (psychology)Linear operatorsMatrix representationTheoretical computer scienceQuantum many-body systemsMachine Learning in Materials ScienceModel Reduction and Neural Networks