Expressivity of Deep Neural Networks
Ingo Gühring, Mones Raslan, Gitta Kutyniok
Abstract
In this chapter, we give a comprehensive overview of the large variety of approximation results for neural networks. Approximation rates for classical function spaces as well as the benefits of deep neural networks over shallow ones for specifically structured function classes are discussed. While the main body of existing results is for general feedforward architectures, we also review approximation results for convolutional, residual and recurrent neural networks.
Topics & Concepts
Computer scienceResidualArtificial neural networkFeedforward neural networkVariety (cybernetics)Artificial intelligenceConvolutional neural networkDeep learningExpressivityDeep neural networksFunction (biology)Feed forwardAlgorithmEngineeringControl engineeringBiologyGeneticsEvolutionary biologyNeural Networks and ApplicationsModel Reduction and Neural NetworksFuzzy Logic and Control Systems