Neural Network with Smooth Activation Functions and without Bottlenecks Is Almost Surely a Morse Function
С. В. Курочкин
Abstract
It is proved that a neural network with sigmoidal activation functions is a Morse function for almost all, with respect to the Lebesgue measure, sets of parameters (weights) in the case when the network architecture has no bottlenecks, i.e., layers with fewer neurons than in the adjacent layers. It is shown by examples that the requirement for no bottlenecks is essential.
Topics & Concepts
Morse codeMathematicsFunction (biology)Activation functionArtificial neural networkMorse theoryMathematical analysisPure mathematicsApplied mathematicsArtificial intelligenceComputer scienceTelecommunicationsBiologyEvolutionary biologyNeural Networks and ApplicationsFace and Expression RecognitionTopological and Geometric Data Analysis