The Global Landscape of Neural Networks: An Overview
Ruoyu Sun, Dawei Li, Shiyu Liang, Tian Ding, R. Srikant
Abstract
One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.
Topics & Concepts
Computer scienceArtificial neural networkArtificial intelligenceNeural Networks and ApplicationsStochastic Gradient Optimization TechniquesFerroelectric and Negative Capacitance Devices