Convergence rates of deep ReLU networks for multiclass classification
Thijs Bos, Johannes Schmidt-Hieber
Abstract
For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.
Topics & Concepts
MathematicsConvergence (economics)Margin (machine learning)Artificial neural networkMulticlass classificationArtificial intelligenceZero (linguistics)Deep neural networksEntropy (arrow of time)Class (philosophy)AlgorithmPattern recognition (psychology)Machine learningApplied mathematicsComputer scienceSupport vector machineEconomic growthPhilosophyEconomicsQuantum mechanicsLinguisticsPhysicsMachine Learning and AlgorithmsAdversarial Robustness in Machine LearningDomain Adaptation and Few-Shot Learning