Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms
Noor Fatima
2020ADCAIJ ADVANCES IN DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE JOURNAL54 citationsDOIOpen Access PDF
Abstract
Adopting the most suitable optimization algorithm (optimizer) for a Neural Network Model is among the most important ventures in Deep Learning and all classes of Neural Networks. It’s a case of trial and error experimentation. In this paper, we will experiment with seven of the most popular optimization algorithms namely: sgd, rmsprop, adagrad, adadelta, adam, adamax and nadam on four unrelated datasets discretely, to conclude which one dispenses the best accuracy, efficiency and performance to our deep neural network. This work will provide insightful analysis to a data scientist in choosing the best optimizer while modelling their deep neural network.
Topics & Concepts
Artificial neural networkComputer scienceArtificial intelligenceDeep learningDeep neural networksMachine learningOptimization algorithmAlgorithmMathematical optimizationMathematicsAdvanced Neural Network ApplicationsAnomaly Detection Techniques and ApplicationsMachine Learning and Data Classification