SwarmCNN: An efficient method for CNN hyperparameter optimization using PSO and ABC metaheuristic algorithms
Özkan İni̇k
Abstract
Abstract Convolutional neural networks (CNNs) have become very popular, as they can successfully solve problems in many areas by obtaining representations of input data at different layers with tuned hyperparameters. A CNN’s hyperparameters include design parameters (DPs), which describe the depth of the CNN and order of layers; layer parameters (LPs), which are used for each CNN layer; and training parameters, which are used for training the CNN. The performance of CNNs depends on these hyperparameters, but setting them properly remains a very difficult and important problem. Although there are studies in the literature that optimize each of these three parameter groups separately, there is a lack of methodologies for simultaneous optimization of DPs and LPs in a nested framework. This study proposes a novel method called SwarmCNN, which combines particle swarm optimization and artificial bee colony algorithms to optimize both DPs and LPs. The effectiveness of SwarmCNN was evaluated across various datasets, including Mnist, Mnist RD, Mnist BN, Mnist BI, Mnist RD + BI, Convex, Rectangles, Mnist Fashion, and CIFAR-10. The results demonstrate promising accuracy rates: 99.58%, 96.20%, 97.56%, 96.39%, 83.39%, 96.92%, 100%, 93.47%, and 84.77%, respectively. Comparative analysis against numerous competitors revealed SwarmCNN’s superiority on five datasets and its second-place ranking on four datasets. The results demonstrated that SwarmCNN emerges as a powerful and competitive solution for optimizing hyperparameters and conducting neural architecture searches with high accuracy on various datasets.