A Training-free Genetic Neural Architecture Search
Mengting Wu, Hung-I Lin, Chun‐Wei Tsai
Abstract
The so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical issue for most of the NAS techniques is in that it is computationally very expensive because several complete/partial training processes are involved in evaluating the goodness of a neural architecture during the process of NAS. To mitigate this problem for evaluating a single neural architecture found by the search algorithm of NAS, we present an efficient NAS in this study, called genetic algorithm and noise immunity for neural architecture search without training (GA-NINASWOT). The genetic algorithm (GA) in the proposed algorithm is used to search for high potential neural architectures while a modified scoring method based on the neural architecture search without training (NASWOT) is used to replace the training process of each neural architecture found by the GA for measuring its quality. To evaluate the performance of GA-NINASWOT, we compared it with several state-of-the-art NAS techniques, which include weight-sharing methods, non-weight-sharing methods, and NASWOT. Simulation results show that GA-NINASWOT outperforms all the other state-of-the-art weight-sharing methods and NASWOT compared in this study in terms of the accuracy and computational time. Moreover, GA-NINASWOT gives a result that is comparable to those found by the non-weight-sharing methods while reducing 99% of the search time.