Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS
Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS
Petro B. Liashchynskyi,Pavlo Liashchynskyi
TLDR
This paper compares the three most popular algorithms for hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm) and attempts to use them for neural architecture search (NAS) and uses these algorithms for building a convolutional neural network (search architecture).
Abstract
In this paper, we compare the three most popular algorithms for hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm) and attempt to use them for neural architecture search (NAS). We use these algorithms for building a convolutional neural network (search architecture). Experimental results on CIFAR-10 dataset further demonstrate the performance difference between compared algorithms. The comparison results are based on the execution time of the above algorithms and accuracy of the proposed models.