Progressive Neural Architecture Search
Abstract
We propose a new method for learning the structure of convolutional
neural networks (CNNs) that is more efficient than recent
state-of-the-art methods based on reinforcement learning and evolutionary
algorithms. Our approach uses a sequential model-based optimization
(SMBO) strategy, in which we search for structures in order of increasing
complexity, while simultaneously learning a surrogate model to guide the
search through structure space. Direct comparison under the same search
space shows that our method is up to 5 times more efficient than the RL
method of Zoph et al. (2018) in terms of number of models evaluated,
and 8 times faster in terms of total compute. The structures we discover
in this way achieve state of the art classification accuracies on CIFAR-10
and ImageNet.
neural networks (CNNs) that is more efficient than recent
state-of-the-art methods based on reinforcement learning and evolutionary
algorithms. Our approach uses a sequential model-based optimization
(SMBO) strategy, in which we search for structures in order of increasing
complexity, while simultaneously learning a surrogate model to guide the
search through structure space. Direct comparison under the same search
space shows that our method is up to 5 times more efficient than the RL
method of Zoph et al. (2018) in terms of number of models evaluated,
and 8 times faster in terms of total compute. The structures we discover
in this way achieve state of the art classification accuracies on CIFAR-10
and ImageNet.
Research Areas
