Paper Digest - Neural Architecture Search: A Survey

Original paper link: https://arxiv.org/pdf/1808.05377.pdf

---

Introduction

NAS is a rather broad topic. The author of the paper categorize methods for NAS according to three dimensions.

• search space - what structure to search from
• search strategy - how to explore search space
• performance estimation strategy - method on reduce the cost of performance estimation.

Search Space

A simple search space is chain-structured neural network. The search space is then parametrized by:

• (i) the (maximum) number of layers n (possibly unbounded);
• (ii) the type of operation every layer executes, e.g., pooling, convolution, or more advanced operations like depthwise separable convolutions (Chollet, 2016) or dilated convolutions (Yu and Koltun, 2016);
• (iii) hype