在有机器学习和深度学习的基础上，如何自学AutoML算法？

作者：FedAI联邦学习
链接：https://www.zhihu.com/question/334021426/answer/840727058
来源：知乎
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。
 

AutoML技术有很多种，包括:

• 神经网络架构搜索
• 优化器搜索
• 自动添加
• 元学习
• 超参数优化
• ......

如果有机器学习和深度学习经验的话，下边这些“干货”就可以直接入手了，咱们挨着说：

一、参考学习论文

AutoML调研

1.A Survey on Neural Architecture Search (Wistuba et al. 2019)

A Survey on Neural Architecture Search | Request PDF​www.researchgate.net

2.Neural Architecture Search: A Survey (Elsken et al. 2019)

Neural Architecture Search: A Survey​arxiv.org

 

3.Taking Human out of Learning Applications: A Survey on Automated Machine Learning (Yao et al. 2018)

https://arxiv.org/abs/1810.13306v1​arxiv.org

 

神经网络架构搜索

1.Neural Architecture Search with Reinforcement Learning (Zoph and Le. 2016)

Neural Architecture Search with Reinforcement Learning​xueshu.baidu.com

 

2. Learning Transferable Architectures for Scalable Image Recognition (Zoph et al. 2017)

Learning Transferable Architectures for Scalable Image Recognition​arxiv.org

 

3. AdaNet: Adaptive Structural Learning of Artificial Neural Networks (Cortes et al. 2017)

AdaNet: Adaptive Structural Learning of Artificial Neural Networks​xueshu.baidu.com

 

4.Regularized Evolution for Image Classifier Architecture Search (Real et al. 2018)

Regularized Evolution for Image Classifier Architecture Search​xueshu.baidu.com

 

5. Differentiable Architecture Search (Liu et al. 2018)

DARTS: Differentiable Architecture Search​arxiv.org

 

6.MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks (Gordon et al. 2018)

MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks​xueshu.baidu.com

 

7.MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning (Hsu et al. 2018)

MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning​arxiv.org

 

8.Neural Architecture Search: A Survey (Elsken et al. 2018)

Neural Architecture Search: A Survey​arxiv.org

 

9.Task-Driven Convolutional Recurrent Models of the Visual System (Nayebi et al. 2018)

Task-Driven Convolutional Recurrent Models of the Visual System​xueshu.baidu.com

 

10.Stochastic Adaptive Neural Architecture Search for Keyword Spotting (Véniat et al. 2018)

Stochastic Adaptive Neural Architecture Search for Keyword Spotting​xueshu.baidu.com

 

11.IRLAS: Inverse Reinforcement Learning for Architecture Search (Guo et al. 2018)

IRLAS: Inverse Reinforcement Learning for Architecture Search​xueshu.baidu.com

 

12.NeuNetS: An Automated Synthesis Engine for Neural Network Design (Sood et al. 2019)

NeuNetS: An Automated Synthesis Engine for Neural Network Design​xueshu.baidu.com

 

13.The Evolved Transformer (So et al. 2019)

The Evolved Transformer​arxiv.org

 

14.Evolutionary Neural AutoML for Deep Learning (Liang et al. 2019)

Evolutionary Neural AutoML for Deep Learning​xueshu.baidu.com

 

15.InstaNAS: Instance-aware Neural Architecture Search (Cheng et al. 2019)

InstaNAS: Instance-aware Neural Architecture Search​xueshu.baidu.com

 

16.The Evolved Transformer (So et al. 2019)

The Evolved Transformer​arxiv.org

 

17.Single Path One-Shot Neural Architecture Search with Uniform Sampling (Guo et al. 2019)

Single Path One-Shot Neural Architecture Search with Uniform Sampling​xueshu.baidu.com

 

18.MixConv: Mixed Depthwise Convolutional Kernels (Tan et al. 2019)

MixConv: Mixed Depthwise Convolutional Kernels​arxiv.org

 

19.DetNAS: Backbone Search for Object Detection (Chen et al. 2019)

DetNAS: Backbone Search for Object Detection​arxiv.org

 

优化器搜索

1.Neural Optimizer Search with Reinforcement Learning (Bello et al. 2017)

https://arxiv.org/abs/1709.07417​arxiv.org

 

自动添加

1.Learning Data Augmentation Strategies for Object Detection (Zoph et al. 2019)

Learning Data Augmentation Strategies for Object Detection​arxiv.org

 

2.Fast AutoAugment (Lim et al. 2019)

Fast AutoAugment​arxiv.org

 

元学习

1.Learning to Learn with Gradients (Chelsea Finn PhD disseration 2018)

Learning to Learn with Gradients​escholarship.org

2.On First-Order Meta-Learning Algorithms (OpenAI Reptile by Nichol et al. 2018)

On First-Order Meta-Learning Algorithms​xueshu.baidu.com

 

3.Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML by Finn et al. 2017)

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks​arxiv.org

 

4.A simple neural attentive meta-learner (Mishra et al. 2017)

https://arxiv.org/abs/1707.03141​arxiv.org

 

超参数优化

1.Google Vizier: A Service for Black-Box Optimization (Golovin et al. 2017)

https://www.researchgate.net/publication/318920618_Google_Vizier_A_Service_for_Black-Box_Optimization​www.researchgate.net

 

2.Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (Li et al. 2016)

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization​arxiv.org

 

模型压缩

1.AMC: AutoML for Model Compression and Acceleration on Mobile Devices (He et al. 2018)

AMC: AutoML for Model Compression and Acceleration on Mobile Devices​arxiv.org

 

二、可使用工具和开源项目

1.ATM: Auto Tune Models: A multi-tenant, multi-data system for automated machine learning (model selection and tuning)

2.Adanet: Fast and flexible AutoML with learning guarantees: Tensorflow package for AdaNet

3.Microsoft Neural Network Intelligence (NNI): An open source AutoML toolkit for neural architecture search and hyper-parameter tuning

4.Dragonfly: An open source python library for scalable Bayesian optimisation

5.H2O AutoML: Automatic Machine Learning by H2O.ai

6. Kubernetes Katib: hyperparameter Tuning on Kubernetes inspired by Google Vizier

7.TransmogrifAI: automated machine learning for structured data by Salesforce

8.AutoSklearn: an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator

9.Ludwig: a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code

10.AutoWeka: hyperparameter search for Weka

11.SMAC: Sequential Model-based Algorithm Configuration

12.Hyperopt-sklearn: hyper-parameter optimization for sklearn

13.Spearmint: a software package to perform Bayesian optimization

14.TOPT: one of the very first AutoML methods and open-source software packages

15.MOE: a global, black box optimization engine for real world metric optimization by Yelp

16.Optuna: define-by-run hypterparameter optimization framework

17.RoBO: a Robust Bayesian Optimization framework

18.HPOlib2: a library for hyperparameter optimization and black box optimization benchmarks

19.Hyperopt: distributed Asynchronous Hyperparameter Optimization in Python

20.ExploreKit: a framework forautomated feature generation

21.FeatureTools: An open source python framework for automated feature engineering

22.PocketFlow: use AutoML to do model compression (open sourced by Tencent)

23.DEvol (DeepEvolution): a basic proof of concept for genetic architecture search in Keras

三、一些商业化产品

1.Amazon SageMaker

2.Google Cloud AutoML

3.Google Cloud ML Hyperparameter Turning

4.Microsoft Azure Machine Learning Studio

5.SigOpt

四、公开Presentation

1.ICML 2019 Tutorial: Recent Advances in Population-Based Search for Deep Neural Networks by Evolving AI Lab

2.Automatic Machine Learning by Frank Hutter and Joaquin Vanschoren

3.Advanced Machine Learning Day 3: Neural Architecture Search by Debadeepta Dey (MSR)

4.Neural Architecture Search by Quoc Le (Google Brain)

五、值得一看的书籍教材

1.AUTOML: METHODS, SYSTEMS, CHALLENGES

AutoML: Methods, Systems, Challenges (new book)​www.automl.org

2.NIPS 2018 3rd AutoML Challenge: AutoML for Lifelong Machine Learning

六、落地应用一则

1.AutoML: Automating the design of machine learning models for autonomous driving by Waymo

英：

https://medium.com/waymo/automl-automating-the-design-of-machine-learning-models-for-autonomous-driving-141a5583ec2a​medium.com

 

中：

AutoML：无人驾驶机器学习模型设计自动化 - 云+社区 - 腾讯云​cloud.tencent.com

 

 

以上，如果对各位有此意向的朋友有用记得点个赞同，感谢！