pytorch贝叶斯网络_使用贝叶斯优化快速调试pytorch中的超参数的快速教程

pytorch贝叶斯网络

Hyperparameters are the parameters in models that determine model architecture, learning speed and scope, and regularization.

超参数是确定模型架构，学习速度和范围以及正则化的模型中的参数。

The search for optimal hyperparameters requires some expertise and patience, and you’ll often find people using exhausting methods like grid search and random search to find the hyperparameters that work best for their problem.

寻找最佳超参数需要一定的专业知识和耐心，并且您经常会发现一些人使用诸如网格搜索和随机搜索之类的精疲力尽的方法来找到最适合其问题的超参数。

快速教程 (A quick tutorial)

I’m going to show you how to implement Bayesian optimization to automatically find the optimal hyperparameter set for your neural network in PyTorch using Ax.

我将向您展示如何实现贝叶斯优化，以使用Ax在PyTorch中自动为您的神经网络找到最佳超参数集。

We’ll be building a simple CIFAR-10 classifier using transfer learning. Most of this code is from the official PyTorch beginner tutorial for a CIFAR-10 classifier.

我们将使用迁移学习构建一个简单的CIFAR-10分类器。 大部分代码来自CIFAR-10分类器的官方PyTorch初学者教程。

I won’t be going into the details of Bayesian optimization, but you can study the algorithm on the Ax website, read the original paper or the 2012 paper on its practical use.

我不会介绍贝叶斯优化的细节，但是您可以在Ax网站上研究该算法，阅读原始论文或2012年论文的实际用法。

首先，通常 (Firstly, the usual)

Install Ax using:

使用以下方法安装Ax：

pip install ax-platform

Import all the necessary libraries:

导入所有必需的库：

import matplotlib.pyplot as plt
import numpy as np
import torch
import torchvision
import torchvision.transforms as transforms
import torch.optim as optim
import torch.nn as nn
import torch.nn.functional as F


from ax.plot.contour import plot_contour
from ax.plot.trace import optimization_trace_single_method
from ax.service.managed_loop import optimize
from ax.utils.notebook.plotting import render
from ax.utils.tutorials.cnn_utils import train, evaluate


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

Download the datasets and construct the data loaders (I would advise adjusting the training batch size to 32 or 64 later):

下载数据集并构造数据加载器(我建议稍后将训练批处理大小调