CS231n Solver.py 详解

最新推荐文章于 2021-03-19 12:23:35 发布

weixin_30632883

最新推荐文章于 2021-03-19 12:23:35 发布

阅读量426

点赞数

文章标签： python

原文链接：http://www.cnblogs.com/lijiajun/p/5582789.html

版权

CS231n Solver.py 详解

Solver是一个类，该类用于接收数据与标签，对权值进行相应求解，在solver类中调整一些超参数以达到最好的训练效果。

成员函数

初始化函数

 1 def __init__(self, model, data, **kwargs):
 2     """
 3     Construct a new Solver instance.
 4     
 5     Required arguments:
 6     - model: A model object conforming to the API described above
 7     - data: A dictionary of training and validation data with the following:
 8       'X_train': Array of shape (N_train, d_1, ..., d_k) giving training images
 9       'X_val': Array of shape (N_val, d_1, ..., d_k) giving validation images
10       'y_train': Array of shape (N_train,) giving labels for training images
11       'y_val': Array of shape (N_val,) giving labels for validation images
12       
13     Optional arguments:
14     - update_rule: A string giving the name of an update rule in optim.py.
15       Default is 'sgd'.
16     - optim_config: A dictionary containing hyperparameters that will be
17       passed to the chosen update rule. Each update rule requires different
18       hyperparameters (see optim.py) but all update rules require a
19       'learning_rate' parameter so that should always be present.
20     - lr_decay: A scalar for learning rate decay; after each epoch the learning
21       rate is multiplied by this value.
22     - batch_size: Size of minibatches used to compute loss and gradient during
23       training.
24     - num_epochs: The number of epochs to run for during training.
25     - print_every: Integer; training losses will be printed every print_every
26       iterations.
27     - verbose: Boolean; if set to false then no output will be printed during
28       training.
29     """
30     self.model = model
31     self.X_train = data['X_train']
32     self.y_train = data['y_train']
33     self.X_val = data['X_val']
34     self.y_val = data['y_val']
35     
36     # Unpack keyword arguments
37     self.update_rule = kwargs.pop('update_rule', 'sgd')
38     self.optim_config = kwargs.pop('optim_config', {})
39     self.lr_decay = kwargs.pop('lr_decay', 1.0)
40     self.batch_size = kwargs.pop('batch_size', 100)
41     self.num_epochs = kwargs.pop('num_epochs', 10)
42 
43     self.print_every = kwargs.pop('print_every', 100)
44     self.verbose = kwargs.pop('verbose', True)
45 
46     # Throw an error if there are extra keyword arguments
47     if len(kwargs) > 0:
48       extra = ', '.join('"%s"' % k for k in kwargs.keys())
49       raise ValueError('Unrecognized arguments %s' % extra)
50 
51     # Make sure the update rule exists, then replace the string
52     # name with the actual function
53     if not hasattr(optim, self.update_rule):
54       raise ValueError('Invalid update_rule "%s"' % self.update_rule)
55     self.update_rule = getattr(optim, self.update_rule)
56 
57     self._reset()

初始化函数接收的变量有：

（1）模型model，这本是一个类对象，定义了网络的结构特征，和数据，优化方法等没有关系，就是单纯的一个网络结构，包含了网络前向后向的计算函数。

（2）数据data，这是一个结构体，包含了训练集：X_train。验证集X_val。训练标签：y_train。验证标签：y_val

（3）第三个参数**kwargs是指将输入的量写成一个字典的形式。在初始化函数中会依次进行pop，如果没有设定某些值就赋予一个默认值

重置函数

 1 def _reset(self):
 2     """
 3     Set up some book-keeping variables for optimization. Don't call this
 4     manually.
 5     """
 6     # Set up some variables for book-keeping
 7     self.epoch = 0
 8     self.best_val_acc = 0
 9     self.best_params = {}
10     self.loss_history = []
11     self.train_acc_history = []
12     self.val_acc_history = []
13 
14     # Make a deep copy of the optim_config for each parameter
15     self.optim_configs = {}
16     for p in self.model.params:
17       d = {k: v for k, v in self.optim_config.iteritems()}
18       self.optim_configs[p] = d

重置函数对一些solver类中的变量进行了重置。特别注意的是新建了一个

optim_configs字典来存储优化的参数，之前的优化参数保存在self.optim_config字典中，这两个是完全不一样的！！

_step函数

 1 def _step(self):
 2     """
 3     Make a single gradient update. This is called by train() and should not
 4     be called manually.
 5     """
 6     # Make a minibatch of training data
 7     num_train = self.X_train.shape[0] %确定有多少个训练集样本
 8     batch_mask = np.random.choice(num_train, self.batch_size) % 从中随机选择出batch_size这么多个
 9     X_batch = self.X_train[batch_mask] % 从训练集中截取
10     y_batch = self.y_train[batch_mask] % 截取对应的标志
11 
12     # Compute loss and gradient %计算损失函数和梯度
13     loss, grads = self.model.loss(X_batch, y_batch) % 调用模型的loss函数进行计算
14     self.loss_history.append(loss) % 将loss值存入一个向量中，后面会plot出来。注意每一个loss都是用一个batch这么多数据求出来的
15 
16     # Perform a parameter update
17     for p, w in self.model.params.iteritems():
18       dw = grads[p]
19       config = self.optim_configs[p]
20       next_w, next_config = self.update_rule(w, dw, config)% 注意这里！！，之前使用过getattr函数，所以成了一个函数
21       self.model.params[p] = next_w
22       self.optim_configs[p] = next_config

check_accuracy函数

 1 def check_accuracy(self, X, y, num_samples=None, batch_size=100):
 2     """
 3     Check accuracy of the model on the provided data.
 4     
 5     Inputs:
 6     - X: Array of data, of shape (N, d_1, ..., d_k)
 7     - y: Array of labels, of shape (N,)
 8     - num_samples: If not None, subsample the data and only test the model
 9       on num_samples datapoints.
10     - batch_size: Split X and y into batches of this size to avoid using too
11       much memory.
12       
13     Returns:
14     - acc: Scalar giving the fraction of instances that were correctly
15       classified by the model.
16     """
17     
18     # Maybe subsample the data
19     N = X.shape[0] % 输入例子的个数
20     if num_samples is not None and N > num_samples: % 例子太多随机抽取一些子类
21       mask = np.random.choice(N, num_samples)
22       N = num_samples
23       X = X[mask] % 随机抽取一些子例子
24       y = y[mask]
25 
26     # Compute predictions in batches
27     num_batches = N / batch_size % 看看N可以分成几个batch
28     if N % batch_size != 0: %如果不能整除
29       num_batches += 1 % 分成的份数加1
30     y_pred = [] %预测值
31     for i in xrange(num_batches): %对每一份例子进行循环
32       start = i * batch_size % 选出当前的例子：这是开头
33       end = (i + 1) * batch_size % 选出当前的例子： 这是结尾
34       scores = self.model.loss(X[start:end]) % 对开头结尾之间的例子进行预测
35       y_pred.append(np.argmax(scores, axis=1)) %将预测后的值取最大值代表该例子的类别，并链接
36     y_pred = np.hstack(y_pred) %将所有的预测合在一起
37     acc = np.mean(y_pred == y) % 求一个平均，做为准确率
38 
39     return acc % 返回准确率

之所以我们分成batch来求，然后合在一起，是为了防止例子过多，内存装不下。

train函数

 1 def train(self):
 2     """
 3     Run optimization to train the model.
 4     """
 5     num_train = self.X_train.shape[0] % 读取训练的例子的个数
 6     iterations_per_epoch = max(num_train / self.batch_size, 1) % 在下面进行解释
 7     num_iterations = self.num_epochs * iterations_per_epoch
 8 
 9     for t in xrange(num_iterations): % 对每一个iteration进行循环！！
10       self._step() % 更新一下。每次更新都是从所有例子中，抽取batch_size个例子，所以batch越小，要想覆盖所有的数据集
所需要的迭代次数越多，也就解释了上面的iterations_per_epoch的来源
11 
12       # Maybe print training loss
13       if self.verbose and t % self.print_every == 0: % 在计算过程中观察中间结果，
14         print '(Iteration %d / %d) loss: %f' % ( %可见print_every后面是迭代的次数
15                t + 1, num_iterations, self.loss_history[-1]) % 不是epoch的次数
16 
17       # At the end of every epoch, increment the epoch counter and decay the
18       # learning rate.
19       epoch_end = (t + 1) % iterations_per_epoch == 0 由于每个epoch是由一些iteration组成
20       if epoch_end: %如果到达了足够多的iteration，也就是epoch结束了
21         self.epoch += 1 % epoch加 1
22         for k in self.optim_configs:  % 所有的learning_rate都要decay
23           self.optim_configs[k]['learning_rate'] *= self.lr_decay
24 
25       # Check train and val accuracy on the first iteration, the last
26       # iteration, and at the end of each epoch.
27       first_it = (t == 0) % 在第一个和最后一个iteration，以及epoch结束时检查acc
28       last_it = (t == num_iterations + 1) %
29       if first_it or last_it or epoch_end: % 计算train和val的acc
30         train_acc = self.check_accuracy(self.X_train, self.y_train,
31                                         num_samples=1000)
32         val_acc = self.check_accuracy(self.X_val, self.y_val)
33         self.train_acc_history.append(train_acc)% 将两个的acc进行记录
34         self.val_acc_history.append(val_acc)
35 
36         if self.verbose:
37           print '(Epoch %d / %d) train acc: %f; val_acc: %f' % (
38                  self.epoch, self.num_epochs, train_acc, val_acc)
39 
40         # Keep track of the best model
41         if val_acc > self.best_val_acc:
42           self.best_val_acc = val_acc
43           self.best_params = {}
44           for k, v in self.model.params.iteritems():
45             self.best_params[k] = v.copy()
46 
47     # At the end of training swap the best params into the model
48     self.model.params = self.best_params