关闭

(Caffe)基本类Solver、Caffe、Batch(二)

296人阅读 评论(0) 收藏 举报
分类:

本文地址:http://blog.csdn.net/mounty_fsc/article/details/51088173

1 Solver

1.1 简介

其对网络进行求解,其作用有:

  1. 提供优化日志支持、创建用于学习的训练网络、创建用于评估的测试网络
  2. 通过调用forward / backward迭代地优化,更新权值
  3. 周期性地评估测试网络
  4. 通过优化了解model及solver的状态

1.2 源代码

<code class="language-c++ hljs cpp has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/**
 * @brief An interface for classes that perform optimization on Net%s.
 *
 * Requires implementation of ApplyUpdate to compute a parameter update
 * given the current state of the Net parameters.
 */</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> Solver {
 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>:
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">explicit</span> Solver(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> SolverParameter& param,
      <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Solver* root_solver = NULL);
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">explicit</span> Solver(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">string</span>& param_file, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Solver* root_solver = NULL);
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Init(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> SolverParameter& param);
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> InitTrainNet();
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> InitTestNets();
 ...
  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// The main entry of the solver function. In default, iter will be zero. Pass</span>
  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// in a non-zero iter number to resume training for a pre-trained net.</span>
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Solve(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">char</span>* resume_file = NULL);
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Solve(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">string</span> resume_file) { Solve(resume_file.c_str()); }
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Step(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> iters);
...

 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span>:
  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// Make and apply the update value for the current iteration.</span>
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> ApplyUpdate() = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>;
  ...

  SolverParameter param_;
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> iter_;
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> current_step_;
  <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">shared_ptr</span><Net<Dtype> > net_;
  <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">shared_ptr</span><Net<Dtype></span> > > test_nets_;
  <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Callback*></span> callbacks_;
  <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Dtype></span> losses_;
  Dtype smoothed_loss_;

  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// The root solver that holds root nets (actually containing shared layers)</span>
  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// in data parallelism</span>
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Solver* <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> root_solver_;
...
};</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li></ul>

说明:

  1. shared_ptr<Net<Dtype>> net_为训练网络的指针,vector<shared_ptr<Net<Dtype>>> test_nets为测试网络的指针组,可见测试网络可以有多个

  2. 一般来说训练网络跟测试网络在实现上会有区别,但是绝大部分网络层是相同的。

  3. 不同的模型训练方法通过重载函数ComputeUpdateValue( )实现计算update参数的核心功能

  4. caffe.cpp中的train( )函数训练模型,在这里实例化一个Solver对象,初始化后调用了Solver中的Solve( )方法。而这个Solve( )函数主要就是在迭代运行下面这两个函数。ComputeUpdateValue(); 
    net_->Update();

1.3 Solver的方法

  • Stochastic Gradient Descent (type: “SGD”)
  • AdaDelta (type: “AdaDelta”)
  • Adaptive Gradient (type: “AdaGrad”)
  • Adam (type: “Adam”)
  • Nesterov’s Accelerated Gradient (type: “Nesterov”)
  • RMSprop (type: “RMSProp”)

详细参见引用1

2 Caffe类

Caffe类为一个包含常用的caffe成员的单例类。如caffe使用的cuda库cublas,curand的句柄等,以及生成Caffe中的随机数等。

<code class="language-c++ hljs cs has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// common.hpp</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// A singleton class to hold common caffe stuff, such as the handler that</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// caffe is going to use for cublas, curand, etc.</span>
class Caffe {
 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>:
  ~<span class="hljs-title" style="box-sizing: border-box;">Caffe</span>();

  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// Thread local context for Caffe. Moved to common.cpp instead of</span>
  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// including boost/thread.hpp to avoid a boost/NVCC issues (#1009, #1010)</span>
  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// on OSX. Also fails on Linux with CUDA 7.0.18.</span>
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> Caffe& Get();

  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">enum</span> Brew { CPU, GPU };
...

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span>:
#ifndef CPU_ONLY
  cublasHandle_t cublas_handle_;
  curandGenerator_t curand_generator_;
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#<span class="hljs-keyword" style="box-sizing: border-box;">endif</span></span>
  shared_ptr<RNG> random_generator_;

  Brew mode_;
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> solver_count_;
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span> root_solver_;

 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span>:
  // The <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> constructor to avoid duplicate instantiation.
  <span class="hljs-title" style="box-sizing: border-box;">Caffe</span>();
  DISABLE_COPY_AND_ASSIGN(Caffe);
};</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li></ul>
<code class="language-c++ hljs r has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">//common.cpp

namespace caffe {

// Make sure each thread can have different values.
static boost::thread_specific_ptr<Caffe> thread_instance_;

Caffe& Caffe::Get() {
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (!thread_instance_.get()) {
    thread_instance_.reset(new Caffe());
  }
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> *(thread_instance_.get());
}

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>
Caffe::Caffe()
    : cublas_handle_(<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">NULL</span>), curand_generator_(<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">NULL</span>), random_generator_(),
    mode_(Caffe::CPU), solver_count_(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), root_solver_(true) {
  // Try to create a cublas handler, and report an error <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> failed (but we will
  // keep the program running as one might just want to run CPU code).
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (cublasCreate(&cublas_handle_) != CUBLAS_STATUS_SUCCESS) {
    LOG(ERROR) << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Cannot create Cublas handle. Cublas won't be available."</span>;
  }
  // Try to create a curand handler.
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (curandCreateGenerator(&curand_generator_, CURAND_RNG_PSEUDO_DEFAULT)
      != CURAND_STATUS_SUCCESS ||
      curandSetPseudoRandomGeneratorSeed(curand_generator_, cluster_seedgen())
      != CURAND_STATUS_SUCCESS) {
    LOG(ERROR) << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Cannot create Curand generator. Curand won't be available."</span>;
  }
}
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>

}  // namespace caffe
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li></ul>

说明:

  1. Caffe类为一个单例类,构造方法私有
  2. 该单例由static boost::thread_specific_ptr<Caffe> thread_instance_维护,确保多线程环境下,不同的线程有不同的Caffe单例版本
  3. 获取该单例由Get()方法执行,即Caffe::Get()方法返回thread_instance_维护的单例,
  4. thread_instance_的初值为NULL,若是第一次访问,则new Caffe()
  5. new Caffe()执行构造方法,其实只是创建了cublas,curand的句柄
  6. 单步调试可发现cublasCreate()创建cublas的句柄,生成了额外的两个线程

3 Batch

<code class="language-c++ hljs css has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">template</span> <<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">typename</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">Dtype</span>>
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">class</span> <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">Batch</span> <span class="hljs-rules" style="box-sizing: border-box;">{
 <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">public</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);">
  Blob<Dtype> data_, label_</span></span>;
<span class="hljs-rule" style="box-sizing: border-box;">}</span></span>;</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

说明:

  • Batch是对一个样本的封装,与Datum不同,Datum是面向数据库的,且一个Datum对应一个样本(图像、标签);而Batch是面向网络的,一个Batch对应一批样本

[1].http://caffe.berkeleyvision.org/tutorial/solver.html


0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:65695次
    • 积分:1560
    • 等级:
    • 排名:千里之外
    • 原创:55篇
    • 转载:56篇
    • 译文:14篇
    • 评论:18条
    最新评论